• excral@feddit.org
    link
    fedilink
    arrow-up
    1
    ·
    9 months ago

    I’ve said it time and time again: AIs aren’t trained to produce correct answers, but seemingly correct answers. That’s an important distinction and exactly what makes AIs so dangerous to use. You will typically ask the AI about something you yourself are not an expert on, so you can’t easily verify the answer. But it seems plausible so you assume it to be correct.

    • pankuleczkapl@lemmy.dbzer0.com
      link
      fedilink
      arrow-up
      0
      ·
      9 months ago

      Thankfully, AI is bad at maths for exactly this reason. You don’t have to be an expert on a very specific topic to be able to verify a proof and - spoiler alert - most of the proofs ChatGPT 5 has given me are plain incorrect, despite OpenSlop’s claims that it is vastly superior to previous models.

      • amorpheus@lemmy.world
        link
        fedilink
        arrow-up
        1
        ·
        9 months ago

        most of the proofs ChatGPT 5 has given me are plain incorrect, despite OpenSlop’s claims that it is vastly superior to previous models

        Both of those can be true.

  • stoy@lemmy.zip
    link
    fedilink
    English
    arrow-up
    1
    ·
    9 months ago

    I suspect this will happen all over with in a few years, AI was good enough at first, but over time reality and the AI started drifting apart

    • Spezi@feddit.org
      link
      fedilink
      arrow-up
      1
      ·
      9 months ago

      And then, the very same CEOs that demanded the use of AI in decision making will be the ones that blame it for bad decisions.

        • jj4211@lemmy.world
          link
          fedilink
          arrow-up
          1
          ·
          9 months ago

          Probably more this, the idea to use LLMs is good, but the employees just didn’t know how to use it right. To say the LLMs are the problem is to admit being wrong, or worse, being gullible in the face of marketing material. The one thing that is drilled in as a first principle of business leadership is to never ever look weak by being wrong or tricked.

    • Kirp123@lemmy.world
      link
      fedilink
      arrow-up
      1
      ·
      9 months ago

      AI is literally trained to get the right answer but not actually perform the steps to get to the answer. It’s like those people that trained dogs to carry explosives and run under tanks, they thought they were doing great until the first battle they used them in they realized that the dogs would run under their own tanks instead of the enemy ones, because that’s what they were trained with.

  • mr_sunburn@lemmy.ml
    link
    fedilink
    arrow-up
    0
    ·
    edit-2
    9 months ago

    I raised this as a concern at the corporate role I work in when an AI tool that was being distributed and encouraged for usage showed two hallucinated data points that were cited in a large group setting. I happened to know my area well, the data was not just marginally wrong but way off, and I was able to quickly check the figures. I corrected it in the room after verifying on my laptop and the reaction in the room was sort of a harmless whoops. The rest of the presentation continued without a seeming acknowledgement that the rest of the figures should be checked.

    When I approached the head of the team that constructed the tool after the meeting and shared the inaccuracies and my concerns, he told me that he’d rather have more data fluency through the ease of the tool and that inaccuracies were acceptable because of the convenience and widespread usage.

    I suspect stories like this are happening across my industry. Meanwhile, the company put out a press release about our AI efforts (literally using Gemini’s Gem tool and custom ChatGPTs seeded with Google Drive) as something investors should be very excited about.

    • squaresinger@lemmy.world
      link
      fedilink
      arrow-up
      1
      ·
      9 months ago

      When I approached the head of the team that constructed the tool after the meeting and shared the inaccuracies and my concerns, he told me that he’d rather have more data fluency through the ease of the tool and that inaccuracies were acceptable because of the convenience and widespread usage.

      “I prefer more data that’s completely made up over less data that is actually accurate.”

      This tells you everything you need to know about your company’s marketing and data analysis department and the whole corporate leadership.

      Potemkin leadership.

  • maplesaga@lemmy.world
    link
    fedilink
    arrow-up
    0
    arrow-down
    1
    ·
    9 months ago

    This is why AI will only be used in subjective things that dont need to be perfect, like art. You cant create intelligence by regurgitating reddit posts via an LLM.

      • maplesaga@lemmy.world
        link
        fedilink
        arrow-up
        0
        arrow-down
        1
        ·
        9 months ago

        True, though its still extremely useful, and its impossible to determine where its stealing from for the average consumer. Its objectively useful.

  • kraxla@piefed.social
    link
    fedilink
    English
    arrow-up
    0
    arrow-down
    1
    ·
    5 months ago

    Your high tech tool produces wrong data?

    You cant not use it, your competitor will.

    So you must do something else to make it more reliable, something small that the Firm can do.

    And if you only caught it 3 months in, that firm is incompetent as fuck.

    But its probably not even a real post.