• vatlark@lemmy.world
    link
    fedilink
    English
    arrow-up
    15
    ·
    3 days ago

    Other than websites, I can never get a citation that has any connection to reality from an LLM. They are either poisoning the training data or it’s just too much detail for the model to capture.

    • Trimatrix@lemmy.world
      link
      fedilink
      English
      arrow-up
      1
      arrow-down
      5
      ·
      3 days ago

      My theory? Its the abstract that trips it up. I don’t know much about neural networks or RAG searching, but the abstract is information dense in a short amount of space. My guess, an LLM scans through a document, finds areas of the document that has the largest quantity of information relevant to the prompt and then branches out until its confident enough in answering the question. While great for most written text, I think the LLM starts with the abstract, gets way too much confidence that it knows enough and then begins spouting bullshit. In my opinion it explains why it creates fictitious citations. Probably sees a citation in an abstract and then conjures up a title from the information in the abstract.

      My only evidence is that I try using LLMs to find information quickly in very dense datasheets for electronic components and most times it will rattle off information found in the component overview section rather than telling me about information found in the application, or electronic characteristics section.

      • AlecSadler@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        6
        ·
        3 days ago

        Are you running your own programmatic LLMs? There is a thing called temperature, and it is typically more lenient for public facing LLMs. But leverage that same LLM via APIs and you can adjust the temperature and reduce or eliminate hallucinations.

        Ultimately, a little variance (creativity) is somewhat good and passing it through levels of agentic validations can help catch hallucinations and dial in the final results.

        That said, I doubt the WH did this, they probably just dumped shit into some crappy public-facing ChatGPT model.

        • Mirshe@lemmy.world
          link
          fedilink
          English
          arrow-up
          3
          ·
          3 days ago

          It’s Grok, it’s always Grok. Musk was there, there’s no way they weren’t dumping terabytes of government data into it in pursuit of some mythical AGI.

          • Trimatrix@lemmy.world
            link
            fedilink
            English
            arrow-up
            3
            ·
            3 days ago

            Last I heard, they use Llama-2 since its the only one approved for GOV work.

            But its probably Grok because they don’t seem keen on approvals

          • AlecSadler@sh.itjust.works
            link
            fedilink
            English
            arrow-up
            1
            ·
            3 days ago

            We tried Grok for some things at a job once and it was absolutely the worst of all of the major ones (for what we were doing). It hallucinated way too much even when attempting to get it not to.

            So you’re probably right.

        • vatlark@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          3 days ago

          Interesting, I haven’t played much with the APIs, I just started messing with running some locally with ollama

  • Lvxferre [he/him]@mander.xyz
    link
    fedilink
    English
    arrow-up
    16
    arrow-down
    3
    ·
    3 days ago

    I’m wondering if Musk is going to be impeached, alongside his first lady Donald Trump-Musk. There’s only a certain level of incompetence that the elite admits from a despot without topping them off, plus they’re slow but surely pissing off even the tech bros.

    • jonne@infosec.pub
      link
      fedilink
      English
      arrow-up
      13
      ·
      3 days ago

      Musk doesn’t hold any office, so he can’t be impeached. He probably stole a bunch of government data, so that might be an avenue for prosecution if the US ever gets an AG that cares about crime again as opposed to anyone in that post for the last decade.

      • Lvxferre [he/him]@mander.xyz
        link
        fedilink
        English
        arrow-up
        4
        arrow-down
        1
        ·
        3 days ago

        Musk doesn’t hold any office, so he can’t be impeached.

        I know. It’s kind of a joke that Musk is the actual president, and Trump his first lady. The question is actually about Trump.

        [Still, thanks for the info!]

  • jballs@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    3
    ·
    3 days ago

    Aw man, seeing this title and that it was from the NYT had me excited for a minute. I was like "oohhh shit, the mainstream media is calling them out for using LLM in the headline?!”

    Turns out, the actual heading is just “White House Health Report Included Fake Citations”.

    They have a few people comment on how LLMs can result in this problem, but still not nearly as dramatic as calling out some bullshit in the headline.