They don’t have short memory, they have NO MEMORY AT ALL.

These are statistical word generation machines, that’s what LLMs are right now. They are REALLY good at this.

But they do not have memory, they do not learn, they do not make decisions. Which means they are incapable of cooperation as such a thing cannot exist without memory or the ability to learn, and decisions cannot be made without either of those.

These tools provide the illusion of such attributes.

• There is no memory, the whole context is sent on every request, the LLM does not have knowledge of prior conversations. It only knows what it is provided in that request only.
  • Lots of tricks and hacks to make this illusion really good in incredibly small scales. But it’s still an illusion. Outside of fine-tuning and retraining new LLMs, which is not feasible to do on a frequency of communication.
• There is no learning.
  • Without memory learning is impossible. Learning requires retraining a model, and to a degree fine tuning. Both of these are resource intensive and are static. And only provide the illusion of learning as it cannot happen in real time.