It stands to reason that if you have access to an LLM’s training data, you can influence what’s coming out the other end of the inscrutable AI’s network. The obvious guess is that…
I’m sure that’s true, but it is also noteworthy that any and every consideration that goes into the initial inclusion of the data before it is fed into the model introduces intended and unintended consequences on the training.
Furthermore, the proliferation of the LLMs themselves is putting negative pressure on survival of the places where all the good data is sourced from in the first place. When traffic to a place like stackoverflow is way down because everyone’s reading LLM answers (that the LLM training dataset got from stack overflow), there are less good conversations on stackoverflow to read. Some of these sources of training data may even be caused to cease to exist entirely.
I’m sure that’s true, but it is also noteworthy that any and every consideration that goes into the initial inclusion of the data before it is fed into the model introduces intended and unintended consequences on the training.
Furthermore, the proliferation of the LLMs themselves is putting negative pressure on survival of the places where all the good data is sourced from in the first place. When traffic to a place like stackoverflow is way down because everyone’s reading LLM answers (that the LLM training dataset got from stack overflow), there are less good conversations on stackoverflow to read. Some of these sources of training data may even be caused to cease to exist entirely.