There’s a problem with an overabundance of PhD candidates. There’s a massive reproducibility crisis. Journals are a for profit enterprise.

It is a shame we don’t bake in a reproduction of some research as a gate. We don’t really incentivize reproduction. Imagine finding bunk research via reproduction being rewarded similarly to security peeps reporting vulnerabilities.

EDIT:

Imagine thinking a LLM can create new research.

I was having this same argument with a friend recently. His position is that stuffing all known human knowledge into the clankers will inevitably lead us down paths we’ve not seen yet by drawing the inferences and connections we’ve not made. Sure, but that’s merely additive, not novel. My rebuttal was that the current models will never able to output a token that isn’t in their vocabulary, never utter a truly out of distribution novel thought or concept. Or as Tom Zahavy said, LLMs can’t jump: https://www.tomzahavy.com/files/llms-cant-jump.pdf