That doesn’t work. TTD works because you’re writing the code you’re testing, so you know exactly what’s in there.
If the LLM hallucinates a feature that wasn’t in the requirements and is somewhat hidden, reviewing the tests won’t cut it. The tests may cover all the requirements, but what about all the code doing stuff that’s not required?
Basically, don’t just test for requirements, also test that what isn’t required isn’t there. And that’s basically an infinite amount of possibilities to test for. The only way you detect that stuff is by checking that the code isn’t doing anything extra it shouldn’t be doing.
That doesn’t work. TTD works because you’re writing the code you’re testing, so you know exactly what’s in there.
If the LLM hallucinates a feature that wasn’t in the requirements and is somewhat hidden, reviewing the tests won’t cut it. The tests may cover all the requirements, but what about all the code doing stuff that’s not required?
Basically, don’t just test for requirements, also test that what isn’t required isn’t there. And that’s basically an infinite amount of possibilities to test for. The only way you detect that stuff is by checking that the code isn’t doing anything extra it shouldn’t be doing.