The code for the tests, yeah. But not the code for the app itself. If you’ve written tests that you are satisfied fully cover all requirements including edge cases, you can satisfy yourself that the app does what you want it to without looking at the app code.
Test-driven development but with an LLM doing the actual development.
That doesn’t work. TTD works because you’re writing the code you’re testing, so you know exactly what’s in there.
If the LLM hallucinates a feature that wasn’t in the requirements and is somewhat hidden, reviewing the tests won’t cut it. The tests may cover all the requirements, but what about all the code doing stuff that’s not required?
Basically, don’t just test for requirements, also test that what isn’t required isn’t there. And that’s basically an infinite amount of possibilities to test for. The only way you detect that stuff is by checking that the code isn’t doing anything extra it shouldn’t be doing.
The code for the tests, yeah. But not the code for the app itself. If you’ve written tests that you are satisfied fully cover all requirements including edge cases, you can satisfy yourself that the app does what you want it to without looking at the app code.
Test-driven development but with an LLM doing the actual development.
That doesn’t work. TTD works because you’re writing the code you’re testing, so you know exactly what’s in there.
If the LLM hallucinates a feature that wasn’t in the requirements and is somewhat hidden, reviewing the tests won’t cut it. The tests may cover all the requirements, but what about all the code doing stuff that’s not required?
Basically, don’t just test for requirements, also test that what isn’t required isn’t there. And that’s basically an infinite amount of possibilities to test for. The only way you detect that stuff is by checking that the code isn’t doing anything extra it shouldn’t be doing.