When I tried to fix the security issues, I quickly realized how this whole thing was a trap. Since I didn’t wrote it, I didn’t have a good bird’s eye view of the code and what it did.

This is where he lost me. Why didn’t he have a good birds-eye view of the code? So you just blindly accept contributions from the junior devs you work with, too? You should be reviewing every line of code the LLM produces, just like you do with everything else

I was impressed. It really felt like I had superpowers! But then I had the idea to audit the code the LLM just produced, like I did at my $dayjob for a Vue application. Feeling that uploading files could be a source of security issues, I asked the same LLM to focus on this specific topic.

It found several dangers: directory traversal attacks, file size limits, system file overwrite, etc. I had no idea the initial code was this unsafe. I had reviewed the code, but without enough experience in backend development, how could I identify issues I didn’t know existed? And why, if it knew about all those dangers, did the LLM produced unsafe code in the first place?

When I tried to fix the security issues, I quickly realized how this whole thing was a trap. Since I didn’t wrote it, I didn’t have a good bird’s eye view of the code and what it did. I couldn’t make changes quickly, which started to frustrated me. The easiest route was asking the LLM to do the fixes for me, so I did. More code was changed and added. It worked, but again I could not tell if it was good or not.

I try to read these things sympathetically but… If it can slip that many very basic and common vulnerabilities past you without you noticing, and you have no ability to evaluate either the original code or the AI’s “fixes” to it, aren’t you just telling us you lack the skills to do the job competently in the first place? This sounds like a case of “all I know is how to bolt Vue components to one another, and outside of that I’m lost.” It tells us more about your own skill level than about how useful the AI would be to someone who understood more about programming.