Bad article, the number of issues aren’t normalized to anything other than PRs. I expect AI authorized PRs to be somewhat bigger than a fully human authorized one.
Ideally they’d compare time to write + time to fix. My experience is that if you use test driven development, LLM isn’t too bad. No worse than an intern.
I think it comes down to who is using the LLM. I had a junior dev once “presumably” AI gen a ton of code (broken trash). Then to fix it, they wrapped each function in a try catch block that dropped the error. Unit tests were mocked out to the extent they didn’t test anything.
When I use an LLM, I have tests and hard constraints on the LLM. It isn’t good enough to do everything, but it can generate about 80% of a simple app
Why would I voluntarily subject myself to constantly working with a virtual intern. The only reason to work with someone of that level is because they grow into someone more useful
For LLM generated code, it can also take a whole to read and understand. When I write code myself, I understand the intention, architecture, and so on. Machine written code is very different. I need to understand how it works. There’s often extraneous stuff in there or weird patterns.
I find that it basically can’t do decent architecture. My last attempt to use it ended with it using casbin, but then rewriting it’s own authorization framework and trying to use both at the same time 😶.
I think there is a lot of power here, but it needs very heavy guidance and handholding to do it well. Otherwise it makes very stupid intern level decisions
While I can get speedups for rapid prototypes where the code doesn’t need to be good, just mostly working, to get actually good code it’s inevitably significantly faster for me to write everything than to fix what an LLM starts.
I fully concur with the intern metaphor. That’s what an LLM is - an intern with VAST amounts of lexical knowledge but little understanding of large scale system design, or generally, experience.
LLM generated code is mostly okay. You do have to review it, like you would an intern’s contributions, but generally, it’s good quality code and well documented.
Yeah I don’t think we should be pushing to have LLMs generate code unsupervised. It’s an unrealistic standard. It’s not even a standard most companies would entrust their most capable programmers with. Everything needs to be reviewed.
But just because it’s not working alone doesn’t mean it’s useless. I wrote like 5 lines of code this week by hand. But I committed thousands of lines. And I reviewed and tweaked and tended to every one of them. That’s how it should be.
It’s also kinda good¿ For writing scripts and write regex, specially claude !
If have some very basic knowledge in bash and not enough lifetime to invest into a 1 year full bash course, python or regex patterns… It’s already very time consuming to self-host everything :/.
And while it mostly does what I want/need I also learned a few cool bash tricks i wasn’t aware… Like how to write and use functions !
Bad article, the number of issues aren’t normalized to anything other than PRs. I expect AI authorized PRs to be somewhat bigger than a fully human authorized one.
Ideally they’d compare time to write + time to fix. My experience is that if you use test driven development, LLM isn’t too bad. No worse than an intern.
I think it comes down to who is using the LLM. I had a junior dev once “presumably” AI gen a ton of code (broken trash). Then to fix it, they wrapped each function in a try catch block that dropped the error. Unit tests were mocked out to the extent they didn’t test anything.
When I use an LLM, I have tests and hard constraints on the LLM. It isn’t good enough to do everything, but it can generate about 80% of a simple app
Why would I voluntarily subject myself to constantly working with a virtual intern. The only reason to work with someone of that level is because they grow into someone more useful
For LLM generated code, it can also take a whole to read and understand. When I write code myself, I understand the intention, architecture, and so on. Machine written code is very different. I need to understand how it works. There’s often extraneous stuff in there or weird patterns.
I find that it basically can’t do decent architecture. My last attempt to use it ended with it using casbin, but then rewriting it’s own authorization framework and trying to use both at the same time 😶.
I think there is a lot of power here, but it needs very heavy guidance and handholding to do it well. Otherwise it makes very stupid intern level decisions
While I can get speedups for rapid prototypes where the code doesn’t need to be good, just mostly working, to get actually good code it’s inevitably significantly faster for me to write everything than to fix what an LLM starts.
I fully concur with the intern metaphor. That’s what an LLM is - an intern with VAST amounts of lexical knowledge but little understanding of large scale system design, or generally, experience.
LLM generated code is mostly okay. You do have to review it, like you would an intern’s contributions, but generally, it’s good quality code and well documented.
Yeah I don’t think we should be pushing to have LLMs generate code unsupervised. It’s an unrealistic standard. It’s not even a standard most companies would entrust their most capable programmers with. Everything needs to be reviewed.
But just because it’s not working alone doesn’t mean it’s useless. I wrote like 5 lines of code this week by hand. But I committed thousands of lines. And I reviewed and tweaked and tended to every one of them. That’s how it should be.
It’s also kinda good¿ For writing scripts and write regex, specially claude !
If have some very basic knowledge in bash and not enough lifetime to invest into a 1 year full bash course, python or regex patterns… It’s already very time consuming to self-host everything :/.
And while it mostly does what I want/need I also learned a few cool bash tricks i wasn’t aware… Like how to write and use functions !
Why would you expect that?
Because an LLM can generate a lot of (garbage?) code quickly.