Structural, not temporary
These aren't complaints about a particular tool or a particular month's model. Each failure below traces to something inherent: the model predicts plausible text, it can't see what you didn't show it, and it has no stake in the code existing next year. Better models make each less frequent. None make them go away.
The six
1. Inventing APIs that should exist
A method with the right name, the right arguments, and no existence. This happens because a plausible method name is exactly what the model produces, and the plausible name is frequently the one a sensible library would have chosen. Cheap to catch — types and a run find it immediately — and it costs beginners hours because the code looks right.
2. Solving the stated problem, not the real one
Ask why a function returns undefined and you get a null check added at the return. The undefined came from three layers up. The symptom is gone, the cause is intact, and the null check is now permanent.
3. Ignoring conventions it wasn't shown
You have an error shape, a date helper, a fetch wrapper. It writes new ones — reasonable, idiomatic, and a duplicate. Over months this is the largest hidden cost, because no individual instance looks wrong in review.
4. Over-engineering small things
Asked for a helper, it returns a configurable class with an options object and three strategies. Training data over-represents library code, where generality is correct. In your application it's five times the code for one caller.
5. Confidence unrelated to correctness
The tone is identical whether the answer is well-established or a guess. There's no reliable signal in the output that distinguishes them, which removes the cue you'd normally get from a colleague's hesitation.
6. Deleting things it doesn't understand
A guard clause with no comment looks like dead code. It was added after an incident. This is the strongest argument for comments that record why, and for reviewing deletions at least as carefully as additions.
Adjusting the review
| Failure | Countermeasure |
|---|---|
| Invented APIs | Types, a linter, and running it — automatic |
| Symptom fixes | Ask where the bad value came from before accepting a fix |
| Convention drift | Keep a conventions file the tool reads; point at an existing example |
| Over-engineering | Say "one caller, keep it simple" in the request |
| Misplaced confidence | Ask for the uncertain parts to be listed explicitly |
| Deletions | Review removed lines first in every diff |
Keeping the benefit
None of this argues against using these tools. The speed increase on well-understood work is real and large. The argument is about where the saved time goes: into review, into the specific parts of your product nobody else has built, and into the tests that make the next change safe.
Teams that get durable value from AI-assisted development mostly aren't using better prompts. They're spending the saved time on review instead of on more output.
Common mistakes
- Assuming a newer model removed the need to check
- Accepting a fix without asking where the bad value originated
- Letting three implementations of the same helper accumulate
- Reviewing additions carefully and skimming deletions
- Spending all the saved time on producing more code
Key takeaways
- The failures are structural — from prediction, missing context, and no long-term stake
- Invented APIs, symptom fixes, convention drift, over-engineering, false confidence, unexplained deletions
- Most countermeasures are cheap, and some are automatic
- Spend the time saved on review, not on more output
Try it yourself
Look back at the last three AI-assisted changes in your repository. Which of the six appeared? Most people find the same one or two recur — and knowing which is yours is worth more than a general warning.