Yesterday (November 27), Habr hosted an 'Author's Fireside Chat'.
It was very interesting, and one of the speaker's statements struck me. It was that AI can help write simple pieces of code but doesn't work with complex things. Thus, large language models are likened to a junior programmer.
I decided to write an article about it this morning, drawing on my knowledge and experience in computational mathematics. Let me know what you think.
I think this is the main myth of vibe-coding. It's exactly the opposite — AI is good at writing quite complex things and retrieving important information that is difficult to find on your own. But it gets confused in the most elementary things. It's a reverse junior.
The problem is that this is a dangerous illusion, and I will now clearly explain why, and how it can be dangerous. Brew some coffee and get ready for a debunking that might save your millions, your career, or even human lives in the future.
The Gist
LLMs like ChatGPT or Claude are actually quite good at imitating the 'complex' stuff — architecture, patterns, dataclasses, even decent modularization. But when it comes to the 'simple' stuff — basic math, loops, numerical methods, physics — they consistently make mistakes that render the result garbage, no matter how many times you draw a UML diagram.
To avoid making unsubstantiated claims, I'll analyze a specific, real-life example: a simulation of the flight of a supersonic ballistic missile, which an AI generated for me. The full Jupyter notebook with the code is in a file I'll link to at the end. Obviously, this mile-long code won't fit into the article, so I'll only include characteristic fragments here.
Of course, the speed at which AI creates such a large amount of code is impressive. But here we will analyze why it's sometimes easier to write everything yourself than to clean up after it.
And yes, an important point: the latest paid version of ChatGPT, if asked to review this code, does indeed find some of the problems — about Euler's method, the integration step, and other 'adult' things. But it remains quite satisfied with places where much more mundane, yet fatal, bugs lie. That's what we'll talk about.
The 'Horns & Code LLC' Effect
To make it clear, let's introduce a term: the 'Horns & Code LLC' effect.
Imagine a hypothetical outsourcing company with that name. They are brilliant at one thing: writing code that looks very convincingly like the real deal.
You open the repository — it's beautiful. There are layers like core, domain, infrastructure. The code is peppered with UserRepository, PaymentService, Factory, Strategy, and sometimes even AggregateRoot. In a three-hundred-line file, there's not a single goto and a lone TODO. The manager is happy, the client is calm, the architect is neatly moving squares in Miro.
And then the system goes to production. Under load, quadratic complexity emerges on the hot path. Money is lost in pennies somewhere because 'well, I used double — what's the big deal.' Retry is implemented with an infinite loop and sleep(100) in hopes of magic.
The facade is senior-level. The filling is a typical student who was given a book on patterns but not a course on algorithms and systems analysis. Today's LLM is exactly that: 'Horns & Code LLC on steroids.' A very convincing form with completely random quality of content where you need precise numbers, not words.
An Example from Personal Experience
I've been tutoring for a long time, and besides high school students, I have a wide variety of clients. A few years ago, I was approached by employees from a typical company in Russia involved in software import substitution. They were creating a CAD system for calculating and designing pipe connections. The specific people who contacted me were creating an import-substituted computational geometry library for this CAD system. They gave me high school plane geometry problems, paying 3000 rubles per problem, which I easily solved and fully documented in 15 minutes.
It turned out that these CAD developers:
had absolutely no grasp of not only matrices and derivatives, but also the basics of 7th-8th grade school geometry
they had never even heard of the computational stability of algorithms
were unaware that everything they had been doing for the last six months could be downloaded as open-source, properly written code from the internet. When I showed them, they said it wouldn't work because they didn't understand it, and their colleagues would immediately realize they had downloaded the code instead of writing it themselves, and then they wouldn't get paid. Most importantly, they had to hide from their management that they were doing something that could be downloaded from the internet, otherwise they would simply be laid off for being useless.
in six months, they wrote a lot of code that would fail on obvious tests. But they said that specialists at their work create the tests, and their code passed all of them successfully, so there were no problems. I'd be interested to see who creates these tests; I think even an LLM couldn't write tests that poorly.
I wanted to be helpful, to explain to them that not only formulas are important, but also the stability of the computational algorithm, but they didn't even want to listen — the code passes all the tests they are given, so why complicate things? And such 'high abstractions' as condition number, were something they didn't even want to delve into: it was very scary, incomprehensible, and in practice, they didn't need it at all.
I think there are many firms on the market that can be called 'Horns & Code LLC'.
Moreover, before becoming a tutor (that is, before 2014), I worked at a company. We received funding for projects related in one way or another to the creation of fundamentally new types of weapons: swarms of UAVs that correct artillery fire. Back in 2012, there was a lot of talk about this, and some high-ranking military official even told us at the company that with this new weapon, Russia would be able to win the inevitable war with NATO in the 2020s–2030s. I worked there for only a year and a half, managing to work on the design of mechanical switches for microwave lines (replacing transistor ones with them would increase the UAV's resistance to electronic warfare) and antennas.
I Saw Enough
For one thing, the company mainly employed students from MIPT, and there was no one working in their field of study (I myself worked there from the end of my 5th year at MIPT; my master's specialty is the physics of fundamental interactions). This was because they hired people with absolutely no experience and paid very little. In fact, that's why I left: preparing children for the OGE and EGE exams is much more profitable, even if you are a beginner tutor charging minimum rates for home visits, as I was in 2014.
Or for another, that almost all other participants in this government procurement market were various firms like 'Horns & Code LLC,' against which even we looked like cool experts. Simply because we were MIPT students, we knew school and university mathematics, understood physics and computational methods, and wrote the code ourselves (rather than copying it from the internet). In other firms, the employees were mostly people who lacked even basic knowledge.
Now let's move on to analyzing the example of the AI's work that I generated this morning.
Supersonic Missile: What the AI Generated
I decided to use an LLM (I used Gemini and Claude) to solve the problem of modeling a missile's flight through the atmosphere during its active supersonic phase.
First, I discussed with the AI how this could be modeled in general — it allows for writing models of varying levels of detail and complexity. I settled on the following problem statement (a more complex one could be made, but this is sufficient for an illustrative article on Habr).
There is an atmosphere with altitude, temperature, pressure, density, and speed of sound. There are tabular coefficients for aerodynamic drag and lift as functions of the Mach number. There is engine thrust, which depends on fuel consumption and pressure difference. The missile's mass decreases linearly over time. The task is to numerically integrate a system of ODEs for velocity, trajectory angle, coordinates, angular velocity, and pitch angle. It is desirable to compare several integration methods. The output should be graphs of the trajectory, velocities, angles, and atmospheric parameters.
I can say that I didn't generate this in one go, but acted as an experienced vibe-coder (and I do have some experience). I broke the task into small steps, double-checked and tested each step with the LLM, asked clarifying questions, requested links and the source of its tables (and corrected some things), and gradually brought it to a working state.
In the end, the problem was solved. Inside, it has everything you'd expect from a good term paper. It also produced detailed flowcharts and even interactive diagrams (Claude can do that).
This is very convenient - there's not only the code in the notebook, but also a web page like this, a screenshot of which I've included, where you can click on buttons to open code snippets.
Here there are functions atm_T, atm_p, atm_rho, atm_a, which calculate standard atmosphere parameters based on geopotential altitude.
There are tabular arrays Mtable, Cxa, C_ya_alpha and interpolation of coefficients using interp1d, parabolic, and linear approximation.
There is a calculation for dynamic pressure q = 0.5 rho V**2, drag and lift forces, and moment about the axis. There are clean functions compute_dot_V, compute_dot_theta_c, compute_dot_x_c, compute_dot_y_c, compute_dot_omega_z, compute_dot_vartheta, each responsible for its own component of the system.
All of this is packaged into three integration methods. For example, Euler's method is implemented like this:
def euler_method(f, y0, t0, t_end, h, params): n = int(0.01 + (t_end - t0) / h) # для ровного значения шага t_values = np.linspace(t0, t_end, n + 1) y_values = np.zeros((n + 1, len(y0))) y_values[0] = y0 for i in range(n): y_values[i + 1] = y_values[i] + h * np.array( [f(t_values[i], y_values[i], params) for f in [f1, f2, f3, f4, f5, f6]] ) return t_values, y_values
There's a modified Euler and a Runge-Kutta with a similar style. Below that is a bunch of graphs: velocity vs. time, altitude vs. range, angles, Mach number, and everything else.
If you are a manager, a client, or just looking at it without the habit of analyzing numerical methods, it will most likely seem like a decent piece of engineering work. Especially since the code doesn't match the typical 'copy-paste from Stack Overflow' — it has an explicit atmosphere, manually entered tables, analytical formulas, and neat labels.
And this is where it gets interesting: let's look for bugs in this code. We'll turn off the 'vibe-coder' (who relies on crafting prompts, clarifications, and a feedback loop) and turn on the computational mathematics specialist.
What the LLM Itself Was Able to Find
To be fair, I gave this same notebook to the latest paid version of ChatGPT and asked it to 'analyze what's wrong.' At a high level, it did a pretty good job.
It noted that using the explicit Euler method for a stiff nonlinear system, where aerodynamic drag increases as (
), and the atmosphere changes significantly with altitude, is a risky solution, to put it mildly.
It said that for stability, one should either take a very small step h, or switch to more stable methods, or even use solve_ivp with an adaptive step. It noted that a step of h = 0.1 seconds for this type of problem looks suspiciously large and that a loss of stability is possible at low altitudes.
That is, at the level of general reasoning about numerical methods, the AI already 'understands' something. At least, it reproduces correct ideas from a good textbook. Moreover, I'll note that the errors found here are far from the simplest ones.
But when it comes to basic machine arithmetic, indices, types, and specific lines of code, the picture deteriorates sharply. What follows is what it didn't find fault with at all, but should have. For the sake of a clean experiment, I asked it about other errors several times, but it found nothing more — and there are many of these errors, and they are in elementary things.
The Error with int() and the Number of Steps
Let's look at the line:
n = int(0.01 + (t_end - t0) / h)
At first glance, this is a well-known 'trick': a small constant is added to the division result to compensate for representation errors in float, and then you take int, which acts as a floor. This is how many people wrote code for educational problems before proper libraries appeared.
Let's look at the actual numbers used in the problem. In the notebook, for example, it is set as:
t0 = 0.0 t_end = 4.23 h = 0.1
The machine representation of 4.23 / 0.1 is not a neat 42.3, but something like 42.300000000000004. After adding 0.01 we get 42.31000000000001. Converting to int we suddenly get 42.
Next, the time grid is constructed:
t_values = np.linspace(t0, t_end, n + 1)
That is, np.linspace(0.0, 4.23, 43) gives an array from 0.0 to 4.23 with a step of approximately 0.100714.... However, the integration in the loop proceeds with a step of exactly h = 0.1. The state y_values[i] corresponds to the time t = t0 + i * h, but we plot the graph using t_values, which runs over a completely different grid.
As a result, the last point according to t_values is 4.23 seconds, while the physical state corresponding to it is 4.2 seconds (42 steps of 0.1).
A small difference, but a systematic one. In a toy educational problem, it's fine. In a real task where, for example, you want to analyze values at the exact moment of engine shutdown, this is already a serious desynchronization. And the paid ChatGPT missed this. It spoke willingly about the method's stability, but not about float and int().
This is where the problem is very clear. It reproduces mathematically 'complex' text. 'Boring' floating-point arithmetic — it does not.
Inconsistent Time Grid
The error with int() leads us to the next one. The time grid is created using np.linspace, while the integration proceeds with a fixed step h. The idea, judging by the code, was to get an even number of steps and neatly plot the values.
The correct approach would be: either we don't store t_values at all, and in each right-hand side we calculate the time as t0 + i h, or, if we really want an array, we build the grid with a direct loop t[i] = t0 + i h and don't 'stretch' it anywhere.
But the code ends up with a hybrid: we calculate values at certain moments in time and plot them as if they were at others. Formally, the difference is a fraction of a percent.
But what's important is that it appears not as a conscious approximation, but as a side effect of a clumsy manipulation with int() and linspace. Such errors are later hunted down in logs for a long time (and sometimes never found!) if something suddenly doesn't match the experiment.
Pseudo-abstraction: The f Parameter That Doesn't Work
Further in the code, we have Euler's method and its declared 'universality':
def euler_method(f, y0, t0, t_end, h, params): ... for i in range(n): y_values[i + 1] = y_values[i] + h * np.array( [f(t_values[i], y_values[i], params) for f in [f1, f2, f3, f4, f5, f6]] )
The signature suggests that you can pass any system of ODEs to euler_method via the f argument. Inside, however, this f is immediately shadowed in the list comprehension for f in [f1, f2, f3, f4, f5, f6]. As a result, the actual system is hardcoded into the global f1...f6, and the f parameter is just a prop that is never used.
At an architectural level, this is an important symptom. The LLM sees many examples where a numerical method takes a right-hand side function and copies that pattern. But it's incapable of checking whether the argument is actually used in the calculation. The result is a pseudo-abstraction: it looks nice and flexible, but inside, everything is hardwired to a specific global implementation.
Silent Loss of Precision in Tables and Pseudo-rounding
In the code fragment that prepares tabular data for output, there's a trick:
P_table_01 = np.array([0.0] * L_01 ) ... for i in range(len(y_euler_01)): massa_table_01[i] = massa(m_0, dot_m, t_euler[i]) P_table_01[i] = calculate_P(calculate_P0(dot_m), S_a, atm_p(0), atm_p(method[i, 3])) P_table_01[i] = int((P_table_01 // 1)[i] + ((P_table_01[i] % 1) // 0.5))
At first glance, the LLM wanted to get neat thrust values in the table, rounded to 0.5 N. But the mechanism chosen is very strange.
First, where the array was created in the original version using [0], numpy made it an integer array, and then the results of calculate_P, which are floating-point numbers, were written to it. The fractional part was silently lost.
In the fragment shown, I have already corrected it to [0.0], so that at least the type is explicitly float, but the AI didn't do this in the original source.
Second, the pseudo-rounding formula itself:
int((P_table_01 // 1)[i] + ((P_table_01[i] % 1) // 0.5))
If P_table_01 is an integer, then P_table_01 // 1 gives the same thing, P_table_01[i] % 1 is always 0, and this whole construction turns into int(P_table_01[i]).
In other words, the person who generated all this only needed to run this fragment by hand once and look at the result to understand that there is no rounding happening. But what percentage of vibe-coders would think to do that?
In the actual code, this line does exactly zero useful work, only confusing the reader. Again, not a tragedy in itself, but it's telling: the LLM generated a 'rounding-like' template without checking if it actually rounds anything.
Aerodynamics Disabled with a Single Line
A special gem is the handling of the angle of attack:
def alpha(y): return 0 # случай идеального управления
Later, the angle of attack is used in the calculation of lift and moment:
def calculate_Ya(M, alpha, y, S_m, V): return C_ya_alpha(M) * alpha * S_m * calculate_q(y, V) def calculate_Mz(Y_a, X_a, alpha, l_d): return -(Y_a * np.cos(alpha) + X_a * np.sin(alpha)) * l_d
Since alpha(y) always returns zero, our lift is zeroed out, the moment disappears, and all angular motion in the model becomes a decorative add-on. Meanwhile, in another place, beautiful graphs of the angle of attack, pitch angle, and angular velocity are plotted. It's clear from the code that the LLM wanted to link alpha with the difference in angles vartheta - theta_c, but at some point replaced it with the constant 'ideal control case' and left it that way.
As a result, we have a missile that flies like a perfectly stabilized slug, while we honestly plot graphs supposedly reflecting its angular behavior. One simple line disables an entire subsystem of equations.
Incorrect 'Velocity Vector Angle'
Another characteristic detail concerns the visualization of the velocity vector angle.
The notebook has a function:
def get_speed_vector_angle(y): return radians_to_degrees(np.arctan2(y[:, 3], y[:, 2])) # arctan(y_c / x_c)
The comment says that the inclination angle of the velocity vector is calculated here. In reality, it takes the coordinates x and y and calculates arctan2(y, x), which is the angle of the radius vector from the origin to the current position. This has no more to do with the velocity vector than atan2(latitude, longitude) has to do with the wind direction.
The correct velocity angle should be calculated from the derivatives of the coordinates, i.e., from (dx/dt, dy/dt). In some parts of the code, the author has the corresponding quantities (dot_x, dot_y), but in the final picture, 'whatever is simpler' is used instead, just to have a graph.
And this, again, is not an error at the level of 'complex physics,' but at the level of 'put the arctangent arguments in the wrong place'.
What All This Adds Up To
If you put it all together, the picture is this. The AI generated a substantial framework for the simulation: atmosphere, interpolation of tabular coefficients, aerodynamic forces and moments, three numerical methods, and visualization of the results. At the level of structure and formatting, the code gives the impression of a serious engineering work, even in compliance with GOST standards.
When this same AI was asked to review the code, it honestly found general problems: the choice of the explicit Euler method, a large integration step, the lack of an adaptive step.
On paper, it all looks even smarter: you might even think, 'wow, it already understands numerical methods.' But the very same AI paid no attention to the fact that:
the number of integration steps is calculated using
int()on a floating-point division, causing the last time point not to correspond to the final state of the system;the time grid is built using
np.linspace, which yields a different step size than the one actually used to integrate the equations;the
fargument ineuler_methodis excluded from use, and the system is hardwired to the globalf1...f6, despite the nice signature;the table with flight parameters loses the fractional part of values due to implicit type conversion and uses pseudo-rounding that doesn't round anything;
the angle of attack is hardcoded to zero, which means the angle-dependent aerodynamics are simply turned off;
the velocity vector angle on the graph is actually the angle of the radius vector from the origin.
possibly (and probably) something else that I didn't notice during a quick assessment
Each of these bugs lives in one or two lines of code. Each of these lines looks 'simple.' And each one, on its own, is capable of completely devaluing all the effort.
To avoid making unsubstantiated claims, I ran this code and compared it with a slightly more correct solution (I used the Runge-Kutta method with an adaptive step and removed the errors described above from the code using a prompt). The result on the graph below speaks for itself:
Where Else AI Consistently Fails at the Simple Stuff
The missile story is not unique. Exactly the same patterns emerge in other tasks.
If you ask an LLM to write a binary search manually, it will very often mix up strict and non-strict comparisons, incorrectly update low and high, leave one element unchecked, or get the loop stuck in an infinite execution in one of the edge cases. By signature and structure, it will be exactly the binary search you've seen in textbooks. But the boundary conditions, where '<' should be replaced with '<=', end up in the wrong place.
With floating-point numbers, it's even more fun. The construct if a == b: for calculation results appears in generated code with enviable regularity. Yes, you can see this in live projects. Yes, the world somehow hasn't burned down yet. But that's no reason to artificially multiply such instances in someone else's code. Proper comparison using a tolerance, working with scales, normalization — this is again that same 'boring math' that the neural network pays no attention to.
AI loves to build SQL queries using f-strings. The logic will be honest: the query is formed, parameters are substituted, results are read. The fact that such a construction throws the door wide open for injections doesn't bother the model. It sees a thousand examples of code with string substitution and does it 'like everyone else.' Parameterized queries, prepared statements — that's a different, less representative sample.
Array boundaries, empty containers, mixed-up radians and degrees, off-by-one errors in loops — all of this constantly pops up in generated code. Including in places where any experienced human programmer would spot it immediately, simply because they've seen the same bug a hundred times in a real system.
What Follows from This
Phrases like 'let the AI write the routine stuff, and we'll handle the architecture' sound reassuring. In reality, the opposite is happening for now.
The architectural facade — modules, classes, dataclasses, separation of concerns — is something LLMs already generate quite decently. Usually, much more decently than a live junior who has been copy-pasting everything from Stack Overflow for a year and a half.
But where the 'simple' stuff begins — a specific value of dt, the loop exit condition, floating-point number comparison, calculating the number of steps, preserving invariants like 'mass is always positive, energy does not increase in a conservative system' — this is precisely where neural networks continue to miss the mark time and again. Meanwhile, the volume of code grows, and trust in each individual line falls.
The fact that the latest paid version of ChatGPT was able to find problems in its own code at the level of choosing a numerical scheme is good. But the fact that it simultaneously missed a trivial error with int() from division and an inconsistent time grid is much more telling. It is learning to talk like a good numerical methods professor, but it has not learned to be a meticulous executor at the level of specific code.
In practice, I draw four simple conclusions from this.
First, AI cannot be considered the author of the code. It is a draft generator. The architectural framework, wrappers around libraries, dead-boring boilerplate — yes.
Everything involving mathematics, physics, or discrete logic must be checked exactly as if it were written by a person completely illiterate in all these things.
Second, you cannot delegate 'simple' things to AI.
On the contrary, it is precisely the simple things — steps, indices, types, invariants — that will have to be checked especially carefully. It will draw the architecture for you. But it will calculate dt using int() from a division and be off by 0.03 seconds.
Third, juniors still need to learn the fundamentals, not just 'how to correctly formulate a prompt for a neural network.' If a person doesn't understand the difference between the explicit Euler method and Runge-Kutta, or what numerical stability is, they won't see the problems in the integration scheme, the discrepancy in time grids, or the disabled pieces of aerodynamics. And they need to learn the fundamentals especially well, starting from the very basic things, like errors due to the peculiarities of machine arithmetic.
Fourth, the fact that AI is sometimes able to find its own mistakes does not make it safe. It is a useful tool for a second opinion. But the responsibility for ensuring that the missile in your model doesn't turn into a numerical firework still rests on you.
The AI will indeed sketch out the architecture. But it will lose the dt in the integral, calculate the step strangely, mix up the angle, and compare floats using ==. Check the 'simple' stuff. That's where the most expensive bugs live now.
The rule of thumb for trusting AI: the more primitive the operation (addition, loop, condition), the higher the probability that the LLM messed up there. Assume the code is written by a philosophy professor who skipped arithmetic in elementary school.
The entire source Jupyter notebook with the code I'm talking about here is available at this link.
With the help of the LLM, I also compiled a description of the algorithms and conclusions there, so that it looks like what is happening is more or less clearly described.
I ask everyone who has read this to write their personal examples of finding similar errors from LLMs in the comments. You can also dig around in the notebook that I quickly vibe-coded. You will surely find some more errors there.