Unreadable AI Code Still Needs a Human Owner

Futurism reports that Jeremy Nixon, founder of chip optimization software company Infinity, described engineers struggling to understand AI-generated code. The report also cites SemiAnalysis researcher Jordan Nanos describing OpenAI engineers reviewing a graphics processing unit kernel associated with DeepSeek. Nanos said the engineers understood the hardware but could not explain what the kernel did line by line. He also said the generated kernels produced correct results and performed well.

Those observations identify a handoff problem, not proof that every confusing program is defective. Organizations can accept generated software faster than employees can inspect it. A benchmark supplies a result. Deployment assigns consequences to customers, operators, and the employee who answers the incident call. The machine wrote the code; accountability did not compile itself.

Unfamiliar syntax and unverifiable behavior are different problems. A specialist may understand code that another competent engineer finds opaque. Tests can compare an optimized kernel against a trusted implementation across specified inputs. Nanos’s account deserves credit for reporting successful results, but the supplied report does not establish the breadth of those tests. Correctness on tested cases is useful evidence, not an unlimited warranty.

Formal verification can establish specified properties when the code and the verification method permit it. That is an available engineering approach, not a safeguard documented in this report. Tests and proofs also need explicit requirements. If the requirement omits an important condition, verification can certify the wrong promise with considerable precision. Asking the generating model to approve its own output does not create independent scrutiny.

Maintenance changes the question. Someone must explain the program’s inputs, assumptions, dependencies, and failure modes well enough to modify or replace it. That does not require every employee to decode every instruction. It does require a capable owner and documentation that survives beyond the chat session. Otherwise, the next hardware change or bug report starts a fresh guessing exercise.

Security review needs its own evidence. Fast execution does not establish safe memory access, appropriate permissions, or careful handling of sensitive data. Futurism also mentions Amazon’s 90-day code safety reset after outages disrupted customer orders. The excerpt does not establish that unreadable AI-generated code caused those outages. The example illustrates the cost of software failure without resolving that causal question.

Engineering leaders can set acceptance standards before deployment: a named maintainer, documented requirements, independent checks proportionate to the risk, a security review, and a tested rollback path. Leaders can also restrict opaque components to uses where operators can detect failure and replace the component safely. If a team cannot explain how it will recognize a bad result or stop the software, the next available decision is to withhold deployment.