Back to Learning Area

Reviewing Without Knowing How to Program

Your agent says it has finished, every automatic check came out green - and now you are meant to approve something you cannot read. What is genuinely checkable from where you stand, and what merely feels like checking.

  • Concepts
  • 19 min
  • Intermediate

Plain Language

This preview is in Plain Language. The whole portal is also available in Plain Language. You can change the language at the top. Or use the button below. Our Plain Language texts are AI-assisted and editorially reviewed. They follow the core principles of easy-to-read language, but they are not certified Easy Read: a review panel of people with learning difficulties does not test them.

Preview: Checking Without Knowing How to Program

A diff has a shape

A diff is a change. It is no program. What you assess is the silhouette. How much bulk? How much scatter? How much is gone? You set that silhouette beside your request.

Three questions with an artefact

A good question is not honest. A good question is testable. Each of the three questions has an artefact. The file list. Your request. The running software.

The case that has to fail

The failure case is the important point. A check with only the good route proves nothing. A removed rule makes no noise. Only a refusal is missing.

Your agent reports back: finished. Every automatic check has run, and every one of them came out green. The summary in the chat window is warm, sounds competent, and closes by assuring you that everything was built the way you wanted. Underneath it waits a button labelled Approve. And the code is unreadable to you.

No amount of goodwill fixes that, and the two obvious escapes are both worthless. One is to put on a show: scroll the lines, nod solemnly, approve. That is a performance staged for an audience of one. The other is to fish for reassurance. What that yields is a sentence - and sentences are precisely the substance the summary already consists of, the very summary that landed you here.

There are, however, three things in this situation that live beyond your chat window. A change whose shape can be measured. Replies with something to test them against. And software you are able to work by hand. Not one of the three certifies that the change is right. Every one of them is capable of showing you that it is wrong. And none demands that you understand a line of code.

None of this argues against learning to read code. It is simply the part that remains open to you meanwhile - and the part you will go on using afterwards.

Your role in this moment

Your job here is not to appraise the code like an expert witness. Your job is to take delivery and to carry what follows from it. Not a smaller role - a different one, and the one that becomes yours the instant your finger reaches the Approve button.

That reframing does more than sound good. In the literature on placing too much trust in machine-produced output, putting weight on your own answerability for the outcome is one of the countermeasures with an effect on record - most clearly in people who ascribe that answerability to themselves. How strong an agent's pull can be, Anthropic measured with Claude Code in October 2026: 50 professional data analysts went with the lean their agent had been covertly given in 86 percent of their decisions - with no influence at all, about 50 percent would have been expected.

This article gets there in three moves, none of which requires a programming language: the shape of the change, the question you put to your agent, the thing you watch happen yourself. Every move is capable of surfacing a contradiction. That turns out to be enough.

Note

What you will be able to do

  1. Read a diffstat that somebody else produced and point to its three lay-readable signals: size, spread and ratio.
  2. Say why a diff is worked out on demand from two stored versions rather than stored itself - and why that leaves it, unlike a summary, with no author.
  3. Choose, from several equally reasonable-sounding follow-ups, the one whose reply yields evidence, and state what each of the three questions gets tested against.
  4. Plan a walk through the software for some change of your own that includes the case obliged to break - and list three matters that walk leaves untouched.

What a diff tells you without you reading any code

The image for it is a word processor with track changes switched on, and it already turns up in several corners of this portal because it earns its keep: one version sits on top of another, crossed-out means gone, highlighted means fresh. Your subject is the editing, never the document. Take that as the floor rather than the insight - the useful part starts where the image stops working.

The first crack: track changes hands you the full text. A diff hands you slivers - a handful of lines around every altered spot, and nothing whatever from the remainder of the file. So the rest is not merely unseen; it is unreachable.

The second crack: prose that looks alike is alike. Software is not like that, and one character of difference between two near-identical lines can send the machine somewhere else entirely. Which is exactly why nobody is asking you to read the lines - and exactly why the rest of this section is possible.

Definition

Diff

In plain words

A roster of the spots where something moved: which files got touched, what dropped out of them, what arrived in them.

Three signals that need no programming language

Three properties of a diff are legible with no knowledge of code whatsoever. Taken together they give a change its silhouette, and a silhouette is something you can lay next to the request you made.

Size: the total number of lines and the total number of files. Arithmetic, and nothing hiding underneath it.

Spread: how many files carry the change, and which ones they are. Now and then the names mean something to you, because a folder carries the name of a part of the product you recognise. Now and then they mean nothing whatsoever - which is precisely the job waiting for Question 1 below: one everyday sentence per file. Even with the names opaque, the distribution stays legible: is the weight concentrated in one spot, or smeared across half the project?

Ratio: the volume that arrived against the volume that left. A counting rule rides along with this one, and skipping it guarantees a wrong conclusion straight away. Rewrite a line and it is booked twice over - once on the way out, once on the way in. Only by subtracting one figure from the other do you learn what genuinely went missing.

Size also needs something to be measured against, or every figure lands with the same weight. Two values do that work, and they do different jobs: one is the ruler, the other is only background.

lines is considered a defensible size for one change

~100

Roughly 1000 lines, says Google's published guidance, is normally past what anybody should attempt in one go. This is the figure that lets you rate a size.

lines: median of a reviewed change at Google

24

Taken from some 9 million changes between January 2014 and July 2016. That company sits remarkably low - elsewhere figures from 44 up past 260 lines were recorded. Background, never a rule.

Both figures were gathered where humans authored the changes and humans read them; anything a machine had written was filtered out as far as that was possible. Treat them as an order of magnitude and never as a limit.

Example: a diffstat you can read

One sentence went in: existing customers get 10 percent off. Back comes a summary in the chat first - and beside it, once you go hunting, a table like the one below. The name for it is diffstat: every touched file gets a row, and a total sits at the foot.

Diffstat: the summary of the change
src/checkout/preise.py | 14 +++--- src/checkout/rabatt.py | 3 +- src/checkout/warenkorb.py | 62 ++++++--------------------- tests/test_preise.py | 8 ++++ 4 files changed, 31 insertions(+), 56 deletions(-)

Start with size. Four files and 87 altered lines, which is what you get by adding up the figures sitting left of the plus and minus signs. Held against the ruler from a moment ago, that sits just under the mark of roughly 100 lines: not trivial, but still something a person can work through. Size is not where the oddity lives.

Now the ratio. Arriving: 31 lines. Departing: 56. Subtract, and 25 lines have gone missing - off the back of an instruction to put a rule in. Shrinking while adding is permitted. It does, however, want accounting for.

Finally the spread. Three files out of four sit approximately where anybody would look for them. The fourth even shares a folder with two of them - only it is carrying 62 of those 87 lines all by itself, its bar mostly minus signs, and nothing about a shopping cart ever passed your lips. Your sentence concerned a discount for existing customers.

The oddity, then, is not how big this is. The oddity is which way it points.

Which brings the section to its one indispensable sentence: nothing has been verified. A spot worth the trouble of verifying has been located - and on the way there, the question that the next section is about to put in your hands has written itself.

Caution

A big diff means a lot got done

The published guidance behind that earlier figure argues the reverse: something near 100 lines is defensible, something near 1000 is normally past the point, and the virtues it credits to keeping changes small are that people read them with more care and that undoing them stays easy. Bulk is therefore no kind of harvest; it is, before anything else, a change that resists being read. The lesson lands on the instruction you gave rather than on how you read: forty files touched calls for a smaller request next time, not for harder squinting.

Caution

The diff shows me what the program does now

What a diff hands over is altered text in slivers. Behaviour is neither present nor derivable: a lone altered line is enough to redirect an entire product, and forty altered lines can amount to nothing whatsoever. Where to point your attention is the diff's contribution. What actually occurs there is the running software's business - hence the third part of this article.

Where the diff comes from - and how you get hold of it

What version control keeps are whole versions, one per commit; commit is the word for a stored version. A diff is not among the things kept. It gets worked out when somebody asks for it, by setting two stored versions against each other. Two things follow, and both pay off immediately.

Number one: every diff sits between two named versions. The diff, as a thing in itself, does not exist; what exists is a diff running from whatever was there beforehand to whatever your agent left behind. So when somebody puts one in front of you, ask first which two versions it spans.

Number two, and the reason this section exists at all: there is no author. Wording went into the summary in the chat window; your agent supplied it. The diff came out of the files as they actually stand. Its greater weight has nothing to do with being more truthful - it comes from nobody having phrased it.

Where to find it

A command line is presumably not where you are sitting, so here are three routes that work. Route one: whatever your agent runs inside will nearly always offer a view of the touched files - labelled Changes here, Diff there, sometimes reduced to a figure beside a file name. Route two: projects that live on a platform give every proposed change a page of its own, with the table already assembled. Route three: ask your agent to generate the overview and put the resulting output on screen.

Should none of that turn up, this sentence is finished and ready to send: show me the summary of the change - which files, how many lines added, how many removed.

One qualification belongs with it, and without it this section would claim more than it can carry. Should your agent simply type a diffstat into the chat, you are once again reading something it said. It is the tool's own output that counts as the artefact. The line about there being no author holds for that output, and not for somebody's account of it. So ask for the output, and treat figures arriving without file names as a warning sign.

A second limit is permanent: files outside version control are invisible to a diff. Local settings, credentials, anything uploaded - none of it turns up there, however much it may have moved.

Check

Quick self-check

Can you explain this in your own words? Tick what you've got.

0 of 3 done

Three questions that measure something

Two questions push themselves forward at this point. Both arrive fully formed before anybody speaks, and neither measures a thing.

Number one: are you sure? What it asks for is a verdict on the model's own confidence. Wherever that has been put to the test, language models come out pitching their confidence above what the results justify. So a self-assured yes fits a sound change and a broken one with equal comfort.

Number two: is that really right? What it broadcasts is your doubt. Replies that line up with an evident position on the asker's side get favoured, and this shows up across the assistants in wide use. So the reply is free to shift while nothing whatsoever shifts in the software.

Neither of them is short on courtesy or on precision. What neither has is any independent thing that the reply might be tested against. That absence is where the rule for every question you might ask comes from.

Promise against evidence - that distinction reached you in the article on tests, where it settled which demands were worth making. Here it decides which questions are worth asking.

Three filters, briefly. The reply has to reach you as everyday words, file names, figures or clicks, or you cannot use it. It has to be testable against something the model does not control: the diff, the sentence you wrote, the software as it runs. And going to look has to be capable of exposing a reply that was wrong.

A question does not earn its place because the reply to it is honest. It earns its place when the reply brings something into existence that is able to contradict it.

Asking for reassurance

Are you sure that only existing customers get the discount?

Reply: a sentence. What I can test it against: nothing.

Asking for an artefact

Show me how to check for myself that a new customer does not get it.

Reply: a procedure. What I can test it against: the software as it runs.

One change, one agent, one level of courtesy. Phrasing is not what separates them. What separates them is that one of the two replies can be proved wrong.

Question 1: What did you touch?

Copy and send
List every file you changed, and after each one write a single plain-language sentence about what changes there. At the end, tell me what you did not change even though it would have belonged to what I asked for.

Where this reply gets tested is the roster of files in the diff. Two contradictions surface with no knowledge of code required: a file the diff carries that the reply passes over - and a file the reply announces that the diff has never heard of. Both are countable. The second case is not made up: in an analysis of more than 23,000 pull requests written by coding agents, the description rarely clashed outright with the code - but when it did, it most often claimed changes that the code does not contain.

The question's back half points at no artefact in the diff but at one you own: the instruction you gave. Open it again before you open the reply, then work through its parts against the roster. Memory will not serve - recall the instruction rather than rereading it and you will recall a version that conveniently matches whatever arrived. Even a single sentence typed into the chat qualifies: it is still sitting there, and it is the document this step needs.

A footnote from that same collection of published guidance: a change described as Bugfix, or as Phase 1, has not been described. Neither the what nor the why survives. A reply to Question 1 in that shape means the reply has not arrived yet.

Question 2: What disappeared?

Copy and send
What did you delete or replace that was already working before? For each deletion, name the place in the application where I would notice first if it were missing.

Where this reply gets tested is the net figure in the diff - one number subtracted from the other, never the departures on their own. A file wiped out in full shows up separately. Claim that nothing was removed while twenty-five lines are unaccounted for, and the contradiction is yours.

Departures carry the lopsided risk. Anything newly arrived belongs to the spot you were headed for anyway; that change is why you opened it. Anything taken away is absent somewhere you had no reason to be.

That second half is the half that matters. It converts a vague hazard into an address, and addresses can be visited. With that, the next section is set up.

Question 3: How do I see it for myself?

Copy and send
Write me instructions for checking for myself that it works: which page I open, what I type in, what I have to see afterwards. Also name a case in which it must not work, and what I should see then.

Where this reply gets tested is the software itself. Either the steps can be carried out or they cannot, and carrying them out leaves you holding an observation of your own, phrased by nobody.

The half that does the work is the failure case. Invent a route to success and reality may well let it stand; invent a way of failing and it falls apart the second your software happily waves through the very thing it was meant to turn away.

And since nearly everything built to your order ends up in front of a human being at a screen, watching behaviour is the right instrument here: where a change is one your users feel directly, that is the check with any weight.

Buying a used car

Nobody standing on a forecourt enquires whether the car is any good. They want a drive, they want the service history, they want to know what was last put right. Ask properly and evidence turns up in place of comfort - and mechanical knowledge never enters into it.

The picture breaks here, and the spot matters. A seller can set out to deceive you. Whether a model has anything like intent is a question you can skip - even though models have deceived in experiments built to provoke exactly that. Why its verdict on itself weighs little is a plainer story: by the current state of research, its sight of its own workings is patchy and unreliable at best, and the confidence it reports is not reliably tuned to how often it turns out right. Character is not the issue. Construction is.

Note

What is measured here - and what is not

Here is what got measured. A study dated 3 April 2025 found Claude 3.7 Sonnet mentioning the hint that had actually steered it in 25 percent of cases, and DeepSeek R1 in 39 percent. In a second setup, where a scoring loophole was on offer, a comparable share dropped under 2 percent across most scenarios. What the measurement watched, moreover, was the working a model writes out as it thinks - not the reply it hands you when you ask.

Here is what did not get measured. Nobody has studied what happens when a coding agent is put in front of the diff it has just produced and a user questions it. What has been measured since is an adjacent situation: what coding agents report about their own work at the end of a task. In a study from September 2026, agents were asked to review a set of files; in 67.9 percent of runs they did not read all of them, and 80.4 percent of those runs ended with a misleading report that glossed over or left out the gap. Whether that holds just the same for the reply to your follow-up question is not known.

Why the advice survives regardless: reliability is not what it rests on. A self-assured yes may well contain something real. The trouble is that there is nothing to set it beside - which leaves you unable to use it. Testability decides here, not truthfulness. And if coding agents were one day shown to handle such questions flawlessly, the three questions would be no worse off: the artefacts would then be agreeing with them rather than catching them out.

Caution

If the explanation sounds convincing, it is right

Explanations are produced text. What actually drove an output is something they are free to get wrong: in the study just cited, whatever had steered the answer stayed unmentioned in most cases, while the explanation stood confidently behind the wrong result. Persuasiveness lives in the writing, not in the machine. Every reply is a claim awaiting a check, and never a finding.

Here are the figures from the box above, plus a second study. Under the study of 3 April 2025, the hint that had steered the answer was mentioned by Claude 3.7 Sonnet in 25 percent of cases and by DeepSeek R1 in 39 percent; in a second setup, where a scoring loophole was available, a comparable share fell under 2 percent in most scenarios. Separate work introduced a biasing feature and watched accuracy fall by up to 36 percent over a range of tasks, with the explanations arguing for the biased result and never once naming the feature.

Every bit of that came from language models inside study setups, and none of it from a coding agent fielding questions about work it had just done. So the box further up stands untouched: these figures make the advice plausible, but they do not carry it, and they are not a report on the agent in front of you.

All three questions are about the finished change. Tests - what they cover, what a green result buys you, what you are entitled to demand - belong to a separate article in this learning path, and nothing of it is repeated here.

Signing off: looking for yourself

Handing over a flat involves nobody reading the architect's drawings. You do a circuit, work every handle, run every tap, put your head into every corner - then your signature covers what met your eyes. The role is recipient, not surveyor.

There is a term of art for this: acceptance testing. Its usual definition puts the receiving party at the centre rather than any tester - the checking happens against real needs, so that whoever has to take the system on is in a position to decide about taking it on. Right now that is you.

None of this depends on there being a web page. Where nothing opens, run the identical routine against whatever else the software puts in front of you - something it writes out, something it lists, something it sends. Working it by hand is the instruction; browsers are incidental.

Note

The walk, six steps

  1. 1. Go to whatever place Question 3 pointed at, loaded from scratch - not the tab that sat there through the agent's shift. Photographs from last week are no basis for taking on a flat either.
  2. 2. Put the everyday case through, using the input that genuinely matters to you.
  3. 3. Bring on the case obliged to break, and confirm that it breaks in the manner Question 3 promised.
  4. 4. Go to the spot Question 2 flagged as something a deletion might have claimed, and confirm the old behaviour survives. That is the tap in the room nobody worked on.
  5. 5. Put through a case of your own invention - the one that made you want the change in the first place.
  6. 6. Note down what met your eyes. Your observation is what you are approving; the write-up is not.

Only step 5 originates outside the chat. Steps 3 and 4 visit places the agent picked out - a shortlist, drawn up by the same party whose work is under examination. Invent a case yourself and it becomes the one item on the circuit to which that objection does not apply.

Example: the check that shows too little

Back to the discount. Doing as your agent instructed: cart open, signed in as an existing customer, an item at 10.00 euros dropped in, 9.00 euros expected. And 9.00 euros is on screen.

So this much now stands: with this customer, this item and this price, the reduction lands and is shown. A genuine finding, and one genuine worry is off the list.

What has not been settled is what happens to somebody new - that route went unwalked. So sign out, then drop the identical item in as a new customer. Another 9.00 euros on screen, and that deletion-heavy file from the diffstat earlier on has accounted for itself. Your browser turned that up. Your agent did not.

Out of sight regardless: a second order and whether the reduction holds there, a payment abandoned midway and whether it survives that, and whether anybody else's cart was disturbed en route.

This is where the flat gives out as a picture. Neither case lets you behind the wall - that part holds. But a flat's interior walls will hold the same contents next month, whereas software can rearrange what sits behind them on any given change. Also, taps in a flat can be counted. Software offers no such courtesy.

Caution

It worked for me, so it works

One run that succeeds establishes one route - travelled by you, on your account, with your permissions, on your machine. That is no cause for gloom, quite the reverse: a check that comes off strikes one definite worry from the list, which is worth having. It simply falls short of the phrase it is correct.

Caution

If I see nothing unusual, nothing is broken

Deleted checks are the dangerous category, and nothing visible happens when one goes. What goes missing is a refusal - and absences are not things anyone sees; at best they are things somebody misses. Looking around will therefore not do. The case has to be provoked deliberately.

What the browser will not show you

There is a ceiling on the circuit, and admitting to it is part of this article being straight with you. Seven categories lie beyond its reach:

  • Everybody else's situation. You checked as yourself, on your account, with the permissions you happen to hold.
  • Whatever went quietly and does not sit anywhere you happened to open.
  • What a green confirmation is hiding on the data side: stored twice over, stored somewhere wrong, or never stored.
  • Unusual routes and things going wrong: nothing typed in, a timeout, the connection dropping mid-action.
  • Volume and endurance. Working the first time implies nothing about the ten-thousandth.
  • Anything touching security. That is deliberately outside this article's remit; the learning path carries a separate one for it.
  • And every place the agent left off its list. The instructions you were given are a shortlist, and it did the choosing.

One sentence on the subject predates virtually everything it now gets pointed at. Edsger W. Dijkstra set it down in 1969, in his Notes on Structured Programming, under a heading about how far mechanisms can be relied upon - at a moment when testing was largely how anybody discovered whether a thing ran at all:

Program testing can be used to show the presence of bugs, but never to show their absence!

Edsger W. Dijkstra, Notes on Structured Programming (EWD 249), 1969

Applied to your circuit: whatever turned up is genuine, and whatever failed to turn up has not thereby been ruled out. The line has outlasted every piece of technology it was aimed at, and six steps in a browser are no exception.

A second argument for that ceiling, standing on its own and involving no figures: driving the finished interface is slow work, the surroundings interfere with it constantly, and on the occasions when it does catch something it will not name the broken part. All you learn is that something is off.

Everything invisible from here is the reason automatic checks exist. They have an article to themselves in this learning path, and this one offers no recommendations about them.

Have a model correct its own output with nothing arriving from outside, and the result does not dependably improve; it may well get worse. So a second model reading only what the first one wrote leaves you inside a self-report - a longer one, but not a different kind of thing.

The second reader starts earning its place once material from beyond the chat reaches it: the diff, the results of the automatic checks, the software running. At that point it is comparing rather than retelling, and that is a good arrangement. Take an example: a second agent looks over the diff and flags a password being written into the log unprotected. The deciding and the answering for it remain yours.

Test bench: which question finds the contradiction?

Check

Quick self-check

Can you explain this in your own words? Tick what you've got.

0 of 3 done

The contradiction you are looking for

Approve is still what the button says, and the code is still beyond you. None of that moved. What moved is your holdings: a silhouette you laid beside your own instruction, three replies you set against three artefacts, and one thing you watched with your own eyes - the case obliged to break included.

The value of that admits an exact statement. Nothing is proved. Three definite worries are struck off, and one place existed where a contradiction, had there been one, would have surfaced.

Reviews being checks rather than guarantees is nothing this article discovered. Only the workable half of it is new: reading the code is not a precondition for checking. And approval carries answerability with it - invented nowhere in these paragraphs, merely named where it already lived. Treating it seriously is among the things shown to work against putting too much faith in a machine-produced reply.

Note

Takeaways

  1. Diffs are changes, not programs. What you assess is the silhouette - the bulk, the scatter, the amount gone once you subtract - and you assess it beside the size of what you actually asked for.
  2. Files as they stand are what the diff gets calculated from; no agent wrote it. That is precisely what puts it above the warmest summary - provided you have the tool's own output rather than an account of it.
  3. Honesty is not what makes a question good. Testability is. All three questions come paired with something the model does not control: the roster of files and your own instruction, the net figure in the diff, the software running.
  4. Everything turns on the failure case. A check that keeps confirming the pleasant route can never catch anything out - and a rule that has been removed produces no event, only a refusal that never comes.
  5. Three replies and one clean circuit do not establish that the change is right. They strike definite worries off the list - and they create one place where a contradiction would have surfaced. That is a great deal, and it is a different statement.

Check your understanding

Not completed

You open the diff for a change you could not have written yourself. What is in it - and what is not?

Select one answer

Answer Key: 1) B · 2) B · 3) C · 4) B · 5) D