Better Questions, Not Better Prompts¶
The advice for getting more out of AI has been remarkably consistent for a few years now. Be specific. Give it context. Define the role. State the constraints. Say how you want the answer formatted. All of that genuinely helps, and I still do most of it.
But when I look back at the sessions where an AI coding assistant was actually useful — where it found the real problem rather than a plausible one — almost none of them turned on the opening prompt. They turned on what I asked next.
I'm starting to think we've been naming the wrong skill.
AI doesn't have a knowledge problem¶
Modern coding models can inspect an enormous amount of a system — searching repositories, tracing call paths, reading tests, inspecting configuration, running commands, comparing competing implementations. Access to all of that isn't the same as knowing which part of it matters, and on a large enough problem there are dozens of plausible directions to investigate.
I've watched a model chase the failing exception when the real defect was an invalid state transition three steps upstream. I've watched one optimize a query when the more useful question was why that service owned the data at all. I've watched one work diligently through every comment a reviewer left without ever asking whether the reviewer's assumptions held.
In none of those cases was the model incapable of solving the problem. It just never looked at the part of the system where the answer was.
Its attention was pointed in the wrong direction.
When that happens, a longer prompt rarely rescues it. Adding more instruction to a misdirected investigation mostly produces a more thorough misdirected investigation. What changes the outcome is a question.
Instructions control actions. Questions control attention.¶
This is the distinction I keep coming back to. Instructions tell a model what to do — change this method, add a test, refactor this class, fix the exception. They're about action, and models follow them well. Questions do something different. They tell the model what deserves thought before any action is taken at all.
Compare asking why is this broken? with asking who is supposed to be authoritative for this state? The second question doesn't contain the answer. It doesn't even assert that authority is the problem. But it redirects the entire investigation before a line of code is read, because now the model is looking for an owner rather than a stack trace.
A handful of questions I reach for constantly, each of which shifts attention somewhere specific:
- What would have to be true for this implementation to be correct? — sends it looking for unstated assumptions.
- What evidence would disprove our current hypothesis? — stops it accumulating confirmation and makes it look for contradictions.
- What happens if this event arrives twice? — moves it toward idempotency, ordering, and replay.
- Are we fixing the symptom, or the lifecycle that produced it? — turns a local defect into a systems question.
None of those supply a conclusion. They don't even narrow the answer much.
They supply a lens.
Where the sharper questions come from¶
It's tempting to file this under being good at prompting, but nobody's reaching for secret syntax. What's being deployed is a catalogue of failure modes accumulated over a long time: distributed systems that broke because messages arrived out of order, databases that quietly became accidental sources of truth, caches treated as authority, harmless-looking retries producing duplicate side effects.
Years of that trains something like an attention policy — a sense of which questions collapse the search space fastest. What's strange is how little it feels like knowledge from the inside. It arrives as mild discomfort rather than as a hypothesis. That sounds like an ownership problem. Before we change this, I want to know where that value comes from. When I wrote about ownership in distributed systems, I was really unpacking one of those instincts slowly enough to see its shape.
The obvious reading of that is that experienced engineers get more out of AI and that's the end of it. Among the people I work with it does seem to hold today, but I don't think it's a property of expertise so much as of having seen this particular failure before — a much less permanent advantage than it sounds, and the part I'd most like to see AI take over.
Questions are how context gets built¶
This changed how I think about context, which is the part I find most practically useful.
We tend to talk about context as though more is strictly better. Give the model the repository. Give it the ticket, the architecture documentation, the schema, every related pull request, the whole prior conversation. But a large pile of material isn't context. It's availability. The model still has to work out what matters, and now it has to do that across a much larger surface.
What actually seems to work is a loop:
Concretely, on a state-corruption problem I might start with where is this state mutated? The model investigates and comes back with three write paths. That answer is now context — small, specific, and earned. So the next question becomes which of those three is supposed to be authoritative?, and the answer to that is ownership context. Then can the other two produce a conflicting state?, which yields consistency context. Then what invariant would prevent that?, which is architectural context.
Four questions in, the model is holding a fairly rich picture of the problem. The important part is that every piece of it was pulled in because a question needed it. Nothing arrived speculatively. That's a different thing from filling a context window up front and hoping the relevant fragment gets noticed.
It also suggests the constraint we talk about most isn't the binding one. Token budgets and context windows are real, but a model can hold a million tokens and still spend its entire reasoning budget on the wrong problem, while a model holding very little can do well if the investigation is aimed. If that's right, the interesting design question for coding agents shifts from how do we give the agent everything it might need? to how does the agent work out what it needs to know next?
When the lens is wrong¶
The obvious objection to all of this is that a lens narrows, and narrowing is only good if you're pointed somewhere useful.
A well-aimed question collapses the search space. A confidently wrong one collapses it just as efficiently, around a false premise — and a capable model will then build a coherent, well-evidenced, entirely misdirected investigation on top of it, which is harder to detect than obvious flailing. A model searching broadly at least stays near the truth by accident.
This gets worse as the questions improve, because a good question makes its answer feel earned. I've had chains where four sensible questions produced four grounded answers, all of it hanging off an assumption I'd smuggled in with the first one. Noticing required telling a complete-sounding answer from a correct one — exactly the judgement I'm claiming is scarce. The argument is somewhat circular, then: questions direct attention well, provided you can already tell a good question from a bad one. I don't think that's fatal, but it's why I treat this as a discipline rather than a technique.
What if the questions came from a model?¶
If good questions are what's scarce, it's worth asking whether a human has to generate them. I don't think so, and that's where this starts to interest me beyond my own workflow.
Imagine separating two responsibilities. One agent works the problem. Another watches the task, the evidence, and the reasoning, and asks a single thing: what is the most important unanswered question right now? Its only job is to find the question whose answer would most reduce uncertainty or most change the decision. It might notice that the implementation agent has been treating a database table as authoritative without ever establishing that it is, and ask what evidence distinguishes it from a projection. It might notice that a proposed fix handles the immediate exception but not re-execution, and ask what happens the second time reconciliation runs against the same state.
Anyone who's worked on a strong engineering team has seen this happen in a room. A discussion circles for twenty minutes, somebody asks one question, and everything stops while the problem quietly changes shape.
That also makes me wonder whether we're describing multi-agent coding systems at the wrong level. When I wrote about review pipelines I defined the stages by role — security reviewer, architecture reviewer, correctness reviewer — and that's still how I'd build one. But roles may be a proxy for something more fundamental: what a security reviewer really contributes is a class of question. Can this input cross a trust boundary? Architecture contributes which capability owns this state? Correctness contributes what happens under retry, concurrency, or partial failure? If that's right, the orchestration problem isn't which agent runs next.
It's which question deserves the system's attention next.
And if a model can generate those questions, the advantage I described earlier — having seen the failure before — stops being something you need fifteen years to accumulate. I'd rather have that outcome than be right about seniority.
Architecture that says what to ask¶
The same idea applies to architecture documentation, which we usually treat as information the model should have. Feed it the diagram and the service boundaries, hope the relevant part surfaces at the relevant moment.
Architecture could do something more useful: say which questions become important when particular parts of the system change. If a change touches an authoritative fact, ask whether this component owns it. If a capability spans multiple deployables, ask where the transactional boundary is. If an event is consumed asynchronously, ask what happens under duplicate or out-of-order delivery. If a read model is modified, ask whether anything downstream could mistake the projection for authority.
Written that way, architecture stops being passive reference material and becomes a way of directing reasoning — encoding not just what the system looks like, but what to worry about when a specific part of it moves. I suspect that's worth considerably more to a model than the diagram is.
Loops, not incantations¶
None of this means prompts don't matter. Clear instructions matter, context matters, constraints matter. My objection to "prompt engineering" is narrower: the phrase makes proficiency sound like knowing the right incantation, when most of what separates a useful session from a wasted one is judgement. Knowing when an answer is incomplete. Knowing which assumption was never established. Knowing when the model has become absorbed in a detail that stopped mattering, and which question forces the problem back into focus.
That's the part I've had least success handing over. A model can generate candidate questions and even rank them by how much uncertainty each would remove — but deciding whether the whole investigation is still heading toward the thing that matters has stayed mine. It's also the part that compounds. Good questions direct attention, directed attention produces better answers, better answers become grounded context, and grounded context is what makes the next question sharper than the last.
I used to think of a session with AI as a series of instructions. I've started thinking of it as a search that I'm steering.
The compounding isn't in the answers. It's in what each answer lets me ask next.