A Question Isn’t a Query

Every AI tool takes plain English now. That fixed the wording of a question. It left the harder part, working out what the question actually is, exactly where it was.

The questions that matter in a company rarely arrive in a form anything can run. They arrive as is our growth real? — a sentence with the grammar of a question and none of its specifications. Growth in what: sign-ups, paying accounts, revenue, usage? Real as opposed to what: a handful of heavy accounts carrying the number, one channel having a good month, a seasonal bump that will reverse? Measured over which period, for which people, against what baseline?

None of that is in the sentence. All of it has to be decided before a single number means anything. And once it has been decided, the computation that follows is close to clerical. The analysis already happened, in the deciding.

The step that stayed with the person

In July, The Workflow Has Two Halves. Only One of Them Has a Product. laid out the steps between a business question and a decision, and the first of them was this one: the question has to be decomposed into something answerable. That piece’s point was that every step after the data has always had a person rather than a product.

Meanwhile, AI tools have converged on a simple promise: ask in plain English, get an answer. It reads like the first step, finally built. It is worth looking closely at which step they actually took.

They took the query. An AI that translates a sentence into SQL is a real advance, and on a question that is already analyzable it is fast and genuinely good. But translation starts from the assumption that the decomposition is finished — that by the time you type, you have already decided what growth means, what real would look like, and which window counts. It turns a specified question into code. It does not specify the question.

An earlier piece here put the analyst’s real job this way: the query was the toll, not the work. Translation abolished the toll. It did not touch the work.

This is not only a view from the outside. Some of the people who build and evaluate these systems have said it about their own benchmarks.

A team of researchers introducing a text-to-SQL dataset called PRACTIQ put it plainly: “Previous text-to-SQL datasets and systems have primarily focused on user questions with clear intentions that can be answered. However, real user questions can often be ambiguous with multiple interpretations or unanswerable due to a lack of relevant data.” When they tested current systems against questions like that, their finding was that “state-of-the-art systems struggle to handle ambiguous and unanswerable questions effectively.”

That is the field noticing that its yardstick assumed the hard part away. The honest version goes one step further than their paper does. The ambiguity they measure — a word that maps to two columns, a phrase with two readings — is the small, tractable end of a much larger gap. Is our growth real? is not ambiguous between two readings of a table. It is not yet a question about any table. Turning it into one is formulation, and formulation is the part of analysis that decides whether the answer will mean anything.

Two questions, not one

There is a distinction worth holding onto here, because it is easy to blur and expensive to get wrong.

Every real question contains two things. One is what matters: what the company is trying to decide, what it already suspects, what result would change its mind. That part belongs to the person asking. It depends on context no system has, and as The Question Is the Scarce Asset Now argued, it is the part that becomes more valuable as answers get cheaper, not less.

The other is the runnable form: which metric stands in for growth, which population, which period, which definition of an active account, what would count as evidence that the growth is broad rather than concentrated. That part is labor. Skilled labor, but labor — and it is the part every generation of tooling has handed back to the person, usually without saying so.

The trouble is what happens when the person does not do it, which is most of the time, because nobody told them it was theirs to do. Unless it is built to stop and ask, a translation layer fills the gaps with defaults. It picks a definition of growth, picks a window, picks a population, and returns a confident number for a question slightly to the side of the one that was asked.

Those silent picks are exactly the failures You Do Not Need to Read SQL to Catch a Wrong Number described: not errors of arithmetic but errors of premise, claims about the business made by something that does not know the business. That piece was about catching them after the fact. This is about where they are made. They are made at formulation, by default, in a step nobody noticed was a step.

“It asks a clarifying question now”

The strongest pushback is that the better tools already handle this. Give them an ambiguous request and they ask what was meant. This is real, and it is the remedy the PRACTIQ researchers build toward: their dataset is organized around exactly that move — a question, a clarification, then the answer.

A clarifying question is worth having. It is also narrower than it looks. It chooses between readings the data already offers: did “customers” mean accounts or users, did “last quarter” mean calendar or fiscal. That works when the question is one specification away from being runnable.

The questions that matter are rarely one specification away. Nobody can answer “what do you mean by real?” on the spot, because the honest answer is that nobody knows yet. Finding out is the point. What real growth would have to look like in this company, and which data could show it, usually only becomes clear after a first look: the number is up, so is it broad or concentrated, new accounts or old ones, one channel or several? That is not a prerequisite to the investigation. It is the investigation, already under way.

Sometimes the honest end of that process is a refusal. If the question is whether growth is translating into revenue, and no revenue data is connected, the right output is not a clarifying question and not a number. It is this can’t be settled with what you have, and here is why. A system that only translates never gets far enough into the question to know that.

Formulation in the open

If formulation is the work, then the requirement for a system is not that it skips the step, and not that it hands the step back to the person. It is that it does the step where the person can see it.

That means stating the question it actually decided to answer, in the language of the business rather than the language of the query. Growth, taken here as paying accounts rather than sign-ups. Real, taken to mean the increase is not carried by a few large accounts and not confined to one channel. Measured against the same period last year, because the business is seasonal. Each of those is a choice. Each is something a founder who has never written a line of SQL can look at and say no, that’s not what we mean.

That is the division of labor the two-question split implies. The system carries the labor of making the question runnable. The person keeps what was always theirs: knowing what matters and recognizing a wrong reading of it on sight. Nothing about that makes the person less necessary. It makes their judgment the thing the whole answer turns on, which it always was.

Where the next sorting happens

For a generation, the hard part of getting an answer from data was the mechanics: the query, the join, the pipeline. Plain-English interfaces have mostly dissolved that, and the dissolving is real progress. Every tool in this market can now take a sentence.

What that does is expose the step underneath, the one that was always there and was always someone’s evening. When every tool can take the sentence, taking the sentence stops distinguishing anything. What will distinguish the next generation is what a system does with the meaning: whether it notices that a question has not been specified yet, whether it works out the runnable form in the open, whether it can tell when the data in front of it cannot answer at all.

A question isn’t a query. The tools worth asking will be the ones built for the difference.

Key Takeaways

Plain-English interfaces fixed the wording of a question. The hard part, working out what the question actually is, never lived in the wording.
Translating a question into a query abolished the toll and left the work. It assumes the decomposition is already done, and fills the gaps with silent defaults when it isn’t.
Researchers are saying it about their own benchmarks: text-to-SQL datasets and systems have primarily focused on questions with clear intentions that can be answered. Real ones often aren’t.
Every real question holds two things: what matters, which stays the person’s, and the runnable form, which is labor that every generation of tooling handed back to them.
A clarifying question chooses between readings the data already offers. The questions that matter are not one specification away, and working them out is the investigation, not a prerequisite to it.

The research referenced is Dong et al., “PRACTIQ: A Practical Conversational Text-to-SQL dataset with Ambiguous and Unanswerable Queries” (arXiv preprint, 2024; revised 2026). This piece continues the argument in “The Workflow Has Two Halves. Only One of Them Has a Product.”

A Question Isn't a Query