Who Answers Changes the Number

The U.S. Census Bureau’s Business Trends and Outlook Survey asks firms whether they used artificial intelligence in the last two weeks, and it records the job title of whoever answers. When the respondent was a top executive or owner, the answer was yes 20.8% of the time. When it was someone in administrative support, 9.9%. With fixed effects for firm size, industry, state, and survey period, administrative respondents still reported AI use 10.3 points less often than executives, on 106,600 observations. The authors cannot tell whether executives know more or overreport, and they say so. What they do instead is assign visibility by role: operations and general management “may be best for answering questions about how AI is changing specific tasks”.

A Federal Reserve note published the same month compared three national estimates of AI adoption: 18%, 41%, and 78%. It is tempting to read those as firms, workers, and executives disagreeing. The author does not. The three numbers measure different units with different weights: a share of firms, a share of the labor force, and the share of employment in firms whose leaders report adoption. Sampling and units of analysis, he concludes, are the main drivers, and information asymmetries between respondents are “likely not the primary driver”. His conclusion is still the right one for this paper: “Which estimate researchers and policymakers look to depends on the question they are seeking to answer.”

When the questions are identical, the gap does not close. In a 2026 NBER survey, employees asked the same questions as executives expected AI to raise employment at their firms by about 0.5% over three years. U.S. executives expected it to cut employment by 1.2%. Those are forecasts, with no truth yet to check them against. They show that position predicts the answer, not which position is right.

After the 2005 explosion at BP’s Texas City refinery, an independent panel chaired by James Baker surveyed the workforce of BP’s five U.S. refineries: 7,451 of 10,298 employees and contractors, a 72% response rate, with results reported by functional group. One item asked whether process safety was a long-term commitment not compromised by short-term financial goals. At Toledo, 45% of operators and 56% of maintenance technicians disagreed. Among operations managers at the same refinery, 19% did. On whether operational pressures lead to cutting corners on process safety, the Toledo figures were 49% of operators and 14% of operations managers.

The gap widened with distance. At Toledo, 52% of operators said they could not challenge refinery management’s decisions without fear of negative consequences; about a quarter said the same of their own supervisors. On procedures, 50% of Toledo’s maintenance technicians said written operating procedures were not followed regularly, while engineering professionals and operations management gave negative responses ranging from zero to 15%. The panel’s summary of the company as a whole was that “a substantial gulf appears to have existed” between the actual performance of its process safety management systems and the company’s perception of that performance, and BP’s own investigation had found that its audits “focused on documented management systems and processes rather than actual practices”.

The Chemical Safety Board’s investigation of the same accident found the mechanism at the top. Executive leadership had considered further intervention at Texas City, but “did not act because the personal injury rates had improved”. The indicator leadership was watching answered a different question from the one that mattered. In its investigation of Macondo, the Board used the same distinction that runs through this series, between work as imagined by well designers and managers and work as done by the crew, and documented a critical check, the negative pressure test before displacing with seawater, that existed as the offshore installation manager’s personal policy rather than in the plan.

None of this is peculiar to refining. A survey of 722 workers on ten UK offshore installations found not one safety culture but fragmented “safety subcultures”, varying mainly with seniority, occupation, age, shift, and accident history, and its authors suggested that a strong, cohesive culture is not necessarily a healthy one.

One mechanism is well documented. In interviews with 40 employees, Milliken, Morrison, and Hewlin found that 34 had at least once felt unable to raise an issue they considered important with their boss, and that 35% felt unable to speak up about problems with organizational processes or performance. Of the 34, 74% said other employees were aware of the issue and also felt unable to raise it. The knowledge existed below, it was shared, and it did not travel up.

The sample is small and drawn from office work. The refinery numbers point in the same direction at industrial scale: the share of operators who felt unable to challenge a decision roughly doubled between the immediate supervisor and refinery management. A diagnosis that collects its evidence through the line inherits that gradient.

The strongest objection runs as follows. Averaging works: when estimates fall on both sides of the truth, the average must outperform the average judge. Groups know things individuals do not: in the classic studies of informant accuracy, “about half of what informants report is probably incorrect”, yet the group at large could still identify its most popular members. And operators are wrong too. In a randomized trial with 16 experienced developers and 246 tasks, the developers believed AI had cut their completion time by 20%; it had increased it by 19%.

Two of these points should be conceded. The people who do the work are not authoritative on magnitudes, how long a step takes or how much a tool changed it; those are measured, which is the argument of the first paper in this series. And management is not always the optimistic party. At Texas City, 61% of maintenance management said workers sometimes worked around process safety concerns, against 7% to 21% at the other refineries.

The averaging argument, though, depends on its conditions. Larrick and Soll’s result holds when judges estimate the same quantity and bracket the truth. A process owner describing the process as designed and an operator describing it as run are not estimating one quantity with noise. Sources that share information are worth far less than their count: with positive dependence, Clemen and Winkler showed, the value of additional sources “can decrease very rapidly”. Heterogeneity of information is often more important than measurement noise, which is what an average assumes away. And majority methods are biased toward widely shared information “at the expense of novel or specialized knowledge that is not widely shared” — the knowledge of the night shift, or of the one person who handles the exception. Even the multiple-informant literature asks how consensus among informants should be judged before it asks how their responses should be combined.

Starbuck and Mezias, reviewing why managers’ perceptions so often look inaccurate, distinguished questions about facts, questions that ask for interpretations of facts, and questions that “invite respondents to invent facts where no facts can possibly exist”. They hypothesized that perceptual error is smaller when a manager’s functional area relates to the variable in question. Proximity is the organizing idea we propose for operational diagnostics.

The process owner is authoritative for the process as intended: its rules, its ownership, what is supposed to happen. The people who run it are authoritative for the steps actually executed, the exceptions, and the workarounds, as facts they witnessed in specific episodes. Durations and frequencies belong to measurement, not to either. When the two accounts disagree, the disagreement is reported, not resolved by a vote.

That requires telling positional disagreement from noise. Our working criterion: a disagreement is positional when it repeats by position across sites or teams, as the operator–manager gap repeated across five refineries in the Baker survey. A disagreement that does not repeat is a candidate for noise. This criterion is our synthesis, not a tested result.

A single number per site is the natural output of a diagnostic, and the wrong one when the disagreement underneath it is positional. Averaging Toledo’s operators and managers produces a respondent who does not exist.

The strongest industry evidence comes from refining, which is downstream, and from the UK North Sea. We found no quantitative comparison of office and field accounts of the same process in U.S. upstream oil and gas.

Surveys measure perceptions. The Baker panel says so itself: survey data “generally reflect impressions, beliefs, and opinions of the group being surveyed”. They establish that position predicts the answer, not which position is right.

In the Census study, the questions about firm-level and worker-level AI use were likely answered by the same person in each firm, so the job-title effect is a difference between respondents across firms, not between levels within one.

Authority by type of claim, and the criterion for positional disagreement, are a synthesis of the sources cited, not an empirical result.

The evidence establishes that who answers changes the number. It leaves open the questions that matter most to an operator.

Plexo builds evidence-based operational diagnostics, with a focus on oil and gas operators and service companies. Its interviews weight evidence by source: process owners for the process as intended, the people who run it for the process as done. Disagreements between them are reported as findings, and every claim traces to a verbatim quote.

We intend to publish how often disagreement is positional only under a stated methodology, with customer consent, and on a sample large enough to support the claim. This is the second paper in a monthly series on evidence in operational diagnostics.