Composite 36Trend 20
The agent evaluation gap: Enterprise AI organizations have a reality-alignment problem, not a coverage problem
Across 157 enterprises, organizations are granting AI agents more autonomy while trusting the evaluations meant to gate that autonomy less. Half have already shipped an agent that passed their internal evaluations and then failed a customer in production; only one in twenty fully trusts automated evaluation today; and the most-cited weakness is that evaluations do not align with real-world outcomes. Yet two-thirds already allow, or are actively engineering toward, deploying agent changes to production on automated evaluation alone — with no human in the loop. The result is an evaluation gap — the distance between how much autonomy enterprises are handing their agents and how far they trust the tests that are supposed to catch the failures.
This wave of VentureBeat Pulse Research examines how technical leaders measure agent performance: which reliability and evaluation platforms they use, how they select and trust them, what breaks in production, and how far they are willing to let agents run without a human in the loop.
The central finding is an evaluation gap — the distance between the autonomy enterprises are granting their agents and the trust they place in the evaluations meant to govern it. Half of organizations (50%) have, in the past year, deployed an agent or LLM feature that passed their internal evaluations and then caused a customer-facing failure, and a quarter have seen it happen more than once. Trust in the tests themselves is thin: only 5% say they fully trust automated evaluation today, and the single most-cited limitation is that evaluations align poorly with real-world outcomes (29%). Enterprises are discovering that a passing eval is not the same as a working agent.
What makes the gap consequential is the direction of travel. Two-thirds of organizations (66%) already permit fully automated, zero-human-in-the-loop deployment for low-risk agents (34%) or are actively engineering their pipelines to allow it within twelve months (33%). At the same time, the evaluation stack that would have to earn that trust is fragmented and immature: the most common primary tools are the model providers’ native evals, tied with having no dedicated tooling at all (17% each); and only about a quarter of enterprises run real-time quality checks on live production traffic. The autonomy is arriving faster than the assurance.
Methodology
VentureBeat fielded this survey as part of its ongoing Pulse Research series, this survey — the Agentic Reliability & Evals tracker — focused on how technical leaders evaluate agent performance and reliability. Responses are filtered to organizations with 100 or more employees (n=157), drawn from a single survey in June 2026; because this is one wave rather than a pooled multi-month sample, the report reads cross-sectionally and does not infer month-over-month trends. Where questions were multiple-select, those shares can sum to more than 100%.
By role the sample is senior and buyer-credible: 38% are final decision-makers for AI purchases and another 34% recommenders or influencers. Product and program managers (15%), consultants and advisors (10%), directors of engineering/IT (8%), and CIOs/CTOs/CISOs (8%) lead the named titles, alongside a large “Other” function (37%). By organization size the sample is mid-market-weighted: 100–499 (37%) and 500–2,499 (27%) employees lead, with 2,500–9,999 (20%), 10,000–49,999 (10%), and 50,000+ (6%) above them. Technology/Software is the largest industry at 23%, followed by Retail/Consumer (15%), Healthcare/Life Sciences (12%), and Manufacturing (10%).
At 157 respondents the sample is large enough to read directionally but should be treated as a directional signal rather than a precise measurement; it is self-selected and is not a probability sample. It skews toward the mid-market, so it is best read as the view from organizations actively standing up agent evaluation practices rather than from the largest operators.
Note: This survey was rebuilt for the June wave from the earlier “LLM observability and evaluations” survey; because the questions and sample differ, no comparisons are made to the April–May data.
Finding 1: A passing eval is not a working agent
Half have shipped an agent that passed evals, then failed a customer
We asked whether, in the past 12 months, organizations had deployed an agent or LLM feature that passed their internal evaluations but then caused a customer-facing failure. Half of those that run evaluations had.
This is the report’s defining number. Half of organizations (50%) have shipped an AI feature that cleared their internal evaluations and then failed in front of a customer — an incorrect output, a broken workflow, or a quality incident — and a quarter have seen it happen more than once. Only 36% report no such failure, and the remainder either run no pre-deployment evaluations (8%) or don’t track the root cause closely enough to know (6%). The failure is precise and expensive: the evaluation said the agent was ready, and it was not. Everything that follows — how enterprises trust their evals, what they monitor, and how much autonomy they grant — is shaped by this experience.
Finding 2: Almost no one fully trusts automated evaluation
The top complaint: Evals don't match real-world outcomes
We asked which limitation most reduces trust in automated agent evaluations today. Only a sliver of enterprises had no complaint at all.
Trust in automated evaluation is scarce, and specific. Only 5% of organizations say they fully trust automated evaluation as it stands — meaning 95% name a limitation that holds them back. The most common, at 29%, is the one that most directly explains Finding 1: evaluations align poorly with real-world outcomes, passing agents that later fail. Bias or inconsistency (21%) and a lack of explainability (18%) follow — enterprises cannot always tell why an evaluation reached its verdict — and 17% cite data-leakage or privacy concerns in the evaluation process itself. The tests meant to certify agents are not yet trusted to certify them, which is precisely why the autonomy trajectory in Finding 3 is so striking.
Finding 3: The autonomy ceiling is rising anyway
Two-thirds already allow, or are building toward, zero-human deployment
We asked whether organizations would let an autonomous agent deploy a code or system change to production on automated evaluation results alone, with no human-in-the-loop validation. The trajectory runs straight through the trust gap.
Here is the paradox at the heart of the report. Even though almost no one fully trusts automated evaluation (Finding 2), two-thirds of organizations (66%) either already allow zero-human-in-the-loop deployment for low-risk agents (34%) or are actively engineering their pipelines to permit it within a year (33%). Only 22% rule it out for the foreseeable future. The direction is unambiguous: enterprises are moving to let evaluations gate production autonomously — removing the human check — at the same moment they say those evaluations don’t reliably match reality. The autonomy ceiling is rising faster than the assurance beneath it, which is the mechanism by which the false-confidence failures of Finding 1 will scale rather than shrink.
Notably, the autonomy bet is not just a small company phenomenon. Splitting the sample by company size, larger enterprises are slightly further down the path toward zero human review than smaller companies (70% versus 64%) and slightly more likely to have shipped an evaluation-passing agent that then failed a customer (54% versus 48%). The assumption that large, regulated organizations are holding the human in the loop longest is, in this sample, backwards. To be sure, these are directional figures, since the survey was not a huge sample — 57 respondents from companies with 2,500+ employees and 100 from companies smaller than that.
Finding 4: The evaluation stack is fragmented and provider-led
Provider-native evals lead — tied with no dedicated tool at all
We asked which agent reliability or evaluation platform enterprises primarily use today. The market has no clear leader — and a large share has nothing dedicated.
The evaluation layer is early and unconsolidated. Provider-native tooling leads — OpenAI’s native evals and traces (17%) and Anthropic’s Claude Console evals (13%) together outweigh any independent platform — but it is tied at the top by a striking answer: 17% of enterprises use no dedicated agent-evaluation tooling at all, a notable gap for organizations shipping agents to customers. The specialist evaluation vendors — DeepEval (12%), Braintrust (8%), LangSmith, Weave, Promptfoo, Langfuse, Arize — are scattered across single to low double digits, and 11% have built their own. No independent platform has yet become the category standard, which leaves most enterprises evaluating agents with provider-native tools, home-grown scripts, or nothing.
Finding 5: Production monitoring rarely watches output quality
Only a quarter run real-time quality checks on live traffic
Production monitoring for an AI agent can watch two very different things. It can watch whether the system is functioning — is the agent up and responding, did each request complete, how fast, at what cost, with any errors. Or it can watch whether the agent's output is correct — automated checks that evaluate the content of each answer as it goes out: did the agent give the right answer, take the right action, stay within policy. The distinction matters because a confidently wrong answer is invisible to the first kind of monitoring: the request completes, the response is fast, no error is thrown, and every functioning-metric reads healthy. We asked organizations which kind their live production monitoring is built for today.
Grouped by what is actually being watched, the split is stark: 51% of organizations monitor only whether the agent is functioning, while 23% monitor whether its answers are right. Counting the ad-hoc reviewers and the don't-knows, roughly three-quarters of organizations run no automated, real-time evaluation of output correctness in production — they can see that the system is up and what it costs, and they are taking the correctness of its answers on faith. That blind spot is the runtime counterpart to the pre-deployment gap in Finding 1: the same organizations engineering the human out of the deployment decision mostly cannot see, in real time, when the deployed agent starts getting things wrong.
Finding 6: Bought on cost, measured on consistency
Price and integration drive selection; evaluation consistency is the goal
We asked what most influenced enterprises’ choice of an evaluation vendor, and what they treat as their primary measure of success. Both answers are pragmatic.
Enterprises buy evaluation tooling on economics and trust it on repeatability. Cost of evaluations (28%) narrowly leads selection, just ahead of ease of integration (27%) and evaluation accuracy (24%) — breadth of observability (13%) and vendor roadmap (4%) matter far less. On what success looks like, more than a third (36%) name evaluation consistency — getting the same verdict on the same behavior every time — well ahead of speed of experimentation (19%), reduction in failures (18%), production visibility (13%), and compliance (11%). The emphasis on consistency is telling: before enterprises can trust an evaluation’s verdict, they need it to be stable — the very property whose absence (bias and inconsistency) ranked among the top trust limitations in Finding 2. Satisfaction with current tooling is only moderate, averaging 3.8 on a five-point scale across overall satisfaction, ease of implementation, and value for money.
Finding 7: The next dollar goes to humans and observability
Investment is flowing to oversight, not just automation
We asked which reliability and evaluation investment will grow most over the next year. The money is going toward watching agents more closely — including with people.
The second-largest planned investment — behind only production observability — is human review workflows, at 26%. Read against Finding 1, that is the report's quietest contradiction: at the same moment two-thirds of enterprises are engineering the human out of the deployment decision, more of them plan to grow spending on human reviewers (26%) than on the automated evaluation pipelines (16%) that would replace them. The zero-human trajectory and the human-review budget are rising in the same companies at the same time. Indeed, only 8% report that their budget is not increasing.
Taken together, enterprises are hedging: building toward autonomy while spending to watch agents more closely and keep humans available for the calls that automated evaluation cannot yet be trusted to make.
Finding 8: A tooling reshuffle is coming
Nearly two-thirds plan to adopt or switch platforms within a year
We asked whether enterprises plan to adopt a new, additional, or replacement evaluation platform, and which they are considering. Few intend to stand pat.
The evaluation market is wide open. While 36% have no plans to change, a clear majority (64%) intend to adopt a new, additional, or replacement platform within twelve months, and 31% within the next quarter. The consideration set points where current usage is thinnest: Confident AI’s DeepEval leads what enterprises are evaluating (20%), ahead of OpenAI’s native evals (13%) and Braintrust (9%) — the open-source specialists drawing more interest than their present footprint.
Given that so many enterprises today rely on provider-native tools or nothing at all (Finding 4), this is less a defection than a first real wave of tooling adoption — the moment the evaluation layer starts to consolidate. Which platforms earn that trust, in a market where almost no one trusts automated evaluation yet, is the open question this series will keep tracking.
The bottom line: An evaluation gap that autonomy will widen, not close
Organizations with 100 or more employees are granting AI agents more independence than they trust their evaluations to support. Half have already shipped an agent that passed its evals and then failed a customer; almost none fully trust automated evaluation, chiefly because it doesn’t match real-world outcomes; and most watch production for uptime and cost rather than for whether the agent’s answers are right. Yet two-thirds already allow, or are actively building toward, deploying to production on automated evaluation alone.
The vendor market is early and unsettled: the most common primary evaluation tools are provider-native evals, tied with no dedicated tooling at all, and a clear majority plan to adopt or switch platforms within the year. Encouragingly, the next dollar is going to observability and — pointedly — human review, suggesting enterprises sense the gap even as they engineer past it. At 157 respondents in a single wave this is a directional read, skewed toward the mid-market — but the direction is clear: autonomy is being granted on the strength of evaluations that the people granting it do not yet trust. The evaluation gap is not a coverage problem that more tests alone will close; it is a problem of evaluations that reflect reality and can be trusted to gate it. The open question for later waves is whether assurance catches up to autonomy — or whether the false-confidence failures move from customer incidents into changes that deploy themselves.
Based on survey responses from 157 qualified enterprise respondents (100+ employees), drawn from a single June 2026 wave. This is a directional read rather than a precise measurement — the sample is self-selected, not a probability sample, and skews toward the mid-market. Respondents include product and program managers, consultants and advisors, directors of engineering/IT, and CIOs/CTOs/CISOs, among other functions, across technology/software, retail/consumer, healthcare/life sciences, manufacturing, and other industries.
Composite 55Trend 10
The agent security gap: 54% of enterprises have already had an AI agent incident, and most still let agents share credentials
Across 107 enterprises, AI agents are being given real access to systems and data while the controls meant to contain them lag behind. More than half have already had a confirmed agent security incident or a near-miss; only about a third give every agent its own scoped identity, and most agents still share credentials; and only three in ten isolate their highest-risk agents. The security stack is overwhelmingly borrowed from the model providers and hyperscalers rather than purpose-built for agents, spending remains a thin slice of the security budget, and enterprises are evenly split on whether their defenses are keeping pace with AI-enabled attackers. The result is an agent security gap — autonomous agents proliferating faster than the identity, isolation, and enforcement controls needed to hold them.
This wave of VentureBeat Pulse Research examines how enterprises secure their AI agents: what tooling they run, how they manage agent identity and isolation, what has already gone wrong, how much they spend, and whether they believe their defenses are keeping pace with AI-enabled attackers.
The central finding is an agent security gap — the distance between the autonomy enterprises are granting their agents and the controls in place to contain them. More than half of organizations (54%) have already experienced a confirmed agent security incident (18%) or a near-miss caught before harm (36%). The structural weakness beneath those numbers is identity: only about a third (32%) give every agent its own scoped, managed identity, while the rest report that some agents share credentials or that agents mostly run on shared API keys and human or service-account credentials. When agents share credentials, a single compromised or over-permissioned agent carries a wide blast radius — and only three in ten enterprises (30%) isolate their highest-risk agents in sandboxes to bound that radius.
What makes the gap notable is how comfortable enterprises are inside it. The security stack is overwhelmingly provider-native — OpenAI’s guardrails (51%), Google’s and Microsoft’s cloud controls, and Anthropic’s managed-agent controls dominate, while the dedicated agent-security specialists barely register — and satisfaction with that borrowed stack is high, averaging 4.2 out of 5. Yet spending remains a thin slice of the security budget, only a third of enterprises believe their AI defenses are ahead of AI-enabled attackers, and a clear majority plan to change tooling within the year. Enterprises are satisfied with controls they are simultaneously preparing to replace.
Methodology
VentureBeat fielded this survey as part of its ongoing Pulse Research series, this instrument focused on enterprise agent security — the tooling, identity, isolation, and enforcement controls organizations use to secure autonomous AI agents. Responses are filtered to organizations with more than 100 employees (n=107; the survey’s smallest size band, 1–100 employees, is excluded), drawn from a single June 2026 wave. Because this is one wave rather than a pooled multi-month sample, the report reads cross-sectionally and does not infer month-over-month trends. Several questions were multiple-select, so those shares can sum to more than 100%.
By role the sample is senior and buyer-credible: 45% are final decision-makers for AI purchases and another 30% recommenders or influencers. Managers (43%), individual contributors (24%), VPs and directors (15%), and the C-suite (11%) make up the seniority mix. By organization size the sample is mid-market-weighted: 251–1,000 (42%) and 101–250 (25%) employees lead, with 1,001–5,000 (19%), 5,001–10,000 (8%), and 10,001+ (7%) above them. Technology/Software is the largest industry at 23%, followed by Manufacturing (15%), Retail/E-commerce (14%), and Healthcare/Life Sciences (13%).
At 107 respondents the sample is large enough to read directionally but should be treated as a directional signal rather than a precise measurement; it is self-selected and is not a probability sample. It skews toward the mid-market, so it is best read as the view from organizations actively standing up agent security rather than from the largest operators.
Satisfaction ratings are computed on the respondents who answered each rating question; the overall satisfaction score reflects 82 of the 107 qualified respondents.
Finding 1: The incidents are already here
More than half have had an agent security incident or near-miss
We asked whether organizations had experienced an agent security incident — a confirmed breach, or a near-miss caught before harm. Most that run agents in production had.
This is the report’s defining number. More than half of organizations (54%) have already had an agent security event — 18% a confirmed incident and 36% a near-miss caught before it caused harm. Only 42% report nothing, and a small remainder either run no agents in production or don’t track such events. That so many report near-misses rather than only confirmed incidents is telling: enterprises are catching problems, but they are catching them close to the edge. The controls examined in the rest of this report — identity, isolation, enforcement — are what determine whether the next near-miss stays a near-miss.
Exposure scales with company size, but containment does not. The incident-or-near-miss rate rises from 49% in the mid-market (companies with 101-1,000 employees) to 63% at larger enterprises (above 1,000 employees), while sandbox isolation of high-risk agents falls from 35% to 20%, and satisfaction with security tooling drops from 4.36 to 3.97. The organizations running the most agents across the most systems carry the most incidents and the least of the one control that bounds an incident's blast radius.
Finding 2: The identity gap
Only a third give every agent its own scoped identity
We asked how enterprises manage the identity of their AI agents — whether each agent has its own credentials, or agents share them. Full per-agent identity is the exception.
Rolled together, the overlapping answers show 69% of enterprises (74 of 107) with credential sharing somewhere in the agent fleet. Identity is the structural weakness beneath the incidents. Only about a third of enterprises (32%) give every agent its own scoped, managed identity — the precondition for least-privilege access and clean attribution. Nearly half (48%) say some agents have scoped identities but many still share credentials, and another 32% say agents mostly run on shared API keys or borrowed human and service-account credentials. (Respondents could describe more than one pattern across their agent fleet, so these overlap.)
The consequence is direct: when agents share credentials, an over-permissioned or compromised agent can act with far more reach than intended, and forensics after an incident cannot cleanly tell which agent did what. The non-human identity problem — giving every agent its own governed identity — is the single largest unfinished piece of enterprise agent security.
Moreover, a company’s agent credential posture is correlated with incidents. Organizations with credential sharing anywhere in the fleet were hit — with an incident or a near-miss in the past twelve months — at 63.5% (47 of 74). Organizations where every agent carries its own scoped identity were hit at 40.9% (9 of 22). The fully-scoped group is small, so for now the relationship is an association rather than proven causation, and the gap is concentrated in the mid-market — but within a single survey, a twenty-three point difference in incident rate suggests significance.
Finding 3: Observe and enforce, but rarely isolate
Only three in 10 sandbox their highest-risk agents
We asked what an organization’s agent security posture looks like in practice — whether they observe, enforce, isolate, or some combination. The control that bounds damage is the least common.
Monitoring and enforcement are reasonably common; containment is not. Roughly half of enterprises observe agent activity (47%) or enforce scoped permissions at runtime (49%), but only 30% isolate their highest-risk agents in sandboxes that bound the blast radius when the other controls fail. That ordering is backwards from a defense-in-depth standpoint: observation tells you what happened, enforcement tries to prevent it, but isolation is what limits the damage when prevention fails — and it is the control enterprises have adopted least. Combined with the identity gap in Finding 2, the picture is of agents that are watched and permissioned but rarely boxed in, which is precisely the configuration in which a single failure propagates.
Finding 4: Security runs on borrowed, provider-native controls
Guardrails from OpenAI, Google and Microsoft dominate; specialists barely register
We asked which agent security tooling enterprises use, and which is their primary layer. The answer favors the model providers and hyperscalers over the dedicated security vendors.
Enterprises are securing agents with tools that came bundled with their models and clouds. OpenAI’s guardrails lead at 51%, followed by Google’s and Microsoft’s cloud-native controls and Anthropic’s managed-agent controls — and when asked to name their single primary security layer, 82% name one of these provider-native offerings. The purpose-built agent-security category — Palo Alto’s Prisma AIRS, CrowdStrike, Cisco AI Defense, Zenity, HiddenLayer, Check Point’s Lakera, Okta for AI Agents, non-human identity platforms — barely registers, each in the low single digits, and only 5% run no dedicated tooling at all. As with retrieval and evaluation elsewhere in this series, the provider bundle is winning the default: enterprises reach first for the guardrails their platform ships, and the independent security layer that would address the identity and isolation gaps has not yet been adopted at scale.
The provider-default pattern is consistent across both Q2 survey waves. In April–May (n=110), usage was led by the same names — OpenAI's controls at 26%, Azure at 15%, AWS at 14%, Google at 12% — with every dedicated agent-security specialist at 3% or below and one in ten using no dedicated tooling at all. The common finding from the two surveys: Enterprises are defaulting to the solutions provided by the platform they’re using, and the specialist category vendors have yet to become big players here.
(A note on reading these shares. As described in the methodology section, the respondent sample is self-selected and skews mid-market, and the usage question counted every vendor or approach a respondent has in place — so the figures measure presence in the security stack rather than spending or exclusivity. Individual vendor percentages therefore carry all the usual sample caveats. The structural pattern, however, held across both Q2 waves on two differently worded questions: provider-native and hyperscaler controls lead, and dedicated agent-security specialists remain in low single digits. Read the individual shares loosely and the pattern with confidence.)
Finding 5: And enterprises are comfortable with it
Satisfaction is high, even as incidents mount and identity lags
We asked how satisfied enterprises are with their current agent security tooling. The comfort is notably out of step with the exposure documented above.
Satisfaction with agent security tooling is high — 4.2 out of 5 overall, and 4.1 for value for money — among the most positive readings in this series. That is the striking part: enterprises are highly satisfied with a stack that is mostly borrowed provider guardrails, even though more than half have already had an incident or near-miss and only a third give their agents scoped identities. The comfort appears to rest on the convenience and low friction of provider-native controls rather than on demonstrated containment. It is a false comfort in the making — the same enterprises expressing satisfaction are, as Finding 8 shows, a clear majority planning to change tooling within the year, which suggests the confidence is thinner than the score implies.
Finding 6: Budgets haven’t caught up
Most spend under a tenth of the security budget on agents
We asked what share of the security budget enterprises allocate to securing AI agents. For a fast-emerging risk, the allocation is modest.
Spending on agent security is still a thin slice. The most common allocation is 6–10% of the security budget (46%), and a third of enterprises (34%) spend 5% or less; only a quarter (24%) devote more than a tenth. Given the incident rate in Finding 1 and the identity and isolation gaps in Findings 2 and 3, the budget looks like a lagging indicator — the risk has arrived faster than the funding to address it. The enterprises spending more than a tenth of their security budget on agents are a distinct minority, and they are likely the ones building the scoped-identity and isolation controls the rest have not.
Finding 7: The arms race is even, at best
Only a third think their AI defenses are ahead of AI-enabled attackers
We asked how enterprises assess the balance between their AI-enabled defenses and AI-enabled attackers. Confidence is far from settled.
Enterprises are split on whether they are winning. Only about a third (35%) believe their AI-enabled defenses are ahead of AI-enabled attackers; the rest are less sure — 32% call it roughly even, 21% think attackers are ahead, and another 21% say it is too early to tell. Taken together, a clear majority (53%) rate the balance as even or tilted toward the attacker. That uncertainty sits uneasily beside the high satisfaction of Finding 5: enterprises are content with their tooling yet unconvinced it is winning the contest it exists to win. In a domain where the offense is also compounding with AI, an even race is not a comfortable place to be.
Finding 8: A security reshuffle is coming
Nearly six in 10 plan to adopt or switch tooling within a year
We asked whether enterprises plan to adopt a new, additional, or replacement agent security solution, and which they are considering. Few intend to stand pat.
The security stack is not settled. While 41% have no plans to change, a clear majority (59%) intend to adopt a new, additional, or replacement agent security solution within twelve months, and 29% within the next quarter — a strong signal that, high satisfaction notwithstanding, enterprises know the current stack is provisional. Incidents are what start the buying cycle.
Among organizations that have been hit, 42.1% plan to adopt, add, or replace agent security tooling within the next ninety days, against 14.0% of organizations with no incident — and after a confirmed incident it becomes majority behavior, at 52.6%. Getting hit also changes the threat assessment: 33.3% of hit organizations say AI-armed attackers are ahead of their defenses, against 8.0% of the unhit. Experience, in this data, is the strongest predictor of both urgency and pessimism.
The consideration set still leans provider-native (OpenAI 34%, Google 30%, Anthropic 29%, Azure 25%), but the dedicated security vendors — Cloudflare, Cisco, Palo Alto, Okta, Check Point’s Lakera — draw early interest in the mid-to-high single digits, more than their current footprint.
What the shopping does not yet include is the identity layer specifically. Twelve percent of the respondents include an agent-identity product — Okta for AI Agents, Microsoft Entra Agent ID, or a non-human identity platform — anywhere in their consideration set, and among the credential-sharing organizations that have already had an incident, identity consideration is essentially unchanged, at roughly one in ten. The control most directly implicated by the incident data is the one largely missing from the purchase plans. Whether this wave hardens the provider-native default or finally opens the door to purpose-built agent security — the identity and isolation controls the incidents call for — is the question this series will keep tracking.
The bottom line: A security gap that autonomy will test first
Organizations with more than 100 employees are giving AI agents real reach into systems and data while securing them with controls built for something else. More than half have already had an incident or near-miss; only a third give every agent its own scoped identity, and most still share credentials; only three in ten isolate their highest-risk agents; and the stack doing this work is overwhelmingly borrowed from the model providers and hyperscalers rather than purpose-built for agents.
The uncomfortable pairing is confidence with exposure: satisfaction with the current tooling is among the highest in this series, yet spending is a thin slice of the security budget, only a third believe their defenses are ahead of AI-enabled attackers, and a clear majority are already planning to replace what they have. At 107 respondents in a single wave this is a directional read, skewed toward the mid-market — but the direction is clear: agent adoption is running ahead of agent security, and the controls that matter most when something fails — scoped identity and isolation — are the ones enterprises have built least. The agent security gap is not a coverage problem that a provider guardrail will close on its own; it is a problem of identity, isolation, and enforcement built for autonomous software. The open question for later waves is whether enterprises close it deliberately — or whether a confirmed incident closes it for them.
Based on survey responses from 107 qualified enterprise respondents (100+ employees), drawn from a single June 2026 wave. This is a directional read, not a precise measurement — the sample is self-selected and skews mid-market, so it's best read as the view from organizations actively standing up agent security rather than from the largest operators. Respondents are senior and buyer-credible (45% final decision-makers, 30% recommenders/influencers), spanning managers through the C-suite, and drawn primarily from Technology/Software, Manufacturing, Retail/E-commerce, and Healthcare/Life Sciences.
Composite 55Trend 10
The AI context gap: Enterprise AI organizations have a trust problem, not a retrieval problem
Across 101 enterprises, the infrastructure that feeds AI agents their business context is being built faster than it can be trusted. Retrieval-augmented generation is already the default context source, and provider-native retrieval has quietly overtaken the dedicated vector databases that define the category — yet a majority of enterprises have already watched their agents produce confident, wrong answers traced to missing or inconsistent context. A governed semantic layer is emerging as the fix, but most are still building it; the field is converging on hybrid retrieval; and even as provider-native tools lead in practice, a plurality say they intend to keep best-of-breed. The result is a context gap — agents that sound authoritative running on a foundation their owners do not yet fully trust.
This wave of VentureBeat Pulse Research examines the enterprise RAG and context layer: what feeds AI agents their business context, which retrieval systems enterprises run, how they buy and measure them, where the architecture is heading, and — most revealingly — how often that context is already failing them.
The central finding is a context gap — the distance between how confidently enterprise agents answer and how reliable the context beneath them actually is. A majority of enterprises (57%) report that in the past six months their AI agents produced confident but wrong answers they traced to missing or inconsistent business context, and more than half of those said it happened more than once. This is not a fringe failure: retrieval is the primary context source for 38% of enterprises, more than any other approach, so when retrieval is thin or inconsistent, the errors it produces are wearing the agent’s authority. The infrastructure to fix it is being built — 58% already run or are building a governed semantic layer — but for most it is not yet in production.
Underneath, the market is consolidating in a direction that surprises. Provider-native retrieval — OpenAI’s file search (40%) and Google’s Vertex AI Search (38%) — already leads every dedicated vector database, and enterprises expect hybrid retrieval to dominate by the end of 2026 (34%). Yet a plurality (36%) say they intend to keep best-of-breed standalone tools rather than consolidate onto a provider’s native context stack, and a majority (57%) plan to switch or add a provider within the year. Stated preference and actual usage are pulling in opposite directions — the market is buying provider-native while insisting it wants independence.
Methodology
VentureBeat fielded this survey as part of its ongoing Pulse Research series. This survey focused on enterprise RAG infrastructure and the context layer — the retrieval systems, semantic layers, and context sources that feed AI agents. Responses are filtered to organizations with more than 100 employees (n=101); the survey drew no responses from organizations of 100 or fewer, so the full sample qualifies. All responses are from a single Q2 2026 (June) wave, so the report reads cross-sectionally and does not infer month-over-month trends. Several questions were multiple-select, so those shares can sum to more than 100%.
By organization size the sample concentrates in the mid-market: 251–1,000 employees (31%) and 101–250 (31%) lead, with 1,001–5,000 (20%), 5,001–10,000 (12%), and 10,001+ (7%) above them. By role it spans managers (39%), individual contributors (27%), the C-suite (16%), and VPs and directors (14%); on purchasing authority it is buyer-credible, with 46% final decision-makers and another 26% recommenders or influencers. Technology/Software is the largest industry at 20%, followed by Healthcare/Life Sciences (11%) and a broad spread across retail, transportation, financial services, manufacturing, and education.
At 101 respondents this is a modest sample and should be read as a directional signal rather than a precise measurement; it is self-selected and is not a probability sample. It is best read as the view from organizations actively standing up RAG and context infrastructure rather than from the largest operators.
Finding 1: Confident and wrong
More than half have traced agent errors to bad context
We asked whether, in the past six months, enterprises had traced a confident but wrong agent answer to missing or inconsistent business context. Most had.
This is the report’s defining number. A majority of enterprises (57%) have already had an AI agent produce a confident, wrong answer they traced to bad context — wrong metrics, stale definitions, or missing documents — and more than half of those have seen it happen more than once. Only 28% report no such failure, and a small remainder either don’t run agents on enterprise data or don’t trace root cause closely enough to know.
The failure mode is specific and dangerous: the model is not obviously hallucinating; it is confidently wrong because the context feeding it was thin or inconsistent. Everything else in this report — what enterprises retrieve, how they govern it, and what they plan to build — is downstream of this problem.
Finding 2: RAG is the default context source
Retrieval feeds more agents than any other method
We asked what an enterprise’s AI agents primarily use to understand its data. Retrieval leads by a wide margin.
Retrieval is the backbone of enterprise context. For 38% of organizations, RAG over documents or a vector index is the primary way agents understand the business — nearly twice the share of the next approach, a governed semantic layer or ontology (21%). Mixed approaches (14%), direct live-system queries (10%), and long-context loading (6%) fill out the rest, and only 2% let agents run on the model’s general knowledge alone. The concentration matters in light of Finding 1: because so much enterprise context flows through retrieval, the quality of that retrieval is the quality of the answer. When RAG is the default source, thin retrieval is not an edge case — it is the main failure surface.
One approach is notable for its absence from these answers: customizing model weights, also known as fine-tuning. Every leading source of business context is injected at run time. Our most recent direct measurement of fine-tuning comes from our April–May survey wave (a separate survey, n=136), where fine-tuning capabilities ranked last of six factors in model selection at 5% — even as 26% of that sample still named fine-tuning and customization an investment they expect to grow. Fine-tuning has fallen out of the primary selection conversation; context injection is how enterprises make agents knowledgeable about their business.
Finding 3: Provider-native retrieval already leads the vector databases
OpenAI file search and vertex AI search top the dedicated tools
We asked which retrieval systems enterprises run in production today. The answer favors the model providers and hyperscalers over the specialists.
The dedicated vector database is no longer the center of the RAG stack. OpenAI’s file search (40%) and Google’s Vertex AI Search (38%) lead — provider-native and hyperscaler-native retrieval — ahead of every purpose-built vector database. Among the specialists, the most-used is the one enterprises already run for other reasons (Elasticsearch/OpenSearch, 20%) and the open, embedded option (pgvector, 12%); the pure-play vector databases that define the category — Weaviate, Qdrant, Pinecone, Milvus — each sit in single digits to low double digits. Notably, 13% of enterprises say they still run no production RAG at all. As with the platforms in the parallel infrastructure wave, enterprises are gravitating to retrieval that comes bundled with tools they already buy.
The shape of this finding held across both Q2 waves. In April–May (n=161), provider-built retrieval led usage there too, while every dedicated vector database remained marginal — the most-used standalone vector database peaked at 8% of that sample — and the hybrid, pluralistic future was already the consensus expectation (34% expected hybrid retrieval to dominate, with another 29% expecting multiple architectures by use case). Two waves, consistent picture: the category that coined the “vector database” term is being collected by the platforms enterprises already buy from.
Finding 4: But they say they want to keep best-of-breed
A plurality resist consolidating onto a provider’s native stack
We asked how enterprises will respond as model providers bundle retrieval, memory, and orchestration into their platforms. Their stated intent cuts against their current usage.
Here is the tension at the heart of the stack. Even as provider-native retrieval leads in practice (Finding 3), a plurality of enterprises (36%) say they intend to keep best-of-breed standalone tools rather than consolidate onto a provider’s native context stack — well ahead of the 21% who plan to consolidate. Another 21% expect a mix, and 9% intend to build and own the layer themselves. The gap between what enterprises run and what they say they want is the strategic question of the category: they are adopting bundled retrieval for convenience while asserting they will preserve independence. Which impulse wins — the pull of the provider bundle or the stated preference for modular control — will shape the retrieval market more than any single tool.
Finding 5: Hybrid retrieval is the consensus bet
Vector-only retrieval is already seen as insufficient
We asked which retrieval architecture enterprises expect to dominate their production RAG systems by the end of 2026. The field is converging — with a large share still unsure.
The architecture is settling on hybrid. A third (34%) expect hybrid retrieval — embeddings combined with reranking and access controls — to dominate their production systems by the end of 2026, three times the 11% who expect vector-only retrieval to prevail. That is a notable signal: the pure vector-search approach that launched the category is already viewed as insufficient on its own, superseded by pipelines that add reranking for accuracy and access controls for governance — the very access controls whose absence produces the failures in Finding 1. Tellingly, the second-largest answer is uncertainty: 17% simply don’t know, and another 14% expect to move beyond a dedicated vector layer entirely toward tool-first or long-context retrieval. The consensus is not a single tool but a layered pipeline — and it is not yet fully formed.
Finding 6: The governed context layer is being built now
Most run or are building a semantic layer — few in production
We asked whether enterprises use a governed semantic or context layer to give agents and BI a shared understanding of their data. Most are on the path; fewer have arrived.
The fix for the context gap is under construction. Well over half of enterprises (58%) either run a governed semantic layer in production (25%) or are piloting and building one (34%), and a further 17% are actively evaluating — meaning three-quarters are engaged with the idea in some form. But the balance is telling: more are building than have shipped, so for most enterprises the shared, governed definition layer that would prevent the "confident but wrong" failures of Finding 1 is still a work in progress. The semantic layer is the industry’s answer to inconsistent context; this wave catches it mid-construction, ambition well ahead of production.
Finding 7: Bought on ingestion and simplicity, watched for correctness
Selection favors operability; monitoring favors correctness and security
We asked what matters most when enterprises choose a retrieval system, and what they track once it is running. Both answers lean practical.
Enterprises choose retrieval systems on operability. Ease of data ingestion (36%), latency and performance (32%), and operational simplicity (29%) lead the selection criteria — ahead of retrieval accuracy and access control (23% each), the two factors most directly tied to the failures in Finding 1. Once systems are running, the emphasis shifts toward trust: the most-tracked metrics are response correctness (42%) and security and access control (38%), ahead of latency (28%), operational stability (27%), and answer relevance (23%).
Satisfaction with current systems is moderately positive but not enthusiastic — on a five-point scale, overall satisfaction averages 4.0, with ease of implementation and value for money both near 3.9. Enterprises buy for how easily a system runs and watch it for whether it can be trusted.
Finding 8: A retrieval reshuffle is coming
A majority plan to change providers — and the vector specialists are gaining interest
We asked whether enterprises plan to change or add a retrieval provider, and which they are considering. The consideration set differs from today’s stack.
The retrieval stack is not settled. While 43% have no plans to change, a small majority (57%) intend to switch or add a provider within twelve months, and a quarter (26%) within the next quarter. The consideration set is where it gets interesting: provider-native retrieval still leads what enterprises are evaluating (OpenAI 22%, Vertex AI Search 21%), but the open-source vector specialists punch above their current footprint — Qdrant (14%) and Milvus (13%) draw more switching interest than their present usage (10% and 6%) would suggest. Read with Finding 4, the picture is a market in flux: enterprises run provider-native today, are evaluating a broader field, and say they want to keep their options open. The reshuffle ahead will test whether best-of-breed intent survives contact with the convenience of the bundle.
The bottom line: A context gap that more retrieval alone won’t close
Organizations with more than 100 employees are wiring agents into their business faster than they can guarantee the context those agents run on. Retrieval is the default source of enterprise context, and it increasingly comes from the model providers and hyperscalers rather than the dedicated vector databases — yet a majority of enterprises have already watched agents answer confidently and wrongly because that context was thin or inconsistent. The failure is not exotic; it is the predictable result of pointing authoritative-sounding agents at an unreliable foundation.
The industry’s answer — a governed semantic layer, hybrid retrieval with reranking and access controls — is being built but is mostly not yet in production, and enterprises are pulled between the convenience of provider-native bundles and a stated preference for best-of-breed independence. At 101 respondents in a single Q2 wave this is a directional read, skewed toward the mid-market — but the direction is clear: the context layer is the next contested tier of the AI stack, and right now agents are running ahead of it. The context gap is not a retrieval-volume problem that more documents or bigger indexes will solve on their own; it is a problem of governed, consistent, access-aware context. The open question for later waves is whether enterprises finish building that layer before the confident-but-wrong failures move from the lab into decisions that matter.
Based on survey responses from 101 qualified enterprise respondents (100+ employees), drawn from a single Q2 2026 (June) wave. At this sample size the results should be read as a directional signal rather than a precise measurement — it's a self-selected sample, not a probability sample, and skews toward the mid-market. Respondents include managers, individual contributors, VPs/directors, and the C-suite, with strong purchasing authority, across technology, healthcare, retail, transportation, financial services, manufacturing, and education.
Composite 58Trend 10
The AI compute gap: Enterprises are buying infrastructure faster than they can measure what it costs
Across 107 enterprises, AI infrastructure spending is accelerating well ahead of the ability to see or steer its economics. Most organizations run their AI on a familiar base of hyperscalers and model-provider APIs, yet the next dollar is aimed at specialized compute almost none of them use today; a majority intend to switch or add providers within the year, many within a quarter. Buying decisions turn on integration and total cost of ownership rather than headline token price — which is fortunate, because most enterprises cannot yet see their unit economics clearly: GPUs sit at half utilization or less, and fewer than half rigorously track what their compute actually costs. The result is a compute gap — heavy, fast-moving investment running ahead of the visibility needed to control it.
This wave of VentureBeat Pulse Research examines enterprise AI infrastructure and compute: where organizations are in their deployment journey, what they run AI on today, how satisfied they are, what would make them switch, where they plan to evaluate their investments, and — most revealingly — how well they can measure and control the economics of the compute underneath it all.
The central finding is a compute gap — the distance between how aggressively enterprises are investing in AI infrastructure and how little of its economics they can see. Only about one in five (21%) run AI in production at scale, yet spending intentions are outrunning that maturity: the single largest planned area enterprises plan to evaluate over the next year is AI-specialized clouds (45%), a layer almost none of these enterprises use today. Meanwhile the compute already in place runs cold — 83% report GPU utilization of 50% or less — and fewer than half (44%) can rigorously track what their AI compute costs. Enterprises are buying more infrastructure faster than they can account for what they already own.
Enterprises are not settled on their infrastructure vendors, either: A clear majority (64%) plan to switch or add an infrastructure provider within twelve months, and 38% within the next quarter — unusually high churn intent for a category this foundational. When they choose, they choose on integration with the existing stack (41%) and total cost of ownership (35%), not on headline price: cost per million tokens is the deciding factor for just 8%. And the frontier constraint that will shape the next round of decisions — the shift from GPU compute to memory bandwidth as inference scales — is barely on the radar, with roughly one in five enterprises either unaware of it or yet to address it.
Methodology
VentureBeat fielded this survey as part of its ongoing Pulse Research series, this survey focused on enterprise AI infrastructure, compute, and inference economics. Responses are filtered to organizations with more than 100 employees (n=107; the survey’s smallest size band, 1–100 employees, is excluded), drawn from a single Q2 2026 (June) wave. Because this is one wave rather than a pooled multi-month sample, the report reads cross-sectionally and does not infer month-over-month trends. Several questions were multiple-select, so those shares can sum to more than 100%.
By organization size the sample concentrates in the mid-market: 101–250 employees (36%) and 251–1,000 (27%) lead, with 1,001–5,000 (22%), 5,001–10,000 (8%), and 10,001+ (7%) above them. By role it spans managers (38%), individual contributors (28%), VPs and directors (19%), and the C-suite (13%); on purchasing authority it is buyer-credible, with 45% final decision-makers and another 30% recommenders or influencers for AI solutions. Technology/Software is the largest industry at 26%, followed by Healthcare/Life Sciences (15%), Financial Services (13%), and Retail/E-commerce (12%).
At 107 respondents the sample is large enough to read directionally but should be treated as a directional signal rather than a precise measurement; it is self-selected and is not a probability sample. It also skews toward the mid-market and toward earlier-stage adopters, so it is best read as the view from organizations actively building out AI infrastructure rather than from the largest hyperscale operators.
Finding 1: Ambition outpaces production
Only one in five run AI in production at scale
We asked where organizations sit in their AI deployment journey. Most are still building toward production rather than operating at scale.
38%
are experimenting — running proofs of concept, not yet in production
37%
have some workloads in production, but not across the organization
21%
run AI in production at scale — the mature minority
4%
are not yet running AI workloads at all
The maturity curve is front-loaded. Three-quarters of enterprises (76%) are either experimenting or running only some workloads in production, and just 21% describe AI in production at scale. This matters for everything that follows: the infrastructure decisions in this report are being made largely by organizations still early in deployment, whose compute footprint — and whose costs — are about to grow. The evaluation and switching intentions in Findings 3 and 4 are the leading edge of that build-out, not the settled preferences of operators who have already found what works.
Finding 2: Enterprises run on hyperscalers and model APIs
The specialized GPU clouds barely register — today
We asked which providers and platforms enterprises currently use to run their AI. The answer is a familiar one: the incumbents.
48%
use Google Cloud — the most-used platform overall (Microsoft Azure 29%, AWS 22%, Oracle Cloud 22%)
41%
use Google’s Gemini models, with OpenAI close behind at 40% and Anthropic at 12%
6%
run their own on-prem or co-located GPU clusters; 4% a custom open-source self-managed stack
<2%
each use the specialized AI clouds — CoreWeave, Lambda, Crusoe, Nebius, Together, Fireworks and peers
The current stack is hyperscaler-and-API. Google Cloud leads at 48%, and the general-purpose clouds (Google, Microsoft, AWS, Oracle) together with the major model APIs (Gemini, OpenAI, Anthropic) account for essentially all current deployment. The specialized “neocloud” GPU providers that dominate AI-infrastructure headlines — CoreWeave, Lambda, Crusoe, Nebius and peers — register at or near zero among these enterprises today. Only 6% run their own on-prem GPU clusters and 4% a custom open-source stack. Enterprises are, for now, running AI on the providers they already buy from — which makes the evaluation intentions in Finding 3 all the more striking.
(A note on reading these shares. As described in the methodology section, this sample is self-selected and skews mid-market, and this question counted every provider a respondent uses — an average of 2.1 selections each — so the figures measure presence in the stack rather than spending or primary status. A sample built this way will show a different provider mix than a spend-weighted census of the broader market; Google's strength here, for example, is consistent with its long-standing position among smaller enterprises building on AI. Read these shares as a portrait of what this AI-active cohort runs today, and treat gaps between these figures and industry-wide market share estimates as a property of the sample rather than a contradiction of either.)
Finding 3: The next dollar goes to infrastructure they don’t yet run
AI-specialized clouds top the evaluations list
We asked where enterprises planned to evaluate AI infrastructure over the next 12 months. Their answers point away from the stack they run today.
45%
AI-specialized clouds (CoreWeave, Lambda, Crusoe, Nebius) — the top planned evaluation area
32%
non-NVIDIA accelerators (AWS Trainium, Google TPU, AMD Instinct, Intel Gaudi, in-house ASICs)
28%
Nvidia Blackwell (GB300) / next-generation GPUs
16%
decentralized or distributed compute networks
11%
sovereign or region-specific compute; 9% say none of the above
Here is the report’s sharpest tension. The single most-cited planned evaluation area — AI-specialized clouds, at 45% — is the very category almost none of these enterprises use today (Finding 2). Nearly a third (32%) intend to evaluate non-Nvidia accelerators, and 28% in next-generation Nvidia silicon; even decentralized compute networks (16%) and sovereign compute (11%) draw meaningful interest. Read against current usage, this is not incremental — it is the leading edge of a re-platforming. The direction-of-travel question tells the same story: every infrastructure approach is net-expanding, but specialized AI clouds carry the highest net momentum (+24), edging out even the hyperscalers (+22). Enterprises are preparing to move a meaningful share of AI compute off the general-purpose cloud.
This continues a trend we saw in our April-May survey wave. Back then, usage of the AI-specialized clouds was equally marginal — CoreWeave at 3%, Lambda at 4%, Crusoe at 2% of enterprises. When we asked enterprises what change they planned in their AI infrastructure strategy over the next twelve months, the most-cited answer was moving workloads to specialized AI clouds, at 33%. Asked in April-May which emerging compute option they were most likely to evaluate AI-specialized clouds again drew the most responses. Two waves, two differently worded questions, one consistent picture: the type of cloud enterprises are most eager to assess is the type they have barely begun to use.
Finding 4: A switching wave is building
Six in 10 plan to change providers within a year — many within a quarter
We asked whether and when enterprises plan to switch or add an infrastructure provider. Very few intend to stand still.
38%
plan to change within the next 0–3 months — tied for the most common answer
36%
have no plans to change
22%
plan to change within 3–6 months
7%
plan to change within 6–12 months
For a category as foundational as compute, this is a remarkable amount of intended movement. Only 36% have no plans to change, meaning a clear majority (64%) intend to switch or add a provider within twelve months — and 38% within the next quarter alone. Where that interest points is telling: the providers drawing the most switching consideration are again the incumbents — Microsoft Azure and Google Cloud (33% each), OpenAI (30%), and Gemini (22%) — which suggests much of the near-term movement is reshuffling among the majors and consolidating spend rather than defecting to new entrants. The neocloud interest in Finding 3 is a 12-month evaluation thesis; the switching in the next quarter is mostly incumbents trading share.
(Method note: Respondents who selected both "no plans to change" and a specific switching window are counted as switchers, on the logic that naming a timeframe is the more specific answer; three respondents were reclassified under this rule.)
Finding 5: Nobody buys on token price
Integration and total cost of ownership decide — not sticker price
We asked what matters most when enterprises select an AI infrastructure provider. Headline price finished last.
41%
integration with the existing cloud and data stack — the top factor
35%
total cost of ownership (TCO)
24%
performance — latency and throughput
19%
each cite security/compliance, autoscaling for spiky workloads, and GPU access/availability
8%
cost per 1M tokens — the least-cited factor
Enterprises do not buy AI infrastructure on pricing, which is the place vendors compete on hardest. Integration with the existing stack (41%) and total cost of ownership (35%) dominate, while the headline metric — cost per million tokens — is the deciding factor for just 8%, dead last. The pattern is coherent: buyers are optimizing for how a provider fits and what it truly costs to operate, not for the advertised unit rate. It also foreshadows Finding 7 — enterprises say TCO matters most, yet most cannot yet measure it rigorously. The stated priority and the measured capability are out of step.
Finding 6: Expensive GPUs, idle most of the time
83% report GPU utilization of 50% or less
We asked what share of their GPU capacity enterprises actually utilize. The answer is a well-known but rarely quantified inefficiency.
37%
run at 26–50% utilization
34%
run at 10–25% utilization
15%
run under 10% utilization
12%
run over 50% — the efficient minority
8%
don’t measure utilization at all; a further 7% consume via API and run no GPUs of their own
Disclosure: Band percentages count every selection against all 107 qualified respondents; 14 respondents selected more than one band, so bands overlap. At the respondent level, 83 of the 100 GPU-operating enterprises reported utilization at or below 50%
The compute already in place runs cold. Adding the bands at or below half capacity, 83% of enterprises that operate GPUs report utilization of 50% or less, and nearly half (49%) run at 25% or below. Only 12% clear the 50% mark, and a further 8% do not measure utilization at all. Idle accelerators are expensive accelerators, and this is the clearest single measure of the compute gap: enterprises are planning to buy more GPUs and specialized compute (Finding 3) while the capacity they already own sits substantially unused. The efficiency headroom in the current fleet is large — and largely unmeasured.
Finding 7: Spending fast, measuring slowly
Fewer than half rigorously track what their compute costs
We asked whether enterprises can quantify the cost and return of their AI infrastructure spend, and how satisfied they are with what they run. Confidence in the ledger lags the spending.
44%
track compute cost and ROI rigorously
39%
track it only partially
20%
can’t quantify it yet
6%
say it isn’t a priority
Measurement trails money. Fewer than half of enterprises (44%) rigorously track the cost and return of their AI compute; the majority track only partially (39%), cannot quantify it yet (20%), or have not prioritized it (6%). That gap is consequential given Finding 5, where total cost of ownership was the second-ranked buying criterion — enterprises are choosing providers on an economic basis they mostly cannot yet measure. Satisfaction with current infrastructure is moderately positive but not enthusiastic: on a five-point scale, overall satisfaction averages 4.0, with ease of implementation (3.8) and value for money (3.9) trailing slightly — the softness landing, tellingly, on cost. Enterprises are spending quickly and accounting slowly.
Finding 8: The next bottleneck few are watching
As inference shifts from compute to memory, the field scatters
Finally, we asked how enterprises would address the emerging constraint in large-scale inference — the shift from GPU compute to memory, specifically KV-cache capacity. The responses reveal a frontier that is not yet a priority.
31%
would rely on Dell (PowerScale / Project Lightning) — the leading single answer
16%
would rely on Nvidia (Dynamo / ICMSP)
18%
are not aware of this as a constraint (9%) or haven’t addressed inference-memory limits yet (8%)
10%
Hammerspace (Tier Zero); 9% DDN (Infinia); the rest split across open-source KV-cache tooling, model-level efficiency, VAST Data, and WEKA
The memory frontier is real but barely governed. Asked which approach they would rely on as the binding constraint in inference shifts from compute to memory bandwidth, enterprises scatter: Dell leads at 31%, Nvidia follows at 16%, and the rest fragments across storage vendors, open-source tooling, and model-level efficiency techniques. Most telling is that roughly one in five (18%) either do not recognize the constraint or have not begun to address it. For a shift that will reshape inference cost and architecture, this is an early and unsettled market — and, consistent with the measurement gap in Finding 7, one where many enterprises simply do not yet have a view. It is the next chapter of the compute gap, arriving before most have closed the current one.
The bottom line: A compute gap that faster spending will widen, not close
Organizations with more than 100 employees are investing in AI infrastructure faster than they can measure it. Most are still early in deployment, yet their spending intentions point past their current stack — toward specialized clouds and alternative accelerators almost none of them run today — and a clear majority intend to change providers within the year. They buy on integration and total cost of ownership rather than headline price, which is rational; the difficulty is that most cannot yet see those economics clearly.
The visibility gap is concrete. The GPUs enterprises already own run at half utilization or less for the overwhelming majority, and fewer than half can rigorously track what their compute costs or returns. Satisfaction is decent but unenthusiastic, softest on value for money — the dimension hardest to judge without measurement. And the next constraint, the shift from compute to memory in large-scale inference, is arriving while most enterprises are still unaware of it. At 107 respondents in a single Q2 wave this is a directional read, skewed toward the mid-market and earlier-stage adopters — but the direction is consistent: the appetite to spend is running well ahead of the instrumentation to spend well. The compute gap is not a capacity problem that more hardware will solve on its own; it is, first, a problem of seeing what the hardware already costs. The open question for later waves is whether enterprises build that visibility before the re-platforming arrives — or buy the next layer of infrastructure as blind to its economics as the last.
Based on survey responses from 107 qualified enterprise respondents (100+ employees), drawn from a single Q2 2026 (June) wave. Because this is one wave rather than a pooled multi-month sample, the results read cross-sectionally rather than as a month-over-month trend, and at 107 respondents this is a directional signal rather than a precise measurement — the sample is self-selected, skews mid-market, and leans toward earlier-stage adopters rather than the largest hyperscale operators. Respondents include managers, individual contributors, VPs/directors, and the C-suite, with buyer-credible purchasing authority, across Technology/Software, Healthcare/Life Sciences, Financial Services, Retail/E-commerce, and other industries.
Composite 52Trend 10
Agentic orchestration: Enterprise AI organizations have a deployment problem, not a platform problem
Across 101 enterprises, agent orchestration is consolidating onto model-provider platforms — Anthropic’s Claude leads by a wide margin — chosen for the gravity of the underlying model and judged on reliable multi-step execution. But the ambition runs well ahead of the reality: most deployed “agents” are still chatbot wrappers, the control plane enterprises expect is deliberately hybrid to avoid lock-in, and real-time fiscal control over token burn remains the exception.
This wave of VentureBeat Pulse Research examines enterprise agent orchestration: which platforms enterprises run on, what drives the choice, what they optimize for, how they expect agent control to be structured, and — most revealingly — how orchestrated their deployed “agents” actually are and how tightly they control the cost of running them.
The central finding is a gap between orchestration ambition and orchestration reality. Enterprises are consolidating fast onto the major model platforms: Anthropic’s Claude is the primary platform for 40%, more than double any rival, followed by Microsoft (18%) and OpenAI (13%). The choice is driven by “model gravity” — native alignment with a state-of-the-art base model (21%) — and success is judged by reliable, multi-step execution (task completion reliability 32%, multi-step workflow management 28%). Yet asked to assess their portfolios honestly, 71% say a quarter or fewer of their deployed “agents” are true multi-step orchestrated workflows rather than single-prompt chatbot wrappers, and only 10% have crossed the halfway mark. The orchestration layer is being built well ahead of the orchestrated portfolio it is meant to run.
That gap shapes the architecture enterprises are putting in place. By the end of 2026 a clear majority (51%) expect a hybrid control plane — provider-native plus external orchestration — and only 6% expect to hand control to a provider-managed service, because vendor lock-in (35%) is the risk they fear most if control lives inside a model provider. Investment follows the build-out: agent workflow tooling leads the spend (34%), with security and permissions enforcement (25%) behind. And fiscal control lags throughout — more than a quarter (27%) have no real-time way to stop a runaway agent before the bill arrives.
Methodology
VentureBeat fielded this survey as part of its ongoing Pulse Research series, this instrument focused on enterprise agent orchestration. Responses are filtered to organizations with 100 or more employees (n=101), drawn from a single June 2026 wave; because this is one wave rather than a pooled multi-month sample, the report reads cross-sectionally and does not infer month-over-month trends.
By organization size the sample is spread evenly across the enterprise bands: 100–499 employees, 2,500–9,999, and 50,000+ (21% each), with 10,000–49,999 and 500–2,499 (19% each). By role it is senior and buyer-credible: product and program managers (15%), CIO/CTO/CISO (13%), consultants and advisors (13%), and a spread of data, AI, and engineering directors and VPs, with an “Other” function at 18%. On purchasing, 81% are recommenders, influencers, or final decision-makers for AI solutions (66% recommender/influencer, 15% final decision-maker). Technology/Software is the largest industry at 44%, followed by Financial Services (17%) and Healthcare/Life Sciences (8%).
At 101 respondents the sample is robust enough to read directionally with reasonable confidence, though it remains self-selected and is not a probability sample.
Finding 1: Orchestration runs on model-provider platforms
Anthropic’s Claude leads; open frameworks are marginal
We asked which agent orchestration platform enterprises primarily use today. The answer concentrates on the major model providers — and on one in particular.
A note on reading these shares. As described in the methodology section, the respondents are self-selected, and this question asked them for a single primary platform — so the figures measure which platform leads each enterprise's deployment, within a self-selected audience of AI-active technical decision-makers. A sample built this way can diverge substantially from spend-weighted market measures, and each VB Pulse survey draws its own sample with its own company-size mix, so vendor figures should not be compared across our surveys either. Read these shares as a portrait of where this cohort has placed its primary orchestration bet today, rather than as market share.
The model platforms dominate. Anthropic, Microsoft, OpenAI, Google, and Amazon together account for roughly 80% of deployments (81 of 101), while the open frameworks (LangChain/LangGraph) and custom in-house builds that anchor engineering discussion sit in single digits. Anthropic’s lead — 40%, more than double the next platform — mirrors the “model gravity” selection logic in Finding 2: enterprises are choosing the orchestration layer that comes with the model they want to build on. As with the security vendors in the prior agent-security wave, the tools that define the category in technical circles are not yet where enterprise deployment concentrates. A small 3% are not orchestrating at all.
Respondents rate the platforms they run at 3.94 out of 5 overall (109 answered), with “value for money” specifically at 3.94 and “ease of implementation” the weakest score, at 3.85 — placing orchestration near the bottom of our five-tracker satisfaction range, ahead of only evaluation tooling. A rating just under 4 out of 5, from users of whom 96% plan to change their orchestration approach within the year, reads as provisional acceptance: the platforms work well enough to run today, and not well enough to stop the search for something better. The ratings sit alongside near-universal intent to change; this is a layer enterprises tolerate more than they love.
Finding 2: Model gravity drives platform selection
The base model, not the tooling, decides the platform
We asked what most influenced the orchestration platform choice. The single largest factor is the pull of the underlying model — though flexibility and ease of development follow close behind.
Model gravity leading is the selection-side explanation for Anthropic’s platform lead: enterprises pick the orchestration environment closest to the frontier model they have standardized on. But the next tier complicates the picture — flexibility across models and tools (17%) and ease of development (17%) say enterprises also want to avoid being trapped by that choice, foreshadowing the lock-in fear in Finding 6. Security and permissions (14%) and total cost of ownership (11%) round out a pragmatic buying logic. Performance (latency/memory) sits last at 4%, a reminder that at this stage of adoption the binding constraints are model fit and optionality, not raw speed.
Finding 3: The job is reliable multi-step execution
Enterprises just orchestration by whether it completes the work
We asked what enterprises optimize for — their primary success metric for orchestration. Reliability and multi-step workflow management dominate; developer- and user-facing metrics trail.
Task completion reliability (32%) and multi-step workflow management (28%) together account for 59% of responses (60 of 101): orchestration succeeds, in the enterprise view, when it reliably carries a task through multiple steps to completion. Developer productivity (17%) matters but is secondary — the inverse of its prominence in framework discussion — and end-user experience (9%) is a minor concern, consistent with orchestration being an internal execution problem rather than a UX one. This reliability-first standard is exactly what makes the Chatbot Trap finding so pointed: enterprises define success as dependable multi-step execution, yet most of their deployed “agents” do not yet do multi-step work at all.
The trap is not evenly distributed. Splitting the sample by organization size, 77% of smaller enterprises say a quarter or fewer of their agents do true multi-step work, against 62% of larger ones. Larger enterprises are meaningfully further into genuine multi-step deployment; the chatbot trap is, directionally, a mid-market condition.
Finding 4: Consolidate, productionize, and build in-house
Three strategic moves are nearly tied for the year ahead
We asked what major change enterprises anticipate in their orchestration strategy over the next 12 months. Three moves cluster at the top, almost evenly split.
The top three — building in-house control (25%), standardizing on one framework (24%), and moving agents from sandbox to production (23%) — are statistically indistinguishable and tell a single story: enterprises are moving from experimentation to operational consolidation. They want fewer frameworks, more production exposure, and more ownership of the control layer; only 4% expect no change. The appetite for custom in-house control planes is notable alongside the platform concentration in Finding 1 — enterprises are standardizing on model-provider platforms while simultaneously planning to wrap them in control logic they own, the hybrid posture that Finding 6 makes explicit.
Finding 5: Investment flows to workflow tooling
Tooling and permissions lead the spend; monitoring trails
We asked which orchestration-related investment will grow most next year. Agent workflow tooling leads, with security and permissions enforcement behind.
Workflow tooling leading (34%) is the budget-side expression of the reliability-and-multi-step priority in Finding 3: the money is going to the machinery that strings steps together dependably. Security and permissions enforcement (25%) and scaling infrastructure (20%) follow — the investments required to take agents from sandbox into production, the strategic move in Finding 4. Monitoring and debugging draws a smaller 11%, with another 11% reporting flat budgets. The weight on tooling, permissions, and scaling over pure observability signals that enterprises are spending to build and harden orchestration, not merely to watch it run.
Finding 6: The control plane will be hybrid — and lock-in is why
Enterprises expect to split control between providers and their own layer
We asked where enterprises expect the primary control plane for agents to live by the end of 2026, and what worries them most if that control sits inside a model-provider platform. A clear majority expect a hybrid model — and vendor lock-in is the reason.
Hybrid control is the dominant expectation by a wide margin (51%), and only 6% expect to hand control to a provider-managed service outright. Read together, the hybrid, custom, and externally-abstracted options — every architecture that keeps control at least partly outside the provider — sum to 88% (89 of 101). The reason surfaces directly when we asked about the risk of provider-resident control: vendor lock-in leads at 35% (35 of 101), ahead of security and permissioning limitations (28%) and inflexibility across models and tools (21%). The pattern echoes the prior wave’s “don’t trust the model to police itself” posture — here, enterprises will build on a provider’s platform but decline to be governed entirely by it. The hybrid control plane is the architectural hedge against the lock-in they most fear.
The June figure asserting a preference for a hybrid control plane marks movement from earlier. In the April–May survey (n=145), only 34% expected a hybrid control plane, and a greater number (12%) expected to hand control fully to a provider-managed service. These two snapshots don’t yet measure a confirmed longitudinal trend — but the direction of the conversation is unambiguous: toward keeping control.
Lock-in is also a new arrival as a top concern. In the April–May wave, the leading concern was security and permissioning limitations (32%), with lock-in second at 24%; by June the two had traded places. The worry about provider platforms appears to be maturing from whether they can be secured to whether they can be replaced.
Finding 7: The chatbot trap — most “agents” aren’t agents yet
Enterprises admit most deployments are still chatbot wrappers
We asked enterprises to assess their portfolios honestly: what share of their deployed “agents” are true multi-step orchestrated workflows versus simple single-prompt chatbot wrappers. The answer is the defining finding of this wave.
This is the gap at the center of the report. Combining the bottom two bands, 71% of enterprises (72 of 101) say a quarter or fewer of their deployed “agents” are genuinely orchestrated — and just 10% (10 of 101) have crossed the halfway mark. The ambition documented in the earlier findings — model-provider platforms, reliability-first success metrics, production rollouts, a deliberate control architecture — runs well ahead of the deployed reality, which remains overwhelmingly single-prompt assistants dressed as agents. This is less a contradiction than a roadmap: the platforms, budgets, and strategies are being put in place precisely because the orchestrated portfolio is still so thin. The open question for later waves is how fast the reality closes on the ambition.
Finding 8: Fiscal control is still reactive
Only a minority can stop a runaway agent before the bill arrives
Finally, we asked how enterprises enforce fiscal control over agent token consumption — the risk that an autonomous loop exhausts a budget before anyone intervenes. Most rely on native caps or after-the-fact monitoring; real-time programmatic control is the exception.
More than a quarter of enterprises (27%) admit they have no real-time, programmatic way to stop an agent before a budget-breaking bill arrives — they learn of it from the logs afterward. Another 32% lean entirely on the native caps and throttles built into their primary platform, a control only as good as the provider’s tooling and one that ties back to the lock-in concern of Finding 6. The enterprises building custom gateways (23%) or exploiting cross-model routing to arbitrage cost (19%) are the ones treating token burn as an engineering problem to be controlled deterministically. As with orchestration maturity, fiscal control is an area where the operational reality lags the ambition: agents are moving toward production faster than the cost-control plane around them is being built.
It’s worth noting, a split appears according to company size: roughly one in three enterprises under 2,500 employees (34%) exercises only reactive control of agent spend, against 20% of larger enterprises — directional figures, but consistent with the chatbot-trap split. The mid-market is running the least mature agents on the least instrumented budgets.
The bottom line: The layer is real; most of the agents aren't yet
Organizations with 100 or more employees describe an orchestration strategy that is consolidating quickly and maturing slowly. They are standardizing on model-provider platforms — Anthropic’s Claude leads at 40% — chosen for the gravity of the underlying model, and they judge success by reliable multi-step execution. Investment is flowing to workflow tooling and permissions, the strategy is to consolidate frameworks and push agents into production, and the control plane they expect is deliberately hybrid, because vendor lock-in is the risk they fear most.
But the honest self-assessment punctures the ambition. Seventy-one percent say a quarter or fewer of their deployed “agents” are truly orchestrated, only 10% are past the halfway mark, and more than a quarter cannot stop a runaway agent in real time. The orchestration layer — the platforms, the budgets, the control architecture — is being built ahead of the orchestrated portfolio it is meant to run. At 101 respondents in a single June wave this reads as a clear directional signal rather than a precise measurement: enterprises have decided how they want to orchestrate agents well before most of their agents are doing anything an orchestration layer is for. The question for subsequent waves is whether the deployed reality closes the gap on the ambition — or whether the chatbot trap proves stickier than the roadmap assumes.
Based on survey responses from 101 qualified enterprise respondents (100+ employees), drawn from a single June 2026 wave. Because this is one wave rather than a pooled multi-month sample, results read directionally rather than as a confirmed trend. Respondents include product and program managers, CIOs, CTOs and CISOs, consultants and advisors, and directors and VPs of data, AI, and engineering, across Technology/Software, Financial Services, Healthcare, and other sectors.
Composite 59Trend 10
Google just redesigned the search box for the first time in 25 years — here’s why it matters more than you think.
For a quarter century, the Google search box has been one of the most recognizable interfaces in computing: a thin white rectangle, a blinking cursor, a few typed words, and a list of blue links. On Tuesday, Google will formally retire that paradigm.
At its annual I/O developer conference, Google announced a sweeping redesign of the search box itself — the literal text field where billions of queries begin every day — transforming it from a simple keyword input into a dynamic, AI-driven conversation starter that can accept text, images, PDFs, videos, and even open Chrome tabs as inputs. The company is also merging its AI Overviews and AI Mode features into a single, seamless search flow, eliminating the friction that previously forced users to choose between a traditional results page and an AI-forward experience.
Liz Reid, Google's vice president and head of Search, called it "the biggest upgrade to our iconic search box since its debut over 25 years ago" during a press briefing on Monday.
The announcement arrived alongside a blizzard of other news — new Gemini models, a personal AI agent called Spark, an intelligent shopping cart, a reimagined developer platform — but the search box redesign may prove to be the most consequential. It is the clearest signal yet that Google views the future of its flagship product not as a place where users type fragmented keywords, but as an interface where they hold open-ended, multimodal conversations with an AI system backed by the entire web.
The new search box expands, accepts files, and coaches you on what to ask
The changes show a fundamental shift in how Google expects people to interact with the product that generates the vast majority of Alphabet's revenue.
The box itself now dynamically expands to accommodate longer, more conversational queries. Where the old interface subtly encouraged brevity — a narrow field suited to two- or three-word keyword strings — the new design invites users to fully articulate complex questions in granular detail. It also now supports multimodal inputs directly. Users can upload images, PDFs, files, and videos, or drag in content from Chrome tabs, right from the main search interface. Previously, some of these capabilities existed in AI Mode, but reaching them required extra steps. Now they sit at the primary entry point.
Google is also deploying what it describes as an AI-powered query suggestion system that "goes beyond autocomplete." Rather than simply predicting the next word a user might type based on popular searches, the system helps users formulate complex, nuanced queries — essentially coaching them toward the kind of detailed questions that AI Mode handles best.
The new search box is starting to roll out immediately in all countries and languages where AI Mode is available.
Google is merging AI overviews and AI mode into one seamless experience
Perhaps more significant than the box itself is the architectural change happening behind it. Google is unifying AI Overviews — the AI-generated summary panels that appear atop traditional search results — with AI Mode, the more immersive conversational search experience the company launched at I/O one year ago.
Starting Tuesday, this merged experience will be live across mobile and desktop worldwide. A user can type a question, receive an AI Overview alongside traditional results, and then continue directly into a back-and-forth AI Mode conversation to ask follow-up questions — all without navigating to a separate interface.
Reid explained the logic during the press briefing: the new AI search box is "an upgrade of our traditional search box, and so the results take you directly to main search rather than AI mode." She noted that while some power users actively sought out AI Mode, "for most users, they don't actually want to have to think about, do they want more of a traditional page or an AI-forward search experience."
The goal, she said, was to ensure that "for most users, they don't have to think about where to go, they can just go to the search box they're familiar with, and it feels like they get the best experience afterwards."
One billion users and doubling queries reveal how fast search behavior is shifting
Google's decision to redesign the foundational interface of its most important product did not happen in a vacuum. The company shared a set of usage statistics during the briefing that reveal just how rapidly user behavior is already changing.
AI Mode, which launched in the United States at I/O 2025, has surpassed one billion monthly users in its first year. AI Mode queries have been doubling every quarter since launch. AI Overviews, the lighter-weight AI summaries, now reach more than 2.5 billion monthly users. And overall search query volume hit an all-time high last quarter — a data point the company had previously disclosed on its earnings call.
Sundar Pichai, Google's CEO, framed these figures as evidence that AI features are additive, not cannibalistic, to search usage. "When people use our AI-powered features in search, they use search more," he said. He added that he loves "how search has become less about individual queries and feels more like an ongoing conversation, giving users deeper insights and connecting you with the vastness of the web."
Reid reinforced the point: "It's not just that people are searching more, it's that they're searching differently. They're fully expressing their questions in granular detail, asking those follow-up questions and searching across modalities."
Gemini 3.5 Flash gives Google's AI search the speed it needs to work at scale
Under the hood, the new search experience runs on Gemini 3.5 Flash, Google's newest AI model, which the company also introduced at I/O. Google upgraded AI Mode's underlying model to 3.5 Flash to deliver what Reid described as "an even more powerful AI search experience."
Gemini 3.5 Flash is the workhorse of this year's announcements. Google claims it outperforms its previous frontier model, Gemini 3.1 Pro, on nearly all benchmarks while running four times faster in output tokens per second than comparable frontier models. Pichai described it as being "in a league of its own in the top right quadrant" of the Artificial Analysis index, which plots intelligence against speed — meaning it delivers near-frontier quality at dramatically lower latency.
That speed matters enormously for search. A conversational AI search experience that feels sluggish would be dead on arrival for a product that serves billions of queries daily. By coupling the redesigned interface with a model optimized for both quality and throughput, Google is attempting to make AI-powered search feel as instantaneous as the old keyword experience — while being dramatically more capable.
Search can now build interactive visuals and custom mini apps on the fly
The redesigned search box is also the gateway to a set of new capabilities that push search far beyond text-based answers. Google announced what it calls "generative UI" — the ability for search to dynamically build custom widgets, interactive visualizations, and even mini applications in real time, tailored to a user's specific question.
Reid offered a concrete example during the briefing: a user could ask "How do black holes affect space time?" and receive an interactive visual in an AI Overview that brings the concept to life. Follow-up questions would trigger the system to dynamically generate entirely new visuals in real time. This is possible, she explained, because of "a novel real-time code generation system we built in partnership with the Google DeepMind team" that runs on Gemini 3.5 Flash. Generative UI capabilities will roll out to everyone this summer, free of charge.
But Google is going further still. For ongoing tasks — planning a wedding, organizing a move, tracking a fitness routine — users will be able to build what the company describes as customizable, stateful experiences within search, powered by its Antigravity development platform. These require no coding expertise. Users simply describe what they want in natural language, and search builds it. Those experiences will be available in coming months, starting with Google AI Pro and Ultra subscribers in the United States.
AI agents that monitor the web around the clock are coming to search results
The redesign also opens the door to what Google calls "information agents" — AI agents that users can configure directly within search to monitor the web 24/7 for specific conditions and deliver synthesized updates when those conditions are met.
A user could, for example, set up an agent to track market movements in a particular sector with specific parameters. The agent would create a monitoring plan, tap into real-time finance data, and proactively notify the user when conditions are met — complete with links and context for further research. Other use cases include apartment hunting, tracking sneaker drops, or monitoring any topic a user cares about. Information agents will launch first for Google AI Pro and Ultra subscribers this summer.
These agents sit within a much larger strategic pivot that Google articulated throughout the briefing: the company is going all-in on AI systems that don't just answer questions but proactively take actions on users' behalf. Beyond search, Google introduced Gemini Spark, a 24/7 personal AI agent that runs on dedicated virtual machines in Google Cloud. It unveiled the Universal Cart, an intelligent cross-merchant shopping cart. It announced the Agent Payments Protocol for agents to make secure purchases. And it expanded its Antigravity developer platform into a full ecosystem for building autonomous AI agents.
Publishers, advertisers, and SEO professionals face a new reality
The redesign raises profound questions for the sprawling ecosystem — publishers, advertisers, SEO professionals — that has been built around the old model of keyword search and blue links.
If users increasingly express their needs as full, conversational sentences rather than fragmented keywords, the entire discipline of search engine optimization will need to evolve. Keyword-density strategies become less relevant when the AI is parsing natural language intent rather than matching strings. Content that answers deep, nuanced questions in authoritative ways becomes more valuable; content engineered to rank for two-word keyword fragments becomes less so.
For publishers, the stakes are existential. AI Overviews already synthesize information from across the web and present it directly in search results, reducing the need for users to click through to source material. The new seamless AI Mode integration deepens that dynamic: users can now get an AI-generated answer and ask multiple follow-up questions without ever leaving the search page. Google has consistently maintained that its AI features drive more traffic to publishers, but the redesign puts that claim under renewed scrutiny as the search results page becomes more self-contained.
For advertisers — who fund the vast majority of Google's revenue — the shift from keywords to conversations changes the calculus of ad targeting. Conversational queries contain richer intent signals, which could make ad targeting more precise and valuable. But they also create new ambiguities: when a user is in the middle of a multi-turn conversation with AI Mode, where does an ad naturally fit? Google did not detail changes to its advertising model during the briefing, but the structural shift in the interface will inevitably reshape how ads are surfaced and measured.
The search box was always more than a product — it was a habit for billions of people
There is a reason Google chose to redesign the search box rather than simply adding new features behind it. The search box is not just a product element at this point; it is a cultural artifact — one of the few pieces of digital infrastructure used by essentially the entire internet-connected world. Changing it sends an unmistakable message about where the company believes computing is headed.
For 25 years, the search box trained billions of people to think in keywords — to compress their curiosity into the shortest possible string of words. The new box invites them to do the opposite: to think out loud, to upload what they're looking at, to ask follow-up questions, to let an AI system handle the compression.
Pichai tied the company's broader ambitions to a striking statistic: Google's surfaces now process over 3.2 quadrillion tokens per month, up seven-fold from a year ago. The company expects capital expenditures of approximately $180 to $190 billion in 2026 — roughly six times the $31 billion it spent four years ago — largely to support the infrastructure required for this AI transformation. When asked about the future of traditional search, he was direct. "Search is the most used AI product in the world," he said.
The blinking cursor in Google's search box still invites you to type. But after 25 years of teaching the world to speak in keywords, Google is now asking it to speak in sentences — and betting roughly $190 billion that it will.
Composite 25Trend 10
The creator of Claude Code just revealed his workflow, and developers are losing their minds
When the creator of the world's most advanced coding agent speaks, Silicon Valley doesn't just listen — it takes notes.
For the past week, the engineering community has been dissecting a thread on X from Boris Cherny, the creator and head of Claude Code at Anthropic. What began as a casual sharing of his personal terminal setup has spiraled into a viral manifesto on the future of software development, with industry insiders calling it a watershed moment for the startup.
"If you're not reading the Claude Code best practices straight from its creator, you're behind as a programmer," wrote Jeff Tang, a prominent voice in the developer community. Kyle McNease, another industry observer, went further, declaring that with Cherny's "game-changing updates," Anthropic is "on fire," potentially facing "their ChatGPT moment."
The excitement stems from a paradox: Cherny's workflow is surprisingly simple, yet it allows a single human to operate with the output capacity of a small engineering department. As one user noted on X after implementing Cherny's setup, the experience "feels more like Starcraft" than traditional coding — a shift from typing syntax to commanding autonomous units.
Here is an analysis of the workflow that is reshaping how software gets built, straight from the architect himself.
How running five AI agents at once turns coding into a real-time strategy game
The most striking revelation from Cherny's disclosure is that he does not code in a linear fashion. In the traditional "inner loop" of development, a programmer writes a function, tests it, and moves to the next. Cherny, however, acts as a fleet commander.
"I run 5 Claudes in parallel in my terminal," Cherny wrote. "I number my tabs 1-5, and use system notifications to know when a Claude needs input."
By utilizing iTerm2 system notifications, Cherny effectively manages five simultaneous work streams. While one agent runs a test suite, another refactors a legacy module, and a third drafts documentation. He also runs "5-10 Claudes on claude.ai" in his browser, using a "teleport" command to hand off sessions between the web and his local machine.
This validates the "do more with less" strategy articulated by Anthropic President Daniela Amodei earlier this week. While competitors like OpenAI pursue trillion-dollar infrastructure build-outs, Anthropic is proving that superior orchestration of existing models can yield exponential productivity gains.
The counterintuitive case for choosing the slowest, smartest model
In a surprising move for an industry obsessed with latency, Cherny revealed that he exclusively uses Anthropic's heaviest, slowest model: Opus 4.5.
"I use Opus 4.5 with thinking for everything," Cherny explained. "It's the best coding model I've ever used, and even though it's bigger & slower than Sonnet, since you have to steer it less and it's better at tool use, it is almost always faster than using a smaller model in the end."
For enterprise technology leaders, this is a critical insight. The bottleneck in modern AI development isn't the generation speed of the token; it is the human time spent correcting the AI's mistakes. Cherny's workflow suggests that paying the "compute tax" for a smarter model upfront eliminates the "correction tax" later.
One shared file turns every AI mistake into a permanent lesson
Cherny also detailed how his team solves the problem of AI amnesia. Standard large language models do not "remember" a company's specific coding style or architectural decisions from one session to the next.
To address this, Cherny's team maintains a single file named CLAUDE.md in their git repository. "Anytime we see Claude do something incorrectly we add it to the CLAUDE.md, so Claude knows not to do it next time," he wrote.
This practice transforms the codebase into a self-correcting organism. When a human developer reviews a pull request and spots an error, they don't just fix the code; they tag the AI to update its own instructions. "Every mistake becomes a rule," noted Aakash Gupta, a product leader analyzing the thread. The longer the team works together, the smarter the agent becomes.
Slash commands and subagents automate the most tedious parts of development
The "vanilla" workflow one observer praised is powered by rigorous automation of repetitive tasks. Cherny uses slash commands — custom shortcuts checked into the project's repository — to handle complex operations with a single keystroke.
He highlighted a command called /commit-push-pr, which he invokes dozens of times daily. Instead of manually typing git commands, writing a commit message, and opening a pull request, the agent handles the bureaucracy of version control autonomously.
Cherny also deploys subagents — specialized AI personas — to handle specific phases of the development lifecycle. He uses a code-simplifier to clean up architecture after the main work is done and a verify-app agent to run end-to-end tests before anything ships.
Why verification loops are the real unlock for AI-generated code
If there is a single reason Claude Code has reportedly hit $1 billion in annual recurring revenue so quickly, it is likely the verification loop. The AI is not just a text generator; it is a tester.
"Claude tests every single change I land to claude.ai/code using the Claude Chrome extension," Cherny wrote. "It opens a browser, tests the UI, and iterates until the code works and the UX feels good."
He argues that giving the AI a way to verify its own work — whether through browser automation, running bash commands, or executing test suites — improves the quality of the final result by "2-3x." The agent doesn't just write code; it proves the code works.
What Cherny's workflow signals about the future of software engineering
The reaction to Cherny's thread suggests a pivotal shift in how developers think about their craft. For years, "AI coding" meant an autocomplete function in a text editor — a faster way to type. Cherny has demonstrated that it can now function as an operating system for labor itself.
"Read this if you're already an engineer... and want more power," Jeff Tang summarized on X.
The tools to multiply human output by a factor of five are already here. They require only a willingness to stop thinking of AI as an assistant and start treating it as a workforce. The programmers who make that mental leap first won't just be more productive. They'll be playing an entirely different game — and everyone else will still be typing.
Composite 25Trend 10
Nous Research's NousCoder-14B is an open-source coding model landing right in the Claude Code moment
Nous Research, the open-source artificial intelligence startup backed by crypto venture firm Paradigm, released a new competitive programming model on Monday that it says matches or exceeds several larger proprietary systems — trained in just four days using 48 of Nvidia's latest B200 graphics processors.
The model, called NousCoder-14B, is another entry in a crowded field of AI coding assistants, but arrives at a particularly charged moment: Claude Code, the agentic programming tool from rival Anthropic, has dominated social media discussion since New Year's Day, with developers posting breathless testimonials about its capabilities. The simultaneous developments underscore how quickly AI-assisted software development is evolving — and how fiercely companies large and small are competing to capture what many believe will become a foundational technology for how software gets written.
type: embedded-entry-inline id: 74cSyrq6OUrp9SEQ5zOUSl
NousCoder-14B achieves a 67.87 percent accuracy rate on LiveCodeBench v6, a standardized evaluation that tests models on competitive programming problems published between August 2024 and May 2025. That figure represents a 7.08 percentage point improvement over the base model it was trained from, Alibaba's Qwen3-14B, according to Nous Research's technical report published alongside the release.
"I gave Claude Code a description of the problem, it generated what we built last year in an hour," wrote Jaana Dogan, a principal engineer at Google responsible for the Gemini API, in a viral post on X last week that captured the prevailing mood around AI coding tools. Dogan was describing a distributed agent orchestration system her team had spent a year developing — a system Claude Code approximated from a three-paragraph prompt.
The juxtaposition is instructive: while Anthropic's Claude Code has captured imaginations with demonstrations of end-to-end software development, Nous Research is betting that open-source alternatives trained on verifiable problems can close the gap — and that transparency in how these models are built matters as much as raw capability.
How Nous Research built an AI coding model that anyone can replicate
What distinguishes the NousCoder-14B release from many competitor announcements is its radical openness. Nous Research published not just the model weights but the complete reinforcement learning environment, benchmark suite, and training harness — built on the company's Atropos framework — enabling any researcher with sufficient compute to reproduce or extend the work.
"Open-sourcing the Atropos stack provides the necessary infrastructure for reproducible olympiad-level reasoning research," noted one observer on X, summarizing the significance for the academic and open-source communities.
The model was trained by Joe Li, a researcher in residence at Nous Research and a former competitive programmer himself. Li's technical report reveals an unexpectedly personal dimension: he compared the model's improvement trajectory to his own journey on Codeforces, the competitive programming platform where participants earn ratings based on contest performance.
Based on rough estimates mapping LiveCodeBench scores to Codeforces ratings, Li calculated that NousCoder-14B's improvemen t— from approximately the 1600-1750 rating range to 2100-2200 — mirrors a leap that took him nearly two years of sustained practice between ages 14 and 16. The model accomplished the equivalent in four days.
"Watching that final training run unfold was quite a surreal experience," Li wrote in the technical report.
But Li was quick to note an important caveat that speaks to broader questions about AI efficiency: he solved roughly 1,000 problems during those two years, while the model required 24,000. Humans, at least for now, remain dramatically more sample-efficient learners.
Inside the reinforcement learning system that trains on 24,000 competitive programming problems
NousCoder-14B's training process offers a window into the increasingly sophisticated techniques researchers use to improve AI reasoning capabilities through reinforcement learning.
The approach relies on what researchers call "verifiable rewards" — a system where the model generates code solutions, those solutions are executed against test cases, and the model receives a simple binary signal: correct or incorrect. This feedback loop, while conceptually straightforward, requires significant infrastructure to execute at scale.
Nous Research used Modal, a cloud computing platform, to run sandboxed code execution in parallel. Each of the 24,000 training problems contains hundreds of test cases on average, and the system must verify that generated code produces correct outputs within time and memory constraints — 15 seconds and 4 gigabytes, respectively.
The training employed a technique called DAPO (Dynamic Sampling Policy Optimization), which the researchers found performed slightly better than alternatives in their experiments. A key innovation involves "dynamic sampling" — discarding training examples where the model either solves all attempts or fails all attempts, since these provide no useful gradient signal for learning.
The researchers also adopted "iterative context extension," first training the model with a 32,000-token context window before expanding to 40,000 tokens. During evaluation, extending the context further to approximately 80,000 tokens produced the best results, with accuracy reaching 67.87 percent.
Perhaps most significantly, the training pipeline overlaps inference and verification — as soon as the model generates a solution, it begins work on the next problem while the previous solution is being checked. This pipelining, combined with asynchronous training where multiple model instances work in parallel, maximizes hardware utilization on expensive GPU clusters.
The looming data shortage that could slow AI coding model progress
Buried in Li's technical report is a finding with significant implications for the future of AI development: the training dataset for NousCoder-14B encompasses "a significant portion of all readily available, verifiable competitive programming problems in a standardized dataset format."
In other words, for this particular domain, the researchers are approaching the limits of high-quality training data.
"The total number of competitive programming problems on the Internet is roughly the same order of magnitude," Li wrote, referring to the 24,000 problems used for training. "This suggests that within the competitive programming domain, we have approached the limits of high-quality data."
This observation echoes growing concern across the AI industry about data constraints. While compute continues to scale according to well-understood economic and engineering principles, training data is "increasingly finite," as Li put it.
"It appears that some of the most important research that needs to be done in the future will be in the areas of synthetic data generation and data efficient algorithms and architectures," he concluded.
The challenge is particularly acute for competitive programming because the domain requires problems with known correct solutions that can be verified automatically. Unlike natural language tasks where human evaluation or proxy metrics suffice, code either works or it doesn't — making synthetic data generation considerably more difficult.
Li identified one potential avenue: training models not just to solve problems but to generate solvable problems, enabling a form of self-play similar to techniques that proved successful in game-playing AI systems. "Once synthetic problem generation is solved, self-play becomes a very interesting direction," he wrote.
A $65 million bet that open-source AI can compete with Big Tech
Nous Research has carved out a distinctive position in the AI landscape: a company committed to open-source releases that compete with — and sometimes exceed — proprietary alternatives.
The company raised $50 million in April 2025 in a round led by Paradigm, the cryptocurrency-focused venture firm founded by Coinbase co-founder Fred Ehrsam. Total funding reached $65 million, according to some reports. The investment reflected growing interest in decentralized approaches to AI training, an area where Nous Research has developed its Psyche platform.
Previous releases include Hermes 4, a family of models that we reported "outperform ChatGPT without content restrictions," and DeepHermes-3, which the company described as the first "toggle-on reasoning model" — allowing users to activate extended thinking capabilities on demand.
The company has cultivated a distinctive aesthetic and community, prompting some skepticism about whether style might overshadow substance. "Ofc i'm gonna believe an anime pfp company. stop benchmarkmaxxing ffs," wrote one critic on X, referring to Nous Research's anime-style branding and the industry practice of optimizing for benchmark performance.
Others raised technical questions. "Based on the benchmark, Nemotron is better," noted one commenter, referring to Nvidia's family of language models. Another asked whether NousCoder-14B is "agentic focused or just 'one shot' coding" — a distinction that matters for practical software development, where iterating on feedback typically produces better results than single attempts.
What researchers say must happen next for AI coding tools to keep improving
The release includes several directions for future work that hint at where AI coding research may be heading.
Multi-turn reinforcement learning tops the list. Currently, the model receives only a final binary reward — pass or fail — after generating a solution. But competitive programming problems typically include public test cases that provide intermediate feedback: compilation errors, incorrect outputs, time limit violations. Training models to incorporate this feedback across multiple attempts could significantly improve performance.
Controlling response length also remains a challenge. The researchers found that incorrect solutions tended to be longer than correct ones, and response lengths quickly saturated available context windows during training — a pattern that various algorithmic modifications failed to resolve.
Perhaps most ambitiously, Li proposed "problem generation and self-play" — training models to both solve and create programming problems. This would address the data scarcity problem directly by enabling models to generate their own training curricula.
"Humans are great at generating interesting and useful problems for other competitive programmers, but it appears that there still exists a significant gap in LLM capabilities in creative problem generation," Li wrote.
The model is available now on Hugging Face under an Apache 2.0 license. For researchers and developers who want to build on the work, Nous Research has published the complete Atropos training stack alongside it.
What took Li two years of adolescent dedication to achieve—climbing from a 1600-level novice to a 2100-rated competitor on Codeforces—an AI replicated in 96 hours. He needed 1,000 problems. The model needed 24,000. But soon enough, these systems may learn to write their own problems, teach themselves, and leave human benchmarks behind entirely.
The question is no longer whether machines can learn to code. It's whether they'll soon be better teachers than we ever were.
Composite 25Trend 10
Anthropic launches Cowork, a Claude Desktop agent that works in your files
Anthropic released Cowork on Monday, a new AI agent capability that extends the power of its wildly successful Claude Code tool to non-technical users — and according to company insiders, the team built the entire feature in approximately a week and a half, largely using Claude Code itself.
The launch marks a major inflection point in the race to deliver practical AI agents to mainstream users, positioning Anthropic to compete not just with OpenAI and Google in conversational AI, but with Microsoft's Copilot in the burgeoning market for AI-powered productivity tools.
"Cowork lets you complete non-technical tasks much like how developers use Claude Code," the company announced via its official Claude account on X. The feature arrives as a research preview available exclusively to Claude Max subscribers — Anthropic's power-user tier priced between $100 and $200 per month — through the macOS desktop application.
For the past year, the industry narrative has focused on large language models that can write poetry or debug code. With Cowork, Anthropic is betting that the real enterprise value lies in an AI that can open a folder, read a messy pile of receipts, and generate a structured expense report without human hand-holding.
How developers using a coding tool for vacation research inspired Anthropic's latest product
The genesis of Cowork lies in Anthropic's recent success with the developer community. In late 2024, the company released Claude Code, a terminal-based tool that allowed software engineers to automate rote programming tasks. The tool was a hit, but Anthropic noticed a peculiar trend: users were forcing the coding tool to perform non-coding labor.
According to Boris Cherny, an engineer at Anthropic, the company observed users deploying the developer tool for an unexpectedly diverse array of tasks.
"Since we launched Claude Code, we saw people using it for all sorts of non-coding work: doing vacation research, building slide decks, cleaning up your email, cancelling subscriptions, recovering wedding photos from a hard drive, monitoring plant growth, controlling your oven," Cherny wrote on X. "These use cases are diverse and surprising — the reason is that the underlying Claude Agent is the best agent, and Opus 4.5 is the best model."
Recognizing this shadow usage, Anthropic effectively stripped the command-line complexity from their developer tool to create a consumer-friendly interface. In its blog post announcing the feature, Anthropic explained that developers "quickly began using it for almost everything else," which "prompted us to build Cowork: a simpler way for anyone — not just developers — to work with Claude in the very same way."
Inside the folder-based architecture that lets Claude read, edit, and create files on your computer
Unlike a standard chat interface where a user pastes text for analysis, Cowork requires a different level of trust and access. Users designate a specific folder on their local machine that Claude can access. Within that sandbox, the AI agent can read existing files, modify them, or create entirely new ones.
Anthropic offers several illustrative examples: reorganizing a cluttered downloads folder by sorting and intelligently renaming each file, generating a spreadsheet of expenses from a collection of receipt screenshots, or drafting a report from scattered notes across multiple documents.
"In Cowork, you give Claude access to a folder on your computer. Claude can then read, edit, or create files in that folder," the company explained on X. "Try it to create a spreadsheet from a pile of screenshots, or produce a first draft from scattered notes."
The architecture relies on what is known as an "agentic loop." When a user assigns a task, the AI does not merely generate a text response. Instead, it formulates a plan, executes steps in parallel, checks its own work, and asks for clarification if it hits a roadblock. Users can queue multiple tasks and let Claude process them simultaneously — a workflow Anthropic describes as feeling "much less like a back-and-forth and much more like leaving messages for a coworker."
The system is built on Anthropic's Claude Agent SDK, meaning it shares the same underlying architecture as Claude Code. Anthropic notes that Cowork "can take on many of the same tasks that Claude Code can handle, but in a more approachable form for non-coding tasks."
The recursive loop where AI builds AI: Claude Code reportedly wrote much of Claude Cowork
Perhaps the most remarkable detail surrounding Cowork's launch is the speed at which the tool was reportedly built — highlighting a recursive feedback loop where AI tools are being used to build better AI tools.
During a livestream hosted by Dan Shipper, Felix Rieseberg, an Anthropic employee, confirmed that the team built Cowork in approximately a week and a half.
Alex Volkov, who covers AI developments, expressed surprise at the timeline: "Holy shit Anthropic built 'Cowork' in the last... week and a half?!"
This prompted immediate speculation about how much of Cowork was itself built by Claude Code. Simon Smith, EVP of Generative AI at Klick Health, put it bluntly on X: "Claude Code wrote all of Claude Cowork. Can we all agree that we're in at least somewhat of a recursive improvement loop here?"
The implication is profound: Anthropic's AI coding agent may have substantially contributed to building its own non-technical sibling product. If true, this is one of the most visible examples yet of AI systems being used to accelerate their own development and expansion — a strategy that could widen the gap between AI labs that successfully deploy their own agents internally and those that do not.
Connectors, browser automation, and skills extend Cowork's reach beyond the local file system
Cowork doesn't operate in isolation. The feature integrates with Anthropic's existing ecosystem of connectors — tools that link Claude to external information sources and services such as Asana, Notion, PayPal, and other supported partners. Users who have configured these connections in the standard Claude interface can leverage them within Cowork sessions.
Additionally, Cowork can pair with Claude in Chrome, Anthropic's browser extension, to execute tasks requiring web access. This combination allows the agent to navigate websites, click buttons, fill forms, and extract information from the internet — all while operating from the desktop application.
"Cowork includes a number of novel UX and safety features that we think make the product really special," Cherny explained, highlighting "a built-in VM [virtual machine] for isolation, out of the box support for browser automation, support for all your claude.ai data connectors, asking you for clarification when it's unsure."
Anthropic has also introduced an initial set of "skills" specifically designed for Cowork that enhance Claude's ability to create documents, presentations, and other files. These build on the Skills for Claude framework the company announced in October, which provides specialized instruction sets Claude can load for particular types of tasks.
Why Anthropic is warning users that its own AI agent could delete their files
The transition from a chatbot that suggests edits to an agent that makes edits introduces significant risk. An AI that can organize files can, theoretically, delete them.
In a notable display of transparency, Anthropic devoted considerable space in its announcement to warning users about Cowork's potential dangers — an unusual approach for a product launch.
The company explicitly acknowledges that Claude "can take potentially destructive actions (such as deleting local files) if it's instructed to." Because Claude might occasionally misinterpret instructions, Anthropic urges users to provide "very clear guidance" about sensitive operations.
More concerning is the risk of prompt injection attacks — a technique where malicious actors embed hidden instructions in content Claude might encounter online, potentially causing the agent to bypass safeguards or take harmful actions.
"We've built sophisticated defenses against prompt injections," Anthropic wrote, "but agent safety — that is, the task of securing Claude's real-world actions — is still an active area of development in the industry."
The company characterized these risks as inherent to the current state of AI agent technology rather than unique to Cowork. "These risks aren't new with Cowork, but it might be the first time you're using a more advanced tool that moves beyond a simple conversation," the announcement notes.
Anthropic's desktop agent strategy sets up a direct challenge to Microsoft Copilot
The launch of Cowork places Anthropic in direct competition with Microsoft, which has spent years attempting to integrate its Copilot AI into the fabric of the Windows operating system with mixed adoption results.
However, Anthropic's approach differs in its isolation. By confining the agent to specific folders and requiring explicit connectors, they are attempting to strike a balance between the utility of an OS-level agent and the security of a sandboxed application.
What distinguishes Anthropic's approach is its bottom-up evolution. Rather than designing an AI assistant and retrofitting agent capabilities, Anthropic built a powerful coding agent first — Claude Code — and is now abstracting its capabilities for broader audiences. This technical lineage may give Cowork more robust agentic behavior from the start.
Claude Code has generated significant enthusiasm among developers since its initial launch as a command-line tool in late 2024. The company expanded access with a web interface in October 2025, followed by a Slack integration in December. Cowork is the next logical step: bringing the same agentic architecture to users who may never touch a terminal.
Who can access Cowork now, and what's coming next for Windows and other platforms
For now, Cowork remains exclusive to Claude Max subscribers using the macOS desktop application. Users on other subscription tiers — Free, Pro, Team, or Enterprise — can join a waitlist for future access.
Anthropic has signaled clear intentions to expand the feature's reach. The blog post explicitly mentions plans to add cross-device sync and bring Cowork to Windows as the company learns from the research preview.
Cherny set expectations appropriately, describing the product as "early and raw, similar to what Claude Code felt like when it first launched."
To access Cowork, Max subscribers can download or update the Claude macOS app and click on "Cowork" in the sidebar.
The real question facing enterprise AI adoption
For technical decision-makers, the implications of Cowork extend beyond any single product launch. The bottleneck for AI adoption is shifting — no longer is model intelligence the limiting factor, but rather workflow integration and user trust.
Anthropic's goal, as the company puts it, is to make working with Claude feel less like operating a tool and more like delegating to a colleague. Whether mainstream users are ready to hand over folder access to an AI that might misinterpret their instructions remains an open question.
But the speed of Cowork's development — a major feature built in ten days, possibly by the company's own AI — previews a future where the capabilities of these systems compound faster than organizations can evaluate them.
The chatbot has learned to use a file manager. What it learns to use next is anyone's guess.
Composite 25Trend 10
Salesforce rolls out new Slackbot AI agent as it battles Microsoft and Google in workplace AI
Salesforce on Tuesday launched an entirely rebuilt version of Slackbot, the company's workplace assistant, transforming it from a simple notification tool into what executives describe as a fully powered AI agent capable of searching enterprise data, drafting documents, and taking action on behalf of employees.
The new Slackbot, now generally available to Business+ and Enterprise+ customers, is Salesforce's most aggressive move yet to position Slack at the center of the emerging "agentic AI" movement — where software agents work alongside humans to complete complex tasks. The launch comes as Salesforce attempts to convince investors that artificial intelligence will bolster its products rather than render them obsolete.
"Slackbot isn't just another copilot or AI assistant," said Parker Harris, Salesforce co-founder and Slack's chief technology officer, in an exclusive interview with Salesforce. "It's the front door to the agentic enterprise, powered by Salesforce."
From tricycle to Porsche: Salesforce rebuilt Slackbot from the ground up
Harris was blunt about what distinguishes the new Slackbot from its predecessor: "The old Slackbot was, you know, a little tricycle, and the new Slackbot is like, you know, a Porsche."
The original Slackbot, which has existed since Slack's early days, performed basic algorithmic tasks — reminding users to add colleagues to documents, suggesting channel archives, and delivering simple notifications. The new version runs on an entirely different architecture built around a large language model and sophisticated search capabilities that can access Salesforce records, Google Drive files, calendar data, and years of Slack conversations.
"It's two different things," Harris explained. "The old Slackbot was algorithmic and fairly simple. The new Slackbot is brand new — it's based around an LLM and a very robust search engine, and connections to third-party search engines, third-party enterprise data."
Salesforce chose to retain the Slackbot brand despite the fundamental technical overhaul. "People know what Slackbot is, and so we wanted to carry that forward," Harris said.
Why Anthropic's Claude powers the new Slackbot — and which AI models could come next
The new Slackbot runs on Claude, Anthropic's large language model, a choice driven partly by compliance requirements. Slack's commercial service operates under FedRAMP Moderate certification to serve U.S. federal government customers, and Harris said Anthropic was "the only provider that could give us a compliant LLM" when Slack began building the new system.
But that exclusivity won't last. "We are, this year, going to support additional providers," Harris said. "We have a great relationship with Google. Gemini is incredible — performance is great, cost is great. So we're going to use Gemini for some things." He added that OpenAI remains a possibility as well.
Harris echoed Salesforce CEO Marc Benioff's view that large language models are becoming commoditized: "You've heard Marc talk about LLMs are commodities, that they're democratized. I call them CPUs."
On the sensitive question of training data, Harris was unequivocal: Salesforce does not train any models on customer data. "Models don't have any sort of security," he explained. "If we trained it on some confidential conversation that you and I have, I don't want Carolyn to know — if I train it into the LLM, there is no way for me to say you get to see the answer, but Carolyn doesn't."
Inside Salesforce's internal experiment: 80,000 employees tested Slackbot with striking results
Salesforce has been testing the new Slackbot internally for months, rolling it out to all 80,000 employees. According to Ryan Gavin, Slack's chief marketing officer, the results have been striking: "It's the fastest adopted product in Salesforce history."
Internal data shows that two-thirds of Salesforce employees have tried the new Slackbot, with 80% of those users continuing to use it regularly. Internal satisfaction rates reached 96% — the highest for any AI feature Slack has shipped. Employees report saving between two and 20 hours per week.
The adoption happened largely organically. "I think it was about five days, and a Canvas was developed by our employees called 'The Most Stealable Slackbot Prompts,'" Gavin said. "People just started adding to it organically. I think it's up to 250-plus prompts that are in this Canvas right now."
Kate Crotty, a principal UX researcher at Salesforce, found that 73% of internal adoption was driven by social sharing rather than top-down mandates. "Everybody is there to help each other learn and communicate hacks," she said.
How Slackbot transforms scattered enterprise data into executive-ready insights
During a product demonstration, Amy Bauer, Slack's product experience designer, showed how Slackbot can synthesize information across multiple sources. In one example, she asked Slackbot to analyze customer feedback from a pilot program, upload an image of a usage dashboard, and have Slackbot correlate the qualitative and quantitative data.
"This is where Slackbot really earns its keep for me," Bauer explained. "What it's doing is not just simply reading the image — it's actually looking at the image and comparing it to the insight it just generated for me."
Slackbot can then query Salesforce to find enterprise accounts with open deals that might be good candidates for early access, creating what Bauer called "a really great justification and plan to move forward." Finally, it can synthesize all that information into a Canvas — Slack's collaborative document format — and find calendar availability among stakeholders to schedule a review meeting.
"Up until this point, we have been working in a one-to-one capacity with Slackbot," Bauer said. "But one of the benefits that I can do now is take this insight and have it generate this into a Canvas, a shared workspace where I can iterate on it, refine it with Slackbot, or share it out with my team."
Rob Seaman, Slack's chief product officer, said the Canvas creation demonstrates where the product is heading: "This is making a tool call internally to Slack Canvas to actually write, effectively, a shared document. But it signals where we're going with Slackbot — we're eventually going to be adding in additional third-party tool calls."
MrBeast's company became a Slackbot guinea pig—and employees say they're saving 90 minutes a day
Among Salesforce's pilot customers is Beast Industries, the parent company of YouTube star MrBeast. Luis Madrigal, the company's chief information officer, joined the launch announcement to describe his experience.
"As somebody who has rolled out enterprise technologies for over two decades now, this was practically one of the easiest," Madrigal said. "The plumbing is there. Slack as an implementation, Enterprise Tools — being able to turn on the Slackbot and the Slack AI functionality was as simple as having my team go in, review, do a quick security review."
Madrigal said his security team signed off "rather quickly" — unusual for enterprise AI deployments — because Slackbot accesses only the information each individual user already has permission to view. "Given all the guardrails you guys have put into place for Slackbot to be unique and customized to only the information that each individual user has, only the conversations and the Slack rooms and Slack channels that they're part of—that made my security team sign off rather quickly."
One Beast Industries employee, Sinan, the head of Beast Games marketing, reported saving "at bare minimum, 90 minutes a day." Another employee, Spencer, a creative supervisor, described it as "an assistant who's paying attention when I'm not."
Other pilot customers include Slalom, reMarkable, Xero, Mercari, and Engine. Mollie Bodensteiner, SVP of Operations at Engine, called Slackbot "an absolute 'chaos tamer' for our team," estimating it saves her about 30 minutes daily "just by eliminating context switching."
Slackbot vs. Microsoft Copilot vs. Google Gemini: The fight for enterprise AI dominance
The launch puts Salesforce in direct competition with Microsoft's Copilot, which is integrated into Teams and the broader Microsoft 365 suite, as well as Google's Gemini integrations across Workspace. When asked what distinguishes Slackbot from these alternatives, Seaman pointed to context and convenience.
"The thing that makes it most powerful for our customers and users is the proximity — it's just right there in your Slack," Seaman said. "There's a tremendous convenience affordance that's naturally built into it."
The deeper advantage, executives argue, is that Slackbot already understands users' work without requiring setup or training. "Most AI tools sound the same no matter who is using them," the company's announcement stated. "They lack context, miss nuance, and force you to jump between tools to get anything done."
Harris put it more directly: "If you've ever had that magic experience with AI — I think ChatGPT is a great example, it's a great experience from a consumer perspective — Slackbot is really what we're doing in the enterprise, to be this employee super agent that is loved, just like people love using Slack."
Amy Bauer emphasized the frictionless nature of the experience. "Slackbot is inherently grounded in the context, in the data that you have in Slack," she said. "So as you continue working in Slack, Slackbot gets better because it's grounded in the work that you're doing there. There is no setup. There is no configuration for those end users."
Salesforce's ambitious plan to make Slackbot the one 'super agent' that controls all the others
Salesforce positions Slackbot as what Harris calls a "super agent" — a central hub that can eventually coordinate with other AI agents across an organization.
"Every corporation is going to have an employee super agent," Harris said. "Slackbot is essentially taking the magic of what Slack does. We think that Slackbot, and we're really excited about it, is going to be that."
The vision extends to third-party agents already launching in Slack. Last month, Anthropic released a preview of Claude Code for Slack, allowing developers to interact with Claude's coding capabilities directly in chat threads. OpenAI, Google, Vercel, and others have also built agents for the platform.
"Most of the net-new apps that are being deployed to Slack are agents," Seaman noted during the press conference. "This is proof of the promise of humans and agents coexisting and working together in Slack to solve problems."
Harris described a future where Slackbot becomes an MCP (Model Context Protocol) client, able to leverage tools from across the software ecosystem — similar to how the developer tool Cursor works. "Slack can be an MCP client, and Slackbot will be the hub of that, leveraging all these tools out in the world, some of which will be these amazing agents," he said.
But Harris also cautioned against over-promising on multi-agent coordination. "I still think we're in the single agent world," he said. "FY26 is going to be the year where we started to see more coordination. But we're going to do it with customer success in mind, and not demonstrate and talk about, like, 'I've got 1,000 agents working together,' because I think that's unrealistic."
Slackbot costs nothing extra, but Salesforce's data access fees could squeeze some customers
Slackbot is included at no additional cost for customers on Business+ and Enterprise+ plans. "There's no additional fees customers have to do," Gavin confirmed. "If they're on one of those plans, they're going to get Slackbot."
However, some enterprise customers may face other cost pressures related to Salesforce's broader data strategy. CIOs may see price increases for third-party applications that work with Salesforce data, as effects of higher charges for API access ripple through the software supply chain.
Fivetran CEO George Fraser has warned that Salesforce's shift in pricing policy for API access could have tangible consequences for enterprises relying on Salesforce as a system of record. "They might not be able to use Fivetran to replicate their data to Snowflake and instead have to use Salesforce Data Cloud. Or they might find that they are not able to interact with their data via ChatGPT, and instead have to use Agentforce," Fraser said in a recent CIO report.
Salesforce has framed the pricing change as standard industry practice.
What Slackbot can do today, what's coming in weeks, and what's still on the roadmap
The new Slackbot begins rolling out today and will reach all eligible customers by the end of February. Mobile availability will complete by March 3, Bauer confirmed during her interview with VentureBeat.
Some capabilities remain works in progress. Calendar reading and availability checking are available at launch, but the ability to actually book meetings is "coming a few weeks after," according to Seaman. Image generation is not currently supported, though Bauer said it's "something that we are looking at in the future."
When asked about integration with competing CRM systems like HubSpot and Microsoft Dynamics, Salesforce representatives declined to provide specifics during the interview, though they acknowledged the question touched on key competitive differentiators.
Salesforce is betting the future of work looks like a chat window—and it's not alone
The Slackbot launch is Salesforce's bet that the future of enterprise work is conversational — that employees will increasingly prefer to interact with AI through natural language rather than navigating traditional software interfaces.
Harris described Slack's product philosophy using principles like "don't make me think" and "be a great host." The goal, he said, is for Slackbot to surface information proactively rather than requiring users to hunt for it.
"One of the revelations for me is LLMs applied to unstructured information are incredible," Harris said. "And the amount of value you have if you're a Slack user, if your corporation uses Slack — the amount of value in Slack is unbelievable. Because you're talking about work, you're sharing documents, you're making decisions, but you can't as a human go through that and really get the same value that an LLM can do."
Looking ahead, Harris expects the interfaces themselves to evolve beyond pure conversation. "We're kind of saturating what we can do with purely conversational UIs," he said. "I think we'll start to see agents building an interface that best suits your intent, as opposed to trying to surface something within a conversational interface that matches your intent."
Microsoft, Google, and a growing roster of AI startups are placing similar bets — that the winning enterprise AI will be the one embedded in the tools workers already use, not another application to learn. The race to become that invisible layer of workplace intelligence is now fully underway.
For Salesforce, the stakes extend beyond a single product launch. After a bruising year on Wall Street and persistent questions about whether AI threatens its core business, the company is wagering that Slackbot can prove the opposite — that the tens of millions of people already chatting in Slack every day is not a vulnerability, but an unassailable advantage.
Haley Gault, the Salesforce account executive in Pittsburgh who stumbled upon the new Slackbot on a snowy morning, captured the shift in a single sentence: "I honestly can't imagine working for another company not having access to these types of tools. This is just how I work now."
That's precisely what Salesforce is counting on.
Composite 25Trend 10
Listen Labs raises $69M after viral billboard hiring stunt to scale AI customer interviews
Alfred Wahlforss was running out of options. His startup, Listen Labs, needed to hire over 100 engineers, but competing against Mark Zuckerberg's $100 million offers seemed impossible. So he spent $5,000 — a fifth of his marketing budget — on a billboard in San Francisco displaying what looked like gibberish: five strings of random numbers.
The numbers were actually AI tokens. Decoded, they led to a coding challenge: build an algorithm to act as a digital bouncer at Berghain, the Berlin nightclub famous for rejecting nearly everyone at the door. Within days, thousands attempted the puzzle. 430 cracked it. Some got hired. The winner flew to Berlin, all expenses paid.
That unconventional approach has now attracted $69 million in Series B funding, led by Ribbit Capital with participation from Evantic and existing investors Sequoia Capital, Conviction, and Pear VC. The round values Listen Labs at $500 million and brings its total capital to $100 million. In nine months since launch, the company has grown annualized revenue by 15x to eight figures and conducted over one million AI-powered interviews.
"When you obsess over customers, everything else follows," Wahlforss said in an interview with VentureBeat. "Teams that use Listen bring the customer into every decision, from marketing to product, and when the customer is delighted, everyone is."
Why traditional market research is broken, and what Listen Labs is building to fix it
Listen's AI researcher finds participants, conducts in-depth interviews, and delivers actionable insights in hours, not weeks. The platform replaces the traditional choice between quantitative surveys — which provide statistical precision but miss nuance—and qualitative interviews, which deliver depth but cannot scale.
Wahlforss explained the limitation of existing approaches: "Essentially surveys give you false precision because people end up answering the same question... You can't get the outliers. People are actually not honest on surveys." The alternative, one-on-one human interviews, "gives you a lot of depth. You can ask follow up questions. You can kind of double check if they actually know what they're talking about. And the problem is you can't scale that."
The platform works in four steps: users create a study with AI assistance, Listen recruits participants from its global network of 30 million people, an AI moderator conducts in-depth interviews with follow-up questions, and results are packaged into executive-ready reports including key themes, highlight reels, and slide decks.
What distinguishes Listen's approach is its use of open-ended video conversations rather than multiple-choice forms. "In a survey, you can kind of guess what you should answer, and you have four options," Wahlforss said. "Oh, they probably want me to buy high income. Let me click on that button versus an open ended response. It just generates much more honesty."
The dirty secret of the $140 billion market research industry: rampant fraud
Listen finds and qualifies the right participants in its global network of 30 million people. But building that panel required confronting what Wahlforss called "one of the most shocking things that we've learned when we entered this industry"—rampant fraud.
"Essentially, there's a financial transaction involved, which means there will be bad players," he explained. "We actually had some of the largest companies, some of them have billions in revenue, send us people who claim to be kind of enterprise buyers to our platform and our system immediately detected, like, fraud, fraud, fraud, fraud, fraud."
The company built what it calls a "quality guard" that cross-references LinkedIn profiles with video responses to verify identity, checks consistency across how participants answer questions, and flags suspicious patterns. The result, according to Wahlforss: "People talk three times more. They're much more honest when they talk about sensitive topics like politics and mental health."
Emeritus, an online education company that uses Listen, reported that approximately 20% of survey responses previously fell into the fraudulent or low-quality category. With Listen, they reduced this to almost zero. "We did not have to replace any responses because of fraud or gibberish information," said Gabrielli Tiburi, Assistant Manager of Customer Insights at Emeritus.
How Microsoft, Sweetgreen, and Chubbies are using AI interviews to build better products
The speed advantage has proven central to Listen's pitch. Traditional customer research at Microsoft could take four to six weeks to generate insights. "By the time we get to them, either the decision has been made or we lose out on the opportunity to actually influence it," said Romani Patel, Senior Research Manager at Microsoft.
With Listen, Microsoft can now get insights in days, and in many cases, within hours.
The platform has already powered several high-profile initiatives. Microsoft used Listen Labs to collect global customer stories for its 50th anniversary celebration. "We wanted users to share how Copilot is empowering them to bring their best self forward," Patel said, "and we were able to collect those user video stories within a day." Traditionally, that kind of work would have taken six to eight weeks.
Simple Modern, an Oklahoma-based drinkware company, used Listen to test a new product concept. The process took about an hour to write questions, an hour to launch the study, and 2.5 hours to receive feedback from 120 people across the country. "We went from 'Should we even have this product?' to 'How should we launch it?'" said Chris Hoyle, the company's Chief Marketing Officer.
Chubbies, the shorts brand, achieved a 24x increase in youth research participation—growing from 5 to 120 participants — by using Listen to overcome the scheduling challenges of traditional focus groups with children. "There's school, sports, dinner, and homework," explained Lauren Neville, Director of Insights and Innovation. "I had to find a way to hear from them that fit into their schedules."
The company also discovered product issues through AI interviews that might have gone undetected otherwise. Wahlforss described how the AI "through conversations, realized there were like issues with the the kids short line, and decided to, like, interview hundreds of kids. And I understand that there were issues in the liner of the shorts and that they were, like, scratchy, quote, unquote, according to the people interviewed." The redesigned product became "a blockbuster hit."
The Jevons paradox explains why cheaper research creates more demand, not less
Listen Labs is entering a massive but fragmented market. Wahlforss cited research from Andreessen Horowitz estimating the market research industry at roughly $140 billion annually, populated by legacy players — some with more than a billion dollars in revenue — that he believes are vulnerable to disruption.
"There are very much existing budget lines that we are replacing," Wahlforss said. "Why we're replacing them is that one, they're super costly. Two, they're kind of stuck in this old paradigm of choosing between a survey or interview, and they also take months to work with."
But the more intriguing dynamic may be that AI-powered research doesn't just replace existing spending — it creates new demand. Wahlforss invoked the Jevons paradox, an economic principle that occurs when technological advancements make a resource more efficient to use, but increased efficiency leads to increased overall consumption rather than decreased consumption.
"What I've noticed is that as something gets cheaper, you don't need less of it. You want more of it," Wahlforss explained. "There's infinite demand for customer understanding. So the researchers on the team can do an order of magnitude more research, and also other people who weren't researchers before can now do that as part of their job."
Inside the elite engineering team that built Listen Labs before they had a working toilet
Listen Labs traces its origins to a consumer app that Wahlforss and his co-founder built after meeting at Harvard. "We built this consumer app that got 20,000 downloads in one day," Wahlforss recalled. "We had all these users, and we were thinking like, okay, what can we do to get to know them better? And we built this prototype of what Listen is today."
The founding team brings an unusual pedigree. Wahlforss's co-founder "was the national champion in competitive programming in Germany, and he worked at Tesla Autopilot." The company claims that 30% of its engineering team are medalists from the International Olympiad in Informatics — the same competition that produced the founders of Cognition, the AI coding startup.
The Berghain billboard stunt generated approximately 5 million views across social media, according to Wahlforss. It reflected the intensity of the talent war in the Bay Area.
"We had to do these things because some of our, like early employees, joined the company before we had a working toilet," he said. "But now we fixed that situation."
The company grew from 5 to 40 employees in 2024 and plans to reach 150 this year. It hires engineers for non-engineering roles across marketing, growth, and operations — a bet that in the AI era, technical fluency matters everywhere.
Synthetic customers and automated decisions: what Listen Labs is building next
Wahlforss outlined an ambitious product roadmap that pushes into more speculative territory. The company is building "the ability to simulate your customers, so you can take all of those interviews we've done, and then extrapolate based on that and create synthetic users or simulated user voices."
Beyond simulation, Listen aims to enable automated action based on research findings. "Can you not just make recommendations, but also create spawn agents to either change things in code or some customer churns? Can you give them a discount and try to bring them back?"
Wahlforss acknowledged the ethical implications. "Obviously, as you said, there's kind of ethical concerns there. Of like, automated decision making overall can be bad, but we will have considerable guardrails to make sure that the companies are always in the loop."
The company already handles sensitive data with care. "We don't train on any of the data," Wahlforss said. "We will also scrub any sensitive PII automatically so the model can detect that. And there are times when, for example, you work with investors, where if you accidentally mention something that could be material, non public information, the AI can actually detect that and remove any information like that."
How AI could reshape the future of product development
Perhaps the most provocative implication of Listen's model is how it could reshape product development itself. Wahlforss described a customer — an Australian startup — that has adopted what amounts to a continuous feedback loop.
"They're based in Australia, so they're coding during the day, and then in their night, they're releasing a Listen study with an American audience. Listen validates whatever they built during the day, and they get feedback on that. They can then plug that feedback directly into coding tools like Claude Code and iterate."
The vision extends Y Combinator's famous dictum — "write code, talk to users" — into an automated cycle. "Write code is now getting automated. And I think like talk to users will be as well, and you'll have this kind of infinite loop where you can start to ship this truly amazing product, almost kind of autonomously."
Whether that vision materializes depends on factors beyond Listen's control — the continued improvement of AI models, enterprise willingness to trust automated research, and whether speed truly correlates with better products. A 2024 MIT study found that 95% of AI pilots fail to move into production, a statistic Wahlforss cited as the reason he emphasizes quality over demos.
"I'm constantly have to emphasize like, let's make sure the quality is there and the details are right," he said.
But the company's growth suggests appetite for the experiment. Microsoft's Patel said Listen has "removed the drudgery of research and brought the fun and joy back into my work." Chubbies is now pushing its founder to give everyone in the company a login. Sling Money, a stablecoin payments startup, can create a survey in ten minutes and receive results the same day.
"It's a total game changer," said Ali Romero, Sling Money's marketing manager.
Wahlforss has a different phrase for what he's building. When asked about the tension between speed and rigor — the long-held belief that moving fast means cutting corners — he cited Nat Friedman, the former GitHub CEO and Listen investor, who keeps a list of one-liners on his website.
One of them: "Slow is fake."
It's an aggressive claim for an industry built on methodological caution. But Listen Labs is betting that in the AI era, the companies that listen fastest will be the ones that win. The only question is whether customers will talk back.
Composite 25Trend 10
Claude Code costs up to $200 a month. Goose does the same thing for free.
The artificial intelligence coding revolution comes with a catch: it's expensive.
Claude Code, Anthropic's terminal-based AI agent that can write, debug, and deploy code autonomously, has captured the imagination of software developers worldwide. But its pricing — ranging from $20 to $200 per month depending on usage — has sparked a growing rebellion among the very programmers it aims to serve.
Now, a free alternative is gaining traction. Goose, an open-source AI agent developed by Block (the financial technology company formerly known as Square), offers nearly identical functionality to Claude Code but runs entirely on a user's local machine. No subscription fees. No cloud dependency. No rate limits that reset every five hours.
"Your data stays with you, period," said Parth Sareen, a software engineer who demonstrated the tool during a recent livestream. The comment captures the core appeal: Goose gives developers complete control over their AI-powered workflow, including the ability to work offline — even on an airplane.
The project has exploded in popularity. Goose now boasts more than 26,100 stars on GitHub, the code-sharing platform, with 362 contributors and 102 releases since its launch. The latest version, 1.20.1, shipped on January 19, 2026, reflecting a development pace that rivals commercial products.
For developers frustrated by Claude Code's pricing structure and usage caps, Goose represents something increasingly rare in the AI industry: a genuinely free, no-strings-attached option for serious work.
Anthropic's new rate limits spark a developer revolt
To understand why Goose matters, you need to understand the Claude Code pricing controversy.
Anthropic, the San Francisco artificial intelligence company founded by former OpenAI executives, offers Claude Code as part of its subscription tiers. The free plan provides no access whatsoever. The Pro plan, at $17 per month with annual billing (or $20 monthly), limits users to just 10 to 40 prompts every five hours — a constraint that serious developers exhaust within minutes of intensive work.
The Max plans, at $100 and $200 per month, offer more headroom: 50 to 200 prompts and 200 to 800 prompts respectively, plus access to Anthropic's most powerful model, Claude 4.5 Opus. But even these premium tiers come with restrictions that have inflamed the developer community.
In late July, Anthropic announced new weekly rate limits. Under the system, Pro users receive 40 to 80 hours of Sonnet 4 usage per week. Max users at the $200 tier get 240 to 480 hours of Sonnet 4, plus 24 to 40 hours of Opus 4. Nearly five months later, the frustration has not subsided.
The problem? Those "hours" are not actual hours. They represent token-based limits that vary wildly depending on codebase size, conversation length, and the complexity of the code being processed. Independent analysis suggests the actual per-session limits translate to roughly 44,000 tokens for Pro users and 220,000 tokens for the $200 Max plan.
"It's confusing and vague," one developer wrote in a widely shared analysis. "When they say '24-40 hours of Opus 4,' that doesn't really tell you anything useful about what you're actually getting."
The backlash on Reddit and developer forums has been fierce. Some users report hitting their daily limits within 30 minutes of intensive coding. Others have canceled their subscriptions entirely, calling the new restrictions "a joke" and "unusable for real work."
Anthropic has defended the changes, stating that the limits affect fewer than five percent of users and target people running Claude Code "continuously in the background, 24/7." But the company has not clarified whether that figure refers to five percent of Max subscribers or five percent of all users — a distinction that matters enormously.
How Block built a free AI coding agent that works offline
Goose takes a radically different approach to the same problem.
Built by Block, the payments company led by Jack Dorsey, Goose is what engineers call an "on-machine AI agent." Unlike Claude Code, which sends your queries to Anthropic's servers for processing, Goose can run entirely on your local computer using open-source language models that you download and control yourself.
The project's documentation describes it as going "beyond code suggestions" to "install, execute, edit, and test with any LLM." That last phrase — "any LLM" — is the key differentiator. Goose is model-agnostic by design.
You can connect Goose to Anthropic's Claude models if you have API access. You can use OpenAI's GPT-5 or Google's Gemini. You can route it through services like Groq or OpenRouter. Or — and this is where things get interesting — you can run it entirely locally using tools like Ollama, which let you download and execute open-source models on your own hardware.
The practical implications are significant. With a local setup, there are no subscription fees, no usage caps, no rate limits, and no concerns about your code being sent to external servers. Your conversations with the AI never leave your machine.
"I use Ollama all the time on planes — it's a lot of fun!" Sareen noted during a demonstration, highlighting how local models free developers from the constraints of internet connectivity.
What Goose can do that traditional code assistants can't
Goose operates as a command-line tool or desktop application that can autonomously perform complex development tasks. It can build entire projects from scratch, write and execute code, debug failures, orchestrate workflows across multiple files, and interact with external APIs — all without constant human oversight.
The architecture relies on what the AI industry calls "tool calling" or "function calling" — the ability for a language model to request specific actions from external systems. When you ask Goose to create a new file, run a test suite, or check the status of a GitHub pull request, it doesn't just generate text describing what should happen. It actually executes those operations.
This capability depends heavily on the underlying language model. Claude 4 models from Anthropic currently perform best at tool calling, according to the Berkeley Function-Calling Leaderboard, which ranks models on their ability to translate natural language requests into executable code and system commands.
But newer open-source models are catching up quickly. Goose's documentation highlights several options with strong tool-calling support: Meta's Llama series, Alibaba's Qwen models, Google's Gemma variants, and DeepSeek's reasoning-focused architectures.
The tool also integrates with the Model Context Protocol, or MCP, an emerging standard for connecting AI agents to external services. Through MCP, Goose can access databases, search engines, file systems, and third-party APIs — extending its capabilities far beyond what the base language model provides.
Setting Up Goose with a Local Model
For developers interested in a completely free, privacy-preserving setup, the process involves three main components: Goose itself, Ollama (a tool for running open-source models locally), and a compatible language model.
Step 1: Install Ollama
Ollama is an open-source project that dramatically simplifies the process of running large language models on personal hardware. It handles the complex work of downloading, optimizing, and serving models through a simple interface.
Download and install Ollama from ollama.com. Once installed, you can pull models with a single command. For coding tasks, Qwen 2.5 offers strong tool-calling support:
ollama run qwen2.5
The model downloads automatically and begins running on your machine.
Step 2: Install Goose
Goose is available as both a desktop application and a command-line interface. The desktop version provides a more visual experience, while the CLI appeals to developers who prefer working entirely in the terminal.
Installation instructions vary by operating system but generally involve downloading from Goose's GitHub releases page or using a package manager. Block provides pre-built binaries for macOS (both Intel and Apple Silicon), Windows, and Linux.
Step 3: Configure the Connection
In Goose Desktop, navigate to Settings, then Configure Provider, and select Ollama. Confirm that the API Host is set to http://localhost:11434 (Ollama's default port) and click Submit.
For the command-line version, run goose configure, select "Configure Providers," choose Ollama, and enter the model name when prompted.
That's it. Goose is now connected to a language model running entirely on your hardware, ready to execute complex coding tasks without any subscription fees or external dependencies.
The RAM, processing power, and trade-offs you should know about
The obvious question: what kind of computer do you need?
Running large language models locally requires substantially more computational resources than typical software. The key constraint is memory — specifically, RAM on most systems, or VRAM if using a dedicated graphics card for acceleration.
Block's documentation suggests that 32 gigabytes of RAM provides "a solid baseline for larger models and outputs." For Mac users, this means the computer's unified memory is the primary bottleneck. For Windows and Linux users with discrete NVIDIA graphics cards, GPU memory (VRAM) matters more for acceleration.
But you don't necessarily need expensive hardware to get started. Smaller models with fewer parameters run on much more modest systems. Qwen 2.5, for instance, comes in multiple sizes, and the smaller variants can operate effectively on machines with 16 gigabytes of RAM.
"You don't need to run the largest models to get excellent results," Sareen emphasized. The practical recommendation: start with a smaller model to test your workflow, then scale up as needed.
For context, Apple's entry-level MacBook Air with 8 gigabytes of RAM would struggle with most capable coding models. But a MacBook Pro with 32 gigabytes — increasingly common among professional developers — handles them comfortably.
Why keeping your code off the cloud matters more than ever
Goose with a local LLM is not a perfect substitute for Claude Code. The comparison involves real trade-offs that developers should understand.
Model Quality: Claude 4.5 Opus, Anthropic's flagship model, remains arguably the most capable AI for software engineering tasks. It excels at understanding complex codebases, following nuanced instructions, and producing high-quality code on the first attempt. Open-source models have improved dramatically, but a gap persists — particularly for the most challenging tasks.
One developer who switched to the $200 Claude Code plan described the difference bluntly: "When I say 'make this look modern,' Opus knows what I mean. Other models give me Bootstrap circa 2015."
Context Window: Claude Sonnet 4.5, accessible through the API, offers a massive one-million-token context window — enough to load entire large codebases without chunking or context management issues. Most local models are limited to 4,096 or 8,192 tokens by default, though many can be configured for longer contexts at the cost of increased memory usage and slower processing.
Speed: Cloud-based services like Claude Code run on dedicated server hardware optimized for AI inference. Local models, running on consumer laptops, typically process requests more slowly. The difference matters for iterative workflows where you're making rapid changes and waiting for AI feedback.
Tooling Maturity: Claude Code benefits from Anthropic's dedicated engineering resources. Features like prompt caching (which can reduce costs by up to 90 percent for repeated contexts) and structured outputs are polished and well-documented. Goose, while actively developed with 102 releases to date, relies on community contributions and may lack equivalent refinement in specific areas.
How Goose stacks up against Cursor, GitHub Copilot, and the paid AI coding market
Goose enters a crowded market of AI coding tools, but occupies a distinctive position.
Cursor, a popular AI-enhanced code editor, charges $20 per month for its Pro tier and $200 for Ultra—pricing that mirrors Claude Code's Max plans. Cursor provides approximately 4,500 Sonnet 4 requests per month at the Ultra level, a substantially different allocation model than Claude Code's hourly resets.
Cline, Roo Code, and similar open-source projects offer AI coding assistance but with varying levels of autonomy and tool integration. Many focus on code completion rather than the agentic task execution that defines Goose and Claude Code.
Amazon's CodeWhisperer, GitHub Copilot, and enterprise offerings from major cloud providers target large organizations with complex procurement processes and dedicated budgets. They are less relevant to individual developers and small teams seeking lightweight, flexible tools.
Goose's combination of genuine autonomy, model agnosticism, local operation, and zero cost creates a unique value proposition. The tool is not trying to compete with commercial offerings on polish or model quality. It's competing on freedom — both financial and architectural.
The $200-a-month era for AI coding tools may be ending
The AI coding tools market is evolving quickly. Open-source models are improving at a pace that continually narrows the gap with proprietary alternatives. Moonshot AI's Kimi K2 and z.ai's GLM 4.5 now benchmark near Claude Sonnet 4 levels — and they're freely available.
If this trajectory continues, the quality advantage that justifies Claude Code's premium pricing may erode. Anthropic would then face pressure to compete on features, user experience, and integration rather than raw model capability.
For now, developers face a clear choice. Those who need the absolute best model quality, who can afford premium pricing, and who accept usage restrictions may prefer Claude Code. Those who prioritize cost, privacy, offline access, and flexibility have a genuine alternative in Goose.
The fact that a $200-per-month commercial product has a zero-dollar open-source competitor with comparable core functionality is itself remarkable. It reflects both the maturation of open-source AI infrastructure and the appetite among developers for tools that respect their autonomy.
Goose is not perfect. It requires more technical setup than commercial alternatives. It depends on hardware resources that not every developer possesses. Its model options, while improving rapidly, still trail the best proprietary offerings on complex tasks.
But for a growing community of developers, those limitations are acceptable trade-offs for something increasingly rare in the AI landscape: a tool that truly belongs to them.
Goose is available for download at github.com/block/goose. Ollama is available at ollama.com. Both projects are free and open source.
Composite 25Trend 10
Railway secures $100 million to challenge AWS with AI-native cloud infrastructure
Railway, a San Francisco-based cloud platform that has quietly amassed two million developers without spending a dollar on marketing, announced Thursday that it raised $100 million in a Series B funding round, as surging demand for artificial intelligence applications exposes the limitations of legacy cloud infrastructure.
TQ Ventures led the round, with participation from FPV Ventures, Redpoint, and Unusual Ventures. The investment values Railway as one of the most significant infrastructure startups to emerge during the AI boom, capitalizing on developer frustration with the complexity and cost of traditional platforms like Amazon Web Services and Google Cloud.
"As AI models get better at writing code, more and more people are asking the age-old question: where, and how, do I run my applications?" said Jake Cooper, Railway's 28-year-old founder and chief executive, in an exclusive interview with VentureBeat. "The last generation of cloud primitives were slow and outdated, and now with AI moving everything faster, teams simply can't keep up."
The funding is a dramatic acceleration for a company that has charted an unconventional path through the cloud computing industry. Railway raised just $24 million in total before this round, including a $20 million Series A from Redpoint in 2022. The company now processes more than 10 million deployments monthly and handles over one trillion requests through its edge network — metrics that rival far larger and better-funded competitors.
Why three-minute deploy times have become unacceptable in the age of AI coding assistants
Railway's pitch rests on a simple observation: the tools developers use to deploy and manage software were designed for a slower era. A standard build-and-deploy cycle using Terraform, the industry-standard infrastructure tool, takes two to three minutes. That delay, once tolerable, has become a critical bottleneck as AI coding assistants like Claude, ChatGPT, and Cursor can generate working code in seconds.
"When godly intelligence is on tap and can solve any problem in three seconds, those amalgamations of systems become bottlenecks," Cooper told VentureBeat. "What was really cool for humans to deploy in 10 seconds or less is now table stakes for agents."
The company claims its platform delivers deployments in under one second — fast enough to keep pace with AI-generated code. Customers report a tenfold increase in developer velocity and up to 65 percent cost savings compared to traditional cloud providers.
These numbers come directly from enterprise clients, not internal benchmarks. Daniel Lobaton, chief technology officer at G2X, a platform serving 100,000 federal contractors, measured deployment speed improvements of seven times faster and an 87 percent cost reduction after migrating to Railway. His infrastructure bill dropped from $15,000 per month to approximately $1,000.
"The work that used to take me a week on our previous infrastructure, I can do in Railway in like a day," Lobaton said. "If I want to spin up a new service and test different architectures, it would take so long on our old setup. In Railway I can launch six services in two minutes."
Inside the controversial decision to abandon Google Cloud and build data centers from scratch
What distinguishes Railway from competitors like Render and Fly.io is the depth of its vertical integration. In 2024, the company made the unusual decision to abandon Google Cloud entirely and build its own data centers, a move that echoes the famous Alan Kay maxim: "People who are really serious about software should make their own hardware."
"We wanted to design hardware in a way where we could build a differentiated experience," Cooper said. "Having full control over the network, compute, and storage layers lets us do really fast build and deploy loops, the kind that allows us to move at 'agentic speed' while staying 100 percent the smoothest ride in town."
The approach paid dividends during recent widespread outages that affected major cloud providers — Railway remained online throughout.
This soup-to-nuts control enables pricing that undercuts the hyperscalers by roughly 50 percent and newer cloud startups by three to four times. Railway charges by the second for actual compute usage: $0.00000386 per gigabyte-second of memory, $0.00000772 per vCPU-second, and $0.00000006 per gigabyte-second of storage. There are no charges for idle virtual machines — a stark contrast to the traditional cloud model where customers pay for provisioned capacity whether they use it or not.
"The conventional wisdom is that the big guys have economies of scale to offer better pricing," Cooper noted. "But when they're charging for VMs that usually sit idle in the cloud, and we've purpose-built everything to fit much more density on these machines, you have a big opportunity."
How 30 employees built a platform generating tens of millions in annual revenue
Railway has achieved its scale with a team of just 30 employees generating tens of millions in annual revenue — a ratio of revenue per employee that would be exceptional even for established software companies. The company grew revenue 3.5 times last year and continues to expand at 15 percent month-over-month.
Cooper emphasized that the fundraise was strategic rather than necessary. "We're default alive; there's no reason for us to raise money," he said. "We raised because we see a massive opportunity to accelerate, not because we needed to survive."
The company hired its first salesperson only last year and employs just two solutions engineers. Nearly all of Railway's two million users discovered the platform through word of mouth — developers telling other developers about a tool that actually works.
"We basically did the standard engineering thing: if you build it, they will come," Cooper recalled. "And to some degree, they came."
From side projects to Fortune 500 deployments: Railway's unlikely corporate expansion
Despite its grassroots developer community, Railway has made significant inroads into large organizations. The company claims that 31 percent of Fortune 500 companies now use its platform, though deployments range from company-wide infrastructure to individual team projects.
Notable customers include Bilt, the loyalty program company; Intuit's GoCo subsidiary; TripAdvisor's Cruise Critic; and MGM Resorts. Kernel, a Y Combinator-backed startup providing AI infrastructure to over 1,000 companies, runs its entire customer-facing system on Railway for $444 per month.
"At my previous company Clever, which sold for $500 million, I had six full-time engineers just managing AWS," said Rafael Garcia, Kernel's chief technology officer. "Now I have six engineers total, and they all focus on product. Railway is exactly the tool I wish I had in 2012."
For enterprise customers, Railway offers security certifications including SOC 2 Type 2 compliance and HIPAA readiness, with business associate agreements available upon request. The platform provides single sign-on authentication, comprehensive audit logs, and the option to deploy within a customer's existing cloud environment through a "bring your own cloud" configuration.
Enterprise pricing starts at custom levels, with specific add-ons for extended log retention ($200 monthly), HIPAA BAAs ($1,000), enterprise support with SLOs ($2,000), and dedicated virtual machines ($10,000).
The startup's bold strategy to take on Amazon, Google, and a new generation of cloud rivals
Railway enters a crowded market that includes not only the hyperscale cloud providers—Amazon Web Services, Microsoft Azure, and Google Cloud Platform—but also a growing cohort of developer-focused platforms like Vercel, Render, Fly.io, and Heroku.
Cooper argues that Railway's competitors fall into two camps, neither of which has fully committed to the new infrastructure model that AI demands.
"The hyperscalers have two competing systems, and they haven't gone all-in on the new model because their legacy revenue stream is still printing money," he observed. "They have this mammoth pool of cash coming from people who provision a VM, use maybe 10 percent of it, and still pay for the whole thing. To what end are they actually interested in going all the way in on a new experience if they don't really need to?"
Against startup competitors, Railway differentiates by covering the full infrastructure stack. "We're not just containers; we've got VM primitives, stateful storage, virtual private networking, automated load balancing," Cooper said. "And we wrap all of this in an absurdly easy-to-use UI, with agentic primitives so agents can move 1,000 times faster."
The platform supports databases including PostgreSQL, MySQL, MongoDB, and Redis; provides up to 256 terabytes of persistent storage with over 100,000 input/output operations per second; and enables deployment to four global regions spanning the United States, Europe, and Southeast Asia. Enterprise customers can scale to 112 vCPUs and 2 terabytes of RAM per service.
Why investors are betting that AI will create a thousand times more software than exists today
Railway's fundraise reflects broader investor enthusiasm for companies positioned to benefit from the AI coding revolution. As tools like GitHub Copilot, Cursor, and Claude become standard fixtures in developer workflows, the volume of code being written — and the infrastructure needed to run it — is expanding dramatically.
"The amount of software that's going to come online over the next five years is unfathomable compared to what existed before — we're talking a thousand times more software," Cooper predicted. "All of that has to run somewhere."
The company has already integrated directly with AI systems, building what Cooper calls "loops where Claude can hook in, call deployments, and analyze infrastructure automatically." Railway released a Model Context Protocol server in August 2025 that allows AI coding agents to deploy applications and manage infrastructure directly from code editors.
"The notion of a developer is melting before our eyes," Cooper said. "You don't have to be an engineer to engineer things anymore — you just need critical thinking and the ability to analyze things in a systems capacity."
What Railway plans to do with $100 million and zero marketing experience
Railway plans to use the new capital to expand its global data center footprint, grow its team beyond 30 employees, and build what Cooper described as a proper go-to-market operation for the first time in the company's five-year history.
"One of my mentors said you raise money when you can change the trajectory of the business," Cooper explained. "We've built all the required substrate to scale indefinitely; what's been holding us back is simply talking about it. 2026 is the year we play on the world stage."
The company's investor roster reads like a who's who of developer infrastructure. Angel investors include Tom Preston-Werner, co-founder of GitHub; Guillermo Rauch, chief executive of Vercel; Spencer Kimball, chief executive of Cockroach Labs; Olivier Pomel, chief executive of Datadog; and Jori Lallo, co-founder of Linear.
The timing of Railway's expansion coincides with what many in Silicon Valley view as a fundamental shift in how software gets made. Coding assistants are no longer experimental curiosities — they have become essential tools that millions of developers rely on daily. Each line of AI-generated code needs somewhere to run, and the incumbents, by Cooper's telling, are too wedded to their existing business models to fully capitalize on the moment.
Whether Railway can translate developer enthusiasm into sustained enterprise adoption remains an open question. The cloud infrastructure market is littered with promising startups that failed to break the grip of Amazon, Microsoft, and Google. But Cooper, who previously worked as a software engineer at Wolfram Alpha, Bloomberg, and Uber before founding Railway in 2020, seems unfazed by the scale of his ambition.
"In five years, Railway [will be] the place where software gets created and evolved, period," he said. "Deploy instantly, scale infinitely, with zero friction. That's the prize worth playing for, and there's no bigger one on offer."
For a company that built a $100 million business by doing the opposite of what conventional startup wisdom dictates — no marketing, no sales team, no venture hype—the real test begins now. Railway spent five years proving that developers would find a better mousetrap on their own. The next five will determine whether the rest of the world is ready to get on board.
