Most people already understand what generative AI can do. But enterprises run into problems when they need to give a model access to information that cannot leave their own environment, such as a patient record, a customer’s financial details, or a company’s most valuable intellectual property.

Sending that data to a cloud or SaaS service means it crosses external networks and is processed on infrastructure run by another organization, creating additional concerns about control, accountability, and exposure. That’s where AI enthusiasm collides with production realities. Despite its productivity potential, enterprise AI still faces a fundamental gap in trust and control.

Organizations need to know whether a system will expose information it should protect, act as intended, meet security and performance requirements, and behave safely at machine speed.

Alon Horev, CTO and co-founder of AI operating system company VAST Data, tells The New Stack that the challenge is particularly acute when AI systems handle sensitive customer information. “Even if you ask the model today to obfuscate a conversation or redact PII from a conversation, it’s hard to have 100% confidence that’s the case, and that it worked.”

“Even if you ask the model today to obfuscate a conversation or redact PII from a conversation, it’s hard to have 100% confidence that’s the case, and that it worked.”

Consider a customer support agent that needs access to an individual’s profile to provide a useful, personalized answer. The organization must ensure that information isn’t exposed to another customer, while also considering whether those conversations can be used for training or system improvement. They might contain personally identifiable information (PII) or other protected details, and the consequences of mishandling them ultimately fall on the organization and the people whose information it holds.

Confidential AI architectures: solving a two-sided trust problem

Enterprise AI has two parties to satisfy: organizations must keep sensitive data under their control, while model builders need to protect the weights and software that represent substantial investments in research, engineering, and IP. They’re understandably reluctant to place those assets in environments where customers, infrastructure operators, or attackers might gain access. That mutual need for control has created a stalemate. How can organizations bring advanced models to sensitive data without asking either side to surrender control?

Horev has seen that the most capable models are increasingly delivered as SaaS services, because that’s the simplest way for their creators to distribute and protect them. Even when a provider offers compliance controls, the enterprise might still shoulder the consequences of a breach, misuse, or regulatory violation. Sending information across the WAN also places it in the hands of more systems, connections, and operators, increasing the number of points that must be trusted and governed. Organizations may also be unwilling, or legally unable, to rely on a provider’s assurances that it will not retain, reuse, or expose their data beyond the intended service. 

For organizations in regulated or data sovereignty-sensitive sectors, that could be an unacceptable trade-off. “Naturally, many organizations are adopting a hybrid strategy,” Horev tells The New Stack. “Some applications and datasets can go to the cloud, while others must remain on-premises, sometimes even in the building, or in the country.”

This is where confidential AI comes in. Encryption at rest and in transit protects data while it’s stored or moving between systems. Confidential computing extends that protection into the processing environment, using hardware-isolated execution to create a protected enclave in which the data and model weights can remain encrypted until they’re released to an approved workload.

Cryptographic attestation verifies the hardware, virtual machine (VM), software, and configuration requesting access before releasing keys. The model builder can encrypt its model using the public key of a specific confidential VM. Only that VM’s corresponding private key can decrypt it within protected memory, enabling the customer to use the model without accessing its weights.

Independent key control preserves the separation between the two sides. The enterprise retains control of the keys governing its data, while the model builder retains control of the keys governing its model. While the workload is running, the infrastructure operator doesn’t control either set of keys.

As AI becomes more agentic, those controls will matter more. Agents will need to access more data, systems and tools, and might act on that information with far less human intervention.

“This world of agentic AI is moving extremely fast, and we need to limit what an agent can see and do.”

Those that can’t establish strong privacy and governance assurances for today’s models will find it even harder to deploy agents safely in the future. “This world of agentic AI is moving extremely fast, and we need to limit what an agent can see and do,” Horev tells The New Stack.

From architecture to ecosystem

Many businesses simply cannot manage the integration, security, and maintenance of the entire AI stack, because it requires working separately with each model provider to engineer something that suits both parties. Turning confidential AI architecture into something organizations can deploy is the challenge VAST DataEnclave, which was launched on September 22, intends to address.

As a capability of the VAST AI Operating System, the goal is to bring the model, application layer, and data platform together under customer-controlled operating conditions. The architecture is designed to protect both sides of the equation: the enterprise’s data and the model builder’s weights. The customer retains control of its infrastructure and data keys, while the model provider can make its software available without handing over the underlying intellectual property.

“We’re trying to close the trust and control gap by working with world-class model builders such as Cohere, Deepgram, Factory, Fundamental and TwelveLabs, who continue to innovate and build their expertise,” says Horev. The ecosystem also includes infrastructure and security providers such as Nvidia, CrowdStrike, Fortanix, Nscale, Cisco, and Supermicro. The range reflects the practical challenge: confidential AI needs more than a protected GPU. It requires models, applications, accelerated hardware, data infrastructure, and operational support to work together.

That control also changes the cost conversation, without automatically making AI cheaper. Hosted models can make budgets harder to predict as token consumption varies with usage patterns, agent loops, model architecture, and workload volume. Customer-controlled infrastructure gives enterprises a more defined capacity and cost base: they can plan around GPU clusters they own or have already budgeted for, instead of allowing inefficient model choices or uncontrolled agent activity to generate an open-ended token bill.

“…instead of allowing inefficient model choices or uncontrolled agent activity to generate an open-ended token bill.”

The cluster also imposes a natural ceiling on throughput, which helps organizations understand how much work their infrastructure can handle within a given period. Model providers can then price access by token, task, or license, while the enterprise retains greater visibility into its total operating cost.

Why the data platform is paramount

Confidential AI protects data and model weights during inference, but it’s only part of the production challenge. Real-world AI systems are living environments in which data moves between storage, databases, GPUs, networks, applications, and agents.

That’s why confidential AI can’t be bolted onto a fragmented stack. Businesses need to protect the model, the data, and the infrastructure connecting them as one system. As Horev says: “You need to build security in multiple layers of the platform,” with someone accountable for rapidly updating compromised components.

Confidentiality is only useful if the resulting system can also be operated, monitored, and improved. As AI infrastructure becomes more distributed, it becomes harder to tell what’s happening when something goes wrong and where the fault lies.

Horev recommends a “single pane of glass” across storage, networking, and compute, so teams can see what’s happening and keep resolution times low. If a network port is intermittently failing in a data center, for example, an agent could help identify the root cause, provided it has access to the right operational data and tightly controlled permissions. Those permissions should govern the infrastructure it can inspect, the data it can retrieve, and the actions it can take.

The same applies to monitoring AI workloads. Teams need visibility into performance, failures, and access patterns without exposing the customer data or model weights. Agent sandboxes can limit the systems and tools an agent can reach, while data platform observability can log which data it accessed, what it did with that data, and how it interacted with downstream systems.

Evaluation, therefore, becomes part of production discipline. Teams must observe systems, measure behavior, govern access, and manage change in ways that demonstrate progress. Confidentiality, data-level policy, observability and correctness have to work together.

The emerging ecosystem suggests demand for models that can run securely under customer control, wherever sensitive data resides. These are “living systems,” says Horev. “It’s not just leveraging a feature inside of a wider platform.” 

Visit the VAST Data Confidential AI solution page to learn more about the architecture, ecosystem, and availability.

The post How confidential AI splits control between data and model owners — and opens new opportunities for both appeared first on The New Stack.

A security researcher testing a 300-person B2B company with a global footprint discovered an internet-exposed database with weak authentication during a routine scan. The database appeared to be a prime target for attackers; it had critical severity and was an obvious first-fix candidate. Upon further inspection, however, researchers learned it was a resettable test database used to test job candidates, rather than a system containing client data.

The episode represents a critical problem: Scanners and security researchers can’t infer the real cost of a compromise on their own. Increasingly, companies rely on an external partner to handle cloud vulnerability triage and keep teams from becoming overwhelmed. Scanners produce excessive noise, requiring focus and attention from engineers to understand alerts and complex attack techniques, tune out false positives, and work with teams on remediation, reducing the time they have to build new features, work with customers, or scale up their tech environment.

Tech employees have more on their to-do lists than ever: Product teams face an infinite stream of feature requests and engineering teams carry more technical debt than they can clear. When you factor in a reorg, many employees may have larger project scopes and leaner teams, all while constantly monitoring, correcting, and mentoring AI agents. Some reports cite 90-hour work weeks.

Beyond today’s structural challenges, security teams are drowning in a different type of data. Programs absorb identity events, firewall logs, endpoint alerts, vendor feeds, and threat intelligence, producing far more analysis data than even a few years ago. Every item can look urgent, high-risk, and worthy of immediate attention. With finite hours, it’s hard to know where to act first.

Jon Rose, founder of the information security and risk management advisory firm IOmergent,  has seen firsthand how AI and near-universal tooling are producing more findings than teams can triage. 

“Within the span of security work, there’s an unending list of things you could tackle, and you’re pulled in so many different directions… But you have to be ruthless about prioritizing and investing your time.”

“Within the span of security work, there’s an unending list of things you could tackle, and you’re pulled in so many different directions,” Rose tells The New Stack. “But you have to be ruthless about prioritizing and investing your time.”

The challenge, then, isn’t remediation. It’s allocation: Deciding where limited engineering and security capacity will make the greatest impact is the most important question security leaders answer every day. A technical severity score, used without threat and environmental context, cannot answer the business question: What should we fix first? Effective vulnerability prioritization turns raw findings into business-aware priorities, and that judgment is the real work.

CVSS limitations: a starting point, not a decision

For security teams, a Common Vulnerability Scoring System (CVSS) provides a useful baseline. Its base metrics classify a vulnerability’s severity — attack vector, complexity, required privileges, and potential impact on confidentiality, integrity, and availability. But a severity score is designed to be stable across environments, so it can’t tell a team whether an asset is exposed to the internet, shielded by compensating controls, or central to the business.

Essentially, treating a base severity score as an automatic fix-first ticket is problematic, rather than using CVSS on its own.

CVSS can include Threat and Environmental metrics that account for evolving exploit conditions and organization-specific context. However, risk-based vulnerability management still depends on accurate knowledge of the environment — and on someone applying that context consistently. 

“The piece that’s missing from any of these tools is the grounding in the business, the understanding of what actually matters,” Rose says.

Escalate the internet-facing medium

A smart decision framework deprioritizes the urgency of a score and probes reachability: whether an attacker could actually reach the vulnerable component. 

“Is the affected service exposed to the public internet or not? Or is it isolated behind network controls and accessible only to a limited set of internal users?” Rose says. “Often, issues will get flagged, but it’s not in a position where it could be triggered.”

Next is consequence. An exposed flaw on a disposable test system might be a genuine security concern. Still, it doesn’t carry the same weight as a weakness on the application that processes customer transactions, stores sensitive information, or underpins a company’s main revenue stream. So while the severity might be alarming, the business outcome might not.

Teams should also ask whether the weakness is attracting active attacker interest and where it could lead. A vulnerability listed in CISA’s Known Exploited Vulnerabilities catalog deserves urgent scrutiny, because there’s evidence of exploitation in the wild.

The Exploit Prediction Scoring System provides a forward-looking signal: An estimate of the likelihood that exploitation of a particular vulnerability will be observed over the next 30 days. Neither replaces a business decision, but both help distinguish a theoretical risk from one that demands attention now. However, as AI-driven exploitation accelerates, the window between what is known to be vulnerable and what is actively exploited is shrinking because the cost of building and weaponizing exploits is dropping.

The relevant attack path might also extend beyond the affected machine. A comparatively modest flaw can become an immediate concern if it provides a route into privileged accounts, a production system, or customer data. Conversely, a high-severity finding can be deprioritized — temporarily — if it is not reachable, has limited impact, and is protected by reliable controls.

“The speed and the depth of research and investigation into those security issues are going faster,” Rose says. “So it can change really quickly.” The worst state is unacknowledged risk sitting in the backlog. Even the best detection degrades when no one owns the trend line. Teams should track exceptions, set review dates, and name an owner. Accepted risk is still risk; the difference is that it’s explicit, time-bound, and revisited. 

An operator capability, not a weekend project

Security teams are leaning more on AI to prioritize and triage threats, and they’re uncovering vulnerabilities at an unprecedented pace. Yet the sheer volume of findings is overwhelming, even for the most efficient of humans.

It’s clear, too, that AI can make it easier to discover less obvious paths to exploitation, and to introduce new ones. An academic study of 20,000+ issues fixed by AI found that LLMs introduce nearly 9x as many new vulnerabilities as developers, exhibiting unique patterns not found in developers’ code. The answer is to use AI at the outcome level. For every alert, the goal is to have the SOC analyst’s standard questions answered quickly: new or known, what’s exposed, what data is at risk, prod or dev, and how it has evolved.

“Effective programs start by aligning with executive teams to understand the business — where the company is going — so allocation and adjustments track the actual risk, not just the score.”

“That’s how teams get thousands of alerts down to 10 to 20 prioritized tickets,” Rose tells The New Stack. “Effective programs start by aligning with executive teams to understand the business — where the company is going — so allocation and adjustments track the actual risk, not just the score.”

As to-do lists grow ever longer, business-context judgment that’s applied every day by someone who owns it is something worth dedicating more resources.

If you think you’d benefit from managed cloud security, book a 30-minute cloud security scoping call with IOmergent.

The post The critical vulnerability was a test database. That’s the whole triage problem. appeared first on The New Stack.

As enterprises move from experimenting with AI to running autonomous agents in production, an infrastructure problem is emerging: rising telemetry costs. Non-deterministic, iterative, and capable of generating data at machine speed, agents are far harder to monitor — and their costs far harder to predict — than conventional applications.

Many companies are struggling to attribute and defend their telemetry bills. In fact, 59% of organizations have already terminated or delayed an agentic AI deployment due to monitoring costs, according to a survey of more than 300 enterprise IT decision-makers in North America and Western Europe, commissioned by Apica and conducted by Omdia/Informa TechTarget. 

The agents most affected are often in some of the most high-stakes deployments: think cybersecurity, compliance, and fraud detection. As monitoring bills explode, deployments aren’t necessarily getting killed by engineering teams. More often than not, it’s finance pulling the plug.

Andi Mann, chief product and technology officer at Apica, recently saw this play out at a large bank. The organization couldn’t pin down exactly what it was spending on its AI programs.

“They knew they couldn’t afford to keep going on the same trajectory, so they had no choice but to cancel certain AI programs,” Mann tells The New Stack. “It’s a pattern I have seen before, because AI projects are cannibalizing typical budgets.” 

“It’s a pattern I have seen before, because AI projects are cannibalizing typical budgets.”

The implications are huge. As people, funding, and monitoring resources are diverted toward new AI workloads, other parts of the business start to suffer. Mann says he’s seeing outages, downtime, penetration attacks, and DDoS protection compete for the same resources.

The problem is already showing up in research. Most (54%) enterprises have seen telemetry volume triple in the past year alone, with 43% of that growth coming from AI/ML workloads — by far the largest driver. Businesses are under pressure to address this crisis as their observability bills climb.

They report spending an average of $3.17 million on observability, with that figure growing 28% year over year and showing no obvious ceiling. No wonder then that 83% rank AI observability as a top priority for the year ahead.

The coming wave could be catastrophic

With a new cloud service, database, or application, you’d expect the monitoring burden and telemetry data to rise by a relatively predictable increment. But enterprises now foresee an average 9.5X increase in telemetry data within two years.

“Imagine if your credit card or grocery bill went up more than nine times — that is not a marginal amount,” Mann says. “This isn’t a gentle ramp; it’s a skyscraper, and it’s prompting panic.”

“This isn’t a gentle ramp; it’s a skyscraper, and it’s prompting panic.”

Some 44% of organizations expect their telemetry data to increase by 6X to 100X.

The reason is that an agent task isn’t the same as a single application request. A customer-support task might generate a top-level trace, several model calls, retrieval operations, tool calls, retries, and loops. But if the agent delegates work to another agent, that adds another branch to the trace. 

Each model can produce token, latency, cost, and provider data, while each tool call generates its own records for arguments, results, status, and downstream activity. Identifiers such as tool_name, agent_id, and trace_id also create high cardinality, making data harder to aggregate and more expensive to index, with costs compounding at every stage.

That creates an uncomfortable gap between AI ambition and infrastructure readiness. Although 35% of enterprises claim widespread agentic AI deployment, operating and managing those systems is very different. Nearly two-thirds are only somewhat prepared for the shift. Unlike conventional applications, agents can call multiple models and tools, repeat tasks, or expand a workflow in unpredictable ways, making both capacity and monitoring costs difficult to forecast.

From extreme telemetry costs to an upstream control layer

The answer, according to Mann, is to intervene earlier. “You can’t keep sending essentially useless data to an expensive central analytics or storage platform because there’s no point analyzing data which says everything is fine,” he says. “As early as possible, get the pipeline to use data collectors and manage those at source.”

Legacy observability platforms were built around collecting data, ingesting it, storing and indexing it, and then analyzing it. That model worked for human-driven, dashboard-queried workloads, but agentic AI requires decisions to be made before telemetry reaches the most expensive parts of the stack.

A pipeline-first architecture can sample repetitive successful events while retaining failures, retries, policy violations, and unusually slow traces. It can enrich records with agent, session, model, tool, token, and estimated-cost data, redact sensitive prompts and identifiers, and aggregate metrics and long-term records to destinations with different cost and retention profiles.

A compact metric or sampled span might represent a routine successful tool call, while a failed call retains its parent trace, error details, retry history, and security context. The aim is to preserve the information needed to explain an agent’s behavior while reducing redundant data and limiting what gets indexed.

Agents also need millisecond-level context for autonomous decisions. Processing telemetry close to its source allows organizations to quickly detect retry storms, excessive tool loops, or unusual token consumption, rather than waiting for data to be ingested and indexed centrally. 

Without upstream control, businesses risk feeding fragmented, unnecessary telemetry into platforms that charge for every additional gigabyte, index, and retained record.

Architecture that separates winners from cancellations

The payoff for rethinking the telemetry pipeline is hard to ignore. Enterprises with a telemetry pipeline are 50% more likely to be prepared for the growth of agentic AI data. Among mature agentic AI organizations, pipeline adoption is what sets them apart: these organizations are 80% more likely to have avoided the operational cost challenges hampering their peers.

Rather than treating the observability platform as a catch-all destination, enterprises can make decisions about telemetry before it gets there.

The answer is to move the intelligence upstream. Rather than treating the observability platform as a catch-all destination, enterprises can make decisions about telemetry before it gets there — filtering out noise, enriching what matters, and routing data according to its value and purpose. That means less data hitting expensive storage and analytics systems, while the information that does make it through is more useful and available in real time.

Crucially, this isn’t about ripping out the observability platforms enterprises already rely on. It’s about putting a smarter control layer in front of them: deciding what data deserves to be ingested, where it should go, and how much it should cost.

Existing observability platforms still have an important role to play. “The pipeline can’t do everything, but it can act as a first responder,” says Mann. “You still work with the big analytics platforms, but you’re saving money, reducing risk, and improving your compliance performance.”

The Apica study finds that its pipeline control, metrics foundation, and data readiness services, for example, can reduce the total cost of ownership by 40% compared to legacy observability platforms. Of course, though, the actual savings will depend on telemetry volumes, retention policies, sampling rules, routing decisions, existing contracts, and the proportion of data that can be processed before ingestion.

The window to rethink the architecture is now. Some 68% of enterprises plan to evaluate changes to their observability stack within the next six months, while almost a quarter say existing vendor relationships won’t be a significant factor in those decisions. The next phase of observability will be won by the ability to handle what agentic AI throws at the infrastructure.

That means the pipeline can no longer be treated as plumbing that moves telemetry from A to B. It’s becoming the control layer for an increasingly autonomous, data-hungry environment. 

Organizations that establish agentic-ready infrastructure will be better positioned to reduce observability costs and improve risk performance. Mann doesn’t think platform engineering and SRE teams have much choice. “This has already become a board-level decision,” he says. “Ultimately, it’s a choice about how smart you can afford to make your business.”

Download the Omdia research report: “The Agentic AI Telemetry Crisis: Are You Ready for What’s Coming?”

The post How telemetry pipelines keep AI agent costs under control appeared first on The New Stack.