As organizations invest in AI, many are discovering a new bottleneck: obtaining data that is both protected and useful. Engineering teams need realistic, production-like data to validate AI-generated changes, train models, test applications, and generate business insights. Yet, privacy initiatives can sometimes make that data harder to access, less representative of real-world conditions, or unable to preserve critical relationships between records. 

The Perforce Delphix “2026 State of AI and Data Privacy Report” highlights this challenge. Among surveyed organizations, 26% say privacy controls make production-quality data harder to obtain, 25% struggle to preserve relationships across data entities, and 51% cite data quality challenges. 

“Protecting data isn’t enough if it can no longer support the systems that depend on it.”

Protecting data isn’t enough if it can no longer support the systems that depend on it. For engineering teams, the question is whether their data protection strategies can preserve the qualities that make data valuable in the first place. You need a well-rounded data strategy with tools that maintain referential integrity and relationships across your environments.

What these statistics mean for practitioners

At first glance, statistics from the report, like “51% of enterprises cite data quality challenges,” may sound like a purely governance issue.

In practice, they represent engineering problems. Low-quality datasets can produce:

• Inaccurate analytics.
• Poorly trained AI models.
• Incomplete test coverage.
• Increased rework.
• Delayed releases.
• Reduced confidence in data automation.

“When an AI model is trained on incomplete or distorted data, its outputs become less reliable.”

When an AI model is trained on incomplete or distorted data, its outputs become less reliable. When test environments contain unrealistic data, defects can escape into production. When analytics datasets lack consistency, teams spend more time validating results than acting on them.

Data protection and utility are not opposing goals

A common misconception is that organizations must choose between privacy and innovation, but the most successful organizations know that compliance, quality, and speed can and need to work together.

Protected data still needs to be:

• Realistic enough for testing and validation.
• Representative enough for analytics.
• Accessible enough for engineering teams.
• Governed enough for regulatory requirements.
• Connected enough to preserve referential integrity.

There’s a two-fold goal in the AI era: reduce sensitive data exposure while also creating trusted data that remains valuable after protection. All too often, enterprises sacrifice compliance for innovation or speed. That’s a big reason 84% of respondents in our report have a data privacy exception in their non-production environments.

“There’s a two-fold goal in the AI era: reduce sensitive data exposure while also creating trusted data that remains valuable after protection.”

Organizations can only move at AI speed when they have access to trustworthy data that accurately represents production conditions. When privacy controls degrade quality, limit realism, or restrict access to representative datasets, the data layer becomes the new bottleneck.

Why referential integrity matters more than ever

Many discussions about data privacy focus on masking sensitive fields. However, masked data that loses referential integrity between entities can create a different kind of risk.

A customer, order, or payment record may still exist, but if the relationships connecting those records break during protection processes, the data no longer resembles reality. Take billing validation, for example. It needs referential integrity when a customer has multiple products, charges, and invoices across several database tables to produce the correct products or make accurate charges. 

Broken relationships are especially problematic for modern AI and analytics systems. Analytics pipelines depend on consistent identifiers to join information across sources. AI and machine learning workflows depend on complete business context to identify patterns and make predictions. Software testing depends on realistic relationships between records to validate application behavior accurately.

When referential integrity is lost:

• Analytics can produce incomplete or misleading results.
• AI models can learn from flawed datasets.
• Testing environments can fail to expose production issues.
• Teams lose trust in protected datasets.

Importantly, these failures are often difficult to detect. Pipelines may continue running successfully while quietly producing degraded outcomes, which can be a very costly mistake.

This helps explain why 25% of surveyed organizations cited preserving relationships across data entities as a significant challenge. For practitioners, that statistic is a warning that privacy controls can unintentionally undermine the quality of AI and analytics initiatives if they fail to preserve business context.

Keep in mind that not all data protection solutions are created equal — enterprise-grade masking algorithms are key to preserving relationships. When applied consistently and at scale, these algorithms ensure the same input produces the same masked output across your systems and environments. Other masking approaches might be done piecemeal, resulting in broken relationships.

What engineering teams should measure

Many organizations measure privacy success through compliance metrics alone. However, AI-driven environments require a broader definition of success. Engineering leaders should evaluate privacy initiatives against several dimensions:

• Data quality: Does the protected dataset accurately reflect production conditions?
• Realism: Can developers, data scientists, and analysts use the data confidently for their intended purpose?
• Referential integrity: Do relationships remain consistent across applications, tables, environments, and data sources?
• Accessibility: Can teams obtain compliant data without introducing delays?
• Provisioning speed: How quickly can trusted datasets be delivered when needed?

These measurements help organizations determine whether privacy efforts enable AI outcomes or create new obstacles.

Designing for governance by default

As AI adoption grows, privacy cannot remain a separate process that occurs after development begins. Organizations should instead map the entire data lifecycle, from data request and discovery to reuse and retirement. 

This approach helps ensure governance is built into workflows rather than applied as a late-stage checkpoint. It also provides stronger auditability, reduces compliance exceptions, and gives teams greater confidence that protected datasets remain fit for purpose.

Most importantly, it aligns privacy objectives with business outcomes instead of treating them as competing priorities.

Trusted data will become a competitive differentiator

The need for test data — in volume, coverage, and scale — is booming as agentic development continues to rise. AI has also increased the volume of change enterprises can generate, but the challenge of validating that change remains unsolved.

That responsibility still belongs to data. The organizations that gain the most value from AI will not necessarily be the ones with the most advanced models. They will be the ones with the most trustworthy data foundations — using a portfolio approach that combines data virtualization for speed, masking for security, and synthetic data for coverage as needed.

“In the AI era, trusted data, not fast model output, may be the ultimate competitive advantage.”

The research points to a clear lesson: If privacy controls uphold realism, quality, relationships between records, or access to representative datasets, they can enable AI success.

As enterprises continue investing in AI and data privacy, the real objective should be ensuring protection and utility coexist. Because in the AI era, trusted data, not fast model output, may be the ultimate competitive advantage.

The post Can enterprises protect data without making AI less reliable? appeared first on The New Stack.

Not your keys, not your coins. Not your model, not your data?

Over the summer, the tech industry was consumed by a debate about AI use in the enterprise and the need to protect IP. If an enterprise used proprietary models, was data leakage a necessary evil?

Companies seemed to have two options: They could use state-of-the-art, proprietary models and risk losing control of their data, or they could use open-weight models and never kiss the frontier.

Thankfully, a third option is emerging.

Consider the concern: Company A wants to use LLM B from AI Lab C, and they want to avoid training AI Lab C how to eat Company A’s lunch by building its capabilities into LLM B. A good way to resolve the tension would be to let Company A run LLM B on its own infrastructure, so there’s no risk of its information fleeing on the wind.

AI agents are “creating a whole different set of requirements at the data layer.”
–Vast Data co-founder Jeff Denworth

But that raises another problem: AI Lab C doesn’t want to allow Company A to run LLM B on its own GPUs because it doesn’t want to hand over its model weights. It’s the same IP issue the company ran into, in reverse. You have to solve the trust problem in both directions!

Enter VAST Data co-founder Jeff Denworth and a new product called DataEnclave, which aims to let AI labs and enterprise-scale companies deploy proprietary models in secure compute environments without risking data transfer in either direction. (DataEnclave uses Nvidia’s Confidential Computing technology to make the system tick; Vast Data’s core product is AI OS, infrastructure that fits beneath a company’s AI applications.) 

The New Stack had Denworth on the podcast to chat about the confidential computing market. I was curious about timing. Why did Vast build DataEnclave now? Nvidia began rolling out Confidential Computing in a serious way in 2024, after all. Denworth argues that the market needed the core technology, yes, but also demand.

And until late 2025, AI demand was modest compared to today’s token totals. Once agentic coding tools took off, corporate demand for AI products soared. This led to the pricing crisis we saw in early 2026, and the secure AI usage debate we endured over the summer. 

Performance drove demand, demand drove usage, and usage dug up fresh problems to solve. Now the question for the market is whether or not DataEnclave has solved enough concerns on both sides of the proprietary AI-proprietary data equation. The market will sort that out as it moves through early access and into general availability.

Our conversation goes deep into the arc of AI, where companies are in their AI journey today, and how much data remains to be unlocked inside the enterprise. If you want to feel the acceleration, it’s a fun one!

The post A third option is emerging in the fight over AI and your data appeared first on The New Stack.

Most people already understand what generative AI can do. But enterprises run into problems when they need to give a model access to information that cannot leave their own environment, such as a patient record, a customer’s financial details, or a company’s most valuable intellectual property.

Sending that data to a cloud or SaaS service means it crosses external networks and is processed on infrastructure run by another organization, creating additional concerns about control, accountability, and exposure. That’s where AI enthusiasm collides with production realities. Despite its productivity potential, enterprise AI still faces a fundamental gap in trust and control.

Organizations need to know whether a system will expose information it should protect, act as intended, meet security and performance requirements, and behave safely at machine speed.

Alon Horev, CTO and co-founder of AI operating system company VAST Data, tells The New Stack that the challenge is particularly acute when AI systems handle sensitive customer information. “Even if you ask the model today to obfuscate a conversation or redact PII from a conversation, it’s hard to have 100% confidence that’s the case, and that it worked.”

“Even if you ask the model today to obfuscate a conversation or redact PII from a conversation, it’s hard to have 100% confidence that’s the case, and that it worked.”

Consider a customer support agent that needs access to an individual’s profile to provide a useful, personalized answer. The organization must ensure that information isn’t exposed to another customer, while also considering whether those conversations can be used for training or system improvement. They might contain personally identifiable information (PII) or other protected details, and the consequences of mishandling them ultimately fall on the organization and the people whose information it holds.

Confidential AI architectures: solving a two-sided trust problem

Enterprise AI has two parties to satisfy: organizations must keep sensitive data under their control, while model builders need to protect the weights and software that represent substantial investments in research, engineering, and IP. They’re understandably reluctant to place those assets in environments where customers, infrastructure operators, or attackers might gain access. That mutual need for control has created a stalemate. How can organizations bring advanced models to sensitive data without asking either side to surrender control?

Horev has seen that the most capable models are increasingly delivered as SaaS services, because that’s the simplest way for their creators to distribute and protect them. Even when a provider offers compliance controls, the enterprise might still shoulder the consequences of a breach, misuse, or regulatory violation. Sending information across the WAN also places it in the hands of more systems, connections, and operators, increasing the number of points that must be trusted and governed. Organizations may also be unwilling, or legally unable, to rely on a provider’s assurances that it will not retain, reuse, or expose their data beyond the intended service. 

For organizations in regulated or data sovereignty-sensitive sectors, that could be an unacceptable trade-off. “Naturally, many organizations are adopting a hybrid strategy,” Horev tells The New Stack. “Some applications and datasets can go to the cloud, while others must remain on-premises, sometimes even in the building, or in the country.”

This is where confidential AI comes in. Encryption at rest and in transit protects data while it’s stored or moving between systems. Confidential computing extends that protection into the processing environment, using hardware-isolated execution to create a protected enclave in which the data and model weights can remain encrypted until they’re released to an approved workload.

Cryptographic attestation verifies the hardware, virtual machine (VM), software, and configuration requesting access before releasing keys. The model builder can encrypt its model using the public key of a specific confidential VM. Only that VM’s corresponding private key can decrypt it within protected memory, enabling the customer to use the model without accessing its weights.

Independent key control preserves the separation between the two sides. The enterprise retains control of the keys governing its data, while the model builder retains control of the keys governing its model. While the workload is running, the infrastructure operator doesn’t control either set of keys.

As AI becomes more agentic, those controls will matter more. Agents will need to access more data, systems and tools, and might act on that information with far less human intervention.

“This world of agentic AI is moving extremely fast, and we need to limit what an agent can see and do.”

Those that can’t establish strong privacy and governance assurances for today’s models will find it even harder to deploy agents safely in the future. “This world of agentic AI is moving extremely fast, and we need to limit what an agent can see and do,” Horev tells The New Stack.

From architecture to ecosystem

Many businesses simply cannot manage the integration, security, and maintenance of the entire AI stack, because it requires working separately with each model provider to engineer something that suits both parties. Turning confidential AI architecture into something organizations can deploy is the challenge VAST DataEnclave, which was launched on September 22, intends to address.

As a capability of the VAST AI Operating System, the goal is to bring the model, application layer, and data platform together under customer-controlled operating conditions. The architecture is designed to protect both sides of the equation: the enterprise’s data and the model builder’s weights. The customer retains control of its infrastructure and data keys, while the model provider can make its software available without handing over the underlying intellectual property.

“We’re trying to close the trust and control gap by working with world-class model builders such as Cohere, Deepgram, Factory, Fundamental and TwelveLabs, who continue to innovate and build their expertise,” says Horev. The ecosystem also includes infrastructure and security providers such as Nvidia, CrowdStrike, Fortanix, Nscale, Cisco, and Supermicro. The range reflects the practical challenge: confidential AI needs more than a protected GPU. It requires models, applications, accelerated hardware, data infrastructure, and operational support to work together.

That control also changes the cost conversation, without automatically making AI cheaper. Hosted models can make budgets harder to predict as token consumption varies with usage patterns, agent loops, model architecture, and workload volume. Customer-controlled infrastructure gives enterprises a more defined capacity and cost base: they can plan around GPU clusters they own or have already budgeted for, instead of allowing inefficient model choices or uncontrolled agent activity to generate an open-ended token bill.

“…instead of allowing inefficient model choices or uncontrolled agent activity to generate an open-ended token bill.”

The cluster also imposes a natural ceiling on throughput, which helps organizations understand how much work their infrastructure can handle within a given period. Model providers can then price access by token, task, or license, while the enterprise retains greater visibility into its total operating cost.

Why the data platform is paramount

Confidential AI protects data and model weights during inference, but it’s only part of the production challenge. Real-world AI systems are living environments in which data moves between storage, databases, GPUs, networks, applications, and agents.

That’s why confidential AI can’t be bolted onto a fragmented stack. Businesses need to protect the model, the data, and the infrastructure connecting them as one system. As Horev says: “You need to build security in multiple layers of the platform,” with someone accountable for rapidly updating compromised components.

Confidentiality is only useful if the resulting system can also be operated, monitored, and improved. As AI infrastructure becomes more distributed, it becomes harder to tell what’s happening when something goes wrong and where the fault lies.

Horev recommends a “single pane of glass” across storage, networking, and compute, so teams can see what’s happening and keep resolution times low. If a network port is intermittently failing in a data center, for example, an agent could help identify the root cause, provided it has access to the right operational data and tightly controlled permissions. Those permissions should govern the infrastructure it can inspect, the data it can retrieve, and the actions it can take.

The same applies to monitoring AI workloads. Teams need visibility into performance, failures, and access patterns without exposing the customer data or model weights. Agent sandboxes can limit the systems and tools an agent can reach, while data platform observability can log which data it accessed, what it did with that data, and how it interacted with downstream systems.

Evaluation, therefore, becomes part of production discipline. Teams must observe systems, measure behavior, govern access, and manage change in ways that demonstrate progress. Confidentiality, data-level policy, observability and correctness have to work together.

The emerging ecosystem suggests demand for models that can run securely under customer control, wherever sensitive data resides. These are “living systems,” says Horev. “It’s not just leveraging a feature inside of a wider platform.” 

Visit the VAST Data Confidential AI solution page to learn more about the architecture, ecosystem, and availability.

The post How confidential AI splits control between data and model owners — and opens new opportunities for both appeared first on The New Stack.

Salesforce introduced its Salesforce Enterprise AI Harness on Thursday as a formalized amalgamation of the AI harness concepts and infrastructure the company has been working to align.

The organization said that “no single system has the complete answer” to complete a straightforward business task, such as completing a customer order; i.e., CRM knows the customer, ERP knows the inventory, FSM (field service management) knows the delivery, and the support team processes… and so on. 

As such, a form of AI leakage pervades throughout modern enterprises, where individual agents and their harnesses do their best to enact automation intelligence, albeit in comparatively siloed chunks.

Harnessing a six-pack of toolsets

Salesforce’s answer is to coalesce what it calls “six trusted capabilities” (from its own platform toolset collection) alongside a new AI control plane, built to underpin an open and composable AI ecosystem.

The Salesforce Enterprise AI Harness encompasses core technologies across Data 360 (a unified customer data platform tool), Informatica (data integration and governance), MuleSoft and Agent Fabric (API connectivity and multi-agent orchestration), Tableau (visual business analytics), Agentforce (an agent platform), Salesforce Guardian (security and compliance), and the Salesforce platform itself through a common, composable architecture and unified experience.

“The Agentic Enterprise won’t be defined by which model a company chooses. Models will continue to change, and intelligence will increasingly be available everywhere. What will differentiate an enterprise is the trusted, proprietary context it brings to that intelligence — starting with the customer — and its ability to securely turn that context into action,” said Rohan Kumar, Salesforce president & chief platform and engineering officer, during press briefing.

“The Agentic Enterprise won’t be defined by which model a company chooses… what will differentiate an enterprise is the trusted, proprietary context it brings to that intelligence.”

This is not Salesforce’s first-ever harness

To be clear, it hasn’t taken Salesforce until late 2026 to ever produce or work with a harness; subsystems within the six pack, such as Agentforce Vibes (a natural language vibe coding tool), make use of specialized execution harnesses, including Mastra and the Claude Agent SDK, to manage local agent execution loops. This is — as suggested — a more formalized, total platform-wide development.

Alongside the six-way alignment spanning context, agency, action, governance, security, and models, Salesforce is offering a new AI control plane to give developers a place to view, manage, and control agents. The company confirms that software engineers can “use the six together as one system or take only what they need,” and create deployments with Salesforce technology, other third-party existing technology, or both.

The big question here is simple: Is this cosmetic packaging designed to disseminate wider Salesforce DNA into software developers’ production environments, or is it a genuinely useful simplification and unification process that will be met with interest and perhaps even gratitude?

Working engineers commenting on sites including the G2 developer forum and B2B software review portal have provided some insight.

What developers and operations professionals think of the Salesforce stack

Commenting on the use of Agentforce as a standalone tool, operations associate Ashish B. noted in August this year, “One area that could be improved is the initial setup and configuration process. Building effective agents can take some customization and a solid understanding of the workflow. The platform would be even better with simpler configuration options and clearer, more straightforward guidance on setting up agents for specific business use cases.”

Salesforce may have been listening. It said the Enterprise AI Harness connects reasoning to business rules, policies, and controls required for predictable execution. It then makes those capabilities reusable across the enterprise so that context can be shared across agents and models. This means actions and workflows can be securely invoked wherever they’re needed, and governance and security can be applied consistently as AI moves across the business.

“The platform would be even better with simpler configuration options and clearer, more straightforward guidance on setting up agents for specific business use cases.”

Writing about the Informatica user experience on Gartner Peer Insights in March of this year, a DevOps engineer said that the platform “works well” for integrating multiple data sources and supports both batch and real-time processing. But they caution, “Debugging and monitoring pipelines can be difficult in complex workflows. Initial setup is challenging for new users and requires some learning curve. [The] User Interface could be improved for better usability and faster navigation.”

Possibly taking into account such feedback, the new AI control plane that accompanies Enterprise AI Harness claims to give businesses a common place to see, manage, and control agents and AI across the enterprise.

“It enables companies to discover and register agents and AI capabilities, establish identity and policy, manage lifecycle, evaluate performance, observe behavior and outcomes, and control cost — across Salesforce and third-party AI. This gives enterprises a consistent layer of visibility and control as AI expands across teams, applications, models, and systems — without requiring every agent or AI experience to be managed separately,” pledged Salesforce.

Six pillars of trust

The whole premise of this Enterprise AI Harness hinges around what Salesforce calls six trusted capabilities. 

Trusted Context combines customer context with data, metadata, semantics, knowledge, real-time signals, memory, and an understanding of how work gets done across the enterprise. Trusted Agency gives agents reasoning, planning, state, memory, and orchestration functions, combining flexible AI reasoning and deterministic controls where certainty is required. Trusted Action securely connects AI to applications, APIs, workflows, tools, and business processes. 

As its name suggests, Trusted Governance governs the data, metadata, policies, and processes that AI relies on, with lineage, quality, guardrails, and controls. Trusted Security applies identity, permissions, privacy, data protection, and runtime security to what AI can access and what agents can do. Trusted Models provides security with intelligent model routing based on accuracy, performance, cost, and requirements. 

Integrations with Claude, Slack, Teams, etc.

The Enterprise AI Harness is being built headlessly from the ground up, with capabilities accessible through technologies including MCP, APIs, skills, and plug-ins. The company said this will let Salesforce capabilities extend beyond traditional Salesforce applications, into services such as Claude, Slack, and Microsoft Teams.

Many of the technologies that form the foundation of Salesforce’s Trusted Enterprise AI Harness are available today, with new capabilities and the unified experience planned to begin rolling out in early fiscal year 2028.

The post “Six tools, one harness”: Salesforce loops together a six-pack of favorites appeared first on The New Stack.

Working with large datasets can lead to slow queries and out-of-memory errors. Polars, an open-source library that developers and data analysts use to clean, combine, and analyze tables of data, promises to ease both problems in its upcoming 2.0 release. But the first release candidate, out last week, comes with a catch: The new default can change the order of returned rows, potentially affecting code that depends on that order.

In announcing the first release candidate for Polars 2.0, the company says that calling collect on any LazyFrame query will now default to the streaming engine. Per Polars, users can expect “massive memory and performance improvements on most queries,” with the streaming engine expected to be “easily 5x faster” in aggregate.

But they need to keep an eye out for changes in row order. 

Move fast — and maybe re-order things? 

Improved memory usage and performance are obvious upgrades for Polars users who rely on the library for data processing and analysis, and it’s the streaming engine that’s bringing it. 

“Streaming engine doesn’t guarantee row-order by default for certain operations.”

With streaming, Polars says it can execute lazy queries in batches, rather than processing all data at once. This way, users can process datasets that don’t fit into available memory. 

But changing how those queries execute could potentially lead to trouble down the line, as the streaming engine can also change the order in which rows are returned. 

As Polars explains, the “streaming engine doesn’t guarantee row-order by default for certain operations.” That includes operations such as join, group_by, and unpivot. 

In its Version 2.0-rc user guide, the company explicitly calls out the migration hazard and underscores its risk in a red “danger” box, acknowledging that the change “may silently impact the results of your pipelines.”

For users whose code expects rows to appear in a certain order, that could create more problems for downstream processes. 

You can enforce row order, but there’s a chance it may cost you some speed

All is not lost, though. If users are working with code that depends on incidental ordering or observable row order, Polars offers guidance on mitigating the migration risk that comes with the new default. 

The change “may silently impact the results of your pipelines.”

There are two main options: Sort explicitly or set maintain_order=True where applicable.

Alternatively, users can keep the in-memory engine as default by setting the engine affinity. 

What else is coming in Polars 2.0 

Making all LazyFrame queries default to the streaming engine isn’t the only change users can expect from Polars 2.0. Per the announcement, the biggest changes in the upcoming release are improved defaults (the streaming engine being the most significant) and a better API. 

In the pre-release post, Polars explains that 2.0 also removes many ambiguous casts.

For example, it directs users to use .str.to_date()/.str.to_datetime() to parse strings to temporal data types. This way, Polars says users get “one obvious way to parse data.” More examples of improvements to strictness are in the migration guide. 

Why the pre-release before the upcoming Polars 2.0? Because Polars says it “[doesn’t] gate new features” and prefers to ship them as soon as they’re ready.

That said, the company assured users there’s more to look forward to for 2.x, hinting at a new IO-plugin design, a faster S3 reader, a cost-based planner, join reordering, and big SQL coverage improvements, among others.

For developers exploring the release candidate now, the takeaway is clear: Better memory and performance are worth getting excited about, but don’t forget to watch that row order.

The post Polars 2.0 pre-release comes with a 5x speed boost — but it could change row order appeared first on The New Stack.

On Monday, Google launched TimesFM-3, a 330-million-parameter time-series forecasting model trained on over a trillion real-world and synthetic data time points.

The new model is now available on Hugging Face under a non-commercial license.

Large language models are great at predicting the next word. For businesses, time-series forecasting models essentially try to do the same thing, but for data. Over the last few years, there’s been a lot of work in building better forecasting models. Last year saw the launch of models like Chronos-2 from Amazon and Moirai 2.0 from Salesforce, while more recently, Datadog launched its Toto 2.0 model.

These models represent a relatively new breed of forecasting models, as they can ingest multiple time series. As Google research scientists Ayush Jain and Rajat Sen explain in the announcement, “most real-world forecasting problems are inherently multivariate: where multiple time series and auxiliary external features jointly impact the future forecast of a time series.”

Past sales, they explain, only tell part of the story. “A good forecast should also draw on sales of related products (e.g., ice cream cones, syrups), historical foot traffic, and known future events like weather forecasts, promotions, and holidays.”

TimesFM-3 is Google’s first model that was natively pre-trained to handle multiple time series and to do so with zero-shot generalization. This also allows it to forecast multiple related time series in parallel and to include historical data, such as past foot traffic.

In the benchmarks Google shared, TimesFM-3 outperforms all of these, often by a significant margin. The team looked at Salesforce’s Gift-Eval, Amazon/AutoGluon’s FEV-Bench, and Time.

What’s maybe the most surprising here is that TimesFM-2.5, which was state-of-the-art when it launched in September 2025, is now at the bottom of the benchmarks. That’s how fast this field is developing.

The architecture

Like its predecessors, TimesFM-3 is a decoder-only transformer that chops each time series into patches of 32 data points and treats them roughly the way a language model treats tokens.

What’s new is that these tokens now flow through two alternating kinds of attention layers. The first one looks backward across time within a single series and keeps things strictly causal, so the model can’t see values it shouldn’t know yet. The other looks sideways across all series at a given moment, which is how a promotion in one product line, for example, can inform the forecast for another.

Decoding changed, too. Earlier versions generated forecasts one patch at a time, Google’s researchers explain, which added latency and compounded errors along the way.

Instead, TimesFM-3 appends masked placeholder tokens for the entire forecast horizon and then fills them all in with a single forward pass.

The non-commercial license

Google decided to launch the new model under a non-commercial license. That’s becoming a bit of a trend in the world of model builders.

TimesFM-2.5 still shipped with the Apache 2.0 license — as do Toto 2.0 and Chronos-2.

The TimesFM-3 source code is still under the Apache license. Still, Google notes that “for the time being, TimesFM 3.0 pretrained weights are distributed under the separate timesfm-non-commercial-license-v1.0 license and are restricted to non-commercial, non-production use. Commercial or production use of the default pretrained weights is not permitted.”

Google will soon replace TimesFM-2.5 as the model that powers BitQuery’s AI.FORECAST command, so the company is actively monetizing these models.

That’s not unusual, of course. Every player in this market already integrates its forecasting models into its own platform, but Google restricting the state-of-the-art weights while also opening a paid path through its data warehouse is a pretty clear signal of where these labs think the money is in the long run.

The post Google’s new forecasting model beats everyone. You can’t use it at work (yet). appeared first on The New Stack.

Observability is entering a new phase now that OpenTelemetry has standardized instrumentation for data collection. Unfortunately, the observability industry still lacks a cost-effective way to store, retain, search, and analyze full-fidelity telemetry data. This results in blind spots in observability and many teams operating without full operational visibility.

As AI systems generate more logs, traces, and metrics — thereby making the blind spots issue worse — Bronto, a Dublin, Ireland, firm offering an intelligent data observability platform, is betting that the next observability platform battle will be won at the data layer, not the dashboard layer.

Bolt-ons and incremental efficiency aren’t enough

Trevor Parsons, co-founder and co-CEO of Bronto, tells The New Stack that the industry has been optimizing at the edges rather than rebuilding the economics and architecture of telemetry storage. The industry has introduced a wide array of “hacks” and “capabilities” to avoid tackling this issue head-on and ultimately to protect their margins. 

“If you are a couple of times cheaper or 50% cheaper, that ain’t going to cut it,” Parsons says, because data volumes, especially AI telemetry, are growing so quickly, on top of already stretched observability budgets and inefficient datastores. 

Promises, Promises, Promises…

Parsons elaborates, “Observability has always and continues to have a data problem.”

The eternal promise of observability has been delivering teams a clearer view of what’s happening inside their systems.

“Observability has always and continues to have a data problem.”

But in practice, that view is often incomplete, expensive, and short-lived. For too many teams, observability has become less about asking better questions and more about fighting the cost and complexity of storing the data they already need. 

“Sometimes people frame that as a cost problem, where they’ll say observability is up to 20 or 30% of your infrastructure spend,” Parsons says. “I actually think this minimizes the issue; it’s much bigger than that. Teams are actually paying 10, 20, 30% of their infrastructure spend for access to only a sliver of their data.” 

Noel Ruane, co-founder and co-CEO of Bronto, frames the challenge that organizations face and tells The New Stack, “Agents and applications are generating more logs, traces, and metrics each day. The software landscape has accelerated, but are observability vendors keeping pace? No, they’re offering workarounds, bolted-on features, and asking teams to accept blind spots.” In short, Ruane says, they’ve failed to solve the data problem.

Out with the old observability model 

“Customers are not getting access to all of their observability data, Parsons explains. “They have to cut their retention from 30 days to seven days to three days. They have to sample data. They have to rehydrate data.”

In other words, today’s tools make customers choose which parts of their own data they’re allowed to see, and you may only get to see it for a short amount of time.” 

“The solutions that are being put in front of customers to give them their data are always full of compromises, forcing teams to choose between cost, coverage, and speed of data access. The burden is always put on the customer by vendors.”

“The solutions that are being put in front of customers to give them their data are always full of compromises, forcing teams to choose between cost, coverage, and speed of data access,” Parsons says. “The burden is always put on the customer by vendors.

“But really this should be the other way around; it’s the vendors’ job to innovate on behalf of the customer” 

OpenTelemetry: Collection solved, storage problem exposed

Severin Neumann, head of community at Bronto, tells The New Stack that OpenTelemetry has helped standardize instrumentation and data collection, while reducing reliance on proprietary agents.

But that success has created a new bottleneck, says Neumann, who is also an OpenTelemetry maintainer and member of the OpenTelemetry governance committee. Now that organizations can collect more telemetry, they need somewhere affordable and useful to put it.

“We have fixed the instrumentation problem,” Neumann says. He cautions that enterprises now need ways to handle all this data. And if enterprises can’t store it and instead throw away large parts of it, humans and agents can not make sense of it.

The observability business model doesn’t align with customer value

The legacy observability tool business model charges customers for data storage, rather than the value teams get from their data, Parsons says. Customers tell him the same thing constantly: “I pay the same price even if I never search my data.” In many cases, customers find existing tools difficult to use and feel that their observability solution is just a really expensive data store that they do not get a lot of value from.”‘

Noel Ruane assessed the market by saying, “Traditional vendors like Datadog know their pricing model isn’t sustainable. They’ve introduced defensive features like ‘Flex Logs’ and a new ClickHouse partnership to try to keep customers from jumping ship, but they’ve only added new complexity for their customers.” 

Especially in the AI era, Ruane adds, the traditional business model charges teams in ways that discourage them from capitalizing on their data. Customers should pay much less for data that sits idle and more when they actually derive value from it with queries and analysis.

Bronto’s technical differentiation

Bronto isn’t selling another observability dashboard. It argues that observability is a storage problem before it’s a visualization problem, and that’s where the company went.

Underneath the platform is a custom-built polymorphic data store called BrontoDB, specifically designed for observability data. The pitch: enterprises can keep more than 100 times the observability data they hold now, and it won’t get slower or harder to use.

Why that matters comes down to how the three signals break. Metrics, logs, and traces each hit a wall at different points, and Bronto says it built BrontoDB to tackle these issues head-on. Parsons is blunt about two of them.

“With metrics, we’ve solved the high cardinality problem where costs traditionally explode with high cardinality metrics,” Parsons says. “With logging, we’ve solved the indexing problem where there was always a trade-off between fast logs and paying through the nose for it or having slow logs and getting them slightly cheaper.”

• High cardinality is what wrecks metrics pricing. Add enough unique dimensions and the bill lands somewhere nobody forecast. Bronto says it was built specifically to take that surprise out.
• Logs have always been pick-your-poison: fast and expensive, or cheap and slow. Bronto says that choice goes away — sub-second search across petabytes, no shortened retention windows, no rehydrating cold data, no waiting.
• Traces, Bronto argues, shouldn’t be sampled at all. Sampling exists because tools and pricing models couldn’t handle the full stream. Bronto says teams can send it all.

Billing works differently, too. Most vendors charge for data sitting in storage, whether anyone touches it or not. Bronto charges closer to what teams actually search and analyze. That’s the piece that has to hold up if full-fidelity observability is going to be affordable at AI scale.

AI is what raises the stakes, Parsons says. AI systems are non-deterministic and trace-heavy. They throw off more telemetry, and the data has to stick around longer if you want to debug effectively. 

He points to an upside as well. As operations become more automated, telemetry data becomes more useful because agents can chew through volumes of history that no SRE would ever read manually.

AI raises both the volume and the stakes, according to Parsons. AI systems create more telemetry because they are non-deterministic, trace-heavy, and require longer retention for troubleshooting. At the same time, AI-enabled operations will make historical telemetry more valuable because agents can analyze far more data than human SRE teams could manually inspect.

“If AI is the intersection of where data meets intelligence, you can not apply intelligence if you do not have the data.”

“If AI is the intersection of where data meets intelligence, you can not apply intelligence if you do not have the data,” Parsons says.

The next observability battle 

AI is unlikely to fix observability’s data problem. In fact, it will produce more telemetry, create more edge cases, and increase the cost of missing the right signal at the wrong time.

For Bronto, the data layer is the next major battleground. Dashboards still matter, but in an AI-heavy production environment, the more important question may be whether teams have access to all their data for as long as they need so that they can apply AI to it. 

The post Observability has a data problem. AI is about to make it worse. appeared first on The New Stack.