Container security controls often fail less because organizations lack standards, scanners, or best practices, and more. After all, every service tends to use its own Dockerfile, building images in slightly different ways.

Buildpacks are often pitched as a way to make containerization more developer-friendly, standardized, and maintainable by eliminating the Dockerfile pain points. They can also help improve security posture in practice, especially in larger, multi-team environments.

In this article, we will explore how Buildpacks provide a standard application-image path where platform teams can centrally govern build inputs, runtime images, Software Bill of Materials (SBOM) production, artifact metadata, and update workflows–making container controls easier to enforce and operate at scale.

The patch that never reached production

What does a patch integration process look like in the real world?

In more than one enterprise, I’ve seen the story look roughly like this: a critical vulnerability is fixed in the organization’s approved runtime image. In a perfect world, a patched base image is detected centrally, rebuilt, tested, published, and automatically propagated to every affected application. But we do not live in a perfect world. A week later, some production workloads still run on the vulnerable base.

“A container security control isn’t useful on its own. It must be applied consistently, observable in the running estate, and maintainable when images, dependencies, and vulnerabilities change.”

Typical reasons include:

• Inconsistent Dockerfiles using different base image tags, Linux versions, and runtime versions, making it harder to determine which service is affected;
• Application images may start from a patched base but reinstall vulnerable dependencies because developers manually install packages;
• Uneven rebuild cadences: some teams rebuild more frequently, others only rebuild when the application code changes, so a service with no recent feature work may run an old vulnerable image for months;
• No centralized inventory, SBOMs, image metadata, or deployment tracking, so organizations cannot reliably determine which applications remain vulnerable or measure patch propagation. 

This story has not yet found its happy ending because the organization is only halfway into establishing a vulnerability management process. A container security control in place, such as “all services must use approved patched base images,” isn’t useful on its own. It must be applied consistently, observable in the running estate, and maintainable when images, dependencies, and vulnerabilities change.

That’s the stage where many teams fall short.

Why container security controls drift

Container security controls are hard to implement largely because of established containerization practices. In many organizations, no single governed build path exists. Each repository produces its own image, usually through a Dockerfile maintained by the application team.

Dockerfiles offer flexibility. They also push many security decisions onto developers: which base images to use, which packages to install, how to configure the runtime user, how minimal the image should be, how to track patches, which CI policies to apply. Over time, as services multiply and teams change, these choices drift. It’s not unusual to see different repositories using different base images, update cycles, and even different interpretations of what “secure” means in practice.

“The problem is expecting every developer to have enough container expertise to do this consistently across hundreds of repositories.”

Dockerfiles are not the problem by themselves. A well-written Dockerfile can produce a minimal, hardened image. The problem is expecting every developer to have enough container expertise to do this consistently across hundreds of repositories.

This approach also makes patch propagation unreliable. Platform teams may publish a patched base image, but each application team must still notice the update, modify its Dockerfile, rebuild, test, and redeploy. Some do it quickly. Others do it late or not at all. The control exists, but adoption remains uneven.

Buildpacks help close this gap by moving common containerization decisions out of individual repositories and into a shared, governed build path.

The shift from repository-specific builds to a governed build platform

Buildpacks turn application source code into a production-ready OCI container image without a Dockerfile. They detect the application type, select the required buildpacks, provide the necessary runtime and dependencies, and produce a runnable image.

For container security, the main benefit is not simply removing Dockerfiles. Buildpacks tend to provide a standard way to construct images, at least for typical application stacks.

“Buildpacks turn application source code into a production-ready OCI container image without a Dockerfile.”

This gives platform and security teams a central point for enforcing controls. Instead of asking every team to choose an approved base image, configure the runtime, manage layers, generate metadata, and track updates, the organization can encode much of this work in shared builders and buildpacks. Developers own their code, dependencies, and service behavior. Producing a compliant image becomes a platform responsibility.

Because Cloud Native Buildpacks is a CNCF graduated project, its specifications and reference implementations undergo rigorous community review and long-term maintenance, making them suitable as the foundation for enterprise security controls.

With that foundation in place, applying and maintaining specific container security controls becomes easier at scale.

Four container security controls buildpacks make it easier to operate

Standardization of build inputs

The first control is standardizing what goes into the build.

In Cloud Native Buildpacks, the key unit is the builder. A builder packages the buildpacks, lifecycle, build-time base image, and runtime base image used to create the final application image. This establishes the builder as a controlled definition of how application images are produced. Developers do not choose a random base image on Docker Hub to build their applications; the Buildpacks ecosystem defines a set of build and run images.

This builder-based approach introduces an important concept. A developer cannot change the base OS layer in a builder with a single line of code because compatibility isn’t guaranteed. With one line of code, however, they can swap builders and still get a compatible, working image.  

This is where standardization takes place. Developers keep using their preferred languages and frameworks, but the platform team builds images from a controlled set of approved builders. Extension paths for cases where buildpacks require customization can also be standardized.

“By controlling the builder, the organization also controls the buildpacks, runtime image family, lifecycle version, and build paths allowed in CI/CD.”

The security benefit is straightforward. By controlling the builder, the organization also controls the buildpacks, runtime image family, lifecycle version, and build paths allowed in CI/CD. The policy is defined once at the platform level and then applied across many services.

Best container security practices by default

After the builder standardization, the next question is: what does the resulting image look like? This is where buildpacks help. They improve the output by applying several container security practices as part of the normal image creation flow:

• Non-root build and execution. Cloud Native Buildpacks require buildpack code to run as a non-root user. Platforms, such as pack, also produce images configured to run applications as the non-root user defined by the run image. Non-root defaults reduce the blast radius of a compromise and make container escape, filesystem tampering, and exploitation of the image build process more difficult.
• Separation of build and runtime environments. Buildpacks mark layers as build-only or launch-time. Only launch layers are included in the final image, so compilers, npm tooling, build caches, and similar tools remain outside production, reducing the attack surface.
• Restricted modifications of the base image. Because buildpacks run without root privileges, they cannot simply install OS packages or modify the base filesystem. OS changes must be provided through the controlled build/run images or image extensions.
• Isolation of sensitive build privileges. When using an untrusted builder, sensitive lifecycle phases can run separately from the phases that execute buildpack code. This prevents an untrusted buildpack from receiving capabilities such as registry credentials and access to the container daemon.

Some builder providers offer additional security features. For example, Paketo buildpacks for Spring Boot provide base images without a shell. Another example is BellSoft’s hardened builder for Paketo buildpacks based on BellSoft Hardened Images.  

These defaults reduce the manual effort and make secure container builds part of a standard process.

SBOM Generation

A Software Bill of Materials (SBOM) lists all software components in an application or image. An SBOM is required for vulnerability tracking, license reviews, and audits, and many regulations require it.

Teams often add SBOM generation as a separate CI step, with separate tools, formats, storage rules, and ownership. That makes coverage uneven, especially across many repositories.

Cloud Native Buildpacks reduce this friction by emitting an SBOM for all dependencies they provide, in formats such as CycloneDX, SPDX, or Syft JSON. As a result, platform and security teams get consistent image inventory data without requiring every application team to build its own SBOM process.

For a typical Java service, that might mean SBOM entries for the JRE version, Spring Boot version, and key libraries pulled in at build time. For Node.js services, it might include the Node runtime version and major npm dependencies. The exact content will vary, but the pattern is the same: SBOM data comes from the build itself, not a separate, manually maintained process.

Patching at scale

As discussed earlier, the hard part is often not writing or receiving a patch, but getting it into the running workloads. Buildpacks shorten that path by using shared builders, buildpacks, and run images. Platform teams can then update these common inputs once, instead of waiting for every application team to repeat the same change.

Another powerful feature of Buildpacks that promotes rapid patching is rebasing. When OS-level fixes become available, the runtime base layers can be replaced with layers from a newer run image without rebuilding the application from source.

Kubernetes tools such as kpack can automate this process. They track image resources and trigger rebuilds when the source, builder, buildpacks, or stack changes.

Rebasing has limits: it updates only the run-image layers. Dependencies added by buildpacks, such as a JRE or Node.js runtime, usually require a rebuild with updated buildpacks or dependency versions. For a Node.js service, a rebuild might mean picking up a new Node.js runtime version and updated npm packages, not just a newer OS base.

So, we are not talking about magical automatic patching here. The key benefit is a more centralized, repeatable patch-propagation path across the image estate—though teams still need to handle testing, rollouts, and exceptions.

The new patch path after buildpacks

Let’s return to the original story. We left the organization in a state where vulnerability management lacked consistent patch integration. Suppose the enterprise has migrated their workflows to Buildpacks — how has the process changed?

Once again, a critical vulnerability is discovered. But after adopting buildpacks, the response looks different: the remediation starts with shared build inputs.

The platform team updates the approved run image, builder, or buildpacks. Images are then rebased or rebuilt:

• Rebase when the fix affects the run-image OS layers;
• Rebuild when the affected component is runtime, like a JRE or Node.js runtime, or application dependencies.

Of course, Buildpacks do not remove the need for testing or deployment controls. Application teams still own their code, dependencies, and compatibility testing. SRE teams still promote, deploy, monitor, and, when necessary, roll back the patched images.

“Buildpacks give organizations one controlled way to build container images instead of leaving every repository to define its own process.”

The most important change is the patch path. Platform teams maintain the shared build inputs and automation. Security defines scan policies and exception rules. Compliance defines the evidence that must be retained.

This path also promotes shifting security left, as it becomes easier for developers to meet the in-house security requirements and container security best practices.

Function	Responsibility
Platform	Builders, run images, buildpacks, and rebuild automation
Security	Vulnerability policy and exceptions
Application teams	Code, dependencies, and compatibility testing
SRE	Promotion, rollout, monitoring, and rollback
Compliance	Audit evidence and retention

Some workloads still need custom images. But you don’t need to choose between Buildpacks and Dockerfiles; you can use both. When required, you can create a custom buildpack and extend the build-time base image with a Dockerfile. The result is not a rigid “Buildpacks only” model, but a governed image strategy with controlled customizations. 

Buildpacks give organizations one controlled way to build container images instead of leaving every repository to define its own process. This makes security controls–such as approved images, safe defaults, SBOMs, and patching–easier to scale. The result is reduced drift, clearer ownership, and faster updates.

If you’d like to get started with Buildpacks, try them on one service and compare the workflow with your current Dockerfile process.

The post How buildpacks help enterprises finally operate container security controls at scale appeared first on The New Stack.

It’s 2 a.m., and you’re restoring a PostgreSQL cluster from last night’s backup. Its data directory lives on one PersistentVolumeClaim and its write-ahead log on another, a common split for I/O isolation. Every volume snapshot reported success. The pods come back. Then Postgres refuses to start because the WAL on one volume references pages that were never captured in the data files on the other. The backup wasn’t corrupted in transit. It was inconsistent the moment it was taken.

If you run stateful workloads on Kubernetes, this failure mode has been silently lurking in your backups for years. It has nothing to do with your backup tool crashing, and everything to do with a guarantee you gave up when you moved off traditional storage.

The consistency group you lost on the way to Kubernetes

Enterprise storage arrays solved this problem decades ago with a feature called a consistency group. You told the array which LUNs belonged to the same application, and when you snapshotted the group, the array froze them all at the same instant. Every volume captured the same point in time. Restores were coherent by construction.

“The backup wasn’t corrupted in transit. It was inconsistent the moment it was taken.”

That guarantee didn’t survive the move to cloud native. The Container Storage Interface (CSI) standardized snapshots around a single object, the VolumeSnapshot, scoped to a single PersistentVolumeClaim (PVC). One PVC, one snapshot. For a stateless service with one volume, that model is fine. For anything that spreads its state across multiple volumes (a database with separate data and log disks, a sharded datastore, most real applications), the per-PVC model can’t tell which volumes belong together.

As teams migrated off proprietary SANs and virtualization stacks onto Kubernetes-native storage, they gained enormous flexibility but lost the consistency group. Most never notice since the gap only shows up at restore time after an incident, when it is far too late to do anything about it.

How individual PVC snapshots break

When protecting a multi-volume application, backup tools enumerate the PVCs and issue a VolumeSnapshot for each one, in sequence. Snapshot volume A, volume B, then volume C.

Each snapshot is individually crash-consistent, equivalent to pulling the power cord on that one volume. But they are not consistent with one another. Between snapshotting A and snapshotting B, the application keeps writing. A transaction can land in the log on volume B that references data never captured on volume A, because A was frozen a few hundred milliseconds earlier. The busier the application and the more volumes involved, the wider the inconsistency window.

“The result is a set of snapshots that each looks healthy and collectively describes a state that never existed.”

The result is a set of snapshots that each looks healthy and collectively describes a state that never existed. You can quiesce the application to close the window (freeze I/O, flush buffers, snapshot, unfreeze), but at production scale, freezing a busy database for the duration of a multi-volume snapshot is exactly the disruption backups are supposed to avoid.

VolumeGroupSnapshot: consistency groups as a Kubernetes API

VolumeGroupSnapshot is the missing primitive, and as of Kubernetes v1.36 (May 2026), it is generally available. It brings the consistency group back as a first-class, vendor-neutral Kubernetes API rather than a proprietary array feature.

The model has three objects. A VolumeGroupSnapshotClass, defined by an administrator, describes how group snapshots are created for a given CSI driver. A VolumeGroupSnapshot is the user’s request, and it carries a label selector that picks out every PVC belonging to the application. A VolumeGroupSnapshotContent tracks the provisioned result. Under the hood, the CSI driver takes one atomic, point-in-time snapshot across every selected volume: a real consistency group, with no application quiescence required, provided the underlying storage supports it.

“Under the hood, the CSI driver takes one atomic, point-in-time snapshot across every selected volume.”

The label selector is the important design choice. You don’t enumerate volumes; you describe them. A selector like `app=postgres` picks up the data and logs PVCs together, and the group boundary is expressed in Kubernetes terms that survive adding or resizing volumes over time.

Wiring it into backup: what changes

An API that produces consistent snapshots only helps if your backup workflow uses it. When I implemented VolumeGroupSnapshot support in Velero, the CNCF project that has become the de facto standard for Kubernetes backup and restore, the core change was replacing “iterate over PVCs and snapshot each” with “group the PVCs that belong together, snapshot the group as one operation, then track the per-volume members for restore.”

That last part matters. A group snapshot fans back out into individual volume snapshots, one per member, so restore still rehydrates each PVC independently, but now every member shares a single point in time. Velero was among the first backup projects to build directly on the upstream VolumeGroupSnapshot API, rather than a proprietary grouping scheme, so the consistency guarantee rides on a standard the whole ecosystem shares instead of a format locked to one tool.

Individual snapshots vs. group snapshots: which to use

This is not a wholesale replacement. Individual PVC snapshots remain the right tool for single-volume workloads and for volumes that are genuinely independent, since snapshotting those as a group buys you nothing and adds coordination overhead. Reach for VolumeGroupSnapshot when correctness depends on multiple volumes sharing a point in time.

A quick decision guide:

• One volume, or several fully independent volumes: individual VolumeSnapshots.
• Multiple volumes with cross-volume write ordering (data plus WAL, data plus index): VolumeGroupSnapshot.
• Unsure whether a skewed or partial restore would corrupt the application? Treat it as a group.

Day 2 notes

A few things to check before relying on this in production. Group snapshot support is per CSI driver: the API is standard, but the driver has to implement it, and adoption is still spreading. A growing set of CSI drivers implement it (Ceph CSI among them), so check the driver’s release notes for upstream VolumeGroupSnapshot support, and create the VolumeGroupSnapshotClass before needing it. The atomicity guarantee is only as strong as the storage backend behind the driver, so validate it by restoring, not by reading success statuses. Audit existing backups now: by protecting multi-volume applications with per-PVC snapshots today, you likely have inconsistent restore points that have never been tested under a real failure.

“With VolumeGroupSnapshot now GA and backup tooling adopting it upstream, multi-volume backups on Kubernetes are finally consistent by construction, not by luck.”

The broader arc is that Kubernetes storage is catching up to what enterprise arrays offered for years, but as an open standard rather than a capability locked to one vendor’s hardware. Consistency groups were one of the last missing pieces. With VolumeGroupSnapshot now GA and backup tooling adopting it upstream, multi-volume backups on Kubernetes are finally consistent by construction, not by luck.

The post Kubernetes 1.36 restores a lost guarantee for database backups appeared first on The New Stack.