====== Storage and Compute Layer Separation ======
**Storage and Compute Layer Separation** is an architectural pattern that decouples persistent data storage infrastructure from transactional compute resources in modern database systems. This design paradigm enables independent scaling, resource optimization, and operational flexibility while introducing specialized challenges in encryption management and data [[consistency|consistency]] coordination.

===== Architectural Overview =====
Storage and compute layer separation represents a fundamental shift from monolithic database architectures where storage and processing occur within a single system. In this pattern, storage responsibilities are delegated to dedicated services optimized for [[durability|durability]] and availability, while compute layers—such as PostgreSQL instances—handle transactional processing and query execution independently(([[https://www.databricks.com/blog/take-control-customer-managed-keys-lakebase-postgres|Databricks - Storage and Compute Layer Separation in Lakebase (2026]]))

The separation creates distinct operational concerns. Storage layers typically consist of specialized components like [[pageserver|Pageserver]] (responsible for page management and durability) and Safekeeper (handling write-ahead logging and durability guarantees), while compute layers run PostgreSQL-compatible instances that execute queries and maintain transaction semantics(([[https://www.databricks.com/blog/take-control-customer-managed-keys-lakebase-postgres|Databricks - Storage and Compute Layer Separation in Lakebase (2026]])).

This architectural decision draws from established patterns in cloud-native systems, where separation of concerns enables horizontal scaling of either component without resource constraints in the other. The pattern has become increasingly prevalent in serverless and multi-tenant database implementations where workload variability and cost optimization are primary objectives.

===== Elastic Scaling and Serverless Operations =====
A primary advantage of storage-compute separation is the ability to scale each layer independently according to actual demands. Compute instances can be provisioned, deprovisioned, or resized based on query workload patterns without affecting storage infrastructure, while storage capacity can expand to accommodate data growth independent of computational requirements(([[https://www.databricks.com/blog/take-control-customer-managed-keys-lakebase-postgres|Databricks - Storage and Compute Layer Separation in Lakebase (2026]]))

Serverless database operations become feasible through this separation. Compute instances can be paused during idle periods or spun up on-demand, while persistent storage remains available and consistent. This capability enables cost-proportional billing models where customers pay for actual compute utilization rather than provisioned capacity.

The separation also facilitates multi-tenant deployments where multiple compute instances can access shared storage infrastructure, amortizing storage costs across workloads while maintaining computational [[isolation|isolation]]. Query caching at the compute layer can operate independently from storage caching mechanisms, allowing optimization strategies tailored to each layer's performance characteristics.

===== Encryption and Security Management =====
Storage-compute separation introduces complexity in encryption architecture, particularly when implementing customer-managed key (CMK) systems where organizations maintain control over encryption key material. Encryption management must be coordinated across multiple components: the storage layer (Pageserver, [[safekeeper|Safekeeper]]), compute instances running PostgreSQL, and intermediate caches at both layers(([[https://www.databricks.com/blog/take-control-customer-managed-keys-lakebase-postgres|Databricks - Storage and Compute Layer Separation in Lakebase (2026]]))

Implementing end-to-end encryption requires establishing secure key distribution mechanisms, ensuring that encryption keys are available to authorized compute instances while preventing unauthorized access to key material. Cache coherency becomes a security consideration—encrypted data may be cached at multiple layers, requiring consistent encryption policy enforcement across all cache tiers.

Key rotation procedures must account for data distributed across independent storage and compute systems. Cache invalidation policies ensure that stale encrypted data is not served when keys have been rotated or revoked. These operational requirements demand sophisticated key management infrastructure and careful coordination between layers to maintain security guarantees while preserving performance characteristics.

===== Data Consistency and Coordination =====
Maintaining transactional semantics and data consistency across separated layers requires robust coordination mechanisms. The storage layer must provide durability guarantees through mechanisms like write-ahead logging (handled by Safekeeper), while compute layers must maintain awareness of storage state to ensure consistent transaction processing(([[https://www.databricks.com/blog/take-control-customer-managed-keys-lakebase-postgres|Databricks - Storage and Compute Layer Separation in Lakebase (2026]]))

Pageserver acts as an intermediary that manages page versioning and consistency, ensuring that compute instances accessing pages retrieve versions consistent with transaction visibility rules. This requires versioning schemes that track logical time across the distributed system, often implemented through log sequence numbers (LSNs) or similar monotonically increasing identifiers.

Cache invalidation and coherency protocols must ensure that modifications made by one compute instance are properly reflected in the storage layer and visible to other compute instances respecting isolation level semantics. This coordination becomes particularly complex in multi-version [[concurrency_control|concurrency control]] (MVCC) implementations where different transactions may require visibility into different snapshots of data state.

===== Operational Implications =====
The separation pattern introduces new operational considerations beyond traditional monolithic database administration. Monitoring and observability must span multiple independent systems with different failure modes and performance characteristics. Storage layer degradation may not immediately impact compute availability, while compute instance failures do not affect data durability or availability to other compute instances.

Disaster recovery and business continuity procedures must account for independent failure scenarios at storage and compute layers. Backup and restore operations may operate differently when storage and compute are decoupled, potentially enabling point-in-time recovery at the storage layer independent of compute state.

Resource contention between multiple compute instances accessing shared storage requires careful throttling and priority queue management to prevent resource exhaustion. Query planning and optimization must consider storage layer latency and bandwidth constraints rather than assuming local data access patterns.


===== See Also =====

  * [[proprietary_vs_open_storage|Proprietary Storage Layers vs Open Table Formats]]
  * [[isolation|Isolation]]
  * [[row_vs_columnar_storage|Row-Oriented vs Columnar Storage]]
  * [[durability|Durability]]
  * [[row_oriented_storage|Row-Oriented Storage]]

===== References =====