The Databricks Key Manager Service is an internal encryption key management system developed by Databricks that handles the lifecycle of Key Encryption Keys (KEKs) and performs cryptographic wrap/unwrap operations within the Databricks platform. The service implements a security architecture that minimizes direct exposure to customer-managed encryption keys while maintaining the ability to protect data at rest across Databricks infrastructure.

The Databricks Key Manager Service functions as an intermediary encryption layer between Databricks infrastructure and customer Key Management Systems (KMS). Rather than directly accessing customer plaintext Customer Master Keys (CMKs), the service operates on pre-wrapped encryption keys that have been protected by the customer's external KMS. This design follows defense-in-depth principles, ensuring that Databricks infrastructure never maintains access to unencrypted CMKs ¹⁾.

The architecture separates key management responsibilities between the customer's KMS and Databricks' internal systems. Customers retain full control over their master encryption keys within their preferred KMS platform, while Databricks manages the operational aspects of key wrapping and unwrap operations required for day-to-day data protection activities.

The primary responsibilities of the Databricks Key Manager Service include managing Key Encryption Keys (KEKs) and executing wrap/unwrap cryptographic operations. The service stores wrapped keys—encryption keys that have been encrypted by the customer's KMS—and performs the necessary cryptographic transformations to enable data encryption and decryption operations across the platform.

A critical aspect of the service's operation is its interaction model with customer KMS systems. The service contacts the customer's KMS exclusively to perform unwrap operations when needed, retrieving plaintext keys for immediate use in cryptographic operations. Importantly, the Databricks infrastructure never stores, logs, or maintains persistence of unwrapped plaintext CMKs. This design ensures that even if Databricks systems are compromised, attackers cannot retrieve customer master encryption keys that remain protected by the customer's KMS ²⁾.

The Databricks Key Manager Service enables customer-managed encryption, giving customers direct control over their encryption keys while leveraging Databricks' infrastructure for key management operations. This model contrasts with fully Databricks-managed key scenarios, where encryption key generation and storage remain entirely within Databricks' control.

By implementing the Key Manager Service with customer-managed keys, organizations maintain the ability to revoke access to their data by disabling or deleting their CMKs in their KMS. Since Databricks cannot access plaintext keys without contacting the customer's KMS, disabling key access at the source immediately prevents further data access through Databricks systems. This capability provides customers with explicit control over data lifecycle and compliance with key management requirements.

The Databricks Key Manager Service integrates with Databricks LakeBase and other data platform components to provide transparent encryption for data at rest. The service operates seamlessly within Databricks' infrastructure without requiring application-level changes, as encryption and decryption operations occur automatically when data is written to or read from Databricks storage systems.

The service architecture supports compliance with regulatory frameworks that mandate encryption key separation and customer control over encryption materials. Organizations subject to standards such as HIPAA, GDPR, PCI-DSS, or SOC 2 can leverage customer-managed keys through the Databricks Key Manager Service to meet encryption and key management requirements while maintaining the operational efficiency of a managed platform.

The separation of key management between customer KMS systems and Databricks infrastructure introduces latency considerations for key unwrap operations. Each encryption or decryption operation requiring access to the plaintext key necessitates a call to the customer's KMS, which may introduce network latency depending on KMS availability and configuration. Organizations must ensure reliable connectivity to their KMS systems to prevent service interruptions.

Additionally, the service requires customers to maintain operational responsibility for their KMS systems. Key management operations depend on KMS availability, authentication credentials, and proper access control configuration. Misconfiguration of KMS access policies or credential management can prevent the Databricks Key Manager Service from functioning correctly.

• Databricks
• Customer Managed Keys (CMK)
• Key Wrapping and Unwrapping Operations
• Databricks Apps
• AWS Key Management Service (AWS KMS)