Master Encryption Key Rotation

Important Support Note: Rotating the master encryption key is a sensitive operation. While we have detailed the steps in this topic, we urge you to contact Support to work with a support representative before attempting any MEK rotation to ensure all pre-requisites have been met. Please provide at least a five-day notice in advance of when you want to conduct the rotation.

Overview

When Secret Server is first installed, a unique random AES256 Master Encryption Key (MEK) is generated and saved in a file, encryption.config. The MEK protects anything sensitive in Secret Server that is not associated with a specific secret, as well as each secret’s unique AES256 key when an HSM is not used.

For added security, you can rotate the MEK, re-encrypting protected data with the new key.

Geo-Replication, DPAPI, or an active secret key rotation must not be in effect when rotating the MEK.

MEK rotation fully supports using a Hardware Security Module (HSM) for Secret Server On-Premises or key management for Secret Server Cloud. This feature requires Enterprise Plus edition or the advanced encryption license. If you are using an HSM, disable it for your first MEK rotation. You can then re-enabled it, and you can run future MEK rotations without disabling it.

If the rotation button is disabled then DPAPI also needs to be disabled, otherwise it will cause the process to fail. The DPAPI needs to be disabled on all nodes.

Rotation Procedure

To perform a MEK rotation:

Important: For Secret Server on Premises, back up the current MEK file encryption.config in the application directory on each node.

Do not continue until you back up your encryption key. Failure to do so could cause extensive permanent data loss.

For Secret Server on Premises, if this is the first MEK rotation, also backup any encrypted session recording videos saved to disk. They are only updated during the very first MEK rotation. Failure to do so could cause extensive permanent data loss.
Important: For Secret Server on Premises, back up your database. See the first task on the Moving the Microsoft SQL Server Database to Another Machine topic for details.

Do not continue until you back up your database. Failure to do so could cause extensive permanent data loss.
Go to Admin > Configuration > Security. The Configuration page appears:
Click the Security tab.
Go to the Master Encryption Key Rotation section (not the Key Rotation section):
Click the Rotate Encryption Keys button. The Master Encryption Key Rotation popup appears:
Carefully read the text, especially for the check box itself.
When you are finished, click to select the check box to acknowledge having read it.
Click the Continue button. The server goes into maintenance mode to perform MEK rotation. Secrets and configuration settings cannot be updated while in maintenance mode. Secrets cannot be updated while in this mode. Processing time will vary, depending on the hardware, number of secrets, and HSM key size.
The rotation begins.
Secret Server updates the encryption.config file on the current server. The file now contains the new MEK, as well as the previous MEK, which is needed to re-encrypt all of the data. The encryption.config file is only updated on the node you are connected to when you click the Rotate Encryption Keys button, so, depending on whether you have clustered servers, pick one of the following procedures:

If you are not running a cluster of On-Premises servers:
1. When prompted to restart the IIS application, run iisreset to stop and restart the IIS server.
2. Open Secret Server to restart it.
If you are running a cluster of On-Premises servers:
1. When prompted to restart the IIS server, run command iisreset /stop on all nodes in an elevated command prompt. This stops but does not restart IIS on the nodes.
2. Copy the encryption.config file from the updated node to all other nodes.
  
  Note: If you are not sure which node performed the rotation, check the "modified time" of the encryption.config files.
3. Run iisreset /start on all nodes. This restarts IIS on the nodes.
4. Open Secret Server to restart it. The first Secret Server node you run continues the rotation process.
The new MEK takes effect, and any new records encrypted use the new key. Any data still using the old key continues to work until the rotation fully finishes.
Secret Server's background worker (Secret Server-BWSR.log) enables maintenance mode, preventing changes to secrets and other configuration settings, and continues the rotation process.
Secret Server marks the data needing an update in the database and begins rotating everything. A progress bar appears on the Security tab.
After all data is rotated except for session recording videos, maintenance mode is disabled.
Secret Server's session recording role (Secret Server-SRWSR.log) starts rotating any existing session recording videos encrypted by the MEK. The Last Master Encryption Key Rotation Status text changes to Pending - Encrypting Session Data. Depending on the number of videos, this may take some time. This only applies to the first MEK rotation—further rotations do not update the recordings again.
Once the process finishes, the Last Master Encryption Key Rotation Status text changes to Completed - Master Encryption Keys Rotated.

Because we only track the current and previous MEK, the MEK rotation must be fully successful before you can run another one. A Retry button appears if the process times out or needs restarting. Monitor the rotation status and contact support if it fails to complete. Failure to do so could cause extensive permanent data loss.