Hardware Security Modules

Introduction

Secret Server integrates with hardware security modules (HSMs). When Secret Server is configured to use an HSM, the Secret Server encryption key is protected by that HSM.

HSMs offer several security features that traditional servers cannot. Depending on the model and design of the HSM, most HSMs are designed to be physically tamper-proof. HSMs may also be independent hardware on a network, which allows physically placing the HSM is a more secure location that might otherwise be too inconvenient for a server.

To provide broad support for HSMs, Secret Server supports any HSM that can be configured with Microsoft's Cryptography Next Generation (CNG) provider or Public-Key Cryptography Standards #11 (PKCS #11). CNG is a layer provided by Windows Server 2008 and later that HSM manufacturers can interface with. PKCS #11 is an API provided by each HSM vendor that Secret Server can interface with to perform cryptographic operations. If your HSM properly supports CNG or PKCS #11 and supports compatible algorithms, Secret Server can use it.

HSM Requirements

Each HSM must provide support for these algorithms

For CNG:

• RSA 4096: Support for RSA with 4096-bit keys is required. The HSM must also support RSA for encryption and decryption, in addition to signing.

• Padding Type: The HSM must support one of the following for RSA encryption:

  • PKCS#1 v1.5 padding
  • OAEP padding

For PKCS #11:

• AES 256: Support for AES 256-bit keys is required. The HSM must also support a CKM_AES_CBC_PAD mechanism for encryption and decryption.

Additionally, closely follow the requirements and recommendations of the HSM vendor for things such as minimum latency, redundancy, and operating environment.

Silent HSM Operation

Because Secret Server is a Web application with no one physically present at the server at most times, Secret Server interacts with the HSM in "silent" mode. This prevents the HSM from attempting to interact with any users logged onto the server.

Some HSM features require interaction. If the HSM is configured in a way that requires interaction, Secret Server cannot communicate with the HSM and fails during the configuration steps.

For example, Operator Card Sets (OCS) in Thales network HSMs are such a configuration. If the Thales CNG provider is configured to use an OCS for key protection instead of module protection, someone must be physically present at the HSM and the server to insert their operator card when the key is needed. If the OCS quorum is more than a single card, Secret Server cannot interact with the HSM because it requires inserting and removing the OCS cards.

In that case, we recommend that Thales' CNG provider is configured to use module protection instead of an OCS. It is possible to use an OCS with Secret Server if the quorum is exactly one card and the card is left in the HSM at all times.

Consult your HSM vendor and their documentation to ensure that the HSM and their CNG provider are able to operate in silent mode and are configured to do so.

Configuring HSM Integration

To configure the HSM integration, go to the Admin > Configuration menu and click HSM in the Configuration section.

For CNG:

You can find the list of available CNG Providers by querying for the list of registered CNG providers. Each provider must correctly report that it is a "Hardware" provider, and that it is not a Smart Card reader. If an error occurs while querying the CNG provider for its properties, it will not appear in the list; however the error is reported to Secret Server's system log. If the desired CNG provider does not appear in the list of CNG providers, ensure that the provider is correctly registered and that IIS has been restarted after the CNG registration. Also check that an error is not occurring while querying the HSM by examining the system log.

For PKCS #11:

Each HSM vendor provides a cryptoki library (dll) for PKCS #11 that Secret Server can interface with to perform cryptographic operations. Please note:

• This DLL must be copied to c:\inetpub\wwwroot\SecretServer\pkcs11 folder for Secret Server to see it.

• This DLL must support CKM_AES_KEY_GEN and CKM_AES_CBC_PAD mechanisms and the slot must be a hardware slot.

Along with the DLL, a token label and the user PIN are needed for Secret Server setup. Please refer to your HSM vendor's documentation for obtaining these values.

Once the options are selected, Secret Server simulates encryption and decryption operations and verifies the results to check that it is functioning properly. Finally, Secret Server verifies the selected providers, and then enables HSM integration.

Enabling HSM

To enable the HSM:

1. Navigate to Administration > Configuration > HSM.

2. Click the Enable HSM button.

3. Important: back up the encryption.config file before proceeding.

4. Click the Next button.

5. Select the API type.

6. Select the required options:

   • For CNG, select the HSM Persistent Provider, Key Size, and Padding Type.

   • For PKCS11, select the Library Name and type the token label and user PIN.

7. Click the Next button. This performs a test to ensure the HSM can be used.

8. Click the Next button.

9. Verify the HSM configuration.

10. Click the Save button. The enabled configuration looks like this:
    
    
    

11. Do an IIS reset or application pool recycle. This starts the rotation.

Rotating the HSM Key

If HSM is enabled, Secret Server can now rotate the HSM key to ensure the secret keys are always protected by an HSM key. Rotating the HSM key only decrypts the secret keys and then re-encrypts them with the new HSM key. We recommend performing a secret key rotation after the HSM key has been rotated.

To rotate the HSM key:

1. Navigate to Admin > Configuration > HSM.

2. Click the Rotate HSM Key button.

3. Important: back up the encryption.config file before proceeding.

4. Click the Next button.

5. Select the API type.
   
   

   Secret Server is unable to rotate to a different API type. So the selection will be the same API type as currently used.
   
   

6. For CNG, select the HSM Persistent Provider, Key Size, and Padding Type.

7. For PKCS11, select the Library Name and type the token label and user PIN.

8. Click the Next button. This performs a test to ensure the HSM can be used.

9. Click the Next button.

10. Verify the new HSM configuration:
    
    
    

11. Click the Save button.

12. Do an IIS reset or application pool recycle. This starts the rotation.

Disabling HSMs

To disable an HSM:

1. Navigate to Administration > Configuration > HSM.

2. Important: back up the encryption.config file before proceeding.

3. Click the Disable HSM button. A warning popup appears:
   
   
   

4. Decide if you want to delete the key from the HSM. Keep in mind deleting the key makes any backups using this key unusable.

5. Click Continue.

6. Do an IIS reset. This starts the rotation.

Stopping Other Nodes in a Clustered Environment

HSMs cannot be enabled, disabled, or rotated, in a clustered Secret Server environment. Follow these steps to stop the other nodes while updating the HSM configuration:

1. Follow these steps on all nodes except for one:

   1. Open IIS Manager on server node.

   2. Expand the server.

   3. Select Application Pools.

   4. Right-click on the Secret Server application pool.

   5. Select Stop.

2. Follow these steps on the one running node:

   1. Navigate to Administration > Configuration > Backup.

   2. Click on Run Backup Now. This backs up the Secret Server database and application files.

   3. Navigate to Administration > Configuration > HSM.

   4. Click on these HSM options: Enable HSM, Disable HSM, or Rotate HSM Key and follow the previous instructions.

   5. Once completed, go back to IIS Manager and recycle the Secret Server application pool, or open a command prompt window as an administration and perform an IIS reset.

   6. Confirm that accessing secrets in Secret Server is successful.

3. Follow these steps on each of the other server nodes:

   1. Copy the updated encryption.config file from the one running node to the c:\inetpub\wwwroot\SecretServer folder.

   2. Open IIS Manager.

   3. Expand the server.

   4. Select Application Pools.

   5. Right-click on the Secret Server application pool.

   6. Select Start.

   7. Confirm that logging in and accessing secrets in Secret Server is successful.

Securing HSM Integration

The wizard to enable, rotate, and disable HSM integration is protected by the "Administer HSM" role permission in Secret Server. The permission should be carefully assigned—if at all. Additionally, you can create an event subscription that sends alerts when the role permission is assigned or unassigned from a role.

Configuring the HSM also has its own event subscriptions for when the HSM integration is enabled, rotated, or disabled.

Additionally, an you can add an application setting to Secret Server to prevent changes to HSM configuration. Enabling, rotating, and disabling this requires direct access to the file system where Secret Server is installed.

To enable this, edit the web-appSettings.config file within Secret Server to contain a key called LockHsmConfiguration with a value of True as follows:

<?xml version="1.0" encoding="utf-8" ?>
<appSettings>
  <add key="LockHsmConfiguration" value="True" />
</appSettings>

This prevents access to the HSM configuration pages, regardless of role permissions. The only way to gain access is to remove this setting, thus proving you, at a minimum, have access to the server where Secret Server is installed.

HSM Redundancy

HSM redundancy varies from HSM to HSM. Please refer to the vendor's documentation on how to back up the HSM. Backups are typically either made to common file location, another HSM, or onto a smart card with the HSM's built-in smart card reader.

As long as the HSM provider is installed on the server and a key exists on the HSM with the same identifier, Secret Server attempts to use that key.

Testing HSM CNG Configuration

Secret Server does its own testing and verification of the HSM and its CNG provider before the HSM integration can be enabled. To further diagnose any issues with the HSM, the certutil command line utility, which is part of Windows, can test the HSM with the –csptest option specified. An example output may contain something like this:

Provider Name: SafeNet Key Storage Provider

   Name: SafeNet Key Storage Provider

……

Asymmetric Encryption Algorithms:

  RSA

      BCRYPT_ASYMMETRIC_ENCRYPTION_INTERFACE -- 3

      NCRYPT_ASYMMETRIC_ENCRYPTION_OPERATION -- 4

      NCRYPT_SIGNATURE_OPERATION -- 10 (16)

    NCryptCreatePersistedKey(SafeNet Key Storage Provider, RSA)

        Name: cngtest-6166f8fe-8caf-4e30-8e5c-a-24575
……

Pass

Examine the output of the test by looking for your CNG provider's name for your HSM and verifying the result. We recommend running this test using the same account as the application pool Secret Server is using. If the testing tool reports errors, consult your HSM's vendor or documentation for resolution.

PKCS #11 Log

The PKCS #11 logs are at c:\inetpub\wwwroot\SecretServer\pkcs11\log\SSPKCS11.log.

This captures the PKCS #11 activity between Secret Server and the HSM when enabling, rotating, and disabling.

Also, each HSM vendor will have a way to output the PKCS #11 logging to a file. Please refer to your HSM vendor’s documentation on how to set up the logging.

Troubleshooting

If this error message “Thycotic.HSM.CNG.CngException: Failed to operate transformation” occurs, do the following:

1. Go to the Secret Server directory.

2. Open the web-appSettings.config file located at c:\inetpub\wwwroot\SecretServer.

3. Add the following tag in that file:

   Copy

   <?xml version="1.0" encoding="utf-8" ?>
   <appSettings>
       <add key="RecycleAppPoolOnHSMError" value="true" />
   </appSettings>

4. Restart IIS.