The new EC2 Nitro Enclaves enable virtual machines to process private data without exposing its encryption key to the parent instance. In this post we will explore how Nitro Enclaves are used to securely process private keys stored in ACM.

This is part 2 in a two-part article. In the first part we review why Nitro Enclaves matter and how they can benefit your sensitive workloads: ACM for Nitro Enclaves - It’s a Big Deal.

Overview

This article follows the steps outlined in AWS’ documentation: AWS Certificate Manager for Nitro Enclaves. In their article, AWS builds the following architecture.

As discussed in part 1, it’s essential for private keys stored in ACM to never be exposed. As such, it’s interesting to see how AWS keeps these private keys secure, while still hosting them on your (relatively insecure) EC2 instance. The answer, of course, lies in the new Nitro Enclaves.

The two main topics for this post are:

• How are private keys transmitted (and can we intercept them)?
• How are permissions assigned to the Nitro Enclave (and can we assume them)?

Launching an EC2 instance

The first step is to launch an Enclave-enabled EC2 instance. From the requirements we learn this can be any Nitro instance with an Intel or AMD processor. Further digging shows that the minimum size is an m5a.xlarge.

The operating system needs to be Linux. On Amazon Linux 2 the Nitro CLI is available in a yum repository. Installation instructions for other operating systems can be found on the Nitro Enclaves CLI Github page.

Amazon has provided ready-to-go AMIs with NginX and the Nitro CLI pre-installed, so for this article we will use those. We will assign an IAM role with admin permissions to the instance so we won’t be limited in exploring access methods.

With the pre-build AMI deployed in a default VPC and an IAM role with admin permissions, our current architecture looks like this:

Creating an ACM certificate

ACM certificates are free, but we do need a valid domain name. I’ve once purchased vpcdemo.net, so that’s what I will use for this article.

After we’ve moved through the steps of requesting a certificate, it shows as Issued and has an ARN. We will need this ARN in the next step.

Associating the IAM role with the certificate

This is where it gets interesting. The next step in the process is to “Associate the role with the ACM certificate”.

The command to achieve this is aws ec2 --region [region] associate-enclave-certificate-iam-role --certificate-arn [certificate_ARN] --role-arn [role_ARN]

This means we’re telling ACM that our EC2 instance role is allowed to access this certificate and private key. But we haven’t created a Nitro Enclave yet, and this role is assigned to an EC2 instance we control. Does that mean we will be able to access the private key from our instance? Let’s find out.

Running the command above yields the following output:

aws ec2 --region eu-central-1 associate-enclave-certificate-iam-role --certificate-arn arn:aws:acm:eu-central-1:123412341234:certificate/f2bb1a6e-5704-4702-beb7-a2c3f36e7103 --role-arn arn:aws:iam::123412341234:role/admin-role
{
    "EncryptionKmsKeyId": "cb8e3d89-cd82-4560-867c-641c0008fab2", 
    "CertificateS3BucketName": "aws-ec2-enclave-certificate-eu-central-1-prod", 
    "CertificateS3ObjectKey": "arn:aws:iam::123412341234:role/admin-role/arn:aws:acm:eu-central-1:123412341234:certificate/f2bb1a6e-5704-4702-beb7-a2c3f36e7103"
}

This output obviously refers to three things:

• An S3 bucket (owned by AWS)
• An S3 object (that likely contains our certificate and private key)
• A KMS key (that is likely used to decrypt sensitive data)

In the next step, AWS describes we should assign new permissions to our EC2 instance’s IAM role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
        "Effect": "Allow",
        "Action": [
            "s3:GetObject"
        ],
        "Resource": ["arn:aws:s3:::aws-ec2-enclave-certificate-eu-central-1-prod/*"]
    },
    {
        "Effect": "Allow",
        "Action": [
            "kms:Decrypt"
        ],
        "Resource": "arn:aws:kms:eu-central-1:*:key/cb8e3d89-cd82-4560-867c-641c0008fab2"
    }
  ]
}

These permissions allow our role to fetch an object from the AWS-owned S3 bucket, and to use the AWS-owned KMS key to decrypt data. Let’s update our architecture diagram with these new components.

Retrieving the file from S3

The CertificateS3BucketName and CertificateS3ObjectKey clearly identify where the ACM files are stored. Since our IAM Role now has all the necessary permissions, we can try to download the file.

[ec2-user@ip-172-31-42-108 ~]$ aws s3 cp s3://aws-ec2-enclave-certificate-eu-central-1-prod/arn:aws:iam::123412341234:role/admin-role/arn:aws:acm:eu-central-1:123412341234:certificate/f2bb1a6e-5704-4702-beb7-a2c3f36e7103 .
download: s3://aws-ec2-enclave-certificate-eu-central-1-prod/arn:aws:iam::123412341234:role/admin-role/arn:aws:acm:eu-central-1:123412341234:certificate/f2bb1a6e-5704-4702-beb7-a2c3f36e7103 to ./f2bb1a6e-5704-4702-beb7-a2c3f36e7103

Lo and behold: it worked! We now have the ACM certificate files on our local machine. The million dollar question is what they contain.

Analyzing the file contents

We’ll output the file and run it through jq:

The contents are in JSON format, with four keys:

• certificate
• certificateChain
• encryptedPrivateKey
• encryptionMethod

The first two values are unencrypted, but these are the public certificate files anyone visiting vpcdemo.net would receive. No secrets there.

The third value is the private key we’re looking for. Obviously, it’s been encrypted. However, we also got access to a KMS key… Let’s see if we can use that to decrypt this value!

Decrypting the private key

First we’ll store the encryptedPrivateKey in a variable: PRIVKEY=$(cat f2bb1a6e-5704-4702-beb7-a2c3f36e7103 | jq -r '.encryptedPrivateKey').

Then we’ll run the private key through KMS: aws kms --region eu-central-1 decrypt --ciphertext-blob fileb://<(echo $PRIVKEY | base64 -d) --output text --query Plaintext.

Unfortunately, but expectedly, this results in the following output:

An error occurred (AccessDeniedException) when calling the Decrypt operation: The ciphertext refers to a customer master key that does not exist, does not exist in this region, or you are not allowed to access.

So this doesn’t work. Let’s find out why. We’ll start by looking at CloudTrail. It shows an interesting line:

"errorMessage": "User: arn:aws:sts::123412341234:assumed-role/admin-role/i-025be0b6a191a2cde is not authorized to perform: kms:Decrypt on resource: arn:aws:kms:eu-central-1:194321236082:key/cb8e3d89-cd82-4560-867c-641c0008fab2",

This shows us that the KMS key is stored in account 194321236082, and that our role was not allowed to use it. This is interesting, because we did exactly follow AWS’ instructions, which added permissions for our role to use this KMS key. The answer must lie somewhere in the Nitro Enclaves.

Running NginX with Nitro Enclaves

After walking through the last few steps in the AWS docs we’ve got a running server with HTTPS:

➜  ~ curl -I -XGET https://vpcdemo.net
HTTP/1.1 200 OK
Server: nginx/1.18.0
Date: Fri, 30 Oct 2020 13:24:26 GMT
Content-Type: text/html
Content-Length: 3520
Last-Modified: Wed, 24 Jun 2020 18:17:11 GMT
Connection: keep-alive
ETag: "5ef398a7-dc0"
Accept-Ranges: bytes

In the NginX configuration at /etc/pki/nginx/nginx-acm.conf we see the following lines:

ssl_certificate_key "engine:pkcs11:pkcs11:model=p11ne-token;manufacturer=Amazon;token=nginx-acm-token;id=%01;object=acm-key;type=private?pin-value=8c9d293b5fcbe9bc5f70fa400822a936";
ssl_certificate "/run/nitro_enclaves/acm/nginx-cert-6e67696e782d61636d2d746f6b656e.pem";

So the Nitro Enclave was able to download and decrypt my certificate, even though the parent instance wasn’t. The next question is how AWS has secured their KMS key so the parent instance can’t use it, but the Nitro Enclave using the same IAM Role can.

Attestation

The answer can be found in Jeff Barr’s blog post and the Cryptographic attestation chapter in the documentation. When a new enclave image file (the OS and code that runs in the enclave) is created, it will automatically hash its contents in various ways. These are then returned as platform configuration registers (PCRs). There are eight PCRs:

• PCR[0]: a hash of the enclave image file
• PCR[1]: a hash of the Linux kernel and bootstrap
• PCR[2]: a hash of the application
• PCR[3]: a hash of the IAM role assigned to the parent instance
• PCR[4]: a hash of the Instance ID of the parent instance
• PCR[8]: a hash of the Enclave image file signing certificate

The first one (PCR0) can be used in a KMS condition. From the KMS docs:

The kms:RecipientAttestation:ImageSha384 condition key allows the kms-decrypt, kms-generate-data-key, and kms-generate-random operations from an enclave only when the image hash from the signed attestation document in the request matches the value in the condition key. The ImageSha384 value corresponds to PCR[0] in the attestation document. This condition key is effective only when you call these APIs from an enclave using the Nitro Enclaves SDK.

And from Jeff Barr’s blog post: “In a real-world environment, I would create a KMS key policy that checks the PCR value as part of a Condition statement:”

"Condition": {
"StringEqualsIgnoreCase": {
 "kms:RecipientAttestation:ImageSha384": "ecfd7aa6d1dcca1e0bba646e7d49ede2761651c68f13cee68b1141c182cd836baae37d05dd8e6260aa847369a7b27e24"
}

In simple terms: every enclave image has a signature that changes when the content of the enclave image changes. By setting the PCR[0] value at the time the image was built as a condition in KMS, the Decrypt operation will only be allowed when executed by this exact version of the enclave image. When somebody tampers with the image - or tries to use the role from the parent instance as we did above - the PCR0 will change, the KMS condition will no longer match, and access will be denied.

Conclusion

In this post we have analyzed the steps AWS has taken to keep the private keys stored in ACM secure. We’ve seen that the ACM certificates and keys are stored in an S3 bucket, and that your EC2 instance role is granted access to that bucket. The same EC2 instance role is also allowed to use a KMS key to decrypt the ACM private key, but through attestation and conditions in the KMS policy, the Decrypt call will only succeed when executed from a trusted enclave.

You can use the same mechanism to build your own enclaves and securely interact with KMS. However, keep in mind that the entire process is only as secure as the single Condition in the KMS trust policy. One error here, and your private data might as well be out in the open.

If you haven’t read it, part 1 of this article explores why the Nitro Enclaves are a great new feature for securing private data. The first part is far less technical and provides a nice 30.000-foot overview of Nitro Enclaves.

I share posts like these and smaller news articles on Twitter, follow me there for regular updates! If you have questions or remarks, or would just like to get in touch, you can also find me on LinkedIn.