CloudFront and S3 Bucket CloudFormation Stack
AWS CDK and CloudFormation Stack to deploy a CloudFront distribution and an S3 Bucket
Chris GThis is a common use case if you use Wagtail, but also has other applications.
This is a common use case if you use Wagtail, the recommendation is to create an S3 "website" to store all of the images, along with an IAM user (API access only) to allow reading and writing files to that bucket.
Doing this manually is tedious and somewhat error-prone, so I wanted to automate this process. The code used in this article can also be found in this GitHub repository.
The AWS CDK is a great tool, it allows you to programmatically create the resources that you want to create, in a format that is much easier to understand compared to working directly with your CloudFormation template. Even though the CDK is also capable of deploying these resources in your AWS account, I use it to generate a traditional CloudFormation template, I think it is a more repeatable process.
CDK Input Parameters
In order to make this stack reusable, it will ask for two parameters, a DNS Domain Name, and an IP address.
- The DNS Domain Name is quite obviously the name of the Domain that you want your CloudFront Distribution to be configured for. It assumes that you have control of that DNS name and that it isn't registered as a Route53 Domain, so we'll be using DNS validation to create the SSL Certificate in AWS Certificate Manager.
Tags.of is a really neat CDK construct, it tells CDK to create Tags for every resource deployed with the name and values provided.
site_domain_name = CfnParameter(self,
id='domain_name',
description="Domain Name",
default="static.example.com",
type="String",
).value_as_string
stack_name = self.stack_name
Tags.of(self).add('Project', "Wagtail Images");
Tags.of(self).add('stackName', stack_name);
Tags.of(self).add('Domain', site_domain_name);The S3 Bucket
We'll then create the S3 bucket, configured with default encryption, and blocking public web access, following the latest best practices from AWS. Only the CloudFront Distribution will be able to read from that bucket and serve files via a public website.
We'll set the Removal Policy to RETAIN as a precaution, which means that when you delete the CloudFormation deployment, the S3 Bucket and the objects will NOT be deleted and you won't lose your data. Feel free to change the Removal Policy to DESTROY if you are just testing and want to minimize cleanup.
We'll also create two S3 lifecycle policies, see Cost-effective S3 Buckets for additional details.
myBucket = aws_s3.Bucket(
self,
's3_bucket',
bucket_name = site_domain_name,
encryption=aws_s3.BucketEncryption.S3_MANAGED,
access_control=aws_s3.BucketAccessControl.PRIVATE,
public_read_access=False,
block_public_access=aws_s3.BlockPublicAccess.BLOCK_ALL,
# removal_policy=RemovalPolicy.DESTROY,
removal_policy=RemovalPolicy.RETAIN,
auto_delete_objects=False,
lifecycle_rules = [
aws_s3.LifecycleRule(
id="multipart",
enabled=True,
abort_incomplete_multipart_upload_after=Duration.days(1),
expired_object_delete_marker=True,
),
aws_s3.LifecycleRule(
id="IA",
enabled=True,
transitions=[{
"storageClass": aws_s3.StorageClass.INTELLIGENT_TIERING,
"transitionAfter": Duration.days(0)
}]
)
]
)AWS IAM
Now, the fun part, the IAM user and access Policies.
We'll first create an IAM user without AWS Console access, and generate a secret access key. We'll store that secret access key in AWS Secrets Manager for now,
We'll also grant that user access to manage its own access keys (more on that later), and also access to manage files in the S3 bucket we just created.
user = iam.User(self, "User")
access_key = iam.AccessKey(self, "AccessKey", user=user)
secret = secretsmanager.Secret(self, "Secret",
secret_string_value=access_key.secret_access_key
)
policy = iam.Policy(self, 'myPolicy', statements=[iam.PolicyStatement(
resources=[user.user_arn],
actions=["iam:ListAccessKeys",
"iam:CreateAccessKey",
"iam:DeleteAccessKey"
]
)]
)
policy.attach_to_user(user)
myBucket.add_to_resource_policy(
iam.PolicyStatement(
sid="AllowUserManageBucket",
effect=iam.Effect.ALLOW,
actions=["s3:ListBucket",
"s3:GetBucketLocation",
"s3:ListBucketMultipartUploads",
"s3:ListBucketVersions"],
resources=[myBucket.bucket_arn],
principals=[iam.ArnPrincipal(user.user_arn)],
)
)
myBucket.add_to_resource_policy(
iam.PolicyStatement(
sid="AllowUserManageBucketObjects",
effect=iam.Effect.ALLOW,
actions=["s3:*Object"],
resources=[myBucket.arn_for_objects("*")],
principals=[iam.ArnPrincipal(user.user_arn)],
)
)The SSL Certificate
Now, we'll use the AWS Certificate Manager to create a free SSL Certificate for our CloudFront Distribution. We'll use the DNS Validation method, which will require you to create a new CNAME DNS record in your Domain DNS to prove to AWS that you own this Domain. If you use Route53 for your DNS, you may want to choose a different option.
Note hat the CloudFormation deployment will NOT be complete until you perform this step.
certificate = cm.Certificate(self, "MyCertificate",
domain_name=site_domain_name,
certificate_name=site_domain_name,
validation=cm.CertificateValidation.from_dns()
)The CloudFront Distribution
Now, let's create the CloudFront Distribution. Note that we'll use the new Origin access control settings, which are now recommended by AWS, instead of the Legacy CloudFront origin access identity (OAI) to access the S3 bucket. Since this is still not fully supported by the CDK we'll have to implement a workaround for now.
We'll also configure a few Behaviors for caching objects in our distribution, compression, default and error response, etc.
cfn_origin_access_control = cloudfront.CfnOriginAccessControl(self, "MyCfnOriginAccessControl",
origin_access_control_config=cloudfront.CfnOriginAccessControl.OriginAccessControlConfigProperty(
name=site_domain_name,
origin_access_control_origin_type="s3",
signing_behavior="always",
signing_protocol="sigv4",
description="CF S3 Origin Access Control"
)
)
distribution = cloudfront.Distribution(
self, "CloudFrontDistribution",
default_behavior=cloudfront.BehaviorOptions(
compress=True,
allowed_methods=cloudfront.AllowedMethods.ALLOW_GET_HEAD_OPTIONS,
cached_methods=cloudfront.CachedMethods.CACHE_GET_HEAD,
cache_policy=cloudfront.CachePolicy.CACHING_OPTIMIZED,
origin_request_policy=cloudfront.OriginRequestPolicy.CORS_S3_ORIGIN,
viewer_protocol_policy=cloudfront.ViewerProtocolPolicy.REDIRECT_TO_HTTPS,
origin=origins.S3Origin(
myBucket, origin_id='s3-static-frontend',
)
),
domain_names=[site_domain_name],
certificate=certificate,
http_version=cloudfront.HttpVersion.HTTP2,
enabled=True,
default_root_object="index.html",
price_class=cloudfront.PriceClass.PRICE_CLASS_ALL,
error_responses=[
cloudfront.ErrorResponse(
http_status=404, response_page_path="/index.html", response_http_status=200,
ttl=Duration.seconds(60)),
]
)
# Override the default behavior to use OAC and not OAI
# Because CDK does not have full support for this feature yet
# Based on: https://github.com/aws/aws-cdk/issues/21771#issuecomment-1282780627
cf_cfn_distrib = t.cast(
cloudfront.CfnDistribution,
distribution.node.default_child,
)
cf_cfn_distrib.add_property_override(
'DistributionConfig.Origins.0.OriginAccessControlId',
cfn_origin_access_control.get_att('Id'),
)
cf_cfn_distrib.add_property_override(
'DistributionConfig.Origins.0.S3OriginConfig.OriginAccessIdentity',
'',
)
myBucket.add_to_resource_policy(iam.PolicyStatement(
effect=iam.Effect.ALLOW,
actions=['s3:GetObject', 's3:ListBucket'],
resources=[f'{myBucket.bucket_arn}/*', myBucket.bucket_arn],
principals=[iam.ServicePrincipal("cloudfront.amazonaws.com")],
conditions={
"StringEquals": {
"aws:SourceArn": f"arn:aws:cloudfront::{Aws.ACCOUNT_ID}:distribution/{distribution.distribution_id}"
}
}
))CloudFormation Outputs
Finally, we'll output some specific details of the key components that were deployed in the CloudFormation Output. These details will be important later.
# Add stack outputs
CfnOutput(
self,
"userNAME",
value=user.user_name,
)
CfnOutput(
self,
"accessKeyId",
value=access_key.to_string(),
)
CfnOutput(
self,
"SiteBucketName",
value=myBucket.bucket_name,
)
CfnOutput(
self,
"DistributionId",
value=distribution.distribution_id,
)
CfnOutput(
self,
"distributionDomainName",
value=distribution.distribution_domain_name,
)
CfnOutput(
self,
"CertificateArn",
value=certificate.certificate_arn,
)CloudFormation Deployment
You can use the AWS CDK to deploy these resources in your account, but I prefer to use the AWS CDK to generate the CloudFormation template.
After installing all of the Python requirements, as well as the AWS CLI, you can synthesize the CloudFormation template for this CDK code using the commands below.
After that, you can use the CloudFormation.yaml file to deploy this CloudFormation stack.
python3 -m venv .env
source .env/bin/activate
pip install -r requirements.txt
cdk synth --path-metadata false --version-reporting false > CloudFormation.yamlPost Deployment Steps
After a successful deployment of our Stack, we'll have a few manual actions to take.
- The SSL Certificate that was created by AWS uses DNS Validation to verify ownership of the DNS name. The Stack will not complete the deployment until the DNS validation has been completed.
- You need to create a DNS name for your Domain as a CNAME of the CloudFront distributionDomainName.
- The aws_secret_access_key for the IAM User account that is created is stored in AWS Secrets Manager. Retrieve the value and delete the secret if you don't want to keep being charged for it.
- Populate the correct values in your environment's .env environment variable file.
DJANGO_SETTINGS_MODULE=myapp.settings.production
AWS_ACCESS_KEY_ID=<removed>
AWS_SECRET_ACCESS_KEY=<removed>
AWS_REGION=us-east-1
AWS_STORAGE_BUCKET_NAME=static.example.com
AWS_S3_CUSTOM_DOMAIN=static.example.com
DJANGO_SECRET_KEY=<removed>The complete CloudFormation template is available at:
https://github.com/Christophe-Gauge/cdk-s3-cloudfront/blob/main/CloudFormation.yaml
Tagged with:
AWS Wagtail