Are you utilizing AWS S3 to host your static website? It's a smart choice, given the scalability and reliability that Amazon Web Services (AWS) offers. However, recent changes to AWS S3 bucket permissions have brought new challenges to the forefront, particularly in terms of access control and security. Ensuring the privacy of sensitive files while maintaining a publicly accessible website can be a daunting task.
I like to create my own Nodejs scripts and modules to automate many of my tasks, especially when it comes to creating AWS services. My script which has worked for nearly a decade suddenly failed over the weekend due to recent AWS policy changes.
This means using a provisioning script that worked before may now create AWS S3 Access Denied errors, leaving you and me very frustrated.
![Solving the new AWS S3 Access Denied Issue for Websites]()
In this article, we will dive deep into the evolving landscape of AWS S3 bucket permissions and shed light on how these changes impact static websites. We will explore the significance of proper access control and security measures in ensuring the integrity and confidentiality of your website's assets.
Join me on this journey as we navigate through the intricacies of AWS S3 and discover effective solutions to address the challenges posed by the new default security settings. By understanding the implications of these changes and learning how to implement robust access control measures, you'll be well-equipped to strike the right balance between a publicly accessible website and the protection of sensitive data.
Get ready to unravel the mysteries surrounding AWS S3 bucket permissions and discover practical solutions to safeguard your static website's assets. Let's dive in and explore the world of access control and security in the realm of AWS S3.
Changes to AWS S3 Bucket Permissions
AWS has recently implemented new default security settings for Amazon S3, aimed at enhancing the overall security posture of S3 buckets. As of April 2023, AWS automatically enables the Block Public Access feature and disables the use of Access Control Lists (ACLs) on newly created S3 buckets. This update ensures that S3 buckets are private by default, preventing accidental public access and strengthening the control over bucket access policies. Website owners hosting static websites on S3 buckets should take note of these changes and adjust their access control strategies to leverage the recommended security features provided by AWS.
In the past, you could just set the bucket's ACL to 'public-read'.
// Set the website bucket's ACL to public
await s3.putBucketAcl({
Bucket,
ACL: "public-read",
}).promise();
Or set individual files as they were uploaded to 'public-read'. But the changes prevent this.
Explanation of the New Default S3 Bucket Security Settings
In April 2023, AWS implemented two key changes to the default bucket security settings in Amazon S3. These changes are designed to strengthen the security posture of S3 buckets and promote best practices in access control.
The first change involves the automatic enablement of S3 Block Public Access for all new S3 buckets. This feature helps prevent unintended public access to S3 buckets and mitigates the risk of data breaches. By enabling S3 Block Public Access, AWS ensures that new buckets are private by default, allowing only the bucket owner to access the contents unless explicitly granted access.
The second change pertains to the deprecation of S3 Access Control Lists (ACLs) for all new buckets. Instead of relying on ACLs for access control, AWS recommends leveraging AWS Identity and Access Management (IAM) policies as a more flexible and robust alternative. IAM policies provide fine-grained control over the bucket and object-level permissions, enabling website owners to define access rules based on specific user roles or groups.
Implications for Static Websites Hosted on S3 Buckets
The introduction of these new default bucket security settings has several implications for static websites hosted on S3 buckets. It is crucial for website owners to understand these implications to ensure the proper configuration and security of their websites.
-
Limited Public Access: With S3 Block Public Access enabled by default, newly created buckets restrict public access to the bucket and its objects. This means that by default, only the bucket owner has access to the contents. It becomes essential for website owners to carefully plan their access control policies and explicitly grant public read access to the necessary objects to make the website publicly accessible.
-
ACL Deprecation: The deprecation of ACLs in favor of IAM policies introduces a shift in access control management for S3 buckets. Website owners must adapt their access control mechanisms to utilize IAM policies effectively. This involves defining appropriate policies that allow the required access for website visitors while maintaining the necessary level of security.
-
Automation Considerations: Website provisioning and deployment processes that relied on setting public ACLs or other legacy access control mechanisms may require adjustments to align with the new default security settings. Automation scripts, deployment pipelines, and infrastructure configuration tools need to be updated to utilize IAM policies and account for restricted public access by default.
By embracing these changes and understanding their implications, website owners can ensure their static websites hosted on S3 buckets maintain a strong security posture while offering the desired level of public accessibility. In the next section, we will explore the solutions to navigate these challenges and effectively manage the access control and security of static websites on AWS S3.
Impact on Static Websites
The recent changes in AWS S3 bucket permissions have had a profound impact on the hosting of static websites. It is crucial to fully grasp the implications of the new default security settings in order to effectively manage access control and security for these websites.
In the context of website hosting, the primary objective is to allow public read access to the site's assets in the vast majority of cases. This is one of the key advantages of hosting static websites behind progressive web applications. By protecting sensitive data through authorization tokens and an API, the static assets necessary to power the website or web application can remain public.
However, with the removal of the 'public-read' ACL in S3, the process of provisioning public websites has become considerably more complex. This change has presented challenges for developers seeking a solution to enable public access to their newly created sites.
To address this issue, I have explored various strategies and options to make S3 website buckets publicly accessible. By leveraging a combination of S3 bucket policies, CloudFront distributions, and proper access control configurations, I arrived at a robust solution. This solution allows developers to create and manage static websites on S3 while maintaining the desired level of public accessibility.
Let's start by reviewing why the changes were made and why they are ultimately good.
Understanding the Default Private Nature of S3 Buckets and Website Accessibility
With the new default security settings, S3 buckets are set to private by default, limiting public access to the bucket and its objects. This means that without explicit configuration, static websites hosted on S3 buckets are not accessible to the public. While this default private nature enhances security, it requires website owners to configure proper access controls to make their websites publicly available.
Website accessibility depends on configuring the appropriate access policies to allow public read access to the required objects. Website owners need to identify the specific assets, such as HTML, CSS, JavaScript files, and images, that need to be publicly accessible and configure the necessary permissions accordingly.
Advantages of the New Default Security Settings for Bucket Access Control
The introduction of S3 Block Public Access as the default setting brings several advantages for access control and security. By making buckets private by default, AWS ensures that website owners have full control over their content and reduces the risk of accidental public exposure.
The use of IAM policies instead of ACLs offers more flexibility and granularity in access control management. IAM policies allow website owners to define specific permissions for different user roles or groups, enabling fine-grained control over who can access the website's assets. This allows for a more secure and tailored approach to access control, aligning with best practices in security and compliance.
Challenges Faced by Developers in Making Static Website Assets Publicly Available
While the new default security settings enhance security, developers may face challenges in making static website assets publicly available. Previously, using ACLs, developers could easily set public read access for objects during the upload process. However, with the deprecation of ACLs, developers need to utilize IAM policies and other techniques to grant public read access selectively.
This change requires developers to update their workflows and scripts to incorporate the necessary steps to configure IAM policies and manage public access permissions. It may involve adjustments in deployment processes, build pipelines, or content management systems to ensure the correct permissions are applied to the website's assets.
Developers also need to consider the implications of restricted public access by default when designing automation solutions for website provisioning and deployment. These solutions must incorporate the configuration of IAM policies to allow public read access to the required objects while maintaining a secure environment.
In the next section, we will explore the solutions and best practices to overcome these challenges and effectively manage the access control and security of static websites hosted on AWS S3.
Addressing the Challenges
With AWS deprecating ACLs for setting permissions on newly created buckets, including the ability to assign public read access during upload, I ran into a brick wall. The deprecation aims to improve security and align with best practices by promoting the use of more robust and scalable mechanisms for access control, such as IAM policies. But, at least I did not feel, there was an adequate explanation of what to do now.
While the use of ACLs is discouraged for newly created buckets, it's important to note that existing buckets and applications utilizing ACLs will continue to function as expected. However, transitioning to IAM policies is recommended for better access control management and compatibility with AWS services and features.
This is where my frustration began.
Developers often encountered frustrating roadblocks while attempting to upload files with restricted access. One particular exception message, 'AccessDenied: Access Denied,' became a recurring obstacle, leaving them scratching their heads for a solution.
As I was working on updating scripts I had been using for nearly a decade without issue to create new websites, I suddenly hit a roadblock where an Access Denied exception was being returned from the AWS Nodejs SDK:
AccessDenied: Access Denied
at Request.extractError ({path to my project}\node_modules\aws-sdk\lib\services\s3.js:699:35)
at Request.callListeners ({path to my project}\node_modules\aws-sdk\lib\sequential_executor.js:106:20)
at Request.emit ({path to my project}\node_modules\aws-sdk\lib\sequential_executor.js:78:10)
at Request.emit ({path to my project}\node_modules\aws-sdk\lib\request.js:688:14)
at Request.transition ({path to my project}\node_modules\aws-sdk\lib\request.js:22:10)
at AcceptorStateMachine.runTo ({path to my project}\node_modules\aws-sdk\lib\state_machine.js:14:12)
at {path to my project}\node_modules\aws-sdk\lib\state_machine.js:26:10
at Request.<anonymous> ({path to my project}\node_modules\aws-sdk\lib\request.js:38:9)
at Request.<anonymous> ({path to my project}\node_modules\aws-sdk\lib\request.js:690:12)
at Request.callListeners ({path to my project}\node_modules\aws-sdk\lib\sequential_executor.js:116:18) {message: 'Access Denied', code: 'AccessDenied', region: null, time: Sun May 14 2023 20:33:29 GMT-0400 (Eastern Daylight Time), requestId: 'YZ5W35K99P0W3N1W', …}
First, I hate when I get error messages like this, the ones where the call stack lists code I do not own. Plus if you have ever tried to debug into the Nodejs SDK it is impossible. No clue how Amazon does, but that is another topic.
As a developer trying to do something that was just easy a few days before it provides no help. That is where my weekend got consumed trying to solve the issue created by the change to the s3 permission model. While the change is good, they provided no guidance or examples of how someone like me should now use the Nodejs SDK to create a new website bucket.
I need sites to be publically available, almost all the time. Now all my websites were restricted, as in not available at all in the browser, which defeats the purpose of converting an s3 bucket to a web server.
After hours, and I mean a lot of hours over the weekend working through the problem with ChatGPT and the Bing AI I finally got a solution.
Introduction of Alternative Solutions for Controlled Access to Specific Files
To address the need for controlled access to specific files in publicly inaccessible S3 buckets, developers can leverage alternative solutions that provide finer-grained control over access permissions. Two such solutions are CloudFront signed URLs and signed cookies. Let's look at these before I get to the real solution I used.
- Using CloudFront Signed URLs for Temporary, Controlled Access
CloudFront signed URLs enable developers to generate unique URLs that grant temporary access to specific files in an S3 bucket via a CloudFront distribution. These URLs have a limited validity period and can be generated with specific access permissions, such as read-only access.
By generating signed URLs for restricted files, developers can share these URLs with authorized users or embed them in their applications, granting temporary access to the files while ensuring that unauthorized users cannot access them directly. This solution offers controlled and time-limited access to sensitive or private files, adding an extra layer of security to the static website.
I have used this solution for a few applications I built, but it is not for a public website. A real-world example of where this applies is when you generate a link to share a private file in OneDrive or Google Drive with a friend.
For most websites, this is not a viable solution, because let's face it we need the files on the site to be accessible for everyone, all the time.
- Generating Signed Cookies for Finer-Grained Access Control
Another alternative solution is the use of signed cookies. Signed cookies allow developers to specify granular access control rules for S3 objects by associating access permissions with cookies. These cookies are generated and signed by the application or a trusted authority and are then passed to the user's browser.
When the user requests a protected file, the browser includes the signed cookie, which is verified by the server. Based on the access control rules defined in the cookie, the server determines whether to allow or deny access to the requested file.
Signed cookies provide developers with more flexibility in defining access policies and can be useful for scenarios where access control decisions are based on dynamic factors such as user roles or specific conditions. This solution offers enhanced control over file access and ensures that only authorized users can retrieve the protected files.
Again, this is not a viable solution. Cookies are being phased out and you can't just issue cookies for everyone before the visit.
Now onto what I actually did to solve my problem and I think it will help you too.
My Solution, Creating and Assigning a Bucket Access Policy
The previous options, while possible just are not viable for a normal website, especially the way I use websites.
To overcome the challenge of restricted access to static website assets, I chose a straightforward solution: enabling public access to the S3 website bucket through IAM policies. By allowing public access, the website's files and resources become readily available to visitors. Let's explore the step-by-step process to implement this solution.
This first step is the real key. I was banging my head on that brick wall for hours trying to do step 2, only to get rejected by AWS repeatedly. So make sure you configure your bucket with these parameters first.
- Enable Public Access to the Bucket:
const s3 = new AWS.S3({});
await s3.putPublicAccessBlock({
Bucket: Bucket,
PublicAccessBlockConfiguration: {
BlockPublicAcls: false,
IgnorePublicAcls: false,
BlockPublicPolicy: false,
RestrictPublicBuckets: false
}
}).promise();
The putPublicAccessBlock method is used to configure the public access settings for an S3 bucket. It allows you to enable or disable various settings that control public access to the bucket and its objects.
In the provided code snippet, the following settings are specified in the PublicAccessBlockConfiguration object:
- BlockPublicAcls: false: This setting determines whether public access is blocked for ACLs (Access Control Lists). Setting it to false means that ACLs are not blocked, allowing public access through ACLs.
- IgnorePublicAcls: false: This setting determines whether public access is ignored for objects that have public ACLs. Setting it to false means that public ACLs are not ignored, meaning they are considered when determining access permissions.
- BlockPublicPolicy: false: This setting determines whether public access is blocked for bucket policies. Setting it to false means that bucket policies can allow public access.
- RestrictPublicBuckets: false: This setting determines whether public access is restricted for the bucket itself. Setting it to false means that the bucket is not restricted from being public.
By setting all these options to false, the code ensures that public access is not blocked or restricted, allowing the bucket and its objects to be publicly accessible.
After configuring the desired public access settings, the method returns a promise, which is awaited using .promise() to wait for the operation to complete before proceeding with the next steps. I like to use asynchronous programming (promises) and this is how you enable any AWS Nodejs SDK function to be asynchronous.
Remember to replace ${Bucket} with the actual name of your S3 bucket in the Bucket property.
- Configure Bucket Policy for Public Read Access:
const bucketPolicy = {
Version: '2012-10-17',
Statement: [
{
Sid: 'PublicReadGetObject',
Effect: 'Allow',
Principal: '*',
Action: 's3:GetObject',
Resource: `arn:aws:s3:::${Bucket}/*`
}
]
};
await s3.putBucketPolicy({
Bucket: Bucket,
Policy: JSON.stringify(bucketPolicy)
}).promise();
The provided code is used to set a bucket policy that allows public read access to objects in the specified S3 bucket. You can read more about the putBucketPolicy method in the SDK documentation.
The bucket policy is defined by the bucketPolicy object, which includes the following properties:
- Version: '2012-10-17': Specifies the version of the policy language being used.
- Statement: An array containing one or more policy statements. In this case, there is a single statement defined.
- Sid: 'PublicReadGetObject': A unique identifier for the policy statement.
- Effect: 'Allow': Specifies that the defined actions are allowed.
- Principal: '*': Indicates that the action is allowed for all users.
- Action: 's3:GetObject': Specifies the action that is allowed, which is retrieving objects from the bucket.
- Resource: arn:aws:s3:::${Bucket}/*``: Defines the resource to which the policy statement applies. In this case, it allows access to all objects in the specified S3 bucket.
The putBucketPolicy method is then called to apply the bucket policy to the specified bucket. The Bucket property is set to the name of the S3 bucket, and the Policy property is set to the JSON representation of the bucketPolicy object.
By executing these code snippets, the bucket's public access is enabled, and a bucket policy is set to allow public read access to its objects. This ensures that your static website files are accessible to visitors without any unnecessary access restrictions.
Implementing this solution not only simplifies the process of hosting a publicly accessible static website but also ensures a seamless user experience for your website visitors.
Taking it One More Step: Controlling Read Access by File Type
In AWS S3, enabling public-read access by file types allows you to control access to specific types of files within your S3 bucket. This ensures that only the designated file types are publicly accessible while restricting access to other file types. This approach is similar to how you can configure file type access in IIS, the web server commonly used with ASP.NET applications.
In IIS, you can achieve a similar outcome by configuring MIME types and corresponding access permissions. MIME types are used to identify the file types being served by the web server, and IIS allows you to set access permissions at the file type level.
For example, if you want to enable public access to .zip and .pdf files hosted on an IIS server, you would configure IIS to serve these file types with appropriate permissions. This typically involves setting the appropriate MIME types for .zip and .pdf files in IIS and ensuring that the access permissions allow public read access to these specific file types.
By doing so, you can ensure that only the designated file types, such as .zip and .pdf, are accessible to the public while maintaining restricted access to other file types.
Just like in AWS S3, this approach in IIS allows you to have fine-grained control over the accessibility of different file types in your web applications, ensuring a secure and tailored approach to access control.
Please note that the specific steps to configure file type access in IIS may vary depending on the version of IIS you are using. It's recommended to consult the official documentation or resources specific to your IIS version for detailed instructions on configuring file type access permissions.
The following policy demonstrates how to configure access permissions for different file types in an S3 bucket. The policy uses IAM policies to control public-read access to specific file types while denying public access to files in a specific key suffix.
const bucketPolicy = {
Version: '2012-10-17',
Statement: [
{
Sid: 'AllowPublicReadForSpecificFileTypes',
Effect: 'Allow',
Principal: '*',
Action: 's3:GetObject',
Resource: `arn:aws:s3:::${Bucket}/*`,
Condition: {
StringLikeIfExists: {
's3:prefix': [
'*.html',
'*.gif',
'*.jpg',
'*.png',
'*.bmp',
'*.webp',
'*.svg',
'*.ico',
'*.css',
'*.js',
'*.woff',
'*.ttf',
'*.manifest',
'*.json'
]
}
}
},
{
Sid: 'DenyPublicReadForSpecificKeySuffix',
Effect: 'Deny',
Principal: '*',
Action: 's3:GetObject',
Resource: `arn:aws:s3:::${Bucket}/private-eyes/*`
}
]
};
await s3.putBucketPolicy({
Bucket: Bucket,
Policy: JSON.stringify(bucketPolicy)
}).promise();
In this updated bucket policy, the first statement with the Sid "AllowPublicReadForSpecificFileTypes" grants public-read access to files that match the specified file types. The Condition block uses the StringLikeIfExists operator to allow public-read access for file types like .html, .gif, *.jpg, and so on.
The second statement with the Sid "DenyPublicReadForSpecificKeySuffix" denies public-read access to files in the "private-eyes/*" path. By specifying the key prefix as "private-eyes/" in the Resource property, any files within the "private-eyes" directory or its subdirectories will be denied public-read access.
Just a note here, you may be used to configuring access by mime-type, as I mentioned about IIS. There is no way to define a policy by mime-type in AWS S3. If you want to use a mime-type approach you could map mime-types to file extensions to create the policy. There are several mime-type npm modules available to help with this.
By utilizing this modified bucket policy, you can ensure that specific file types are publicly accessible while restricting access to files in the "private-eyes" path, maintaining control over your website's security and access permissions.
Feel free to customize the file types and key suffixes based on your specific requirements.
Conclusion
In conclusion, the recent changes made by AWS to S3 bucket permissions have brought about stricter default security settings, highlighting the importance of implementing proper access control and security measures for static websites hosted on S3 buckets. With the introduction of S3 Block Public Access and the deprecation of ACLs, the landscape of bucket access and website accessibility has undergone significant transformations.
The default private nature of S3 buckets now provides enhanced security by preventing unintended public access to website content. However, this shift has posed challenges for developers seeking to make specific files publicly accessible on their static websites. While ACLs are no longer applicable for file-level access control during upload, there are alternative solutions available.
One such solution is the utilization of CloudFront signed URLs, which offer a secure and controlled method of granting temporary access to specific files. By generating signed URLs, developers can define the exact expiration time, permissions, and access restrictions for each URL, allowing fine-grained control over file access.
While other approaches like signed cookies also exist, they may not be suitable for public websites due to their limitations. Therefore, applying a custom access IAM policy to the bucket is the recommended approach to achieving the desired level of control and security.
It's important to note that the examples and approach presented in this article specifically target Node.js and utilize the official AWS Node.js SDK. If you're working with a different programming language, it's advisable to consult the documentation specific to that SDK. Although the documentation may not explicitly address this particular issue, you can adapt the solution presented here to suit your requirements.
In summary, while the changes in AWS's S3 bucket permissions bring new considerations for hosting static websites, the solution of leveraging CloudFront signed URLs provides an effective means of uploading non-publicly accessible files while maintaining robust security measures. By understanding and harnessing these tools, developers can ensure the successful deployment and management of static websites on AWS S3 buckets.
FAQ
Q: Can I still make my entire static website publicly accessible using the new default bucket security settings?
Answer: Yes, you can still make your entire static website publicly accessible by properly configuring the S3 bucket's website settings. The default security settings primarily focus on preventing unintended public access to individual files within the bucket.
Q: How do I set the bucket policy to make all files in my S3 bucket publicly accessible?
A: To make all files in your S3 bucket publicly accessible, you can configure the bucket policy to allow the s3:GetObject action for the * principal on the bucket's resources. This effectively grants public read access to all files in the bucket.
Q: Can I restrict public access to specific file types in my S3 bucket?
A: Yes, you can define the allowed file types in the bucket policy by specifying conditions based on the object's key suffix or prefix. For example, you can allow public read access only to files with the .html or .css extensions by using the appropriate conditions in your bucket policy.
Q: How can I prevent public access to certain directories or paths within my S3 bucket?
A: To restrict public access to specific directories or paths within your S3 bucket, you can use the bucket policy to deny public access to objects matching a certain key prefix or suffix. For instance, you can deny public access to all files in the "private/" directory by specifying a deny statement in the bucket policy.
Q: What if I want to allow public access to some files but keep others private within the same S3 bucket?
A: In such cases, you can use a combination of allow and deny statements in your bucket policy to control public access to different files based on their key prefixes or suffixes. By specifying specific conditions and matching patterns, you can finely tune the access permissions for various files within your bucket.
Q: Can I apply the bucket policy to an existing S3 bucket or only during the bucket creation process?
A: You can apply or modify the bucket policy at any time, regardless of whether the bucket already exists or is being created. Using the AWS Management Console, AWS CLI, or SDKs, you can update the bucket policy settings to configure public access permissions for your existing S3 bucket without affecting the stored objects or website configuration.