Manage Amazon S3 Buckets Using Event Notifications With AWS SQS

Recently I came across a problem where the admin team should get notified when some user deletes any object from S3 object. If you also have a similar requirement where the team should get notification whenever an object is created, deleted, or modified, this is where event notifications comes into the picture.

You can use AWS S3 Event notifications feature to receive notifications to following destinations when certain events happen in the AWS S3 bucket.

Amazon Simple Notification Service (Amazon SNS) topics
Amazon Simple Queue Service (Amazon SQS) queues
AWS Lambda function

In this article, I will be configuring AWS SQL queues as a destination to track delete event from my S3 bucket.

Configure AWS SQL queues to track delete event from S3 bucket

Step 1

Step 2

Go to S3 bucket and create a new bucket if it does not exist. I have already created a bucket with name ‘mydemobucket198’.

Step 3

Go to Properties.

Step 4

Scroll down to the page until section Event Notification appears. Click on Create event notification button.

Step 5

A window appears to fill details to create a new event notification. I have put ‘s3_notify’ as the event name.

Step 6

A list will appear for events that are currently supported like Object creation, Object removal, etc. I checked Object Removal for this article so whenever any user deletes any S3 object I will get a notification.

Step 7

Go to Destination section where you can choose a destination to publish the event.

I have chosen SQL queues so notification will be sent to SQS queues that can be read later by a server and perform action accordingly.

Choose SQL queues if it's already created or create a new one. As I don't have any existing SQL queues, I will create a new one.

Step 8

Go to Amazon SQS service and click on Create queue.

Step 9

Enter name of SQS queue as ‘s3_delete_notify’ and choose all settings as default.

Step 10

Once the SQS queue is created, go to Access Policy.

Step 11

We need to modify the Access Policy to send a message from S3 bucket when any object gets deleted.

Step 12

Click on Edit and paste below Access Policy code -

{
"Version": "2012-10-17",
"Id": "Policy1651140347168",
"Statement": [
{
"Sid": "Stmt1651140341677",
"Effect": "Allow",
"Principal": "*",
"Action": "sqs:SendMessage",
"Resource": "arn:aws:sqs:ap-south-1:464473132183:s3_delete_notify"
}
]
}

Don't forget to replace the ARN value with your SQL queue.

Step 13

Refresh the page and select the SQS queue that we have just created in the above steps. Click on Save Changes.

Step 14

Go to S3 bucket and upload some files.

Step 15

Select any file and click on Delete button. I have selected photo-1.jpeg and deleted that file.

Step 16

Go to SQS queue ‘s3_delete_notify’ and click on Send and receive message button.

Step 17

Click on Poll for messages button to pull messages.

Step 18

As you can see, selected ID is related to S3 object that has been deleted.

There is another ID which was created by AWS to test initially when queue was created.

Step 19

After you click on Message ID, you can see the details in Message Body. In our case, we can see S3 bucket name with S3 object name photo-1.jpeg which was deleted.

Conclusion

Using S3 Event notification you can enable notification for events like creating, deleting, and modifying S3 objects using destinations Amazon SNS, Amazon SQS and AWS Lambda.