How To Read Files From S3 Bucket Using Lambda Function And Python

Introduction

In this article, you will learn how to read the file from S3 bucket and return the JSON response using Lambda function with Python. For this article, I have uploaded a sample JSON file to S3 bucket and using the API endpoint, the results are returned in JSON format.

Sample.json

JSON file uploaded to S3 bucket, and this file contains the following content.

[
    {
        "Section": "Scope",
        "Category": "Goals",
        "#": 1,
        "Question": "What is being considered in scope and what is out of scope?",
        "Response Type": "Text"
    },
    {
        "Section": "Scope",
        "Category": "Goals",
        "#": 2,
        "Question": "What are the primary drivers of these objectives?",
        "Response Type": "Text"
    }
]

API Endpoint

https://1xxxxx2xx3.execute-api.us-east-1.amazonaws.com/poc/inputs?bucket=demo23012023&key=sample.json

Note: parameters bucket and key value need to be replaced with the correct values.

Output Response

JSON response.

Pre-requisites

1. Lambda function created. Refer to Building Lambda functions with Python
2. API Endpoint created. Refer to Build an API Gateway REST API with Lambda integration
3. Required permissions and roles created to access S3. Refer to Lambda execution role

Code Snippet

Update the code in lambda_function.py with the following.

import io
import boto3
import json

def lambda_handler(event, context):
    s3 = boto3.resource('s3')
    bucket = event['params']['querystring']['bucket']
    key = event['params']['querystring']['key']
    obj = s3.Object(bucket, key)
    body = obj.get()['Body'].read().decode('utf-8')
    return json.loads(body)

Click Deploy to update the changes in the Lambda function.

Test the API

Open Postman and execute the following API to get the results as shown below.

Summary

This article describes how to read the file from S3 bucket and return the JSON response using the Lambda function with Python.