AWS Lambda

This project is designed using AWS Lambda and AWS Cloudwatch for an automatic data collection and storage system with server availability monitoring.

Tech Stack

Language: Python
Libraries: requests,mysql-connector-python,boto3
SQL Database:: AWS RDS MySQL
Amazon Web Service: AWS Cloudwatch, AWS SNS, AWS lambda

Storing and Fetching data in AWS RDS MySQL Database

The data are fetched from - API using requests python library.The fetched data from API are then stored to AWS MySQL database using mysql-connector-python python library.

Database Configuration

Please replace with your own Username,Password,AWS Database host name and Database name string in /open-api/lambda_function.py file while running the code in your AWS lambda.

Slack Configuration

Please replace with your own Slack channel Webhook and Channel name string in /slacknotifier/lambda_function.py file while running the code in your AWS lambda.

Lambda Creation

Open Api folder lambda_function.py file is used to fetch data and store in AWS RDS MySQL database.

Slacknotifier folder lambda_function.py is used to send error message to Slack channel.

Important points to note:

1. While running your code in AWS Lambda, python libraries other than the few standard python libraries (example: boto3) need to be uploaded as folders along with your lambda function code.

2. Always keep your code file name as lambda_function.py and default function name as lambda_handler. If not, it needs to be changed in Runtime settings.

AWS Cloudwatch Event

The Event is configured to trigger Open API lambda function for every 1 minute

AWS Cloudwatch Alarm

This Alarm is configured to monitor the Open API lambda function for a period of every 1 minute and trigger AWS SNS if the Open API lambda function throws an error.

Amazon Simple Notification Service

The subscription is configured to send an email to a mail ID and also trigger the Slacknotifier lambda function to send messages to a Slack channel.

Error Messages

In AWS Cloudwatch Alarm

In SLACK Channel

In Email

WORKFLOW

1. AWS Cloudwatch Event trigger Open API lambda function for every 1 minute to fetch data from API and store it to AWS RDS MySql database.
2. AWS Cloudwatch Alarm will monitor Open API lambda function for a period of every 1 minute.
3. AWS Cloudwatch Alarm will trigger AWS SNS if Open API lambda function throws an error.
4. AWS SNS send an email to a mail ID and also trigger the Slacknotifier lambda function to send messages to a Slack channel.

Sample Results :

Data stored in My SQL database at 11:36 AM

Data stored in My SQL database at 11:37 AM

LinkedIn Post

• https://www.linkedin.com/posts/rjs-balajee-389a8215a_github-balajeerjsawslambda-activity-7052605552350154753-VOw8?utm_source=share&utm_medium=member_desktop