Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* @ABMC831 @OlivieFranklova @Zejnilovic
25 changes: 25 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
name: Bug report
about: Create a report to help us improve
labels: 'bug'

---

## Describe the bug
A clear and concise description of what the bug is.

## To Reproduce
Steps to reproduce the behavior OR commands run:
1. Go to '...'
2. Click on '....'
3. Enter value '...'
4. See error

## Expected behavior
A clear and concise description of what you expected to happen.

## Screenshots
If applicable, add screenshots to help explain your problem.

## Additional context
Add any other context about the problem here.
16 changes: 16 additions & 0 deletions .github/ISSUE_TEMPLATE/epic_task.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
name: Epic
about: A larger task consisting of more deliverables
labels: 'epic'
---

## Background
A clear and concise intro into the situation.

## Goal
The goal that epic wants to achieve.

```[tasklist]
### Tasks
- [ ] Add a draft title or issue reference here
```
21 changes: 21 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
name: Feature request
about: Suggest an idea for this project
labels: 'enhancement'

---

## Background
A clear and concise description of where the limitation lies.

## Feature
A description of the requested feature.

## Example [Optional]
A simple example if applicable.

## Proposed Solution [Optional]
Solution Ideas:
1.
2.
3.
13 changes: 13 additions & 0 deletions .github/ISSUE_TEMPLATE/question.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
name: Question
about: Ask a question
labels: 'question'

---

## Background [Optional]
A clear explanation of the reason for raising the question.
This gives us a better understanding of your use cases and how we might accommodate them.

## Question
A clear and concise inquiry
36 changes: 36 additions & 0 deletions .github/ISSUE_TEMPLATE/spike_task.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
---
name: Spike
about: Issue template for spikes, research and investigation tasks
labels: 'spike'

---

## Background
A clear and concise description of the problem or a topic we need to understand.

Feel free to add information about why it's needed and what assumptions you have at the moment.

## Questions To Answer

1.
2.
3.

## Desired Outcome

The list of desired outcomes of this spike ticket.

```[tasklist]
### Tasks
- [ ] Questions have been answered or we have a clearer idea of how to get to our goal
- [ ] Discussion with the team
- [ ] Documentation
- [ ] Create recommendations and new implementation tickets
- [ ] item here..
```

## Additional Info/Resources [Optional]

1.
2.
3.
64 changes: 63 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,63 @@
# EventGate
# EventGate
Python lambda for sending well-defined messages to confluent kafka
assumes AWS Deployment with API Gateway exposure of endpoint

<!-- toc -->
- [Lambda itself](#lambda-itself)
- [API](#api)
- [Config](#config)
- [Terraform Deplyoment](#terraform-deplyoment)
- [Scripts](#scripts)
<!-- tocstop -->

## Lambda itself
Hearth of the solution lies in the Src folder

## API
POST 🔒 method is guarded by JWT token in standard header "bearer"

| Method | Endpoint | Info |
|---------|-----------------------|------------------------------------------------------------------------------|
| GET | `/token` | forwards (HTTP303) caller to where to obtain JWT token for posting to topic |

| GET | `/topics` | lists available topics |
| GET | `/topics/{topicName}` | schema for given topic |
| POST 🔒 | `/topics/{topicName}` | posts payload (after authorization and schema validation) into kafka topic |
| POST | `terminate` | kills lambda - useful for when forcing config reload is desired |


## Config
There are 3 configs for this solution (in conf folder)

- config.json
- this one needs to live in the conf folder
- defines where are other resources/configs
- access.json
- this one could be local or in AWS S3
- defines who has access to post to individual topics
- topics.json
- this one could be local or in AWS S3
- defines schema of the topics, as well as enumerates those


## Terraform Deplyoment
Whole solution expects to be deployed as lambda in AWS,
there are prepared terraform scripts to make initial deplyoment, and can be found in "terraform" fodler
All that is needed is supplementing variables for
- aws_region
- vpc_id
- resource prefix - all terraform resources would be prefixed my this prefix, usefull when mixed-in with something else
- lambda_role_arn - the role for the lambda, should be able to make HTTP calls to wherever kafka server lives
- lambda_vpc_subnet_ids

Once tfvars are supplied, go terraform apply and you are done

## Scripts
Useful scripts for dev and Deployment

### Notebook
Jupyter notebook, with one cell for lambda initialization and one cell per method, for testing purposes
Obviously using it requires correct configs to be in place (PUBLIC key is being loaded during initilization)

### Preapare Deployment
shell script for fetching pithon requirements and ziping it together with sources and config into lambda archive, ready to be used by terraform
4 changes: 2 additions & 2 deletions conf/topics.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
"description": "Environment",
"type": "string"
},
"id": {
"guid": {
"description": "Unique identifier for the event (GUID)",
"type": "string"
},
Expand Down Expand Up @@ -76,7 +76,7 @@
"description": "Format of the data",
"type": "string"
},
"id": {
"guid": {
"description": "Unique identifier for the event (GUID)",
"type": "string"
},
Expand Down
8 changes: 8 additions & 0 deletions notebook.ipynb → scripts/notebook.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,14 @@
"source": [
"%load_ext autoreload\n",
"%autoreload 2\n",
"\n",
"# Jump out of the \"Scripts\" box for the lambda source\n",
"import sys\n",
"import os\n",
"parent_dir = os.path.abspath(os.path.join(os.getcwd(), os.pardir))\n",
"sys.path.insert(0, parent_dir)\n",
"if not os.path.exists('src'):\n",
" os.chdir('..')\n",
"import src.event_gate_lambda"
]
},
Expand Down
1 change: 1 addition & 0 deletions prepare.deplyoment.sh → scripts/prepare.deplyoment.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
cd ..
pip3 install -r src/requirements.txt -t dependencies/ --platform manylinux2014_x86_64 --python-version 3.12 --only-binary=:all:
zip -r lambda_function.zip conf
cd src
Expand Down