SQS functions

[mikeAdamss]

Not - Need AWS access before this can be picked up. Mike to show how to create and use a queue in bleed environment while developing.

What is this

When working with AWS we want a simple mechanism for sending message to and pulling messages from the AWS SQS system.

All we want for this task is a simple client to:

• send a msg to a queue
• receive a msg from a queue

We are assuming we want the message to be json, so in python terms we want to be able to:

(a) pass a dictionary into the queue
(b) get a dictionary back out of the queue

A key point here is that reading a message from a queue is non destructive, so for (b) we'll want to read the message -> get the dict -> delete the message from the queue (otherwise it'll persist and we'll end up doing the same task over and over again) -> give the developer the dict that they're after.

What to do

Note - use the boto3 sqs client for this.

There are some simple an well organised examples from AWS here.

• Remove the SNS directory here (SNS and SQS are slightly different things).
• Create a /awsmessaging/sqs directory with a main.py and the following functions in it.

def publish_message_dict(topic: str, msg: Dict):
       # The "topic" is the user friendly name for the queue, eg "files-that-need-proceessing". that kinda thing
       # Use the topic to get the the full url of the queue, boto3 should have a "get_queue" function of some kind for this.
       # Now you have a queue url and a dictionary you want to turn into a message - wrap up the boto3 sqs
       # example code (there will lots of examples if you google - we're aiming for a developer to be able to easily
       # send a dictionary to the queue without having to care about how it works.
       # At all points, catch errors and log (not raise) them if you get one. 
       
def receive_delete_message_dict(topic: str, wait_time: int) -> Dict:
      # This will largely be the "Receive messages from a queue" example except:
      # - we always want exactly one message 
      # - we want to use our own logger, and we don't want to raise an exception.
      # - confirm that the received object is the expected message class (whatever that is, you'll need to poke around)
      # - confirm that the message content is in the form of a dictionary
      # - get the dictionary out of the message
      # - delete the message from the queue (see "Delete a message from a queue" example
      # return the dictionary

Acceptance Criteria

• Can publish a dictionary to a queue
• Can read a dictionary from a queue (and delete the message in question)
• Tests to confirm these behaviours.