Introduce randomness into the scheduler.

[urbanslug]

Environment

All

Expected behavior

We want to be able to message different contacts within the same group at different and random times. Moreover, we would like to stagger the number of messages per week to something like 3 messages a week.

Actual behavior

Messages can only be sent out at a fixed time every day as a minimum.

---

Implementation plan

• Update rapidpro code or write a script to make celery run the trigger task at different times for each user in a group.
• Make changes to the RapidPro UI that reflects this functionality

[urbanslug]

Karma:

Problem

I think it would be wasting effort to improve schedules for the current bot but rather go straight ahead to working on bot 1 and 2 and how to schedule them. This is because the time it will take to build out the schedules for the current iteration which I estimate will be by end of this week and to be deployed by the beginning of the next one. Which will then overlap with the schedules for the next iteration which should be done by the end of next week. I tend to avoid Friday deploys.

So with this in mind let me give my implementation plan for what I feel has most value (scheduling of bot 1 and bot 2):

• We register a start date for each user consider this day 0.
• For Bot 1 we want to message them x3 a week and for Bot 2 x1 week this translates to 4 conversations a week. 4 doesn't fit well into 7 day weeks so let's consider 9 day periods to be weeks instead of 7 day periods since the first interaction.
• Bot 1 message 0 days (the initial one), 3 days and 6 days after the first interaction then Bot 2 will message them 9 days after the first interaction.
• We then run this over 45 days for each user with 3 day intervals and 9 day iteration periods which translates to 16 total interactions. 12 interactions of Bot 1 and 4 interactions of Bot 2.

Okay that out of the way let's talk about scheduling of the messaging hour/time. We have randomly generated the time for each of the 16 interactions. Which is indistinguishable from if they were generated randomly in real time. For example: message them at 10 am after 3 days, 8 pm after 6 days, 2 pm after 9 days, 1pm after 12 days and so forth.

Since there will be be 16 total interactions (I'm ignoring the initial non scheduled one) and we have a 12 hour period from 09:00 to 21:00 in which to message them we can initiate 13 interactions using the 13 hours from 09:00 to 21:00 and still have 3 interactions to repeat hours that we feel have a lot of value.

https://docs.google.com/a/ona.io/spreadsheets/d/1i3FdOH400MSEU0tVkm6y5-co1A8o9t68CdHw3hhYe4I/edit?usp=sharing

Solution

Use RapidPro campaigns

Rationale

1. Requires no extra code, deployment or configuration
2. We already have our users in RapidPro groups
3. We can use the registration_date contact field to determine the start date for initiating messages to users
4. Given we know the messaging intervals we can generate random time between 09:00 and 21:00 to set for each campaign event.

[urbanslug]

Scheduling engineering implementation

We want to be able to message users in a predetermined schedule (intervals) or/and at a time of day that they or we specify (think of a setting reminder).

Real time random scheduling in terms of time of day is indistinguishable from predetermined random scheduling as long as the messaging interval is not random. Otherwise, random intervals can also be p regenerated and then saved in which case they are also indistinguishable than if they were generated in real time.

In other words, real time random scheduling is either impossible or not worth the effort.

Problem/current requirements

Karma

Has two conversations.

Conversation 1	Conversation 2
x3 per week	x1 per week

Sofia

Has 5 conversations aimed at different users entirely. Schedules not yet determined.

FCI.	PSED	DMC III	SRQ 20	Caregiver Knowledge	Procedural Memory	Pre-academic skills

Possible future requirements

Have users set the time that they want to be messaged (like setting a reminder).

Implementation ideas

What can be used:

1. RapidPro
2. A scheduling tool from the internet
3. Writing our own scheduling tool/script

The scheduling tool should work with our contact/user management tool which for now is RapidPro

RapidPro

Use RapidPro to manage not only the users but the messaging schedules.
We do this by creating a flow with one node which makes a webhook request to our bot.

Could there be something better than scheduling a flow?

Advantages

• it already manages our contacts
• It's easy to use
• We have a fairly good understanding of RapidPro at Ona.

Drawbacks

• RapidPro is not a scheduler and scheduling is not a core RapidPro objective
• It only offers two ways to do scheduling:
  1. Triggers
  2. Campaigns

Triggers

• They can repeat a flow:
  • Monthly
  • Weekly
  • Daily
• They can run for a specific user or a group which is a good thing.
• You can't set them using the RapidPro API, that is, there's no API endpoint to perform CRUD on triggers.

Campaigns

Campaigns are a RapidPro feature that allows you to run a flow over a group.
Campaigns use campaign events. These are the specific times to message contacts.

• There's a CRUD endpoint to manipulate them
• You can set a campaign event based on contact fields such as registration_date
• Run only on groups
• You have to create a campaign event for each time you want to send a message

A scheduling tool from the internet

Writing our own scheduling tool

Requirements/good to have

• API only
• cron job
• Can set work for a specific user or a group of users
• Can integrate well with a user management tool such as RapidPro contacts

Components

• Celery
• Redis key value store of users and when to message them

Redis scheduler DB schema

Key	Value
RapiPro UUID	Time and Status Object { time: <datetime> status: <lapsed/active> }

Redis -> Celery -> RapidPro

Platform permissions

Messenger

We need pages_messaging_subscriptions permission to initiate messaging with user more than once in 24 hours for now we are able to use pages_messaging.

Facebook permissions:

• pages_messaging: Enables your app to send and receive messages using a Facebook Page.
• pages_messaging_subscriptions: Enables your app to send messages using Facebook Pages at any time after the first user interaction. This permission can't be used to send advertising or promotional content.
• pages_messaging_phone_number: (USA only) Enables your app to match phone numbers you already have to Messenger accounts. In order to use this permission, your app must be already approved for the Send/Receive API and based in the US.

[urbanslug]

Custom scheduling tool flowchart:

[pld]

This is a great write-up 😄, it sounds like we're committing to using RapidPro campaigns, but to use them for our particular use-case we'll need to write code on our end to manage communicating with the campaign API.

I.e. we'll still be writing something that creates campaign events for a group based on a schedule, so aren't we still responsible for writing that scheduler? And then I'm not sure if the answer is yes, and the parts about the custom scheduler are describing how we'd build that, which something that uses the RapidPro campaigns

[urbanslug]

Yes!

For Karma we'll use the pre-determined schedule going straight down to create RapidPro campaign.

[pld]

Cool, so we need to add some details, or a separate implementation issue describing how the tool that creates campaigns based on the pre-determined schedule will work