Usage in an Express app

[tomblanchard]

Hey @willhoag - this looks like a nice package to make calls to the Shopify API. I'm developing a pretty simple Express app which has authentication, will I be able to use this package to give each user their own limit?

For example, if I open my app in two separate incognito windows with two different users authenticated and make a bunch of calls to the Shopify API (each user provides different API credentials), can I use this package to throttle the calls to stay within the Shopify API limits?

[willhoag]

ATM you'd have to make an instance per user.

Implementation might look something like this (not tested):

const queues = {}
const QUEUE_LIFE = 30 * 60 * 1000 // 30 mins
const JOB_FREQUENCY = 60 * 60 * 1000 // 1hr
const queueConf = {
  calls: 6,
  perInterval: 60 * 1000,
  maxConcurrent: 4,
  tickFrequency: 100
}

// JOB to clear old queues
setInterval(() => {
  const now = new Date()
  Object.keys(queues).forEach(id => {
    const q = queues[id]
    if (now - q.lastUsed > QUEUE_LIFE) {
      q.queue.stop()
      delete queues[id]
    }
  })
}, JOB_FREQUENCY)

// Get wrapped queue for users
const getUserQueue = (id) => {
  if (!queues[id]) {
    queues[id] = {
      queue: BucketQueue(queueConf).start(),
      lastUsed: new Date()
    }
  }

  return {
    add: (...props) => {
      const queue = queues[id]
      queue.lastUsed = new Date()
      return queue.add(...props)
    }
  }
}

getUserQueue(user.id).add(fetch, 'https://api.shopify.com/url', { method: 'POST' })

[tomblanchard]

Hey @willhoag - oh wow, thanks so much for that detailed example, that helps a lot!

I'm wondering how much a solution like this will scale though, I tried out a similar solution a few months back and a coworker told me this would fail in production due to the code being deployed to multiple CPU stores, causing each thread to contain a different singleton with different instances. I'm unsure of how accurate this is though, do you think a solution like this is production-ready?

He suggested that I opt for a database-backed rate limiter solution but I can't seem to find anything which supports both the Shopify "leaky bucket" limit algorithm and the ability to define separate limits per user.

I'm gonna try out your solution and do some stress-testing because I don't see any reason why it's not production-ready.

Cheers again mate.

[willhoag]

Yeah, technically the package itself is storing state in process memory, so if you spin your app up multiple times (whether on cpus or on machines w/ load balancing), you'll have multiple instances of this memory. You'd have to restrict users to the same process to maintain a single leaky bucket state for them.

Your friend has a good point. If you plan on scaling horizontally (cpu / machine), you might want to consider storing that state in an in-memory database like Redis or something, instead of in the process memory of a BucketQueue instance.

[tomblanchard]

@willhoag Oh right, that's good to know. I'm guessing I won't be able to use this package while storing the state in a DB? Do you happen to know of any other solutions which support both Shopify's "leaky bucket" limit algorithm and storing state in a DB such as Redis?

Thanks again for all your help, I'm more used to communicating with APIs via one-off migration scripts via the command line and whatnot where I don't have to worry about different users and scaling.

[willhoag]

I believe the leaky bucket packages were limited when I wrote this a couple years ago. Did a quick NPM search via: https://www.npmjs.com/search?q=leaky+bucket

And found a couple packages:
https://www.npmjs.com/package/dripping-bucket
https://www.npmjs.com/package/relimit

I haven't vetted these, but seem like they'd do the trick if you're looking for this kind of scaleability.