ci: apply concurrency limit for github workflows

[mscheong01]

Right now, if we push multiple commits, all our CI workflows pile up in the queue, causing a bit of a jam. We can cut down on this by setting up a concurrency limit. This way, any new workflow that hits the same branch will cancel the previous ones still in queue. But, if we don't want this to happen with our master branch workflows, we can make an exception. Here's how we could set it up:

concurrency: 
  group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event.inputs.sha }}
  cancel-in-progress: true

This setup checks if the workflow is targeting the master branch. If it is, it assigns a unique group name to avoid cancellation. Otherwise, it groups workflows by their name and the branch they're targeting, cancelling any in-progress ones when a new one starts. Let me know if this adjustment is required.

more info: https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#concurrency

[phymbert]

@ggerganov This is a must have, it's almost impossible to merge something during EMEA business hours 😅

[phymbert]

Hi @mscheong01 @ggerganov, since we introduced this change, master builds are also cancelled every time we merge. I think we need to introduce an exception for the master branch. I believe every commit here should have a complete CI check.

@mscheong01 Can you please have a look ?

[mscheong01]

@phymbert
using this setting instead of the current one will exempt the master branch workflows from being cancelled:

concurrency: 
  group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event.inputs.sha }}
  cancel-in-progress: true

I'll get this tested on my fork tomorrow just in case and open a PR.

[phymbert]

Thanks, would you please mind to open a PR ?

[mscheong01]

@phymbert I've tested this on my fork and opened a new PR #6486 😄