Skip to content

Add arbitrary key/value data store to the nflog#4899

Merged
SuperQ merged 4 commits intoprometheus:mainfrom
Spaceman1701:feature/nflog-data
Jan 28, 2026
Merged

Add arbitrary key/value data store to the nflog#4899
SuperQ merged 4 commits intoprometheus:mainfrom
Spaceman1701:feature/nflog-data

Conversation

@Spaceman1701
Copy link
Contributor

This change adds a new receiver_data field to the nflog entry protobuf message. receiver_data is a string -> int|double|string map.

Integrations can interact with receiver data using the new nflog.Store struct which acts like a mutable persistent key/value store. Every time the notifier is called, the context contains a *nflog.Store with any data set by previous runs of the notifier. Since the data is sourced from the nflog, everything works as expected in HA mode and through restarts.

I've chosen to support single key/value pairs (as opposed to key -> bytes or just a raw bytes field to reduce complexity for the notifier. Nflog entries are persisted through restarts and upgrades, so it's important to avoid breaking backwards compatibility. Short of making every integration specify its own proto, we don't have a good way to avoid compatibility breaks. I think keeping the scope of custom data more limited will help discourage storing complicated structures in the nflog.

The nflog store is automatically cleared after an nflog message containing only resolved alerts. This allows "new" notifications to start from a fresh store. If a notifier doesn't mutate the store, the old values are persisted into the next nflog entry.

Using the store is optional and easy:

func (n *MyNotifier) Notify(ctx context.Context, alerts ...*types.Alert) (bool, error) {
    // store is always set
    store := notify.NFLogStore(ctx)
    
    persisted_data, ok := store.GetStr("something_useful")
    if !ok {
        persisted_data = n.generateNewPersistedData()
        store.SetStr("something_useful", persisted_data)
    }
   
    // use persisted data when sending the notification
}

In HRT, we use this system for storing the first notification's Slackthread_ts in our slack notifier. This allows future notifications to update the existing message or respond in the thread. I haven't included any uses of the store in this PR since HRT's slack notifier is a whole different can of worms.

@Spaceman1701 Spaceman1701 changed the title Add arbitrary key/value data store the nflog Add arbitrary key/value data store to the nflog Jan 18, 2026
Signed-off-by: Ethan Hunter <ehunter@hudson-trading.com>
Copy link
Contributor

@siavashs siavashs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added minor suggestions, but otherwise LGTM 👍
I also ran some code analysis on the tests and the recommendation was to add the following test cases:

// Test: Transition from resolved-only to firing alerts should clear receiver data
// Test: Entry with firing alerts should preserve receiver data  
// Test: Verify NflogStore contains expected data based on entry state

@Spaceman1701
Copy link
Contributor Author

Added minor suggestions, but otherwise LGTM 👍 I also ran some code analysis on the tests and the recommendation was to add the following test cases:

// Test: Transition from resolved-only to firing alerts should clear receiver data
// Test: Entry with firing alerts should preserve receiver data  
// Test: Verify NflogStore contains expected data based on entry state

I think I do have tests for each of these:

  • Transition from resolved-only to firing alerts should clear receiver data -> TestNFLogStore_NoLeakBetweenNotificationSequences
  • Entry with firing alerts should preserve receiver data -> TestReceiverData_PreserveWhenNotifierDoesNotUpdate and TestDedupStageExtractsReceiverData_*
  • Verify NflogStore contains expected data based on entry state -> TestDedupStageExtractsReceiverData_*

But I'm very happy to add some more test cases if something's missing here

Signed-off-by: Ethan Hunter <ehunter@hudson-trading.com>
Signed-off-by: Ethan Hunter <ehunter@hudson-trading.com>
Signed-off-by: Ethan Hunter <ehunter@hudson-trading.com>
@SuperQ SuperQ merged commit b5df726 into prometheus:main Jan 28, 2026
7 checks passed
@SoloJacobs SoloJacobs mentioned this pull request Jan 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants