Skip to content
This repository was archived by the owner on Dec 11, 2025. It is now read-only.
This repository was archived by the owner on Dec 11, 2025. It is now read-only.

Allow feedback submission without requiring .create() completion call (manual session/session_id registration) #91

@ccocks

Description

@ccocks

Currently, the Not Diamond SDK requires calling client.chat.completions.create() to generate a session_id before submitting feedback via metric.feedback(). This tightly couples feedback submission to completion generation through the SDK or API.

In many real-world use cases, users may already have completions generated externally (e.g., from self-hosted models, other APIs, or offline inference) and want to:

Submit feedback on those completions

Use Not Diamond’s routing and feedback learning without invoking .create() or re-generating completions

Avoid wasting compute/time by duplicating requests just to get session IDs

Current limitation:
Feedback submission requires a valid session_id, which currently can only be obtained by calling .create(). There is no public API or SDK method to:

Manually register a session (session ID + model/provider info)

Submit feedback linked to that manually registered session

Proposed solution:
Add support for:

A new SDK/API method to manually create/register sessions with arbitrary metadata, allowing users to associate existing completions with sessions.

Allow submitting feedback referencing these manually registered sessions without calling .create().

Possibly support a “dry-run” or “session-only” mode in .create() that skips actual generation but returns a valid session ID.

Benefits:

Enables users to integrate Not Diamond feedback with completions generated outside the SDK/API.

Avoids duplicate generation calls and wasted compute.

Makes Not Diamond routing usable in more diverse real-world scenarios (self-hosted, batch-generated, multi-cloud, etc.).

Example use case:
I have a custom LLM inference pipeline outside Not Diamond. I want to use ND’s routing feedback system without re-generating completions. Currently, I have to call .create() anyway to get a session ID, which causes wasted calls and costs.

Additional context:
Code inspection shows that metric.feedback() requires a session_id obtained only from .create(). There is no public API to register a session independently.

Thank you!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions