[Bug]: get_operation_guided prompt asks to review OpenAPI spec but does not provide it

[OwenMaine]

What happened?

The code sends a prompt to the LLM saying: "Review the OpenAPI specification and infer the operation ID.It never actually sends the OpenAPI specification in that message. The LLM receiving this request is "blind." It doesn't have the spec in its context window. It will have to rely on its pre-trained knowledge of Paystack (which might be outdated) or simply hallucinate.
To fix this I propose you need to include a summarized list of endpoints in the prompt or use an embedding search (RAG) to find the relevant operation.

Version

1.0.2 (Default)

Relevant log output

Code of Conduct

• I agree to follow the project's Code of Conduct

[tolu-paystack]

Hey @OwenMaine, it's actually not "blind". The spec is long, and rather than load it in the context window each time, we added functions that let the LLM fetch the relevant parts of the spec when it needs to.

There's a resource that gets the list of available operations from the spec file and all the logic for handling the OpenAPI spec can be found in openapi-parser.ts.

[OwenMaine]

Hey @OwenMaine, it's actually not "blind". The spec is long, and rather than load it in the context window each time, we added functions that let the LLM fetch the relevant parts of the spec when it needs to.

There's a resource that gets the list of available operations from the spec file and all the logic for handling the OpenAPI spec can be found in openapi-parser.ts.

Hey @tolu-paystack get_paystack_operation_guided tool does not use the Client LLM directly. It uses an MCP feature called sampling/createMessage.

When sampling/createMessage is called, the MCP server asks the connected AI to start a brand new, isolated inference.

In your code, the only thing passed into that new inference is the text string: "Review the OpenAPI specification and infer., it do not pass the OpenAPI spec into that specific message array, nor do it pass the tool definitions into the sampling parameters.

Therefore, the LLM answering that specific sub-query truly is blind."It cannot magically reach out and use the paystack_operation_list resource because it hasn't been given the context or the tools to do so within that isolated sampling request.

[tolu-paystack]

@OwenMaine sorry for the initial confusion. I was thinking about get_paystack_operation (the non-guided version), which does work as intended but you clearly specified get_paystack_operation_guided.

I've reviewed the code and you're absolutely right. The sampling/createMessage call creates an isolated inference context, and we aren't passing the user's request nor are we passing the spec file.

I've shared this with the team and suggested we remove the tool entirely. get_paystack_operation works as-is, since it has the operation IDs and descriptions included in the tool description now. Will update this issue as we decide on the path forward. Thanks for raising this!

[OwenMaine]

@OwenMaine sorry for the initial confusion. I was thinking about get_paystack_operation (the non-guided version), which does work as intended but you clearly specified get_paystack_operation_guided.

I've reviewed the code and you're absolutely right. The sampling/createMessage call creates an isolated inference context, and we aren't passing the user's request nor are we passing the spec file.

I've shared this with the team and suggested we remove the tool entirely. get_paystack_operation works as-is, since it has the operation IDs and descriptions included in the tool description now. Will update this issue as we decide on the path forward. Thanks for raising this!

@tolu-paystack you are welcome, should the team need assistance in orchestrating the idea I am here to help, Great initiative doing this, excited for the ease it would bring .