Skip to content

Remove get_paystack_operation_guided tool with broken inference context#16

Merged
Andrew-Paystack merged 3 commits intomainfrom
copilot/fix-guided-prompt-issue
Feb 20, 2026
Merged

Remove get_paystack_operation_guided tool with broken inference context#16
Andrew-Paystack merged 3 commits intomainfrom
copilot/fix-guided-prompt-issue

Conversation

Copy link
Contributor

Copilot AI commented Feb 20, 2026

The get_paystack_operation_guided tool creates an isolated sampling/createMessage context that asks the LLM to "Review the OpenAPI specification and infer the operation ID" but never provides the spec or user request. The LLM receives no context and must hallucinate.

Changes

  • Removed src/tools/get-operation-guided.ts
  • Removed registration from src/tools/index.ts

Unaffected Functionality

The working get_paystack_operation tool remains and already includes all operation IDs in its tool description, providing proper context for operation selection without requiring a separate inference step.

// The non-guided version that continues to work correctly
server.registerTool(
  "get_paystack_operation",
  {
    description: `Get Paystack API operation details by operation ID. Available
    operations are:
    ${Object.keys(operations).map(operation => operation).join(", ") }`,
    // ... operation IDs visible to LLM in description
  }
)
Original prompt

This section details on the original issue you should resolve

<issue_title>[Bug]: get_operation_guided prompt asks to review OpenAPI spec but does not provide it</issue_title>
<issue_description>### What happened?

The code sends a prompt to the LLM saying: "Review the OpenAPI specification and infer the operation ID.It never actually sends the OpenAPI specification in that message. The LLM receiving this request is "blind." It doesn't have the spec in its context window. It will have to rely on its pre-trained knowledge of Paystack (which might be outdated) or simply hallucinate.
To fix this I propose you need to include a summarized list of endpoints in the prompt or use an embedding search (RAG) to find the relevant operation.

Version

1.0.2 (Default)

Relevant log output

Code of Conduct

  • I agree to follow the project's Code of Conduct</issue_description>

<agent_instructions>remove the get_operation_guided while ensuring any related actions and functions aren't broken. Raise the related functions for my review

</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@tolu-paystack Hey @OwenMaine, it's actually not "blind". The spec is long, and rather than load it in the context window each time, we added functions that let the LLM fetch the relevant parts of the spec when it needs to.

There's a resource that gets the list of available operations from the spec file and all the logic for handling the OpenAPI spec can be found in openapi-parser.ts.

@tolu-paystack @OwenMaine sorry for the initial confusion. I was thinking about `get_paystack_operation` (the non-guided version), which does work as intended but you clearly specified `get_paystack_operation_guided`.

I've reviewed the code and you're absolutely right. The sampling/createMessage call creates an isolated inference context, and we aren't passing the user's request nor are we passing the spec file.

I've shared this with the team and suggested we remove the tool entirely. get_paystack_operation works as-is, since it has the operation IDs and descriptions included in the tool description now. Will update this issue as we decide on the path forward. Thanks for raising this!


🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

Copilot AI and others added 2 commits February 20, 2026 23:26
Co-authored-by: Andrew-Paystack <78197464+Andrew-Paystack@users.noreply.github.com>
Co-authored-by: Andrew-Paystack <78197464+Andrew-Paystack@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix get_operation_guided prompt by including OpenAPI spec Remove get_paystack_operation_guided tool with broken inference context Feb 20, 2026
@Andrew-Paystack Andrew-Paystack marked this pull request as ready for review February 20, 2026 23:36
@Andrew-Paystack Andrew-Paystack requested a review from a team February 20, 2026 23:36
@Andrew-Paystack Andrew-Paystack merged commit d462c99 into main Feb 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: get_operation_guided prompt asks to review OpenAPI spec but does not provide it

2 participants