Skip to content

Report dollars spent on OpenAI#52

Merged
chenweize1998 merged 3 commits intoOpenBMB:mainfrom
kierangilliam:report-usage
Oct 9, 2023
Merged

Report dollars spent on OpenAI#52
chenweize1998 merged 3 commits intoOpenBMB:mainfrom
kierangilliam:report-usage

Conversation

@kierangilliam
Copy link
Copy Markdown
Contributor

The user does not have visibility into the cost of each session. This PR adds a report_metrics callback for environments to display useful metrics to the user.

This has been implemented for the TaskSolving environment. Try "brainstorming" to see the result.

$ python main_tasksolving_cli.py --task tasksolving/brainstorming
...
Agent spend:  
Agent (Role: AGENT_TYPES.ROLE_ASSIGNMENT) role assigner: $0.001631  
Agent (Role: AGENT_TYPES.SOLVER) A mechanical engineer specialized in hydrogen storage technologies: $0.01  
Agent (Role: AGENT_TYPES.CRITIC) An electrical engineer with expertise in renewable energy integration: $0.002690  
Agent (Role: AGENT_TYPES.CRITIC) A regulatory specialist familiar with hydrogen storage regulations: $0.002037  
Agent (Role: AGENT_TYPES.CRITIC) An infrastructure planner with experience in large-scale projects: $0.002518  
Agent (Role: AGENT_TYPES.EXECUTION) Dummy Executor: $0.000000  
Agent (Role: AGENT_TYPES.EVALUATION) Evaluator: $0.003592  
Total spent: $0.018993 

@kierangilliam kierangilliam marked this pull request as draft October 8, 2023 16:19
@kierangilliam kierangilliam marked this pull request as ready for review October 8, 2023 16:23
@chenweize1998
Copy link
Copy Markdown
Collaborator

Awesome! Thanks for your help

@chenweize1998 chenweize1998 merged commit abd444e into OpenBMB:main Oct 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants