You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Then at the end of each task, it got saved by Controller to the final State (e.g., state.costs: list[dict], maybe with a state.total_cost_usd: float that sums up the total cost) so we can have a better estimation of evaluation costs.
The text was updated successfully, but these errors were encountered:
Now, we only print out the costs of each LLM call.
It would be preferable if we could build this cost-logging mechanism into the
LLM
class itself (https://github.com/OpenDevin/OpenDevin/blob/main/opendevin/llm/llm.py) and then have it log the cost for eachchat_completion
response (alist[dict]
).Then at the end of each task, it got saved by
Controller
to the finalState
(e.g.,state.costs: list[dict]
, maybe with astate.total_cost_usd: float
that sums up the total cost) so we can have a better estimation of evaluation costs.The text was updated successfully, but these errors were encountered: