Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AgentOps Runtime Logging Implementation #2682

Open
wants to merge 89 commits into
base: main
Choose a base branch
from

Conversation

bboynton97
Copy link
Collaborator

@bboynton97 bboynton97 commented May 14, 2024

Why are these changes needed?

This PR adds AgentOps as an optional integration with autogen

  • Extends the Runtime Logger to support additional logging calls
  • Adds a source field to a number of logging functions
  • Adds agentops_logging.py for agent observability
  • A few type issues
  • A few typos

Checks

@Hk669
Copy link
Collaborator

Hk669 commented May 25, 2024

As mentioned by @ekzhu , it will be difficult for us to maintain this in our codebase. Instead, we can pass this as a custom logger.

Having a tutorial on how to pass this as a custom logger would make sense. Thanks

cc @sonichi

@bboynton97
Copy link
Collaborator Author

Can i get permanent approval on workflows for this PR? :)

@bboynton97
Copy link
Collaborator Author

As mentioned by @ekzhu , it will be difficult for us to maintain this in our codebase. Instead, we can pass this as a custom logger.

Agreed on this @Hk669. I moved the logger to our SDK and now it's passed in outside of the LoggerFactory.

website/docs/tutorial/tool-use.ipynb Outdated Show resolved Hide resolved
website/docs/topics/llm-observability.md Outdated Show resolved Hide resolved
website/docs/topics/llm-observability.md Outdated Show resolved Hide resolved
website/docs/topics/llm-observability.md Outdated Show resolved Hide resolved
website/docs/topics/llm-observability.md Show resolved Hide resolved
autogen/oai/client.py Outdated Show resolved Hide resolved
website/docs/topics/llm-observability.md Outdated Show resolved Hide resolved
website/docs/topics/llm-observability.md Outdated Show resolved Hide resolved
website/docs/topics/llm-observability.md Outdated Show resolved Hide resolved
website/docs/tutorial/tool-use.ipynb Outdated Show resolved Hide resolved
@codecov-commenter
Copy link

codecov-commenter commented May 31, 2024

Codecov Report

Attention: Patch coverage is 42.42424% with 19 lines in your changes are missing coverage. Please review.

Project coverage is 19.28%. Comparing base (f9d3fda) to head (a4e01a5).
Report is 4 commits behind head on main.

Files Patch % Lines
autogen/logger/file_logger.py 33.33% 6 Missing ⚠️
autogen/logger/sqlite_logger.py 37.50% 5 Missing ⚠️
autogen/runtime_logging.py 42.85% 4 Missing ⚠️
autogen/agentchat/conversable_agent.py 33.33% 2 Missing ⚠️
autogen/oai/client.py 0.00% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #2682       +/-   ##
===========================================
- Coverage   33.88%   19.28%   -14.61%     
===========================================
  Files          87       87               
  Lines        9422     9433       +11     
  Branches     2011     1973       -38     
===========================================
- Hits         3193     1819     -1374     
- Misses       5961     7503     +1542     
+ Partials      268      111      -157     
Flag Coverage Δ
unittests 19.27% <42.42%> (-14.62%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants