Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Inference] Expose metadata #524

Open
SBrandeis opened this issue Mar 1, 2024 · 1 comment
Open

[Inference] Expose metadata #524

SBrandeis opened this issue Mar 1, 2024 · 1 comment
Labels
enhancement New feature or request inference @huggingface/inference related widgets

Comments

@SBrandeis
Copy link
Contributor

The Inference API exposes some metadata about the inference computation via HTTP headers,
mainly:

  • x-compute-time
  • x-compute-type

Along with some task-specific info such as x-compute-characters for text-generation

It would be great to expose that metadata through the HfInference client. It will be especially useful when we integrate the inference client in widgets (#360 )
@SBrandeis SBrandeis added enhancement New feature or request inference @huggingface/inference related widgets labels Mar 1, 2024
@SBrandeis
Copy link
Contributor Author

cc @coyotte508 for viz

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request inference @huggingface/inference related widgets
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@SBrandeis