Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

expose SRIOV information as PCI devices #315

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

ffromani
Copy link
Collaborator

@ffromani ffromani commented May 1, 2022

Expose informations about SRIOV devices. After some more design iterations (#230 (comment)), we fixed these goals:

  • model SRIOV devices as special PCI devices, avoiding a new top-level package
  • to shape the API, we start checking what the k8s SRIOV operator needs
  • to further refine later, we will check what other projects like the k8s node feature discovery consume
  • the goal is to enable such projects to switch to ghw if/when they want.

There are few more noteworthy items in this PR:

  • due to lack of resources, initial support is for linux only. Nothing linux-specific should have sneaked in the API. Should the current API design make unnecessarily hard to add support on other platforms (e.g. windows) this should be treated as bug
  • the code prefers Physical Functions for discovery, but we acknowledge that forcing consumers to traverse the physical functions to learn about virtual functions is awkward, so the API provides shallow references to the Virtual Functions for the sake of practicality.
  • this is an alternate implementation of issue Grab information on VFs for SR-IOV PFs #92 , conflicting with expose SRIOV information #230

Fixes: #92

Signed-off-by: Francesco Romani fromani@redhat.com

@ffromani ffromani mentioned this pull request May 1, 2022
@ffromani ffromani changed the title expose SRIOV information as PCI devices WIP: expose SRIOV information as PCI devices May 1, 2022
@ffromani
Copy link
Collaborator Author

ffromani commented May 1, 2022

this PR is reviewable, but tests are still WIP. The alternate API is pretty much stable though.

@jaypipes
Copy link
Owner

@fromanirh did you want to rebase this and have it reviewed?

@ffromani
Copy link
Collaborator Author

@fromanirh did you want to rebase this and have it reviewed?

Sure, I'll rebase shortly! other than that this is not very urgent. Up to us if we like more this direction or the original one in #230

@ffromani ffromani changed the title WIP: expose SRIOV information as PCI devices expose SRIOV information as PCI devices Jun 10, 2022
@ffromani
Copy link
Collaborator Author

note that most of the tests are missing - deferred until we settle about the direction. Should not be merged without tests, though.

Copy link
Owner

@jaypipes jaypipes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fromanirh apologies for the delayed review.

I think I would prefer to have the SRIOV-specific attributes added to the pkg/pci.Device struct itself instead of having an "extended" Function struct that derives from Device.

You could have a helper method on pkg/pci.Info called GetSRIOVDevices() that would return only the PCI devices that were SR-IOV capable. Something like this?

// GetSRIOVDevices returns only the PCI devices that are
// Single Root I/O Virtualization (SR-IOV) capable -- either
// physical of virtual functions.
func (i *Info) GetSRIOVDevices() []*Device {
    res := []*Device{}
    for _, dev := range i.Devices {
        if dev.Parent != nil || len(dev.Functions) != 0 {
            res = append(res, dev)
        }
    }
    return res
}

README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
cmd/ghwc/commands/sriov.go Outdated Show resolved Hide resolved
pkg/pci/function.go Outdated Show resolved Hide resolved
@ffromani
Copy link
Collaborator Author

ffromani commented Jul 6, 2022

@fromanirh apologies for the delayed review.

I think I would prefer to have the SRIOV-specific attributes added to the pkg/pci.Device struct itself instead of having an "extended" Function struct that derives from Device.

You could have a helper method on pkg/pci.Info called GetSRIOVDevices() that would return only the PCI devices that were SR-IOV capable. Something like this?

// GetSRIOVDevices returns only the PCI devices that are
// Single Root I/O Virtualization (SR-IOV) capable -- either
// physical of virtual functions.
func (i *Info) GetSRIOVDevices() []*Device {
    res := []*Device{}
    for _, dev := range i.Devices {
        if dev.Parent != nil || len(dev.Functions) != 0 {
            res = append(res, dev)
        }
    }
    return res
}

No worries @jaypipes and thank you for the review. I'm kinda reluctant to conflate even more data in the pci.Device struct (if we do than sooner or later we should also dissolve the gpu pkg into pci - which I'd like! :) ) but I don't feel so strongly to oppose this direction, so let's try and see how it looks.

@ffromani ffromani force-pushed the net-sriov-pci branch 2 times, most recently from 1e5efcf to c72700e Compare July 21, 2023 10:48
@ffromani
Copy link
Collaborator Author

@jaypipes eventually I come to like this approach. We may need a couple more iteration to sort out the details of the API (naming, printing) but overall I'm happy to pursue this direction. PR is updated, rebased and ready for review

@ffromani
Copy link
Collaborator Author

depends on #351

simplify the code without change in behaviour, using
a flow without `else` to reduce the blocks.

Signed-off-by: Francesco Romani <fromani@redhat.com>
README.md Outdated Show resolved Hide resolved
Add support to report SRIOV devices.
Differently from GPU devices, we model SRIOV devices as special
PCI devices, extending the `pci` package instead of introducing
a new top-level package.
This design emerged during the review of a previous proposal:
jaypipes@9058f61#r755312597

SRIOV devices are either Physical Functions or Virtual functions.
The preferred representation for ghw is Physical Functions, whose
dependent devices will be Virtual Functions; however, for the sake of
practicality, the API also exposes soft references to Virtual Functions,
so consumers of the API can access them directly and not navigating
the parent devices.

This patch also adds support in `ghwc`, to report the sriov information,
and in the `snapshot` package, to make sure to capture all the files
in sysfs that ghw cares about.

Last but not least, lacking access to suitable non-linux systems,
support is provided only on linux OS, even though the API tries hard
not to be linux-specific.

Resolves: jaypipes#92
Signed-off-by: Francesco Romani <fromani@redhat.com>
Comment on lines +912 to +913
Virtual Functions are available both as entries in the `pci.Functions` slice and as properties of their parent Physical Functions.
Both references are aliases to the same object.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yuck, missed to update the rest of the docs :\

@ffromani
Copy link
Collaborator Author

Now that we agreed on the direction, I need to add tests. Will do ASAP.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Grab information on VFs for SR-IOV PFs
2 participants