zypper does not have an equivalent to "dnf whatprovides"

[Firstyear]

On fedora and rhel you can use dnf whatprovides to locate a specific file that is provided by a package, even if that package is not installed.

zypper has no equivalent to this. zypper se -f seems to only search already installed packages.

An example of why this is useful is you can type "dnf whatprovides '*/ldapsearch'" and get an answer. There is, easy to access and obvious equivalent in zypper.

Having something like zypper whatprovides would be a great equaliser and accesibility tool.

[luc14n0]

Our zypper se -f works for not-installed packages too, but only for a couple of prefixes:

-f, --file-list

    Search in the file list of packages. Note that the
    full file list is available for installed packages
    only. For remote packages only an abstract of their
    file list is available within the metadata (files
    containing /etc/, /bin/, or /sbin/).

And we kind of have an equivalent to dnf whatprovides (whatprovides is an alias to dnf provides, while our what-provides, to zypper search --provides --match-exact):

what-provides (wp) capability

    List all packages providing the specified capability (case-
    insensitive search). See also the install command for info
    about specifying capabilities.
    The command line is automatically transformed into the
    corresponding search command:

        $ zypper what-provides 'zypper>1.6'
                $ zypper search --provides --match-exact 'zypper>1.6'

        For a case-sensitive search call

                $ zypper search --provides --match-exact --case-sensitive 'zypper>1.6'

I'm not sure, but I think DNF distros -- Fedora/CentOS/RHEL -- generates capabilities for each file in their packages, opposed to what we do, defining file provisions in OBS project configurations, like we see in openSUSE:Factory:

FileProvides: /bin/bash bash
FileProvides: /usr/bin/bash bash
FileProvides: /bin/sh bash-sh busybox-sh
FileProvides: /usr/bin/sh bash-sh busybox-sh
...

And I wonder why Zypper doesn't search for other locations as well, when the right package is not installed, aside from /etc, /bin and /sbin (we should update the man to append /usr appropriately there). We have the filelist.xml, containing what seems to be the full list of files, available in the repository metadata.

[luc14n0]

And I wonder why Zypper doesn't search for other locations as well, when the right package is not installed, aside from /etc, /bin and /sbin

Answering myself here, I hope. It seems that Zypper gets the filelist from libsolv, and that filelist is filtered. So, the search for files in non-installed packages was implemented, primarily, for dependency resolution of binary files, which makes sense. I'm probably missing something, though.

Nonetheless, if Zypper would implement searches for any file in non-installed packages, it has to use something different than what zypper search -f, to interact with filelist.xml, I suppose. That is, something different "behind the curtains", at least.

---

P.S.

But I wonder if something else would do too, a standalone script to parse the xml file wouldn't be that hard to come up with.

[mlandres]

The main reason is that we do not download/refresh the repositories filelists.xml (TW-oss ~48M for TW-oss ) and other.xml (~37MB for TW-oss; contains the changelogs) at all. Due to their size it is very expensive to keep them up-to-date.

We could think about a flag to include them if your bandwith is big enough. Or we could try to download them on demand only , though this would delay searches for files/chanelogs a lot. For listing changelog/files of a specific package we could download the packages .rpm header on demand and query this. But a free search in all filelists/changelogs will be expensive.

Currently we just see the files which are explicitly provided by package. Those provides are injected when building the repo metadata on the server and they live in the repos primary.xml . So the client (zypper) can actually not tell which directories (/etc/, /bin/,...) the server covers. OBS AFAIK focuses on files which may actually be required by other packages.

[Firstyear]

I think that if someone requests a what provides, you should download the complete metadata to answer that query. It's what yum/dnf have done in the same situation when you request a what provides.

Second, we should be able to search the full filesystem hierachy, especially because someone may be looking for libraries or similar. I have needed (and so have others) the ability to answer "what provides libblah.so" more than once.

[luc14n0]

The main reason is that we do not download/refresh the repositories filelists.xml (TW-oss ~48M for TW-oss ) and other.xml (~37MB for TW-oss; contains the changelogs) at all. Due to their size it is very expensive to keep them up-to-date.

Yeah, indeed it's a somewhat beefy file. And I wonder what if we used a compression method other than GZIP, maybe XZ or even the fast decompression ratios ZSTD. I remember reading -- in the Heroes mailing list, if memory serves -- a thread where Bernhard W. played a little with XZ and ZSTD, and compared them. Both XZ and ZSTD had better numbers than GZIP.

We could think about a flag to include them if your bandwith is big enough. Or we could try to download them on demand only , though this would delay searches for files/chanelogs a lot. For listing changelog/files of a specific package we could download the packages .rpm header on demand and query this. But a free search in all filelists/changelogs will be expensive.

Agreed, it's very expensive. Anyone who uses DNF can feel the slowness when it has to refresh repo metadata, even more so when people is using a rolling distribution. Now, here I am wondering if this wouldn't be a job for a secondary tool, one that wouldn't block Zypper. Something like what Debian/Ubuntu did. They have an apt-file tool that won't fetch (meta)data automatically, at least not OOTB, people have to use apt-file update.

Currently we just see the files which are explicitly provided by package. Those provides are injected when building the repo metadata on the server and they live in the repos primary.xml . So the client (zypper) can actually not tell which directories (/etc/, /bin/,...) the server covers. OBS AFAIK focuses on files which may actually be required by other packages.

Yes, the /etc/ coverage might be for the masses, sysadmins looking for configuration files -- if they know about this Zypper feature at all.

In the versions to come, OBS will show us the list of files in the web UI, along with the current symbol provision/requirement view for the generated RPM package. And this, in turn, makes me wonder if we could implement the full filesystem hierarchy search in OSC instead.

[Firstyear]

The main reason is that we do not download/refresh the repositories filelists.xml (TW-oss ~48M for TW-oss ) and other.xml (~37MB for TW-oss; contains the changelogs) at all. Due to their size it is very expensive to keep them up-to-date.

Yeah, indeed it's a somewhat beefy file. And I wonder what if we used a compression method other than GZIP, maybe XZ or even the fast decompression ratios ZSTD. I remember reading -- in the Heroes mailing list, if memory serves -- a thread where Bernhard W. played a little with XZ and ZSTD, and compared them. Both XZ and ZSTD had better numbers than GZIP.

ZSTD is significantly faster than XZ too. we are starting to swap to it for compression in rust packaging.

And to reiterate, you could just download the file when a user does a provides command.

[mlandres]

ZYPP downloads whatever the server provides. ZYPP itself could handle XZ as well as ZCK. We extra added support for ZCK and thought the build service would want to switch to it, but they AFAIK prefer GZ (--rsyncable) because of it 's faster compression.

Benjamin and Andreii are working on #openSUSE/libzypp#417 and also try to get proper zsync data to d.o.o and the mirrors, so we can at least benefit from --rsyncayble and download only the changed ranges. But we also found that d.o.o and some mirros refuse to provide more than a few ranges in one request. So we won't be able to fetch all ranges in one go, but may at least benefit from a reduced download size, if only small parts in the metadata changed.

And yes, donwloading filelist/other on demand might be an option.

[luc14n0]

ZYPP downloads whatever the server provides. ZYPP itself could handle XZ as well as ZCK. We extra added support for ZCK and thought the build service would want to switch to it, but they AFAIK prefer GZ (--rsyncable) because of it 's faster compression.

That's interesting, Zstd has --rsyncable too, but I can't say how well it would perform for openSUSE's use cases, compared to Gzip. And I just learned that Zchunk uses Zstd internally. I'm putting opening a thread in the Heroes mailing list in my To Do, so we can know where they stand regarding this topic.

Benjamin and Andreii are working on #openSUSE/libzypp#417 and also try to get proper zsync data to d.o.o and the mirrors, so we can at least benefit from --rsyncayble and download only the changed ranges. But we also found that d.o.o and some mirros refuse to provide more than a few ranges in one request. So we won't be able to fetch all ranges in one go, but may at least benefit from a reduced download size, if only small parts in the metadata changed.

Oh, this sounds promising to me.