Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Significant performance differences with different numpy installs #524

Open
jpgard opened this issue Jul 24, 2023 · 1 comment
Open

Significant performance differences with different numpy installs #524

jpgard opened this issue Jul 24, 2023 · 1 comment

Comments

@jpgard
Copy link

jpgard commented Jul 24, 2023

Expected behaviour

Please describe how you expected madmom to behave.

I would expect madmom to achieve roughly similar performance with OpenBLAS numpy and MKL-enabled numpy (for example, the numpy installation docs say "MKL is typically a little faster and more robust than OpenBLAS").

Actual behaviour

There is a 30x slowdown for ParallelProcessors when using OpenBLAS numpy (the default on pypi!), compared to MKL numpy (the version on conda), on my 2021 MacBook M1 Max. Processing a ~3min 44.1khz WAV file with an RNNBeatProcessor takes ~300sec with the pip version on numpy installed, while it takes 10sec with the conda version of numpy.

This means that even the same code, executed in an environment constructed from the same requirements.txt file, will produce very different performance results if inside a conda environment vs. in a Python virtualenv (or any other non-conda environment manager) -- even on the same hardware.

Note: I understand that this isn't an issue with madmom (a wonderful piece of software that I am grateful for your efforts developing and sharing!!!). However, it would probably be extremely useful to many users to add a big loud warning to the installation info that this performance gap exists for other environments besides my own (if it does!) -- this is something users can set up themselves with proper knowledge, but if it isn't flagged, it could result in significant and unnecessary performance degradations.

Steps needed to reproduce the behaviour

import madmom
import soundfile as sf
beat_proc = RNNBeatProcessor()
samples, sr = sf.read(path_to_some_file)
beat_acts = beat_proc(samples)

Information about installed software

Python 3.10
madmom 0.17.dev0
numpy 1.24.4

@jpgard
Copy link
Author

jpgard commented Jul 25, 2023

After some more digging -- it actually looks like the issue is due to x86 vs. ARM versions of numpy. Specifically, I observed this in the context of docker environments, where depending on the base image/os, different versions of numpy (with the different instruction sets enabled) were installed by pip.

Again, not a bug per se, but probably something worth highlighting (even if the message is "accelerated linear algebra libraries will accelerate your madmom code").

@jpgard jpgard changed the title Significant performance differences with OpenBLAS vs. MKL numpy Significant performance differences with different versions of numpy Jul 25, 2023
@jpgard jpgard changed the title Significant performance differences with different versions of numpy Significant performance differences with different numpy installs Jul 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant