Consider using the Speex resampler consistenly for every audio device

[johnnovak]

1. Some audio devices use the resampler from the reSIDfp library. This creates an unfortunate dependency on reSID in these unrelated devices, plus we're already using the Speex resampler for the Sound Blaster, and there's generic resampling support in the mixer for any mixer channel.

   

2. The Speex resampler is probably more performant than all these other resamplers bundled with the libraries.

3. Consider rendering OPL and MT-32 output at their original rates too, then use Speex to resample to the host sample rate. Rendering at the host sample rate instead of the device's native sample rate might introduce timing discrepancies and these bundled resamplers might be lower quality and not as performant as Speex.

[weirddan455]

I'll continue our discussion from #3629 here. We probably want to add the flag SDL_AUDIO_ALLOW_FREQUENCY_CHANGE when we open the SDL audio device.

int sdl_allow_flags = 0;

if (negotiate) {
	// We allow chunk size changes (if supported), but don't allow
	// frequency changes because we've only seen them go beyond
	// SDL's desired 48 Khz limit
#if defined(SDL_AUDIO_ALLOW_SAMPLES_CHANGE)
	sdl_allow_flags |= SDL_AUDIO_ALLOW_SAMPLES_CHANGE;
#endif

That's a bit crap. I'd expect to to simply fail with negotiation off if the driver doesn't support that requested rate, and with negotiation on, it should just find the closest driver-supported rate.

The behavior if we add the flag is that it will return the closest thing the platform can do and we have to do the resampling ourselves. With the flag off we're saying "OK SDL this is the type of audio we have, deal with it!" and it will automatically convert the audio if it needs to.

I'd really like disallow any SDL-side resampling whatsoever. Is that possible with some flags? I want it to work as per above, and I want to keep all resampling business in our hands.

I believe this is enough to avoid resampling for our use-case but I'd really have to check the SDL source code to be sure. If we ever use the SDL audio streams, it's a bit different but I'd probably recommend staying away from those and continue using the lower-level callback like we currently are.

There is also some SDL_Hint flags (which can also be set as environment variables). One of which can use a third-party re-sampler library if available (I think it had "rabbit" in the name or something).

[johnnovak]

The behavior if we add the flag is that it will return the closest thing the platform can do and we have to do the resampling ourselves.

That's absolutely what we want! We handle the resampling of all the individual mixer channels to the host rate.

With the flag off we're saying "OK SDL this is the type of audio we have, deal with it!" and it will automatically convert the audio if it needs to.

That's what we absolutely do not want! 😅 No second resampling pass please using some shitty linear interpolation or sample dropping algo 🥶

I believe this is enough to avoid resampling for our use-case but I'd really have to check the SDL source code to be sure.

Great, I'd appreciate that.

If we ever use the SDL audio streams, ...

Extremely unlikely.

...it's a bit different but I'd probably recommend staying away from those and continue using the lower-level callback like we currently are.

Absolutely. We only need a callback that allows us to fill a stereo DAC buffer—and thank you very much SDL, yourd audio duties stop right there 😆

There is also some SDL_Hint flags (which can also be set as environment variables). One of which can use a third-party re-sampler library if available (I think it had "rabbit" in the name or something).

That's good to know, but there's no need for that. We'll just keep using Speex. It does the job very well and has low latency figures, which is super important. I've done a comparison of various resampling libraries, and while, for example, libsamplerate consumes less CPU power, it introduces unacceptably high latencies (in the 30-50ms+ range, from memory—that's a no-go for games).