Developing New Algorithms

[ecarl65]

I'm a total newb to MatX and CUDA in general. I'm a DSP engineer and program in C++ and Python, but just have a hole in my experience regarding GPUs. Anyway, I thought MatX might be a good way to dip my toes in the water.

I'm just wondering a little bit about the model. It's awesome that it has a lot of tools like filtering, FFTs, etc. Let's say I wanted to make a new algorithm, like a Hough transform, for example. I guess there are a couple of ways of going about it?

1. Program entirely using MatX, in developer-friendly manner that doesn't involve low level CUDA calls. Hope that the speed is decent. This is what MatX was designed for, presumably?
2. If you're experienced with CUDA read the CONTRIBUTING.md file and try to contribute a hough transform algorithm with appropriate low-level CUDA calls for addition into the baseline.

Do I have the general idea right?

[cliffburdick]

Hi @ecarl65, thanks for stopping by!

1. Yes, MatX is designed to be the same speed or faster than a novice to mid-level user writing their own CUDA kernels. We do this two ways: compile-time kernel fusion for element-wise kernels, and using popular CUDA libraries for optimized backends (cuBLAS, CUB, etc).

The first is important because using C++17 we're able to convert large algebraic expressions into a single kernel that should be as good or better than what you'd write yourself. It's also a lot more modular because writing a single kernel limits you to data types, sizes, strides, etc, whereas writing it generically in MatX allows you to change those parameters without changing the expression itself.

The second is important because in almost all cases you will not beat the CCCL or CUDA Math libraries, so we can utilize the same types you had in your algebraic expressions but translate it into the CUDA libraries. A good example of this is something like doing a sum(). For an optimal sum() function you need to use CUB. But CUB has its own API separate from other CUDA libraries that allows you to do a lot at compile-time. You can do that all yourself, or you can use something like MatX to do sum(out, in * 4 + sin(b)); and we convert that into an iterator used by CUB that computes the input section on-the-fly. You can do this yourself, but it ends up being a lot of code, and again, it's not modular.

This GTC talk goes into more depth: https://www.nvidia.com/en-us/on-demand/session/gtcfall21-a31410/

2. Yes, we're always open to new contributions. I can look into a Hough transform and see roughly what it would take and hopefully give some guidance.

[ecarl65]

Thanks so much! I look forward to reading this presentation. Also, this project is fantastic! This is exactly the sort of thing we need for easier and better adoption of CUDA technologies to people like me. How do we get this socialized to more people? I only happened to stumble on this from a link from a co-worker, but it seems like abstracting away a lot of the harder programming yet retaining similar execution times is a win-win and what people like me (engineers who aren't as savvy on the software development front) really need!

[cliffburdick]

Thanks so much! I look forward to reading this presentation. Also, this project is fantastic! This is exactly the sort of thing we need for easier and better adoption of CUDA technologies to people like me. How do we get this socialized to more people? I only happened to stumble on this from a link from a co-worker, but it seems like abstracting away a lot of the harder programming yet retaining similar execution times is a win-win and what people like me (engineers who aren't as savvy on the software development front) really need!

Thanks @ecarl65, much appreciated. Right now we rely on word-of-mouth beyond some internal uses we have, so you can help spread the word! We also have a sister project called cuSignal (https://github.com/rapidsai/cusignal) that's focused on GPU signal processing in Python. Many people find it helpful to prototype in Python before moving to C++, and we're happy to help with both.

I forgot to mention that if you think the performance of anything in MatX is not what you expect, please let us know. We expect to be native performance, and if not, then we'd like to know where we're failing.

[ecarl65]

Sounds great, I'll do that. I've already talked to my DSP friends inside and outside my company about it. But I'll start playing around with it soon as well. So far it looks fantastic, and I'm excited to use it. Of course, getting more and more features and approaching numpy functionality will only make it more useful. But that's a long term endeavor. Merely having filtering and FFTs gets me most of what I need, as well as matrix operations and a data type that just handles the CUDA parts.

[leofang]

We also have a sister project called cuSignal (https://github.com/rapidsai/cusignal) that's focused on GPU signal processing in Python. Many people find it helpful to prototype in Python before moving to C++, and we're happy to help with both.

FYI going forward cuSignal will be integrated into CuPy: https://docs.rapids.ai/notices/rsn0032/