[SUGGESTION] Vocabulary types based on View and Array

[dutkalex]

Hi! Looking back on a few years of working on a daily basis in a Kokkos based CFD solver, I have noticed that I have come to write lots of small vocabulary types and functions for dealing with things such as mesh connectivity, small thread-local matrix vector products, binary searches etc. This has come to the point that we are now considering making this "Kokkos toolbox" a separate project because of the size and the highly reusable nature of this part of the codebase for all kinds of Kokkos-based HPC applications.

I believe that having a few vocabulary types for very common needs in HPC applications, based on View and Array which are the bread and butter of Kokkos data structures, would greatly benefit the community as a whole: better productivity for users which could rely on high-quality implementations of these simple yet very useful pieces of code (instead of ad-hoc project local lower-quality implementations), and improved interoperability between Kokkos-based projects. Off the top of my head:

• Kokkos::Array based stack containers for handling vectors and matrices with:
  • component-wise operator+, operator* etc.
  • structured bindings support
  • thread local KOKKOS_FUNCTIONs for matrix vector, matrix matrix products etc.
  • easy read/write operations from and to Kokkos::Views (e.g. Vector<double, 5> vector = some_XbyYby5_view(i_elm, i_pt, Kokkos::ALL);)
• A container for representing indirection tables (i.e. the different flavors of Kokkos::View< Kokkos::View<T, Kokkos::LayoutStride, Kokkos::MemoryTraits<Kokkos::Unmanaged>>* > containers which are very common for filling MPI buffers, imposing boundary bounditions etc. in mesh-based scientific computing applications
• A container for representing mesh connectivities and other irregular data structures which are typically stored as a big raw 1D Kokkos::View, with an indirection table which stores subviews on the raw allocation to provide a usable API.

If you judge this to be something worth developing, I can easily provide "to be refined and hardened" implementations for these vocabulary types. Anyway, thanks for all the amazing work, and looking forward to your feedback!

Best reagrds,
Alex

[maartenarnst]

These additions sound interesting indeed.

• A container for representing mesh connectivities and other irregular data structures which are typically stored as a big raw 1D Kokkos::View, with an indirection table which stores subviews on the raw allocation to provide a usable API.

@romintomasetti and I have also developed such a data structure. We called it a FlatRaggedView. It has some similarities with the StaticCrsGraph that is already in Kokkos too. We'd be happy to be involved in refining, hardening, ...

See also:

• Kokkos static CRS graph row view is too tightly typed #5939

[dutkalex]

@maartenarnst Yes this is an unavoidable data structure when general unstructured meshes are involved, and I am convinced that having such a data structure would prove useful for solving other common problems in a memory efficient way... It is not obvious to me how this could be unified with the StaticCrsGraph class, and I lean more towards a separate type but I am no expert. Before proposing anything I think it is wiser to wait for feedback from the Kokkos team and get their opinion on how to proceed, but I'd be happy to work on a joint PR with you guys! 🙃