Improve CRDB storage performance

[BenjaminPelletier]

Is your feature request related to a problem? Please describe.
A production GCP deployment was observed having relatively high request completion latency and was root-caused to likely be due to the storage type used for CockroachDB.

Cockroach Labs recommends the use of pd-ssd storage to avoid performance bottlenecks due to slow random-access storage. The pd-standard disks used for the standard storage class in GCP are intended for more traditional append-only / sequential read workloads, but we currently use the standard storage class in our deployment.

Describe the solution you'd like
Add a deployment configuration option to select the storage class to be used for CockroachDB nodes (all other deployment components can probably keep the standard storage class), default to premium-rwo (which selects pd-ssd disks), and document that standard or standard-rwo can be selected for cost savings.

Describe alternatives you've considered
We could hardcode the recommended storage solution to match Cockroach Labs' recommendation, but a configurable variable would allow us to serve more use cases, especially R&D instances with low performance needs but high cost sensitivity.

[barroco]

The default is indeed standard for Google Cloud (GKE) at the moment. I will prepare a PR which will change the default storage class to the type recommended by cockroach per cloud provider.

In the meantime, this can be configured depending on the deployment strategy:

Terraform

The storage class can be set using the terraform variable *_kubernetes_storage_class:

• Google Cloud (GKE): google_kubernetes_storage_class
• AWS (EKS): aws_kubernetes_storage_class

Tanka

As documented here, the storage class can be configured using the field cockroach.storage in the main.jsonnet file.

Helm

The storage class can be defined using Helm Chart value: cockroachdb.storage.persistentVolume.storageClass. Schema - Example