This project provides the Stargate GraphQL API - an HTTP service that gives access to data stored in a Cassandra cluster using auto-generated GraphQL interface. (As part of Stargate V2, this service was extracted from the monolithic Stargate V1 Coordinator as a separate microservice.)
The project depends on the sgv2-quarkus-common module, which provides common functionality used by all Stargate V2 APIs.
All issues related to this project are marked with the stargate-v2 and either GraphQL CQL-first or GraphQL schema-first.
Please read the Stargate V2 Shared Concepts in order to get basic concepts shared between all V2 API implementations.
Here is a brief functional overview of the paths supported by the GraphQL service:
/graphql-schemaexposes DDL operations (describe, create table, etc). It can be used for any keyspace (most operations take akeyspaceargument).- for each keyspace, there is a
/graphql/<keyspace>service for DML operations (read and insert data, etc). Initially, its GraphQL schema is automatically generated from the CQL data model. We call this the CQL-first model. /graphql-adminallows users to deploy their own GraphQL schemas. Stargate will alter the CQL schema to match what they want to map. We call this the GraphQL-first or schema-first model. Once a GraphQL-first schema has been deployed in this manner, it replaces the CQL-first one:/graphql/<keyspace>now uses the custom schema, and the generated schema is not available anymore.
We rely extensively on GraphQL Java. Before delving further into the Stargate code, it can be helpful to have a basic understanding of that library:
Random random = new Random();
GraphQLSchema schema =
GraphQLSchema.newSchema() // (1)
.query(
GraphQLObjectType.newObject()
.name("Query")
.field(
GraphQLFieldDefinition.newFieldDefinition()
.name("random")
.type(Scalars.GraphQLInt)
.build())
.build())
.codeRegistry( // (2)
GraphQLCodeRegistry.newCodeRegistry()
.dataFetcher(
FieldCoordinates.coordinates("Query", "random"),
(DataFetcher<Integer>) environment -> random.nextInt())
.build())
.build();
GraphQL graphql = GraphQL.newGraphQL(schema).build(); // (3)
ExecutionResult result = graphql.execute("{ random }");
System.out.println(result.getData().toString()); // prints {random=1384094011}-
newSchema()provides a DSL to create the GraphQL schema programmatically. This example will produce:type Query { random: Int }
-
The code registry provides the logic to execute queries at runtime. It is broken down into data fetchers. Each fetcher handles a field, identified by its coordinates in the schema.
-
Finally, we turn the
GraphQLSchemainto an executableGraphQL.
The service accesses the Stargate persistence via the gRPC bridge. The bridge client
is obtained via StargateRequestInfo (imported from sgv2-quarkus-common), which creates it
from the auth token and (optional) tenant ID passed as HTTP request headers.
Note that we don't use the gRPC client generated by Quarkus (StargateBridge) directly; instead, we
rewrap it in a synchronous / future-based interface: StargateBridgeClient. This was done out of
convenience to simplify the migration of existing code. In the future, it might be possible to make
all the fetcher code asynchronous and use the Quarkus client (but there will still be some wrapping
required because graphql-java only knows of to deal with futures).
They are implemented as REST resources. They all extend GraphqlResourceBase, which handles the various ways to query GraphQL over HTTP: POST vs GET, arguments in the query string vs the body, multipart, etc.
The only thing that changes across subclasses is how we obtain the GraphQL object to query
(see GraphQL layer below), and whether multipart is supported.
StargateGraphqlContext allows us to inject state that will be available later in the data fetchers (see CassandraFetcher). In particular, this is how we pass the reference to the bridge client. The context also handles batching, which will be described below.
PlaygroundResource exposes the GraphQL playground (an in-browser client, served from a static HTML file). FilesResource provides downloadable versions of users' custom schemas, and CQL directive definitions.
GraphqlCache provides the GraphQL instances used by the HTTP layer. The lifecycle of those
objects varies depending on the service:
The DDL service never changes, we only need to cache one instance. The schema is built by DdlSchemaBuilder.
The data fetchers are in the package cqlfirst.ddl.fetchers. Their implementation is pretty straightforward: translate the GraphQL operations into CQL queries for the persistence layer.
Our fetchers are generally loosely typed, a lot of them return raw data as a Map<String, Object>.
But there is a case where that doesn't work, and that's when one of the inner fields can be
parameterized:
query {
keyspace(name: "library") {
books: table(name: "books") { columns { name } }
authors: table(name: "authors") { columns { name } }
}
}In that situation, there is no way to return both tables with a Map<String, Object>, because the
map keys are the actual field types, not their aliases. The solution is to use a strongly-typed DTO,
such as KeyspaceDto. (In hindsight, it would be better to use the strongly-typed approach whenever
possible, that can be done as a housekeeping task in the future but is not high priority.)
We cache one GraphQL per keyspace.
The schema is built by DmlSchemaBuilder. This time things are a bit more dynamic, we read from a
CqlKeyspaceDescribe and generate the schema accordingly. For example if there is a "User" table,
we'll have a User GraphQL type, an insertUser operation, etc.
The schema often references the same types over and over. For example, each time a table can be
queried by an int, the query operation will have an IntFilterInput argument that defines various
search operators (eq, lte, etc). In the builder code, every occurrence of a given type must be
represented by the same GraphQLType instance, so we must keep track of the types we have generated
so far, and reuse them if they appear again. This is handled by FieldTypeCache and its subclasses.
We name our GraphQL types and fields after CQL tables and columns. But GraphQL identifiers have more restrictive naming rules, so we sometimes need to convert names. This is covered by NameMapping, which also caches the results.
CQL also has a wider range of primitive types. CqlScalar defines a number of custom GraphQL
scalars to match the CQL types that have no direct equivalent. See the *Coercing classes for the
details of the type coercion rules.
The data fetchers are in the package cqlfirst.dml.fetchers. There is one fetcher per type of
query, it gets initialized with the CqlTable that this particular operation was created for.
In order to adapt to CQL changes in real time, GraphqlCache re-fetches the CqlKeyspaceDescribe
from the bridge on every call to getDml. If the "hash" field is different, this means that the CQL
schema has changed, and we need to regenerate a new GraphQL instance. That logic can be observed
in the GraphqlHolder internal class in GraphqlCache.
It's also worth noting that both versions of the API share the same cache entry: if a GraphQL-first
schema was deployed for this keyspace, we use it, otherwise we generate a CQL-first schema. Again,
see GraphqlHolder.
The admin service never changes, we only need to cache one instance. The schema is built by AdminSchemaBuilder.
The data fetchers are in the package graphqlfirst.fetchers.admin. Most of the fetchers are trivial, with the exception of DeploySchemaFetcherBase: this is where we process a custom GraphQL schema received from the user.
- we first check if a previous custom schema was deployed for this keyspace. This is stored in a
table
stargate_graphql.schema_source. We also use a lightweight transaction as a rudimentary concurrency control mechanism to ensure that two deployments cannot run concurrently. This is all handled by SchemaSourceDao. - SchemaProcessor parses the user's GraphQL schema, and builds:
- MappingModel, a representation of the equivalent CQL model. This starts in
MappingModel.build(), and then branches out to*ModelBuilderhelper classes for each kind of GraphQL element (types, operations, etc). - the
GraphQLinstance. We use the user's schema directly (it is valid if we got this far), and MappingModel generates the data fetchers.
- MappingModel, a representation of the equivalent CQL model. This starts in
- CassandraMigrator checks if that theoretical CQL model matches the actual contents of the database. If it doesn't, there are various user-configurable migration strategies to handle the differences: error out, or try to alter the CQL model. In the latter case, the check returns a list of migration queries, that we can now execute.
- finally, we insert the new version in
stargate_graphql.schema_source. The newGraphQLinstance is sent to GraphqlCache, which starts to serve it to its clients.
We provide a set of CQL directives that allow users to control certain aspects of their mapping (for
example, use a different table name than the inferred default). They are defined in CqlDirectives,
and referenced throughout the model-building code. You can also download a text version from a
running Stargate instance at graphql-files/cql_directives.graphql.
Once a user has deployed their own GraphQL schema, it replaces the CQL-first schema for that keyspace. As already mentioned, GraphqlCache contains logic to determine which variant to load.
There are two ways that a GraphQL-first GraphQL instance can be created:
- if the deploy operation just happened in this Stargate process, then DeploySchemaFetcherBase
already has the
GraphQL, and puts it directly in the cache withGraphqlCache.putDml. - if the deployment happened via another Stargate instance, or if Stargate just restarted, we need
to reload the schema from
stargate_graphql.schema_source. This uses a simplified version of the deployment process described in the previous section, seeGraphqlCache.GraphqlHolder.
Unlike CQL-first, GraphQL-first does not react to external CQL schema changes: the user is supposed to have full control over the GraphQL schema, so we can't just change it behind their back. So our assumption is that once users go GraphQL-first, they will evolve their CQL schema via successive deploy operations, not by directly altering the database. If things get out of sync, there are no guarantees: GraphqlCache will detect the changes (via the hash mechanism) and try to regenerate the schema-first GraphQL, but that might fail.
The data fetchers are in the package graphqlfirst.fetchers.deployed. They rely on the
representation that was built by MappingModel. For example, the query fetcher relies on a query
model that defines the target table, which parameters of the GraphQL operation will be mapped to
WHERE clauses, etc. This part of the code is quite tedious because our mapping rules are very
flexible: there are many different ways to define operations and map the results. The best way to
get familiar with a fetcher is to look at the integration tests and trace their execution.
Batching is a cross-cutting concern, both CQL-first and GraphQL-first support the @atomic
directive to indicate that a set of mutations must be executed as a single CQL batch:
mutation @atomic {
... // mutations
}StargateGraphqlContext (via its inner class BatchContext) provides coordination services that
allows independent fetchers to accumulate queries, and track which is the last one that must execute
the batch.
The GraphQL API service is packaged as a Docker image available on Docker Hub. For examples of container-based deployments, see the docker-compose or helm folders.
The GraphQL API is also packaged as an uber-jar as part of Stargate v2 releases, along with the accompanying start-graphqlapi.sh script.
The startup script can also be used to start the uber-jar locally, along with other options documented in the Development Guide.
There are two main configuration property prefixes used, stargate. and quarkus..
The quarkus. properties are defined by the Quarkus framework, and the complete list of available properties can be found on the All configuration options page.
In addition, the related guide of each Quarkus extension used in the project provides an overview of the available config options.
The stargate. properties are defined by this project itself and by the sgv2-quarkus-common configuration.
The properties are defined by dedicated config classes annotated with the @ConfigMapping.
The list of currently available properties is documented in the Configuration Guide.
This project uses Quarkus, the Supersonic Subatomic Java Framework. If you want to learn more about Quarkus, please visit its website.
It's recommended that you install Quarkus CLI in order to have a better development experience. See CLI Tooling for more information.
Warning This project uses Java 17, please ensure that you have the target JDK installed on your system.
Before being able to run the application, make sure you install the root ../apis/pom.xml:
cd ../
./mvnw install -DskipTestsTo run your application in dev mode with live coding enabled, use the command:
../mvnw quarkus:devNote
Quarkus now ships with a Dev UI, which is available in dev mode only at http://localhost:8080/star\ gate/dev/.
In development mode, Quarkus starts by default with debug mode enabled, listening to port 5005 without suspending the JVM.
You can attach the debugger at any point, the simplest option would be from IntelliJ using Run -> Attach to Process.
If you wish to debug from the start of the application, start with -Ddebug=client and create a debug configuration in a Listen to remote JVM mode.
See Debugging for more information.
Warning
You need to build the coordinator docker image(s) first. In the Stargate repo directory/apisrun:../mvnw clean install -P dse -DskipTests && ./build_docker_images.sh
Integration tests are using the Testcontainers library in order to set up all needed dependencies, a Stargate coordinator and a Cassandra data store.
They are separated from the unit tests and are running as part of the integration-test and verify Maven phases:
../mvnw integration-testDepending on the active profile, integration tests will target different Cassandra version as the data store. The available profiles are:
cassandra-40(enabled by default) - runs integration tests with Cassandra 4.0 as the data storecassandra-311- runs integration tests with Cassandra 3.11 as the data storedse-68- runs integration tests with DataStax Enterprise (DSE) 6.8 as the data store
The required profile can be activated using the -P option:
../mvnw integration-test -P cassandra-311Warning
You need to build the coordinator docker image(s) first. In the Stargate repo directory/apisrun:../mvnw clean install -P dse -DskipTests && ./build_docker_images.sh
Running integration tests from an IDE is supported out of the box. The tests will use the Cassandra 4.0 as the data store by default. Running a test with a different version of the data store or the Stargate coordinator requires changing the run configuration and specifying the following system properties:
testing.containers.cassandra-image- version of the Cassandra docker image to use, for example:cassandra:4.0.4testing.containers.stargate-image- version of the Stargate coordinator docker image to use, for example:stargateio/coordinator-4_0:v2.0.0-ALPHA-10-SNAPSHOT(must be V2 coordinator for the target data store)testing.containers.cluster-version- version of the cluster, for example:4.0(should be one of3.11,4.0or6.8)testing.containers.cluster-dse- optional and only needed if DSE is used
The integration tests can also be executed against an already running instance of the application.
This can be achieved by setting the quarkus.http.test-host system property when running the tests.
You'll most likely need to specify the authentication token to use in the tests, by setting the stargate.int-test.auth-token system property.
./mvnw verify -DskipUnitTests -Dquarkus.http.test-host=1.2.3.4 -Dquarkus.http.test-port=4321 -Dstargate.int-test.auth-token=[AUTH_TOKEN]
You can skip the integration tests during the maven build by disabling the int-tests profile using the -DskipITs property:
../mvnw verify -DskipITsAlternatively you may want to run only integration tests but not unit tests (especially when changing integration tests). This can be achieved using the command:
../mvnw verify -DskipUnitTests
If your Integration Test run fails with some generic, non-descriptive error like:
[ERROR] CollectionsResourceIntegrationTest » Runtime java.lang.reflect.InvocationTargetException
here are some things you should try:
- Make sure your Docker Engine has enough resources. For example following have been observed:
- Docker Desktop defaults of 2 gigabytes of memory on Mac are not enough: try at least 4
The application can be packaged using:
../mvnw packageThis command produces the quarkus-run.jar file in the target/quarkus-app/ directory.
Be aware that it’s not an über-jar as the dependencies are copied into the target/quarkus-app/lib/ directory.
The application is now runnable using java -jar target/quarkus-app/quarkus-run.jar.
If you want to build an über-jar, execute the following command:
../mvnw package -Dquarkus.package.type=uber-jarThe application, packaged as an über-jar, is now runnable using java -jar target/*-runner.jar.
You can create a Docker image named io.stargate/docsapi using:
../mvnw clean package -Dquarkus.container-image.build=trueOr, if you want to create a native-runnable Docker image named io.stargate/docsapi-native using:
../mvnw clean package -Pnative -Dquarkus.native.container-build=true -Dquarkus.container-image.build=trueIf you want to learn more about building container images, please consult Container images.
The easiest way is to use the built-in GraphQL playground at http://localhost:8080/playground.
You can then interact with the various GraphQL schemas by entering their URL in the playground's address bar, for example http://localhost:8080/graphql/{keyspace}.
This project uses various Quarkus extensions, modules that run on top of a Quarkus application.
You can list, add and remove the extensions using the quarkus ext command.
Note Please check the shared extensions introduced by the sgv2-quarkus-common project.
The Quarkus DI solution.
The project uses Docker for building the Container images. Properties for Docker image building are defined in the pom.xml file. Note that under the hood, the generated Dockerfiles under src/main/docker are used when building the images. When updating the Quarkus version, the Dockerfiles must be re-generated.
The extension setups the health endpoints under /stargate/health.
This project disables the OpenAPI and SwaggerUI, due to the availability of the GraphQL Playground.