Stargate V2 / REST / Schema access optimization: partial caching (f.ex of PKs) #1481
Replies: 3 comments
-
Not having a state-full update mechanism will only work for the cases like the one explained above. For other use cases we can not afford to store schema data in 1min TTL cache, simply because as soon as a schema is changed we would send wrong responses for one minute. Full cache everything is not an option imo. But stateful update mechanism is. Once we need a schema for a tenant we should go grab it and listen on it's updates until the schema is in the cache. How long the schema should stay in the cache should be defined by max size of the cache, meaning that most used entries stay longer. The stateful update mechanism could really be simple in the begging. You can simply only send the schema changed event without actually piping the whole schema with the event. That's enough for the cache invalidation. And that's the event holding 2 strings, quite simple. Bottom line is that schema is not changed so often, thus it's not like these events are flying around all the time. I would avoid doing any localized caches, but implement a solution and a component that can be used everywhere. At the end it's actually gonna save work. |
Beta Was this translation helpful? Give feedback.
-
Good points @ivansenic. I was wondering about that myself; not being sure if this approach makes sense. I'll add 2 counter-points for discussion:
First: implementing full cache solution is not trivial for many reasons, including need to combine both update listening and some form of cache invalidation/refresh (since we cannot count on 100% reliable update mechanism; so either TTL or background scanning also needed). There is also work both on client and server side. Second: some of the work on handling client-side caching would be reusable: that is, client-side cache handling for just some operations is quite similar to that of more pervasive caching (just without background update mechanism). So: considering more limited client-side only caching could still be a stepping-stone even if we eventually have to implement full client-and-server-stateful solution. |
Beta Was this translation helpful? Give feedback.
-
Note: since Olivier is already working on full-feature Schema handling, it seems unlikely we would bother with this. We can consider this an inactive discussion; not sure what is the usual procedure since there doesn't seem to be "Close" option. |
Beta Was this translation helpful? Give feedback.
-
The first version of Stargate V2 REST implementation -- so-called "Steel Thread" -- seems to perform as expected, having perhaps 50% of throughput of SGv1 equivalent, based on initial tests. One obvious additional overhead with SGv2 is that Schema information is needed for almost all operations, adding one more gRPC call on critical path.
To avoid/reduce these calls we will probably need to add caching: but full cache-everything for all tenants caching will be complicated to implement, especially due to multi-tenancy restrictions and possible need to have stateful update mechanism.
But before embarking on implementing transparent "full" caching, perhaps we can find out more localized things to cache.
One specific example I have is that of "getRow()" functionality, in which we only seem to need Schema information to bind positional path parameters into name/type information of primary keys. While we absolutely need this information, it seems that this particular sliver of information
Given this, a "Simple" regular cache with reasonable long TTL (as in, say, 60 seconds) could probably give a very high hit rate and eliminate most lookups, and this without causing problems wrt cache validation.
If we implemented this, we should be able to measure impact quite easily. And if there are clear benefits we could try to figure out other targeted "simple" caching caches to eliminate biggest bottlenecks, possibly avoiding bigger general caching system.
This may or may not be enough to avoid need for other schema access mechanisms but seems worth investigating.
Beta Was this translation helpful? Give feedback.
All reactions