-
Notifications
You must be signed in to change notification settings - Fork 94
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tablets: consider alternative implementation #983
Comments
We are aiming (at least initially) for tablet size - which isn't too small - we'd like to have 5-10GB in size. @avikivity - thoughts? |
Inserts to tablets structure happen any time we receive info about the tablet. This is not only during split / merge but during normal work of a new session. After session creation the vector will be empty, and all queries will be non-token-aware. Then we'll start receiving tablets because we send queries to wrong nodes and we'll insert those tablets to the Deletes will happen when received tablet overlaps with some tablet in the vector - this will happen, iiuc, during split / merge and after topology changes (because tablets will be rebalanced). How many tablets may there be in a single table? |
Aren't we going to batch updates to the tablets lookup structure and then do read-copy-update? I thought that we wanted to heavily optimized for the read case - search in |
We do batching, but it's not related to this issue. The procedure to update tablet structure is:
Batching here means receiving all tablets from channel in step 1, not only 1 tablet. This reduces the amount of times we need to perform step 2, which is beneficial because |
Another performance optimization that I think is best to leave for a follow up: TabletReplicas should do less allocations in a typical case. |
Current implementation of tablets that is going to be merged stores tablets for a given table in
Vec
.This is ok for searching, but insterting / deleting tablets requires moving part of the vector.
The alternative would be implementation based on some tree (maybe
BTreeMap
is enough, maybe we would need external crate, I'm not sure). It would make insertion / deletion logarithmic instead of linear, but could have worse cache behavior.It's hard to guess which version will be faster in practice - we should implement both and benchmark them on some workloads.
The text was updated successfully, but these errors were encountered: