Reduce getNodeByQuery overhead #13221
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fcostaoliveira
changed the title
jhershberg
reviewed
Apr 18, 2024
| for (e = s+1; e < keylen; e++) | ||
| if (key[e] == '}') break; | ||
|
|
||
| /* No '}' or nothing between {} ? Hash the whole key. */ |
There was a problem hiding this comment.
While we're at it, do we want to put an "unlikely" here? I am far from an expert, just asking.
There was a problem hiding this comment.
will to the test of the impact on the results and reply back.
…after defragment (redis#13231) Introducted by redis#13013 After defragmenting the dictionary in the kvstore, if the dict is reallocated, the value of its node in the kvstore rehashing list must be updated.
For my mistake, in the last revert commit in redis#13231, I originally wanted to revert the last one, but reverted the penultimate fix. Now that we have fix another potential memory read issue in [`743f1dd` (redis#13231)](redis@743f1dd), now it just seems to avoid confusion, i will verify in the future whether it will have any impact, if so we will add this PR to backport. Failed CI: https://github.com/sundb/redis/actions/runs/8826731960
Because it does not cause any propagation (arguably it should, see the comment in the tcl file) The motivation for this fix is that in 6.2 if dirty changed without propagation inside MULTI/EXEC it would cause propagation of EXEC only, which would result in the replica sending errors to its master
added reverse history search to redis-cli, use it with the following: * CTRL+R : enable search backward mode, and search next one when pressing CTRL+R again until reach index 0. ``` 127.0.0.1:6379> keys one 127.0.0.1:6379> keys two (reverse-i-search): # press CTRL+R (reverse-i-search): keys two # input `keys` (reverse-i-search): keys one # press CTRL+R again (reverse-i-search): keys one # press CTRL+R again, still `keys one` due to reaching index 0 (i-search): keys two # press CTRL+S, enable search forward (i-search): keys two # press CTRL+S, still `keys one` due to reaching index 1 ``` * CTRL+S : enable search forward mode, and search next one when pressing CTRL+S again until reach index 0. ``` 127.0.0.1:6379> keys one 127.0.0.1:6379> keys two (i-search): # press CTRL+S (i-search): keys one # input `keys` (i-search): keys two # press CTRL+S again (i-search): keys two # press CTRL+R again, still `keys two` due to reaching index 0 (reverse-i-search): keys one # press CTRL+R, enable search backward (reverse-i-search): keys one # press CTRL+S, still `keys one` due to reaching index 1 ``` * CTRL+G : disable ``` 127.0.0.1:6379> keys one 127.0.0.1:6379> keys two (reverse-i-search): # press CTRL+R (reverse-i-search): keys two # input `keys` 127.0.0.1:6379> # press CTRL+G ``` * CTRL+C : disable ``` 127.0.0.1:6379> keys one 127.0.0.1:6379> keys two (reverse-i-search): # press CTRL+R (reverse-i-search): keys two # input `keys` 127.0.0.1:6379> # press CTRL+G ``` * TAB : use the current search result and exit search mode ``` 127.0.0.1:6379> keys one 127.0.0.1:6379> keys two (reverse-i-search): # press CTRL+R (reverse-i-search): keys two # input `keys` 127.0.0.1:6379> keys two # press TAB ``` * ENTER : use the current search result and execute the command ``` 127.0.0.1:6379> keys one 127.0.0.1:6379> keys two (reverse-i-search): # press CTRL+R (reverse-i-search): keys two # input `keys` 127.0.0.1:6379> keys two # press ENTER (empty array) 127.0.0.1:6379> ``` * any arrow key will disable reverse search your result will have the search match bolded, you can press enter to execute the full result note: I have _only added this for multi-line mode_, as it seems to be forced that way when `repl` is called Closes: redis#8277 --------- Co-authored-by: Clayton Northey <clayton@knowbl.com> Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech> Co-authored-by: debing.sun <debing.sun@redis.com> Co-authored-by: Bjorn Svensson <bjorn.a.svensson@est.tech> Co-authored-by: Viktor Söderqvist <viktor@zuiderkwast.se>
…SP3 in MULTI (redis#13255) This test was introducted by redis#13251. Normally we auto transform the reply format of XREADGROUP to array under RESP3 (see trasformer_funcs). But when we execute XREADGROUP command in multi it can't work, which cause the new test failed. The solution is to verity the reply of XREADGROUP in advance rather than in MULTI. Failed validate schema CI: https://github.com/redis/redis/actions/runs/9025128323/job/24800285684 --------- Co-authored-by: guybe7 <guy.benoish@redislabs.com>
…an bug (redis#13150)" (redis#13266) The kernel config `vm.mmap_rnd_bits` had been revert in actions/runner-images#9491, so we can revert the changes from redis#13150. CI only with ASAN passed: https://github.com/sundb/redis/actions/runs/9058263634
…ommand (redis#13276) This PR is based on the commits from PR redis#12944. Allow SPUBLISH command within multi/exec on replica Behavior on unstable: ``` 127.0.0.1:6380> CLUSTER NODES 39ce8aa20f1f0d91f1a88d976ee1926dfefcdf1a 127.0.0.1:6380@16380 myself,slave 8b0feb120b68aac489d6a5af9c77dc40d71bc792 0 0 0 connected 8b0feb120b68aac489d6a5af9c77dc40d71bc792 127.0.0.1:6379@16379 master - 0 1705091681202 0 connected 0-16383 127.0.0.1:6380> SPUBLISH hello world (integer) 0 127.0.0.1:6380> MULTI OK 127.0.0.1:6380(TX)> SPUBLISH hello world QUEUED 127.0.0.1:6380(TX)> EXEC (error) MOVED 866 127.0.0.1:6379 ``` With this change: ``` 127.0.0.1:6380> SPUBLISH hello world (integer) 0 127.0.0.1:6380> MULTI OK 127.0.0.1:6380(TX)> SPUBLISH hello world QUEUED 127.0.0.1:6380(TX)> EXEC 1) (integer) 0 ``` --------- Co-authored-by: Harkrishn Patro <harkrisp@amazon.com> Co-authored-by: oranagra <oran@redislabs.com>
`reclaimFilePageCache` did not set `errno` but `rdbSaveInternal` which is logging the error assumed it did. This makes sure `errno` is set. Fixes redis#13245 Signed-off-by: Ted Lyngmo <ted@lyncon.se>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The following PR does the following changes based upon on CPU profile info. The
getNodeByQueryfunction represents 8.2% of an overhead of 12.3% when comparing single shard cluster with standalone.Proposed changes:
The above changes represent an improvement of approximately 5% on the achievable ops/sec as showcase bellow.
results
steps to reproduce
2 Nodes.
DB node (swap ip) :
Client node:
results unstable (804110a)
results this PR (7e1d2ea)