Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#2341] Initialize slots with empty BitSet in RedisClusterNode's constructors #2852

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

zeze1004
Copy link

@zeze1004 zeze1004 commented May 12, 2024

Issue: #2341
This PR supersedes the previously closed PR #2798

Make sure that:

  • You have read the contribution guidelines.
  • You have created a feature request first to discuss your contribution intent. Please reference the feature request ticket number in the pull request.
  • You use the code formatters provided here and have them applied to your changes. Don’t submit any formatting related changes.
  • You submit test cases (unit or integration tests) that back your changes.

@zeze1004
Copy link
Author

@mp911de

I kindly request your review of this pull request. Your feedback would be greatly appreciated. 🙇🏻

@zeze1004
Copy link
Author

I tested everything locally and it passed, but it failed on GitHub Actions. I'll make the necessary fixes and commit again soon.

@tishun
Copy link
Collaborator

tishun commented May 23, 2024

I tested everything locally and it passed, but it failed on GitHub Actions. I'll make the necessary fixes and commit again soon.

Please ignore the failure of the SslIntegrationTests.pubSubSsl:396 - it is unstable and if you pull the latest sources you will see it is disabled

Copy link
Collaborator

@tishun tishun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this!

Please check my comments and let me know if you need any clarifications or have some concerns.

@@ -103,8 +103,7 @@ public RedisClusterNode(RedisURI uri, String nodeId, boolean connected, String s
this.configEpoch = configEpoch;
this.replOffset = -1;

this.slots = new BitSet(slots.length());
this.slots.or(slots);
this.slots = slots != null ? slots : new BitSet(0);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think part of the idea behind the existing code is to clone the content of the slots, instead of directly assigning the provided value.

This has the added benefit of isolation, because the code that called the constructor could no longer modify the contents of the slot (as a side effect).

However you are correct that a good design should consider this method receiving a null for the slots argument. Perhaps something in the line of:

this.slots = new BitSet(SlotHash.SLOT_COUNT);
this.slots.or(slots);

This way:

  • if the slots argument are null then the or() method handles it correctly
  • the this.slots is always initialized with an empty list of slots

IMHO the consistency of always having BitSet initialized with the SlotHash.SLOT_COUNT pays off for the "waste" of memory.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tishun Thank you for your feedback. I have made the suggested changes.

@@ -123,8 +122,9 @@ public RedisClusterNode(RedisClusterNode redisClusterNode) {
this.replOffset = redisClusterNode.replOffset;
this.aliases.addAll(redisClusterNode.aliases);

if (redisClusterNode.slots != null && !redisClusterNode.slots.isEmpty()) {
this.slots = new BitSet(SlotHash.SLOT_COUNT);
this.slots = redisClusterNode.slots != null ? new BitSet(SlotHash.SLOT_COUNT) : new BitSet(0);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the same logic can be followed here. See my previous comment.


assertThat(copiedNode.getSlots()).containsExactly(1, 2);
}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the most important test is the one described in #2341 and it is missing:

  • create one RedisClusterNode instance from the constructor on line 102-103 (parsed nodes use this constructor)
  • create one RedisClusterNode instance from the constructor on line 112-125 (cloned nodes use this constructor)
  • check if RedisClusterNode::hasSameSlotsAs() returns that they are the same

If you see the last mention of an issue in the original issue you'll see that there is also a 3rd constructor (the one on line 78-79) that calls the setSlotBits() method and it would also result in an null slots field if the array is empty. If we had a test for that we would have caught that this condition is not handled right now.

I think a good design would address this and make sure there is no way we set null to the slots field. What do you think?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. I forgot to add the necessary test. I've added it. Could you please review again?

@zeze1004 zeze1004 force-pushed the fix/RedisClusterNode.hasSameSlotsAs() branch from 25d74c3 to 4fe8d21 Compare June 1, 2024 06:36
@zeze1004 zeze1004 requested a review from tishun June 1, 2024 07:17
Copy link
Collaborator

@tishun tishun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey thanks for addressing the last comments.

There are still two things that I am worried about:

  1. We are still missing the same logic on line 90 (the first of the three constructors)
  2. The test is not really verifying the issue from RedisClusterNode.hasSameSlotsAs() is unreliable  #2341

}

@Test
public void testHasDifferentSlotsAs() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is OK, but it tests something different than what is explained in #2341

In #2341 the problems is that one part of the logic uses RedisClusterNode(RedisURI uri, String nodeId, boolean connected, String slaveOf, long pingSentTimestamp, long pongReceivedTimestamp, long configEpoch, BitSet slots, Set<NodeFlag> flags) to create a RedisClusterNode, while another uses the public RedisClusterNode(RedisClusterNode redisClusterNode) constructor. Both are provided an empty BitSet.

When later on both resulting objects are compared with .hasSameSlotsAs() it would return false, when it should return true, because the first object would have an empty slots BitSet, while the second would have a null slots BitSet.

I think the proper test scenario is :

public void testHasDifferentSlotsAs() {
        BitSet emptySlots = new BitSet(SlotHash.SLOT_COUNT);

        RedisClusterNode node1 = new RedisClusterNode(RedisURI.create("localhost", 6379), "nodeId1", true, "slaveOf", 0L, 0L,
                0L, emptySlots, new HashSet<>());
        RedisClusterNode node2 = new RedisClusterNode(node1);

        assertThat(node1.hasSameSlotsAs(node2)).isTrue();

You can test if this case fails when you remove your changes to RedisClusterNode.java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants