Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add connection contexts to hash table #3036

Merged
merged 84 commits into from
May 28, 2024

Conversation

matthewige
Copy link
Collaborator

@matthewige matthewige commented Nov 9, 2023

Description

  • Currently, the connection contexts are stored in a linked list. This change adds them to a RTL_AVL_TABLE to allow for faster lookups and uses the table for lookup instead of the linked list.
  • The logic to purge stale entries remains, but shifted to just enumerating the entries in the table.
  • This change also adds a pre-allocated list of entries, which are reserved for use when allocating an entry fails. When checking for an existing context, the table will be checked first, and then the linked list (which should only be used under low memory conditions).

Closes #1675 and #2996

Testing

CI/CD. No additional testing was added.

Documentation

N/A

Installation

N/A

@matthewige matthewige marked this pull request as ready for review November 9, 2023 18:24
@matthewige matthewige changed the title [DRAFT] Add connection contexts to hash table Add connection contexts to hash table Nov 9, 2023
@matthewige matthewige linked an issue Nov 9, 2023 that may be closed by this pull request
@matthewige matthewige marked this pull request as ready for review March 28, 2024 19:39
@Alan-Jowett
Copy link
Member

Copilot summary:

This pull request introduces a series of changes to the netebpfext/net_ebpf_ext_sock_addr.c file to improve memory management and performance. The major theme of these changes is the introduction of a new structure for managing connection contexts and the replacement of a simple list with a hash table and a low-memory list. The changes also include the addition of a counter for the number of times a pre-allocated low memory context was used, and the modification of several functions to accommodate these changes.

Memory management and performance improvements:

  • netebpfext/net_ebpf_ext_sock_addr.c: Added a new macro LOW_MEMORY_CONNECTION_CONTEXT_COUNT to define the number of pre-allocated low memory contexts.
  • netebpfext/net_ebpf_ext_sock_addr.c: Introduced a new structure _net_ebpf_ext_sock_addr_connection_contexts to manage connection contexts. This structure includes a hash table for storing blocked connection contexts, a list for storing pre-allocated contexts for low memory conditions, and a list for storing blocked contexts under low memory conditions.
  • netebpfext/net_ebpf_ext_sock_addr.c: Modified _ebpf_sock_addr_context_destroy( to acquire an exclusive lock on _net_ebpf_ext_sock_addr_blocked_contexts.lock and to purge blocked connect contexts.
  • netebpfext/net_ebpf_ext_sock_addr.c: Replaced the function _net_ebpf_sock_addr_initialize_globals() with _net_ebpf_sock_addr_initialize_blocked_connection_contexts() and _net_ebpf_ext_uninitialize_blocked_connection_contexts(), which initialize and uninitialize the blocked connection contexts, respectively. These functions also manage the pre-allocated contexts for low memory conditions.
  • netebpfext/net_ebpf_ext_sock_addr.c: Modified _net_ebpf_extension_connection_context_initialize and _net_ebpf_ext_get_and_remove_connection_context to accommodate the new structure for managing connection contexts. [1] [2]
  • netebpfext/net_ebpf_ext_sock_addr.c: Modified _net_ebpf_ext_insert_connection_context_to_list to insert connection contexts into the hash table or the low-memory list, depending on the memory conditions.
  • netebpfext/net_ebpf_ext_sock_addr.c: Modified net_ebpf_ext_sock_addr_register_providers() and net_ebpf_ext_sock_addr_unregister_providers() to call the new functions for initializing and uninitializing the blocked connection contexts. [1] [2] [3]
  • netebpfext/net_ebpf_ext_sock_addr.c: Modified net_ebpf_extension_sock_addr_authorize_connection_classify and net_ebpf_extension_sock_addr_redirect_connection_classify to use the new structure for managing connection contexts. [1] [2]

netebpfext/net_ebpf_ext_sock_addr.c Outdated Show resolved Hide resolved
netebpfext/net_ebpf_ext_sock_addr.c Outdated Show resolved Hide resolved
@mtfriesen
Copy link
Collaborator

@matthewige we should be clear that the RTL_AVL_TABLE is actually an AVL tree, and is not a hash table. If either an AVL tree or hash table offers acceptable performance, then referring to this RTL data structure simply as a "table" in code comments should be OK and not misleading.

netebpfext/net_ebpf_ext_sock_addr.c Outdated Show resolved Hide resolved
netebpfext/net_ebpf_ext_sock_addr.c Outdated Show resolved Hide resolved
netebpfext/net_ebpf_ext_sock_addr.c Outdated Show resolved Hide resolved
netebpfext/net_ebpf_ext_sock_addr.c Outdated Show resolved Hide resolved
netebpfext/net_ebpf_ext_sock_addr.c Show resolved Hide resolved
@saxena-anurag saxena-anurag added this pull request to the merge queue May 28, 2024
Merged via the queue into microsoft:main with commit 366fd7c May 28, 2024
84 of 86 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
7 participants