Add Active Object Spatial Cache

[ExeVirus]

This PR adds a simple cache to server active objects (SAOs) based on position. This PR was made to solve the problem outlined in #14613, hoping to improve by orders of magnitude our ability to handle more entities in Minetest, and to make more performant ways to get objects in a given area/radius.

This directly addresses section 2.4 of the current roadmap for minetest. It specifically calls out the old version of the issue this task tries to address.

The implementation is best described as a single-layer of buckets implemented using unordered_multimap, which is a fancy term for "It's sorts entities into buckets the same size as mapblocks (16x16x16)". When you ask for entities in a given area/radius, this map allows you to filter out the vast majority of irrelevant entities.

Performance

5000 Entities - all getting objects inside radius

Simple Comparison

Branch	Max Lag (sec)
Master	3.4549450874329
Spatial Map Only	2.0550429821014
Spatial Map + Supposedly More Performant std::map	2.5252630710602

Relevant lua used for testing:

-- simply create 5000 entities in a tight area, ~36 mapblocks
chatcommand("lag", {
    func = function(name, param)
        for i = 1, 5000 do
            local obj = minetest.add_entity({x = math.random(-48,48), y = 15, z = math.random(-48,48)}, "entity_test:test_entity", nil)
        end
        return true
end})

-- Entity wiggles a bit and does a getObjectsInRadius every step
description = "Test Entity",
on_step = function(self)
        local pos = self.object:get_pos()
        local objects = minetest.get_objects_inside_radius(pos, 5)
        pos.z = pos.z + math.random(-1,1)
        pos.x = pos.x + math.random(-1,1)
        self.object:set_pos(pos)
end,

-- To Measure Max Lag
local function measure()
    local lag = minetest.get_server_max_lag()
    minetest.chat_send_all(lag)
    minetest.log(lag)
    minetest.after(1, function(...)
        measure()
    end)
end
minetest.register_on_joinplayer(function(ObjectRef, last_login)
    minetest.clear_objects()
    measure()
end)

Hot Spot benchmarks

Master:

Function Name	Total CPU [ms, %]	Self CPU [ms, %]
minetestserver (PID: 26420)	3872 (100.00%)	0 (0.00%)
server::ActiveObjectMgr::getObjectsInsideRadius	2535 (65.47%)	2350 (60.69%)
ucrtbase.dll!0x00007ff9cde7fde6	157 (4.05%)	157 (4.05%)
ucrtbase.dll!0x00007ff9cde7f05b	125 (3.23%)	125 (3.23%)
ws2_32.dll!0x00007ff9ce49184a	111 (2.87%)	111 (2.87%)
lua_rawseti	108 (2.79%)	105 (2.71%)

Spatial Map Only:

Function Name	Total CPU [ms, %]	Self CPU [ms, %]
minetestserver (PID: 17296)	5937 (100.00%)	0 (0.00%)
ModifySafeMap > >::get	2340 (39.41%)	2336 (39.35%)
std::_Func_impl_no_alloc<<lambda,void,unsigned short>::_Do_call	3455 (58.19%)	936 (15.77%)
server::SpatialMap::getRelevantObjectIds	3880 (65.35%)	387 (6.52%)
server::SpatialMap::equal_range	339 (5.71%)	339 (5.71%)
ucrtbase.dll!0x00007ff9cde7fde6	217 (3.66%)	217 (3.66%)

Spatial Map + Performant std::map

Function Name	Total CPU [unit, %]	Self CPU [unit, %]	Module
minetestserver (PID: 9600)	5375 (100.00%)	0 (0.00%)	minetestserver
phmap::priv::btree::internal_find	1468 (27.31%)	1466 (27.27%)	minetestserver
std::_Func_impl_no_alloc<<lambda,void,unsigned short>::_Do_call	3110 (57.86%)	871 (16.20%)	minetestserver
ModifySafeMap > >::get	2081 (38.72%)	611 (11.37%)	minetestserver
server::SpatialMap::equal_range	340 (6.33%)	340 (6.33%)	minetestserver
server::SpatialMap::getRelevantObjectIds	3469 (64.54%)	336 (6.25%)	minetestserver
ucrtbase.dll!0x00007ff9cde7fde6	190 (3.53%)	190 (3.53%)	ucrtbase
ucrtbase.dll!0x00007ff9cde7f05b	151 (2.81%)	151 (2.81%)	ucrtbase

Builtin Benchmarking [Pending]

To do

This PR is still pending several features/integrations, but is working as is and Ready for Review. (It could be merged as is, but we'd be missing some further performance gains by using the map in places we currently linear iterate instead, as well as optimizations in the algorithm)

• Create performant, simple position based data structure
• Add benchmarks to verify exactly what the tradeoffs are of this versus current master
• Perform side-by-side benchmarking metrics
• Create initial PR to get code cleaned up for merge
• Integrate new cache with server::ActiveObjectManager
• Ensure cache is updated on every entity move, insertion, and deletion.
• Integrate cache capabilities throughout Minetest, such as helping with per-client SAO updates, etc.
• Make code to match Minetest style guidelines, and integrate into the proper C++ files in the engine
• Run with a full test server to verify nothing is broken (beyond unit tests already performed)

How to test

benchmarking:

1. Enable benchmarking in your c++ cmake settings
2. Build
3. run minetest.exe --run-benchmarks
4. checkout master, and bring over this branch's benchmark_activeobjectmgr.cpp for A to B comparison
5. recompile
6. run minetest.exe --run-benchmarks

Stability/Correctness

I have validated that the benchmarks give the same number of entities when using master and this branch, but have not done enough full runtime on a server testing yet.

[appgurueu]

gave it a quick look

[appgurueu]

Early return would help here

[ExeVirus]

The whole function happens within that if statement, so if it fails: that's early return. Difference in style

[appgurueu]

Yes, I'm aware. I would consider an earlier return to be better style-wise - as in, if empty, we're done, return - because it avoids the indentation level. This is getting pretty deeply nested.

[appgurueu]

This method directly removes entries from m_active_objects, side-stepping removeObject. I believe you need to replace m_active_objects.remove(it.first); with removeObject(it.first); to ensure that you don't end up with dead entries in your spatial index data structure. At least that's what I did.