Calculation of the Perlin noise derivatives (gradients).

[CandyFiend]

Problem

For example, I am writing a world generator and when inserting a node in a certain place on the surface of the earth, I would like to know which way the height of the surface will grow and at what speed. And depending on the growth rate and direction, it is up to me to decide whether to insert earth or stone. If the mountain is growing and the growth rate is higher than the threshold I set, I could put a stone on top and I would have high cliffs without earthen nodes spoiling the landscape. That is, we predict in advance how the noise value will change when any of the coordinates change and we can make a decision depending on this.

Solutions

As far as I know, it is enough to write functions for calculating partial derivatives of each coordinate. And second-order derivatives will allow us to find out how quickly the noise value will increase or decrease in a given direction. The advantage of this approach is that all this can be learned by working with the coordinate of one node, without calculating the noise values of the surrounding coordinates and without generating the world "in advance" in order to correct it later.

If we recall the school course of mathematical analysis, then the first derivative shows us which way the graph will be tilted. That is, if x increases, then y increases or decreases. The second derivative will show how much the graph is tilted, i.e. how fast the y coordinate changes depending on the x coordinate.

Alternatives

nothing

Additional context

I think this description is enough to justify the usefulness of this feature, especially for developers of LUA generators. I looked in the documentation and searched for this topic in the issues on github, but found nothing. I hope that I explained everything clearly and did not confuse anything. I learned about this approach when I watched a video about generators of worlds and the author of the video said that the authors of "No man's sky" used it for soil erosion.

[CandyFiend]

If I understood correctly, then the code already has calculations of the noise gradient. Can I call them from the LUA API? I also saw the interpolation and would like to be able to call these functions from the LUA API. For example, I use 2d noise to generate rivers and other 2d noise to generate surfaces. I need to make sure that the river is in a depression and for this I could use interpolation of two noises with different settings. It is possible to write interpolation in LUA, but I am not sure how good an idea this is in terms of performance.

I also think it would be useful to be able to calculate the gradient by selecting the desired octaves. Thus, if necessary, you can filter out unnecessary "noise" and see the trend at the desired level of detail.

Are there any plans to add other noises?
https://github.com/minetest/minetest/blob/master/src/noise.cpp

[SmallJoker]

Would approximating the derivatives though a linear delta calculation not be precise enough? You can already get the noise value at a given position, thus checking the neighbours (table lookup) of the heightmap provided by the mapgen should be feasible.

[CandyFiend]

Would approximating the derivatives though a linear delta calculation not be precise enough? You can already get the noise value at a given position, thus checking the neighbours (table lookup) of the heightmap provided by the mapgen should be feasible.

I am currently studying the mapgen object api, apparently you are writing about it. You suggest trying to calculate the values of the first and second order derivatives from the values of the noise functions of the previous and next integer coordinate values. This is:
https://en.wikipedia.org/wiki/Numerical_differentiation
I am interested in the possibility of calculating the value of the derivative without calculating the noise value in the vicinity of the desired node. I.e., when I give the function the coordinates of the node, so that by calculating one function (or two) it gets a number that would tell me that if I increase x, the noise value will increase or decrease by somethat value (i need to understand the "speed" and direction of change). As I wrote above, the derivatives of the noise function of the first and second order solve this problem (although the 2nd order may not be needed in practice). I understand that in the engine, the noise values at the same point are stored, so if I have to calculate the noise value in the vicinity, the noise values of neighboring nodes will be calculated for one node and they will not be calculated later, but will be taken from storage. And then the question is, what will be faster if I take these values, calculate the approximate value of the derivative from them and use lua to implement my storage and store them there, or add storage in the PerlinNoiseMap object, which will store gradients for each coordinate in the form of a table {x = -1, y=0, z=3}. Similar to get_2d(3d)_map, but with stored gradient values. I've seen it in the sources noise.cpp calculating the gradient, why not just save it to another storage. I don't see such a store in the api documentation.

I was planning to use this mechanism in this mod:
https://content.minetest.net/packages/MisterE/luamap/
Its api implies writing a function that is called for each node and should return the id of the node that will be set at this coordinate. set_mapgen_params = singlenode. I suspect that with this parameter, I will not be able to use heightmap, especially considering that I want to generate the landscape myself, because I am not satisfied with the landscape generation of existing mapgen and I cannot fix it (if I can, please tell me how). The mod gives me access to get_2d(3d)_map_flat, but which of the ways to calculate the gradient is better is not completely clear to me.
I wanted to try superimposing noises with different settings on top of each other and generate river valleys and floodplains with one noise, plains with the second noise, and mountains with the third noise. This is just an example. Standard generators do not give me this opportunity, so I wanted to try to do it using a lua generator.

Questions for clarification:

1. The values of the calculated noises are calculated once and stored in the PerliтNoiseMap object. Or they are saved to a buffer that is cleared when the game is restarted?
2. Does get_2d_map return all values without nil, or will i have to call calc_2d_map for nil values and write the result to this table/array?

[SmallJoker]

The values of the calculated noises are calculated once and stored in the PerliтNoiseMap object. Or they are saved to a buffer that is cleared when the game is restarted?

PerlinNoiseMap is an object created by a minetest.get_perlin_map call:

minetest/src/script/lua_api/l_env.cpp

Lines 1123 to 1140 in c4703a7

	// get_perlin_map(noiseparams, size)
	// returns world-specific PerlinNoiseMap
	int ModApiEnv::l_get_perlin_map(lua_State *L)
	{
	GET_ENV_PTR_NO_MAP_LOCK;
	NoiseParams np;
	if (!read_noiseparams(L, 1, &np))
	return 0;
	v3s16 size = read_v3s16(L, 2);
	s32 seed = (s32)(env->getServerMap().getSeed());
	LuaPerlinNoiseMap *n = new LuaPerlinNoiseMap(&np, seed, size);
	*(void **)(lua_newuserdata(L, sizeof(void *))) = n;
	luaL_getmetatable(L, "PerlinNoiseMap");
	lua_setmetatable(L, -2);
	return 1;
	}

The noise values are calculated only when you use a getter function, such as get_2d_map. After that, these values are only preserved within the Lua state, hence freed by the garbage collector or when shutting down the server.

Does get_2d_map return all values without nil, or will i have to call calc_2d_map for nil values and write the result to this table/array?

It creates a new table noise_value = returned_map[y][x], containing only integer values:

minetest/src/script/lua_api/l_noise.cpp

Lines 155 to 176 in c4703a7

	int LuaPerlinNoiseMap::l_get_2d_map(lua_State *L)
	{
	NO_MAP_LOCK_REQUIRED;
	size_t i = 0;
	LuaPerlinNoiseMap *o = checkObject<LuaPerlinNoiseMap>(L, 1);
	v2f p = readParam<v2f>(L, 2);
	Noise *n = o->noise;
	n->perlinMap2D(p.X, p.Y);
	lua_createtable(L, n->sy, 0);
	for (u32 y = 0; y != n->sy; y++) {
	lua_createtable(L, n->sx, 0);
	for (u32 x = 0; x != n->sx; x++) {
	lua_pushnumber(L, n->result[i++]);
	lua_rawseti(L, -2, x + 1);
	}
	lua_rawseti(L, -2, y + 1);
	}
	return 1;
	}

[CandyFiend]

I understood, then I didn't understand, then I started to figure it out and got into the code. I don't understand the c++ source code by almost 90%.

I realized that get_2d_map (or 3d doesn't matter, the difference is in an additional dimension) counts all values from some position (as indicated in the class ref documentation). I still did not understand to what position it calculates. How is this field defined, what is calculated from here to here. What I understood is that through this function we can get a table with these calculated (perhaps even calculated in different threads) values in lua. Unfortunately, there are no comments in the source code, in the best programming traditions.

The calc_2d_map function calculates the same thing, but stores it internally. What does it mean internally? In a variable without access from LUA? Then why use this function? If I call this function in advance for some non-generated mapblock, predicting the direction of the player's movement, and then try to get the value of the same area through get_2d_map, will I get any benefit from the fact that get_2d_map gets into some buffer variable and gets these values without calculating them? I'm just trying to figure out how it works to design a fast generator.

Well, returning to the question of the derivative, is it possible to add the get_2d_gradient_map function, which returns a table by analogy with get_2d_map with items of the table type item = {x = int, y = int, z = int}. Each coordinate is the value of the first-order partial derivative of this variable.

I have already realized that by calculating noise over a region, I can use these values using numerical differentiation methods to approximately calculate the values of the derivative with some accuracy (depending on the method).
Returning to efficiency, are noise values calculated sequentially or in parallel? If in parallel, then the calculation of the gradient by numerical methods can also be parallelized and gain performance. If the calculations occur sequentially, will the calculations in c++ be more optimized (for example, by the compiler) than in lua? In short, is it better to add these calculations in an optimized version once to the api of the engine or for each author of the generator to implement them separately? I'm already silent about the second orders of derivatives, let's see if the get_2d_gradient_map function will be added in any form?