Status file and Images not generated on a Linux Server

[fabiodinatale]

Hi!

We are trying to run a CFD simulation using a Linux server without graphics interface.
We do not have problems for what concerns the simulation, but with the generation of pictures and of the status file, while the data we would like to collect (i.e. forces and torques) are correctly generated.

Could you please help us in solving this problem?

Below you can find the details of our defines.hpp and of the part of the setup.cpp file with the instructions for generating the images:

defines.hpp

#pragma once

//#define D2Q9 // choose D2Q9 velocity set for 2D; allocates 53 (FP32) or 35 (FP16) Bytes/cell
//#define D3Q15 // choose D3Q15 velocity set for 3D; allocates 77 (FP32) or 47 (FP16) Bytes/cell
#define D3Q19 // choose D3Q19 velocity set for 3D; allocates 93 (FP32) or 55 (FP16) Bytes/cell; (default)
//#define D3Q27 // choose D3Q27 velocity set for 3D; allocates 125 (FP32) or 71 (FP16) Bytes/cell

#define SRT // choose single-relaxation-time LBM collision operator; (default)
//#define TRT // choose two-relaxation-time LBM collision operator

//#define FP16S // compress LBM DDFs to range-shifted IEEE-754 FP16; number conversion is done in hardware; all arithmetic is still done in FP32
#define FP16C // compress LBM DDFs to more accurate custom FP16C format; number conversion is emulated in software; all arithmetic is still done in FP32

//#define BENCHMARK // disable all extensions and setups and run benchmark setup instead

//#define VOLUME_FORCE // enables global force per volume in one direction (equivalent to a pressure gradient); specified in the LBM class constructor; the force can be changed on-the-fly between time steps at no performance cost
#define FORCE_FIELD // enables computing the forces on solid boundaries with lbm.calculate_force_on_boundaries(); and enables setting the force for each lattice point independently (enable VOLUME_FORCE too); allocates an extra 12 Bytes/cell
#define EQUILIBRIUM_BOUNDARIES // enables fixing the velocity/density by marking cells with TYPE_E; can be used for inflow/outflow; does not reflect shock waves
//#define MOVING_BOUNDARIES // enables moving solids: set solid cells to TYPE_S and set their velocity u unequal to zero
//#define SURFACE // enables free surface LBM: mark fluid cells with TYPE_F; at initialization the TYPE_I interface and TYPE_G gas domains will automatically be completed; allocates an extra 12 Bytes/cell
//#define TEMPERATURE // enables temperature extension; set fixed-temperature cells with TYPE_T (similar to EQUILIBRIUM_BOUNDARIES); allocates an extra 32 (FP32) or 18 (FP16) Bytes/cell
#define SUBGRID // enables Smagorinsky-Lilly subgrid turbulence LES model to keep simulations with very large Reynolds number stable
//#define PARTICLES // enables particles with immersed-boundary method (for 2-way coupling also activate VOLUME_FORCE and FORCE_FIELD; only supported in single-GPU)

//#define INTERACTIVE_GRAPHICS // enable interactive graphics; start/pause the simulation by pressing P; either Windows or Linux X11 desktop must be available; on Linux: change to "compile on Linux with X11" command in make.sh
//#define INTERACTIVE_GRAPHICS_ASCII // enable interactive graphics in ASCII mode the console; start/pause the simulation by pressing P
#define GRAPHICS // run FluidX3D in the console, but still enable graphics functionality for writing rendered frames to the hard drive

#define GRAPHICS_FRAME_WIDTH 1920 // set frame width if only GRAPHICS is enabled
#define GRAPHICS_FRAME_HEIGHT 1080 // set frame height if only GRAPHICS is enabled
#define GRAPHICS_BACKGROUND_COLOR 0x000000 // set background color; black background (default) = 0x000000, white background = 0xFFFFFF
//#define GRAPHICS_TRANSPARENCY 0.7f // optional: comment/uncomment this line to disable/enable semi-transparent rendering (looks better but reduces framerate), number represents transparency (equal to 1-opacity) (default: 0.7f)
#define GRAPHICS_U_MAX 0.25f // maximum velocity for velocity coloring in units of LBM lattice speed of sound (c=1/sqrt(3)) (default: 0.25f)
#define GRAPHICS_Q_CRITERION 0.0001f // Q-criterion value for Q-criterion isosurface visualization (default: 0.0001f)
#define GRAPHICS_F_MAX 0.002f // maximum force in LBM units for visualization of forces on solid boundaries if VOLUME_FORCE is enabled and lbm.calculate_force_on_boundaries(); is called (default: 0.002f)
#define GRAPHICS_STREAMLINE_SPARSE 4 // set how many streamlines there are every x lattice points
#define GRAPHICS_STREAMLINE_LENGTH 128 // set maximum length of streamlines
#define GRAPHICS_RAYTRACING_TRANSMITTANCE 0.25f // transmitted light fraction in raytracing graphics ("0.25f" = 1/4 of light is transmitted and 3/4 is absorbed along longest box side length, "1.0f" = no absorption)
#define GRAPHICS_RAYTRACING_COLOR 0x005F7F // absorption color of fluid in raytracing graphics

#define GRAPHICS_T_DELTA 0.25f // maximum velocity for velocity coloring in units of LBM lattice speed of sound (c=1/sqrt(3)) (default: 0.25f)
#define GRAPHICS_RHO_DELTA 0.25f // maximum velocity for velocity coloring in units of LBM lattice speed of sound (c=1/sqrt(3)) (default: 0.25f)


// #############################################################################################################

#define TYPE_S 0b00000001 // (stationary or moving) solid boundary
#define TYPE_E 0b00000010 // equilibrium boundary (inflow/outflow)
#define TYPE_T 0b00000100 // temperature boundary
#define TYPE_F 0b00001000 // fluid
#define TYPE_I 0b00010000 // interface
#define TYPE_G 0b00100000 // gas
#define TYPE_X 0b01000000 // reserved type X
#define TYPE_Y 0b10000000 // reserved type Y

#define VIS_FLAG_LATTICE  0b00000001 // lbm.graphics.visualization_modes = VIS_...|VIS_...|VIS_...;
#define VIS_FLAG_SURFACE  0b00000010
#define VIS_FIELD         0b00000100
#define VIS_STREAMLINES   0b00001000
#define VIS_Q_CRITERION   0b00010000
#define VIS_PHI_RASTERIZE 0b00100000
#define VIS_PHI_RAYTRACE  0b01000000
#define VIS_PARTICLES     0b10000000

#if defined(FP16S) || defined(FP16C)
#define fpxx ushort
#else // FP32
#define fpxx float
#endif // FP32

#ifdef BENCHMARK
#undef UPDATE_FIELDS
#undef VOLUME_FORCE
#undef FORCE_FIELD
#undef MOVING_BOUNDARIES
#undef EQUILIBRIUM_BOUNDARIES
#undef SURFACE
#undef TEMPERATURE
#undef SUBGRID
#undef PARTICLES
#undef INTERACTIVE_GRAPHICS
#undef INTERACTIVE_GRAPHICS_ASCII
#undef GRAPHICS
#endif // BENCHMARK

#ifdef SURFACE // (rho, u) need to be updated exactly every LBM step
#define UPDATE_FIELDS // update (rho, u, T) in every LBM step
#endif // SURFACE

#ifdef TEMPERATURE
#define VOLUME_FORCE
#endif // TEMPERATURE

#ifdef PARTICLES // (rho, u) need to be updated exactly every LBM step
#define UPDATE_FIELDS // update (rho, u, T) in every LBM step
#endif // PARTICLES

#if defined(INTERACTIVE_GRAPHICS) || defined(INTERACTIVE_GRAPHICS_ASCII)
#define GRAPHICS
#define UPDATE_FIELDS // to prevent flickering artifacts in interactive graphics
#endif // INTERACTIVE_GRAPHICS || INTERACTIVE_GRAPHICS_ASCII

setup.cpp

	// ####################################################################### run simulation, export images and data ##############################################################################

	// Get the current time
    std::time_t current_time = std::time(nullptr);

    // Convert the current time to a struct tm (time information)
    struct std::tm time_info;

	//#ifdef _WIN32
	localtime_s(&time_info, &current_time);
	//#else
	//localtime_r(&time_info, &current_time);
	//#endif
	std::string date_str = std::to_string(time_info.tm_year + 1900) + '_' + std::to_string(time_info.tm_mon + 1) + '_' + std::to_string(time_info.tm_mday);
	std::string time_str = std::to_string(time_info.tm_hour) + '_' + std::to_string(time_info.tm_min) + '_' + std::to_string(time_info.tm_sec);
	std::string test_name = date_str + "_" + time_str;
	
	lbm.graphics.visualization_modes = VIS_Q_CRITERION;
	lbm.run(0u); 			// initialize simulation
	lbm.write_status(get_exe_path() + "export/" + test_name + "/");		// write simulation status to file
	
	// set up data export
	#if defined(FP16S)
		const string data_path = get_exe_path()+"export/"+test_name+"/data/FP16S/";
	#elif defined(FP16C)
		const string data_path = get_exe_path()+"export/"+test_name+"/data/FP16C/";
	#else // FP32
		const string data_path = get_exe_path()+"export/"+test_name+"/data/FP32/";
	#endif // FP32
	const uint prec = 6u;																	// precision of data output
	write_file(data_path+"forces.dat",  "#      t\t      CdA\t      ClA\t      CsA\n");		// initialize forces data file
	write_file(data_path+"torques.dat", "#      t\t      CrAl\t      CpAl\t      CyAl\n");	// initialize torques data file

	// MAIN SIMULATION LOOP
	while(lbm.get_t()<=units.t(si_T)) {
		//#if defined(GRAPHICS) && !defined(INTERACTIVE_GRAPHICS)
			if(lbm.graphics.next_frame(units.t(si_T), video_length)) {
				// set up camera position and write frame to file for 60 Hz video acquisition
				// side view
				lbm.graphics.set_camera_centered(-67.0f, 21.0f, 100.0f, 1.25f);	// camera position: (Rx, Ry, FOV, zoom)
				lbm.graphics.write_frame(get_exe_path()+"export/"+test_name+"/png/");
				
				// front view
				lbm.graphics.set_camera_centered(-90.0f, 0.0f, 100.0f, 1.25f);	// camera position: (Rx, Ry, FOV, zoom)
				lbm.graphics.write_frame(get_exe_path() + "export/" + test_name + "/png/");
			}
		//#endif // GRAPHICS && !INTERACTIVE_GRAPHICS
		lbm.run(lbm_dt); // run 1 time step
		
		#if defined(FORCE_FIELD)
			// compute force and torques on robot
		if (lbm.get_t()%10==0) {
			lbm.calculate_force_on_boundaries();
			lbm.F.read_from_device();
			const float3 lbm_force = lbm.calculate_force_on_object(TYPE_S | TYPE_X); // compute total force on object (TYPE_S)
			const float3 lbm_torque = lbm.calculate_torque_on_object(robot_center, TYPE_S | TYPE_X); // compute total torque on object (TYPE_S)
			
			// compute force areas and torque coefficients on robot in SI units
			const float CdA = units.si_F(lbm_force.y) / (0.5f * si_rho * sq(si_u));
			const float ClA = units.si_F(lbm_force.z) / (0.5f * si_rho * sq(si_u));
			const float CsA = -units.si_F(lbm_force.x) / (0.5f * si_rho * sq(si_u));
			const float CrAl = units.si_T(lbm_torque.y) / (0.5f * si_rho * sq(si_u));
			const float CpAl = -units.si_T(lbm_torque.x) / (0.5f * si_rho * sq(si_u));
			const float CyAl = units.si_T(lbm_torque.z) / (0.5f * si_rho * sq(si_u));
			
			// write force and torque data to file
			write_line(data_path + "forces.dat", to_string(lbm.get_t()) + "\t" + to_string(units.si_t(lbm.get_t()), prec) + "\t" + to_string(CdA, prec) + "\t" + to_string(ClA, prec) + "\t" + to_string(CsA, prec) + "\n");
			write_line(data_path + "torques.dat", to_string(lbm.get_t()) + "\t" + to_string(units.si_t(lbm.get_t()), prec) + "\t" + to_string(CrAl, prec) + "\t" + to_string(CpAl, prec) + "\t" + to_string(CyAl, prec) + "\n");
		}
		#endif

	// write simulation status to file and exit
	lbm.write_status(get_exe_path()+"export/"+test_name+"/");
} /**/

The same code on a Windows laptop does not produce the same problem.

Thanks in advance!

[ProjectPhysX]

Hi @fabiodinatale,

I can't reproduce this issue, neither on my Windows 10 system nor on my Linux system. I see the files forces.dat, torques.dat, the .png images, and the status files correctly generated on both systems.

I suspect this to be an issue with an old g++ compiler version, which might not correctly support <filesystem>. In this case, if the filesystem folder tree in which you want to write the files into is not present before writing files, C++ will not write the files at all. Which g++ --version do you have installed? Try updating, to at least g++-9.

Kind regards,
Moritz

[fabiodinatale]

Hi @ProjectPhysX, first of all, thanks for the tests you performed and for your reply.

Running the command g++ --version on the server, this is the result I get:

fdinatale@iitbmp014srv001:~$ g++ --version
g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

i.e. the system has the version 11.4.0 of g++.

I also tried to run the code in a conda environment and in this case the compiler is the following one:

(fluidx3d) fdinatale@iitbmp014srv001:~$ g++ --version
g++ (conda-forge gcc 12.3.0-5) 12.3.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

so it is even newer.

Actually, I discovered that pictures and status file are created but in my root directory. You can see them as they are the ones named 02.png and 02.txt respectively:

(fluidx3d) fdinatale@**********srv001:~$ ls -la
total 280
drwxr-xr-x 14 fdinatale domain users   4096 May  3 09:20 .
drwxr-xr-x 19 root      root           4096 Nov 29 09:46 ..
drwxr-xr-x  3 fdinatale domain users     22 Apr 10 12:58 02.FluidX3D
-rw-r--r--  1 fdinatale domain users 192494 Apr 29 15:27 02.png
-rw-r--r--  1 fdinatale domain users    337 Apr 29 15:26 02.txt
-rw-------  1 fdinatale domain users  25585 May  3 08:58 .bash_history
-rw-r--r--  1 fdinatale domain users    220 Oct 17  2023 .bash_logout
-rw-r--r--  1 fdinatale domain users   4667 Apr 10 12:42 .bashrc
drwx------  3 fdinatale domain users     59 Apr 10 12:46 .cache
drwxr-xr-x  2 fdinatale domain users     30 Apr 10 12:47 .conda
-rw-r--r--  1 fdinatale domain users     52 Apr 10 12:43 .condarc
drwx------  4 fdinatale domain users     70 Apr 19 15:29 .config
-rw-r--r--  1 fdinatale domain users   7571 Apr 19 17:21 .cxlayout.ini
drwxr-xr-x  3 fdinatale domain users     27 Oct 20  2023 .local
drwxr-xr-x 19 fdinatale domain users   4096 Apr 10 12:52 miniforge3
drwx------  3 fdinatale domain users     26 Oct 20  2023 .nv
-rw-r--r--  1 fdinatale domain users    807 Jan  8 14:35 .profile
drwx------  2 fdinatale domain users     60 Oct 20  2023 .ssh
-rw-------  1 fdinatale domain users    244 May  3 09:20 .Xauthority

I don't know if this information can help.

Thanks!

[ProjectPhysX]

The compiler is fine. Can you share your entire file path into which the images should be written, and is there any special characters in there? FluidX3D is doing some string manipulation on the file path, maybe this fails in your case.

[fabiodinatale]

Thanks @ProjectPhysX for your feedback.
The complete path where the images should be generated is: /home/iit.local/fdinatale/02.FluidX3D/FluidX3D/export/test_name/png, where test_name is a string composed by the date and time of the test itself.

Anyhow, I tried to give as input in both write_status and lbm.write_frame function the name of the files to be generated and now both the status file and the pictures are generated in /home/iit.local/fdinatale/02.FluidX3D/FluidX3D.