Rend2 is not correctly building when cross compiling using mingw from Linux to Windows x64 due to compact_glsl issues

[veosar]

Reporting a bug? Please make sure you've given the following information - thanks!

Operating system and version:

Building inside docker on Ubuntu 22.04 with mingw (Dockerfile provided in my repo to reproduce), target is Windows x64 using OpenJK provided toolchain.

Is this for single player or multiplayer?

MP

Description of the bug (and if possible, steps to reproduce the bug):

Here is my fork with additional dockerfile added to show my setup and allow to reproduce in simplest way possible:

https://github.com/veosar/OpenJK-Fixes/tree/issues/rend2-mingw-crosscompile

You can clone the repo or just copy the Dockerfile-Rend2Issue to your local repo. After you do that, go into OpenJK main directory and run command docker build -f Dockerfile-Rend2Issue .. Firstly, it will fail with this issue:

74.13 [ 83%] Building CXX object codemp/rd-vanilla/CMakeFiles/rd-vanilla_x86.dir/__/__/shared/qcommon/safe/string.cpp.obj
74.40 [ 84%] Linking CXX shared library ../../rd-vanilla_x86.dll
75.00 [ 84%] Built target rd-vanilla_x86
75.01 [ 84%] Building CXX object codemp/rd-rend2/CMakeFiles/compact_glsl.dir/glsl/compact.cpp.obj
75.31 In file included from /usr/src/openjk/codemp/rd-rend2/tr_local.h:38,
75.31                  from /usr/src/openjk/codemp/rd-rend2/glsl/compact.cpp:10:
75.31 /usr/src/openjk/codemp/rd-rend2/qgl.h:7:11: fatal error: gl/gl.h: No such file or directory
75.31     7 | # include <gl/gl.h>
75.31       |           ^~~~~~~~~
75.31 compilation terminated.
75.31 make[2]: *** [codemp/rd-rend2/CMakeFiles/compact_glsl.dir/build.make:77: codemp/rd-rend2/CMakeFiles/compact_glsl.dir/glsl/compact.cpp.obj] Error 1
75.31 make[1]: *** [CMakeFiles/Makefile2:503: codemp/rd-rend2/CMakeFiles/compact_glsl.dir/all] Error 2
75.31 make: *** [Makefile:156: all] Error 2
------
Dockerfile-Rend2Issue:15
--------------------
  14 |     # Build x86_64 windows, cross compiled arch
  15 | >>> RUN mkdir /usr/src/openjk/build.x86_64_win &&\
  16 | >>>      cd /usr/src/openjk/build.x86_64_win &&\
  17 | >>>      cmake -DCMAKE_TOOLCHAIN_FILE=/usr/src/openjk/cmake/Toolchains/x86_64-w64-mingw32.cmake -DCMAKE_INSTALL_PREFIX=/opt \
  18 | >>>              -DBuildMPCGame=OFF -DBuildMPEngine=OFF -DBuildMPUI=OFF -DBUildMPDed=OFF\
  19 | >>>              -DBuildSPEngine=OFF -DBuildSPGame=OFF -DBuildSPRdVanilla=OFF \
  20 | >>>              .. &&\
  21 | >>>      make &&\
  22 | >>>      make install
  23 |
--------------------
ERROR: failed to solve: process "/bin/sh -c mkdir /usr/src/openjk/build.x86_64_win &&\tcd /usr/src/openjk/build.x86_64_win &&\tcmake -DCMAKE_TOOLCHAIN_FILE=/usr/src/openjk/cmake/Toolchains/x86_64-w64-mingw32.cmake -DCMAKE_INSTALL_PREFIX=/opt \t\t-DBuildMPCGame=OFF -DBuildMPEngine=OFF -DBuildMPUI=OFF -DBUildMPDed=OFF\t\t-DBuildSPEngine=OFF -DBuildSPGame=OFF -DBuildSPRdVanilla=OFF \t\t.. &&\tmake &&\tmake install" did not complete successfully: exit code: 2

I think this is due to these lines of code:

#elif defined( _WIN32 )
#	include <windows.h>
#	include <gl/gl.h>

Trying to include lowercase gl due to WIN32 flag that do not exist in Linux environment.

I will just change this #include <gl/gl.h> to <GL/gl.h> just to get through this quickly.

After you bypass this issue and continue, you get this error:

73.96 [ 85%] Linking CXX executable ../../compact_glsl.exe
74.16 [ 85%] Built target compact_glsl
74.17 [ 85%] Generating glsl_shaders.cpp, glsl_shaders.h
74.17 /bin/sh: 1: compact_glsl: not found
74.17 make[2]: *** [codemp/rd-rend2/CMakeFiles/rd-rend2_x86.dir/build.make:94: codemp/rd-rend2/glsl_shaders.cpp] Error 127
74.17 make[1]: *** [CMakeFiles/Makefile2:476: codemp/rd-rend2/CMakeFiles/rd-rend2_x86.dir/all] Error 2
74.17 make: *** [Makefile:156: all] Error 2

Probably due to the previous _WIN32 flag (I don't know), compact_glsl seems to be building with .exe extension and therefore cmake can't find it.

I guess it could be fixed by using MINGW flag in if conditions in some capacity. I couldn't find a clean way myself though.

Of course the issue can be bypassed in numerous ways, but it's a fact that in current state it does not work OOTB.

What did you expect to happen instead?

Rend2 files and compact_glsl should recognize correctly that it's being built on Linux against Windows, not being built on Windows. In scenario provided it should build seamlessly, just like RD Vanilla.