Fix windows build.

[redpolline]

Should fix #3092.

[redpolline]

Well that's odd, the Windows CI build doesn't seem to set CMAKE_SYSTEM_NAME to Windows. Maybe try with VCPKG_TARGET_TRIPLET....

[redpolline]

Well that last one was a build system setup error: [NuGet] Response status code does not indicate success: 500 (Internal Server Error).

[redpolline]

OK, The CI is setting CMAKE_SYSTEM_NAME to Windows. So why is it claiming that CMP0144 isn't being set?

[redpolline]

OK, defining the CMP default on the command line works. (I guess CMP0069 isn't working here either then....) So that's one problem down.

Now just I need to figure out why the CI build is rejecting all of the boost libraries. So let's try setting Boost_DEBUG and Boost_VERBOSE.

[redpolline]

OK, it's seemingly due to the Boost_COMPILER being wrong. (vc140 instead of vc143.) Let's try to override it manually and see if it works.

[redpolline]

For best results just have the CI use the vcpkg installed boost.

[Macdu]

Please do not modify the CI files, there is no reason to do so as it works perfectly fine right now.

[Macdu]

Suggested change

	if (NOT VITA3K_Boost_FOUND)
	if (NOT TARGET Boost::boost)

[redpolline]

As I've stated in the long post below, this won't work.

Boost already tries to do this itself in it's boost_*-config.cmake files as the very first functional statement, and again in the various variant files (for me at line 74 in both). In all cases Boost's own attempts fail.

Sadly, that means we have to keep track of this ourselves.

[Macdu]

How can boost try to use both a Debug and Release build? It will by default use a build matching the current target build type

[redpolline]

When cmake is initially called with the preset option, the build configuration isn't set by VS2022. As such everything winds up defaulting to the Debug build as that's the first entry in cmake's default configuration list. This causes Boost to add it's debug dependencies to the build when get_boost() calls find_package().

Later, when cmake is invoked again with the build option, the config option is also given. This sets the build configuration to a specific type. (Whatever the user selected in VS2022's GUI.) As long as that configuration was "Debug" the build will succeed. As get_boost()'s call to find_package() will effectively be a no-op.

If the user has selected a different build configuration however, then the repeated call to find_package() will append a new Boost library and set of include headers to the target. VS2022 can not handle this, and thus we get #3092.

Re-running the build, without changing the build configuration in VS2022's GUI again, however will get rid of the old incorrect dependency and allow the build to succeed, but that's inconvenient to those trying to follow the manual build instructions for Windows. We also cannot remove the incorrect dependency ourselves once it's added, as cmake doesn't provide a reliable option / command to do so. As such our only option is to prevent the second call to find_package() from being made in the first place.

As to my theory as to why this behavior happens and why the work around, well, works:

In my build of Vita3K's external boost::filesystem we have these files at Vita3K\build\windows-vs2022\external\boost\lib\cmake\boost_filesystem-1.82.0:
boost_filesystem-config.cmake
boost_filesystem-config-version.cmake
libboost_filesystem-variant-vc143-mt-gd-x64-1_82-static.cmake
libboost_filesystem-variant-vc143-mt-x64-1_82-static.cmake

The actual call to add_library() / set_target_properties() is in the variant files. (in both variants at line 75 & 77 for me.) With the config files effectively re-implementing cmake's generator expressions via the various flags: Boost_USE_SINGLE_VARIANT / Boost_USE_RELEASE_LIBS / Boost_USE_DEBUG_LIBS / etc. (I guess this predates cmake's generator expressions? But there is limited use of them....)

There is code in there to block calling add_library() / set_target_properties() using the same suggested change you made above, i.e. if(NOT TARGET Boost::filesystem), but it fails here. My guess is that cmake isn't keeping all of the information between cmake invocations because of the changed build configuration type, but cmake is reusing the previous list of libraries and includes for each project as a template. As such, if (NOT TARGET ...) fails as cmake needs to reconfigure it based on the new build configuration.

Which makes sense. That's the entire point of cmake's generator expressions. Cmake's preset stage is used to generate the default dependency list. Then cmake's build stage, with it's defined build configuration type, is only supposed to append what is specifically needed for that specific build configuration. I.e. Cmake's build stage is not supposed to redefine core dependencies common to all build configurations.

[Macdu]

Debug builds are meant to be used for ... debug builds.
As its name implies RelWithDebugInfo is a release build (with debug info), only use release builds dependencies with it.

[redpolline]

OK, turns out that's actually not required to fix #3092. So I've pushed a new commit that removes that segment and it's associated comment.

[redpolline]

Please do not modify the CI files, there is no reason to do so as it works perfectly fine right now.

There are two reasons that I did so:

1. The version of VS installed on the Windows CI is out of date.

This is the reason why this PR has multiple force-pushes against it. The version of Boost in Vita3K's external/ expects vs143 for it to be deemed "usable" by Boost's own cmake files, but the version installed on the CI is vs140. As such, cmake rejects the Boost in external/ and fails the build attempt unless we force it to use the Boost installed by vcpkg. I assume this was the expectation anyway given that the Windows CI was explicitly configured to download and install Boost via vcpkg. Forcing this to happen requires enabling VITA3K_FORCE_SYSTEM_BOOST and setting Boost_ROOT to the vcpkg install directory.

As this is an intentional design decision with Vita3K's CI, and not an issue with vcpkg in general, I decided to place those options in the CI config so that others could use vcpkg to build Vita3K without forcing them to use a system provided Boost library.

2. The cmake policies set in CMakeLists.txt are being unset by the Windows CI.

This seems to be a bug with vcpkg. As I ran some tests before and after against the Windows CI above and confirmed that while the set_policy(CMP_0144) command used in CMakeLists.txt did set the policy as instructed, it was unset at some point before / during the call to find_package().

This behavior doesn't occur under any other build environment for any OS. Not even under VS2022 Community. Regardless, we cannot currently expect that the the policies set in CMakeLists.txt will actually be used by the CI. So we have no other means to set them except as an argument to cmake. I can try to remove them, but that means the old deprecated cmake behavior will be re-enabled and warnings will start showing up again in the CI logs. (Assuming that removing CMP_0144 doesn't break the build again. It was required under VS2022 Community, and is also required by the settings I mentioned above. As CMP_0144 controls the behavior of find_package() when *_ROOT is set. The old deprecated behavior of CMP_0144 is to ignore that setting for compatibility with older CMakeLists.txt files.)

[Macdu]

Using vcpkg is really straightforward, just add the cmake toolchain and use find_package, that's all. There should be no mention of vcpkg in the CMakeList file.
If custom boost is not being used, external/boost shouldn't even be looked at.

About CMP_0144, it was added in 3.27 and we require 3.17, to fix it. You say that if the build system is windows it is at least CMake 3.27 while if it not the cmake version is less than 3.17...
You can just use BOOSTROOT instead of BOOST_ROOT and we will be done (or set the minimum version to 3.17). Also removing CMP_0144 won't break anything, it's just that Boost was already using BOOST_ROOT in its cmake file before this policy was introduced.

The vc140 issue is absolutely not because of the CI being out of date (it is always up to date which can be kind of annoying if vs suddenly decides to break something) but because of this https://github.com/microsoft/vcpkg/blob/3dd44b931481d7a8e9ba412621fa810232b66289/scripts/buildsystems/vcpkg.cmake#L825 vcpkg is assuming that if boost is installed we let him do its stuff which is not the best but fine except if you do weird not useful things like in this PR.

[redpolline]

Using vcpkg is really straightforward, just add the cmake toolchain and use find_package, that's all. There should be no mention of vcpkg in the CMakeList file.

Done. All of the references should be removed in the latest push.

About CMP_0144, it was added in 3.27 and we require 3.17, to fix it. You say that if the build system is windows it is at least CMake 3.27 while if it not the cmake version is less than 3.17...

Done.

The vc140 issue is absolutely not because of the CI being out of date (it is always up to date which can be kind of annoying if vs suddenly decides to break something) but because of this https://github.com/microsoft/vcpkg/blob/3dd44b931481d7a8e9ba412621fa810232b66289/scripts/buildsystems/vcpkg.cmake#L825 vcpkg is assuming that if boost is installed we let him do its stuff which is not the best but fine except if you do weird not useful things like in this PR.

Thanks for the link. I suspected that something like this was going on, but hadn't had the time to hunt it down. (I'll pin that for later reference.)

If custom boost is not being used, external/boost shouldn't even be looked at.

The current behavior in main supports 3 modes for Boost: 1) If VITA3K_FORCE_SYSTEM_BOOST is enabled, then Boost is assumed to be provided by the cmake host system and found by calling find_package() once. (Optionally, it can be given a hint as to where on the host to look for it.) It will cause a fatal error if this assumption is false. 2) If both VITA3K_FORCE_SYSTEM_BOOST and VITA3K_FORCE_CUSTOM_BOOST are disabled, then we attempt to ask the cmake host system for a Boost distribution and use it if it's found. If it's not found we fall back to.... 3) If VITA3K_FORCE_SYSTEM_BOOST is disabled and VITA3K_FORCE_CUSTOM_BOOST is enabled -OR- the cmake host system did not provide a Boost distribution in 2, we automatically fallback to compiling and using the in-tree Boost distribution in external/boost. Most users compiling Vita3K will wind up using mode 3. (Windows / MacOS) as their build instructions don't involve downloading and installing a Boost distribution. The Linux users don't include that in their build instructions either, but Linux users are more likely to have Boost installed anyway by their distro. Which means Linux users can wind up using mode 2 or 3 based on their specific linux distro and what other packages they already have installed on their system. Mode 1 is mostly for package builders / distribution maintainers. Where a specific Boost distribution is mandatory. I.e. Linux Distros and the CI. Are you saying that the default should be mode 1 and that using external/boost (i.e mode 3) should be _explicitly_ enabled if it is to be used? If so, that can be done, but we'll have to change the presets to enable custom mode for most builds and do some kind of detection for the CI so we don't enable mode 3 for it....

[Zangetsu38]

i have tested after clean my build folder broken again build
And now is fixed, no any boost error in link 👌

[Macdu]

Don't set a different minimum boost version depending on whether you are using Windows or Linux, this makes no sense. Just use BOOSTROOT instead of BOOST_ROOT and the warning will go away.

Also as I said the default behavior is:

• look if cmake is able to find itself a boost distribution
• if it isn't able, build boost yourself
  There should be no reason to have to modify the workflow file for this.

Finally the boost custom build makes both the debug and release version. It should really have no issue. Using VITA3K_Boost_FOUND and other stuff sounds really hacky and unnecessary in this case. CMake can be bad but not to this point and same can be said about Boost.

[redpolline]

Don't set a different minimum boost version depending on whether you are using Windows or Linux, this makes no sense. Just use BOOSTROOT instead of BOOST_ROOT and the warning will go away.

Done.

Also as I said the default behavior is: look if cmake is able to find itself a boost distribution if it isn't able, build boost yourself There should be no reason to have to modify the workflow file for this.

The default behavior is broken in this case for the following reasons / issues / upstream kludges. VCPKG: 1) vcpkg doesn't provide a FindBoost.cmake with it's distribution. As such we cannot use the "CONFIG" option to find_package(Boost ...) when building with vcpkg. 2) VCPKG also defines several hooks which override CMake's normal behavior in an attempt force the use of it's libraries. These hooks cannot be overridden by the CMakeLists.txt being executed. Which means the self-built Boost in external/boost we wind up falling back to when using "CONFIG" will break due to the hooks changing specific CMake variables. (The compiler version string.) VS2022: The real issue in VS2022 is three fold: 1) VS2022 calls CMake initially without setting the build configuration. This means Boost picks whatever default it has instead of the one that would make the most sense for the build configuration set in VS2022's GUI. 2) VS2022 cannot handle symbol redefinitions. (Arguably that's a fault of the developer, not the build system, but I digress.) This means that if Boost picks a different link library during the build call to CMake, VS2022 will get a link library dependency list with conflicting symbol definitions it cannot resolve thus breaking the build. This occurs under the VS2022 initial build because of VS2022 issue 1. 3) We need to set CONFIG when calling find_package(Boost ...) for the Boost distribution in external/boost to be used properly. Attempting to call find_package(Boost ...) without "CONFIG" triggers a different dependency resolution path in CMake, and adds the Boost library targets regardless of our attempts to block it. (Yes, even with my "hacky" workaround, the build __will__ fail if "CONFIG" isn't set when calling find_package(Boost ...).) Boost: Also divided into two issues: 1) Boost doesn't support generator expressions properly. It should use them when calling target_link_libraries() / target_include_directories() / etc. Instead of relying on a non-working check in cmake's generator mode to see if it's been called once already or the various BOOST_USE_* flags. This is the real root cause of issue #3092, but as it's a bug in a large upstream dependency there's not much Vita3K can do about it beyond trying to workaround it until a fix is implemented upstream. 2) Boost doesn't support any means to invoke it's find_package() scripts without automatically adding the selected library to the current target. (I.e. There's no way to run find_package(Boost ...) in "detection only" mode.) Boost does support choosing a specific library config to use (Debug or Release), but that's useless to Vita3K in this case because choosing the wrong config for the wrong Vita3K build configuration will still break the build, and we won't know which one to use until VS2022's build call to CMake. (See VS2022's issue 1.) We really need to be able to detect but not set Boost's libraries during VS2022's initial CMake call to avoid VS2022's symbol redefinition error. (See VS2022's issue 2.) While working around Boost issue 1. CMake: 1) CMake doesn't support removing dependencies from a target. This is the underlying cause of Boost's own "solution" being implemented in it's Boost*.cmake scripts, and the lack of an easy workaround for Boost issue 2. 2) CMake wipes some variables in-between VS2022's calls to it. This is the cause of Boost's upstream "solution" failing. As the check when run by Boost's cmake scripts during the build call will always fail due to the undefined state variable. As to why this happens, I believe that it has something to do with the build configuration string being different between CMake calls, but I haven't really spent a lot of effort confirming that belief. TL;DR: All of the above build dependencies have an issue that is further complicated by another issue with the same build dep. That when combined with other build deps limits our attempts at a more "proper" workaround. As for why I chose the workaround that I did: Not using "CONFIG" under VS2022 is one of the causes for the error in issue #3092. (I.e. Removing "CONFIG" breaks the build under VS2022 while fixing it under vcpkg.) This is diametrically opposed behavior. Only one of these can be true in the default behavior. The other must be explicitly enabled by CMake's invoker. As VS2022 is more widely used than VCPKG for building Vita3K, I chose to enable the "CONFIG" option by default as this would be the most seamless for the most people.

Finally the boost custom build makes both the debug and release version. It should really have no issue. Using VITA3K_Boost_FOUND and other stuff sounds really hacky and unnecessary in this case.

As I've already pointed out, it's the exact same "solution" that Boost itself uses, and it's the one you suggested a few posts back. If you've got some other solution that doesn't break the CI, VS2022, and the other builds (Linux / MacOS) while also not changing the CI's config to implement it, I'm all ears. Honestly, I'd be interested in seeing it just for the learning experience. The only other workaround I have (and the one I initially came up with weeks ago) was to explicitly set the CMake build configuration for each Windows build in CMakePresets.json as part of the configurePresets array. That way there would be no question as to what build configuration was being used despite VS2022's issue 1, and we could use Boost's flags to workaround Boost issue 1 without caring about Boost issue 2. The problem with that initial workaround is that it overly complicates CMakePresets.json into a hard to maintain mess, but it _is_ a suggested workaround when Googling a solution for mixing Boost and CMake generator expressions in a CMake project. (And the reason I had those Boost flags in the PR until recently.) I decided against this workaround because I viewed it as too hacky if another means could be found to make the build work, which I did in the current PR.

CMake can be bad but not to this point and same can be said about Boost.

That's true, but the problem is not just limited to CMake and Boost. It's also involving VS2022 and VCPKG. The addition of those two to the mix and their own limitations, in addition to the limitations of the others, is what's exposing these flaws in all of them. All of these flaws make a less hacky workaround much harder to implement. The proper fix would be to fix the issues in their respective upstream projects.