GSoC Final Evaluation: Steam Workshop SDK/Third Party Integration (#2350

)

Closes #1881. This is the second part of my first PR #2343.

This PR is part of the ``Google Summer of Code 2023`` program.

I will be updating my Logs inside my [domain](https://k0t0z.github.io).

#### Note: ✔️ means that SOG/unit tests for the
corresponding function are provided.


---------------------------------------------------------------------------------------------------

✅ [Leaderboards
API](https://github.com/YoYoGames/GMEXT-Steamworks/wiki/Leaderboards)
 - ✅ steam_create_leaderboard();
 - ✅ steam_upload_score();
 - ✅ steam_upload_score_ext();
 - ⬜ steam_upload_score_buffer();
 - ⬜ steam_upload_score_buffer_ext();
 - ✅ steam_download_scores();
 - ✅ steam_download_scores_around_user();
 - ✅ steam_download_friends_scores();
 - ⬜ ``LeaderboardEntry`` data type
 - ✅ ``LeaderboardDisplayType`` constant
 - ✅ ``LeaderboardSortOrder`` constant


---------------------------------------------------------------------------------------------------

✅ [Social
API](https://github.com/YoYoGames/GMEXT-Steamworks/wiki/Social)
 - ✅ steam_set_rich_presence();
 - ✅ steam_set_clear_presence();
 - ✅ steam_user_set_played_with();
 - ⬜ steam_get_friends_game_info();
 - ✅ steam_get_user_avatar();
 - ✅ steam_image_get_size();
 - ✅ steam_image_get_rgba();
 - ✅ steam_image_get_bgra();
 - ✅ steam_image_get_argb();
 - ✅ steam_image_create_sprite();


---------------------------------------------------------------------------------------------------

⬜ [Cloud
API](https://github.com/YoYoGames/GMEXT-Steamworks/wiki/Cloud)
 - ⬜ steam_is_cloud_enabled_for_app();
 - ⬜ steam_is_cloud_enabled_for_account();
 - ⬜ steam_get_quota_total();
 - ⬜ steam_get_quota_free();
 - ⬜ steam_file_exists();
 - ⬜ steam_file_size();
 - ⬜ steam_file_persisted();
 - ⬜ steam_file_write();
 - ⬜ steam_file_write_file();
 - ⬜ steam_file_read();
 - ⬜ steam_file_share();
 - ⬜ steam_file_delete();


---------------------------------------------------------------------------------------------------

## Must Be Done Before Merging:

✅ Adding support for other platforms in the Makefile
file.
✅ Switching the ``Default`` property in the Steamworks
& Json extensions YAML file to ``false``.
✅ Switching the ``cxxflags`` property in
``Compilers/Linux/gcc.ey`` to ``-fdiagnostics-color=always``.
⬜ Reviewing all error messages and documentation in
newly added files.
✅ Adding the license to all newly added files.
✅ Providing installation instructions for all
platforms.
✅ Modifying the extension to call ``steam_init();``
automatically when enabled.
✅ Match all naming conventions with GMS's Steamworks
extension.
⬜ Testing the APIs with the latest version of
Steamworks SDK, Steam, OS version, etc.
⬜ Must test all EDL scripts in the example game
provided.
⬜ Providing SOG/unit tests for all functions written
(this need to be studied carefully as in order to test the C++ wrapper
tester must have steam installed and running or I may use mocks).
✅ Clearing garbage files.
✅ ``DEBUG_MESSAGE()`` must exist only in APIs files.
Try to remove it from wrapper files. Try to make the wrapper independent
on ENIGMA.
✅ Fix example game background design.
⬜ Update the example game to match the latest version
of [GMS Steamworks
extension](https://marketplace.yoyogames.com/assets/10709/steamworks-ext).

:white_check_mark: The compiler must write a file next to the exe.
:white_large_square: Review includes.

:white_check_mark: Commit the example game again and remove it from
``.gitignore`` file.
:white_large_square: The extension and the example must be compatible
with GameMaker.
:white_large_square: Set the title for Steamworks demo to ``Created with
ENIGMA``.
:white_large_square: Monitor GMS's terminal behavior.


---------------------------------------------------------------------------------------------------

## Dependencies:

- [Social
API](https://github.com/YoYoGames/GMEXT-Steamworks/wiki/Social) depends
on this #2309 (#2361).


---------------------------------------------------------------------------------------------------

## Good to have:

:white_large_square: Test the changes with
[RGM](https://github.com/enigma-dev/RadialGM).
:white_large_square: Solve all build warnings.

---------

Signed-off-by: Saif Kandil <74428638+k0T0z@users.noreply.github.com>
Co-authored-by: saifkandil <saifsaalaheldeen2002@gmail.com>
Co-authored-by: = <=>
Co-authored-by: --replace-all <--replace-all>
Co-authored-by: Robert Colton <robertbcolton@gmail.com>

CommandLine/testing/SimpleTests/steam_general.sog/create.edl

@@ -0,0 +1,39 @@
+var steam_api = 0;
+
+if (steam_initialised())
+{
+    if (steam_stats_ready() && steam_is_overlay_enabled())
+    {
+        steam_api = 1;
+    }
+}
+
+gtest_assert_true(steam_api);
+
+var app_id = steam_get_app_id();
+gtest_assert_eq(app_id,  480);
+
+var steam_id = steam_get_user_steam_id();
+gtest_assert_eq(steam_id,  0);
+
+var account_id = steam_get_user_account_id();
+gtest_assert_eq(account_id,  0);
+
+var logged_in = steam_is_user_logged_on();
+gtest_assert_true(logged_in);
+
+var name = steam_get_persona_name();
+gtest_assert_eq(name, "FakeSteamUser");
+
+var user_name = steam_get_user_persona_name(0);
+gtest_assert_eq(user_name, "FakeSteamUser");
+
+var language = steam_current_game_language();
+gtest_assert_eq(language, "FakeLanguage");
+
+var languages = steam_available_languages();
+gtest_assert_eq(languages, "FakeLanguages");
+
+steam_shutdown();
+steam_api = steam_initialised();
+gtest_assert_false(steam_api);

CommandLine/testing/SimpleTests/steam_overlay.sog/create.edl

@@ -0,0 +1,23 @@
+var steam_api = 0;
+
+if (steam_initialised())
+{
+    if (steam_stats_ready() && steam_is_overlay_enabled())
+    {
+        steam_api = 1;
+    }
+}
+
+gtest_assert_true(steam_api);
+
+var is_enabled = steam_is_overlay_enabled();
+gtest_assert_true(is_enabled);
+
+steam_activate_overlay(ov_friends);
+
+var is_activated = steam_is_overlay_activated();
+gtest_assert_true(is_activated);
+
+steam_shutdown();
+steam_api = steam_initialised();
+gtest_assert_false(steam_api);

CommandLine/testing/SmallTests.cpp

@@ -107,7 +107,7 @@ TEST_P(SimpleTestHarness, SimpleTestRunner) {
   // Iterate only platforms, graphics & collision systems for now
   for (TestConfig tc : GetValidConfigs(true, true, false, true, false, false)) {
 
-    tc.extensions = "Alarms,Timelines,Paths,MotionPlanning,IniFilesystem,ParticleSystems,DateTime,DataStructures,libpng,GTest";
+    tc.extensions = "Alarms,Timelines,Paths,MotionPlanning,IniFilesystem,ParticleSystems,DateTime,DataStructures,libpng,GTest,Json,Steamworks";
     int ret = TestHarness::run_to_completion(game, tc);
     if (!ret) continue;
     switch (ret) {

ENIGMAsystem/SHELL/Makefile

@@ -121,6 +121,20 @@ SOURCES := $(wildcard *.cpp) $(wildcard Platforms/General/*.cpp)
 include $(addsuffix /Makefile,$(SYSTEMS) $(EXTENSIONS))
 include Bridges/$(PLATFORM)-$(GRAPHICS)/Makefile
 
+# If Steamworks extension not enabled, enable Steamworks stub.
+# Else build the fake library.
+ifeq (,$(findstring Universal_System/Extensions/Steamworks, $(EXTENSIONS)))
+include Universal_System/StubExtensions/Steamworks/Makefile
+else
+compile_game: $(STEAM_FAKE_LIB)
+clean: steam_clean
+endif
+
+# If DataStructures extension not enabled, enable DataStructures stub.
+ifeq (,$(findstring Universal_System/Extensions/DataStructures, $(EXTENSIONS)))
+include Universal_System/StubExtensions/DataStructures/Makefile
+endif
+
 #This does not work, use a for loop and prepend it to each one not the whole string
 OBJECTS := $(addprefix $(OBJDIR)/,$(patsubst %.m, %.o, $(patsubst %.cpp, %.o, $(patsubst %.c, %.o, $(SOURCES)))))
 #RCFILES := $(addprefix $(WORKDIR),$(RESOURCES))
@@ -137,7 +151,6 @@ else
 	RESOURCEBINARY := $(OBJDIR)/resources.res
 endif
 
-
 ############
 # building #
 ############

ENIGMAsystem/SHELL/Platforms/General/PFmain.cpp

@@ -7,11 +7,17 @@
 #include "Universal_System/roomsystem.h"
 #include "Universal_System/mathnc.h" // enigma_user::clamp
 
+#include "Universal_System/Extensions/Steamworks/steamworks.h"
+
 #include <chrono> // std::chrono::microseconds
 #include <thread> // sleep_for
 
 namespace enigma {
 
+std::queue<std::map<std::string, variant>> posted_async_events;
+
+std::mutex posted_async_events_mutex;
+
 std::vector<std::function<void()> > extension_update_hooks;
 
 bool game_isending = false;
@@ -166,7 +172,27 @@ int updateTimer() {
   return 0;
 }
 
+void fireEventsFromQueue() {
+  // Acquire lock and release it when out of scope of fireEventsFromQueue().
+  std::lock_guard<std::mutex> guard(posted_async_events_mutex);
+  while (!posted_async_events.empty()) {
+    enigma_user::ds_map_clear(enigma_user::async_load);
+
+    std::map<std::string, variant> event = posted_async_events.front();
+
+    posted_async_events.pop();
+
+    for (auto& [key, value] : event) {
+      enigma_user::ds_map_add(enigma_user::async_load, key, value);
+    }
+
+    enigma::fireSteamworksEvent();
+  }
+}
+
 int enigma_main(int argc, char** argv) {
+  enigma_user::async_load = enigma_user::ds_map_create();
+
   // Initialize directory globals
   initialize_directory_globals();
 
@@ -201,6 +227,8 @@ int enigma_main(int argc, char** argv) {
     for (auto update_hook : extension_update_hooks)
       update_hook();
 
+    enigma::fireEventsFromQueue();
+
     ENIGMA_events();
     handleInput();
   }
@@ -215,6 +243,8 @@ int enigma_main(int argc, char** argv) {
 
 namespace enigma_user {
 
+int async_load;
+
 const int os_browser = browser_not_a_browser;
 std::string working_directory = "";
 std::string program_directory = "";

ENIGMAsystem/SHELL/Platforms/General/PFmain.h

@@ -1,5 +1,6 @@
 /** Copyright (C) 2011 Josh Ventura
 *** Copyright (C) 2013-2014 Robert B. Colton
+*** Copyright (C) 2023-2024 Saif Kandil (k0T0z)
 ***
 *** This file is a part of the ENIGMA Development Environment.
 ***
@@ -19,35 +20,61 @@
 #ifndef ENIGMA_PLATFORM_MAIN
 #define ENIGMA_PLATFORM_MAIN
 
+#include <map>
+#include <mutex>
+#include <queue>
 #include <string>
 
+#include "Universal_System/Extensions/DataStructures/include.h"
+#include "Universal_System/var4.h"
+
 namespace enigma {
-  extern bool game_isending;
-  extern int game_return;
-  extern int parameterc;
-  extern std::string* parameters;
-  extern int pausedSteps;
-  extern int current_room_speed;
-  extern int frames_count;
-  extern unsigned long current_time_mcs;
-  extern bool game_window_focused;
-
-  int enigma_main(int argc, char** argv);
-  int game_ending();
-  void Sleep(int ms);
-  void compute_window_scaling();
-  void compute_window_size();
-  void initialize_program_directory();
-  void initialize_directory_globals();
-  void set_program_args(int argc, char** argv);
-  void platform_focus_lost();
-  void platform_focus_gained();
-  void initTimer();
-  int updateTimer();
-  int gameWait();
-  void set_room_speed(int rs);
-}
-
+extern bool game_isending;
+extern int game_return;
+extern int parameterc;
+extern std::string* parameters;
+extern int pausedSteps;
+extern int current_room_speed;
+extern int frames_count;
+extern unsigned long current_time_mcs;
+extern bool game_window_focused;
+
+/**
+   * @brief This queue is used to store the events that are fired from the 
+   *        async event system.
+   * 
+   */
+extern std::queue<std::map<std::string, variant>> posted_async_events;
+
+/**
+   * @brief This mutex is used to lock the posted_async_events queue.
+   * 
+   */
+extern std::mutex posted_async_events_mutex;
+
+int enigma_main(int argc, char** argv);
+int game_ending();
+void Sleep(int ms);
+void compute_window_scaling();
+void compute_window_size();
+void initialize_program_directory();
+void initialize_directory_globals();
+void set_program_args(int argc, char** argv);
+void platform_focus_lost();
+void platform_focus_gained();
+void initTimer();
+int updateTimer();
+int gameWait();
+void set_room_speed(int rs);
+
+/**
+   * @brief This function is used to fire all the events that are stored in the 
+   *        @c posted_async_events queue.
+   * 
+   */
+void fireEventsFromQueue();
+}  // namespace enigma
+
 namespace enigma_user {
 
 extern std::string working_directory;
@@ -58,8 +85,15 @@ extern double fps;
 extern unsigned long delta_time;
 extern unsigned long current_time;
 
+/**
+ * @brief When firing any event inside posted_async_events queue, this variable will 
+ *        contain the data that will be sent to the game.
+ * 
+ */
+extern int async_load;
+
 void sleep(int ms);
-unsigned long get_timer(); // number of microseconds since the game started
+unsigned long get_timer();  // number of microseconds since the game started
 void game_end();
 void game_end(int ret);
 void action_end_game();
@@ -76,17 +110,17 @@ unsigned long long disk_free(std::string drive);
 
 void execute_shell(std::string fname, std::string args);
 void execute_program(std::string fname, std::string args, bool wait);
-std::string execute_shell_for_output(const std::string &command);
+std::string execute_shell_for_output(const std::string& command);
 void url_open(std::string url);
-void action_webpage(const std::string &url);
+void action_webpage(const std::string& url);
 
 void set_program_priority(int value);
 std::string filename_absolute(std::string fname);
 std::string filename_join(std::string prefix, std::string suffix);
 std::string environment_get_variable(std::string name);
-bool environment_set_variable(const std::string &name, const std::string &value);
+bool environment_set_variable(const std::string& name, const std::string& value);
 bool set_working_directory(std::string dname);
 
-} // namespace enigma_user
+}  // namespace enigma_user
 
-#endif //ENIGMA_PLATFORM_MAIN
+#endif  //ENIGMA_PLATFORM_MAIN

ENIGMAsystem/SHELL/Universal_System/Extensions/Json/json.cpp

@@ -190,6 +190,9 @@ namespace enigma_user
 		}
 	}
 
+	/*
+		Check https://github.com/enigma-dev/enigma-dev/pull/2358.
+	*/
 	string json_encode(variant ds_map) {
 		if (!enigma_user::ds_map_exists(ds_map)) {
 				DEBUG_MESSAGE("DS map does not exist", MESSAGE_TYPE::M_ERROR);

ENIGMAsystem/SHELL/Universal_System/Extensions/Steamworks/.gitignore

@@ -0,0 +1,27 @@
+############################################################################
+# Copyright (C) 2023-2024 Saif Kandil (k0T0z)
+#
+# This file is a part of the ENIGMA Development Environment.
+#
+# ENIGMA is free software: you can redistribute it and/or modify it under the
+# terms of the GNU General Public License as published by the Free Software
+# Foundation, version 3 of the license or any later version.
+#
+# This application and its source code is distributed AS-IS, WITHOUT ANY
+# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
+# details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this code. If not, see <http://www.gnu.org/licenses/>
+############################################################################
+
+/.vs/*
+/.vscode/*
+
+# gb stands for Global Backup and it shouldn't be committed to git
+# gb is auto-generated by the example game
+/*.gb*/*
+
+# This is only temporary, it will be removed once the Midterm review is done.
+# /steamworks_demo/*

ENIGMAsystem/SHELL/Universal_System/Extensions/Steamworks/About.ey

@@ -0,0 +1,15 @@
+%e-yaml
+---
+
+Name: Steamworks
+Identifier: Steamworks
+Author: Saif Kandil (https://k0t0z.github.io)
+Description: Steamworks is a set of tools and services that help game developers and publishers build their games and get the most out of distributing on Steam.
+Default: false
+Icon: steam.svg
+
+Depends:
+	Extensions: DataStructures, Json
+Dependencies: None
+Implement: extension_steamworks
+Init: extension_steamworks_init

ENIGMAsystem/SHELL/Universal_System/Extensions/Steamworks/CMakeLists.txt

@@ -0,0 +1,35 @@
+############################################################################
+# Copyright (C) 2023-2024 Saif Kandil (k0T0z)
+#
+# This file is a part of the ENIGMA Development Environment.
+#
+# ENIGMA is free software: you can redistribute it and/or modify it under the
+# terms of the GNU General Public License as published by the Free Software
+# Foundation, version 3 of the license or any later version.
+#
+# This application and its source code is distributed AS-IS, WITHOUT ANY
+# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
+# details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this code. If not, see <http://www.gnu.org/licenses/>
+############################################################################
+
+############################################################################
+# Note that this file isn't tested yet as ENIGMA's engine still works with
+# the old build system. Ignore this file for now.
+############################################################################
+
+add_library(steamapi STATIC IMPORTED GLOBAL 
+    redistributable_bin/steam_api.lib
+)
+
+set_property(TARGET steamapi APPEND PROPERTY IMPORTED_CONFIGURATIONS DEBUG)
+set_target_properties(steamapi PROPERTIES
+        IMPORTED_LINK_INTERFACE_LANGUAGES_DEBUG "CXX"
+        IMPORTED_LOCATION_DEBUG "${CMAKE_SOURCE_DIR}/game_client/Steamv157/sdk/redistributable_bin/steam_api.lib")
+
+target_link_libraries(steamworks 
+    steamapi
+)