Add support for building profiled dynamic way #9900

mpickering · 2024-04-17T10:42:25Z

Add support for profiled dynamic way

New options for cabal.project and ./Setup interface:

* `profiling-shared`: Enable building profiling dynamic way
* Passing `--enable-profiling` and `--enable-executable-dynamic` builds
  profiled dynamic executables.

Support for using `profiling-shared` is guarded behind a constraint
which ensures you are using `Cabal >= 3.13`.

In the cabal file:

* `ghc-prof-shared-options`, for passing options when building in
  profiling dynamic way

Other miscellenious fixes and improvements

* Some refactoring around ways so that which
  ways we should build for a library, foreign library and executable is
  computed by the `buildWays` function (rather than ad-hoc in three
  different places).

* Improved logic for detecting whether a compiler supports compiling
  a specific way. See functions `profilingVanillaSupported`,
  `dynamicSupported`, `profilingDynamicSupported` etc
  These functions report accurate infomation after ghc-9.10.1.

* Fixed logic for determining whether to build shared libraries. (see
  #10050)
  Now, if you explicitly enable `--*-shared`, then that will always take
  effect. If it's not specified then `--enable-executable-dynamic` will
  turn on shared libraries IF `--enable-profiling` is not enabled.

* Remove assumption that dynamically linked compilers can build dynamic
  libraries (they might be cross compilers.

* Query the build compiler to determine which library way is necessary
  to be built for TH support to work.
  (rather than assume all compilers are dynamically linked)

* An extensive test which checks how options for `./Setup` and
  `cabal-install` are translated into build ways.

Fixes #4816, #10049, #10050

mpickering · 2024-04-18T10:29:56Z

I require the version to be bumped to 3.13 before this can be finished.

ulysses4ever · 2024-04-19T14:17:54Z

That's

Bump project version from 3.11 to 3.13 #9897

mpickering · 2024-05-29T09:59:34Z

The test fails on 8.6.5 because of this GHC bug (fixed in 2018): e400b9babdcf11669f963aeec20078fe7ccfca0d

New options for cabal.project and ./Setup interface: * `profiling-shared`: Enable building profiling dynamic way * Passing `--enable-profiling` and `--enable-executable-dynamic` builds profiled dynamic executables. Support for using `profiling-shared` is guarded behind a constraint which ensures you are using `Cabal >= 3.13`. In the cabal file: * `ghc-prof-shared-options`, for passing options when building in profiling dynamic way Other miscellenious fixes and improvements * Some refactoring around ways so that which ways we should build for a library, foreign library and executable is computed by the `buildWays` function (rather than ad-hoc in three different places). * Improved logic for detecting whether a compiler supports compiling a specific way. See functions `profilingVanillaSupported`, `dynamicSupported`, `profilingDynamicSupported` etc These functions report accurate infomation after ghc-9.10.1. * Fixed logic for determining whether to build shared libraries. (see haskell#10050) Now, if you explicitly enable `--*-shared`, then that will always take effect. If it's not specified then `--enable-executable-dynamic` will turn on shared libraries IF `--enable-profiling` is not enabled. * Remove assumption that dynamically linked compilers can build dynamic libraries (they might be cross compilers. * Query the build compiler to determine which library way is necessary to be built for TH support to work. (rather than assume all compilers are dynamically linked) * An extensive test which checks how options for `./Setup` and `cabal-install` are translated into build ways. Fixes haskell#4816, haskell#10049, haskell#10050

sheaf · 2024-05-30T15:14:08Z

Cabal/src/Distribution/Simple/Configure.hs

@@ -807,6 +803,20 @@ computeLocalBuildConfig cfg comp programDb = do
            , relocatable = fromFlagOrDefault False $ configRelocatable cfg
            }

+  -- Dynamic executable, but no shared vanilla libararies


Suggested change

-- Dynamic executable, but no shared vanilla libararies

-- Dynamic executable, but no shared vanilla libraries

sheaf · 2024-05-30T15:14:17Z

Cabal/src/Distribution/Simple/Configure.hs

+        ++ "is not being built. Linking will fail if any executables "
+        ++ "depend on the library."
+
+  -- Profiled dynamic executable, but no shared profiling libararies


Suggested change

-- Profiled dynamic executable, but no shared profiling libararies

-- Profiled dynamic executable, but no shared profiling libraries

sheaf · 2024-05-30T15:16:26Z

Cabal/src/Distribution/Simple/Configure.hs

+            return $ \buildOptions -> apply buildOptions{LBC.withProfLibShared = tryLibProfilingShared}
+            else -- Case 2: Compiler doesn't support profiling shared so turn them off
+            do
+              -- If we wanted to enable profilng shared libraries.. tell the


Suggested change

-- If we wanted to enable profilng shared libraries.. tell the

-- If we wanted to enable profiling shared libraries.. tell the

sheaf · 2024-05-30T15:27:39Z

Cabal/src/Distribution/Simple/Configure.hs

+                      , LBC.withProfLib = profilingVanillaSupportedOrUnknown comp && (tryLibProfilingShared || LBC.withProfLib original_options)
+                      , LBC.withDynExe = if LBC.withProfExe original_options then False else LBC.withDynExe original_options
+                      }
+        return (tryExeProfiling && not (tryLibProfiling || tryLibProfilingShared), apply2)


Can you move the "when exeProfWithoutLibProf " warning into here, and then refactor to not return this boolean? I find it too complicated right now.

sheaf · 2024-05-30T15:34:54Z

Cabal/src/Distribution/Simple/GHC.hs

+          ]
+        whenGHCi $ installOrdinary builtDir targetDir ghciLibName
+      ProfWay -> do
+        installOrdinary builtDir targetDir profileLibName


Why doesn't this code path include any logic to deal with extraBundledLibs that all other paths include? (I know you didn't change this.)

sheaf · 2024-05-30T16:07:52Z

Cabal/src/Distribution/Simple/GHC/Build/ExtraSources.hs

+                    forM_ (neededLibWays isIndef) $ \case
+                      StaticWay -> compileIfNeeded vanillaSrcOpts
+                      DynWay -> compileIfNeeded sharedSrcOpts{ghcOptObjSuffix = toFlag "dyn_o"}
+                      ProfWay -> compileIfNeeded profSrcOpts{ghcOptObjSuffix = toFlag "p_o"}
+                      ProfDynWay -> compileIfNeeded profSharedSrcOpts{ghcOptObjSuffix = toFlag "p_dyn_o"}
+            CFLib flib ->
+              case neededFLibWay (withDynFLib flib) of
+                StaticWay -> compileIfNeeded vanillaSrcOpts
+                DynWay -> compileIfNeeded sharedSrcOpts
+                ProfWay -> compileIfNeeded profSrcOpts
+                ProfDynWay -> compileIfNeeded profSharedSrcOpts
+            -- For the remaining component types (Exec, Test, Bench), we also
+            -- determine with which options to build the objects (vanilla vs shared vs
+            -- profiled), but predicate is the same for the three kinds.
+            _exeLike ->
+              case neededExeWay of
+                StaticWay -> compileIfNeeded vanillaSrcOpts
+                DynWay -> compileIfNeeded sharedSrcOpts
+                ProfWay -> compileIfNeeded profSrcOpts
+                ProfDynWay -> compileIfNeeded profSharedSrcOpts


Good improvement.👍

sheaf · 2024-05-30T16:10:10Z

Cabal/src/Distribution/Types/LocalBuildInfo.hs

+    wantedFLibWay is_dyn_flib =
+      case (is_dyn_flib, withProfExe lbi) of
+        (True, True) -> ProfDynWay
+        (False, True) -> ProfWay
+        (True, False) -> DynWay
+        (False, False) -> StaticWay
+
+    wantedExeWay =
+      case (withDynExe lbi, withProfExe lbi) of
+        (True, True) -> ProfDynWay
+        (True, False) -> DynWay
+        (False, True) -> ProfWay
+        (False, False) -> StaticWay


Good that you got rid of the sketchy if isLib then id else take 1 that was there before. 👍

sheaf · 2024-05-30T16:12:21Z

Cabal/src/Distribution/Types/LocalBuildInfo.hs

+        (False, True) -> ProfWay
+        (False, False) -> StaticWay
+   in
+    (wantedLibWays, wantedFLibWay, wantedExeWay)


What happened to the following logic:

neededWays = wantedWays <> Set.fromList -- TODO: You also don't need to build the GHC way when doing TH if -- you are using an external interpreter!! [defaultGhcWay | doingTH && defaultGhcWay `Set.notMember` wantedWays]

?

sheaf · 2024-05-30T16:14:12Z

cabal-install/src/Distribution/Client/ProjectPlanning.hs


      -- TODO: [code cleanup] move this into the Cabal lib. It's currently open
      -- coded in Distribution.Simple.Configure, but should be made a proper
      -- function of the Compiler or CompilerInfo.
      compilerShouldUseSharedLibByDefault =
        case compilerFlavor compiler of
-          GHC -> GHC.isDynamic compiler
+          GHC -> GHC.compilerBuildWay compiler == DynWay && canBuildSharedLibs


Is it intended that this is false if compilerBuildWay == ProfDynWay?

sheaf · 2024-05-30T16:24:12Z

Cabal/src/Distribution/Simple/Configure.hs

@@ -734,7 +735,7 @@ computeLocalBuildConfig cfg comp programDb = do
            CompilerId GHC _ ->
              -- if ghc is dynamic, then ghci needs a shared
              -- library, so we build one by default.
-              GHC.isDynamic comp
+              GHC.compilerBuildWay comp == DynWay


Is the logic consistent here? If we have --enable-executable-dynamic --enable-profiling, do we want sharedLibsByDefault to be True?
Conversely, why don't we have GHC.compilerBuildWay comp `elem` [DynWay, ProfDynWay]? What will happen with ProfDyn and ghci? Is that supported?

mpickering force-pushed the wip/prof_dyn branch from 680c3a2 to 70b5f74 Compare April 18, 2024 13:52

mpickering force-pushed the wip/prof_dyn branch 6 times, most recently from c1f27fe to cb44290 Compare May 22, 2024 11:36

mpickering mentioned this pull request May 28, 2024

Fix predicate which determines whether to build a shared library #10054

Open

5 tasks

mpickering force-pushed the wip/prof_dyn branch 2 times, most recently from fd810d4 to d47096e Compare May 29, 2024 12:21

mpickering force-pushed the wip/prof_dyn branch from d47096e to 870fa68 Compare May 29, 2024 13:37

mpickering changed the title ~~Draft: Add support for building profiled dynamic way~~ Add support for building profiled dynamic way May 29, 2024

sheaf reviewed May 30, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for building profiled dynamic way #9900

Add support for building profiled dynamic way #9900

mpickering commented Apr 17, 2024 •

edited

mpickering commented Apr 18, 2024

ulysses4ever commented Apr 19, 2024

mpickering commented May 29, 2024

sheaf May 30, 2024

sheaf May 30, 2024

sheaf May 30, 2024

sheaf May 30, 2024

sheaf May 30, 2024

sheaf May 30, 2024

sheaf May 30, 2024

sheaf May 30, 2024

sheaf May 30, 2024

sheaf May 30, 2024 •

edited

	-- Dynamic executable, but no shared vanilla libararies
	-- Dynamic executable, but no shared vanilla libraries

	-- Profiled dynamic executable, but no shared profiling libararies
	-- Profiled dynamic executable, but no shared profiling libraries

	-- If we wanted to enable profilng shared libraries.. tell the
	-- If we wanted to enable profiling shared libraries.. tell the

Add support for building profiled dynamic way #9900

Are you sure you want to change the base?

Add support for building profiled dynamic way #9900

Conversation

mpickering commented Apr 17, 2024 • edited

mpickering commented Apr 18, 2024

ulysses4ever commented Apr 19, 2024

mpickering commented May 29, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sheaf May 30, 2024 • edited

Choose a reason for hiding this comment

mpickering commented Apr 17, 2024 •

edited

sheaf May 30, 2024 •

edited