Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring package to component centered design #6334

Merged
merged 43 commits into from
Nov 14, 2023
Merged
Show file tree
Hide file tree
Changes from 36 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
101c522
refactoring: move getPackageFile to package file module
theobat Apr 10, 2023
73d2fa4
feat: provide component building blocks
theobat Nov 13, 2022
8dc2d52
feat: add component collection fields on Package datatype
theobat Nov 13, 2022
af23d56
feat: replace packageLibraries Package field
theobat Apr 4, 2023
43f09b6
refactoring: packageHasExposedModules Package field refactoring
theobat Apr 4, 2023
e87ff82
feat: remove packageUnknownTools for component based solution
theobat Apr 7, 2023
134c331
feat: remove packageUnknownTools for component based solution
theobat Apr 7, 2023
39a05d0
refactoring: remove package based packageInternalLibraries
theobat Apr 8, 2023
e8932cd
refactoring: remove package based packageExes
theobat Apr 8, 2023
936c2ab
refactoring: remove packageTests and packageBench
theobat Apr 9, 2023
a5ff987
feat: replace package file gathering for cabal aligned design
theobat Nov 10, 2023
b0b3098
refacorting: cleaning package libraries
theobat Nov 10, 2023
c839e91
feat: remove ghc package opts lambda in package type
theobat Nov 12, 2023
e9b5615
ignore .vscode
theobat Nov 12, 2023
da2e5a0
fix: ci layout issues
theobat Nov 12, 2023
92adb32
Merge branch 'package-component' of https://github.com/theobat/stack …
mpilgrem Nov 12, 2023
7b7f1a7
Improve documentation of .gitignore exclusions
mpilgrem Nov 12, 2023
33b3abd
List extensions in alphabetical order
mpilgrem Nov 12, 2023
97b7f6c
Remove PartialTypeSignatures extension
mpilgrem Nov 12, 2023
a84368c
Order imports alphabetically
mpilgrem Nov 12, 2023
fc4cd00
Explicit import lists and consistent formatting
mpilgrem Nov 12, 2023
7560034
Add extensions used to .hlint.yaml
mpilgrem Nov 12, 2023
852a8be
Swap order of packageHasLibrary test, and consistent formatting
mpilgrem Nov 12, 2023
2a0b2d4
Reformat, for consistency
mpilgrem Nov 12, 2023
d5be99c
Reformat, for consistency
mpilgrem Nov 12, 2023
e30bcf0
Add explicit imports and remove redundant imports; reformat
mpilgrem Nov 12, 2023
38a4de9
Add explicit import lists; reformat, for consistency
mpilgrem Nov 12, 2023
6106bee
Reformatting, for consistency
mpilgrem Nov 12, 2023
460e8ad
Update ignored Stan observations for code changes
mpilgrem Nov 13, 2023
7310b24
Reformatting, for consistency
mpilgrem Nov 13, 2023
0b09fea
Prefer type `[a]` to `[] a`
mpilgrem Nov 13, 2023
4cafcdc
Further reformatting, for consistency
mpilgrem Nov 13, 2023
99fd135
Reformatting, for consistency
mpilgrem Nov 13, 2023
ff21030
Reformatting, for consistency
mpilgrem Nov 13, 2023
84a5a4a
Reformatting, for consistency
mpilgrem Nov 13, 2023
118531c
Add Haddock documentation
mpilgrem Nov 13, 2023
52f170f
Prefer `hasBuildableMainLibrary` to `hasMainBuildableLibrary`
mpilgrem Nov 14, 2023
925a6e6
Prefer `packageBenchmarks` for field, and `buildableBenchmarks`
mpilgrem Nov 14, 2023
c64510f
Prefer `buildableExes` to `packageExes`
mpilgrem Nov 14, 2023
2c5c099
Prefer `buildableSubLibs` to `packageSubLibrariesNameSet`
mpilgrem Nov 14, 2023
d63ac92
Introduce `Stack.Package.buildableTestSuites`
mpilgrem Nov 14, 2023
82266a8
Introduce `Stack.Package.buildableForeignLibs`
mpilgrem Nov 14, 2023
85cc8f9
Update ignored Stan observations for code changes
mpilgrem Nov 14, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,9 @@ stan.html
# macOS-related
.DS_Store

# VS Code workspace settings
.vscode

# VS Code Counter (Visual Studio Code Extension)-related
.VSCodeCounter

Expand Down
8 changes: 8 additions & 0 deletions .hlint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -57,13 +57,21 @@
# the GHC2021 set. Other extensions can be added, if need be.
- name:
- NoImplicitPrelude
- ConstraintKinds
- DataKinds
- DefaultSignatures
- DeriveDataTypeable
- DeriveGeneric
- DerivingStrategies
- DisambiguateRecordFields
- DuplicateRecordFields
- FlexibleContexts
- GADTs
- GeneralizedNewtypeDeriving
- LambdaCase
- MultiWayIf
- OverloadedLists
- OverloadedRecordDot
- OverloadedStrings
- QuasiQuotes
- RecordWildCards
Expand Down
4 changes: 2 additions & 2 deletions .stan.toml
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@

# Anti-pattern: Data.ByteString.Char8.pack
[[ignore]]
id = "OBS-STAN-0203-fki0nd-1111:21"
id = "OBS-STAN-0203-fki0nd-1120:21"
# ✦ Description: Usage of 'pack' function that doesn't handle Unicode characters
# ✦ Category: #AntiPattern
# ✦ File: src\Stack\Build\Execute.hs
Expand All @@ -63,7 +63,7 @@

# Anti-pattern: Data.ByteString.Char8.pack
[[ignore]]
id = "OBS-STAN-0203-fki0nd-2656:3"
id = "OBS-STAN-0203-fki0nd-2662:3"
# ✦ Description: Usage of 'pack' function that doesn't handle Unicode characters
# ✦ Category: #AntiPattern
# ✦ File: src\Stack\Build\Execute.hs
Expand Down
2 changes: 1 addition & 1 deletion src/Stack/Build.hs
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ import Stack.Build.Installed ( getInstalled, toInstallMap )
import Stack.Build.Source ( localDependencies, projectLocalPackages )
import Stack.Build.Target ( NeedTargets (..) )
import Stack.FileWatch ( fileWatch, fileWatchPoll )
import Stack.Package ( resolvePackage )
import Stack.Package ( packageExes, resolvePackage )
import Stack.Prelude hiding ( loadPackage )
import Stack.Runners ( ShouldReexec (..), withConfig, withEnvConfig )
import Stack.Setup ( withNewLocalBuildTargets )
Expand Down
24 changes: 13 additions & 11 deletions src/Stack/Build/ConstructPlan.hs
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,10 @@ import Stack.Build.Cache ( tryGetFlagCache )
import Stack.Build.Haddock ( shouldHaddockDeps )
import Stack.Build.Source ( loadLocalPackage )
import Stack.Constants ( compilerOptionsCabalFlag )
import Stack.Package ( applyForceCustomBuild )
import Stack.Package
( applyForceCustomBuild, hasMainBuildableLibrary
, packageExes, packageUnknownTools
)
import Stack.Prelude hiding ( loadPackage )
import Stack.SourceMap ( getPLIVersion, mkProjectPackage )
import Stack.Types.Build
Expand All @@ -45,6 +48,7 @@ import Stack.Types.BuildConfig
( BuildConfig (..), HasBuildConfig (..), stackYamlL )
import Stack.Types.BuildOpts
( BuildOpts (..), BuildOptsCLI (..), BuildSubset (..) )
import Stack.Types.CompCollection ( collectionMember )
import Stack.Types.Compiler ( WhichCompiler (..) )
import Stack.Types.CompilerPaths
( CompilerPaths (..), HasCompiler (..) )
Expand All @@ -67,8 +71,8 @@ import Stack.Types.NamedComponent ( exeComponents, renderComponent )
import Stack.Types.Package
( ExeName (..), InstallLocation (..), Installed (..)
, InstalledMap, LocalPackage (..), Package (..)
, PackageLibraries (..), PackageSource (..), installedVersion
, packageIdentifier, psVersion, runMemoizedWith
, PackageSource (..), installedVersion, packageIdentifier
, psVersion, runMemoizedWith
)
import Stack.Types.ParentMap ( ParentMap )
import Stack.Types.Platform ( HasPlatform (..) )
Expand Down Expand Up @@ -1162,10 +1166,7 @@ addPackageDeps package = do
-- make sure we consider sub-libraries as libraries too
packageHasLibrary :: Package -> Bool
packageHasLibrary p =
not (Set.null (packageSubLibraries p)) ||
case packageLibraries p of
HasLibraries _ -> True
NoLibraries -> False
hasMainBuildableLibrary p || not (null (packageSubLibraries p))

checkDirtiness ::
PackageSource
Expand Down Expand Up @@ -1334,8 +1335,8 @@ checkAndWarnForUnknownTools p = do
-- Check whether the tool is on the PATH or a package executable before
-- warning about it.
warnings <-
fmap catMaybes $ forM unknownTools $ \name@(ExeName toolName) ->
runMaybeT $ notOnPath toolName *> notPackageExe toolName *> warn name
fmap catMaybes $ forM unknownTools $ \toolName ->
runMaybeT $ notOnPath toolName *> notPackageExe toolName *> warn toolName
tell mempty { wWarnings = (map toolWarningText warnings ++) }
pure ()
where
Expand All @@ -1349,8 +1350,9 @@ checkAndWarnForUnknownTools p = do
skipIf $ isRight eFound
-- From Cabal 1.12, build-tools can specify another executable in the same
-- package.
notPackageExe toolName = MaybeT $ skipIf $ toolName `Set.member` packageExes p
warn name = MaybeT . pure . Just $ ToolWarning name (packageName p)
notPackageExe toolName =
MaybeT $ skipIf $ collectionMember toolName (packageExecutables p)
warn name = MaybeT . pure . Just $ ToolWarning (ExeName name) (packageName p)
skipIf p' = pure $ if p' then Nothing else Just ()

-- | Warn about tools in the snapshot definition. States the tool name
Expand Down
80 changes: 43 additions & 37 deletions src/Stack/Build/Execute.hs
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ import Distribution.Types.UnqualComponentName
( mkUnqualComponentName )
import Distribution.Verbosity ( showForCabal )
import Distribution.Version ( mkVersion )
import GHC.Records ( getField )
import Path
( PathException, (</>), addExtension, filename
, isProperPrefixOf, parent, parseRelDir, parseRelFile
Expand Down Expand Up @@ -124,7 +125,11 @@ import Stack.Coverage
, generateHpcUnifiedReport, updateTixFile
)
import Stack.GhcPkg ( ghcPkg, unregisterGhcPkgIds )
import Stack.Package ( buildLogPath )
import Stack.Package
( buildLogPath, hasMainBuildableLibrary
, mainLibraryHasExposedModules, packageExes
, packageSubLibrariesNameSet
)
import Stack.PackageDump ( conduitDumpPackage, ghcPkgDescribe )
import Stack.Prelude
import Stack.Types.ApplyGhcOptions ( ApplyGhcOptions (..) )
Expand All @@ -144,6 +149,10 @@ import Stack.Types.BuildOpts
, CabalVerbosity (..), HaddockOpts (..)
, ProgressBarFormat (..), TestOpts (..)
)
import Stack.Types.CompCollection
( collectionKeyValueList, collectionLookup
, getBuildableListText
)
import Stack.Types.Compiler
( ActualCompiler (..), WhichCompiler (..)
, compilerVersionString, getGhcVersion, whichCompilerL
Expand All @@ -152,6 +161,7 @@ import Stack.Types.CompilerPaths
( CompilerPaths (..), GhcPkgExe (..), HasCompiler (..)
, cabalVersionL, cpWhich, getCompilerPath, getGhcPkgExe
)
import qualified Stack.Types.Component as Component
import Stack.Types.Config
( Config (..), HasConfig (..), buildOptsL, stackRootL )
import Stack.Types.ConfigureOpts
Expand All @@ -175,9 +185,8 @@ import Stack.Types.NamedComponent
)
import Stack.Types.Package
( InstallLocation (..), Installed (..), InstalledMap
, LocalPackage (..), Package (..), PackageLibraries (..)
, installedPackageIdentifier, packageIdentifier
, runMemoizedWith
, LocalPackage (..), Package (..), installedPackageIdentifier
, packageIdentifier, runMemoizedWith
)
import Stack.Types.PackageFile ( PackageWarning (..) )
import Stack.Types.Platform ( HasPlatform (..) )
Expand Down Expand Up @@ -1713,7 +1722,7 @@ singleBuild
&& not isFinalBuild
-- Works around haddock failing on bytestring-builder since it has no
-- modules when bytestring is new enough.
&& packageHasExposedModules package
&& mainLibraryHasExposedModules package
-- Special help for the curator tool to avoid haddocks that are known
-- to fail
&& maybe True (Set.notMember pname . curatorSkipHaddock) mcurator
Expand Down Expand Up @@ -1748,11 +1757,8 @@ singleBuild
(hasLib, hasSubLib, hasExe) = case taskType of
TTLocalMutable lp ->
let package = lpPackage lp
hasLibrary =
case packageLibraries package of
NoLibraries -> False
HasLibraries _ -> True
hasSubLibraries = not . Set.null $ packageSubLibraries package
hasLibrary = hasMainBuildableLibrary package
hasSubLibraries = not . null $ packageSubLibraries package
hasExecutables =
not . Set.null $ exesToBuild executableBuildStatuses lp
in (hasLibrary, hasSubLibraries, hasExecutables)
Expand Down Expand Up @@ -1797,9 +1803,9 @@ singleBuild
-- However, we must unregister any such library in the new snapshot, in case
-- it was built with different flags.
let
subLibNames = Set.toList $ case taskType of
TTLocalMutable lp -> packageSubLibraries $ lpPackage lp
TTRemotePackage _ p _ -> packageSubLibraries p
subLibNames = Set.toList $ packageSubLibrariesNameSet $ case taskType of
TTLocalMutable lp -> lpPackage lp
TTRemotePackage _ p _ -> p
toMungedPackageId :: Text -> MungedPackageId
toMungedPackageId subLib =
let subLibName = LSubLibName $ mkUnqualComponentName $ T.unpack subLib
Expand Down Expand Up @@ -2038,13 +2044,9 @@ singleBuild

cabal0 keep KeepTHLoading $ "haddock" : args

let hasLibrary =
case packageLibraries package of
NoLibraries -> False
HasLibraries _ -> True
packageHasComponentSet f = not $ Set.null $ f package
hasSubLibraries = packageHasComponentSet packageSubLibraries
hasExecutables = packageHasComponentSet packageExes
let hasLibrary = hasMainBuildableLibrary package
hasSubLibraries = not $ null $ packageSubLibraries package
hasExecutables = not $ null $ packageExecutables package
shouldCopy =
not isFinalBuild
&& (hasLibrary || hasSubLibraries || hasExecutables)
Expand Down Expand Up @@ -2093,10 +2095,10 @@ singleBuild
let ident = PackageIdentifier (packageName package) (packageVersion package)
-- only pure the sub-libraries to cache them if we also cache the main
-- library (that is, if it exists)
(mpkgid, subLibsPkgIds) <- case packageLibraries package of
HasLibraries _ -> do
(mpkgid, subLibsPkgIds) <- if hasMainBuildableLibrary package
then do
subLibsPkgIds <- fmap catMaybes $
forM (Set.toList $ packageSubLibraries package) $ \subLib -> do
forM (getBuildableListText $ packageSubLibraries package) $ \subLib -> do
let subLibName = MungedPackageName
(packageName package)
(LSubLibName $ mkUnqualComponentName $ T.unpack subLib)
Expand All @@ -2112,10 +2114,10 @@ singleBuild
case mpkgid of
Nothing -> throwM $ Couldn'tFindPkgId $ packageName package
Just pkgid -> pure (Library ident pkgid Nothing, subLibsPkgIds)
NoLibraries -> do
else do
markExeInstalled (taskLocation task) pkgId -- TODO unify somehow
-- with writeFlagCache?
pure (Executable ident, []) -- don't pure sub-libraries in this case
pure (Executable ident, []) -- don't pure sublibs in this case

case taskType of
TTRemotePackage Immutable _ loc ->
Expand Down Expand Up @@ -2276,7 +2278,9 @@ singleTest topts testsToRun ac ee task installedMap = do

let suitesToRun
= [ testSuitePair
| testSuitePair <- Map.toList $ packageTests package
| testSuitePair <-
(fmap . fmap) (getField @"interface") <$>
collectionKeyValueList $ packageTestSuites package
, let testName = fst testSuitePair
, testName `elem` testsToRun
]
Expand Down Expand Up @@ -2485,9 +2489,11 @@ singleTest topts testsToRun ac ee task installedMap = do
when needHpc $ do
let testsToRun' = map f testsToRun
f tName =
case Map.lookup tName (packageTests package) of
Just C.TestSuiteLibV09{} -> tName <> "Stub"
_ -> tName
case getField @"interface" <$> mComponent of
Just C.TestSuiteLibV09{} -> tName <> "Stub"
_ -> tName
where
mComponent = collectionLookup tName (packageTestSuites package)
generateHpcReport pkgDir package testsToRun'

bs <- liftIO $
Expand Down Expand Up @@ -2696,17 +2702,17 @@ primaryComponentOptions ::
primaryComponentOptions executableBuildStatuses lp =
-- TODO: get this information from target parsing instead, which will allow
-- users to turn off library building if desired
( case packageLibraries package of
NoLibraries -> []
HasLibraries names -> map
T.unpack
( T.append "lib:" (T.pack (packageNameString (packageName package)))
: map (T.append "flib:") (Set.toList names)
)
( if hasMainBuildableLibrary package
then map T.unpack
$ T.append "lib:" (T.pack (packageNameString (packageName package)))
: map
(T.append "flib:")
(getBuildableListText (packageForeignLibraries package))
else []
)
++ map
(T.unpack . T.append "lib:")
(Set.toList $ packageSubLibraries package)
(getBuildableListText $ packageSubLibraries package)
++ map
(T.unpack . T.append "exe:")
(Set.toList $ exesToBuild executableBuildStatuses lp)
Expand Down
32 changes: 17 additions & 15 deletions src/Stack/Build/Source.hs
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,11 @@ import qualified Distribution.PackageDescription as C
import qualified Pantry.SHA256 as SHA256
import Stack.Build.Cache ( tryGetBuildCache )
import Stack.Build.Haddock ( shouldHaddockDeps )
import Stack.Package ( resolvePackage )
import Stack.Package
( hasMainBuildableLibrary, packageBenchmarks, packageExes
, resolvePackage
)
import Stack.PackageFile ( getPackageFile )
import Stack.Prelude
import Stack.SourceMap
( DumpedGlobalPackage, checkFlagsUsedThrowing
Expand All @@ -40,6 +44,7 @@ import Stack.Types.BuildOpts
, TestOpts (..), boptsCLIAllProgOptions
)
import Stack.Types.CabalConfigKey ( CabalConfigKey (..) )
import Stack.Types.CompCollection ( getBuildableSetText )
import Stack.Types.CompilerPaths ( HasCompiler, getCompilerPath )
import Stack.Types.Config ( Config (..), HasConfig (..), buildOptsL )
import Stack.Types.Curator ( Curator (..) )
Expand All @@ -52,10 +57,11 @@ import Stack.Types.NamedComponent
( NamedComponent (..), isCSubLib, splitComponents )
import Stack.Types.Package
( FileCacheInfo (..), LocalPackage (..), Package (..)
, PackageConfig (..), PackageLibraries (..)
, dotCabalGetPath, memoizeRefWith, runMemoizedWith
, PackageConfig (..), dotCabalGetPath, memoizeRefWith
, runMemoizedWith
)
import Stack.Types.PackageFile ( PackageWarning, getPackageFiles )
import Stack.Types.PackageFile
( PackageComponentFile (..), PackageWarning )
import Stack.Types.Platform ( HasPlatform (..) )
import Stack.Types.SourceMap
( CommonPackage (..), DepPackage (..), ProjectPackage (..)
Expand Down Expand Up @@ -315,7 +321,7 @@ loadLocalPackage pp = do
( packageExes pkg
, if boptsTests bopts
&& maybe True (Set.notMember name . curatorSkipTest) mcurator
then Map.keysSet (packageTests pkg)
then getBuildableSetText (packageTestSuites pkg)
else Set.empty
, if boptsBenchmarks bopts
&& maybe
Expand All @@ -334,13 +340,9 @@ loadLocalPackage pp = do
-- individual executables or library") is resolved, 'hasLibrary' is only
-- relevant if the library is part of the target spec.
Just _ ->
let hasLibrary =
case packageLibraries pkg of
NoLibraries -> False
HasLibraries _ -> True
in hasLibrary
|| not (Set.null nonLibComponents)
|| not (Set.null $ packageSubLibraries pkg)
hasMainBuildableLibrary pkg
|| not (Set.null nonLibComponents)
|| not (null $ packageSubLibraries pkg)

filterSkippedComponents =
Set.filter (not . (`elem` boptsSkipComponents bopts))
Expand Down Expand Up @@ -422,7 +424,7 @@ loadLocalPackage pp = do
-- must not be buildable.
, lpUnbuildable = toComponents
(exes `Set.difference` packageExes pkg)
(tests `Set.difference` Map.keysSet (packageTests pkg))
(tests `Set.difference` getBuildableSetText (packageTestSuites pkg))
(benches `Set.difference` packageBenchmarks pkg)
}

Expand Down Expand Up @@ -499,8 +501,8 @@ getPackageFilesForTargets ::
-> Set NamedComponent
-> RIO env (Map NamedComponent (Set (Path Abs File)), [PackageWarning])
getPackageFilesForTargets pkg cabalFP nonLibComponents = do
(components',compFiles,otherFiles,warnings) <-
getPackageFiles (packageFiles pkg) cabalFP
PackageComponentFile components' compFiles otherFiles warnings <-
getPackageFile pkg cabalFP
let necessaryComponents =
Set.insert CLib $ Set.filter isCSubLib (M.keysSet components')
components = necessaryComponents `Set.union` nonLibComponents
Expand Down
Loading