Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce readme files for package managers #185

Merged
merged 2 commits into from
Jan 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions internal/resolution/pm/bower/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Bower resolution logic

The way resolution of bower lock files works is as follows:

1. Run `bower install --save --save-dev --save-exact --allow-root` in order to install all dependencies
2. Run `bower list` to get installed dependencies tree

The result of `bower list` command is then being written into the lock file.
7 changes: 7 additions & 0 deletions internal/resolution/pm/composer/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Composer resolution logic

The way resolution of composer lock files works is as follows:

1. Run `composer update --no-interaction --no-scripts --ignore-platform-reqs --no-autoloader --no-install --no-plugins --no-audit` in order to install all dependencies

Generated `composer.lock` file is then uploaded together with `composer.json` for scanning.
8 changes: 8 additions & 0 deletions internal/resolution/pm/gomod/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Go resolution logic

The way resolution of go lock files works is as follows:

1. Run `go mod graph` in order to create dependency graph
2. Run `go list -mod=readonly -e -m all` to get the list of packages

The results of the commands above are then combined to form the finished lock file.
9 changes: 9 additions & 0 deletions internal/resolution/pm/gradle/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Gradle resolution logic

The way resolution of gradle lock files works is as follows:

1. Generate init script file for project and subprojects
2. Run `gradle --init-script gradle-init-script.groovy debrickedAllDeps` in order to create dependencies graph
3. In case permission to execute gradlew is not granted, fallback to PATHs gradle installation is used: `gradle --init-script gradle-init-script.groovy debrickedFindSubProjectPaths`

The results of the executed command above is then being written into the lock file.
8 changes: 8 additions & 0 deletions internal/resolution/pm/maven/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Maven resolution logic

The way resolution of maven lock files works is as follows:

1. Parse `pom.xml` file
2. Run `mvn dependency:tree -DoutputFile=maven.debricked.lock -DoutputType=tgf --fail-at-end` in order to install all dependencies

The result of the second command above is then written to `maven.debricked.lock` file.
7 changes: 7 additions & 0 deletions internal/resolution/pm/npm/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# NPM resolution logic

The way resolution of NPM lock files works is as follows:

1. Run `npm install --ignore-scripts --audit=false --bin-links=false` in order to install all dependencies

Generated `package-lock.json` file is then uploaded together with `package.json` for scanning.
21 changes: 21 additions & 0 deletions internal/resolution/pm/nuget/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Nuget resolution logic

There are two supported files for resolution of nuget lock files:

### packages.config

We need to convert a `packages.config` file to a `.csproj` file. This is to enable the use of the dotnet restore command
that enables Debricked to parse out transitive dependencies. This may add some additional framework dependencies that
will not show up if we only scan the `packages.config` file. This is done in a few steps:

1. Parse `packages.config` file
2. Run `dotnet --version` to get dotnet version
3. Collect unique target frameworks and packages from the file
4. Create `.nuget.debricked.csproj.temp` file with the collected data

With this done we can move on to the next section

### .csproj

1. Run `dotnet restore <file> --use-lock-file --lock-file-path <lock_file>` in order to restore the dependencies and tools of a project (lock file name can be different depend on which manifest file is being resolved)
2. Cleanup temporary csproj file after lock file is created (for `packages.config` case)
6 changes: 3 additions & 3 deletions internal/resolution/pm/nuget/cmd_factory.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,15 +47,15 @@ var packagesConfigTemplate = `

type CmdFactory struct {
execPath IExecPath
packageConfgRegex string
packageConfigRegex string
4ernovm marked this conversation as resolved.
Show resolved Hide resolved
packagesConfigTemplate string
tempoCsproj string
}

func NewCmdFactory(execPath IExecPath) *CmdFactory {
return &CmdFactory{
execPath: execPath,
packageConfgRegex: PackagesConfigRegex,
packageConfigRegex: PackagesConfigRegex,
packagesConfigTemplate: packagesConfigTemplate,
tempoCsproj: "",
}
Expand All @@ -75,7 +75,7 @@ func (cmdf *CmdFactory) MakeInstallCmd(command string, file string) (*exec.Cmd,

// If the file is a packages.config file, convert it to a .csproj file
// check regex with PackagesConfigRegex
packageConfig, err := regexp.Compile(cmdf.packageConfgRegex)
packageConfig, err := regexp.Compile(cmdf.packageConfigRegex)
if err != nil {
return nil, err
}
Expand Down
12 changes: 6 additions & 6 deletions internal/resolution/pm/nuget/cmd_factory_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -303,7 +303,7 @@ func TestCreateCsprojContent(t *testing.T) {
t.Run(test.name, func(t *testing.T) {
cmd := CmdFactory{
execPath: ExecPath{},
packageConfgRegex: PackagesConfigRegex,
packageConfigRegex: PackagesConfigRegex,
packagesConfigTemplate: test.tmpl,
}
_, err := cmd.createCsprojContentWithTemplate(test.targetFrameworksStr, test.packages)
Expand All @@ -317,8 +317,8 @@ func TestCreateCsprojContent(t *testing.T) {
func TestMakeInstallCmdBadPackagesConfigRegex(t *testing.T) {

cmd, err := (&CmdFactory{
execPath: ExecPath{},
packageConfgRegex: "[",
execPath: ExecPath{},
packageConfigRegex: "[",
}).MakeInstallCmd(nuget, "file")

assert.Error(t, err)
Expand Down Expand Up @@ -364,8 +364,8 @@ func (ExecPathErr) LookPath(file string) (string, error) {
func TestMakeInstallCmdExecPathError(t *testing.T) {

cmd, err := (&CmdFactory{
execPath: ExecPathErr{},
packageConfgRegex: PackagesConfigRegex,
execPath: ExecPathErr{},
packageConfigRegex: PackagesConfigRegex,
}).MakeInstallCmd(nuget, "file")

assert.Error(t, err)
Expand Down Expand Up @@ -421,7 +421,7 @@ func TestConvertPackagesConfigToCsproj(t *testing.T) {

cmd := CmdFactory{
execPath: ExecPath{},
packageConfgRegex: PackagesConfigRegex,
packageConfigRegex: PackagesConfigRegex,
packagesConfigTemplate: tt.packagesConfigTemplate,
}
_, err := cmd.convertPackagesConfigToCsproj(tt.filePath, nugetCommand)
Expand Down
15 changes: 15 additions & 0 deletions internal/resolution/pm/pip/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Pip resolution logic

The way resolution of pip lock files works is as follows:

1. Create a Venv in which we do the installation and run all commands
2. Run `pip install -r <requirements.txt_file>` in order to install all dependencies
3. Run `cat` to get the contents of the requirements.txt file
4. Run `pip list` to get a list of all installed packages
5. Run `pip show <list_of_installed_packages>` to get more in-depth information from each package, including the relations between dependencies

The results of the commands above are then combined to form the finished lock file with the following sections:

1. The contents of the requirements.txt (from cat)
2. The list of all installed dependencies (from pip list)
3. More detailed information on each package with relations (from pip show)
7 changes: 7 additions & 0 deletions internal/resolution/pm/yarn/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Yarn resolution logic

The way resolution of yarn lock files works is as follows:

1. Run `install --non-interactive --ignore-scripts --ignore-engines --ignore-platform --no-bin-link --production=false` in order to install all dependencies

Generated `yarn.lock` file is then uploaded together with `package.json` for scanning.
Loading