Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove prefix dots in debricked custom files #137

Merged
merged 12 commits into from
Oct 27, 2023
14 changes: 8 additions & 6 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,16 @@ coverage.html
node_modules
debricked
dist/
/.debricked-go-dependencies.txt
/.gomod.debricked.lock
/debricked-go-dependencies.txt
/gomod.debricked.lock
/.env
test/resolve/testdata/pip/requirements.txt.venv/
test/resolve/testdata/pip/.requirements.txt.debricked.lock
test/resolve/testdata/pip/requirements.txt.pip.debricked.lock
internal/cmd/scan/testdata/npm/yarn.lock
internal/resolution/pm/gradle/.gradle-init-script.debricked.groovy
test/resolve/testdata/npm/yarn.lock
test/resolve/testdata/nuget/packages.lock.json
test/resolve/testdata/nuget/obj
.debricked.fingerprints.wfp
test/resolve/testdata/nuget/csproj/packages.lock.json
test/resolve/testdata/nuget/packagesconfig/packages.config.nuget.debricked.lock
test/resolve/testdata/nuget/**/obj
debricked.fingerprints.wfp
test/resolve/testdata/gomod/gomod.debricked.lock
5 changes: 3 additions & 2 deletions examples/templates/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,10 @@ In order for us to analyze all dependencies in your project, their versions, and

**Example 1:** If npm is used in your project you will have a `package.json` file, but in order for us to scan all your dependencies we need either `package-lock.json` or `yarn.lock` as well.

**Example 2:** If Maven is used in your project you will have a `pom.xml` file, but in order for us to resolve all your dependencies we need a second file, as Maven does not offer a lock file system. Instead, Maven dependency:tree plugin can be used to create a file called `.maven.debricked.lock`
**Example 2:** If Maven is used in your project you will have a `pom.xml` file, but in order for us to resolve all your dependencies we need a second file, as Maven does not offer a lock file system. Instead, Maven dependency:tree plugin can be used to create a file called `
maven.debricked.lock`

## Debricked CLI dependency resolution
In all templates the manifest file resolution is enabled by default. That means Debricked CLI will attempt to resolve manifest files that belong to package managers that does not offer lock file systems.
For example, if a `pom.xml` is found by Debricked CLI it will attempt to create `.maven.debricked.lock` automatically.
For example, if a `pom.xml` is found by Debricked CLI it will attempt to create `maven.debricked.lock` automatically.
To disable manifest file resolution, add the flag `--no-resolve`.
2 changes: 1 addition & 1 deletion internal/file/finder_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ func TestGetGroupsPIP(t *testing.T) {
manifestsFound = append(manifestsFound, manifestFile)
}
manifestsExpected := []string{"testdata/pip/requirements-dev.txt", "testdata/pip/requirements.txt", "testdata/pip/requirements.test.txt"}
locksExpected := []string{"testdata/pip/requirements-dev.txt.pip.debricked.lock", "testdata/pip/.requirements.txt.pip.debricked.lock"}
locksExpected := []string{"testdata/pip/requirements-dev.txt.pip.debricked.lock", "testdata/pip/requirements.txt.pip.debricked.lock"}
sort.Strings(manifestsExpected)
sort.Strings(locksExpected)
sort.Strings(manifestsFound)
Expand Down
4 changes: 2 additions & 2 deletions internal/file/group_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -97,10 +97,10 @@ func TestGroupMatchFile(t *testing.T) {
var match bool
match = matchFile("package.json", "package-lock.json")
assert.Equal(t, match, true)
match = matchFile("requirements.txt", ".requirements.txt.pip.debricked.lock")
match = matchFile("requirements.txt", "requirements.txt.pip.debricked.lock")
assert.Equal(t, match, true)
match = matchFile("requirements.txt", "requirements.txt.pip.debricked.lock")
assert.Equal(t, match, true)
match = matchFile("requirements-test.txt", ".requirements.txt.pip.debricked.lock")
match = matchFile("requirements-test.txt", "requirements.txt.pip.debricked.lock")
assert.Equal(t, match, false)
}
4 changes: 2 additions & 2 deletions internal/file/groups_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -157,9 +157,9 @@ func TestMatchGroupsExpected(t *testing.T) {
"foo/bar/examples/test/requirements.txt": {},
"foo/asd/requirements-test-dev.txt": {"foo/asd/.requirements-test-dev.txt.pip.debricked.lock"},
"foo/asd/requirements-test.txt": {"foo/asd/.requirements-test.txt.pip.debricked.lock"},
"foo/asd/requirements.txt": {"foo/asd/.requirements.txt.pip.debricked.lock"},
"foo/asd/requirements.txt": {"foo/asd/requirements.txt.pip.debricked.lock"},
"foo/asd/requirements-api.txt": {},
"foo/asd/src/main/event_listeners/requirements.txt": {"foo/asd/src/main/event_listeners/.requirements.txt.pip.debricked.lock"},
"foo/asd/src/main/event_listeners/requirements.txt": {"foo/asd/src/main/event_listeners/requirements.txt.pip.debricked.lock"},
"foo/asd/src/main/util/test/composer.json": {},
}

Expand Down
2 changes: 1 addition & 1 deletion internal/fingerprint/fingerprint.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ var EXCLUDED_FILES = []string{
}

const (
OutputFileNameFingerprints = ".debricked.fingerprints.wfp"
OutputFileNameFingerprints = "debricked.fingerprints.wfp"
)

func isExcludedFile(filename string) bool {
Expand Down
2 changes: 1 addition & 1 deletion internal/resolution/pm/gomod/job.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import (
)

const (
fileName = ".gomod.debricked.lock"
fileName = "gomod.debricked.lock"
)

type Job struct {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,6 @@ allprojects {

allprojects {
task debrickedAllDeps(type: DependencyReportTask) {
outputFile = file('./.gradle.debricked.lock')
outputFile = file('./gradle.debricked.lock')
}
}
2 changes: 1 addition & 1 deletion internal/resolution/pm/maven/cmd_factory.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ func (_ CmdFactory) MakeDependencyTreeCmd(workingDirectory string) (*exec.Cmd, e
Args: []string{
"mvn",
"dependency:tree",
"-DoutputFile=.maven.debricked.lock",
"-DoutputFile=" + lockFileExtension,
"-DoutputType=tgf",
"--fail-at-end",
},
Expand Down
2 changes: 1 addition & 1 deletion internal/resolution/pm/maven/cmd_factory_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ func TestMakeDependencyTreeCmd(t *testing.T) {
args := cmd.Args
assert.Contains(t, args, "mvn")
assert.Contains(t, args, "dependency:tree")
assert.Contains(t, args, "-DoutputFile=.maven.debricked.lock")
assert.Contains(t, args, "-DoutputFile=maven.debricked.lock")
assert.Contains(t, args, "-DoutputType=tgf")
assert.Contains(t, args, "--fail-at-end")
}
4 changes: 4 additions & 0 deletions internal/resolution/pm/maven/job.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ import (
"github.com/debricked/cli/internal/resolution/job"
)

const (
lockFileExtension = "maven.debricked.lock"
)

type Job struct {
job.BaseJob
cmdFactory ICmdFactory
Expand Down
7 changes: 5 additions & 2 deletions internal/resolution/pm/nuget/cmd_factory.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ import (
"strings"
)

const packagesConfigLockfile = "packages.config.nuget.debricked.lock"
const nugetLockfile = "packages.lock.json"

type ICmdFactory interface {
MakeInstallCmd(command string, file string) (*exec.Cmd, error)
GetTempoCsproj() string
Expand Down Expand Up @@ -77,14 +80,14 @@ func (cmdf *CmdFactory) MakeInstallCmd(command string, file string) (*exec.Cmd,
return nil, err
}

fileLockName := "packages.lock.json"
fileLockName := nugetLockfile
if packageConfig.MatchString(file) {
file, err = cmdf.convertPackagesConfigToCsproj(file, command)
cmdf.tempoCsproj = file
if err != nil {
return nil, err
}
fileLockName = ".packages.config.nuget.debricked.lock"
fileLockName = packagesConfigLockfile
}

fileDir := filepath.Dir(file)
Expand Down
2 changes: 1 addition & 1 deletion internal/resolution/pm/nuget/cmd_factory_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ func TestMakeInstallCmdPackagsConfig(t *testing.T) {
assert.Contains(t, args, "restore")
assert.Contains(t, args, "--use-lock-file")
assert.Contains(t, args, "--lock-file-path")
assert.Contains(t, args, ".packages.config.nuget.debricked.lock")
assert.Contains(t, args, "packages.config.nuget.debricked.lock")

// Cleanup: Remove the created .csproj file
if err := os.Remove("testdata/valid/packages.config.nuget.debricked.csproj.temp"); err != nil {
Expand Down
2 changes: 1 addition & 1 deletion internal/resolution/pm/pip/job.go
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ func (j *Job) writeLockContent() error {
return err
}

lockFileName := fmt.Sprintf(".%s%s", filepath.Base(j.GetFile()), lockFileExtension)
lockFileName := fmt.Sprintf("%s%s", filepath.Base(j.GetFile()), lockFileExtension)
lockFile, err := j.fileWriter.Create(util.MakePathFromManifestFile(j.GetFile(), lockFileName))
if err != nil {
return err
Expand Down
81 changes: 63 additions & 18 deletions test/resolve/resolver_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,41 +3,74 @@ package resolve
import (
"os"
"path/filepath"
"strings"
"testing"

"github.com/debricked/cli/internal/cmd/resolve"
"github.com/debricked/cli/internal/wire"
"github.com/stretchr/testify/assert"
)

func removeLines(input, prefix string) string {
lines := strings.Split(input, "\n")
var result []string

for _, line := range lines {
if !strings.HasPrefix(line, prefix) {
result = append(result, line)
}
}

return strings.Join(result, "\n")
}

func TestResolves(t *testing.T) {
cases := []struct {
name string
manifestFile string
lockFileName string
expectedFile string
name string
manifestFile string
lockFileName string
expectedFile string
packageManager string
}{
{
name: "basic package.json",
manifestFile: "testdata/npm/package.json",
lockFileName: "yarn.lock",
expectedFile: "testdata/npm/yarn-expected.lock",
name: "basic package.json",
manifestFile: "testdata/npm/package.json",
lockFileName: "yarn.lock",
expectedFile: "testdata/npm/yarn-expected.lock",
packageManager: "npm",
},
{
name: "basic requirements.txt",
manifestFile: "testdata/pip/requirements.txt",
lockFileName: ".requirements.txt.debricked.lock",
expectedFile: "testdata/pip/expected.lock",
name: "basic requirements.txt",
manifestFile: "testdata/pip/requirements.txt",
lockFileName: "requirements.txt.pip.debricked.lock",
expectedFile: "testdata/pip/expected.lock",
packageManager: "pip",
},
{
name: "basic .csproj",
manifestFile: "testdata/nuget/basic.csproj",
lockFileName: "packages.lock.json",
expectedFile: "testdata/nuget/packages-expected.lock.json",
name: "basic .csproj",
manifestFile: "testdata/nuget/csproj/basic.csproj",
lockFileName: "packages.lock.json",
expectedFile: "testdata/nuget/csproj/packages-expected.lock.json",
packageManager: "nuget",
},
{
name: "basic packages.config",
manifestFile: "testdata/nuget/packagesconfig/packages.config",
lockFileName: "packages.config.nuget.debricked.lock",
expectedFile: "testdata/nuget/packagesconfig/packages.config.expected.lock",
packageManager: "nuget",
},
{
name: "basic go.mod",
manifestFile: "testdata/gomod/go.mod",
lockFileName: "gomod.debricked.lock",
expectedFile: "testdata/gomod/expected.lock",
packageManager: "gomod",
},
}

for _, c := range cases {
for _, cT := range cases {
c := cT
t.Run(c.name, func(t *testing.T) {
resolveCmd := resolve.NewResolveCmd(wire.GetCliContainer().Resolver())
lockFileDir := filepath.Dir(c.manifestFile)
Expand All @@ -53,8 +86,20 @@ func TestResolves(t *testing.T) {

expectedFileContents, fileErr := os.ReadFile(c.expectedFile)
assert.NoError(t, fileErr)
expectedString := string(expectedFileContents)
actualString := string(lockFileContents)

if c.packageManager == "pip" {
// Remove locations as it is dependent on the machine
expectedString = removeLines(expectedString, "Location: ")
actualString = removeLines(actualString, "Location: ")

assert.Equal(t, string(expectedFileContents), string(lockFileContents))
} else if c.packageManager == "nuget" {
// Remove hashes as that is different on different OS
expectedString = removeLines(expectedString, " \"contentHash\":")
actualString = removeLines(actualString, " \"contentHash\":")
}
assert.Equal(t, expectedString, actualString)
})
}
}
Loading