Skip to content

Commit

Permalink
fix: scanning lock due of unresolved deps
Browse files Browse the repository at this point in the history
- Renamed snykConf to projectConfigs, to improve the readability

- Created configsSuccessfullyResolved method that confirms if
  configs sets as canBeResolved=true, can be really resolved or not.

  If there is an error now, while resolving configs set as
  canBeResolved=true, we are no longer blocking the scanning. We ignore
  those configs whose depGraph cannot be computed and move forward with the
  scanning process since it's not a snyk issue but a given gradle project
  resolution config caused by bad config or 3rd party gradle dependencies bad behaving.

By going into `test/fixtures/successful-scan-with-unresolved-custom-configs/build.gradle` and running `gradle -q dependencies` you will see the following message (pic below)

This fixture emulates issue gradle/gradle#6854, **where gradle cannot resolve incremental analysis configurations**.

<img width="638" alt="Screen Shot 2021-01-31 at 18 54 04" src="https://user-images.githubusercontent.com/40601533/106393164-d369dd80-63f5-11eb-9335-bcc1716b890a.png">

What does **FAILED** means? Means any dependency belonging to these configuratios failed to be resolved (compute depGraph)

Continue reading about failed resolution in Gradle Docs.. https://docs.gradle.org/current/userguide/viewing_debugging_dependencies.html#example_rendering_the_dependency_report_for_a_custom_configuration
  • Loading branch information
anthogez committed Jan 31, 2021
1 parent 20b5fdc commit 23169a9
Show file tree
Hide file tree
Showing 5 changed files with 1,022 additions and 26 deletions.
46 changes: 30 additions & 16 deletions lib/init.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import java.util.regex.Matcher
import org.gradle.util.GradleVersion

// Snyk dependency resolution script for Gradle.
// Tested on Gradle versions from 2.14 to 5.4.1
// Tested on Gradle versions from v2.14 to v6.8.1

// This script does the following: for all the projects in the build file,
// generate a merged configuration of all the available configurations,
Expand Down Expand Up @@ -129,6 +129,19 @@ def getSnykGraph(Iterable deps) {
return graph.nodes
}

def configsSuccessfullyResolved(configurations) {
def resolvedConfigurations = [];
configurations.each({ configuration ->
try {
configuration.resolve();
resolvedConfigurations.add(configuration);
} catch(Exception ex) {
println('NOT_RESOLVED ' + ex.toString())
}
})
return resolvedConfigurations;
}

// We are attaching this task to every project, as this is the only reliable way to run it
// when we start with a subproject build.gradle. As a consequence, we need to make sure we
// only ever run it once, for the "starting" project.
Expand Down Expand Up @@ -218,7 +231,8 @@ allprojects { everyProj ->
rootProject.allprojects.findAll(shouldScanProject).each { proj ->
println('SNYKECHO processing project: ' + proj.name)

def snykConf = null
def projectConfigs = null
def filteredProjectConfigs = null

// Gradle v3.0+ contains concepts as attributes, config canBeResolved, that does not exist in legacy versions
final GradleVersion gradleVersionInUse = GradleVersion.current();
Expand All @@ -230,40 +244,40 @@ allprojects { everyProj ->
// we can compute a dependency graph and that contains all the necessary information for resolution to happen.
if (confAttrSpec != null) {
// Drop all the configrations that don't match the attribute filter
snykConf = proj.configurations
.findAll({ it.canBeResolved == true && it.canBeConsumed == false && it.name =~ confNameFilter && matchesAttributeFilter(it) })
filteredProjectConfigs = proj.configurations.findAll({ it.canBeResolved == true && it.canBeConsumed == false && it.name =~ confNameFilter && matchesAttributeFilter(it) })

if(snykConf.size() == 0) {
snykConf = proj.configurations
if(filteredProjectConfigs.size() == 0) {
filteredProjectConfigs = proj.configurations
.findAll({ it.canBeResolved == true && it.canBeConsumed == true && it.name =~ confNameFilter && matchesAttributeFilter(it) })
}

projectConfigs = configsSuccessfullyResolved(filteredProjectConfigs)
} else {
snykConf = proj.configurations
.findAll({ it.canBeResolved == true && it.canBeConsumed == false && it.name =~ confNameFilter })

// if we cannot find dependencies that can be only resolved but not consumable
// we try to find configs that are simultaneously resolvable and consumable
// to prevent dependency resolution conflicts (e.g. Cannot choose between the following variants)
// we avoid the coexistence of (canBeResolved: true, canBeConsumed: false) and (canBeResolved: true, canBeConsumed: true) configs
if(snykConf.size() == 0) {
snykConf = proj.configurations
filteredProjectConfigs = proj.configurations.findAll({ it.canBeResolved == true && it.canBeConsumed == false && it.name =~ confNameFilter })
if(filteredProjectConfigs.size() == 0) {
filteredProjectConfigs = proj.configurations
.findAll({ it.canBeResolved == true && it.canBeConsumed == true && it.name =~ confNameFilter })
}
projectConfigs = configsSuccessfullyResolved(filteredProjectConfigs)
}
} else {
snykConf = proj.configurations.findAll({ it.name =~ confNameFilter })
def configsFilteredByConfName = proj.configurations.findAll({ it.name =~ confNameFilter })
projectConfigs = configsSuccessfullyResolved(configsFilteredByConfName)
}

if (snykConf.size() == 0 && proj.configurations.size() > 0) {
if (projectConfigs.size() == 0 && proj.configurations.size() > 0) {
throw new RuntimeException('Matching configurations not found: ' + confNameFilter +
', available configurations for project ' + proj + ': '
+ proj.configurations.collect { it.name })
}

if (snykConf != null) {
println('SNYKECHO resolving configuration ' + snykConf.name)
def gradleFirstLevelDeps = snykConf.resolvedConfiguration.firstLevelModuleDependencies
if (projectConfigs != null) {
println('SNYKECHO resolving configuration ' + projectConfigs.name)
def gradleFirstLevelDeps = projectConfigs.resolvedConfiguration.firstLevelModuleDependencies.findAll({ it.size() > 0 })
println('SNYKECHO converting gradle graph to snyk-graph format')
projectsDict[proj.name] = [
'targetFile': findProject(proj.path).buildFile.toString(),
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
plugins {
id 'io.gatling.gradle' version "3.5.0"
}

build.dependsOn gatlingClasses

dependencies {
gatling group: 'org.scalaj', name: 'scalaj-http_2.13', version: '2.4.2'
}

gatling {
simulations = {
exclude "**/paxos/BaseSimulation.scala"
exclude "**/paxos/Configuration.scala"
}
}
Loading

0 comments on commit 23169a9

Please sign in to comment.