Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Plugin Support #794

Merged
merged 92 commits into from
Apr 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
078f441
code structure
CodingDepot Nov 20, 2023
9088273
new build file
CodingDepot Nov 20, 2023
566a5a8
added spotbugs plugin sarif4k dependency
CodingDepot Nov 29, 2023
d8c4a5f
added and moved parser implementation
CodingDepot Nov 29, 2023
afded52
added and moved aggregate implementation
CodingDepot Nov 29, 2023
8d26e75
change include to fit renamed module
CodingDepot Nov 29, 2023
7d9eb27
moved comments
CodingDepot Nov 29, 2023
3069c98
add reports used in testing
CodingDepot Nov 29, 2023
cca931d
add Executor for SpotBugs
CodingDepot Dec 4, 2023
aa01b07
add dummy files for testing
CodingDepot Dec 4, 2023
adc6c75
add many TODOs and an initial method that aims to fix rule references…
CodingDepot Dec 6, 2023
6048333
move result.ruleID and result.ruleIndex into the result.rule object i…
CodingDepot Dec 6, 2023
e22cfbb
add new executor using the FindSecBugs-Plugin
CodingDepot Dec 11, 2023
94e6ef2
initial implementation of PMDExecutor
CodingDepot Dec 11, 2023
ee66096
narrow down dependency error
CodingDepot Dec 13, 2023
d46bcdc
fix clashing dependencies
CodingDepot Dec 13, 2023
0e85789
add more vulnerable demo code
CodingDepot Dec 13, 2023
51fb32c
fix test code
CodingDepot Dec 13, 2023
6c32844
experiment with plugin application
CodingDepot Dec 13, 2023
521c0ef
revert example file
CodingDepot Dec 18, 2023
accbe1a
fix FindSecBugsExecutor
CodingDepot Dec 18, 2023
8020509
completely overhaul test structure
CodingDepot Dec 18, 2023
461b8ea
fix name of test
CodingDepot Dec 18, 2023
d063573
remove SpotbugsExecutor.kt as it is incorporated in FindSecBugsExecut…
CodingDepot Jan 8, 2024
6c20d1f
rename plugins
CodingDepot Jan 8, 2024
e1f89b7
add plugins to codyze configuration
CodingDepot Jan 10, 2024
78a8389
establish plugins as subcommands instead of cli options
CodingDepot Jan 15, 2024
db140bb
add necessary gradle dependency
CodingDepot Jan 15, 2024
4e51ce5
try aggregating the results
CodingDepot Jan 15, 2024
8b63637
register all executors during use
CodingDepot Jan 17, 2024
cebaa97
fix issues
CodingDepot Jan 17, 2024
a8fbcd8
ignore the physical artifact location in tests
CodingDepot Jan 17, 2024
0b8946f
Merge branch 'main' into rh/aggregator
fwendland Jan 18, 2024
7272f76
Merge remote-tracking branch 'origin/rh/aggregator' into rh/aggregator
CodingDepot Jan 22, 2024
8996a65
fix codyze as Aggregate driver and create custom invocation object
CodingDepot Jan 22, 2024
6b4a5a7
move necessary test resources
CodingDepot Jan 22, 2024
59c9233
add context option for plugins
CodingDepot Jan 22, 2024
7d82874
add context option for plugins #2
CodingDepot Jan 22, 2024
29cac15
fix missing context reference in case main class was not called (e.g.…
CodingDepot Jan 22, 2024
103d480
remove EmptyPlugin.kt as it is not needed with the subcommand approach
CodingDepot Jan 22, 2024
c4b3769
correctly resolve output path
CodingDepot Jan 22, 2024
b1c4514
resolve all detekt issues in codyze-core
CodingDepot Jan 22, 2024
a995b12
resolve all detekt issues in codyze-core:Test
CodingDepot Jan 22, 2024
7c13582
resolve all detekt issues in codyze-plugins
CodingDepot Jan 24, 2024
f2b34c7
resolve all detekt issues in codyze-cli
CodingDepot Jan 24, 2024
9762c9f
resolve last detekt issue in codyze-core:Test
CodingDepot Jan 24, 2024
f1769c0
add documentation for the Plugins
CodingDepot Jan 24, 2024
fbf5cf8
add documentation for the FindSecBugs plugin
CodingDepot Jan 24, 2024
bcc78b3
add codyze-plugins to code-coverage-report dependencies
CodingDepot Jan 29, 2024
97f87b7
debug: upload test report as artifact
CodingDepot Jan 31, 2024
c21a3f6
debug: upload whole tet directory
CodingDepot Jan 31, 2024
7e66b81
fix cleanup path
CodingDepot Jan 31, 2024
030cffb
create file before trying to get its writer
CodingDepot Jan 31, 2024
5049426
remove debug step
CodingDepot Jan 31, 2024
3d2c7ee
re-add debug step
CodingDepot Jan 31, 2024
29c6251
FindSecBugs Plugin now creates any necessary parent directories for i…
CodingDepot Jan 31, 2024
b3a9959
remove debug step
CodingDepot Jan 31, 2024
64f353a
Merge branch 'main' into rh/aggregator
CodingDepot Jan 31, 2024
dad8ead
increase test coverage for the Aggregate
CodingDepot Feb 5, 2024
3f6d74d
remove unused import
CodingDepot Feb 5, 2024
8713a17
Merge branch 'main' into rh/aggregator
CodingDepot Feb 5, 2024
0ea248f
improve tests
CodingDepot Feb 5, 2024
6f10f67
fix all detekt issues
CodingDepot Feb 5, 2024
1abd7c9
Merge branch 'main' into rh/aggregator
CodingDepot Feb 7, 2024
e572a40
exclude Program entry point and abstract Plugin class from coverage r…
CodingDepot Mar 7, 2024
3ce89da
fix typo in main file path
CodingDepot Mar 7, 2024
56c65b5
Merge branch 'main' into rh/aggregator
fwendland Mar 25, 2024
9a42f16
Merge branch 'main' into rh/aggregator
fwendland Mar 26, 2024
86b69fa
Merge branch 'main' into rh/aggregator
fwendland Apr 2, 2024
a30a9b1
Merge branch 'main' into rh/aggregator
fwendland Apr 3, 2024
559e1f6
Merge branch 'main' into rh/aggregator
fwendland Apr 3, 2024
58adce1
Merge branch 'main' into rh/aggregator
fwendland Apr 5, 2024
addf1bd
Merge branch 'main' into rh/aggregator
fwendland Apr 11, 2024
ab1cc98
Move base for plugin support into codyze-core
fwendland Apr 15, 2024
2836960
Make plugin module optional using flag
fwendland Apr 15, 2024
c255345
Load plugins dynamically and load findsecbugs-plugin through dependen…
fwendland Apr 15, 2024
24b2dc8
mark feature as experimental in the docs
CodingDepot Apr 15, 2024
0c44f41
fix detekt issues
CodingDepot Apr 15, 2024
543ecff
Comment cleanup
CodingDepot Apr 15, 2024
6cb503d
Merge branch 'main' into rh/aggregator
fwendland Apr 15, 2024
21ba7dd
ignore experimental Plugin feature
CodingDepot Apr 15, 2024
30f560d
Merge remote-tracking branch 'origin/rh/aggregator' into rh/aggregator
CodingDepot Apr 15, 2024
f8cb9fb
Merge branch 'main' into rh/aggregator
fwendland Apr 15, 2024
3f852c6
Merge branch 'main' into rh/aggregator
fwendland Apr 15, 2024
98927a1
delete separate tool result file if it was not specified
CodingDepot Apr 15, 2024
5dd59c6
remove wrong constraint for flag that required the existence of the file
CodingDepot Apr 15, 2024
6b409b8
circumvent PMD not handling complex ruleset paths
CodingDepot Apr 15, 2024
26deb3a
do not force cliNames to be lowercase anymore
CodingDepot Apr 15, 2024
0eede6f
Merge remote-tracking branch 'origin/rh/aggregator' into rh/aggregator
CodingDepot Apr 15, 2024
8688a4d
Remove commented out project dependency
fwendland Apr 15, 2024
e8f721f
Pull in feature flag from root project
fwendland Apr 15, 2024
7028ee0
Fix test
fwendland Apr 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -78,3 +78,13 @@ subprojects {
}
}
}

/*
* Optional and experimental features
*/
// this code block also exists in `settings.gradle.kts`
val enablePluginSupport: Boolean by extra {
val enablePluginSupport: String? by project
enablePluginSupport.toBoolean()
}
project.logger.lifecycle("Plugin feature is ${if (enablePluginSupport) "enabled" else "disabled"}")
9 changes: 9 additions & 0 deletions buildSrc/src/main/kotlin/features.gradle.kts
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
plugins {
kotlin("jvm")
}

val enablePluginSupport: Boolean by rootProject.extra

dependencies {
if (enablePluginSupport) runtimeOnly(project(":codyze-plugins"))
}
6 changes: 6 additions & 0 deletions code-coverage-report/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -33,12 +33,18 @@ reporting {
}
}

val enablePluginSupport: Boolean by rootProject.extra
project.logger.lifecycle("Plugin feature is ${if (enablePluginSupport) "enabled" else "disabled"}")

dependencies {
jacocoAggregation(projects.codyzeBackends.cpg)
jacocoAggregation(projects.codyzeCli)
jacocoAggregation(projects.codyzeCore)
jacocoAggregation(projects.codyzeSpecificationLanguages.coko.cokoCore)
jacocoAggregation(projects.codyzeSpecificationLanguages.coko.cokoDsl)

// Optional and experimental features
if (enablePluginSupport) jacocoAggregation(project(":codyze-plugins"))
}

tasks.check {
Expand Down
3 changes: 3 additions & 0 deletions codecov.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
ignore:
- "**/cli/Main.kt"
- "**/codyze/core/plugin/*.kt"
coverage:
status:
project:
Expand Down
1 change: 1 addition & 0 deletions codyze-cli/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
plugins {
id("documented-module")
id("features")
application
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ package de.fraunhofer.aisec.codyze.cli
import com.github.ajalt.clikt.core.CliktCommand
import com.github.ajalt.clikt.core.NoOpCliktCommand
import com.github.ajalt.clikt.core.context
import com.github.ajalt.clikt.core.findOrSetObject
import com.github.ajalt.clikt.output.MordantHelpFormatter
import com.github.ajalt.clikt.parameters.arguments.argument
import com.github.ajalt.clikt.parameters.arguments.multiple
Expand All @@ -27,6 +28,7 @@ import com.github.ajalt.clikt.parameters.options.option
import com.github.ajalt.clikt.parameters.options.versionOption
import com.github.ajalt.clikt.parameters.types.path
import de.fraunhofer.aisec.codyze.core.VersionProvider
import de.fraunhofer.aisec.codyze.core.executor.ExecutorCommand
import java.nio.file.Path
import kotlin.io.path.Path

Expand Down Expand Up @@ -68,7 +70,11 @@ class ConfigFileParser : CliktCommand(treatUnknownOptionsAsArgs = true) {
*/
@Suppress("Unused", "UnusedPrivateMember")
class CodyzeCli(val configFile: Path?) :
NoOpCliktCommand(help = "Codyze finds security flaws in source code", printHelpOnEmptyArgs = true) {
NoOpCliktCommand(
help = "Codyze finds security flaws in source code",
printHelpOnEmptyArgs = true,
allowMultipleSubcommands = true
) {

init {
versionOption(
Expand All @@ -86,4 +92,11 @@ class CodyzeCli(val configFile: Path?) :
private val unusedConfigFile: Path? by configFileOption()

val codyzeOptions by CodyzeOptionGroup()

val usedExecutors by findOrSetObject { mutableListOf<ExecutorCommand<*>>() }

// This run method is only necessary to correctly set the "usedExecutors" variable
override fun run() {
usedExecutors
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,12 @@ import de.fraunhofer.aisec.codyze.core.executor.Executor
import de.fraunhofer.aisec.codyze.core.executor.ExecutorCommand
import de.fraunhofer.aisec.codyze.core.output.OutputBuilder
import de.fraunhofer.aisec.codyze.core.output.SarifBuilder
import de.fraunhofer.aisec.codyze.core.plugin.Plugin
import de.fraunhofer.aisec.codyze.specificationLanguages.coko.dsl.cli.CokoSubcommand
import org.koin.core.module.dsl.factoryOf
import org.koin.dsl.bind
import org.koin.dsl.module
import java.util.ServiceLoader

/**
* Every [Backend] must provide [BackendCommand] to be selectable in the CLI.
Expand All @@ -49,3 +51,8 @@ val executorCommands = module {
val outputBuilders = module {
factoryOf(::SarifBuilder) bind(OutputBuilder::class)
}

/**
* List all available [Plugin]s. They use external tools to extend the analysis.
*/
val plugins = ServiceLoader.load(Plugin::class.java).map { it.module() }
42 changes: 23 additions & 19 deletions codyze-cli/src/main/kotlin/de/fraunhofer/aisec/codyze/cli/Main.kt
Original file line number Diff line number Diff line change
Expand Up @@ -18,20 +18,19 @@ package de.fraunhofer.aisec.codyze.cli
import com.github.ajalt.clikt.core.subcommands
import de.fraunhofer.aisec.codyze.core.backend.BackendCommand
import de.fraunhofer.aisec.codyze.core.executor.ExecutorCommand
import io.github.oshai.kotlinlogging.KotlinLogging
import de.fraunhofer.aisec.codyze.core.output.aggregator.Aggregate
import de.fraunhofer.aisec.codyze.core.plugin.Plugin
import org.koin.core.context.startKoin
import org.koin.java.KoinJavaComponent.getKoin
import java.nio.file.Path

private val logger = KotlinLogging.logger {}

/** Entry point for Codyze. */
fun main(args: Array<String>) {
startKoin { // Initialize the koin dependency injection
// use Koin logger
printLogger()
// declare modules
modules(executorCommands, backendCommands, outputBuilders)
modules(listOf(executorCommands, backendCommands, outputBuilders) + plugins)
}

// parse the CMD arguments
Expand All @@ -44,26 +43,31 @@ fun main(args: Array<String>) {
} finally {
// parse the arguments based on the codyze options and the executorOptions/backendOptions
codyzeCli = CodyzeCli(configFile = configFile)
codyzeCli.subcommands(getKoin().getAll<ExecutorCommand<*>>())
codyzeCli.subcommands(getKoin().getAll<ExecutorCommand<*>>() + getKoin().getAll<Plugin>())
codyzeCli.main(args)
}

// get the used subcommands
val executorCommand = codyzeCli.currentContext.invokedSubcommand as? ExecutorCommand<*>

// allow backendCommand to be null in order to allow executors that do not use backends
val backendCommand = executorCommand?.currentContext?.invokedSubcommand as? BackendCommand<*>
// Following code will be executed after all commands' "run" functions complete

// this should already be checked by clikt in [codyzeCli.main(args)]
requireNotNull(executorCommand) { "UsageError! Please select one of the available executors." }
require(codyzeCli.usedExecutors.isNotEmpty()) { "UsageError! Please select one of the available executors." }
for (executorCommand in codyzeCli.usedExecutors) {
// allow backendCommand to be null in order to allow executors that do not use backends
val backendCommand = executorCommand.currentContext.invokedSubcommand as? BackendCommand<*>

val codyzeConfiguration = codyzeCli.codyzeOptions.asConfiguration()
// the subcommands know how to instantiate their respective backend/executor
val backend = backendCommand?.getBackend() // [null] if the chosen executor does not support modular backends
val executor = executorCommand.getExecutor(codyzeConfiguration.goodFindings, codyzeConfiguration.pedantic, backend)
val codyzeConfiguration = codyzeCli.codyzeOptions.asConfiguration()

val run = executor.evaluate()
// the subcommands know how to instantiate their respective backend/executor
val backend = backendCommand?.getBackend() // [null] if the chosen executor does not support modular backends
val executor = executorCommand.getExecutor(
codyzeConfiguration.goodFindings,
codyzeConfiguration.pedantic,
backend
)

// use the chosen [OutputBuilder] to convert the SARIF format (a SARIF RUN) from the executor to the chosen format
codyzeConfiguration.outputBuilder.toFile(run, codyzeConfiguration.output)
val run = executor.evaluate()
Aggregate.addRun(run)

// use the chosen OutputBuilder to convert the SARIF format (a SARIF Run) from the executor to the chosen format
codyzeConfiguration.outputBuilder.toFile(Aggregate.createRun() ?: run, codyzeConfiguration.output)
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,8 @@
package de.fraunhofer.aisec.codyze.core.config

import de.fraunhofer.aisec.codyze.core.output.OutputBuilder
import io.github.oshai.kotlinlogging.KotlinLogging
import java.nio.file.Path

private val logger = KotlinLogging.logger {}

/**
* Holds the main configuration to run Codyze with
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,7 @@
*/
package de.fraunhofer.aisec.codyze.core.executor

import com.github.ajalt.clikt.core.NoOpCliktCommand
import com.github.ajalt.clikt.core.subcommands
import com.github.ajalt.clikt.core.*
import de.fraunhofer.aisec.codyze.core.backend.Backend
import de.fraunhofer.aisec.codyze.core.backend.BackendCommand
import de.fraunhofer.aisec.codyze.core.backend.BackendOptions
Expand All @@ -29,6 +28,9 @@ import org.koin.java.KoinJavaComponent.getKoin
abstract class ExecutorCommand<T : Executor>(cliName: String? = null) :
NoOpCliktCommand(hidden = true, name = cliName) {

/** Use the global context set in [CodyzeCli] */
private val usedExecutors by findOrSetObject { mutableListOf<ExecutorCommand<*>>() }

abstract fun getExecutor(goodFindings: Boolean, pedantic: Boolean, backend: Backend?): T

/**
Expand All @@ -39,4 +41,8 @@ abstract class ExecutorCommand<T : Executor>(cliName: String? = null) :
inline fun <reified T> registerBackendOptions() {
subcommands(getKoin().getAll<BackendCommand<*>>().filter { it.backend == T::class })
}

override fun run() {
usedExecutors += this
}
}
Loading