Could not connect to Central search (403: Forbidden) when analyzing a .jar file

[Pho3njx]

Hello Dear DependencyCheck team,

I face an issue while executing ODC and analyse a .jar file. The connection to the Maven Central (https://search.maven.org/solrsearch/select does suddenly not work anymore.
The issue is similar to #6994, but it's a different http error. So, I believe that's not a technical issue on maven.org side.

Could it be that Central blocks ODC , because it made too many requests? Have you faced such issue before too and have an idea how I could fix it? => see FAQ here.

In comparison to the ODC CLI execution, I never run into such issue by executing ODC over Maven (mvn ${MVN_CLI_OPTS} -s ${MVN_SETTINGS} dependency-check:check [...] ) in the same project. 🤔
I also couldn't find any rate limits for Central (as we have for NVD e.g.)

Environment (where the issue happens):

• GitLab - pipeline
• I use the GitLab docker-dependency-check with pre-seeded NVD database
• the docker image uses ODC version 11.1.1
• I did not --disableCentralCache, see execution command in 1st line of Error-Log below
• only 1 .jar file is in the target/ folder that will be analyzed by ODC

Error-Log:

$ /usr/share/dependency-check/bin/dependency-check.sh --format ALL --scan ./target/*.jar --project $PROJECT_NAME --nvdApiKey $NVD_API_KEY

[INFO] Checking for updates
[INFO] Skipping the NVD API Update as it was completed within the last 240 minutes
[INFO] Skipping Known Exploited Vulnerabilities update check since last check was within 24 hours.
[INFO] Check for updates complete (830 ms)
[INFO] 
Dependency-Check is an open source tool performing a best effort analysis of 3rd party dependencies; false positives and false negatives may exist in the analysis performed by the tool. Use of the tool and the reporting provided constitutes acceptance for use in an AS IS condition, and there are NO warranties, implied or otherwise, with regard to the analysis or its use. Any use of the tool and the reporting provided is at the user's risk. In no event shall the copyright holder or OWASP be held liable for any damages whatsoever arising out of or in connection with the use of this tool, the analysis performed, or the resulting report.
   About ODC: https://jeremylong.github.io/DependencyCheck/general/internals.html
   False Positives: https://jeremylong.github.io/DependencyCheck/general/suppression.html
💖 Sponsor: https://github.com/sponsors/jeremylong
[INFO] Analysis Started
[INFO] Finished Archive Analyzer (1 seconds)
[INFO] Finished File Name Analyzer (0 seconds)
[INFO] Finished Jar Analyzer (0 seconds)
[ERROR] Could not connect to Central search. Analysis failed.
java.io.IOException: Finally failed connecting to Central search. Giving up after 7 tries.
	at org.owasp.dependencycheck.analyzer.CentralAnalyzer.fetchMavenArtifacts(CentralAnalyzer.java:362)
	at org.owasp.dependencycheck.analyzer.CentralAnalyzer.analyzeDependency(CentralAnalyzer.java:228)
	at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze(AbstractAnalyzer.java:131)
	at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:88)
	at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:37)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.io.IOException: https://search.maven.org/solrsearch/select?q=1:0e02407d19971bfa241441212901dd327a37722b&wt=xml - Server status: 403 - Server reason: Forbidden
	at org.owasp.dependencycheck.utils.Downloader.fetchAndHandle(Downloader.java:613)
	at org.owasp.dependencycheck.data.central.CentralSearch.searchSha1(CentralSearch.java:179)
	at org.owasp.dependencycheck.analyzer.CentralAnalyzer.fetchMavenArtifacts(CentralAnalyzer.java:339)
	... 8 common frames omitted
[WARN] An error occurred while analyzing '/tmp/dctempdde304bf-3670-4fb7-bcde-c6002b916baf/check17303026260557696910tmp/1/BOOT-INF/lib/java-jwt-4.4.0.jar' (Central Analyzer).

[...]

Thank you so much for your help!
Best Regards!

[jeremylong]

I would try again today. We have seen rare issues with the availability of Central.

[Pho3njx]

Hi @jeremylong,
many thanks for your reply.
I'll keep an eye on the behavior the next days. I see search.maven is down currently... ⏳

[Pho3njx]

While Maven Search is still down, I question myself if it would be an idea to change ODC from requesting search.maven.org to central.sonatype.com. I made a short test by setting --centralUrl https://central.sonatype.com/solrsearch/select and it seems to work.

Why changing the URL:

• central.sonatype.com seems to be much more stable than search.maven.org (by comparing the stats).
• when I open search.maven.org in the browser I'm forwarded to the central.sonatype.com anyway
• identical requests seem to work on central.sonatype.com too (e.g.: ODC Maven-Search request <>
  Central Search)

My logs from successful job:

$ /usr/share/dependency-check/bin/dependency-check.sh --format ALL --scan ./target/*.jar --project $MY_PROJECT --nvdApiKey $NVD_API_KEY --centralUrl https://central.sonatype.com/solrsearch/select
[INFO] Checking for updates
[INFO] Skipping the NVD API Update as it was completed within the last 240 minutes
[INFO] Skipping Known Exploited Vulnerabilities update check since last check was within 24 hours.
[INFO] Check for updates complete (929 ms)
[INFO] 
Dependency-Check is an open source tool performing a best effort analysis of 3rd party dependencies; false positives and false negatives may exist in the analysis performed by the tool. Use of the tool and the reporting provided constitutes acceptance for use in an AS IS condition, and there are NO warranties, implied or otherwise, with regard to the analysis or its use. Any use of the tool and the reporting provided is at the user's risk. In no event shall the copyright holder or OWASP be held liable for any damages whatsoever arising out of or in connection with the use of this tool, the analysis performed, or the resulting report.
   About ODC: https://jeremylong.github.io/DependencyCheck/general/internals.html
   False Positives: https://jeremylong.github.io/DependencyCheck/general/suppression.html
💖 Sponsor: https://github.com/sponsors/jeremylong
[INFO] Analysis Started
[INFO] Finished Archive Analyzer (1 seconds)
[INFO] Finished File Name Analyzer (0 seconds)
[INFO] Finished Jar Analyzer (0 seconds)
[INFO] Finished Central Analyzer (6 seconds)
[INFO] Finished Dependency Merging Analyzer (0 seconds)
[INFO] Finished Hint Analyzer (0 seconds)
[INFO] Finished Version Filter Analyzer (0 seconds)
Dec 10, 2024 11:[45](https://gitlab....):22 AM org.apache.lucene.util.HotspotVMOptions <clinit>
WARNING: Lucene cannot optimize algorithms or calculate object sizes for JVMs that are not based on Hotspot or a compatible implementation.
WARNING: A restricted method in java.lang.foreign.Linker has been called
WARNING: java.lang.foreign.Linker::downcallHandle has been called by the unnamed module
WARNING: Use --enable-native-access=ALL-UNNAMED to avoid a warning for this module
Dec 10, 2024 11:45:22 AM org.apache.lucene.store.MemorySegmentIndexInputProvider <init>
INFO: Using MemorySegmentIndexInput and native madvise support with Java 21 or later; to disable start with -Dorg.apache.lucene.store.MMapDirectory.enableMemorySegments=false
Dec 10, 2024 11:45:22 AM org.apache.lucene.internal.vectorization.VectorizationProvider lookup
WARNING: Java runtime is not using Hotspot VM; Java vector incubator API can't be enabled.
[INFO] Created CPE Index (2 seconds)
[INFO] Finished CPE Analyzer (4 seconds)
[INFO] Finished False Positive Analyzer (0 seconds)
[INFO] Finished NVD CVE Analyzer (0 seconds)
[INFO] Finished RetireJS Analyzer (4 seconds)
[INFO] Finished Sonatype OSS Index Analyzer (0 seconds)
[INFO] Finished Vulnerability Suppression Analyzer (0 seconds)
[INFO] Finished Known Exploited Vulnerability Analyzer (0 seconds)
[INFO] Finished Dependency Bundling Analyzer (0 seconds)
[INFO] Finished Unused Suppression Rule Analyzer (0 seconds)
[INFO] Analysis Complete (19 seconds)
[INFO] Writing XML report to: ./dependency-check-report.xml
[INFO] Writing HTML report to: ./dependency-check-report.html
[...]

[jeremylong]

I can guarantee that --centralUrl https://central.sonatype.com/solrsearch/select does not produce accurate results with the current version of ODC. It may not throw an exception, but you will get FP/FN on Java dependencies.

[jeremylong]

Using --centralUrl https://central.sonatype.com/solrsearch/select current is equivelent to disabling the central analyzer.

[weihan1394]

@jeremylong thanks for the sharing. is there any alternative that we can use since the legacy link is not stable?

[weihan1394]

@jeremylong want to follow up with you, is there any vulnerabilities that u found that contains the FP/FN on java dependencies? I saw from maven website that seems like they are planning to retire the legacy search.maven.org.

Based on: https://central.sonatype.org/faq/what-happened-to-search-maven-org/#is-it-possible-to-continue-using-searchmavenorg

Thanks for the help

[Pho3njx]

@jeremylong: Thanks a lot for you analysis!
I also can confirm that I have many FP/FN when using the other Central API URL.
But I had no http response errors from Central the whole time 😅

So, let's get back to the actual problem.
I switched to the default URL today and I had some successful ODC scans. But later I ran again into the described issue and ODC fails.

Does someone have another idea how to fix this error and make ODC work?

$ /usr/share/dependency-check/bin/dependency-check.sh --format ALL --scan ./target/*.jar --project $MY_PROJECT --nvdApiKey $NVD_API_KEY

[...]

[INFO] Analysis Started
[INFO] Finished Archive Analyzer (1 seconds)
[INFO] Finished File Name Analyzer (0 seconds)
[INFO] Finished Jar Analyzer (0 seconds)
[ERROR] Could not connect to Central search. Analysis failed.
java.io.IOException: Finally failed connecting to Central search. Giving up after 7 tries.
	at org.owasp.dependencycheck.analyzer.CentralAnalyzer.fetchMavenArtifacts(CentralAnalyzer.java:362)
	at org.owasp.dependencycheck.analyzer.CentralAnalyzer.analyzeDependency(CentralAnalyzer.java:228)
	at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze(AbstractAnalyzer.java:131)
	at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:88)
	at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:37)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.io.IOException: https://search.maven.org/solrsearch/select?q=1:0e02407d19971bfa241441212901dd327a37722b&wt=xml - Server status: 403 - Server reason: Forbidden
	at org.owasp.dependencycheck.utils.Downloader.fetchAndHandle(Downloader.java:613)
	at org.owasp.dependencycheck.data.central.CentralSearch.searchSha1(CentralSearch.java:179)
	at org.owasp.dependencycheck.analyzer.CentralAnalyzer.fetchMavenArtifacts(CentralAnalyzer.java:339)
	... 8 common frames omitted
[WARN] An error occurred while analyzing '/tmp/dctemp4c2ae483-5a52-48f0-b7c4-107c1b4706d2/check15565132295481931586tmp/1/BOOT-INF/lib/java-jwt-4.4.0.jar' (Central Analyzer).

[...]

[INFO] Finished CPE Analyzer (7 seconds)
[INFO] Finished False Positive Analyzer (0 seconds)
[INFO] Finished NVD CVE Analyzer (0 seconds)
[INFO] Finished RetireJS Analyzer (4 seconds)
[INFO] Finished Sonatype OSS Index Analyzer (4 seconds)
[INFO] Finished Vulnerability Suppression Analyzer (0 seconds)
[INFO] Finished Known Exploited Vulnerability Analyzer (0 seconds)
[INFO] Finished Dependency Bundling Analyzer (0 seconds)
[INFO] Finished Unused Suppression Rule Analyzer (0 seconds)
[INFO] Analysis Complete (1833 seconds)
[INFO] Writing XML report to: ./dependency-check-report.xml
[INFO] Writing HTML report to: ./dependency-check-report.html
[INFO] Writing JSON report to: ./dependency-check-report.json
[INFO] Writing CSV report to: ./dependency-check-report.csv
[INFO] Writing SARIF report to: ./dependency-check-report.sarif
[INFO] Writing JENKINS report to: ./dependency-check-jenkins.html
[INFO] Writing JUNIT report to: ./dependency-check-junit.xml
[INFO] Writing GITLAB report to: ./dependency-check-gitlab.json
[ERROR] Could not connect to Central search. Analysis failed.
[ERROR] Could not connect to Central search. Analysis failed.

[...]

[ERROR] Could not connect to Central search. Analysis failed.
Uploading artifacts for failed job
00:02
Uploading artifacts...
dependency-check-report.html: found 1 matching artifact files and directories 
dependency-check-report.json: found 1 matching artifact files and directories 
Uploading artifacts as "archive" to coordinator... 201 Created  id=208929888 responseStatus=201 Created token=abc-64
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: command terminated with exit code 14