Possible race condition when reading Java DB files after download

[JanMosigItemis]

Description

The other day, we encountered Trivy crashes when scanning our Java based service images. The scan fails because of problems finding downloaded Java DB files. The problem is not 100% reproducible but appears on a regular basis.

As a workaround, we are now downloading the DBs separately before initiating the scan. This seems to work 100% of the time.

Desired Behavior

Trivy scan is successful.

Actual Behavior

The scan fails with one of two kinds of errors: Either open failed or chmod failed. Both because a file was missing. The missing file is not always the same.

Examples:

trivy image --cache-dir "${CI_PROJECT_DIR}/.cache/trivy" --no-progress --timeout 300s --format template --template mytemplate.tpl -o report.file --ignorefile ".trivyignore" --exit-code 1 myimage:version
2023-10-23T09:05:26.177Z	INFO	Need to update DB
2023-10-23T09:05:26.177Z	INFO	DB Repository: ghcr.io/aquasecurity/trivy-db
2023-10-23T09:05:26.177Z	INFO	Downloading DB...
2023-10-23T09:05:28.909Z	INFO	Vulnerability scanning is enabled
2023-10-23T09:05:28.909Z	INFO	Secret scanning is enabled
2023-10-23T09:05:28.909Z	INFO	If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2023-10-23T09:05:28.909Z	INFO	Please see also https://aquasecurity.github.io/trivy/v0.46/docs/scanner/secret/#recommendation for faster secret detection
2023-10-23T09:05:38.945Z	INFO	JAR files found
2023-10-23T09:05:38.945Z	INFO	Java DB Repository: ghcr.io/aquasecurity/trivy-java-db:1
2023-10-23T09:05:38.945Z	INFO	Downloading the Java DB...
2023-10-23T09:05:39.869Z	INFO	JAR files found
2023-10-23T09:05:39.869Z	INFO	Java DB Repository: ghcr.io/aquasecurity/trivy-java-db:1
2023-10-23T09:05:39.869Z	INFO	Downloading the Java DB...
2023-10-23T09:06:00.941Z	INFO	JAR files found
2023-10-23T09:06:00.942Z	INFO	Java DB Repository: ghcr.io/aquasecurity/trivy-java-db:1
2023-10-23T09:06:00.942Z	INFO	Downloading the Java DB...
2023-10-23T09:06:08.237Z	INFO	The Java DB is cached for 3 days. If you want to update the database more frequently, the '--reset' flag clears the DB cache.
2023-10-23T09:06:08.238Z	INFO	Analyzing JAR files takes a while...
2023-10-23T09:06:14.953Z	INFO	The Java DB is cached for 3 days. If you want to update the database more frequently, the '--reset' flag clears the DB cache.
2023-10-23T09:06:14.953Z	INFO	Analyzing JAR files takes a while...
2023-10-23T09:06:14.956Z	FATAL	image scan error: scan error: scan failed: failed analysis: analyze error: pipeline error: failed to analyze layer (sha256:mysha): post analysis error: post analysis error: Unable to initialize the Java DB: Java DB update failed: Java DB update error: Java DB metadata error: unable to open a file: open /builds/myproject/.cache/trivy/java-db/metadata.json: no such file or directory

chmod is exactly the same with slightly different error

2023-10-23T09:26:36.008Z	FATAL	image scan error: scan error: scan failed: failed analysis: analyze error: pipeline error: failed to analyze layer (sha256:mysha): post analysis error: post analysis error: Unable to initialize the Java DB: Java DB update failed: Java DB update error: DB download error: oci download error: download error: failed to download: chmod /builds/myproject/.cache/trivy/java-db/trivy-java.db: no such file or directory

Reproduction Steps

Not exactly sure.

We are running Trivy in a AWS EKS based environment like this:

trivy image --cache-dir "${CI_PROJECT_DIR}/.cache/trivy" --no-progress --timeout 300s --format template --template mytemplate.tpl -o report.file --ignorefile ".trivyignore" --exit-code 1 myimage:version

Target

Container Image

Scanner

Vulnerability

Output Format

Template

Mode

Standalone

Debug Output

Could not be saved or reproduced. However, the one debug log we saw did not contain any errors except for the ones already posted.

Operating System

Linux

Version

A docker image with 0.46.0 at the time of writing.

Checklist

• Run trivy image --reset
• Read the troubleshooting

[knqyf263]

You can track the problem here.
#5441