Add: scap-sync

[maxdollinger]

What

small programm to handle incremental syncs (max 1day) of scap-data to an already setup postgres instance.

Why

For incremental refactor / rebuild of the scap-data flows the current code need's to stay the same.
And the CLI approach is to complex for what is need in this case.

References

DEVOPS-1930

Checklist

• Tests (integration test e2e added)

[greenbonebot]

Scanning the following files:

.github/workflows/ci-python.yml
greenbone/scap_sync/cpe/__init__.py
greenbone/scap_sync/cpe/sync.py
greenbone/scap_sync/cpe/upsert.py
greenbone/scap_sync/cve/__init__.py
greenbone/scap_sync/cve/sync.py
greenbone/scap_sync/cve/upsert.py
greenbone/scap_sync/main.py
greenbone/scap_sync/utils/db.py
greenbone/scap_sync/utils/env.py
greenbone/scap_sync/utils/time.py
poetry.lock
pyproject.toml
tests/cpe/cli/test_download.py
tests/cpe/cli/test_find.py
tests/cpe_match/cli/test_json_download.py
tests/cpe_match/producer/test_db.py
tests/cpe_match/producer/test_nvd_api.py
tests/cpe_match/worker/test_db.py
tests/cpe_match/worker/test_json.py
tests/cve/cli/test_download.py
tests/scap_sync/__init__.py
tests/scap_sync/test_main.py
tests/tags.py
tests/test_timer.py
tests/test_version.py

Scan: '.github/workflows/ci-python.yml'

Nothing detected in .github/workflows/ci-python.yml
Scan took 0.00 seconds

Scan: 'greenbone/scap_sync/cpe/init.py'

Nothing detected in greenbone/scap_sync/cpe/init.py
Scan took 0.00 seconds

Scan: 'greenbone/scap_sync/cpe/sync.py'

Nothing detected in greenbone/scap_sync/cpe/sync.py
Scan took 0.00 seconds

Scan: 'greenbone/scap_sync/cpe/upsert.py'

Nothing detected in greenbone/scap_sync/cpe/upsert.py
Scan took 0.00 seconds

Scan: 'greenbone/scap_sync/cve/init.py'

Nothing detected in greenbone/scap_sync/cve/init.py
Scan took 0.00 seconds

Scan: 'greenbone/scap_sync/cve/sync.py'

Nothing detected in greenbone/scap_sync/cve/sync.py
Scan took 0.00 seconds

Scan: 'greenbone/scap_sync/cve/upsert.py'

Nothing detected in greenbone/scap_sync/cve/upsert.py
Scan took 0.00 seconds

Scan: 'greenbone/scap_sync/main.py'

Nothing detected in greenbone/scap_sync/main.py
Scan took 0.00 seconds

Scan: 'greenbone/scap_sync/utils/db.py'

Nothing detected in greenbone/scap_sync/utils/db.py
Scan took 0.00 seconds

Scan: 'greenbone/scap_sync/utils/env.py'

Nothing detected in greenbone/scap_sync/utils/env.py
Scan took 0.00 seconds

Scan: 'greenbone/scap_sync/utils/time.py'

Nothing detected in greenbone/scap_sync/utils/time.py
Scan took 0.00 seconds

Scan: 'poetry.lock'

Nothing detected in poetry.lock
Scan took 0.02 seconds

Scan: 'pyproject.toml'

Nothing detected in pyproject.toml
Scan took 0.00 seconds

Scan: 'tests/cpe/cli/test_download.py'

Nothing detected in tests/cpe/cli/test_download.py
Scan took 0.00 seconds

Scan: 'tests/cpe/cli/test_find.py'

Nothing detected in tests/cpe/cli/test_find.py
Scan took 0.00 seconds

Scan: 'tests/cpe_match/cli/test_json_download.py'

Nothing detected in tests/cpe_match/cli/test_json_download.py
Scan took 0.00 seconds

Scan: 'tests/cpe_match/producer/test_db.py'

Nothing detected in tests/cpe_match/producer/test_db.py
Scan took 0.00 seconds

Scan: 'tests/cpe_match/producer/test_nvd_api.py'

Nothing detected in tests/cpe_match/producer/test_nvd_api.py
Scan took 0.00 seconds

Scan: 'tests/cpe_match/worker/test_db.py'

Nothing detected in tests/cpe_match/worker/test_db.py
Scan took 0.00 seconds

Scan: 'tests/cpe_match/worker/test_json.py'

Nothing detected in tests/cpe_match/worker/test_json.py
Scan took 0.00 seconds

Scan: 'tests/cve/cli/test_download.py'

Nothing detected in tests/cve/cli/test_download.py
Scan took 0.00 seconds

Scan: 'tests/scap_sync/init.py'

Nothing detected in tests/scap_sync/init.py
Scan took 0.00 seconds

Scan: 'tests/scap_sync/test_main.py'

Nothing detected in tests/scap_sync/test_main.py
Scan took 0.00 seconds

Scan: 'tests/tags.py'

Nothing detected in tests/tags.py
Scan took 0.00 seconds

Scan: 'tests/test_timer.py'

Nothing detected in tests/test_timer.py
Scan took 0.00 seconds

Scan: 'tests/test_version.py'

Nothing detected in tests/test_version.py
Scan took 0.00 seconds

[github-actions]

Conventional Commits Report

Type	Number
Added	9
Bug Fixes	2

🚀 Conventional commits found.

[codecov]

Codecov Report

❌ Patch coverage is 79.35943% with 58 lines in your changes missing coverage. Please review.
✅ Project coverage is 62.89%. Comparing base (ccd55fa) to head (1dc985b).

Files with missing lines	Patch %	Lines
greenbone/scap_sync/cve/upsert.py	75.63%	16 Missing and 13 partials ⚠️
greenbone/scap_sync/cpe/upsert.py	81.13%	5 Missing and 5 partials ⚠️
greenbone/scap_sync/utils/db.py	71.42%	8 Missing ⚠️
greenbone/scap_sync/main.py	75.00%	5 Missing and 1 partial ⚠️
greenbone/scap_sync/utils/env.py	80.76%	3 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #135      +/-   ##
==========================================
+ Coverage   60.53%   62.89%   +2.36%     
==========================================
  Files          31       39       +8     
  Lines        2002     2283     +281     
  Branches      208      244      +36     
==========================================
+ Hits         1212     1436     +224     
- Misses        762      798      +36     
- Partials       28       49      +21     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.