Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(deps): updates tests to CRS v4.5 and Albedo as backend #161

Merged
merged 7 commits into from
Aug 3, 2024

Conversation

jcchavezs
Copy link
Member

No description provided.

ftw/Dockerfile.ftw Outdated Show resolved Hide resolved
jcchavezs and others added 2 commits July 25, 2024 08:42
1.0.x requires CRS 4.5

Co-authored-by: Matteo Pace <pace.matteo96@gmail.com>
fzipi
fzipi previously approved these changes Aug 2, 2024
@fzipi fzipi requested a review from M4tteoP August 2, 2024 17:35
@fzipi fzipi enabled auto-merge (squash) August 2, 2024 17:35
@M4tteoP M4tteoP disabled auto-merge August 3, 2024 10:28
M4tteoP
M4tteoP previously requested changes Aug 3, 2024
Copy link
Member

@M4tteoP M4tteoP left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#152 has been merged updating back go-ftw to v1.0.3.

CRS v4.3 still comes with the old test format expected to be run with go-ftw 0.6.4.

I don't get it from a go-ftw standpoint, but running old test formats with the latest go-ftw just leads to all the tests passed.

Run with go-ftw 0.6.4:

➕ run 3714 total tests in 17.949088553s
⏭  skipped 6 tests
☝  ignored 95 tests
👎 15 test(s) failed to run: ["920201-2" "920202-2" "920290-4" "930110-7" "951220-2" "951230-2" "951240-2" "953101-1" "953101-2" "953101-3" "953101-4" "953101-5" "955100-1" "955100-2" "955100-3"]
Error: failed 15 tests

Run with go-ftw 1.0.3:

➕ run 3904 total tests in 8.533989878s
⏭ skipped 0 tests
☝ ignored 95 tests
🎉 All tests successful!

This PR requires either directly bumping CRS to 4.5 and also updating the backend to Albedo, or stick with the previous go-ftw. I'm working on the first option

Copy link

sonarcloud bot commented Aug 3, 2024

@M4tteoP M4tteoP changed the title chore(deps): upgrades deps. chore(deps): updates tests to CRS v4.5 and Albedo as backend Aug 3, 2024
@M4tteoP
Copy link
Member

M4tteoP commented Aug 3, 2024

Some numbers about the CRS tests ( Caddy v2.8.4, Coraza v3.2.1, CRS v4.5, Albedo v0.0.15):

🚀 53 excluded tests are now working.
😕 13 new failing tests.
⚠️ 60 CRS failing tests are still to be investigated.

Overall we got a good -40 failing tests, but more work is definitely needed to chase down the to-be-investigated ones.

@M4tteoP M4tteoP dismissed their stale review August 3, 2024 11:33

Requested changes have been committed

@M4tteoP M4tteoP requested a review from fzipi August 3, 2024 11:34
Copy link
Member

@fzipi fzipi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work!

@fzipi fzipi merged commit fc7a826 into main Aug 3, 2024
6 checks passed
@fzipi fzipi deleted the upgrades_deps branch August 3, 2024 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants