Skip to content

Commit

Permalink
feat: Implements Go e2e tests (#802)
Browse files Browse the repository at this point in the history
* adds health checks, CRS e2e tests

* adds config check logic and comments around configs

* NULLED_BODY, removed references to Envoy, minor optimizations

* moves parameters to flags

* adds e2e test run, refactors e2e folder

* chore: adds improvements to e2e runner.

* fix calling e2e tests

* httpbin attempt

* feat: uses a mux to emulate a proxy.

* removes CRS and recommeneded dependencies from e2e test

* nits around e2e expected directives and comments

* tests: adds content length check.

* adds e2e to readme, fix application/x-www-form-urlencoded tests, adds comment about this needed content-type

---------

Co-authored-by: José Carlos Chávez <jcchavezs@gmail.com>
  • Loading branch information
M4tteoP and jcchavezs committed Jun 19, 2023
1 parent 5ffc3a3 commit 1b8a007
Show file tree
Hide file tree
Showing 10 changed files with 463 additions and 3 deletions.
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,20 @@ the operator with `plugins.RegisterOperator` to reduce binary size / startup ove
* `coraza.rule.multiphase_valuation` - enables evaluation of rule variables in the phases that they are ready, not
only the phase the rule is defined for.

## E2E Testing

[`Http/e2e/`](./http/e2e) provides an utility to run e2e tests.
It can be used standalone against your own waf deployment:
```shell
go run github.com/corazawaf/coraza/http/e2e@main --proxy-hostport localhost:8080 --httpbin-hostport localhost:8081
```
or as a library by importing:
```go
"github.com/corazawaf/coraza/v3/http/e2e/pkg"
```
As a reference for library usage, see [`testing/e2e/e2e_test.go`](.testing/e2e/e2e_test.go).
Expected directives that have to be loaded and available flags can be found in [`http/e2e/main.go`](./examples/http/e2e/main.go).

## Tools

* [Go FTW](https://github.com/coreruleset/go-ftw): Rule testing engine
Expand Down
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ require (
github.com/corazawaf/libinjection-go v0.1.2
github.com/foxcpp/go-mockdns v1.0.0
github.com/magefile/mage v1.15.0
github.com/mccutchen/go-httpbin/v2 v2.8.0
github.com/petar-dambovaliev/aho-corasick v0.0.0-20211021192214-5ab2d9280aa9
github.com/tidwall/gjson v1.14.4
golang.org/x/net v0.11.0
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ github.com/foxcpp/go-mockdns v1.0.0 h1:7jBqxd3WDWwi/6WhDvacvH1XsN3rOLXyHM1uhvIx6
github.com/foxcpp/go-mockdns v1.0.0/go.mod h1:lgRN6+KxQBawyIghpnl5CezHFGS9VLzvtVlwxvzXTQ4=
github.com/magefile/mage v1.15.0 h1:BvGheCMAsG3bWUDbZ8AyXXpCNwU9u5CB6sM+HNb9HYg=
github.com/magefile/mage v1.15.0/go.mod h1:z5UZb/iS3GoOSn0JgWuiw7dxlurVYTu+/jHXqQg881A=
github.com/mccutchen/go-httpbin/v2 v2.8.0 h1:5Ld/mII532zWV6Dn9UpOUZebfD8chkYmx3UIW3VZET8=
github.com/mccutchen/go-httpbin/v2 v2.8.0/go.mod h1:+DBHcmg6EOeoizuiOI8iL12VIHXx+9YQNlz+gjB9uxk=
github.com/miekg/dns v1.1.25/go.mod h1:bPDLeHnStXmXAq1m/Ch/hvfNHr14JKNPMBo3VZKjuso=
github.com/miekg/dns v1.1.50 h1:DQUfb9uc6smULcREF09Uc+/Gd46YWqJd5DbpPE9xkcA=
github.com/miekg/dns v1.1.50/go.mod h1:e3IlAVfNqAllflbibAZEWOXOQ+Ynzk/dDozDxY7XnME=
Expand Down
3 changes: 3 additions & 0 deletions go.work.sum
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,7 @@ github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f h1:KUppIJq7/+
github.com/npillmayer/nestext v0.1.3 h1:2dkbzJ5xMcyJW5b8wwrX+nnRNvf/Nn1KwGhIauGyE2E=
github.com/oklog/run v1.0.0 h1:Ru7dDtJNOyC66gQ5dQmaCa0qIsAUFY3sFpK1Xk8igrw=
github.com/pascaldekloe/goe v0.1.0 h1:cBOtyMzM9HTpWjXfbbunk26uA6nG3a8n06Wieeh0MwY=
github.com/pelletier/go-toml v1.9.1/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c=
github.com/pierrec/lz4 v2.0.5+incompatible h1:2xWsjqPFWcplujydGg4WmhC/6fZqK42wMM8aXeqhl0I=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/posener/complete v1.2.3 h1:NP0eAhjcjImqslEwo/1hq7gpajME0fTLTezBKDqfXqo=
Expand All @@ -111,7 +112,9 @@ github.com/ryanuber/columnize v2.1.0+incompatible h1:j1Wcmh8OrK4Q7GXY+V7SVSY8nUW
github.com/ryanuber/go-glob v1.0.0 h1:iQh3xXAumdQ+4Ufa5b25cRpC5TYKlno6hsv6Cb3pkBk=
github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 h1:nn5Wsu0esKSJiIVhscUtVbo7ada43DJhG55ua/hjS5I=
github.com/sirupsen/logrus v1.6.0 h1:UBcNElsrwanuuMsnGSlYmtmgbb23qDR5dG+6X6Oo89I=
github.com/spf13/cobra v1.6.1/go.mod h1:IOw/AERYS7UzyrGinqmz6HLUo219MORXGxhbaJUqzrY=
github.com/stretchr/objx v0.1.1 h1:2vfRuCMp5sSVIDSqO8oNnWJq7mPa6KVP3iPIwFBuy8A=
github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/yuin/goldmark v1.4.13 h1:fVcFKWvrslecOb/tg+Cc05dkeYx540o0FuFt3nUVDoE=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
go.etcd.io/etcd/api/v3 v3.5.4 h1:OHVyt3TopwtUQ2GKdd5wu3PmmipR4FTwCqoEjSyRdIc=
Expand Down
57 changes: 57 additions & 0 deletions http/e2e/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
// Copyright 2023 Juan Pablo Tosso and the OWASP Coraza contributors
// SPDX-License-Identifier: Apache-2.0

package main

import (
"flag"
"fmt"
"os"

e2e "github.com/corazawaf/coraza/v3/http/e2e/pkg"
)

// Flags:
// --nulled-body: Interruptions at response body phase are allowed to return 200 (Instead of 403), but with a body full of null bytes. Defaults to "false".
// --proxy-hostport: Proxy endpoint used to perform requests. Defaults to "localhost:8080".
// --httpbin-hostport: Upstream httpbin endpoint, used for health checking reasons. Defaults to "localhost:8081".

// Expected Coraza configs:
/*
SecRuleEngine On
SecRequestBodyAccess On
SecResponseBodyAccess On
SecResponseBodyMimeType application/json
# Custom rule for Coraza config check (ensuring that these configs are used)
SecRule &REQUEST_HEADERS:coraza-e2e "@eq 0" "id:100,phase:1,deny,status:424,log,msg:'Coraza E2E - Missing header'"
# Custom rules for e2e testing
SecRule REQUEST_URI "@streq /admin" "id:101,phase:1,t:lowercase,log,deny"
SecRule REQUEST_BODY "@rx maliciouspayload" "id:102,phase:2,t:lowercase,log,deny"
SecRule RESPONSE_HEADERS:pass "@rx leak" "id:103,phase:3,t:lowercase,log,deny"
SecRule RESPONSE_BODY "@contains responsebodycode" "id:104,phase:4,t:lowercase,log,deny"
# Custom rules mimicking the following CRS rules: 941100, 942100, 913100
SecRule ARGS_NAMES|ARGS "@detectXSS" "id:9411,phase:2,t:none,t:utf8toUnicode,t:urlDecodeUni,t:htmlEntityDecode,t:jsDecode,t:cssDecode,t:removeNulls,log,deny"
SecRule ARGS_NAMES|ARGS "@detectSQLi" "id:9421,phase:2,t:none,t:utf8toUnicode,t:urlDecodeUni,t:removeNulls,multiMatch,log,deny"
SecRule REQUEST_HEADERS:User-Agent "@pm grabber masscan" "id:9131,phase:1,t:none,log,deny"
*/

func main() {
// Initialize variables
var (
nulledBody = flag.Bool("nulled-body", false, "Accept a body filled of empty bytes as an enforced disruptive action. Default: false")
proxyHostport = flag.String("proxy-hostport", "localhost:8080", "Configures the URL in which the proxy is running. Default: \"localhost:8080\"")
httpbinHostport = flag.String("httpbin-hostport", "localhost:8081", "Configures the URL in which httpbin is running. Default: \"localhost:8081\"")
)
flag.Parse()

err := e2e.Run(e2e.Config{
NulledBody: *nulledBody,
ProxiedEntrypoint: *proxyHostport,
HttpbinEntrypoint: *httpbinHostport,
})

if err != nil {
fmt.Printf("[Fail] %s\n", err)
os.Exit(1)
}
}
Loading

0 comments on commit 1b8a007

Please sign in to comment.