[Enhancement] Easy checking of CLI output

[shubham-cmyk]

What this PR does / why we need it:

This PR would add an declarative way to check the output of the structs.

This types of command could be used now :

apiVersion: kuttl.dev/v1beta1
kind: TestStep
commands:
- command: echo "Hello World"
  output:
    stdout:
      contains: "Hello World"
    stderr:
      contains: "Hello World"

Fixes #22

[eddycharly]

@shubham-cmyk you have linter issues

[shubham-cmyk]

@eddycharly

I am trying to fix this but this is not fixing :

gofmt -w /pkg/test/utils/kubernetes.go

[eddycharly]

How about something like:

apiVersion: kuttl.dev/v1beta1
kind: TestStep
commands:
- command: echo "Hello World"
  output:
    stdout:
      expected: "Hello ?*"
      match: Wildcard
    stderr:
      expected: "Hello World"
      match: Equals

[eddycharly]

Looks like it would be more extensible 🤷

[shubham-cmyk]

apiVersion: kuttl.dev/v1beta1
kind: TestStep
commands:
- command: echo "Hello World"
  output:
    stdout:
      expected: "Hello ?*"
      match: Wildcard
    stderr:
      expected: "Hello World"
      match: Equals

Okay, the Match could be Equal ( strict ) and contains ( just string contains check ) a wildcard (wildcard check could be passed) .....

Or we should keep match to equals and wildcard

[eddycharly]

I think Equals and Wildcard are enough to cover 99% cases.
We might need Contains in very specific situations though and it shouldn't rely on wildcards (when the expected output contains * or ? but we don't want to treat them as wildcard patterns...)

[eddycharly]

Something like this will not work well:

apiVersion: kuttl.dev/v1beta1
kind: TestStep
commands:
- command: echo "Hello World"
  output:
    stdout:
      expected: "How are you today ?"
      match: Wildcard

[shubham-cmyk]

So let's push contains it would be good 👍

[eddycharly]

So let's push contains it would be good 👍

Ok, fine.

[chipzoller]

Any schema changes just be sure that the initial post in this PR has a full and complete description of what's available.

[shubham-cmyk]

These below structs are added. ( edited )

type CommandOutput struct {
	Stdout ExpectedOutput `json:"stdout"`
	Stderr ExpectedOutput `json:"stderr"`
}

type ExpectedOutput struct {
	MatchType string `json:"match"`
	ExpectedValue string `json:"expected"`
}

Now we could have something like :

For only wild-card match.

apiVersion: kuttl.dev/v1beta1
kind: TestStep
commands:
- command: echo "Hello World"
  output:
    stdout:
      match: wildcard
      expected:  " any pattern "

Strictly the stdout should be equal what I define.

apiVersion: kuttl.dev/v1beta1
kind: TestStep
commands:
- command: echo "Hello World"
  output:
    stdout:
      match: equals
      expected: "Hello World"

Check whether the the stdout contains the "hello"

apiVersion: kuttl.dev/v1beta1
kind: TestStep
commands:
- command: echo "Hello World"
  output:
    stdout:
      match: contains
      expected:  "Hello"

[eddycharly]

@shubham-cmyk the changes here do not reflect this:

  output:
    stdout:
      expected: "How are you today ?"
      match: Wildcard

[shubham-cmyk]

@eddycharly

[eddycharly]

It would be nice to cover this with tests too 🙏

[shubham-cmyk]

It would be nice to cover this with tests too 🙏

Added unit_test to cover this function.

[eddycharly]

@shubham-cmyk please raise an error if background is true and output checks are present

[shubham-cmyk]

@shubham-cmyk please raise an error if background is true and output checks are present

Done