Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

k6runner: add check metadata and type to remote runner requests #928

Merged
merged 3 commits into from
Oct 22, 2024

Conversation

roobre
Copy link
Collaborator

@roobre roobre commented Oct 7, 2024

Similar approach to #904, but I chose to make a distinction between pure metadata, that is, data that makes it only to the telemetry and does not take part in any decision making, and actual information that is used for decision making. The latter is strongly typed.

@roobre roobre marked this pull request as ready for review October 7, 2024 10:23
@roobre roobre requested a review from a team as a code owner October 7, 2024 10:23
@roobre roobre marked this pull request as draft October 7, 2024 10:57
@roobre roobre marked this pull request as ready for review October 7, 2024 12:20
Comment on lines +44 to +52
// CheckInfo holds information about the SM check that triggered this script.
type CheckInfo struct {
// Type is the string representation of the check type this script belongs to (browser, scripted, multihttp, etc.)
Type string `json:"type"`
// Metadata is a collection of key/value pairs containing information about this check, such as check and tenant ID.
// It is loosely typed on purpose: Metadata should only be used for informational properties that will make its way
// into telemetry, and not for making decision on it.
Metadata map[string]any `json:"metadata"`
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is where I'd like to have more reviewer attention, as these changes are hard to revert.

I chose to separate actionable information (Type), which will be used downstream to make decisions, from non-actionable data that will be basically used to enrich logging. I think this makes the relevant information "strongly typed", if that makes sense. Looking forward to your thoughts on this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Metadata should only be used for informational properties that will make its way into telemetry

That's kind of the reason for using map[string]string instead of map[string]any.

For the purposes of info metrics, you are dealing with strings. Preserving the type doesn't gain you much.

Type is the string representation of the check type this script belongs to (browser, scripted, multihttp, etc.)

The only strong thing about the type here is that it requires it to be a) a field with a specific name; b) a string. Other than that, it's a free-form string, which defeats the strong type argument.

Note that you can unmarshal a map[string]any as anything you want on the other side. What I mean by that is that on the receiving side you can do e.g.

type CheckInfo {
   Type string `json:"type"`
   TenantId int64 `json:"tenantId"`
   RegionId int64 `json:"regionId"`
}

What I mean by this is that having a map on the producer side doesn't prevent you from having a well defined struct on the client side. Without some kind of validation, just having a field is not enough to force users of this code to actually set a type.

Unless you introduce actual coupling between the agent and the runner, which is not going to happen, this is all in internal/, the "strong type" argument is not very strong. I'm not saying it's wrong, just that it's not giving you what you say it gives you.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's kind of the reason for using map[string]string instead of map[string]any.

I'm fine-ish with map[string]string for metadata. I leaned towards any because that will show integers as integers in the runner logs, instead of strings. If at some point we want to correlate logs, I think it would be nice that what is logged as an int in one place is also logged as an int on another.

Unless you introduce actual coupling between the agent and the runner, which is not going to happen, this is all in internal/, the "strong type" argument is not very strong. I'm not saying it's wrong, just that it's not giving you what you say it gives you.

Practically it's giving nothing, but I think the separation conveys meaning to readers. They will notice that particular field is special and think twice before adding or removing it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I'm good with that, I just wanted to make sure the expectations are correct.

Do note that JSON numbers are super weird, because the JS in JSON stands for JavaScript, and JavaScript has no integers, only floats.

JSON mixed with Go is even weirder, because, when dealing with any, Go says "if JSON numbers are floats then I will decode them into float64 because that is what makes sense". You can override that using UseNumber (which requires using a json.Decoder not just json.Unmarshal) plus asking nicely for the specific conversion you want.

Do note this has an implication when marshaling elements from map[string]any into JSON via zerolog: the JSON decoder placed a float64 in the map's value, so logger.Any(...) sees a float, and it will log it out as a float. If you go thru the json.Number route, I think that should do the right thing (because json.Number's underlying type is string 😆).

@roobre roobre force-pushed the metadata branch 2 times, most recently from 2d2d5cb to 05bf1cd Compare October 8, 2024 10:06
internal/k6runner/k6runner.go Outdated Show resolved Hide resolved
Comment on lines +44 to +52
// CheckInfo holds information about the SM check that triggered this script.
type CheckInfo struct {
// Type is the string representation of the check type this script belongs to (browser, scripted, multihttp, etc.)
Type string `json:"type"`
// Metadata is a collection of key/value pairs containing information about this check, such as check and tenant ID.
// It is loosely typed on purpose: Metadata should only be used for informational properties that will make its way
// into telemetry, and not for making decision on it.
Metadata map[string]any `json:"metadata"`
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Metadata should only be used for informational properties that will make its way into telemetry

That's kind of the reason for using map[string]string instead of map[string]any.

For the purposes of info metrics, you are dealing with strings. Preserving the type doesn't gain you much.

Type is the string representation of the check type this script belongs to (browser, scripted, multihttp, etc.)

The only strong thing about the type here is that it requires it to be a) a field with a specific name; b) a string. Other than that, it's a free-form string, which defeats the strong type argument.

Note that you can unmarshal a map[string]any as anything you want on the other side. What I mean by that is that on the receiving side you can do e.g.

type CheckInfo {
   Type string `json:"type"`
   TenantId int64 `json:"tenantId"`
   RegionId int64 `json:"regionId"`
}

What I mean by this is that having a map on the producer side doesn't prevent you from having a well defined struct on the client side. Without some kind of validation, just having a field is not enough to force users of this code to actually set a type.

Unless you introduce actual coupling between the agent and the runner, which is not going to happen, this is all in internal/, the "strong type" argument is not very strong. I'm not saying it's wrong, just that it's not giving you what you say it gives you.

internal/k6runner/k6runner.go Outdated Show resolved Hide resolved
internal/k6runner/k6runner.go Outdated Show resolved Hide resolved
internal/k6runner/k6runner.go Outdated Show resolved Hide resolved
Copy link
Contributor

@mem mem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is good, after adding the region_id

@roobre
Copy link
Collaborator Author

roobre commented Oct 14, 2024

@mem Painstakingly added model.Check everywhere, just want to double-check if this is what you had in mind.

Copy link
Contributor

@mem mem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@roobre roobre merged commit ce37f32 into main Oct 22, 2024
2 checks passed
@roobre roobre deleted the metadata branch October 22, 2024 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants