Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NETOBSERV-1217: Added flag to enable multiCluster configuration #482

Merged
merged 1 commit into from
Nov 29, 2023
Merged

NETOBSERV-1217: Added flag to enable multiCluster configuration #482

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions api/v1alpha1/flowcollector_webhook.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,13 @@ func (r *FlowCollector) ConvertTo(dstRaw conversion.Hub) error {
if restored.Spec.Processor.Metrics.DisableAlerts != nil {
dst.Spec.Processor.Metrics.DisableAlerts = restored.Spec.Processor.Metrics.DisableAlerts
}
if restored.Spec.Processor.ClusterName != "" {
dst.Spec.Processor.ClusterName = restored.Spec.Processor.ClusterName
}
if restored.Spec.Processor.MultiClusterDeployment != nil {
dst.Spec.Processor.MultiClusterDeployment = restored.Spec.Processor.MultiClusterDeployment
}

dst.Spec.Processor.Metrics.Server.TLS.InsecureSkipVerify = restored.Spec.Processor.Metrics.Server.TLS.InsecureSkipVerify
dst.Spec.Processor.Metrics.Server.TLS.ProvidedCaFile = restored.Spec.Processor.Metrics.Server.TLS.ProvidedCaFile

Expand Down
1 change: 1 addition & 0 deletions api/v1alpha1/zz_generated.conversion.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions api/v1beta1/flowcollector_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -480,6 +480,10 @@ type FlowCollectorFLP struct {
// `clusterName` is the name of the cluster to appear in the flows data. This is useful in a multi-cluster context. When using OpenShift, leave empty to make it automatically determined.
ClusterName string `json:"clusterName,omitempty"`

//+kubebuilder:default:=false
// Set `multiClusterDeployment` to `true` to enable multi clusters feature. This will add clusterName label to flows data
MultiClusterDeployment *bool `json:"multiClusterDeployment,omitempty"`

// `debug` allows setting some aspects of the internal configuration of the flow processor.
// This section is aimed exclusively for debugging and fine-grained performance optimizations,
// such as `GOGC` and `GOMAXPROCS` env vars. Users setting its values do it at their own risk.
Expand Down
2 changes: 2 additions & 0 deletions api/v1beta1/zz_generated.conversion.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions api/v1beta1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions api/v1beta2/flowcollector_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -472,6 +472,10 @@ type FlowCollectorFLP struct {
// `clusterName` is the name of the cluster to appear in the flows data. This is useful in a multi-cluster context. When using OpenShift, leave empty to make it automatically determined.
ClusterName string `json:"clusterName,omitempty"`

//+kubebuilder:default:=false
// Set `multiClusterDeployment` to `true` to enable multi clusters feature. This will add clusterName label to flows data
MultiClusterDeployment *bool `json:"multiClusterDeployment,omitempty"`

// `debug` allows setting some aspects of the internal configuration of the flow processor.
// This section is aimed exclusively for debugging and fine-grained performance optimizations,
// such as `GOGC` and `GOMAXPROCS` env vars. Users setting its values do it at their own risk.
Expand Down
5 changes: 5 additions & 0 deletions api/v1beta2/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 12 additions & 0 deletions bundle/manifests/flows.netobserv.io_flowcollectors.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4941,6 +4941,12 @@ spec:
type: object
type: object
type: object
multiClusterDeployment:
default: false
description: Set `multiClusterDeployment` to `true` to enable
multi clusters feature. This will add clusterName label to flows
data
type: boolean
port:
default: 2055
description: Port of the flow collector (host port). By convention,
Expand Down Expand Up @@ -7817,6 +7823,12 @@ spec:
type: object
type: object
type: object
multiClusterDeployment:
default: false
description: Set `multiClusterDeployment` to `true` to enable
multi clusters feature. This will add clusterName label to flows
data
type: boolean
port:
default: 2055
description: Port of the flow collector (host port). By convention,
Expand Down
12 changes: 12 additions & 0 deletions config/crd/bases/flows.netobserv.io_flowcollectors.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4927,6 +4927,12 @@ spec:
type: object
type: object
type: object
multiClusterDeployment:
default: false
description: Set `multiClusterDeployment` to `true` to enable
multi clusters feature. This will add clusterName label to flows
data
type: boolean
port:
default: 2055
description: Port of the flow collector (host port). By convention,
Expand Down Expand Up @@ -7803,6 +7809,12 @@ spec:
type: object
type: object
type: object
multiClusterDeployment:
default: false
description: Set `multiClusterDeployment` to `true` to enable
multi clusters feature. This will add clusterName label to flows
data
type: boolean
port:
default: 2055
description: Port of the flow collector (host port). By convention,
Expand Down
1 change: 1 addition & 0 deletions controllers/flowcollector_controller_iso_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ func flowCollectorIsoSpecs() {
ConversationHeartbeatInterval: &metav1.Duration{Duration: time.Second},
ConversationEndTimeout: &metav1.Duration{Duration: time.Second},
ConversationTerminatingTimeout: &metav1.Duration{Duration: time.Second},
MultiClusterDeployment: ptr.To(true),
ClusterName: "testCluster",
Debug: flowslatest.DebugConfig{},
LogTypes: &outputRecordTypes,
Expand Down
33 changes: 20 additions & 13 deletions controllers/flowlogspipeline/flp_pipeline_builder.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ const (
conntrackTerminatingTimeout = 5 * time.Second
conntrackEndTimeout = 10 * time.Second
conntrackHeartbeatInterval = 30 * time.Second
clusterNameLabelName = "K8S_ClusterName"
)

type PipelineBuilder struct {
Expand Down Expand Up @@ -53,6 +54,10 @@ func (b *PipelineBuilder) AddProcessorStages() error {

indexFields, lastStage = b.addConnectionTracking(indexFields, lastStage)

if b.desired.Processor.MultiClusterDeployment != nil && *b.desired.Processor.MultiClusterDeployment {
indexFields = append(indexFields, clusterNameLabelName)
}

// enrich stage (transform) configuration
enrichedStage := lastStage.TransformNetwork("enrich", api.TransformNetwork{
Rules: api.NetworkTransformRules{{
Expand Down Expand Up @@ -316,19 +321,21 @@ func (b *PipelineBuilder) addTransformFilter(lastStage config.PipelineBuilderSta
var clusterName string
transformFilterRules := []api.TransformFilterRule{}

if b.desired.Processor.ClusterName != "" {
clusterName = b.desired.Processor.ClusterName
} else {
//take clustername from openshift
clusterName = string(globals.DefaultClusterID)
}
if clusterName != "" {
transformFilterRules = []api.TransformFilterRule{
{
Input: "K8S_ClusterName",
Type: "add_field_if_doesnt_exist",
Value: clusterName,
},
if b.desired.Processor.MultiClusterDeployment != nil && *b.desired.Processor.MultiClusterDeployment {
if b.desired.Processor.ClusterName != "" {
clusterName = b.desired.Processor.ClusterName
} else {
//take clustername from openshift
clusterName = string(globals.DefaultClusterID)
}
if clusterName != "" {
transformFilterRules = []api.TransformFilterRule{
{
Input: clusterNameLabelName,
Type: "add_field_if_doesnt_exist",
Value: clusterName,
},
}
}
}

Expand Down
18 changes: 18 additions & 0 deletions docs/FlowCollector.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7537,6 +7537,15 @@ TLS client configuration for Loki URL.
`Metrics` define the processor configuration regarding metrics<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>multiClusterDeployment</b></td>
<td>boolean</td>
<td>
Set `multiClusterDeployment` to `true` to enable multi clusters feature. This will add clusterName label to flows data<br/>
<br/>
<i>Default</i>: false<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>port</b></td>
<td>integer</td>
Expand Down Expand Up @@ -12707,6 +12716,15 @@ TLS client configuration for Loki URL.
`Metrics` define the processor configuration regarding metrics<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>multiClusterDeployment</b></td>
<td>boolean</td>
<td>
Set `multiClusterDeployment` to `true` to enable multi clusters feature. This will add clusterName label to flows data<br/>
<br/>
<i>Default</i>: false<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>port</b></td>
<td>integer</td>
Expand Down
8 changes: 8 additions & 0 deletions hack/cloned.flows.netobserv.io_flowcollectors.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3414,6 +3414,10 @@ spec:
type: object
type: object
type: object
multiClusterDeployment:
default: false
description: Set `multiClusterDeployment` to `true` to enable multi clusters feature. This will add clusterName label to flows data
type: boolean
port:
default: 2055
description: Port of the flow collector (host port). By convention, some values are forbidden. It must be greater than 1024 and different from 4500, 4789 and 6081.
Expand Down Expand Up @@ -5395,6 +5399,10 @@ spec:
type: object
type: object
type: object
multiClusterDeployment:
default: false
description: Set `multiClusterDeployment` to `true` to enable multi clusters feature. This will add clusterName label to flows data
type: boolean
port:
default: 2055
description: Port of the flow collector (host port). By convention, some values are forbidden. It must be greater than 1024 and different from 4500, 4789 and 6081.
Expand Down
Loading