Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

allows controller to be configured via environment variable #31

Merged
merged 3 commits into from
Sep 21, 2024
Merged

allows controller to be configured via environment variable #31

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9272ce9
allows controller to be configured via environment variable
Peac36 Aug 24, 2024
1ad312e
add go-flags library
Peac36 Sep 1, 2024
6399339
minor fixes
Peac36 Sep 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,20 @@ Out of tree implementation of https://github.com/kubernetes/enhancements/tree/ma
It allows users to use an ipam-controller that allocates IP ranges to Nodes, setting the node.spec.PodCIDRs fields.
The ipam-controller is configured via CRDs

## Config

| Command line | Environment | Default | Description |
|-----------------------------|-----------------------------|-----------------------|----------------------------------------------------------------------------------------------------------------------|
| apiserver | IPAM_API_SERVER_URL | | The address of the Kubernetes API server. Overrides any value in kubeconfig. Only required if out-of-cluster. |
| kubeconfig | IPAM_KUBECONFIG | | Path to a kubeconfig. Only required if out-of-cluster. |
| health-probe-address | IPAM_HEALTH_PROBE_ADDR | :8081 | Specifies the TCP address for the health server to listen on. |
| enable-leader-election | IPAM_ENABLE_LEADER_ELECTION | true | Enable leader election for the controller manager. Ensures there is only one active controller manager. |
| leader-elect-lease-duration | IPAM_LEASE_DURATION | 15s | Duration that non-leader candidates will wait to force acquire leadership (duration string). |
| leader-elect-renew-deadline | IPAM_RENEW_DEADLINE | 10s | Interval between attempts by the acting master to renew a leadership slot before it stops leading (duration string). |
| leader-elect-retry-period | IPAM_RESOURCE_LOCK | 2s | Duration the clients should wait between attempting acquisition and renewal of a leadership (duration string). |
| leader-elect-resource-lock | IPAM_RESOURCE_LOCK_NAME | leases | The type of resource object that is used for locking. Supported options are 'leases', 'endpoints', 'configmaps'. |
| leader-elect-resource-name | IPAM_RESOURCE_NAME | node-ipam-controller | The name of the resource object that is used for locking. |

## Build

To build the binary for node-ipam-controller:
Expand Down Expand Up @@ -56,3 +70,4 @@ Install node-ipam-controller in the cluster via helm:
```sh
helm install node-ipam-controller ./charts/node-ipam-controller --create-namespace --namespace nodeipam --set image.tag=local
```

3 changes: 2 additions & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ go 1.21
require (
github.com/evanphx/json-patch v5.6.0+incompatible
github.com/golangci/golangci-lint v1.54.2
github.com/jessevdk/go-flags v1.6.1
github.com/onsi/ginkgo/v2 v2.13.2
github.com/onsi/gomega v1.30.0
github.com/stretchr/testify v1.8.4
Expand Down Expand Up @@ -210,7 +211,7 @@ require (
golang.org/x/net v0.17.0 // indirect
golang.org/x/oauth2 v0.8.0 // indirect
golang.org/x/sync v0.4.0 // indirect
golang.org/x/sys v0.14.0 // indirect
golang.org/x/sys v0.21.0 // indirect
golang.org/x/term v0.13.0 // indirect
golang.org/x/text v0.13.0 // indirect
golang.org/x/time v0.3.0 // indirect
Expand Down
4 changes: 4 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,8 @@ github.com/imdario/mergo v0.3.6/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJ
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
github.com/jessevdk/go-flags v1.6.1 h1:Cvu5U8UGrLay1rZfv/zP7iLpSHGUZ/Ou68T0iX1bBK4=
github.com/jessevdk/go-flags v1.6.1/go.mod h1:Mk8T1hIAWpOiJiHa9rJASDK2UGWji0EuPGBnNLMooyc=
github.com/jgautheron/goconst v1.5.1 h1:HxVbL1MhydKs8R8n/HE5NPvzfaYmQJA3o879lE4+WcM=
github.com/jgautheron/goconst v1.5.1/go.mod h1:aAosetZ5zaeC/2EfMeRswtxUFBpe2Hr7HzkgX4fanO4=
github.com/jingyugao/rowserrcheck v1.1.1 h1:zibz55j/MJtLsjP1OF4bSdgXxwL1b+Vn7Tjzq7gFzUs=
Expand Down Expand Up @@ -824,6 +826,8 @@ golang.org/x/sys v0.4.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.14.0 h1:Vz7Qs629MkJkGyHxUlRHizWJRG2j8fbQKjELVSNhy7Q=
golang.org/x/sys v0.14.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.21.0 h1:rF+pYz3DAGSQAxAu1CbC7catZg4ebC4UIeIhKxBZvws=
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.1.0/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
Expand Down
87 changes: 51 additions & 36 deletions main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,65 +25,80 @@ import (
"os"
"time"

metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/tools/clientcmd"
"k8s.io/component-base/logs"
logsapi "k8s.io/component-base/logs/api/v1"
clientset "sigs.k8s.io/node-ipam-controller/pkg/client/clientset/versioned"
informers "sigs.k8s.io/node-ipam-controller/pkg/client/informers/externalversions"
"sigs.k8s.io/node-ipam-controller/pkg/controller/ipam"

"sigs.k8s.io/node-ipam-controller/pkg/leaderelection"
"sigs.k8s.io/node-ipam-controller/pkg/signals"

"github.com/jessevdk/go-flags"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
kubeinformers "k8s.io/client-go/informers"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"
logsapi "k8s.io/component-base/logs/api/v1"
clientset "sigs.k8s.io/node-ipam-controller/pkg/client/clientset/versioned"
informers "sigs.k8s.io/node-ipam-controller/pkg/client/informers/externalversions"
"sigs.k8s.io/node-ipam-controller/pkg/controller/ipam"

// Import all Kubernetes client auth plugins (e.g. Azure, GCP, OIDC, etc.)
// to ensure that exec-entrypoint and run can make use of them.
_ "k8s.io/client-go/plugin/pkg/client/auth"
"k8s.io/client-go/tools/clientcmd"
_ "k8s.io/component-base/logs/json/register"
"k8s.io/klog/v2"
)

func main() {
var (
apiServerURL string
kubeconfig string
healthProbeAddr string
enableLeaderElection bool
leaseDuration time.Duration
renewDeadline time.Duration
retryPeriod time.Duration
resourceLock string
resourceName string
)
type config struct {
ApiServerURL string `long:"apiserver" description:"The address of the Kubernetes API server. Overrides any value in kubeconfig. Only required if out-of-cluster." env:"IPAM_API_SERVER_URL"`
Kubeconfig string `long:"kubeconfig" description:"Path to a kubeconfig. Only required if out-of-cluster." env:"IPAM_KUBECONFIG"`
HealthProbeAddr string `long:"health-probe-address" default:":8081" description:"Specifies the TCP address for the health server to listen on." env:"IPAM_HEALTH_PROBE_ADDR"`
EnableLeaderElection bool `long:"enable-leader-election" description:"Enable leader election for the controller manager. Ensures there is only one active controller manager." env:"IPAM_ENABLE_LEADER_ELECTION"`
LeaseDuration time.Duration `long:"leader-elect-lease-duration" default:"15s" description:"Duration that non-leader candidates will wait to force acquire leadership (duration string)." env:"IPAM_LEASE_DURATION"`
RenewDeadline time.Duration `long:"leader-elect-renew-deadline" default:"10s" description:"Interval between attempts by the acting master to renew a leadership slot before it stops leading (duration string)." env:"IPAM_RENEW_DEADLINE"`
RetryPeriod time.Duration `long:"leader-elect-retry-period" default:"2s" description:"Duration the clients should wait between attempting acquisition and renewal of a leadership (duration string)." env:"IPAM_RESOURCE_LOCK"`
ResourceLock string `long:"leader-elect-resource-lock" default:"leases" description:"The type of resource object that is used for locking. Supported options are 'leases', 'endpoints', 'configmaps'." env:"IPAM_RESOURCE_LOCK_NAME"`
ResourceName string `long:"leader-elect-resource-name" default:"node-ipam-controller" description:"The name of the resource object that is used for locking." env:"IPAM_RESOURCE_NAME"`
}

flag.StringVar(&kubeconfig, "kubeconfig", "", "Path to a kubeconfig. Only required if out-of-cluster.")
flag.StringVar(&apiServerURL, "apiserver", "", "The address of the Kubernetes API server. Overrides any value in kubeconfig. Only required if out-of-cluster.")
flag.StringVar(&healthProbeAddr, "health-probe-address", ":8081", "Specifies the TCP address for the health server to listen on.")
flag.BoolVar(&enableLeaderElection, "enable-leader-election", true, "Enable leader election for the controller manager. Ensures there is only one active controller manager.")
flag.DurationVar(&leaseDuration, "leader-elect-lease-duration", 15*time.Second, "Duration that non-leader candidates will wait to force acquire leadership (duration string).")
flag.DurationVar(&renewDeadline, "leader-elect-renew-deadline", 10*time.Second, "Interval between attempts by the acting master to renew a leadership slot before it stops leading (duration string).")
flag.DurationVar(&retryPeriod, "leader-elect-retry-period", 2*time.Second, "Duration the clients should wait between attempting acquisition and renewal of a leadership (duration string).")
flag.StringVar(&resourceLock, "leader-elect-resource-lock", "leases", "The type of resource object that is used for locking. Supported options are 'leases', 'endpoints', 'configmaps'.")
flag.StringVar(&resourceName, "leader-elect-resource-name", "node-ipam-controller", "The name of the resource object that is used for locking.")
func (c *config) load() error {
// allows using true/false in the parameters
_, err := flags.NewParser(c, flags.Default|flags.AllowBoolValues).ParseArgs(os.Args)
if err != nil {
return err
}
return nil
}

func main() {
c := logsapi.NewLoggingConfiguration()
logsapi.AddGoFlags(c, flag.CommandLine)
flag.Parse()

conf := config{EnableLeaderElection: true}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not familiar with this framework, does the library sets this to false if the flag is set and ignores it if the flag is not set @mneverov

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We discussed that by default the controller should run in leader election mode. The go-flags library does not have default values for boolean flags and suggests to use reverted flags like --disable-leader-election which will be on by default and can be overwritten.
To workaround this and provide the same flag semantics as in #20 @Peac36 added the line above such that if --enable-leader-election flag is not provided the controller will still run in leader election mode (again as before).
To disable leader election one can use --enable-leader-election=false.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sense

/lgtm

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the new library differentiate between flag not set and set to false

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When --enable-leader-election or --enable-leader-election=true are provided as well as when the flag is not provided at all the controller will start in leader election mode.
Disabling leader election is possible via --enable-leader-election=false

err := conf.load()
if err != nil {
var flagError *flags.Error
if errors.As(err, &flagError) {
if flagError.Type == flags.ErrHelp {
os.Exit(0)
}
os.Exit(1)
}
mneverov marked this conversation as resolved.
Show resolved Hide resolved
fmt.Fprintf(os.Stderr, "unable to parse config: %q", err)
os.Exit(1)
}

logs.InitLogs()
if err := logsapi.ValidateAndApply(c, nil); err != nil {
fmt.Fprintf(os.Stderr, "%v\n", err)
os.Exit(1)
}

ctx, cancel := context.WithCancel(signals.SetupSignalHandler())
defer cancel()
logger := klog.FromContext(ctx)

server := startHealthProbeServer(healthProbeAddr, logger)
cfg, err := clientcmd.BuildConfigFromFlags(apiServerURL, kubeconfig)
server := startHealthProbeServer(conf.HealthProbeAddr, logger)
cfg, err := clientcmd.BuildConfigFromFlags(conf.ApiServerURL, conf.Kubeconfig)
if err != nil {
logger.Error(err, "failed to build kubeconfig")
klog.FlushAndExit(klog.ExitFlushTimeout, 1)
Expand All @@ -95,14 +110,14 @@ func main() {
klog.FlushAndExit(klog.ExitFlushTimeout, 1)
}

if enableLeaderElection {
if conf.EnableLeaderElection {
logger.Info("Leader election is enabled.")
leaderelection.StartLeaderElection(ctx, kubeClient, cfg, logger, cancel, runControllers, leaderelection.Config{
LeaseDuration: leaseDuration,
RenewDeadline: renewDeadline,
RetryPeriod: retryPeriod,
ResourceLock: resourceLock,
ResourceName: resourceName,
LeaseDuration: conf.LeaseDuration,
RenewDeadline: conf.RenewDeadline,
RetryPeriod: conf.RetryPeriod,
ResourceLock: conf.ResourceLock,
ResourceName: conf.ResourceName,
})
} else {
logger.Info("Leader election is disabled.")
Expand Down
Loading