Skip to content

Commit

Permalink
Poll synctracker HasSynced() during startup
Browse files Browse the repository at this point in the history
The "upstream" informer trackers do not always return HasSynced() == true
immediately, even if we have already processed all events the informer
dispatched us.  Change to polling so that we can progress also in that case.

Signed-off-by: Tero Saarni <tero.saarni@est.tech>
  • Loading branch information
tsaarni committed Aug 16, 2024
1 parent f9b6f6f commit ed7b7de
Showing 1 changed file with 10 additions and 19 deletions.
29 changes: 10 additions & 19 deletions internal/contour/handler.go
Original file line number Diff line number Diff line change
Expand Up @@ -135,11 +135,14 @@ func (e *EventHandler) Start(ctx context.Context) error {
// yet included in a DAG rebuild.
outstanding int

// timer holds the timer which will expire after e.HoldoffDelay
timer *time.Timer
// Polling interval during startup, for the initial informer synchronization.
initialSyncPollPeriod = 100 * time.Millisecond

// timer triggers delayed DAG rebuilds.
timer *time.Timer = time.NewTimer(initialSyncPollPeriod)

Check failure on line 142 in internal/contour/handler.go

View workflow job for this annotation

GitHub Actions / lint

var-declaration: should omit type *time.Timer from declaration of var timer; it will be inferred from the right-hand side (revive)

// pending is a reference to the current timer's channel.
pending <-chan time.Time
pending <-chan time.Time = timer.C

Check failure on line 145 in internal/contour/handler.go

View workflow job for this annotation

GitHub Actions / lint

var-declaration: should omit type <-chan time.Time from declaration of var pending; it will be inferred from the right-hand side (revive)

// lastDAGRebuild holds the last time rebuildDAG was called.
// lastDAGRebuild is seeded to the current time on entry to
Expand All @@ -153,12 +156,6 @@ func (e *EventHandler) Start(ctx context.Context) error {
return
}

// It may be that there are no resources at all to process in watched namespaces.
// Initial (empty) DAG build is not needed and we can mark it as built immediately to allow the XDS server to start.
if e.syncTracker.HasSynced() {
e.initialDagBuilt.Store(true)
}

for {
// In the main loop one of four things can happen.
// 1. We're waiting for an event on op, stop, or pending, noting that
Expand Down Expand Up @@ -196,21 +193,15 @@ func (e *EventHandler) Start(ctx context.Context) error {
if updateOpAdd, ok := op.(opAdd); ok {
if updateOpAdd.isInInitialList {
e.syncTracker.Finished()

// If this was the last event in the initial list but none of the events triggered DAG rebuild,
// then we can mark the (empty) DAG as built to allow the XDS server to start.
if e.syncTracker.HasSynced() && timer == nil {
e.initialDagBuilt.Store(true)
}
// Note that we don't need to check syncTracker.HasSynced() here, as it sometimes returns false
// for a short period after the initial list is processed.
}
}
case <-pending:
// Ensure informer caches are synced.
// Schedule a retry for dag rebuild if cache is not synced yet.
// Ensure informer caches are synced, schedule a retry if cache is not synced yet.
// Note that we can't block and wait for the cache sync as it depends on progress of this loop.
if !e.syncTracker.HasSynced() {
e.Info("skipping dag rebuild as cache is not synced")
timer.Reset(e.holdoffDelay)
timer.Reset(initialSyncPollPeriod)
break
}

Expand Down

0 comments on commit ed7b7de

Please sign in to comment.