Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trigger show-tech in case of VM postProcess timeout #349

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

pau-hedgehog
Copy link
Contributor

@pau-hedgehog pau-hedgehog commented Jan 27, 2025

We are facing several CI failure conditions where show-tech is not executed and thus we cannot diagnose the root cause

Related to #333

@pau-hedgehog pau-hedgehog self-assigned this Jan 27, 2025
Signed-off-by: Pau Capdevila <pau@githedgehog.com>
Signed-off-by: Pau Capdevila <pau@githedgehog.com>
Signed-off-by: Pau Capdevila <pau@githedgehog.com>
@pau-hedgehog pau-hedgehog marked this pull request as ready for review January 31, 2025 11:35
@pau-hedgehog
Copy link
Contributor Author

pau-hedgehog commented Jan 31, 2025

Maybe we should also try to capture show-tech when other on-ready commands fail:

https://github.com/githedgehog/fabricator/actions/runs/13070506705/job/36471259713

In principle it's there but not working:

                case OnReadyInspect:
                    appliedFor := 2 * time.Minute
                    if opts.ControlUpgrade {
                        appliedFor = 4 * time.Minute
                    }       
    
                    if err := c.Inspect(ctx, vlab, InspectOpts{WaitAppliedFor: appliedFor}); err != nil {
                        slog.Warn("Failed to inspect", "err", err)
    
                        if opts.CollectShowTech {
                            if err := c.VLABShowTech(ctx, vlab); err != nil {
                                slog.Warn("Failed to collect show-tech diagnostics", "err", err)
    
                                return fmt.Errorf("getting show-tech: %w", err)
                            }
                        }
    
                        return fmt.Errorf("inspecting: %w", err)
                    }
                }
            } 

@@ -400,25 +400,26 @@ func (c *Config) VLABRun(ctx context.Context, vlab *VLAB, opts VLABRunOpts) erro
if vm.Type == VMTypeServer || vm.Type == VMTypeControl {
postProcesses.Add(1)
group.Go(func() error {
defer postProcesses.Done()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think having defer ...Done() will break the logic of post actions, I need to dig deeper to understand if it's a problem

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants