Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(CLI): cache CDK API data in disk #1529

Merged
merged 10 commits into from
Feb 14, 2024
Merged

chore(CLI): cache CDK API data in disk #1529

merged 10 commits into from
Feb 14, 2024

Conversation

PengyuanZhao
Copy link
Contributor

@PengyuanZhao PengyuanZhao commented Feb 1, 2024

Summary

  • All lacework component commands will call LoadCatalog. It will try to load the components from the CLI cache. If not found, send a API request.

How did you test this change?

Manually run all the lacework component commands

Issue

https://lacework.atlassian.net/browse/GROW-2481

@PengyuanZhao PengyuanZhao requested a review from a team as a code owner February 1, 2024 23:33
Copy link
Contributor

@jon-stewart jon-stewart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found a problem.

> bin/lacework component install component-example --version 0.8.1
...
> bin/lacework component show component-example
          STATUS                  NAME          VERSION        DESCRIPTION
--------------------------+-------------------+---------+------------------------
  Tainted (Please update)   component-example   0.8.1     Component description

The following versions of this component are available to install:
 - 0.8.0
 - 0.8.1 (installed)
 - 0.8.2

Fix for this is:

line 517 cli/cmd/component.go: catalog, err := lwcomponent.NewCatalog(cli.LwApi, lwcomponent.NewStageTarGz, true). Pass true instead of false.

lwcomponent/cdk_component.go Outdated Show resolved Hide resolved
@jon-stewart
Copy link
Contributor

jon-stewart commented Feb 5, 2024

Get this error thrown when I delete the cdk_cache file.

> bin/lacework component show component-example
          STATUS                  NAME          VERSION        DESCRIPTION
--------------------------+-------------------+---------+------------------------
  Tainted (Please update)   component-example   0.8.1     Component description

The following versions of this component are available to install:
 - 0.8.0
 - 0.8.1 (installed)
 - 0.8.2
{"level":"error","ts":"2024-02-05T10:20:05Z","caller":"cmd/component.go:531","msg":"unable to save component component-example into cdk_cache: open /Users/j0n/.config/lacework/components/cdk_cache: no such file or directory","stacktrace":"github.com/lacework/go-sdk/cli/cmd.showComponent.func1\n\t/Users/j0n/Workspace/Lacework/go-sdk/cli/cmd/component.go:531\ngithub.com/lacework/go-sdk/cli/cmd.showComponent\n\t/Users/j0n/Workspace/Lacework/go-sdk/cli/cmd/component.go:548\ngithub.com/lacework/go-sdk/cli/cmd.runComponentsShow\n\t/Users/j0n/Workspace/Lacework/go-sdk/cli/cmd/component.go:507\ngithub.com/spf13/cobra.(*Command).execute\n\t/Users/j0n/Workspace/Lacework/go-sdk/vendor/github.com/spf13/cobra/command.go:940\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/Users/j0n/Workspace/Lacework/go-sdk/vendor/github.com/spf13/cobra/command.go:1068\ngithub.com/spf13/cobra.(*Command).Execute\n\t/Users/j0n/Workspace/Lacework/go-sdk/vendor/github.com/spf13/cobra/command.go:992\ngithub.com/lacework/go-sdk/cli/cmd.Execute\n\t/Users/j0n/Workspace/Lacework/go-sdk/cli/cmd/root.go:166\nmain.main\n\t/Users/j0n/Workspace/Lacework/go-sdk/cli/main.go:34\nruntime.main\n\t/opt/homebrew/Cellar/go/1.21.3/libexec/src/runtime/proc.go:267"}

Different error when I touch the file.

"msg":"unable to save component component-example into cdk_cache: unexpected end of JSON input"

@jon-stewart jon-stewart closed this Feb 5, 2024
@jon-stewart jon-stewart reopened this Feb 5, 2024
@jon-stewart
Copy link
Contributor

jon-stewart commented Feb 5, 2024

With caching/persist API info, we shouldn't need to call the API if the cache file is present. Example line 316 in cdk_component.go - don't need to call client.V2.Components.ListComponents.

Seems like we are calling the API when the info is cached.

lwcomponent/catalog.go Outdated Show resolved Hide resolved
@PengyuanZhao PengyuanZhao changed the title chore(CLI): persist CDK API data in disk chore(CLI): cache CDK API data in disk Feb 5, 2024
Copy link
Contributor

@jon-stewart jon-stewart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nicely done!

lwcomponent/cdk_component.go Show resolved Hide resolved
cli/cmd/component.go Outdated Show resolved Hide resolved
cli/cmd/component.go Outdated Show resolved Hide resolved
cli/cmd/root.go Outdated Show resolved Hide resolved
cachedComponentsApiInfo map[string]ApiInfo,
) (*Catalog, error) {
if stageConstructor == nil {
return nil, errors.New("nil Catalog StageConstructor")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error is not helpful to the user 🤔 what should the user do in this case?¿

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to "StageConstructor is not specified to create new catalog". Not exactly sure what else to say and why we inject stage constructor here. Will there be other types of stagers? cc @jon-stewart

lwcomponent/catalog.go Outdated Show resolved Hide resolved
lwcomponent/catalog.go Outdated Show resolved Hide resolved
Copy link
Contributor

@afiune afiune left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PengyuanZhao PengyuanZhao merged commit e969371 into main Feb 14, 2024
5 checks passed
@PengyuanZhao PengyuanZhao deleted the pengyuan/GROW-2481 branch February 14, 2024 21:42
@lacework-releng lacework-releng mentioned this pull request Feb 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants