Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement analyze mode for embed-code-go. #35

Merged
merged 79 commits into from
May 28, 2024
Merged

Conversation

vb-td
Copy link
Collaborator

@vb-td vb-td commented May 22, 2024

In this PR, the analyze mode was added.

Now, it is possible to run program to find the code embeddings which don't have corresponding code fragments for them. All corresponding information is written to the build/analytics/embeddings-not-found-files.txt file.
Here is example of the typical record in this file:

error for file C:/Users/user/project/_i18n/en/docs/8/guides/authentication.md, missing embeddings: 
java/guides/authentication/Examples.java — Proxy check
kotlin/kt/guides/authentication/Examples.kt — Proxy check
java/guides/authentication/Examples.java — Basic auth
kotlin/kt/guides/authentication/Examples.kt — Basic auth
java/guides/authentication/Examples.java — Cancel auth
kotlin/kt/guides/authentication/Examples.kt — Cancel auth
java/guides/authentication/Examples.java — Select client certificate
kotlin/kt/guides/authentication/Examples.kt — Select client certificate
java/guides/authentication/Examples.java — Cancel certificate auth
kotlin/kt/guides/authentication/Examples.kt — Cancel certificate auth
java/guides/authentication/Examples.java — Select custom certificate
kotlin/kt/guides/authentication/Examples.kt — Select custom certificate

Also, the information about changed embeddings collected as well. All corresponding information is written to the build/analytics/embeddings-changed-files.txt file.
Here is example of some typical records in this file:

en\docs\8\guides\authentication.md : EmbeddingInstruction[file=`kotlin/kt/guides/authentication/Examples.kt`, fragment=`Select custom certificate`, start=`<nil>`, end=`<nil>`]
en\docs\8\guides\browser.md : EmbeddingInstruction[file=`java/guides/browser/Examples.java`, fragment=`Create browser from profile`, start=`<nil>`, end=`<nil>`]
pt\docs\tutorials\integration\selenium.md : EmbeddingInstruction[file=`/tutorials/selenium/launcher/src/main/java/SeleniumLauncher.java`, fragment=``, start=`Pattern *import*`, end=`<nil>`]

Additionally, the error-handling flow has been modified. Previously, if any error occurred during the processing, the entire flow would stop. This behavior made it impossible to scan all embeddings and collect comprehensive analytics. Now, if any error occurs, the flow continues, the encountered errors are collected as artifacts, and no embeddings are performed as a result.

@vb-td vb-td self-assigned this May 22, 2024
@vb-td vb-td requested a review from armiol May 27, 2024 15:34
Copy link
Collaborator

@armiol armiol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with the respect of some comments to address first.

Copy link
Collaborator

@dmytro-kashcheiev dmytro-kashcheiev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vb-td LGTM with some minor comments

Comment on lines +61 to +63
if err != nil {
problemEmbeddingsLines = append(problemEmbeddingsLines, err.Error())
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have continue here or maybe else if since it take time to understand why we need to check changedEmbeddings even in case of errors.

Copy link
Collaborator Author

@vb-td vb-td May 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is confusing, but it can be many embeddings in one file, some of them can cause an error and some of them can be changed. So it is made by intention. I can comment this part or just don't check the file for changed embeddings if any error occurs.

@vb-td vb-td merged commit 9d753ad into embed-code-go May 28, 2024
@vb-td vb-td deleted the embed-code-go-analyze branch May 28, 2024 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants