Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Included an unshaded guava specifically for BigQuery usage. #177

Merged
merged 1 commit into from
Oct 29, 2024

Conversation

nika-qubit
Copy link
Contributor

@nika-qubit nika-qubit commented Oct 28, 2024

Note that the guava versions used by Flink and BigQuery can be incompatible with each other. The final uber jar of a consumer should include both shaded versions so that Flink and BigQuery connectors can function correctly with their own guava dependency. Without this, Maven will resolve to a single version of guava randomly picked from some transitive dependency and may result in runtime exceptions of a Flink job.

Minor: expanded tabs to whitespaces for the touched poms.

Before the change, guava is a transitive dep (introduced by BigQuery storage) of this project (common jar):
Screenshot 2024-10-28 at 11 09 11 AM

After the change, guava is a direct dep (introduced by the common jar of this project):
Screenshot 2024-10-28 at 1 51 12 PM

The build process (top-right), new jar structure (bot-left) and the old jar structure (bot-right):
Screenshot 2024-10-28 at 4 08 39 PM

You can see that the only addition is the guava jar, which looks like this:
Screenshot 2024-10-28 at 4 09 57 PM

Note that the guava versions used by Flink and BigQuery can be incompatible with each other.
The final uber jar of a consumer should include both versions so that Flink and BigQuery connectors
can function correctly with their own guava dependency. Without this, Maven will resolve to a
single version of guava randomly picked from some transitive dependency and may result in runtime
exceptions of a Flink job.
@nika-qubit nika-qubit changed the title Included a shaded guava specifically for BigQuery usage. Included an unshaded guava specifically for BigQuery usage. Oct 28, 2024
@prashastia
Copy link
Collaborator

/gcbrun

@jayehwhyehentee jayehwhyehentee merged commit e9ca296 into GoogleCloudDataproc:main Oct 29, 2024
4 checks passed
prashastia pushed a commit that referenced this pull request Oct 29, 2024
Reverts #175. Desired change was implemented in #177.

/gcbrun
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants