forked from GoogleCloudPlatform/DataflowTemplates
-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
sync main #123
Merged
Merged
sync main #123
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: pawankashyapollion <v-pawan.kumar@ollion.com>
* Adding test table for Map types * Adding support for cassandra map
* add inf fix to ts fixing bug on timestamp type * Update FormatDatastreamRecordToJson.java use equals not ==
* Upgrade Beam version to 2.63.0 * add cache factory to local spanner io * remove cache pass to ReadChangeStreamPartitionDoFn * Add unimplemented/unused stubs to TestChangeStreamMutation * Use java Instant instead of threeten Instant * Fix low watermark setter call * fix testchangestreammutation * Recreate validation PR * fix import order * linux-env requirements files * fix v1 pom
* Add load test for cross db txn * Change test timeout to 2 days * Revert spanner-pr.yml changes
* Report Lineage for CsvToBigQuery template * Reply beampr-32662 to CsvConverters
* Changes to write read interleave type from information schema, and write/read the type to/from avro. No change to tests yet. * Fix warnings, bug in InformationSchemaScanner, and bug in AvroSchemaToDdlConverter. Also properly default to IN PARENT when emitting ddl, in case the interleave type is not set (really only necessary for tests, since otherwise it will always be set. * Set interleaveType in InfoSchemaScanner * Style fixes, and only generate INTERLEAVE IN ddl for gsql. * another style fix - remove unused import * Make conditions more readable and add comments * Tests * Fix condition * Add interleave in table to ExportPipelineIT
Added SCRAM-SHA-512 authentication support to Kafka to Kafka, Kafka to GCS, and Kafka to BigQuery templates.
#2218) * Update java requirement * Generate docs
* Uncomment kinesis * bump protoc to 4 in v2
* changes * Changes * changes * changes * docs
…y template (#2084) * Support partitioned reads for DateTime column type * minor changes * Support backward compatibility and timezone in lower/upper bounds * removed test cases for bounds in unit tests as derby does not support timezone * Fixed typo's in JdbcToBigQuery.java Corrected testcases expected output * Corrected unit test cases expected output. Removed DateTime integration test cases. * Corrected a typo * Added default value for partitionColumnType * spotless apply
* disabling flaky test to unblock dataflow release * minor change
* skip runnerv2 tests in TemplateTestBase * add logic to skip use_runner_v2 experiment in the launchTemplate * Remove redundant property skipRunnerV2Test in pom and update logic in launchTemplate * Fix format violations using mvn spotless:apply
* SpannerExceptionClassifier class and IT * Unit tests and Integration tests * Formatting changes * Correcting UT * Unit test for SpannerResourceManager * Creating Spanner Migration Exception * Formatting changes * Correcting tests * Addressing comments * Correcting a UT * Addressing comments * Changing SpannerMigrationException to extend RuntimeException
defector221
approved these changes
Mar 3, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
pawankashyapollion
approved these changes
Mar 3, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.