Skip to content

Release-2.3.0
Compare
Choose a tag to compare
@leonardBang leonardBang released this 10 Nov 01:56
· 7 commits to release-2.3 since this release
release-2.3.0
c65f9e0

Release Notes

Documentation: https://ververica.github.io/flink-cdc-connectors/release-2.3

Features and Improvements

  1. [mysql] Improve error message under the case that start reading from earliest but schema change happened before (#1724)
  2. [mysql] Use event filter to drop events earlier than the specified timestamp in timestamp startup mode (#1724)
  3. [docs][oracle] Add new introduced config options from incremental snapshot framework (#1718)
  4. [docs][oceanbase][zh] Update outdated sections and translate them to Chinese
  5. [hotfix][docs] Add link for Oceanbase ZH docs
  6. [docs][oceanbase] Add Chinese documents for oceanbase-cdc connector(#1611)
  7. [db2] Fix stream api example in db2 doc. (#1725)
  8. [oracle] Add missing supported config options for oracle table source (#1720)
  9. [mysql] Fix MySqlBinlogSplitAssigner may reassign split after exception restart (#1713)
  10. [docs] Repair Connector Data Type links
  11. [docs] Update the connector download link in the document
  12. [docs] Update the flink cdc picture with supported database vendors.
  13. [tidb] Fix unstable TiDB region changed test. (#1702)
  14. [docs][mongodb] Add docs for MongoDB incremental source
  15. [oracle][mysql] Improve the Oracle all data types test and clean up debug logs
  16. [oracle] Properly support TIMESTAMP_LTZ type for oracle cdc connector
  17. [oracle]support use user given column as split column
  18. [oracle][minor] Clean up useless methods and classes
  19. [oracle] Parallel read supports startup of startupOptions
  20. [oracle] Add full type field test
  21. [oracle]OracleTableSource support splitSize, fetchSize ,connectMaxRetries, connectionPoolSize and add test
  22. [hotfix][oracle][db2] Update the license header
  23. [cdc-base][oralce] Remove useless method of JdbcSourceFetchTaskContext
  24. [cdc-base][oracle] Improve the base methods base on Oracle specific requirements
  25. [oracle] Support custom url for incremental snapshot source
  26. [oracle] Introduce SCAN_INCREMENTAL_SNAPSHOT_ENABLED option for OracleTableSourceFactory
  27. [minor][oracle] Improve tests and code format
  28. [oracle] Ignore testCheckpointAndRestore util DBZ-5245 and DBZ-4936 fix
  29. [oracle][test] Add tests for Oracle incremental snapshot source
  30. [oracle] Use ROWID as split column and replace limit syntax with ROWNUM
  31. [oracle] Let LogMinerStreamingChangeEventSource do not rely on oracle table LOG_MINING_FLUSH
  32. [oracle] Use Incremental Snapshot Framework for Oracle CDC Connector (#1079)
  33. [docs] Bump Flink version to 1.16.0
  34. [common] Bump Flink version to 1.16.0
  35. [docs][db2] Add db2 to README.md (#1699)
  36. [tidb] Checkpoint is not updated long after a task has been running (#1686)
  37. [hotfix] Add method getMaxResolvedTs back to class CDCClient. (#1695)
  38. [docs] Bump connector version to flink 1.15.2 in docs (#1684)
  39. [tidb] Fix data lost when region changed (#1632)
  40. [hotfix][docs] Correct reference link for DB2 docs (#1683)
  41. [mysql] Update docs of specifying starting offset feature of MySQL CDC source
  42. [hotfix][mysql] Remove unused constructor in MySqlTableSource
  43. [mysql] Support starting reading binlog from earliest offset / specific offset / timestamp (#1661)
  44. [mongodb] Support incremental snapshot for MongoDB CDC Connector
  45. [cdc-base] Extract a higher level IncrementalSource independent of JDBC dialects
  46. [hotfix][docs] Fix db2 doc download link (#1681)
  47. [docs][db2] Add Db2 CDC connector doc link
  48. [docs][db2] Add Db2 cdc connector docs
  49. [build][db2] Add Db2 pipeline script
  50. [e2e][db2] Add Db2 e2e test
  51. [db2] Add sql connector for Db2 CDC (#1626)
  52. [db2] Introduce Db2 cdc connector (#450)
  53. [tidb] Fix data loss in tidb-cdc when switch from snapshot read to change event read (#1207)
  54. [hotfix][cdc-base] Fix OffsetFactory is not initialized when restored from checkpoint (#1657)
  55. [common] Bump Flink version to 1.15.2 (#1504)
  56. [cdc-base] Abstract naming for variables of incremental framework (#1646)
  57. [mysql] ChunkSplitter supports to snapshot when splitting chunks for a table (#1463)
  58. [cdc-base] Fix StreamSplit's endingOffset of StreamSplitAssigner (#1659)
  59. [cdc-base] JdbcSourceConfig can be template type of DataSourceDialect (#1639)
  60. [mysql] Use the latest received meta of binlog split when request the next meta (#1479)
  61. [minor] Use Collections.singletonList instead of Arrays.asList if possible (#1569)
  62. [oceanbase] Add new deserialization schema with runtime converter (#1356)
  63. [mysql] Avoid duplicate split requests when add new table (#1156)
  64. [postgres] Upgrade org.postgresql:postgresql from 42.2.14 to 42.2.26 for CVE-2022-31197 (#1616)
  65. [test][e2e] Upgrade org.postgresql:postgresql from 42.3.1 to 42.2.26 for CVE-2022-31197(#1616)
  66. [hotfix][typo] Fix typo from schemaLess to schemaless
  67. [cdc-base] Reduce the memory usage of JM by sharing table schemas between splits
  68. [cdc-base] Release the scan fetcher thread resource after reading finished (#1619)
  69. [cdc-base] Optimize pure binlog phase check logic to improve performance (#1620)
  70. [cdc-base] Let all records of one snapshot split don't cross checkpoints (#1622)
  71. [postgres] Introduce changelog mode configuration with upsert/all modes
  72. [common] Introduce changelog mode configuration with upsert/all modes
  73. [hotfix][docs] Fix docs for scan.incremental.snapshot.chunk.size (#1617)
  74. [cdc-base] Adopt streaming merge during snapshotSplit scan to avoid OOM
  75. [docs][oceanbase][zh] Add video tutorials that synchronizes data from OceanBase to Elasticsearch (#1604)
  76. [docs][oceanbase] Add video tutorials that synchronizes data from OceanBase to Elasticsearch (#1605)
  77. [mysql] Fix parse error when ALTER SQL statement contains default string value and collate. (#1544)
  78. [docs][mysql] Add regex examples for multiple tables (#1564)
  79. [oceanbase] Shade apache commons dependency to avoid class conflict (#1532)
  80. [postgres] add geometry type support. (#1554)
  81. [build] Polish CI scripts and remove useless code (#1540)
  82. [mysql] fix parsing ddl error when use string default values for numeric types (#1522)
  83. [mysql] Enumerate captured tables lazily (#1507)
  84. [mysql] Reduce the memory usage of JM by sharing table schemas between splits (#1523)
  85. [hotfix][docs] Fix comment typo in MysqlSnapshotSplit (#1513)
  86. [mysql] Let all records of one snapshot split don't cross checkpoints (#1505)
  87. [tests] Remove usage of deprecated EnvironmentSettings.Builder#useBlinkPlanner (#1493)
  88. [cdc-base][mysql] Fix split key comparison when may contain different types (#1499)
  89. [docs][mysql] Update the description of mysql server-time-zone option (#1497)
  90. [mysql] Use local timezone as the default value of 'server-time-zone' option (#1407)
  91. [docs][postgres] Add two frequently debezium options in Postgres connector document (#1142)
  92. [mongodb] Allow mongo ARRAY to be converted to string type in Flink (#1475)
  93. [hotfix][docs] Fix the page links in MySQL Chinese document (#1466)
  94. [oceanbase][test] Use memory mode of obcdc and double startup timeout duration (#1474)
  95. [mysql][polardbx] Stop container timely in tests (#1468)
  96. [mysql][ploardbx] Support all charsets in column level (#1166)
  97. [mongodb] Expose updateLookup option for stream source. (#1461)
  98. [oceanbase] Use utc offset as 'server-time-zone' option's default value (#1443)
  99. [mysql][minor] Skip capture newly added tables according snapshotassigner's status (#1449)
  100. [polardbx][test] Bump the polardb-x image version from 2.0.1 to 2.10 (#1423)
  101. [build][test] Specify surefire provider to surefire-junit47 (#1423)
  102. [hotfix][docs] Fix typo in bug report doc (#1460)
  103. [minor][oracle] Rename JdbcConfigurationUtil to OracleJdbcUrlUtils
  104. [oracle] Support custom JDBC URL for Oracle (#1258)
  105. [docs] Correct the data type of 'server-id' option to String (#1444)
  106. [hotfix][test] Fix MySqlSourceITCase failure due to StreamExecutionEnvironment#getConfiguration returned unmodifiable object (#1446)
  107. [test] Fix tests missed 'update_before' records if the primary keys of source and sink are different (#1451)
  108. [hotfix][oracle] Fix the shell script interpreter assignment should be at first line (#1452)
  109. [hotfix][tidb-cdc] TiDB CDC dealUnsignedColumnValue throw NullPointerException. (#1448)
  110. [polardbx][test] Rename PolardbxSourceTCase to PolardbxSourceITCase (#1447)
  111. [mysql] Enable pass custom jdbc properties for debezium mysql connection (#1217)
  112. [mysql] Release the debezium reader thread resources after reading finished (#1358)
  113. [oceanbase] Introduce 'table-list' option to support capture list of tables (#1369)
  114. [cdc-base] Flink CDC base registers the identical history engine on multiple tasks (#1340)
  115. [hotfix][mysql] Fix compile error due to merge conflict
  116. [mysql] Generates multiple chunks when approximate row count is bigger than chunk size (#1193)
  117. [cdc-base] Fix NPE during snpashot scan phase (#1339)
  118. [docs] Improve MySQL ZH document and update the outdated links (#1435)
  119. [docs] Translate mysql cdc doc to Chinese (#1391)
  120. [mysql] Introduce 'scan.incremental.snapshot.chunk.key-column' option to support splitting table by custom column (#1415)
  121. [mysql] Optimize pure binlog phase check logic to improve performance (#1392)
  122. [oracle] Fix mining session stopped due to 'No more data to read from socket'
  123. [common] Ignore retriableException to make the debezium engine retry works
  124. [test][oracle] Fix unstable tests OracleE2eITCase due to reuse container
  125. [tidb] Fix NPE when the specified table does not exist (#1411)
  126. [docs] Add step up SQLServer Database cdc document in sqlserver-cdc.md.
  127. [mongodb] Expose batch.size config option for table source. (#1387)
  128. [mongodb] Example tests for MongoDBSource (#1388)
  129. [build] Use correct year and copyright owner in license files (#1390)
  130. [mongodb] Allow document field defined as string type in flink sql (#1385)
  131. [mysql] Fix dead loop when splitting chunks. (#1107)
  132. [docs] Fix a typo in sqlserver-cdc.md (#1382)
  133. [docs] Fix typo in mysql-cdc.md (#1364)
  134. [docs] Fix a typo in tidb-cdc.md (#1215)
  135. [docs] Add all supported type mapping for mysql cdc connector (#1373)
  136. [build] Build document site only if current PR modified files under docs (#1230)
  137. [oracle] Support XMLType for Oracle CDC Connector (#1342)
  138. [build] Bump Flink version to 1.14.4 (#1284)
  139. [mysql] Adopt streaming merge during snapshotSplit scan to avoid OOM (#1219)
  140. [oceanbase] Introduce more config options of log proxy and improve the oceanbase connector(#1234)
  141. [docs][cdc-base] Optimize annotations (#1320)
  142. [mysql] Connection pool id should contains username (#1328)
  143. [tidb-cdc] Monitor tidb table lose data, which table has one more regions. (#1196)
  144. [dbz] Patch for DBZ-5126 to fix memory leak problem. (#939)
  145. [hotfix][docs] Fix typo in java docs. (#1305)
  146. [mongodb] Permission optimization of single database is specified (#1240)
  147. [hotfix][cdc-base] Fix typo 'currenFetcher' -> 'currentFetcher' (#1299)
  148. [hotfix][oceanbase] Fix the default string encoding(#1294)
  149. [mongodb][doc] Fix meta field collection_name of metadata sql example (#1290)
  150. [mongodb] Fix UnsupportedTemporalTypeException when define Timestamp as String (#1268)
  151. [mysql] Support optional numeric unsigned zerofill types (#1197)
  152. [mysql] Add heartbeatInterval field to equals() and hashCode() (#1116)
  153. [mysql] Catch and hold exception in chunk splitter thread and rethrow in getNext() (#1109)
  154. [mysql] Use the gtid set firstly when compare the binlog offset (#761)
  155. [hotfix][tidb] Fix RowKind.INSERT judgement condition (#1073)
  156. [oracle] Fix duplicated records when restoration (#1100)
  157. [mysql] Throw RuntimeException timely in snapshot scan phase (#1098)
  158. [docs][mongodb] Update document about MongoDB database and collection regex filter (#1089)
  159. [common] Apply filter on ResolvedSchema to keep physical columns only (#1092)
  160. [oracle][test] Use prebuilt docker image in tests
  161. [common] Bump debezium version to 1.6.4.Final
  162. [cdc-base] Improve Incremental snapshot framework code style. (#1054)
  163. [tidb] Test TiDB 6.0.0 and add it to supported list (#1057)
  164. [mysql] Serialize isSuspended flag for MySqlBinlogSplit (#996)
  165. [docs][mysql] Fix connector document link error (#1043)
  166. [sqlserver] Add more supported(tested) SqlServer versions (#1050)
  167. [docs] Update the supported sink vendors
  168. [oceanbase] Remove useless dependency and code of test. (#1018)
  169. [docs][mysql] Add note to make the tableList usage and dependencies clearer (#1019)
  170. [hotfix][docs] Fix misspell of docs title
  171. [docs] Add example for Scan Newly Added tables feature (#1016)
  172. [hotfix][docs] Fix jar link for PolarDB-X doc. (#1012)

Download

Contributor List (sorted alphabetically)

  1. 01410172
  2. Amber Moe
  3. Dezhi Cai
  4. Enoch
  5. Hang Ruan
  6. He Wang
  7. JiaJia
  8. Jiabao Sun
  9. Junwang Zhao
  10. Kyle Dong
  11. Leonard Xu
  12. Matrix42
  13. Paul Lin
  14. Qingsheng Ren
  15. Qishang Zhong
  16. Rinka
  17. Sergey Nuyanzin
  18. Tigran Manasyan
  19. camelus
  20. dujie
  21. ehui
  22. empcl
  23. fbad
  24. gongzhongqiang
  25. hehuiyuan
  26. hele.kc
  27. hsldymq
  28. jiabao.sun
  29. legendtkl
  30. leixin
  31. leozlliang
  32. lidoudou1993
  33. lincoln lee
  34. lxxawfl
  35. lzshlzsh
  36. molsion
  37. molsionmo
  38. pacino
  39. rookiegao
  40. skylines
  41. sunny
  42. vanliu
  43. wangminchao
  44. wangxiaojing
  45. xieyi888
  46. yurunchuan
  47. zhmin
  48. 阿洋
  49. 莫贤彬
Assets 2
Loading