Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate latest changes in CS into stable branch #9271

Closed
zverevgeny opened this issue Sep 16, 2024 · 0 comments · Fixed by #9268
Closed

Integrate latest changes in CS into stable branch #9271

zverevgeny opened this issue Sep 16, 2024 · 0 comments · Fixed by #9268
Assignees

Comments

@zverevgeny
Copy link
Collaborator

notation:
(+!) - already integrated, conflict when cherry-pick again
(+!-) - already integrated, when cherry-pick again tries to rollback previous changes
good - cherry-picked with no conflict
(+0) - already integrated, no changes when cherry-pick again
ignore - do not cherry-pick this time
conflict - there was some conflicts

e3c9723 zverevgeny Sun Sep 15 09:20:22 Turn on GCCountersNormalizer (#8166)
revert ccf98e2

good 991079d zverevgeny Mon Jul 22 09:14:55 delete unused ISSTransactionOperator (#6908)
(+!) 51ddf40 ivanmorozov333 Mon Jul 22 11:46:09 unify indexation (for committed data) and compaction (#6909)
(+!) 0662944 ivanmorozov333 Mon Jul 22 16:50:51 separate merger logic for make new one (#6931)
good 298b189 Semyon Mon Jul 22 21:57:11 Fix flaky KqpOlapScheme.DropTable (#6795)
(+0) 2ca7153 zverevgeny Tue Jul 23 08:15:22 Extend DisabledOnSchemeShard FF for column tables on dedicated bases (#6498)
good 669605c zverevgeny Wed Jul 24 09:30:10 increase CS read timeout (#6963)
(+0) edea306 zverevgeny Wed Jul 24 18:30:33 fix unused (#7046)
good 4598038 ivanmorozov333 Fri Jul 26 10:16:10 add stats test for internal column table from tables store (#6684)
(+0) 9785f6a ivanmorozov333 Fri Jul 26 19:06:17 dont create delete flag column in indexation (#7082)
(+0) 0671f74 zverevgeny Sat Jul 27 11:22:09 Disable modification operations in CS by default (#7129)
conflict ae7146c ivanmorozov333 Sun Jul 28 08:32:13 implement sparse data accessor (#7055)

good 46db233 ivanmorozov333 Mon Jul 29 07:24:00 additional test and addressing fixes (#7168)
(+0) 123a335 ivanmorozov333 Mon Jul 29 14:33:37 fix compaction intervals construction (#7176)
good 13b5991 ivanmorozov333 Mon Jul 29 16:40:28 tx operations start unification (#7178)
good 33bdce5 ivanmorozov333 Mon Jul 29 17:48:17 asymmetric access for sorted key on find position (#7177)
good 75ab2b2 ivanmorozov333 Tue Jul 30 07:01:57 improve performace for stat requests (sysview) (#7211)
good 49a8f16 ivanmorozov333 Tue Jul 30 13:58:38 speed up merging with correct pointers operation (#7230)
good a87900c ivanmorozov333 Tue Jul 30 13:58:54 speed up tables manager in schemeshard (#7229)
(+!-) ee419db ivanmorozov333 Tue Jul 30 17:03:46 fix segfault in case normalization different versions (#7239)
(+0) 74bba08 ivanmorozov333 Wed Jul 31 11:52:15 fix leak on release internal local tx operation (#7280)
good 407a550 ivanmorozov333 Wed Jul 31 16:40:34 fix pk checker (#7266)
good 88d40e9 ivanmorozov333 Wed Jul 31 17:10:46 correct not null flags usage in tests (#7293)
good 40be822 ivanmorozov333 Wed Jul 31 21:16:06 signals for tablet initialization (#7281)

fix_unused: see #7330 and run tests

85d4495 Ivan Jul 15 15:29:47 [KIKIMR-21472] Output the final AST in explain query (#6487)
good ff5d894 ivanmorozov333 Wed Jul 31 21:28:48 switch on aggregations pushdown into CS (#7305)

(+!-) ae8c7f9 ivanmorozov333 Thu Aug 1 06:51:24 fix empty column usage (#7334)
(conflict) b1c5908 ivanmorozov333 Thu Aug 1 11:55:47 fix columns normalization for absent (#7340)
good 6b1225c zverevgeny Thu Aug 1 15:53:05 Move new flag to env from config for compatibility with musty arc recipy (#7272)
good c0c6ab2 Semyon Thu Aug 1 23:42:52 add missing table stats for column shards (#7140)
good d34dcf3 zverevgeny Fri Aug 2 11:34:44 fix clickbench with CS (#7393)
good d4149ad zverevgeny Sun Aug 4 10:00:20 move test TShardReader to a common place (#7441)
good 899ccd7 ivanmorozov333 Sun Aug 4 13:24:09 Overlimit read memory (#7408)
good f319dde ivanmorozov333 Sun Aug 4 13:56:43 make local internal CS persistent snapshot pinger (#7431)
good a1f1ecc ivanmorozov333 Mon Aug 5 11:47:37 Blob memory interval limit (#7444)
conflict 7d65202 Artem Alekseev Mon Aug 5 16:22:49 Fix KqpOlapAggregations.Aggregation_ResultCountAll_Filter (#7423)
good 58580f8 Semyon Tue Aug 6 14:53:20 add tx progress counters (#7407)
good 7ad9e1e Semyon Wed Aug 7 17:07:35 add missing monitoring metrics for column shards (#7314)
good 9dbab08 ivanmorozov333 Thu Aug 8 11:24:19 fix segfault in deserializer stream usage (#7542)
!!!check spaces good 78d1439 Artem Alekseev Thu Aug 8 18:09:29 Add simple stability test for olap queries (#7543)
(+!-) b16060e zverevgeny Fri Aug 9 08:48:32 Delete empty portions normalizer (#7596)
good c59279a Artem Alekseev Fri Aug 9 15:51:38 Fix integer overflow in TTimeSliceLogic::DoCalcWeight (#7606)
conflict a2e8760 ivanmorozov333 Mon Aug 12 09:44:16 change memory planner for CS scan (#7372)
good ef2fc0f zverevgeny Mon Aug 12 19:26:23 delete non-existing ut from muted (#7672)
good 3734eda ivanmorozov333 Tue Aug 13 19:31:50 different request variants need different memory profiles (#7664)
(+!-) 555ebf3 ivanmorozov333 Tue Aug 13 19:32:43 improve chunks splitter performance (#7665)
ignore 63c9b91 ildar-khisambeev Wed Aug 14 16:43:49 add status processing (#6609)
good cb4e391 zverevgeny Wed Aug 14 22:44:30 unmute incorrectly muted olap tests (#7680)
(+0) 009d3c9 ivanmorozov333 Thu Aug 15 12:31:42 scan error processing on restore data (#7813)
good 3d60e1e ivanmorozov333 Thu Aug 15 15:32:47 fix snapshot tracing signals (#7821)
good 752095e ivanmorozov333 Thu Aug 15 16:11:33 fix snapshot control in case sys view requests (#7824)
(+!-) 1159d25 ivanmorozov333 Thu Aug 15 18:06:20 Optional columns feature flags (#7814)
good 83b9579 ivanmorozov333 Fri Aug 16 07:08:28 dont use cpu for not abortable chunks (#7865)
(+0) 8796c8a ivanmorozov333 Fri Aug 16 11:09:10 fix portion kind modification (#7868)
good ac738f9 Semyon Fri Aug 16 14:44:00 Show RuntimeFeatures on portion_stats sys-view (#7673)
(+0) 04beb79 ivanmorozov333 Fri Aug 16 15:36:39 replace as default write modification type (#7894)
good f6476d4 ivanmorozov333 Fri Aug 16 16:08:29 speed up compilation for request processing (#7900)
good c68f701 Artem Alekseev Fri Aug 16 16:34:33 Fix kikimr_runner for ubuntu 20.04 (#7895)
(+0) af37b14 ivanmorozov333 Fri Aug 16 16:56:43 additional tx signals (#7876)
(+0) 7b528af ivanmorozov333 Fri Aug 16 22:21:46 Fix chunks scanner (#7913)
(+0) c03e8b2 ivanmorozov333 Sat Aug 17 07:50:57 speed up filters construction (#7934)
(+0) b524a82 ivanmorozov333 Sun Aug 18 09:29:45 timeout for long_tx operations (#7961)
good 5522a96 ivanmorozov333 Mon Aug 19 07:52:55 fix race on scan start with indexation cleaning blobs (#7968)
good 4226868 ivanmorozov333 Mon Aug 19 13:05:19 refactoring for ranges control (#7758)
good a61fa8d Semyon Tue Aug 20 17:10:31 Add a scenario test for altering tiering config (#7636)
<-- cs progress 3

ignore b4533a2 Artem Alekseev Tue Aug 20 17:19:31 Add olap_workload to stability_tests (#8047)
ignore 7ab6040 zverevgeny Wed Aug 21 00:01:01 Simplify tpc-ds q34, q73 a bit to help an optimizer (#7969)
good f2788b5 ivanmorozov333 Thu Aug 22 07:58:10 multi heads allocations manager (#8126)
good c1c1c23 Semyon Thu Aug 22 10:48:52 Add unit-test for data eviction in tiering (#7971)
good bb1c803 ivanmorozov333 Thu Aug 22 13:07:13 correct portions index usage for control memory scan intervals (#8135)
good a5de48c Semyon Thu Aug 22 13:08:17 Don't skip null values on COUNT(*) in column shards (#8024)
good baf6413 Artem Alekseev Thu Aug 22 13:56:25 Add scan/insert while alter test (#8094)
good 3e3ffd1 Semyon Thu Aug 22 13:58:29 Add column shard Controller parameters to AppData (#8000)
good b4a5ff5 Alexander Avdonkin Fri Aug 23 11:51:02 Added sparsed columns tests for standalone tables (#8167)
good c249b27 Semyon Fri Aug 23 15:54:32 Forbid creating empty tiering rules in column shards (#8154)
ignore 78bb468 ildar-khisambeev Fri Aug 23 18:39:09 count-min-sketch as index (#7978)
good b5dba78 ivanmorozov333 Sat Aug 24 08:15:09 dont create temporary sparsed array on merging (#8174)
ignore d425c83 ildar-khisambeev Mon Aug 26 14:20:36 Revert add status PROCESSING to statistics proto (#8253)
ignore 1c09c5d Semyon Tue Aug 27 14:57:32 Add concurrent tiering updates to OLAP scenario test (#8262)
ignore 14d8e1b ildar-khisambeev Tue Aug 27 17:25:34 fix tests (#8323)
good f5de1a7 ivanmorozov333 Wed Aug 28 19:02:58 speed up merging for sparsed columns (#8365)
good 8d6b42b ivanmorozov333 Thu Aug 29 12:17:43 signals about long_tx writing (#8407)
good bf8c633 Semyon Thu Aug 29 17:21:13 Fix table stats aggregation for OLAP tables (#8417)
good 297d5b4 Alexander Avdonkin Fri Aug 30 09:48:56 Multicolumn sparsed test (#8284)
good 684bdae ivanmorozov333 Sun Sep 1 11:58:21 validations for prevent incorrect data in local db (#8477)
conflict f4b2c61 Semyon Sun Sep 1 19:05:25 Add unit test: drop then add column with different type (#8446)

(локи)conflict 1f29b7c (из stable-24-3-analytics) Mon Sep 9 18:48:34 [Stable 24 3 8 analytics] EvWrite & CTAS (#8861) Nikita Vasilev

(локи)conflict 1af55df ivanmorozov333 Mon Sep 2 12:26:39 tx interactions manager (#8050)
<--cs_progress_5
pacify py linter 33195d6

good 0532ed6 Artem Alekseev Mon Sep 2 13:00:37 Add registrator to TTxChainSession (#8535)
good 68720d1 ivanmorozov333 Mon Sep 2 14:54:55 Fix column livecycle (#8580)
(локи)good 5f2c357 ivanmorozov333 Mon Sep 2 14:55:23 provide correct path id to lock info (#8591)
good 892228d ivanmorozov333 Mon Sep 2 18:32:30 fix race on enqueue tx_commit in case long synchronization (#8588)
(локи)good b8d1289 ivanmorozov333 Mon Sep 2 18:32:39 remove validation throught available reask in timeout (#8609)
(локи)good 6fca8c1 ivanmorozov333 Mon Sep 2 19:27:00 we have to notify cs at first to add lock information (#8614)
good 4a4dbe2 Artem Alekseev Tue Sep 3 10:58:31 Add UpsertWhileSplitTest (#8472)
(локи)conflict b666aa2 ivanmorozov333 Wed Sep 4 09:30:05 immediate writing with no tx (#8697)
good 6180869 ivanmorozov333 Wed Sep 4 10:58:06 reduce logging rate for stats info (#8683)
good a29e93b ivanmorozov333 Wed Sep 4 11:07:56 switch on mkql factory for cs (#8682)
good 766ee48 Semyon Wed Sep 4 12:01:31 Fix partition count in column table description (#8589)
conflict 65e10f5 Semyon Wed Sep 4 12:02:58 remove blob columns not present in current scheme
good 53df48f ivanmorozov333 Wed Sep 4 12:05:24 use ss tablet id for schemeshard and sender for tx_proxy (#8684)
good 15da817 ivanmorozov333 Wed Sep 4 14:52:23 Fix broken txs (#8714)
good ba7abc4 ivanmorozov333 Wed Sep 4 15:00:26 fix case indexation removed table (#8713)
good 2fb4cac ivanmorozov333 Wed Sep 4 16:00:59 split factor increased to avoid tests interaction by timeout (#8728)
good 0ba24ba ivanmorozov333 Wed Sep 4 17:23:24 additional signals for commit timings control (#8729)
conflict f9b4a7d Artem Alekseev Wed Sep 4 18:25:23 Add EnableAlterSharding option (#8707)
conflict dc3cf10 Semyon Thu Sep 5 15:44:02 add feature flag: enable olap tiering (#8719)
good 0923723 ivanmorozov333 Thu Sep 5 18:13:34 fix import build (#8769)
good 1257b2f Semyon Fri Sep 6 15:50:18 add missing enum value to win-specific code (#8854)
ignore 3ca8730 Artem Alekseev Fri Sep 6 16:06:26 Fix Date32 json serialization for column store (#8848)

c6d4d1b Nikita Vasilev Sat Sep 7 19:39:15 2024 +0300 Olap TableStore read/write test (#8858)

conflict c0d60de ivanmorozov333 Sun Sep 8 14:04:25 remove table only for empty insert table (#8822)
good 0ab0d7a ivanmorozov333 Mon Sgiep 9 09:50:46 Fix write ids usage (#8909)
good e546733 ivanmorozov333 Mon Sep 9 10:29:25 speed up column ids checkers and finder (#8906)
ignore 68c8f3f Nikita Vasilev Mon Sep 9 11:48:12 2024 +0300 Fix locks tests (#8733)
good 2ab0fd8 Artem Alekseev Mon Sep 9 16:02:54 Add PARTITIONS_COUNT setting to yql (#7602)
good 4efd77c Alexander Avdonkin Mon Sep 9 21:48:46 Implemented limit on olap table columnt count (#8742)
good 7001202 ivanmorozov333 Tue Sep 10 12:17:22 skip long tx writing to in control locks (#8924)
conflict 26c9911 Artem Alekseev Tue Sep 10 17:25:20 Fix PARTITION_COUNT option name and add tests (#9012)
ignore 4238641 zverevgeny Tue Sep 10 19:37:02 CODEOWNERS CS (#9025)
conflict 189aa4b Vladislav Gogov Tue Sep 10 22:12:56 Add feature flag: enable olap compression (#8726)
conflict 9b1e630 ivanmorozov333 Wed Sep 11 11:17:51 Normalizer insert records (#9010)
(локи)good 618f5cf ivanmorozov333 Wed Sep 11 11:18:55 correct EvWrite locks processing (#8931)
conflict 7d436f8 Artem Alekseev Wed Sep 11 11:39:12 Add decimal tests for columnshard (#8959)

<-- 4f8fa3d cs_progress_6 tiny diff with main 7d436f8 in ydb/core/tx/columnshard/ ydb/core/formats/

ignore 71c4756 Nikita Vasilev Wed Sep 11 11:39:47 2024 +0300 HTAP: Query Processor (#8917)
ignore 99bd033 Vladislav Gogov Wed Sep 11 15:15:46 2024 +0300 Added test with disabled feature flag "EnableOlapCompression" (#9077)
good 3db740c Nikita Vasilev Wed Sep 11 15:22:43 2024 +0300 Disable CTAS by default (#9057)
good 658b27b ivanmorozov333 Thu Sep 12 12:11:11 2024 +0300 speed up logging exp (#9087)
conflict ca964ba Alexander Avdonkin Thu Sep 12 14:04:41 2024 +0300 Sparsed columns are disabled by default and can be optionally enabled (#9002)
good 8585364 ivanmorozov333 Thu Sep 12 16:37:13 2024 +0300 speed up scan initialization (#9086)
ignore 67cacf0 ivanmorozov333 Thu Sep 12 20:07:19 2024 +0300 htap locks (#9121)
ignore 8fca2df Artem Alekseev Fri Sep 13 14:19:46 2024 +0300 Fix verify in TSharingTransactionOperator::DoParse (#9158)
ignore acee834 Nikita Vasilev Fri Sep 13 15:37:03 2024 +0300 Fix sink index column order (#9190)
itnore 35e326d Nikita Vasilev Fri Sep 13 15:45:17 2024 +0300 HTAP: single arbiter (#9199)
ignore 1960e5f Nikita Vasilev Fri Sep 13 18:16:02 2024 +0300 Smaller memory footprint for shards info (#9207)
e3c9723 zverevgeny Sun Sep 15 09:20:22 2024 +0300 Turn on GCCountersNormalizer (#8166)

@zverevgeny zverevgeny linked a pull request Oct 15, 2024 that will close this issue
@zverevgeny zverevgeny self-assigned this Oct 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant