Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

increase insert xxx select xxx speed #452

Closed
wants to merge 1 commit into from

Conversation

csuopen
Copy link

@csuopen csuopen commented Nov 14, 2023

when increase polar_bulk_read_size to 64, but "insert xxx select" not getting faster, because ring_size is low,so increase ring_size in function GetAccessStrategy.

test:

postgres=# reset polar_bulk_read_size;
RESET
Time: 0.200 ms
postgres=# show polar_bulk_read_size;
 polar_bulk_read_size
----------------------
 128kB
(1 row)

Time: 0.216 ms
postgres=# select count(*) from test2;
   count
-----------
 200000000
(1 row)

Time: 96621.768 ms (01:36.622)

postgres=# set polar_bulk_read_size to 64;
SET
postgres=# set max_parallel_workers_per_gather to 0;
SET
postgres=# \timing
Timing is on.
postgres=# select count(*) from test2;
   count
-----------
 200000000
(1 row)

Time: 73271.417 ms (01:13.271)

postgres=# set max_parallel_workers_per_gather to 8;
SET
Time: 0.173 ms
postgres=# explain select count(*) from test2;
                                         QUERY PLAN
---------------------------------------------------------------------------------------------
 Finalize Aggregate  (cost=9009314.43..9009314.44 rows=1 width=8)
   ->  Gather  (cost=9009313.60..9009314.41 rows=8 width=8)
         Workers Planned: 8
         ->  Partial Aggregate  (cost=9008313.60..9008313.61 rows=1 width=8)
               ->  Parallel Seq Scan on test2  (cost=0.00..8945812.48 rows=25000448 width=0)
(5 rows)

Time: 0.454 ms
postgres=# select count(*) from test2;
   count
-----------
 200000000
(1 row)

Time: 92410.190 ms (01:32.410)

postgres=# set polar_bulk_read_size to 64;
SET
Time: 0.168 ms
postgres=# show polar_bulk_read_size;
 polar_bulk_read_size
----------------------
 512kB
(1 row)

Time: 0.154 ms
postgres=# select count(*) from test2;
   count
-----------
 200000000
(1 row)

Time: 75722.028 ms (01:15.722)

@polardb-bot
Copy link

polardb-bot bot commented Nov 14, 2023

Hi @csuopen ~ Congratulations to your first PR to PolarDB. 🎉

Please make sure that your PR conforms the standard, and has passed all the checks.

We will review your PR as soon as possible.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@polardb-bot
Copy link

polardb-bot bot commented Nov 14, 2023

Hey @csuopen :

Congratulations~ 🎉 Your commit has passed all the checks. Please wait for further manual review.

@polardb-bot polardb-bot bot added the ci/success CI status is success label Nov 14, 2023
@osdba osdba deleted the csudev branch November 20, 2023 09:14
@csuopen
Copy link
Author

csuopen commented Nov 24, 2023

mistake something, PR should not be to ApsaraDB:POLARDB_11_STABLE, need to DEV branch. so close is, and create new pr to dev branch.

@csuopen csuopen closed this Nov 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci/success CI status is success
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants