-
Notifications
You must be signed in to change notification settings - Fork 2
using sbarbett.pihole.manage_lists with large blocklists takes really long #3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Hi Marlena, There's a reason for the delay, as you can see here: https://github.com/sbarbett/pihole-ansible/blob/main/roles/manage_lists/tasks/process_list.yml When I was developing the module and role for managing local DNS records, I ran into some issues with the PiHole API that, at least at a glance, appeared to be rate limiting. To work around this, I added a 1-second delay between each transaction. When I later added the role for managing lists, I copied that logic under the assumption that similar issues might arise. However, I haven’t actually tested it with a large number of lists. Can you try running your list updates with the following section in - name: Pause for 1 second
command: sleep 1
changed_when: false
no_log: true If omitting the delay doesn’t cause any issues, feel free to submit a PR with it removed, and I’ll merge it. You could also try something shorter, like 100ms. Thanks, |
Hey, already tested that, it speeds up around 30% (2 instead of 3seconds) but still very slow. I started bulding a mix of your and my solution here: https://github.com/muller2002/pihole-ansible/blob/main/plugins/modules/bulk_list_update.py Please let me know, if you have any thougts about this |
This will lead to a lot faster updates, fixes sbarbett#3
After some consideration, I revised the lower module and implemented your approach in v1.1.0. Eliminating the unnecessary API calls should considerably reduce the time it takes to run. |
Hey,
when trying out version 1.0.5 i wanted to use the new sbarbett.pihole.manage_lists.
I have a large collection of ~200 Blocklists. Each blocklist takes around 3 seconds to update, so around 10 minutes per host.
The solution proposed in #2 was faster (around 30seconds per run).
I propose to add a new module, which does the handling of bulk updates and change the role to just call this module for every pihole
The text was updated successfully, but these errors were encountered: