fw-addr-lists script hangs

[ackstorm23]

Although it appeared to be working at first install, I've noticed the fw-addr-lists script I enabled never completes anymore. It hangs a long time, and then timeouts.

How do I debug what's going on so I can get to the bottom of it?

[eworm-de]

First of all, can you share your configuration, please?

For large lists it takes quite some time to complete.

[ackstorm23]

This is my overlay. Nothing else from the default config has been changed.

# Overlay for global configuration by RouterOS Scripts
# Copyright (c) 2013-2024 Christian Hesse <mail@eworm.de>
# https://git.eworm.de/cgit/routeros-scripts/about/COPYING.md
#
# global configuration, custom overlay
# https://git.eworm.de/cgit/routeros-scripts/about/#editing-configuration

# Copy relevant configuration from global-config, paste and modify it here.
# https://git.eworm.de/cgit/routeros-scripts/about/global-config.rsc

# This defines the settings for firewall address-lists (fw-addr-lists).
:global FwAddrLists {
#  "allow"={
#    { url="https://git.eworm.de/cgit/routeros-scripts/plain/fw-addr-lists.d/allow";
#      cert="E1"; timeout=1w };
#  };
  "block"={
#    { url="https://git.eworm.de/cgit/routeros-scripts/plain/fw-addr-lists.d/block";
#      cert="E1" };
    { url="https://feodotracker.abuse.ch/downloads/ipblocklist_recommended.txt";
      cert="GlobalSign Atlas R3 DV TLS CA 2022 Q3" };
    { url="https://sslbl.abuse.ch/blacklist/sslipblacklist.txt";
      cert="GlobalSign Atlas R3 DV TLS CA 2022 Q3" };
    { url="https://www.dshield.org/block.txt"; cidr="/24";
      cert="R3" };
    { url="https://lists.blocklist.de/lists/strongips.txt";
      cert="Certum Domain Validation CA SHA2" };
#    { url="https://www.spamhaus.org/drop/drop.txt";
#      cert="Cloudflare Inc ECC CA-3" };
#    { url="https://www.spamhaus.org/drop/edrop.txt";
#      cert="Cloudflare Inc ECC CA-3" };
  };
#  "mikrotik"={
#    { url="https://git.eworm.de/cgit/routeros-scripts/plain/fw-addr-lists.d/mikrot>
#      cert="E1"; timeout=1w };
#  };
};
:global FwAddrListTimeOut 1d;

The script just runs forever, eventually timing out the next day.

[eworm-de]

Hmm, this is just copied, but no additional lists added, no? All devices should be capable of handling that just fine. (BTW, after latest update you have to update certificate names, or just copy over again.)

Possibly a more general problem... Please give the output of:

:put [ $IsFullyConnected ];
:put [ $IsDefaultRouteReachable ];
:put [ $IsDNSResolving ];
:put [ $IsTimeSync ];

[ackstorm23]

Correct, I only copy and pasted the config over with the intend that i might make changes later, but never got around to it.

It ran correctly the first time, as I saw it populate the list and saw firewall blocks for it's rules. But that is not true anymore.

I ran the commands, it looks as though it's entirely broken.

[admin@MikroTik] > :put [ $IsFullyConnected ];

[admin@MikroTik] > :put [ $IsDefaultRouteReachable ];

[admin@MikroTik] > :put [ $IsDNSResolving ];

[admin@MikroTik] > :put [ $IsTimeSync ];

I'll re-run the install steps and see it the problem resolves itself.

[ackstorm23]

fetch if cert works fine

[admin@MikroTik] > /tool/fetch "https://git.eworm.de/cgit/routeros-scripts/plain/certs/ISRG-Root-X2.pem" dst-path="ISRG-Root-X2.pem";
      status: finished
  downloaded: 1KiBC-z pause]
    duration: 1s

nothing new gets imported as the cert already exists

(/certificate/import file-name=ISRG-Root-X2.pem passphrase=""; which isn't printed)
[admin@MikroTik] >
     certificates-imported: 0
     private-keys-imported: 0
            files-imported: 0
       decryption-failures: 0
  keys-with-no-certificate: 0

fingerprint

[admin@MikroTik] > /certificate/set name="ISRG-Root-X2" [ find where common-name="ISRG Root X2" ];
[admin@MikroTik] > /certificate/print proplist=name,fingerprint where fingerprint="69729b8e15a86efc177a57afb7171dfc64add28c2fca8cf1507e34453ccb1470";
Flags: T - TRUSTED
Columns: NAME, FINGERPRINT
#   NAME          FINGERPRINT
0 T ISRG-Root-X2  69729b8e15a86efc177a57afb7171dfc64add28c2fca8cf1507e34453ccb1470
[admin@MikroTik] > /file/remove [ find where name="ISRG-Root-X2.pem" ];

grabbing script failures because it already exists

[admin@MikroTik] > :foreach Script in={ "global-config"; "global-config-overlay"; "global-functions" } do={ /system/script/add name=$Script owner=$Script source=([ /tool/fetch check-certificate=yes-without-crl ("https://git.eworm.de/cgit/routeros-scripts/plain/" . $Script . ".rsc") output=user as-value]->"data"); };
failure: item with such name already exists

loading it doesn't throw any errors

[admin@MikroTik] > /system/script { run global-config; run global-functions; };
info: Loaded on RB5009UPr+S+ with RouterOS 7.16beta2 (testing).

Interestingly the startup script add does NOT error, but I remember seeing that startup script in my scheduler before...hm...

[admin@MikroTik] /system/script> /system/scheduler/add name="global-scripts" start-time=startup on-event="/system/script { run global-config; run global-functions; }";

adding the job for updating the script failures because it already exists

[admin@MikroTik] /system/script> /system/scheduler/add name="ScriptInstallUpdate" start-time=startup interval=1d on-event=":global ScriptInstallUpdate; \$ScriptInstallUpdate;";
failure: item with this name already exists

now it's working again.

[admin@MikroTik] /system/script> /system/script/run fw-addr-lists
info: list: block (436) -- added: 0 - renewed: 436 - removed: 0     

It would seem as though the start-up job never added properly, or was removed or broken in some way. If I run into this again, I will check for that problem first.

[eworm-de]

Thanks for detailed explanation, and have fun with the scripts!