ValueError: Invalid severity value 'severity'

[jukefr]

Not sure what I am doing wrong sorry if this is a bad issue but I'm not all that familiar with python and enums...

I made a config based on the sample one:

blocklist_instance_sources = []

blocklist_url_sources = [
  { url = 'https://thebad.space/exports/mastodon', format = 'csv' },
]

allowlist_url_sources = []

blocklist_instance_destinations = [{ domain = 'REDACTED', token = 'REDACTED' }]

save_intermediate = true
savedir = './tmp'
blocklist_savefile = './tmp/merged_blocklist.csv'
blocklist_auditfile = './tmp/domain_counts_list.csv'

mergeplan = 'min'

import_fields = [
  'public_comment',
  'reject_media',
  'reject_reports',
  'obfuscate',
]

export_fields = ['public_comment']

Basically want to sync my instance blocks with thebadspace, using the "minimum" severity because I already imported the csv via the mastodon interface but it automatically suspends everything.
But I get this error:

2023-09-10 18:07:37,878 INFO Fetching domain blocks from URLs...
Traceback (most recent call last):
  File "/home/user/.local/bin/fediblock-sync", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/user/.local/pipx/venvs/fediblockhole/lib/python3.11/site-packages/fediblockhole/__init__.py", line 773, in main
    sync_blocklists(args)
  File "/home/user/.local/pipx/venvs/fediblockhole/lib/python3.11/site-packages/fediblockhole/__init__.py", line 65, in sync_blocklists
    blocklists.extend(fetch_from_urls(conf.blocklist_url_sources,
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/fediblockhole/lib/python3.11/site-packages/fediblockhole/__init__.py", line 147, in fetch_from_urls
    bl = parse_blocklist(rawdata, url, listformat, import_fields, max_severity)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/fediblockhole/lib/python3.11/site-packages/fediblockhole/blocklists.py", line 264, in parse_blocklist
    return parser.parse_blocklist(blockdata, origin)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/fediblockhole/lib/python3.11/site-packages/fediblockhole/blocklists.py", line 72, in parse_blocklist
    block = self.parse_item(blockitem)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/fediblockhole/lib/python3.11/site-packages/fediblockhole/blocklists.py", line 159, in parse_item
    block = DomainBlock(**blockitem)
            ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/fediblockhole/lib/python3.11/site-packages/fediblockhole/const.py", line 131, in __init__
    self.severity = severity
    ^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/fediblockhole/lib/python3.11/site-packages/fediblockhole/const.py", line 148, in severity
    self._severity = BlockSeverity(sev)
                     ^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/fediblockhole/lib/python3.11/site-packages/fediblockhole/const.py", line 25, in __init__
    self._level = self.str2level(severity)
                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/fediblockhole/lib/python3.11/site-packages/fediblockhole/const.py", line 53, in str2level
    raise ValueError(f"Invalid severity value '{severity}'")
ValueError: Invalid severity value 'severity'

I am not exactly sure what I am doing wrong here

I also tried downloading and specifying the path with file:///... and removing the first line from the export which i thought was the issue but then I get this error:

2023-09-10 18:15:04,465 INFO Fetching domain blocks from URLs...
Traceback (most recent call last):
  File "/home/user/.local/bin/fediblock-sync", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/user/.local/pipx/venvs/fediblockhole/lib/python3.11/site-packages/fediblockhole/__init__.py", line 773, in main
    sync_blocklists(args)
  File "/home/user/.local/pipx/venvs/fediblockhole/lib/python3.11/site-packages/fediblockhole/__init__.py", line 65, in sync_blocklists
    blocklists.extend(fetch_from_urls(conf.blocklist_url_sources,
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/fediblockhole/lib/python3.11/site-packages/fediblockhole/__init__.py", line 147, in fetch_from_urls
    bl = parse_blocklist(rawdata, url, listformat, import_fields, max_severity)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/fediblockhole/lib/python3.11/site-packages/fediblockhole/blocklists.py", line 264, in parse_blocklist
    return parser.parse_blocklist(blockdata, origin)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/fediblockhole/lib/python3.11/site-packages/fediblockhole/blocklists.py", line 72, in parse_blocklist
    block = self.parse_item(blockitem)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/fediblockhole/lib/python3.11/site-packages/fediblockhole/blocklists.py", line 159, in parse_item
    block = DomainBlock(**blockitem)
            ^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: DomainBlock.__init__() missing 1 required positional argument: 'domain'

I'm not sure exactly what is going wrong the csv list imports fine on Mastodon backend so I was guessing that the format of it is "correct" and what is expected but I'm not sure.

I also tried setting the format to mastodon_csv but same issues I am really unsure what I am doing wrong

[jpwarren]

The first line of the CSV file is important, because it tells the parser which columns are which fields. That'll be why you get the TypeError when you're using the file method.

The ValueError of your first method indicates that the severity column is maybe parsing the header line twice, or maybe there's a row with severity as the severity level, not one of the supported severity codes.

Can you share a copy of the blocklist you're trying to import? That'll help figure out what's going on here.

Looks like some more helpful error messages are needed. These are a bit confusing and weird for newcomers.

[jukefr]

Hey thank you for the very fast response!
It was indeed a duplicate "header" line (domain, severity etc) at midway through the file (~line 2049, a simple search for "severity" revealed it pretty fast i did not think of that 😅).
The export is the url in the config file provided at the top of the issue if you want it (https://thebad.space/exports/mastodon)

[jpwarren]

Glad you sorted it out!

Still worth us looking at improving the errors, and/or ignoring non-fatal ones as mentioned in #48.

And helpful to have this here in case other people encounter the same problem.