Skip to content

Commit

Permalink
[bug] use single-quotes for string literals in sqlite queries
Browse files Browse the repository at this point in the history
Historically, sqlite accepts double-quoted string literals, which is in
contravention of the sql standard (which says single quotes should be
used for string literals, and double quotes for identifiers like column
names).

The sqlite authors consider this a misfeature, and are making efforts to
change it:
- Since sqlite 3.27.0 (2019-02-07), using double-quotes for a string
  literal causes a warning to be printed to the error log.
- Since sqlite 3.29.0 (2019-07-10), the sqlite authors recommend
  compiling sqlite with the -DSQLITE_DQS=0 option, which causes use of
  double quotes for string literals to be an error.

More details here: https://www.sqlite.org/quirks.html#double_quoted_string_literals_are_accepted

My sqlite is compiled with this recommended option, so prior to this
change, running, eg, `beet ls`, produced this error:

    Traceback (most recent call last):
      File "/usr/local/bin/beet", line 33, in <module>
        sys.exit(load_entry_point('beets==1.6.0', 'console_scripts', 'beet')())
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/beets/ui/__init__.py", line 1285, in main
        _raw_main(args)
      File "/usr/local/lib/python3.11/site-packages/beets/ui/__init__.py", line 1272, in _raw_main
        subcommand.func(lib, suboptions, subargs)
      File "/usr/local/lib/python3.11/site-packages/beets/ui/commands.py", line 1089, in list_func
        list_items(lib, decargs(args), opts.album)
      File "/usr/local/lib/python3.11/site-packages/beets/ui/commands.py", line 1084, in list_items
        for item in lib.items(query):
                    ^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/beets/library.py", line 1529, in items
        return self._fetch(Item, query, sort or self.get_default_item_sort())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/beets/library.py", line 1503, in _fetch
        return super()._fetch(
               ^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/beets/dbcore/db.py", line 1094, in _fetch
        rows = tx.query(sql, subvals)
               ^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/beets/dbcore/db.py", line 859, in query
        cursor = self.db._connection().execute(statement, subvals)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    sqlite3.OperationalError: no such column: "" - should this be a string literal in single-quotes?

Making the appropriate change to library.py allowed `beet ls` to
continue to this error:

    Traceback (most recent call last):
      File "/usr/local/bin/beet", line 33, in <module>
        sys.exit(load_entry_point('beets==1.6.0', 'console_scripts', 'beet')())
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/beets/ui/__init__.py", line 1285, in main
        _raw_main(args)
      File "/usr/local/lib/python3.11/site-packages/beets/ui/__init__.py", line 1272, in _raw_main
        subcommand.func(lib, suboptions, subargs)
      File "/usr/local/lib/python3.11/site-packages/beets/ui/commands.py", line 1089, in list_func
        list_items(lib, decargs(args), opts.album)
      File "/usr/local/lib/python3.11/site-packages/beets/ui/commands.py", line 1084, in list_items
        for item in lib.items(query):
                    ^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/beets/library.py", line 1529, in items
        return self._fetch(Item, query, sort or self.get_default_item_sort())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/beets/library.py", line 1503, in _fetch
        return super()._fetch(
               ^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/beets/dbcore/db.py", line 1093, in _fetch
        rows = tx.query(sql, subvals)
               ^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/beets/dbcore/db.py", line 858, in query
        cursor = self.db._connection().execute(statement, subvals)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    sqlite3.OperationalError: no such column: "text" - should this be a string literal in single-quotes?

The change to query.py allowed `beet ls` to succeed.

As for testing, I frankly don't think it's worthwhile to build an
automated test harness for beets that recompiles sqlite differently. I
think it's sufficient in this case to observe that the existing tests
pass. I've been running with this patch for 6 months or so, and haven't
had any issues.
  • Loading branch information
amonks committed Sep 13, 2024
1 parent 0a985fd commit c716905
Show file tree
Hide file tree
Showing 3 changed files with 4 additions and 3 deletions.
4 changes: 2 additions & 2 deletions beets/dbcore/query.py
Original file line number Diff line number Diff line change
Expand Up @@ -1040,8 +1040,8 @@ def order_clause(self) -> str:
if self.case_insensitive:
field = (
"(CASE "
'WHEN TYPEOF({0})="text" THEN LOWER({0}) '
'WHEN TYPEOF({0})="blob" THEN LOWER({0}) '
'WHEN TYPEOF({0})=\'text\' THEN LOWER({0}) '
'WHEN TYPEOF({0})=\'blob\' THEN LOWER({0}) '
"ELSE {0} END)".format(self.field)
)
else:
Expand Down
2 changes: 1 addition & 1 deletion beets/library.py
Original file line number Diff line number Diff line change
Expand Up @@ -313,7 +313,7 @@ def order_clause(self):
collate = "COLLATE NOCASE" if self.case_insensitive else ""
return (
"(CASE {0}_sort WHEN NULL THEN {0} "
'WHEN "" THEN {0} '
'WHEN \'\' THEN {0} '
"ELSE {0}_sort END) {1} {2}"
).format(field, collate, order)

Expand Down
1 change: 1 addition & 0 deletions docs/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ Bug fixes:
as a numpy array. Update ``librosa`` dependency constraint to prevent similar
issues in the future.
:bug:`5289`
* Fixed changed double to single quotes in two queries to work with DSQLITE_DQS=0.

For packagers:

Expand Down

0 comments on commit c716905

Please sign in to comment.