v4.0.4

Moonbase59 · Jul 1, 2024 · 2561bc5 · 2561bc5
1 parent 2136277
commit 2561bc5
Show file tree

Hide file tree

Showing 4 changed files with 112 additions and 47 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,12 @@
 # autocue changelog
 
+### 2024-07-01 – v4.0.4
+
+- Allow to override results via JSON even _after_ having done a fresh analysis (automatic or forced), for ultimate flexibility when using `cue_file` for pre-processing. You can now add fades, ramp or hook points, or do other calculations and feed the results into `cue_file` for tagging. **Use with care**, because some values are dependent on others. In any case, `cue_file` will ever _only_ write tags beginning with `liq_` and (if requested) `replaygain_`.
+- Prevent some strange errors that could happen when piping something into `cue_file` and JSON input was not `stdin`. We now use `ffmpeg -nostdin` to prevent it reading input that was meant for `cue_file`.
+- Don’t write _all_ `liq_*` tags (could have side effects), but only those _known_ (see `cue_file --help` for current list).
+- Streamlined tag conversion code a little.
+
 ### 2024-06-18 – v4.0.3
 
 - Changed default of `-x`/`--extra` and `settings.autocue.cue_file.overlay_longtail` from `-15.0` LU to `-12.0` LU, requested by @RM-FM and the community. Together with the `-d`/`--drop` default change from `60.0` to `40.0`, this makes for a "tighter" playout and doesn’t lose too much of long or sustained endings.

diff --git a/FAQ.md b/FAQ.md
@@ -1,5 +1,5 @@
 ---
-date: 2024-06-26
+date: 2024-07-01
 author: Matthias C. Hormann (Moonbase59)
 ---
 # FAQ – Frequently Asked Questions
@@ -16,6 +16,7 @@ author: Matthias C. Hormann (Moonbase59)
 - [How to pre-process more than one file at a time ("mass tagging")?](#how-to-pre-process-more-than-one-file-at-a-time-mass-tagging)
 - [What tagging software to use?](#what-tagging-software-to-use)
 - [Can I use `cue_file` to replaygain my files?](#can-i-use-cue_file-to-replaygain-my-files)
+- [Can I use `cue_file` to _manually_ add/overwrite tags when pre-processing?](#can-i-use-cue_file-to-manually-addoverwrite-tags-when-pre-processing)
 - [How to make transitions _tighter_, i.e. overlay earlier?](#how-to-make-transitions-tighter-ie-overlay-earlier)
 - [How to make transitions _longer_, i.e. overlay later and keep every bit of a song ending?](#how-to-make-transitions-longer-ie-overlay-later-and-keep-every-bit-of-a-song-ending)
 - [Can I completely _disable_ the "sustained endings" feature?](#can-i-completely-disable-the-sustained-endings-feature)
@@ -156,6 +157,36 @@ As always, you should _know what you’re doing_, and set up these tools appropr
   |replaygain_track_range|dB|
 
 
+## <a name="can-i-use-cue_file-to-manually-addoverwrite-tags-when-pre-processing"></a>Can I use `cue_file` to _manually_ add/overwrite tags when pre-processing? <a href="#toc" class="goToc">⇧</a>
+
+- _Yes_, you can, even _after_ forcing a re-analysis (v4.0.4+).
+- _Only_ tags from `cue_file`’s list of _known_ tags (see `cue_file --help`) will ever been written by `cue_file`. So no overriding artist or title here—that’s what should have been done in an earlier step, using a good tagging software.
+- **Use with care!** Some values are dependent on others, you could easily mess up something.
+- You **must** create _well-formed JSON_ and can then use `cue_file` with the `-j`/`--json` switch to let your tags override or add to what’s already there.
+
+#### Example: Adding fade-in and fade-out, using `echo` and `stdin`
+
+```bash
+echo '{"liq_fade_in": 0.1, "liq_fade_out": 0.1}' | cue_file -j - -fwr "filename.ext"
+```
+- `-j -` — sets JSON input to `stdin`
+- `-fwr` — _force_ re-analysis, _write_ tags, write _replaygain_
+
+#### Example: using a JSON file `fades.json`
+
+```json
+{
+  "liq_fade_in": 0.10,
+  "liq_fade_out": 0.10
+}
+```
+```bash
+cue_file -j fades.json -fwr "filename.ext"
+```
+- `-j fades.json` — read JSON data from file `fades.json`
+- `-fwr` — _force_ re-analysis, _write_ tags, write _replaygain_
+
+
 ## <a name="how-to-make-transitions-tighter-ie-overlay-earlier"></a>How to make transitions _tighter_, i.e. overlay earlier? <a href="#toc" class="goToc">⇧</a>
 
 - First, try to _decrease_ `-d`/`--drop`/`settings.autocue.cue_file.sustained_loudness_drop` _gradually_. The default is `40.0`%, so maybe go down in 10% increments and see how you like it.
@@ -308,7 +339,7 @@ Nothing is definite yet, Liquidsoap 2.3.0 is still under heavy development.
 
 Most certainly, as of 2024-06-23…
 
-- you will need a _new version_ 5.x.x of both `autocue.cue_file` and `cue_file`, because Liquidsoap will change the tags and API.
+- you will need a _new version_ of both `autocue.cue_file` and `cue_file`, because Liquidsoap will change the tags and API.
 - you’ll need to _pre-process your files again_, if you have used that feature.
 - I will see that `cue_file` will be able to _remove obsolete tags_.
 - if possible, I’ll do some _checking_ so that you don’t run `autocue.cue_file` (and thus `cue_file`) under an incompatible Liquidsoap version. The `check_autocue_setup()` function will take care of that.

diff --git a/autocue.cue_file.liq b/autocue.cue_file.liq
@@ -37,6 +37,7 @@
 # 2024-06-16 - Moonbase59 - v4.0.2 - Allow `-8.33dB` type values with no blank
 # 2024-06-18 - Moonbase59 - v4.0.3 - Changed overlay_longtail from -15 to -12,
 #                           most people seem to want transitions a bit tighter
+# 2024-07-01 - Moonbase59 - v4.0.4 - Sync with cue_file version
 
 # Lots of debugging output for AzuraCast in this, will be removed eventually.
 
@@ -50,7 +51,7 @@ let settings.autocue.cue_file.version =
   settings.make(
     description=
       "Software version of autocue.cue_file. Should coincide with `cue_file`.",
-    "4.0.3"
+    "4.0.4"
   )
 
 # Internal only! Not a user setting.

diff --git a/cue_file b/cue_file
@@ -58,13 +58,18 @@
 #                         for slightly tighter/denser playout (community wish)
 # 2024-06-18 Moonbase59 - v4.0.3 Change LONGTAIL_EXTRA_LU from -15 to -12,
 #                         most people seem to want transitions a bit tighter
+# 2024-07-01 Moonbase59 - v4.0.4 Fix JSON override after analysis
+#                       - Add `-nostdin` to ffmpeg commands, prevents strange
+#                         errors when piping something to cue_file
+#                       - streamline tag conversion code a little
+#                       - only write known tags, not all `liq_*`
 #
 # Originally based on an idea and some code by John Warburton (@Warblefly):
 #   https://github.com/Warblefly/TrackBoundaries
 # Some collaborative work with RM-FM (@RM-FM): Sustained ending analysis.
 
 __author__ = 'Matthias C. Hormann'
-__version__ = '4.0.3'
+__version__ = '4.0.4'
 
 import os
 import sys
@@ -250,6 +255,56 @@ def amplify_correct(target, loudness, true_peak_dB, noclip):
     return amplify, amplify_correction
 
 
+# remove " dB", " LU", " dBFS", " dBTP" and " LUFS" suffixes from
+# tags_found
+def remove_suffix(tags):
+    suffixed_tags = [
+        "liq_amplify", "liq_amplify_adjustment",
+        "liq_loudness", "liq_loudness_range", "liq_reference_loudness",
+        "replaygain_track_gain", "replaygain_track_range",
+        "replaygain_reference_loudness",
+        "liq_true_peak_db",
+        "liq_true_peak",  # in case old " dBFS" values were stored in v1.2.3
+    ]
+    for tag in suffixed_tags:
+        if tag in tags and isinstance(tags[tag], str):
+            # No need to check for unit name, only using defined tags
+            m = re.search(r'([+-]?\d*\.?\d+)', tags[tag])
+            if m is not None:
+                tags[tag] = m.group()
+
+    return tags
+
+
+# convert tags into their typed variants, ready for calculations
+def convert_tags(tags):
+    items = tags.items()
+
+    # make keys lowercase, include only tags in tags_to_check
+    tags = {
+        k.lower(): v for k,
+        v in items if k.lower() in tags_to_check}
+
+    # remove suffixes from several tags
+    tags = remove_suffix(tags)
+
+    # convert tag string values to the correct types, listed in tags_to_check
+    tags = {k: tags_to_check[k](v) for k, v in tags.items()}
+
+    return tags
+
+# override a (typed) result with JSON overrides
+def override_from_JSON(tags, tags_json={}):
+    # get tags in JSON override file
+    tags_in_json = convert_tags(tags_json)
+
+    # unify, right overwrites left if key in both
+    # tags_found = tags_in_stream | tags_in_format | tags_in_json
+    tags = {**tags, **tags_in_json}
+
+    return tags
+
+
 def read_tags(
         filename,
         tags_json={},
@@ -282,28 +337,20 @@ def read_tags(
 
     # get tags in stream #0 (mka, opus, etc.)
     try:
-        stream_items = result['streams'][0]['tags'].items()
+        stream_tags = result['streams'][0]['tags']
     except KeyError:
-        stream_items = {}
+        stream_tags = {}
 
     # get tags in format (flac, mp3, etc.)
     try:
-        format_items = result['format']['tags'].items()
+        format_tags = result['format']['tags']
     except KeyError:
-        format_items = {}
+        format_tags = {}
 
-    # get tags in JSON override file
-    json_items = tags_json.items()
+    tags_in_stream = convert_tags(stream_tags)
+    tags_in_format = convert_tags(format_tags)
+    tags_in_json = convert_tags(tags_json)
 
-    tags_in_stream = {
-        k.lower(): v for k,
-        v in stream_items if k.lower() in tags_to_check}
-    tags_in_format = {
-        k.lower(): v for k,
-        v in format_items if k.lower() in tags_to_check}
-    tags_in_json = {
-        k.lower(): v for k,
-        v in json_items if k.lower() in tags_to_check}
     # unify, right overwrites left if key in both
     # tags_found = tags_in_stream | tags_in_format | tags_in_json
     tags_found = {**tags_in_stream, **tags_in_format, **tags_in_json}
@@ -320,29 +367,6 @@ def read_tags(
         except KeyError:
             pass
 
-    # remove " dB", " LU", " dBFS", " dBTP" and " LUFS" suffixes from
-    # tags_found
-    def remove_suffix(tags):
-        suffixed_tags = [
-            "liq_amplify", "liq_amplify_adjustment",
-            "liq_loudness", "liq_loudness_range", "liq_reference_loudness",
-            "replaygain_track_gain", "replaygain_track_range",
-            "replaygain_reference_loudness",
-            "liq_true_peak_db",
-            "liq_true_peak",  # in case old " dBFS" values were stored in v1.2.3
-        ]
-        for tag in suffixed_tags:
-            if tag in tags and isinstance(tags[tag], str):
-                # No need to check for unit name, only using defined tags
-                m = re.search(r'([+-]?\d*\.?\d+)', tags[tag])
-                if m is not None:
-                    tags[tag] = m.group()
-
-        return tags
-
-    # remove suffixes from several tags
-    tags_found = remove_suffix(tags_found)
-
     # create replaygain_track_gain from Opus R128_TRACK_GAIN (ref: -23 LUFS)
     if "r128_track_gain" in tags_found:
         rg = float(tags_found["r128_track_gain"]) / 256 + (target - -23.0)
@@ -353,9 +377,6 @@ def read_tags(
             "replaygain_track_gain" in tags_found):
         tags_found["liq_amplify"] = tags_found["replaygain_track_gain"]
 
-    # convert tag string values to the correct types, listed in tags_to_check
-    tags_found = {k: tags_to_check[k](v) for k, v in tags_found.items()}
-
     # Handle old RG1/mp3gain positive loudness reference
     # "89 dB" (SPL) should actually be -14 LUFS, but as a reference
     # it is usually set equal to the RG2 -18 LUFS reference point
@@ -445,7 +466,7 @@ def add_missing(tags_found, target=TARGET_LUFS, blankskip=0.0, noclip=False):
         tags_found["liq_amplify"] = tags_found["replaygain_track_gain"]
 
     if "liq_amplify_adjustment" not in tags_found:
-        tags_found["liq_amplify_adjustment"] = "0.00 dB"
+        tags_found["liq_amplify_adjustment"] = 0.00  # dB
 
     if "liq_loudness" not in tags_found:
         tags_found["liq_loudness"] = target - \
@@ -501,6 +522,7 @@ def analyse(
         FFMPEG,
         "-v",
         "quiet",
+        "-nostdin",
         "-y",
         "-i",
         filename,
@@ -841,7 +863,7 @@ def write_tags(filename, tags={}, replaygain=False):
         # copy only `liq_*`, float with 2 decimals, bools and strings lowercase
         tags_new = {k: "{:.2f}".format(v)
                     if isinstance(v, float) else str(v).lower()
-                    for k, v in tags.items() if k.startswith("liq_") or k in rg_tags
+                    for k, v in tags.items() if k in tags_to_check or k in rg_tags
                     }
         # liq_true_peak & replaygain_track_peak have 6 decimals, fix it
         if "liq_true_peak" in tags_new:
@@ -955,6 +977,7 @@ def write_tags(filename, tags={}, replaygain=False):
             args = [
                 FFMPEG,
                 '-v', 'quiet',
+                '-nostdin',
                 '-y',
                 '-i', str(filename.absolute()),
                 '-map_metadata', '0',
@@ -1197,6 +1220,9 @@ if args.force or not skip_analysis:
         nice=args.nice,
         noclip=args.noclip
     )
+    # allow to override even the analysis results
+    if args.json:
+        result = override_from_JSON(result, tags_json)
 else:
     result = add_missing(tags_found, args.target, args.blankskip, args.noclip)