Skip to content

Commit

Permalink
Merge pull request #29 from Moonbase59/master
Browse files Browse the repository at this point in the history
Add changes in master to v5.0.0-dev
  • Loading branch information
Moonbase59 authored Jul 1, 2024
2 parents de5c010 + 2734438 commit 922aa6e
Show file tree
Hide file tree
Showing 9 changed files with 152 additions and 50 deletions.
2 changes: 1 addition & 1 deletion .github/FUNDING.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# These are supported funding model platforms

custom: ["https://www.paypal.me/Moonbase59"]
custom: ["https://www.paypal.com/donate/?hosted_button_id=PBPR63362LDEU"]
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# autocue changelog

### 2024-07-01 – v4.0.4

- Allow to override results via JSON even _after_ having done a fresh analysis (automatic or forced), for ultimate flexibility when using `cue_file` for pre-processing. You can now add fades, ramp or hook points, or do other calculations and feed the results into `cue_file` for tagging. **Use with care**, because some values are dependent on others. In any case, `cue_file` will ever _only_ write tags beginning with `liq_` and (if requested) `replaygain_`.
- Prevent some strange errors that could happen when piping something into `cue_file` and JSON input was not `stdin`. We now use `ffmpeg -nostdin` to prevent it reading input that was meant for `cue_file`.
- Don’t write _all_ `liq_*` tags (could have side effects), but only those _known_ (see `cue_file --help` for current list).
- Streamlined tag conversion code a little.

### 2024-06-18 – v4.0.3

- Changed default of `-x`/`--extra` and `settings.autocue.cue_file.overlay_longtail` from `-15.0` LU to `-12.0` LU, requested by @RM-FM and the community. Together with the `-d`/`--drop` default change from `60.0` to `40.0`, this makes for a "tighter" playout and doesn’t lose too much of long or sustained endings.
Expand Down
66 changes: 64 additions & 2 deletions FAQ.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
date: 2024-06-23
date: 2024-07-01
author: Matthias C. Hormann (Moonbase59)
---
# FAQ – Frequently Asked Questions
Expand All @@ -16,6 +16,7 @@ author: Matthias C. Hormann (Moonbase59)
- [How to pre-process more than one file at a time ("mass tagging")?](#how-to-pre-process-more-than-one-file-at-a-time-mass-tagging)
- [What tagging software to use?](#what-tagging-software-to-use)
- [Can I use `cue_file` to replaygain my files?](#can-i-use-cue_file-to-replaygain-my-files)
- [Can I use `cue_file` to _manually_ add/overwrite tags when pre-processing?](#can-i-use-cue_file-to-manually-addoverwrite-tags-when-pre-processing)
- [How to make transitions _tighter_, i.e. overlay earlier?](#how-to-make-transitions-tighter-ie-overlay-earlier)
- [How to make transitions _longer_, i.e. overlay later and keep every bit of a song ending?](#how-to-make-transitions-longer-ie-overlay-later-and-keep-every-bit-of-a-song-ending)
- [Can I completely _disable_ the "sustained endings" feature?](#can-i-completely-disable-the-sustained-endings-feature)
Expand All @@ -25,6 +26,7 @@ author: Matthias C. Hormann (Moonbase59)
- [Why should I use `check_autocue_setup`, and what does it do?](#why-should-i-use-check_autocue_setup-and-what-does-it-do)
- [What are good first steps to use Autocue in my own Liquidsoap script?](#what-are-good-first-steps-to-use-autocue-in-my-own-liquidsoap-script)
- [What will happen when I switch to the upcoming Liquidsoap 2.3.0?](#what-will-happen-when-i-switch-to-the-upcoming-liquidsoap-230)
- [What can I do if some files or a remote playlist simply _don’t play?_](#what-can-i-do-if-some-files-or-a-remote-playlist-simply-dont-play)
<!-- Generated by gh-toc, https://moonbase59.github.io/gh-toc/ -->
<!-- ToC end -->

Expand Down Expand Up @@ -155,6 +157,36 @@ As always, you should _know what you’re doing_, and set up these tools appropr
|replaygain_track_range|dB|


## <a name="can-i-use-cue_file-to-manually-addoverwrite-tags-when-pre-processing"></a>Can I use `cue_file` to _manually_ add/overwrite tags when pre-processing? <a href="#toc" class="goToc">⇧</a>

- _Yes_, you can, even _after_ forcing a re-analysis (v4.0.4+).
- _Only_ tags from `cue_file`’s list of _known_ tags (see `cue_file --help`) will ever been written by `cue_file`. So no overriding artist or title here—that’s what should have been done in an earlier step, using a good tagging software.
- **Use with care!** Some values are dependent on others, you could easily mess up something.
- You **must** create _well-formed JSON_ and can then use `cue_file` with the `-j`/`--json` switch to let your tags override or add to what’s already there.

#### Example: Adding fade-in and fade-out, using `echo` and `stdin`

```bash
echo '{"liq_fade_in": 0.1, "liq_fade_out": 0.1}' | cue_file -j - -fwr "filename.ext"
```
- `-j -` — sets JSON input to `stdin`
- `-fwr`_force_ re-analysis, _write_ tags, write _replaygain_

#### Example: using a JSON file `fades.json`

```json
{
"liq_fade_in": 0.10,
"liq_fade_out": 0.10
}
```
```bash
cue_file -j fades.json -fwr "filename.ext"
```
- `-j fades.json` — read JSON data from file `fades.json`
- `-fwr`_force_ re-analysis, _write_ tags, write _replaygain_


## <a name="how-to-make-transitions-tighter-ie-overlay-earlier"></a>How to make transitions _tighter_, i.e. overlay earlier? <a href="#toc" class="goToc">⇧</a>

- First, try to _decrease_ `-d`/`--drop`/`settings.autocue.cue_file.sustained_loudness_drop` _gradually_. The default is `40.0`%, so maybe go down in 10% increments and see how you like it.
Expand Down Expand Up @@ -307,7 +339,7 @@ Nothing is definite yet, Liquidsoap 2.3.0 is still under heavy development.

Most certainly, as of 2024-06-23…

- you will need a _new version_ 5.x.x of both `autocue.cue_file` and `cue_file`, because Liquidsoap will change the tags and API.
- you will need a _new version_ of both `autocue.cue_file` and `cue_file`, because Liquidsoap will change the tags and API.
- you’ll need to _pre-process your files again_, if you have used that feature.
- I will see that `cue_file` will be able to _remove obsolete tags_.
- if possible, I’ll do some _checking_ so that you don’t run `autocue.cue_file` (and thus `cue_file`) under an incompatible Liquidsoap version. The `check_autocue_setup()` function will take care of that.
Expand All @@ -322,3 +354,33 @@ Most certainly, as of 2024-06-23…
See also:

- https://github.com/savonet/liquidsoap/pull/3931#issuecomment-2184937173


## <a name="what-can-i-do-if-some-files-or-a-remote-playlist-simply-dont-play"></a>What can I do if some files or a remote playlist simply _don’t play?_ <a href="#toc" class="goToc">⇧</a>

This most often happens with _large files_, _large remote files_, and files that are _hard to analyse_ because they’re already clipping (too loud).

Check your Liquidsoap log for entries like

```
2024/06/26 10:01:43 [request:2] Time limit exceeded by 16.00 secs!
```

This can happen if the _downloading/preparing is slow_, or the _files are large_ (recordings, prerecorded shows, DJ sets, podcasts) or _hard to analyse_. These factors are out of our control. If a request timeout happens, the file will simply _not be played_, and the _next available used_.

In such cases, I suggest to _pre-tag_ files using the `cue_file` tool, but this might not always be possible.

In the above case, I’d suggest _increasing the Autocue timeout slightly_ from the default `60.0` seconds to `60` + timeout shown (`16.0`) + a small "safety margin", say another `10` seconds, making for `86` seconds. Let’s round that up to `90` seconds.

In AzuraCast’s _Edit Liquidsoap Configuration_, in the _second input box_, add this parameter:
```
settings.autocue.cue_file.timeout := 90.0
```

**Note:** Please do _not_ use arbitrary large values like `480.0` "just to prevent it happening"! Doing so could well have adverse side effects.

_Save Changes_ and _Restart Broadcasting_.

Keep watching your logs for timeouts, just in case even larger files arrive. You can search for `Time limit exceeded` to quickly find such entries in the log.

If the above should _not_ help, you should look for a _faster/higher bandwidth connection_ to the source you download the files from. To be able to _work_ with a file, Liquidsoap has to download a complete copy first, before autocueing and playout can begin.
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# <a name="autocue"></a>autocue <a href="#toc" class="goToc">⇧</a>

**If you like what you got, please consider to [![Donate with Paypal](https://www.paypalobjects.com/en_US/i/btn/btn_donate_LG.gif)](https://www.paypal.com/donate/?hosted_button_id=PBPR63362LDEU). Thank you! ❤️**

On-the-fly JSON song cue-in, cue-out, overlay, replaygain calculation for Liquidsoap, AzuraCast and other AutoDJ software.

**Check out the [presentation](https://moonbase59.github.io/autocue/presentation/autocue.html) for an introduction, the [FAQ – Frequently Asked Questions](FAQ.md), and the [changelog](CHANGELOG.md) for latest information!**
Expand Down
1 change: 1 addition & 0 deletions autocue.cue_file.liq
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
# 2024-06-16 - Moonbase59 - v4.0.2 - Allow `-8.33dB` type values with no blank
# 2024-06-18 - Moonbase59 - v4.0.3 - Changed overlay_longtail from -15 to -12,
# most people seem to want transitions a bit tighter
# 2024-07-01 - Moonbase59 - v4.0.4 - Sync with cue_file version
# 2024-07-01 - Moonbase59 - v5.0.0-dev - New JSON API

# Lots of debugging output for AzuraCast in this, will be removed eventually.
Expand Down
112 changes: 69 additions & 43 deletions cue_file
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,11 @@
# for slightly tighter/denser playout (community wish)
# 2024-06-18 Moonbase59 - v4.0.3 Change LONGTAIL_EXTRA_LU from -15 to -12,
# most people seem to want transitions a bit tighter
# 2024-07-01 Moonbase59 - v4.0.4 Fix JSON override after analysis
# - Add `-nostdin` to ffmpeg commands, prevents strange
# errors when piping something to cue_file
# - streamline tag conversion code a little
# - only write known tags, not all `liq_*`
# 2024-07-01 Moonbase59 - v5.0.0-dev
# - New more compatible JSON output
#
Expand Down Expand Up @@ -252,6 +257,56 @@ def amplify_correct(target, loudness, true_peak_dB, noclip):
return amplify, amplify_correction


# remove " dB", " LU", " dBFS", " dBTP" and " LUFS" suffixes from
# tags_found
def remove_suffix(tags):
suffixed_tags = [
"liq_amplify", "liq_amplify_adjustment",
"liq_loudness", "liq_loudness_range", "liq_reference_loudness",
"replaygain_track_gain", "replaygain_track_range",
"replaygain_reference_loudness",
"liq_true_peak_db",
"liq_true_peak", # in case old " dBFS" values were stored in v1.2.3
]
for tag in suffixed_tags:
if tag in tags and isinstance(tags[tag], str):
# No need to check for unit name, only using defined tags
m = re.search(r'([+-]?\d*\.?\d+)', tags[tag])
if m is not None:
tags[tag] = m.group()

return tags


# convert tags into their typed variants, ready for calculations
def convert_tags(tags):
items = tags.items()

# make keys lowercase, include only tags in tags_to_check
tags = {
k.lower(): v for k,
v in items if k.lower() in tags_to_check}

# remove suffixes from several tags
tags = remove_suffix(tags)

# convert tag string values to the correct types, listed in tags_to_check
tags = {k: tags_to_check[k](v) for k, v in tags.items()}

return tags

# override a (typed) result with JSON overrides
def override_from_JSON(tags, tags_json={}):
# get tags in JSON override file
tags_in_json = convert_tags(tags_json)

# unify, right overwrites left if key in both
# tags_found = tags_in_stream | tags_in_format | tags_in_json
tags = {**tags, **tags_in_json}

return tags


def read_tags(
filename,
tags_json={},
Expand Down Expand Up @@ -284,28 +339,20 @@ def read_tags(

# get tags in stream #0 (mka, opus, etc.)
try:
stream_items = result['streams'][0]['tags'].items()
stream_tags = result['streams'][0]['tags']
except KeyError:
stream_items = {}
stream_tags = {}

# get tags in format (flac, mp3, etc.)
try:
format_items = result['format']['tags'].items()
format_tags = result['format']['tags']
except KeyError:
format_items = {}
format_tags = {}

# get tags in JSON override file
json_items = tags_json.items()
tags_in_stream = convert_tags(stream_tags)
tags_in_format = convert_tags(format_tags)
tags_in_json = convert_tags(tags_json)

tags_in_stream = {
k.lower(): v for k,
v in stream_items if k.lower() in tags_to_check}
tags_in_format = {
k.lower(): v for k,
v in format_items if k.lower() in tags_to_check}
tags_in_json = {
k.lower(): v for k,
v in json_items if k.lower() in tags_to_check}
# unify, right overwrites left if key in both
# tags_found = tags_in_stream | tags_in_format | tags_in_json
tags_found = {**tags_in_stream, **tags_in_format, **tags_in_json}
Expand All @@ -322,29 +369,6 @@ def read_tags(
except KeyError:
pass

# remove " dB", " LU", " dBFS", " dBTP" and " LUFS" suffixes from
# tags_found
def remove_suffix(tags):
suffixed_tags = [
"liq_amplify", "liq_amplify_adjustment",
"liq_loudness", "liq_loudness_range", "liq_reference_loudness",
"replaygain_track_gain", "replaygain_track_range",
"replaygain_reference_loudness",
"liq_true_peak_db",
"liq_true_peak", # in case old " dBFS" values were stored in v1.2.3
]
for tag in suffixed_tags:
if tag in tags and isinstance(tags[tag], str):
# No need to check for unit name, only using defined tags
m = re.search(r'([+-]?\d*\.?\d+)', tags[tag])
if m is not None:
tags[tag] = m.group()

return tags

# remove suffixes from several tags
tags_found = remove_suffix(tags_found)

# create replaygain_track_gain from Opus R128_TRACK_GAIN (ref: -23 LUFS)
if "r128_track_gain" in tags_found:
rg = float(tags_found["r128_track_gain"]) / 256 + (target - -23.0)
Expand All @@ -355,9 +379,6 @@ def read_tags(
"replaygain_track_gain" in tags_found):
tags_found["liq_amplify"] = tags_found["replaygain_track_gain"]

# convert tag string values to the correct types, listed in tags_to_check
tags_found = {k: tags_to_check[k](v) for k, v in tags_found.items()}

# Handle old RG1/mp3gain positive loudness reference
# "89 dB" (SPL) should actually be -14 LUFS, but as a reference
# it is usually set equal to the RG2 -18 LUFS reference point
Expand Down Expand Up @@ -447,7 +468,7 @@ def add_missing(tags_found, target=TARGET_LUFS, blankskip=0.0, noclip=False):
tags_found["liq_amplify"] = tags_found["replaygain_track_gain"]

if "liq_amplify_adjustment" not in tags_found:
tags_found["liq_amplify_adjustment"] = "0.00 dB"
tags_found["liq_amplify_adjustment"] = 0.00 # dB

if "liq_loudness" not in tags_found:
tags_found["liq_loudness"] = target - \
Expand Down Expand Up @@ -503,6 +524,7 @@ def analyse(
FFMPEG,
"-v",
"quiet",
"-nostdin",
"-y",
"-i",
filename,
Expand Down Expand Up @@ -843,7 +865,7 @@ def write_tags(filename, tags={}, replaygain=False):
# copy only `liq_*`, float with 2 decimals, bools and strings lowercase
tags_new = {k: "{:.2f}".format(v)
if isinstance(v, float) else str(v).lower()
for k, v in tags.items() if k.startswith("liq_") or k in rg_tags
for k, v in tags.items() if k in tags_to_check or k in rg_tags
}
# liq_true_peak & replaygain_track_peak have 6 decimals, fix it
if "liq_true_peak" in tags_new:
Expand Down Expand Up @@ -957,6 +979,7 @@ def write_tags(filename, tags={}, replaygain=False):
args = [
FFMPEG,
'-v', 'quiet',
'-nostdin',
'-y',
'-i', str(filename.absolute()),
'-map_metadata', '0',
Expand Down Expand Up @@ -1199,6 +1222,9 @@ if args.force or not skip_analysis:
nice=args.nice,
noclip=args.noclip
)
# allow to override even the analysis results
if args.json:
result = override_from_JSON(result, tags_json)
else:
result = add_missing(tags_found, args.target, args.blankskip, args.noclip)

Expand Down
7 changes: 5 additions & 2 deletions docs/presentation/autocue.html
Original file line number Diff line number Diff line change
Expand Up @@ -959,8 +959,11 @@ <h1>Links</h1>
<section id="thanks" class="title-slide slide level1">
<h1>Thanks!</h1>
<p><img data-src="images/matthias.png" /><br />
Matthias C. Hormann<br />
(“Moonbase59”)</p>
Matthias C. Hormann (“Moonbase59”)</p>
<p><strong>If you like what you got, please consider to <a
href="https://www.paypal.com/donate/?hosted_button_id=PBPR63362LDEU"><img
data-src="https://www.paypalobjects.com/en_US/i/btn/btn_donate_LG.gif"
alt="Donate with Paypal" /></a>. Thank you! ❤️</strong></p>
</section>
</div>
</div>
Expand Down
5 changes: 3 additions & 2 deletions docs/presentation/autocue.md
Original file line number Diff line number Diff line change
Expand Up @@ -745,5 +745,6 @@ GitHub repo:
# Thanks!

![ ](images/matthias.png)
Matthias C. Hormann
(“Moonbase59”)
Matthias C. Hormann (“Moonbase59”)

**If you like what you got, please consider to [![Donate with Paypal](https://www.paypalobjects.com/en_US/i/btn/btn_donate_LG.gif)](https://www.paypal.com/donate/?hosted_button_id=PBPR63362LDEU). Thank you! ❤️**
Binary file modified docs/presentation/autocue.pdf
Binary file not shown.

0 comments on commit 922aa6e

Please sign in to comment.