[API Request] Add bitDepth and samplingRate to songs

[GioF71]

Type of change

API extension

Proposal description

For the endpoints which return Songs, along with the existing "bitRate" field, it would be nice to also have extra fields for bitDepth and samplingRate.

• bitDepth could be an interger with values: 8, 16, 24, 32 and maybe 0 for lossy format (it has no meaning in this case AFAIU). Or, if necessary to return something different for lossy formats, the data type might be a string instead of an integer
• samplingRate should be an integer. Possible values would be 44100, 48000, etc.

Backward compatibility impact

There should be no backward compatibility impact as there is no change to existing fields.

Backward compatibility

• No backward compatibility impact.

API details

The change should extend to all the endpoint which return Songs, directly or for album and/or playlists.
Simply two more fields with bitDepth and sampleRate should be added in order to carry the additional detail about the song (as a media file).

Security impacts

None

Potential issues

None

Alternative solutions

There is no alternative solution, because on the client side, it is not granted that we can have access to the files and that we even have the full path.
Also even if it was the case, having to access the file in order to get those information would be very expensive and it would be preferable to include in the server scan phase.

[dweymouth]

Similarly, I think we should add codec as well. Right now, the extension and MIME type are exposed which both do not tell the client what the codec is, since they can be ambiguous (eg .m4a and audio/mp4 can be either AAC or ALAC) Though it might be more work in the scanner for servers to detect these fields so there may be some pushback

[Tolriq]

New fields are optional so no real pushback if at least one server want to support this.
But yes codec is important information specially when we'll start to improve the transcoding part.

[epoupon]

Btw, what about format? Is it important for clients or is it all about codec? (and is the current "suffix" field enough?)

[Tolriq]

By format you mean container ? In theory suffix and mimetype is enough. But we can expose all data as others including format subtypes everything needed for transcoding décisions.

[epoupon]

Yes I meant container: I already hit a .opus file which was actually a webm or mastroska container (don't remember), and not a ogg one. Not sure if this is legit or not, I have not checked, hence the question.

[dweymouth]

EDIT: this list conflated codec and container and should not be used

For codec, I propose we include a list of well-known codec values to help coordinate expectations between client and server for what the values mean. I also propose that, since the stream endpoint already uses short-form codec strings as a de-facto standard, that we go with short codec strings. I'll propose this list to start with, but we can tweak it through discussion if needed.

• mp3: standard MPEG-layer 3 audio file
• wav: standard uncompressed WAV format audio
• aiff: standard uncompressed AIFF format audio
• flac: FLAC audio in a standard FLAC file container
• opus: Opus format audio in an OGG container (usually .opus file extension)
• vorbis: Vorbis format audio in an OGG container
• aac: Advanced Audio Coding (AAC) audio in a MP4 container (usual extension .m4a). Could be any profile of AAC (AAC-LC, HE-AAC, etc)
• alac: Apple Lossless Audio Codec in a MP4 container (usual extension .m4a)
• wma: Windows Media Audio (lossy)
• ape: Monkey's Audio
• wavpack: Wavpack lossless audio
• dsf: DSD (Direct Stream Digital) audio in a DSF file (usual extension .dsf)
• ac3: Dolby Digital (AC3) audio
• eac3: Extended AC3 format (Dolby DIgital Plus). Commonly used to carry Dolby Atmos audio
• mpc Musepack format (uncommon)
• oggflac: FLAC audio in an OGG container (uncommon)
• shn: Shorten format (usually .shn extension) (uncommon)
• wmal: Windows Media Audio Lossless (uncommon)

[Tolriq]

You mix codec and container this is not the way.
Codec are actual audio codec, not container.

We can expose both if mimetype is not enough for the container, but certainly not mix them in arbitrary values.

[dweymouth]

Yes, we can go that way. If we do that the new stream endpoint extension needs to support clients supplying a list of codecs and a list of containers that they can accept for playback. I was thinking that, since in audio, with few exceptions there is typically only one container format that is used for a given codec, we could roll the two together into a single "format" concept, if we were explicit about exactly what a format string was calling for.

[Tolriq]

You mix codec as information about the media and params for transcoding. They are not related even if can be commonly defined.

[dweymouth]

Alright, I'll give a second go at a list of well-known codec values

[dweymouth]

Alright, second try, since Tolriq pointed out that we probably don't want to conflate codec and container:

• lpcm: uncompressed linear PCM audio (commonly .wav and .aiff)
• mp3: MP3
• aac: Advanced Audio Coding (any profile: HE-AAC, AAC-LC, etc)
• opus: Opus codec
• vorbis: Vorbis codec
• alac: Apple Lossless Audio Codec
• wma: Windows Media Audio lossy codec
• wmal: Windows Media Audio lossless codec (uncommon)
• ape: Monkey's Audio
• wavpack: Wavpack (lossless) codec
• ac3: AC-3 (aka Dolby Digital) codec
• eac3: Enhanced AC-3 (aka Dolby Digital Plus)
• dsd: Direct Stream Digital (DSD) audio
• dca: DTS Coherent Acoustics aka DTS
• shorten: Shorten codec (uncommon)
• musepack: Musepack codec (uncommon)
• speex: Speex codec (uncommon)
• atrac: ATRAC, any profile [usually ATRAC3] (uncommon)

[Tolriq]

BTW we should also have a unknown value, as empty would be not available. Unknown means check the full codec name if you understand it.

[dweymouth]

To avoid payload size bloat, would we want to have only the short codec value as a property on subsonic.Child, and introduce a new API for retrieving the full format + codec info for a given track if the client needs it?

[Tolriq]

Well for my need I'd say no :) I use and store all the data and doing 1 query per song does not really work.

Future endpoints will have field selection so the payload size won't be a major issue.

[dweymouth]

Future endpoints will have field selection so the payload size won't be a major issue.

Fair point, though in this case maybe I would propose holding off on adding long-form codec/container info to every track until we have a field selection API in place

[Tolriq]

I would not as it would allow clients to already only request transcoding when it knows it does not know the full codec.

As a compromise we can define that the fullCodec / container should only be returned in the getSong end point and not in the generic Child anwer.

But as said before extracting the shortCodec or the long, sample rate, ... should be done in one scan by the server, already defining how everything is handled from the start is better for both servers and clients.

I do not want to have to full rescan 15TB of songs each time we add something related to data extraction, if we can avoid it.

[GioF71]

EDIT: moved to the replies of the previous entry

[dweymouth]

Probably in its own page that's linked to from the doc for Child (ie song). But I don't think this list is finalized yet

[GioF71]

Hello guys, would it be helpful if we reduce the scope of the discussion to just bitDepth and samplingRate, and discuss the outstanding topics in another discussion?
Maybe this might help some changes to go through more rapidly.

About samplingRate, it was mentioned here that we did not agree on setting "full" values or not.
I am proposing to use an integer value which leaves the client the possibility to display the information as preferred without having to remap a string which would contain a value like "44.1" or "44".
So for example for 44.1kHz the field "samplingRate" would have the value of 44100 as an integer.
Of course this is just my opinion, if you prefer a contracted string, I will make the change.

Thank you.

[Tolriq]

As said a couple of times, all the values are related and extracted at the same time by the servers, defining them all together makes every one gain time in implementing the extraction and storage of the data. From server to users that do not have to rescan all their files.

There is no race to add fields, it took me a very very long time and tons of efforts to bring the project where it is, it won't disappear, but yes things moves slowly.

[GioF71]

Ok @Tolriq, no rush then.