MusicBrainz JSON-LD: Retrieval and Data Structure

[malajvan]

MusicBrainz provide access to their json-ld files.

Curl Command

To access data from the MusicBrainz JSON-LD API, you can use the following curl command:

curl -H 'Accept: application/ld+json' -o <output_file> <musicbrainz_link>

Entities of Interest

• There are several entities available in MusicBrainz, including release, release group, event, and artist.

• Most of the relevant information can be found in the release entity, especially when dealing with albums.

Release Entity Example

Here is an example of JSON-LD data for a release:

{
  "@type": "MusicRelease",
  "catalogNumber": "PH 2005",
  "@context": "https://schema.org/docs/jsonldcontext.json",
  "@id": "http://musicbrainz.org/release/20027ad4-6667-49b6-bd07-5ac12d57f8dd",
  "creditedTo": "Johnny McGreevy & Séamus Cooley",
  "sameAs": "https://www.amazon.com/gp/product/B001RF74KG",
  "releaseOf": {
    "@type": "MusicAlbum",
    "byArtist": [
      {
        "@id": "http://musicbrainz.org/artist/de8f8c4b-a768-460f-ae9e-1ca8bbaf426f",
        "@type": ["Person", "MusicGroup"],
        "name": "Johnny McGreevy"
      },
      {
        "name": "Séamus Cooley",
        "@type": ["Person", "MusicGroup"],
        "@id": "http://musicbrainz.org/artist/5439eef7-2d99-4024-9b21-482a4c8550b3"
      }
    ],
    "track": [
      {
        "@id": "http://musicbrainz.org/recording/d506fa4b-733e-4864-8b41-29c5c510024e",
        "@type": "MusicRecording",
        "trackNumber": "1.1",
        "name": "The Broken Pledge / Julia Delaney (reels)"
      },
      {
        "trackNumber": "1.2",
        "@id": "http://musicbrainz.org/recording/b59c6d66-e6c1-4902-a673-953f84c17577",
        "@type": "MusicRecording",
        "name": "Seán sa Ceo / Michael Preston's (reels)"
       .....
      }
    ]
  }
}

Track Information

When examining specific tracks within a release, the information largely remains the same as in the elements in track from the previous JSON-LD.

      {
        "@id": "http://musicbrainz.org/recording/d506fa4b-733e-4864-8b41-29c5c510024e",
        "@type": "MusicRecording",
        "trackNumber": "1.1",
        "name": "The Broken Pledge / Julia Delaney (reels)"
      }

Release Group Entity

See the difference between the concepts of release and release group here.

Another entity of interest is the release group, which has less detailed information than the release entity. It doesn't include track, region, release format, and other information. However, it has links to other databases.

Here is an example of the "sameAs" field in a release group:

"sameAs": [
  "https://thesession.org/recordings/2070",
  "https://www.irishtune.info/album/McGrCool/"
]

Sometimes they even have links to wikidata

Main Entity Proposal

My proposal is to use the MusicBrainz release entity as the main entity to retrieve JSON-LD data from the API. Additional information can be merged or expanded using the release group entity or any other necessary entities to obtain more comprehensive data.

Reconciling with Wikidata

• For reconciling values such as musicians, file formats, etc., we can use OpenRefine.

  • Options:
    • Keep the MusicBrainz ID (either as another key or the main @id key).
    • Discard the MusicBrainz links and IDs.

• Regarding keys, MusicBrainz follows the schema.org context, which doesn't link to Wikidata. We can consider overriding this context and adding our properties where appropriate.

To start, I'll collect a list of releases (I'll try to find overlaps with our other dbs) and try to expand the jsonld at certain interesting metada. I'll then reconcile with wikidata using OpenRefine, overriding their contexts where needed. For values like musicians, cities, genres, I'll get the main @id as a wikidata iri, and add the musicbrainz id as another field in the json document.

[malajvan]

Commenting on structures in openrefine

• OpenRefine handles JSON-LD pretty weirdly, this might be because I read somewhere that OpenRefine only supports up to 1 level nesting? Result to handle it as a normal json instead
• Choose the outer most json document as a record path
• After creating the project in OpenRefine, each nested record (each track, artist, etc) will be expanded when we try to export them. I don't see a feature for record-view exporting, everything seems to be handle as rows
• Change to row view, choose fill down for all columns.
• Reconcile - (tbc)
• Export with this template? (Will still have to do cleanups)

{
  "@id": {{jsonize(cells["_ - @id"].value)}},
  "@type": {{jsonize(cells["_ - @type"].value)}},
  "@context": {{jsonize(cells["_ - @context"].value)}},
  "name": {{jsonize(cells["_ - name"].value)}},
  "musicReleaseFormat": {{jsonize(cells["_ - musicReleaseFormat"].value)}},
  "creditedTo": {{jsonize(cells["_ - creditedTo"].value)}},
  "duration": {{jsonize(cells["_ - duration"].value)}},
  "catalogNumber": {{jsonize(cells["_ - catalogNumber"].value)}},
  "inLanguage": {{jsonize(cells["_ - inLanguage"].value)}},
  "sameAs": {{jsonize(cells["_ - sameAs"].value)}},
  "gtin14": {{jsonize(cells["_ - gtin14"].value)}},
  "track": [
    {
      "name": {{jsonize(cells["_ - track - _ - name"].value)}},
      "@id": {{jsonize(cells["_ - track - _ - @id"].value)}},
      "trackNumber": {{jsonize(cells["_ - track - _ - trackNumber"].value)}},
      "@type": {{jsonize(cells["_ - track - _ - @type"].value)}},
      "duration": {{jsonize(cells["_ - track - _ - duration"].value)}},
      "isrcCode": {{jsonize(cells["_ - track - _ - isrcCode"].value)}},
      "sameAs": {{jsonize(cells["_ - track - _ - sameAs"].value)}},
      "sameAs - sameAs": {{jsonize(cells["_ - track - _ - sameAs - sameAs"].value)}},
      "contributor": [
        {
          "@type": {{jsonize(cells["_ - track - _ - contributor - _ - @type"].value)}},
          "roleName": {{jsonize(cells["_ - track - _ - contributor - _ - roleName"].value)}},
          "contributor": [
            {
              "@id": {{jsonize(cells["_ - track - _ - contributor - _ - contributor - @id"].value)}},
              "name": {{jsonize(cells["_ - track - _ - contributor - _ - contributor - name"].value)}},
              "@type": {{jsonize(cells["_ - track - _ - contributor - _ - contributor - @type"].value)}},
              "@type - @type": {{jsonize(cells["_ - track - _ - contributor - _ - contributor - @type - @type"].value)}}
            }
          ]
        }
      ],
      "producer": [
        {
          "name": {{jsonize(cells["_ - track - _ - producer - name"].value)}},
          "@id": {{jsonize(cells["_ - track - _ - producer - @id"].value)}},
          "@type - @type": {{jsonize(cells["_ - track - _ - producer - @type - @type"].value)}},
          "_ - @id": {{jsonize(cells["_ - track - _ - producer - _ - @id"].value)}},
          "_ - name": {{jsonize(cells["_ - track - _ - producer - _ - name"].value)}},
          "_ - @type - @type": {{jsonize(cells["_ - track - _ - producer - _ - @type - @type"].value)}}
        }
      ],
      "isrcCode - isrcCode": {{jsonize(cells["_ - track - _ - isrcCode - isrcCode"].value)}}
    }
  ],
  "image": [
    {
      "encodingFormat": {{jsonize(cells["_ - image - encodingFormat"].value)}},
      "representativeOfPage": {{jsonize(cells["_ - image - representativeOfPage"].value)}},
      "@type": {{jsonize(cells["_ - image - @type"].value)}},
      "contentUrl": {{jsonize(cells["_ - image - contentUrl"].value)}},
      "thumbnail": [
        {
          "encodingFormat": {{jsonize(cells["_ - image - thumbnail - _ - encodingFormat"].value)}},
          "contentUrl": {{jsonize(cells["_ - image - thumbnail - _ - contentUrl"].value)}},
          "@type": {{jsonize(cells["_ - image - thumbnail - _ - @type"].value)}}
        }
      ]
    }
  ],
  "releaseOf": [
    {
      "creditedTo": {{jsonize(cells["_ - releaseOf - creditedTo"].value)}},
      "albumProductionType": {{jsonize(cells["_ - releaseOf - albumProductionType"].value)}},
      "@type": {{jsonize(cells["_ - releaseOf - @type"].value)}},
      "albumReleaseType": {{jsonize(cells["_ - releaseOf - albumReleaseType"].value)}},
      "@id": {{jsonize(cells["_ - releaseOf - @id"].value)}},
      "name": {{jsonize(cells["_ - releaseOf - name"].value)}},
      "byArtist": [
        {
          "@id": {{jsonize(cells["_ - releaseOf - byArtist - @id"].value)}},
          "name": {{jsonize(cells["_ - releaseOf - byArtist - name"].value)}},
          "@type": {{jsonize(cells["_ - releaseOf - byArtist - @type"].value)}},
          "name": {{jsonize(cells["_ - releaseOf - byArtist - _ - name"].value)}},
          "@id": {{jsonize(cells["_ - releaseOf - byArtist - _ - @id"].value)}},
          "@type": {{jsonize(cells["_ - releaseOf - byArtist - _ - @type"].value)}},
          "@type - @type": {{jsonize(cells["_ - releaseOf - byArtist - _ - @type - @type"].value)}}
        }
      ]
    }
  ],
  "hasReleaseRegion": [
    {
      "releaseDate": {{jsonize(cells["_ - hasReleaseRegion - _ - releaseDate"].value)}},
      "@type": {{jsonize(cells["_ - hasReleaseRegion - _ - @type"].

[malajvan]

Leaning more on just getting the data psql dump and reconciliation from CSVs... seems like more work and more prone to error if we try to merge json data.
If anyone has experience reconciling json file structures on OpenRefine would love to get some expertise.

[malajvan]

@dchiller I talked about the different json-ld structure and how to get them from musicbrainz here if you're interested

[malajvan]

After discussions with Alistair, he suggested using MusicBrainz Json Dumps (https://data.metabrainz.org/pub/musicbrainz/data/json-dumps/) and instead of using OpenRefine, bypass its inability to handle structured data by using the reconciliation service api (https://www.w3.org/community/reports/reconciliation/CG-FINAL-specs-0.2-20230410/)

[malajvan]

refer to #27