You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+3-3Lines changed: 3 additions & 3 deletions
Original file line number
Diff line number
Diff line change
@@ -6,7 +6,7 @@ This is a simple personal project to extract and model music album data using Az
6
6
7
7
## Data: 1001 Albums To Hear Before You Die
8
8
9
-
Data is sourced from an API provided by the webapp [1001albumsgenerator](https://1001albumsgenerator.com/), based off
9
+
Data is sourced from an API provided by the webapp [1001albumsgenerator](https://1001albumsgenerator.com/), based on
10
10
the book `1001 Albums You Must Hear Before You Die` by Robert Dimery.
11
11
12
12
Every day a new music album is listened to and rated. The API tracks the albums listened to and the rating assigned,
@@ -30,6 +30,8 @@ pipeline `get_albums`.
30
30
The pipeline then calls an Azure Databricks notebook called `load_albums_delta` that loads today's json file into a
31
31
delta table in an ADB workspace.
32
32
33
+
In addition, each week a maintainence script optimizes and vacuums the delta table.
34
+
33
35
Storage account access is managed via an API call to an Azure Key Vault that hold the details of a storage account
34
36
key to be used by Databricks to connect to.
35
37
@@ -46,5 +48,3 @@ visualisation.
46
48
47
49
In the ADB workspace, a Dashboard visualisation uses the Gold layer of the dbt warehouse and provides simple
48
50
visualisations and analysis.
49
-
50
-
Users with permission can access the visualisation dashboard via the ADB workspace [here](https://adb-2359489148887710.10.azuredatabricks.net/dashboardsv3/01ef83092f1b1403b7967bea7000d543/published?o=2359489148887710)
0 commit comments