Collect the Datasets

To be able to work with the datasets, they must first be loaded into the code. To be able to do this, a server must be used as data must be retrieved.

Jump to:

• Survey Dataset
• RDW Datasets

Survey dataset

First of all, of course, I started with the survey data of the CMD-students who followed the information design semester. This was an excel file. Fortunately, Jonah Meijers had already converted this to a JSON file so that it is easier to load it into the code.

To load this dataset I started by setting up a local server. I tried this by using my node for this. Because of block tech I have already worked with a node server. Only this didn't quite work the way I wanted. I saw nothing in my console, but I did see in the terminal. With the help of Lauren's explanation I started a local server via python. I started with the code below: $ node script.js

Because I couldn't get anything in the console, I used the local python server that Laurens told about: $ python -m SimpleHTTPServer 8800

Because of this I suddenly saw my output in the console. So I started with the script that I have shown below.

let fetchJSONdata = async () => {
  const response = await fetch('../data/userData.json');
  const data = await response.json();
  return data;
}

fetchJSONdata().then(fullData => {
  console.log('Data is loaded!');
  console.log(fullData);
})

Since this code was still very illogical (the workflow is not good and I used a variable that were unnecessary) I changed this to the code below. This shows that the workflow has been improved (at the top of the function that is called and at the bottom of the function itself). I also wrote the function names in a more general way so that I could use this function more often when I start using multiple datasets.

Furthermore, I use a global variable 'endpoint' so that I don't have to go through all of my code when it changes later. It can also be seen that I am using an async await function. This is because I have worked with this before in block tech and I personally find this more pleasant and cleaner to read than promises.

const endpoint = '../data/userData.json';

fetchAllData(endpoint)
  .then(fullData => {
    console.log("Data is loaded!");
    console.log(fullData);
  })

async function fetchAllData (url) {
  const response = await fetch(url);
  return await response.json();
}

RDW datasets

Now that I have learned more about loading datasets, I started looking at the datasets I need for my final visualization. First I worked with the same function that I had written for the survey data, namely the code below:

async function fetchAllData (url) {
  const response = await fetch(url);
  return await response.json();
}

The problem I now ran into was that I wanted to fetch two APIs. Since retrieving both datasets also takes some time, it was important to wait for this. I then solved this by writing my code like this, with the help of Jordy Fronik his code:

fetchData(API_1, API_2)
  .then((data) => {
    console.log(data);
    data.forEach((column) => {
      column.chargingpointCapacity = parseInt(column.chargingpointCapacity);
      column.parkingCapacity = parseInt(column.parkingCapacity);
    });
    createViz(data);
  })

// Function from Jordy Fronik:
// Source: https://github.com/joordy/frontend-data
async function fetchData(url, url2) {
  const response1 = await fetch(url);
  const dataset1 = await response1.json();

  const response2 = await fetch(url2);
  const dataset2 = await response2.json();

  const newDataArray = dataset2.map((item) => {
    const specs = dataset1.find((obj) => item.areaid === obj.areaid);
    if (specs !== undefined) {
      item.chargingpointCapacity = specs.chargingpointcapacity;
      item.parkingCapacity = specs.capacity;
    }
    return item;
  })
  return newDataArray;
} 

Only I did not find this code very pleasant and structured to read. For this reason I started writing the code to the code below:

const API_1 = 'https://opendata.rdw.nl/resource/b3us-f26s.json';
const API_2 = 'https://opendata.rdw.nl/resource/t5pc-eb34.json';

start();

async function start () {
  const facilitiesData = await fetchData(API_1);
  const locationData = await fetchData(API_2);
}

async function fetchData(url) {
  const response = await fetch(url);
  return await response.json();
}

With these lines of code, this fetch function has also become more functional. Namely, I call the function fetchData twice in the async await function start. In other words: the function fetchData can be used more often.