Data-Ingestion-and-Analysis-using-UK-Police-Crime-API

Hello! Please refer to the MDA (Modern Data Architectures) PDF.

This is a lab to understand batch data ingestion, while using the OSBDET environment. This is a virtual machine project which packages many big data open source projects such as Hadoop, Spark, Kafka or Nifi. https://github.com/raulmarinperez/osbdet

In this case we will be using the configuration file .xml as a template to gather the data from the UK Police Crime Free Rest-API.

Methodology

1. Import the .xml configuration template file to Nifi
2. Create a HDFS directory to safe the files
3. Run the NiFi Flow to gather the data
4. Run the .ipynb to watch the data being analysed.

BAM! That's it! I recommend you check the PDF, it is an easy way to look at the analysis without having to replicate all the steps.