pySpark Project

This task was given to explore pySpark

Project Description

Read the CSV/excel file.
Check the table definition.
Create schema based on the table definition.
Take the given CSV file as input
Process the data based on the given definition of each table.
Save different CSV files based on the table definition.
Full description here.