Skip to content
/ Redshift_ingestion Public

Working on PySpark and writing a structured .parquet file on Amazon Redshift after some basic cleaning and formatting

0 stars 0 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

Data5ci3nc3/Redshift_ingestion

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
Project_Redshift_ingestion_completed.ipynb
Project_Redshift_ingestion_completed.ipynb
 
 
README.md
README.md
 
 

Repository files navigation

Redshift_ingestion project

Working on PySpark and writing a structured file on Amazon Redshift.

Goals to be achieved

  • Load a parquet file from Amazon S3 using boto3 into a PySpark DataFrame
  • Understand the underlying concepts of Spark for big data management
  • Loading data onto Redshift
  • Understand the difference between a data lake and a data warehouse

About

Working on PySpark and writing a structured .parquet file on Amazon Redshift after some basic cleaning and formatting

Topics

aws python3 pyspark redshift

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Languages