Skip to content
datacorner edited this page Oct 4, 2023 · 24 revisions

The pipelite Project

The purpose of this solution is to build and execute Data Pipelines. Nothing new under the sun of course, however it aims to make this task very easy and by just using some configuration to build those data pipelines. in other words by using pipelite you can create quite powerful data intergation stuff without typing any lines of code !

pipelite is a Data Pipeline solution by design !

The way this solution is built is also totally extensible and enables all developers to extend its capabilities by addin new Data connectors and/or data pipelite.transformers.

πŸš€ Currently this solution provides data access and load from these data sources :
βœ… External file (csv)
βœ… External Excel Spreadsheet (xls, xlsx, xlsm, xlsb, odf, ods and odt) (in Progress)
βœ… External XES File (in Progress)
βœ… ODBC Data Sources (checked with SQL Server, SQLite) by using an configurable SQL query (Read Only)
βœ… SAP Read Table via SAP RFC (Read Only / In Progress)

πŸš€ And provides those transformers
βœ… Nothing !
βœ… Concat 2 Data sources
βœ… Lookup with 2 data sources

But, I just told you ... this is the beggining and pipelite is designed to be extensible ... so if you want to collaborate by creating new connectors and/or transformers, just join the community ;-)

🏠 Home
πŸ”‘ Main concepts
πŸ’» Installation
πŸ”¨ Configuration
πŸš€ Running

Supported Data Sources
πŸ“„ CSV File
πŸ“‘ XES File
πŸ“ƒ Excel File
πŸ“€ ODBC
🏒 SAP
🎒 ABBYY Timeline

Supported Transformations
πŸ”€ Pass Through
πŸ“Ά Dataset Profiling
πŸ”‚ Concat 2 Data sources
πŸ†– SubString
πŸ†’ Column Transformation
πŸ”ƒ Join data sources
πŸ”ƒ Lookup
πŸ”€ Rename Column Name

Extending pipelite
βœ… how to
βœ… Adding new Data sources
βœ… Adding new Transformers
βœ… Adding new Pipelines

Clone this wiki locally