DataStorm is a software that insert into a database the data of a CSV or Excel file. The software is able to insert the data into a new table or into an existing table and optimise the type of the columns to reduce the size of the database. Using C++ and Qt, the software can insert the data very efficiently and into different database management systems like MySQL, PostgreSQL or SQLite.
Insert data into a new tableInsert data into an existing table
Optimise the type of the columns (VARCHAR(MAX_LENGHT))
From CSV file
Don't require ANY privilege to insert the data
These drivers are required to connect to one of the databases:
- mysql-connector for qt
- postgresql-connector for qt
- sqlite-connector for qt
- odbc-connector for qt
To install the software, you need to follow the steps below:
- Clone the repository:
git clone https://github.com/Maxime-Cllt/DataStorm.git- Create a build directory:
mkdir build
cd build- Generate the build files:
cmake ..- Compile the software:
make- Run the software:
./DataStorm- Does not use LOAD DATA INFILE because it's required to have the FILE privilege to use it.
- Does not use the COPY command because it's required to have the SUPERUSER privilege to use it.
- Use the INSERT command to insert the data into the database despite not being the most efficient way.
- Does not use the BULK INSERT command because it's only available for SQL Server.
| Size (Ko) | Time to import (ms) | DBMS |
|---|---|---|
| 2345 | 132 | SQLite |
| 2345 | 893 | MySQL |
| 21 105 | 1148 | SQLite |
| 21 105 | 7623 | MySQL |