CSV Reorganiser

This utility tool allows you to produce quickly reorganised CSV from a YAML configuration definition.

Description

Ever had to manipulate hundred of CSV files, change ordering from exports, or whatever to produce a final result? Then this tool was made for you.

Stop to the long hours trying to export, reorganise and separate fields values. All you need is Java and a YAML configuration file and that tool will make the job for you.

Getting Started

Dependencies

Java 21 is required for runtime.

This tool was made with/for Java 21 and Apache Maven as dependency manager.

It is using the following dependencies (already included as-is in the jar):

• SnakeYAML
• Apache Commons CSV
• Apache Commons Lang
• Apache Commons IO
• Apache Log4J
• Reflections

Testing dependencies:

• JUnit 5 (Jupiter)
• Mockito
• Hamcrest 2

Documentation dependencies (by and for):

• Javaparser

Check the pom.xml file for dependencies versions.

Compilation

Download the code, you should have Java 21+ and Apache Maven to be able to compile the project. Follow typical Maven goals regarding your intention (compile, package, install, ...).

Installing

Just download the jar file associated to the release or run mvn package (at least) from the repository. Jar file should be provided under target/csv-reorganiser-<VERSION>.jar

Executing software

Run the following command:

java -jar <path-to-jar>/csv-reorganiser-X.Y.jar <yaml-cfg> <csv-sources..> <csv-target>

With:

• yaml-cfg the YAML configuration file to be used for the reorganisation. Examples are provided in the examples/ folder, documentation provided on the wiki.
• csv-sources the base CSV files to be reorganised. If multiple ones are provided, they will all be processed sequentially and the results compiled into the same target file. Each source file must have the same structure.
• csv-target the target file to be written. If existing, it will be overwritten.

Engines

There are 2 engines in the CSV reorganiser. For each engine, the configuration file is written in a slight different way.

• The first one (id 1 or chess), is using a simple structure and performs only simple operations.
• The second one (id 2 or hyde), is an experimental engine using a text based definition of operations, with text filters directly included in the operation definition (still in implementation).

You can change which engine you are using by using the option flag -engine <engine-id> or -e <engine-id> where <engine-id> is the engine name or number. You can also specify the engine in the configuration file via the header, using the tag engine which behaves the same as the option.

If no engine is specified, chess will be assumed.

Check the wiki for more information.

Executing unit tests

A simple mvn test should suffice. This project has been built with the idea of achieving at least 80% of code coverage through unit tests.

If you want to skip unit tests to package or install, run command for installing (see before) with option -DskipTests.

Help

Check out the wiki.

Authors

• Sylvain Domenjoud aka "Sylordis" (creator and maintainer)

Version History

• v1.1 - in progress (but usable with chess engine)
  • Some unit tests were left behind (disabled).
  • Multiple source files processing.
  • New operation: Concat, Substring.
  • Preparation for Hyde.
• v1.0
  • First official release.
  • Unit-tested (most of it) along with some integration tests.
  • 3 Operations: Get, Value and RegReplace.

License

This project is licensed under the Apache License v2 - see the LICENSE file for details

Links

Project website: https://github.com/sylordis/csv-reorganiser

Known issues

Check the wiki/Known issues.