feat: add common SQL database support

.env.template

@@ -0,0 +1,21 @@
+# Environment variables for testing
+
+CI=true
+
+MARIADB_HOST=
+MARIADB_PORT=
+MARIADB_USER=
+MARIADB_PASSWORD=
+MARIADB_DATABASE=
+
+MYSQL_HOST=
+MYSQL_PORT=
+MYSQL_USER=
+MYSQL_PASSWORD=
+MYSQL_DATABASE=
+
+POSTGRESQL_HOST=
+POSTGRESQL_PORT=
+POSTGRESQL_USER=
+POSTGRESQL_PASSWORD=
+POSTGRESQL_DATABASE=

.github/workflows/ci-workflow.yml

@@ -42,6 +42,8 @@ jobs:
 
       - name: Run tests
         run: npm run test
+        env:
+          CI: true
 
       - name: Upload coverage report
         if: always()

.vscode/settings.json

@@ -0,0 +1,27 @@
+{
+  /**
+       ----- Linting configurations
+     */
+  // Prevent editor formatting on save for certain file types
+  "[javascript]": {
+    "editor.formatOnSave": false,
+    "editor.formatOnPaste": false
+  },
+  "[typescript]": {
+    "editor.formatOnSave": false,
+    "editor.formatOnPaste": false
+  },
+  // Configure eslint to report + fix errors for correct file types
+  "eslint.validate": ["javascript", "typescript"],
+  "editor.codeActionsOnSave": {
+    "source.fixAll.eslint": "always"
+  },
+  // Configuration cspell
+  "cSpell.language": "en",
+  "cSpell.caseSensitive": false,
+  "cSpell.languageSettings": [
+    { "languageId": "typescript", "caseSensitive": false }
+  ],
+  "cSpell.words": ["Alcaraz", "batchjs", "Martínez"],
+  "liveServer.settings.port": 5501
+}

docs/common-api.md

@@ -0,0 +1,128 @@
+# Common API
+
+In this documentation, we will focus on the common API. This module includes the core of BatchJS. This API allows you to create your own custom jobs and steps using the common interface.
+
+------------
+
+<!--Auto generated Documentation PLEASE DO NOT MODIFY THIS FILE. MODIFY THE JSDOC INSTEAD-->
+
+## Table of Contents
+
+  - [AbstractBatchEntityReaderStream](#abstractbatchentityreaderstream)
+  - [AbstractBatchEntityWriterStream](#abstractbatchentitywriterstream)
+
+## AbstractBatchEntityReaderStream
+
+`extends ObjectReadable` 
+
+Class that enable to implement classes to read data in batches of a specified size in different types of data storage.
+
+
+
+  ### Constructor
+  | Name       | Description                             | Type                         |
+  |------------|-----------------------------------------|------------------------------|
+  | **options** | The options for the AbstractBatchEntityReaderStream. | AbstractBatchEntityReaderStreamOptions |
+  | **options.batchSize** | The maximum number of elements in a batch. | number |
+
+
+
+### _read (function)
+
+
+
+Reads a batch of data from the data storage and pushes it to the consumer stream.
+If the size parameter is not specified, it reads the number of entities specified in the batchSize option.
+If the size parameter is specified, it reads the minimum of the size and the batchSize option.
+If no data is available, it pushes null to the consumer stream to signal that the end of the stream has been reached.
+If an error occurs while reading data, it emits an error event to the stream.
+
+  #### Parameters
+  | Name       | Description                             | Type                         |
+  |------------|-----------------------------------------|------------------------------|
+  | **size** | The size parameter for controlling the read operation. | number |
+
+  #### Returns
+  | Type       | Description                             |
+  |------------|-----------------------------------------|
+  | Promise.&lt;void&gt; | A promise that resolves when the data has been read and pushed to the consumer stream. |
+
+
+### _flush (function)
+
+`private` 
+
+Flushes the buffer by pushing its content to the consumer stream. If the consumer stream is not ready to receive data, it waits for the drain event and flushes the buffer again when it is emitted.
+This function is recursive and will keep flushing the buffer until it is empty.
+
+  #### Returns
+  | Type       | Description                             |
+  |------------|-----------------------------------------|
+  | Promise.&lt;void&gt; | A promise that resolves when the buffer is flushed. |
+
+
+## AbstractBatchEntityWriterStream
+
+`extends ObjectWritable` 
+
+Class that enable to implement classes to write data in batches of a specified size in different types of data storage.
+
+
+
+  ### Constructor
+  | Name       | Description                             | Type                         |
+  |------------|-----------------------------------------|------------------------------|
+  | **options** | The options for the AbstractBatchEntityWriterStream. | AbstractBatchEntityWriterStreamOptions |
+  | **options.batchSize** | The maximum number of elements in a batch. | number |
+
+
+
+### _write (function)
+
+
+
+A method to write data to the stream, push the chunk to the buffer, and execute the callback.
+
+  #### Parameters
+  | Name       | Description                             | Type                         |
+  |------------|-----------------------------------------|------------------------------|
+  | **chunk** | The data chunk to write to the stream. | T |
+  | **encoding** | The encoding of the data. | BufferEncoding |
+  | **callback** | The callback function to be executed after writing the data. | WriteCallback |
+
+  #### Returns
+  | Type       | Description                             |
+  |------------|-----------------------------------------|
+  | Promise.&lt;void&gt; | This function does not return anything. |
+
+
+### _final (function)
+
+
+
+Finalizes the stream by pushing remaining data batches, handling errors,
+and executing the final callback.
+
+  #### Parameters
+  | Name       | Description                             | Type                         |
+  |------------|-----------------------------------------|------------------------------|
+  | **callback** | The callback function to be executed after finalizing the stream. | WriteCallback |
+
+  #### Returns
+  | Type       | Description                             |
+  |------------|-----------------------------------------|
+  | Promise.&lt;void&gt; | This function does not return anything. |
+
+
+### _flush (function)
+
+`private` 
+
+Creates a batch of data from the buffer and flushes it to the storage.
+
+  #### Returns
+  | Type       | Description                             |
+  |------------|-----------------------------------------|
+  | Promise.&lt;void&gt; |  |
+
+

docs/mariadb-api.md

@@ -0,0 +1,150 @@
+# Common API
+
+In this documentation, we will focus on the common API. This module includes the core of BatchJS. This API allows you to create your own custom jobs and steps using the common interface.
+
+------------
+
+<!--Auto generated Documentation PLEASE DO NOT MODIFY THIS FILE. MODIFY THE JSDOC INSTEAD-->
+
+## Table of Contents
+
+  - [MariadbBatchEntityReader](#mariadbbatchentityreader)
+  - [MariadbBatchEntityWriter](#mariadbbatchentitywriter)
+
+## MariadbBatchEntityReader
+
+`extends AbstractBatchEntityReaderStream` 
+
+Class that read data in batches of a specified size in Mariadb databases.
+
+
+
+  ### Constructor
+  | Name       | Description                             | Type                         |
+  |------------|-----------------------------------------|------------------------------|
+  | **options** | The options for the MariadbBatchEntityReader. | MariadbBatchEntityReaderOptions |
+  | **options.pool** | The MariadbQL connection pool. | Pool |
+  | **options.query** | SQL query to be executed (without LIMIT and OFFSET). | string |
+
+
+
+### fetch (function)
+
+`private` 
+
+Fetches a batch of data from the database.
+
+  #### Parameters
+  | Name       | Description                             | Type                         |
+  |------------|-----------------------------------------|------------------------------|
+  | **size** | The size of the batch to fetch. | number |
+
+  #### Returns
+  | Type       | Description                             |
+  |------------|-----------------------------------------|
+  | Promise.&lt;BatchData.&lt;T&gt;&gt; | A promise that resolves with the batch of data. |
+
+
+### _destroy (function)
+
+`private` 
+
+Destroys the writer by finalizing the statement used to read entities and
+closing the database connection. This method should be called when the
+writer is no longer needed to free up resources.
+
+  #### Parameters
+  | Name       | Description                             | Type                         |
+  |------------|-----------------------------------------|------------------------------|
+  | **error** | The error that caused the destruction. | Error, null |
+  | **callback** | The callback function to be executed after destroying the reader. | ReadCallback |
+
+
+### connectDatabase (function)
+
+`private` 
+
+Connects to the database by creating a new database connection if none
+already exists, or by reusing an existing connection.
+
+  #### Returns
+  | Type       | Description                             |
+  |------------|-----------------------------------------|
+  | Promise.&lt;PoolConnection&gt; | A promise that resolves with the database connection. |
+
+
+### disconnectDatabase (function)
+
+
+
+Disconnects from the database by closing the active database connection
+and setting the connection reference to null. This method should be called
+when the reader is no longer needed to free up resources.
+
+  #### Returns
+  | Type       | Description                             |
+  |------------|-----------------------------------------|
+  | Promise.&lt;void&gt; | A promise that resolves when the database connection
+is successfully closed. |
+
+
+### prepareStatement (function)
+
+`private` 
+
+Prepares a statement for fetching entities. If the statement has already been
+prepared, it is reused.
+
+  #### Parameters
+  | Name       | Description                             | Type                         |
+  |------------|-----------------------------------------|------------------------------|
+  | **db** | The database connection. | Mariadb.Database |
+
+  #### Returns
+  | Type       | Description                             |
+  |------------|-----------------------------------------|
+  | Promise.&lt;Mariadb.Statement&gt; | The prepared statement. |
+
+
+### finalizeStatement (function)
+
+`private` 
+
+Finalizes the statement used to fetch entities. This method should be
+called when the reader is no longer needed to free up resources.
+
+
+## MariadbBatchEntityWriter
+
+`extends AbstractBatchEntityWriterStream` 
+
+Class that writes data in batches of a specified size in MariadbQL databases.
+
+
+
+  ### Constructor
+  | Name       | Description                             | Type                         |
+  |------------|-----------------------------------------|------------------------------|
+  | **options** | The options for the MariadbBatchEntityWriter. | MariadbBatchEntityWriterOptions |
+  | **options.pool** | The MariadbQL connection pool. | Pool |
+  | **options.prepareStatement** | Insert SQL prepared statement to be executed. | String |
+
+
+
+### batchWrite (function)
+
+`protected` 
+
+Writes a batch of data to the storage.
+
+  #### Parameters
+  | Name       | Description                             | Type                         |
+  |------------|-----------------------------------------|------------------------------|
+  | **chunk** | The batch of data to write to the storage. | BatchData.&lt;T&gt; |
+
+  #### Returns
+  | Type       | Description                             |
+  |------------|-----------------------------------------|
+  | Promise.&lt;void&gt; |  |
+
+