#6 Implement metric pipeline to analyze model performance

[botirk38]

Why?

We need to implement this feature to analyze the performance of our pipeline according to specified metrics. The primary use cases are:

1. To evaluate the quality of reconstructed data compared to original data using standardized metrics (e.g., BLEU score).
2. To provide a flexible and configurable way to apply different metrics to various data columns.
3. To integrate metric analysis seamlessly into our SONAR pipelines for Hugging Face datasets.

This implementation will allow us to:

• Quantitatively assess the performance of our data processing or model outputs.
• Easily switch between different metrics for evaluation.
• Identify areas where the pipeline's performance falls below a specified threshold.

How?

Key technical decisions made in this implementation:

1. Created a MetricPipelineConfig class extending PipelineConfig to encapsulate metric-specific configuration.
2. Implemented a MetricAnalyzerPipeline class that inherits from the base Pipeline class.
3. Utilized the Hugging Face evaluate library to load and compute metrics.
4. Implemented batch processing to efficiently handle large datasets.
5. Added logging throughout the pipeline for better monitoring and debugging.
6. Used type hints and dataclasses for improved code readability and maintainability.

Key components:

• MetricOverwrites: Allows for easy overriding of metric-specific configurations.
• compute_metric: Calculates the metric score for a given set of original and reconstructed data.
• process_batch: Applies the metric computation to each specified column in a batch.
• __call__: Processes the entire dataset using the configured metric.

Work in Progress (WIP) parts:

• Error handling could be expanded, especially in the process_batch method.
• The results attribute in MetricAnalyzerPipeline is initialized but not used in the current implementation.

Test plan

To test these changes, we will:

1. Create unit tests for individual components of the MetricAnalyzerPipeline.
2. Implement integration tests to ensure proper functioning within the SONAR pipeline ecosystem.
3. Test with various metrics and dataset types to ensure flexibility and robustness.

[antoine-tran]

This PR looks good to me. I think some code needs to be improved but once the comments are addressed (and make linter happy ofc), we can land this