v9

FEATURES:
MET-5712 Add status for problem pattern analysis (#162)
MET-5806 Oembed for generic resources (#164)
MET-5961 DeBias status and dataset transitions (#161)
MET-5962 DeBias detected terms storage and report (#167)
MET-5962 DeBias Save detected terms to database and serve them
MET-6199 DeBias Optimisation and improvements (#168)
MET-6231 DeBias add LLM and NER flags (#172)
MET-6239 DeBias Consistency when processing one dataset at a time (#171)

DEBT:
MET-6117 Remove deprecations from oai harvesting (#165)

BUGS:
MET-6037 Fix Sandbox tier statistics don't correspond to record tier calculation
MET-6244 Add vacuum procedure to repositories (#170)
MET-6244 Fix dataset clean up routine (#169)

MISCELLANEOUS:
MET-5649 Update metis-actions branch from jdk21 to main
MET-5966 Align with latest harvesting changes in metis framework. (#160)
MET-6040 Add indices and update queries for metrics
MET-6103 Cleanup deprecations on http harvesting (#163)
MET-6284 Bump io.netty:netty-codec-http from 4.1.100.Final to 4.1.108.Final (#158)
MET-6284 POM update for release
Add missing github token
Update unit tests after fix date range indexing update from metis-framework
Update POM dependencies from javax to jakarta for jakarta.xml.bind
Set new snapshot version 9-SNAPSHOT

v8

FEATURES:
MET-5633 update sandbox with metis framework feature (#155)
MET-5637 Implement new dataset info endpoint (#148)
MET-5598 independent metis normalization version (#145)
MET-4736 view harvesting parameters (#141)

DEBT:
MET-4976 upgrade to spring boot 3 and framework 6 (#157)
MET-5498 Replace IP address hash function (#154)
MET-5622 technical debt sandbox (#149)
MET-5237 update morphia and testcontainers dependency (#139)
MET-5633 revert distributionManagement to point to local repositories
MET-5474: Update to jdk21 build (#151)
MET-5473 upgrade to jdk17 (#150)
Revert "MET-5473: Upgrade to jdk 21 builder"
MET-5473: Upgrade to jdk 21 builder
MET-5471: Update docker jre to temurin 21 (#146)
MET-5057 add media software dependencies to docker image (#143)
MET-4305 Migration to Spring Boot and Dockerization (#140)

BUGS:
MET-5687 Changed creation_date type from LocalDateTime to ZonedDateTime (#153)
MET-5670 fix allow empty setspec for oai protocol (#152)
MET-5057: Fix configuration naming and values inconsistencies (#144)

MISCELLANEOUS:
Merge branch 'release/v8'
Set release version 8
Updated versions in the pom file for release
Fixing metis version
Update elastic apm
Updated pom file with the latest corelib version
Bump io.netty:netty-codec-http2 from 4.1.94.Final to 4.1.100.Final (#147)
Fix project-path
Add upload of jars
Set new snapshot version 8-SNAPSHOT

v7

FEATURES:
MET-5383 Add rate control to validation workflow (#135)
MET-5382 Validation and field warnings endpoint (#134)
MET-5362 Create Validation Workflow API and update database for record tiers (#132)

DEBT:
MET-5444 Optimization of validation workflow (#138)
MET-5381 Updated code with new changes (#137)
MET-5305 Accept .tar and .tar.gz uploads harvesting (#130)

BUGS:
MET-5373 Periodic cleanup does not work fixed (#133)

MISCELLANEOUS:
Merge branch 'release/v7'
Set release version 7
Update metis version for release
Adding missing annotation
[maven-release-plugin] prepare for next development iteration

Dataset wide error and concurrency consumer per queue support

FEATURES:
MET-5035 Support for dataset wide errors
MET-5253 Download option for the published record
MET-5250 Add flag records publish successfully
MET-5125 Configurable concurrent consumers allocation per queue

BUGS:
MET-5231 Fixed concurrency mutex mechanism error

MISCELLANEOUS:
MET-5173 Update of Metis Core 10 libraries, postgresql 42.6.0 and netty-core-http 4.1.91.Final library versions added release action.
MET-5231 Update to use Spring framework 2.7.11 & Sprindoc 1.7.0

Record Selection with step size for harvesting, Update sandbox with latest feature from dereference/enrichment, Adapted sandbox for multiple nodes scenarios and Integration tests with testcontainers

FEATURES:
MET-4931 Do not display "fake" title values in message report for problem pattern P7
MET-2018 Update sandbox with new dereference/enrichment feature from metis
MET-5001 Record selection with step size. Improvement of record selection algorithm

BUGS:
MET-4845 Fix harvesting with dataset with xslt and handle record duplicates
MET-4901 Fix slowness of integration tests

MISCELLANEOUS:
MET-4655 Fix integration tests in DatasetController class
MET-4813 Ensure secure temporary file creation
MET-4657 Updated sandbox with metis changes in Enrichment REST service
MET-4785 More robust creation of temporary ID's for zip records
MET-4900 Updated postgres, testcontainers and dockercompose versions
MET-4783 Added queue sizes metrics
MET-4904 Made setting record ID safe for multiple nodes and fixed harvesting identifiers status
MET-5010 Sonarqube cleanup. Removed NullPointer exception bug reported by sonar
MET-4506 Make problem patterns interactions safe for multiple nodes
MET-4970 XsltUrlUpdateService safe multiple nodes
Update to the latest version of netty
Update the variable to unify version for netty and reactor
Update dependencies due entity api client v2
Update spring boot version to use updated version of spring webflux
Added complement to webflux dependency
Updated of pom file to fix deployment

Added record tier 0 data, fixed bugs during dataset processing and added performance tests to project

FEATURES:
MET-4417 Records tier 0 information is retrieved for the user
MET-4627 Added performance tests into project
MET-4625 Added elastic APM and custom metrics for performance tests

BUGS:
MET-4706 Fix SQL Error during validation step
MET-4718 Fixed issue where enrichment gets stuck because of Jibx excution
MET-4740 Removed path information from temporary record ids that were saved in the database
MET-4705 P1 problem pattern now finds duplicated titles with ignoring case
MET-4701 records are now retrieved from correct (harvest) step

MISCELLANEOUS:
Update readme file
Update perf.yml for performance tests
Update performance test scenarios to receive parameters
Update performance test parameters
Update performance tests polling time
Update performance test scenarios with better numbering to improve reporting
Update performance test scenarios to generate the right amount of datasets
Update performance scenarios maximum
update performance scenarios thread group action
Update threshold thread life span

Implementation of pattern analysis, 3D content tier calculation, update to North Macedonia, fix OAIPMH library dependency and race condition.

FEATURES:
MET-4450, MET-4451 Implementation of P2 and P6 problem patterns.
MET-4457 Implementation of P3, P5, P7, P9 and P12 problem patterns.
MET-4458 Implementation of P1 problem pattern.
MET-4624 Introduction of Spring boot actuator.
MET-4473 Update sandbox with 3D content tier calculation display.

BUGS:
MET-4607 Fix record and title limits on problem pattern P1 display.
MET-4599 Fix cache overwrite in xslt transformation in case of error.
MET-4592 Fix sorting issue of lists within problem patterns.
MET-4362 Fix error reporting for OAIPMH records.
MET-4358 Fix dependency on OAIPMH library.
MET-4470 Fix race condition in pattern analysis.
MET-4270 Fix publish link malfunction for failed datasets.
MET-4263 Fix record duplicates handling.

MISCELLANEOUS:
MET-4436 Parameter sanitation in dataset REST controller.
MET-4553 Re-integrate sandbox integration tests.
MET-4328 Use enum values for status and step display.
MET-4483 Add purge to problem patterns data in database.
MET-4418 Change Macedonia to North Macedonia.
MET-4499 Update API call for pattern analysis by dataset id.
MET-4422 Removed provider-record-url-template variable in configuration.

Tier Calculation, Apply transformation to EDM, new and Asynchronous harvesting processes

FEATURES:
MET-4052 Asynchronous OAI-PMH harvesting when submitting a new dataset
MET-4159 Remaining harvesting processes run asynchronously
MET-4038 Allow user to apply XSLT conversion to EDM before validation external
MET-4157 Add tier calculation - Metadata tier breakdown
MET-4158 Add tier calculation - Content tier breakdown
MET-4156 Add tier calculation - tier calculation overview
MET-3983 Allow user to do OAI-PMH harvesting. Implementation of REST API call for OAI-PMH harvesting
MET-3926 Allow user to do HTTP harvesting. Implementation of REST APT call for HTTP harvesting

BUGS:
MET-4357 Fix resource leak on Media processing
MET-4248 Implementation of OAI support for deleted records
MET-4183 Findings and fixes of OutOfMemoryError in Tomcat application
MET-3929 Fix of lost connection to postgres database
MET-4242 Fix of sandbox for not going further than Media Processing
MET-4093 Fix of sandbox stopping processing when one record was invalid
Fix of xslt file upload
Fix of tests to work both in Windows and Linux
Fix for Integration Tests
Make the repositories https. This fixes the issue with maven version > 3.8.1
Fix log4j logger name spelling
Handle possible null on resource thumbnail
Fix default xslt initialization
Fix building issue

MISCELLANEOUS:
MET-4325 Improvement of record id list in sandbox error report
MET-4284 Improvement of synchronization report in sandbox
MET-4217 Fixed error report when harvest fails
MET-4260 Decommission of the preview environment
MET-4083 Move XSLT conversion to EDM step into queues
MET-4071 Limiting the number of record to process up to 1000. Return error message if is surpassed.
MET-4196 Adding last small changes to tier calculation functionality
MET-4106 Update log4j version to 2.17.1 that fixes CVE-2021-44832
MET-4193 Fix integration tests that were failing because of bean initialization
MET-4008 Add caching of the default xslt for transformation
MET-4064 Performing XSLT transformation to EDM during harvesting
MET-3984 Implementation of dataset processing with OAI-PMH
MET-2622 Review of Sandbox exception hierarchy
MET-4005 Updated metis version
MET-4009 Update CORS mapping configuration
Update of database schema
Update of pom file dependencies
Update of configuration application file
Update ci workflow and use temurin.
Add readme badges
Update ci with sonar
Add build action

V1

FEATURES:
MET-3501 Updated Language labels
MET-3536 Added name, language, country and creation data to dataset summary
MET-3489 Created new API calls for Languages and Countries.

BUGS:

MISCELLANEOUS:
MET-3848 Updated metis libraries
MET-3419 Updated Sandbox with new configuration from metis library