Skip to content

Fixes for New Backend

Compare
Choose a tag to compare
@dylanmcreynolds dylanmcreynolds released this 03 Oct 16:46
· 28 commits to main since this release
67b0556

What's Changed

Full Changelog: v0.4.2...v0.4.3

Several issues were discovered in ingesting datasets to the new backend:

  • The new backend provides a more consistent usage of 404 return codes than the old backend did. In the new backend, 404 seems to only be returned when the path to and endpoint is not found. The old backend sometimes returned 404 when requests did not bring back content, and there was some code to deal with that discrepency, which has been removed.
  • There is a breaking API change in this version related to creating OrigDataBlock. The signature has changed, and a new model object has been introduced:

The OLD way to upload an `OrigDataBlock':

 # Datablock with DataFiles
data_file = DataFile(path="/foo/bar", size=42)
data_block = Datablock(
       size=42,
       version=1,
       datasetId=dataset_id,
       dataFileList=[data_file],
        **ownable.dict()
   )
scicat.upload_dataset_origdatablock(data_block)

The NEW way:

 # Datablock with DataFiles
 data_file = DataFile(path="/foo/bar", size=42)
 data_block_dto = CreateDatasetOrigDatablockDto(
       size=42,
       datasetId=dataset_id,
        dataFileList=[data_file],
  )
  scicat.upload_dataset_origdatablock(dataset_id, data_block_dto)

This change was necessary for uploading OrigDataBlock because the new backend introduces the CreateDatasetOrigDatablockDto model object, which has fewer fields then the old backends OrigDatablock model, and validation rejects the old mechanism.