Problem: it's hard to differentiate between Ingest and Storage concerns

[jraddaoui]

After the analysis on issues #1076, #1081 and other discussions outside Github, we realized there is no clear separation between the ingest and storage concerns in Enduro. We tried to provide a unified experience in the user interface but, while doing so, we mixed actions from storage in the ingest application. A clear example is the "move package" operation, check this comment to know more.

This issue will cover the solutions we decided to implement as a first step to help us address #1076 and #1081 with a better separation of concerns.

Dashboard

• Add Ingest and Storage headings in sidebar.
• Rename Packages to SIPs on Ingest (sidebar, URLs, breadcrumbs, pages).
• Include Locations and AIPs on Storage.
• Rename Packages to AIPs on Storage (sidebar, URLs, breadcrumbs, pages).
• Add Ingest/Storage to breadcrumbs and URLs.
• Have different pages for SIPs and AIPs (browse and index).
  • Show ingest details and link to AIP on SIP page.
  • Show storage details on AIP page, no link back to SIP.
• Wire move operation to Storage endpoints below and only show on AIP page.
• Use new API endpoints detailed below.
• Use UUIDs in URLs.

API

Current:

Method	Endpoint	Attributes
GET	/package	package:list
GET	/package/{id}	package:read
POST	/package/{id}/confirm	package:review
GET	/package/{id}/move	package:move
POST	/package/{id}/move	package:move
GET	/package/{id}/preservation-actions	package:listActions
POST	/package/{id}/reject	package:review
POST	/package/upload	package:upload
GET	/storage/location	storage:location:list
POST	/storage/location	storage:location:create
GET	/storage/location/{uuid}	storage:location:read
GET	/storage/location/{uuid}/packages	storage:location:listPackages
POST	/storage/package	storage:package:create
GET	/storage/package/{aip_id}	storage:package:read
GET	/storage/package/{aip_id}/download	storage:package:download
POST	/storage/package/{aip_id}/reject	storage:package:review
GET	/storage/package/{aip_id}/store	storage:package:move
POST	/storage/package/{aip_id}/store	storage:package:move
POST	/storage/package/{aip_id}/submit	storage:package:submit
POST	/storage/package/{aip_id}/update	storage:package:submit

Expected:

Method	Endpoint	Attributes
GET	/ingest/sip	ingest:sip:list
GET	/ingest/sip/{uuid}	ingest:sip:read
POST	/ingest/sip/{uuid}/confirm	ingest:sip:review
GET	/ingest/sip/{uuid}/preservation-actions	ingest:sip:listActions
POST	/ingest/sip/{uuid}/reject	ingest:sip:review
POST	/ingest/sip/upload	ingest:sip:upload
GET	/storage/location	storage:location:list
POST	/storage/location	storage:location:create
GET	/storage/location/{uuid}	storage:location:read
GET	/storage/location/{uuid}/aips	storage:location:listAips
GET	/storage/aip	storage:aip:list
POST	/storage/aip	storage:aip:create
GET	/storage/aip/{uuid}	storage:aip:read
GET	/storage/aip/{uuid}/download	storage:aip:download
GET	/storage/aip/{uuid}/preservation-actions	storage:aip:listActions
POST	/storage/aip/{uuid}/reject	storage:aip:review
GET	/storage/aip/{uuid}/store	storage:aip:move
POST	/storage/aip/{uuid}/store	storage:aip:move
POST	/storage/aip/{uuid}/submit	storage:aip:submit
POST	/storage/aip/{uuid}/update	storage:aip:submit

Databases

Current schema:

Ingest DB:

Storage DB:

Changes:

• Rename package table to sip on Ingest.
• Rename package table to aip on Storage.
• Duplicate preservation_action and preservation_task on Storage.
• Add uuid column to sip table.
• Rename aip_id column to aip_uuid on sip table.
• Rename aip_id column to uuid on aip table.
• Remove workflow_id, run_id and location_id from sip table.
• Remove move package preservation action type from Ingest.
• Use move aip preservation action type on Storage.

Workflows

• Remove move workflow from ingest.
• Create preservation actions and task on storage move workflow.
• Generate SIP UUID before SIP creation.
• Consider recording Storage upload workflow as a preservation action.

---

Alternatives and additional context

See Miro board if you have access. Below the solutions analysis there are wireframes and notes for the changes outlined in this issue.