4. If we decided to create another django-admin command, we should align with Katello to see if it makes sense for them to run a management command reading data from the storage.

From the Katello side, we're okay with the command being the same as before or a new one. Choose whatever is most efficient from the Pulp side.

Also I wanted to clarify to be super sure -- OCI Image Index manifests will always have a null arch, OS, and size? I'm not super clear since, according to https://specs.opencontainers.org/image-spec/image-index/?v=v1.0.1, the optional platform property can have an arch and an OS,

Also I wanted to clarify to be super sure -- OCI Image Index manifests will always have a null arch, OS, and size? I'm not super clear since, according to https://specs.opencontainers.org/image-spec/image-index/?v=v1.0.1, the optional platform property can have an arch and an OS,

Thank you for bringing this!
We reviewed and discussed the os and architecture fields from manifest list, and we will work on it in this PR.
If the manifest list (or oci index) contains the platform (an optional) field, we will populate pulp manifest with arch and os (whenever platform is defined, os and arch are required).

After re-reading the specs, I realized that manifestlist or oci-index do not have a platform field. Actually, platform is a field from manifests. In this link, we can see it better: https://github.com/opencontainers/image-spec/blob/main/image-index.md (platform is a bullet inside manifests).

So, please, ignore my last comment and yes, "OCI Image Index manifests will always have a null arch, OS, and size".

After re-reading the specs, I realized that manifestlist or oci-index do not have a platform field. Actually, platform is a field from manifests. In this link, we can see it better: https://github.com/opencontainers/image-spec/blob/main/image-index.md (platform is a bullet inside manifests).

So, please, ignore my last comment and yes, "OCI Image Index manifests will always have a null arch, OS, and size".

Nice catch! I didn't notice it was under manifests, so that's perfect.

If we need to store the "architecture" and "os" fields on the Manifest model, we should try to fetch the values from a manifest list first. An OCI Index has the "architecture" and "os" fields required. There is no need to read the data from a config blob if a manifest is listed within an index (https://github.com/opencontainers/image-spec/blob/main/image-index.md#image-index-property-descriptions).

I did some investigation in each workflow, and here are my findings:

• for the sync and pull-through tasks, the first declarative content that we pass to the next pipeline stage is/are blobs/configblobs
  

  pulp_container/pulp_container/app/tasks/sync_stages.py

  Lines 330 to 345 in 1a8fe63

  	async def handle_blobs(self, manifest_dc, content_data):
  	"""
  	Handle blobs.
  	"""
  	manifest_dc.extra_data["blob_dcs"] = []
  	for layer in content_data.get("layers") or content_data.get("fsLayers"):
  	if not self._include_layer(layer):
  	continue
  	blob_dc = self.create_blob(layer)
  	manifest_dc.extra_data["blob_dcs"].append(blob_dc)
  	await self.put(blob_dc)
  	layer = content_data.get("config", None)
  	if layer:
  	blob_dc = self.create_blob(layer, deferred_download=False)
  	manifest_dc.extra_data["config_blob_dc"] = blob_dc
  	await self.put(blob_dc)

  • after that, in the resolve_flush method, we then send the manifests and manifest lists in this order:
    

    pulp_container/pulp_container/app/tasks/sync_stages.py

    Lines 274 to 306 in 1a8fe63

    	async def resolve_flush(self):
    	"""Resolve pending contents dependencies and put in the pipeline."""
    	# Order matters! Things depended on must be resolved first.
    	for manifest_dc in self.manifest_dcs:
    	config_blob_dc = manifest_dc.extra_data.get("config_blob_dc")
    	if config_blob_dc:
    	manifest_dc.content.config_blob = await config_blob_dc.resolution()
    	await sync_to_async(manifest_dc.content.init_labels)()
    	manifest_dc.content.init_image_nature()
    	for blob_dc in manifest_dc.extra_data["blob_dcs"]:
    	# Just await here. They will be associated in the post_save hook.
    	await blob_dc.resolution()
    	await self.put(manifest_dc)
    	self.manifest_dcs.clear()
    	for manifest_list_dc in self.manifest_list_dcs:
    	for listed_manifest in manifest_list_dc.extra_data["listed_manifests"]:
    	# Just await here. They will be associated in the post_save hook.
    	await listed_manifest["manifest_dc"].resolution()
    	await self.put(manifest_list_dc)
    	self.manifest_list_dcs.clear()
    	for tag_dc in self.tag_dcs:
    	tagged_manifest_dc = tag_dc.extra_data["tagged_manifest_dc"]
    	tag_dc.content.tagged_manifest = await tagged_manifest_dc.resolution()
    	await self.put(tag_dc)
    	self.tag_dcs.clear()
    	for signature_dc in self.signature_dcs:
    	signed_manifest_dc = signature_dc.extra_data["signed_manifest_dc"]
    	signature_dc.content.signed_manifest = await signed_manifest_dc.resolution()
    	await self.put(signature_dc)
    	self.signature_dcs.clear()

• for the push workflow, we first push the blobs and the corresponding manifest for that image, and, just after pushing all images, the manifest-list/oci-index is pushed

I couldn't find how to fetch these values from the manifest list in advance. I'm not sure if I overlooked something or misunderstood the code workflow.

• Mirroring:
  I believe you can extract the information about os/architecture inside the create_listed_manifest method:

  pulp_container/pulp_container/app/tasks/sync_stages.py

  Line 480 in 1a8fe63

  man_dc = DeclarativeContent(content=manifest)

  .

• Pushing:
  When it comes to pushing, we may want to get this information from the already uploaded config blob:

  pulp_container/pulp_container/app/registry_api.py

  Line 1295 in 1a8fe63

  config_blob = found_config_blobs.first()

  .

  If we are fine (we are mostly not) with updating existing manifests, we could potentially update the objects here:

  pulp_container/pulp_container/app/registry_api.py

  Line 1221 in 1a8fe63

  manifest_to_list = models.ManifestListManifest(

  .

Thank you for the help and the suggestions/optimizations!