POC: Refactor image artifact storage & execution profiles

[asmacdo]

POC implementation of #284

• Closes Add skopeo-based adapter for working with OCI images #277

The changes were substantial, so I wanted to go ahead and try it out to test sanity. Some implementation
did change the design doc.

All 3 phases are complete:

• Phase 1: OCI image storage via Skopeo, versioned paths, Docker loading
• Phase 2: --image and --exec flags for containers-run
• Phase 3: YAML execution profiles with inheritance (clobber semantics)

Breaking changes from master

• docker:// now stores as OCI directory (was: Singularity SIF)
• cmdexec not set by containers-add (commented out, TODO Phase 4)
• Old URL schemes commented out: dhub://, oci:, shub:// (TODO Phase 4)

New features

• containers-profiles command to list available profiles
• Profiles support extends: for inheritance
• CLI args (--image, --exec) override profile fields
• Base profiles (no image) work with --image flag
• Early validation: error if profile references missing image

Can be further improved by:

• Additional substitutions, e.g. {binds} so profiles can be better composable
• containers-init or similar to install default profiles (docker-default, apptainer-default)
• Phase 4: Backwards compatibility for old URL schemes
• Include profile reference in provenance metadata

ReproNim/containers integration

To fully replace the singularity_cmd shim, profiles would need additional features:

• pre-run / post-run hooks for dynamic setup/cleanup scripts
• env: block for static environment variables
• {env.VARNAME} placeholder for dynamic env var expansion

These are documented in the design doc as optional upstream RFEs. The current implementation covers the
core use case; ReproNim could adopt profiles now and request these extensions as needed.

[asmacdo]

Added a demo.sh for easy review.

./demo.sh

#!/bin/bash
# demo.sh - Demonstrate datalad-container images/profiles refactor
#
# Usage: ./demo.sh [base_dir]
#        base_dir defaults to /tmp

set -e

BASE_DIR="${1:-/tmp}"
DEMO_DIR="$BASE_DIR/datalad-container-demo-$$"

section() {
    echo ""
    echo "════════════════════════════════════════════════════════════"
    echo "  $1"
    echo "════════════════════════════════════════════════════════════"
    echo ""
}

info() {
    echo "► $1"
}

warn() {
    echo "⚠ $1"
}

run_cmd() {
    echo "\$ $1"
    eval "$1"
}

show_last_commit() {
    echo ""
    info "Last commit:"
    git --no-pager log -1 --oneline
    echo ""
}

# ============================================================================
section "SETUP: Create demo dataset"
# ============================================================================

info "Creating demo directory: $DEMO_DIR"
mkdir -p "$DEMO_DIR"
cd "$DEMO_DIR"

run_cmd "datalad create ."

info "Adding alpine image..."
run_cmd "datalad containers-add alpine:latest --url docker://alpine:latest"

# ============================================================================
section "PHASE 1: containers-add + datalad run (raw docker command)"
# ============================================================================

info "Using datalad run with explicit docker command (no containers-run)"
run_cmd "datalad run --input .datalad/containers/images/alpine/latest/image --output phase1-output.txt -- docker run --rm datalad-container/alpine:latest sh -c 'echo Phase 1: raw datalad run > /dev/stdout'"

# Create output manually since docker stdout doesn't redirect to file easily
echo "Phase 1: raw datalad run" > phase1-output.txt
git add phase1-output.txt && git commit --amend --no-edit

show_last_commit

# ============================================================================
section "PHASE 2: containers-run --image --exec"
# ============================================================================

info "Using containers-run with explicit --image and --exec flags"
run_cmd "datalad containers-run --image alpine:latest --exec 'docker run --rm --user \$(id -u):\$(id -g) -v \$(pwd):/work -w /work {img} {cmd}' --output phase2-output.txt --expand outputs -- sh -c 'echo Phase 2: containers-run with --image --exec > {outputs}'"

show_last_commit

# ============================================================================
section "PHASE 3a: Create profile with image, containers-run --profile"
# ============================================================================

info "Creating profiles directory and docker-alpine profile..."
mkdir -p .datalad/containers/profiles

cat > .datalad/containers/profiles/docker-alpine.yaml << 'EOF'
image: alpine:latest
exec: docker run --rm --user $(id -u):$(id -g) -v $(pwd):/work -w /work {img} {cmd}
EOF

info "Profile contents:"
cat .datalad/containers/profiles/docker-alpine.yaml

info "Saving profile to dataset..."
run_cmd "datalad save -m 'Add docker-alpine profile' .datalad/containers/profiles/"

run_cmd "datalad containers-run --profile docker-alpine --output phase3a-output.txt --expand outputs -- sh -c 'echo Phase 3a: profile with image > {outputs}'"

show_last_commit

# ============================================================================
section "PHASE 3b: containers-list and containers-profiles output"
# ============================================================================

info "Listing containers:"
run_cmd "datalad containers-list"

echo ""
info "Listing profiles:"
run_cmd "datalad containers-profiles"

# ============================================================================
section "PHASE 3c: Profile with extends (inheritance)"
# ============================================================================

info "Creating docker-alpine-env profile that extends docker-alpine..."
cat > .datalad/containers/profiles/docker-alpine-env.yaml << 'EOF'
extends: docker-alpine
exec: docker run --rm --user $(id -u):$(id -g) -v $(pwd):/work -w /work -e MY_VAR=hello-from-extended-profile {img} {cmd}
EOF

info "Profile contents:"
cat .datalad/containers/profiles/docker-alpine-env.yaml

run_cmd "datalad save -m 'Add docker-alpine-env profile' .datalad/containers/profiles/"

run_cmd "datalad containers-run --profile docker-alpine-env --output phase3c-output.txt --expand outputs -- sh -c 'echo Phase 3c: MY_VAR=\$MY_VAR > {outputs}'"

info "Output file contents:"
cat phase3c-output.txt

show_last_commit

# ============================================================================
section "PHASE 3d: CLI override --exec"
# ============================================================================

info "Using --profile but overriding --exec to add environment variable..."
run_cmd "datalad containers-run --profile docker-alpine --exec 'docker run --rm --user \$(id -u):\$(id -g) -v \$(pwd):/work -w /work -e CLI_OVERRIDE=yes {img} {cmd}' --output phase3d-output.txt --expand outputs -- sh -c 'echo Phase 3d: CLI_OVERRIDE=\$CLI_OVERRIDE > {outputs}'"

info "Output file contents:"
cat phase3d-output.txt

show_last_commit

# ============================================================================
section "PHASE 3e: CLI override --image"
# ============================================================================

info "Adding busybox image..."
run_cmd "datalad containers-add busybox:latest --url docker://busybox:latest"

info "Using docker-alpine profile but overriding --image to use busybox..."
run_cmd "datalad containers-run --profile docker-alpine --image busybox:latest --output phase3e-output.txt --expand outputs -- sh -c 'echo Phase 3e: running busybox instead of alpine > {outputs}'"

info "Output file contents:"
cat phase3e-output.txt

show_last_commit

# ============================================================================
section "PHASE 3f: Error case - missing image (early validation)"
# ============================================================================

info "Creating profile that references non-existent image..."
cat > .datalad/containers/profiles/bad-profile.yaml << 'EOF'
image: nonexistent:v999
exec: docker run --rm {img} {cmd}
EOF

run_cmd "datalad save -m 'Add bad-profile for testing' .datalad/containers/profiles/"

info "Attempting to use bad-profile (should fail early)..."
echo ""
if datalad containers-run --profile bad-profile -- echo "this should not run" 2>&1; then
    warn "ERROR: Command should have failed!"
else
    info "Command failed as expected (early validation works)"
fi

# Clean up bad profile
rm .datalad/containers/profiles/bad-profile.yaml
run_cmd "datalad save -m 'Remove bad-profile' .datalad/containers/profiles/"

# ============================================================================
section "PHASE 3g: Base profiles without image (require --image flag)"
# ============================================================================

info "Creating base profiles for different runtimes (no image specified)..."

cat > .datalad/containers/profiles/docker-default.yaml << 'EOF'
exec: docker run --rm --user $(id -u):$(id -g) -v $(pwd):/work -w /work {img} {cmd}
EOF

cat > .datalad/containers/profiles/apptainer-default.yaml << 'EOF'
exec: apptainer exec oci:{img_path} {cmd}
EOF

cat > .datalad/containers/profiles/podman-default.yaml << 'EOF'
exec: podman run --rm --userns=keep-id -v $(pwd):/work:Z -w /work {img} {cmd}
EOF

run_cmd "datalad save -m 'Add base runtime profiles' .datalad/containers/profiles/"

info "Listing all profiles:"
run_cmd "datalad containers-profiles"

# Test docker-default (should work)
info "Testing docker-default profile with --image alpine:latest..."
run_cmd "datalad containers-run --profile docker-default --image alpine:latest --output phase3g-docker.txt --expand outputs -- sh -c 'echo Phase 3g: docker-default profile > {outputs}'"
show_last_commit

# Test apptainer-default if available
if command -v apptainer &> /dev/null || command -v singularity &> /dev/null; then
    info "Testing apptainer-default profile..."
    run_cmd "datalad containers-run --profile apptainer-default --image alpine:latest --output phase3g-apptainer.txt --expand outputs -- sh -c 'echo Phase 3g: apptainer-default profile > {outputs}'"
    show_last_commit
else
    warn "Skipping apptainer test (apptainer/singularity not installed)"
fi

# Test podman-default if available
if command -v podman &> /dev/null; then
    info "Testing podman-default profile..."
    run_cmd "datalad containers-run --profile podman-default --image alpine:latest --output phase3g-podman.txt --expand outputs -- sh -c 'echo Phase 3g: podman-default profile > {outputs}'"
    show_last_commit
else
    warn "Skipping podman test (podman not installed)"
fi

# ============================================================================
section "DEMO COMPLETE"
# ============================================================================

info "Demo directory: $DEMO_DIR"
echo ""
info "Final git log:"
git --no-pager log --oneline

echo ""
info "All containers:"
datalad containers-list

echo ""
info "All profiles:"
datalad containers-profiles

echo ""
echo "Demo completed successfully!"

Output

════════════════════════════════════════════════════════════
  SETUP: Create demo dataset
════════════════════════════════════════════════════════════

► Creating demo directory: /tmp/datalad-container-demo-1114184
$ datalad create .
create(ok): /tmp/datalad-container-demo-1114184 (dataset)
► Adding alpine image...
$ datalad containers-add alpine:latest --url docker://alpine:latest
[INFO] Saving OCI image from docker://alpine:latest 
Getting image source signatures
Copying blob sha256:014e56e613968f73cce0858124ca5fbc601d7888099969a4eea69f31dcd71a53
Copying config sha256:7acffee03fe864cd6b88219a1028855d6c912e7cf6fac633aa4307529fd0cc08
Writing manifest to image destination
[WARNING] Skipping non-annexed layer: /tmp/datalad-container-demo-1114184/.datalad/containers/images/alpine/latest/image/blobs/sha256/014e56e613968f73cce0858124ca5fbc601d7888099969a4eea69f31dcd71a53 
[INFO] Loaded image into Docker daemon: datalad-container/alpine:latest 
add(ok): .datalad/containers/images/alpine/latest/image/blobs/sha256/014e56e613968f73cce0858124ca5fbc601d7888099969a4eea69f31dcd71a53 (file)
add(ok): .datalad/containers/images/alpine/latest/image/blobs/sha256/7acffee03fe864cd6b88219a1028855d6c912e7cf6fac633aa4307529fd0cc08 (file)
add(ok): .datalad/containers/images/alpine/latest/image/blobs/sha256/a107a3c031732299dd9dd607bb13787834db2de38cfa13f1993b7105e4814c60 (file)
add(ok): .datalad/containers/images/alpine/latest/image/index.json (file)
add(ok): .datalad/containers/images/alpine/latest/image/oci-layout (file)
add(ok): .datalad/config (file)
save(ok): . (dataset)
action summary:
  add (ok: 6)
  save (ok: 1)
add(ok): .datalad/containers/images/alpine/latest/image/blobs/sha256/014e56e613968f73cce0858124ca5fbc601d7888099969a4eea69f31dcd71a53 (file)
add(ok): .datalad/containers/images/alpine/latest/image/blobs/sha256/7acffee03fe864cd6b88219a1028855d6c912e7cf6fac633aa4307529fd0cc08 (file)
add(ok): .datalad/containers/images/alpine/latest/image/blobs/sha256/a107a3c031732299dd9dd607bb13787834db2de38cfa13f1993b7105e4814c60 (file)
add(ok): .datalad/containers/images/alpine/latest/image/index.json (file)
add(ok): .datalad/containers/images/alpine/latest/image/oci-layout (file)
add(ok): .datalad/config (file)
save(ok): . (dataset)
containers_add(ok): /tmp/datalad-container-demo-1114184/.datalad/containers/images/alpine/latest/image (file)
action summary:
  add (ok: 6)
  containers_add (ok: 1)
  save (ok: 1)

════════════════════════════════════════════════════════════
  PHASE 1: containers-add + datalad run (raw docker command)
════════════════════════════════════════════════════════════

► Using datalad run with explicit docker command (no containers-run)
$ datalad run --input .datalad/containers/images/alpine/latest/image --output phase1-output.txt -- docker run --rm datalad-container/alpine:latest sh -c 'echo Phase 1: raw datalad run > /dev/stdout'
[INFO] Making sure inputs are available (this may take some time) 
[INFO] == Command start (output follows) ===== 
Phase 1: raw datalad run
[INFO] == Command exit (modification check follows) ===== 
run(ok): /tmp/datalad-container-demo-1114184 (dataset) [docker run --rm datalad-container/alpine...]
[master 3937679] [DATALAD] Configure containerized environment 'alpine:latest'
 Date: Wed Dec 3 14:25:21 2025 -0600
 7 files changed, 8 insertions(+)
 create mode 120000 .datalad/containers/images/alpine/latest/image/blobs/sha256/014e56e613968f73cce0858124ca5fbc601d7888099969a4eea69f31dcd71a53
 create mode 120000 .datalad/containers/images/alpine/latest/image/blobs/sha256/7acffee03fe864cd6b88219a1028855d6c912e7cf6fac633aa4307529fd0cc08
 create mode 120000 .datalad/containers/images/alpine/latest/image/blobs/sha256/a107a3c031732299dd9dd607bb13787834db2de38cfa13f1993b7105e4814c60
 create mode 120000 .datalad/containers/images/alpine/latest/image/index.json
 create mode 120000 .datalad/containers/images/alpine/latest/image/oci-layout
 create mode 100644 phase1-output.txt

► Last commit:
3937679 [DATALAD] Configure containerized environment 'alpine:latest'


════════════════════════════════════════════════════════════
  PHASE 2: containers-run --image --exec
════════════════════════════════════════════════════════════

► Using containers-run with explicit --image and --exec flags
$ datalad containers-run --image alpine:latest --exec 'docker run --rm --user $(id -u):$(id -g) -v $(pwd):/work -w /work {img} {cmd}' --output phase2-output.txt --expand outputs -- sh -c 'echo Phase 2: containers-run with --image --exec > {outputs}'
[INFO] Making sure inputs are available (this may take some time) 
[INFO] == Command start (output follows) ===== 
[INFO] == Command exit (modification check follows) ===== 
run(ok): /tmp/datalad-container-demo-1114184 (dataset) [docker run --rm --user $(id -u):$(id -g)...]
add(ok): phase2-output.txt (file)
save(ok): . (dataset)
action summary:
  add (ok: 1)
  get (notneeded: 1)
  run (ok: 1)
  save (ok: 1)

► Last commit:
9a869d2 [DATALAD RUNCMD] docker run --rm --user $(id -u):$(id -g)...


════════════════════════════════════════════════════════════
  PHASE 3a: Create profile with image, containers-run --profile
════════════════════════════════════════════════════════════

► Creating profiles directory and docker-alpine profile...
► Profile contents:
image: alpine:latest
exec: docker run --rm --user $(id -u):$(id -g) -v $(pwd):/work -w /work {img} {cmd}
► Saving profile to dataset...
$ datalad save -m 'Add docker-alpine profile' .datalad/containers/profiles/
add(ok): .datalad/containers/profiles/docker-alpine.yaml (file)
save(ok): . (dataset)
action summary:
  add (ok: 1)
  save (ok: 1)
$ datalad containers-run --profile docker-alpine --output phase3a-output.txt --expand outputs -- sh -c 'echo Phase 3a: profile with image > {outputs}'
[INFO] Making sure inputs are available (this may take some time) 
[INFO] == Command start (output follows) ===== 
[INFO] == Command exit (modification check follows) ===== 
run(ok): /tmp/datalad-container-demo-1114184 (dataset) [docker run --rm --user $(id -u):$(id -g)...]
add(ok): phase3a-output.txt (file)
save(ok): . (dataset)
action summary:
  add (ok: 1)
  get (notneeded: 1)
  run (ok: 1)
  save (ok: 1)

► Last commit:
d68f9ab [DATALAD RUNCMD] docker run --rm --user $(id -u):$(id -g)...


════════════════════════════════════════════════════════════
  PHASE 3b: containers-list and containers-profiles output
════════════════════════════════════════════════════════════

► Listing containers:
$ datalad containers-list
alpine:latest -> .datalad/containers/images/alpine/latest/image

► Listing profiles:
$ datalad containers-profiles
docker-alpine -> alpine:latest

════════════════════════════════════════════════════════════
  PHASE 3c: Profile with extends (inheritance)
════════════════════════════════════════════════════════════

► Creating docker-alpine-env profile that extends docker-alpine...
► Profile contents:
extends: docker-alpine
exec: docker run --rm --user $(id -u):$(id -g) -v $(pwd):/work -w /work -e MY_VAR=hello-from-extended-profile {img} {cmd}
$ datalad save -m 'Add docker-alpine-env profile' .datalad/containers/profiles/
add(ok): .datalad/containers/profiles/docker-alpine-env.yaml (file)
save(ok): . (dataset)
action summary:
  add (ok: 1)
  save (ok: 1)
$ datalad containers-run --profile docker-alpine-env --output phase3c-output.txt --expand outputs -- sh -c 'echo Phase 3c: MY_VAR=$MY_VAR > {outputs}'
[INFO] Making sure inputs are available (this may take some time) 
[INFO] == Command start (output follows) ===== 
[INFO] == Command exit (modification check follows) ===== 
run(ok): /tmp/datalad-container-demo-1114184 (dataset) [docker run --rm --user $(id -u):$(id -g)...]
add(ok): phase3c-output.txt (file)
save(ok): . (dataset)
action summary:
  add (ok: 1)
  get (notneeded: 1)
  run (ok: 1)
  save (ok: 1)
► Output file contents:
Phase 3c: MY_VAR=hello-from-extended-profile

► Last commit:
9fe0639 [DATALAD RUNCMD] docker run --rm --user $(id -u):$(id -g)...


════════════════════════════════════════════════════════════
  PHASE 3d: CLI override --exec
════════════════════════════════════════════════════════════

► Using --profile but overriding --exec to add environment variable...
$ datalad containers-run --profile docker-alpine --exec 'docker run --rm --user $(id -u):$(id -g) -v $(pwd):/work -w /work -e CLI_OVERRIDE=yes {img} {cmd}' --output phase3d-output.txt --expand outputs -- sh -c 'echo Phase 3d: CLI_OVERRIDE=$CLI_OVERRIDE > {outputs}'
[INFO] Making sure inputs are available (this may take some time) 
[INFO] == Command start (output follows) ===== 
[INFO] == Command exit (modification check follows) ===== 
run(ok): /tmp/datalad-container-demo-1114184 (dataset) [docker run --rm --user $(id -u):$(id -g)...]
add(ok): phase3d-output.txt (file)
save(ok): . (dataset)
action summary:
  add (ok: 1)
  get (notneeded: 1)
  run (ok: 1)
  save (ok: 1)
► Output file contents:
Phase 3d: CLI_OVERRIDE=yes

► Last commit:
ecc03bf [DATALAD RUNCMD] docker run --rm --user $(id -u):$(id -g)...


════════════════════════════════════════════════════════════
  PHASE 3e: CLI override --image
════════════════════════════════════════════════════════════

► Adding busybox image...
$ datalad containers-add busybox:latest --url docker://busybox:latest
[INFO] Saving OCI image from docker://busybox:latest 
Getting image source signatures
Copying blob sha256:e59838ecfec5e79eb4371e9995ef86c8000fe1c67d7b9fa7b57e996d9ba772ff
Copying config sha256:08ef35a1c3f050afbbd64194ffd1b8d5878659f5491567f26d1c814513ae9649
Writing manifest to image destination
[WARNING] Skipping non-annexed layer: /tmp/datalad-container-demo-1114184/.datalad/containers/images/busybox/latest/image/blobs/sha256/e59838ecfec5e79eb4371e9995ef86c8000fe1c67d7b9fa7b57e996d9ba772ff 
[INFO] Loaded image into Docker daemon: datalad-container/busybox:latest 
add(ok): .datalad/containers/images/busybox/latest/image/blobs/sha256/08ef35a1c3f050afbbd64194ffd1b8d5878659f5491567f26d1c814513ae9649 (file)
add(ok): .datalad/containers/images/busybox/latest/image/blobs/sha256/870e815c3a50dd0f6b40efddb319c72c32c3ee340b5a3e8945904232ccd12f44 (file)
add(ok): .datalad/containers/images/busybox/latest/image/blobs/sha256/e59838ecfec5e79eb4371e9995ef86c8000fe1c67d7b9fa7b57e996d9ba772ff (file)
add(ok): .datalad/containers/images/busybox/latest/image/index.json (file)
add(ok): .datalad/containers/images/busybox/latest/image/oci-layout (file)
add(ok): .datalad/config (file)
save(ok): . (dataset)
action summary:
  add (ok: 6)
  save (ok: 1)
add(ok): .datalad/containers/images/busybox/latest/image/blobs/sha256/08ef35a1c3f050afbbd64194ffd1b8d5878659f5491567f26d1c814513ae9649 (file)
add(ok): .datalad/containers/images/busybox/latest/image/blobs/sha256/870e815c3a50dd0f6b40efddb319c72c32c3ee340b5a3e8945904232ccd12f44 (file)
add(ok): .datalad/containers/images/busybox/latest/image/blobs/sha256/e59838ecfec5e79eb4371e9995ef86c8000fe1c67d7b9fa7b57e996d9ba772ff (file)
add(ok): .datalad/containers/images/busybox/latest/image/index.json (file)
add(ok): .datalad/containers/images/busybox/latest/image/oci-layout (file)
add(ok): .datalad/config (file)
save(ok): . (dataset)
containers_add(ok): /tmp/datalad-container-demo-1114184/.datalad/containers/images/busybox/latest/image (file)
action summary:
  add (ok: 6)
  containers_add (ok: 1)
  save (ok: 1)
► Using docker-alpine profile but overriding --image to use busybox...
$ datalad containers-run --profile docker-alpine --image busybox:latest --output phase3e-output.txt --expand outputs -- sh -c 'echo Phase 3e: running busybox instead of alpine > {outputs}'
[INFO] Making sure inputs are available (this may take some time) 
[INFO] == Command start (output follows) ===== 
[INFO] == Command exit (modification check follows) ===== 
run(ok): /tmp/datalad-container-demo-1114184 (dataset) [docker run --rm --user $(id -u):$(id -g)...]
add(ok): phase3e-output.txt (file)
save(ok): . (dataset)
action summary:
  add (ok: 1)
  get (notneeded: 1)
  run (ok: 1)
  save (ok: 1)
► Output file contents:
Phase 3e: running busybox instead of alpine

► Last commit:
3f07f05 [DATALAD RUNCMD] docker run --rm --user $(id -u):$(id -g)...


════════════════════════════════════════════════════════════
  PHASE 3f: Error case - missing image (early validation)
════════════════════════════════════════════════════════════

► Creating profile that references non-existent image...
$ datalad save -m 'Add bad-profile for testing' .datalad/containers/profiles/
add(ok): .datalad/containers/profiles/bad-profile.yaml (file)
save(ok): . (dataset)
action summary:
  add (ok: 1)
  save (ok: 1)
► Attempting to use bad-profile (should fail early)...

run(error): /tmp/datalad-container-demo-1114184 (dataset) [Profile 'bad-profile' references image 'nonexistent:v999' but no image found at /tmp/datalad-container-demo-1114184/.datalad/containers/images/nonexistent/v999/image]
► Command failed as expected (early validation works)
$ datalad save -m 'Remove bad-profile' .datalad/containers/profiles/
delete(ok): .datalad/containers/profiles/bad-profile.yaml (symlink)
save(ok): . (dataset)
action summary:
  delete (ok: 1)
  save (ok: 1)

════════════════════════════════════════════════════════════
  PHASE 3g: Base profiles without image (require --image flag)
════════════════════════════════════════════════════════════

► Creating base profiles for different runtimes (no image specified)...
$ datalad save -m 'Add base runtime profiles' .datalad/containers/profiles/
add(ok): .datalad/containers/profiles/apptainer-default.yaml (file)
add(ok): .datalad/containers/profiles/docker-default.yaml (file)
add(ok): .datalad/containers/profiles/podman-default.yaml (file)
save(ok): . (dataset)
action summary:
  add (ok: 3)
  save (ok: 1)
► Listing all profiles:
$ datalad containers-profiles
apptainer-default -> 
docker-alpine-env (extends: docker-alpine) -> alpine:latest
docker-alpine -> alpine:latest
docker-default -> 
podman-default -> 
► Testing docker-default profile with --image alpine:latest...
$ datalad containers-run --profile docker-default --image alpine:latest --output phase3g-docker.txt --expand outputs -- sh -c 'echo Phase 3g: docker-default profile > {outputs}'
[INFO] Making sure inputs are available (this may take some time) 
[INFO] == Command start (output follows) ===== 
[INFO] == Command exit (modification check follows) ===== 
run(ok): /tmp/datalad-container-demo-1114184 (dataset) [docker run --rm --user $(id -u):$(id -g)...]
add(ok): phase3g-docker.txt (file)
save(ok): . (dataset)
action summary:
  add (ok: 1)
  get (notneeded: 1)
  run (ok: 1)
  save (ok: 1)

► Last commit:
286bdef [DATALAD RUNCMD] docker run --rm --user $(id -u):$(id -g)...

► Testing apptainer-default profile...
$ datalad containers-run --profile apptainer-default --image alpine:latest --output phase3g-apptainer.txt --expand outputs -- sh -c 'echo Phase 3g: apptainer-default profile > {outputs}'
[INFO] Making sure inputs are available (this may take some time) 
[INFO] == Command start (output follows) ===== 
INFO:    Using cached SIF image
[INFO] == Command exit (modification check follows) ===== 
run(ok): /tmp/datalad-container-demo-1114184 (dataset) [apptainer exec oci:.datalad/containers/i...]
add(ok): phase3g-apptainer.txt (file)
save(ok): . (dataset)
action summary:
  add (ok: 1)
  get (notneeded: 1)
  run (ok: 1)
  save (ok: 1)

► Last commit:
eada7f7 [DATALAD RUNCMD] apptainer exec oci:.datalad/containers/i...

► Testing podman-default profile...
$ datalad containers-run --profile podman-default --image alpine:latest --output phase3g-podman.txt --expand outputs -- sh -c 'echo Phase 3g: podman-default profile > {outputs}'
[INFO] Making sure inputs are available (this may take some time) 
[INFO] == Command start (output follows) ===== 
[INFO] == Command exit (modification check follows) ===== 
run(ok): /tmp/datalad-container-demo-1114184 (dataset) [podman run --rm --userns=keep-id -v $(pw...]
add(ok): phase3g-podman.txt (file)
save(ok): . (dataset)
action summary:
  add (ok: 1)
  get (notneeded: 1)
  run (ok: 1)
  save (ok: 1)

► Last commit:
7dde79a [DATALAD RUNCMD] podman run --rm --userns=keep-id -v $(pw...


════════════════════════════════════════════════════════════
  DEMO COMPLETE
════════════════════════════════════════════════════════════

► Demo directory: /tmp/datalad-container-demo-1114184

► Final git log:
7dde79a [DATALAD RUNCMD] podman run --rm --userns=keep-id -v $(pw...
eada7f7 [DATALAD RUNCMD] apptainer exec oci:.datalad/containers/i...
286bdef [DATALAD RUNCMD] docker run --rm --user $(id -u):$(id -g)...
895d197 Add base runtime profiles
69024cf Remove bad-profile
dd36c88 Add bad-profile for testing
3f07f05 [DATALAD RUNCMD] docker run --rm --user $(id -u):$(id -g)...
8fe8897 [DATALAD] Configure containerized environment 'busybox:latest'
ecc03bf [DATALAD RUNCMD] docker run --rm --user $(id -u):$(id -g)...
9fe0639 [DATALAD RUNCMD] docker run --rm --user $(id -u):$(id -g)...
439acfb Add docker-alpine-env profile
d68f9ab [DATALAD RUNCMD] docker run --rm --user $(id -u):$(id -g)...
5b76d11 Add docker-alpine profile
9a869d2 [DATALAD RUNCMD] docker run --rm --user $(id -u):$(id -g)...
3937679 [DATALAD] Configure containerized environment 'alpine:latest'
767d7a6 [DATALAD] new dataset

► All containers:
alpine:latest -> .datalad/containers/images/alpine/latest/image
busybox:latest -> .datalad/containers/images/busybox/latest/image

► All profiles:
apptainer-default -> 
docker-alpine-env (extends: docker-alpine) -> alpine:latest
docker-alpine -> alpine:latest
docker-default -> 
podman-default -> 

Demo completed successfully!

[asmacdo]

This PR served its purpose-- I'm proceeding on the design doc PR with fewer breaking changes.