Skip to content

feat: Demote PARTICIPANT_ID_MISMATCH to a warning for now #3320

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nellh merged 6 commits into from
Feb 12, 2025
Merged

feat: Demote PARTICIPANT_ID_MISMATCH to a warning for now #3320

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
5e2dbd1
feat: Demote PARTICIPANT_ID_MISMATCH to a warning for now
effigies Feb 4, 2025
2be2eda
feat: Sync datalad validation task configuration to app worker
effigies Feb 11, 2025
54898f7
chore: Make the linter happy about valid JSON for validator configura…
nellh Feb 12, 2025
20d2a89
fix: Consolidate validator configuration to one JSON file and symlinks
nellh Feb 12, 2025
13565b6
fix(worker): --blacklistModalities must be final argument
nellh Feb 12, 2025
8477fd0
fix: Remove validator-config symlink for Deno version and link app to…
nellh Feb 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions cli/src/commands/upload.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import type { CommandOptions } from "@cliffy/command"
import { getRepoAccess } from "./git-credential.ts"
import { readConfig } from "../config.ts"
import { createDataset } from "../graphq.ts"
import validatorConfig from "../validator-config.json" with { type: "json" }

async function getRepoDir(url: URL): Promise<string> {
const LOCAL_STORAGE_KEY = `openneuro_cli_${url.hostname}_`
Expand Down Expand Up @@ -71,6 +72,7 @@ export async function uploadAction(
const schemaResult = await validate(
await readFileTree(dataset_directory_abs),
options,
validatorConfig,
)
console.log(consoleFormat(schemaResult))

Expand Down
11 changes: 11 additions & 0 deletions cli/src/validator-config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"error": [
{ "code": "NO_AUTHORS" },
{ "code": "SUBJECT_FOLDERS" },
{ "code": "EMPTY_DATASET_NAME" }
],
"warning": [
{ "code": "PARTICIPANT_ID_MISMATCH" }
],
"ignore": []
}
9 changes: 2 additions & 7 deletions packages/openneuro-app/src/scripts/workers/schema.worker.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,9 @@
import { fileListToTree, validate } from "@bids/validator/main"
import type { ValidationResult } from "@bids/validator/main"
import type { Config, ValidatorOptions } from "@bids/validator/options"
import validatorConfig from "./validator-config.json"

const config: Config = {
error: [
{ code: "NO_AUTHORS" },
{ code: "SUBJECT_FOLDERS" }, // bids-standard/bids-specification#1928 downgrades to warning
{ code: "EMPTY_DATASET_NAME" },
],
}
const config: Config = validatorConfig

const options: ValidatorOptions = {
datasetPath: "browser",
Expand Down
1 change: 1 addition & 0 deletions packages/openneuro-app/src/scripts/workers/validator-config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../../../../../cli/src/validator-config.json
2 changes: 1 addition & 1 deletion packages/openneuro-app/tsconfig.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"tsBuildInfoFile": "../../.build-cache/app.tsbuildinfo"
},
"include": ["./src"],
"files": ["./src/lerna.json"],
"files": ["./src/lerna.json", "./src/scripts/workers/validator-config.json"],
"references": [
{ "path": "../openneuro-components" }
]
Expand Down
11 changes: 11 additions & 0 deletions services/datalad/datalad_service/tasks/assets/validator-config.json
View file
Open in desktop
Copy link
Copy Markdown
Contributor Author

@effigies effigies Feb 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we could symlink this one too, then?

Copy link
Copy Markdown
Contributor

@nellh nellh Feb 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to get that to work but gave up on it. I've also had to go back to using a non-symlinked one for the JSR CLI, so there's just going to be several copies.

Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"error": [
{ "code": "NO_AUTHORS" },
{ "code": "SUBJECT_FOLDERS" },
{ "code": "EMPTY_DATASET_NAME" }
],
"warning": [
{ "code": "PARTICIPANT_ID_MISMATCH" }
],
"ignore": []
}
8 changes: 6 additions & 2 deletions services/datalad/datalad_service/tasks/validator.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import asyncio
from pathlib import Path
import json
import logging
import os
import re

import requests
Expand Down Expand Up @@ -46,10 +46,14 @@ async def validate_dataset_deno_call(dataset_path, ref, logger=logger):

Runs the deno bids-validator process.
"""
# Sync to packages/openneuro-app/src/scripts/workers/schema.worker.ts
config_path = Path(__file__).parent / 'assets' / 'validator-config.json'
return await run_and_decode(
['deno', 'run', '-A',
f'jsr:@bids/validator@{DENO_VALIDATOR_VERSION}',
'--json', dataset_path],
'--config', str(config_path),
'--json', dataset_path,
'--blacklistModalities', 'micr'],
logger=logger
)

Expand Down