Skip to content

Commit

Permalink
fix(server): Limit validation to 50k issues
Browse files Browse the repository at this point in the history
Prevents failing to report validation for datasets with larger than 16MB BSON issues.
  • Loading branch information
nellh committed Jan 15, 2025
1 parent 794f576 commit ddfd375
Show file tree
Hide file tree
Showing 2 changed files with 31 additions and 0 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
import { validationSeveritySort } from "../validation"

vi.mock("../../../config.ts")

describe("validation resolvers", () => {
describe("validationSeveritySort", () => {
it("sorts severity error before warning", () => {
const issues = [
{ severity: "warning" },
{ severity: "error" },
{ severity: "error" },
{ severity: "warning" },
{ severity: "error" },
]
const sortedIssues = issues.sort(validationSeveritySort)
expect(sortedIssues[0].severity).toBe("error")
expect(sortedIssues[1].severity).toBe("error")
expect(sortedIssues[2].severity).toBe("error")
expect(sortedIssues[3].severity).toBe("warning")
})
})
})
9 changes: 9 additions & 0 deletions packages/openneuro-server/src/graphql/resolvers/validation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,12 +38,21 @@ export const snapshotValidation = async (snapshot) => {
.exec()
}

export function validationSeveritySort(a, b) {
return a.severity.localeCompare(b.severity)
}

/**
* Save issues data returned by the datalad service
*
* Returns only a boolean if successful or not
*/
export const updateValidation = (obj, args) => {
// Limit to 50k issues with errors sorted first
if (args.validation.issues.length > 50000) {
args.validation.issues.sort(validationSeveritySort)
args.validation.issues = args.validation.issues.slice(0, 50000)
}

Check warning on line 55 in packages/openneuro-server/src/graphql/resolvers/validation.ts

View check run for this annotation

Codecov / codecov/patch

packages/openneuro-server/src/graphql/resolvers/validation.ts#L51-L55

Added lines #L51 - L55 were not covered by tests
return Validation.updateOne(
{
id: args.validation.id,
Expand Down

0 comments on commit ddfd375

Please sign in to comment.