Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Nov 1, 2025

Implements Approach 2 (Domain Model First) for migrating from string parsing to JSON output (--json flag) with clear separation between data retrieval and parsing. Provides complete architecture for future-proofing against multiple data sources (binaries, slurmrestd REST API, files).

Current Architecture

  • Fetchers call Slurm binaries → custom parsers extract key=value pairs → TableData (rows/columns)
  • OpenAPI schemas (v0.0.40-43) and generated models already present in internal/slurmapi/
  • Provider pattern: BaseProvider[T] with concrete implementations (JobsProvider, NodesProvider, etc.)

Implemented Architecture

Five-layer separation of concerns:

  1. Data Sources (internal/datasource/) - SlurmDataSource interface with implementations:

    • BinaryJSONSource - Calls Slurm binaries with --json flag
    • FileSource - Reads JSON from files (testing/offline mode)
    • RestAPISource - Placeholder for future slurmrestd support
  2. JSON Parsers (internal/parser/) - Convert OpenAPI models to domain models:

    • ParseJobsJSON() - V0043OpenapiJobInfoRespdomain.Jobs
    • ParseNodesJSON() - V0043OpenapiNodesRespdomain.Nodes
    • ParsePartitionsJSON() - V0043OpenapiPartitionRespdomain.Partitions
  3. Domain Models (internal/domain/) - API-independent business entities:

    • domain.Job - Comprehensive job representation (40+ fields)
    • domain.Node - Complete compute node model
    • domain.Partition - Partition configuration and limits
  4. Adapters (internal/model/adapters.go) - Transform domain models to display format:

    • JobsToTableData() - Converts domain.JobsTableData
    • NodesToTableData() - Converts domain.NodesTableData
    • PartitionsToTableData() - Converts domain.PartitionsTableData
  5. V2 Providers - Demonstrate the complete pattern:

    • JobsProviderV2, NodesProviderV2, PartitionsProviderV2
    • Flow: DataSource → Parser → Domain Model → Adapter → TableData

Key Benefits

  • Clear separation: Data retrieval, parsing, and presentation are completely independent
  • Multiple sources: Easy to switch between binary calls, REST API, or file-based sources
  • Type safety: Uses generated OpenAPI models for JSON unmarshaling
  • Future-proof: Adding slurmrestd support requires only implementing RestAPISource
  • Backward compatible: V1 providers remain functional; V2 providers coexist as examples
  • Testable: Each layer can be tested independently

Documentation

Complete architecture documentation added in docs/architecture-json-migration.md including:

  • Detailed layer explanations
  • Usage examples
  • Migration path
  • Design decisions
  • Next steps for production use

Testing

  • ✅ All code compiles and builds successfully
  • ✅ Type-safe JSON parsing using generated OpenAPI models
  • ✅ Clean separation verified across all layers
  • ✅ V2 providers demonstrate complete data flow

Future work: Add unit tests for parsers, integration tests with mock data, gradual V1 to V2 migration.


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI changed the title [WIP] Refactor data parsing to use JSON for better reliability Analysis: Architecture options for JSON-based Slurm command parsing Nov 1, 2025
Copilot AI requested a review from Antvirf November 1, 2025 03:24
@Antvirf
Copy link
Owner

Antvirf commented Nov 1, 2025

@copilot approach 2 pls

Copilot AI and others added 3 commits November 1, 2025 03:38
Co-authored-by: Antvirf <26534322+Antvirf@users.noreply.github.com>
Co-authored-by: Antvirf <26534322+Antvirf@users.noreply.github.com>
Co-authored-by: Antvirf <26534322+Antvirf@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Nov 1, 2025

@copilot approach 2 pls

Approach 2 (Domain Model First) implemented! ✅

Complete architecture with 5 clean layers:

  1. Data Sources - SlurmDataSource interface with BinaryJSONSource (uses --json), FileSource (testing), and RestAPISource placeholder
  2. JSON Parsers - Convert OpenAPI models → domain models (ParseJobsJSON, ParseNodesJSON, ParsePartitionsJSON)
  3. Domain Models - API-independent business entities (domain.Job, domain.Node, domain.Partition)
  4. Adapters - Transform domain → TableData for existing UI
  5. V2 Providers - Demonstrate the pattern (JobsProviderV2, NodesProviderV2, PartitionsProviderV2)

Clear separation achieved:

  • Getting JSON: Data source implementations (binary/REST/file)
  • Parsing JSON: Parser layer uses generated OpenAPI models
  • Internal structure: Domain models are display-agnostic, adapters handle table conversion

Future-proof: Adding slurmrestd support = just implement RestAPISource

See docs/architecture-json-migration.md for complete details.

Commits: 0248506 (core), 0fd3c43 (providers), 16b1336 (docs)

Copilot AI changed the title Analysis: Architecture options for JSON-based Slurm command parsing Implement Domain Model First architecture for JSON-based Slurm command parsing Nov 1, 2025
@Antvirf Antvirf closed this Nov 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants