Skip to content

[Improvement-18019][task-sql] Support SQL from resource file and parameter placeholders#18020

Open
macdoor wants to merge 13 commits intoapache:devfrom
macdoor:feature/task-sql-resource-file
Open

[Improvement-18019][task-sql] Support SQL from resource file and parameter placeholders#18020
macdoor wants to merge 13 commits intoapache:devfrom
macdoor:feature/task-sql-resource-file

Conversation

@macdoor
Copy link
Contributor

@macdoor macdoor commented Mar 4, 2026

Was this PR generated or assisted by AI?

YES.
The implementation and refactoring were done by me, with AI assistance mainly for:

Purpose of the pull request

[Improvement][task-sql] Support executing SQL from Resource Center files as well as inline scripts, and apply parameter placeholders (${...}, $[...]) consistently for SQL tasks.

This makes it easier to manage SQL scripts (version control, reuse across workflows) and aligns task-sql with other resource-aware task types.

close #18019

Brief change log

Backend

  • SqlParameters (task-api)

    • Add fields:
      • sqlSource: "SCRIPT" (default) or "FILE"
      • sqlResource: resource path in Resource Center
    • Keep existing sql for inline SQL (backward compatible).
    • Update:
      • checkParameters() to accept either sql or sqlResource
      • getResourceFilesList() to return a ResourceInfo when sqlResource is set, so Worker downloads the file.
  • SqlTask (task-sql)

    • Add ensureSqlContent():
      • If sql is empty and sqlResource is set:
        • Use TaskExecutionContext.getResourceContext() to resolve local file path.
        • Read SQL text from the downloaded resource file and set it into SqlParameters.sql.
    • Keep the existing split/bind/execute logic unchanged.
    • Improve title placeholder replacement:
      • Build a placeholder map even when prepareParamsMap is null (fall back to Collections.emptyMap()).
      • Apply ParameterUtils.convertParameterPlaceholders for title so time placeholders (e.g. $[yyyyMMdd]) still work even without custom params.
  • Tests

    • Add SqlTaskTest:
      • testSqlLoadedFromResourceFileWhenSqlIsEmpty:
        • Creates a temp SQL file.
        • Configures sqlResource to point to it while leaving sql empty.
        • Uses ResourceContext on TaskExecutionContext.
        • Verifies SqlTask loads the content into SqlParameters.sql.

Frontend (UI)

  • Task form for SQL task

    • use-sql.ts:
      • Extend model with:
        • sqlSource: 'SCRIPT' | 'FILE' (default 'SCRIPT')
        • sqlResource: selected resource path.
    • fields/use-sql.ts:
      • Add a SQL Source radio group:
        • SCRIPT: show the existing SQL editor (field: 'sql').
        • FILE: hide the editor and show a Resource Center file selector bound to sqlResource (single-select).
      • Keep original validation for sql when sqlSource === 'SCRIPT'.
  • Localization

    • Add i18n keys in en_US and zh_CN:
      • sql_source, sql_source_script, sql_source_file, sql_resource_file.

Verify this pull request

This change added tests and can be verified as follows:

  • Unit tests

    • SqlParametersTest:
      • Verifies getResourceFilesList() returns a non-empty list when sqlResource is set.
    • SqlTaskTest:
      • testSqlLoadedFromResourceFileWhenSqlIsEmpty: verifies SqlTask loads SQL content from a Resource Center file via ResourceContext when sql is empty and sqlResource is set.
  • Manual verification

    • Build backend and UI on dev with this branch.
    • In UI:
      • Create a SQL task with:
        • SQL Source = SCRIPT, edit inline SQL as before → task behavior unchanged.
        • SQL Source = FILE, select a .sql file from Resource Center → task executes the SQL from that file.
      • Use ${...} and $[...] placeholders in both modes to confirm they are resolved as expected.

Pull Request Notice

Pull Request Notice

This PR does not introduce incompatible changes to existing SQL tasks:

  • Existing tasks without sqlSource/sqlResource continue to run as before (treated as SCRIPT + sql).
  • New fields are backward compatible and optional.

closes #18019

@github-actions github-actions bot added UI ui and front end related backend test labels Mar 4, 2026
@SbloodyS SbloodyS changed the title [Improvement][task-sql] Support SQL from resource file and parameter placeholders [Improvement-18019][task-sql] Support SQL from resource file and parameter placeholders Mar 5, 2026
@SbloodyS SbloodyS added this to the 3.4.2 milestone Mar 5, 2026
@SbloodyS SbloodyS added the improvement make more easy to user or prompt friendly label Mar 5, 2026
- Use SqlSourceType enum instead of String for sqlSource field
- Fix SqlTaskTest.java: merge new test into existing class, remove duplicate license/class block
- Remove Chinese comment from frontend use-sql.ts slots
- Fix missing closing brace in SqlTask.printReplacedSql method

Made-with: Cursor
@macdoor macdoor requested a review from SbloodyS March 6, 2026 03:31
Copy link
Member

@SbloodyS SbloodyS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM Overall. Just some NIT.

@macdoor macdoor requested a review from SbloodyS March 7, 2026 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend improvement make more easy to user or prompt friendly test UI ui and front end related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Improvement][task-sql] Support executing SQL from resource center file

2 participants