[Improvement-18019][task-sql] Support SQL from resource file and parameter placeholders#18020
Open
macdoor wants to merge 13 commits intoapache:devfrom
Open
[Improvement-18019][task-sql] Support SQL from resource file and parameter placeholders#18020macdoor wants to merge 13 commits intoapache:devfrom
macdoor wants to merge 13 commits intoapache:devfrom
Conversation
added 4 commits
March 4, 2026 23:22
Made-with: Cursor
Made-with: Cursor
Made-with: Cursor
Made-with: Cursor
SbloodyS
requested changes
Mar 5, 2026
...-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/parameters/SqlParameters.java
Outdated
Show resolved
Hide resolved
dolphinscheduler-ui/src/views/projects/task/components/node/fields/use-sql.ts
Outdated
Show resolved
Hide resolved
...cheduler-task-sql/src/test/java/org/apache/dolphinscheduler/plugin/task/sql/SqlTaskTest.java
Outdated
Show resolved
Hide resolved
- Use SqlSourceType enum instead of String for sqlSource field - Fix SqlTaskTest.java: merge new test into existing class, remove duplicate license/class block - Remove Chinese comment from frontend use-sql.ts slots - Fix missing closing brace in SqlTask.printReplacedSql method Made-with: Cursor
SbloodyS
reviewed
Mar 7, 2026
Member
SbloodyS
left a comment
There was a problem hiding this comment.
LGTM Overall. Just some NIT.
...-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/parameters/SqlParameters.java
Outdated
Show resolved
Hide resolved
Made-with: Cursor
SbloodyS
reviewed
Mar 12, 2026
...hinscheduler-task-sql/src/main/java/org/apache/dolphinscheduler/plugin/task/sql/SqlTask.java
Show resolved
Hide resolved
…eview Made-with: Cursor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Was this PR generated or assisted by AI?
YES.
The implementation and refactoring were done by me, with AI assistance mainly for:
Purpose of the pull request
[Improvement][task-sql] Support executing SQL from Resource Center files as well as inline scripts, and apply parameter placeholders (
${...},$[...]) consistently for SQL tasks.This makes it easier to manage SQL scripts (version control, reuse across workflows) and aligns
task-sqlwith other resource-aware task types.close #18019
Brief change log
Backend
SqlParameters (task-api)
sqlSource:"SCRIPT"(default) or"FILE"sqlResource: resource path in Resource Centersqlfor inline SQL (backward compatible).checkParameters()to accept eithersqlorsqlResourcegetResourceFilesList()to return aResourceInfowhensqlResourceis set, so Worker downloads the file.SqlTask (task-sql)
ensureSqlContent():sqlis empty andsqlResourceis set:TaskExecutionContext.getResourceContext()to resolve local file path.SqlParameters.sql.prepareParamsMapisnull(fall back toCollections.emptyMap()).ParameterUtils.convertParameterPlaceholdersfortitleso time placeholders (e.g.$[yyyyMMdd]) still work even without custom params.Tests
SqlTaskTest:testSqlLoadedFromResourceFileWhenSqlIsEmpty:sqlResourceto point to it while leavingsqlempty.ResourceContextonTaskExecutionContext.SqlTaskloads the content intoSqlParameters.sql.Frontend (UI)
Task form for SQL task
use-sql.ts:modelwith:sqlSource:'SCRIPT' | 'FILE'(default'SCRIPT')sqlResource: selected resource path.fields/use-sql.ts:SCRIPT: show the existing SQL editor (field: 'sql').FILE: hide the editor and show a Resource Center file selector bound tosqlResource(single-select).sqlwhensqlSource === 'SCRIPT'.Localization
en_USandzh_CN:sql_source,sql_source_script,sql_source_file,sql_resource_file.Verify this pull request
This change added tests and can be verified as follows:
Unit tests
SqlParametersTest:getResourceFilesList()returns a non-empty list whensqlResourceis set.SqlTaskTest:testSqlLoadedFromResourceFileWhenSqlIsEmpty: verifiesSqlTaskloads SQL content from a Resource Center file viaResourceContextwhensqlis empty andsqlResourceis set.Manual verification
devwith this branch.SQL Source = SCRIPT, edit inline SQL as before → task behavior unchanged.SQL Source = FILE, select a.sqlfile from Resource Center → task executes the SQL from that file.${...}and$[...]placeholders in both modes to confirm they are resolved as expected.Pull Request Notice
Pull Request Notice
This PR does not introduce incompatible changes to existing SQL tasks:
sqlSource/sqlResourcecontinue to run as before (treated asSCRIPT+sql).closes #18019