Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat:流水线变量语法支持两种风格 #10576 #10961

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

mingshewhe
Copy link
Collaborator

@mingshewhe mingshewhe commented Sep 13, 2024

方言设计:

  1. 项目增加pipeline_dialect字段,为什么不放在properties里面,因为要在界面展示,而且修改审批时能够对比
  2. 流水线设置增加pipelineDialect字段,值放在pipelineAsSetting字段中
  3. 抽象方言对象com.tencent.devops.common.pipeline.dialect.IPipelineDialect,实现类ClassicPipelineDialect和ConstrainedPipelineDialect

引擎执行设计

  1. 在构建时,将方言值存储在var表中,key为BK_CI_PIPELINE_DIALECT
  2. 构建运行时,直接从var表中从方言取值,如果方言有改动,那么重试还应该继续保持构建时的值
  3. 变量引用方式:
    • 如果支持${}引用,那么在com.tencent.devops.process.engine.service.vmbuild.EngineVMBuildService#claim,领取任务是在服务端就会解析一次
    • worker领取到任务后,会再次解析,比如构建机上运行时才有的变量,如${workspace},解析完后,如果还有${{}}引用的变量,会运行表达式计算
  4. 变量不存在
    • 表达式引擎里面,通过传参判断当变量不存在时是否抛出异常,传参由方言控制
    • 变量不存在有几个小点,会导致构建直接取消
      • 自定义环境变量:job和task级别,如果解析报错就直接抛错了,
      • buildEnv环境变量:
      • 人工审核:人工审核变量不存在,也直接失败
      • matrix:如果matrix解析中报错,也会失败
      • 互斥组:互斥组使用变量,变量不存在,也会报错
  5. 变量超长
    • 启动变量:
    • 运行时插件输出:

@@ -1598,6 +1608,24 @@ abstract class AbsProjectServiceImpl @Autowired constructor(
deptName: String?
)

override fun getPipelineDialect(projectId: String): String {
Copy link
Member

@irwinsun irwinsun Sep 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

此方法不需要,统一由项目VO缓存,在修改项目处增加方案提示:注意:该配置可能会影响项目下所有流水线行为,配置更新需要10分钟

@@ -297,7 +297,7 @@ class SubPipelineRepositoryService @Autowired constructor(
val trigger = stages.getOrNull(0)
?: throw ErrorCodeException(errorCode = ProcessMessageCode.ERROR_PIPELINE_MODEL_NEED_JOB)
// 检查触发容器
val paramsMap = defaultModelCheckPlugin.checkTriggerContainer(trigger)
val paramsMap = defaultModelCheckPlugin.checkTriggerContainer(trigger, false)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check

var newValue = value
if (dialect.supportUseSingleCurlyBracesVar()) {
newValue = ObjectReplaceEnvVarUtil.replaceEnvVar(newValue, contextMap).let {
JsonUtil.toJson(it, false)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

原有逻辑是否json转换

@@ -31,3 +31,4 @@ const val EXCHANGE_PROJECT_COUNT_LOGIN = "exchange_project_count_login"
const val ROUTE_PROJECT_COUNT_LOGIN = "route_project_count_login"
const val QUEUE_PROJECT_COUNT_LOGIN = "queue_project_count_login"
const val SECRECY_PROJECT_REDIS_KEY = "SECRECY_PROJECT_REDIS_KEY" // 保密项目Key
const val PROJECT_PIPELINE_DIALECT_REDIS_KEY = "project:pipeline:dialect"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不需要此redis 缓存key

@@ -378,4 +378,13 @@ interface ServiceProjectResource {
@QueryParam("englishName")
englishName: List<String>
): Result<List<String>?>

@Operation(summary = "获取项目级流水线方言, 流水线编辑/修改时调用")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

del; 此方法不需要了,方言统一由项目返回

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants