Implement 2 level parameter overrides in Azure YAML Pipelines - Stack Overflow

admin管理员组

文章数量:1334908

In a pretty complex YAML setup using templates, I'm trying to define a parameter object containing some default service settings and optional overrides at environment level, like this (example test.yml):

trigger:
- none

pool: TESTPOOL

extends:
  template: pipeline.yml
  parameters:
    DefaultSettings:
      Service1:
        Setting1: Value1
        Setting2: Value2
      Service2:
        Setting1: Value1
        Object:
          Prop1: PropValue1
        Array:
        - ArrayValue1
        Dictionary: #try-to-remove-this
        - Key1: Value1

    Environments:
      Development:
        Settings:
          Service1:
            Setting2: DevelopmentValue2
          Service2:
            Setting1: DevelopmentValue1

      Test:
        Settings:
          Service2:
            Setting1: TestValue1

The idea is to have the usual environment settings defined once and only override where there are differences. The settings are grouped per service because later on in the templates each service is then handled individually.

In pipeline.yml I'm then iterating over environments:

parameters:
- name: DefaultSettings
  type: object
  default: []
- name: Environments
  type: object
  default: []

stages:
- stage: Pipeline
  jobs:
  - job: job
    steps:
    - script: |
        setlocal EnableDelayedExpansion
        echo ENVIRONMENTS: !ENVIRONMENTS!
        echo DEFAULTSETTINGS: !DEFAULTSETTINGS!
      env:
        ENVIRONMENTS: ${{ convertToJson(parameters.Environments) }}
        DEFAULTSETTINGS: ${{ convertToJson(parameters.DefaultSettings) }}

- ${{ each environment in parameters.Environments }}:
  - template: environment.yml
    parameters:
      EnvironmentName: ${{ environment.Key }}
      Environment: ${{ environment.Value }}
      DefaultSettings: ${{ parameters.DefaultSettings }}

And in environment.yml I'm trying to combine the defaults and the overrides into a final Settings object to be passed further into templates doing the actual work. I'm rebuilding this object based on the structure of DefaultSettings:

parameters:
- name: EnvironmentName
  type: string
- name: Environment
  type: object
  default: []
- name: DefaultSettings
  type: object
  default: []

stages:
- stage: ${{ parameters.EnvironmentName }}
  dependsOn:
  - Pipeline
  jobs:
  - job: job
    steps:
    - script: |
        setlocal EnableDelayedExpansion
        echo ENVIRONMENT: !ENVIRONMENT!
        echo DEFAULTSETTINGS: !DEFAULTSETTINGS!
      env:
        ENVIRONMENT: ${{ convertToJson(parameters.Environment) }}
        DEFAULTSETTINGS: ${{ convertToJson(parameters.DefaultSettings) }}
    - template: environment-steps.yml
      parameters:
        Settings:
          ${{ each service in parameters.DefaultSettings }}:
            ${{ service.Key }}:
              ${{ each setting in service.Value }}:
                ${{ setting.Key }}: ${{ coalesce(parameters.Environment.Settings[service.Key][setting.Key], setting.Value) }}

environment-steps.yml just prints the end result:

parameters:
- name: Settings
  type: object
  default: []

steps:
- script: |
    setlocal EnableDelayedExpansion
    echo SETTINGS: !SETTINGS!
  env:
    SETTINGS: ${{ convertToJson(parameters.Settings) }}

The problem is that when I try to run this pipeline, Azure DevOps throws an error that doesn't really help:

/YAML/test/environment.yml (Line: 32, Col: 37): Unable to convert the object to a template token. Actual type 'System.Collections.Generic.List`1[[System.Collections.Generic.KeyValuePair`2[[Microsoft.TeamFoundation.DistributedTask.Pipelines.Yaml.ObjectTemplating.Tokens.ScalarToken, Microsoft.TeamFoundation.DistributedTask.Pipelines.Yaml, Version=19.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a],[Microsoft.TeamFoundation.DistributedTask.Pipelines.Yaml.ObjectTemplating.Tokens.TemplateToke[...]

If I remove the Dictionary setting (see #try-to-remove-this in test.yml), the pipeline runs, but the computed Settings object looks weird when printing convertToJson(parameters.Settings) (see Object and Array properties), although convertToJson(parameters.DefaultSettings) looks fine:

I would guess something goes wrong at the coalesce() but I don't have other ideas.

Do you?

In a pretty complex YAML setup using templates, I'm trying to define a parameter object containing some default service settings and optional overrides at environment level, like this (example test.yml):

trigger:
- none

pool: TESTPOOL

extends:
  template: pipeline.yml
  parameters:
    DefaultSettings:
      Service1:
        Setting1: Value1
        Setting2: Value2
      Service2:
        Setting1: Value1
        Object:
          Prop1: PropValue1
        Array:
        - ArrayValue1
        Dictionary: #try-to-remove-this
        - Key1: Value1

    Environments:
      Development:
        Settings:
          Service1:
            Setting2: DevelopmentValue2
          Service2:
            Setting1: DevelopmentValue1

      Test:
        Settings:
          Service2:
            Setting1: TestValue1

The idea is to have the usual environment settings defined once and only override where there are differences. The settings are grouped per service because later on in the templates each service is then handled individually.

In pipeline.yml I'm then iterating over environments:

parameters:
- name: DefaultSettings
  type: object
  default: []
- name: Environments
  type: object
  default: []

stages:
- stage: Pipeline
  jobs:
  - job: job
    steps:
    - script: |
        setlocal EnableDelayedExpansion
        echo ENVIRONMENTS: !ENVIRONMENTS!
        echo DEFAULTSETTINGS: !DEFAULTSETTINGS!
      env:
        ENVIRONMENTS: ${{ convertToJson(parameters.Environments) }}
        DEFAULTSETTINGS: ${{ convertToJson(parameters.DefaultSettings) }}

- ${{ each environment in parameters.Environments }}:
  - template: environment.yml
    parameters:
      EnvironmentName: ${{ environment.Key }}
      Environment: ${{ environment.Value }}
      DefaultSettings: ${{ parameters.DefaultSettings }}

And in environment.yml I'm trying to combine the defaults and the overrides into a final Settings object to be passed further into templates doing the actual work. I'm rebuilding this object based on the structure of DefaultSettings:

parameters:
- name: EnvironmentName
  type: string
- name: Environment
  type: object
  default: []
- name: DefaultSettings
  type: object
  default: []

stages:
- stage: ${{ parameters.EnvironmentName }}
  dependsOn:
  - Pipeline
  jobs:
  - job: job
    steps:
    - script: |
        setlocal EnableDelayedExpansion
        echo ENVIRONMENT: !ENVIRONMENT!
        echo DEFAULTSETTINGS: !DEFAULTSETTINGS!
      env:
        ENVIRONMENT: ${{ convertToJson(parameters.Environment) }}
        DEFAULTSETTINGS: ${{ convertToJson(parameters.DefaultSettings) }}
    - template: environment-steps.yml
      parameters:
        Settings:
          ${{ each service in parameters.DefaultSettings }}:
            ${{ service.Key }}:
              ${{ each setting in service.Value }}:
                ${{ setting.Key }}: ${{ coalesce(parameters.Environment.Settings[service.Key][setting.Key], setting.Value) }}

environment-steps.yml just prints the end result:

parameters:
- name: Settings
  type: object
  default: []

steps:
- script: |
    setlocal EnableDelayedExpansion
    echo SETTINGS: !SETTINGS!
  env:
    SETTINGS: ${{ convertToJson(parameters.Settings) }}

The problem is that when I try to run this pipeline, Azure DevOps throws an error that doesn't really help:

/YAML/test/environment.yml (Line: 32, Col: 37): Unable to convert the object to a template token. Actual type 'System.Collections.Generic.List`1[[System.Collections.Generic.KeyValuePair`2[[Microsoft.TeamFoundation.DistributedTask.Pipelines.Yaml.ObjectTemplating.Tokens.ScalarToken, Microsoft.TeamFoundation.DistributedTask.Pipelines.Yaml, Version=19.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a],[Microsoft.TeamFoundation.DistributedTask.Pipelines.Yaml.ObjectTemplating.Tokens.TemplateToke[...]

If I remove the Dictionary setting (see #try-to-remove-this in test.yml), the pipeline runs, but the computed Settings object looks weird when printing convertToJson(parameters.Settings) (see Object and Array properties), although convertToJson(parameters.DefaultSettings) looks fine:

I would guess something goes wrong at the coalesce() but I don't have other ideas.

Do you?

• azure-pipelines-yaml

Share Improve this question Follow edited Nov 26, 2024 at 19:27 jazzman asked Nov 20, 2024 at 22:14 jazzmanjazzman 4177 bronze badges 2

• Hi @azzman, can you share the completed yaml files of using objects or arrays? So that I can test and investigate further in my pipeline. – Ziyang Liu-MSFT Commented Nov 25, 2024 at 7:34
• @ZiyangLiu-MSFT can you reproduce it now with the full YAML files? – jazzman Commented Dec 5, 2024 at 8:17

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

Testing based on the yaml files shared by you, I don't have the same error and the pipeline works fine.

azure-pipelines.yml:

trigger:
- none

pool:
  vmImage: ubuntu-latest

extends:
  template: pipeline.yml
  parameters:
    DefaultSettings:
      Service1:
        Setting1: Value1
        Setting2: Value2
      Service2:
        Setting1: Value1

    Environments:
      Development:
        Settings:
          Service1:
            Setting2: DevelopmentValue2
          Service2:
            Setting1: DevelopmentValue1
      Test:
        Settings:
          Service2:
            Setting1: TestValue1

pipeline.yml:

parameters:
- name: DefaultSettings
  type: object
  default: []
- name: Environments
  type: object
  default: []

stages:
- stage: Pipeline
  jobs:
  - job: job
    steps:
    - script: |
        echo "ENVIRONMENTS: ${ENVIRONMENTS}"
        echo "DEFAULTSETTINGS: ${DEFAULTSETTINGS}"
      env:
        ENVIRONMENTS: ${{ convertToJson(parameters.Environments) }}
        DEFAULTSETTINGS: ${{ convertToJson(parameters.DefaultSettings) }}

- ${{ each environment in parameters.Environments }}:
  - template: environment.yml
    parameters:
      Environment: ${{ environment.Value }}
      DefaultSettings: ${{ parameters.DefaultSettings }}

environment.yml:

parameters:
- name: Environment
  type: object
  default: []
- name: DefaultSettings
  type: object
  default: []

stages:
- stage:
  jobs:
  - job: job
    steps:
    - script: |
        echo "ENVIRONMENT: ${ENVIRONMENT}"
        echo "DEFAULTSETTINGS: ${DEFAULTSETTINGS}"
      env:
        ENVIRONMENT: ${{ convertToJson(parameters.Environment) }}
        DEFAULTSETTINGS: ${{ convertToJson(parameters.DefaultSettings) }}


- template: environment-stages.yml
  parameters:
    Settings:
      ${{ each service in parameters.DefaultSettings }}:
        ${{ service.Key }}:
          ${{ each setting in service.Value }}:
            ${{ setting.Key }}: ${{ coalesce(parameters.Environment.Settings[service.Key][setting.Key], setting.Value) }}

environment-stages.yml:

parameters:
- name: Settings
  type: object
  default: []

stages:
- stage: 
  jobs:
  - job: job
    steps:
    - script: |
        echo "Settings: ${SETTINGS}"
      env:
        SETTINGS: ${{ convertToJson(parameters.Settings) }}

Result:

本文标签： Implement 2 level parameter overrides in Azure YAML PipelinesStack Overflow