use authentification RSA instead of password in devOps azure using flyway on snowflake - Stack Overflow

admin管理员组

文章数量:1376079

jobs:
  - deployment: deployUat
    timeoutInMinutes: 120
    variables: 
      private_key: $(UAT_SF_PRIVATE_KEY_test)
    pool: 
      vmImage: 'windows-latest'
    environment: "$(System.TeamProject)-uat"
    strategy:
      runOnce:
        deploy:
          steps:
          - task: PowerShell@2
            displayName: 'Clear Flyway Working Directory'
            inputs:
              targetType: 'inline'
              script: |
                Get-ChildItem -Path "$(Pipeline.Workspace)" -Directory -Recurse -Force | 
                Where-Object { $_.Name -like "*.flyway*" } | 
                Remove-Item -Recurse -Force -Verbose
                
          - task: PowerShell@2
            displayName: "Format Private Key"
            name: setPrivateKey  
            inputs:
              targetType: inline
              script: |
                $formattedKey = "$(UAT_SF_PRIVATE_KEY_test)" -replace "@", "`n"
                Write-Host "##vso[task.setvariable variable=formattedKey;isOutput=true]$formattedKey"
                Write-Host "Formatted key value stored"
                
          - task: PowerShell@2
            displayName: Get JDBC URL
            inputs:
              targetType: 'inline'
              script: |
                Write-Host "Formatted key value: $(setPrivateKey.formattedKey)"
                $jdbcUrl = "jdbc:snowflake://xx.xx-europe.azure.snowflakecomputing/?warehouse=$(UAT_SF_WAREHOUSE)&role=$(UAT_SF_ROLE)&db=UAT_$(SF_DATABASE)&authenticator=snowflake_jwt"
                Write-Host "Generated JDBC URL: $jdbcUrl"
                Write-Host "##vso[task.setvariable variable=jdbcUrl;isOutput=true]$jdbcUrl"
            name: property

When i put the key on hard code it works but when i try to get it from library it does not work , i tried to print the value of the variable in the task before but it return nothing. Error : Message : Private key provided is invalid or not supported: Use java.security.interfaces.RSAPrivateCrtKey.class for the private key

Caused by: net.snowflake.client.jdbc.SnowflakeSQLLoggedException: Private key provided is invalid or not supported: Use java.security.interfaces.RSAPrivateCrtKey.class for the private key

jobs:
  - deployment: deployUat
    timeoutInMinutes: 120
    variables: 
      private_key: $(UAT_SF_PRIVATE_KEY_test)
    pool: 
      vmImage: 'windows-latest'
    environment: "$(System.TeamProject)-uat"
    strategy:
      runOnce:
        deploy:
          steps:
          - task: PowerShell@2
            displayName: 'Clear Flyway Working Directory'
            inputs:
              targetType: 'inline'
              script: |
                Get-ChildItem -Path "$(Pipeline.Workspace)" -Directory -Recurse -Force | 
                Where-Object { $_.Name -like "*.flyway*" } | 
                Remove-Item -Recurse -Force -Verbose
                
          - task: PowerShell@2
            displayName: "Format Private Key"
            name: setPrivateKey  
            inputs:
              targetType: inline
              script: |
                $formattedKey = "$(UAT_SF_PRIVATE_KEY_test)" -replace "@", "`n"
                Write-Host "##vso[task.setvariable variable=formattedKey;isOutput=true]$formattedKey"
                Write-Host "Formatted key value stored"
                
          - task: PowerShell@2
            displayName: Get JDBC URL
            inputs:
              targetType: 'inline'
              script: |
                Write-Host "Formatted key value: $(setPrivateKey.formattedKey)"
                $jdbcUrl = "jdbc:snowflake://xx.xx-europe.azure.snowflakecomputing/?warehouse=$(UAT_SF_WAREHOUSE)&role=$(UAT_SF_ROLE)&db=UAT_$(SF_DATABASE)&authenticator=snowflake_jwt"
                Write-Host "Generated JDBC URL: $jdbcUrl"
                Write-Host "##vso[task.setvariable variable=jdbcUrl;isOutput=true]$jdbcUrl"
            name: property

When i put the key on hard code it works but when i try to get it from library it does not work , i tried to print the value of the variable in the task before but it return nothing. Error : Message : Private key provided is invalid or not supported: Use java.security.interfaces.RSAPrivateCrtKey.class for the private key

Caused by: net.snowflake.client.jdbc.SnowflakeSQLLoggedException: Private key provided is invalid or not supported: Use java.security.interfaces.RSAPrivateCrtKey.class for the private key

• azure-devops
• snowflake-cloud-data-platform

Share Improve this question Follow edited Mar 28 at 12:43 DatagGirl asked Mar 27 at 17:13 DatagGirlDatagGirl 2777 bronze badges 5

• Where do you hard code it and where do you use variables? In "Format Private Key" task or in "FlywayCLI@0" task? – Ziyang Liu-MSFT Commented Mar 28 at 7:05
• in flyway task like that - task: FlywayCLI@0 displayName: 'Flyway Repair (Pre-Deploy)' inputs: command: 'repair' workingDirectory: '$(Pipeline.Workspace)/drop/${{ parameters.rootFolder }}' url: '$(property.jdbcUrl)' user: $(UAT_SF_USERNAME) password: | -----BEGIN PRIVATE KEY----- MIIEvAIBADANBgkqhkiG9w0BAQE ... -----END PRIVATE KEY----- commandOptions: – DatagGirl Commented Mar 28 at 9:44
• 1 When you store the same private key in a variable group and pass it to password as a variable, you will encounter an error? Have you tried echo the variable's value inside the pipeline? Is it the same as the hard-coded value? – Ziyang Liu-MSFT Commented Mar 28 at 9:53
• yes i tried,i update my question with what i tried. like teh variable it does not pass from one task to another. im wondering if it because i do not have jobs – DatagGirl Commented Mar 28 at 12:45
• "im wondering if it because i do not have jobs" - No. If you use the output variable in the same job, there is no need to add job when referring to the job. Based on the current situation, we need to check which step is having issue. See my answer below for better troubleshooting. – Ziyang Liu-MSFT Commented Mar 31 at 8:14

Add a comment  | 

2 Answers 2

Sorted by: Reset to default 1

Add some commands in your YAML to print the values of variables to find out what is wrong. For example,

jobs:
  - deployment: deployUat
    timeoutInMinutes: 120
    environment: "$(System.TeamProject)-uat"
    variables: 
      private_key: $(UAT_SF_PRIVATE_KEY_test)
    pool: 
      vmImage: 'windows-latest'
    strategy:
      runOnce:
        deploy:
          steps:
          - task: CmdLine@2
            inputs:
              script: 'echo The value of UAT_SF_PRIVATE_KEY_test is $(UAT_SF_PRIVATE_KEY_test)'
          - task: PowerShell@2
            name: setPrivateKey  
            inputs:
              targetType: 'inline'
              script: |
                $formattedKey = "$(UAT_SF_PRIVATE_KEY_test)" -replace "@", "`n"
                Write-Output "The value of formattedKey is: $formattedKey"
                Write-Host "##vso[task.setvariable variable=newformattedKey;isOutput=true]$formattedKey"

          - task: CmdLine@2
            inputs:
              script: 'echo The value of newformattedKey is $(setPrivateKey.newformattedKey)'

• If it's wrong in the first CmdLine@2 task, check the variable in your variable group. You can share an example of your UAT_SF_PRIVATE_KEY_test , I want to know the format of it.

• If it's wrong in the second PowerShell@2 task, verify whether the way to format secret is correct.

Based on your scripts, you are trying to replace @ in the UAT_SF_PRIVATE_KEY_test with `n . If so, I am not sure whether the actual value of formattedKey is what you expect. Assume the value of UAT_SF_PRIVATE_KEY_test is test@abc.

• The value of $formattedKey in the PowerShell@2 task is multi-line, but output variables do not support multi-line variables, so the value of the output variable is the part before the newline character. As shown in the screenshot:

• In this example, the actual value of output variable newformattedKey is test.

I had problem to pass the variable from one task (powershell) to another one-flyway ) so i used a config file like that :

 - task: PowerShell@2
            displayName: "Format Private Key"
            inputs:
              targetType: inline
              script: |
                $tempFile = "$(Pipeline.Workspace)/formattedKey.txt"
                "$(private_key)" | Out-File $tempFile
                Write-Host "##vso[task.setvariable variable=keyFilePath]$tempFile"

  
          - task: PowerShell@2
            displayName: Get JDBC URL
            inputs:
              targetType: 'inline'
              script: |
                

                $jdbcUrl = "jdbc:snowflake://xx.xx-xx.azure.snowflakecomputing/?warehouse=$(PRD_SF_WAREHOUSE)&role=$(PRD_SF_ROLE)&db=PRD_$(SF_DATABASE)"
                Write-Host "Generated JDBC URL: $jdbcUrl"  # This will display the generated JDBC URL
                Write-Host "##vso[task.setvariable variable=jdbcUrl;isOutput=true]$jdbcUrl"
                
                #private key process
                $formattedKey = Get-Content "$(keyFilePath)" -Raw
                Write-Host "Private key loaded successfully : $formattedKey"

                # Process the key formatting in separate statements

                $formattedKey = $formattedKey -replace "-----BEGIN PRIVATE KEY-----", ""
                $formattedKey = $formattedKey -replace "-----END PRIVATE KEY-----", ""
                $formattedKey = $formattedKey -replace "\s+", " "
                $formattedKey = $formattedKey.Trim()

               
                $formattedKey = "@|@-----BEGIN PRIVATE KEY-----@$formattedKey@-----END PRIVATE KEY-----@"
                Write-Host "Private key formated loaded successfully1: $formattedKey"
                $formattedKey = $formattedKey -replace "@", "`n"
                Write-Host "Private key formated loaded successfully2: $formattedKey"

                $pemPath = "$(Pipeline.Workspace)/snowflake_key.pem" -replace "\\", "/"  
                [System.IO.File]::WriteAllText($pemPath, $formattedKey, [System.Text.Encoding]::UTF8)
                Write-Host "##vso[task.setvariable variable=SNOWFLAKE_KEY_PATH]$pemPath"
                Write-Host "Successfully created PEM file at: $pemPath"  
            name: property

          - task: FlywayCLI@0
            displayName: Flyway Repair (Pre-Deploy)
            inputs:
              command: 'repair'
              workingDirectory: '$(Pipeline.Workspace)/drop/${{parameters.rootFolder}}'
              url: "jdbc:snowflake://xx.xx-xx.azure.snowflakecomputing/?warehouse=$(PRD_SF_WAREHOUSE)&role=$(PRD_SF_ROLE)&db=PRD_$(SF_DATABASE)&private_key_file=$(SNOWFLAKE_KEY_PATH)"
              user: $(PRD_SF_USERNAME)
              ...

本文标签： use authentification RSA instead of password in devOps azure using flyway on snowflakeStack Overflow