python - Handling UTF-8 Special Characters in GitLab CICD Pipeline on Windows - Stack Overflow

admin管理员组

文章数量:1387448

I'm creating a merge request in gitlab with a gitlab pipeline. I set special characters in .gitlab-ci.yml and provide it to a python script, which is creating the merge request.
But in the merge request, the characters "äöüß" are displayed as "├ñ├╢├╝├ƒ"

.gitlab-ci.yml

test-job:
  stage: test
  script:
    - python script.py -Test "äöüß"
  
  tags:
    - shell:powershell
    - tools:python

script.py

import argparse
import os
import gitlab

# arguments
parser = argparse.ArgumentParser()
parser.add_argument("-Test", type=str, required=True)
args = parser.parse_args()

# gitlab project
gl = gitlab.Gitlab("https://xxx/", private_token=os.getenv("xxx"), ssl_verify=os.getenv("GITLAB_SSL_CERT"))
project = gl.projects.get(os.getenv("CI_PROJECT_ID"))

# create test merge request
mr = project.mergerequests.create({
    "source_branch": os.getenv("CI_COMMIT_BRANCH"),
    "target_branch": "master",
    "title": "Test-Title",
    "description": args.Test
    })

If I add print(f"input: {args.Test}") to script.py it returns this error:

Traceback (most recent call last):
  File "D:\GitLab-Runner\builds\yxFfC2Egu\0\SPS_Entwicklung\test\ci-dev-test\script.py", line 14, in <module>
    print(f"input: {args.Test}")
    ~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python313\Lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'charmap' codec can't encode character '\u251c' in position 7: character maps to <undefined>

I have a Windows 11 runner with powershell as shell.

• .gitlab.ci.yml is encoded as utf-8
• I have tried different ways of encode()/decode()
• checked the encoding with chardet, what returned utf-8.

I have no clue what I can try next or what I am doing wrong or what I'm not understanding here. Strings in the .gitlab.ci.yml and in python should be handled as utf-8, right? But powershell handels strings as windows-1252 per default? Is this the Problem? I tried also [Console]::OutputEncoding = [Text.UTF8Encoding]::UTF8 but nothing changed. I am happy for every hint.

本文标签： pythonHandling UTF8 Special Characters in GitLab CICD Pipeline on WindowsStack Overflow