Cache Python Playwright dependencies in Azure Pipelines - Stack Overflow

admin管理员组

文章数量:1122846

I have a playwrite/python automation framework and I'm trying to configure the yaml file. However I'm having no luck. My goal is to have a stage to install all dependencies and then use that stage as a dependency for any stage that I'll be creating.

Here is my current code. It doesn't have any error but doesn't look like it's installing the specified dependencies and running the test on the stages as well.

trigger:
  branches:
    include:
      - main  # Adjust the branch name as needed

pool:
  vmImage: 'windows-latest'  # Same VM image for all stages

stages:
  - stage: InstallDependencies
    displayName: "Install Dependencies"
    jobs:
      - job: Install
        displayName: "Install Dependencies"
        steps:
          # Use the same Python version
          - task: UsePythonVersion@0
            inputs:
              versionSpec: '3.x'  # Adjust according to your Python version requirements
              addToPath: true

          # Set up the virtual environment and install dependencies
          - script: |
              python -m venv .venv
              .venv\Scripts\activate
              python.exe -m pip install --upgrade pip
              python.exe -m pip install behave
              python.exe -m pip install -r requirements.txt
              python -m playwright install
            displayName: "Set up Python Virtual Environment and Install Dependencies"

          # Publish the virtual environment as an artifact
          - task: PublishBuildArtifacts@1
            inputs:
              PathtoPublish: '.venv'  # Path to the virtual environment
              ArtifactName: 'venv'   # Artifact name
              publishLocation: 'Container'

  # Run Inventory Feature Tests
  - stage: InventoryFeature
    dependsOn: InstallDependencies
    displayName: "Run Inventory Feature Tests"
    jobs:
      - job: RunInventory
        displayName: "Run Inventory Feature Tests"
        steps:
          # Use the same Python version
          - task: UsePythonVersion@0
            inputs:
              versionSpec: '3.x'
              addToPath: true

          # Download the virtual environment artifact
          - task: DownloadBuildArtifacts@0
            inputs:
              artifactName: 'venv'
              downloadPath: '$(System.DefaultWorkingDirectory)'

          # Activate the virtual environment and run tests
          - script: |
              $(System.DefaultWorkingDirectory)\venv\Scripts\activate
              behave features/inventory.feature --junit --junit-directory reports/inventory
            displayName: "Run Inventory Feature Tests"

          # Publish test results
          - task: PublishTestResults@2
            inputs:
              testResultsFiles: 'reports/inventory/*.xml'
              testRunTitle: "Inventory Feature Tests"

  # Run Stores Feature Tests
  - stage: StoresFeature
    dependsOn: InstallDependencies
    displayName: "Run Stores Feature Tests"
    jobs:
      - job: RunStores
        displayName: "Run Stores Feature Tests"
        steps:
          # Use the same Python version
          - task: UsePythonVersion@0
            inputs:
              versionSpec: '3.x'
              addToPath: true

          # Download the virtual environment artifact
          - task: DownloadBuildArtifacts@0
            inputs:
              artifactName: 'venv'
              downloadPath: '$(System.DefaultWorkingDirectory)'

          # Activate the virtual environment and run tests
          - script: |
              $(System.DefaultWorkingDirectory)\venv\Scripts\activate
              behave features/stores.feature --junit --junit-directory reports/stores
            displayName: "Run Stores Feature Tests"

          # Publish test results
          - task: PublishTestResults@2
            inputs:
              testResultsFiles: 'reports/stores/*.xml'
              testRunTitle: "Stores Feature Tests"

  # Run Colors Feature Tests
  - stage: ColorsFeature
    dependsOn: InstallDependencies
    displayName: "Run Colors Feature Tests"
    jobs:
      - job: RunColors
        displayName: "Run Colors Feature Tests"
        steps:
          # Use the same Python version
          - task: UsePythonVersion@0
            inputs:
              versionSpec: '3.x'
              addToPath: true

          # Download the virtual environment artifact
          - task: DownloadBuildArtifacts@0
            inputs:
              artifactName: 'venv'
              downloadPath: '$(System.DefaultWorkingDirectory)'

          # Activate the virtual environment and run tests
          - script: |
              $(System.DefaultWorkingDirectory)\venv\Scripts\activate
              behave features/colors.feature --junit --junit-directory reports/colors
            displayName: "Run Colors Feature Tests"

          # Publish test results
          - task: PublishTestResults@2
            inputs:
              testResultsFiles: 'reports/colors/*.xml'
              testRunTitle: "Colors Feature Tests"

or if anyone has better suggestion to achieve the same goal? appreciate any guidance.

I have a playwrite/python automation framework and I'm trying to configure the yaml file. However I'm having no luck. My goal is to have a stage to install all dependencies and then use that stage as a dependency for any stage that I'll be creating.

Here is my current code. It doesn't have any error but doesn't look like it's installing the specified dependencies and running the test on the stages as well.

trigger:
  branches:
    include:
      - main  # Adjust the branch name as needed

pool:
  vmImage: 'windows-latest'  # Same VM image for all stages

stages:
  - stage: InstallDependencies
    displayName: "Install Dependencies"
    jobs:
      - job: Install
        displayName: "Install Dependencies"
        steps:
          # Use the same Python version
          - task: UsePythonVersion@0
            inputs:
              versionSpec: '3.x'  # Adjust according to your Python version requirements
              addToPath: true

          # Set up the virtual environment and install dependencies
          - script: |
              python -m venv .venv
              .venv\Scripts\activate
              python.exe -m pip install --upgrade pip
              python.exe -m pip install behave
              python.exe -m pip install -r requirements.txt
              python -m playwright install
            displayName: "Set up Python Virtual Environment and Install Dependencies"

          # Publish the virtual environment as an artifact
          - task: PublishBuildArtifacts@1
            inputs:
              PathtoPublish: '.venv'  # Path to the virtual environment
              ArtifactName: 'venv'   # Artifact name
              publishLocation: 'Container'

  # Run Inventory Feature Tests
  - stage: InventoryFeature
    dependsOn: InstallDependencies
    displayName: "Run Inventory Feature Tests"
    jobs:
      - job: RunInventory
        displayName: "Run Inventory Feature Tests"
        steps:
          # Use the same Python version
          - task: UsePythonVersion@0
            inputs:
              versionSpec: '3.x'
              addToPath: true

          # Download the virtual environment artifact
          - task: DownloadBuildArtifacts@0
            inputs:
              artifactName: 'venv'
              downloadPath: '$(System.DefaultWorkingDirectory)'

          # Activate the virtual environment and run tests
          - script: |
              $(System.DefaultWorkingDirectory)\venv\Scripts\activate
              behave features/inventory.feature --junit --junit-directory reports/inventory
            displayName: "Run Inventory Feature Tests"

          # Publish test results
          - task: PublishTestResults@2
            inputs:
              testResultsFiles: 'reports/inventory/*.xml'
              testRunTitle: "Inventory Feature Tests"

  # Run Stores Feature Tests
  - stage: StoresFeature
    dependsOn: InstallDependencies
    displayName: "Run Stores Feature Tests"
    jobs:
      - job: RunStores
        displayName: "Run Stores Feature Tests"
        steps:
          # Use the same Python version
          - task: UsePythonVersion@0
            inputs:
              versionSpec: '3.x'
              addToPath: true

          # Download the virtual environment artifact
          - task: DownloadBuildArtifacts@0
            inputs:
              artifactName: 'venv'
              downloadPath: '$(System.DefaultWorkingDirectory)'

          # Activate the virtual environment and run tests
          - script: |
              $(System.DefaultWorkingDirectory)\venv\Scripts\activate
              behave features/stores.feature --junit --junit-directory reports/stores
            displayName: "Run Stores Feature Tests"

          # Publish test results
          - task: PublishTestResults@2
            inputs:
              testResultsFiles: 'reports/stores/*.xml'
              testRunTitle: "Stores Feature Tests"

  # Run Colors Feature Tests
  - stage: ColorsFeature
    dependsOn: InstallDependencies
    displayName: "Run Colors Feature Tests"
    jobs:
      - job: RunColors
        displayName: "Run Colors Feature Tests"
        steps:
          # Use the same Python version
          - task: UsePythonVersion@0
            inputs:
              versionSpec: '3.x'
              addToPath: true

          # Download the virtual environment artifact
          - task: DownloadBuildArtifacts@0
            inputs:
              artifactName: 'venv'
              downloadPath: '$(System.DefaultWorkingDirectory)'

          # Activate the virtual environment and run tests
          - script: |
              $(System.DefaultWorkingDirectory)\venv\Scripts\activate
              behave features/colors.feature --junit --junit-directory reports/colors
            displayName: "Run Colors Feature Tests"

          # Publish test results
          - task: PublishTestResults@2
            inputs:
              testResultsFiles: 'reports/colors/*.xml'
              testRunTitle: "Colors Feature Tests"

or if anyone has better suggestion to achieve the same goal? appreciate any guidance.

• python
• python-3.x
• azure-devops
• azure-pipelines
• playwright

Share Improve this question Follow edited yesterday bryanbcook 17.5k22 gold badges4444 silver badges7373 bronze badges asked yesterday Fadhl Jamal Al-MuhsinFadhl Jamal Al-Muhsin 3133 bronze badges

Add a comment  | 

2 Answers 2

Sorted by: Reset to default 0

Microsoft-hosted agents are run in individual VMs, which are re-imaged after each run.

This means that tools and software not included in the agents must be installed on each job where these are needed.

As @Rui-Jarimba points out, build agents in the Microsoft provided agent pool are recycled between each use so you cannot make assumptions about the runtime environment between jobs.

In your scenario, you could leverage the Cache@2 task to download python dependencies between pipeline executions. The Cache@2 task can persist files based on a "key". In this case, your requirements.txt file. See the pipeline caching documentation for more details on the syntax and behaviour.

The PIP_CACHE_DIR variable is recognized by python tooling. At the end of the pipeline job, the Cache@2 task uploads the contents of the cache directory as an artifact and restores it upon subsequent runs.

variables:
  PIP_CACHE_DIR: $(Pipeline.Workspace)/.pip

stages:
- stage: Test
  jobs:
  - job: RunInventory
    steps:

    # download code from the repository so you have
    # access to your feature files...
    - checkout: self

    # Set the python runtime.
    # if the runtime is one of the supported runtimes on the 
    # the build agent, it will be reused. Otherwise it will be 
    # downloaded and installed
    - task: UsePythonVersion@0
      inputs:
        versionSpec: '3.x'
        addToPath: true
    
    # Setup/Download cache if it exists
    - task: Cache@2
      inputs:
        key: 'python | "$(Agent.OS)" | requirements.txt'
        restoreKeys: | 
          python | "$(Agent.OS)"
          python
        path: $(PIP_CACHE_DIR)
        cacheHitVar: CACHE_RESTORED

    # install dependencies, if needed
    - script: |
        python.exe -m pip install --upgrade pip
        python.exe -m pip install behave
        python.exe -m pip install -r requirements.txt
        python.exe -m playwright install
      displayName: Install Dependencies
      # add condition to only perform this step 
      # only when the cache was not restored
      condition: ne(variables['CACHE_RESTORED'], true)

    - ... # steps to execute your scripts 

This assumes that the dependencies you are installing are included as part of the cache directory. According to the playwright documentation, browser binaries are cached at %USERPROFILE%\AppData\Local\ms-playwright, so you may need two Cache@2 tasks.

本文标签： Cache Python Playwright dependencies in Azure PipelinesStack Overflow