azure - how to set Dynamic add content in Compression type in copy activity - Stack Overflow

admin管理员组

文章数量:1336632

I created the dynamic dataset and linker service. In copy activity i used that dynamic dataset and want to pass parameter in copy activity or use Compression type option as parameter/Dynamic, when i click on Compression type there is drop down menu and found option of add dynamic add content, but unable how to add or use this option .

I implement the answer in azure and found some issue , can you please have a look.

first i created two parameters

After parameters i edit the json as well, you can review it

and pass the parameter as string

But the. main issue is that i am unable to get these parameters into datasets You can check this picture.

I created the dynamic dataset and linker service. In copy activity i used that dynamic dataset and want to pass parameter in copy activity or use Compression type option as parameter/Dynamic, when i click on Compression type there is drop down menu and found option of add dynamic add content, but unable how to add or use this option .

I implement the answer in azure and found some issue , can you please have a look.

first i created two parameters

After parameters i edit the json as well, you can review it

and pass the parameter as string

But the. main issue is that i am unable to get these parameters into datasets You can check this picture.

• azure
• azure-pipelines
• azure-data-factory

Share Improve this question Follow edited Nov 28, 2024 at 11:53 rbrayb 46.8k3434 gold badges118118 silver badges179179 bronze badges Recognized by Microsoft Azure Collective asked Nov 20, 2024 at 22:04 Junaid BilalJunaid Bilal 1755 bronze badges 8

• 1 Can you add images of your pipeline for better understanding – Pratik Lad Commented Nov 21, 2024 at 3:19
• images has been shared i post , thank you for your support and consideration @Prakit Lad – Junaid Bilal Commented Nov 21, 2024 at 6:56
• 1 @JunaidBilal, you can use dynamic expression by editing the dataset json. please check the below answer. – Rakesh Govindula Commented Nov 21, 2024 at 8:18
• 1 Ya, @RakeshGovindula. Thank you for to all community members, i am doing POC on this answer and will share my result with all of you soon. – Junaid Bilal Commented Nov 21, 2024 at 8:45
• 1 i. would like to thank you for your support and time because your reply and provided solution has solved my problem. Thank you @RakeshGovindula – Junaid Bilal Commented Nov 27, 2024 at 17:31

 |  Show 3 more comments

1 Answer 1

Sorted by: Reset to default 1

This behavior might be a bug in the datasets of ADF. You can achieve your requirement by changing the dataset JSON as shown below.

First create the required dataset parameters.

Now, go to the dataset JSON by clicking on the icon {} at the top right corner of the dataset and edit the JSON to add the dynamic content expression.

Here, I have used Binary dataset, and I have edited the dataset JSON as shown below.

{
    "name": "Binary_source_compression",
    "properties": {
        "linkedServiceName": {
            "referenceName": "ADLS_nov20th_test",
            "type": "LinkedServiceReference"
        },
        "parameters": {
            "comp_type": {
                "type": "string"
            },
            "comp_level": {
                "type": "string"
            }
        },
        "annotations": [],
        "type": "Binary",
        "typeProperties": {
            "location": {
                "type": "AzureBlobFSLocation",
                "fileName": "mainzip.zip",
                "fileSystem": "zipsinputcon"
            },
            "compression": {
                "type": {
                    "value": "@dataset()p_type",
                    "type": "Expression"
                },
                "level": {
                    "value": "@dataset()p_level",
                    "type": "Expression"
                }
            }
        }
    }
}

Click on ok and now it will show the dynamic expression for these properties at the dataset level.

Give the required values to these parameters in the copy activity.

Run the pipeline and it will give the desired results.

UPDATE:

For the csv file, you can edit the dataset JSON as below.

        "compressionCodec": {
                "value": "@dataset()p",
                "type": "Expression"
            },
            "compressionLevel": {
                "value": "@dataset()p_level",
                "type": "Expression"
            }

本文标签： azurehow to set Dynamic add content in Compression type in copy activityStack Overflow