azure - Dataflow in Synapse pipeline is not reading CSV file correctly - Stack Overflow

admin管理员组

文章数量:1278977

I have a CSV file and it looks like below: (Column Delimiter : +| and row delimiter : &|\n)

This file has two problems : 1st - Both row and column delimiter is 2 characters, and this does not work in both dataflow and copydata activity.

2nd- If I pass only the first character of the delimiter, I get the below preview in dataflow :

Why am I getting these rectangles in the output and why there is only one row?

If I try the same in copy activity, I get the correct output but I don't want to complicate the process and want to do all inside dataflow. Below is the copy activity:

Below are my connection settings:

So my ask is simple, I need to read files that have 2 character column delimiter and 3 char row delimiter, and I want to read these files from dataflow as there are some transformations that I need to do after reading these files. However, the output is not coming correct.

Thanks in advance

I have a CSV file and it looks like below: (Column Delimiter : +| and row delimiter : &|\n)

This file has two problems : 1st - Both row and column delimiter is 2 characters, and this does not work in both dataflow and copydata activity.

2nd- If I pass only the first character of the delimiter, I get the below preview in dataflow :

Why am I getting these rectangles in the output and why there is only one row?

If I try the same in copy activity, I get the correct output but I don't want to complicate the process and want to do all inside dataflow. Below is the copy activity:

Below are my connection settings:

So my ask is simple, I need to read files that have 2 character column delimiter and 3 char row delimiter, and I want to read these files from dataflow as there are some transformations that I need to do after reading these files. However, the output is not coming correct.

Thanks in advance

• azure
• azure-pipelines
• azure-synapse
• azure-synapse-analytics
• azure-synapse-pipeline

Share Improve this question Follow edited Feb 24 at 19:58 sam asked Feb 24 at 1:40 samsam 1,30433 gold badges1515 silver badges3838 bronze badges 2

• @Do you see any ERROR? – Dileep Raj Narayan Thumula Commented Feb 24 at 3:27
• @DileepRajNarayanThumula: No, there is no error. Dataflow is behaving weirdly. Firstly it's giving only first row and second it is printing special characters in the output – sam Commented Feb 24 at 3:36

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

I have used the sample data like you have given below is the image:

Why are you seeing rectangles in Data Flow output? When you specify only the first character of the delimiter + instead of +| for columns or & instead of &| for rows

I have tried the below approach: I have used below in the Column delimiter AND Row delimiter

+|

&|

Below are the parameter values:

Results:

本文标签： azureDataflow in Synapse pipeline is not reading CSV file correctlyStack Overflow