Hello Fabric Users!!!
In this article, I will demonstrate how to dynamically load a CSV file with space-delimited headers into a Lakehouse as Delta tables by using Dataflow Gen-2
This is my Fabric workspace in the csvload_LH ,I have Sales Order.csv file (1) and am using Load to Tables (2) and creating New table(3)
Now I will click on Load (4)
It throws an error (5) and clearly telling that, column headers having spaces not allowed. We have to use (6) underscore(_)
This is the dataset I'm trying to load. Here column headers(7) having the spaces. Now here we have only few columns may be we will add underscore manually but what if we have 200+ coulmns in real time.
Now I will create a Dataflow Gen2 called Dataingestion_DF(8) and click on Get data from another source(9)
Now click on OneLake (10) and click on csvload_LH (11)
Now under Files dropdown select Sales Orders.csv (12) and clcik on Create (13)
Now Column headers as showing as rows (14) click on Use first row as headers (15) and renamed as Sales_Orders_TB (16)
Now we have column headers (17) and click on + symbole to add destination (18)
Select Lakehouse (19)
Click on Create (20)
Now choose New table(21) select csvload_LH (22), check the tbale name(23) and click on Next (24)
Now all the column headers having space(25) automaically converted to underscore(_) (26) and click on Save settings (27)
Now destination added (28) and click on Publish (29)
Dataflow published successfully (30) and click on Lakehosue (31)
Now Sales_Orders_TB (32) populated in Tables Section as Delta table. If you see all columns (33) properly aligned with underscore (_)
This is one of way to load the csv files that having column headers with spaces by using dataflows in Fabric.
I hope it will be clear for all.
Thank you for reading this blog.
Regards
Inturi Suparna Babu