Copy Data from Vertipaq to Lakehouse

[rks]

Hi Community,
Semantic Models in PowerBI (if they are in Import Mode) offer the option to enable OneLake Integration. In that way they create Parquet-Files (https://learn.microsoft.com/de-de/power-bi/enterprise/onelake-integration-overview) in OneLake.

I have a similar scenario: migration. I have now successfully migrated the Semantic Model to a DirectLake Model but the underlying lakehouse is still empty. I don't want to import all data from my onPrem into the lakehouse again, but I would rather like to pull data into the lakehouse from the (vertipaq)-model that is already available in Fabric.
Does semantic-link or any other tool also offer an option to migrate the pure data from Vertipaq-Tables to OneLake-Tables?

Best regards,
Konstantin

[v-vpabbu]

Hi @rks,

As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided for the issue worked? or Let us know if you need any further assistance?
If our response addressed, please mark it as Accept as solution and click Yes if you found it helpful.

Regards,
Vinay Pabbu

[v-vpabbu]

Hi @rks,

May I ask if you have gotten this issue resolved?

If it is solved, please mark the helpful reply or share your solution and accept it as solution, it will be helpful for other members of the community who have similar problems as yours to solve it faster.

Regards,
Vinay Pabbu

[v-vpabbu]

Hi @rks,

As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided for the issue worked? or Let us know if you need any further assistance?
If our response addressed, please mark it as Accept as solution and click Yes if you found it helpful.

Regards,
Vinay Pabbu

[rks]

Hi Vinay, thanks for the answer. But just for understanding this correctly: Isn't the oneLake integration (https://learn.microsoft.com/de-de/power-bi/enterprise/onelake-integration-overview) providing a native way to create parquet-files from an import model?

Assuming this is correct: Let me rephrase the question, can I disable the automatic sync-process and just initially (migrate) copy the data from vertipaq to parquet? Or: Could I keep the parquet files even if I later on disable the integration button?

This statement - according to the docs - copies the data data from vertipaq to onelake, but only if the button is enabled (and thus all subsequent refreshes are reflected in the one lake as well).

{
"export": {
"layout": "delta",
"type": "full",
"objects": [
{
"database": "databaseName"
}
]
}
}

Best regards,
Konstantin

[v-vpabbu]

Hi @rks,

OneLake Integration is meant for continuous sync, not one-time migration. If enabled, it creates Parquet files from an Import Mode semantic model. However, if disabled, it’s unclear whether the existing Parquet files remain available.

If you need a one-time migration, you can manually copy the Parquet files to another location before turning off OneLake Integration.

Regards,

Vinay Pabbu

[rks]

Thanks @v-vpabbu , I will try again. I believe that also the manual copied data will be deleted (at least if it's in the same workspace). 

I will try that on Wednesday and report back to this thread 🙂

[rks]

@v-vpabbu I was able to try out your suggestion. 

1) I enabled OneLake integration for the dataset and all relevant parquet files were created

2) By using the onelake integration for Windows Explorer I copied all parquet files onto my HD

3) I created a Lakehouse in Fabric on a different workspace

4) I copied all the files into the files directory of the lakehouse

5) I used the fabric feature "load to table" to create the table

So here's what I observed: 

The files and the data were loaded into lakehouse-tables. However, there are two issues:
1) the table names are different from the folder names (all small caps instead of camel case)

2) the column headers were lost and were ignored

So it seems despite the fact the data was loaded, it can only be properly used by a high investment to "clean" the data?

Best regards,
Konstantin

[v-vpabbu]

Hi @rks,

1) the table names are different from the folder names (all small caps instead of camel case)

This is a default behavior of Delta Lake and Fabric. Fabric Lakehouse automatically converts table names to lowercase when loading from Parquet. 

2) the column headers were lost and were ignored

This might be a schema detection issue, where column names are being treated as data instead of headers.

Use Dataflow Gen2 to define the schema before loading.

Regards,

Vinay Pabbu

[v-vpabbu · Accepted solution]

Hi @rks,

Thank you for reaching out to Microsoft Fabric Community Forum.

There’s no native way to migrate data from VertiPaq to OneLake, as it stores data in a compressed, internal format optimized for querying, not for extraction.

The best way to get the data into the Lakehouse is to reload it from the original source. While you could use DAX queries to pull data, they’re not practical for large datasets.

If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it!

Regards,
Vinay Pabbu