Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Microsoft is giving away 50,000 FREE Microsoft Certification exam vouchers. Get Fabric certified for FREE! Learn more

Reply
Anonymous
Not applicable

Mirroring in Microsoft Fabric

Hi,

 

I am some ambiguity in understanding Fabric Mirroring. I know that it enables users to effortlessly replicate external data warehouses or databases. But I want to understand does it copy the physical data to Fabric and if no then how does it work?

 

Hoping for a reply.

1 ACCEPTED SOLUTION

Hi @Anonymous 

 

I have checked with the internal team. The internal team has confirmed that:

Mirroring does physically copy the data from source database to OneLake delta parquet format as well as maintain any data changes in the OneCopy in Fabric OneLake.

 

I hope this information helps. Please do let us know if you have any further questions.

 

Thanks.

View solution in original post

10 REPLIES 10
purushotham15
Regular Visitor

Looks like its similar to HTAP capability in synapse where the data can be ingested/copied automatically in dedicated pool whenever there is a change in source system(cosmos/azure sql). It works for all entities which has PK columns. Is it same here? any difference in handling?

v-cboorla-msft
Microsoft Employee
Microsoft Employee

Hi @Anonymous 

 

Thanks for using Microsoft Fabric Community.

No, Microsoft Fabric Mirroring does not physically copy the data to Microsoft Fabric. Instead, it employs a sophisticated process to replicate data without moving it physically. Here’s how it works:

 

Change Data Capture (CDC): Fabric's Mirroring feature uses a technology called Change Data Capture (CDC) to act like a detective. It constantly monitors your external data sources (like databases) and keeps a watchful eye for any changes, such as new entries, updates, or deletions. 

Delta Tables: Instead of copying everything, Mirroring only captures these identified changes. It then cleverly transforms them into a format specifically designed for analysis, often using Delta tables. These tables are like condensed summaries, focusing only on the differences.

OneLake Storage Layer: The transformed changes are then placed in a secure storage area within Fabric called OneLake. OneLake is designed for efficiency, ensuring the data is organized and readily accessible for your analysis needs.

Intelligent Logic: To optimize performance, Fabric employs intelligent logic. It can figure out when your source data has changed and only triggers synchronization for those specific updates. This minimizes unnecessary processing power usage.

Granular Controls: Mirroring offers you granular control. You can choose which specific data (like tables or entities) you want to be reflected in Fabric. This allows you to focus on the most relevant information for your analysis.

For more details please refer : Introducing Mirroring in Microsoft Fabric.

                                               : What's new in Microsoft Fabric?

vcboorlamsft_0-1710746739466.png

 

I hope this information helps. Please do let us know if you have any further questions.

 

Thanks.

 

 

Anonymous
Not applicable

Hi @v-cboorla-msft ,

 

I get the CDC part of mirroring but how is the inital snapshot of data handeled by Fabric. I just want to develop an understanding of it.

 

Hoping for a reply.

Hi @Anonymous 

 

You're right, the initial snapshot in Fabric Mirroring is a crucial step to understand. While CDC handles ongoing changes, the initial snapshot brings your external data into Fabric for the first time. Here's how it's likely handled:

Prioritizing Efficiency: Fabric Mirroring is likely designed for speed and minimal resource usage during the initial data transfer. This could involve:

  • Bulk Loading: Transferring large chunks of data at once instead of individual records, significantly accelerating the process.
  • Partitioned Transfers: Particularly for very large datasets, dividing the data into smaller subsets allows for faster and more efficient transfer.

Potential Configurability:

  • Partial Snapshots: Depending on your configuration, you might have the option to specify which tables or subsets of data to include in the initial snapshot. This allows you to focus on specific data relevant to your needs.

Leveraging Source Capabilities:

  • It's possible that Fabric Mirroring utilizes existing export functionalities within your source database (like Azure Cosmos DB or SQL Server) to expedite the initial data transfer process. This can leverage existing tools optimized for your specific source.

Additional Considerations:

  • Temporary Storage (Possible): The initial data transfer might be staged in temporary storage optimized for speed before being transformed and stored in OneLake, Fabric's data storage solution. This temporary storage would facilitate a faster initial transfer.
  • Gradual Population (Speculative): For exceptionally large datasets, the initial data population might occur gradually to minimize any impact on the source database's performance. This would ensure a smooth initial synchronization process.

 

I hope this information helps. Please do let us know if you have any further questions.

 

Thanks.

Hi @Anonymous 

 

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others.
Otherwise, will respond back with the more details and we will try to help.


Thanks.

Anonymous
Not applicable

Hi @v-cboorla-msft ,

 

So during the inital snapshot Mirroring does copy the data from source to Fabric, correct me if I am wrong.

Hi @Anonymous 

 

Yes, during the initial snapshot, mirroring establishes a synchronized copy of the source data within the Microsoft Fabric Lakehouse. This process ensures that the data in the lakehouse is synchronized with the source system. It’s like creating a mirror image of the data, capturing its current state. Once this initial snapshot is complete, the mirroring process efficiently captures and integrates incremental data updates, enabling efficient data synchronization and ongoing data management. 

 

For details please refer : Fabric Mirroring - public preview

vcboorlamsft_0-1710829357648.png

 

I hope this information helps. Please do let us know if you have any further questions.

 

Thanks.

Anonymous
Not applicable

Hi @v-cboorla-msft ,

 

So mirroring does physically copy the data during the initial snapshot and then ensures that the data is synchronized with the source, right?

Hi @Anonymous 

 

I have checked with the internal team. The internal team has confirmed that:

Mirroring does physically copy the data from source database to OneLake delta parquet format as well as maintain any data changes in the OneCopy in Fabric OneLake.

 

I hope this information helps. Please do let us know if you have any further questions.

 

Thanks.

Hi @Anonymous 

 

Glad that your query got resolved.

Please continue using Fabric Community for any help regarding your queries.

 

Thanks.

Helpful resources

Announcements
MarchFBCvideo - carousel

Fabric Monthly Update - March 2025

Check out the March 2025 Fabric update to learn about new features.

March2025 Carousel

Fabric Community Update - March 2025

Find out what's new and trending in the Fabric community.

"); $(".slidesjs-pagination" ).prependTo(".pagination_sec"); $(".slidesjs-pagination" ).append("
"); $(".slidesjs-play.slidesjs-navigation").appendTo(".playpause_sec"); $(".slidesjs-stop.slidesjs-navigation").appendTo(".playpause_sec"); $(".slidesjs-pagination" ).append(""); $(".slidesjs-pagination" ).append(""); } catch(e){ } /* End: This code is added by iTalent as part of iTrack COMPL-455 */ $(".slidesjs-previous.slidesjs-navigation").attr('tabindex', '0'); $(".slidesjs-next.slidesjs-navigation").attr('tabindex', '0'); /* start: This code is added by iTalent as part of iTrack 1859082 */ $('.slidesjs-play.slidesjs-navigation').attr('id','playtitle'); $('.slidesjs-stop.slidesjs-navigation').attr('id','stoptitle'); $('.slidesjs-play.slidesjs-navigation').attr('role','tab'); $('.slidesjs-stop.slidesjs-navigation').attr('role','tab'); $('.slidesjs-play.slidesjs-navigation').attr('aria-describedby','tip1'); $('.slidesjs-stop.slidesjs-navigation').attr('aria-describedby','tip2'); /* End: This code is added by iTalent as part of iTrack 1859082 */ }); $(document).ready(function() { if($("#slides .item").length < 2 ) { /* Fixing Single Slide click issue (commented following code)*/ // $(".item").css("left","0px"); $(".item.slidesjs-slide").attr('style', 'left:0px !important'); $(".slidesjs-stop.slidesjs-navigation").trigger('click'); $(".slidesjs-previous").css("display", "none"); $(".slidesjs-next").css("display", "none"); } var items_length = $(".item.slidesjs-slide").length; $(".slidesjs-pagination-item > button").attr("aria-setsize",items_length); $(".slidesjs-next, .slidesjs-pagination-item button").attr("tabindex","-1"); $(".slidesjs-pagination-item button").attr("role", "tab"); $(".slidesjs-previous").attr("tabindex","-1"); $(".slidesjs-next").attr("aria-hidden","true"); $(".slidesjs-previous").attr("aria-hidden","true"); $(".slidesjs-next").attr("aria-label","Next"); $(".slidesjs-previous").attr("aria-label","Previous"); //$(".slidesjs-stop.slidesjs-navigation").attr("role","button"); //$(".slidesjs-play.slidesjs-navigation").attr("role","button"); $(".slidesjs-pagination").attr("role","tablist").attr("aria-busy","true"); $("li.slidesjs-pagination-item").attr("role","list"); $(".item.slidesjs-slide").attr("tabindex","-1"); $(".item.slidesjs-slide").attr("aria-label","item"); /*$(".slidesjs-stop.slidesjs-navigation").on('click', function() { var itemNumber = parseInt($('.slidesjs-pagination-item > a.active').attr('data-slidesjs-item')); $($('.item.slidesjs-slide')[itemNumber]).find('.c-call-to-action').attr('tabindex', '0'); });*/ $(".slidesjs-stop.slidesjs-navigation, .slidesjs-pagination-item > button").on('click keydown', function() { $.each($('.item.slidesjs-slide'),function(i,el){ $(el).find('.c-call-to-action').attr('tabindex', '-1'); }); var itemNumber = parseInt($('.slidesjs-pagination-item > button.active').attr('data-slidesjs-item')); $($('.item.slidesjs-slide')[itemNumber]).find('.c-call-to-action').attr('tabindex', '0'); }); $(".slidesjs-play.slidesjs-navigation").on('click', function() { $.each($('.item.slidesjs-slide'),function(i,el){ $(el).find('.c-call-to-action').attr('tabindex', '-1'); }); }); $(".slidesjs-pagination-item button").keyup(function(e){ var keyCode = e.keyCode || e.which; if (keyCode == 9) { e.preventDefault(); $(".slidesjs-stop.slidesjs-navigation").trigger('click').blur(); $("button.active").focus(); } }); $(".slidesjs-play").on("click",function (event) { if (event.handleObj.type === "click") { $(".slidesjs-stop").focus(); } else if(event.handleObj.type === "keydown"){ if (event.which === 13 && $(event.target).hasClass("slidesjs-play")) { $(".slidesjs-stop").focus(); } } }); $(".slidesjs-stop").on("click",function (event) { if (event.handleObj.type === "click") { $(".slidesjs-play").focus(); } else if(event.handleObj.type === "keydown"){ if (event.which === 13 && $(event.target).hasClass("slidesjs-stop")) { $(".slidesjs-play").focus(); } } }); $(".slidesjs-pagination-item").keydown(function(e){ switch (e.which){ case 37: //left arrow key $(".slidesjs-previous.slidesjs-navigation").trigger('click'); e.preventDefault(); break; case 39: //right arrow key $(".slidesjs-next.slidesjs-navigation").trigger('click'); e.preventDefault(); break; default: return; } $(".slidesjs-pagination-item button.active").focus(); }); }); // Start This code is added by iTalent as part of iTrack 1859082 $(document).ready(function(){ $("#tip1").attr("aria-hidden","true").addClass("hidden"); $("#tip2").attr("aria-hidden","true").addClass("hidden"); $(".slidesjs-stop.slidesjs-navigation, .slidesjs-play.slidesjs-navigation").attr('title', ''); $("a#playtitle").focus(function(){ $("#tip1").attr("aria-hidden","false").removeClass("hidden"); }); $("a#playtitle").mouseover(function(){ $("#tip1").attr("aria-hidden","false").removeClass("hidden"); }); $("a#playtitle").blur(function(){ $("#tip1").attr("aria-hidden","true").addClass("hidden"); }); $("a#playtitle").mouseleave(function(){ $("#tip1").attr("aria-hidden","true").addClass("hidden"); }); $("a#play").keydown(function(ev){ if (ev.which ==27) { $("#tip1").attr("aria-hidden","true").addClass("hidden"); ev.preventDefault(); return false; } }); $("a#stoptitle").focus(function(){ $("#tip2").attr("aria-hidden","false").removeClass("hidden"); }); $("a#stoptitle").mouseover(function(){ $("#tip2").attr("aria-hidden","false").removeClass("hidden"); }); $("a#stoptitle").blur(function(){ $("#tip2").attr("aria-hidden","true").addClass("hidden"); }); $("a#stoptitle").mouseleave(function(){ $("#tip2").attr("aria-hidden","true").addClass("hidden"); }); $("a#stoptitle").keydown(function(ev){ if (ev.which ==27) { $("#tip2").attr("aria-hidden","true").addClass("hidden"); ev.preventDefault(); return false; } }); }); // End This code is added by iTalent as part of iTrack 1859082