Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Microsoft is giving away 50,000 FREE Microsoft Certification exam vouchers. Get Fabric certified for FREE! Learn more

Reply
eduardo_schmitz
Regular Visitor

Is it possible to insert data into fabric datawarehouse using a script that isnt in fabric notebook?

I have a small pipeline that gets data from mongo using lambda functions and glue and save these files in S3. Then i get the parquet files in S3 using fabric connectors. But I think It would be better to simply use the script to insert data into my fabric DW. Is there a way to do it from AWS?

9 REPLIES 9
v-veshwara-msft
Community Support
Community Support

Hi @eduardo_schmitz ,

We’re following up once more regarding your query. If it has been resolved, please mark the helpful reply as the Accepted Solution to assist others facing similar challenges.

If you still need assistance, please let us know.
Thank you.

alia12
Regular Visitor

Yes, it's possible to insert data into Microsoft Fabric Data Warehouse without using Fabric Notebooks. Since your pipeline already retrieves data from MongoDB using AWS Lambda and AWS Glue, you have a few options to push data directly into Fabric DW:

 

1. You can set up an ODBC/JDBC connection from AWS to Fabric DW and execute INSERT INTO or MERGE statements. This method requires configuring a secure connection on the AWS side.

 

2. If you’re using Azure services in combination with AWS, Synapse Link can streamline data movement from AWS storage to Fabric DW.

 

Another option is to use Microsoft Fabric’s REST APIs to push data into your DW without relying on Fabric notebooks.

 

If you prefer a script-based approach directly from AWS, using ODBC/JDBC or APIs would be the best choice. Have you considered trying a direct connection from Glue or Lambda via ODBC?

yes, a script-based approach would be ideal. My data requires little transformation, so I would much rather push it directly into the DW Fabric using Lambda or Glue.

However i'm not familiar with using ODBC/JDBC or fabric API just yet. Thank you very much for the suggestion. I will study this approach.
if you have an article about this, i'd appreciate, otherwise i'm alreayding accepting this as solution.

 

you can consider also the brand new spark connector for warehouse.
Spark connector for Microsoft Fabric Data Warehouse - Microsoft Fabric | Microsoft Learn


--
Riccardo Perico
BI Architect @ Lucient Italia

Blog | GitHub

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Sure! Since your data doesn’t need heavy transformation, a script-based insert makes total sense. You can definitely do this using Python or Node.js from AWS Lambda or Glue, you'd just need to first exchange credentials for an access token (using Microsoft Entra ID), and then call the Fabric REST API to insert the data. It’s a bit of a setup, but once you’ve done the OAuth flow and understand the structure of the Fabric API, it becomes straightforward. I haven’t seen a full walkthrough yet, but the official docs - https://learn.microsoft.com/en-us/fabric/ cover the endpoints and auth setup well.

 

 

Hi @eduardo_schmitz ,
Thanks for the update,
To help you get started with ODBC/JDBC connections to Fabric Data Warehouse, here are some useful resources:

1. Connectivity to data warehousing - Microsoft Fabric | Microsoft Learn

2. Set up your Odbc connection - Microsoft Fabric | Microsoft Learn

3. Microsoft ODBC Driver for SQL Server - ODBC Driver for SQL Server | Microsoft Learn

4. Microsoft JDBC Driver for SQL Server - JDBC Driver for SQL Server | Microsoft Learn

 

If you're considering Fabric's REST API as an alternative, you can explore:
Microsoft Fabric REST API references - Microsoft Fabric REST APIs | Microsoft Learn

 

Hope these help. Please reach out for further assistance.

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly and a kudos would be appreciated.

Thank you.

Hi @eduardo_schmitz ,

We haven’t heard from you since our last response and wanted to check if the provided solutions met your needs. If they did, please consider accepting the solution to help others benefit. Otherwise, feel free to reach out for further assistance.

Thank you.

Hi @eduardo_schmitz ,

Just checking in again as we haven’t heard back from you. If the provided responses addressed your issue, kindly mark the helpful reply as the Accepted Solution to assist others with similar queries.

If your issue is still unresolved or you need further help, feel free to reach out.
Thank you.

v-veshwara-msft
Community Support
Community Support

Hi @eduardo_schmitz ,

Thank you for reaching out in Microsoft Fabric Community.
Based on your scenario, there are multiple ways to ingest data from AWS into Microsoft Fabric Data Warehouse:

1. Direct Data Ingestion from AWS
You can establish a connection from AWS services (such as AWS Lambda, AWS Glue) to Fabric Data Warehouse using ODBC/JDBC. This allows data to be inserted directly into Fabric DW using INSERT INTO or MERGE statements. However, setting up and maintaining the connection falls under AWS configurations.

 

2. Using Fabric Data Pipelines.
Microsoft Fabric offers Data Pipelines to ingest data from AWS S3 using Fabric's S3 connector.
This method enables a no-code/low-code approach to move data into Fabric Data Warehouse efficiently.
Set up your Amazon S3 connection - Microsoft Fabric | Microsoft Learn

Since setting up a direct connection from AWS to Fabric DW primarily involves AWS-side configurations, I recommend posting this in the AWS Community Forum for more insights from AWS experts:
  AWS re:Post Community → https://repost.aws/

 

Hope this helps. Please reach out for further assistance.

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly and a kudos would be appreciated.

 

Best Regards,
Vinay.

Helpful resources

Announcements
MarchFBCvideo - carousel

Fabric Monthly Update - March 2025

Check out the March 2025 Fabric update to learn about new features.

Notebook Gallery Carousel1

NEW! Community Notebooks Gallery

Explore and share Fabric Notebooks to boost Power BI insights in the new community notebooks gallery.

April2025 Carousel

Fabric Community Update - April 2025

Find out what's new and trending in the Fabric community.

"); $(".slidesjs-pagination" ).prependTo(".pagination_sec"); $(".slidesjs-pagination" ).append("
"); $(".slidesjs-play.slidesjs-navigation").appendTo(".playpause_sec"); $(".slidesjs-stop.slidesjs-navigation").appendTo(".playpause_sec"); $(".slidesjs-pagination" ).append(""); $(".slidesjs-pagination" ).append(""); } catch(e){ } /* End: This code is added by iTalent as part of iTrack COMPL-455 */ $(".slidesjs-previous.slidesjs-navigation").attr('tabindex', '0'); $(".slidesjs-next.slidesjs-navigation").attr('tabindex', '0'); /* start: This code is added by iTalent as part of iTrack 1859082 */ $('.slidesjs-play.slidesjs-navigation').attr('id','playtitle'); $('.slidesjs-stop.slidesjs-navigation').attr('id','stoptitle'); $('.slidesjs-play.slidesjs-navigation').attr('role','tab'); $('.slidesjs-stop.slidesjs-navigation').attr('role','tab'); $('.slidesjs-play.slidesjs-navigation').attr('aria-describedby','tip1'); $('.slidesjs-stop.slidesjs-navigation').attr('aria-describedby','tip2'); /* End: This code is added by iTalent as part of iTrack 1859082 */ }); $(document).ready(function() { if($("#slides .item").length < 2 ) { /* Fixing Single Slide click issue (commented following code)*/ // $(".item").css("left","0px"); $(".item.slidesjs-slide").attr('style', 'left:0px !important'); $(".slidesjs-stop.slidesjs-navigation").trigger('click'); $(".slidesjs-previous").css("display", "none"); $(".slidesjs-next").css("display", "none"); } var items_length = $(".item.slidesjs-slide").length; $(".slidesjs-pagination-item > button").attr("aria-setsize",items_length); $(".slidesjs-next, .slidesjs-pagination-item button").attr("tabindex","-1"); $(".slidesjs-pagination-item button").attr("role", "tab"); $(".slidesjs-previous").attr("tabindex","-1"); $(".slidesjs-next").attr("aria-hidden","true"); $(".slidesjs-previous").attr("aria-hidden","true"); $(".slidesjs-next").attr("aria-label","Next"); $(".slidesjs-previous").attr("aria-label","Previous"); //$(".slidesjs-stop.slidesjs-navigation").attr("role","button"); //$(".slidesjs-play.slidesjs-navigation").attr("role","button"); $(".slidesjs-pagination").attr("role","tablist").attr("aria-busy","true"); $("li.slidesjs-pagination-item").attr("role","list"); $(".item.slidesjs-slide").attr("tabindex","-1"); $(".item.slidesjs-slide").attr("aria-label","item"); /*$(".slidesjs-stop.slidesjs-navigation").on('click', function() { var itemNumber = parseInt($('.slidesjs-pagination-item > a.active').attr('data-slidesjs-item')); $($('.item.slidesjs-slide')[itemNumber]).find('.c-call-to-action').attr('tabindex', '0'); });*/ $(".slidesjs-stop.slidesjs-navigation, .slidesjs-pagination-item > button").on('click keydown', function() { $.each($('.item.slidesjs-slide'),function(i,el){ $(el).find('.c-call-to-action').attr('tabindex', '-1'); }); var itemNumber = parseInt($('.slidesjs-pagination-item > button.active').attr('data-slidesjs-item')); $($('.item.slidesjs-slide')[itemNumber]).find('.c-call-to-action').attr('tabindex', '0'); }); $(".slidesjs-play.slidesjs-navigation").on('click', function() { $.each($('.item.slidesjs-slide'),function(i,el){ $(el).find('.c-call-to-action').attr('tabindex', '-1'); }); }); $(".slidesjs-pagination-item button").keyup(function(e){ var keyCode = e.keyCode || e.which; if (keyCode == 9) { e.preventDefault(); $(".slidesjs-stop.slidesjs-navigation").trigger('click').blur(); $("button.active").focus(); } }); $(".slidesjs-play").on("click",function (event) { if (event.handleObj.type === "click") { $(".slidesjs-stop").focus(); } else if(event.handleObj.type === "keydown"){ if (event.which === 13 && $(event.target).hasClass("slidesjs-play")) { $(".slidesjs-stop").focus(); } } }); $(".slidesjs-stop").on("click",function (event) { if (event.handleObj.type === "click") { $(".slidesjs-play").focus(); } else if(event.handleObj.type === "keydown"){ if (event.which === 13 && $(event.target).hasClass("slidesjs-stop")) { $(".slidesjs-play").focus(); } } }); $(".slidesjs-pagination-item").keydown(function(e){ switch (e.which){ case 37: //left arrow key $(".slidesjs-previous.slidesjs-navigation").trigger('click'); e.preventDefault(); break; case 39: //right arrow key $(".slidesjs-next.slidesjs-navigation").trigger('click'); e.preventDefault(); break; default: return; } $(".slidesjs-pagination-item button.active").focus(); }); }); // Start This code is added by iTalent as part of iTrack 1859082 $(document).ready(function(){ $("#tip1").attr("aria-hidden","true").addClass("hidden"); $("#tip2").attr("aria-hidden","true").addClass("hidden"); $(".slidesjs-stop.slidesjs-navigation, .slidesjs-play.slidesjs-navigation").attr('title', ''); $("a#playtitle").focus(function(){ $("#tip1").attr("aria-hidden","false").removeClass("hidden"); }); $("a#playtitle").mouseover(function(){ $("#tip1").attr("aria-hidden","false").removeClass("hidden"); }); $("a#playtitle").blur(function(){ $("#tip1").attr("aria-hidden","true").addClass("hidden"); }); $("a#playtitle").mouseleave(function(){ $("#tip1").attr("aria-hidden","true").addClass("hidden"); }); $("a#play").keydown(function(ev){ if (ev.which ==27) { $("#tip1").attr("aria-hidden","true").addClass("hidden"); ev.preventDefault(); return false; } }); $("a#stoptitle").focus(function(){ $("#tip2").attr("aria-hidden","false").removeClass("hidden"); }); $("a#stoptitle").mouseover(function(){ $("#tip2").attr("aria-hidden","false").removeClass("hidden"); }); $("a#stoptitle").blur(function(){ $("#tip2").attr("aria-hidden","true").addClass("hidden"); }); $("a#stoptitle").mouseleave(function(){ $("#tip2").attr("aria-hidden","true").addClass("hidden"); }); $("a#stoptitle").keydown(function(ev){ if (ev.which ==27) { $("#tip2").attr("aria-hidden","true").addClass("hidden"); ev.preventDefault(); return false; } }); }); // End This code is added by iTalent as part of iTrack 1859082
Top Solution Authors