Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Microsoft is giving away 50,000 FREE Microsoft Certification exam vouchers. Get Fabric certified for FREE! Learn more

Reply
BiJoe
Helper II
Helper II

Transform data using dbt tutorial, not working

After successfully starting the Hello Airflow job example, I am trying a slightly modified version of this tutorial, in a new Airflow job:

 

https://learn.microsoft.com/en-us/fabric/data-factory/apache-airflow-jobs-dbt-fabric

 

The modification is that I am connecting to the default Warehouse with sample data, instead of the mentioned Contoso warehouse.

 

The problem is that the DAG does not run, after trying multiple schedule intervals. 

 

 

 

Hopefully I have configured the DAG correctly, the only doubt I have is the database field in the profiles YAML file. Should that be the name of the warehouse?

 

EDIT: 

I just discovered that there is a schedule interval set to daily, which means that the DAG will run at midnight on the day it is started, right? Is this a particulary beginner friendly setting for a tutorial? The tutorial also says to "press play", but this is a scheduled DAG right?

 

EDIT 2: I have set the schedule interval to continuous,  and start date two days ago, catchup to False still no DAGs are run. Where can I find the logs?

 

Screenshots:

 

BiJoe_0-1727954804379.png

 

 

BiJoe_2-1727947361476.png

 

BiJoe_1-1727947247973.png

 

BiJoe_0-1727958849181.png

 

 

 

 

 

 

 

1 ACCEPTED SOLUTION

Hi @BiJoe,

Yes, you're correct! When a DAG is set with a daily schedule interval, it will run at midnight on the day it is started. While this can be a common setting, it might not be the most beginner-friendly option for a tutorial since it can lead to confusion about immediate execution.

 

  •  Regarding the "press play" instruction, that typically refers to manually triggering the DAG. You can start a DAG by clicking on the three dots in the Airflow UI and selecting "Trigger DAG."

 

  • As for your edited question about finding the logs, you can access the logs within the Airflow UI. Once you trigger a DAG, you can click on the DAG name, then go to the "Graph View" or "Gantt View," and click on the task you want to inspect. This will lead you to the logs for that specific task or click on the logs button inside a DAG in Airflow UI.

If you still encounter issues with your DAGs not running, double-check your configurations and the Airflow scheduler status.

 

Let me know if you need further assistance!

 

Best regards,
Sai Prudhvi Neelakantam
Data Engineer | 3x Microsoft Certified

LinkedIn: in/saiprudhvineelakantam | geekindata.com

If this helps, kindly mark this as the accepted solution! 👍

View solution in original post

5 REPLIES 5
Floris_Loeffen
New Member

Hi @BiJoe ,

 

Could you please tell me what was the exact solution? I have the same problem you had and I cannot really understand from the 'solved' reply what was the solution....

Hi @Floris_Loeffen I doubt that I set the status so "Solved" myself, not really sure. It was not working properly IIRC. Haven't touched this since back then. 

v-nuoc-msft
Community Support
Community Support

Hi @BiJoe 

 

In Fabric, the database field should be the name of the warehouse you want to connect to. Make sure it matches the default warehouse name in the Fabric with the sample data.

 

If the scheduling interval is set to daily, the DAG will run at midnight (UTC time) on the day it starts.

 

Even if the DAG is a scheduled task, you can trigger it manually by pressing the Play button in Airflow UI.

 

About Monitoring DAG:

 

https://learn.microsoft.com/en-us/fabric/data-factory/apache-airflow-jobs-run-fabric-item-job#monito...

 

Regards,

Nono Chen

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

 

@v-nuoc-msft thanks for replying. 

 

Unfortunately there is no Play button in Airflow UI for the DAG I have replicated from the tutorial. 

 

Thanks for the link about monitoring DAGs, but that does not work so long the DAG does not run properly.

Hi @BiJoe,

Yes, you're correct! When a DAG is set with a daily schedule interval, it will run at midnight on the day it is started. While this can be a common setting, it might not be the most beginner-friendly option for a tutorial since it can lead to confusion about immediate execution.

 

  •  Regarding the "press play" instruction, that typically refers to manually triggering the DAG. You can start a DAG by clicking on the three dots in the Airflow UI and selecting "Trigger DAG."

 

  • As for your edited question about finding the logs, you can access the logs within the Airflow UI. Once you trigger a DAG, you can click on the DAG name, then go to the "Graph View" or "Gantt View," and click on the task you want to inspect. This will lead you to the logs for that specific task or click on the logs button inside a DAG in Airflow UI.

If you still encounter issues with your DAGs not running, double-check your configurations and the Airflow scheduler status.

 

Let me know if you need further assistance!

 

Best regards,
Sai Prudhvi Neelakantam
Data Engineer | 3x Microsoft Certified

LinkedIn: in/saiprudhvineelakantam | geekindata.com

If this helps, kindly mark this as the accepted solution! 👍

Helpful resources

Announcements
MarchFBCvideo - carousel

Fabric Monthly Update - March 2025

Check out the March 2025 Fabric update to learn about new features.

"); $(".slidesjs-pagination" ).prependTo(".pagination_sec"); $(".slidesjs-pagination" ).append("
"); $(".slidesjs-play.slidesjs-navigation").appendTo(".playpause_sec"); $(".slidesjs-stop.slidesjs-navigation").appendTo(".playpause_sec"); $(".slidesjs-pagination" ).append(""); $(".slidesjs-pagination" ).append(""); } catch(e){ } /* End: This code is added by iTalent as part of iTrack COMPL-455 */ $(".slidesjs-previous.slidesjs-navigation").attr('tabindex', '0'); $(".slidesjs-next.slidesjs-navigation").attr('tabindex', '0'); /* start: This code is added by iTalent as part of iTrack 1859082 */ $('.slidesjs-play.slidesjs-navigation').attr('id','playtitle'); $('.slidesjs-stop.slidesjs-navigation').attr('id','stoptitle'); $('.slidesjs-play.slidesjs-navigation').attr('role','tab'); $('.slidesjs-stop.slidesjs-navigation').attr('role','tab'); $('.slidesjs-play.slidesjs-navigation').attr('aria-describedby','tip1'); $('.slidesjs-stop.slidesjs-navigation').attr('aria-describedby','tip2'); /* End: This code is added by iTalent as part of iTrack 1859082 */ }); $(document).ready(function() { if($("#slides .item").length < 2 ) { /* Fixing Single Slide click issue (commented following code)*/ // $(".item").css("left","0px"); $(".item.slidesjs-slide").attr('style', 'left:0px !important'); $(".slidesjs-stop.slidesjs-navigation").trigger('click'); $(".slidesjs-previous").css("display", "none"); $(".slidesjs-next").css("display", "none"); } var items_length = $(".item.slidesjs-slide").length; $(".slidesjs-pagination-item > button").attr("aria-setsize",items_length); $(".slidesjs-next, .slidesjs-pagination-item button").attr("tabindex","-1"); $(".slidesjs-pagination-item button").attr("role", "tab"); $(".slidesjs-previous").attr("tabindex","-1"); $(".slidesjs-next").attr("aria-hidden","true"); $(".slidesjs-previous").attr("aria-hidden","true"); $(".slidesjs-next").attr("aria-label","Next"); $(".slidesjs-previous").attr("aria-label","Previous"); //$(".slidesjs-stop.slidesjs-navigation").attr("role","button"); //$(".slidesjs-play.slidesjs-navigation").attr("role","button"); $(".slidesjs-pagination").attr("role","tablist").attr("aria-busy","true"); $("li.slidesjs-pagination-item").attr("role","list"); $(".item.slidesjs-slide").attr("tabindex","-1"); $(".item.slidesjs-slide").attr("aria-label","item"); /*$(".slidesjs-stop.slidesjs-navigation").on('click', function() { var itemNumber = parseInt($('.slidesjs-pagination-item > a.active').attr('data-slidesjs-item')); $($('.item.slidesjs-slide')[itemNumber]).find('.c-call-to-action').attr('tabindex', '0'); });*/ $(".slidesjs-stop.slidesjs-navigation, .slidesjs-pagination-item > button").on('click keydown', function() { $.each($('.item.slidesjs-slide'),function(i,el){ $(el).find('.c-call-to-action').attr('tabindex', '-1'); }); var itemNumber = parseInt($('.slidesjs-pagination-item > button.active').attr('data-slidesjs-item')); $($('.item.slidesjs-slide')[itemNumber]).find('.c-call-to-action').attr('tabindex', '0'); }); $(".slidesjs-play.slidesjs-navigation").on('click', function() { $.each($('.item.slidesjs-slide'),function(i,el){ $(el).find('.c-call-to-action').attr('tabindex', '-1'); }); }); $(".slidesjs-pagination-item button").keyup(function(e){ var keyCode = e.keyCode || e.which; if (keyCode == 9) { e.preventDefault(); $(".slidesjs-stop.slidesjs-navigation").trigger('click').blur(); $("button.active").focus(); } }); $(".slidesjs-play").on("click",function (event) { if (event.handleObj.type === "click") { $(".slidesjs-stop").focus(); } else if(event.handleObj.type === "keydown"){ if (event.which === 13 && $(event.target).hasClass("slidesjs-play")) { $(".slidesjs-stop").focus(); } } }); $(".slidesjs-stop").on("click",function (event) { if (event.handleObj.type === "click") { $(".slidesjs-play").focus(); } else if(event.handleObj.type === "keydown"){ if (event.which === 13 && $(event.target).hasClass("slidesjs-stop")) { $(".slidesjs-play").focus(); } } }); $(".slidesjs-pagination-item").keydown(function(e){ switch (e.which){ case 37: //left arrow key $(".slidesjs-previous.slidesjs-navigation").trigger('click'); e.preventDefault(); break; case 39: //right arrow key $(".slidesjs-next.slidesjs-navigation").trigger('click'); e.preventDefault(); break; default: return; } $(".slidesjs-pagination-item button.active").focus(); }); }); // Start This code is added by iTalent as part of iTrack 1859082 $(document).ready(function(){ $("#tip1").attr("aria-hidden","true").addClass("hidden"); $("#tip2").attr("aria-hidden","true").addClass("hidden"); $(".slidesjs-stop.slidesjs-navigation, .slidesjs-play.slidesjs-navigation").attr('title', ''); $("a#playtitle").focus(function(){ $("#tip1").attr("aria-hidden","false").removeClass("hidden"); }); $("a#playtitle").mouseover(function(){ $("#tip1").attr("aria-hidden","false").removeClass("hidden"); }); $("a#playtitle").blur(function(){ $("#tip1").attr("aria-hidden","true").addClass("hidden"); }); $("a#playtitle").mouseleave(function(){ $("#tip1").attr("aria-hidden","true").addClass("hidden"); }); $("a#play").keydown(function(ev){ if (ev.which ==27) { $("#tip1").attr("aria-hidden","true").addClass("hidden"); ev.preventDefault(); return false; } }); $("a#stoptitle").focus(function(){ $("#tip2").attr("aria-hidden","false").removeClass("hidden"); }); $("a#stoptitle").mouseover(function(){ $("#tip2").attr("aria-hidden","false").removeClass("hidden"); }); $("a#stoptitle").blur(function(){ $("#tip2").attr("aria-hidden","true").addClass("hidden"); }); $("a#stoptitle").mouseleave(function(){ $("#tip2").attr("aria-hidden","true").addClass("hidden"); }); $("a#stoptitle").keydown(function(ev){ if (ev.which ==27) { $("#tip2").attr("aria-hidden","true").addClass("hidden"); ev.preventDefault(); return false; } }); }); // End This code is added by iTalent as part of iTrack 1859082
Top Kudoed Authors