When I first heard about the term “real-time intelligence” I was envisioning instantaneous data retrieval and large visuals flowing past at breakneck speeds. And, while that might be a part of it, in the world of Microsoft Fabric it can also mean an event driven experience. This means that data can be monitored and automatically provide information based on changes in real time. This means that you do not have to have a process that runs on a certain schedule. When the inputs change you get updated information.
This is facilitated in Microsoft Fabric by the Real Time hub. The hub is the place for all the in-motion (live or near live) data. The hub is designed in Fabric to manage, discover and ingest all of your data streams. There are already a large number of data connectors built into the Real Time hub. The Real Time hub is a single location in the Tenant for all your data streams. Also, you can include Fabric events along with the steaming data.
As of the writing of this article, the following connectors are supported. I would expect this list to grow in the future but this is a great starting point.
|
Streaming data from other clouds |
|
|
Kafka Clusters |
|
|
Database Change Data Capture (CDC) feeds |
|
|
Microsoft streaming sources |
|
|
Fabric events |
|
Once you are within the Real Time hub, you can create Event Streams and you use the Event House or KQL for analytics and to use queries. This leads to dashboards and using the Data Activator. This is a summary of some of the Microsoft documentation about those steps:
- Real-Time hub, a single place to discover, manage and use event steaming data from Fabric, other Microsoft data sources, and external data sources
- Eventstreams for capturing, transforming, and routing real-time events to various destinations with a no-code experience.
- Eventhouse and KQL database are the ideal analytics engine to process data in motion.
- KQL queryset to run queries, view, and customize query results on data.
- Real-time dashboards, which contains a collection of tiles\
- Data Activator that integrates with Real-Time hub, eventstreams, real-time dashboards and KQL querysets, making it seamless to trigger on any patterns or changing in real-time data.
As a Power BI user, I believe that a picture is worth a thousand words. And so, here is a visual that Microsoft provided to help explain this process:
There is already quite a bit of documentation, and the number of articles and Microsoft learning items will only increase over time.
However, a great place to start is this documentation hub for real-time intelligence:
Real-Time Intelligence in Microsoft Fabric documentation - Microsoft Fabric | Microsoft Learn