Choosing the Right Data Ingestion Method with Microsoft Fabric

Microsoft Fabric provides various options for data ingestion, each with its own strengths and use cases. Understanding which option to use and when can significantly impact your data workflow efficiency. In this video, we'll explore the different data ingestion methods offered by Microsoft Fabric—Dataflow (with and without Fast Copy), Data Pipeline, and Notebooks—and discuss their pros and cons.

Dataflow:

• Fast Copy is not effective with lower volumes of data.

• However, with larger volumes of data, using Fast Copy is recommended.

• Dataflow is the quickest to market but also the most expensive and least performant option.

Data Pipeline:

• Offers low cost and great performance.

• Power User and Developer friendly.

Notebooks:

• Offers low cost and great performance.

• Developer-friendly experience.

• High time to market.

• Notebooks can be used not only for data ingestion but also for tasks like Data Science and Data Engineering.

In order to provide a benchmark for comparison, I conducted tests by copying data from Azure SQL to Lakehouse with no transformations. I compared data ingestion for two different volumes of data—approximately 2 million rows and approximately 70 million rows—using two different SKUs: F64 and F4.

For a detailed analysis and further insights, check out the video.

With this information, you'll be better equipped to choose the right data ingestion method for your specific needs and optimize your data workflow with Microsoft Fabric.

Exploring Copilot's Role in Dataflow: A Dive into its Functionality

In the realm of Power BI Dataflow, we're accustomed to the richness of functionality and the ease with which these functionalities can transform data. But what about Copilot? Can it help us navigate complex operations such as append, merge, pivot, unpivot, and group by?

While we may not need assistance with basic operations like filtering and renaming columns, Copilot's potential in tackling more complex tasks remains to be explored. In this video, we'll delve into Copilot's capabilities and determine its effectiveness in handling these intricate data manipulation operations.

Let's find out together...

How to remove blanks and repeats form tables and charts in parent child hierarchies #powerbi #dax

Should core PowerBI visuals natively support Parent Child hierarchies? Yes... do they? No... We have to deal with blanks and repeats and all kinds of things to make our charts and tables look nice. Well, in this video I offer you my approach to clean up your visualizations so they look exactly the way you want them to look.

this is the DAX code that i used in this video, first we need to create a measure that allows us to understand what level of hierarchy we currently are in.

Current Level = 
ISFILTERED(Org[Level 1]) + 
ISFILTERED(Org[Level 2]) +
ISFILTERED(Org[Level 3]) +
ISFILTERED(Org[Level 4]) + 
ISFILTERED(Org[Level 5]) + 
ISFILTERED(Org[Level 6]) 

Then we write another measure that will allow us to say if we want to keep the current hierarchy member in the visual or not.

mustHide = 
SWITCH
(
    [Current Level],
    1, FALSE(),
    2, ISBLANK(SELECTEDVALUE(Org[Level 2])),
    3, ISBLANK(SELECTEDVALUE(Org[Level 3])),
    4, ISBLANK(SELECTEDVALUE(Org[Level 4])),
    5, ISBLANK(SELECTEDVALUE(Org[Level 5])),
    6, (SELECTEDVALUE(Org[Level 5]) = SELECTEDVALUE(Org[Name/Role])) || ISBLANK(SELECTEDVALUE(Org[Level 5]))
)

And lastly, we take our business measure that we would like to display in our table, matrix or other chart.

Headcount = 
IF
(
    [mustHide], 
    BLANK(), 
    IF
    (
        ISBLANK([Direct Reports]),
        1,
    
        var _path = [Selected Leader]

        RETURN
        CALCULATE
        (
            COUNTROWS(Org), 
            FILTER
            (
                ALL(Org),
                PATHCONTAINS(Org[path], _path) && _path <> Org[ID]
            )
        )
    )
)

and now you are cooking with gas :)