/Logos/ae-logo-pos-rgb.svg){kind=logo}
In early 2024, Microsoft hosted its inaugural Microsoft Fabric Community Conference in Las Vegas, and in the autumn, the European edition followed. The purpose of these events was to bring together the vibrant Microsoft Fabric community and share the latest insights and innovations surrounding the product and its ecosystem. With this in mind, AE’s Microsoft Fabric experts, Thibauld Croonenborghs and Jeroen Ruytings, attended the Stockholm event to gather key learnings. In this blog, they share their observations and reflections.
Their story is split into four sections:
1. Why should your CIO and CDO consider Microsoft Fabric?
2. What are the key benefits for business users?
3. How can developers make their Fabric journey more successful?
4. What can admins do to better maintain and organise Fabric instances?
Why should your CIO and CDO consider adopting Microsoft Fabric?
A unified solution for data and analytics
Microsoft Fabric stands out as a unified solution in the realm of data and analytics. It offers a straightforward starting point for organisations, allowing them to scale gradually. Features like mirroring and shortcuts simplify the process of integrating existing data into OneLake without duplicating it. This ease of integration extends to other platforms like AWS, GCP, and in future updates, even SAP and Oracle. Microsoft has also improved compatibility with Databricks and Snowflake, further establishing Fabric as a central player in the data ecosystem.
Security and protection
Building on this, Microsoft Fabric ensures robust security. Previously, organisations had to manually set up their infrastructure when building a data platform, but Fabric now takes care of this, managing authentication, network configuration, and anomaly detection in user behaviour.
Moreover, with Microsoft Purview, data protection is further enhanced. By defining business domains, adding sensitivity labels, and attaching policies, organisations can control who can access or modify their data. While Purview involves some costs, the investment is worth it when compared to the potential risks of a data breach.
Enhancing data quality
Data quality is another advantage offered by Purview. It allows data domain owners to set natural language quality rules, which can then be monitored through dashboards to ensure consistency. As we all know, poor data leads to inaccurate insights and flawed AI models. Given the rapid pace of new feature releases in Microsoft Fabric, organisations currently using Synapse should already be planning their transition.
Preparing for the AI era
And let’s not forget the rise of AI. Organisations that delay integrating AI risk falling behind, while Microsoft Fabric offers a secure and efficient way to deploy AI models, ensuring sensitive data is protected and AI applications are used responsibly.
What are the key benefits of Microsoft Fabric for business users?
A tailored experience for business users
Microsoft has taken significant steps to simplify interactions for business users by creating personas that engage with Fabric in a more focused way. Previously, business users were often overwhelmed by data lakes and warehouses. Now, they have a streamlined interface that allows them to work with data prepared by technical teams, enabling them to concentrate on deriving insights.
The power of Copilot
In addition, Microsoft has introduced Copilot, an innovative tool that allows users to interact with data platforms using natural language. Business users can now ask questions like, “What was my revenue in 2023?” or even request a dashboard on revenue trends over the last decade.
This feature holds great promise, although early testing indicates it doesn’t always perform as expected. Furthermore, Copilot comes with additional costs, and as with any data-driven tool, ensuring high data quality is crucial to obtaining reliable results.
Empowering data ownership with Purview
Purview further empowers business users by providing a clean platform to manage their data products within specific business domains. Teams can define policies, procedures, and data quality standards, enabling them to take ownership of their data. This also fosters collaboration, as data sets can be endorsed and shared across different teams within the organisation.
Activating data with no-code solutions
Beyond that, business teams can utilise no-code solutions like Data Activator to respond quickly to critical business events. For instance, they can receive email or Teams notifications when a specific event occurs in their data, allowing swift action. Similarly, alerts are triggered when data quality falls, ensuring timely intervention.
Real-Time solutions for critical insights
The advancements in real-time solutions are also notable, with technology becoming more mature in this area. Several real-time use cases were demonstrated at the conference, showing how organisations can better monitor manufacturing processes, maintain machines, and track events in the entertainment industry.
However, it's essential to evaluate whether real-time is the best fit for your use case, as not all implementations have been successful.
How can developers make their Fabric journey more successful?
Infrastructure as code
Microsoft continues to innovate, making life easier for developers working with Fabric. The introduction of a Terraform provider is a game-changer, enabling developers to deploy infrastructure using code. This simplifies the process of setting up infrastructure for new departments or moving projects from development to production. Support for ARM, Bicep, Python, and Java SDKs is also in the pipeline, further enhancing development capabilities.
Enhancements in the spark world
In the world of Spark, high-concurrency mode was introduced, allowing developers to reuse Spark sessions between notebooks. While this feature is not yet available for Spark job definitions, Microsoft has promised a fourfold speed increase with its native Spark engine—a feature we are eager to test.
Improved testing capabilities
Testing remains a cornerstone of good software engineering, and developers now have better tools for this. A newly released Python package offers a powerful solution for unit testing Data Factory Pipelines. Additionally, Git integration across Microsoft Fabric means proper version management is now possible, with multiple branching strategies available to suit different workflows. Developers can choose between more UI-driven or code-driven approaches, ensuring flexibility in how they manage their projects.
New features in PowerBI
And on a lighter note—dark mode is now available for PowerBI!
What can admins do to better maintain and organise Fabric instances?
Security and log management
For administrators, security remains a top priority. Microsoft Fabric makes it easier to delegate security responsibilities to business domain owners through Microsoft Purview, allowing them to define access policies and manage data sensitivity levels. A key task for admins is ensuring that logs are properly collected and stored securely, as Microsoft services only retain logs for 30 days.
Capacity management
Capacity management is another area where admins play a crucial role. During the conference, we heard stories of businesses accidentally burning through capacity units due to poor monitoring. To avoid such issues, it's vital to monitor capacity usage closely, educate users about the implications of their actions, and pause capacity when it’s not in use. Capacity reservations are also worth exploring as a cost-saving measure.
Automating tasks with APIs
Admins can further streamline operations by leveraging Microsoft Fabric’s APIs, which allow for automating tasks such as user management, workspace setup, and security configuration. Combined with the Terraform provider, admins can now manage infrastructure as code, making processes repeatable and more efficient.
Optimising costs
Lastly, admins are in a unique position to optimise costs. With access to detailed billing overviews, they can monitor dashboard refresh frequencies and ensure resources are used wisely, preventing unnecessary expenses. If you want to learn more about saving costs with Fabric, check out our blog post.
Want to learn more?
If you're eager to stay updated on all things Microsoft Fabric, make sure to subscribe to AE’s Data & AI mailings. We promise only relevant content—no boring sales pitches!
