What is Azure Synapse Analytics and how it differs from Data Factory

Challenges and innovations in the IT world on Advanced Technology Days

Advanced Technology Days was held in Zagreb for the 17th time! The conference has become a traditional gathering of IT enthusiasts in the SEE region with an emphasis on new technologies and innovations in the field. This year Unitfly had two presenters: our COO talked about Azure Synapse Analytics, an Azure platform that combines enterprise data warehouse and big data analytics to ensure centralized management of data lakes and warehouses. Seemingly opposite, our Software Engineer Dino Grgic presented the challenges of OCR, which you can read here.

COO Alan Debijadi / Unitfly

We will now get in-depth into Azure Synapse Analytics.

Bulking up with Azure Synapse Analytics

Azure Synapse Analytics is an enterprise analytics service that offers a more efficient way to gain insights across data warehouses and big data systems. It offers the key features of multiple solutions: ETL from data warehouses, big data analytics, and reporting, as well as visualization (achieved by accessing Power BI within the service).

Difference between Synapse Analytics and Data Factory

Azure Synapse Analytics can help you turn big, unstructured data into actionable insights, while Data Factory ensures numerous integrations without the use of code. The main difference between the two services is that Synapse Analytics is an analytics service, and Data Factory is a hybrid data integration service that simplifies the ETL at scale. Data Factory offers the integration of different data sources, but Synapse Analytics serves as a platform from which you can manage, prepare and serve data for BI and Machine Learning purposes with reporting capabilities.

Azure Data Factory offers features such as:

  • real-time integration
  • parallel processing
  • data chunker

On the other hand, Azure Synapse provides:

  • Complete T-SQL-based analytics
  • deeply integrated Apache Spark
  • hybrid data integration

What does Azure Synapse Analytics do?

Ingest – all functionalities of Data Factory and more

Synapse Analytics offers all the possibilities of Data Factory such as the integration of different data sources, but with added functionalities of monitoring, management, alerting, and security in one place.

Explore and analyze – using Synapse SQL

Synapse SQL combines distributed query processing capabilities with Azure Storage to achieve high performance and scalability, offering serverless and dedicated resource models.

Serverless SQL pool

Serverless SQL pool is a query service over the data in your data lake. It enables you to access your data through these functionalities:

  • a familiar T-SQL syntax to query data in place without the need to copy or load data into a specialized store
  • integrated connectivity via the T-SQL interface that offers a wide range of business intelligence and ad-hoc querying tools, including the most popular drivers

Dedicated SQL pool (formerly SQL DW)

Dedicated SQL pool (formerly SQL DW) is a collection of analytic resources that are provisioned when using Synapse SQL. The size of a dedicated SQL pool is determined by Data Warehousing Units (DWU).

The analysis results can go to worldwide reporting databases or applications. Business analysts can then gain insights to make well-informed business decisions.

The other available services are Apache Spark and Data Explorer (still in preview).

Visualization

The main appeal of Synapse Analytics lies in the ability to do everything in one place. Thanks to the native integration with Power BI, data can be instantly visualized in the platform.

Conclusion

Azure Synapse Analytics offers a way to have the whole end-to-end process in one place, from managing, preparing, and serving data for BI and machine learning purposes. Without the need to include additional platforms to import data from different sources, it positions itself as a must-have solution for data engineers.

Introduction to Azure Data Factory

Every successful company bases its decision-making process on data. To make use of all the data flowing around us, we need to understand its core meaning, gather, and analyze it. To ensure you have the data transferred, organized and the whole process scheduled and monitored, the best solution is Azure Data Factory.