Challenges and innovations in the IT world on Advanced Technology Days
Advanced Technology Days was held in Zagreb for the 17th time! The conference has become a traditional gathering of IT enthusiasts in the SEE region with an emphasis on new technologies and innovations in the field. This year Unitfly had two presenters: our COO talked about Azure Synapse Analytics, an Azure platform that combines enterprise data warehouse and big data analytics to ensure centralized management of data lakes and warehouses. Seemingly opposite, our Software Engineer Dino Grgic presented the challenges of OCR, which you can read here.
We will now get in-depth into Azure Synapse Analytics.
Bulking up with Azure Synapse Analytics
Azure Synapse Analytics is an enterprise analytics service that offers a more efficient way to gain insights across data warehouses and big data systems. It offers the key features of multiple solutions: ETL from data warehouses, big data analytics, and reporting, as well as visualization (achieved by accessing Power BI within the service).
Difference between Synapse Analytics and Data Factory
Azure Synapse Analytics can help you turn big, unstructured data into actionable insights, while Data Factory ensures numerous integrations without the use of code. The main difference between the two services is that Synapse Analytics is an analytics service, and Data Factory is a hybrid data integration service that simplifies the ETL at scale. Data Factory offers the integration of different data sources, but Synapse Analytics serves as a platform from which you can manage, prepare and serve data for BI and Machine Learning purposes with reporting capabilities.
Azure Data Factory offers features such as:
- real-time integration
- parallel processing
- data chunker
On the other hand, Azure Synapse provides:
- Complete T-SQL-based analytics
- deeply integrated Apache Spark
- hybrid data integration
What does Azure Synapse Analytics do?
Ingest – all functionalities of Data Factory and more
Synapse Analytics offers all the possibilities of Data Factory such as the integration of different data sources, but with added functionalities of monitoring, management, alerting, and security in one place.
Explore and analyze – using Synapse SQL
Synapse SQL combines distributed query processing capabilities with Azure Storage to achieve high performance and scalability, offering serverless and dedicated resource models.
Serverless SQL pool
Serverless SQL pool is a query service over the data in your data lake. It enables you to access your data through these functionalities:
- a familiar T-SQL syntax to query data in place without the need to copy or load data into a specialized store
- integrated connectivity via the T-SQL interface that offers a wide range of business intelligence and ad-hoc querying tools, including the most popular drivers
Dedicated SQL pool (formerly SQL DW)
Dedicated SQL pool (formerly SQL DW) is a collection of analytic resources that are provisioned when using Synapse SQL. The size of a dedicated SQL pool is determined by Data Warehousing Units (DWU).
The analysis results can go to worldwide reporting databases or applications. Business analysts can then gain insights to make well-informed business decisions.
The other available services are Apache Spark and Data Explorer (still in preview).
Visualization
The main appeal of Synapse Analytics lies in the ability to do everything in one place. Thanks to the native integration with Power BI, data can be instantly visualized in the platform.
Conclusion
Azure Synapse Analytics offers a way to have the whole end-to-end process in one place, from managing, preparing, and serving data for BI and machine learning purposes. Without the need to include additional platforms to import data from different sources, it positions itself as a must-have solution for data engineers.
