Big Data Engineering-Welcome to the course
Introduction to the course
()
Course syllabus
How to be successful in this course
Big Data Engineering-Understand big data engineering with Apache Spark in Azure Synapse Analytics
What is an Apache Spark pool in Azure Synapse Analytics?
()
How do Apache Spark pools in Azure Synapse Analytics?
()
When do you use Apache Spark pools in Azure Synapse Analytics?
Lesson summary
()
Big Data Engineering-Ingest data with Apache Spark notebooks in Azure Synapse Analytics
Introduction to spark notebooks
()
Understand the use-cases for spark notebooks
()
Create a spark notebook in Azure Synapse Analytics
Discover supported languages in spark notebooks
Develop spark notebooks
Develop spark notebooks
Run spark notebooks
()
Run spark notebooks
Load data in Spark notebooks
Load data in Spark notebooks
Save Spark notebooks
Lesson summary
()
Big Data Engineering-Transform data with DataFrames in Apache Spark Pools in Azure Synapse Analytics
Introduction to DataFrames in Spark pools in Azure Synapse Analytics
()
Load data in a Spark DataFrame
()
Load data into a Spark DataFrame
Create an Apache Spark table
Flatten nested structures and explode arrays with Apache Spark
()
Flatten nested structures and explode arrays with Apache Spark in synapse
Lesson summary
()
Query pools and manage workloads in Azure Synapse Analytics-Integrate SQL and Apache Spark pools in Azure Synapse Analytics
Describe the integration methods between SQL and Spark pools in Azure Synapse Analytics
()
Understand the use-cases for SQL and Spark pools integration
()
Authenticate in Azure Synapse Analytics
()
Transfer data between SQL and Spark pool in Azure Synapse Analytics
Authenticate between Spark and SQL pool in Azure Synapse Analytics
Integrate SQL and Spark pools in Azure Synapse Analytics
Externalize the use of Spark pools within Azure Synapse Workspace
()
Transfer data outside the Synapse workspace using the PySpark connector
Transfer data outside the Synapse workspace using the PySpark connector
()
Lesson summary
()
Query pools and manage workloads in Azure Synapse Analytics-Monitor and manage data engineering workloads with Apache Spark in Azure Synapse Analytics
Monitor Spark pools in Azure Synapse Analytics
()
Base-line Apache Spark performance with Apache Spark history server in Azure Synapse Analytics
Optimize Apache Spark jobs in Azure Synapse Analytics
()
Automate scaling of Apache Spark pools in Azure Synapse Analytics
Lesson summary
()
Practice Exam on Perform data engineering with Azure Synapse Apache Spark Pools-Course practice exam
About the practice exam
Course 6 recap
()
Practice Exam on Perform data engineering with Azure Synapse Apache Spark Pools-Course wrap up
Course wrap up
()
Next steps