Introduction: Designing Data Processing Systems
Google Professional Data Engineer course overview
()
Onboard to GCP
()
1. Storage Technology Selection
Open-source vs. Google Cloud managed services
()
Pros and cons of open-source data engineering tools
()
Google Cloud analytics services
()
2. Data Pipeline Design
Data engineering pipelines
()
Google Cloud storage strategy
()
3. Data Warehousing and Processing Migration
Overview GCP storage
()
Optimize for GCP database solutions
()
Prompt engineering for BigQuery
()
Using Google BigQuery with Google Colab
()
Exploring data with Google BigQuery
()
Conclusion: Designing Data Processing Systems
Next steps
()
1. Storage System Implementation
Demo: Google Cloud Shell
()
Demo: Google Cloud Editor
()
Demo: Google CLI SDK
()
Demo: Google gcloud CLI tool
()
Storage comparison
()
2. Pipeline Building and Operationalization
Jack and the Beanstalk as a data pipeline
()
Compare compute offerings
()
Demo: Compute volatility on GCP
()
3. Processing Infrastructure Implementation
The challenges of big data
()
Demo: Extending GCP Cloud Functions
()
Data pipeline triggers
()
Conclusion: Building and Operationalizing Data Processing Systems
Next steps
()
Introduction: Operationalizing Machine Learning (ML) Models
Course overview
()
1. Pre-built ML Models as a Service
Google Colab with TensorFlow Hub
()
Using GCP NLP from the CLI
()
2. Training and Serving Infrastructure Selection
PyTorch pretrained model overview
()
Demo: PyTorch pretrained model
()
Understanding TPUs
()
TPUs as part of technology transition
()
Getting started with Vertex AI
()
Using GCP ML API vision from CLI
()
3. ML Model Measurement, Monitoring, and Troubleshooting
Plan-do-check-act methodology
()
Demo: Load testing with Locust
()
MLOps on GCP
()
Conclusion: Operationalizing Machine Learning Models
Using Google machine learning courses
()
Next steps
()
Introduction: Ensuring Solution Quality
Course overview
()
1. Security and Compliance Design
Integrated data security
()
Understand Rust crate audits by Google
()
The Rust language is secure by design
()
2. Scalability and Efficiency Assurance
Using Bard to enhance productivity
()
Copilot-enabled Rust
()
Continuous integration with Rust and GitHub actions
()
Demo unit test Rust
()
Energy efficiency of Python vs. Rust
()
3. Flexibility and Portability Assurance
What is distroless?
()
Demo: Build and deploy Rust Microservice Cloud Run
()
Demo: App Engine Rust deploy
()
Conclusion: Ensuring Solution Quality
Next steps
()