Data Engineering Vol2 AWS Data Processing - Spark & Kafka
Build batch and streaming data pipelines with Spark, Kafka, AWS EMR, MSK, and Iceberg
Created by Soumyadeep Dey
Explore how to build robust data pipelines using Spark and Kafka on AWS. You'll work with both batch and streaming data, learning to process large datasets and connect storage with analytics tools. Real-world projects help you develop practical skills for modern data engineering challenges.
Packt | Jul 2026 | 2847 min
What You Will Learn
You'll start with foundational concepts in big data processing, then dive into Spark architecture, transformations, and SQL. Step-by-step demonstrations show how to use Kafka for streaming and Spark for batch workloads on AWS. Hands-on projects reinforce your understanding by tackling real data engineering scenarios.
Key Features
- Design and optimize Spark and Kafka pipelines for batch and streaming data processing
- Integrate AWS services like EMR, MSK, S3, and Redshift for scalable workflows
- Apply tuning and troubleshooting techniques to handle large datasets efficiently
Target Audience
If you're a Python developer, data engineer, analyst, or architect looking to expand your skills in cloud-based data processing, this course is for you. Some experience with AWS and SQL will help you get the most out of the material. Ideal for those aiming to design, build, and optimize modern data pipelines.
Related courses
