Data Engineering
Querying BigQuery Tables with Serverless Spark SQL
Dataproc Serverless
Nov 2026 · 8 min
gcp
bq
Dataproc Serverless
Noebooks on Dataproc
This blog BigQuery Notebooks
End-to-end pipeline ingesting 1M+ events/day using Kafka, Spark Streaming, and Delta Lake.
LLM-powered document assistant using vector embeddings and retrieval augmented generation.
Modular data warehouse transformation layer with CI/CD, lineage, and automated testing.
Time series ML model for commodity price prediction, deployed on AWS SageMaker.