End-to-end ML pipeline orchestration combining Airflow DAGs with MLflow tracking.
Connect numerous services and agents, enabling them to communicate asynchronously and reliably, even across different event formats and schemas.
How I built a 1M events/day pipeline using Kafka and Spark Streaming.
End-to-end pipeline ingesting 1M+ events/day using Kafka, Spark Streaming, and Delta Lake.
LLM-powered document assistant using vector embeddings and retrieval augmented generation.
Modular data warehouse transformation layer with CI/CD, lineage, and automated testing.
Time series ML model for commodity price prediction, deployed on AWS SageMaker.