Built a scalable data warehouse for Large Language Model (LLM) fine-tuning with a team of 5. The goal: empower Amharic customer-support and engagement using LLM capabilities. We fine-tuned the Llama model on a dataset of Amharic news articles scraped from multiple sources, then stood up an Airflow + Faust + Docker pipeline to keep data flowing and models retrained.
Scalable Data Warehouse for LLM Fine-tuning
Amharic-language customer-support LLM. Fine-tuned Llama on scraped Amharic news with a production-grade Airflow + Faust + Docker pipeline.
Llama 2Hugging FaceApache AirflowFaustDocker
// impact
- +30% sentiment analysis accuracy.
- +40% pipeline efficiency.
- 90% uptime for consistent service delivery.
