AI
Exploring the intersection of LLMs, agentic workflows, and real-world utility.
Philosophy
I prioritize building AI systems that augment human capabilities safely and transparently.
Engineering Logs
A chronological record of milestones, learnings, and builds.
2026-05-15Learning
Fine-Tuning Llama 3 for Code Generation
Spent the weekend exploring fine-tuning techniques for local code generation.
Why Llama 3?
I chose the 8B parameter model because it offers the perfect balance of reasoning capability and VRAM requirements for local execution. Running inference on a single RTX 4090 was surprisingly smooth.
The Pipeline
- Data Collection: Curated a dataset of 10k highly-rated open-source React components.
- Pre-processing: Stripped out comments and standardized formatting.
- LoRA Configuration: Set rank to 16 and alpha to 32.
Results
After 4 epochs, the model successfully learned to output tailwind classes matching my specific design system. However, context window limitations meant it struggled with multi-file architectures. Next up: RAG integration!
Next Steps
- Build an LSP integration
- Test on a 70B model via cloud API
- Publish findings