Empirical Optimization of Meta-Llama-3.2-3B for the Malaysian TVET Ecosystem: A Case Study in Sovereign AI Specialization

Abstract

The General-purpose frontier models frequently exhibit Out-of-Distribution (OOD) hallucinations when applied to localized institutional domains, such as the Malaysian POLYCC (Polytechnics and Community Colleges) and TVET Madani ecosystems. This study presents an empirical hyperparameter optimization of the Meta-Llama-3.2-3B Small Language Model (SLM) to bridge this domain knowledge gap. Utilizing a Supervised Fine-Tuning (SFT) pipeline on AWS SageMaker platform, we specialized the architecture using a high-density dataset of 2,700 synthetically distilled instruction-response pairs. Through a systematic 10-trial experimental sweep, we isolated the interaction between LoRA Rank (ꭇ), learning rate, and regularization. Performance was validated via a competitive leaderboard-driven evaluation against 50 hidden domain-specific questions. Our findings identify an optimal performance frontier (Trial 7), where a high-capacity configuration (ꭇ=256), a stable learning rate (5 x 10⁵), and a precise dropout (0.03) yielded a winning 64% blind win rate. These results demonstrate that a 3B-parameter model can achieve high reasoning density and factual accuracy when its hyperparameter architecture is aligned with the logical complexity of the target domain. This study provides a validated technical roadmap for the deployment of sovereign, localized AI assistants within the Malaysian TVET ecosystem, establishing a scalable blueprint for broader institutional AI adoption.