Efficient Language Model Compression For Mobile Deployment : A Study on Qwen1.5-1.8b-chat and Phi-3-Mini-4K-Instruct

Abstract

Large language models (LLMs) offer impressive capabilities but are often too computationally demanding for deployment on resource-constrained devices such as mobile platforms. This project investigates the effectiveness and limitations of compression techniques applied to two representative models, Qwen1.5-1.8B-Chat and Phi-3-Mini-4K-Instruct, using different approaches for each architecture. For Qwen1.5-1.8B-Chat, structured pruning and INT8 quantization were applied, achieving significant size reduction with high benchmark retention (88.4% MMLU, 95.9% ARC-Challenge). However, functional evaluation revealed severe degradation, including deterministic output collapse and loss of generative ability, indicating gaps in benchmark-driven evaluation. In contrast, Phi-3-Mini-4K-Instruct was compressed using EntroLLM mixed quantization and entropy-based methods, which preserved both accuracy and generative behavior, demonstrating greater deployment reliability. These findings highlight that compression outcomes are highly model-dependent and that standard benchmarks may obscure critical failures. This work contributes technical insights by identifying architecture-specific vulnerabilities to compression as well as methodological lessons that underscore the need for comprehensive, deployment-aware evaluation frameworks to ensure reliable LLM performance in practice.