It is being widely written that there is growing interest in Small Language Models (SLMs) as recent trends indicate increased popularity. Nvidia recently stated “the small, rather than large, language models are the future of agentic AI.” Its position is that “small language models are sufficiently powerful, inherently more suitable, and necessarily more economical for many invocations in agentic systems.” Small models “provide significant benefits in cost-efficiency, adaptability, and deployment flexibility and possess greater operational flexibility in comparison to LLMs.”
SLMs are gaining traction due to their potential for high efficiency. Technically speaking, Deloitte’s Jim Rowan states in a WSJ article: “SLMs offer a way to unlock a whole category of AI features and functionality offline and at the edge, generally requiring less expensive hardware and consuming far less energy.” Supporting this, Tirias Research found that moving 20% of the GenAI workload to the edge would save $16 billion dollars in 2028.
IBM defines SLMs as being smaller in scale and scope than large language models and are more compact and efficient. SLMs are ideal for resource-constrained environments such as edge devices and mobile apps, or when responses to a user’s query must be done offline. Hugging Face notes that SLMs typically range from 1 million to 10 billion parameters, compared to large language models that have “hundreds of billions – or even trillions – of parameters,” while still retaining core NLP capabilities (text generation, summarization, translation, and question-answering).
Reasons Researchers Will Want To Keep An Eye on SLM Development
Promising elements of SLMs for business researchers to consider include the usage of higher quality and more domain specific data, less hallucinations, and greater capabilities around autonomy.
- Quanta Magazine’s Stephen Ornes writes that through the knowledge distillation approach, SLMs use high-quality data from knowledge transferred from a large model to a smaller one. “The reason [SLMs] get so good with such small models and such little data is that they use high-quality data instead of the messy stuff.”
- HBR notes that SLMs are trained on “a narrower range of data,” which can be “fine-tuned for specific tasks or industries, like healthcare, legal work, or managing supply chains,” thus making them less prone to hallucinate or generate incorrect results.
- Stu Robarts noted in Verdict via Forbes, that because SLMs are trained on deep, domain-specific knowledge, AI agents will be enabled to make decisions with greater autonomy. For example, in finance, an AI agent will not only generate market insights, but will actively trade based on real-time data. In logistics, AI will not only track supply chains but autonomously “optimizes delivery routes and inventory levels.”
- The designers of open source llmware BLING and DRAGON created high-quality business-focused small models (less than 6B parameters and 6–9B parameters). One excellent training objective characteristic they developed is “Better to say ‘I don’t know’ than make something up.” A “Not Found” response is generated for questions that “can not be answered by the grounded source, rather than using background knowledge or often times undefined behavior — reducing a common source of hallucinations.”
Real Life Use cases
- Smaller models, as observed by Gartner, are being used in early stage R&D, “such as molecular property prediction, right through to analysing regulatory requirements.” PA Consulting built an SLM for the Sellafield nuclear processing site to help them keep up with constant regulatory changes, by determining “which changes are relevant and which documents are affected, giving the engineers something to evaluate.”
- Microsoft is using an internal application for cloud supply chain fulfillment, which is an SLM that facilitates natural language interactions within their supply chain processes. Results show that these smaller models “not only outperformed larger ones in accuracy but also offered reduced running times, even when fine-tuned on limited datasets, showcasing the potential of SLMs to deliver efficient and effective solutions in specialized domains.”
Image by rawpixel from Pixabay
September 19, 2025
