Singapore – New research has examined how large language models (LLMs) can identify when available information is insufficient and select reasoning approaches based on language and task requirements.
The research, conducted by Appier‘s AI Research team, focuses on capabilities that could affect the reliability of enterprise Agentic AI, particularly in situations where systems need to make decisions using retrieved information or operate across different linguistic and cultural environments.
One study examined whether LLMs can recognise when none of the available answers is valid.
Appier’s AI Research team tested 28 LLMs of different sizes using “None of the Above” (NA) options to represent situations where no valid answer was available. According to the company, model accuracy declined by between 30% and 50% when “none of the above” was the correct response.
The findings indicate that models can select an incorrect or suboptimal answer even when the information available to them does not support any of the choices. Appier said this ability to recognise information gaps is particularly relevant to tasks involving business ethics and other areas where several plausible options may require broader assessment.
The research team used Supervised Fine-Tuning (SFT) and Direct Preference Optimisation (DPO) to train models to identify scenarios where none of the available answers was correct. DPO, which trains models using both preferred and non-preferred responses, increased accuracy in identifying questions without a correct answer by nearly 30 percentage points, according to Appier.
The researchers also noted that the “none of the above” approach is more suitable for questions where answers are clearly defined and options are mutually independent. For enterprise applications, Appier said systems could combine improved retrieval with checkpoints to determine whether sufficient information is available before taking action, with additional searches or human escalation used where necessary.
The second paper examined how the language used during reasoning can influence the performance of large reasoning models (LRMs).
Appier’s research found that models frequently default to high-resource languages such as English when reasoning, even when the original prompt is provided in another language. For some models, the reasoning language differed from the response language in more than 90% of cases.
The researchers used a “text prefilling” technique to encourage models to reason in a specified language and then assessed the effect on different types of tasks.
The study found that high-resource languages such as English generally produced stronger results for mathematics and knowledge-based tasks. For tasks involving cultural understanding, however, reasoning in the local language was found to capture local context more effectively.
The research also found differences in safety performance. According to Appier, local-language reasoning was more effective at identifying harmful or illegal queries during safety testing.
The findings suggest that the most effective reasoning language may vary according to the task. Appier said this could support the development of “reasoning-language routing”, in which an Agentic AI system dynamically selects a reasoning language based on factors such as task type, market and cultural context while continuing to communicate with users in their preferred language.
Implications for enterprise AI
Appier said the two studies point to the need for AI evaluation to extend beyond whether a model produces a correct answer.
“These two papers redefine the standard for evaluating AI. As AI moves from answering questions to making autonomous decisions, measuring whether a model produces the correct answer is no longer enough. We must also assess whether it can recognise insufficient information, adjust its actions accordingly, and select the reasoning approach best suited to each task,” said Chih Han Yu, CEO and Co-founder of Appier.
“These capabilities will help Agentic AI evolve from simply executing instructions into a reliable decision-making system capable of navigating real-world complexity. Through sustained foundational research, Appier aims to turn these critical questions into measurable and improvable AI capabilities, enabling enterprises across markets and languages to adopt Agentic AI with greater confidence,” he continued.
The company said its AI Research team will continue research into LLMs and Agentic AI, including potential applications across its Ad Cloud, Personalisation Cloud and Data Cloud product lines.
The research comes as enterprises increasingly explore Agentic AI for tasks involving autonomous decision-making, information retrieval and operations across multiple markets and languages. Appier’s studies highlight two areas that could influence the deployment of such systems: their ability to recognise when available information is insufficient and their ability to adapt reasoning to the requirements of a particular task.
