Studies performed within the last year have analyzed the correlation between AI tool usage and the impact on critical thinking. One study focuses on the effect of the use of GenAI tools on critical thinking in knowledge work. Researchers for Microsoft and Carnegie Mellon analyzed close to a thousand real-world GenAI tool use examples to find out how knowledge workers engage in critical thinking. Knowledge workers, they found, engage in critical thinking primarily in three ways when using AI tools (mostly directly quoted):
- For recall and comprehension, the focus shifts from information gathering to information verification.
- For application, the emphasis shifts from problem-solving to AI response integration
- For analysis, synthesis, and evaluation, effort shifts from task execution to task stewardship (oversight) This refers to translating intentions into queries, steering AI responses, and assessing if the AI response meets their quality standards for work, while retaining accountability for the work.
In addition, utilizing AI tools raises concerns about long-term reliance on AI and the decline of independent problem-solving.
In another study by SBS Swiss Business School’s Michael Gerlich, it was found there was a significant negative correlation between frequent AI tool usage and critical thinking abilities when offloading certain cognitive tasks. Those more dependent on AI tools were younger and scored lower in critical thinking compared to older participants. In addition, regardless of AI usage, the more highly educated exhibited better critical thinking skills. AI tools offer increased efficiency but there is risk of deep, reflective thinking being diminished.
What do these study results mean for business researchers who provide information that is used for critical decision making in the organization? The ability to critically think is a core skill of successful business researchers. This encompasses the knowledge and discipline to seek out the most qualified sources that meet the research need at the time. To further clarify, we can look to the quality data and critical thinking cycle, which is comprised of four continuous truths:
- Optimal decision making is achieved through critical thinking
- Critical thinking is enabled by the use of quality data/information
- Quality data is as good as the quality of the resources referenced
- The more quality resources consulted, the greater the ability to connect data threads to enable optimal decision making
If relying on AI tools for recall and comprehension (as stated above) shifts the focus from information gathering to information verification, truths 2, 3, and 4 (especially 4) in the cycle are greatly diminished. The decreased focus in gathering information will lead to fewer quality resources being consulted. Verification of information is crucial, but it is likely too much time will be spent verifying sources that are not optimal in meeting the immediate research need at the time. The pool of sources from which the researcher is verifying is limited, with key sources likely being missed or overlooked. Attention that is being diverted away from the crucial gathering stage, already puts the other foundational truths at rick. Here we are again looking at the issues surrounding AI and using open sources via the internet. In my post on Gemini Deep Research, I discuss the type of sources that GenAI tools do not pick up when using public access data, such as those behind paywalls (which tend to be more authoritative) and are below the “surface level,” requiring additional clicks.
The studies mentioned above, and ones similar help identify very real concerns about using AI tools for cognitive tasks. However, the hard part is just beginning, which is mastering the delicate balance between using AI tools for efficiency while at the same time ensuring critical thinking skills are maintained. And that is another post.
Photo by Jonas Leupe on Unsplash
