Why critical thinking matters

It is likely that scepticism about the benefits of AI is:

Related to higher intelligence

Inversely correlated with intelligence

None of the other statements is likely correct

Likely higher in people who are more religious or more spiritual

Over-reliance on AI was identified as a significant problem. Which of the following statements is most likely to be true?

Under-reliance on AI may also be an issue for some professionals, ie that they reject AI advice when the AI advice is actually correct

Under-reliance on AI is unlikely to be an issue at the moment as human judgement is usually better than that of AI models

Over-reliance on AI is likely to be more of a problem for organisations than under-reliance on AI

Under-reliance on AI (when the human is wrong and overrides correct AI advice) is likely to cost organisations less money than over-reliance on AI

A consulting firm called Data Consulting Indicators announced in a press release just a few months that its consultants are consistently using a proprietary AI for 21 per cent of their work. As an observer, what should concern you most about this announcement?

The fact that they are using a proprietary AI may mean that the AI model has been trained on an insufficiently large data set to be useful

A particular usage metric does not guarantee that the firm is automatically delivering more useful recommendations

The targets are low given that most other consultancies have announced that they are using AI for a larger proportion of their work

Two of the other statements are correct

A risk manager presents data to her colleagues showing that her team has performed strongly over the last 28 months using AI-assisted analysis. She argues that this proves that her oversight procedures are working well. Based on the psychological evidence mentioned in the article, what is the most important counterargument?

Her argument lacks substance as she has not mentioned adoption rates based on volumes of interactions with the AI

Her confidence in the data does not necessarily mean that the team has actually gained any real competence in using AI models

28 months is too short a period to draw any conclusions

The absence of major problems over this period is still compatible with the argument that critical thinking skills may be declining gradually

What is a major conclusion that can be drawn from the article?

Professionals who are receptive to the use of AI should be prevented from using it

Organisations are more likely to experience competitive advantage when they intentionally slow the adoption of AI

AI adoption is most successful when leaders instruct employees on its use on a one-to-one basis

The traits that make professionals open to using AI may also make them vulnerable to its errors

A newly recruited finance director called Raina is trying to create a set of guidelines for the finance department’s successful deployment of AI models. Which of the following best reflects the article’s argument?

The team should be aiming to use AI to a similar extent as is used by its competitors within the same sector and industry

Employees should be encouraged to defer to AI on data-intensive tasks while reserving human judgement for situations that require evaluation of more qualitative factors

Given the accuracy of current AI models, members of the team should be encouraged to trust AI recommendations unless there are strong quantitative reasons to override them

Employees should be encouraged to challenge and further test AI recommendations whenever they have doubts about its accuracy

Research by Dunning and Kruger has led to the so-called Dunning-Kruger effect. Which of the following is the best illustration of how this might arise in a finance setting?

Several finance business partners lack the confidence to challenge the department’s AI models even when their instincts later prove to have been correct

Two of the other answers are correct

A team leader insists on volume targets for AI use even though different individuals within the team have greatly varying degrees of skill in applying it

An analyst confidently interprets AI outputs but makes recommendations to stakeholders that contain systematic errors

A CFO sets a policy stating that all AI-assisted recommendations must be signed off by the finance manager who originated the request. Several of the finance managers argue that this policy is unfair as they cannot always guarantee that the AI will be correct. How should the CFO respond?

Reassure the finance managers that the AI is accurate enough that sign-off is largely a formality

Explain that the finance managers have more than enough critical thinking ability for this not to be an issue.

Explain that the policy exists because it is important for a human to have oversight and ultimate responsibility for all decisions that originate from the finance department

Acknowledge that the policy may need to change so that members of the financial team with less training and experience with the AI should be exempted from the policy

A CFO proposes measuring her team’s AI performance by measuring how often analysts accept AI recommendations versus override them, with a target acceptance rate of 30 per cent. What is the biggest flaw in this approach?

A fixed acceptance rate cannot distinguish between appropriate reliance versus over-reliance

Members of the team may try to game the system by accepting recommendations they would otherwise question

The acceptance rate is likely too low given the accuracy of current AI models

The target acceptance rate does not account for the much higher error rates that are likely for complex, ambiguous tasks versus simple, straightforward tasks

At a widely publicised international conference, the finance director of a public sector organisation states: 'The best defence against AI errors is to hire people who are naturally sceptical of technology.' What is the biggest error in this line of reasoning?

Instinctive scepticism toward AI may cause its own set of problems. Appropriate use of AI requires judgement and evaluation rather than blanket suspicion

Sceptical employees may be more cynical about all forms of change, which may impede their willingness to adapt when presented with other organisational initiatives that do not relate to AI

Technologically sceptical people may have lower skills relating to all forms of technology – not just AI

The finance director’s announcement simply encourages applicants thinking of working for this public sector organisation to lie or exaggerate their scepticism about the use of AI