Artificial intelligence (AI) promises to make accountants faster, but in tax and finance the stakes are accuracy. Experts warn that generic AI chatbots, often produce answers that are outdated, misleading or incorrect without careful prompting and oversight. In other words, speed is pointless if the results aren’t reliable.

The pitfalls of generic AI

Using off-the-shelf AI can introduce serious risks. For example:

• Outdated or incorrect data. Many AI chatbots are trained on broad internet sources and can cite stale or incomplete information. In a BBC report, 19% of AI answers introduced factual errors.
• Fabricated (hallucinated) answers. AI can confidently invent plausible but false facts. In a UK High Court case (Al-Haroun v Qatar National Bank [2025] EWHC 1383), a lawyer submitted 18 non-existent case citations generated by AI. In a recent tax case, HMRC v Gunnarsson [2025] UKUT 247 (TCC) an unrepresented taxpayer used a chatbot to justify his argument, citing three fake cases and was rapped by the tribunal judge for wasting HMRC’s time researching non-existent cases. These examples highlight a broader, global pattern of AI hallucinations in professional contexts.
• Lack of nuance. AI can handle straightforward queries reasonably well, but it often struggles with context and complexity. In areas such as tax, audit, or accounting, where rules are highly technical and depend on precise circumstances, AI may fail to provide accurate or reliable analysis, leading to oversimplified or misleading conclusions. In practice, this means an AI’s answer might violate subtle rules or misapply a regulation.
• Unverified output. Generic AI models typically do not cite their sources, meaning their conclusions cannot be independently traced or validated. Without verifiable references, users must treat outputs as unsubstantiated claims rather than authoritative information, which poses risks in fields that demand accuracy, such as law, tax, auditing, and accounting.

In domains built on accuracy, trust, and compliance such as tax and finances, these problems matter. It is important to have a policy on AI within your business and monitor the use of AI for business purposes regularly.

Don’t know where to start with your policy? Download this free AI policy template from Croner. ACCA recommends that you obtain advice from your professional indemnity insurance providers on any risks related to the use of AI within your professional work before implementing any policies.

Trust through verifiable sources

The real game changer in AI is not simply speed, but the use of models built on verified, expert-curated data rather than the fastest internet scrape.

Croner’s balanced and transparent approach helps professionals make more informed decisions while reducing the risks associated with unverified or overconfident AI-generated outputs.

Explore Croner’s AI solutions and take advantage of Croner’s exclusive ACCA member rate

This article was supplied by Croner-i. Find out more at Croner's website or call 0800 231 5199.