Yellow diamond-shaped 'Dead End' street sign against a blue sky on a suburban road, symbolizing a path that runs out

The Proving Problem Why a Self-built AI Assistant Runs Out of Road in Social Performance Work

Somewhere in most Australian mining, energy and utility businesses right now, someone on the community or land team has built themselves an AI assistant. They pointed a general-purpose model at a folder of consultation notes, agreements and minutes, wrote a page of instructions, and started asking it questions.

Whoever built it understood their own problem well enough to act without waiting for procurement. The question is not whether AI belongs in this work, because it does. The narrower question is what happens when the work moves from answering questions to proving something.

What the DIY approach gets right

A general AI assistant pointed at your documents is very good at drafting. Consultation summaries, meeting notes, first-pass correspondence, report narrative for a sustainability disclosure: it will produce a usable draft faster than a person will. It will find the clause you need in a 200-page agreement, rewrite community materials in plain language, and pull readable themes out of a batch of feedback.

All of that is real value, and none of it is the part that gets audited. These are drafting tasks sitting on top of a record, and the question is what the record is.

An assistant pointed at a folder is a reader. It answers questions about what it can see, when someone thinks to ask. A social performance function does not have a reading problem. It has a proving problem: proving a commitment made at a hearing three years ago was met, proving a grievance was closed inside the window your own framework promised, proving the right landholders were contacted before works started.

That is a record problem, and a reader cannot solve it. Four things break.

1. Nothing happens until somebody asks

An assistant produces answers. It does not hold the register. Commitments, grievances, land agreements and consent records are governed artefacts, each with an owner, a state and usually a deadline, and each has to exist whether or not anyone asks.

Australian frameworks assume exactly that. Signatories to the Clean Energy Council’s Best Practice Charter report publicly against their engagement commitments every two years, and The Energy Charter has energy businesses disclose annually, now voluntary from 2026, under a decentralised accountability process that replaced the earlier Independent Accountability Panel. A commitment you report against publicly, on a fixed cycle, is a register with a clock on it.

In a chat-based setup, when nobody asks, nothing happens. The commitment nobody thought to prompt about this week is the one that lapses, so risk surfaces on a schedule set by whoever remembers to type.

Worth asking your team: how does an obligation get actioned if nobody thinks to ask about it that week?

2. Chat transcripts alone are not evidence

Approval conditions, public inquiries, native title processes, lender reviews and industry assurance all want the same three things: attribution, immutable timestamps, and version history. Who recorded what, when, and what the record said before somebody edited it.

This is not a soft expectation. IFC Performance Standard 1, for example, requires an environmental and social management system that includes stakeholder engagement and an effective grievance mechanism, with documented engagement and appropriate external communication to affected communities. ICMM’s Assurance and Validation Procedure requires members to conduct annual independent assurance of sustainability performance and periodic third-party validation of Performance Expectations, with public disclosure, effectively making commitments and obligations registers part of the assurable system. And the Dyer review of renewables engagement led to a voluntary national developer rating scheme now run independently by Equifax, so engagement conduct is something a third party assesses rather than something the developer describes.

An answer generated on the day is not a record of what was known at the time. It is a fresh interpretation of whatever documents were in the folder that morning. So when a condition gets tested, the team spends two weeks reconstructing a consultation history from documents, email and chat logs, and that is where the gaps get found.

Worth asking your team: if a condition of approval were tested next quarter, what is the artefact we would hand over, and who would attest to it?

3. The counting questions

Stakeholders, grievances, commitments, parcels and agreements are related records, not documents, and the value sits in the relationships between them. This person raised this issue, which became this grievance, linked to this commitment, on this parcel, under this agreement.

Under Queensland’s land access framework, a resource authority holder must give each landholder written notice at least 10 business days before entering for preliminary activities, and generally needs a conduct and compensation agreement before advanced activities. Which landholders were noticed, on what date, and which parcels have an executed agreement are questions with dates and counts as answers.

A reader summarises the documents in front of it. It will not hold that chain together across eight years of interactions, will not recognise the same landholder under four different spellings, and cannot reliably answer a question whose answer is a number. A plausible wrong number is more dangerous than none, because it goes into a board pack.

Worth asking your team: when we ask how many commitments are overdue, how do we check the number is right, and who does that today?

4. No state, no clock

Grievance response times, escalation paths, assigned owners, reminders and sign-off. IPIECA’s manual on community grievance mechanisms, written by the oil and gas industry for itself, sets the expectation plainly: a clear procedure with an indicative timeframe for each stage, accessible to all affected stakeholders, and closed out consistently.

An assistant has no state and no clock. It cannot escalate a grievance past its response window, chase the community liaison officer who owes a close-out, or stop a commitment being marked complete by somebody with no authority to say so. So the team runs the real workflow in a spreadsheet alongside the AI, which is the situation they were trying to leave.

Worth asking your team: who gets told when a grievance passes its response window, and how do they find out?

The objection that deserves a straight answer

“AI is improving every month, this will be solved soon.” Agreed, and not grudgingly. The gap is not model capability. It is that there is no governed record for the model to be right about, and a better model applied to an incomplete register produces wrong answers faster and more convincingly.

That is now a live governance question. Writing in InformationWeek, Pam Baker reported that regulators are moving toward expecting organisations to preserve the retrieval trail behind an AI answer: the source documents, their versions, what was retrieved, when, and who reviewed it. Audit readiness, as one contributor put it, means showing what informed it and why, so regulators get evidence, not interpretation.

Where this lands

This is not an argument for less AI. Borealis has Integrated AI in it, and the direction of travel is obvious. The distinction that matters is between AI grounded in a governed record and AI inferring from a folder of documents. The first tells you what is overdue and what the file said in March 2023. The second tells you what the documents appear to say today, which holds up right until somebody official asks how you know.

So the question for your team is not whether the AI assistant is helpful. It clearly is. It is this: when the inquiry, the lender or the auditor asks how you know a commitment was met, what do you hand them?

This is the problem Borealis, now part of Irth has spent 20 years solving, working with hundreds of users across mining, gas, energy and transportation to build exactly this kind of governed record.

If your team is getting asked these proving questions right now, talk to us about what that could look like for your operation.

Get started with
Borealis stakeholder engagement software today!