Searching in the Dark

How three government agencies are ignoring twenty years of e-discovery technology — and using their own backwardness against OIA requesters

Preface

In an article published in the New Zealand Herald in February 2026 I considered the operation of the Official Information Act. I observed that avoidance strategies persist: repeated requests for “clarification” that reset statutory clocks, fragmented record-keeping, and opaque decision-making. The cumulative effect is delay, frustration, and diminished public confidence. I also noted that most records are now electronic, searchable, and capable of rapid review using modern tools, including AI-assisted search and e-discovery techniques. Claims that requests are unduly burdensome because they involve hundreds of documents ring hollow when compared to contemporary information-management capabilities. Indeed, the digital paradigm should have made compliance easier, not harder.

I decided to follow up on this. I requested official information from the Department of Prime Minister and Cabinet, the Department of Internal Affairs and the Ministry of Justice about the way that they process OIA requests and the extent to which technology is used to assist in processing those requests. This article deals with the outcome of that request.

Ironically the most evasive reply came from the Ministry of Justice. They refused to provide the information sought because it was in the public domain. I was referred to a previous (similar) request and a link was provided. When I accessed the link I was provided with a similar refusal. The information was in the public domain. However, that communication provided four links to pdf documents three of which worked and one of which was dead. A Google search enabled me to find the missing document.

This was an example of the Official Obfuscation Act in action. The MOJ could have denied my request advising the information was in the public domain and including the links that would lead me to it. But no. In a stunning example of abstruse bureaucratic obscurantism they set me off on a paper chase.

The OIA process is complex and beset with delays but it could be a lot simpler. This article explains why.

Introduction

Ask three government agencies what technology they use to find documents responsive to an Official Information Act request, and the collective answer — on the evidence of recent OIA responses from the Ministry of Justice, the Department of Internal Affairs and the Department of the Prime Minister and Cabinet — is this: a search bar, some spreadsheets, and a lot of people reading.

That answer would have been unremarkable in 2005. In 2026 it is an admission of institutional failure. The legal profession confronted exactly this problem — isolating relevant documents from large volumes of electronic material — and solved it more than a decade ago. The tools are mature, court-approved and commercially available. Litigators use them daily. The agencies that administer the public’s statutory right to information have simply not bothered.

What the responses actually reveal

DPMC’s response is the most detailed about method, and therefore the most damning. OIA requests are managed in MOTO, a workflow system, linked to iManage, a document repository.

To find information within scope, staff search iManage by title, document number, author or keyword, with filters for date range and document type. That is the entire toolkit. Strip away the branding and it is Ctrl-F with a due-date column. MOTO itself, for all its QA rounds and sign-out chains, is administrative choreography: it tracks the request while contributing precisely nothing to the one task that matters — determining which documents are actually responsive.

The Ministry of Justice material is worse, because it is more honest about the drudgery. Its process guide instructs advisors to run reports from the JAX database, download them, collate them “in your own spreadsheet,” and email that spreadsheet around the building asking subject matter experts what falls within scope.

Staff are taught to rename files with #1, #2, #3 prefixes so they sort properly, to drag emails into folders one at a time, to combine PDFs in Adobe Pro, and to mark up redactions by hand. There is a section on what to do when Adobe chokes on long file names.

And the actual identification of relevant information — the step technology could transform — rests entirely on human recall: check the inboxes, check the physical files, check the weekly reports, ask the subject expert whether anything has been missed. This is not a search methodology. It is a scavenger hunt with a statutory deadline.

DIA at least engages with the question, and its candour is useful. The Department concedes it has no guidance or policy at all on the use of digital tools to isolate information for OIA responses. Its approved AI tools are Microsoft Copilot and NotebookLM — general-purpose office assistants, not review platforms — and most staff in the very teams that service official correspondence lack the premium licence needed to search the Department’s own content management system.

DIA does not track whether Copilot is used for OIA work at all. Read that again: the department responsible for the machinery of government cannot say whether its one relevant tool is ever used.

The technology the agencies are pretending not to know about

None of this is how document-heavy legal work has been done for years. Electronic discovery in civil litigation faced precisely this problem, at far greater scale, and produced technology assisted review: predictive coding, in which a knowledgeable reviewer codes a sample of documents and statistical models rank the remaining corpus by likelihood of relevance; continuous active learning, which sharpens the model as review proceeds; email threading; near-duplicate detection; concept clustering that groups documents by subject regardless of the keywords they happen to contain.

These are not exotic tools. Judicial approval of predictive coding dates to 2012 in the United States and 2016 in England, and New Zealand’s own High Court Rules contemplate their use where proportionate.

The research findings are settled: trained models are not merely faster than exhaustive manual review but more consistent and frequently more accurate, because human reviewers tire, drift and disagree.

Every technique in the e-discovery toolkit maps directly onto a task these agencies currently perform by hand — or perform incompletely, because the manual burden makes thoroughness impractical.

Lawyers in private practice were forced to adopt these tools by the economics of litigation. Government agencies, facing no such discipline, have declined to adopt them — at the public’s expense.

The s 18(f) racket

This is where the technology gap stops being an efficiency question and becomes a question of integrity.

Section 18(f) of the OIA lets an agency refuse a request where the information cannot be made available without “substantial collation or research.” DIA invoked the spectre of that ground in this very response, warning that a request for practitioner-level guidance “would be likely to be refused under section 18(f)… due to the substantial manual collation and research that it would necessitate.” The Ministry of Justice’s process guide coaches staff through the same reasoning: how many documents, how many hours, how much reading.

Note the word doing the work: manual. The burden that justifies refusal is substantial only because the agencies have chosen labour-intensive methods.

An agency that could train a review model in an afternoon and rank a hundred thousand documents by relevance overnight would find very few requests genuinely oppressive.

When the collation burden is an artefact of the agency’s own technological choices, invoking that burden to refuse requests — or to buy more time, as the Ministry of Justice did with this very request under s 15A(1)(a) because it “requires a search through a large quantity of information” — is not a statutory safeguard. It is a racket: refuse to modernise, then charge the cost of your obsolescence to the requester in delay and denial.

The Act’s timeframes assume agencies will make reasonable efforts to find information. A twenty-working-day limit stretched by self-inflicted inefficiency is not compliance with the Act; it is the slow strangulation of it.

DIA adds an irony of its own. Its AI policy prohibits enterprise AI from materially contributing to decisions that could adversely affect an individual — expressly including “determining a response to a request.”

As a principle for the release decision, which involves judgment, withholding grounds and the public interest test, that is sound. But the policy’s shadow has evidently fallen over the entirely different task of finding the documents in the first place.

Locating potentially relevant material is not a decision adverse to anyone; it is the mechanical precondition to one. Litigation practice has kept this distinction clear for fifteen years: the software finds, the lawyer decides. A policy that cannot tell the two apart is not caution. It is an alibi.

The excuses do not survive contact

The predictable defences write themselves, so they may as well be answered now.

Privacy? Purpose-built review platforms are not consumer chatbots: they run in controlled or sovereign environments, keep data in place, and log every coding decision.

Accountability? Technology assisted review produces what manual review never can — measurable recall and precision, a documented and auditable search protocol. An Ombudsman reviewing a refusal would be far better served by statistical validation of a search than by an assurance that somebody remembered to check the right inboxes.

Cost? The Ministry of Justice pays trained advisors to rename PDFs one at a time. The status quo is not free; its cost is merely hidden in salaries and in the public’s forgone right to know.

The agencies’ processes are conscientious in their own terms — DPMC’s workflow discipline, the Ministry’s scoping plans, DIA’s governance framework all reflect people trying to do the job properly with the tools they have been given. That is exactly the indictment.

The failure is not at the desk of the advisor dragging emails into folders. It sits with the officials who decided, year after year, that the twenty-first century was somebody else’s problem.

The Ministry of Justice’s own OIA policy declares that, as administrator of the Act, “the Ministry should be an exemplar.”

An exemplar does not respond to a request about search technology by extending the deadline because searching is hard. Government departments hold information volumes rivalling major commercial litigation, under deadlines shorter than most discovery timetables, on behalf of a public whose right to that information is constitutional in character.

The private sector answered volume with intelligence because it had no choice. Government has answered it with spreadsheets, extensions and refusals — because, so far, nobody has made it choose otherwise.

The law of official information has not changed. The technology for honouring it has. The only thing keeping pace with neither is the willingness of these agencies to notice.