This article is part of the Iris Ethics Knowledge Base; you can find a directory of all articles here.
A Delphi study develops considered judgement from a panel, usually of experts or people with specific standing in a field. Panellists answer a first round of questions. The evaluation or research team synthesises those responses, feeds the synthesis back, and asks a further set of questions informed by what the panel said. This continues, typically across two to four rounds, until the panel reaches an agreed level of consensus or until the remaining disagreement is itself the finding.
Two features define it. Responses are anonymous between panellists, so a senior voice does not dominate the room. And each round's instrument is built from the round before it. That second feature is the one that bears consideration for ethics review.
The National Statement already accounts for Delphi and similar methods
Nothing in the National Statement requires every instrument to be finalised before approval. Where the total project cannot be described in advance because the design and detail of successive stages will be informed by preceding stages, evaluators and researchers should provide a description of the stages that are foreseen and how they intend to seek ethics approval for each stage (NS 3.1.8). The National Statement also acknowledges that some designs will be shaped by the experience and insights of participants, and that this can be a valid and powerful approach (NS 3.1.3).
So the question for a committee is not "where is round three?" It is "do we understand how round three will come into existence, and is that process itself ethically sound?"
What we approve, and what we do not need to see
Iris Ethics HREC will approve a Delphi study without sighting the later round instruments, provided two conditions are met.
First, the round one instrument is complete and clearly articulated. This is the round we can actually assess. It is where the framing risks sit, where leading questions do their damage, and where participants form their impression of what they have agreed to.
Second, there is a stated methodology for how later rounds will be developed. This substitutes for seeing the instruments, and it needs to be specific on:
- how responses will be analysed and synthesised;
- what will be fed back to panellists and in what form;
- the threshold that defines consensus;
- the criterion for retiring an item;
- the maximum number of rounds; and
- the stopping rule.
If generative AI tools will be used in synthesis, say so and say how.
An application that says "subsequent rounds will be developed based on round one findings" has not met this test. One that states a defined agreement threshold above which items are treated as settled, a floor below which they are discarded, the form in which the remainder returns to the panel, and a maximum number of rounds, has. The thresholds are a matter for the design and the field. What we assess is whether the rule was set in advance and applied consistently, not whether it matches a particular number.
Where later rounds depart materially from that stated methodology, that is an amendment, and it should come back to us.
Pre-rounds
Some Delphi studies begin with a scoping or pre-round, often qualitative, that generates the item pool for round one. This is a sound design choice and we can accept it. We will approve the pre-round on the same basis as any other data collection, then review the round one design before it is implemented, because round one is the instrument we would otherwise have assessed at the outset. Once it is approved, the logic above applies to what follows. This is a sequential approval pathway, not a series of separate applications, and it is the arrangement NS 3.1.8 contemplates.
Consent across rounds
A recurring question is whether panellists consent once, at the start, or afresh at each round. There is no single answer, and applications that assume one are usually the applications that get asked about it. Our position is that consent should be proportionate to risk. The higher the risk to participants, the stronger the case for renewing consent per round, particularly where later rounds may move into territory that was not foreseeable at recruitment. A Delphi that begins with technical measurement questions and converges on judgements about the conduct of identifiable organisations has changed what it is asking of people, and the consent should keep pace.
Where risk is lower, a single consent at the outset can be entirely appropriate, provided the requirements of NS 2.2 are met on the information given up front: the round structure, the number of rounds anticipated, what participation involves at each round, and the realistic time commitment across the study. The National Statement also recognises that projects involving multiple stages may require consent to be revisited over time (NS 3.1.23) and possibly renegotiarted (NS 2.2.8). The failure mode is not asking too often. It is recruiting people to what sounds like a survey, and them discovering three rounds later that they have committed to hours of work nobody mentioned.
Where the actual risks are
Anonymity for small panels
Applications routinely promise anonymity between panellists. But an expert panel is small by design and often drawn from a field where everyone knows who else works in it. A synthesis reporting that "one panellist with a regulatory background noted" may be functionally identifying that individual. The National Statement requires that participants understand whether third parties will know who has been approached and who has participated (NS 3.1.26), and that where there are significant risks if participatory status becomes known, the consent strategy must mask identity (NS 3.1.27). State plainly whether panel composition will be published, whether panellists will know who else is on the panel, and how feedback will be written to avoid attribution. Promising anonymity the design cannot deliver is worse than not promising it.
Withdrawal is genuinely complicated
A panellist who withdraws after round two has already shaped the round three instrument, and their contribution cannot be extracted from a synthesis the panel has already seen. This does not make withdrawal impossible, but the information given to participants must be accurate about it. Tell panellists what they can withdraw (their individual responses, from a point forward) and what they cannot. Attrition also carries a merit dimension: if half the panel leaves by round three, the consensus reported is a consensus among those who stayed, and it should be reported that way.
Consensus can be manufactured
The feedback mechanism that makes a Delphi work is also a pressure mechanism. Repeatedly showing a panellist they are an outlier, then asking them to reconsider, can produce convergence that reflects social pressure rather than changed judgement. That is both a merit problem (NS 3.1.2) and a burden on participants. Good designs mitigate it by reporting minority rationales rather than distributions alone, by making it legitimate to hold a position across rounds, and by not treating non-convergence as failure.
Two short examples
A policy Delphi on service standards. A department commissions a panel of practitioners, peak body representatives, and people with lived experience to develop minimum standards for a support service. The application submits the round one item pool in full, states that panellists will see anonymised distributions plus de-identified rationales, and sets a maximum number of rounds. The specific risk is that practitioner panellists are drawn from funded providers and may be reluctant to endorse standards their organisation cannot currently meet. The design should address whether their employer will know they participated.
An evaluation indicator-setting panel. An evaluation team runs a pre-round of interviews with program staff to generate candidate outcome indicators, then puts those to a small Delphi panel across two rounds. The pre-round is approved first, and the round one instrument built from it comes back for review before implementation. The specific risk is that the panel is small enough that a distinctive service model mentioned in someone's rationale identifies both the panellist and their site.
Summary
Delphi methods are well established, and their iterative design is not an obstacle to ethical approval. What a reviewer needs is a complete round one instrument and a specific methodology for how later rounds will be generated, including consensus thresholds and stopping rules. Where a pre-round informs the first Delphi round, we can approve that pre-round and review the round one design before implementation. Consent should be proportionate to risk. The risks worth spending time on are not procedural: they are the fragility of anonymity in small panels, the practical limits on withdrawal once rounds have built on each other, and the capacity of the feedback mechanism to produce convergence that is social rather than substantive.
This page was last updated on 7 October 2026.
Was this helpful? We'd appreciate your feedback!
This article makes reference to the National Statement on Ethical Conduct in Human Research 2025. It is intended as practical guidance and does not constitute legal advice.
AI Disclosure: Resources in this library were prepared using Large Language Models with input from Iris Ethics staff who guided the scope and design. Subsequent revisions and final versions were developed and approved by Iris Ethics staff.