ATTORNEY-GENERAL v PROBLEM GAMBLING FOUNDATION OF NEW ZEALAND [2016] NZCA 609

ATTORNEY-GENERAL v PROBLEM GAMBLING FOUNDATION OF NEW ZEALAND [2016] NZCA 609 [16 December 2016]IN THE COURT OF APPEAL OF NEW ZEALANDCA467/2015[2016] NZCA 609BETWEEN THE ATTORNEY-GENERALAppellantAND PROBLEM GAMBLING FOUNDATIONOF NEW ZEALANDRespondentHearing: 2 and 3 August 2016Court: Cooper, Winkelmann and Venning JJCounsel: Solicitor-General U R Jagose QC and J K Gorman for AppellantM Chen and S L Mead for RespondentJudgment: 16 December 2016 at 10 amJUDGMENT OF THE COURTA The appeal is allowed.B The judgment entered in the High Court in favour of the respondent is set aside.C Judgment is entered in favour of the appellant.D The costs orders in favour of the respondent made in the High Court are set aside.E Costs in the High Court are to be fixed by that Court in accordance with the outcome of this judgment.F The respondent must pay the appellant costs for a standard appeal on a band A basis and usual disbursements.____________________________________________________________________REASONS OF THE COURT(Given by Winkelmann J)Table of ContentsPara NoBackground [3]The request for proposals [9]The process [13]High Court proceedings [23]First ground of appeal: did the Judge apply wrong scope of review? [27]Relevant principles [30]Legal test applied in the High Court [39]Discussion [41]Second ground of appeal: did the Judge err in applying judicial testfor apparent bias and in finding unconscious bias? [55]Relevant facts [57]High Court judgment [62]Discussion [72]Third ground of appeal: was the Judge was wrong to find lack ofprobative evidence? [86]The Judge's approach [88]Discussion [93]Fourth ground of appeal: was the Judge wrong to find breach ofMandatory Rules and breach of legitimate expectation? [100]High Court judgment: pre-scoring and consensus scoring [106]Discussion [112]Breach of legitimate expectation/Mandatory Rules in pre-scoringand consensus scoring [113]High Court judgment: Moderation phase [128]Discussion [137]Result [143][1] The Ministry of Health is charged with implementing an integrated problem gambling strategy. In July 2013 it requested proposals for services from private providers to assist with the delivery of that strategy and, in March 2014, it made and announced decisions as to the providers with which the Ministry would contract for services for the period from 1 January 2014 until 30 June 2016. Disappointed with those decisions, the Problem Gambling Foundation of New Zealand (the Foundation) challenged their lawfulness by way of judicial review. A three-day hearing ensued in the High Court in September 2014, with judgment released some 10 months later on23 July 2015.1 Woodhouse J granted the application for judicial review on three grounds: (1) breach of legitimate expectation and mandatory rules; (2) mistake of fact/lack of probative evidence; and (3) inadequately managed conflicts of interest. He set the decisions aside.[2] The Attorney-General brings this appeal on behalf of the Ministry as the litigation relates to actions of Ministry officials or delegates. For ease of comprehension, we therefore refer to the appellant as the Ministry rather than as the Attorney-General. The Ministry argues first that the Judge applied the wrong level of review to what were contracting decisions by the Ministry. Secondly, that the Judge was wrong to find the errors he did regardless of the appropriate level of review to be applied.Background[3] The Foundation is a charitable trust — a non-profit organisation which provides both public health and clinical problem gambling services. It has provided problem gambling services since 1988.[4] The Ministry is the government department charged with responsibility under the Gambling Act 2003 (the Act) for developing, managing and implementing an"integrated problem gambling strategy".2 The strategy the Ministry develops and implements has to address certain matters specified in s 317(2), which include:(a) measures to promote public health by preventing and minimising the harm from gambling (public health services); and(b) services to treat and assist problem gamblers and their family and whānau (clinical services).[5] To comply with its statutory obligations, the Ministry routinely adopts a six-year strategic plan and two three-year service plans. Each six-year strategic plan1 Problem Gambling Foundation of New Zealand v Attorney-General [2015] NZHC 1701 [HC judgment].2 Gambling Act 2003, ss 317–318.describes the strategic context and each three-year service plan details the strategy for the upcoming three-year period. Cabinet approves each service plan. The Ministry consults extensively when developing strategic and service plans and involves all key stakeholders in those consultations, including the Foundation.[6] It is also relevant to mention that the public health and clinical services provided by the Ministry pursuant to these plans are funded by an appropriation.3[7] The 2013–2016 service plan recorded that all government agencies and the non-government agencies they fund are expected to strive to enhance their efficiency and effectiveness, and that the Ministry expected this goal to be a "key driver"throughout the 2013–2016 period. The plan also recorded the Ministry's intention totest the market for the primary prevention component of public health services and for the "psychosocial intervention support" component of intervention services in order to establish the potential to enhance efficiency and effectiveness. This service plan was approved by Cabinet on 15 April 2013.[8] Although the Act provides a detailed framework for the development of the service plan, it imposes no obligation on the Ministry as to how it is to implement its service plan once approved. The Ministry could provide the services itself or contract one or more private providers to deliver the services contemplated by the plans. The Ministry has chosen the latter course.The request for proposals[9] The request for proposals (RFP) for the above services was developed to undertake the foreshadowed testing of the market. It was the first nationwide contestable procurement process in the 10 years the Ministry had been responsible for problem gambling.[10] The RFP documentation distributed to providers and potential providers in July 2013 — which lies at the heart of this proceeding — was clear in its terms that, although the RFP might result in negotiations with a view to entering a contract for3 Although s 319(1) of the Gambling Act provides for a problem gambling levy to be applied to gambling operations, that is paid into the Crown account and is not paid to the Ministry.services, it was "of itself not an offer that Potential providers accept bysubmitting Proposals". It did not create contractual relations between the Ministry and any entity which submitted a proposal. The Ministry reserved the right to accept or reject all or any proposals and to negotiate and/or complete a formal contract with any party, whether or not that party had submitted a proposal. It is therefore common ground that the RFP procedure did not give rise to process contracts of the type sometimes found in tendering situations.[11] It is also not at issue that, as well as the terms and conditions contained within the RFP documentation, the RFP process was subject to the Mandatory Rules for Procurement by Departments. These were rules endorsed by Cabinet in 2006 which, according to the introduction to the rules, set out "mandatory standards andprocedural requirements for the conduct of procurement by governmentdepartments" that "reflect and reinforce New Zealand's established policy ofopenness and transparency in government procurement".4[12] The RFP set out criteria and weightings for the evaluation of the proposals against quality and commercial considerations. Quality criteria were said to be weighted as to 70 per cent and price as to 30 per cent. The RFP allowed proposals to be submitted for the provision of regional or national services, and for providers to join together in the presentation of a proposal.The process[13] The Ministry received 32 proposals in response to the RFP, representing 86 service combinations. The Foundation submitted two proposals: one to provide services in the majority of the regions as a sole provider and the other as"lead agency" in a partnership with Hapai te Hauora Ltd.[14] The Ministry selected a panel of people from within and outside the Ministry to evaluate proposals received against the published criteria and to make4 Although the Mandatory Rules were operative at the time, they have been replaced by the Government Rules of Sourcing, now in their third edition: New Zealand Government Procurement and Ministry of Business, Innovation & Employment Government Rules of Sourcing: Rules for planning your procurement, approaching the market and contracting(3rd ed, 2015).recommendations to the Ministry's ultimate decision-maker, Mr Bartling, who was not part of the selection panel. Mr Bartling was at the time the group manager of Mental Health Service Improvement.[15] The evaluation process used was as follows. Individual panel members scored each proposal according to the RFP criteria between one and 10 using standard scoring/evaluation sheets. All panel members returned electronic evaluation sheets. These sheets were referred to as pre-scoring sheets and the process was called the pre-scoring phase. Pre-scores were then submitted to and compiled by the non-scoring chair of the panel, Mr Levy, who was a senior contract manager at the Ministry at the time. After Mr Levy completed his compilation the panel members met and, in a discussion facilitated by Mr Levy, discussed their pre-scores and the reasons for differences in individual pre-scores before agreeing on a single consensus score for each proposal on each criterion.[16] The panel then undertook a moderation process where, as a group, they reflected upon whether the results made sense. This was a re-ranking process, in which the scores were not changed but rankings were shifted. The moderation phase resulted in a significant re-ordering from that achieved through the pre-scoring and consensus-scoring phases. The re-ordered scores formed the basis for the panel'srecommendation.[17] Following this process, the recommendations were provided to Mr Bartling.5If accepted, the effect of the recommendations would be that, apart from a verylimited secondary role in Gisborne and Canterbury, the Foundation's proposals were rejected. This was against the background that, during the preceding 10-year period, the Foundation had been the largest provider of problem gambling services. And the Foundation was not the only existing provider for which the recommendations had significant negative implications.[18] In evidence provided for the judicial review, Mr Bartling said he was aware that accepting the recommendations would bring about significant change within the5 Mr Bartling, the decision-maker, had the consensus scores, the moderated scores, and the information prepared as to what was taken into account when re-ranking during the moderation stage.sector. He therefore sought and obtained reassurance from Mr Levy that the recommendations reflected the true consensus view of the panel and that the management of conflicts of interest by the panel was robust and fair. Having received that reassurance, Mr Bartling nevertheless sought an additional reassurance that the process met good practice standards for government procurement processes.[19] The Ministry instructed PricewaterhouseCoopers (PwC) to provide an independent probity report in connection with the RFP process. Although PwC identified aspects of the process requiring fuller documentation, once that documentation was attended to, PwC issued a report confirming there were no unresolved concerns regarding the RFP process.[20] Following the receipt of that advice, Mr Bartling sought advice from the team leader for gambling harm minimisation, Mr Thompson, as to how to transition to the proposed new service mix with the least possible disruption and also sought reassurance that the decisions the panel had recommended "made sense". UltimatelyMr Thompson advised him that the panel's recommendations, with a few minor adjustments, ought to be accepted.[21] The decisions Mr Bartling then made in March 2014 reflected most but not all of the panel's recommendations. In the case of the Foundation, contrary to thepanel's recommendations, Mr Bartling decided to contract with the Foundation for Asian problem gambling services. The effect of the contracting decisions was nevertheless to radically reduce the scope and value of the services provided by the Foundation.[22] Because of the Foundation's legal challenge to these decisions, the contracts have never been concluded. Instead the contracts of the incumbent providers have been extended to 1 July 2017.High Court proceedings[23] The Foundation's amended statement of claim contained causes of action alleging first, breach of natural justice/procedural expectation and, second, mistakeof fact/decision not supported by probative evidence. These claims were reformulated by Woodhouse J in his judgment.6[24] As to the first cause of action, the Foundation alleged the Ministry had failed to follow the evaluation processes and criteria set out in, or indicated by, the RFP and to comply with the process requirements imposed by the Mandatory Rules. The Foundation argued that, if the Ministry was to depart from the process described in the RFP, the Mandatory Rules required that it notify those who had submitted proposals to provide an opportunity for them to amend their proposals. The Ministry did not do this. These deficiencies, it was said, gave rise to claims founded on breach of natural justice, breach of procedural expectations, and breach of the Mandatory Rules. The Foundation alleged numerous deficiencies in process, some of which were rejected by the High Court. We do not therefore describe them. Of relevance to this appeal, however, is that the Foundation argued that:(a) when evaluating the proposals the Ministry used criteria, and weightings for those criteria, different to those recorded or indicated by the RFP;(b) the moderation process was not part of the process set out in, or indicated by, the RFP and was in fact inconsistent with it; and(c) the Ministry failed to address conflicts of interests as required by the Mandatory Rules.[25] As to the second cause of action (mistake of fact/lack of probative evidence), the Foundation argued that the panel's evaluation methodology was so flawed that it produced unreliable information, which in turn tainted the decision made in reliance on that information. The challenged decisions, it alleged, were made either on no or inadequate probative evidence or, alternatively, on the basis of factual mistakes.[26] In the High Court, Woodhouse J found in favour of the Foundation on each of these grounds. In doing so, he rejected an argument advanced on behalf of the6 HC judgment, above n 1, at [9].Ministry that a narrow scope of judicial review was appropriate in this case because the proceeding involved decisions made by the Crown in a contracting context.7 The Crown argued that both of the pleaded causes of action fell outside that narrow scope of review.First ground of appeal: did the Judge apply wrong scope of review?[27] It is common ground between the parties that the first issue to address on this appeal is the appropriate scope of review to be applied in this proceeding and, in particular, whether the standard of review formulated by the Privy Council inMercury Energy Ltd v Electricity Corp of New Zealand Ltd (Mercury Energy) and applied by the Court of Appeal in Lab Tests Auckland Ltd v Auckland District Health Board (Lab Tests) was appropriate in these circumstances.8 If the Crown is correct as to the scope of review, that is determinative of this appeal — none of the pleaded allegations would fall within that narrow scope.[28] The Solicitor-General accepts that judicial review of the Crown's contracting decisions is available but says that, in a commercial context, review will usually only succeed where there is evidence of fraud, corruption, bad faith, or any analogous situation such as where the integrity of the process has been undermined by a conflict of interest where insider information is used to disadvantage rivals in a tender. On the Ministry's case, that is the general rule as to the available scope of review of commercial contracting decisions. But the Ministry accepts there is an exception to this general rule — broader judicial review will be available in respect of decisions where public law obligations, which require the Court's supervision, arise from the particular context. It says the starting point is that the scope of reviewof the Crown's commercial contracting decisions is narrow, absent such contextual indications. The Ministry says the Judge erred by abandoning this starting point, instead undertaking an expansive assessment of all of the circumstances and giving no particular weight to the contractual nature of the decision before settling upon a broad scope of review.7 HC judgment, above n 1, at [110].8 Mercury Energy Ltd v Electricity Corp of New Zealand Ltd [1994] 2 NZLR 385 (PC) andLab Tests Auckland Ltd v Auckland District Health Board [2008] NZCA 385, [2009] 1 NZLR 776.[29] For its part, the Foundation supports the Judge's analysis. Ms Chen forthe Foundation points to the recent Supreme Court decision in Ririnui v Landcorp Farming Ltd (Ririnui) as authority for the proposition that not all contracting decisions by the Crown or state-owned enterprises are subject to only narrow review— only ordinary commercial transactions are.9 It says that outside of ordinary commercial transactions the scope of review is determined by the context within which the decision-maker operates, whether the decision-maker was acting in accordance with its statutory obligations and whether there is a public interest component. Here the Crown was contracting for the provision of public healthservices. This and other contextual indicators support the Judge's assessment thatthis was not an ordinary commercial transaction.Relevant principles[30] We begin with a review of the key authorities relied upon by the parties in this area. First in terms of chronology is Mercury Energy, which involved a judicial review challenge to a contracting decision of a state-owned enterprise. Deliveringthe Board's advice, Lord Templeman accepted that the decisions of state-owned enterprises were, in principle, amenable to judicial review but said:10It does not seem likely that a decision by a State-owned enterprise to enter into or determine a commercial contract to supply goods or services will ever be the subject of judicial review in the absence of fraud, corruption or bad faith.[31] Next in the chronology is Pratt Contractors Ltd v Transit New Zealand(Pratt Contractors).11 In that case the plaintiff pursued a claim based upon a contract arising out of a tender process conducted by Transit New Zealand, a Crown entity. In the Court of Appeal it was accepted that Transit was bound by a process contract and that the contract included an implied contractual term to treat tenderers even-handedly. However, McGrath J cautioned against the application of "standardsakin to those required in judicial review proceedings involving the exercise ofstatutory powers" when assessing such contractual claims.129 Ririnui v Landcorp Farming Ltd [2016] NZSC 62, [2016] 1 NZLR 1056.10 Mercury Energy Ltd v Electricity Corp of New Zealand Ltd, above n 8, at 391.11 Pratt Contractors Ltd v Transit New Zealand [2003] UKPC 83, [2005] 2 NZLR 433.12 Transit New Zealand v Pratt Contractors Ltd [2002] 2 NZLR 313 (CA) at [98].[32] On appeal Lord Hoffman, delivering the Board's advice, agreed with Finn J'sdictum in Hughes Aircraft Systems International v Airservices Australia that an implied duty of good faith and fair dealing:13 does not as such impose on [the employer] under the guise of contract law, the obligation to avoid making its decision or otherwise conducting itself in ways which would render it amenable to judicial review of administrative action.[33] We then come to Lab Tests Auckland Ltd v Auckland District Health Board. Three district health boards undertook a RFP process for community laboratory services for the Auckland region. Although Diagnostic Medlab was the incumbent provider, through that process Lab Tests emerged as the preferred tenderer and ultimately the three health boards entered into contracts with Lab Tests.[34] Drawing upon Mercury Energy, Pratt Contractors and other authorities, this Court in Lab Tests stated the following principles:14(a) Public bodies must exercise their powers, including contracting powers, in accordance with their empowering statute. If they do not, then their decisions are amenable to review on the grounds of illegality.(b) The procedural obligations of a body performing a public function will vary with context so that a public body exercising a particular statutory power may be bound by natural-justice obligations when making a particular decision but those obligations may have less, or even no, relevance to the same body when making another type of decision under the statute.(c) "Context" for these purposes "includes the nature of the decision being made, the nature of the body making the decision and the statutory setting within which the decision is made".1513 Pratt Contractors Ltd v Transit New Zealand, above n 11, at [46]–[47] citing Hughes Aircraft Systems International v Airservices Australia (1997) 76 FCR 151 (FCA) at 197G.14 Lab Tests Auckland Ltd v Auckland District Health Board, above n 8, at [55]–[59].15 At [58].(d) The courts will intervene by way of judicial review in relation to contracting decisions made by public bodies in a commercial context in limited circumstances, although subject to the contextual point just mentioned. In a commercial context, judicial review will normally only be available where there is fraud, corruption or bad faith, or in analogous situations.16 A broad-based probity-in-public-decision- making review is not to be applied.17 The imposition of more onerousprocedural obligations may unduly fetter a public body's ability tonegotiate effectively.18 Generally, other accountability mechanisms such as ministerial control and parliamentary oversight are likely to be seen as more appropriate.19[35] Finally in this line of authorities we mention Ririnui, a decision of the Supreme Court released after the judgment the subject of this appeal.20 In Ririnuithe focus of the application for judicial review was decisions taken in the course ofLandcorp's sale of a particular block of land called Whārere. Landcorp proposed tosell Whārere because it was no longer a fit with Landcorp's other operations.Landcorp was party to a protocol with the Office of Treaty Settlements, whereby Landcorp would give early warning of proposed land sales to the Office to enable the Crown to consider whether the land was of interest to it for settlement purposes. Where land was of interest, the protocol set out a process by which the Crown could purchase that land in order to bank it for future Treaty settlements.[36] Landcorp went through the protocol process in respect of Whārere but the Crown disclaimed any interest in buying the land because it believed, wrongly as it transpired, that Treaty claims to the land had been settled. Landcorp then entered into a binding commitment to sell Whārere to private individuals. Mr Ririnui brought judicial review proceedings on behalf of a Treaty claimant who wished for, but had been denied, a proper opportunity to buy the land.16 At [91].17 At [85].18 At [88].19 At [59].20 Ririnui v Landcorp Farming Ltd, above n 9.[37] Arnold J, giving the leading judgment of the majority, referred to the general proposition of law that a decision by a state-owned enterprise to enter into a commercial contract was unlikely to be reviewable in the absence of fraud, corruption, bad faith or some analogous circumstance.21 But he said, "even if that proposition is accepted, it does not necessarily apply to all contracting decisions made by state-owned enterprises".22[38] In reaching the view that a broader scope of review was appropriate in the case before the Supreme Court, Arnold J identified various factors which heconsidered revealed a close connection between Landcorp's own land-owning activities and the Crown's fulfilment of its Treaty obligations to Māori. He referred to Landcorp's statement of corporate intent, which included references to the protocol and to the reconciliation of Treaty of Waitangi settlement processes with its commercial objectives. He also took into account various steps taken by the Landcorp board in the course of sale of the land, which reflected that the board did not view the sale process as simply a commercial one but rather recognised it had a substantial public interest component.23 Accordingly, Arnold J saw the case as falling outside the general proposition because it was not an ordinary commercialtransaction "given the special context of former Crown land, the Treaty and Maori interests".24 The majority allowed the appeal in part, granting the application for judicial review on grounds that various decisions made in connection with the sale were made under the influence of a mistaken view that Treaty claims to Whārere had been settled.Legal test applied in the High Court[39] Woodhouse J did not accept there was a prima facie rule that only narrow grounds of review are available in respect of commercial contracting decisions. He said of the contextual inquiry outlined in Lab Tests:2521 At [65].22 At [65].23 See [74].24 At [65].25 HC judgment, above n 1, at [63]. in my judgment the Court, in its reference to Mercury Energy, was notstating what might be described as a prima facie rule "subject to context".The starting point is context. With context as the starting point, it may be that certain types of decision are so plainly founded on existing contractual arrangements that there is no scope for the application of broader public law procedural standards.[40] The matters Woodhouse J identified as supporting his conclusion that a broad scope of review was appropriate in this case were as follows:(a) The RFP was for the provision of services which would enable the Ministry to discharge its statutory duty to implement a problem gambling strategy concerned with all health aspects of problem gambling: public health and clinical aspects. The Judge considered this feature moved the case a considerable distance from Lab Tests(where the tender was for clinical services) and some — perhaps all— of the other cases.26 He said most of the cases endorsing a narrow scope of review involved tenders or proposals for specific areas which were generally small parts of much broader areas of public activity.27Although the parties directly affected were the proposers —successful and unsuccessful — the Judge said this "should notobscure the relevant point of context — the Ministry was making a decision bearing on all aspects of public health and clinical servicesfor problem gambling".28(b) Added to this was the absence of statutory provisions requiring the Ministry to act commercially, such as those present in Lab Tests and in s 4 of the State-Owned Enterprises Act 1986, the latter governing the contracting situation in the Mercury Energy case.29(c) The Judge attached significance to the absence in the Act of prescriptions as to the process to be followed in implementing the strategy — statutory provisions of the type governing the tendering26 At [75].27 At [89].28 At [90].29 At [77].process in Lab Tests.30 He drew support from the decision of Collins J in Telco Technology Services Ltd v Ministry of Education (Telco) in concluding that the vacuum created by an absence of specific legislative provisions may be filled by public law principles such as natural justice and procedural fairness.31(d) The Mandatory Rules supported a broad scope of review because of their emphasis upon the need for tender documentation to be complete in terms of information and for the process to be fair, and the requirement in the Rules that tenderers be advised of any modification in the essential requirements and evaluation criteria.32(e) Although the fact the decision-maker was a core government department was not conclusive, he said "when it is combined withother matters of context, including the absence of express statutory constraints of the sort considered in Lab Tests, this does providefurther support for the Foundation's primary argument as to thebroader scope of judicial review".33(f) The absence of any contractual rights for proposers and corresponding contractual obligations on the Ministry.34(g) There was no evidence that the imposition of procedural obligations of the sort contained in the arguments for the Foundation wouldimpede the Ministry's ability to make a decision on the RFP as it would not limit any relevant ability to negotiate.35 The Judge said:36The steps leading to the decision, and the decision itself, did not as a matter of fact involve any negotiation with any of the organisations making proposals. What is more, the RFP did not, at least in any express way, put any obligation on the Ministry to consult tenderers during the decision making process, although it had the ability to do so30 At [76].31 Telco Technology Services Ltd v Ministry of Education [2014] NZHC 213 at [37].32 HC judgment, above n 1, at [86].33 At [92].34 At [94].35 At [99].36 At [99].if it chose to. This is to be compared with the express obligation on the Ministry, contained in mandatory rule 28, to notify any modifications of essential requirements and evaluation criteria in the RFP.(h) The Judge accepted the decision the subject of challenge was too complex to be the subject of a merits-based review.37 But, he said, theFoundation's grounds of review did not raise issues going to the substantive merit of the decisions.(i) The Judge considered that there were no other effective accountability mechanisms, rejecting as suitable alternatives an inquiry by the Auditor-General under s 18 of the Public Audit Act 2001 or a complaint to the Ombudsman.38Discussion[41] We agree with the Crown that where the decision the subject of review is a procurement (contracting) decision made in a commercial context, that is the starting point for consideration of the appropriate scope of review.39 We understand the High Court Judge's analysis to be that the decisions were not taken in a commercial context. However, that analysis gave no weight to the fact that these were procurement decisions. We agree with the Ministry that where decisions are made by the Crown in the course of a procurement process, that will usually provide the commercial context. It follows the prima facie position will be that only narrow review is appropriate, subject to any relevant contextual matters indicating a need for the High Court to have broader powers of review.[42] This Court identified relevant contextual matters in Lab Tests, noted above. To those we would also add the nature of the interest sought to be protected by the party seeking judicial review. We say this because it may be that a decision taken in a commercial context by a state actor does entail wider public interest considerations, suggesting that a broader scope of review will be appropriate, as the Crown acknowledges. But to avail itself of that broader scope of review, the37 At [104].38 At [109].39 Lab Tests Auckland Ltd v Auckland District Health Board, above n 8.applicant for review must raise issues relevant to that public interest and not just be a disappointed commercial party, seeking to take advantage of public remedies in a commercial context.[43] In our view, the critical contextual consideration in this case is that the party seeking review, the Foundation, is a disappointed participant complaining about the RFP process through which it hoped to secure a contract to provide paid services to the Ministry. We acknowledge that, as the Judge identified, the RFP process was for the provision of services to enable the Ministry to implement its problem gambling strategy and there is indeed a public interest in the Ministry discharging its statutory duty to formulate a plan and provide those services.40 But we think the Judge was wrong to conflate the public interest in the Ministry discharging its duty to provide those services with the need for broad review in this case. He was wrong to discountas "too abstract to be of assistance" the Ministry's argument that the Foundation'sclaim did not truly engage the public interest it sought to invoke in support of broad review.41[44] We also do not agree with his view that the services at issue in this proceeding are relevantly distinguishable from the clinical services in Lab Tests. The Judge saw it as relevant that the Ministry was contracting out most of the implementation of its problem gambling strategy and was contracting out public health as well as clinical services.42 There were very broad public health implications in the provision of laboratory testing for the population covered by the three health boards in Lab Tests. It would be a peculiar outcome if the scope of review is determined by the percentage of contracting out that occurs.[45] Ms Chen for the Foundation argues that you cannot split up actions taken in the course of pursuing a public interest objective — on her approach, ensuring the problem gambling strategy is adhered to requires the presence of the full scope of review at each stage of its implementation, including down to the level of the RFP process. We do not accept the logic of her argument. Although the development of the problem gambling strategy was a public act with public consequences, entering40 See HC judgment, above n 1, at [75].41 At [88].42 At [89].into individual contracts through a procurement process to implement that strategy does not have public consequences which call for the full panoply of judicial review. And to the extent there is public interest in the procurement process, this challenge does not engage that interest as the complaint is the prejudice to the Foundation'scontracting prospects.[46] We contrast this challenge with that in Rirunui. Although Ririnui also involved a party disappointed in their hopes of securing a contract, a feature of that case missing in the present was an alignment of the claimant's grievance with the broader public interests in play — in Ririnui, the resolution of Treaty claims. The Foundation's claim, in contrast, is grounded firmly upon unfairness to it in the process and disappointment of its own private interests. It is a complaint which could have been advanced through contract if the process had given rise to a process contract. The only way in which its allegations engage with the public interest in the provision of public health services is the Foundation's attempted invocation of that contextual feature in favour of broad judicial review.[47] We also do not see the various absences the Judge identifies (see [40] above) as supporting the conclusion he reached as to the scope of review. Although we agree the absence of a statutory direction to act commercially is a factor to be considered, as we have noted, the fact that this was a procurement process is a powerful indicator that the context was commercial. So too was the direction in the Mandatory Rules that procurement decisions were to be made on the basis of value for money of services to be supplied in terms of the essential requirements and evaluation criteria.[48] The absence of statutory procedural protection does not point, in a commercial context, toward a need for the imposition of public law procedural protection. We do not agree with the approach articulated in Telco that the existence of a "vacuum created by an absence of specific legislative provisions" in some wayinvites public law procedural principles to fill it.43 We do not see the logic in that conclusion; it is surely the presence of onerous procedural obligations which might arguably take a contract out of the normal run of commercial contracts. In this case43 Telco Technology Services Ltd v Ministry of Education, above n 31, at [37].the absence of such provisions is neutral. While it is true the Act does not stipulate process, it does not address at all how the responsible Crown agency is to discharge its obligations to provide the relevant services.[49] The absence of contractual obligations created by the RFP process also does not assist the Foundation. It weighs on the opposite side of the scale, in favour of narrow review. The Ministry's stipulation that the RFP documentation did not create contractual obligations, and its reservation of the power to contract with a party not participating in that process or providing a non-complying proposal, makes plain the informal nature of the process.[50] Ms Chen argues that broad review is necessary to ensure that, as a public body, the Ministry is held to a high standard of conduct. We accept the Solicitor'ssubmission that this is to invite a probity assessment of public administration, an assessment rejected in Lab Tests.44 We agree there is a public interest that agencies show integrity and competence in the discharge of public powers and duties. But we consider that alternative avenues existed for the Foundation to address concerns regarding the process — complaints to the Ombudsman's office or to the Auditor-General. The Judge discounted these as viable remedies. We consider he was wrong to do so. These two bodies can and do subject the processes of Crown agencies to scrutiny and hold them to best practice as embodied at the time in the Mandatory Rules, which are now replaced by the Government Rules of Sourcing.45And the Ministry is responsible to Parliament for how it spends the appropriation it receives for delivery of the problem gambling strategy.[51] We accept that the Mandatory Rules applied to this RFP process. The Judge saw the presence of these rules as weighing in favour of broad review, even though he treated the absence of statutory and contractual rules as also favouring that level of review. The introduction to the Mandatory Rules records that they "set outmandatory standards and procedural requirements for the conduct of procurement bygovernment departments", for the purpose of facilitating "competitive participation by domestic and foreign suppliers in New Zealand's government procurement44 Lab Tests Auckland Ltd v Auckland District Health Board, above n 8, at [85].45 See Public Audit Act 2001, ss 16 and 18; and Ombudsmen Act 1975, s 13.market". It would be strange if the application of these rules, designed to be used in an inherently commercial context of procurement, were to be construed as evidence that this was not a commercial context. The Rules were, as the Ministry submits, the executive government's non-legal directive to its agencies as to standards and conduct required to be observed to ensure fairness, quality and integrity in procurement processes. While they bind the agency, as we come to later, we see no reason to interpret the Rules as imposing, in this case, a procedural obligation enforceable by the disappointed potential contractors; the Ministry did not bind itself to a process contract to that effect. The public interest that undoubtedly exists in the quality and integrity of procurement processes is, in this case, met by the availability of the mechanisms we have already referred to. The Mandatory Rules show unsurprisingly that the executive itself seeks to achieve those ends.[52] The Judge saw nothing in the Ministry's submission that, to echo theconcerns expressed in Lab Tests, the imposition of broader procedural obligations might handicap the Ministry in meeting its objectives of providing the public health services and that it might simply provide ammunition to "determined private sector service providers".46 The facts of this case, however, rather prove the Ministry'spoint. These court proceedings have now stretched out beyond the date on which the three-year service plan would have expired. In light of the challenge, the Ministry elected to leave existing arrangements in place, which meant that the Foundation has had the benefit of the status quo, which the decisions were to replace, for the entire time. A three-day hearing in the High Court, a long wait for a judgment from that Court and now the further delay of appeal have all disrupted the implementation of the original six-year strategic plan. Indeed, so great has been the delay occasioned by this proceeding that its outcome is, at least in the one sense, moot. The contracting period covered by the RFP expired on 30 June 2016. The extension ofthe incumbents' contracts to 2017 has delayed the introduction of what the Ministry had concluded was the preferred mix of services and has delivered, instead, the status quo.46 HC judgment, above n 1, at [95] citing Lab Tests Auckland Ltd v Auckland District Health Board, above n 8, at [78].[53] To conclude, the following factors overwhelmingly favour the narrow scope of review identified in Lab Tests in this case:(a) The decision the subject of review was to contract for services following a procurement process.(b) The Foundation is a disappointed party in that procurement process. Participants to the procurement process agreed that participation in the process would not create contractual relations.We do not consider there are any compelling contextual factors that shift the scope of review from this position. As we earlier noted, this finding effectively disposes of this appeal as none of the grounds of review come within the narrow scope of review that applies in these circumstances.[54] We now turn to apply this analysis to the grounds upheld in the High Court.Second ground of appeal: did the Judge err in applying judicial test for apparent bias and in finding unconscious bias?[55] As this Court said in Lab Tests, evidence of a conflict of interest may lead a court to grant review even in "narrow-scope" cases if the conflict of interest creates a situation analogous to fraud, corruption or bad faith, thereby compromising the integrity of the contracting process.47 The Crown submits, and we accept, that to ground a successful challenge the conflict would have to be at such a level that it involved financial interests, genuine insider knowledge (such that it might lead to corruption) or clear personal interests in a tender outcome (such as a close familial relationship with a tenderer) such that the conflict was unable to be managed.[56] The Foundation did not argue in the High Court that there was any conflict of interest which could be described as analogous to bad faith. It argued, however, that the decision-making was tainted by apparent bias. The Judge accepted that47 Lab Tests Auckland Ltd v Auckland District Health Board, above n 8, at [91].argument, applying the standard for apparent bias used in judicial proceedings.48The Crown says he was wrong to do so.Relevant facts[57] In the Ministry's RFP process, panel members were required to complete aconflict-of-interest form and answer "yes", "no" or "potentially" to the question ofwhether they had a conflict of interest (which included "perceived" conflicts). Two external members declared conflicts. Ms West declared she had recently left the employment of the Foundation, having worked there for about eight years. The Ministry decided that as she was no longer in the employment of the Foundation and had no involvement with or interest (financial or otherwise) in the Foundation, she could continue as a member of the panel.[58] Mr Pereira declared a conflict of interest in connection with one proposal; he had a direct family member on the board of trustees of an entity submitting a joint proposal. Subsequently, the Ministry established from the relevant proposal that the entity in question was not a joint venturer; it was only a sub-contractor to the provider submitting the proposal. But on the basis of his original declaration the Ministry addressed this conflict of interest by resolving that Mr Pereira notparticipate in scoring that entity's proposal.[59] Of the internal panel members, Mr Levy, the non-scoring chair, considered he might have a perceived conflict of interest as a senior contract manager for the Ministry. He let Mr Bartling, his group manager, know of the perceived conflict before the first panel meeting and, at Mr Bartling's request, sought advice from a Ministry procurement adviser. Mr Levy was advised that as a non-scoring chair he need not complete a conflict-of-interest-management plan.[60] Another Ministry employee, Mr Williams, declared a perceived conflict of interest as a senior contract manager for the Ministry. The Ministry considered that both Mr Levy's and Mr Williams' perceived conflicts were adequately managed48 HC judgment, above n 1, at [315].through the Ministry's ethical code of conduct and because Mr Levy was a non-scoring member of the panel.[61] Mr Everist declared he was previously employed by a Crown agency which engaged with the problem gambling sector. The Ministry decided that he could continue as a member of the panel as he was no longer in the employment of that agency and had no involvement with or interest (financial or otherwise) in that agency.High Court judgment[62] The Foundation argued that the Ministry failed to adequately address the actual or potential conflicts of interest and that, in the circumstances, this constituted an error of law.[63] Woodhouse J identified what he described as "four sets of rules" binding upon the panel and its members, which addressed in some way conflicts of interest and bias:49(a) The Mandatory Rules which included:Rule 14:Departments must have in place policies and procedures to eliminate any potential conflict of interest on the part of those engaged in or having influence over a procurement.Rule 43:Departments must receive, open and evaluate all tenders under procedures that guarantee the fairness and impartiality of the procurement process.(b) The Ministry's ethical code of conduct, which applied to the internal panel members employed by the Ministry and which in substance required that:As with all panel members, relevant discussion and consideration of all proposals are to be based on the content49 At [287].of the proposals presented. Prior knowledge or comments outside of the proposal content were to be excluded.(c) Directions to panel members in the declarations they were required to sign as to actual or perceived conflicts of interest.(d) What the Judge characterised as a "voluntary code of conduct"— the "personal-knowledge-exclusion rule" that all membersagreed to for the purposes of the panel process. The panel members agreed that when assessing the proposals they would have regard only to their content and exclude all prior knowledge.[64] The Judge addressed the issues of apparent conflicts of interest and apparent bias together. As to the standard to which the Ministry was to be held, he recognisedthat "[m]ore often than not the standard to apply to administrative decision-makerswill be lower, and often substantially lower, than that applied to judges".50 However, in this case, the Judge determined that the applicable test was that which applies to judicial decision-making, the test for apparent bias enunciated by the Supreme Court in Saxmere Co Ltd v Wood Board Disestablishment Co Ltd (Saxmere):51A judge should not sit if a fair-minded and informed lay observer would have a reasonable apprehension that the judge might not bring an impartial mind to the resolution of the question the judge is required to decide.(Footnote omitted.)[65] The Judge applied the two-step analysis described by Blanchard J in Saxmerefor the identification of apparent bias:52(a) first, the identification of what it is said might lead a judge to decide a case other than on its legal and factual merits; and(b) secondly, there must be "an articulation of the logical connectionbetween the matter and the feared deviation from the course ofdeciding the case on its merits".(Footnote omitted.)[66] The Judge said that the Saxmere test applied because of certain matters of context: the Mandatory Rules, which required the elimination of conflicts of50 At [280].51 Saxmere Co Ltd v Wood Board Disestablishment Co Ltd [2009] NZSC 72, [2010] 1 NZLR 35 at [89].52 At [4] (footnote omitted).interests; the Ministry's ethical code of conduct; the personal-knowledge-exclusion rule and the effective direction to panel members in the conflict declaration. He said that when those rules were assessed together, they made two things clear:53[T]he standard required to be met by the panel was a high one; and that it is a standard which requires the question whether there was apparent bias in this case to be determined by applying the principles outlined in Saxmere, as earlier discussed [67] The Judge said that the "personal-knowledge-exclusion rule" was not very different from what is required of a jury in a criminal trial.54 In light of that rule, there was no obvious reason as to why people with pre-existing knowledge of some of the organisations making proposals were selected. He observed:55There was no evidence in this case that the Ministry had no option but to select panel members who not only may have known about proposers in a general way, but who had actually had dealings with one or more of the organisations and, it appears in some cases, to a reasonable extent.[68] As to the first step in the Saxmere test — what might have led a panel member to evaluate proposals other than in accordance with the rules — the Judge said that was obvious:56[W]ith all of the panel members who acknowledged potential bias or conflicts, it was existing knowledge of, and in some cases existing or prior association with, one or more of the organisations making proposals.[69] He said the second step, articulation of the logical connection between the matter and the feared deviation from the course of deciding the case on its merits, was equally plain.57 It was the use of existing knowledge of, and prior associations with, the organisations submitting proposals.[70] The Judge concluded that apparent bias had been established:[335] That is a conclusion which I consider can properly be founded on therules coupled with the Ministry's own evidence that five of the seven panelmembers had relevant knowledge. This evidence is what is critical when related to the rules. There is other evidence for the Ministry that the panel's53 HC judgment, above n 1, at [315].54 At [316].55 At [321].56 At [323].57 At [324].decisions were unaffected by prior knowledge and prior associations, but that is not to be taken into account. Also irrelevant to the ultimate question as to whether there was apparent bias is the evidence from the Ministry that the Ministry officers considered that declarations of potential conflicts or bias had been adequately managed. If the question of management of conflicts was relevant, then in my judgment this matter was not properly managed. On an objective assessment there was apparent bias because the Ministry established an evaluation panel with five of its seven members armed at the outset with knowledge they were then required somehow to eradicate, not just from their conscious thoughts, but also from their subconscious minds. It is a real possibility that these panel members may not have brought impartial and unprejudiced minds to their evaluations. All of them expressly stated, for the various reasons they also stated, that there was the possibility that, in effect, they might not meet the standard. If this had happened with a judge, the judge would be disqualified. If it happened with a juror, the juror would be stood aside. The reasons for this are the reasons why I consider the declarations to be important evidence in support of the conclusion already reached that there was apparent bias.[71] The Judge said the following evidence supported his conclusion that there was apparent bias:(a) Mr Levy's evidence was that many of the organisations making proposals were known to members of the panel and it was clear to him that, in pre-scoring, panel members had "in some cases, relied on orinferred information relating to particular criteria and sub-criteria that simply was not present in the proposal being considered". The Judgesaid this evidence was reasonably telling in indicating "the presenceof apparent bias or, which in this context is the same thing, breach ofthe various rules".58(b) Evidence from the PwC probity report, which noted a possible"exception" in relation to potential conflicts of interest declared by two unnamed panel members; from the context, Mr Pereira and Ms West. PwC personnel had compared the scoring patterns of these two panel members to the scoring of other panel members to see"whether there was any significant deviation in their scoring which might indicate any bias in their scoring". PwC found no such58 At [311].evidence but the Judge saw it as significant that PwC nevertheless concluded:59Despite finding no evidence, the [Ministry] should remain aware that the market could perceive that these two panel members could haveunduly influenced other panel members' scoring of proposals in thejoint panel moderation sessions.(c) The evidence of Mr Mullins, a statistician, who provided evidence for the Foundation as an expert witness. The Judge set out lengthy portions of Mr Mullins' evidence in his judgment, which he said"pointed to objective indicators, from statistical analysis, of at least the possibility of bias". An example of the Judge's treatment of thisevidence is as follows:[302] Much of Mr Mullins' evidence as to the possibility of biasdid not identify individuals, at least by name, but I am satisfied that, if the evidence does establish the possibility of bias, it is not necessary in this case to identify individuals. The concern is with decision making by a panel. If it is possible that a conclusion at one or more of the panel evaluation stages was affected by bias, to an extent that makes the particular conclusion unreliable, that, in my judgment, is enough. There was further evidence from Mr Mullins which wentbeyond opinions of possible bias affecting the panel's conclusions inthis general way. There was analysis of individual scoring, with indications of the possibility of bias arising at that point. One prominent example of this, explained in some detail, was what Mr Mullins described as "inter-rater variability" and which I described as "aberrant scores". Mr Mullins did explain that things such as this might have arisen for reasons other than bias. He referred on a number of occasions, as did Dr Jury, to indications of a lack of adequate training. But the existence of more than one possible cause of aberrations in scoring, or other indicators of things going wrong, does not remove the possibility of bias having had a material influence.(d) The evidence of another expert witness called by the Foundation, Dr Jury.60 Dr Jury has extensive experience overseeing the planning and funding of health services for public and private health bodies including procurement processes. He was asked by the Foundation to comment on the RFP process. The Judge narrated aspects of Dr Jury'sevidence in which Dr Jury was very critical of that process, highlighting what Dr Jury considered to be discrepancies in scoring.59 At [312].60 At [304].The context of this discussion in his judgment shows that the Judge'spurpose was to link these defects in the process to the possibility of unconscious bias.Discussion[72] It follows from our earlier discussion regarding the scope of review that we consider the Judge applied the wrong test to determine whether a conflict of interest existed of such a nature to justify the grant of relief in this circumstance. As the Judge recorded, it was not argued by the Foundation that there was in fact any conflict of interest which could be described as analogous to bad faith or corruption. In this case, that fact alone disposes of this ground of review.[73] Having reached that view it is difficult to proceed past that point. It is implicit in that finding that agencies acting in the procurement sphere should not be able to be held (by disappointed prospective providers) to a higher standard of conduct in managing conflicts of interest.61 The position is summarised by Professor McLean:62Decision-makers are allowed to bring strongly held favourable or adverse views to the process; decision-makers are not required to disqualify themselves or to extend an opportunity to the party to correct any prejudicial views held about the tenderers by the decision-makers.Even where there is a process contract, fairness and good faith only require honesty and a willingness to consider information which might change their view.63[74] There are, however, errors in the Judge's reasoning process on whichcomment is necessary. Even if minded to subject this decision-making to a broader scope of review, we would not accept that the Ministry should be held to a judicial standard of conduct in its decision-making in this case. We say this for the following reasons.61 We note that the Judge rejected that proposition at [314]. We consider he was wrong to do so.62 Janet McLean "Convergence in Public and Private Law Doctrines — the Case of PublicContracts" [2016] NZ L Rev 5 at 10.63 Pratt Contractors Ltd v Transit New Zealand, above n 11, at [47].[75] First, this was a decision made at the end of a procurement process to enter into negotiations to contract. The selection of a potential contracting party is far removed from a judicial process. The fact that the Mandatory Rules refer to elimination of conflicts is no reason to hold the Ministry to the judicial standard for apparent bias. These are, after all, rules which apply to all procurement processes including those, one assumes, for the purchase of the most prosaic items.[76] Secondly, the "no previous knowledge" direction given by Mr Levy, to which the Judge attached so much weight, was no more than good practice — an attempt to ensure, as Mr Levy said, that incumbency was not given weight beyond that provided for in the criteria. It can hardly elevate this to a quasi-judicial process.[77] Thirdly, we also do not consider that the directions to panel members in the declaration forms are properly characterised as a fourth set of rules. They were merely part of the same methodology utilised by the Ministry to improve the quality of the decision-making.[78] The second matter upon which we comment is this. Even if he were correct to apply the Saxmere test, the Judge erred in his consideration of the evidence of Dr Jury and Mr Mullins. He treated that as evidence of bias (and in this context he could only mean actual albeit unconscious bias) contaminating the process. The test for apparent bias does not involve, at any point, an attempt to predict or inquire into the actual thought processes of a judge.64[79] The third issue is that, even if the Saxmere test were appropriate, it is difficult to see how a case for that standard was made out. Judges frequently hear cases involving parties who have appeared before them previously. They are able to bring an impartial mind to bear by excluding from their consideration that earlier knowledge, just as the panel members agreed to do in this case. Prior knowledge of, or prior dealing with, a party by one of the panel members would not, on our assessment, cause a fair-minded lay observer to reasonably apprehend that the panel member might not bring an impartial mind to bear on the evaluation. What was64 Ebner v Official Trustee in Bankruptcy [2000] HCA 63, (2000) 205 CLR 337 at [7]; andSaxmere Co Ltd v Wool Board Disestablishment Co Ltd, above n 51, at [4].imposed on this panel was a standard higher than that usually imposed on those fulfilling a judicial function. That was, of course, an unworkable standard. It would have required a panel made up of those with no previous involvement in the problem gambling sector. That would clearly not be in the public interest as such a panel would not have been able to properly evaluate these services.[80] The Foundation attempts on appeal to make something of Mr Pereira'sconflict declaration, describing it both as a family connection and a pecuniary interest. This was not an issue relied upon by the Judge in his findings. The connection was in any case remote — a relative working for a subcontractor to one of the prospective providers. The proposal revealed the true nature of that connection. It was not a pecuniary interest. It could not, we consider, meet even the stringent Saxmere test.[81] Finally, we say something about the admissibility of the evidence of Dr Jury and Mr Mullins. The appellant argues on appeal that this evidence was inadmissible because it was unhelpful.[82] Dr Jury comments upon the process adopted by the Ministry in selecting the panel, the training (or rather lack thereof) of panel members, the scoring processes at the pre-score, consensus and moderation stages, and the quality of the recommendations. This evidence was inadmissible in a judicial review proceeding. It seems that Dr Jury was given a very broad instruction — simply to "comment on the processes adopted" by the Ministry in the RFP. This no doubt explains, at leastin part, how discursive his evidence is. As the appellant submits, Dr Jury expresses his opinion on the very issues the Foundation invited the Court to consider — for example, whether the processes complied with the Mandatory Rules and whether the Ministry was under procedural obligations to notify changes in its process such as the inclusion of the moderation stage. Although evidence on the ultimate issue to be determined may be admissible if helpful to the fact-finder, in this case these are matters upon which the Court is equally well equipped to reach a view.65 Further, Dr Jury's evidence takes the form of a wide-ranging commentary upon theMinistry's evidence, without any apparent limitation as to the scope of what he could65 See Evidence Act 2006, s 25(2).or should comment upon. In the course of that commentary, Dr Jury gives his view as to the merits of the decision reached through the RFP. An inquiry into the merits is beyond the scope of this judicial review.[83] The appellant also objected to the admissibility of the evidence of Mr Mullins, a lecturer in statistics at the University of Auckland. There are many parts to this very extensive evidence. In broad-brush terms, Mr Mullins:(a) questions the appropriateness of the scoring system, saying that the Ministry did not have the "expertise required to evaluate competitivetenders of this sort, nor do they have the requisite statistical or qualityassurance skills necessary to ensure a fair and transparent process";(b) comments upon the pre-scoring and consensus scoring undertaken by the panel by delving down to consideration of pre-scoring by each panel member, identifying fundamental methodological errors in pre-scoring and consensus scoring, and commenting at one point"[t]he more I look at the documents, the more I find wrong with theprocess";(c) comments upon the moderation process; and(d) describes appropriate statistical approaches to correct bias in scoring and identifies how the process should have been constructed from a statistical point of view.[84] We do not consider this evidence was admissible on any view of the scope of the review as again, in substance, it comments upon the quality of the decision-making and so is not substantially helpful.66 Moreover, as we come to, the proposition that the Ministry should have applied an academic statistician's skills toits assessment of the scoring is unsustainable.66 See Evidence Act, s 25(2).[85] The respondent takes the point that in the High Court the Ministry objected to the admissibility of only limited passages of the evidence of Dr Jury and Mr Mullins. While that may be true, this cannot alter the position that evidence was admitted which ought not to be have been. It was unhelpful for the reasons we have outlined and other reasons we come to. In any case, on the view we have taken, the admissibility of this evidence makes no difference to the outcome of the appeal.Third ground of appeal: was the Judge wrong to find lack of probative evidence?[86] The appellant argues that the Judge created a novel ground of review based on his assessment of the validity of the statistical methodology employed in thepanel's evaluation of the proposals. Although the Judge purported to review thepanel's methodology on the basis of lack of probative evidence/mistake of fact, his findings were based on the material unreliability of findings at various stages of the procurement process. The Ministry says that in reality this was a review based upon the substantive merits of the decision.[87] The appellant also says that, even if such a ground of review could be invoked in a case such as this, the Judge's reliance upon Mr Mullins' evidence aboutstatistical deficiencies in the process misconceives the nature of the RFP process. This was not a statistical assessment. Rather, it was a human process in whichproposals were judged against criteria and weightings at the Ministry's solediscretion.The Judge's approach[88] The Judge said that the Foundation's claim based on mistake of fact or lackof probative evidence came within the broad principles justifying review under the general heading of mistake.67 The Foundation argued panel members were mistaken in their apparent beliefs that they were using a well-designed evaluation process, that they were correctly implementing that process and that, in consequence, their67 HC judgment, above n 1, at [225].conclusions at each stage were reliable and could be used as a reliable foundation for decision. He said:68When considering whether flawed methodology of this nature can be the subject of judicial review, it is of importance that the Foundation's evidencethat I have considered, and will outline shortly, is not evidence directed to the substantive merit of the decision, or to the substantive merit of preliminary conclusions reached by the panel at each stage. As already indicated, it is evidence to the effect that, because of flawed methodology, various conclusions were unreliable.[89] The Judge said the ground of review advanced was similar to that discussed by the Privy Council in Re Erebus Royal Commission in the following passage:69The first rule is that the person making a finding in the exercise of such a jurisdiction must base his decision upon evidence that has some probativevalue in the sense described below. The technical rules of evidence applicable to civil or criminal litigation form no part of the rules of natural justice. What is required by the first rule is that the decision to make the finding must be based upon some material that tends logically to show the existence of facts consistent with the finding and that the reasoning supportive of the finding, if it be disclosed, is not logically self-contradictory.[90] The Judge accepted the Foundation's argument that the evaluation process, other than moderation, was statistical in nature, and the reliability of the process, and the conclusions reached, were properly assessed by a statistician.70 He attached particular significance to para 48 of the RFP which recorded the weightings to be applied by the Ministry when assessing proposals. He also attached significance to the individual score sheets utilised by evaluators, which required them to allocate a numerical score against standard criteria at the pre-scoring stage. His acceptance of the Foundation's characterisation of the process was bolstered by Mr Mullins'opinion that it was a statistical process and by the absence of any expert evidence for the Ministry contradicting this.[91] The Judge conducted an extensive review of Mr Mullins' evidence. Although we regard the evidence, as we have recorded, as irrelevant, we summarise the portions relied upon by the Judge as necessary to his reasoning:68 At [226].69 Re Erebus Royal Commission [1983] NZLR 662 (PC) at 671.70 HC judgment, above n 1, at [234].(a) The panel was instructed to create a number of numerical scores which were summarised using a statistical measure of averaging the data, which was then used for the purposes of producing a consensus score and deriving a ranking for each of the providers. This was a statistical process.(b) Because all the sub-criteria other than price are measured on a qualitative basis, rather than a strict measurement basis, they aresubject to much more "noise" than a quantitative measure might be.Mr Mullins described noise as corresponding "to the averagedifference between the raw observations [of individual evaluators] and their overall average, or possibly an overall score obtained by a'consensus' process". Mr Mullins examined the "sensitivity" of thefinal score to small changes in one or more of the scoring criteriabecause that "tells us something about the level of 'noise' present inthe qualitative measurements". As a consequence of that review, Mr Mullins considered that the main criteria were very sensitive to small changes in the raw average scores for sub-criteria. He said:This sensitivity is important, as each evaluator, in the absence of formal well-defined criteria for allocating a score (as was the case here), may hold quite a different opinion as to an appropriate score for any given tender, which can have a significant impact on the total score achieved.(c) "Inter-rater variability" has already been referred to. It is Mr Mullins'expression to describe differences in the pre-scores between panel members. His broad proposition was that flawed methodology is indicated if there are marked differences in the scores and that this is compounded if there are no formulae in place to make appropriate adjustments where there are such marked variations.(d) The use of the consensus process resulted in large modifications to some scores. This suggests, Mr Mullins said, that some of the panel members from the Ministry may have had a considerable influence over others in reaching consensus scores. In Mr Mullins' opinion:The use of a consensus decision-making process undermines the initial independence of the various evaluators. Compounding the issues with consensus scoring is the fact that the individual scoreswere made in the presence of 'noise' as described above.The magnitude of this noise is both unknown and unknowable. This magnitude may be great enough (and in my opinion is likely to be great enough) to obscure any real distinction between tenderers. In addition to this noise, there is also the possibility of biases arising through the mis-application of the chosen statistical measures (e) Mr Mullins' opinion proceeded on the basis that Mr Levy entered a zero where an individual evaluator had entered no score at the pre-score stage. Mr Mullins said this use of zeros undermined the reliability of the consensus scores which the decision-maker drew on to reach his decision.71(f) The moderation process was poorly documented. Moderation produced very significant shifts from the rankings achieved through pre-scoring and consensus scoring which showed, Mr Mullins said,"the possibility of bias among evaluators is very real".[92] On the basis of this evidence, the Judge accepted Mr Mullins' conclusion that there was material unreliability in the process which generated the recommendations.72 Since the Ministry conceded that nothing occurred after the panel process and prior to decision which could cure any fundamental deficiency in the recommendations, he found the ground of review made out. He concluded that the evaluation process was a careful step-by-step process.73 Because of this the evidential foundation for a valid final decision was absent if there was, as here, a series of unreliable conclusions at various stages of the process.7471 The Ministry explained, and the Judge was satisfied, that Mr Mullins proceeded upon a mistakenview as to the treatment of missing values and the use of "replacement zeros". Nevertheless, the Judge accepted Mr Mullins' evidence that the use of replacement zeros was a source of unreliability.72 HC judgment, above n 1, at [274].73 At [271].74 At [273].Discussion[93] It follows from our conclusion as to the scope of review that mistake of fact/absence of evidence was not an available ground of review in this case. For that reason we deal only very briefly with this ground of review.[94] As the Judge said, this ground of review did not fit easily into the ground of mistake of fact or law. At best it can be characterised as an error of law of the variety described in Bryson v Three Foot Six Ltd:75[26] An ultimate conclusion of a fact-finding body can sometimes be so insupportable — so clearly untenable — as to amount to an error of law: proper application of the law requires a different answer. That will be the position only in the rare case in which there has been, in the well-known words of Lord Radcliffe in Edwards v Bairstow, a state of affairs "in which there is no evidence to support the determination" or "one in which theevidence is inconsistent with and contradictory of the determination" or "onein which the true and only reasonable conclusion contradicts thedetermination". Lord Radcliffe preferred the last of these three phrases but he said that each propounded the same test. In Lee Ting Sang itself the Privy Council concluded that reliance upon dicta of Denning LJ in two cases "of a wholly dissimilar character" may have misled the Courts in Hong Kong inthe assessment of the facts and amounted in the circumstances to an error of law justifying setting aside concurrent findings of fact. Their Lordships were of the opinion that the facts pointed so clearly to the existence of a contract of service that the finding that the applicant was working as an independent contractor was, quoting the words of Viscount Simonds inEdwards v Bairstow, "a view of the facts which could not reasonably be entertained", which was to be regarded as an error of law. In Lee Ting Sangthe facts demonstrated so clearly that the applicant was an employee that it was the true and only reasonable conclusion.(Footnotes omitted.)[95] We do not accept the fundamental proposition upon which the Judge proceeded — that this was a statistical process, the quality of which was to beassessed by the application of a statistician's skills. It seems to us an improbable conclusion that, merely because the Ministry elected to use a number-based system to assist the panel with its evaluation of the proposals, it thereby committed itself to undertaking a statistically defensible analysis, with all the mathematical complexity that follows from that, through to the point of recommendation. The RFP providesthat "proposals will be evaluated against the criteria and weightings, which will be75 Bryson v Three Foot Six Ltd [2005] NZSC 34, [2005] 3 NZLR 721.judged by the Ministry at its sole discretion". It did not stipulate how that evaluation was to take place. It was therefore for the Ministry to decide how it would evaluate each proposal against those criteria and weightings indicated in the RFP. There was no commitment to prospective providers to utilise a statistically sound analysis, indeed any particular process, beyond the commitment to evaluate against the criteria and weightings. Although the use of numbers and percentages might make this process something that is recognised by a statistician, that cannot, we consider, bind the Ministry to a statistically accurate process leading up to the point of recommendation.[96] It is also not seriously arguable that the Ministry proceeded upon the basis that the contracting recommendations it received from the panel were the product of a statistical process. The use of a scoring system is a common approach in evaluative processes. It is a technique used to encourage focused analysis by panel members of the critical parts of proposals and also to achieve a level of consistency of analysis, although surely not a consistency that would withstand a statistician'sscrutiny.[97] Supporting the view that this was not a statistical process in the sense that Mr Mullins deploys the concept is the Ministry's inclusion of a moderation step in the process. At that point panel members were required to step back from their consensus scores and ask themselves, as a panel, if those consensus scores made sense. A significant re-ranking then occurred. This rather makes the case that the Ministry saw the numerical scoring part of the process as just that — a part of an overall process to assist with, but not dictate, the recommendation.[98] This was, as the Solicitor submitted, an intensely human process, bringing together people with different expertise and knowledge to evaluate the proposals. They were assisted at the initial stages by a scoring process undertaken individually, then as a consensus, which achieved focus on critical content in each proposal. The criteria against which proposals were assessed were qualitative not quantitative. This was not a strictly statistical analysis, which would take the panel ineluctably toward only one right answer based on statistical scores.[99] We also consider that the Ministry is right when it submits that the Judge did delve into the merits under this head of review. It is hard to conclude otherwisewhen his finding is based on the "material unreliability" of findings at various stagesof the process. In short, because he treated the evaluation process as a mathematical process, he admitted evidence that was concerned with and assessed the quality of that process. We consider he erred in doing so.Fourth ground of appeal: was the Judge wrong to find breach of Mandatory Rules and breach of legitimate expectation?[100] Again, in light of our findings as to the appropriate scope of review, we do not consider that breach of legitimate (procedural) expectations and breach of mandatory rules was an available ground of review. But even had these been available grounds of review, we are satisfied the Judge erred in finding them made out. We consider this latter issue briefly.[101] The Judge accepted both of the Foundation's arguments.76 The Ministry says that the Judge was wrong to find:(a) the Foundation had a legitimate expectation that the Ministry would not depart at all from criteria and weightings of criteria signalled in a particular part of the RFP document and that it would not include a moderation process in its evaluation;77(b) the Mandatory Rules created rights enforceable by the Foundation;78and(c) the Ministry departed, in any material respect, from the process as outlined in the RFP and so did not comply with/breached the Mandatory Rules.7976 He rejected other allegations advanced by the Foundation under this general heading which are not now advanced on appeal.77 HC judgment, above n 1, at [179].78 At [116].79 See [179].[102] The Foundation argued in the High Court the following gave rise to breach of its legitimate expectations and the Mandatory Rules:(a) in evaluating the proposals in the pre-scoring and consensus scoring phases, the Ministry used criteria and weightings differing from those recorded or indicated in the RFP; and(b) the moderation process was not part of the RFP.[103] In the High Court the Foundation's claim was in large part based upon thestructure of the RFP. The RFP is a lengthy document comprised of several parts and appendices. The parts are made up as follows: Part A, introduction and background; Part B, procedure and timetables for the RFP process; Part C, a description of the services the Ministry sought to procure; Part D, a list of the proposal evaluation criteria; and Part E, a description of the format of the information required. Part Dlinks to para 48 in Part A. Paragraph 48 of Part A sets out the "quality and commercial aspects of the evaluation" together with their weightings. In evaluatingthe proposals the Ministry would attach 70 per cent weight to quality considerations and 30 per cent to price considerations. Under those two broad headings there were separately weighted sub-criteria — one sub-criterion under price and seven under quality.[104] The Foundation argued that Part D provided a complete list of evaluation criteria, effectively carrying forward (and replacing) the para 48 criteria and weightings, and binding the Ministry as if a code. Part D sets out various criteria as well as more detailed sub-criteria under those. But the pre-scoring and consensus scoring sheets made clear the Ministry had changed its evaluation criteria from those advised in Part D. This was a breach of the Mandatory Rules, as was the addition of a moderation phase, which was not signalled in the RFP.[105] We consider first the issues in connection with pre-scoring and consensus scoring, and then the findings in respect of the inclusion of a moderation phase.High Court judgment: pre-scoring and consensus scoring[106] The Judge said the RFP process gave rise to legitimate procedural expectations that the terms of the RFP would be complied with by the Ministry.80The Judge said that the criteria and sub-criteria contained in Part D were the criteria the Ministry had represented in the RFP the proposals would be evaluated against and were, in that sense, determinative.81[107] The Judge said the Mandatory Rules had "legal force" because they weremandatory standards and procedural requirements, imposed on all government departments with the authority of Cabinet.82 He was satisfied that a breach of the Mandatory Rules, unless immaterial, would vitiate the contracting decision.[108] The Judge noted the content of the following Mandatory Rules:83(a) Rule 28, which directed the Ministry to put into the RFP all information necessary for proposers to prepare and submit their proposals including the essential requirements and evaluation criteria for evaluation of the proposals.(b) Rule 31, which required that any modification of the essential requirements and evaluation criteria in the RFP had to be published on the GETS (Government Electronic Tenders Service) website or advised to providers in writing.(c) Rule 45, which gave emphasis to the significance of recording in theRFP "the essential requirements and evaluation criteria".[109] In short, he was satisfied that the combined effect of the RFP and the Mandatory Rules was to bind the Ministry to strict compliance with the terms of the RFP. He also considered that the Mandatory Rules placed the onus on the Ministryto ensure that the RFP was clear and rejected the Ministry's argument that any80 HC judgment, above n 1, at [134] and [142].81 At [134].82 At [116].83 At [113].ambiguity or lack of clarity in the RFP was inconsequential because a prospective provider had the opportunity to seek clarification.84[110] He conducted a detailed comparison between the sub-criteria set out in Part D and those against which the proposals were evaluated by the panel.85He concluded that the variations identified established a material breach of the Mandatory Rules and of the Foundation's legitimate expectations.86 In particular, he found that the Ministry had:(a) Added a new sub-criterion — "Financial viability" — to the"Requirements" criterion in Part D. This new sub-criterion represented 25 per cent of the "Requirements" weighting and approximately three per cent of the total weighting.87(b) Added a new sub-criterion — "Viable organisational structure" — to those under the "Delivery" criterion in Part D. This new sub-criterion represented 15 per cent of the "Delivery" criterion and approximately2 per cent of the final score.88(c) Deleted a sub-criterion — "Any proposed subcontractors experience" — from the sub-criteria listed under the "Delivery"criterion in Part D.89(d) Added a new sub-criterion — "Alignment with strategic plan" — tothe "Alignment" criterion in Part D. This new sub-criterion represented 30 per cent of the "Alignment" criterion and three per cent of the final score.9084 At [136].85 At [144]–[179].86 At [185].87 At [144]–[151].88 At [152]–[156].89 At [152]–[156].90 At [160]–[161].(e) Inadequately described the "Outputs and Outcomes" criterion listed in Part D.91 The description provided in Part D was "Performance measures and quality measures proposed". The Foundation said from that it understood the Ministry wanted information relating to external outputs but it became apparent, following discovery, that the panelhad been looking for information about an organisation's internalsystems. Although the Judge accepted Part E included the statement next to outputs and outcomes — "[f]or example, you could describe your internal quality assurance processes" — he said that single sentence could not govern the interpretation of the RFP and in any case fell a long way short of making clear that a criterion called"Outputs and Outcomes" in fact related to internal systems.92(f) Removing the sub-criterion — "Ability to deliver both public health and clinical services in preferred region" — from under the "Purchase Units" criterion in Part D and adding it as a sub-criterion to the"Price" criterion.93 The Judge said that, under the "Purchase Units"criterion, the "Ability to deliver" sub-criterion would have contributed a maximum of three per cent to the total score. By moving it to the"Price" criterion it contributed 7.5 per cent of the total score.[111] None of these changes to the sub-criteria stipulated in Part D were notified to those making proposals. The Judge said the changes were material because they altered the notified weightings and, where differences in scores between providers were small, could determine the outcome.94 They were particularly material because of their cumulative effect. The addition of some criteria was, on the evidence of the Foundation, material because it could have addressed them if notified of the changes.9591 At [166]–[170].92 At [170].93 At [171]–[178].94 At [129].95 At [128].Discussion[112] To found a claim of breach of legitimate expectation, the Foundation had to meet the criteria espoused by this Court in Comptroller of Customs v Terminals (NZ) Ltd:96[125] Where legitimate expectation is raised, the inquiry generally has three steps. The first is to establish the nature of the commitment made by the public authority whether by a promise or settled practice or policy. This is a question of fact to be determined by reference to all the surrounding circumstances. A promise or practice that is ambiguous in nature is unlikely to be treated as giving rise to a legitimate expectation in administrative law terms.[126] The second is to determine whether the plaintiff's reliance on thepromise or practice in question is legitimate. This involves an inquiry as to whether any such reliance was reasonable in the context in which it was given.[127] The third, and often most difficult part of the inquiry, is to decide what remedy, if any, should be provided if a legitimate expectation is established.Breach of legitimate expectation/Mandatory Rules in pre-scoring and consensus scoring[113] The Judge's finding of breach of legitimate expectation, in our view, rested upon the following propositions or conclusions:(a) Part D set out the only evaluation criteria and sub-criteria the panel could take into account. The Ministry could not add to or subtract from those unless it notified those making proposals that it was doing so.(b) The criteria were to be narrowly construed and any ambiguity resolved in favour of the Foundation.(c) The Ministry did add and subtract from the criteria — to the prejudice of the Foundation.96 Comptroller of Customs v Terminals (NZ) Ltd [2012] NZCA 598, [2014] 2 NZLR 137.(d) The evaluation process was a statistical one so alterations to it of any significance were material.[114] The first proposition — that Part D was determinative — rested upon the following:97(a) Paragraph 4 in Part A, which stated "[t]he Ministry will select the preferred Proposal(s) based on its evaluation of the Proposals against the valuation criteria specified in Part D", and which made no reference to Part E.(b) Part E was headed "Format and Information Required" while, in contrast, Part D was explicitly linked to evaluation criteria in that it began "[s]ubject to complying with the Minimum Standards the following criteria will be used when assessing the Proposalsreceived".(c) Paragraph 48 in Part A was linked to Part D in content. All of the criteria and sub-criteria in Part D can be matched to para 48 (although the language does not quite match) but the match between Part E and para 48 was not complete and, where there was a link, it was imprecise.[115] In our view, the Judge's interpretation that Part D is determinative is inconsistent with the RFP. Paragraph 48, not Part D, provides the critical information as to evaluation criteria and as to weightings. The preamble to para 48says "[t]he criteria and weightings for the quality and commercial aspects of theevaluation are set out below". The following chart then appears:97 See the Judge's discussion in HC judgment, above n 1, at [133]–[143].Price / Quality Ratio Criteria WeightingPrice 30% Ability to deliver a full range of requiredservices across a region at a reasonableprice per FTE [full time equivalentemployee]100%Quality 70%Organisational strength and stabilityrequirementsAbility to deliver the required Services12.5%15%Successful experience in delivery ofsimilar services 7%Capability of staff proposed to deliverthe required Services 15%Alignment of Services with the healthand social service sector 10%Performance and Quality measures toensure the quality of Services 7.5%Ability to deliver all clinical and publichealth services required in a region3%[116] The criteria in para 48 are very brief and are clearly intended to take their content and colour, at least in part, from other parts of the RFP. We do not consider that the RFP restricts those relevant parts to Part D.[117] Part D does expand upon the information in para 48, adding sub-criteria to the eight criteria. But it is described in Part A as "[a]n overview of the RFPevaluation criteria", included to "assist Potential providers to understand the relative importance of the price and non-price attributes".[118] It is clear that there are other parts of the RFP which those making proposals had to take into account in formulating their proposal to meet the para 48 criteria beyond Part D. Part E is clearly linked to para 48 because, as described in Part A, Part E sets "out the information to be provided by Potential providers in their Proposals". There is also information elsewhere as to what will be evaluated in the proposals — information such as targeting priority populations, the characteristics of dedicated Pacific, Māori and Asian services, and the need for the successful party to maintain an information system that enables proper performance monitoring and reporting.[119] The Ministry did then clearly represent that it would be using the basic evaluation framework set out in para 48 but, to understand what they were required to address under those eight headings, a party needed to read the entire RFP and not just Part D. We are satisfied that anyone reading this document would understand that, to provide a good proposal, they needed to complete Part E as well as read and closely understand not only Part D but also Parts A and C. For these reasons we consider the Foundation could have had no legitimate expectation that only the Part D sub-criteria would be considered under para 48. The Judge's construction of Part D as the exclusive list of sub-criteria to be considered under para 48 is insupportable.[120] There is another aspect to the Judge's approach that we find problematic, which relates to the second proposition we have set out at [113] above. When assessing how prospective providers would read the RFP the Judge treated the document as if it were a contract to be construed in the case of ambiguity against the Ministry.98 But the RFP is clearly not a contractual document — it is not drafted with the precision of a commercial contract and should not be construed as such. The parties agreed that the RFP did not create contractual rights and obligations. It is moreover a document in which the Ministry explicitly reserved to itself the discretion as to how it would evaluate the proposals against the criteria. And it reserved to itself the ability to accept non-complying proposals and to waive any irregularity in a proposal received.[121] We also are satisfied that none of the particular changes identified by the Judge were, properly construed, changes in notified criteria. It was clear from the overall document that all of the sub-criteria considered by the evaluation panel were to be considered. If matters were not touched upon in Part D, they were highlighted in Part A or in Part C, which stipulated the services required, or were required information in terms of Part E. As to the particular variations identified by the Judge we comment as follows:(a) The addition of "Financial viability" as a sub-criterion to the"Requirements" criterion. If regard is had to the weighted criteria in98 See for example HC judgment, above n 1, at [136] and [166]–[170].para 48, 12.5 per cent is attributed to "organisational strength andstability". It is hard to see how it could be a surprise to the Foundation that, in presenting its information as to its organisational strength and stability, it should have addressed the financial viability of its organisation.(b) The addition of the "Viable organisational structure" sub-criterion to the "Delivery" criterion. It is again implicit in the criteria appearing in para 48 — on this occasion "Ability to deliver the requiredservices" — that a prospective provider would expect to satisfy the Ministry that it had a viable organisational structure.(c) The deletion of the "Any proposed subcontractors" sub-criterion from the "Delivery" criterion. We cannot see how the Foundation could be prejudiced by a failure to evaluate an unweighted Part D sub-criterion.(d) The addition of the "Alignment with strategic plan" sub-criterion to the "Alignment" criterion. The Alignment criterion identified in para 48 reads: "Alignment of Services with the health and socialservice sector". Given the entire history of the RFP, which was, to the knowledge of the Foundation, intended to enable the Ministry to deliver its strategic plan, it is hard to see how the Foundation could have been surprised to learn that it should address, under this heading,its proposal's alignment with the strategic plan. Moreover, question 28 in Part E asks the party submitting the proposal to"Describe how your Services will align at a local, regional and national level with the Preventing and Minimising Gambling Harm Strategy 201/11 to 2015/16".(e) The provision of inadequate information to enable prospective providers to understand what was required in respect to the "Outputs and Outcomes" criterion in Part D. This criterion links to the"Performance and quality measures to ensure the quality of services"criterion in para 48. Part E gives as an example of the informationthat could be provided, "[f]or example, you could describe your internal quality assurance processes". There is moreover a clear emphasis in Part A on the need for systems to monitor and report on outcomes. Against this background the Foundation's surprise that the panel should be evaluating, under this heading, an organisation'sinternal systems is not something which can be laid at the door of the Ministry.(f) The movement of a sub-criterion from the "Purchase Units" criterion to the "Price" criterion. Again the Foundation cannot claim to have been prejudiced by the removal of non-weighted sub-criteria from Part D. As to the addition, the Foundation had notice that the sub-criterion which was added would be taken into account as it was simply shifted from one criterion to another.[122] The Judge's conclusion that these were material departures from the RFP also, as we have noted, entailed the proposition that the RFP set out a statistical scheme for evaluation intended to produce, at the end of it, recommendations for decision. We have already rejected that conclusion. But his analysis has a further defect, as it attributed significance to the addition and removal of unweighted sub-criteria.[123] This takes us to the Judge's finding that the evaluation involved a breach ofthe Mandatory Rules. Given our findings there were no departures from the RFP, including the finding we shortly come to in relation to the moderation process, this issue is not in any sense determinative. If there was no material departure from the RFP, there was no need to notify potential providers of that departure and therefore no breach of the Mandatory Rules.[124] The Judge referred to the following statement by this Court in Chiu v Minister of Immigration, concerning misinterpretation of voluntarily adopted rules:9999 HC judgment, above n 1, at [114] citing Chiu v Minister of Immigration [1994] 2 NZLR 541 (CA) at 550.The administrative law significance of misinterpreting voluntarily adopted rules or guidelines depends upon the context in which the misinterpretationoccurs: see further Baldwin and Houghton, "Circular Arguments: The Status and Legitimacy of Administrative Rules" [1986] Public Law 239 and deSmith's Judicial Review of Administrative Action (4th ed, 1980) pp 317 and 507. In the majority of cases the misinterpretation will vitiate the decision upon the ground that it constitutes an error of law (R v Chief Immigration Officer, Gatwick Airport, ex parte Kharrazi [1980] 1 WLR 1396 (CA);Fitchett (Contractors) Ltd v Secretary of State for the Environment (1988) 56 P & CR 380; Re Preston [1985] AC 835 at p 866 explaining HTV Ltd v Price Commission [1976] ICR 170), produces unreasonableness in the administrative law sense (R v Secretary of State for the Home Department, ex parte Khan [1985] 1 All ER 40, 52 (CA)), frustrates a legitimate expectation (ibid), or causes a delegate to stray beyond the authorised scope of his delegation (discussed, although held not applicable on the facts, inBroadbridge v Stammers (1987) 76 ALR 339, 343)). The present case is not the occasion for extended analysis of the law on this topic. It is sufficient to say that although the result could probably be justified under more than one heading in this case, we would see unreasonableness as a sufficient groundfor vitiating the officer's decision even if it had turned uponmisinterpretation of the manual. Whatever decision might have been proper if based upon other sources, we do not consider that any reasonable decision maker attempting to apply para 14(e) of these guidelines could have arrived at the result reached by the officer.[125] We agree with the Judge that this discussion is helpful and also with his observation that the Mandatory Rules were not voluntarily adopted rules. They were mandatory rules imposed upon Government departments by Cabinet. But we do not see that it follows from this that the Rules had the force of law. They are not legislation or subordinate legislation. A breach of them, even a material breach, does not automatically give rise to an illegality so as to vitiate a decision, as the Judge seems to have assumed. The consequence of a material breach of non-legislative/non-regulatory rules depends upon the nature of the rules and, as was said in Chiu, the circumstances of the breach.100[126] Given that we have found no material departure from the terms of the RFP and that the Rules were not breached, we do not propose to give this issue further consideration. It is artificial to do so in the absence of factual findings which give colour to the claim.100 For further discussion on this point see Robert Baldwin and John Houghton "Circular Arguments: the Status and Legitimacy of Administrative Rules" [1986] PL 239.[127] To conclude on the issue of evaluation criteria at the pre-scoring and consensus-scoring stages, we are satisfied the Judge was wrong to conclude that the Ministry breached the Foundation's legitimate expectation or the Mandatory Rules in the use of the evaluation criteria in the scoring sheets at these stages of the evaluation process.High Court judgment: Moderation phase[128] The Judge next turned to whether the moderation stage was a deviation from the RFP which had to be notified to those submitting proposals in accordance with the requirements of Mandatory Rule 31. The Foundation argued that the moderation process was not indicated by the RFP, that it was contrary to the evaluation process as expressly recorded and that, in consequence, the principles that were applied were both unknown to proposers and in conflict with what was recorded in the RFP.[129] The Judge was satisfied that the moderation stage occurred in breach of the Mandatory Rules and in breach of the Foundation's legitimate expectation as to boththe general nature of the processes and the criteria that would be used for evaluation. He said:[199] Of particular concern was that the Panel utilised principles in this process which were not part of the Part D criteria, and were not recorded until after the Panel process had been completed.[202] My conclusion that the moderation process, and its principles, involved a major shift from the RFP is in fact implicit in some of what Mr Levy said. In particular his statement:Once the panel had agreed consensus scored, it stood back and considered whether the results made sense.What that necessarily means, and it is borne out by what happened in numbers of instances, is that the carefully structured evaluation of proposals up to that point, based not only on all of the criteria and sub-criteria in Part D (or in Part E for that matter), and with the carefully structured weightings, could be ignored.[203] There was strong criticism of the moderation stage, and in particularfrom Dr Jury, in relation to proper evaluation processes and fromMr Mullins from a statistician's perspective. I agree with the criticism.Moderation was a major departure from the RFP.[130] The principles were first reduced to writing by the panel chair, Mr Levy, after completion of the panel's work and in response to a request by PwC as part of the probity review. During the probity review, PwC had observed that there was no documentation of the methods used by the panel to moderate the results and to arrive at the final recommendations as to providers:In particular, nothing had been formally documented to evidence how in seven out of thirteen regions, some of the recommended providers were selected ahead of providers whose proposals had scored higher in the proposal evaluation results.[131] The document Mr Levy produced in response to the request was referred to in the High Court as Appendix 7. The Judge set out the following relevant extracts in his judgment as follows:101SummaryHaving evaluated all the individual regions and proposals, the panel were in a position to understand how regional and national providers reflect the service mix and need. The panel reviewed the outcome comparing the result of the regional and national providers.A moderation process was undertaken to enable the Panel to review the preferred provider(s), first for appropriateness to the region and, secondly whether a national provider was able to provide the same or better service for that region.The following principles were important dynamics that assisted the discussion and balance reached in the final recommendation set out in Appendix 1:Features• Regional providers can range in size but often will have a localised focus.• Some regional providers have significant infrastructure and support networks to sustain a significant local presence and profile.• By comparison, an incumbent national provider should have an acute awareness of the importance of identifying and relating their particular service delivery model in local conditions.101 HC judgment, above n 1, at [189].Regional / local providers• Consideration was placed by the panel in the first instance on the regional provider(s) ability, capability and presence in the region.• For an incumbent provider, the expectation of clear evidence and service delivery should be taken in the context of additional considerations such as the accessibility of the region, the population needs and the ability of the provider to service the region.• • • • • • Where there was no clear preferred regional provider, the panel moved to consider the service delivery by a national provider.[190] The Foundation, together with some other organisations such as the Salvation Army, were not classified as "regional/local providers" but as "national providers". The remaining subsection of appendix 7 relates tonational providers. It is as follows:National providers• For national providers, there is a competitive advantage with efficiencies, size and overall resilience.• The panel considered how the national provider would best serve the local region.• Examples include the ability to connect with the local community, would the national providers' reputationresonate with the local community, could the service be better or superior to a local provider?• For an incumbent provider [that is to say, an incumbent national provider], the Panel agreed that ideally the expectation and standard needed to be significantly higher than the regional provider.• For example, the addition of multiple co-existing problems, advanced referral pathways, facilitation of clients, and local key local connections [sic] were all considered reasonable expectations from a national provider.• For new national providers, the standard needed to be equal or superior to that demonstrated by an incumbent national provider.• Where more than one national provider was delivering in the same region, the panel revisited how that region was specifically described and would support local clients.• While the response template asked for the identification of the regions that would be delivered, the national provider needed to be specific and demonstrate exactly how that region was going to be serviced.• This distinction is important as it recognises that while it is expected that there is national delivery framework [sic], that there are in fact regional differences — which are evidenced by the regional proposals that were assessed.[132] The Judge placed particular emphasis upon the following passage of Mr Levy's evidence:10272. Moderation represented the final step in the Panel's role. What thePanel sought to do was to ensure that its recommendations reflectedmembers' best collective view of the mix of different types ofservice providers that would best serve the needs of their communities and across New Zealand.73. Once the Panel had agreed consensus scores, it stood back and considered whether the results made sense. This process is captured in the Panel Memorandum. As with the Consensus scoring, this steprequired the exercise of the Panel's collective judgement. However,at this step, the Panel looked beyond the terms of the proposals to focus on, as Appendix 7 of the Panel Memorandum sets out, theappropriateness of the providers that scored highest at the Consensus scoring stage for their regions, and whether a national provider was able to provide the same or better service for that region.74. The principles adopted were that where there was no clear preferred regional provider, the Panel considered the service delivery by a national provider. For an incumbent national provider, ideally the expectation and standard needed to be significantly higher than the regional provider. In essence, these principles were intended to assist the Panel to identify the most efficient and effective mix of provider(s) in each region and nationally, taking into account eachregion's demographic and gambling harm profile as outlined in the needs assessment and the indicative clinical and public health FTE numbers and the indicative mix of services for each region set out in Part E of the RFP.75. This was a re-ranking process, in which the scores were not changed, but some of the rankings were shifted. The principles adopted102 At [191].ensured a consistency in approach, in order to achieve sound, robust recommendations that would work in practice.[133] Mr Levy then went on to say:76. In many cases, there were no changes, or no material changes, to the Consensus scoring rankings. For example, in Northland, application of the moderation principles resulted in no change to any of the ranking of the proposals.77. In other regions, there were changes. For example in Waikato, the Salvation Army, a national general service provider, was ranked first at the Consensus scoring step. The Panel noted that there was asignificant Maori population in Waikato and a "high proportion ofMaori clients experiencing gambling harm". The second-ranked provider at the Consensus scoring stage was a local Maori provider which the Panel considered best met the needs of the Maori population, so it changed the ranking. However, bearing in mind the demographics of the region, it recommended that the Ministry also contract the Salvation Army to provide general services. In fact, an error I made transcribing numbers meant that the outcome after thePanel's Moderation actually reflected what should have been the ranking on the Consensus scoring — that is, local Maori provider first and Salvation Army second.78. The needs of regional Maori and Pacific populations were reflectedin the Panel's recommendations in other regions. For example, in the Bay of Plenty, during Moderation the Panel ranked first the Maori provider that had been ranked sixth following the Consensus scoring (but also recommended that the Salvation Army be contracted to provide general services). In Wellington during Moderation the Panel ranked second the Pacific Provider that had been ranked seventh following the Consensus scoring (but also recommended that the Ministry contract the national general service provider and the Maori provider that had been ranked first and second following the Consensus scoring).[134] The Judge concluded that the principles used to establish final rankings at the moderation stage involved a major departure from what was expressly recorded in the RFP:103What that necessarily means, and it is borne out by what happened in numbers of instances, is that the carefully structured evaluation of proposals up to that point, based not only on all of the criteria and sub-criteria in Part D (or in Part E for that matter), and with the carefully structured weightings, could be ignored.103 At [202].[135] The Judge did not expressly identify the principles he said were wrongly taken into account in the moderation phase. On appeal Ms Chen for the Foundation submits that it is implicit in his judgment, and the evidence shows, that the panel took into account two principles not recorded in the RFP and which were inconsistent with it:(a) a preference for regional providers over national providers, which she argues contradicts the "delivery of services" criterion; and(b) a requirement that incumbent providers had to achieve a higher standard than new regional providers.[136] The Solicitor argued that there were no such principles applied during the moderation process and that the Ministry was free to use the moderation process as it was not inconsistent with the RFP.Discussion[137] We agree with the Ministry that there is nothing in the RFP which precludes the use of a moderation process. The Ministry had a discretion as to how it evaluated proposals against the requirements notified in the RFP. The RFP was silent as to the process the Ministry was to follow in that regard. Given the complexity of the undertaking, all prospective providers must have contemplated that a process would be used to bring together the recommendations of the panel members and also to arrive at a mix of regional and national providers across the country. We have already found that the Ministry was under no obligation to use a statistically sound process for this.[138] The next issue is whether the panel did take into account moderation principles as found by the Judge and as argued by the Foundation and, if so, whether those were inconsistent with the RFP.[139] Contrary to the Solicitor's submission, Appendix 7 read together with Mr Levy's evidence does suggest that certain principles were applied at the moderation stage. These seem to have been as follows. If the preferred providerwas not regional, then a national provider would be considered. But when comparing a regional and incumbent national provider in areas of high Māori population of the same ranking, the regional provider would be preferred unless the incumbent national provider could provide services of a significantly higher standard.[140] From the content of Appendix 7, it seems that the latter principle was applied because of the natural advantages identified where regional services are provided by a regional provider, particularly in areas with a high Māori population. In selecting between regional and national providers, the panel was entitled to take into account advantages it believed would accrue to it from the provision of services by locally based and connected providers. It was entitled to bring to bear its collective skill and knowledge to that effect. We see nothing in the RFP that precluded these principles or with which these principles are inconsistent. Parts C and E made clear that the Ministry anticipated contracting with a mix of service providers regionally and nationally. Since it was not inconsistent with the RFP this was not a change that needed to be notified in accordance with the Mandatory Rules.[141] We also note that this is not an argument that assists the Foundation, givenMr Levy's evidence that "there were no regions in which either of the [Foundation's]proposals would have been recommended following the Consensus scoring but notafter Moderation".[142] Accordingly, the Judge erred in finding that the moderation process was inbreach of the Foundation's legitimate expectations and in breach of theMandatory Rules, and therefore void.Result[143] For these reasons, the appeal is allowed. The judgment entered in favour of the Foundation is set aside and judgment is entered in favour of the appellant.Although Woodhouse J had set aside the Ministry's decisions made on or about20 March 2014 as to contract, those decisions are now spent as to effect. There is no point, as we apprehend it, in their reinstatement.[144] In the High Court, the Foundation was awarded costs.104 That costs award is now set aside. Costs in the High Court are to be fixed by that Court in accordance with the outcome of this judgment.[145] The Foundation says that, in the event the appeal is allowed, no costs should be awarded as the outcome is now moot and the appeal was pursued by the Ministry as a matter of principle because of the public interest there is in the standard of review to be applied to Crown contracting decisions. But the Foundation pursued this litigation for its own interests — it was not acting in the public interest. And although in a sense the outcome of this appeal is moot, we accept the Solicitor'sargument that the Ministry had no alternative but to pursue the appeal because of the principles which were arguably established in the High Court judgment.[146] In those circumstances we are satisfied that respondent must pay the appellant costs for a standard appeal on a band A basis and usual disbursements.Solicitors:Crown Law Office, Wellington for AppellantChen Palmer, Auckland for Respondent104 At [343].