Beyond the Questionnaire: A Four-Pillar Reference Model for Continuous Assurance of Public Sector AI Systems

  1. Rajeev Chakraborty

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

Background. UK central government has built a substantial body of pre-deployment AI assurance practice, anchored in departmental assurance questionnaires, the Artificial Intelligence Playbook for the UK Government (Government Digital Service, 2025), the Algorithmic Transparency Recording Standard (ATRS), and instruments such as Data Protection and Equality Impact Assessments. These instruments establish a strong foundation for assessing whether an AI system is fit to enter live service. The next step, building on this strength, is to extend the same structured rigour across the rest of the system lifecycle.Aim. This paper proposes and theoretically grounds a four-pillar reference model for responsible AI (RAI) assurance designed to augment existing UK government instruments across the full lifecycle: Pre-Deployment, Model Activation, Operational Response, and Closed-Loop Learning. The distinctive moves are the treatment of Model Activation as a discrete baseline-setting phase and the specification of Closed-Loop Learning as a cross-government or cross-deployer institutional function, both of which are thinly addressed in existing reference frameworks.Approach. The model is developed using a design science research approach. It is derived through structured comparative document analysis of the four reference frameworks UK departments most directly encounter (the NIST AI Risk Management Framework, the OECD AI Principles, the EU AI Act, and the UK AI Playbook) using the AI system lifecycle as the analytical organising frame. It is theoretically anchored in Value Sensitive Design (Friedman, Kahn and Borning, 2006; Friedman and Hendry, 2019; Umbrello and van de Poel, 2021) and refined through expert-informed practitioner engagement at the Department for Science, Innovation and Technology (DSIT) and a second UK central government department in early 2026.Findings. The comparative analysis shows that the four reference frameworks converge densely at pre-deployment, address operational response with varying degrees of prescriptiveness, are thin on activation as a distinct lifecycle phase across three of the four frameworks, and consistently underspecify closed-loop learning. The four-pillar model addresses the cells where the leading frameworks are weakest, in a form compatible with each.Contribution. The paper contributes a lifecycle reference model for public sector AI assurance, a cell-level mapping of four leading frameworks against that lifecycle, and an institutional fit analysis for UK government adoption. Its distinctive contribution lies in specifying activation as calibration and closed-loop learning as institutional memory. The paper is offered as a basis for cross-government conversation and as the design phase of a research programme whose evaluation phase is described in Section 8.

Article activity feed

  1. PREreview

    This Zenodo record is a permanently preserved version of a PREreview. You can view the complete PREreview at https://prereview.org/reviews/23176872.

    PREreview: Beyond the Questionnaire: A Four-Pillar Reference Model for Continuous Assurance of Public Sector AI Systems

    Reviewed Preprint: Chakraborty, R. (2026). Beyond the Questionnaire: A Four-Pillar Reference Model for Continuous Assurance of Public Sector AI Systems. Preprints.org, DOI: 10.20944/preprints202606.0177.v1.

    Reviewer: Julian Rodriguez, Jr., FRSA, MRes, M.ISRM (ORCID: 0009-0007-9332-0140)

    What This Paper Does Well

    Rajeev Chakraborty tackles a major headache in government AI rollout: the reliance on one-and-done checklists. Right now, departments fill out launch questionnaires, check compliance boxes, and launch their AI tools—but rarely have a structured system to monitor how those algorithms actually perform over time.

    To fix this, Chakraborty compares existing frameworks (NIST, OECD, EU AI Act, and the UK Government AI Playbook) and builds a sensible, Four-Pillar Model for continuous oversight:

    1. Pre-Deployment: Getting the initial risk and compliance baseline right before launch.

    2. Model Activation: Treating the actual turn-on phase as a distinct calibration period.

    3. Operational Response: Monitoring real-time performance and catching errors as the live system runs.

    4. Closed-Loop Learning: Sharing lessons across departments so the rest of the government doesn't repeat the same mistakes.

    His biggest contribution here is treating Model Activation as its own setup phase and making Closed-Loop Learning a shared institutional duty. These two areas are usually brushed past in standard governance policies.

    Key Strengths

    • Exposes the "Checklist Trap": The paper rightly points out that passing a pre-launch test doesn't mean an AI will stay accurate in the real world. Separating launch from ongoing operation forces teams to track errors early.

    • Fixes Government Silos: Making "closed-loop learning" a cross-department requirement stops different teams from making identical, costly errors in isolation.

    • Practical UK Government Fit: The model plugs right into existing tools used by UK agencies like DSIT, making it directly useful for real-world policy work.

    Room for Improvement & Practical Suggestions

    1. Accounting for Administrative Delay and "Systemic Intent Shadows": The model assumes government teams can adjust their tools quickly when errors pop up. In reality, bureaucratic delays create administrative latency. During this gap, a "Systemic Intent Shadow" (Rodriguez, 2025) emerges—the discrepancy between intended algorithmic governance and real-world operational drift. Integrating explicit measures to quantify latency would prevent unmonitored risk from accumulating during administrative lag.

    2. Navigating the "Zero-Defect Paradox" in Public Risk Culture: Public sector managers are often terrified of publicly admitting a system drifted or made a mistake. Under this zero-defect expectation (Rodriguez, 2025), continuous assurance disclosures can unintentionally disincentivize transparent reporting. The author should address how departments can build a psychological safety mechanism so teams feel safe reporting minor operational glitches without fear of political backlash.

    3. Clearer Measurement Metrics: In future testing, it would help to see specific success metrics—like how much faster a team spots model errors compared to using standard yearly questionnaires.

    Final Thoughts

    This paper is a strong, highly practical step forward for public sector tech governance. It moves government AI oversight away from passive paperwork and toward active, real-world accountability.

    Recommendation: Excellent work; ready for publication with minor additions!

    References

    • Chakraborty, R. (2026). Beyond the Questionnaire: A Four-Pillar Reference Model for Continuous Assurance of Public Sector AI Systems. Preprints.org. DOI: 10.20944/preprints202606.0177.v1.

    • Rodriguez, J., Jr. (2025). Systemic Intent Shadow (SIS) Theory and Administrative Latency in Governance Architectures. Zenodo / Preprints.org.

    • Rodriguez, J., Jr. (2025). The Zero-Defect Paradox: Institutional Risk Aversion and Structural Friction in Modern Compliance Systems. SSRN / Zenodo.

    Competing interests

    The author declares that they have no competing interests.

    Use of Artificial Intelligence (AI)

    The author declares that they used generative AI to come up with new ideas for their review.

  2. Version published to 10.20944/preprints202606.0177.v1