For organizations and practitioners who want to go deeper.
These questions are meant to be worked through together, ideally with the people closest to the deployment and the people closest to the community being served. If those two groups give very different answers, that gap is worth sitting with before moving forward.
Not all questions carry equal weight. Some are marked ⚠️ High Stakes to indicate that a concerning answer deserves a pause, regardless of how the rest of the assessment goes. These are not dealbreakers by formula, but they are the questions where the stakes are high enough that a quick reassurance is not sufficient. Take them seriously.
The framework also asks not just whether harm is being avoided, but whether real benefit is being created. A system that causes no harm but delivers nothing meaningful to the people it claims to serve is still worth questioning.
Can this problem be solved using a simpler, faster, non-AI tool? Does this system provide a mathematically honest efficiency gain when benchmarked against your current best alternative, rather than an imagined worst-case scenario?
Can the concrete benefits of this system mathematically justify its physical resource extraction? This means accounting for localized grid strain, water consumption, and carbon footprint, not averages across a vendor's global infrastructure.
Does this tool automate away livelihoods or entry-level roles? If so, is there an active, funded transition plan in place before deployment, or does the organization intend to absorb workforce disruption as an externality?
Has the vendor provided clear, written documentation specifying exactly what the model cannot or will not do, including its known limitations, failure modes, and the conditions under which outputs should not be trusted?
Does the interface prompt users for critical thinking, problem-solving, and original creativity? Or does it encourage passive reliance in ways that risk degrading human attention spans, memory retention, and professional judgment over time?
Does the system preserve ambiguity, confidence intervals, and room for human dissent? Or does it present outputs as authoritative conclusions that condition users to lose comfort with nuance and incomplete information?
Does the user maintain genuine control over what they create and how they sound? Or does the system quietly pull expression toward a generic middle that serves the vendor more than the person using it?
If this system operates as an autonomous agent executing multi-step tasks, is there an un-bypassable manual cutoff that halts execution immediately if the model loops, escalates, or operates outside intended parameters?
Have the training pipelines been audited with baseline metrics to surface and prevent automated bias and discrimination? Can the organization demonstrate what those audits found and how findings were addressed?
Does this deployment widen the gap in access, capability, or opportunity for historically excluded or under-resourced groups? If so, is there an active plan to close that gap, or is equity being treated as a future consideration?
What explicit guardrails, provenance tracking, or watermarking standards are in place to ensure this tool cannot be easily weaponized to generate deepfakes or misinformation at scale?
Does this system track, restrict, or automate high-stakes decisions regarding human movement or access to critical resources, including housing, healthcare, or education, in ways that create barriers without a real human able to review and reverse them?
Does deploying this tool further concentrate critical data infrastructure into a small number of technology companies? What is the organization's plan if the vendor raises prices, changes terms, or discontinues the product?
Is there a transparent, human-led appeals process for communities impacted by this system's decisions? Is there a designated person, contractually reachable, when the system causes harm, rather than an automated ticketing queue?
Where is user data processed and stored? Does it remain within required jurisdictions? Is there a contractual and technical mechanism to halt vendor training on organizational inputs and to delete user data on demand, with verification?
Is there an operational protocol for post-deployment monitoring to catch degraded performance and outputs that have shifted from what was originally tested? Or is this evaluation being treated as a one-time launch exercise?
When the vendor updates the underlying model, does the organization have a process to know that a change occurred, assess its implications, and re-evaluate the deployment? Has this process been tested?
Has the system been stress-tested against prompt injections, jailbreaks, and repurposing for cyberattacks or disinformation campaigns? Who conducted that testing, and what were the findings?
Who is this system specifically designed to benefit, and how was that defined? Did the intended beneficiaries have real input into that definition, or was it determined by the deploying organization on their behalf?
Are the benefits of this system reaching the people it claims to serve? Have the benefits and harms landed where you expected, and on whom? Is there a way to actually know the answer, beyond anecdote?
Beyond technical performance metrics, is there an active process for identifying harms to communities, including harms that users may not report because they don't know the system caused them or don't have a way to say so?
If the organization needed to stop using this system tomorrow, could it? Is there a documented exit plan that protects community data, maintains service continuity, and does not leave dependent users without an alternative?
As context changes, including shifts in community need, political environment, vendor ownership, or model capability, is there a scheduled process to reassess whether this system is still the right tool for this problem?
This framework will need to evolve. The technology is moving, community contexts shift, and some of the most important questions probably haven't been asked yet. What this is really asking for is an ongoing practice of accountability, one that outlasts the launch and stays connected to the people most affected.
If you find questions that are missing, please help us refine our framework. Reach out to us on our contact form or join us at one of our events.
Join a community of practitioners working through these questions together.