Forget Sarah Connor and the other heroes of the Terminator series. In the real world, there’s only one team with the power to beat irresponsible AI: Procurement.
Elon Musk recently remarked that “we don’t want a Terminator outcome” when it comes to the trajectory of artificial intelligence. While the comment carries his signature flair for the dramatic, it highlights a very real anxiety regarding the speed of AI development.
As computation power for these systems grows at a staggering 4.2x annual rate, the gap between what these machines can do and our ability to oversee them is widening.
The responsibility to bridge this gap falls to procurement professionals. The function has an opportunity to position itself as the primary gatekeeper of responsible AI, but we have to move fast. The reality of the tech industry is that no one else is going to do it. End users are focused on utility, and vendors are focused on market share.
In other words, procurement is the last line of defense. If there are no buyers for irresponsible AI, the scale of the problem becomes much easier to manage.
The Oversight Vacuum
Public trust in tech self-regulation is at an all-time low. Research from Americans for Responsible Innovation indicates that 82% of the public does not trust tech executives to regulate AI on their own. This skepticism is well-founded. Without external pressure, the default setting for AI deployment is often speed over safety.
ARI recently warned the General Services Administration that existing "any lawful use" clauses in procurement regulations essentially grant AI systems a blank check to operate without meaningful guardrails once they are inside government systems. Relying on these vague legalities is a recipe for disaster, but procurement holds the unique leverage of the contract to demand better.
Lessons From the New Orleans Loophole
A major hurdle for AI governance is the tendency for these tools to bypass formal solicitation entirely. A case study highlighted by Tech Policy Press regarding the New Orleans Police Department illustrates the danger of "shadow" acquisitions. The department used predictive policing software for years without public knowledge.
How did it bypass due process? Because the vendor provided it for free.
Because no money changed hands, the deal sidestepped the city's usual oversight processes. The city did not treat philanthropic gifts like traditional purchases, leaving city officials and council members in the dark about the partnership.
This case shows that procurement must govern the entry of all technology, regardless of the price tag. Whether it is a free trial, a university partnership, or a low-cost tool bought on a government purchasing card, the risks remain the same.
Responsible AI Frameworks
Everyone is navigating this territory for the first time. There is no ancient playbook for AI procurement, but that shouldn’t be an excuse for inaction.
Note that it’s unlikely to be procurement’s job to actually develop this framework. A Chief Technology Officer or a Legal Head might sign off on the principles, but procurement is the only department that can embed those principles into the RFP, the contract, and the supplier scorecard.
A responsible AI framework will generally be built around the following pillars.
Data Privacy
This is the bedrock of trust. Procurement must ensure that any AI vendor has rigorous protocols for how data is collected, stored, and used. The system should protect sensitive information and comply with evolving privacy regulations without exception.
Safety and Reliability
AI systems must perform as intended without causing unintended harm. This requires vetting for technical robustness and ensuring that the models are resilient against manipulation or hallucinations that could lead to faulty decision-making.
Human Oversight
No AI should operate in a total vacuum. A responsible framework mandates that humans remain in the loop, especially for high-stakes decisions. This ensures that there is always a kill switch and a person responsible for the system’s output.
Contextual Adaptability
An AI tool that works for marketing may be dangerous if applied to HR or law enforcement without adjustments. The framework must evaluate whether the AI is fit for its specific purpose and if the vendor can adapt the tool to the organization's unique environment.
User-Centricity
Technology should serve the people using it. Responsible AI is designed with the end user in mind, ensuring the interface is intuitive and that the tool actually solves a problem rather than adding unnecessary complexity.
Bias, Ethics, and Fairness
This is perhaps the most critical pillar for modern procurement. It involves auditing AI for algorithmic bias to ensure it doesn't discriminate based on race, gender, or other protected characteristics. A fair system is one that produces equitable outcomes for everyone it touches.
Auditability, Clarity, and Accountability
You cannot manage what you cannot see. The framework must require "explainable AI", meaning the vendor (or the AI itself) can clearly explain how it (the AI) reached a specific conclusion. Every step of the process must be auditable, with clear lines of accountability for when things go wrong.
Image: An example of a Responsible AI Framework from CAI.io.
Practical Steps for the New Gatekeepers
Three steps are necessary for procurement to evolve into gatekeepers of responsible AI.
- First, procurement must mandate structured safety evaluations. As ARI suggested, any lab building "frontier" models should be required to demonstrate that its systems have been vetted for misuse potential before they qualify for a contract. This turns safety into a prerequisite for profit.
- Second, organizations should move toward centralized oversight models for AI acquisition. Research published at the 2025 ACM Conference on Fairness, Accountability, and Transparency shows that the most successful governance happens when every software acquisition passes through IT staff and AI experts. Decentralized systems, where individual departments or entities manage their own tech portfolios, often lack the specialized capacity to spot algorithmic bias or security flaws.
- Third, contracts must be updated to address the "feature creep" of modern software. Vendors frequently roll out new AI capabilities into existing contracts without notifying the client. Procurement must insist on transparency and re-evaluation whenever a system’s core functionality changes.
Time for Procurement to Guard the Gate
Procurement is where the most consequential decisions about AI are made. As the UK and the EU implement stringent new sanctions and compliance mandates for digital assets, the pressure for the US to secure its own technological infrastructure is mounting.
If we want to ensure that AI serves the public interest, we must look at how it was acquired in the first place. Procurement professionals can step up, asserting control at the point of purchase to ensure values like safety, privacy, and non-discrimination are non-negotiable.
Meanwhile, the gate remains wide open.
Need help gatekeeping your indirect spend? Una members have access to the highest-quality, reliable, and pre-vetted suppliers who are equipped to handle the largest volumes while remaining fully compliant with industry standards.
By streamlining your tactical spend through a foundation of trust and transparency, we free you to concentrate on the bigger challenges…including the future of AI.
