Part 1: What Are Red Lines for AI and Why Are They Important?

This article is the first part of a three-part explainer series on Global Red Lines for AI. You can find the series overview here, part 2 here, and part 3 here.

The idea of a red line is simple. In principle, we know that setting clear and reasonable limits helps keep people safe. Take road safety as an example. Regulators require car manufacturers to include vehicles with reliable, safety-tested brakes, and many vehicles are even equipped with electronic speed limiters, capping how fast they can go even if the engine could go faster. These limits collectively reduce danger and significantly improve everyone’s chances of reaching their destination safely.

So how does this idea translate to artificial intelligence? As AI capabilities rapidly advance, we risk crossing into hazardous territory if we fail to establish guardrails now. Like cars, AI offers remarkable opportunities, but without well-defined limits, it presents serious dangers. Red lines can serve as the “rules of the road” for AI: boundaries that ensure these technologies don’t cause harm and remain aligned with the public interest.

This explainer, part one of a three-part series, dives into what red lines for AI are and why it’s crucial to address them today.

What is a red line?

In basic terms, a red line is a limit that shouldn’t be crossed. We rely on red lines in many areas of life, like setting maximum levels for toxic chemicals to protect workers’ health, or capping pollution to protect the environment. While these red lines don’t eliminate all risks, they define what society agrees is unacceptable.

Some specific examples of international red lines include:

Table 1: Examples of international red lines in other fields

What is a red line for AI?

There is no universally agreed upon definition of “AI red lines” but broad agreement can be seen regarding their key characteristics. The OECD defines AI red lines as a “threshold defined in terms of unacceptable model capabilities regardless of mitigations,” while the World Economic Forum refers to them as “specific boundaries that AI systems must not cross.” 

Red lines for AI
For the purpose of this piece, we define red lines in AI governance as specific, non-negotiable prohibitions on certain AI behaviors or AI uses that are deemed too dangerous, high-risk, or unethical to permit. These boundaries are intended to protect the survival, security, and liberty of humankind.

AI red lines can fall into two categories:
1. AI behaviors: limits on certain behaviors that AI systems should not exhibit (e.g., developers must prove that their systems will not exhibit self-replication or improvement without human control, even if the systems are technically capable of doing so).
2. AI uses: limits on how humans can use the AI system (e.g., a prohibition on using AI to manipulate or surveil children, even if technologies enabling doing so exist)

In practice, different terms are often used to describe what are essentially “red lines” for AI. While these terms vary slightly in meaning and emphasis, they share a common goal: identifying behaviors or uses of AI that are considered too dangerous, unethical, or harmful to be allowed. Common examples include “risk thresholds” which refer to specific levels of risk for which there are specific consequences; “unacceptable risks” which highlight outcomes that are not tolerable under any circumstances; and “prohibitions” which make an explicit ban. Though the language may differ across legal, technical, and policy contexts, these concepts all reflect a growing recognition that some AI behaviors and uses should be off limits.

What are some specific examples of red lines for AI? 

Some uses and behaviors of AI pose such serious risks that they are increasingly seen as non-negotiable. Below are a few examples of AI red lines that many experts and governments around the world are beginning to coalesce around. Part 2 of this series examines concrete policies where these red lines are starting to take shape.

How these red lines are enforced may vary depending on the specific threat. Options include licensing mechanisms that block market access for developers who can’t demonstrate their systems are safe, or penalties for violations.

Table 2: Potential red lines for AI

Who wants red lines for AI? 

Momentum for establishing AI red lines is rapidly building among experts and the public alike.

In March 2024, global AI scientists gathered in Beijing for the International Dialogue on AI Safety (IDAIS), where they jointly called for red lines in AI development to prevent catastrophic and potentially existential risks.

As part of the official process for the AI Action Summit held in Paris in February 2025, The Future Society conducted a global consultation with over 10,000 citizens and more than 200 expert organizations across five continents. One of the top priorities identified by experts was the need to establish “clear and enforceable red lines for advanced AI development.”

Civil society organizations (CSOs) are also pushing this issue to the forefront. In a February 2025 poll conducted by The Future Society, 44 CSOs ranked their highest priority as “establishing legally binding ‘red lines’ prohibiting high-risk or uncontrollable AI systems incompatible with human rights.”

Why are global red lines for AI important?

Establishing global red lines is no longer just desirable; it is necessary. On a global level, we must act to define and prohibit a set of AI behaviors and uses that pose disproportionate, cross-border risks. These red lines can serve as a critical backstop in global governance: setting clear prohibitions on developments we agree must not proceed under any circumstances, regardless of commercial incentives or national interests.

There are three core reasons why such global boundaries matter:

1. AI risks are systemic and transnational.

As AI is increasingly embedded in critical systems, daily life, and decision-making processes, the world is becoming more dependent on these technologies. This growing reliance means that when harm occurs, the consequences can be far-reaching and deeply disruptive. As the International AI Safety Report 2025 emphasizes, “system flaws can have a rapid global impact: When a single general-purpose AI system is widely used across sectors, problems or harmful behaviours can affect many users simultaneously. These impacts can manifest suddenly, such as through model updates or initial release, and can be practically irreversible.” 

Importantly, AI risks do not respect borders. Regardless of where a system is developed or deployed, its impacts can quickly spill over into other countries, similar to financial market shocks that trigger global recessions or the uncontrolled release of an engineered pathogen leading to a worldwide pandemic. These risks are often interrelated: a failure or misuse in one domain can trigger cascading effects in others, amplifying the harm worldwide.

2. The current global landscape rewards speed of AI development over safety. 

Without global coordination on AI red lines, we risk a “race to the bottom,” where countries and companies prioritize speed over safety and the protection of fundamental rights. This dynamic is already playing out with the emergence of AI systems that exhibit agent-like capabilities, including the ability to reason, make plans, and take actions in both digital and physical environments. The pursuit of artificial general intelligence (AGI), driven by a handful of powerful actors with immense commercial and strategic incentives, has only intensified these pressures, often sidelining safety. This isn’t necessarily due to bad intentions; it’s a reflection of the current incentives, which reward rapid progress over caution and disclaim liability. 

In a context shaped by both corporate and geopolitical competition, globally agreed upon red lines for AI could serve as a minimum baseline to define what must not be done. In the absence of a broad, universal definition of safety, red lines offer a clear way to define compliance by marking obviously unsafe and unacceptable behaviors. Even major tech companies recognize this need: for example, in 2023, Microsoft asked for “safety brakes” when risks become unacceptable. Reflecting growing consensus, in 2024 at the Seoul AI Safety Summit, 27 countries and the EU committed to establish shared thresholds of risk beyond which AI labs won’t release their models. 

3. Enforceable boundaries are critical for public trust, so we can actually use AI for good.

Lastly, if advanced AI systems are to truly serve the public interest, people must have confidence that their development is subject to meaningful constraints. Normative principles and voluntary commitments are not effective enough. AI users need informed trust. Binding red lines can signal that certain behaviors and uses are out of bounds. When those limits are clearly defined, globally agreed upon, and enforced, they can help build the legitimacy of AI governance efforts and ensure continued accountability and oversight. Building public trust this way is essential if we want societies to embrace and benefit from AI, rather than fear or resist it.

This all sounds good in theory. But what about in practice?

In Part 2 of this series, we expand on where red lines for AI already exist or are starting to take shape. Drawing from binding treaties and corporate policies, we highlight real-world examples that could serve as a solid foundation for global boundaries against specific AI behaviors and uses.