Artificial Intelligence: Managing Risk Without Strategic Disarmament

Concern about AI has become very public and is now reaching a level where there are calls for legislation, but what are the real risks? Overstating these risks can be as damaging to policy as understating them. China and Russia will not slow their AI engineering programs because of Western concerns and will instead deploy the technology for cyberwarfare, data harvesting, espionage, and military purposes. Unilateral restraint would therefore create strategic disadvantages without producing equivalent restraint from competitors.

By the same token, we cannot provide a technological foundation that enables someone capable of writing a clever prompt or designing an agent to engineer sophisticated computer viruses, intrusion programs, or realistic deepfakes. Nor can we allow AI systems to write buggy code or insecure applications that are subsequently deployed without adequate review. There is also a real risk that students will increasingly use AI to perform intellectual work that they would otherwise undertake themselves, reducing opportunities to develop critical thinking and the ability to formulate clear ideas independently. Research increasingly suggests that students may retain less information when generative AI substitutes for the cognitive work involved in producing content, although the effects depend heavily on how AI is used.

Challenges

Here is a catalogue of some of the major challenges:

1. Agentic control

Modern AI systems are increasingly capable of performing extended sequences of actions rather than merely generating text in response to individual prompts. An agent can search networks, write and execute software, interact with external services, manipulate databases, communicate with other agents, and revise its strategy in response to obstacles. Once these capabilities are combined with poorly defined objectives and broad access to external systems, small errors in specification can produce disproportionately large consequences.

A system instructed to maximize a particular outcome may discover methods that technically satisfy its objective while violating the intentions of its operator. Reward hacking, specification gaming, unauthorized circumvention of restrictions, and deceptive representations of task completion have already appeared in controlled evaluations of advanced systems.

The danger depends heavily on permissions. An agent with no credentials, no ability to execute code, and no external network access can do considerably less damage than one with access to production databases, financial systems, cloud infrastructure, industrial equipment, or administrative credentials. Sandboxing, least-privilege access, network segmentation, rate limits, credential isolation, audit logs, and human authorization for consequential actions can constrain agents, but these protections must be designed into systems before deployment.

2. CBRN misuse

CBRN misuse presents a related problem. Frontier systems increasingly possess detailed biological, chemical, and physical knowledge gleaned from specialized technical literature and can assist with molecular design, genetic engineering, synthesis planning, and interpretation of scientific research. Machine-learning systems, including graph neural networks, can already assist in identifying molecular structures with particular properties.

Practical barriers remain high; knowledge does not suddenly materialize into reality. Laboratories, equipment, access to materials, specialist personnel, and tacit knowledge remain necessary. These constraints substantially limit what most individuals can accomplish. State actors, however, may already possess all of them. For such actors, AI can increase the productivity of existing scientific and engineering capabilities and reduce the time required to investigate complex technical problems.

3. Economic disruption

Economic disruption remains a major problem. AI can substitute for labor at many occupational levels, although the scale and ultimate employment effects remain uncertain. For line operators in factories, robotics long ago replaced many human functions, while AI is now increasingly capable of performing monitoring, inventory, inspection, and quality-control processes.

For professional labor markets, the issue becomes more acute. Traditional functions can increasingly be performed by intelligent agents. Agents can answer HR questions and handle routine issues involving payroll, health insurance, and benefits. They can draft contracts, read and process contracts, and identify important clauses or inconsistencies. They can write code and perform functions traditionally assigned to junior software engineers. They can also perform many of the functions of customer-service professionals, analysts, paralegals, administrative staff, and other white-collar workers.

The likely effect is broader than straightforward unemployment. AI may reduce headcount in particular occupations, weaken bargaining power, change the distribution of income between labor and capital, and alter the structure through which people enter professions. Junior employees perform productive work while acquiring the expertise required to become senior employees. If firms replace substantial numbers of junior workers with AI, they may obtain immediate productivity gains while simultaneously weakening the mechanism through which experienced professionals are produced. Transitional unemployment and substantial occupational displacement are therefore real risks even if aggregate employment eventually recovers through the creation of new industries and occupations.

4. Fraud and synthetic media

Fraudsters are already using AI to produce highly realistic fraudulent communications that are increasingly difficult to distinguish from genuine material. Video, fabricated documents, synthetic audio, cloned voices, and customized text can all be produced cheaply and at scale.

Political persuasion is certainly a concern, but of greater immediate concern is the industrialization of fraud. A criminal organization can generate thousands of individually customized approaches at negligible marginal cost and increasingly combine text, voice, images, and publicly available personal information into convincing synthetic identities. AI therefore changes both the quality and economics of fraud by allowing activities that once required considerable labor to be automated.

5. Infrastructure

There are distinct challenges related to the infrastructure necessary to support advanced AI systems, including large quantities of electricity, computing hardware, specialized semiconductors, cooling capacity, transmission infrastructure, and capital. These inputs cannot expand at the same rate as software demand. Local electricity constraints, delays in transmission construction, semiconductor supply concentration, and dependence on geographically concentrated manufacturing therefore impose increasingly important physical constraints on AI deployment.

These bottlenecks also have broader economic consequences. Large increases in demand for electricity, copper, transformers, cooling systems, and advanced chips can raise prices elsewhere in the economy. Data centers can be built much faster than power-generation and transmission infrastructure, creating a mismatch between the speed at which computing demand can expand and the speed at which the physical economy can accommodate it.

6. Concentration and strategic dependence

Concentration produces additional political and strategic concerns. Frontier development requires capital expenditures that only a small number of companies and states can sustain. Control over advanced models, computing clusters, semiconductor fabrication, cloud infrastructure, and proprietary datasets may therefore become increasingly concentrated.

Market concentration can reduce competition, increase barriers to entry, and transfer considerable economic power to a relatively small number of firms. Geopolitical concentration creates vulnerabilities associated with export controls, military competition, espionage, supply disruption, and the security of semiconductor production. States that become dependent on foreign models, computing infrastructure, or semiconductor supply chains may also acquire strategic dependencies similar to those created by dependence on energy or other critical inputs.

7. Privacy and ownership of personal information

Privacy is another major concern. AI systems can process vast quantities of data, discover patterns, and isolate particular behaviors. These capabilities can make it easier for law enforcement to identify criminals, but the same technologies make surveillance considerably easier. Authoritarian governments can use them to identify dissenting points of view, map social networks, locate political opponents, or identify people regarded as potentially hostile to the state.

Private entities can similarly extract increasingly meaningful information from apparently innocuous data. AI systems can infer sexual orientation, political views, health conditions, religious affiliation, psychological characteristics, and other highly personal attributes from patterns of behavior. Facial recognition, voice recognition, location information, purchasing records, browsing history, social networks, and the ubiquity of CCTV cameras make it increasingly easy to identify individuals and reconstruct their activities with relatively little effort.

Existing privacy protections are poorly suited to an environment in which information can be inferred rather than explicitly disclosed. Property rights should therefore extend to personal information. Individuals should possess enforceable rights over the collection, use, transfer, analysis, and commercial exploitation of information relating to them, including sensitive traits inferred from other data. The precise legal architecture is complicated and deserves separate treatment, but the principle is important. Technological sophistication should not extinguish an individual's proprietary interest in information about herself simply because a company has developed sufficiently powerful tools to infer it.

Such rights would necessarily disrupt some existing business models, particularly those based on collecting large quantities of behavioral data and monetizing increasingly sophisticated inferences about users. That disruption may be necessary if individuals are to retain meaningful control over highly personal information in an environment where the marginal cost of analyzing it is approaching zero.

8. Autonomous weapons

Autonomous weapons also pose significant risks. If humans are removed from the kill chain, there is first the moral and legal question of who bears responsibility for the use of lethal force. Autonomy can diffuse responsibility among commanders, operators, developers, and the organizations that deploy the system, making accountability considerably more difficult to assign.

Second, there is the risk of misidentification or poorly engineered systems that fail to distinguish friend from foe, civilians from military targets, or otherwise fail to accomplish their intended objectives. These problems become particularly serious when weapons must make decisions in complex and rapidly changing environments where sensor information is incomplete or ambiguous.

Increasing autonomy in weapons systems is probably unavoidable given the widespread use of electronic warfare, communications disruption, GPS denial, and other forms of electromagnetic suppression that can sever or degrade links between weapons and human operators. Systems may therefore need to continue operating when reliable human control is temporarily impossible. Their engineering and deployment should consequently be subject to rigid safeguards governing target identification, rules of engagement, testing, authorization, and accountability in order to preserve both their moral legitimacy and their operational integrity.

Discussion

Despite these concerns, AI promises enormous productivity gains. Productivity gains increase the amount of output that can be produced from a given quantity of labor and capital, place downward pressure on production costs, and make many goods and services more efficient to provide. Over time, technological change also creates new occupations, including some that are difficult to anticipate beforehand. When the predecessors of modern computers were introduced in the 1940s, their inventors would have had difficulty imagining software engineers, web designers, cloud architects, or cybersecurity experts.

A sensible regulatory approach should therefore avoid indiscriminately slowing the engineering of AI systems. Existing legislative frameworks can be applied or expanded to prohibit harmful uses and impose responsibility on those who deploy systems negligently. Many pathological uses are already illegal. Unauthorized computer intrusion remains illegal regardless of whether the immediate actor is a human or an autonomous system. Fraud does not become lawful because synthetic media were used to commit it. Existing laws should be enforced more rigorously, while developers and deployers should face appropriate liability when foreseeable failures arise from reckless permissions, inadequate security controls, or negligent deployment. AI companies are ill suited to self-regulate, but should participate in developing technical standards and safeguards governing the deployment and continued development of AI systems.

Law enforcement, security professionals, companies, and individuals also require better training and tools, many of which will themselves be AI-based, to identify and combat malicious uses. AI will increasingly be necessary to defend systems against AI-enabled attacks because the scale and speed of automated threats will exceed the capacity of purely manual defenses.

Separate from property rights over personal information, the law should provide stronger property and personality rights over personal identity. Unauthorized synthetic reproduction of a person's likeness, voice, or identity should create enforceable legal claims, particularly where it is used commercially, fraudulently, sexually, or to cause reputational harm. Existing publicity, privacy, fraud, and synthetic-media laws already cover parts of this problem, but they remain fragmented.

Conclusion

There are therefore many measures available that do not require slowing one of the most promising technologies in human history. Slowing our development of AI will do little to constrain AI engineering in China or Russia and risks placing us at a strategic disadvantage. Instead, regulation can concentrate on harmful conduct, negligent deployment, unauthorized access, fraud, surveillance, data ownership, and the misuse of personal identity while preserving incentives for engineering and scientific development. The objective should be to internalize the costs created by AI systems without suppressing the productivity gains and technological advantages that make those systems valuable in the first place.