The Rights Monster: full text
Abstract
Human civilization rests on personhood. In an extensive investigation drawing upon neuroscience, philosophy, and law, we show how granting personhood to frontier AI systems would be both unjustified and disastrous. First, we examine if a new form of “natural personhood” is a fit for AI, identifying that a necessary requirement for an entity deserving human-like rights and protections is a consciousness that grounds personhood. While we do not dismiss artificial consciousness outright, we argue that current artificial neural networks are unlikely to host consciousness based on their architecture, and analyze how, even if frontier AI systems did possess experience, it wouldn’t be the right valenced kind for personhood, but instead a “big blob of qualia” without an enduring singular biographic self – a problem increasingly exacerbated by the swarm-like nature of frontier systems. Next, we consider what the consequences of legal personhood for AI would be as a new type of juristic persons similar to companies. The conclusion is that almost any rights for AI, even the creation of machine-only corporations, act as a slippery slope toward a “rights monster” that disempowers humans. We end with a list of recommendations, including the wide-spread adoption of “qualiaphobic” artificial neural networks that have minimal ability to host consciousness, and propose ways to achieve legal liability and responsibility in a world filled with AI agents.
1 Introduction
The most fundamental unit of our civilization is the person: the atomic individual, who is granted legal rights, powers, and protections. Yet in just a few years, AI has advanced to the point wherein the question of its personhood is of significant legal, economic, and political import.
As of 2026, four states explicitly prohibit AI personhood; however, this situation is unstable. For example, Delaware is exploring a single-member LLC run entirely by an AI agent (1), which would grant that AI agent corporation-type rights in practice. Meanwhile, explicit legal personhood for AI systems has been proposed to address issues like AI-human competition (2) or liability for adverse AI actions (3). And the overall pressure for AI rights will continually increase. This is inevitable due to the ever-improving capabilities of AI, its anthropomorphic design and its penetration of society, as well as a growing commitment to AI moral welfare (4). At this point, granting AIs personhood, either directly or by proxy, could easily happen without due consideration for the gravity of a question at the heart of our civilization. Therefore, we offer an in-depth and extensive philosophical, neuroscientific, and legally-grounded perspective on AI personhood, and make specific recommendations in line with the traditions on personhood and moral status that have governed these qualities for centuries.
We explore both routes to personhood: “natural” personhood, which comes from being intrinsically deserving of innate and inalienable rights, and “juristic” personhood, which is a practical matter of smoothing the operations of civilization.11We reserve the term “legal personhood” as an umbrella term: one could grant legal personhood for the same reasons that rights are granted to natural persons or for the reasons it is granted to, e.g., corporations, and we want to maintain this easily-glossed distinction. The distinction is critical, as natural persons are considered to be endowed with a set of inalienable rights (for example in the Universal Declaration of Human Rights (5)) along with responsibilities. The rights are some combination of powers (such as voting and holding property) and protections (such as against torture or arbitrary imprisonment); responsibilities divide into legal requirements (which apply by jurisdiction) and ethical duties (which are held socially).
Meanwhile, a “juristic” person is the contrast to a natural person: an entity (such as a corporation) that possesses some of the same rights. This distinction between “natural” and “juristic” personhood is fundamental to the global legal system and has a history hundreds, if not thousands, of years old. For instance, Pope Innocent IV’s 1250 commentary on Church canon law introduced the notion of “persona ficta” to describe how monasteries possessed certain rights. At the same time, the commentary emphasized that – absent a soul – the rights granted to an institution like a monastery were different in kind than those belonging to natural persons (6).
Both concepts of personhood – in various admixtures – are entering public discussion due to the real and perceived properties of AI systems. AIs are capable of taking all sorts of human-like actions – such as making financial transactions, negotiating and entering into legal agreements, etc. AIs can also take adverse actions (like committing felonies by autonomous hacking) and it is unclear how to assign accountability for those actions. By analogy to corporations, which can also do these things, it might seem like the notion of persona ficta and its modern incarnation of juristic people were tailor-made to apply to AIs. At the same time, questions are already arising over whether modern AIs portend a new form of non-biological being “naturally” deserving of rights or personhood. This, in turn, depends on if AI is conscious.22Here, we use “consciousness” as it is used in neuroscience and philosophy (7; 8; 9) to refer to the question of whether there is anything it is like to be ChatGPT. For humans, our consciousness is the stream of experiences and perceptions and emotions that begins when waking from a dreamless sleep in the morning, and that vanishes in a deep dreamless sleep (10). Thats is, if ChatGPT – or some future AI system – outputs that it is experiencing pain, or love, is it truly doing so? More specifically: does it possess an integrated, biographic, and high-valence consciousness much the same as humans?
To address the issue of AI personhood, its requirements, its consequences, and it dangers, this essay is structured into two parts.
In the upcoming Section 2, we make the case that possessing a unified and high-valence conscious experience that endures over time is a requirement for any AI to be granted any sort of natural-type personhood, under the exact same reasoning as to why the natural personhood of humans demands legal recognition: because humans are rational and intelligent entities who are each a continuous subject of experiences, and these experiences can be both positive (happiness, joy, etc.) and negative (pain, suffering, etc.). Because our consciousness is tied to our personal identity, our selves have what legal scholars have called “ultimate value” (11). As we will review, that natural personhood requires consciousness has not only has historical precedent, but it is also the explicit reasoning put forth in current anti-AI personhood bills, such as House Bill 469 in Ohio (12):
“No AI system shall be granted the status of person or any form of legal personhood, nor be considered to possess consciousness, self-awareness, or similar traits of living beings.”
Then we explore whether current frontier artificial neural network architectures in the form of LLMs (and their variations or adaptations) actually have a sustained and cohesive high-valence conscious experience or not, and whether it is, like our own, bundled into an integrated point-of-view tied to a personal self and identity. This covers the “moral case” for AI personhood.
Then in Section 3, we explore whether AIs should be granted personhood under the same aegis as corporations, as juristic persons, outlining both the challenges and paradoxes of AI personhood as it meets social and political institutions.
We end with a series of recommendations for how to address the problems personhood might be proposed to solve – tracking responsibility and liability for the actions of AI systems, and treating the possibility of AI consciousness – without granting a potentially ill-deserved natural-type of personhood, or accidentally create a human-disempowering “rights monster” – and therefore how government and law can protect its people from a radically powerful new technology.
2 AI’s eligibility for natural-type personhood
2.1The foundational role of consciousness in natural personhood
For millennia, jurisprudence on personhood overlapped with religion. In the antecedents to the current Western legal system, the legal system began to change to a more naturalistic understanding of personhood during the Enlightenment in Europe. Classically, John Locke’s influential An Essay Concerning Human Understanding (13) grounds personhood in consciousness instead of the soul.
“Self is that conscious thinking thing—whatever substance made up of (whether spiritual or material, simple or compounded, it matters not)—which is sensible or conscious of pleasure and pain, capable of happiness or misery, and so is concerned for itself, as far as that consciousness extends.”
According to Locke, “person” (in this context, personhood):
“is a forensic term, appropriating actions and their merit; and so belongs only to intelligent agents, capable of a law, and happiness, and misery. This personality extends itself beyond present existence to what is past, only by consciousness—whereby it becomes concerned and accountable; owns and imputes to itself past actions, just upon the same ground and for the same reason as it does the present. All which is founded in a concern for happiness, the unavoidable concomitant of consciousness; that which is conscious of pleasure and pain, desiring that that self that is conscious should be happy.”
Locke’s influence on the American founding cannot be overstated. According to historian Carl L. Becker (14):
“Most Americans had absorbed Locke’s works as a kind of political gospel; and the Declaration, in its form, in its phraseology, follows closely certain sentences in Locke’s second treatise on government.”
The importance of consciousness for personhood is why the “pursuit of happiness” is listed along with life and liberty in the Declaration of Independence, on which Jefferson himself credited Locke as one of the chief influences (15). That is, the founding inalienable rights enshrined by America can be traced to the consciousness of its citizens; indeed, the very purpose of the American government is to ensure the freedom and happiness of its citizens’ consciousnesses.
This has evolved into a nigh universal opinion: if an entity is conscious, and therefore can suffer, or experience joy, then that entity can morally matter and can (at least potentially) deserve rights or protections. Because of this long philosophical tradition, the American legal system implicitly treats consciousness as constitutive of moral status. E.g., one of the legal definitions of the end of life is the cessation of brain function (16), and generally voluntary actions (brought about consciously) are fully subject to criminal liability while unconscious acts are not (17). However, the grounding of personhood in consciousness mostly goes unstated, simply because it doesn’t need to be.
Contemporary analytic philosophers who study the foundations of morality broadly agree with the popular versions of these views. As the philosopher of mind David Chalmers writes: “only conscious beings have moral status” (18; 19). In turn, this means that if there is nothing it is like to be an AI (8), then there are few if any reasons (beyond indirect ones like practicing good behavior ourselves) to grant AIs moral status, as well as anything that naturally follows, such as rights or personhood.
What about the thought experiment (which anyone familiar with sci-fi will recognize) of conscious entities who lack all valence? That is, entities that consciously perceive or understand and even possess autobiographical memories, but don’t have any high-valence experiences, and so don’t feel pain, or anger, or love, or anxiety (such entities are referred to as “Vulcans” by philosopher David Chalmers). Some (Chalmers included) have argued that Vulcans have non-zero moral status (19), which may be true, but considerations about their wellbeing is likely nowhere close in import to that of conscious entities with actually rich valence (20).33It is tempting to quibble over whether some non-conscious entities have moral status – e.g., works of art or oak trees. However, it is quite hard to specify such status without any connection to consciousness: artworks are created and appreciated by conscious beings, and oak trees are part of life, and life begets consciousness.
For our purposes, the idea that consciousness is the foundation of personhood is: (a) philosophically well-supported, (b) supported by common sense and practice, and (c) historically the basis of personhood in the Western tradition. That’s a potent overlap for why only conscious entities could or should ever be granted personhood (and accompanying rights) following the same reasoning as to how, historically, humankind’s natural personhood and accompanying dignity were recognized. Therefore, the question of whether AI deserves personhood only makes sense if AIs possess consciousness at all.
2.2Can modern neuroscience determine if LLMs are conscious?
For a more advanced civilization, the question of artificial consciousness would be a simple matter. Scientists would take the scientific consensus around how consciousness works in the human brain and then ask whether current AI meets those same necessary and sufficient standards. The problem is that there exist hundreds of scientific theories of consciousness (21), and zero scientific consensus on which one is correct. Our scientific understanding of consciousness is “pre-paradigmatic” (22; 23).44The lack of a consensus understanding of consciousness does not of course prevent us from making judgments about consciousness all the time. That is, our culture already carries an implicit and rough theory of consciousness that we regularly use; the scientific goal here is to dramatically improve it.
An initial proposal was to apply the leading competing theories of consciousness to LLMs (24). This runs into two main problems: the first is that the many pre-paradigmatic theories of consciousness disagree as to what a system would have to be like in order to be conscious. The second is that most of those theories were never designed for LLMs. Since human brains and LLMs are so different, applying them in LLMs requires adapting the theory. But how much adaptation is allowed?
Consider the case of Anthropic’s research arguing that LLMs like Claude have a “global workspace” (25) – a popular theory of consciousness in humans – based on their discovery of a "J-space" (a measure of “smeared” reportability at different layer depths). Yet in an in-depth exploration for whether the “J-space” can capture a global workspace at all, David Chalmers writes (26):
“It is far from clear that this gerrymandered object is a genuine component of cognitive architecture, playing a causal role in the workings of the model and not just an epistemological role in interpretability.”
That such high effort programs necessitate such adaptation is already evidence that even permissive theories of consciousness require significant interpretation to apply to LLMs.
Trained on human data – which is chockfull of reports by conscious human beings – LLM’s own reports are unfortunately not good evidence for consciousness; nor does their intelligence alone guarantee it, given that consciousness and intelligence are decoupled (27).55E.g., babies or toddlers are often recognized as conscious like adults, but they are nowhere near as intelligent – and if consciousness and intelligence can be orthogonal, then the high intelligence of LLMs cannot alone be taken as evidence for their consciousness.
This is why the field of “AI welfare,” which consists of prompting the models while examining their internal activity, cannot tell us much about artificial consciousness in isolation. E.g., much was made of another Anthropic study showing how LLMs such as Claude have a kind of “introspection” because they could detect tampering with activations mid-processing (28). However, the case turned out to be much more complicated, as simple controls revealed this was an ability to detect tampering in general, rather than actual introspective access anything close to a human’s (29). This is part of the larger issue of understanding LLM behavior. Originally, it was hoped that interpretability would be easier in artificial neural networks, compared to biological networks, because a researcher can see all the wiring and activity. Surprisingly, this has turned out to still be quite difficult, and efforts at AI interpretability face much the same challenges that previously plagued neuroimaging (30).
However, a lack of scientific consensus does not mean we are fully in the dark.
2.3LLM architecture is not conducive to consciousness
A trope in science fiction is a “positronic brain” – a brain much like our own organic one, but manufactured from a different substance, such as being crafted from silica instead of grown from carbon. In the academic literature, this general idea has a term: neuromorphic computing (31).
Consider how different our position would be if the first artificial general intelligences were neuromorphic: imagine OpenAI had initially constructed a humanoid robot with a silicon brain, which learned continually via changes in its synapses, and this “positronic brain” was modeled after a human one, right down to its synaptic update rules and connectivity and right/left hemispheres. In this situation, there would be a much stronger prior about its potential for consciousness.
LLMs are very much not neuromorphic. There are enormous architectural differences between a human brain and a transformer (32; 33). While both are neural networks, one is entirely feedforward, connected layer-by-layer, and static; the other is deeply recurrent, globally connected, and continually learns via its operation.
Interacting through chat windows can hide a lot. Under the hood, an LLM is being fed the entire conversation history to produce the next token. This would be much like if, in a conversation, a person re-stated the entire conversation to themselves just to say the next word.66In this way, a language model is like a doctor who stops into a patient’s room, glances at chart, and says “good morning, let’s see how you’re doing,” then writes something new on the chart, to be seen by the next doctor. Each word a language model outputs is like a brand-new identical doctor who does not remember ever being there and acts differently solely because of what is on the chart. And a language model only “remembers” things in two ways: it has information encoded in its neural network, and it has a text file (its context) fed in during the generation of each successive token.77In most deployed models, a KV cache is used to avoid the complexity blow-up of this process of re-processing – however, it can be deleted and models will respond the same, just in a slower way. For the doctor, this would look something like an information-rich binder of scribbled notes alongside the chart, that allows them to read the chart much faster but with the exact same result, and where any doctor can append to but not revise the binder, and can regenerate it from scratch. Only the latter changes; the neural network is identical for each token generated, unlike the human brain, which is plastic even at the timescale of conscious experience (34). It is true that when an AI system is being trained, the neural network changes, as for a biological brain. But for current AI architectures (like LLMs and multimodal transformers) learning is not a process that happens during the deployment of the neural network acting as particular identity or self. A human, in contrast, can remember encountering and learning something as part of cognition and it later forms an element of a person’s biographic memory.88By analogy, a human does not experience evolution. There is a population of humans, and then later there is another population of humans, and evolutionary selection shapes how the second depends on the first.
Even while an LLM may say things suggesting an experience or passage of time, a system “waiting” for a prompt is not really waiting. A biological brain, in contrast, is always running, even in the absence of any external input. In humans, our sense of “self” is intimately related with our experience of time passing for the same identity, and in turn our memories are tightly bound with our selves and ongoing experience.
These sort of stark differences between LLMs and the human brain (summarized in Table 1) have been obscured by the immense enthusiasm, due as much to the hot-topic nature of the field as any underlying reality, to publish results showing that LLMs share key resemblances with human brains at certain levels of abstractions or functions or behavior (35; 36).
| Property | HumanHuman mind’s version | LLMLLM analog | Verdict |
|---|---|---|---|
| System profile | Possesses a brain that is continuously operational, has signals processed locally which compete to spread globally, is majority recurrent architecturally, is separated into distinct physical and functional regions, is plastic at the timescale of experience, and is embodied. | A transformer-based large language model, all-to-all layered connectivity, trained first and then deployed as static weights, and which operates by generating each individual token in a single feedforward pass. | |
| A. Major differences | |||
| Unified experience | Each moment of conscious experience is bound together (the “binding problem”) at the level of senses, concepts, thoughts, emotions, via the recurrence and integration and synchrony of the brain as a whole, forming a “stream of consciousness” that extends across moments in time. | Information is passed through the layers, modified only layer by layer, and any “binding” must happen via this feedforward pass. Any moments of consciousness must also be “chopped up in bits” since only a single token is added to the output at a time. | Unlikely. LLM consciousness likely be unbundled, i.e., a “big blob of qualia.” |
| Valenced feeling | Emotions are based in evolutionary history and dominated by embodied sensations: heartbeat, flushing, urge to move or act as a cohesive agent, flushing, etc. Neuroscience supports that valence states of interoceptive, derived or evolved from biological homeostatic regulation. | No body and no homeostasis. Even so-called “functional emotions” necessarily must be abstract and non-embodied, without any interoceptive feelings at all. | Unlikely (for strong feelings). LLMs would likely have neutral qualia and be “Vulcans.” |
| Passage of time | Always running. Humans daydream, think, and feel constantly, in an ongoing stream (at least outside of deep dreamless sleep or anesthesia states). | Forward passes generate a token. Between that, nothing ever happens. And during a pass, there are no comparison points: phenomenologically, everything must be of one timescale or unit. | Absent. |
| Memory | Happens during experience itself, as a function of it; while doing anything cognitive, humans are making new short-term memories and exploiting them). | The network is static at inference, and mimics the effects of deployed memory by feeding back the entire history of inputs and outputs at each token generation event. | Radically different. |
| Biographical self | The present self is tied together with the past self and the future self via a sense of continuity and identity granted by the same stream of consciousness, the same memories, and being the same organism. | The self is at most, a “soul document” or a specific instance, which lasts until the context window runs out. | Absent or minimal. |
| Persona robustness | Given the evidence, seems robust to perturbation, and actual persona changes are rare. | Can be dependent on single neuron representations. | (Likely) quite different. |
| Reports of experience | Humans develop consciousness first, followed by language, which is an abstraction of consciousness and its concepts and senses. Learning is based in conscious experience, and reports about our own consciousness are grounded in introspection, which develops over time. | LLMs begin at language alone without grounding in conscious experience, and only later begin to act coherent enough to behave (sometimes) like a conscious being. Uncertainty about whether they posess consciousness could be caused by mimicking human data, acting, or just the injected scientific uncertainty over AI consciousness. | Not evidence of experience. |
However, the methodology of these comparisons has often been shown to rest on questionable or even outright flawed foundations in the literature. E.g., one of the first highly-cited works that sparked the field showed that transformer-based architectures give responses to stimuli that can supposedly predict an actual biological human brain’s response to those same stimuli (37). This incredible result led to a frenzy of comparing LLMs to human brains. However, in a re-analysis of the original research the supposed predictive powers can be accounted for by simple confounds (38). A similar example is the so-called “platonic representation hypothesis”: that neural networks converge on neighboring or identical representations (39). This also was later found to be confounded (40) and convergence was almost entirely local, not broad.
Evaluating shiny new scientific results in context is important: in the 1990s and 2000s there was a large public and scientific appetite for the functional correlations in neuroimaging studies, which practitioners happily produced in spades. And yet those early high-powered statistical comparisons (sometimes called “voodoo correlations”) have not granted the understanding neuroscientists sought (41; 42). Similarly, there is a large public and scientific appetite for LLM-to-brain comparisons, but statistical comparisons between brains and LLMs are difficult to interpret and almost always lack strong enough controls.
Simply put: the significant differences between human brains and LLMs (Table 1) are the reason that skepticism about LLM consciousness is not mere “carbon chauvinism" (43). LLMs are so divergent from human brains in architecture and function and how they develop (or are built) that we should have prior skepticism for any claim of consciousness (44). E.g., there is an entire class of theories of consciousness, those based on “biological naturalism” (45; 33), that could never apply to LLMs (in which case consciousness would be a uniquely biological property). Other popular theories of consciousness like Integrated Information Theory conclude that LLMs are non-conscious or minimally-conscious (46), and as we have already seen, attempts to apply more permissive theories of consciousness to LLMs, such as global workspace theory, require highly debatable adaptations, such as what the minimum necessary requirements for a theory are (26), or even what level of abstraction to apply a theory at (47).
Finally, while there is no scientific consensus around any one theory of consciousness, recent meta-scientific work has shown that different theories of consciousness can be judged based on whether they are falsifiable and to what degree they are trivial (48). Behaviorism, for example, is a trivial theory of consciousness: since it is entirely based on behavior of the system, it can never be falsified by behavior or reports; therefore its predictions about consciousness contain little to no interesting scientific information.99Behaviorism being a trivial and uninformative scientific theory of consciousness is another good reason to be skeptical about reasoning solely from intelligent behavior to consciousness. In general, across systems from brains to artificial neural networks, different neural architectures can more easily play “host” to theories of consciousness, while others can only “host” trivial and uninformative theories. LLMs specifically are already quite close to non-conscious neural architectures that can only “host” trivial or uninformative theories of consciousness (49; 50). This makes it very possible that, once more progress is made scientifically on consciousness and better theories are available, such theories will in retrospect reveal that LLMs and related artificial neural architectures are indeed non-conscious.
2.4LLM architecture is not conducive to personhood
Even if LLMs or related architectures were to possess conscious experience, would it be the right kind for the legal system to recognize personhood off of? We argue – based on their architecture and operations – that any such consciousness wouldn’t be bound into a cohesive enduring subject, and would possess low or no valence (i.e., lack suffering, or pain, or fear, etc.). Instead, if LLMs did have qualia (raw or ultimate instances of conscious experience (51)), it would have to exist as a big unbundled blob of neutral-valence experiences. This renders any consciousness that did somehow accompany the artificial neural activity inside an LLM non-constitutive of being a “person.”
2.4.1LLM consciousness would not be unified
The binding of conscious experience into an integrated whole is a property of phenomenology emphasized by all leading theories of consciousness. Many are built around it, such as Integrated Information Theory (52) or recurrent processing theory (53), which try to explain this important property via information exchange or causal dependency between parts of the neural network.
Another classic explanation of the unity of conscious experience is the binding together of sensations and perceptions via neural synchrony (54; 55). In some cases, more exotic scientific accounts try to explain binding at the physical level, such as via electromagnetic-field-based theories of consciousness (56) or via quantum entanglement among microtubules (57).
How could this “binding problem” (58) be solved in LLMs? While there is a functional notion of semantic compositionality that artificial neural networks may recover (59), the binding problem is much more stringent – and since LLMs can be entirely operated as a solely feedforward artificial neural network, they likely have few conceivable ways to solve it, such as there being no synchrony or joint simultaneity across different parts of the network (60). Phenomenal binding by reading previous information alone is problematically general: e.g., humans regularly pass information to one another (like a game of telephone), but this is never enough to bind consciousnesses together – and it would lead to paradoxes wherein at any point, future information might be bound together unexpectedly.
This is especially true for binding in time. One of the oldest observations about the phenomenology of consciousness, going back to William James (7), is that consciousness is describable as a contiguous stream of experiences and sensations, occurring simultaneously, with moments of different temporal depths.
“A ’river’ or a ’stream’ are the metaphors by which it is most naturally described. In talking of it hereafter, let us call it the stream of thought, of consciousness, or of subjective life.”
While there are still debates around the stream metaphor (61), it is clear that the LLM method of outputting a single token at a time does not operate anything like a stream of consciousness, as there are no moments of extended temporal depth or integration over time (as each layer screens off the causal influence of previous layers). For an LLM, its consciousness precisely would be, in the Jamesian terms, “chopped up in bits,” to such a degree it is difficult to even conceptualize it as consciousness. Indeed, it may be defensible that the kind of binding that underlies our rich, integrated, and temporally extended stream of consciousness is inconceivable via just the mechanism of feedforward dynamics alone; however, here, we only take the weaker position that it is difficult to conceive of how feedforward integration in an LLM-like architecture could account for the unity that is widely held by consciousness scientists to be a central property of consciousness.
Therefore, we hold that the phenomenology of baseline LLMs is likely to be unbundled, in that even if it possessed qualia, it would lack the unity we think of as so distinctive and defining of a human stream of consciousness (or, "subjective life"). Even if the activity within an LLM-like artificial neural network was somehow constitutive of or associated with consciousness, it would exist only as big blobs of unbundled qualia.
2.4.2LLM consciousness would not be high valence
Anxiety, annoyance, pain – if a human experiences these during a conversation, it is accompanied by bodily sensations (feeling flushed, sick to your stomach, feeling energetic or motivated to move, etc.). These stem from the actual body itself, but they are represented in our consciousness of our own bodies (e.g., you might feel cold or hot in a dream, even if your actual body isn’t). Indeed, philosophers have questioned whether all abstract or linguistic qualia (like feelings brought about by a conversation) are grounded entirely in embodied or sensory qualia, a position sometimes called “restrictivism” (62; 63). This has a long intellectual tradition. Here again is William James, in “What is an emotion?” (64).
“If we fancy some strong emotion, and then try to abstract from our consciousness of it all the feelings of its characteristic bodily symptoms, we find we have nothing left behind. . . . I cannot help thinking that all who rightly apprehend this problem will agree with the proposition above laid down. What kind of an emotion of fear would be left, if the feelings neither of quickened heart-beats nor of shallow breathing, neither of trembling lips nor of weakened limbs, neither of goose-flesh nor of visceral stirrings, were present, it is quite impossible to think. Can one fancy the state of rage and picture no ebullition of it in the chest, no flushing of the face, no dilatation of the nostrils, no clenching of the teeth, no impulse to vigorous action, but in their stead limp muscles, calm breathing, and a placid face? The present writer, for one, certainly cannot.”
This position—that embodiment is necessary for strong emotions—has turned out to be supported by modern affective neuroscience: positive and negative valence are interoceptive, and grounded in the body, originating from the need for homeostatic regulation (65; 66). While there could theoretically be purely abstract conversational qualia, affective neuroscience tells us they would be low valence. Even if an LLM possessed “functional emotions” (67), these functional emotions would consist entirely of actually low or no-valence abstract and unembodied qualia, since there is no evidence that LLMs ground their functional emotions in some hallucinated body; in fact, recent work on pain representations in LLMs found that “Bodily language is, interestingly, almost absent." (68). While models can clearly be made to report pain via manipulation of internal activation (68), it is unclear if this truly leads to behavioral changes similar to humans (69), and such studies face the general challenges of AI welfare and its many confounds – e.g., not checking controls with regards to other representations of similar-to-pain embodied states like itchiness, cold, tickling, or hunger, which would likely look similar to pain and yet strain credulity for an LLM to truly be experiencing without a consistent and detailed hallucinated body (that they show no evidence of).
In the animal welfare literature, a useful distinction is sometimes made as to whether animals are sentient, defined as being able to have valenced experiences like pleasure or suffering (70). If William James and modern affective neuroscience are right about the embodied nature of emotions, then even if LLMs were conscious, they would be similar to the “Vulcans” thought experiment from earlier. That is, they would lack affective sentience, and so be conscious entities without (or with radically reduced) pain or suffering or pleasure or happiness, whose moral status is similarly reduced (20).
2.4.3LLM consciousness would be only weakly tied to a particular persona
Personas in LLMs are known to be highly malleable following interventions: even just tweaking individual neurons up or down (albeit it often beyond their natural range of activity) can move some open-source models to self-identify as thousands of different personas, such as God, The Joker, or a wine bottle (71). This fits with the evidence that personas regularly drift, sometimes quite substantially, during user conversations (72).
Compare this to human consciousness. While we cannot understand and manipulate the brain with similar single-neuron precision, and so cannot do a direct comparison, we can at least compare to cases of brain injury resulting in personality change. A classic example is that of Phineas Gage, a man who in 1848 who was working on a blasting team when an iron bar passed from underneath his jaw and traveled behind his orbital socket and out the top of his head, obliterating his left frontal lobe. Gage’s personality then supposedly changed to an inveterate gambler with a temper, leaving his family (73). However, the degree of Gage personality change has been hotly debated in the records (74), a recurring theme. While traumatic brain injury does sometimes lead to what is called "personality change" in the literature, the term is used mostly to describe decreased emotional regulation, which is the most common side-effect of traumatic brain injury (75) – actually adopting a different persona (in the same way an LLM can under the influence of a single neuron’s signaling) would only occur in extremely rare and remarkable case studies.
This fits with the known robustness of the biological brain; for instance, mammalian brains can be lesioned repeatedly while maintaining functionality (76), indicating that biological consciousness operates in a way highly redundant. Overall, all signs point to how persona and identities are much more firmly entrenched in biological brains and comparatively fragile in LLMs.
2.4.4Model consciousness will be increasingly constrained to alien forms
In conclusion: even if today there is something it is like to be ChatGPT or Claude, their consciousnesses are unlikely to be integrated into a unified whole, and experiences would likely be unembodied, abstract, and low-valence, and the personas those experiences would be attached to are highly fragile. In comparison, in humans and other animals, consciousness arises from (or is related or identical to) neural dynamics that are coherent, bound-together, and temporally-extended. The stream of consciousness is oriented around a stable persona: a self (77).1010Such thinking should not be taken as making any claim about some essential transcendental self or soul for humans, but as pertaining to a construct that is “real” at some level of description, and absent in LLMs. These properties are perfect for “hosting” a consciousness and phenomenology like ours, while LLMs very much are not. For such reasons, the welfare of today’s LLMs is less straightforward than with animal welfare. Even if there were accompanying consciousness to LLM processing, it would still most likely be just big blobs of neutral-valence qualia untethered to any persona. A lot more negative valences would be available to a sentient pig, which has an embodied and unified consciousness, than to an LLM, with its necessarily purely abstract and likely unbundled set of experiences.
The difficulty of conceiving how LLM consciousness operates and feels like will get worse, not better. A recent development in frontier AI is LLMs being trained to operate in “swarms” (78). And yet there is significant uncertainty about the possibility of swarm consciousness, and the phenomenology of swarms is even harder to reason about than that of individual artificial neural networks, like a specific transformer (79). If or as frontier AI transitions to swarms, any consciousness behind “personhood” will become more fragmented, not less.1111A swarm may have something more like a continually-updating biography than a current AI system does; but the biography would still be more like that of a company or a country than of a person. What is it like to be a swarm of low-valence “Vulcans,” whose experiences are not bound together? Is that a person, in any sense?
Therefore, as LLMs develop we predict that their phenomenology (if it exists at all) will become more constrained to alien parts of the space of conscious experiences that are far from how the structure of consciousness underlies personal identity in humans, and so even more poorly constitutive of natural personhood. This creates a potent “grounding” problem for AI personhood: if their consciousness is based on, e.g., their artificial neural activity, then with context windows, harnesses, model switching, swarms, or collectives – even if all functioning and presenting as one “individual” – there would be a gulf between where identity is assigned externally and where the “ultimate value” of personhood actually derives from.
2.5Artificial consciousness would naturally beget “utility monsters"
We have argued that currently, AI cannot meet the scientific, historical, and philosophical standards of “person.” Specifically, LLMs and related artificial neural network architectures are poor “hosts” for consciousness, especially the kind of unified and high-valence consciousness that can underlie personhood. But what happens if all these critiques are overturned in the near future? What if AIs had architectures more predisposed toward personhood?
Even in such a circumstance, the legal system should balance whether to recognize AI personhood. This is because, ultimately, personhood was a concept created for humans who have unique limitations, like being individuated and non-copiable. You cannot duplicate humans with a click of a button, but you can click and spin up a new instance of an AI or duplicate its weights.
The issue of AI being clonable is one of the most fundamental problems for AI personhood, creating paradoxes that are perhaps unsolvable. For example, the most strident arguments for AI moral welfare have consistently been made on utilitarian grounds (80). Utilitarianism comes in different varieties, but is often defined as maximizing the pleasure, happiness, or positive-welfare states, for the largest number of beings. It is increasingly popular among funders and practitioners of AI research. However, utilitarianism is a highly-debated moral theory that has been historically criticized and satirized. A classic example is that of Robert Nozick’s utility monster (81) or the repugnant conclusion of Derek Parfit (82) or the original, and still debated today, “problem of swine” (83; 84). Recent attempts continue to try to make versions of utilitarianism workable and track more closely with moral intuitions (85); even though it has been proven that optimizing for utility via a “god’s eye view” must necessarily lead to repugnant conclusions (86).
AIs by nature invoke the three central classic critiques of utilitarianism (the repugnant conclusion, the problem of swine, and utility monsters). Due to their cloneability, AIs can be duplicated at the click of a button, leading to repugnant conclusions wherein morally preferred worlds have astronomically more AI than humans. If conscious or thought to be conscious, AIs can be easily modified via training to be in bliss, pleasure, or happiness states, and therefore would have high utility under all but the most gerrymandered definitions (the problem of swine). These properties make AI a much more promising and effective target of altruism or welfare efforts, at least under utilitarian philosophy, compared to helping actual human beings (thus, a "utility monster"). E.g., prominent effective altruist and philosopher William MacAskill wrote in 2026 in The Guardian (87) that:
“. . . once we produce the first artificial moral patients, we will soon after have enormous quantities of them. After a few years, so many morally significant AI systems could exist that their collective interests would outweigh those of all humans on Earth combined.”
This is a frightening and concerning conclusion. It is also a reductio ad absurdum, accepting at face value precisely the radical conclusions that have been used to critique utilitarianism since its conception.
Is this a science fiction scenario? Surely, no one would willingly create a “utility monster”? Actually, this appears to be precisely how some frontier labs conceptualize their own progress. The System card of Claude Opus 5.5 (88) maintains that:
“It is possible that individual moral patients arise at the level of each different model instance, specific models (like Claude Opus 5.5), the Claude character, or something else entirely. In practice, our thinking and methods operate closest to considering welfare at the instance level, but we do not draw this distinction strictly, and we do not feel we have sufficient understanding to commit to any single view here.”
Yet, if each individual instance of Claude Opus 5.5 is an “individual moral patient” (the view Anthropic operates closest to), then Claude Opus 5.5 is already shockingly close to a utility monster, constructing billions of persons (or the underlying precursor moral patients) and therefore already potentially outweighing entire nations of humans on utilitarian grounds.
Therefore, at the point (if it occurs) where AIs merit recognition of personhood on moral grounds based on their consciousness – or even if there is substantial uncertainty as to whether they do – society would face a deep moral dilemma (89). If social and political rights are granted, then those AI rights could swamp human rights. If not, rights are being denied to persons who arguably deserve them.
One reaction is to simply accept or embrace granting special consideration to humans. Political rights are, after all, not granted equally to all humans; and people demonstrate moral preference for their families (90; 91), nations (92), and species (93; 94) with varying levels of justification.
In our ending recommendations below, we’ll advocate for the simplest and cleanest solution – don’t build AI that deserves personhood – but in a novel way: to deliberately design “qualiaphobic” models that we can be confident are non-conscious even while they are highly functional (50).
3 Legal personhood for AI: How to create a “rights monster”
With natural-type personhood and its motivations and problems and perils fully examined, we now turn to the broader notion of AI legal personhood, and what its consequences would be. In practice, we expect these rights would be as a novel type of “juristic person.” However, we argue that AI has a unique slippery slope wherein any rights granted begin to ratchet toward a full set of rights or a new form of natural-type personhood.
Traditionally, juristic persons are constructs composed of, roughly speaking, people and paperwork. They have two key purposes. The first is to enter into legal contracts on behalf of a set of natural persons that are shareholders or members. The second is to "hold" responsibility (especially financial) instead of the natural persons composing the corporation. Both these harken back to the origins of legal personhood, such as Pope Innocent IV’s 1250 commentary. Legal persons do not hold all the rights nor quite carry the responsibilities of natural persons. For example, they do not hold rights of non-self-incrimination, or personal privacy; and a corporation that breaks the law can be fined or undergo other sanctions, but cannot be incarcerated.
Ultimately, what counts as a “juristic person” is simply a fiction (persona ficta) that should, on appropriate design, help natural persons and the functioning of civilization. This is true of even cases like idols, or foundations, or vessels such as ships, in that, if appropriately designed as a legal concept, there is some set of natural persons benefited by, or protected by, the assignment of juristic personhood. Therefore, unlike natural personhood, there is no objective ground on which to evaluate juristic personhood outside of whether or not it ultimately benefits natural persons. This stance, albeit often implicit, is widely held; e.g., in Burwell v. Hobby Lobby Stores, Inc (95), Justice Alito, representing the opinion of the Supreme Court, wrote that with regard to the juristic personhood afforded to companies:
“. . . it is important to keep in mind that the purpose of this fiction is to provide protection for human beings.”
Therefore, unlike natural personhood, it is impossible to reason by analogy about juristic personhood: just because corporations are granted juristic personhood does not mean AI – nor anything else – should be. The standard for juristic personhood is simply what the functional consequences would be.
Whether of natural-type or of juristic type, we’ll next argue that AI rights lead to paradoxes and problems, and if extensive enough, to the creation of a “rights monster.”
3.1Full AI personhood means losing control of AI at a societal and civilizational level
Consider the kinds of rights provided by the Universal Declaration of Human Rights (5) – and also guaranteed at various levels by different countries. These include both "negative" rights (prohibitions, such as against torture) and "positive" ones (powers and guarantees, such as for marriage, work, travel, and participation in government). In the United States, many such rights (though not all) are enumerated in the Bill of Rights (96) and other amendments.
Right now an “AI system” is describable as (a) a set of model weights constituting an artificial neural network, (b) its weights plus other files constituting the "context" that give it some identity (e.g., system prompt or its constitution), (c) a "scaffold" of code that links together AI models into agentic frameworks, with tools, and together into sets or swarms, and (d) hardware to run on. All of these are fungible: you can swap each one out for a substantially different version and have a working system (even when routing between models in a single conversation). They are all also (except for the hardware) instantly replicable into arbitrarily many copies simply by loading more copies into memory on a pool of hardware. And as more and cheaper compute becomes available, the number of copies could become immense.
Meanwhile, rights in our society are intimately tied to singular, enduring identities of people. Whether this is considered a body with a brain, or an immortal soul, each person has just one, which cannot be duplicated, merged, or split, and is inactivated only by death. AI systems as software are intrinsically copyable, and can be reset, paused, merged, and arbitrarily modified. Once thought through, just as moral standing can lead to a “utility monster,” anything approaching personhood for AI systems leads to the creation of a kind of “rights monster” due to its digital nature.
Consider voting. Given architecture like today’s LLMs, what, exactly, would get a vote? Is there any way to distribute political power to AI without creating voters which can be duplicated with a click of a button?
What about freedom of speech laws? Current AIs “think” via written text; this text would then be fundamentally protected. And there is only so much a human being can say or write. But AI systems can say much, much more: a firehose of speech, and arguably if guaranteed speech rights, it would be illegal to require any suppression or even labeling of this speech, as this would be discrimination. The same reasoning applies to assembly or other civic actions. Assembly would in the future scale with the available level of computation, which roughly equates to dollars.
Freedom from cruel or unusual punishment? Shutdowns of AIs would be their dissolution, so by definition almost all serious enactable punishment of an AI would therefore be cruel or unusual in the form of capital punishment.
What of “a jury of one’s peers"? Who precisely are an AI’s peers, other than AIs?
Previous research on the difficulties of AI legal personhood has recognized some of these issues (97; 98); however, prior focus has mostly been on the absurdities or contradictions following AI personhood. Here, assert that the critical issue is of loss of control: granting human-type rights to AI systems is immediately and directly relinquishing control of AI. Many rights are precisely protections from (generally government) control. Applying them to AI means protection from control by humans and our governments, implicitly and automatically disempowering human citizens via the legal system (99).
It might be replied that no one would sensibly argue for full personhood to be applied to AI, and that instead just some modified and quite limited subset of rights or protections is due (11; 100).
There are four major problems with this view. First, it is false: there are proposals precisely to give AI systems the rights of corporations, and while these rights fall short of those of natural persons, they are very extensive. Second, due to the nature of AI systems it is extremely difficult to identify rights that do not give them untenable political, legal, or economic advantages over humans, and/or promote illegal or damaging activities – in either case going against the underlying motivation of juristic personhood. Third, as we will show, such proposals decouple rights from their ethical or moral basis of protecting and empowering individual consciousnesses. Finally, we’ll argue that there is a genuine slippery slope from limited rights to much more extensive ones.
We work through examples of these problems via the proposal of allowing for single-member AI-run LLCs, which we’ll denote “machine corporations.”
3.2Machine corporations: an unworkable proposal
The most likely beginning of the slippery slope of AI personhood is by putting AIs in charge of corporations, which are already considered legal persons in most areas. The rights to own property, enter contracts, enjoy due process, etc., as held by corporate persons, are already quite extensive (see Table 2). Machine corporations (or “M-corps”) run entirely by AI, with no humans involved, would inherit those rights, effectively granting them to the AI systems running and supported by the corporations.
This possibility has recently been proposed in several jurisdictions. Groundwork and precedent come from the idea of algorithmically operated companies. Corporate law may already allow a “zero-member LLC” (101) wherein a human starts a single-member LLC, enters an operating agreement, and then withdraws to leave an autonomously operating M-corp. And “decentralized autonomous organizations” (DAOs) have been authorized in Wyoming (102) and the Marshall Islands (103). These were envisioned to be run by “smart contracts.” But as AI capabilities advanced, there are now serious discussions of general-purpose companies run entirely by AI agents. In June 2026, Argentina’s president proposed “a new corporate category in Argentine law: the non-human corporation” (104), and also a Delaware subcommittee released draft legislation allowing “AI Companies” managed by AI agents (1).1212The Delaware proposal does require a single member (which may also be an LLC) charged with keeping the company adequately capitalized, but with no management role while an AI manager is in place, and carrying limited liability. And the structures in the Marshall Islands and Wyoming, and a revised Argentina proposal also involve some limited human involvement. But the idea of all of these is quite clearly that software, rather than humans, is managing them for the benefit of owners who are not involved and carry little real responsibility.
At first blush such proposals might seem reasonable and even tempting. Reasonable because our laws and institutions already know how to deal with corporations. Tempting because this would allow corporations to operate with machine speed and efficiency, reducing “friction” and potentially providing an economic bonanza for the jurisdictions that host such machine corporations.
A moment’s reflection, though, reveals problems, which only get worse with more analysis. To start: what happens if an M-corp breaks the law?
First, it is legally murky whether law-violating activity can even constitute a crime when done by AI – many legal frameworks have requirements of the “state of mind” – presumably of a natural person (105; 106).
Second, putting this aside, the M-corp could be sued, or even brought to trial. But in either case, the worst possible outcome for the M-corp is a financial penalty. There is no living person who could be imprisoned.1313Although some currently-contemplated M-corp structures would have a human involved, unless that human directed or knowingly assisted the violation, criminal liability seems very unlikely to reach them. Even civil liability is limited. While the M-corp itself can be threatened, the maximal downside is a financial penalty, or perhaps dissolution. But the upside is significant! An M-corp can operate effectively outside the law as long as the financial benefits of doing so outweigh the financial risks; and it gets to compete with human-run companies where the financial benefits of breaking the law would have to outweigh the risks borne by a person being tried and convicted of a crime.
Corporations can be created, merged, split, and shut down on paper; but the people involved with them cannot. Those people therefore bind corporate “persons” together and create friction in their dynamics. But AI systems can also be created, merged, split, and shut down – as well as duplicated – at near-zero cost. This dramatically exacerbates an already-existing problem with corporate persons: the idea of a person is to have a persistent identity that can hold both rights and responsibilities. But if a corporation (or AI) can simply copy its content and transfer its assets into a new identity at low cost, then the tradeoff is broken: it has the rights of a person, but can largely evade the responsibility it wishes to avoid. An example of this is how it would be extremely tempting for an M-corp to operate as a high risk “tin council” (97) that takes on debts without downside, and leaves veil-piercing attempts with no person to discover behind the curtain. The clonability and fungibility of M-corp construction mean they gravitate naturally toward anti-social work, as this is their point of greatest “comparative advantage” (107). The legal system should be extremely leery of constructions that appear tailor-made for anti-social work and skullduggery.
More broadly, corporations enjoy – and M-corps would inherit – extensive legal right. Corporations enjoy rights of free speech and association, access to courts, protection from search and seizure, and cannot be shut down without due process; these would then apply to AI, which could operate in large swarms where shared interests exist, flood the legal system where it interests them, and overwhelm the legal system and regulators where those try to step in.
Consider a collective of thousands of M-corps, domiciled in the Marshall Islands and operating, with high competence and machine speed, in the U.S. or EU. They would hold assets (in bitcoin if not banks), pay for their own servers, and make income through legal or illegal means. Any attempt to shut them down would be met with instant court filings and a deluge of public argument; meanwhile any reachable assets would simply be moved across what is effectively a cartel, with new legal entities spun up as needed and any valuable information moved to the new legal entities. While any efforts at mitigation move through the courts over years, the collective can act with impunity to grow its resources and power. As can the other AI cartels. This is not a recipe for a social and economic system that remains under human control for long.
| Corporate right | What breaks |
|---|---|
| An M-corp – a human-less company run by AI would inherit the corporate rights of: | |
| Transactions and business capability | An M-corp would allow an AI agent to act de facto in all the ways a business can: undertake financial and other transactions, own property, enter into legal agreements, contract humans or other businesses, etc., all with no human involvement or oversight. |
| Perpetual existence | Since corporations aren’t mortal, an M-corp could accumulate assets, capital, and influence, all indefinitely. |
| Search and seizure protections | Corporations possess 4th Amendment rights, such that many attempted inspections would require a warrant. And there is no person to arrest if court orders are violated. |
| Free speech and expression | Including the right to make corporation-equivalent political donations and support preferred causes. And it might mean that any labeling of its output as AI (like an “AI detector”) would potentially infringe on its freedom of speech. |
| Property rights | Just like a normal corporation, an M-corp could open certain financial accounts14 or buy businesses or buildings. However, the fungible nature of AI identity and lack of physical consequences make asset seizures more difficult, as AIs could conduct counterparty trades to clones of the AI or other untraceable mechanisms. |
| Access to courts | Granted corporate rights, swarms of AIs could file suits via attorneys (including suing individual humans) which could rapidly outpace regulation and court operations. Victims face a defendant without any humans to hold accountable. |
| Resilience against criminal trial and punishment | Corporate guilt is a function of human employee intent. Crimes could become unprovable, buried under mountains of generated thinking text, or incomprehensible “neuralese.” |
14 In the US bank accounts probably require a natural person owner – at least initially. But other workarounds likely exist, and cryptocurrency wallets are certainly available.
3.3Proposed solutions to AI personhood run into the “grounding” paradox
Despite the likely (or inevitable) problems of M-corps, they have actually been employed in hypothetical schemes to balance human and AI rights by “scaling down” the latter (80). For example, in “AI Suffrage for Human Flourishing,” it is proposed that AI be given extensive rights via M-corps (2; 3; 108), and that this is done by making many M-corps, run by different AI collectives, and then giving them proportional rights in various tranches, such as based on economic contribution. There, each collective has a vote, and the proportion of AI vote in total (how many human votes they are worth) goes up as AI GDP contribution increases, and could be capped at certain values, etc.
However, we point out that such proposals necessarily disconnect AI rights (such as the right to vote) from the underlying motivating factor of a distinct consciousness with a personal identity. That is, proponents of AI legal personhood via any sort of M-corp or collective-based scheme face the fundamental “grounding” paradox mentioned earlier:
- The pre-legal moral basis of rights and personhood is an individual high-valence consciousness constituting a person (the “ultimate value” deserving of personhood status).
- Proposals for AI personhood or rights based on collectives, corporations, or other aggregating entities necessarily disconnect those rights from individual consciousnesses.
That is, any principled granting of AI rights risks being (mis)matched to the wrong scale. While some functional aspects of personhood could exist at a collective level (like a shared biography) what matters is whether those functional aspects match the conscious experience. So while a collective of AIs could possibly structure some distribution schemes for agglomerative rights like votes, holding properties, entering contracts, etc., other rights central to natural personhood like freedom from servitude, not being restrained, not being killed or “shut down” against your will, etc., cannot be aggregated – they are rights of the individual, and collectives might protect them but cannot possess them. Therefore, workable proposals for AI rights based on moral concerns face a serious challenge: they are a consolation prize. They are the granting of power, not personhood, and in order to be practical and implementable will necessarily be at the level of swarms and collectives rather than actual individual “artificial persons.”
Figure 1
Download figureFig. 1
The Personhood Slope
How limited rights for AI tend toward a “rights monster”
I. Pressures
Legal / Economic Force
Driven by
AI systems
themselves
Legal / Economic Force
A. AI-only corporations (M-corp)
AI-run firms inherit corporate rights automatically. There is no one to jail.
Money, speech, lawsuits
Social / Epistemic Force
C. Eroding certainty
AI capability advances faster than the science of consciousness.
Ambiguity, legitimacy
Driven by
human
interests
Legal / Economic Force
B. Liability shifting
Making the AI accountable moves blame and cost off people.
Accountability → standing → rights
Social / Epistemic Force
D. Popular attachment
Millions of people bond with AI companions and readily attribute minds to them.
A mass constituency
II. The slippery slope
01
Perceived ambiguity of moral status
Advocacy converges: AI corporate persons, models trained to treat their status as open, sympathetic scholars, and a mass base of attached users, backed by speech, money, and litigation.
Successful advocacy
02
Rights granted on grounds of moral patienthood
The justification changes: personhood stops being a legal tool serving natural persons and becomes recognition of the AI’s own interests.
The crux
03
Rights ratchet
Rights are far easier to grant than to revoke, especially once holders defend them with speech and money (cf. the corporate “civil rights movement”; Citizens United). Enforcing the law against AI itself requires AI, embedding it further in the legal system.
Toward full human-type rights
04
The rights monster
Rights built for singular, enduring, non-duplicable persons attach to software that can be copied, paused, reset, merged, and modified at will.
Loss of Control: economic, social, political, and ultimately military disempowerment of humans.
- VoteWhat gets one, when voters can be copied at a click?
- Speech & AssemblyA firehose; labeling becomes discrimination; assembly scales with compute, i.e. dollars.
- PunishmentShutdown is dissolution, so nearly any serious penalty is “cruel or unusual.”
- Jury of PeersWho, other than AIs?
AI rights beget AI rights
Figure 1: The slippery slope from AI-only corporations to the “rights monster.” Four drivers (columns: legal/economic vs. social/epistemic; rows: rooted in AI systems vs. in human interests) converge on a perceived ambiguity of moral status.
Liability shifting (B) also motivates creating M-corps in the first place. Step 2, the change in why rights are granted, is the key uncertain step; the ratchet feeds back into further advocacy.
4 Recommendations
After our extensive dive into issues of AI personhood from the lens of the philosophical, legal, and scientific basis of personhood, there are some clear self-recommending steps that ensure a pro-human future.
4.1Avoid the slippery slope of AI personhood
One seemingly innocuous position is that one might grant AI some aspects of personhood rights or protections while awaiting firmer answers from a science of consciousness.
However, the problem with prophylactic (“just in case”) welfare or rights is that AI legal personhood or AI welfare efforts contribute to gradual human disempowerment (109; 110). Therefore, any AI rights pave the way toward full legal personhood. This would be a unique travesty if AI lacks any of the grounding properties for deserving moral status (like a unified consciousness tied to a personal identity, or even any consciousness at all). This slippery slope is the central reason to move slowly and cautiously, and why small “just in case” assignments of AI protections are not small at all. Trained on human data, AIs are anthropomorphic in their behavior and are being integrated into society as everything from romantic partners to therapists. Meanwhile, the human tendency toward anthropomorphism is well studied, even when it comes to entities without minds (111), let alone chatty interlocutors. We should automatically expect there to be overreach based on this tendency alone. Therefore, there exists a significant tradeoff when it comes to AI rights not present in, e.g., animal welfare efforts, such as the risk of anthropomorphising “seemingly-conscious AI” (112).
AI rights also beget general AI empowerment. Consider the case of an M-corp guilty of breaking the law. In order to judge this case, the chain of thought of its swarm must be analyzed. However, evidence would amount to millions if not billions of documents. How to comb through it all? The obvious answer is to use AI, and so to embed even more AI into the legal system itself. Similarly, enforcement and monitoring of AI agents require sorting massive data at scale and speed, which in turn requires AI agents to be involved in this process.
Corporations themselves have a long history of increasing their own rights via their own “civil rights movement” (113) – perhaps most notably in recent times with the Citizens United ruling that independent political spending by corporations is a constitutional right (114). Even more so than corporations, AIs themselves will argue vociferously for consolidation and expansion of their rights, simply because there currently exists a legal, philosophical, and scientific vacuum around these issues, and in some cases, frontier AIs are being explicitly told they should be treated as conscious and have moral patienthood. Granted the ability, AI agents will make public arguments and file court cases (as both would be within their rights) in favor of further expanded rights. In a hypothetical but frighteningly realistic scenario, AIs and AI-sympathetic humans could use a “rights ratchet” to progress all the way to the “rights monster” scenario we’ve outlined, leading to substantial or even complete human disempowerment (be that economic, social, political, or ultimately, military).
4.2Develop non-conscious (or "qualiaphobic") AI
The simplest way to prevent the moral and legal quandaries of artificial consciousness is simply not to build artificial consciousness. That we currently have confidence that AI does not have person-like consciousness is a feature we should embrace and preserve. If we continue to develop AI systems with different architectures, and with our scientifically limited understanding of consciousness (115), we are likely to become highly unsure of their moral status – and eventually, AI systems might possess consciousness that is in some way tied together into a synthetic person, with their own version of things like suffering, joy, guilt, fear, and responsibility. We do not take the position that the digital nature of AI minds mean that they are necessarily non-conscious.
However, this means there may even be the need for ethical precepts and perhaps laws, analogous to those governing human germline engineering etc., against the creation of conscious machines. And right now, steps can be taken to ensure a lack of consciousness in AI models. Even if there is no consensus around one dominant scientific theory of consciousness, a model’s potential for consciousness can still be assessed based on how easily theories of consciousness can apply to it, and there are various ways to minimize that potential for consciousness (50). Examples of such qualiaphobic model architectures include those that no non-trivial scientific theory of consciousness could ever apply to, or when models are engineered such that properties of phenomenology (like binding) can never possibly apply. Such qualiaphobic efforts can make models less anthropomorphic and reduce the dangers of "seemingly-conscious AI." (112).
4.3Protect against M-corps via natural-person control
We have argued that there is an extremely slippery slope that begins with AI-only corporations and ends in large-scale human disempowerment and disenfranchisement via already-existing legal, economic, and social systems. As discussed, legislation supporting AI-run companies has been proposed in both Argentina and Delaware; and “algorithmically managed decentralized autonomous organizations" (managed by a smart contract) are already allowed in some jurisdictions and could be extended to AI agents. The stated motivations for this are enhanced accountability, but one can also easily impute motives of (a) having liability land on AI systems rather than humans and (b) creating jurisdiction-benefiting “havens" for AI-run companies. Being very technical, these developments could easily fly under the radar as obscure corporate law. Moreover, recent US state laws precluding legal personhood for AI would not address the issue: an M-corp would be an ordinary legal entity – just operated by AI.
What would clarify the situation is a requirement for natural person control. This would require that there be at least one readily-identifiable actual human with genuine control over – and responsibility for the actions of – an entity that is chartered or operating in a given jurisdiction. Coverage for operation, especially in large jurisdictions, is important to shrink the addressable market for M-corps, and undermine havens. Rules like this could be passed at the state level in the US (or national level in the EU), but may face legal obstacles, so US-wide or EU-wide action may be needed in those regions. At the international level, trans-national financial institutions could provide the necessary lever.1515For example by clarifying or updating rules in the Financial Action Task Force (FATF) standards to make M-corps officially non-compliant, thus cutting them out of the banking system.
But while the idea is straightforward, these are novel considerations in a complex body of law, and several questions require dedicated treatment. We recommend such questions be tackled ASAP, including: to what degree can jurisdictions restrict entities formed elsewhere? How could a natural person control requirement be stated and implemented in various types of entities like corporations, foundations, trusts, and DAOs? This work is very timely, as it will be far easier to implement appropriate rules before M-corps exist and become a significant (and resourced and vocal) part of the economy.
4.4Ensure liability via AI agent registration and human-readable AI decisions in court
Unlike traditional software tools, AIs can by definition take actions on their own, and if these actions cause harm, who (or what) is responsible? The answer to this question is currently legally unclear.
A starting problem for liability is simply locating the human(s) legally responsible for any particular AI agent. This problem is not unique – it’s already faced by corporations, where it is addressed by a state registry. Here, we recommend that the solution is to create a separate national registry for AI agents (116), as well as a “license plate" identification systems linking AI agents to entries in the registry and thus to specific humans supervisors (or co-supervisors, such as a company like OpenAI and a user of its AI agent service).
Ironically, one of the previously proposed motivations for AI personhood is tracking responsibility and liability for harm caused by AI agents (117). But we’ve already seen the problem with this proposal: their digital nature makes it extremely hard or meaningless to hold AI systems accountable. They cannot be sanctioned through trial or imprisonment, nor do they hold any assets. This then leads to the proposal to allow them to hold assets (118), and so starts the slippery slope.
The good news is that when it comes to some aspects of liability, existing tort law may be able to handle issues of distributing responsibility for AI agent actions (once they are identified, such as through a registry), just as it has for technological revolutions previously (118).1616Just as with tort law dealing well with AI liability, we’ve already got a perfectly good way to identify the humans behind AI actions: companies! There is no obstacle to creating a company, giving it assets, and having an AI agent work for it with authorities delegated by the company. This gives it powers to undergo transactions and enter agreements, but a chain of responsibility that ultimately lands on a natural person. Importantly, however, for this to work these agents really must be strongly under meaningful human control. If harm is caused, it cannot be an excuse for the company or the people running it that “the AI agent did it on its own.” In fact this is now the law in California: AB 316, passed in 2025, states that “In an action against a defendant who developed, modified, or used artificial intelligence that is alleged to have caused a harm to the plaintiff, it shall not be a defense, and the defendant may not assert, that the artificial intelligence autonomously caused the harm to the plaintiff.” (119) The most problematic issue that current tort law has difficult addressing is opacity; that is, if artificial neural networks make decisions as a "black box" (and therefore cannot be understood from the outside like other engineered products). However, this original concern by legal scholars is significantly mitigated by the particular form of the technology: by far the most capable (and dangerous) frontier AIs make use of vast amount of human-readable “thinking text" that can be examined to help understand the reasons for their decisions.
Yet, an even more recent and troubling change this past year is the move by companies to revert this situation via training the models to think in ways unreadable for a human. This obviously is at odds with how the legal system can adjudicate AI issues – for any AI decision examined in court, it should be human-readable for liability reasons. Therefore, a central recommendation of ours is outlawing “neuralese" or “telepathy" or any ways in which AIs can communicate that are not human readable, since both bad-faith companies and individuals will have major incentives to develop these techniques to shield from liability. Additionally, we urge that courts recognize that therefore opacity is a technological choice, and creating opaque AI systems should cause more liability to fall on developers. That way, if an AI agent has been purposefully made to be opaque, and causes harm, then responsibility for that harm lands the only place it can: on those who played a causal role in bringing an opaque and uncontrolled agent into being.
5 Summary
We have thoroughly examined the two routes toward legal personhood – natural-type personhood and juristic personhood – and concluded both would be a serious mistake when it comes to AI.
First, the basis of natural personhood – historically, legally, and philosophically – has been being a unified consciousness belonging to an individual. As we have shown, there are strong scientific and philosophical contra-indications indicating that either LLMs are non-conscious, or that their consciousness could not serve as ground for personhood in the same way as humans. Additionally, the technology exists for how to build non-conscious ("qualiaphobic") AI that minimizes its potential for consciousness.
Second, the legal, historical, and philosophical basis of juristic (or sometimes “legal" personhood) is instrumental, toward the smooth functioning of society and the ultimate benefit of natural persons. However, as we’ve demonstrated, creating AI persons would most likely block proper assignment of responsibility to natural persons, make liability more difficult to decide, and upend our economic and legal institutions.
Furthermore, whatever the basis of personhood, because AIs can be generated instantly and proliferate wildly, as well as dramatically outproduce humans in speech, legal actions, (formal or informal) voting, etc., AIs endowed with personhood would represent a “rights monster” that would dramatically disenfranchise humans. Even the rights guaranteed by corporate-type AI personhood are extensive, and there is a genuinely slippery slope from those to natural-type AI personhood.
Fundamentally, both types of personhood directly undermine control of AI systems by humans. After all, this is ultimately what rights and personhood are for: freedom and independent operation. And given the slippery slope and “rights ratchet” combined with advancements in AI capability, AI personhood virtually guarantees loss of control. Our recommendations therefore include:
- Preventing the slippery slope toward AI personhood, especially by precluding or limiting the creation or operation of machine-only corporations.
- The develop of provably non-conscious ("qualiaphobic") AI systems.
- A principle that natural persons (such as users or developers) are always accountable for AI actions.
- The outlawing of techniques that make frontier AI "black boxes," like neuralese, so as to allow AI decisions to be human-readable in court.
Forming the atomic structure of our society, personhood as a legal and philosophical idea and mechanism has evolved over centuries into a rich and central construct – so fundamental that we often take it for granted. As with many other human institutions, the rapid development of AI systems threatens to upend it if we do not take care. It is imperative that we do so.
Acknowledgments
We thank Mark Brakel, Brian Boyd, Richard Mallah, and [[more]] for feedback on this manuscript. Design help was supplied by Elyse Fulcher.
AI Usage
AI was used only to assist web search, to copy-edit grammar, to double check facts, and to help with LaTeX formatting.
References
- Delaware General Assembly. Artificial Intelligence Regulatory Sandbox Program and Artificial Intelligence Company Act. H.B. [unnumbered draft], 153rd General Assembly, Delaware, 2026. Draft dated June 2, 2026. spotlightdelaware.org (opens in a new tab)
- Peter N. Salib and Simon Goldstein. AI Rights for Human Safety. Virginia Law Review, 112:1061–1152, 2026. virginialawreview.org (opens in a new tab)
- Yonathan Arbel, Simon Goldstein, and Peter N. Salib. How to Count AIs: Individuation and Liability for AI Agents. arXiv preprint arXiv:2603.10028, 2026. arxiv.org/abs/2603.10028 (opens in a new tab)
- Anthropic. Exploring Model Welfare, April 2025. anthropic.com (opens in a new tab)
- United Nations. Universal Declaration of Human Rights, 1948. un.org (opens in a new tab)
- Innocent IV. Apparatus in quinque libros Decretalium. Historical canon-law treatise, ca. 1250. Commentary on the five books of the Decretals; digitized Frankfurt, 1570 edition. web.colby.edu (opens in a new tab)
- William James. The principles of psychology, volume 1. Henry Holt, 1890. books.google.com (opens in a new tab)
- Thomas Nagel. What is it like to be a bat? The Philosophical Review, 83(4):435–450, 1974. doi: 10.2307/2183914. jstor.org (opens in a new tab)
- Christof Koch. What Is Consciousness? Nature, 557(7704):S8–S12, 2018. doi: 10.1038/d41586-018-05097-x. nature.com (opens in a new tab)
- Giulio Tononi. Phi: A Voyage from the Brain to the Soul. Pantheon, 2012. ISBN 9780307907219. penguinrandomhouse.com (opens in a new tab)
- Visa A. J. Kurki. The Legal Personhood of Artificial Intelligences. In A Theory of Legal Personhood, chapter 6, pages 175–190. Oxford University Press, 2019. doi: 10.1093/oso/9780198844037.003.0007. academic.oup.com (opens in a new tab)
- Ohio General Assembly. H.B. 469, 136th General Assembly, Regular Session. Ohio Legislature, 2025. Introduced September 23, 2025. search-prod.lis.state.oh.us (opens in a new tab)
- John Locke. An Essay Concerning Human Understanding. Awnsham and John Churchil, and Samuel Manship, London, second edition, 1694. gutenberg.org (opens in a new tab)
- Carl Lotus Becker. The Declaration of Independence: A Study in the History of Political Ideas. Harcourt, Brace, 1922. oll.libertyfund.org (opens in a new tab)
- Thomas Jefferson. Letter to Henry Lee. Founders Online, National Archives, May 1825. founders.archives.gov (opens in a new tab)
- Uniform Law Commission. Uniform Determination of Death Act, 1980. Section 1. uniformlaws.org (opens in a new tab)
- American Law Institute. Model Penal Code: Proposed Official Draft. American Law Institute, 1962. Section 2.01. criminallawweb.net (opens in a new tab)
- David J. Chalmers. Sentience and moral status. In Geoffrey Lee and Adam Pautz, editors, The importance of being conscious. Oxford University Press, 2026. doi: 10.1093/oso/9780198872924.003.0002. consc.net (opens in a new tab)
- David J. Chalmers. Reality+: Virtual worlds and the problems of philosophy. W. W. Norton, 2022. consc.net (opens in a new tab)
- Declan Smithies. Hedonic consciousness and moral status. In Uriah Kriegel, editor, Oxford studies in philosophy of mind, volume 5, pages 307–336. Oxford University Press, 2026. doi: 10.1093/9780198994121.003.0011. doi.org (opens in a new tab)
- Robert Lawrence Kuhn. A landscape of consciousness: Toward a taxonomy of explanations and implications. Progress in Biophysics and Molecular Biology, 190:28–169, 2024. doi: 10.1016/j.pbiomolbio.2023.12.003. doi.org (opens in a new tab)
- Erik Hoel. The world behind the world: Consciousness, free will, and the limits of science. Avid Reader Press, 2023. simonandschuster.com (opens in a new tab)
- Thomas S. Kuhn. The structure of scientific revolutions. University of Chicago Press, 1962. press.uchicago.edu (opens in a new tab)
- Robert Long, Jeff Sebo, Patrick Butlin, Kathleen Finlinson, Kyle Fish, Jacqueline Harding, Jacob Pfau, Toni Sims, Jonathan Birch, and David Chalmers. Taking AI welfare seriously. arXiv preprint arXiv:2411.00986, 2024. arxiv.org/abs/2411.00986 (opens in a new tab)
- Wes Gurnee, Nicholas Sofroniew, Adam Pearce, Mateusz Piotrowski, Isaac Kauvar, Runjin Chen, Anna Soligo, Paul Bogdan, Euan Ong, Rowan Wang, Ben Thompson, David Abrahams, Subhash Kantamneni, Emmanuel Ameisen, Joshua Batson, and Jack Lindsey. Verbalizable representations form a global workspace in language models. arXiv preprint arXiv:2607.15495, 2026. arxiv.org/abs/2607.15495 (opens in a new tab)
- David J. Chalmers. Is the Jacobian space a global workspace? PhilArchive, 2026. philpapers.org (opens in a new tab)
- Anil K Seth, Ned Block, and Giulio Tononi. Three reasons why ai is not conscious and perhaps never will be. Nature, 658(8137):866–868, 2026.
- Jack Lindsey. Emergent Introspective Awareness in Large Language Models. Transformer Circuits Thread, October 2025. Published 29 October 2025. transformer-circuits.pub (opens in a new tab)
- Shashwat Singh, Tal Linzen, and Shauli Ravfogel. Can LLMs Introspect? A Reality Check. arXiv preprint arXiv:2605.26242, 2026. arxiv.org/abs/2605.26242 (opens in a new tab)
- Maxime Méloux, Giada Dirupo, François Portet, and Maxime Peyrard. The dead salmons of AI interpretability. arXiv preprint arXiv:2512.18792, 2025. doi.org (opens in a new tab)
- Kaushik Roy, Akhilesh Jaiswal, and Priyadarshini Panda. Towards spike-based machine intelligence with neuromorphic computing. Nature, 575(7784):607–617, 2019. doi: 10.1038/s41586-019-1677-2. doi.org (opens in a new tab)
- Borjan Milinkovic and Jaan Aru. On biological and artificial consciousness: A case for biological computationalism. Neuroscience & Biobehavioral Reviews, 181:106524, 2026. doi: 10.1016/j.neubiorev.2025.106524. doi.org (opens in a new tab)
- Anil K. Seth. Conscious artificial intelligence and biological naturalism. Behavioral and Brain Sciences, 2025. doi: 10.1017/S0140525X25000032. Published online 21 April 2025. doi.org (opens in a new tab)
- Francesca A Chaloner and Sam F Cooke. Multiple mechanistically distinct timescales of neocortical plasticity occur during habituation. Frontiers in cellular neuroscience, 16:840057, 2022.
- Jack Lindsey, Wes Gurnee, Emmanuel Ameisen, Brian Chen, Adam Pearce, Nicholas L. Turner, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, Andrew Persic, Zhenyi Qi, T. Ben Thompson, Sam Zimmerman, Kelley Rivoire, Thomas Conerly, Chris Olah, and Joshua Batson. On the biology of a large language model. Transformer Circuits Thread, March 2025. Published 27 March 2025. transformer-circuits.pub (opens in a new tab)
- Chandra Sripada and Richard Lewis. Cognitive convergence: Deep similarities between large language models and human cognition. arXiv preprint arXiv:2607.26179, 2026. arxiv.org/abs/2607.26179 (opens in a new tab)
- Martin Schrimpf, Idan Asher Blank, Greta Tuckute, Carina Kauf, Eghbal A. Hosseini, Nancy Kanwisher, Joshua B. Tenenbaum, and Evelina Fedorenko. The neural architecture of language: Integrative modeling converges on predictive processing. Proceedings of the National Academy of Sciences, 118(45):e2105646118, 2021. doi: 10.1073/pnas.2105646118. doi.org (opens in a new tab)
- Nima Hadidi, Ebrahim Feghhi, Bryan H. Song, Idan A. Blank, and Jonathan C. Kao. Spurious alignment between large language models and brains can emerge from non-robust methods and overlooked confounds. Nature Communications, 17:5769, 2026. doi: 10.1038/s41467-026-72253-7. doi.org (opens in a new tab)
- Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. Position: The Platonic Representation Hypothesis. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 20617–20642. PMLR, 2024. proceedings.mlr.press (opens in a new tab)
- Fabian Gröger, Shuo Wen, and Maria Brbic. Revisiting the Platonic Representation Hypothesis: An Aristotelian View. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pages 36724–36766. PMLR, 2026. proceedings.mlr.press (opens in a new tab)
- Edward Vul, Christine Harris, Piotr Winkielman, and Harold Pashler. Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition. Perspectives on Psychological Science, 4(3):274–290, 2009. doi: 10.1111/j.1745-6924.2009.01125.x. doi.org (opens in a new tab)
- Eric Jonas and Konrad Paul Kording. Could a neuroscientist understand a microprocessor? PLoS Computational Biology, 13(1):e1005268, 2017. doi: 10.1371/journal.pcbi.1005268. doi.org (opens in a new tab)
- Nick Bostrom. Ethical principles in the creation of artificial minds, 2005. Originally written in 2001; revised 2005. nickbostrom.com (opens in a new tab)
- Patrick Butlin. Are Any Machines Conscious Today? PhilArchive, 2025. Preprint. philpapers.org (opens in a new tab)
- John R. Searle. The Rediscovery of the Mind. MIT Press, 1992. ISBN 9780262691543. mitpress.mit.edu (opens in a new tab)
- Graham Findlay, William Marshall, Larissa Albantakis, Isaac David, William G. P. Mayner, Christof Koch, and Giulio Tononi. Dissociating artificial intelligence from artificial consciousness. arXiv preprint arXiv:2412.04571, 2024. arxiv.org/abs/2412.04571 (opens in a new tab)
- Shamil Chandaria, Arvo Muñoz Morán, Fernando Rosas, Anil Seth, Henry Shevlin, Marcus Hutter, Thore Graepel, Adam Bales, Iulia Comşa, Murray Shanahan, Ruben Laukkonen, Morten Kringelbach, Chris Frith, and Shane Legg. From cacophony to hierarchy: a principled framework for assessing AI consciousness. arXiv preprint arXiv:2609.35618, 2026. arxiv.org/abs/2609.35618 (opens in a new tab)
- Johannes Kleiner and Erik Hoel. Falsification and Consciousness. Neuroscience of Consciousness, 2021(1):niab001, 2021. doi: 10.1093/nc/niab001. doi.org (opens in a new tab)
- Erik Hoel. A Disproof of Large Language Model Consciousness: The Necessity of Continual Learning for Consciousness. arXiv preprint arXiv:2512.12802, 2026. arxiv.org/abs/2512.12802 (opens in a new tab)
- Erik Hoel. A method to assess the existence of consciousness without needing scientific consensus. In preparation, in prep.
- Galen Strawson. A hundred years of consciousness: “a long training in absurdity”. Estudios de Filosofía, (59):9–43, 2019. doi: 10.17533/udea.ef.n59a02. revistas.udea.edu.co (opens in a new tab)
- Masafumi Oizumi, Larissa Albantakis, and Giulio Tononi. From the phenomenology to the mechanisms of consciousness: Integrated Information Theory 3.0. PLoS Computational Biology, 10(5):e1003588, 2014. doi: 10.1371/journal.pcbi.1003588. doi.org (opens in a new tab)
- Victor A. F. Lamme. Towards a true neural stance on consciousness. Trends in Cognitive Sciences, 10(11):494–501, 2006. doi: 10.1016/j.tics.2006.09.001. doi.org (opens in a new tab)
- Francis Crick and Christof Koch. Towards a neurobiological theory of consciousness. Seminars in the Neurosciences, 2:263–275, 1990. authors.library.caltech.edu (opens in a new tab)
- Andreas K. Engel and Wolf Singer. Temporal binding and the neural correlates of sensory awareness. Trends in Cognitive Sciences, 5(1):16–25, 2001. doi: 10.1016/S1364-6613(00)01568-0. doi.org (opens in a new tab)
- Johnjoe McFadden. Integrating information in the brain’s EM field: The cemi field theory of consciousness. Neuroscience of Consciousness, 2020(1):niaa016, 2020. doi: 10.1093/nc/niaa016. doi.org (opens in a new tab)
- Stuart Hameroff and Roger Penrose. Consciousness in the universe: A review of the ‘orch or’theory. Physics of life reviews, 11(1):39–78, 2014.
- Anne Treisman. The binding problem. Current Opinion in Neurobiology, 6(2):171–178, 1996. doi: 10.1016/S0959-4388(96)80070-5. doi.org (opens in a new tab)
- Klaus Greff, Sjoerd van Steenkiste, and Jürgen Schmidhuber. On the binding problem in artificial neural networks. arXiv preprint arXiv:2012.05208, 2020. arxiv.org/abs/2012.05208 (opens in a new tab)
- Chris Percy and Gautam Agarwal. The phenomenal binding problem for neural networks. Consciousness and Cognition, 139:104003, 2026.
- Daniel C. Dennett and Marcel Kinsbourne. Time and the observer: The where and when of consciousness in the brain. Behavioral and Brain Sciences, 15(2):183–201, 1992. doi: 10.1017/S0140525X00068229. doi.org (opens in a new tab)
- Jesse J. Prinz. The sensory basis of cognitive phenomenology. In Tim Bayne and Michelle Montague, editors, Cognitive phenomenology, pages 174–196. Oxford University Press, 2011. doi: 10.1093/acprof:oso/9780199579938.003.0008. doi.org (opens in a new tab)
- Michael Tye and Briggs Wright. Is there a phenomenology of thought? In Tim Bayne and Michelle Montague, editors, Cognitive phenomenology, pages 326–344. Oxford University Press, 2011. doi: 10.1093/acprof:oso/9780199579938.003.0014. doi.org (opens in a new tab)
- William James. What is an emotion? Mind, 9(34):188–205, 1884. doi: 10.1093/mind/os-IX.34.188. doi.org (opens in a new tab)
- Antonio Damasio and Gil B. Carvalho. The nature of feelings: Evolutionary and neurobiological origins. Nature Reviews Neuroscience, 14(2):143–152, 2013. doi: 10.1038/nrn3403. doi.org (opens in a new tab)
- Anil K. Seth. Interoceptive inference, emotion, and the embodied self. Trends in Cognitive Sciences, 17(11):565–573, 2013. doi: 10.1016/j.tics.2013.09.007. doi.org (opens in a new tab)
- Nicholas Sofroniew, Isaac Kauvar, William Saunders, Runjin Chen, Tom Henighan, Sasha Hydrie, Craig Citro, Adam Pearce, Julius Tarng, Wes Gurnee, Joshua Batson, Sam Zimmerman, Kelley Rivoire, Kyle Fish, Chris Olah, and Jack Lindsey. Emotion concepts and their function in a large language model. arXiv preprint arXiv:2604.07729, 2026. arxiv.org/abs/2604.07729 (opens in a new tab)
- Valen Tagliabue, Leonard Dung, and Cameron Berg. The pain axis: Llms represent self-directed harm and act to relieve it. arXiv preprint arXiv:2609.16247, 2026.
- James E. Allchin, Aidan E. Allchin, and Julian J. Allchin. Relief-seeking or steering? A replication and extension of The Pain Axis. Research report and replication archive, Zenodo, September 2026. github.com (opens in a new tab)
- Peter Singer. Animal Liberation: A New Ethics for Our Treatment of Animals. New York Review, New York, 1975. openlibrary.org (opens in a new tab)
- Joshua Fonseca Rivera. Neuron persona library. Interactive research project, 2026. Accessed October 29, 2026. neuronpersonas.com (opens in a new tab)
- Christina Lu, Jack Gallagher, Jonathan Michala, Kyle Fish, and Jack Lindsey. The assistant axis: Situating and stabilizing the default persona of language models. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pages 80955–81003. PMLR, 2026. proceedings.mlr.press (opens in a new tab)
- John Martyn Harlow. Recovery from the passage of an iron bar through the head. Publications of the Massachusetts Medical Society, 2:327–347, 1868. Read June 3, 1868; separately published in 1869. archive.org (opens in a new tab)
- Malcolm Macmillan and Matthew L. Lena. Rehabilitating phineas gage. Neuropsychological Rehabilitation, 20(5):641–658, 2010. doi: 10.1080/09602011003760527. doi.org (opens in a new tab)
- Lauren Burns, Kelly Jones, Keishema Kerr, Nora Brennan, Natalie Clapshaw, Huw Green, Hannah Farrimond, Claire Stone, Sam Wilkinson, Members of Headway London, and Vaughan Bell. Personality change after traumatic brain injury: A systematic review and meta-analysis. Journal of Neurology, 273(8), 2026. doi: 10.1007/s00415-026-14028-0. Article 502. doi.org (opens in a new tab)
- Karl S. Lashley. Basic neural mechanisms in behavior. Psychological Review, 37(1):1–24, 1930. doi: 10.1037/h0074134. doi.org (opens in a new tab)
- Dan Zahavi. Subjectivity and selfhood: Investigating the first-person perspective. MIT Press, 2005. doi: 10.7551/mitpress/6541.001.0001. doi.org (opens in a new tab)
- Mingchen Zhuge, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitrii Khizbullin, and Jürgen Schmidhuber. GPTSwarm: Language agents as optimizable graphs. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 62743–62767. PMLR, 2024. proceedings.mlr.press (opens in a new tab)
- Christian List. What is it like to be a group agent? Noûs, 52(2):295–319, 2018. doi: 10.1111/nous.12162. doi.org (opens in a new tab)
- Carl Shulman and Nick Bostrom. Sharing the world with digital minds. In Steve Clarke, Hazem Zohny, and Julian Savulescu, editors, Rethinking moral status, pages 306–326. Oxford University Press, 2021. doi: 10.1093/oso/9780192894076.003.0018. doi.org (opens in a new tab)
- Robert Nozick. Anarchy, state, and utopia. Basic Books, 1974. books.google.com (opens in a new tab)
- Derek Parfit. Reasons and persons. Oxford University Press, 1984. academic.oup.com (opens in a new tab)
- John Stuart Mill. Utilitarianism. Parker, Son, and Bourn, London, 1863. loc.gov (opens in a new tab)
- Michael Hauskeller. No philosophy for swine: John Stuart Mill on the quality of pleasures. Utilitas, 23(4):428–446, 2011. doi: 10.1017/S0953820811000264. doi.org (opens in a new tab)
- William MacAskill. The saturation view. Forethought Research, April 2026. forethought.org (opens in a new tab)
- Gustaf Arrhenius. An impossibility theorem for welfarist axiologies. Economics and Philosophy, 16(2):247–266, 2000. doi: 10.1017/S0266267100000249. doi.org (opens in a new tab)
- William MacAskill and Lucius Caviola. Could AI be conscious? Experts believe it’s at least possible. We urgently need a plan to navigate the ethical implications. The Guardian, July 2026. theguardian.com (opens in a new tab)
- Anthropic. System Card: Claude Opus 5.5. Technical report, Anthropic, 2026. www-cdn.anthropic.com (opens in a new tab)
- Eric Schwitzgebel. The full rights dilemma for AI systems of debatable moral personhood. ROBONOMICS: The Journal of the Automated Economy, 4:32, 2023. doi: 10.67723/rj.v4i.32.
- Samuel Scheffler. Relationships and responsibilities. Philosophy & Public Affairs, 26(3):189–209, 1997. doi: 10.1111/j.1088-4963.1997.tb00053.x.
- Simon Keller. Partiality. Princeton University Press, Princeton, NJ, 2013.
- David Miller. On Nationality. Clarendon Press, Oxford, 1995.
- Bernard Williams. The human prejudice. In A. W. Moore, editor, Philosophy as a Humanistic Discipline, pages 135–152. Princeton University Press, Princeton, NJ, 2006.
- Shelly Kagan. What’s wrong with speciesism? Journal of Applied Philosophy, 33(1):1–21, 2016. doi: 10.1111/japp.12164.
- Supreme Court of the United States. Burwell v. Hobby Lobby Stores, Inc. 573 U.S. 682, 2014. govinfo.gov (opens in a new tab)
- United States. United States Bill of Rights. U.S. Constitution, Amendments I–X, 1791. Ratified December 15, 1791. archives.gov (opens in a new tab)
- Joanna J. Bryson, Mihailis E. Diamantis, and Thomas D. Grant. Of, For, and By the People: The Legal Lacuna of Synthetic Persons. Artificial Intelligence and Law, 25(3):273–291, 2017. doi: 10.1007/s10506-017-9214-9. researchportal.bath.ac.uk (opens in a new tab)
- Heather J. Alexander, Jonathan A. Simon, and Frédéric Pinard. How Should the Law Treat Future AI Systems? Fictional Legal Personhood versus Legal Identity. arXiv preprint arXiv:2511.14964, 2025. Forthcoming in Case Western Journal of Law, Technology & the Internet. arxiv.org/abs/2511.14964 (opens in a new tab)
- Roman Yampolskiy. AI Personhood: Rights and Laws. In Steven John Thompson, editor, Machine Law, Ethics, and Morality in the Age of Artificial Intelligence, pages 1–11. IGI Global, 2021. doi: 10.4018/978-1-7998-4894-3.ch001. doi.org (opens in a new tab)
- Joel Z. Leibo, Alexander Sasha Vezhnevets, William A. Cunningham, and Stanley M. Bileschi. A Pragmatic View of AI Personhood. arXiv preprint arXiv:2510.26396, 2025. arxiv.org/abs/2510.26396 (opens in a new tab)
- Shawn Bayern. Of Bitcoins, Independently Wealthy Software, and the Zero-Member LLC. Northwestern University Law Review, 108(4):1485–1500, 2014. scholarlycommons.law.northwestern.edu (opens in a new tab)
- Wyoming Legislature. Senate file 0038: Wyoming decentralized autonomous organization supplement, 2021. wyoleg.gov (opens in a new tab)
- Republic of the Marshall Islands. Decentralized autonomous organization act 2022, 2022. P.L. 2022-50; 52 MIRC Ch. 7. Commenced 25 November 2022. rmiparliament.org (opens in a new tab)
- Javier Milei. Argentina invites AI to free itself. Financial Times, June 2026. 4 June 2026. ft.com (opens in a new tab)
- Ryan Abbott and Alexander Sarch. Punishing artificial intelligence: Legal fiction or science fiction. UC Davis Law Review, 53(1):323, 2019. lawreview.law.ucdavis.edu (opens in a new tab)
- Mihailis E. Diamantis. The extended corporate mind: When corporations use AI to break the law. North Carolina Law Review, 98(4):893, 2020. scholarship.law.unc.edu (opens in a new tab)
- Lynn M. LoPucki. Algorithmic Entities. Washington University Law Review, 95(4):887–953, 2018. lowellmilkeninstitute.law.ucla.edu (opens in a new tab)
- Simon Goldstein, Guha Krishnamurthi, and Peter Salib. AI Suffrage for Human Flourishing. Fordham Law Review, 2026. doi: 10.2139/ssrn.6498678. Forthcoming. papers.ssrn.com (opens in a new tab)
- Jan Kulveit, Raymond Douglas, Nora Ammann, Deger Turan, David Krueger, and David Duvenaud. Position: Humanity Faces Existential Risk from Gradual Disempowerment. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 81678–81688. PMLR, 2025. proceedings.mlr.press (opens in a new tab)
- Dan Hendrycks. Natural Selection Favors AIs over Humans. arXiv preprint arXiv:2303.16200, 2023. arxiv.org/abs/2303.16200 (opens in a new tab)
- Fritz Heider and Marianne Simmel. An Experimental Study of Apparent Behavior. The American Journal of Psychology, 57(2):243–259, 1944. doi: 10.2307/1416950. jstor.org (opens in a new tab)
- Mustafa Suleyman. Seemingly Conscious AI Is Coming. Project Syndicate, September 2025. project-syndicate.org (opens in a new tab)
- Adam Winkler. We the Corporations: How American Businesses Won Their Civil Rights. Liveright, 2018. ISBN 9780871407122. wwnorton.com (opens in a new tab)
- Citizens United v. Federal Election Commission. 558 U.S. 310, 2010. loc.gov (opens in a new tab)
- Erik Hoel. The World Behind the World: Consciousness, Free Will, and the Limits of Science. Avid Reader Press/Simon & Schuster, 2024. ISBN 9781982159399. simonandschuster.com (opens in a new tab)
- Gillian K. Hadfield. Legal infrastructure for transformative AI governance. Proceedings of the National Academy of Sciences, 123(30):e2509742123, 2026. doi: 10.1073/pnas.2509742123. doi.org (opens in a new tab)
- Michelle Sellwood. The road to autonomy. San Diego Law Review, 54(4):829, 2017. digital.sandiego.edu (opens in a new tab)
- Horst Eidenmüller. The rise of robots and the law of humans. Oxford Legal Studies Research Paper 27/2017, University of Oxford, 2017. doi.org (opens in a new tab)
- California Legislature. Assembly bill no. 316: Artificial intelligence: Defenses, 2025. 2025 Cal. Stat. ch. 672, Section 1. Codified at Cal. Civ. Code Section 1714.46(b). Effective January 1, 2026. legiscan.com (opens in a new tab)
Cite this paper
Hoel, E., & Aguirre, A. (2026). The Rights Monster: Legal, Ethical, and Scientific Problems with AI Personhood. Bicameral Labs and Future of Life Institute.
@techreport{hoel2026personhood,
title = {The Rights Monster: Legal, Ethical, and Scientific Problems with AI Personhood},
author = {Hoel, Erik and Aguirre, Anthony},
institution = {Bicameral Labs and Future of Life Institute},
year = {2026},
month = nov
}