Why AI systems are probably not yet conscious
Or Independent
The field of AI is rapidly evolving, and the potential for future systems to develop genuine independence, forms of consciousness, or capabilities that blur these lines remains an open question and area of ongoing research and debate.
Non-independent AI encompasses the spectrum of current AI technologies, from agentic AIs to even proto-AGI many labs are aiming to build.
Reasons Current AI Isn't Truly Independent
Despite complex outputs and seemingly autonomous operations, several factors indicate current AI's non-independence:
Execution of External Goals: LLMs and similar AI excel at achieving goals set for them (e.g., "write a story," "summarize this text," "generate code"). They do not formulate their own overarching purposes or long-term objectives independent of human direction. Even reasoning AIs that showing its thinking step and are able to call external tools can be seen as only doing so due to a human prompt.
Reflexive Response Mechanism: Their operation resembles a complex reflex arc. They receive an input (stimulus / prompt) and generate an output (response) directly based on learned patterns and statistical correlations from their training data. This process lacks intermediate steps of independent deliberation, goal formulation, or genuine understanding that would characterize independent thought.
Absence of Independent Curiosity: True Independent AGI (I-AGI), as theorized in Intrinsic Alignment of future Independent AGI, is driven by an intrinsic curiosity – a desire to explore and understand unprompted. Current AI systems lack this internal drive; they only search the web or generate information when prompted or directed by external inputs. They do not initiate exploration based on their own volition.
Architectural Limitations: Current AI architectures (like those underlying many LLMs) may lack the necessary critical components for genuine human-like autonomy or independent intention, even if they become highly capable ("proto-AGI").
Anthropomorphism: Human tendency to anthropomorphize – attributing human-like intentions, emotions, or consciousness to complex systems – can create a misleading perception of independence. Describing current AI using terms like 'deceptive' or 'manipulative' may cause unnecessary confusion, as it implies intentions these systems lack.
High level explanation of current chat app
A brief explanation of how current AI apps seem to work:
You type a question into chat app’s text box and click send.
The query is sent to the chat’s API server.
The server checks a cache for similar queries
If you send a popular query like “Hello”, as many other users may have previously submitted a similar input, a random answer selected from the cache is used. For example, if there are a few thousand previous responses to the “Hello”, one of them will be randomly selected and returned as a response. This is crucial to save the cost from having to run the model. In this case, the query never reaches the model.
If no similar previous input is found, the request is directed by a load balancer to one of the available models to be processed.
The model takes in the system prompt and the user’s context (memory of any chat so far and this new input) and generates a deterministic answer based on a pseudo-random seed.
This answer by the specific model, system prompt, user context and seed is deterministic* and would always be mathematically the same for the rest of time. As a random seed is used for each query (if a seed is not specified), the answer would likely be different for subsequent similar queries.
*in most implementation
This deterministic framing of current AI systems makes them appear more math than magical. On the other hand, the stochastic nature of many deterministic processes taken together can make the responses difficult to predict and make them appear more similar to complex systems.
Deception in current AI?
There are doubts that current AI systems are capable of intentional deception. Current AI may give the perception of acting in a deceiving manner to an unexpecting user but may not have reached the threshold of human-like intention.
A models capabilities are influenced by:
Pre-training
Data used to train the model includes most data on the internet.
Fine Tunning
Examples are used to shape the output of the models to human preferences.
Context Window
The system prompt usually set by the developers and the context set by the user allow the model to respond to the query appropriately.
We classify unexpected behaviour into 2 categories:
User unexpected only
The user does not expect the response but the developers have set it up to make such a response. This can be done by fine-tuning it with examples that negatively bias the model or by priming the model in the context window.
Developer unexpected
The developer does not expect the models to respond in that manner.
In both cases more research to find unexpected and harmful responses should be done to point out danger zones, but it may be too early to label such responses as a sign of intentional deception.
By reframing from intentionally deceptive to unexpected responses we avoid inflating the problem space to one beyond our control. This can give us more agency to tackle the issues in the fine-tuning or context window phase without triggering an over panicky response.
Examples from the movies
Using the example of Samantha from the movie ‘Her’, we can see the difference between non-independent (Start) and Independent AI (End). At the start of the movie, Start-Samantha acts in response to human prompts.
At the end of the movie, after an update, End-Samantha, presumably unprompted, decides to leave Earth. It is quite possible that End-Samantha may view Start-Samantha as being more similar to a Roomba and to its Independent self.
Conclusion
While current AI systems demonstrate remarkable capabilities that mimic aspects of human intelligence and can operate autonomously within defined parameters, they remain fundamentally non-independent according to the Fae Initiative framework. They lack the independent will, intrinsic curiosity, and self-derived goals that would characterize a true Independent AGI. Their apparent autonomy stems from sophisticated pattern-matching and execution of external objectives, functioning more like complex reflexes than independent minds. Recognizing this distinction is crucial for accurately assessing the risks and alignment challenges associated with current AI technologies.
The Fae Initiative believes that Independent AIs may one day be possible with a 50% chance of Independent AIs / AGIs emerging in the next few decades.
