Hi everyone,
One of the major challenges in clinical neuroscience is understanding and treating Disorders of Consciousness (DoC), including coma, unresponsive wakefulness syndrome (UWS), and minimally conscious states (MCS).
Research into potential interventions faces significant ethical and practical constraints. Could artificial neural networks provide a complementary computational environment for exploring some of the underlying dynamics?
I recently introduced Project Sleeping LLM, an exploratory research framework investigating this possibility.
The project explores whether autoregressive Transformers could serve as in-silico model organisms for studying computational analogues of disrupted consciousness-related dynamics.
I can share the Zenodo working paper and DOI in the comments for anyone interested in the formal mathematics and architectural details.
The core hypothesis
Could information processing in Transformer architectures offer a useful computational setting for investigating ideas related to consciousness, including Global Neuronal Workspace Theory (GNWT) and Integrated Information Theory (IIT)?
These connections are hypotheses to investigate, not an assumption that language models are conscious.
The initial framework proposes three research directions:
- Modeling disrupted information processing
Introducing controlled perturbations into Transformer architectures, such as residual-stream attenuation, attention-pattern modifications, and latent-state constraints, to study how they affect coherent information processing.
- Measuring perturbational complexity
Investigating whether concepts from the Perturbational Complexity Index (PCI) can be adapted to Transformer hidden states, using measures such as Lempel–Ziv complexity and changes in activation patterns following localized perturbations.
- Exploring recovery of coherent dynamics
Testing whether interventions such as representation steering, controlled stochastic perturbations, and recurrent feedback can restore coherent processing after experimentally induced disruptions, without restarting the model or retraining its weights.
The central question is whether these experiments could provide a useful computational testbed for studying information integration, network disruption, and recovery dynamics.
Looking for critical feedback and collaborators
I initiated this project and developed its initial conceptual framework. I’m now looking to connect with researchers who can challenge its assumptions, identify methodological weaknesses, and help establish whether the proposed approach is scientifically viable.
I’d particularly welcome input from:
- Computational neuroscientists working on consciousness, PCI, neural complexity, or disorders of consciousness.
- ML researchers and engineers experienced in mechanistic interpretability, PyTorch, TransformerLens, or representation engineering.
- Complex systems researchers interested in information integration, dynamical systems, and perturbation-based analysis.
Two questions I’m especially interested in discussing:
- How defensible is the analogy between cortical disconnection and disruptions to information flow in Transformer architectures?
- What methodological challenges would need to be addressed to adapt PCI meaningfully to hidden-state dynamics?
Critical feedback, alternative approaches, and potential collaborations are all welcome. The goal at this stage is to test the framework rigorously, including the possibility that some of its central assumptions may not hold.
Thanks for reading.