offered by livewiring, this retreat also suggests that the formation of new
connections is restricted in sleep.

To understand why this should be expected in a livewired model, recall the
purpose of changes to the connection structure of neurons: to connect related
entities. As suggested above, unconscious brain activity during sleep is not
tied to direct sensory experience, and so can become disordered.

If the brain were to attempt to remember these experiences, then, it may
generate connections between entities that were only coincidentally activated
by chance rather than due to input from the external environment. This could
generate random connections with little predictive power of the external
environment, and so it is reasonable for the brain to de-prioritize the
development of new connections during sleep to prevent the formation of such
spurious connections. This would explain why dreams are difficult to remember;
we do not lay down new pathways during sleep to recall such experiences.

###  5.8 Livewiring and Brain Dysfunction

Assuming livewiring mediated by glial cells plays a significant role in the
development process of the brain, one would expect that brain disorders would
in many cases be related to dysfunctions in the glial cells which help
regulate the brain’s network structure. This expectation matches observations
of brains across a wide variety of disorders, and efforts to better understand
the role of glial dysfunction in disease remains an active area of research
(Barres 2008).

For instance, Alzheimer’s has been found to be associated with the dysfunction
of microglia, possibly involving them eliminating synapses too aggressively
(Mosher & Wyss-Coray 2014). Similarly, overactive pruning of synapses by
microglia and astrocytes have been implicated in Parkinson’s disease (Tremblay
et al. 2019). More generally, dysfunction of glial cells, particularly
astrocytes, has been found to be associated with a broad variety of
neurodegenerative disorders (Oksanen et al. 2019).

The end result of many of these varied disorders is a substantial alteration
to the structure of the connections between neurons, typically involving the
destruction of many of the prior connections. This reinforces the importance
of the network structure of the brain in determining its function.

However, in exploring brain pathologies, potential dysfunctions under distinct
failure modes are not the only source of insight into brain function;
effective treatments can also offer clarity into the core mechanisms upon
which the brain relies.

For instance, it has been found that there is a great deal of variability in
the severity of symptoms experienced by patients with neuronal hypertrophy
consistent with Alzheimer’s; patients with substantial brain lesions who lead
a more active lifestyle tend to develop less severe symptoms. This has led to
speculation that despite the loss of connections caused by disease, the brain
remains able to form new connections to replace the lost ones, thereby
maintaining greater functionality (Iacono et al. 2009).

In the case of severe depression, electroconvulsive therapy, one of the most
effective treatments for severe cases, has been shown to generate increased
dendritic arborization in the dentate gyrus (thought to be responsible for
encoding episodic memories) in animals, and has been shown to update the
functional connections from this region in humans (Takamiya et al. 2019).

From all of these cases of brain dysfunction and crude remedies, we see that
alterations to the livewiring process have a significant impact on brain
behavior. This further reinforces the importance of focusing future research
on ways in which the behavior of glial cells can be better regulated to
maintain healthy brain functionality.

###  5.9 Self-Supervised Learning, Efficient Encoding, and Transfer Learning

Assuming livewiring networks give rise to operations on symbols as argued
above, it is natural to conclude that such models will achieve far more
compact representations of the environment, especially environments that
evolve over time, as operations on symbols allow for far more efficient
transition models for environments with stable entities (see Yi et al. 2018
for an example of such operations on manually defined entities). More
efficient world models should naturally lead to an improved ability to
efficiently encode the most important information in the environment to form
predictions about the future.

Any improvements in the efficiency with which livewired networks represent the
environment should generate corresponding efficiency gains in self-supervised
models which attempt to reason about future world states on the basis of past
observations. In particular, performing operations on efficiently encoded
symbol-like entities should greatly reduce the space of future world states
which must be examined.

To emphasize the importance of the efficiency of representations to enabling
effective inference, it should be noted that even in humans, the ability to
retain information in working memory is constrained by the efficiency with
which information is organized. Organizing such items in working memory
efficiently is referred to as “chunking” (Thalmann et al. 2019).

Moreover, the emphasis livewired networks place on learning latent
representations of recurring entities in the environment may also enable such
models to achieve more efficient multimodal learning (Ngiam et al. 2011)
across different domains. In particular, as the same latent entities in the
environment should persist across sensory modalities (to the extent to which
such entities can be sensed by separate modalities), information from one
modality should enable the identification of latent entities and thus the
projection of future experience of other sensory modalities. Additionally, a
more symbol-driven learning system may enable greater context dependence by
identifying latent entities which influence the interpretation of sensory
inputs.

Perhaps the greatest tool humans use to organize multimodal learning is
language; language by its construction enables humans to develop identical
latent representations of the environment, and to share such representations
with others without the need for directly shared sensory experience. Moreover,
as language is designed to describe the entities in the environment which are
most relevant to human experience, it provides a powerful organizing tool
around which people can structure their experiences. Indeed, translation is
only possible because all languages take advantage of approximately the same
structures in the environment.

Any human-level learning agent which has been exposed to language can thus be
expected to develop a world model which is built around the same language
structure as speakers of that language, organizing internal representations
around the same latent entities as suggested by the language. The importance
of exposure to language can be seen in the impaired development of individuals
who are not exposed to language during early development when the brain is
most plastic (Hall 2017).

Language also provides a tool by which novel experiences can be characterized
in terms of existing knowledge. In particular, learning agents which operate
on symbols should be capable of applying analogies, and language provides a
useful level of abstraction at which this process can be accomplished. This
should greatly facilitate the transfer of learning between problem domains.

Attention models have had great success in advancing state-of-the-art
performance on tasks requiring semantic understanding (Vaswani et al. 2017,
Chaudhari et al. 2019). Assuming livewired networks greatly improve the
efficiency of information retention in such systems, such models should be
expected to become yet more effective.

###  5.10 Attention Models and Online Self-Supervised Learning

The persistence of calcium waves in astrocytes for 5-30 min. (Scemes & Giaume
2006) suggests an agent pursuing online learning should maintain the strongest
activations in individual nodes in the priority queue for future connection
growth well after the initial activation. To understand why this may be
useful, it helps to consider that in operant conditioning, whereby animals can
be taught to associate behavior in response to stimuli with rewards (or
punishment), the schedule of rewards following an operant is highly
significant to learning (Staddon & Cerutti 2003), enabling stimuli which are
temporally close to be recognized as related, with the strength of the
relation dependent upon the time elapsed.

Analogously, in the case of online learning, it will likely be beneficial to
maintain nodes with especially strong activations in the priority queue of
nodes inspected for new connections. To replicate the temporal dynamics
suggested by operant conditioning, it may suffice to apply a constant
forgetting factor hyperparameter to all prior activations.

Alternatively, the persistence of past items for learning could be achieved by
relying on attention mechanisms to carry over information from past periods,
or by using long-term potentiation to amplify the activations of nodes
recently activated strongly. An attention-based approach may better enable
reward information to guide the persistence of the most important information,
though a combination of these approaches may best enable the persistence of
different types of information.

Regardless of the specific mechanism(s) used to achieve learning across time
periods, one constant across these approaches is that learning should be
strongly linked to an agent’s expectation of reward; in humans, learning is
strongly (if complexly) related to emotional state (Tyng et al. 2017).

To roughly characterize how emotions and rewards should drive the retention of
information across time to enable self-supervised learning, there are a few
forces which should regulate learning: 1) curiosity, which directs agents to
retain more information concerning novel domains to learn about potential
sources of rewards or punishments, 2) fear response to cases of punishment,
and 3) anticipatory response to expectation of rewards.

In each of these cases, the time spent reflecting on experiences, and the
extent to which information about such experiences is retained across time,
should be driven by the anticipated rewards or punishment associated with the
experience. For cases of greater reward signal uncertainty, the drive towards
curiosity should incentivize even more resources to be devoted to learning.
This approach is consistent with observations that uncertain rewards generate
even stronger dopamine responses in humans than certain ones (Niv et al.
2005).

To enable this type of behavior, extensive connections should be established
initially between network regions associated with reward and punishment
expectation, attention, and planning given the functional importance of these
links. This is consistent with the architecture of the human brain, where
components associated with the anticipation of rewards and punishments,
particularly the dopaminergic pathways and the limbic system, are tightly
linked to brain regions largely responsible for working memory and planning
(Arias-Carrion et al. 2010, Messe et al. 2014).

###  5.11 Generative Models and Connecting the Dots

As discussed above, learning in a livewired network can be compared to the
search for novel connections between existing symbols. However, the power of
such an approach will not be limited to discriminative models; generative
models should be expected to be able to make use of the interactions between
symbols to form new representations using existing symbols.

In particular, generative models already allow for style transfer between
images (Karras et al. 2019); analogous methods may allow for applying newly
learned knowledge to an existing knowledge base to generate new knowledge,
relying on symbolic structure to follow a chain of reasoning.

Enabling reward signals to guide the time and resources devoted to the
creation of such novel representations is a promising approach, which would be
consistent with the significance of emotional salience to learning (Tyng et
al. 2017).

To provide additional context for such a potential iterative process of
learning and inference, it should be noted that during periods of rest,
including sleep, the hippocampus replays recent experiences, which is
generally acknowledged to be critical for memory consolidation and learning
(Olafsdottir et al. 2018).

###  5.12 Unpacking the Black Box

Assuming livewired neural networks enable relational models to operate on
symbols, we can begin to see how they might be used to build models of the
world that more directly reflect human thought processes. In particular, by
forming associative representations of entities in the environment, livewired
networks may be capable of forming the same relations as humans find useful in
describing the world.

For instance, livewired networks may be able to use their ability to
manipulate symbols to use language as a model of the world. This in turn may
enable livewired networks to connect entities in the environment with the
words used to describe them. This may enable livewired networks to perform
highly efficient transfer learning by exploiting the learned structure of
language to form a model of the aspects of the environment which language
describes.

Moreover, by enabling models to manipulate symbols, especially language, which
correspond directly to internal representations of entities in the
environment, livewired networks may greatly advance the interpretability of
such models. This in turn could make eliminating bias from a model far more
transparent by directly investigating the associations formed by the model and
increasing the ability of such models to generate explanations for their
behavior.

More specifically, by investigating the associational structure of the
entities represented by a livewired network, a clear interpretation of the
biases of the network could be obtained. To overcome undesirable
representations, training data which explicitly contradicts the objectionable
views could be repeated to the model until its representation matches the
desired behavior. To ensure the model is receptive to updating its prior
beliefs, the learning rate and connection formation rate could be temporarily
increased to encourage the model to keep an open mind.

As with human learners, the representations formed by models will come to
reflect the information that is available to them, and which is incentivized
by reward signals. Great care must then be taken to ensure that information is
presented with appropriate context, and that behavior is carefully monitored
to ensure the proper actions are incentivized.

##  6 Conclusion

Here I have illustrated how livewired neural networks may be expected to give
rise to models which operate on symbol-like entities. I showed how the growth
of new connections between existing neurons, paired with a higher learning
rate for newer connections, may enable such livewired networks to achieve few-
shot learning without overwriting previous learning.

Furthermore, I have demonstrated how this livewired approach could give rise
to a number of emergent behaviors in humans, and provided a plausible
biological mechanism by which such livewiring may be accomplished in the
brain.

Finally, I have provided a number of promising directions for further research
pursuing the premise that livewiring presents a substantially novel model of
how the brain adapts to the environment.

## References

  * [1] Adamsky A et al.  2018\.  Astrocytic Activation Generates De Novo Neuronal Potentiation and Memory Enhancement.  Cell, 174, 59–71.e14.
  * [2] Agulhon C, Sun M-Y, Murphy T, Myers T, Lauderdale K, Fiacco TA.  2012\.  Calcium signaling and gliotransmission in normal vs. reactive astrocytes.  In Frontiers in Pharmacology, 3:139.
  * [3] Alet F, Weng E, Lozano-Perez T, Kaelbling LP.  2019\.  Neural Relational Inference With Fast Modular Meta-learning.  In NeurIPS.
  * [4] Araque A, Parpura V, Sanzgiri RP, Haydon PG.  1999\.  Tripartite synapses: glia, the unacknowledged partner.  In Trends in Neurosciences, 22:208–15 108.
  * [5] Arias-Carrion O, Stamelou M, Murillo-Rodriguez E, Menendez-Gonzalez M, Poppel E.  2010\.  Dopaminergic reward system: a short integrative review.  In International Archives of Medicine, 3:24.
  * [6] Azulay A, Weiss Y.  2018\.  Why do deep convolutional networks generalize so poorly to small image transformations?  arXiv preprint, arXiv:1805.12177.
  * [7] Bach-y-Rita P, Kercel SW.  2003\.  Sensory substitution and the human–machine interface.  In Trends in Cognitive Sciences, 7(12):541–46.
  * [8] Baker N, Lu H, Erlikhman G, Kellman PJ.  2018\.  Deep convolutional networks do not classify based on global object shape.  In PLOS Computational Biology, 14(12):e1006613.
  * [9] Barres BA.  2008\.  The mystery and magic of glia: a perspective on their roles in health and disease.  In Neuron, 60:430–40.
  * [10] Battaglia PW et al.  2018\.  Relational inductive biases, deep learning, and graph networks.  arXiv preprint, arXiv:1806.01261.
  * [11] Bazargani N, Attwell D.  2015\.  Astrocyte calcium signalling: the third wave.  In Nature Neuroscience, 19:182–189.
  * [12] Bellec G, Kappel D, Maass W, Legenstein R.  2017\.  Deep Rewiring: Training Very Sparse Deep Networks.  CoRR, abs/1711.05136
  * [13] Bellesi M, de Vivo L, Tononi G, Cirelli C.  2015\.  Effects of sleep and wake on astrocytes: clues from molecular and ultrastructural studies.  In BMC Biology, 13:66.
  * [14] Bengio Y.  2012\.  Practical recommendations for gradient-based training of deep architectures.  In Neural Networks: Tricks of the Trade.  Springer Berlin Heidelberg, pp. 437–478.
  * [15] Bergmann O, Spalding KL, Frisén J.  2015\.  Adult neurogenesis in humans.  In Cold Spring Harbor Perspectives in Biology, 7(7):a018994.
  * [16] Bernardinelli Y, Randall J, Janett E, Nikonenko I, Konig S, Jones EV, Flores CE, Murai KK, Bochet CG, Holtmaat A, Muller D.  2014\.  Activity-dependent structural plasticity of perisynaptic astrocytic domains promotes excitatory synapse stability.  In Current Biology, 24:1679–1688.
  * [17] Berto S, Mendizabal I, Usui N, Toriumi K, Chatterjee P, Douglas C, Tamminga CA, Preuss TM, Yi SV, Konopka G.  2019\.  Accelerated evolution of oligodendrocytes in the human brain.  In Proceedings of the National Academy of Sciences of the United States of America, 116(48):24334–24342.
  * [18] Biederman I, Cooper EE.  1992\.  Size Invariance in Visual Object Priming.  In Journal of Experimental Psychology, 18:1, 121-133.
  * [19] Billaudelle, Cramer B, Petrovici MA, Schreiber K, Kappel D, Schemmel J, Meier K.  2019\.  Structural plasticity on an accelerated analog neuromorphic hardware system.  arXiv preprint, arXiv:1912.12047.
  * [20] Blomstrand F, Aberg ND, Eriksson PS, Hansson E, Ronnback L.  1999\.  Extent of intercellular calcium wave propagation is related to gap junction permeability and level of connexin-43 expression in astrocytes in primary cultures from four brain regions.  In Neuroscience, 92:255–265.
  * [21] Bottou L.  1998\.  Online algorithms and stochastic approximations.  In Online Learning and Neural Networks.
  * [22] Bottou L.  2014\.  From machine learning to machine reasoning.  In Machine Learning, 94 (2), 133–149.
  * [23] Chaudhari S, Polatkan G, Ramanath R, Mithal V.  2019\.  An attentive survey of attention models.  CoRR, abs/1904.02874.
  * [24] Chen L, Bellec G, Vogginger B, Kappel D, Partzsch J, Neumärker F, Höppner S, Maass W, Furber S, Legenstein R, Mayr CG.  2018\.  Efficient Deep Learning on a SpiNNaker 2 Prototype.  In Frontiers in Neuroscience, 12:840.
  * [25] Choudhury ME, Miyanishi K, Takeda H, Islam A, Matsuoka N, Kubo M, Matsumoto S, Kunieda T, Nomoto M, Yano H, Tanaka J.  2019\.  Phagocytic elimination of synapses by microglia during sleep.  In Glia.
  * [26] Collignon O, Dormal G, Albouy G, Vandewalle G, Voss P, Phillips C, Lepore F.  2013\.  Impact of blindness onset on the functional organization and the connectivity of the occipital cortex.  In Brain, 136:2769–2783.
  * [27] Conrad K.  1958\.  Die beginnende Schizophrenie.  Thieme Verlag, Stuttgart, Germany.
  * [28] Dani JW, Chernjavsky A, Smith SJ.  1992\.  Neuronal activity triggers calcium waves in hippocampal astrocyte networks.  In Neuron, 8:429–440.
  * [29] Debanne D.  2004\.  Information processing in the axon.  In Nature Reviews Neurosciences 5:304–316.
  * [30] Devereux BJ, Clarke A, Tyler LK.  2018\.  Integrated deep visual and semantic attractor neural networks predict fMRI pattern-information along the ventral object processing pathway.  In Scientific Reports, 8(1):10636.
  * [31] Dicke U, Roth G.  2016\.  Neuronal factors determining high intelligence.  In Philosophical Transactions of the Royal Society B: Biological Sciences, 371:20150180.
  * [32] Eagleman D.  2020\.  Livewired: The Inside Story of the Ever-Changing Brain.  Pantheon Books.
  * [33] Eldan R, Shamir O.  2016\.  The Power of Depth for Feedforward Neural Networks.  In COLT.
  * [34] Eriksson J, Vogel EK, Lansner A, Bergström F, Nyberg L.  2015\.  Neurocognitive Architecture of Working Memory.  In Neuron, 88:33–46.
  * [35] Evci U, Elsen E, Castro P, Gale T.  2019\.  Rigging the lottery: Making all tickets winners.  arXiv preprint arXiv:1911.11134.
  * [36] Fields RD et al.  2014\.  Glial biology in learning and cognition.  In Neuroscientist, 20: 426–431.
  * [37] Fields RD, Woo DH, Basser PJ.  2015\.  Glial regulation of the neuronal connectome through local and long-distant communication.  In Neuron, 86: 374–386.
  * [38] Foss DJ.  1982\.  A Discourse on Semantic Priming.  In Cognitive Psychology, 14: 590-607.
  * [39] Foster KR, Kokko H.  2009\.  The evolution of superstitious and superstitious-like behaviour.  In Philosophical Transactions of the Royal Society B: Biological Sciences, 276: 31–37.
  * [40] Franceschi L, Niepert M, Pontil M, He X.  2019\.  Learning discrete structures for graph neural networks.  arXiv preprint, arXiv:1903.11960.
  * [41] Frankle J, Carbin M.  2018\.  The lottery ticket hypothesis: finding sparse, trainable neural networks.  arXiv preprint, arXiv:1803.03635
  * [42] Galloway A, Adeluyi A, O’Donovan B, Fisher ML, Rao CN, Critchfield P, Sajish M, Turner JR, Ortinski PI.  2018\.  Dopamine Triggers CTCF-Dependent Morphological and Genomic Remodeling of Astrocytes.  In Journal of Neuroscience, 38: 4846–4858.
  * [43] Gogtay N, Giedd JN, Lusk L, Hayashi KM, Greenstein D, Vaituzis AC, Nugent TF 3rd, Herman DH, Clasen LS, Toga AW, Rapoport JL, Thompson PM.  2004\.  Dynamic mapping of human cortical development during childhood through early adulthood.  In Proceedings of the National Academy of Sciences of the United States of America, 101: 8174–8179.
  * [44] Graybiel AM., Grafton ST.  2015\.  The striatum: where skills and habits meet.  In Cold Spring Harbor Perspectives in Biology, 7(8):a021691.
  * [45] Greff K, van Steenkiste S, Schmidhuber J.  2020\.  On the binding problem in artificial neural networks.  arXiv preprint, arXiv:2012.05208.
  * [46] Grover A, Zweig A, Ermon S.  2017\.  Graphite: Iterative generative modeling of graphs.  In NIPS Bayesian Deep Learning Workshop.
  * [47] Guo Y, Yao A, Chen Y.  2016\.  Dynamic network surgery for efficient DNNs.  CoRR, abs/1608.04493.
  * [48] Haier RJ, et al.  1992\.  Regional glucose metabolic changes after learning a complex visuospatial/motor task: A positron emission tomographic study.  In Brain Research 570: 134–143.
  * [49] Hall WC.  2017\.  What You Don’t Know Can Hurt You: The Risk of Language Deprivation by Impairing Sign Language Development in Deaf Children.  In Maternal and Child Health Journal, 21(5): 961–5.
  * [50] Han X, Chen M, Wang F, Windrem M, Wang S, Shanz S, Xu Q, Oberheim NA, Bekar L, Betstadt S, Silva AJ, Takano T, Goldman SA, Nedergaard M.  2013\.  Forebrain engraftment by human glial progenitor cells enhances synaptic plasticity and learning in adult mice.  In Cell Stem Cell, 12: 342–53.
  * [51] Hassabis D, Kumaran D, Vann SD, Maguire EA.  2007\.  Patients with hippocampal amnesia cannot imagine new experiences.  In Proceedings of the National Academy of Sciences of the United States of America, 104: 1726–1731.
  * [52] Hawkins J, Ahmad S, Cui Y.  2017\.  A theory of how columns in the neocortex enable learning the structure of the world.  In Frontiers in Neural Circuits, 11: 81.
  * [53] He K, Zhang X, Ren S, Sun J.  2016\.  Deep residual learning for image recognition.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770–778.
  * [54] Hebb D.  1949\.  The Organization of Behavior.  Wiley, pp 63.
  * [55] Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov R.  2012\.  Improving neural networks by preventing co-adaptation of feature detectors.  CoRR, abs/1207.0580.
  * [56] Hu J, Shen L, Albanie S, Sun G, Wu E.  2018\.  Squeeze-and-Excitation Networks.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2018): 7132–7141.
  * [57] Huang G, Liu Z, Weinberger KQ.  2017\.  Densely connected convolutional networks.  In IEEE Conference on Computer Vision and Pattern Recognition.
  * [58] Huttenlocher PR, Dabholkar AS.  1997\.  Regional differences in synaptogenesis in human cerebral cortex.  In The Journal of Comparative Neurology, 387(2): 167–178.
  * [59] Iacono D, Markesbery WR, Gross M, Pletnikova O, Rudow G, Zandi P, Troncoso JC.  2009\.  The Nun Study: Clinically silent AD, neuronal hypertrophy, and linguistic skills in early life.  In Neurology.
  * [60] Ioffe S, Szegedy C.  2015\.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift.  arXiv preprint, arXiv:1502.03167v3.
  * [61] Jauregui-Huerta F, Ruvalcaba-Delgadillo Y, Gonzalez-Castaneda R, Garcia-Estrada J, Gonzalez-Perez O, Luquin S.  2010\.  Responses of glial cells to stress and glucocorticoids.  In Current Immunology Reviews, 6:195–204.
  * [62] Johnson D, Blumstein DT, Fowler JH, Haselton MG.  2013\.  The evolution of error: Error management, cognitive constraints, and adaptive decision-making biases.  In Trends in Ecology & Evolution, 28(8):474–81.
  * [63] Juang BH, Rabiner LR.  1990\.  Hidden Markov models for speech recognition.  In Technometric, 33 (3), 251–272.
  * [64] Kanemaru K, Okubo Y, Hirose K, Iino M.  2007\.  Regulation of neurite growth by spontaneous Ca2+ oscillations in astrocytes.  In Journal of Neuroscience, 27(33):8957–8966.
  * [65] Karras T, Laine S, Aila T.  2019\.  A style-based generator architecture for generative adversarial networks.  arXiv preprint, arXiv:1812.04948 .
  * [66] Kerr JN, Greenberg D, Helmchen F.  2005\.  Imaging input and output of neocortical networks in vivo.  In Proceedings of the National Academy of Sciences of the United States of America, 102:14063–14068.
  * [67] Kirkpatrick J, Pascanu R, Rabinowitz N, Veness J, Desjardins G, Rusu AA, Milan K, Quan J, Ramalho T, Grabska-Barwinska A, et al.  2017\.  Overcoming catastrophic forgetting in neural networks.  arXiv preprint, arXiv:1612.00796.
  * [68] Koller D, Friedman N.  2009\.  Probabilistic Graphical Models: Principles and Techniques.  MIT Press.
  * [69] Lara AH, Wallis JD.  2015\.  The role of prefrontal cortex in working memory: A mini review.  In Frontiers in Systems Neuroscience, 9.
  * [70] Larsson G, Maire M, Shakhnarovich G.  2016\.  Fractalnet: Ultra-deep neural networks without residuals.  arXiv preprint, arXiv:1605.07648.
  * [71] Limbacher T, Legenstein R.  2020\.  Emergence of Stable Synaptic Clusters on Dendrites Through Synaptic Rewiring.  In Frontiers in Computational Neuroscience.
  * [72] Liu C, Zoph B, Shlens J, Hua W, Li LJ, Fei-Fei L, Yuille A, Huang J, Murphy K.  2017\.  Progressive neural architecture search.  arXiv preprint, arXiv:1712.00559.
  * [73] Liu S, Mocanu DC, Matavalam ARR, Pei Y, Pechenizkiy M.  2019\.  Sparse evolutionary Deep Learning with over one million artificial neurons on commodity hardware.  arXiv preprint, arXiv:1901.09181 .
  * [74] Longley-Cook LH.  1962\.  An introduction to credibility theory.  In PCAS, 49, 194-221.
  * [75] Mao J, Gan C, Kohli P, Tenenbaum JB, Wu J.  2019\.  The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision.  In International Conference on Learning Representations (ICLR).
  * [76] Markov NT et al.  2014\.  Anatomy of hierarchy: Feedforward and feedback pathways in macaque visual cortex.  In Journal of Comparative Neurology, 522, 225–259.
  * [77] McCulloch W, Pitts W.  1943\.  A logical calculus of the ideas immanent in nervous activity.  In Bulletin of Mathematical Biophysics, 5.
  * [78] Merabet LB, Pascual-Leone A.  2010\.  Neural reorganization following sensory loss: the opportunity of change.  In Nature Reviews Neurosciences, 11:44–52.
  * [79] Messe A, Rudrauf D, Benali H, Marrelec G.  2014\.  Relating structure and function in the human brain: relative contributions of anatomy, stationary dynamics, and non-stationarities.  In PLoS Compututational Biology, 10:e1003530.
  * [80] Ming GL, Song H.  2011\.  Adult neurogenesis in the mammalian brain: significant answers and significant questions.  In Neuron, 26; 70:687.
  * [81] Minsky, M.  1963\.  Steps toward artificial intelligence.  In Computers and Thought, pages 406–450.  McGraw-Hill, New York.
  * [82] Mocanu DC, Mocanu E, Stone P, Nguyen PH, Gibescu M, Liotta A.  2018\.  Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science.  In Nature Communications, 9(1):2383.
  * [83] Mosher KI, Wyss-Coray T.  2014\.  Microglial dysfunction in brain aging and Alzheimer’s disease.  In Biochemical Pharmacology, 88:594–604.
  * [84] Mostafa H, Wang X.  2019\.  Parameter efficient training of deep convolutional neural networks by dynamic sparse reparameterization.  arXiv preprint, arXiv:1902.05967.
  * [85] Natekin A, Knoll A.  2013\.  Gradient boosting machines, a tutorial.  In Frontiers in Neurorobotics, 7:21.
  * [86] Nedergaard M, Rodríguez JJ, Verkhratsky A.  2010\.  Glial calcium and diseases of the nervous system.  In Cell Calcium, 47, 140–149.
  * [87] Ngiam J, Khosla A, Kim M, Nam J, Lee H, Ng AY.  2011\.  Multimodal deep learning.  In Proceedings of the 28th international conference on machine learning, (ICML-11): 689–696.
  * [88] Nicoll RA.  2017\.  A Brief History of Long-Term Potentiation.  In Neuron, 93:281–290.
  * [89] Niv Y, Duff MO, Dayan P.  2005\.  Dopamine, uncertainty and TD learning.  In Behavioral and Brain Functions, 1:6.
  * [90] Oberheim NA, Takano T, Han X, He W, Lin JH, Wang F, Xu Q, Wyatt JD, Pilcher W, Ojemann JG, Ransom BR, Goldman SA, Nedergaard M.  2009\.  Uniquely hominid features of adult human astrocytes.  In Journal of Neuroscience, 29:3276–87.
  * [91] Oksanen M, Lehtonen S, Jaronen M, Goldsteins G, Hämäläinen RH, Koistinaho J.  2019\.  Astrocyte alterations in neurodegenerative pathologies and their modeling in human induced pluripotent stem cell platforms.  In Cellular and Molecular Life Sciences, 76:2739–2760.
  * [92] Olafsdottir HF, Bush D, Barry C.  2018\.  The role of hippocampal replay in memory and planning.  In Current Biology, 28, R37–R50.
  * [93] Parpura V, Haydon PG.  2000\.  Physiological astrocytic calcium levels stimulate glutamate release to modulate adjacent neurons.  In Proceedings of the National Academy of Sciences of the United States of America, 97(15), 8629-8634.
  * [94] Pascual-Leone A, Amedi A, Fregni F, Merabet LB. Merabet.  2005\.  The plastic human brain cortex.  In Annual Review of Neuroscience, 28:377–401.
  * [95] Penfield W, Boldrey E.  1937\.  Somatic Motor And Sensory Representation In The Cerebral Cortex Of Man As Studied By Electrical Stimulation.  In Brain, 60 (4): 389–443.
  * [96] Prasad AA, Pasterkamp RJ.  2009\.  Axon guidance in the dopamine system.  In Advances in Experimental Medicine and Biology, 651:91–100.
  * [97] Qian N.  1999\.  On the momentum term in gradient descent learning algorithms.  In Neural networks: the official journal of the International Neural Network Society, 12(1):145–151.
  * [98] Quiroga RQ, Reddy L, Kreiman G, Koch C, Fried I.  2005\.  Invariant visual representation by single neurons in the human brain.  In Nature.
  * [99] Qureshi A, Rizvi F, Syed A, Shahid A, Manzoor H.  2014\.  The method of loci as a mnemonic device to facilitate learning in endocrinology leads to improvement in student performance as measured by assessments.  In Advances in Physiology Education, 38 (2) pp. 140-144.
  * [100] Raghu M, Schmidt E.  2020\.  A survey of deep learning for scientific discovery.  arXiv preprint, arXiv:2003.11755.
  * [101] Ramón y Cajal S.  1933\.  ¿Neuronismo o reticularismo? Las preubas objetivas de la unidad anatómico de las celulas nerviosas.  In Archivos de Neurobiologia, Madrid 13: 1-144.
  * [102] Rasch B, Born J.  2013\.  About sleep’s role in memory.  In Physiological Reviews, 93, pp. 681-766.
  * [103] Rosenberg SS, Spitzer NC.  2011\.  Calcium signaling in neuronal development.  In Cold Spring Harbor Perspectives in Biology, 3(10):a004259.
  * [104] Rosenblatt F.  1961\.  Principles of neurodynamics: perceptrons and the theory of brain mechanisms.  Washington, D.C.: Spartan Books.
  * [105] Rumelhart DE, Hinton GE, Williams RJ.  1985\.  Learning internal representations by error propagation.  Technical report, DTIC Document.
  * [106] Sabour S, Frosst N, Hinton GE.  2017\.  Dynamic routing between capsules.  In NeurIPS, pages 3859–3869.
  * [107] Salzman CD, Fusi S.  2010\.  Emotion, cognition, and mental state representation in amygdala and prefrontal cortex.  In Annual Review of Neuroscience, 33:173–202.
  * [108] Santello M, Toni N, Volterra A.  2019\.  Astrocyte function from information processing to cognition and cognitive impairment.  In Nature Neuroscience, 22, 154–166.
  * [109] Santoro A, Raposo D, Barrett DGT, Malinowski M, Pascanu R, Battaglia PW, Lillicrap T.  2017\.  A simple neural network module for relational reasoning.  arXiv preprint, arXiv:1706.01427.
  * [110] Scemes E, Giaume C.  2006\.  Astrocyte calcium waves: what they are and what they do.  In Glia, 54:716–725.
  * [111] Shannon CE.  1948\.  A Mathematical Theory of Communication.  In The Bell System Technical Journal, Vol. 27, pp. 379–423, 623–656.
  * [112] Smith LN.  2017\.  Cyclical Learning Rates for Training Neural Networks.  In Applications of Computer Vision (WACV), IEEE Winter Conference on IEEE, pages 464–472.
  * [113] Smith LN, Topin N.  2018\.  Super-convergence: Very fast training of neural networks using large learning rates.  arXiv preprint, arXiv:1708.07120.
  * [114] Spruston N.  2008\.  Pyramidal neurons: dendritic structure and synaptic integration.  In Nature Reviews Neuroscience, 9:206–221.
  * [115] Srivastava RK, Greff K, Schmidhuber J.  2015\.  Training very deep networks.  arXiv preprint, arXiv:1507.06228.
  * [116] Staddon JER, Cerutti DT.  2003\.  Operant conditioning.  In Annual Review of Psychology, 54:115–144.
  * [117] Stanley KO, Miikkulainen R.  2002\.  Evolving neural networks through augmenting topologies.  In Evolutionary Computation, 10:99–127.
  * [118] Sung F, Yang Y, Zhang L, Xiang T, Torr PHS, Hospedales TM.  2018\.  Learning to compare: Relation network for few-shot learning.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
  * [119] Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A.  2014\.  Going deeper with convolutions.  arXiv preprint, arXiv:1409.4842.
  * [120] Takamiya A, Plitman E, Chung JK, Chakravarty M, Graff-Guerrero A, Mimura M, Kishimoto T.  2019\.  Acute and long-term effects of electroconvulsive therapy on human dentate gyrus.  In Neuropsychopharmacology, 44, 1805–1811.
  * [121] Tan DX, Manchester LC, Sanchez-Barcelo E, Mediavilla MD, Reiter RJ.  2010\.  Significance of high levels of endogenous melatonin in Mammalian cerebrospinal fluid and in the central nervous system.  In Current Neuropharmacology, 8(3):162–7.
  * [122] Tertil M et al.  2018\.  Glucocorticoid receptor signaling in astrocytes is required for aversive memory formation.  In Translational Psychiatry, 8, 255.
  * [123] Thalmann M, Souza AS, Oberauer K.  2019\.  How does chunking help working memory?  In Journal of Experimental Psychology: Learning, Memory, and Cognition, 45(1), 37–55.
  * [124] Tierney AL, Nelson CA.  2009\.  Brain Development and the Role of Experience in the Early Years.  In Zero To Three, 30:9–13.
  * [125] Tishby N, Zaslavsky N.  2015\.  Deep learning and the information bottleneck principle.  In ITW, pp. 1–5.  IEEE.
  * [126] Tomasi DG, Shokri-Kojori E, Volkow ND.  2017\.  Brain network dynamics adhere to a power law.  In Frontiers in Neuroscience, 11:72.
  * [127] Tremblay ME, Cookson MR, Civiero L.  2019\.  Glial phagocytic clearance in Parkinson’s disease.  In Molecular Neurodegeneration, 14(1):16.
  * [128] Tyng CM, Amin HU, Saad MNM, Malik AS.  2017\.  The influences of emotion on learning and memory.  In Frontiers in Physiology 8\.
  * [129] Van Steenkiste S, Chang M, Greff K, Schmidhuber J.  2018\.  Relational neural expectation maximization: Unsupervised discovery of objects and their interactions.  In International Conference on Learning Representations (ICLR).
  * [130] Van Strien NM, Cappaert NL, Witter MP.  2009\.  The anatomy of memory: an interactive overview of the parahippocampal-hippocampal network.  In Nature Reviews Neurosciences 10:272–282.
  * [131] Vardjan N, Verkhratsky A, Zorec R.  2017\.  Astrocytic pathological calcium homeostasis and impaired vesicle trafficking in neurodegeneration.  In International Journal of Molecular Sciences 18:358.
  * [132] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I.  2017\.  Attention is all you need.  In Advances in Neural Information Processing Systems, pages 6000–6010.
  * [133] Veening JG, Barendregt HP.  2010\.  The regulation of brain states by neuroactive substances distributed via the cerebrospinal fluid; a review.  In Cerebrospinal Fluid Research 7\.
  * [134] Wang J, Zhang Z, Xie C, Zhou Y, Premachandran V, Zhu J, Xie L, Yuille A.  2017\.  Visual concepts and compositional voting.  arXiv preprint, arXiv:1711.04451.
  * [135] Wu Z, Pan S, Chen F, Long G, Zhang C, Yu PS.  2019\.  A comprehensive survey on graph neural networks.  arXiv preprint, arXiv:1901.00596.
  * [136] Xie S, Kirillov A, Girshick R, He K.  2019\.  Exploring randomly wired neural networks for image recognition.  In Proceedings of the IEEE International Conference on Computer Vision, pp. 1284–1293.
  * [137] Yi K, Wu J, Gan C, Torralba A, Kohli P, Tenenbaum JB.  2018\.  Neural-Symbolic VQA: Disentangling reasoning from vision and language understanding.  In NeurIPS.
  * [138] You J, Leskovec J, He K, Xie S.  2020\.  Graph structure of neural networks.  In ICML.
  * [139] Yun S, Jeong M, Kim R, Kang J, Kim HJ.  2019\.  Graph Transformer Networks.  In NeurIPS.
  * [140] Zappaterra MW, Lehtinen MK.  2012\.  The cerebrospinal fluid: regulator of neurogenesis, behavior, and beyond.  In Cellular and Molecular Life Sciences 69:2863–2878.
  * [141] Zhu M, Gupta S.  2018\.  To Prune, or Not to Prune: Exploring the Efficacy of Pruning for Model Compression.  In International Conference on Learning Representations Workshop.
  * [142] Zoph B, Vasudevan V, Shlens J, Le QV.  2018\.  Learning transferable architectures for scalable image recognition.  In CVPR.
  * [143] Zundorf G, Reiser G.  2011\.  Calcium dysregulation and homeostasis of neural calcium in the molecular mechanisms of neurodegenerative diseases provide multiple targets for neuroprotection.  In Antioxidants & Redox Signaling, 14(7):1275–1288.


# Conditional Relative Risk:
An association measure for longitudinal data analysis

Lina<EMAIL_ADDRESS>Juan<EMAIL_ADDRESS>Oscar<EMAIL_ADDRESS>
Department of Statistics, Faculty of Sciences
Universidad Nacional de Colombia, Bogota, Colombia

###### Abstract

In this paper, we propose a novel association measure for longitudinal studies
based on the traditional definition of relative risk. In a Markovian fashion,
such a proposal takes into account the information content regarding the
previous time. We derive its corresponding confidence interval by means of the
Delta method having in mind the crude association between factor and event.
Also, we study the properties of our uncertainty quantification scheme through
an exhaustive simulation study. Our findings show that the coverage
probability is quite close to the level of confidence. Finally, our proposal
has a reasonable interpretation from the epidemiological as well as the
statistical point of view.

Keywords: Association measures, association studies, longitudinal studies,
relative risk, repeated measurements.

##  1 Introduction

In many longitudinal studies, it is necessary to measure the outcome of
interest on several occasions throughout the follow-up period (Twisk et al.,,
2005). In these cases, traditional crude association measures as well as those
adjusted through linear models (Rothman et al.,, 2008) are typically not
appropriate, since they do not take into account the existing correlation
among measurements corresponding to the same individual.

Traditionally, either cohort or experimental longitudinal studies with
dichotomous outcomes use the relative risk (RR) or the odds ratio (OR) as
measures of association. These measures can be calculated either adjusted or
crude, depending on whether or not confounding variables are taken into
account (Rothman et al.,, 2008; Szklo and Nieto,, 2019). On one side, when
considering crude association, confidence intervals are typically obtained
through a direct application of the Delta method (Morris and Gardner,, 1988).
However, when controlling for confounding variables is in order, confidence
intervals are based on linear model fitting.

Models for longitudinal data are commonly classified into marginal models,
transition models and random effects models (Diggle and Heagerty,, 2002). To
date, there are available several techniques to provide estimates for
parameters in marginal models mainly relying on the generalized estimation
equations (EEG, Liang and Zeger, 1986). These methods have been extended in
several directions, such as those provided by (Lipsitz et al.,, 1991), who
modified the equations in order to find an estimate based on the OR, and also,
considering not fixed by design follow-up times with outcomes that may depend
on previous outcomes (Fitzmaurice et al.,, 2006). Other types of marginal
models have also been developed, such as structural marginal models, which are
closely linked to causal inference approaches in Epidemiology (e.g., Robins et
al., 2000).

In model-based approaches, correlation effects are implicitly included when
estimating model parameters through a user-prespecified covariance matrix, but
not explicitly when estimating association parameters such as the RR or the
OR. Thus, we propose a way to include directly the correlation between the
occurrence of the event in two moments $k$ and $j$, with $k<j$. To do so, we
develop correlated binary data theory in order to develop two brand new
association measures based on the RR. Specifically, we propose two crude
conditional association measures for $2\times 2$ tables, including point
estimates as well as uncertainty quantification (confidence intervals).

This paper is structured as follows: Section 2 develops theory for correlated
binary data, Section 3 derives novel measures of association for $2\times 2$
tables along with the corresponding confidence intervals, Section 4 makes a
comparison between our proposal and traditional RR, Section 5 shows an actual
application with experimental data, and finally, Section 6 discusses findings
and recommendations for future research.

##  2 Theory for correlated binary data

An association measure for longitudinal studies is the relative risk $RR$,
given by

$RR=\frac{\pi_{E}}{\pi_{NE}}$ (1)

where $\pi_{E}$ is the outcome risk between exposed and $\pi_{NE}$ is the
outcome risk between non-exposed.

In this case, we want to establish an association measure between a
dichotomous exposure with a dichotomous outcome at time $j$, conditioned by
the outcome at time $k$, with $k<j$. The conditional probability of occurrence
of the outcome at $j$ is calculated below for two cases, namely, assuming that
the outcome’s probability remains constant over the follow-up period, and
also, assuming that this probability varies over the follow-up period.

###  2.1 The outcome’s probability remains constant over the follow-up period

Let $\bm{Y}=(Y_{1},Y_{2},\ldots,Y_{t})$ be the random vector with the response
variable over the follow-up period, with $Y_{j}\sim\textsf{Bernoulli}(\pi)$,
where $\pi$ is the probability of occurrence of the outcome at time $j$:

$Y_{j}=\begin{cases}1&\text{ if the outcome happens at time $j$}\\\ 0&\text{
if the outcome does not happens at time $j$.}\\\ \end{cases}$

Thus,

$\textsf{E}\left(Y_{j}\right)=\pi\,,\qquad\textsf{Var}\left(Y_{j}\right)=\pi(1-\pi)\,,\qquad\textsf{Cov}\left(Y_{j},Y_{k}\right)=\rho_{jk}\pi(1-\pi)\,,$
(2)

where $\rho_{jk}$ is the correlation between the outcomes at times $j$ and
$k$. Additionally, since
$\textsf{Cov}\left(Y_{j},Y_{k}\right)=\textsf{E}\left(Y_{j}Y_{k}\right)-\textsf{E}\left(Y_{j}\right)\textsf{E}\left(Y_{k}\right)$
and

$\displaystyle\textsf{E}\left(Y_{j}Y_{k}\right)=\sum_{y_{j},y_{k}}{y_{j}y_{k}\Pr(Y_{j}=y_{j},Y_{k}}=y_{k})=\Pr(Y_{j}=1,Y_{k}=1)\,,$
(3)

Then, from equations (2) and (3), it follows that

$\displaystyle\textsf{Cov}\left(Y_{j},Y_{k}\right)=\Pr\left(Y_{j}=1,Y_{k}=1\right)-{\pi}^{2}=\rho_{jk}\pi(1-\pi)\,.$

Now, in the same spirit of Crowder, (1985), we obtain that

$\displaystyle\Pr\left(Y_{j}=1\mid
Y_{k}=1\right)=\frac{\Pr\left(Y_{j}=1,Y_{k}=1\right)}{\Pr\left(Y_{k}=1\right)}=\pi+\rho_{jk}(1-\pi)\,,$

and similarly,

$\Pr\left(Y_{j}=1\mid Y_{k}=0\right)=(1-\rho_{jk})\pi\,.$

Finally, since both $0\leq\Pr\left(Y_{j}=1\mid Y_{k}=1\right)\leq 1$ and
$0\leq\Pr\left(Y_{j}=1\mid Y_{k}=0\right)\leq 1$, then the correlation between
the outcomes at times $j$ and $k$ is bounded in a way that

$\max\left\\{-\frac{\pi}{1-\pi},-\frac{1-\pi}{\pi}\right\\}\leq\rho_{jk}\leq
1\,.$

###  2.2 The outcome’s probability changes over the follow-up period

As of now, we have made the assumption that the outcome’s probability remains
constant over the follow-up period, but usually the individual’s conditions
change over time, thus the outcome’s probability changes too. So, now we let
$Y_{j}\sim\textsf{Bernoulli}(\pi_{j})$, where $\pi_{j}$ is the outcome’s
probability at time $j$.

Under these conditions, we get that

$\textsf{E}\left(Y_{j}\right)=\pi_{j}\,,\qquad\textsf{V}\left(Y_{j}\right)=\pi_{j}(1-\pi_{j})\,,\qquad\textsf{Cov}\left(Y_{j},Y_{k}\right)=\rho_{jk}\sqrt{\pi_{j}(1-\pi_{j})\pi_{k}(1-\pi_{k})}\,.$

Following the same reasoning as before, it follows that

$\Pr\left(Y_{j}=1\mid Y_{k}=1\right)=\pi_{j}+\
\rho_{jk}\sqrt{\frac{\pi_{j}(1-\pi_{j})(1-\pi_{k})}{\pi_{k}}}$ (4)

as well as

$\Pr\left(Y_{j}=1\mid
Y_{k}=0\right)=\pi_{j}-\rho_{jk}\sqrt{\frac{\pi_{j}(1-\pi_{j})\pi_{k}}{1-\pi_{k}}}\,.$
(5)

Once again, since both $0\leq\Pr\left(Y_{j}=1\mid Y_{k}=1\right)\leq 1$ and
$0\leq\Pr\left(Y_{j}=1\mid Y_{k}=0\right)\leq 1$, then the correlation between
the outcomes at times $j$ and $k$ is bounded in a way that

$-\sqrt{\frac{\pi_{j}\pi_{k}}{\left(1-\pi_{j}\right)\left(1-\pi_{k}\right)}}\leq\rho_{jk}\leq\sqrt{\frac{\pi_{k}\left(1-\pi_{j}\right)}{\pi_{j}\left(1-\pi_{k}\right)}}\,,$

and also,

$-\sqrt{\frac{\left(1-\pi_{j}\right)\left(1-\pi_{k}\right)}{\pi_{j}\pi_{k}}}\leq\rho_{jk}\leq\sqrt{\frac{\left(1-\pi_{j}\right)\left(1-\pi_{k}\right)}{\pi_{j}\pi_{k}}}\,.$

##  3 A new association measure

Taking as starting point the classical definition of relative risk (1) at time
$j$, we are able to include the information about what actually happened at
time $k$, with $k<j$. Using the result in (4), the relative risk given that
the outcome occurred at $k$ is defined as:

$\displaystyle{RR}_{1}$ $\displaystyle=\frac{\pi_{Ej\mid
k=1}}{\pi_{\bar{E}j\mid k=1}}=\frac{{\Pr\left(Y_{j}=1\mid
Y_{k}=1\right)}_{E}}{{\Pr\left(Y_{j}=1\mid Y_{k}=1\right)}_{\bar{E}}}$ (6)
$\displaystyle=\frac{\pi_{jE}+\rho_{jkE}\sqrt{\pi_{jE}(1-\pi_{jE})(1-\pi_{kE})/\pi_{kE}}}{\pi_{j\bar{E}}+\rho_{jk\bar{E}}\sqrt{\pi_{j\bar{E}}(1-\pi_{ij\bar{E}})(1-\pi_{k\bar{E}})/\pi_{k\bar{E}}}}\,,$

where $\pi_{Ej|k=1}$ is the outcome’s probability at time $j$ given that the
outcome occurred at $k$ among the exposed, $\pi_{\bar{E}j|k=1}$ is the
outcome’s probability at time $j$ given that the outcome occurred at $k$ among
the non-exposed, $\pi_{jE}$ is the outcome’s probability at time $j$ among the
exposed, $\pi_{j\bar{E}}$ is the outcome’s probability at time $j$ among the
non-exposed, $\rho_{jkE}$ is the correlation between the dichotomous variables
$Y_{j}$ and $Y_{k}$ for the exposed, and finally, $\rho_{jk\bar{E}}$ is the
correlation for the dichotomous variables $Y_{j}$ and $Y_{k}$ for the non-
exposed. Additionally, we define the relative risk at time $k$ given that the
outcome did not occur at time $j$ as:

$\displaystyle{RR}_{0}$
$\displaystyle=\frac{\pi_{Ej|k=0}}{\pi_{\bar{E}j|k=0}}=\frac{{\Pr\left(Y_{j}=1|Y_{k}=0\right)}_{E}}{{\Pr\left(Y_{j}=1|Y_{k}=0\right)}_{\bar{E}}}$
(7)
$\displaystyle=\frac{\pi_{jE}-\rho_{jkE}\sqrt{\pi_{jE}(1-\pi_{jE})\pi_{kE}/(1-\pi_{kE})}}{\pi_{j\bar{E}}-\rho_{jk\bar{E}}\sqrt{\pi_{j\bar{E}}(1-\pi_{j\bar{E}})(\pi_{k\bar{E}})/(1-\pi_{k\bar{E}})}}\,,$

where $\pi_{Ej|k=0}$, $\pi_{\bar{E}j|k=0}$, $\pi_{jE}$, $\rho_{jkE}$, and
$\rho_{jk\bar{E}}$ are defined in a analogous manner as above.

###  3.1 Inference

Typically, it is impossible to work directly with the entire population, and
as a consequence, we need to estimate the population’s parameters $RR_{1}$ and
$RR_{0}$. In this spirit, we can see the problem as a problem with two tables,
namely, one table for the sample individuals for whom the outcome occurred at
time $k$ (Table 1), and another table for the sample individuals for whom the
outcome did not occur at time $k$ (Table 2), with $k<j$.

Table 1: Sampling table at time $j$ given that the outcome occurred at time $k$.  | Outcome (Yes) | Outcome (Not) | Total
---|---|---|---
Exposed (Yes) | $a_{1}$ | $b_{1}$ | $a_{1}+b_{1}=n_{1E}$
Exposed (Not) | $c_{1}$ | $d_{1}$ | $c_{1}+d_{1}=n_{1\bar{E}}$
Table 2: Sampling table at time $j$ given that the outcome did not occur at time $k$.  | Outcome (Yes) | Outcome (Not) | Total
---|---|---|---
Exposed (Yes) | $a_{0}$ | $b_{0}$ | $a_{0}+b_{0}=n_{0E}$
Exposed (Not) | $c_{0}$ | $d_{0}$ | $c_{0}+d_{0}=n_{0\bar{E}}$

This way, the point estimates for $RR_{1}$ and $RR_{0}$ are given by:

$\widehat{RR}_{1}=\frac{\hat{\pi}_{Ej|k=1}}{\hat{\pi}_{\bar{E}j|k=1}}=\frac{a_{1}/n_{1E}}{c_{1}/n_{1\bar{E}}}=\frac{\hat{\pi}_{jE}+\hat{\rho}_{jkE}\sqrt{\hat{\pi}_{jE}(1-\hat{\pi}_{jE})(1-\hat{\pi}_{kE})/\hat{\pi}_{kE}}\
}{\hat{\pi}_{j\bar{E}}+\hat{\rho}_{jk\bar{E}}\sqrt{\hat{\pi}_{j\bar{E}}(1-\hat{\pi}_{ij\bar{E}})(1-\hat{\pi}_{k\bar{E}})/\hat{\pi}_{k\bar{E}}}}$
(8)

and

$\widehat{RR}_{0}=\frac{\hat{\pi}_{Ej|k=0}}{\hat{\pi}_{\bar{E}j|k=0}}=\frac{a_{0}/n_{0E}}{c_{0}/n_{0\bar{E}}}=\frac{\hat{\pi}_{jE}-\hat{\rho}_{jkE}\sqrt{\hat{\pi}_{jE}(1-\hat{\pi}_{jE})\hat{\pi}_{kE}/(1-\hat{\pi}_{kE})}\
}{\hat{\pi}_{j\bar{E}}-\hat{\rho}_{jk\bar{E}}\sqrt{\hat{\pi}_{j\bar{E}}(1-\hat{\pi}_{j\bar{E}})(\hat{\pi}_{k\bar{E}})/(1-\hat{\pi}_{k\bar{E}})}}\,.$
(9)

Furthermore, it is also possible to estimate the correlations $\rho_{jkE}$ and
$\rho_{jk\bar{E}}$ as follows:

${\hat{\rho}}_{ijkE}=\frac{a_{1}n_{0E}-a_{0}n_{1E}}{\sqrt{n_{1E}n_{0E}(a_{1}+a_{0})(b_{1}+b_{0})}}\quad\text{and}\quad{\hat{\rho}}_{ijk\bar{E}}=\frac{c_{1}n_{0\bar{E}}-c_{0}n_{1\bar{E}}}{\sqrt{n_{1\bar{E}}n_{0\bar{E}}(c_{1}+c_{0})(b_{1}+b_{0})}}\,.$

####  3.1.1 Confidence interval

Under the frequentist paradigm, our proposal is based on the probabilistic
distribution of $\ln{RR_{1}}$ and $\ln{RR_{0}}$ for a large sample sizes
$n_{1E}$ and $n_{0E}$. By the central limit theorem, we have that

$\sqrt{n_{1E}}\left(\hat{\pi}_{Ej|k=1}-\pi_{Ej|k=1}\right)\xrightarrow{d}\textsf{N}\left(0,\pi_{Ej|k=1}(1-\pi_{Ej|k=1})\right)\,,$

and therefore, by taking logarithms and applying the Delta method, we finally
get that

$\sqrt{n_{1E}}\left(\ln\hat{\pi}_{Ej|k=1}-\ln\pi_{Ej|k=1}\right)\xrightarrow{d}\textsf{N}\left(0,\frac{1-\pi_{Ej|k=1}}{\pi_{Ej|k=1}}\right)\,.$

Since $\ln{\widehat{RR}_{1}}=\ln\hat{\pi}_{Ej|k=1}-\ln\hat{\pi}_{j\bar{E}}$,
it follows that an asymptotic confidence interval for $\ln{RR_{1}}$ with
$100(1-\alpha)\%$ confidence is given by

$\ln{\widehat{RR}_{1}}\pm
z_{1-\alpha/2}\sqrt{\frac{1-\hat{\pi}_{Ej|k=1}}{n_{1E}\hat{\pi}_{Ej|k=1}}+\frac{1-\hat{\pi}_{\bar{E}j|k=1}}{n_{1\bar{E}}\hat{\pi}_{\bar{E}j|k=1}}}\,,$
(10)

where $z_{1-\alpha/2}$ is the percentile $100(1-\alpha/2)$ of the standard
normal distribution. Similarly, it follows that an asymptotic confidence
interval for $\ln{RR_{0}}$ with $100(1-\alpha)\%$ confidence is given by

$\ln{\hat{RR}_{0}}\pm
z_{1-\alpha/2}\sqrt{\frac{1-\hat{\pi}_{Ej|k=0}}{n_{0E}\hat{\pi}_{Ej|k=0}}+\frac{1-\hat{\pi}_{\bar{E}j|k=0}}{n_{0\bar{E}}\hat{\pi}_{\bar{E}j|k=0}}}\,.$
(11)

Taking exponential in (10) and (11), the corresponding confidence intervals
for $RR_{1}$ and $RR_{0}$ with $100(1-\alpha)\%$ confidence are respectively:

$\hat{RR}_{1}\exp{\left(\pm
z_{1-\alpha/2}\sqrt{\frac{1-\hat{\pi}_{Ej|k=1}}{n_{1E}\hat{\pi}_{Ej|k=1}}+\frac{1-\hat{\pi}_{\bar{E}j|k=1}}{n_{1\bar{E}}\hat{\pi}_{\bar{E}j|k=1}}}\right)}$
(12)

and

$\hat{RR}_{0}\exp{\left(\pm
z_{1-\alpha/2}\sqrt{\frac{1-\hat{\pi}_{Ej|k=0}}{n_{0E}\hat{\pi}_{Ej|k=0}}+\frac{1-\hat{\pi}_{\bar{E}j|k=0}}{n_{0\bar{E}}\hat{\pi}_{\bar{E}j|k=0}}}\right)}\,.$
(13)

####  3.1.2 Coverage probability

The coverage probability $P_{c}$ of a confidence interval is the probability
that the interval contains the true parameter value. Then, an optimal result
occurs when the coverage probability is near to the corresponding confidence,
i.e., $Pc\approx 1-\alpha$, or alternatively, $1-Pc\approx\alpha$. In order to
compute $P_{c}$ for $RR_{1}$ and $RR_{0}$ considering $100(1-\alpha)\%=95\%$
confidence, we simulate 2025 ($3\times 3\times 5\times 5\times 3\times 3$)
scenarios according to the following cases:

  * •

Sample sizes $n_{E}$ and $n_{\bar{E}}$ vary on $\\{500,1000,2000\\}$ ($3\times
3$ possibilities).

  * •

Occurrence probabilities $\pi_{ijE}$ and $\pi_{ij\bar{E}}$ are considered
constant over the follow up period and vary on $\\{0.1,0.3,0.5,0.7,0.9\\}$
($5\times 5$ possibilities).

  * •

Correlations between exposed $\rho_{ijkE}$ and non-exposed $\rho_{ijk\bar{E}}$
are considered as low medium, and high correlation and vary on
$\\{0.1,0.5,0.9\\}$ ($3\times 3$ possibilities).

We use the following procedure to obtain the corresponding coverage
probability en each of the 2025 scenarios:

  1. 1.

Generate every $2\times 2$ table with entries greater than 0 given that both
the outcome occurred and the outcome did not occur at time $k$, with $k<j$.

  2. 2.

For each table, compute the 95% confidence interval for ${RR}_{k}$.

  3. 3.

Identify if the true value of ${RR}_{k}$ lies within the corresponding
interval.

  4. 4.

Compute $P_{c}$ as

$P_{c}=\sum_{t\in T}p_{t}\,,$ (14)

where $p_{t}$ is the probability of occurrence of table $t$ and $T$ is the set
of those tables whose corresponding interval covers the true value of
${RR}_{k}$.

####  3.1.3 Results

Recall that optimal results are obtained when $1-P_{c}\approx\alpha$. We
obtain in all cases that $1-P_{c}<0.09$. Additionally, for the $RR_{1}$
confidence interval:

  1. 1.

When the outcome’s probability for non-exposed is $\pi_{j\bar{E}}=0.1$ (see
Figure 1):

     * •

For $\rho_{jk\bar{E}}=0.1$, $1-P_{c}$ is lower when $\pi_{jE}$ is very low or
very high. Moreover, $1-P_{c}$ is close to $\alpha$ when $\pi_{jE}$ assumes
intermediate values.

     * •

For $\rho_{jk\bar{E}}=0.5$, in general, $1-P_{c}$ is greater than in the
previous case. As $\pi_{jE}$ increases, $1-P_{c}$ slowly increases.

     * •

For $\rho_{jk\bar{E}}=0.9$, $1-P_{c}$ takes the highest values of them all. As
$\pi_{jE}$ increases, $1-P_{c}$ increases up to values close to $0.09$
approximately, except when $\rho_{jkE}=0.9$, where $1-P_{c}$ increases for
small values of $\pi_{jE}$, and then, decreases for large values of
$\pi_{jE}$.

Figure 1: Complement of the coverage probability of $RR_{1}$ confidence
intervals when $\pi_{j\bar{E}}=0.1$.

  2. 2.

When the outcome’s probability for non-exposed is $\pi_{j\bar{E}}=0.5$ (see
Figure 2):

     * •

For $\rho_{jkE}=0.1$, $1-P_{c}<0.05$ when $\pi_{ijE}$ assumes low values. As
$\pi_{ijE}$ increases, $1-P_{c}$ is close to $\alpha$ and remains almost
constant until the highest values of $\pi_{ijE}$ are reached, case in which
$1-P_{c}$ is greater.

     * •

For $\rho_{jkE}=0.5$, in general, $1-P_{c}$ is greater than in the previous
case.

     * •

For $\rho_{jkE}=0.9$, $1-P_{c}$ takes the highest values of them all. When
$\pi_{jE}=0.1$, $1-P_{c}$ decreases down to values close to $0.05$, unless
$\rho_{jk\bar{E}}=0.9$, where $1-P_{c}$ decreases for intermediate values of
$\pi_{jE}$, and then, it increases to higher values of $\pi_{jE}$.

Figure 2: Complement of the coverage probability of $RR_{1}$ confidence
intervals when $\pi_{j\bar{E}}=0.5$.

  3. 3.

When the outcome’s probability for non-exposed is $\pi_{j\bar{E}}=0.9$ (see
Figure 3):

     * •

Results are very similar to those obtained in the previous instance, but we
can see more variability in the results of $1-P_{c}$, depending on the values
of ${n}_{E}$ and $n_{\bar{E}}$.

     * •

Typically, $1-P_{c}$ assumes high values, especially when $\rho_{ijkE}=0.9$.

     * •

Also, $1-P_{c}$ has a substantial decrease for intermediate values of
$\pi_{ijE}$, with a slight increase for highest values of $\pi_{ijE}$.

Figure 3: Complement of the coverage probability of $RR_{1}$ confidence
intervals when $\pi_{j\bar{E}}=0.9$.

In general, with respect to samples sizes, the largest sample sizes of non-
exposed ($n_{\bar{E}}$) are those that produce the values of $1-P_{c}$ close
to $0.05$ and vice-versa, so that the closest value to $0.05$ is obtained when
$({n}_{E},n\bar{E})=(2000,2000)$.

Finally, results for $RR_{0}$ can be interpreted in a similar fashion as
above. See Figures 4, 5, and 6 for details.

Figure 4: Complement of the coverage probability of $RR_{0}$ confidence
intervals when $\pi_{j\bar{E}}=0.1$. Figure 5: Complement of the coverage
probability of $RR_{0}$ confidence intervals when $\pi_{j\bar{E}}=0.5$. Figure
6: Complement of the coverage probability of $RR_{0}$ confidence intervals
when $\pi_{j\bar{E}}=0.9$.

##  4 Comparison with conventional relative risk measures

In order to evidence the effect of considering the correlation with what
happened in the previous moment, we compare conventional relative risk
measures with our proposal.

###  4.1 $RR_{1}$ versus $RR$

There is an important difference between $RR_{1}$ and $RR$. When the outcome’s
probability is low, the strength of the association would be underestimated
with the $RR$. For example, for the case
${(\pi}_{ijE},\pi_{ij\bar{E}},\rho_{ijkE},\rho_{ijk\bar{E}})=(0.1,0.1,0.9,0.1)$,
we have that $RR=1$, indicating that there is no association between the
outcome and the exposure. However, taking into account the correlation for the
individuals who had the event in the previous time, we obtain that
${RR}_{1}=4.8$, which allow us to conclude that an association exists in such
a way that the risk for those exposed is almost 5 times the risk for those
non-exposed (see Figure 7).

Figure 7: $RR_{1}$ versus $RR$.

For small values of the outcome’s probability of the exposed ($\pi_{ijE}$), we
have that $RR<{RR}_{1}$. In general, the traditional association measures tend
to underestimate the association between the outcome and the exposure in
individuals who also had the event at a previous time $k$ (see Figure 7).

Furthermore, $RR_{1}$ reaches the lowest values when the correlation in the
non-exposed is high, and also, when the correlation in the exposed is low,
being the lowest when $(\rho_{ijkE},\rho_{ijk\bar{E}})=(0.1,0.9)$. The highest
values are obtained for high correlations in the exposed and for low
correlations in the non-exposed, being the highest when
$(\rho_{ijkE},\rho_{ijk\bar{E}})=(0.9,0.1)$ (see Figure 7).

###  4.2 $RR_{0}$ versus $RR$

In general, $RR$ underestimates the association between the outcome and the
exposure for individuals who did not have the event in a previous time $k$.
This underestimation becomes more evident as the outcome’s probability in the
exposed $(\pi_{ijE})$ is greater. For example, for the case
$(\pi_{ijE},\pi_{ij\bar{E}},\rho_{ijkE},\rho_{ijk\bar{E}})=(0.9,0.1,0.1,0.9)$,
we have that $RR=9$, which indicates that the risk among the exposed is 9
times the risk among the non-exposed. However, taking into account the
correlation for the individuals who did not have the event at the previous
time, we obtain that ${RR}_{0}=81$, which allow us to conclude that the risk
for the exposed is 81 times the risk for the non-exposed. Such a phenomenon
occurs because when the outcome’s probability in the exposed is high and the
correlation is low, the conditional probability is high ($\pi_{Ex|k=0}=0.81$).
Whereas when the probability of occurrence in the non-exposed is low and the
correlation is high, the probability that the event occurs at time $j$ must be
very low $(\pi_{\bar{E}j|k=0}=0.01)$ (See figure 8).

Figure 8: $RR_{0}$ vs $RR$

Furthermore, ${RR}_{0}$ grows as the outcome’s probability in the exposed
increases, and then, decreases as the outcome’s probability in the non-exposed
increases. Nevertheless, the behavior in terms of correlations is totally
opposite to that of ${RR}_{1}$, since it reaches the lowest value for low
correlations in the non-exposed and high correlations in the exposed, the
lowest being when $(\rho_{ijkE},\rho_{ijk\bar{E}})=(0.9,0.1)$, and the highest
when the correlation in the exposed is low and in the non-exposed is high,
being the highest when $(\rho{ijkE},\rho_{ijk\bar{E}})=(0.1,0.9)$ (See figure
8).

##  5 Application to Contraception Data

These data have been analyzed in Machin et al., (1988), Fitzmaurice et al.,
(2012) as well as Wakefield, (2013). This dataset comes from a randomized
clinical trial, in which 1151 women were randomized to 100 or 150 mg doses of
depot medroxyprogesterone acetate (DMPA). This contraceptive was injected on
the day of randomization and three more doses at 90-day intervals, the follow-
up period was three months after the last injection. The outcome was
amenorrhea and was measured in the 4 visits. In this analysis, the data of the
women with complete information on the outcome in the 4 visits were included
($n=714$), and patients with doses of 150 mg were considered “exposed” and
with 100 mg as “non-exposed”. In general, the probability of amenorrhea
increased over time in both exposed and non-exposed (see Table 3).

Table 3: Risk of amenorrhea ($\%$) for each visit. Dose | Visit 1 | Visit 2 | Visit 3 | Visit 4
---|---|---|---|---
150 mg | $16.1$ | $29.7$ | $47.9$ | $53.5$
100 mg | $17.7$ | $25.5$ | $37.1$ | $51.8$

For the second visit, considering all patients, no association was found
between the dose and the occurrence of amenorrhea. This conclusion is the same
for those who had had amenorrhea on the first visit, but it is the opposite
for those who had not had it on the first visit, since in this case, patients
with higher dose have a greater risk of amenorrhea. On the third visit, when
considering all the patients, an association was found: the higher dose
produced a greater risk of amenorrhea, the conclusion was the same for those
who did not have the outcome on the second visit. The opposite occurred among
those who had amenorrhea on the second visit, since in this case there was no
association, that is, the dose did not modify the risk of the outcome. On the
fourth visit, something similar to the second occurred, no association was
obtained when considering all the patients or those who had not reported
amenorrhea on the third visit. While for those who had reported it, an
association was found, the higher dose produced a greater risk of occurrence
of amenorrhea (see Table 4).

In this application, we can see how the conclusions would change if we take
into account the correlation with previous times. In general, we would say
that the dose does not seem to affect the risk of amenorrhea; however, we see
that this actually occurs for women who have had it before, but for those who
have not, a higher dose does increase the risk to have the outcome. Therefore,
the recommendation would be to be very careful with the dose in those women
without a history of amenorrhea (see Table 4).

Table 4: Estimation, point and $95\%$ confidence interval, of traditional and conditional relative risks ($\%$) for each visit. | Visit 2 | Visit 3 | Visit 4
---|---|---|---
$\hat{RR}$ | $1.17$ | $1.29$ | $1.07$
$95\%\text{ CI}$ | $0.92-1.48$ | $1.08,1.53$ | $0.93,1.23$
$\hat{RR}_{1}$ | $0.93$ | $1.08$ | $0.93$
$95\%\text{ CI}$ | $0.70-1.23$ | $0.93-1.25$ | $0.83-1.03$
$\hat{RR}_{0}$ | $1.39$ | $1.40$ | $1.03$
$95\%\text{ CI}$ | $1.01-1.93$ | $1.07-1.84$ | $0.66-1.39$
$\hat{\rho}_{E}$ | $0.28$ | $0.43$ | $0.58$
$\hat{\rho}_{\bar{E}}$ | $0.41$ | $0.72$ | $0.73$

##  6 Conclusions

In this paper, two brand new approaches to calculate association measures in
longitudinal studies are proposed. We based such proposals on the definition
of risk, taking into account what happened in previous times. We find that the
probability of occurrence of a dichotomous event, conditioned to what happened
at the previous moment, involves the correlation between these variables.

In general, the coverage probability of the proposed confidence intervals for
${RR}_{1}$ is close to the required value, but when the correlation in the
non-exposed is high, this probability takes smaller values. For the proposed
confidence intervals for ${RR}_{0}$, the coverage probability is close to or
higher than the predetermined one.

The conclusions about association may be different depending on whether or not
the correlation between the times is taken into account, showing the
fundamental importance of our proposal in both observational and experimental
studies. As an example of an experimental study, we consider an actual
application in contraception data, in which the difference between the
analysis with the traditional relative risk and our proposals is more than
evident, since in visits two and four, it would have been concluded that the
dose it is not associated with an increased risk of outcome for all women,
$RR=1.17$ $(95\%CI:0.92-1.48)$ and $RR=1.07$ $(95\%CI:0.93-1.23)$,
respectively, when in fact, the risk is greater with a dose of DMPA 150 mg for
those women who have not previously suffered amenorrhea (see $RR_{0}$ in Table
4).

Future research can be oriented to obtain adjusted measures of association
including the correlation with what happened in the previous time, taking into
account confounding factors. Similarly, the possibility remains open to study
further $RR_{1}$ and $RR_{0}$ considering more than one previous measurement.
Furthermore, confidence intervals can be developed for $RR_{1}$ and $RR_{0}$
when the exposure is not constant over time, and also, when confounding
variables are present. Finally, estimating $RR_{1}$ and $RR_{0}$ in the
context of multilevel modeling is a possibility.

## Statements and Declarations

The authors declare that they have no known competing financial interests or
personal relationships that could have appeared to influence the work reported
in this article.

## References

  * Crowder, (1985) Crowder, M. (1985).  Gaussian estimation for correlated binomial data.  Journal of the Royal Statistical Society: Series B (Methodological), 45(2):229–237.
  * Diggle and Heagerty, (2002) Diggle, P. and Heagerty, P. (2002).  Analysis of longitudinal data.  Oxford University Press, Oxford.
  * Fitzmaurice et al., (2006) Fitzmaurice, G., Lipsitz, S., Ibrahim, J., Gelber, R., and Lipshultz, S. (2006).  Estimation in regression models for longitudinal binary data with outcome-dependent follow-up.  Biostatistics, 48(3):469–485.
  * Fitzmaurice et al., (2012) Fitzmaurice, G. M., Laird, N. M., and Ware, J. H. (2012).  Applied longitudinal analysis.  John Wiley & Sons, New Jersey.
  * Liang and Zeger, (1986) Liang, K. and Zeger, S. (1986).  Longitudinal data analysis using generalized linear models.  Biometrika, 73(1):13–22.
  * Lipsitz et al., (1991) Lipsitz, S., Laird, N., and Harrington, D. (1991).  Estimation in regression models for longitudinal binary data with outcome-dependent follow-up.  Biometrika, 78(1):153–160.
  * Machin et al., (1988) Machin, D., Farley, T. M., Busca, B., Campbell, M. J., and d’Arcangues, C. (1988).  Assessing changes in vaginal bleeding patterns in contracepting women.  Contraception, 38(2):165–179.
  * Morris and Gardner, (1988) Morris, J. A. and Gardner, M. J. (1988).  Statistics in medicine: Calculating confidence intervals for relative risks (odds ratios) and standardised ratios and rates.  British medical journal, 296(6632):1313.
  * Robins et al., (2000) Robins, J., Hernán, M., and Brumback, B. (2000).  Marginal structural models and causal inference in epidemiology.  Epidemiology, 11(5):550–560.
  * Rothman et al., (2008) Rothman, K. J., Greenland, S., Lash, T. L., et al. (2008).  Modern epidemiology, volume 3.  Wolters Kluwer Health/Lippincott Williams & Wilkins Philadelphia.
  * Szklo and Nieto, (2019) Szklo, M. and Nieto, F. J. (2019).  Epidemiology: Beyond the Basics.  Jones & Bartlett Publishers, Burlington.
  * Twisk et al., (2005) Twisk, J. W. R., Smidt, N., and de Vente, W. (2005).  Applied analysis of recurrent events: a practical overview.  Journal of Epidemiology & Community Health, 59(8):706–710.
  * Wakefield, (2013) Wakefield, J. (2013).  Bayesian and Frequentist Regression Methods.  Springer, New York.


# Deep Image prior with StruCtUred Sparsity (DISCUS) for dynamic MRI
reconstruction

###### Abstract

High-quality training data are not always available in dynamic MRI. To address
this, we propose a self-supervised deep learning method called deep image
prior with structured sparsity (DISCUS) for reconstructing dynamic images.
DISCUS is inspired by deep image prior (DIP) and recovers a series of images
through joint optimization of network parameters and input code vectors.
However, DISCUS additionally encourages group sparsity on frame-specific code
vectors to discover the low-dimensional manifold that describes temporal
variations across frames. Compared to prior work on manifold learning, DISCUS
does not require specifying the manifold dimensionality. We validate DISCUS
using three numerical studies. In the first study, we simulate a dynamic
Shepp-Logan phantom with frames undergoing random rotations, translations, or
both, and demonstrate that DISCUS can discover the dimensionality of the
underlying manifold. In the second study, we use data from a realistic late
gadolinium enhancement (LGE) phantom to compare DISCUS with compressed sensing
(CS) and DIP and to demonstrate the positive impact of group sparsity. In the
third study, we use retrospectively undersampled single-shot LGE data from
five patients to compare DISCUS with CS reconstructions. The results from
these studies demonstrate that DISCUS outperforms CS and DIP and that
enforcing group sparsity on the code vectors helps discover true manifold
dimensionality and provides additional performance gain.

Index Terms—  Dynamic MRI, unsupervised learning, manifold learning, deep
image prior (DIP)

##  1 Introduction

Magnetic resonance imaging (MRI) is a routine diagnostic tool with a broad
range of clinical applications. Long scan times, however, are a major hurdle
in MRI as they can reduce patient comfort, decrease throughput, and make
imaging susceptible to motion artifacts. Consequently, accelerating MRI has
been an active area of research [1]. More recently, deep learning (DL)-based
methods have shown great promise for MRI reconstruction. Several recent
studies suggest that such methods can outperform sparsity-based compressed
sensing (CS) methods in terms of image quality [2].

Most DL-based image reconstruction methods require high-quality fully sampled
data for supervised training. However, such data are typically not available
for dynamic MRI. In contrast, self-supervised methods do not rely on fully
sampled data, making them an attractive option for dynamic imaging. Several
self-supervised methods have been proposed for MRI reconstruction, including
deep image prior (DIP) [3]. DIP utilizes the network structure as an implicit
prior. For dynamic imaging, Yoo et al. employed DIP by training a generative
network to map a low-dimensional manifold to an image series [4]. In another
study, Hamilton et al. integrated low-rank constraint with DIP to facilitate
accelerated free-breathing cardiac imaging [5]. Bell et al. proposed a more
robust adaptation of DIP by training the network to function as a denoiser
instead of a generator [6].

In this work, we propose an extension of DIP, called deep image prior with
structured sparsity (DISCUS). DISCUS trains a single network to map a series
of random code vectors to a series of images. In contrast to methods that
require specifying the dimensionality of the manifold [4], which may not known
in advance, DISCUS discovers the dimensionality by imposing group sparsity on
the code vectors. Also, DISCUS does not assume that temporal closeness (order
of acquisition) is tied to image similarity [7]. This makes DISCUS suitable
for free-breathing single-shot applications, where consecutive frames are not
necessarily more similar. In the subsequent sections, we describe DISCUS and
validate it using simulated and measured data.

##  2 Methods

In this section, we first briefly describe DIP and then use that as a starting
point to explain DISCUS.

###  2.1 DIP

DIP can be used to solve a wide range of inverse problems [3]. To apply DIP to
MRI reconstruction, a random code vector $\bm{z}$ is passed through a network
$\bm{G}_{\bm{\theta}}$ to generate an estimate of the true image
$\bm{x}\in{\mathbb{C}}^{N}$. Both the network parameters, $\bm{\theta}$, and
the code vector, $\bm{z}$, are optimized to make the output of the network
consistent with the measured k-space data $\bm{y}\in{\mathbb{C}}^{M}$ through
a known forward operator $\bm{A}\in{\mathbb{C}}^{M\times N}$. This
optimization process can be expressed as

$\displaystyle\hat{\bm{z}},\hat{\bm{\theta}}=\operatorname*{arg\,min}_{\bm{z},\bm{\theta}}{\left\|\bm{A}\bm{G}_{\bm{\theta}}(\bm{z})-\bm{y}\right\|_{2}^{2}}.$
(1)

Once trained, the final image is recovered by
$\hat{\bm{x}}=\bm{G}_{\hat{\bm{\theta}}}(\hat{\bm{z}})$.

###  2.2 DISCUS

The high-level description of DISCUS is provided in Fig. 1. DISCUS attempts to
construct an image series with $T$ frames by solving the following
optimization problem:

$\displaystyle\hat{\bm{z}}_{0},\hat{\bm{z}}_{t},\hat{\bm{\theta}}=\operatorname*{arg\,min}_{\bm{z}_{0},\bm{z}_{t},\bm{\theta}}{\sum_{t=1}^{T}{\left\|\bm{A}_{t}\bm{G}_{\bm{\theta}}(\bm{z}_{0},\bm{z}_{t})-\bm{y}_{t}\right\|_{2}^{2}}+\lambda\left\|\bm{z}_{:}\right\|_{2,1}}$
(2)

where $\bm{y}_{t}\in{\mathbb{C}}^{M}$, $\bm{x}_{t}\in\mathbb{C}^{N}$, and
$\bm{A}_{t}\in\mathbb{C}^{M\times N}$ are measured data, underlying image, and
forward operator for the $t^{\text{th}}$ frame, respectively. Also,
$\bm{z}_{0}\in{\mathbb{R}}^{kN}$ and $\bm{z}_{t}\in{\mathbb{R}}^{N}$ represent
static and dynamic code vectors, respectively, for a user-defined $k>0$. Here,
$\left\|\bm{z}_{:}\right\|_{2,1}={\sum_{n=1}^{N}\sqrt{\sum_{t}z_{t}^{2}[n]}}$
is a hybrid $\ell_{2}$-$\ell_{1}$ norm that first computes elementwise
$\ell_{2}$ norm along the time dimension followed by the $\ell_{1}$ norm along
the remaining dimensions. Here, $z_{t}[n]$ represents the $n^{\text{th}}$
element of $\bm{z}_{t}$.

$\\{\bm{z}_{t}\\}_{t=1}^{T}$$\bm{z}_{0}$$\bm{G}_{\bm{\theta}}$$\\{\bm{x}_{t}\\}_{t=1}^{T}$
Fig. 1: Layout of the DISCUS framework. A CNN-based network
$\bm{G}_{\bm{\theta}}$ takes one static ($\bm{z}_{0}$) and one dynamic
($\bm{z}_{t}$) code vector to generate an estimate of frame $\bm{x}_{t}$.

The static code vector $\bm{z}_{0}$ is common across all frames, while the
dynamic code vector $\bm{z}_{t}$ is specific to the $t^{\text{th}}$ image.
Code vectors $\bm{z}_{0}$ and $\bm{z}_{t}$ are concatenated along the channel
dimension and fed into the generator, $\bm{G}_{\bm{\theta}}$. Once the network
is trained,
$\hat{\bm{x}}_{t}=\bm{G}_{\hat{\bm{\theta}}}(\hat{\bm{z}}_{0},\hat{\bm{z}}_{t})$
produces the final reconstructed image for the $t^{\text{th}}$ frame.

###  2.3 Manifold learning with group sparsity

For dynamic MRI, the reconstruction methods based on manifold learning assume
that the temporal changes due to respiration or other physiological motions
reside on a low-dimensional manifold [8]. Manifold learning can be used in the
context of DIP [7]. This is achieved by restricting the dimensionality of the
input code vectors to the dimensionality of the manifold and then training the
network to map this low-dimensional input to an image series. However, this
approach requires that the dimensionality of the manifold is known precisely,
which is not always the case for MRI. Changes in image contrast due to RF
timing imperfections, heart rate variations, through-plane motion, and bulk
motion can alter the dimensionality of the underlying manifold from its
nominal value.

In DISCUS, we discover the underlying manifold of image series using a hybrid
$\ell_{2}$-$\ell_{1}$ norm that promotes group sparsity (the second term in
Eq. 2) [9]. The group sparsity encourages all dynamic code vectors to have a
support that is not only sparse but also common across all $T$ code vectors;
this support (number of non-zero entries) represents the dimensionality of the
underlying manifold. In this work, we do not enforce additional constraints,
such as temporal smoothness, on the code vectors [7] because such constraints
do not apply to all dynamic applications, including free-breathing single-shot
late gadolinium enhancement (LGE).

##  3 Experiments and results

###  3.1 Phantom study

In this study, we demonstrate that DISCUS can discover the low-dimensional
manifold. To this end, we simulated a 2D Shepp-Logan phantom of size
$128\times 128$. Then, we created three different image series. In the first
series, we generated $T=64$ frames, each with a random rotation within $\pm
3^{\circ}$ with respect to the first frame. In the second series, we generated
$T=64$ frames, each with a random horizontal translation within $\pm 3$ pixels
with respect to the first frame. In the last series, we generated $T=64$
frames, each with a random rotation and translation. To simulate single-coil
k-space data, each frame was Fourier transformed and downsampled using the
mask shown in Fig. 2b.

Each of the three image series was reconstructed using CS, low-rank + sparse
(L+S) [10], and DISCUS. For CS, we imposed $\ell_{1}$-norm minimization in the
spatial wavelet domain. For both CS and L+S, we performed an exhaustive grid
search to fine-tune free parameters. For DISCUS, we used a 7-layer U-Net [3]
and tuned the parameters such as $\lambda$, learning rate, and number of
iterations. The tuning parameters across all methods were optimized based on
normalized mean squared error (NMSE) defined in dB.

As seen in Fig. 2a, DISCUS was able to discover the true manifold
dimensionality, defined by the size of the support (non-zero entries) of
dynamic code vectors. For the image series with only rotations, only
translations, and both rotations and translations, the manifold dimensionality
discovered by DISCUS was 1, 1, and 2, respectively. However, we did not
observe the locations of the non-zero entries to be of any significance. Table
1 presents NMSE and structural similarity index (SSIM) values for the three
image series, with DISCUS outperforming other methods by a wide margin.

(b)TrueCSL+SDISCUS(a) Fig. 2:  (a) One of the dynamic code vectors $\\{\hat{\bm{z}}_{t}\\}_{t=1}^{T}$ from image series with rotations (left), translations (middle), and rotations+translations (right). The number of non-zero entries matches the true dimensionality in each image series. (b) One representative frame from the image series with rotations+translations. The first image in the second row presents the sampling pattern (frames: left-right, phase-encoding: top-bottom, frequency-encoding: not shown), and the remaining images show error maps after fivefold amplification. Metric | Rotation | Translation | Rotation + translation
---|---|---|---
CS | L+S | DISCUS | CS | L+S | DISCUS | CS | L+S | DISCUS
SSIM | 0.883 | 0.831 | 0.960 | 0.883 | 0.835 | 0.962 | 0.882 | 0.834 | 0.922
NMSE | -23.82 | -20.30 | -31.00 | -23.75 | -20.54 | -30.70 | -23.73 | -19.20 | -28.67
Table 1: NMSE and SSIM values from the Shepp-Logan phantom study.

###  3.2 LGE simulation study

In this study, we simulated six distinct LGE image series from the MRXCAT
phantom [11]. Each image series was cropped to $112\times 168$ pixels and had
$T=32$ frames with $4$ receive coils. All data were retrospectively
downsampled at $R=4$ using a realistic golden ratio offset (GRO) Cartesian
sampling mask with fully sampled readout [12]. The parameters defining tissue
contrast, including proton density, T1, inversion time, and gadolinium
concentration, were selected to match routine LGE scans at our institute. To
incorporate breathing-induced motion, a unique breathing pattern was simulated
for each image series. To make the study more realistic, a myocardial scar was
simulated in three of the six image series. White Gaussian noise was added to
the k-space data to yield a signal-to-noise ratio (SNR) of 25 dB. The coil-
combined reference from the fully sampled data was used for the quantitative
assessment of different methods. The network in DISCUS was trained for 12,000
iterations, requiring 90 minutes on a single NVIDIA RTX3090 GPU.

CS L+S DIP DISCUS-GS DISCUS SSIM 0.905 0.922 0.846 0.958 0.978 NMSE -24.25
-22.44 -21.66 -27.06 -28.03

Table 2: NMSE and SSIM values for the simulated LGE data. The numbers are
averaged over six distinct image series. TrueCSL+SDIPDISCUS-GSDISCUS Fig. 3:
Top row: A representative frame from a simulated LGE image series with a
pronounced myocardial scar. Bottom row: sampling mask (frames: left-right,
phase-encoding: top-bottom, frequency-encoding: not shown) and $\times 8$
error maps.

We performed reconstructions using CS, L+S, DIP, DISCUS, and DISCUS without
group sparsity (DISCUS-GS). These methods were compared in terms of NMSE and
SSIM. The results are summarized in Table 2. DISCUS outperforms CS, L+S, and
DIP by a wide margin for both NMSE and SSIM. These results also show that
removing group sparsity worsens the performance by approximately one dB. Fig.
3 shows a representative frame from one of the image series. Compared to
DISCUS, CS exhibits excessive blurring around the myocardial scar, DIP
exhibits excessive noise, and L+S exhibits a more structured error map.

###  3.3 LGE patient study

Five free-breathing image series, each with $T=32$ frames, were collected from
clinical patients on a commercial 1.5T scanner (MAGNETOM Sola, Siemens
Healthcare, Erlangen, Germany) using phase-sensitive inversion recover LGE
sequence [13]. To enable acquisition without acceleration, i.e., $R=1$, the
data were collected with a temporal footprint of 249.6 ms and a compromised
in-plane spatial resolution of $2.4\times 2.9$ mm2. The data were compressed
to eight virtual coils, and the coil sensitivities were estimated using
ESPIRiT [14]. Each image series was retrospectively undersampled using the GRO
sampling mask at $R=2,3,$ and $4$ [12]. The coil-combined reference from the
fully sampled data was used for the quantitative assessment of different
methods. The network in DISCUS was trained for 12,000 iterations, requiring 75
minutes on a single NVIDIA RTX3090 GPU.

Metric | R=2 | R=3 | R=4
---|---|---|---
CS | L+S | DISCUS | CS | L+S | DISCUS | CS | L+S | DISCUS
SSIM | 0.977 | 0.984 | 0.982 | 0.958 | 0.983 | 0.980 | 0.933 | 0.980 | 0.978
NMSE | -23.31 | -20.21 | -25.03 | -20.00 | -19.49 | -23.53 | -17.78 | -19.08 | -22.86
Table 3: NMSE and SSIM values for the patient LGE data. The numbers are
averaged over five image series.

TrueCSL+SDISCUS Fig. 4: Top row: A representative frame from one of the
patients at $R=4$. Bottom row: sampling mask (frames: left-right, phase-
encoding: top-bottom, frequency-encoding: not shown) and $\times 5$ error
maps.

Table 3 compares DISCUS with CS and L+S at three different acceleration rates.
In terms of NMSE, DISCUS offers a significant advantage over CS and L+S at all
three acceleration rates, while L+S marginally outperforms DISCUS in terms of
SSIM. Fig. 4 shows a representative frame from one of the image series at
$R=4$. In this example, DISCUS is able to preserve fine details, while CS
shows excessive smoothing and blocky artifacts and L+S shows a significant
artifact inside the left ventricle, as highlighted by red arrows.

##  4 Conclusions

We have proposed a self-supervised reconstruction method, DISCUS, for dynamic
MRI. A unique feature of DISCUS is that it learns the underlying manifold by
promoting group sparsity among frame-specific code vectors. This feature
enables DISCUS to discover the low-dimensional manifold even when the manifold
dimensionality is not known in advance. Using simulated and measured MRI data
for free-breathing single-shot LGE, we demonstrate that DISCUS outperforms CS
and DIP in terms of image quality. Our future efforts will focus on applying
DISCUS to prospectively undersampled data and to applications other than LGE.

##  5 Additional Information

Acknowledgment: This work was funded by NIH grants R01HL135489, R01HL151697,
and R01EB029957.

Compliance with ethical standards: This study was performed in line with the
principles of the Declaration of Helsinki. Approval was granted by the
Institutional Review Board (IRB) of The Ohio State University (2019H0076).

## References

  * [1] Saiprasad Ravishankar, Jong Chul Ye, and Jeffrey A. Fessler,  “Image reconstruction: From sparsity to data-adaptive methods and machine learning,”  Proceedings of the IEEE, vol. 108, no. 1, pp. 86–109, 2020.
  * [2] Jure Zbontar, Florian Knoll, Anuroop Sriram, Matthew J Muckley, Mary Bruno, Aaron Defazio, Marc Parente, Krzysztof J Geras, Joe Katsnelson, Hersh Chandarana, et al.,  “fastMRI: An open dataset and benchmarks for accelerated MRI,”  arXiv:1811.08839, 2018.
  * [3] Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky,  “Deep image prior,”  in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 9446–9454.
  * [4] Jaejun Yoo, Kyong Hwan Jin, Harshit Gupta, Jerome Yerly, Matthias Stuber, and Michael Unser,  “Time-dependent deep image prior for dynamic MRI,”  IEEE Transactions on Medical Imaging, vol. 40, no. 12, pp. 3337–3348, 2021.
  * [5] Jesse I Hamilton, William Truesdell, Mauricio Galizia, Nicholas Burris, Prachi Agarwal, and Nicole Seiberlich,  “A low-rank deep image prior reconstruction for free-breathing ungated spiral functional CMR at 0.55 T and 1.5 T,”  Magnetic Resonance Materials in Physics, Biology and Medicine, pp. 1–14, 2023.
  * [6] Evan Bell, Shijun Liang, Qing Qu, and Saiprasad Ravishankar,  “Robust self-guided deep image prior,”  in 2023 IEEE International Conference on Acoustics, Speech and Signal Processing, 2023, pp. 1–5.
  * [7] Qing Zou, Abdul Haseeb Ahmed, Prashant Nagpal, Stanley Kruger, and Mathews Jacob,  “Dynamic imaging using a deep generative storm (Gen-SToRM) model,”  IEEE Transactions on Medical Imaging, vol. 40, no. 11, pp. 3102–3112, 2021.
  * [8] Sunrita Poddar and Mathews Jacob,  “Dynamic MRI using smoothness regularization on manifolds (SToRM),”  IEEE Transactions on Medical Imaging, vol. 35, no. 4, pp. 1106–1115, 2015.
  * [9] Ming Yuan and Yi Lin,  “Model selection and estimation in regression with grouped variables,”  Journal of the Royal Statistical Society Series B: Statistical Methodology, vol. 68, no. 1, pp. 49–67, 2006.
  * [10] Ricardo Otazo, Emmanuel Candes, and Daniel K Sodickson,  “Low-rank plus sparse matrix decomposition for accelerated dynamic MRI with separation of background and dynamic components,”  Magnetic resonance in medicine, vol. 73, no. 3, pp. 1125–1136, 2015.
  * [11] Lukas Wissmann, Claudio Santelli, William P Segars, and Sebastian Kozerke,  “MRXCAT: Realistic numerical phantoms for cardiovascular magnetic resonance,”  Journal of Cardiovascular Magnetic Resonance, vol. 16, no. 1, pp. 1–11, 2014.
  * [12] Mihir Joshi, Aaron Pruitt, Chong Chen, Yingmin Liu, and Rizwan Ahmad,  “Technical report (v1.0)–pseudo-random cartesian sampling for dynamic MRI,”  arXiv preprint arXiv:2206.03630, 2022.
  * [13] Peter Kellman, Andrew E Arai, Elliot R McVeigh, and Anthony H Aletras,  “Phase-sensitive inversion recovery for detecting myocardial infarction using gadolinium-delayed hyperenhancement,”  Magnetic Resonance in Medicine, vol. 47, no. 2, pp. 372–383, 2002.
  * [14] Peter J Shin, Peder EZ Larson, Michael A Ohliger, Michael Elad, John M Pauly, Daniel B Vigneron, and Michael Lustig,  “Calibrationless parallel imaging reconstruction based on structured low-rank matrix completion,”  Magnetic Resonance in Medicine, vol. 72, no. 4, pp. 959–970, 2014.