# Enabling High-Level Machine Reasoning with Cognitive Neuro-Symbolic Systems

Alessandro Oltramari

###### Abstract

High-level reasoning can be defined as the capability to generalize over
knowledge acquired via experience, and to exhibit robust behavior in novel
situations. Such form of reasoning is a basic skill in humans, who seamlessly
use it in a broad spectrum of tasks, from language communication to decision
making in complex situations. When it manifests itself in understanding and
manipulating the everyday world of objects and their interactions, we talk
about common sense or commonsense reasoning. State-of-the-art AI systems don’t
possess such capability: for instance, Large Language Models have recently
become popular by demonstrating remarkable fluency in conversing with humans,
but they still make trivial mistakes when probed for commonsense competence;
on a different level, performance degradation outside training data prevents
self-driving vehicles to safely adapt to unseen scenarios, a serious and
unsolved problem that limits the adoption of such technology. In this paper we
propose to enable high-level reasoning in AI systems by integrating cognitive
architectures with external neuro-symbolic components. We illustrate a hybrid
framework centered on ACT-R, and we discuss the role of generative models in
recent and future applications.

## Introduction

A large part of neuro-symbolic systems is based on transforming symbolic
knowledge into sub-symbolic representations that are suitable for infusion in
data-driven learning algorithms: Knowledge Graph Embedding (kge), among the
others, is a prominent approach to reduce knowledge graph (kg) triples to
latent vectors (Wang et al. 2017). Such transformation is instrumental to
efficient computability of kg properties, as well as to application in a
variety of downstream tasks: for instance, in (Wickramarachchi, Henson, and
Sheth 2023) the authors leverage kge methods to label unseen entities in
autonomous driving datasets. Whether the kge process is realized by geometric,
tensor or deep learning models, the purpose is to compress kg structures into
a low-dimensional space, where symbolic statements are replaced with dense,
sub-symbolic expressions. Concatenation, non-linear mapping, attention-like
mechanisms, gating mechanisms, are further methods to adapt knowledge
structures to neural computations – e.g., (Peters et al. 2017; Strub et al.
2018; Margatina, Baziotis, and Potamianos 2019).
While knowledge-infusion can improve neural models, it is not sufficient to
enable high-level reasoning, which is typically required by complex tasks such
as natural language understanding, activity recognition, decision making in
complex scenarios: latent, sub-symbolic expressions can only augment training
signals with features derived from explicit semantic content, but this
infusion process does neither carry any information about the reasoning
mechanisms needed to process the learned knowledge, nor instruct the neural
models on how those should unfold.
But, what do we mean with high-level reasoning and why is it important to
endow artificial intelligent systems with such feature?

## Problem statement

We can define high-level reasoning as the capability to generalize over
knowledge acquired via direct or mediated experience, and to exhibit robust
behavior in novel situations. This definition is inspired by Kahneman’s system
2 mode of thought (Kahneman 2011). When high-level reasoning manifests itself
in understanding and manipulating the everyday world of objects and their
interactions, we talk about common sense or commonsense reasoning. State-of-
the-art AI systems don’t possess such capability: for instance, Large Language
Models (llms) have recently become popular by demonstrating remarkable fluency
in conversing with humans, but they still make trivial mistakes when probed
for commonsense competence (see next section); on a different level, one of
the motivations why the promise of autonomous cars hasn’t panned out yet
concerns performance degradation outside training data, which prevents self-
driving vehicles to safely adapt to unseen scenarios.111A main weakness of
deep learning approaches, as stated in a recent article (Bengio et al. 2019),
is that ‘current methods seem weak when they are required to generalize beyond
the training distribution, which is what is often needed in practice’, such as
in safely maneuvering a vehicle. Humans, on the opposite, are very good at
generalizing from a few examples, and at filling the gaps in experience with
reasoning: for instance, when asked about what happens after a bottle of red
wine is thrown against a concrete wall, even children can answer with the
utmost certainty that the bottle will shatter and the wall will be wet and
red-stained – they can also easily infer that the impact between any fragile
material and any hard surface typically ends with the former being
substantially altered, if not destroyed; analogously, student drivers only
need limited training to learn how to safely maneuver a car, adapting their
knowledge and skills to novel situations. Compared to current AI systems based
on GPU accelerated computing, human reasoning capabilities are impressive,
even more so when we factor in what Herbert Simon used to call ‘bounded
rationality’ (Simon 1955), i.e., the notion that human cognition operates with
limited knowledge and is subject to time constraints – a heritage of evolution
(Santos and Rosati 2015).
As these arguments suggest, a cognitive stance toward designing AI systems
(Lieto 2021) seems to be key to enable high-level reasoning capabilities at
the computational level: accordingly, we propose to complement cognitive
architectures (Kotseruba and Tsotsos 2020; Langley, Laird, and Rogers 2009)
with neuro-symbolic methods. In this paper we illustrate the blueprints of a
cognitive neuro-symbolic reasoning system centered on the ACT-R222Abbreviation
of ‘Adaptive Control of Thought, Rational’. cognitive architecture (Anderson
1996), whose hybrid (symbolic and sub-symbolic) mechanisms are well-suited for
integration with neuro-symbolic algorithms and resources. Note that the
proposed approach is applicable to any cognitive architecture whose properties
are compatible with ACT-R, such as SOAR (Laird 2019) and SIGMA (Rosenbloom,
Demski, and Ustun 2016): in fact, these three architectures have been grouped
into the so-called ‘Standard Model of the Mind’ (Laird, Lebiere, and
Rosenbloom 2017), an idea that has its roots in physics.333For a brief
introduction to the Standard Model of Particle Physics, see this resource from
the U.S. Department of Energy:
https://www.energy.gov/science/doe-explainsthe-standard-model-particle-physics
Note that the Standard Model of the Mind doesn’t prescribe how to implement
cognitively-inspired AI systems; rather, it aims to play the role of a
conceptual framework of reference for developing them.

## Motivations

Over the last decade, deep learning has yielded tremendous advancements in
many AI fields, such as computer vision. For instance, neural models can
achieve high accuracy in object detection when training and testing domains
originate from the same data distribution. However, recent work shows that
minimal/regional modifications implanted in the data at test time cause
significant drop in accuracy (Eykholt et al. 2018; Rosenfeld, Zemel, and
Tsotsos 2018). The examples documented in (Rosenfeld, Zemel, and Tsotsos 2018)
are of particular interest, as they indicate how commonsense
contextualization, by means of incorporating a priori structured knowledge
into deep networks, can mitigate the effect of those perturbations, resulting
in more robust performance (Marino, Salakhutdinov, and Gupta 2016). In
general, a visual model suitably infused with knowledge extracted from
semantic resources like conceptnet (Speer, Chin, and Havasi 2017) can
strengthen the connections holding within instances of the same conceptual
domain (e.g., couch, television, table, lamp are located in living rooms) and
discard out-of-context interpretations (e.g., no real elephants are located in
living rooms, but photographs of elephant may be – figure 1 depicts such
case).

Figure 1: The Elephant in the Room: the probability that a label assigned by
an object detection system is correct increases when the context is factored
in: in this example, the label ‘elephant’ could plausibly denote a picture of
the pachyderm, but not the pachyderm itself.

When shifting to natural language, and to tasks like automated question
answering, the key role played by knowledge-based contextualization for neural
language models stands evident.444We use ‘language model’, ‘neural language
model’ and ‘large language model’ as interchangeable terms, as they commonly
refer to the same neural architecture based on multi-headed self-attention
mechanisms (Vaswani et al. 2017); however, computational power significantly
differs as function of the specific implementations (e.g., bert has 6 blocks
with 12 heads, gpt-3 has 24 blocks and 48 heads), and of the size of training
datasets (chat-gpt has been trained on a massive corpus – 570 GB – of text
data). For instance, it has been demonstrated that using kg triples to
disambiguate textual elements in a sentence, and embed the corresponding
concepts and relations in neural language models (Devlin et al. 2018),
significantly improves performance (Ma et al. 2021). In fact, despite of the
impressive results that llms are producing in Natural Language Processing (Ma
et al. 2019; Bauer and Bansal 2021; Shwartz et al. 2020), basic reasoning
capabilities are still largely missing. This is also the reason why it’s not
appropriate to use ‘Natural Language Understanding’ to denote these tasks,
because it would entail that robust and comprehensive reasoning capabilities
are present (McShane 2017). Let’s expand on this argument and consider a few
representative examples.
In ProtoQA (Boratko et al. 2020), gpt-2 (Dale 2021) fails to select options
like ‘pumpkin’, ‘cauliflower’, ‘cabbage’ as top candidates, for the question
‘one vegetable that is about as big as your head is?’: instead, ‘broccoli’,
‘cucumber’, ‘beet’, ‘carrot’ are predicted. In this case, the different models
learn some essential properties of vegetables from the training data, but do
not seem to acquire the capability of comparing their size to that of other
types of objects, revealing a substantial lack of analogical reasoning (Ushio
et al. 2021). The same issues are observed when chatgpt, a recent popular
version of gpt-3 optimized for conversations, is considered: the main
difference is that chatgpt is capable of generating plausible answers when the
question is submitted literally, but often fails to do so when the verbal
expression ‘about as big as’ is paraphrased with alternative forms like ‘about
the same size’, ‘about the same shape’, ‘comparable to’, etc. This
‘hypersensitivity’ to surface-level linguistic features – an epiphenomenon of
the model’s incapability to generalize over textual variations of the same
content – seem to indicate that the model cannot perform the necessary
(analogical) reasoning steps needed to correctly answer to the question. Along
these lines, recent work (Ettinger 2020) has shown that lack of complex
inferences, role-based event prediction, and understanding the conceptual
impact of negation, are some of the weaknesses diagnosed when bert (Devlin et
al. 2018), one of prominent open source language models, is applied to
benchmark datasets. ProtoQA again provides good examples of these
deficiencies: in general, neural models struggle to correctly interpret the
scope of modifiers like ‘not’ (reasoning under negation), ‘often’ and ‘seldom’
(temporal reasoning). Regarding the latter, in task 14 of bAbI (Weston et al.
2015), a comprehensive benchmark challenge designed by Facebook Research,
neural language systems exhibit variable accuracy in grasping temporal
ordering entailed by prepositions like ‘before’ and ‘after’. Similarly, in
bAbI task 17, which concerns spatial reasoning, llm-based systems fail to
infer basic positional information that require interpreting the semantics of
‘to the left/right of’, ‘above/below’, etc. If such systems are inaccurate
when dealing with common characteristics of the physical world, their
performance doesn’t improve when sentiments are considered: for instance, in
SocialIQA (Sap et al. 2019), given a context like ‘in the school play, Robin
played a hero in the struggle to death with the angry villain’, models are
unable to consistently select ‘hopeful that Robin will succeed’ over ‘sorry
for the villain’ when required to pick the correct answer to ‘how would others
feel afterwards?’. It’s not surprising that reasoning about emotional
reactions represents a difficult task for pure learning systems, when we
consider that such form of inference is deeply rooted in the sphere of human
experiences and social life, which involves a ‘layered’ understanding of
mental attitudes, intentions, motivations, emotions, and of the events that
trigger them.
The qualitative analysis presented above suggests that neural models struggle
to perform well in tasks that require high-level reasoning. But, are neuro-
symbolic approaches sufficient to overcome these limitation? Latent
expressions can augment training signals with sub-symbolic features derived
from explicit semantic content, but knowledge infusion per se doesn’t
determine how inference processes are conducted. Relevant work in this space
shows how deep neural models can replicate logical reasoning (Ebrahimi,
Eberhart, and Hitzler 2021; Garcez et al. 2022), but it doesn’t follow that
any form of logical reasoning that is provably reducible to learning
algorithms, should also be systematically reduced to it – this would be a
requirement only for tightly-coupled neuro-symbolic systems (Kautz 2022;
Garcez and Lamb 2023).
Accordingly, in the next section we make the case for developing an AI
framework where the ACT-R architecture is loosely-coupled with neuro-symbolic
components, to enable high-level reasoning.

## Method

Figure 2: act-r integrated with neuro-symbolic modules.

Cognitive architectures attempt to capture at the computational level the
invariant mechanisms of human cognition, including those underlying the
functions of control, learning, memory, adaptivity, perception and action.
act-r (Anderson 1996), in particular, is designed as a hybrid modular
framework including perceptual, motor and memory components, synchronized by a
procedural module through limited capacity buffers. Over the years, act-r has
accounted for a broad range of tasks at a high level of fidelity, reproducing
aspects of complex human behavior, from everyday activities like event
planning (Somers, Oltramari, and Lebiere 2020) and car driving (Cina and Rad
2023), to highly technical tasks such as piloting an airplane (Chen et al.
2021), and monitoring a network to prevent cyber-attacks (Ben-Asher et al.
2015). act-r has been used as a component in pipelines that include learning
algorithms (e.g., biologically-inspired neural networks (Jilk et al. 2008))
and external semantic resources (e.g., (Oltramari and Lebiere 2012; Emond
2006)): along this line of research, we claim that integrating act-r – or any
compatible cognitive architecture – with neuro-symbolic components is
instrumental to enable high-level machine reasoning.
Figure 2 provides a compact visualization of our proposed framework: the boxes
in blue, enclosed in the grey rectangle, represent the default components of
ACT-R, those in green the neuro-symbolic extensions.
The integration would occur along three main directions:

  * •

knowledge $\leftrightsquigarrow$ memory: the external symbolic module, which
can include background/domain knowledge graphs (kg), lexical resources (lr),
rule bases (rb), and a suitable inference engine, is linked to the declarative
memory. This is a two-way integration: the symbolic module can be read or
written by ACT-R, where the latter operation is triggered when populating or
pruning world knowledge is needed as part of task execution.

  * •

neural $\rightsquigarrow$ perception: the neural module, which can include
convolutional, recurrent, long-short-term memory networks, generative models,
etc., is trained, fine-tuned, or prompted with data processed from the
environment, providing relevant patterns of information to the perceptual or
imaginal module. This integration bypasses the direct connection holding – in
standard ACT-R – between the perceptual module and the environment.555Such
connection assumes symbolic representations of visual and auditory signals
being available to the architecture through pre-processing.

  * •

knowledge $\rightsquigarrow$ neural: adequately-selected embedding mechanisms
govern knowledge-infusion in the neural module, enabling knowledge-based
contextualization of patterns of information distilled from the environment,
which are subsequently channeled into ACT-R buffers.

If the mutual connections between the two intertwined neuro-symbolic modules
and act-r can be used to combine rich semantic contents with scalable learning
functionalities, they don’t per se bring about high-level reasoning: this
capability also requires two features of the integrated framework, namely the
cognitive architecture’s own procedural module and a proper inference engine
in the external symbolic module.
The procedural module matches the content of the other module buffers and
coordinates their activity using production rules, which are ‘condition-
action’ pairs tied to the task at hand. Productions use an utility-based
computation to select, from a set of task-specific plausible rules, the single
rule that is executed at any point in time. For instance, when building a
recommendation system to support a mechanic in troubleshooting a car engine, a
relevant situation that needs to be covered is a vehicle that doesn’t start
but has power; in this example, a high-utility production rule should capture
the following heuristic: if the engine holds compression well, and the fuel
system is working correctly, then the spark plugs should be checked. The
variables in these rule conditions would need to be filled with actual
empirical observations and measurements, as it is often the case when
cognitive architectures are applied in real-world scenarios: in our example,
such evidence could be actually gathered by a real technician using the
recommendation system in a human-machine-teaming fashion, a type of approach
that falls under the ‘cognitive model as oracle’ paradigm (Lebiere et al.
2022).
The inference engine in the symbolic module is used to derive knowledge from
assertions in the semantic resource of reference, a well-known feature of
symbolic AI systems. What is important to stress here, is that – in our
proposal – this form of logic-based reasoning would realize two functions: 1)
provide a combination of asserted and inferred knowledge that act-r
declarative memory can process and pass to the production system; 2) support
knowledge-infusion into neural modules. The first functionality would help to
decouple basic forms of reasoning, e.g. temporal and spatial666E.g., Region-
Connection-Calculus (Cohn et al. 1997) for spatial reasoning, Allen’s axioms
for temporal reasoning (Allen and Ferguson 1994). , from cognitive assessments
performed by the production system on conditional actions. Such feature makes
our proposed system efficient, as act-r productions are not well-suited for
logical reasoning. The second functionality would allow pre-training, fine-
tuning, or prompting a neural-model on both asserted and inferred knowledge:
this can provide act-r perceptual model with more informative patterns than
just those obtained by processing raw data.
It’s worth making a final consideration here: the framework introduced in this
section is complemental to the body of work that investigates how neuro-
symbolic systems can be leveraged to realize human-like cognitive reasoning
(Garcez, Lamb, and Gabbay 2008): in our proposal, act-r is interfaced with
neuro-symbolic components, whereas – in the approaches reviewed by Garcez et
al. – neuro-symbolic frameworks are used to solve cognitive tasks. The
difference lies on whether cognitive processes are considered first class
citizens or not.

## Discussion:
The Role of Generative AI in Cognitive Neuro-Symbolic Reasoning

As seen in the previous section, our proposed framework doesn’t require or
commit on a specific neural architecture. However, generative AI, and
specifically large language models, will play an increasingly relevant role in
enabling high-level reasoning based on cognitive neuro-symbolic systems. In
the next two sections we will briefly outline present research in this field,
and sketch what we think are promising developments.

### Related Work

The importance of integrating cognitive mechanisms into data-driven AI systems
has been recently acknowledged by one of the key figures in deep learning,
Yann LeCun: in a position paper published in 2022 (LeCun 2022), he described a
biologically-inspired cognitive architecture, where a so-called configurator
orchestrates information provided by different modules, such as the perception
module and the world model module, which replicate the functions emerging from
prefrontal-cortical processes. Furthermore, a motivation model – designed to
mimic the role of the amygdala in producing basic emotional states like pain
and pleasure – is used to compute intrinsic costs associated with current and
future actions, a mechanism that is instrumental to inform predictive
capabilities. It’s relevant to point out that there has been extensive
research on mapping cognitive architectures to brain areas/processes – e.g.,
(Borst et al. 2015) – and that an established scientific community has been
working on biologically-inspired approaches to cognitive architectures since
the early 2000’s (the BICA international conference has reached its 14
edition777See: https://bica2023.org/cfp/).
In line with the current trend of investigating computational models of
cognition in the context of large-scale neural networks, a recent blog (Weng
2023) provides an overview of how llms could be used to control autonomous
agents. It goes beyond the scope of our contribution to review in detail the
papers mentioned in the blog, but it’s beneficial to highlight some of the
most interesting topics.
In (Wei et al. 2022) the authors leverage chain-of-thought prompting with palm
540b (Chowdhery et al. 2022) for task-decomposition: despite of their reported
success, using prompting to generate fine-grained reasoning steps does not
always yield consistent results, as shown by (Chen, Zaharia, and Zou 2023) for
different versions of gpt-4 (OpenAI 2023). The same work also indicates that,
even when reasoning steps are correctly reproduced, they don’t always match
with the model selecting the correct solution/answer to a problem/question.
Another paper surveyed in the blog (Park et al. 2023) focuses on using gpt-4
to build believable agents for a sandbox environment 888Inspired by the video-
game ‘The Sims’: https://www.ea.com/games/the-sims. According to the authors,
cognitive architectures would not have the same degree of flexibility (and
scalability) that modern generative models provide when building AI agents, as
the former depend on hand-crafting rules, thus applicable only to narrow,
closed-world contexts. However, this is a partial account of the state of the
art: for instance, production compilation, act-r’s rule learning mechanism,
allows to learn new, task-specific production rules that directly implement
the relevant action(s) for a particular state (Taatgen, Huss, and Anderson
2006); moreover, to assess which stimuli from an environment are relevant for
an agent to act upon, researchers have developed mechanisms like instance-
based learning, a type of reinforcement learning (Sutton and Barto 2018),
which can be plugged into act-r (Gonzalez, Lerch, and Lebiere 2003). One may
also question the claim on generative models’ flexibility: in fact, the scope
of such capability is not the real world, with its ever-changing situations,
but rather some emerging patterns in the text-based training data, which are
biased interpretations of the real world. Incidentally, this lack of
‘grounding’ is also at the origin of llms’ widely-documented hallucination
problem – for an introduction to this phenomenon, see (Ji et al.
2023).999There is an interesting analogy between Plato’s Cave myth (Jowett et
al. 1873), where shadows projected on a blank wall were all that prisoners
could use to understand reality, thus misinterpreting it, and llms’s
generating inaccurate statements about the world, based on biased data
patterns.

Figure 3: Without adequate instructions, chat-gpt opts for a compact model of
counting from 1 to 10, which is not substantially distinct from a simple for
loop (left-side). Interestingly, the chatbot suggests to include additional
cognitive processes and mechanisms to make the model’s behavior more realistic
and accurate (bottom-left): this is what actually happens when the llm is
instructed to use all act-r modules (right-side). Far from being exhaustive,
this example provides some evidence of the feasibility of scaling cognitive
models via llms.

### Future Work

As the overview in the previous section suggests, there are intrinsic
limitations in utilizing a llm as orchestrator for intelligent agents. In this
regard, we can distill two main reasons for selecting a cognitive architecture
over a llm: a) the inner functioning of the former is transparent, whereas the
latter is a ‘black-box’ (Castelvecchi 2016); b) the former is designed to
replicate the invariant mechanisms of human cognition, the latter is
engineered to produce human-grade linguistic behavior, which cognitive
properties can only be ascribed to. By and large, what the state-of-the-art
suggests is that a synergistic integration of these cognitive architectures
and llms can help to maximize their relative strengths and mitigate their
weaknesses, fostering the creation of more advanced AI systems, capable of
high-level reasoning. In particular, (1) scaling cognitive models via llms and
(2) prompt-engineering llms with cognitive models can be seen as novel
approaches in this direction; they would actually be complemental, as (1) is a
method to automatize the creation of cognitive models using generative AI,
whereas (2) is a method to ground generative AI on computational artifacts
that reflect principled cognitive theories.

  1. 1.

Scaling cognitive models via llms. A cognitive architecture is a generic
framework to develop cognitive models, which are, conversely, tied to specific
tasks and domains: the process of developing cognitive models is still largely
manual, and thus affected by lack of scalability. Because llms have proven to
be effective in generating code across a variety of programming languages
(Gozalo-Brizuela and Garrido-Merchan 2023), they could also be leveraged to
produce software implementations of cognitive models. Initial experiments
performed by asking chat-gpt to generate basic cognitive models using a novel
library, i.e., PyACT-R101010https://github.com/jakdot/pyactr, show that the
OpenAI’s signature llm learns to correctly generate compact Python snippets,
although it only makes marginal use of act-r modules and buffers. In order to
achieve such level of sophistication in cognitive model design, chat-gpt needs
to be prompted with relevant instructions about which mechanisms of a
cognitive architecture it should use (see figure 3).

  2. 2.

Prompt-engineering llms with cognitive models. Using llms in domain-specific
applications requires either fine-tuning on a target dataset, or prompt-
engineering with adequate contextual knowledge. In many use cases, well-
curated data are either unavailable or too time-consuming to collect at scale,
making the latter more convenient and efficient. When the goal is to turn a
llm into a reliable decision support system, the ‘grounding’ problem mentioned
earlier also extends to the cognitive dimension: that is, such system would
need to be based on shared interpretations of reality as well as on sound
reasoning steps, from a cognitive-decisional standpoint. In fact, it’d be
difficult to conceive such a system as trustworthy if hallucinations on both
factual knowledge and on inferential mechanisms were widespread. To this end,
prompting a llm with key steps of a cognitive model’s reasoning process, the
so-called trace, would be instrumental to mitigate the second type of
hallucinations. Such steps de facto represent the introspective stages of a
cognitive model, and of a cognitive neuro-symbolic reasoning system based on
it.

## Conclusion

In the current debate on the limits of deep neural networks, the split is
oftentimes between those who think that more data is the panacea, and those
who support designing systems that integrate learning approaches with other
processing elements, such as knowledge representation and reasoning,
statistical algorithms, human-in-the-loop methods. In this paper, which echoes
the second category, we made the case for adopting a cognitive approach to
perform that integration, inspired by the results that architectures like
act-r have produced, over the last decades, in replicating complex human tasks
at the machine level. We described the main components of a cognitive neuro-
symbolic reasoning system, outlined their respective functionalities, and
discussed related and future work in the area of generative AI.
At the end, to paraphrase Yoshua Bengio (Yoshua Bengio 2022), we don’t assume
or prove that using cognitive architectures is the only possibility to equip
machines with high-level, human-like reasoning: however, through a diversity
of scientific explorations, we’ll increase our chances to find the ingredients
we are missing.

## References

  * Allen and Ferguson (1994) Allen, J. F.; and Ferguson, G. 1994.  Actions and events in interval temporal logic.  _Journal of logic and computation_ , 4(5): 531–579.
  * Anderson (1996) Anderson, J. R. 1996.  ACT: A simple theory of complex cognition.  _American psychologist_ , 51(4): 355.
  * Bauer and Bansal (2021) Bauer, L.; and Bansal, M. 2021.  Identify, Align, and Integrate: Matching Knowledge Graphs to Commonsense Reasoning Tasks.  In _Proc. of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , 2259–2272. Online: Association for Computational Linguistics.
  * Ben-Asher et al. (2015) Ben-Asher, N.; Oltramari, A.; Erbacher, R. F.; and Gonzalez, C. 2015.  Ontology-based Adaptive Systems of Cyber Defense.  In _STIDS_ , 34–41.
  * Bengio et al. (2019) Bengio, Y.; Deleu, T.; Rahaman, N.; Ke, R.; Lachapelle, S.; Bilaniuk, O.; Goyal, A.; and Pal, C. 2019.  A meta-transfer objective for learning to disentangle causal mechanisms.  _arXiv preprint arXiv:1901.10912_.
  * Boratko et al. (2020) Boratko, M.; Li, X. L.; Das, R.; O’Gorman, T.; Le, D.; and McCallum, A. 2020.  ProtoQA: A Question Answering Dataset for Prototypical Common-Sense Reasoning.  _arXiv preprint arXiv:2005.00771_.
  * Borst et al. (2015) Borst, J. P.; Nijboer, M.; Taatgen, N. A.; van Rijn, H.; and Anderson, J. R. 2015\.  Using data-driven model-brain mappings to constrain formal models of cognition.  _PLoS One_ , 10(3): e0119673.
  * Castelvecchi (2016) Castelvecchi, D. 2016.  Can we open the black box of AI?  _Nature News_ , 538(7623): 20.
  * Chen et al. (2021) Chen, H.; Liu, S.; Pang, L.; Wanyan, X.; and Fang, Y. 2021.  Developing an improved ACT-R model for pilot situation awareness measurement.  _IEEE Access_ , 9: 122113–122124.
  * Chen, Zaharia, and Zou (2023) Chen, L.; Zaharia, M.; and Zou, J. 2023.  How is ChatGPT’s behavior changing over time?  _arXiv preprint arXiv:2307.09009_.
  * Chowdhery et al. (2022) Chowdhery, A.; Narang, S.; Devlin, J.; Bosma, M.; Mishra, G.; Roberts, A.; Barham, P.; Chung, H. W.; Sutton, C.; Gehrmann, S.; et al. 2022.  Palm: Scaling language modeling with pathways.  _arXiv preprint arXiv:2204.02311_.
  * Cina and Rad (2023) Cina, M.; and Rad, A. B. 2023.  Categorized review of drive simulators and driver behavior analysis focusing on ACT-R architecture in autonomous vehicles.  _Sustainable Energy Technologies and Assessments_ , 56: 103044.
  * Cohn et al. (1997) Cohn, A. G.; Bennett, B.; Gooday, J.; and Gotts, N. M. 1997.  Qualitative spatial representation and reasoning with the region connection calculus.  _geoinformatica_ , 1: 275–316.
  * Dale (2021) Dale, R. 2021.  GPT-3: What’s it good for?  _Natural Language Engineering_ , 27(1): 113–118.
  * Devlin et al. (2018) Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_.
  * Ebrahimi, Eberhart, and Hitzler (2021) Ebrahimi, M.; Eberhart, A.; and Hitzler, P. 2021.  On the Capabilities of Pointer Networks for Deep Deductive Reasoning.  _arXiv preprint arXiv:2106.09225_.
  * Emond (2006) Emond, B. 2006.  WN-LEXICAL: An ACT-R module built from the WordNet lexical database.  In _Proceedings of the Seventh International Conference on Cognitive Modeling_ , 359–360.
  * Ettinger (2020) Ettinger, A. 2020.  What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models.  _Transactions of the Association for Computational Linguistics_ , 8: 34–48.
  * Eykholt et al. (2018) Eykholt, K.; Evtimov, I.; Fernandes, E.; Li, B.; Rahmati, A.; Xiao, C.; Prakash, A.; Kohno, T.; and Song, D. 2018.  Robust physical-world attacks on deep learning visual classification.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 1625–1634.
  * Garcez et al. (2022) Garcez, A. d.; Bader, S.; Bowman, H.; Lamb, L. C.; de Penning, L.; Illuminoo, B.; Poon, H.; and Gerson Zaverucha, C. 2022.  Neural-symbolic learning and reasoning: A survey and interpretation.  _Neuro-Symbolic Artificial Intelligence: The State of the Art_ , 342: 1.
  * Garcez and Lamb (2023) Garcez, A. d.; and Lamb, L. C. 2023.  Neurosymbolic AI: The 3 rd wave.  _Artificial Intelligence Review_ , 1–20.
  * Garcez, Lamb, and Gabbay (2008) Garcez, A. S.; Lamb, L. C.; and Gabbay, D. M. 2008.  _Neural-symbolic cognitive reasoning_.  Springer Science & Business Media.
  * Gonzalez, Lerch, and Lebiere (2003) Gonzalez, C.; Lerch, J. F.; and Lebiere, C. 2003.  Instance-based learning in dynamic decision making.  _Cognitive Science_ , 27(4): 591–635.
  * Gozalo-Brizuela and Garrido-Merchan (2023) Gozalo-Brizuela, R.; and Garrido-Merchan, E. C. 2023.  ChatGPT is not all you need. A State of the Art Review of large Generative AI models.  _arXiv preprint arXiv:2301.04655_.
  * Ji et al. (2023) Ji, Z.; Lee, N.; Frieske, R.; Yu, T.; Su, D.; Xu, Y.; Ishii, E.; Bang, Y. J.; Madotto, A.; and Fung, P. 2023.  Survey of hallucination in natural language generation.  _ACM Computing Surveys_ , 55(12): 1–38.
  * Jilk et al. (2008) Jilk, D. J.; Lebiere, C.; O’Reilly, R. C.; and Anderson, J. R. 2008.  SAL: An explicitly pluralistic cognitive architecture.  _Journal of Experimental and Theoretical Artificial Intelligence_ , 20(3): 197–218.
  * Jowett et al. (1873) Jowett, B.; et al. 1873.  _The dialogues of Plato_ , volume 4.  Scribner, Armstrong.
  * Kahneman (2011) Kahneman, D. 2011.  _Thinking, fast and slow_.  macmillan.
  * Kautz (2022) Kautz, H. 2022.  The third ai summer: Aaai robert s. engelmore memorial lecture.  _AI Magazine_ , 43(1): 105–125.
  * Kotseruba and Tsotsos (2020) Kotseruba, I.; and Tsotsos, J. K. 2020.  40 years of cognitive architectures: core cognitive abilities and practical applications.  _Artificial Intelligence Review_ , 53(1): 17–94.
  * Laird (2019) Laird, J. E. 2019.  _The Soar cognitive architecture_.  MIT press.
  * Laird, Lebiere, and Rosenbloom (2017) Laird, J. E.; Lebiere, C.; and Rosenbloom, P. S. 2017.  A standard model of the mind: Toward a common computational framework across artificial intelligence, cognitive science, neuroscience, and robotics.  _Ai Magazine_ , 38(4): 13–26.
  * Langley, Laird, and Rogers (2009) Langley, P.; Laird, J. E.; and Rogers, S. 2009.  Cognitive architectures: Research issues and challenges.  _Cognitive Systems Research_ , 10(2): 141–160.
  * Lebiere et al. (2022) Lebiere, C.; Cranford, E.; Martin, M.; Morrison, D.; and Stocco, A. 2022.  Cognitive architectures and their applications.  In _Proceedings of IEEE CIC_.
  * LeCun (2022) LeCun, Y. 2022.  A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27.  _Open Review_ , 62.
  * Lieto (2021) Lieto, A. 2021.  _Cognitive design for artificial minds_.  Routledge.
  * Ma et al. (2019) Ma, K.; Francis, J.; Lu, Q.; Nyberg, E.; and Oltramari, A. 2019.  Towards Generalizable Neuro-Symbolic Systems for Commonsense Question Answering.  In _Proc. of the First Workshop on Commonsense Inference in Natural Language Processing_ , 22–32.
  * Ma et al. (2021) Ma, K.; Ilievski, F.; Francis, J.; Bisk, Y.; Nyberg, E.; and Oltramari, A. 2021\.  Knowledge-driven data construction for zero-shot evaluation in commonsense question answering.  In _Proc. of 35th AAAI Conference on Artificial Intelligence_.
  * Margatina, Baziotis, and Potamianos (2019) Margatina, K.; Baziotis, C.; and Potamianos, A. 2019.  Attention-based conditioning methods for external knowledge integration.  _arXiv preprint arXiv:1906.03674_.
  * Marino, Salakhutdinov, and Gupta (2016) Marino, K.; Salakhutdinov, R.; and Gupta, A. 2016.  The more you know: Using knowledge graphs for image classification.  _arXiv preprint arXiv:1612.04844_.
  * McShane (2017) McShane, M. 2017.  Natural language understanding (NLU, not NLP) in cognitive systems.  _AI Magazine_ , 38(4): 43–56.
  * Oltramari and Lebiere (2012) Oltramari, A.; and Lebiere, C. 2012.  Using ontologies in a cognitive-grounded system: automatic action recognition in video surveillance.  In _Proceedings of the 7th International Conference on Semantic Technology for Intelligence, Defense, and Security_. Citeseer.
  * OpenAI (2023) OpenAI, R. 2023.  GPT-4 technical report.  _arXiv_ , 2303–08774.
  * Park et al. (2023) Park, J. S.; O’Brien, J. C.; Cai, C. J.; Morris, M. R.; Liang, P.; and Bernstein, M. S. 2023.  Generative agents: Interactive simulacra of human behavior.  _arXiv preprint arXiv:2304.03442_.
  * Peters et al. (2017) Peters, M. E.; Ammar, W.; Bhagavatula, C.; and Power, R. 2017.  Semi-supervised sequence tagging with bidirectional language models.  _arXiv preprint arXiv:1705.00108_.
  * Rosenbloom, Demski, and Ustun (2016) Rosenbloom, P. S.; Demski, A.; and Ustun, V. 2016.  The Sigma cognitive architecture and system: Towards functionally elegant grand unification.  _Journal of Artificial General Intelligence_ , 7(1): 1.
  * Rosenfeld, Zemel, and Tsotsos (2018) Rosenfeld, A.; Zemel, R.; and Tsotsos, J. K. 2018.  The elephant in the room.  _arXiv preprint arXiv:1808.03305_.
  * Santos and Rosati (2015) Santos, L. R.; and Rosati, A. G. 2015.  The evolutionary roots of human decision making.  _Annual review of psychology_ , 66: 321–347.
  * Sap et al. (2019) Sap, M.; Rashkin, H.; Chen, D.; Le Bras, R.; and Choi, Y. 2019.  Social IQa: Commonsense Reasoning about Social Interactions.  In _Proc. of EMNLP-IJCNLP_ , 4463–4473.
  * Shwartz et al. (2020) Shwartz, V.; West, P.; Le Bras, R.; Bhagavatula, C.; and Choi, Y. 2020.  Unsupervised Commonsense Question Answering with Self-Talk.  In _Proc. of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , 4615–4629. Online: Association for Computational Linguistics.
  * Simon (1955) Simon, H. A. 1955.  A behavioral model of rational choice.  _The quarterly journal of economics_ , 99–118.
  * Somers, Oltramari, and Lebiere (2020) Somers, S.; Oltramari, A.; and Lebiere, C. 2020.  Cognitive Twin: A Cognitive Approach to Personalized Assistants.  In _AAAI Spring Symposium: Combining Machine Learning with Knowledge Engineering (1)_.
  * Speer, Chin, and Havasi (2017) Speer, R.; Chin, J.; and Havasi, C. 2017.  Conceptnet 5.5: An open multilingual graph of general knowledge.  In _Thirty-first AAAI conference on artificial intelligence_.
  * Strub et al. (2018) Strub, F.; Seurin, M.; Perez, E.; De Vries, H.; Mary, J.; Preux, P.; and Pietquin, A. C. 2018.  Visual reasoning with multi-hop feature modulation.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , 784–800.
  * Sutton and Barto (2018) Sutton, R. S.; and Barto, A. G. 2018.  _Reinforcement learning: An introduction_.  MIT press.
  * Taatgen, Huss, and Anderson (2006) Taatgen, N. A.; Huss, D.; and Anderson, J. R. 2006.  How cognitive models can inform the design of instructions.  In _Proceedings of the seventh international conference on cognitive modeling_ , 304–309. Citeseer.
  * Ushio et al. (2021) Ushio, A.; Espinosa-Anke, L.; Schockaert, S.; and Camacho-Collados, J. 2021.  BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?  _arXiv preprint arXiv:2105.04949_.
  * Vaswani et al. (2017) Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, Ł.; and Polosukhin, I. 2017.  Attention is all you need.  _Advances in neural information processing systems_ , 30.
  * Wang et al. (2017) Wang, Q.; Mao, Z.; Wang, B.; and Guo, L. 2017.  Knowledge Graph Embedding: A Survey of Approaches and Applications.  _IEEE Transactions on Knowledge and Data Engineering_ , 29(12): 2724–2743.
  * Wei et al. (2022) Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Xia, F.; Chi, E.; Le, Q. V.; Zhou, D.; et al. 2022.  Chain-of-thought prompting elicits reasoning in large language models.  _Advances in Neural Information Processing Systems_ , 35: 24824–24837.
  * Weng (2023) Weng, L. 2023.  LLM-powered Autonomous Agents.  _lilianweng.github.io_.
  * Weston et al. (2015) Weston, J.; Bordes, A.; Chopra, S.; Rush, A. M.; van Merriënboer, B.; Joulin, A.; and Mikolov, T. 2015.  Towards ai-complete question answering: A set of prerequisite toy tasks.  _arXiv preprint arXiv:1502.05698_.
  * Wickramarachchi, Henson, and Sheth (2023) Wickramarachchi, R.; Henson, C.; and Sheth, A. 2023.  CLUE-AD: A Context-Based Method for Labeling Unobserved Entities in Autonomous Driving Data.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 37, 16491–16493.
  * Yoshua Bengio (2022) Yoshua Bengio. 2022.  DeepLearning.AI.  https://www.deeplearning.ai/the-batch/yoshua-bengio-wants-neural-nets-that-reason/.


11institutetext: Allen Institute for Cell Science, Seattle, Washington

# Densely Decoded Networks with Adaptive Deep Supervision for Medical Image
Segmentation

Suraj Mishra

###### Abstract

Medical image segmentation using deep neural networks has been highly
successful. However, the effectiveness of these networks is often limited by
inadequate dense prediction and inability to extract robust features. To
achieve refined dense prediction, we propose densely decoded networks
(${D\\!D\\!N}$), by selectively introducing ‘crutch’ network connections. Such
‘crutch’ connections in each upsampling stage of the network decoder (1)
enhance target localization by incorporating high resolution features from the
encoder, and (2) improve segmentation by facilitating multi-stage contextual
information flow. Further, we present a training strategy based on adaptive
deep supervision (${A\\!D\\!S}$), which exploits and adapts specific
attributes of input dataset, for robust feature extraction. In particular,
${A\\!D\\!S}$ strategically locates and deploys auxiliary supervision, by
matching the average input object size with the layer-wise effective receptive
fields (${L\\!E\\!R\\!F}$) of a network, resulting in a class of
${D\\!D\\!N}$s. Such inclusion of ’companion objective’ from a specific hidden
layer, helps the model pay close attention to some distinct input-dependent
features, which the network might otherwise ‘ignore’ during training. Our new
networks and training strategy are validated on 4 diverse datasets of
different modalities, demonstrating their effectiveness.

##  1 Introduction

In recent years, convolutional neural networks (${C\\!N\\!N}$) have been
highly successful in medical image segmentation. Every skeletonized
${C\\!N\\!N}$ for image segmentation task, can be viewed as a combination of
an encoder and a decoder. While the encoder extracts features from the image
domain, the decoder maps these extracted features into the mask domain. Such a
mapping is done by assigning a class label to every pixel of the image, which
is also known as localization. However, we observe that the network parameters
may not be fully utilized in predicting dense pixels from coarse encoder
features, which can lead to unrefined spatial localization and poor
segmentation. Further, we believe that image domain specific information can
be exploited for improved feature extraction. Enhancing segmentation accuracy
by (1) distilling the dense prediction task and (2) consolidating feature
extraction with an enhanced training strategy tailored specifically for
biomedical image datasets are the major objectives of this work.

Our first objective of distilling dense prediction has been a topic of
interest. Long et al. [16] proposed a “fully convolutional network” along with
‘skip’ connections to avoid coarse output. Ronneberger et al. [26] modified
the model in [16] by introducing stage wise upsampling to avoid larger
strides. Each upsampling stage is supplemented by a corresponding stage in the
encoder. A similar approach for 3D segmentation was proposed by Milletari et
al. [19]. Despite their success, most of these architectures fail to segment
the target objects accurately because of their inability to capture finer
details. To achieve our first objective, we strategically divide the distilled
dense prediction task into two sub-tasks: (1) enhanced localization and (2)
finer detail generation. To tackle these two tasks, we propose the new densely
decoded networks (${D\\!D\\!N}$s).

${D\\!D\\!N}$s are capable of generating refined dense prediction from coarse
features. To accomplish this, ${D\\!D\\!N}$s implement ‘crutch’ connections in
every upsampling stage of the network. Such crutch connections can be
classified into two major categories: (1) incoming crutch connections
(${I\\!C\\!C}$) and (2) outgoing crutch connections (${O\\!C\\!C}$). While the
commensurate stage inclusion using ’crutch’ connections was proposed in [26],
${I\\!C\\!C}$ (inspired by DenseNet [8]) extends it by including all deeper
stages too. Such inclusion contributes towards enhanced localization, as each
upsampling stage is provided with feature maps in varying degrees of
coarseness, which are combined to generate the output of the stage. While
${I\\!C\\!C}$ is included for enhanced localization, ${O\\!C\\!C}$ is directed
towards finer detail generation. ${O\\!C\\!C}s$ (inspired by [15, 14]) are
extracted from each upsampling stage, and are merged to generate the final
output. A somewhat similar approach to ${O\\!C\\!C}$ was proposed in [4]. We
modify it by generating ${O\\!C\\!C}$ from the stage-wise decoder instead of
the encoder and including ${O\\!C\\!C}$ from every stage.

Robust discriminativeness among features is a vital attribute for any feature
extractor. In addition to accounting for the issue of vanishing gradient,
using auxiliary classifiers to perform deep supervision has been effective in
ensuring discriminativeness of features in hidden layers [11]. Hence, we
leverage deep supervision for our second objective of robust feature
extraction. Deep supervision can be viewed as a feedback generated by a local
output (from a hidden layer) by providing companion objective with auxiliary
classifiers [11]. The combined error is back-propagated, influencing the
hidden layer update process to favor highly discriminative feature maps. The
strategy of how to introduce deep supervision into a network architecture has
been an active research topic [28, 11, 4, 29], but has not been explored
comprehensively. We extend the idea of deep supervision to biomedical image
segmentation by proposing adaptive deep supervision (${A\\!D\\!S}$).
${A\\!D\\!S}$ introduces deep supervision from the specific layer in the
encoder that has the preeminent contribution towards discriminative feature
extraction. ${A\\!D\\!S}$ exploits the relative ‘stable’ nature of biomedical
image data to extract dataset specific information (e.g., the average object
size), and matches this information with the receptive field of the network to
determine the preeminent layer adaptively for different specific datasets.
This results in a class of ‘adaptive’ networks capable of extracting improved
robust features.

We extensively validate each component of our proposed framework using four
datasets of differing modalities (Ultrasound, 3D MRI, RGB, and Grayscale). Our
experiments show that combining ${D\\!D\\!N}$ with ${A\\!D\\!S}$ generates a
class of networks which are capable of being trained end-to-end while
achieving better segmentation accuracy over state-of-the-art methods.

##  2 Method

Figure 1: Our proposed DDN architecture. ICC is shown as dashed colored arrows
while OCC is thick colored arrows. ADS for (a) melanoma, (b) wing disc (red
dotted box for the Obj $>$ LERFN cases), (c) lymph node, (d) BraTS are shown
as dotted colorless arrows. Each learnable layer (except 1x1 conv) is followed
by batch-norm and ReLU activation. Upsampling layers in ICC (similar as in
OCC) are not shown here.

###  2.1 Densely Decoded Networks (DDN)

We propose ${D\\!D\\!N}$, with a U-Net [26] type encoder-decoder architecture,
for biomedical image segmentation. Fig. 1 shows our proposed network along
with ‘crutch’ connections. We first discuss ${I\\!C\\!C}$, followed by the
details of ${O\\!C\\!C}$.

ICC: Consider a ${C\\!N\\!N}$, with $N$ stacked conv-layers in its encoder,
arranged in $M$ stages. Each stage represents a set of conv-layers extracting
features with a fixed feature map size (i.e., in-between two down-sampling
operations). Assuming a U-Net [26] type skeleton network, there is a
corresponding upsampling stage for each downsampling stage. Denoting $y_{m}$
as the output of the $m^{th}$ upsampler $U_{m}$, we can state
$y_{m}=U_{m}(x_{m})$, where $x_{m}$ is the input to $U_{m}$. In ${D\\!D\\!N}$,
each upsampling stage takes ${I\\!C\\!C}$s from the commensurate and all the
deeper stages in the encoder. Hence, $y_{m}=U_{m}(x_{m}\oplus E_{m}\oplus
E_{m+1}\oplus\cdots\oplus E_{M})$, where $E_{i}$ is the output of the encoder
stage $i$ (or ${I\\!C\\!C}$), $i=m,\ldots,M$ and $\oplus$ is the concatenation
operation. Note that U-Net can be viewed as a special case of $DDN$ since for
U-Net, $y_{m}=U_{m}(x_{m}\oplus E_{m})$, with $m\in M$. Each ${I\\!C\\!C}$
acts as crutches to the main input to any upsampling stage (the main input
$x_{m}$, black arrows in Fig. 1), resulting in enhanced localization.
Intuitively, inclusion of feature maps of varying coarseness degrees to
generate the upsampling output is the key here.

OCC: Output of a ${C\\!N\\!N}$ can be stated as $y_{0}=U_{1}(x_{1})$, where
$U_{1}$ is the last upsampler with input $x_{1}$, generating the output
$y_{0}$ (ignoring conv layers between $U_{1}$ and the output). In
${D\\!D\\!N}$, the final output is a combination of outputs (pixel-wise
feature map addition, shown as $\boxplus$ in Fig. 1) from all the upsampling
stages, i.e., $y_{0}=U_{1}(x_{1})+\cdots+U_{M}(x_{M})$ or ${O\\!C\\!C}$s (the
vanilla case, without considering ${I\\!C\\!C}$s). ${O\\!C\\!C}$s can be
viewed as crutches for the final output. Inclusion of upsampled features from
various feature map sizes facilitates multi-stage contextual information flow,
resulting in finer detail generation. Merging both ${I\\!C\\!C}$s and
${O\\!C\\!C}$s, the final output of ${D\\!D\\!N}$ can be reported as
$y_{0}=U_{1}(x_{1}\oplus\cdots\oplus E_{M})+\cdots+U_{M}(x_{m})$, where
$x_{m}=E_{M}$. Intuitively, $DDN$ has a more general architecture with many
current architectures as its special cases.

Figure 2: Our proposed model for ADS. Based on the (LERF, Obj) matching (i.e.,
inputs, shown in green), ADS suggests the location of Lossaux, shown in red
and blue boxes.

###  2.2 Adaptive Deep Supervision (ADS)

For robust feature extraction, we propose ${A\\!D\\!S}$, which deals with the
receptive field of a network, to introduce auxiliary classifiers.
Convolutional layers in ${C\\!N\\!N}$s use only a portion of the input, which
contributes towards the output generation and is known as the receptive field
($RF$) of a layer [10]. For a ${C\\!N\\!N}$ performing segmentation, $RF$ of
the network is regarded as the region of input influencing the output of the
encoder. Luo et al. [17] showed that this region of influence is smaller
(called the effective receptive field or ${E\\!R\\!F}$) with unresponsive
border. Intuitively, convolutional filters extract improved features when the
target object size matches the filters’ ${E\\!R\\!F}$ and the object lies at
the ${E\\!R\\!F}$ center [21, 22, 23, 24].

We propose, for ${C\\!N\\!N}$s performing biomedical image segmentation, using
deep supervision from a specific hidden layer (the preeminent layer) to
introduce an companion objective (or regularization), which forces the network
not to ignore some specific input-dependent features. Our method is based on
the intuition that smaller objects are lost in the network’s sub-sampling
operations, while robust global features for larger objects are never
extracted due to a smaller network ${E\\!R\\!F}$ or field of view. Hence, we
consider the preeminent layer determination as a matching problem, whose
objective is to determine the network layer whose ${E\\!R\\!F}$ matches as
well as possible with the target object size (${O\\!b\\!j}$) of the input
dataset. We modify the method in [17] by proposing layer-wise ${E\\!R\\!F}$
(${L\\!E\\!R\\!F}$) and use it instead of $ERF$ to perform matching (explained
below). The framework for ${A\\!D\\!S}$ is highlighted in Fig. 2. The major
components of ${A\\!D\\!S}$ are: (1) ${O\\!b\\!j}$ calculation, (2)
${L\\!E\\!R\\!F}$ determination, and (3) ${A\\!D\\!S}$ implementation. We
provide detailed explanations of each components in the following sections.

Obj Calculation: Unlike natural scene images, objects in biomedical images are
relatively ‘stable’ (e.g., a dataset is for a specific disease or type of
organs, captured in a specific modality). We exploit this property by
estimating a representative object size (${O\\!b\\!j}$) for a specific dataset
using the mean object size with an acceptable level of accuracy. We use a
connected-component based method for object size calculation on the training
set of labelled images $G_{i}$, $i=1,2,\ldots,g$, where each image has $J_{i}$
objects. Binarized labelled images ($\mathbb{R}^{C}\to\mathbb{R}^{0,1}$) are
used to determine the pixel area of each target object ($T_{i}^{j}$ in
$G_{i}$, i.e, the $j^{th}$ object in the $i^{th}$ image), irrespective of the
target classes $C$, by treating each target object as a connected-component.
The square root of the pixel area is considered as the size of an object
($o_{i}^{j}=\sqrt{PixelArea(T_{i}^{j})}$). The mean value of these object
sizes is taken as the average object size for $G_{i}$. The dataset
characteristic object size is the mean value over all the $g$ images
(${O\\!b\\!j}=\sum_{i}(\sum_{j}o_{i}^{j}/J_{i})/g$).

LERF Determination: We adopt a layer-wise approach for ${E\\!R\\!F}$
calculation (i.e., ${L\\!E\\!R\\!F}$), since our objective is to match
${O\\!b\\!j}$ with the ${E\\!R\\!F}$ of a specific layer. We follow a partial
derivative based approach to measure the influence of the input region, over
an output node (a unit feature map, i.e., a pixel) of a conv layer [17]. For a
CNN with $N$ conv layers in its encoder, ${L\\!E\\!R\\!F}_{h}$ is associated
with layer $h$ ($h=1,2,\ldots,N$). With an input node $p_{i,j}$, our objective
is to determine $\frac{\partial q_{i_{t},j_{t}}}{\partial p_{i,j}}$, where
$q_{i_{t},j_{t}}$ is the target node at the output of the conv layer for which
the influence measurement is being performed. In a ${C\\!N\\!N}$, error
gradients are calculated during back-propagation (i.e., $\frac{\partial
L}{\partial p_{i,j}}$, for a loss function $L$). Hence, by assigning the
target node gradient a non-zero value ($\frac{\partial L}{\partial
q_{i_{t},j_{t}}}=1$ and $\frac{\partial L}{\partial p_{i,j}}=0$, for
$i_{t}\neq i$ and $j_{t}\neq j$) and using the chain rule, the region of
influence can be determined (i.e., all non-zero gradients at the input). This
region has a near-Gaussian distribution [17] with an insignificant border
area. Therefore, the square root of the thresholded region is assigned as the
${L\\!E\\!R\\!F}$ of the conv layer.

ADS Implementation: To implement ${A\\!D\\!S}$, we use ${O\\!b\\!j}$ and
${L\\!E\\!R\\!F}$, to determine $L_{target}$, i.e., the conv layer with
preeminent contribution towards feature extraction. In ${A\\!D\\!S}$
implementation, two possible cases must be tackled separately. Case-1: When
${L\\!E\\!R\\!F}_{N}$ (i.e., ${E\\!R\\!F}$ of the network) is
$\geq{O\\!b\\!j}$, $L_{target}$ is selected as the layer $h*$ where,
$|{O\\!b\\!j}-{L\\!E\\!R\\!F}_{h^{*}}|\leq|{O\\!b\\!j}-{L\\!E\\!R\\!F}_{h}|;\forall~{}1\leq
h\leq N$. Deep supervision is then implemented from $L_{target}$ to inculcate
the hidden features extracted as companion objective for discriminative
feature learning. Our experiments revealed that deep supervision using $OCC$
of the decoder (shown in Fig. 1) that has direct link with $L_{target}$
generates more accurate results. Further, such inclusion restricts additional
network weights since dense features are already upsampled and hence
preferred. When no direct connection between the decoder and $L_{target}$, the
next conv layer $L_{h^{t}}$ is selected with $h^{t}>target$. Case-2: When
${L\\!E\\!R\\!F}_{N}<{O\\!b\\!j}$, we propose stacking $h^{*}$ additional conv
layers on the $Loss_{aux}$ branch extracted from $L_{N}$ (shown as $\star$ in
Fig. 1), such that
$|{O\\!b\\!j}-{L\\!E\\!R\\!F}_{h^{*}}|\leq|{O\\!b\\!j}-{L\\!E\\!R\\!F}_{h}|;\forall~{}1\leq
h\leq N+h^{*}$. This stacking provides a network path with a larger
${E\\!R\\!F}$ compared to the base network, resulting in closer matching to
${O\\!b\\!j}$ and hence better global feature extraction.

Inclusion of ${A\\!D\\!S}$ also changes the total loss ($Loss_{total}$, for
back-propagation) into the sum of the main loss ($Loss_{main}$) and auxiliary
loss ($Loss_{aux}$), i.e.,
$Loss_{total}=Loss_{main}+Loss_{aux}+L_{2}$-$Regularization$. We do not use
any multiplier to reduce the effect of ${A\\!D\\!S}$, instead letting the
network learn by itself. We investigated cross-entropy, Dice, and Jaccard
based $Loss_{main}$ and $Loss_{aux}$ calculations, and provide experimental
details in the next section.

##  3 Experiments and Results

In this section, we first provide details of datasets used to validate our
approach. Then we provide the general experimental setup to implement
${D\\!D\\!N}$ for each dataset. ${A\\!D\\!S}$ implementation for individual
cases are detailed after that. Finally, we discuss results obtained along with
ablation study to corroborate our findings.

(1) Melanoma segmentation in RGB images: The ISIC 2017 skin lesion dataset [5]
contains 2000 training, 150 validation, and 600 test RGB images for melanoma
segmentation. Observing the smaller validation set, we merge the training and
validation sets and randomly select 20% of the merged set for validation as
done in [25]. Images are resized to $320\times 320$ following [12]. (2) Brain
tumor segmentation in 3D MR images: BraTS 2017 training dataset [2, 18, 1]
contains 285 cases (210 HGG and 75 LGG) of volumetric data in four modalities
(T1, T1CE, T2, and FLAIR). Each volume is of size $240\times 240\times 155$
and was annotated into 3 tumor regions. For our experiments, we consider the
more difficult LGG cases. As pre-processing, we normalize each image by
subtracting mean and dividing variance. Following [6], we only consider the
FLAIR modality and extract transverse slices (XY-plane) as training images. 3
classes of tumor regions are combined to generate the whole tumor region and
used as ground truth for training. (3) Lymph node segmentation in ultrasound
images: Lymph node dataset contains ultrasound images of the lymph node areas
of 237 patients. Following [31], we use 137 images for training (20% for
validation) and the rest for testing, assuring no identity overlap. Each image
is resized to $224\times 224$. (4) Wing disc segmentation in grayscale images:
Wing disc pouches of fruit flies are used to explore organ development [13].
996 grayscale wing disc pouch images are investigated by using 889 images for
training (20% for validation) and 107 images for testing. Following [13, 20],
we use images of size $512\times 512$ for the experiments.

Figure 3: Example segmentation results for (row-1) lymph node, (row-2) wing
disc, (row-3) melanoma, and (row-4) BraTS dataset. (a), (c), and (e) show
example test images. (b), (d), and (f) are the ground truth (blue) and
segmentation output (red) obtained using DDN + ADS. Magenta (red + blue)
highlights true positive regions.

DDN Setup: Our proposed ${D\\!D\\!N}$ architecture (shown in Fig. 1) has a
similar number of filter arrangements (or channels) in each layer as U-Net
[26]. Initial experiments with different arrangements of dropout layers [27]
along with different dropout rates reveal that using a single dropout layer at
the end of the encoder with a dropout rate of 50% performs the best. Random
flipping and rotation are used for data augmentation. We use a standard back-
propagation implementing ADAM with a fixed learning rate of 0.00002
($\beta_{1}=0.9,\beta_{2}=0.999,\epsilon=1\mathrm{e}{-8}$). Experiments were
performed on NVIDIA-TITAN and Tesla P100 GPUs, using the PyTorch framework for
a number of epochs (melanoma = 3000, BraTS = 500, lymph = 20000, wing disc =
5000). The batch size for each case is selected as the maximum size
permissible by the GPU. We use similar network initialization as in He et al.
[7]. We experimented with Dice and Jaccard loss for melanoma segmentation
following [30] but did not observe any significant change in accuracy compared
to cross-entropy loss. While cross-entropy is used as $Loss_{main}$ for
melanoma, lymph node, and wing disc segmentation, Dice loss is used for BraTS
segmentation following [6]. ADS Setup: Analyzing the Gaussian distribution of
$RF$ [17], a threshold of 2 standard deviation ($95.4\%$) is used for
${L\\!E\\!R\\!F}$ extraction. To neutralize variations in ${L\\!E\\!R\\!F}$
calculations caused by network nonlinearity, the mean over 20 iterations is
used. Following the implementation details provided in Sec. 2.2, ${O\\!b\\!j}$
for resized images is determined (melanoma $\sim$ 118, BraTS $\sim$ 16, lymph
node $\sim$ 51, wing disc $\sim$ 189). Using ${L\\!E\\!R\\!F}$ values of
${D\\!D\\!N}$ layers, ${A\\!D\\!S}$ locations are decided and are shown in
Fig. 1. We find that stacking conv-layers with 64 filter channels (instead of
1024 or 512 for Case-2) on ${A\\!D\\!S}$ results in better accuracy by
avoiding overfitting and also results in a lesser number of network weights.
We use the same $Loss_{aux}$ as $Loss_{main}$ for every ${A\\!D\\!S}$ case.

Table 1: Segmentation results.

| Lymph Node |  | Wing Disc
---|---|---|---
Method | JAC | PR | RE | F1 | Method | meanIU | F1
U-Net [26] | 0.661 | 0.834 | 0.761 | 0.796 |  U-Net [26] | 0.948 | 0.954
CUMedNet [4] | 0.759 | 0.847 | 0.880 | 0.863 |  CUMedNet [4] | 0.939 | 0.945
Zhang et al. [31] | 0.810 | 0.901 | 0.889 | 0.895 |  Liang et al. [13] | 0.962 | 0.968
$DDN$ | 0.804 | 0.880 | 0.881 | 0.886 | $DDN$ | 0.968 | 0.972
$DDN+ADS$ | 0.827 | 0.903 | 0.893 | 0.908 | $DDN+ADS$ | 0.972 | 0.975

| Melanoma |  | BraTS |
---|---|---|---|---
Method | JAC | SE | SP | Dice | Method | Dice | SE | SP
Li et al. [12] | 0.753 | 0.855 | 0.974 | 0.839 |  FCN [16] |  0.825$\pm$0.002 | 0.818 | 0.998
Yuan et al. [30] | 0.765 | 0.825 | 0.975 | 0.849 |  U-Net [26] |  0.844$\pm$0.003 | 0.834 | 0.998
Bisla et al. [3] | 0.770 | - | - | - |  Dong et al. [6] | 0.840 | - | -
$DDN$ | 0.772 | 0.854 | 0.966 | 0.865 | $DDN$ |  0.851$\pm$0.003 | 0.838 | 0.999
$DDN+ADS$ | 0.784 | 0.852 | 0.976 | 0.872 | $DDN+ADS$ |  0.864$\pm$0.002 | 0.840 | 0.999

Figure 4: (a) A wing-disc test image. (b) Ground truth (GT). (c) GT in red,
FCN (output) in green, and FCN+ADS in blue. (d) GT in red, DDN-ICC in green,
and DDN-ICC+ADS in blue. (e) GT in red, DDN-ICC in green, and DDN+ADS in blue.
(f) GT in red, DDN-OCC+ADS in green, and DDN+ADS in blue. (g) GT in red,
DDN+ADS∗ (ADS from (a) in Fig. 1) in green, DDN+ADS in blue (’-’: without;
’+’: with).

Evaluation and Ablation Study: Table 1 summarizes quantitative results
obtained for all the four datasets, while Fig. 3 shows qualitative results for
example test cases. Similar post-processing and evaluation metrics as the
existing state-of-the-art are used for each case. For lymph node,
${D\\!D\\!N}+{A\\!D\\!S}$ achieves the state-of-the-art accuracy in all the 4
metrics while reducing the number of network parameters by $\sim 50\%$.
${D\\!D\\!N}$ itself attains the state-of-the-art accuracy for wing disc
segmentation. Implementing ${A\\!D\\!S}$ on top of $DDN$ further improves the
performance. For the melanoma dataset, ${D\\!D\\!N}+{A\\!D\\!S}$ shows more
impressive accuracy gain with $\sim 3\%$ increase in Dice coefficient.
Following [30], we explored different inputs like stacking color channels
(RGB+HSV, RGB+HSV+L), but did not observe any significant change in accuracy.
Similar improved accuracy is also observed on BraTS data. Isensee et al. [9]
has reported higher Dice (0.895) compared to ${D\\!D\\!N}+{A\\!D\\!S}$,
however, using both HGG and LGG data. Further, we believe ${D\\!D\\!N}$
implementing 2D filters cannot extract inter-slice context from 3D volumes.
Ablation Study: Benefit of ${A\\!D\\!S}$ is provided in the last two rows of
the respective dataset in Table 1 and is also shown in Fig. 4(c), (d) and (e)
(magenta area highlighting increase in true positive cases). We also performed
similar studies on U-Net [26] and FCN [16] type base networks (Fig. 4(c)).
Similarly contributions of ${I\\!C\\!C}$ and ${O\\!C\\!C}$ are studied and one
example case is provided in Fig. 4(d), (e), and (f). Further experiments on
${A\\!D\\!S}$ location selection (Fig. 4(g)), corroborates our framework. More
results can be found in Suppl. Materials.

##  4 Conclusions

In this work, we introduced densely decoded networks (${D\\!D\\!N}$) with
‘crutch’ connections, for enhanced localization and segmentation output
generation. Further, we proposed adaptive deep supervision (${A\\!D\\!S}$)
that uses input data to guide the auxiliary supervision placement,
demonstrating its ability in robust feature extraction. We exhibited the
effectiveness of ${D\\!D\\!N}$ and ${A\\!D\\!S}$ by conducting experiments on
four datasets of different modalities.

## References

  * [1] Bakas, S., et al.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Scientific Data 4 (09 2017). https://doi.org/10.1038/sdata.2017.117
  * [2] Bakas, S., et al.: Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge. CoRR abs/1811.02629 (2018)
  * [3] Bisla, D., et al.: Skin lesion segmentation and classification with deep learning system. CoRR abs/1902.06061 (2019)
  * [4] Chen, H., et al.: Deep contextual networks for neuronal structure segmentation. In: AAAI (2016)
  * [5] Codella, N.C.F., et al.: Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC). CoRR abs/1710.05006 (2017)
  * [6] Dong, H., et al.: Automatic brain tumor detection and segmentation using U-Net based fully convolutional networks. CoRR abs/1705.03820 (2017)
  * [7] He, K., et al.: Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. arXiv e-prints arXiv:1502.01852 (2015)
  * [8] Huang, G., Liu, Z., Weinberger, K.Q.: Densely connected convolutional networks. CoRR abs/1608.06993 (2016), http://arxiv.org/abs/1608.06993
  * [9] Isensee, F., et al.: Brain tumor segmentation and radiomics survival prediction: Contribution to the BRATS 2017 challenge. CoRR abs/1802.10508 (2018)
  * [10] LeCun, Y., et al.: Backpropagation applied to handwritten zip code recognition. Neural Comput. (1989)
  * [11] Lee, C., Xie, S., et al.: Deeply-supervised nets. arXiv e-prints arXiv:1409.5185 (2014)
  * [12] Li, Y., Shen, L.: Skin lesion analysis towards melanoma detection using deep learning network. CoRR abs/1703.00577 (2017)
  * [13] Liang, P., et al.: A new registration approach for dynamic analysis of calcium signals in organs. In: ISBI (2018)
  * [14] Lin, T., et al.: Feature pyramid networks for object detection. CoRR abs/1612.03144 (2016)
  * [15] Lin, T., et al.: Focal loss for dense object detection. CoRR abs/1708.02002 (2017)
  * [16] Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. CoRR abs/1411.4038 (2014)
  * [17] Luo, W., et al.: Understanding effective receptive field in deep convolutional neural networks. In: NIPS (2016)
  * [18] Menze, B.H., et al.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Transactions on Medical Imaging 34(10), 1993–2024 (Oct 2015)
  * [19] Milletari, F., Navab, N., Ahmadi, S.: V-Net: Fully convolutional neural networks for volumetric medical image segmentation. CoRR abs/1606.04797 (2016)
  * [20] Mishra, S., et al.: CC-NET: Image complexity guided network compression for biomedical image segmentation. In: ISBI 2019. pp. 57–60 (April 2019)
  * [21] Mishra, S., Chen, D.Z., Hu, X.S.: A data-aware deep supervised method for retinal vessel segmentation. In: ISBI. pp. 1254–1257 (2020)
  * [22] Mishra, S., Chen, D.Z., Hu, X.S.: Objective-dependent uncertainty driven retinal vessel segmentation. In: ISBI. pp. 453–457 (2021). https://doi.org/10.1109/ISBI48211.2021.9433774
  * [23] Mishra, S., Zhang, Y., Chen, D.Z., Sharon Hu, X.: Data-driven deep supervision for medical image segmentation. IEEE Transactions on Medical Imaging (2022). https://doi.org/10.1109/TMI.2022.3143371
  * [24] Mishra, S., Zhang, Y., Zhang, L., Zhang, T., Hu, X.S., Chen, D.Z.: Data-driven deep supervision for skin lesion classification. In: MICCAI. pp. 721–731 (2022)
  * [25] Perez, F., Avila, S., Valle, E.: Solo or ensemble? choosing a CNN architecture for melanoma classification. CoRR abs/1904.12724 (2019)
  * [26] Ronneberger, O., et al.: U-Net: Convolutional networks for biomedical image segmentation. ArXiv e-prints (May 2015)
  * [27] Srivastava, N., et al.: Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 15, 1929–1958 (2014)
  * [28] Szegedy, C., Liu, W., et al.: Going deeper with convolutions. CoRR abs/1409.4842 (2014), http://arxiv.org/abs/1409.4842
  * [29] Yu, L., et al.: Automatic 3D cardiovascular MR segmentation with densely-connected volumetric ConvNets. CoRR abs/1708.00573 (2017)
  * [30] Yuan, Y., Lo, Y.: Improving dermoscopic image segmentation with enhanced convolutional-deconvolutional networks. CoRR abs/1709.09780 (2017)
  * [31] Zhang, Y., et al.: Decompose-and-integrate learning for multi-class segmentation in medical images. In: MICCAI (2019)