section[64pt] [] subsection[64pt] *subsubsection[74pt] [ – ]
scale=1,color=black,opacity=1.0,angle=0, contents=

Technical Report RT-MAC-2021-01
Department of Computer Science – IME
University of São Paulo

Yorah Bosse<EMAIL_ADDRESS>
Federal University of Mato Grosso do Sul (Brazil)
Igor Scaliante Wiese<EMAIL_ADDRESS>
Federal Univ.of Technology - Paraná (Brazil)
Marco Aurélio Graciotto Silva<EMAIL_ADDRESS>
Federal Univ.of Technology - Paraná (Brazil)
Nelson Lago<EMAIL_ADDRESS>
University of São Paulo (Brazil)
Leônidas de Oliveira Brandão<EMAIL_ADDRESS>
University of São Paulo (Brazil)
David Redmiles<EMAIL_ADDRESS>
University of California (USA)
Fabio Kon<EMAIL_ADDRESS>
University of São Paulo (Brazil)
Marco A. Gerosa<EMAIL_ADDRESS>
Northern Arizona University (USA) / University of São Paulo (Brazil)

2021

Contents

######

  1. 1 Introduction
  2. 2 Methodology
    1. 2.1 Data Collection
    2. 2.2 Data Analysis
  3. 3 Antipatterns Catalogs
    1. 3.1 Description of the Antipatterns Format
  4. 4 Final Consideration
    1. 4.1 Future Work
  5. Acknowledgment
  6. A Statements of the Chosen Exercises
  7. B Suggested Solution for Professors and Students
  8. C Antipattern Catalog in C
  9. D Antipattern Catalog in Python
  10. E Antipattern Catalog in Both, C and Python

# Catalogs of C and Python
Antipatterns by CS1 Students

Yorah Bosse, Igor Scaliante Wiese, Marco Aurélio Graciotto Silva, Nelson Lago,
Leônidas de Oliveira Brandão, David Redmiles, Fabio Kon, Marco A. Gerosa

###### Abstract.

Understanding students’ programming misconceptions is critical. Doing so
depends on identifying the reasons why students make errors when learning a
new programming language. Knowing the misconceptions can help students to
improve their reflection about their mistakes and also help instructors to
design better teaching strategies. In this technical report, we propose
catalogs of antipatterns for two programming languages: C and Python. To
accomplish this, we analyzed the codes of 166 CS1 engineering students when
they were coding solutions to programming exercises. In our results, we
catalog 41 CS1 antipatterns from 95 cataloged misconceptions in C and Python.
These antipatterns were separated into three catalogs: C, Python, and
antipatterns found in code using both programming languages. For each
antipattern, we present code examples, students’ solutions (if they are
present), a possible solution to avoid the antipattern, among other
information.

Keywords: CS1, misconceptions, antipatterns, programming learning

## 1 Introduction

Learning to program is a complex (mhashi2013difficulties, 11) and arduous task
(robins2003learning, 13, 8, 9), as evidenced by the number of studies
considering the student failure rates shown in the literature
(watson2014failure, 17, 12, 4, 2, 10).

Understanding the mistakes made by CS1 students when learning to program can
provide benefits to the students’ growth. Such mistakes are an essential part
of the learning process, and research has shown that students can learn from
error (streumer2006world, 16, 6).

In this study, that was a part of the thesis entitled “Patterns of
Difficulties Related to Programming Learning” (bosse2020thesis, 3), we
observed how a group of CS1 students approached and solved the programming
exercises proposed throughout their “introduction to programming” class. Our
goal was to identify frequent misconceptions made by them when developing code
and thereby define a set of antipatterns.

A pattern is the description of a problem that occurs repeatedly in our
environment together with the core of a usable solution to that problem
(alexander1977pattern, 1). An antipattern, in turn, is a common response that
looks like an appropriate and effective solution, but resulting in negative
consequences (brown1998antipatterns, 5). (brown1998antipatterns, 5) further
add that an antipattern is the result of lack of knowledge or the application
of a “good” pattern in the wrong context (brown1998antipatterns, 5). In
addition, they state that mistakes that occurred at least three times in
different situations becomes a pattern (antipattern).

In this technical report, we are presenting a set of antipattern catalogs for
C, Python, and the intersection of both, in addition to directions on how to
learn from them. These antipattern catalogs may help instructors to teach and
students to learn how to program, reducing difficulties faced by CS1 students.
They also may play a key role in research that aims to develop programming
oriented teaching-learning systems, giving greater conditions for developing
helpful feedback not only on reducing syntax errors but also on decreasing
semantic errors and misleading programming styles.

## 2 Methodology

The code analyzed in this research came from introductory programming classes
taught for engineering courses at the University of São Paulo, Brazil, in
2017. In the following subsections, we detail the steps of data collection and
analysis of this study.

### 2.1 Data Collection

The data collection happened in a distance learning course with no face-to-
face classes, where all 3 exams were performed on-site. To enroll in this
course, students had to have taken the compulsory CS1 face-to-face course in
the previous year, had to have at least 70% attendance, and had to have failed
it. There were initially 166 students, 34 of them using the C programming
language (Electrical and Computational Engineering students) and 132 students
using the Python language (other engineering courses). Each student had to use
the same language adopted in their previous face-to-face course when they
failed.

The course took 16 weeks. In each week, students had to solve a set of
practical exercises, with automatic evaluation, using Moodle with the VPL
plugin (rodriguez2012virtual, 14). Every compilation had the code and its
result recorded. These exercises were mandatory and were considered in the
final grade, weighting one fifth of the total grade. There were 62 exercises,
approximately 4 per week. The weekly exercises were chosen by the instructor
to address concepts of a particular set of topics covering the main topics in
programming languages (e.g., input/output, variables, selection and repetition
structure, functions, array).

### 2.2 Data Analysis

For the analysis, we chose exercises covering all of the content taught in the
course. The Appendix A present the statements of the 15 chosen exercises.
Exercises were grouped by topic and we chose representative exercises giving
preference for the first ones in each topic. From the 15 chosen exercises, we
analyzed 2,233 student code submissions (1,253 in C and 980 in Python). All
students, $\sim$20% using C and $\sim$80% using Python, solved the same
exercises.

After choosing which exercises to explore, the choice of code submissions to
be analyzed for each of them was random, but giving preference to students’
codes with up to 20 submissions. For each exercise, a minimum of 50
submissions was analyzed.

During the analysis, several misconceptions were found. Each new mistake found
was cataloged in a new datasheet, and those that already existed were added as
a new event in a existing one.

We used the following criteria to classify the 3 types of errors: syntax,
semantics or style. Syntax errors detected by the compiler; semantic errors
identified by the wrong results; and the Style errors considered not
appropriate because they could generate future problems in code development or
poor programming practice (rogers2014aces, 15).

When we finished analysing the codes and classifying the mistakes, the
antipattern catalogs of each language were organized with the errors that
presented themselves in at least three distinct student events. Next, we
organized a catalog with the antipatterns that appear in both languages.

## 3 Antipatterns Catalogs

Codes developed in C and Python were recorded and analyzed during CS1 taught
for engineering courses at the University of São Paulo, Brazil. We found 95
types of misconceptions made by students using C and 44 by students using
Python.

The misconceptions with at least three registered events yielded three
antipattern catalogs (Table 1). The first catalog presents the antipatterns
found in codes developed with the C programming language, a total of 21 (rows
1 to 21). The second catalog presents the antipatterns found in Python,
totaling 11 (rows 31 to 41). The third one presents the 9 that occurred in
both C and Python programming languages (rows 22 to 30).

Table 1. Summary of the antipattern catalogs.

The last five columns of Table 1 show the total of events found, in how many
of these events the student was unable to solve the problem and the average,
median, and mode of submissions necessary to correct the error. Appendices C,
D, and E show the complete datasheets of all antipatterns that make up each
catalog, according to the format explained in the Figures 1, 2, and 3.

### 3.1 Description of the Antipatterns Format

The format adopted for the datasheets is an adaptation of the presentation
template of a pattern suggested by (alexander1977pattern, 1). Each datasheet
of the catalog consists of three parts. The first part serves to identify the
antipattern in a unique way. Figure 1 shows the information that makes up this
part of the datasheets.

Figure 1. Explanation of the first part of the antipatterns datasheets.

The second part is intended to show in detail the occurrences found. There are
three to four events presented to characterize an antipattern. Information
such as in which submission/compilation the mistake occurred and in which it
was solved helps, for example, to know if the student solved the antipattern
and, if so, to show how and how much ’time’ was necessary for that (Figure 2).
Appendix A shows the statements of the exercises identified by item “The
exercise that was being solved.”

Figure 2. Explanation of the second part of the antipatterns datasheets.

The last part serves to suggest a step by step of how professors can present
the antipattern to their students, and also suggestion of a step by step of
how students can study it by themselves (Figure 3). Appendix B shows the
statements of the exercises identified by item “The exercise that was being
solved.”

Figure 3. Explanation of the third part of the antipatterns datasheets.

## 4 Final Consideration

We found 139 types of misconceptions in C and Python, which 41 of them were
identified as antipatterns in the CS1 students’ code. Professors could work
these antipatterns while teaching the corresponding content, avoiding the time
spent looking for and solving problems in the codes developed by their
students.

The data exposed in the last five columns of Table 1 can provide vital
information for professors. With this data, the professor can identify the
antipatterns that most occurred, those that students typically failed to
resolve, those that took a long time for students to find a solution, among
much other information.

These catalogs can be another step in a broader movement to lift and
categorize the specific difficulties faced by students who may help professors
to teach and students to learn programming, as well as researchers in the
development of new research and tools to assist them in these tasks.

### 4.1 Future Work

As a future work, a study that verifies the impact on the CS1 students’ code
in case the antipatterns were constantly highlighted during the introductory
programming classes. Besides, the creation of a questions repository could be
helpful. This repository’s goal would be to address each antipattern in many
different ways, looking to develop some essential skills for programmers, like
finding errors in their code. According to (gomes2014teacher, 7), one way to
help students is with well-designed, carefully thought out exercises,
reflecting how, which, when, and why each exercise would be presented. A
question repository could help the professors when choosing and varying the
exercises used in each situation.

## Acknowledgment

The first author was supported by a scholarship from the CAPES Foundation, an
agency under the Ministry of Education of Brazil – Program name: PDSE–Programa
de Doutorado Sanduiche no Exterior (Doctoral Program Abroad)/Process number:
PDSE – 88881.135066/2016-01), to study in the University of California –
Irvine, and also supported by UFMS – University of Mato Grosso do Sul to
develop the thesis. In addition, this study was financed in part by the
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brazil (CAPES) –
Finance Code 001.

## References

  * (1) Christopher Alexander et al.  “A pattern language”  Oxford University Press, New York, 1977
  * (2) Jens Bennedsen and Michael E Caspersen  “Failure rates in introductory programming: 12 years later”  In _ACM Inroads_ 10.2 ACM, 2019, pp. 30–36
  * (3) Yorah Bosse  “Patterns of Dificulties Related to Programming Learning (Text in Portuguese)”, 2020
  * (4) Yorah Bosse and Marco Aurélio Gerosa  “Why is programming so difficult to learn?: Patterns of Difficulties Related to Programming Learning Mid-Stage”  In _ACM SIGSOFT Software Engineering Notes_ 41.6 ACM, 2017, pp. 1–6
  * (5) William H Brown, Raphael C Malveau, Hays W McCormick and Thomas J Mowbray  “AntiPatterns: refactoring software, architectures, and projects in crisis”  John Wiley & Sons, Inc., 1998
  * (6) Per-Erik Ellström  “Integrating learning and work: Problems and prospects”  In _Human resource development quarterly_ 12.4 Wiley Online Library, 2001, pp. 421–435
  * (7) Anabela Gomes and Antonio Mendes  “A teacher’s view about introductory programming teaching and learning: Difficulties, strategies and motivations”  In _2014 IEEE Frontiers in Education Conference (FIE) Proceedings_ , 2014, pp. 1–8  IEEE  DOI: 10.1109/FIE.2014.7044086
  * (8) Tony Jenkins  “On the difficulty of learning to program”  In _Proceedings of the 3rd Annual Conference of the LTSN Centre for Information and Computer Sciences_ 4.2002, 2002, pp. 53–58  Citeseer
  * (9) AM Kristi  “Problems in learning and teaching programming-a literature study for developing visualizations in the Codewitz-Minerva Project”  In _Codewitz Need Analysis, Institute of Software System, Tampere University of Technology, Finland_ , 2003, pp. 1–12
  * (10) Andrew Luxton-Reilly et al.  “Pass rates in STEM disciplines including computing”  In _Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education_ , 2019, pp. 267–268  ACM
  * (11) Mahmoud M Mhashi and ALIM Alakeel  “Difficulties facing students in learning computer programming skills at Tabuk University”  In _Proceedings of the 12th International Conference on Education and Educational Technology (EDU’13), Iwate, Japan_ , 2013, pp. 15–24
  * (12) Leônidas Oliveira Brandão, Yorah Bosse and Marco Aurélio Gerosa  “Visual programming and automatic evaluation of exercises: An experience with a STEM course”  In _2016 IEEE Frontiers in Education Conference (FIE)_ , 2016, pp. 1–9  IEEE
  * (13) Anthony Robins, Janet Rountree and Nathan Rountree  “Learning and teaching programming: A review and discussion”  In _Computer science education_ 13.2 Taylor & Francis, 2003, pp. 137–172
  * (14) Juan Carlos Rodríguez-del-Pino, Enrique Rubio Royo and Zenón Hernández Figueroa  “A Virtual Programming Lab for Moodle with automatic assessment and anti-plagiarism features”, 2012
  * (15) Stephanie Rogers et al.  “ACES: Automatic evaluation of coding style”, 2014
  * (16) JN Streumer and M Kho  “The world of work-related learning”  In _Work-related learning_ Springer, 2006, pp. 3–49
  * (17) Christopher Watson and Frederick WB Li  “Failure rates in introductory programming revisited”  In _Proceedings of the 2014 conference on Innovation & technology in computer science education_, 2014, pp. 39–44  ACM

section[64pt] – []

##  Appendix A Statements of the Chosen Exercises

Below, we present the statements of the exercises chosen to analyze the codes
generated by the students of introduction to programming.

Exercise 1.1 –

Develop a code that reads an integer entered by the user and prints the value
typed.

Exercise 1.2 –

Develop a code that receives an integer (int), typed by the user, and prints
the square of that number.

Exercise 2.1 –

Develop a code that reads two integers typed by the user, and prints the
largest of the two numbers.

Exercise 2.2 –

Develop a code that reads two integers as inputs (say in variables $a$ and
$b$), and prints 1 if “$a<b$” and $0$ otherwise.

Exercise 3.1 –

Develop a code in C or Python that checks whether 3 numbers can represent the
angles of a triangle or not. Your algorithm should prompt the user to enter
the 3 natural numbers (representing angles in degrees) and print: “Sim” (Yes)
and the 3 numbers in the sequence entered if they add up to 180, and “NAO”
(NO) and the sum of the 3 numbers otherwise.

Exercise 3.2 –

Develop a code that reads three natural numbers as inputs and checks to see if
those numbers are Pythagorean. Three numbers are Pythagorean if the square of
the largest of them (hypotenuse) is equal to the sum of the square of the
other two. Your program should print: if Pythagorean, the value “1” and the
value of the hypotenuse squared; if not Pythagorean, only the value “0”.

Three numbers, represented by $h$, $a$, and $b$ are called Pythagorean if, and
only if, $h^{2}=a^{2}+b^{2}$. The name is because they correspond to the sides
length of a right triangle.

Exercise 3.3 –

Develop a C/Python program that reads a natural (say in the ”total” variable)
and computes the largest sum of consecutive naturals, starting at 1, that is
less than or equal to the value entered ($<=$ total).

Exercise 4.1 –

Develop a program with functions named “soma” (sum) and “media” (average),
both with 2 formal parameters of type “int”, which respectively return the sum
of the parameters and their average. Functions must have 2 integer parameters,
such as “soma(a, b)” and “media(a, b)”. Your main program should receive from
the user 2 integers, invoke with them first the sum function and print their
sum, then invoke media and print their arithmetic mean (invoking the
functions).

Exercise 4.2 –

Make a program in C / Python necessarily implementing a function named $fat$,
with one parameter $n$, which, given an integer $n$, returns the factorial of
$n$ if $n\geq 0$, otherwise return $-1$. Make a ”main” program that prompts
the user to type any integer and using the function $fat$ returns the
factorial of $n$, if $n\geq 0$, otherwise return $-1$.

Remember, the factorial function $fat(n)$ is defined only for naturals
(starting by zero), as follows: $fat(0)=1$ and $fat(n)=n*fat(n-1),n>0$. Thus,
for example, the factorial of $4$ (represented by $4!$) is computed as
follows: $fat(4)=4!=4.3.2.1=24$.

Therefor, it is necessary that your program prints $-1$ whenever the user
enter a negative integer (e.g., if the input is $n=-10$, the output is $-1$;
input $n=-21$ implys output $-1$).

Exercise 4.7 –

Develop a code that receives a positive natural number $n$ ($n\geq 1$) and
then $n$ lists of integer values, each list is ended by a $0$ (ending mark of
the subsequence), calculating the sum of all odd numbers in each subsequence
and print them (as soon as you read each $0$).

Examples: For the subsequences below, all with $n=3$

    Ex. 1. Inputs (including n): 3 0 0 0 => outputs: 0
    Ex. 2. Inputs (including n): 3 2 0 2 0 2 0 => outputs: 0
    Ex. 3. Inputs (including n): 3 1 0 3 0 5 0 => outputs: 1 3 5)

Exercise 7.1 –

Construct an algorithm that receives a positive natural n ($n>0$) and prints
the sum of the first n terms of the harmonic series defined below.

$H=1/1+1/2+1/3+1/4+\dots+1/k+\dots$

Exercise 7.2 –

Build an algorithm that takes a positive natural number and real value ($n$
and $x$), calculates and prints the value of $x\hat{\ }n$ (power from $x$ to
$n$). Use only the sum and product operators.

Exercise 7.4 –

Build a program that receives an integer $n$ and then $n$ characters as input
by printing the ASCII (American Standard Code for Information Interchange)
code for each character.

Exercise 8.1 –

1\. Implement a function called determineSeOrdenado— that takes as parameters
an integer $n$ and array of integers called $vet$. The function should return
$1$, if the elements in (that is, if 2. Make a values that are stored in the
array (which can have up to $40$ elements). Using the value returned by the
function of item 1, your program should print $1$, if the array is in
ascending order, and $0$ otherwise.

Exercise 13.1 –

Make a program in which the user enters positive integers $m$ and $n$ (Then
your program should print each line, followed by the “Thus, you must first
print the value of the equal sign and the sum of the number from line zero.
Then follow lines 1, 2, 3 to m-1.

##  Appendix B Suggested Solution for Professors and Students

Below, we present detailed descriptions of solutions for professors and
students mentioned in the antipattern catalogs.

FOR PROFESSORS

1\. Explanation using blackboard and projector – reinforce the concept using
Kahoot

It is important to make clear to the learner that this anti-pattern exists.
For this, a solution suggestion would be:

  1. 1.

Bring to class code snippets with antipatterns related to previously presented
topics with at most one error per line

  2. 2.

show them to the learners

  3. 3.

ask them to point out some error

  4. 4.

ask if someone in class could explain the consequences of the error for the
program

  5. 5.

reinforce the explanation, if possible, showing other examples of the same
error

  6. 6.

explain the correct way to write the command (line), without the error

  7. 7.

rewrite the line on the blackboard, without the error, or make the correction
visible on the projector

  8. 8.

repeat steps 3 to 7 as long as there are errors in the code Then (it is
necessary for the teacher’s computer and the learners’ phones or computers to
have Internet access):

  9. 9.

show the password for the previously prepared kahoot quiz and ask learners to
con- nect to kahoot with their phone

  10. 10.

If this method has not been used before with them, explain the “game”

  11. 11.

start the “game”, cheer each learner’s victory, and make notes on the
questions they fail most often

  12. 12.

In the end, present the results and go back to the exercises they had the most
diffi- culty with.

P.S: The teacher will obtain a valuable resource, as it will become clear on
which topics students need reinforcement and may address them further during
class.

2\. Groups working with step-by-step execution to understand errors –
reinforce with exercises

It is important to make clear to the learner that this anti-pattern exists.
For this, a solution suggestion would be:

  1. 1.

assemble teams of 2 or 3 students;

  2. 2.

give the teams codes with the error. Not necessarily the same code for all
teams;

  3. 3.

ask them to do a step-by-step execution of the code presented;

  4. 4.

then ask for a team to show the class what they have found and explain how
they got this result;

  5. 5.

ask others with the same snippet if they had different results. If so, ask
them to present to the class;

  6. 6.

now you (instructor), using visuals, simulating computer memory space, for in-
stance, show, correct or reinforce what was presented;

  7. 7.

repeat steps 3 through 6 if you have more than one type of exercise;

  8. 8.

give them exercises to solve in which they could make the mistakes mentioned
above;

  9. 9.

ask them to solve on the blackboard so you can correct them if the mistake
still appears.

3\. Programming in front of the students, making the error appear– reinforce
by asking students to develop new codes

It is important to make clear to the learner that this anti-pattern exists.
For this, a solution suggestion would be:

  1. 1.

ask learners to solve an exercise in which the error you want to address might
happen;

  2. 2.

go to the blackboard and expose a superficial outline of what the code needs
to do and in what sequence;

  3. 3.

use the projector to start programming on the computer in front of the
students;

  4. 4.

make the error appear;

  5. 5.

compile and ask the learners for suggestions on how the code could be fixed;

  6. 6.

test each suggestion and explain if necessary, the reasons why it did not
work;

  7. 7.

finish with working code projected for learners to see;

  8. 8.

give them one more exercise to solve;

  9. 9.

ask a learner to develop their code right on the computer to project for all;

  10. 10.

explain what is happening while the learner is typing;

  11. 11.

repeat steps 8 through 10 as many times as you like, using intensive practice
for learning.

4\. Apply the bench test (table test) in code samples with and without the
error, compare the results – reinforce by asking the students to solve some
exercises

It is important to make clear to the learner that this anti-pattern exists.
For this, a solution suggestion would be:

  1. 1.

show an example of the error to the learner. Make sure the only mistake in
your example is the one you want to address;

  2. 2.

make it very clear, highlighting the mistake with a colored circle or arrow;

  3. 3.

explain why it is wrong;

  4. 4.

do a bench test (table test) showing what happens;

  5. 5.

fix the code, and redo the bench test (table test);

  6. 6.

compare the results;

  7. 7.

make sure the learners understand by asking them to do one or two exercises
like this, but with other code snippets.

  8. 8.

correct during class, in the presence of the learners.

FOR STUDENTS

1\. Solve exercises, add code errors, and ask classmate to find them

Learners may study this antipattern to avoid it in the future with an
exercise:

> Note:It is important that students work with at least one peer and that they
> have a computer to work on the exercise.

  1. 1.

Study some of the antipatterns of a given topic

  2. 2.

Choose similar exercises that depend on the topic and solve them independently

  3. 3.

After the exercises are done and error-free with correct outputs, each one
copies their code to paper with added errors (each one chooses how many errors
to add)

  4. 4.

Switch papers

  5. 5.

Each one corrects their peer’s exercise, pointing out the errors found

  6. 6.

When done, learners check with their peer, who previously solved the exercise,
whether any problem slipped through or if all errors were found

  7. 7.

Each one explains the solution they developed to their peer, showing the
difficulties they had and how they solved them.

2\. Study one or more anti-patterns, introduce to classmate and reinforce with
exercise

Learners may study this antipattern to avoid it in the future with an
exercise:

> Note:It is important to do this with at least one peer.

  1. 1.

Choose a topic already seen in class and select different parts of the topic
for each learner to explain

  2. 2.

Each one is to study the material, prepare a quick presentation, and search
for examples to illustrate the explanation

  3. 3.

Search for at least 1 exercise to be solved after your presentation

  4. 4.

Choose whose turn it is to present

  5. 5.

Start the presentation. The explanation should clarify everyone’s
difficulties. Showcase the possible errors (antipatterns) that may occur. In
case some doubt remains, all may help search for answers if the presenter does
not know enough.

  6. 6.

Present the chosen exercise and ask colleagues to solve it

  7. 7.

Check it together with bench test (table test) of the found solutions

  8. 8.

Analyze what would happen if the antipattern existed in the code

  9. 9.

Repeat steps 4 to 9 as many times as necessary for all to present their part

3\. Introduce the error into a code and understand the consequences it
generates.

Learners may study this antipattern to avoid it in the future with an
exercise:

> Note:It is important to have a computer available to undertake the exercise.

  1. 1.

Copy into the development environment some completed exercise from a previous
class, a book, or the internet where the error (antipattern) may be inserted

  2. 2.

execute the program and reason about what it does

  3. 3.

insert the error (antipattern) in the code and try to execute it again

  4. 4.

If there is a compilation error, note how the compiler present the error, so
you can more easily understand what the compiler points to in cases of error

  5. 5.

If the program executes, note how results are presented, trying to understand
where the error is and how it behaves

  6. 6.

Do this with multiple exercises, until you are certain that you understand the
antipattern

##  Appendix C Antipattern Catalog in C

Below, we present the catalog of antipatterns found in code developed using C
as programming language.

See pages 1 of ./appendix/antipatterns_catalog_c.pdf

See pages 2- of ./appendix/antipatterns_catalog_c.pdf

##  Appendix D Antipattern Catalog in Python

Below, we present the catalog of antipatterns found in code developed using
Python as programming language.

See pages 1 of ./appendix/antipatterns_catalog_python.pdf

See pages 2- of ./appendix/antipatterns_catalog_python.pdf

##  Appendix E Antipattern Catalog in Both, C and Python

Below, we present the catalog of antipatterns found in code developed using
Python and C as well.

See pages 1 of ./appendix/antipatterns_catalog_c_python.pdf

See pages 2- of ./appendix/antipatterns_catalog_c_python.pdf


(eccv) Package eccv Warning: Package ‘hyperref’ is loaded with option
‘pagebackref’, which is *not* recommended for camera-ready version

11institutetext: Tsinghua Shenzhen International Graduate School, Tsinghua
University 22institutetext: College of Computer Science and Software
Engineering, Shenzhen University 33institutetext: Bytedance Inc.
44institutetext: Peng Cheng Laboratory
44email: {cshguo<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>
# MambaIR: A Simple Baseline for Image Restoration with State-Space Model

Hang Guo 11⋆⋆ Jinmin Li Equal Contribution11 Tao Dai 22
Zhihao Ouyang 33 Xudong Ren 11 Shu-Tao Xia 1144

###### Abstract

Recent years have seen significant advancements in image restoration, largely
attributed to the development of modern deep neural networks, such as CNNs and
Transformers. However, existing restoration backbones often face the dilemma
between global receptive fields and efficient computation, hindering their
application in practice. Recently, the Selective Structured State Space Model,
especially the improved version Mamba, has shown great potential for long-
range dependency modeling with linear complexity, which offers a way to
resolve the above dilemma. However, the standard Mamba still faces certain
challenges in low-level vision such as local pixel forgetting and channel
redundancy. In this work, we introduce a simple but effective baseline, named
MambaIR, which introduces both local enhancement and channel attention to
improve the vanilla Mamba. In this way, our MambaIR takes advantage of the
local pixel similarity and reduces the channel redundancy. Extensive
experiments demonstrate the superiority of our method, for example, MambaIR
outperforms SwinIR by up to 0.45dB on image SR, using similar computational
cost but with a global receptive field. Code is available at
https://github.com/csguoh/MambaIR.

###### Keywords:

Image Restoration State Space Model Mamba

##  1 Introduction

Image restoration, aiming to reconstruct a high-quality image from a given
low-quality input, is a long-standing problem in computer vision and further
has a wide range of sub-problems such as super-resolution, image denoising,
_etc_. With the introduction of modern deep learning models such as CNNs [70,
14, 37, 76, 12] and Transformers [8, 36, 9, 34, 11], state-of-the-art
performance has continued to be refreshed in the past few years.

To some extent, the increasing performance of deep restoration models largely
stems from the increasing network receptive field. First, a large receptive
field allows the network to capture information from a wider region, enabling
it to refer to more pixels to facilitate the reconstruction of the anchor
pixel. Second, with a larger receptive field, the restoration network can
extract higher-level patterns and structures in the image, which can be
crucial for some structure preservation tasks such as image denoising.
Finally, Transformer-based restoration methods which possess larger receptive
fields experimentally outperform CNN-based methods, and the recent art [9]
also points out that activating more pixels usually leads to better
restoration results.

Figure 1: The Effective Receptive Field (ERF) visualization [40] for EDSR
[37], RCAN [76], SwinIR [36], HAT [9], and the proposed MambaIR. A larger ERF
is indicated by a more extensively distributed dark area. Only the proposed
MambaIR achieves a significant global effective receptive field.

Despite possessing many attractive properties, it appears that there exists an
inherent choice dilemma between global receptive fields and efficient
computation for current image restoration backbones. For CNN-based restoration
networks [37, 76], although the effective receptive field is limited (as shown
in Fig. 1(a)), it is appropriate for resource-constrained device deployments
due to the favorable efficiency of convolution parallel operations. By
contrast, Transformer-based image restoration methods usually set the number
of tokens to the image resolution [36, 8, 9], therefore, despite the global
receptive field, directly using the standard Transformer [57] will come at an
unacceptable quadratic computational complexity. Moreover, employing some
efficient attention techniques such as shifted window attention [39] for image
restoration, usually comes at the expense of a globally effective receptive
field (as shown in Fig. 1(b)), and does not intrinsically escape out of the
trade-off between a global receptive field and efficient computation.

Recently, structured state-space sequence models (S4), especially the improved
version Mamba, have emerged as an efficient and effective backbone for
constructing deep networks [21, 55, 17, 46, 19]. This development hints at a
potential resolution to balancing global receptive field and computational
efficiency in image restoration. In detail, the discretized state space
equations in Mamba can be formalized into a recursive form and can model very
long-range dependencies when equipped with specially designed structured
reparameterization [20]. This means that Mamba-based restoration networks can
naturally activate more pixels, thus improving the reconstruction quality.
Furthermore, the parallel scan algorithm [19] renders Mamba to process each
token in a parallel fashion, facilitating efficient training on modern
hardware such as GPU. The above promising properties motivate us to explore
the potential of Mamba to achieve efficient long-range modeling for image
restoration networks.

However, the standard Mamba [19], which is designed for 1D sequential data in
NLP, is not a natural fit for image restoration scenarios. First, since Mamba
processes flattened 1D image sequences in a recursive manner, it can result in
spatially close pixels being found at very distant locations in the flattened
sequences, resulting in the problem of local pixel forgetting. Second, due to
the requirement to memorize the long sequence dependencies, the number of
hidden states in the state space equations is typically large, which can lead
to channel redundancy, thus hindering the learning of critical channel
representations.

To address the above challenges, we introduce MambaIR, a simple but very
effective benchmark model, to adapt Mamba for image restoration. MambaIR is
formulated with three principal stages. Specifically, the 1)Shallow Feature
Extraction stage employs a simple convolution layer to extract the shallow
feature. Then the 2)Deep Feature Extraction stage performs with several
stacked Residual State Space Blocks (RSSBs). As the core component of our
MambaIR, the RSSB is designed with local convolution to mitigate local pixel
forgetting when applying the vanilla Mamba to 2D images, and it is also
equipped with channel attention to reduce channel redundancy caused by the
excessive hidden state number. We also employ the learnable factor to control
the skip connection within each RSSB. Finally, the 3)High-Quality Image
Reconstruction stage aggregates both shallow and deep features to produce a
high-quality output image. Through possessing both a global effective
receptive field as well as linear computational complexity, our MambaIR serves
as a new alternative for image restoration backbones.

In short, our main contributions can be summarized as follows:

  * •

We are the first to work to adapt state space models for low-level image
restoration via extensive experiments to formulate MambaIR, which acts as a
simple but effective alternative for CNN- and Transformer-based methods.

  * •

We propose the Residue State Space Block (RSSB) which can boost the power of
the standard Mamba with local enhancement and channel redundancy reduction.

  * •

Extensive experiments on various tasks demonstrate our MambaIR outperforms
other strong baselines to provide a powerful and promising backbone solution
for image restoration.

##  2 Related Work

###  2.1 Image Restoration

Image restoration has been significantly advanced since the introduction of
deep learning by several pioneering works, such as SRCNN [14] for image super-
resolution, DnCNN [70] for image denoising, ARCNN [13] for JPEG compression
artifact reduction, _etc_. Early attempts usually elaborate CNNs with
techniques such as residual connection [29, 67, 6], dense connection [60, 78]
and others [31, 62, 18, 76, 12] to improve model representation ability.
Despite the success, CNN-based restoration methods typically face challenges
in effectively modeling global dependencies. As Transformers have been proven
to be a strong competitor for CNNs in multiple computer vision tasks [5, 15,
39], using Transformers for image restoration appears promising. Despite the
global receptive field, it still faces specific challenges from the quadratic
computational complexity of the self-attention mechanism [57]. To address
this, IPT [8] divides one image into several small patches and processes each
patch independently with self-attention. SwinIR [36] further introduces
shifted window attention [39] to improve the performance. In addition,
progress continues to be made in designing efficient attention mechanisms for
restoration [66, 9, 32, 11, 63, 10]. Nonetheless, efficient attention design
usually comes at the expense of global effective receptive fields, and the
dilemma of the trade-off between efficient computation and global modeling is
not essentially resolved.

###  2.2 State Space Models

State Space Models (SSMs) [22, 21, 55], stemming from classics control theory
[28], are recently introduced to deep learning as a competitive backbone for
state space transforming. The promising property of linearly scaling with
sequence length in long-range dependency modeling has attracted great interest
from searchers. For example, the Structured State-Space Sequence model (S4)
[21] is a pioneer work for the deep state-space model in modeling the long-
range dependency. Later, S5 layer [55] is proposed based on S4 and introduces
MIMO SSM and efficient parallel scan. Moreover, H3 [17] achieves promising
results that nearly fill the performance gap between SSMs and Transformers in
natural language. [46] further improve S4 with gating units to obtain the
Gated State Space layer to boost the capability. More recently, Mamba [19], a
data-dependent SSM with selective mechanism and efficient hardware design,
outperforms Transformers on natural language and enjoys linear scaling with
input length. Moreover, there are also pioneering works that adopt Mamba to
vision tasks such as image classification [38, 81], video understanding [58],
biomedical image segmentation [42] and others [26, 50]. In this work, we
explore the potential of Mamba to image restoration with restoration-specific
designs to serve as a simple but effective baseline for future work.

##  3 Methodology

###  3.1 Preliminaries

The recent advancements of the class of structured state-space sequence models
(S4) are largely inspired by the continuous linear time-invariant (LTI)
systems, which maps a 1-dimensional function or sequence
$x(t)\in\mathbb{R}\rightarrow y(t)\in\mathbb{R}$ through an implicit latent
state $h(t)\in\mathbb{R}^{N}$. Formally, this system can be formulated as a
linear ordinary differential equation (ODE):

$\displaystyle h^{\prime}(t)$
$\displaystyle={\rm\textbf{A}}h(t)+{\rm\textbf{B}}x(t),$ (1) $\displaystyle
y(t)$ $\displaystyle={\rm\textbf{C}}h(t)+{\rm\textbf{D}}x(t),$

where $N$ is the state size, ${\rm\textbf{A}}\in\mathbb{R}^{N\times N}$,
${\rm\textbf{B}}\in\mathbb{R}^{N\times 1}$,
${\rm\textbf{C}}\in\mathbb{R}^{1\times N}$, and
${\rm\textbf{D}}\in\mathbb{R}$.

After that, the discretization process is typically adopted to integrate Eq. 1
into practical deep learning algorithms. Specifically, denote $\rm\Delta$ as
the timescale parameter to transform the continuous parameters A, B to
discrete parameters $\overline{{\rm\textbf{A}}}$,
$\overline{{\rm\textbf{B}}}$. The commonly used method for discretization is
the zero-order hold (ZOH) rule, which is defined as follows:

$\displaystyle\overline{\rm\textbf{A}}$ $\displaystyle={\rm
exp}({\rm{\Delta\textbf{A}}}),$ (2) $\displaystyle\overline{\rm\textbf{B}}$
$\displaystyle=({\rm{\Delta\textbf{A}}})^{-1}({\rm
exp(\textbf{A})}-\textbf{I})\cdot{\rm\Delta\textbf{B}}.$

After the discretization, the discretized version of Eq. 1 with step size
$\rm\Delta$ can be rewritten in the following RNN form:

$\displaystyle h_{k}$
$\displaystyle=\overline{\rm\textbf{A}}h_{k-1}+\overline{\rm\textbf{B}}x_{k},$
(3) $\displaystyle y_{k}$
$\displaystyle={\rm\textbf{C}}h_{k}+{\rm\textbf{D}}x_{k}.$

Furthermore, the Eq. 3 can also be mathematically equivalently transformed
into the following CNN form:

$\displaystyle\overline{\rm\textbf{K}}$
$\displaystyle\triangleq(\mathrm{\textbf{C}}\overline{\mathrm{\textbf{B}}},{\rm{\textbf{C}}}\overline{\rm\textbf{A}}\overline{\rm\textbf{B}},\cdots,{\rm{\textbf{C}}}{\overline{\rm\textbf{A}}}^{L-1}\overline{\rm\textbf{B}}),$
(4) y $\displaystyle={\rm\textbf{x}}\circledast\overline{\rm\textbf{K}},$

where $L$ is the length of the input sequence, $\circledast$ denotes
convolution operation, and $\overline{\rm\textbf{K}}\in\mathbb{R}^{L}$ is a
structured convolution kernel.

The recent advanced state-space model, Mamba [19], have further improved
$\overline{\rm\textbf{B}}$, C and $\rm\Delta$ to be input-dependent, thus
allowing for a dynamic feature representation. The intuition of Mamba for
image restoration lies in its development on the advantages of S4 model.
Specifically, Mamba shares the same recursive form of Eq. 3, which enables the
model to memorize ultra-long sequences so that more pixels can be activated to
aid restoration. At the same time, the parallel scan algorithm [19] allows
Mamba to enjoy the same advantages of parallel processing as Eq. 4, thus
facilitating efficient training.

###  3.2 Overall Architecture

As shown in Fig. 2, our MambaIR consists of three stages: shallow feature
extraction, deep feature extraction, and high-quality reconstruction. Given a
low-quality (LQ) input image $I_{LQ}\in\mathbb{R}^{H\times W\times 3}$, we
first employ a $3\times 3$ convolution layer from the shallow feature
extraction to generate the shallow feature $F_{S}\in\mathbb{R}^{H\times
W\times C}$, where $H$ and $W$ represent the height and width of the input
image, and $C$ is the number of channels. Subsequently, the shallow feature
$F_{S}$ undergoes the deep feature extraction stage to acquire the deep
feature $F_{D}^{l}\in\mathbb{R}^{H\times W\times C}$ at the $l$-th layer,
$l\in\\{1,2,\cdots L\\}$. This stage is stacked by multiple Residual State-
Space Groups (RSSGs), with each RSSG containing several Residue State-Space
Blocks (RSSBs). Moreover, an additional convolution layer is introduced at the
end of each group to refine features extracted from RSSB. Finally, we use the
element-wise sum to obtain the input of the high-quality reconstruction stage
$F_{R}=F^{L}_{D}+F_{S}$, which is used to reconstruct the high-quality (HQ)
output image $I_{HQ}$.

Figure 2: The overall network architecture of our MambaIR, as well as the (a)
Residual State-Space Block (RSSB), the (b) Vision State-Space Module (VSSM),
and the (c) 2D Selective Scan Module (2D-SSM).

###  3.3 Residual State-Space Block

The block design in previous Transformer-based restoration networks [36, 66,
9, 11] mainly follow the Norm $\rightarrow$ Attention $\rightarrow$ Norm
$\rightarrow$ MLP flow. Although Attention and SSM can both model global
dependencies, however, we find these two modules behave differently (see
supplementary material for more details) and simply replacing Attention with
SSM can only obtain sub-optimal results. Therefore, it is promising to tailor
a brand-new block structure for Mamba-based restoration networks.

To this end, we propose the Residual State-Space Block (RSSB) to adapt the SSM
block for restoration. As shown in Fig. 2(a), given the input deep feature
$F_{D}^{l}\in\mathbb{R}^{H\times W\times C}$, we first use the LayerNorm (LN)
followed by the Vision State-Space Module (VSSM) [38] to capture the spatial
long-term dependency. Moreover, we also use learnable scale factor
$s\in\mathbb{R}^{C}$ to control the information from skip connection:

$Z^{l}=\mathrm{VSSM}(\mathrm{LN}(F_{D}^{l}))+s\cdot F_{D}^{l}.$ (5)

Furthermore, since SSMs process flattened feature maps as 1D token sequences,
the number of neighborhood pixels in the sequence is greatly influenced by the
flattening strategy. For example, when employing the four-direction unfolding
strategy of [38], only four nearest neighbors are available to the anchor
pixel (see Fig. 3(a)), i.e., some spatially close pixels in 2D feature map are
instead distant from each other in the 1D token sequence, and this over-
distance can lead to local pixel forgetting. To this end, we introduce an
additional local convolution after VSSM to help restore the neighborhood
similarity. Specifically, we employ LayerNorm to first normalize the $Z^{l}$
and then use convolution layers to compensate for local features. In order to
maintain efficiency, the convolution layer adopts the bottleneck structure,
i.e., the channel is first compressed by a factor $\gamma$ to obtain features
with the shape $\mathbb{R}^{H\times W\times\frac{C}{\gamma}}$, then we perform
channel expansion to recover the original shape.

In addition, SSMs typically introduce a larger number of hidden states to
memorize very long-range dependencies, and we visualize the activation results
for different channels in Fig. 3(b) and find notable channel redundancy. To
facilitate the expressive power of different channels, we introduce the
Channel Attention (CA) [23] to RSSB. In this way, SSMs can focus on learning
diverse channel representations after which the critical channels are selected
by subsequent channel attention, thus avoiding channel redundancy. At last,
another tunable scale factor $s^{\prime}\in\mathbb{R}^{C}$ is used in residual
connection to acquire the final output $F_{D}^{l+1}$ of the RSSB. The above
process can be formulated as:

$F_{D}^{l+1}=\mathrm{CA(Conv(LN}(Z^{l})))+s^{\prime}\cdot Z^{l}.$ (6)

Figure 3: (a) Without using local enhancement will cause spatially close
pixels (area in the red box) get forgotten in the flattened 1D sequence due to
the long distance. (b) We use RELU and global average pooling on the VSSM
outputs from the last layer to get the channel activation values. Most
channels are not activated (i.e., channel redundancy) when channel attention
is not used.

###  3.4 Vision State-Space Module

To maintain efficiency, the Transformer-based restoration networks usually
divide input into small patches [8] or adopt shifted window attention [36],
hindering the interaction at the whole-image level. Motivated by the success
of Mamba in long-range modeling with linear complexity, we introduce the
Vision State-Space Module to image restoration.

The Vision State-Space Module (VSSM) can capture long-range dependencies with
the state space equation, and the architecture of VSSM is shown in Fig. 2(b).
Following [38], the input feature $X\in\mathbb{R}^{H\times W\times C}$ will go
through two parallel branches. In the first branch, the feature channel is
expanded to $\lambda C$ by a linear layer, where $\lambda$ is a pre-defined
channel expansion factor, followed by a depth-wise convolution, SiLU [54]
activation function, together with the 2D-SSM layer and LayerNorm. In the
second branch, the features channel is also expanded to $\lambda C$ with a
linear layer followed by the SiLU activation function. After that, features
from the two branches are aggregated with the Hadamard product. Finally, the
channel number is projected back to $C$ to generate output $X_{out}$ with the
same shape as input:

$\displaystyle X_{1}=\mathrm{LN(2D\text{-}SSM(SiLU(DWConv(Linear}(X))))),$ (7)
$\displaystyle X_{2}=\mathrm{SiLU(Linear}(X)),$ $\displaystyle
X_{out}=\mathrm{Linear}(X_{1}\odot X_{2}),$

where DWConv represents depth-wise convolution, and $\odot$ denotes the
Hadamard product.

###  3.5 2D Selective Scan Module

The standard Mamba [19] causally processes the input data, and thus can only
capture information within the scanned part of the data. This property is well
suited for NLP tasks that involve a sequential nature but poses significant
challenges when transferring to non-causal data such as images. To better
utilize the 2D spatial information, we follow [38] and introduce the 2D
Selective Scan Module (2D-SSM). As shown in Fig. 2(c), the 2D image feature is
flattened into a 1D sequence with scanning along four different directions:
top-left to bottom-right, bottom-right to top-left, top-right to bottom-left,
and bottom-left to top-right. Then the long-range dependency of each sequence
is captured according to the discrete state-space equation. Finally, all
sequences are merged using summation followed by the reshape operation to
recover the 2D structure.

###  3.6 Loss Function

To make a fair comparison with previous works [76, 36, 66], we optimize our
MambaIR with $L_{1}$ loss for image SR, which can be formulated as:

$\mathcal{L}=||I_{HQ}-I_{LQ}||_{1},$ (8)

where $||\cdot||_{1}$ denotes the $L_{1}$ norm. For image denoising, we
utilize the Charbonnier loss [7] with $\epsilon=10^{-3}$:

$\mathcal{L}=\sqrt{||I_{HQ}-I_{LQ}||^{2}+\epsilon^{2}}.$ (9)

##  4 Experiences

###  4.1 Experimental Settings

Dataset and Evaluation. Following the setup in previous works [36, 66], we
conduct experiments on various image restoration tasks, including image super-
resolution (i.e., classic SR, lightweight SR, and real SR) and image denoising
(i.e., Gaussian color image denoising and real-world denoising). We employ
DIV2K [56] and Flickr2K [37] to train classic SR models and use only DIV2K to
train lightweight SR models. Moreover, we use Set5 [4], Set14 [65], B100 [44],
Urban100 [24], and Manga109 [45] to evaluate the effectiveness of different SR
methods. For gaussian color image denoising, we utilize DIV2K [56], Flickr2K
[37], BSD500 [3], and WED [43] as our training datasets. Our testing datasets
for guassian color image denoising includes BSD68 [44], Kodak24 [16], McMaster
[74], and Urban100 [24]. For real image denoising, we train our model with 320
high-resolution images from SIDD [1] datasets, and use the SIDD test set and
DND [52] dataset for testing. Following [36, 76], we denote the model as
MambaIR+ when self-ensemble strategy [37] is used in testing. The performance
is evaluated using PSNR and SSIM on the Y channel from the YCbCr color space.
In addition, we also include other restoration tasks such as JPEG compression
artifact reduction, more experimental results can be seen in the supplementary
material.

Table 1: Ablation experiments for different design choices of RSSB.

settings | #param | MACs | Set5 | Set14 | B100 | Urban100 | Manga109
---|---|---|---|---|---|---|---
(0)baseline | 16.7M | 439G | 38.57 | 34.67 | 32.58 | 34.15 | 40.28
(1)+w/o Conv | 11.9M | 316G | 38.48 | 34.54 | 32.56 | 34.04 | 40.20
(2)+w/o Conv+CA | 11.8M | 315G | 38.55 | 34.64 | 32.57 | 34.06 | 40.14
(3)+replace with MLP | 14.3M | 379G | 38.55 | 34.68 | 32.59 | 34.22 | 40.13

Training Details. In accordance with previous works [36, 9, 66], we perform
data augmentation by applying horizontal flips and random rotations of
$90^{\circ},180^{\circ}$, and $270^{\circ}$. Additionally, we crop the
original images into $64\times 64$ patches for image SR and $128\times 128$
patches for image denoising during training. For image SR, we use the pre-
trained weights from the $\times$2 model to initialize those of $\times$3 and
$\times$4 and halve the learning rate and total training iterations to reduce
training time [37]. To ensure a fair comparison, we adjust the training batch
size to 32 for image SR and 16 for image denoising. We employ the Adam [30] as
the optimizer for training our MambaIR with $\beta_{1}=0.9,\beta_{2}=0.999$.
The initial learning rate is set at $2\times 10^{-4}$ and is halved when the
training iteration reaches specific milestones. Our MambaIR model is trained
with 8 NVIDIA V100 GPUs.

###  4.2 Ablation Study

Effects of different designs of RSSB. As the core component, the RSSB can
improve Mamba with restoration-specific priors. In this section, we ablate
different components of the RSSB. The results, presented in Tab. 1, indicate
that (1) directly using the standard Mamba to process flattened sequences can
lead to local pixel forgetting, and the utilization of simple convolution
layers can effectively enhance the local interaction. (2) Without using
additional convolution and channel attention, i.e., directly employing off-
the-shelf Mamba for restoration, can only obtain sub-optimal results, which
also supports our previous analysis. (3) Replacing Conv+ChanelAttention with
MLP, whose resulted structure will be similar to Transformer, also leads to
unfavorable results, indicating that although both SSMs and Attention have the
global modeling ability, the behavior of these two modules is different and
thus accustomed block structure should be considered for further improvements.

Figure 4: Qualitative comparison of our MambaIR with CNN and Transformer based
methods on classic image SR with scale $\times$4. Table 2: Ablation
experiments for different scan modes in VSSM.

scan mode | #param | MACs | Set5 | Set14 | B100 | Urban100 | Manga109
---|---|---|---|---|---|---|---
one-direction | 16.7M | 439G | 38.53 | 34.63 | 32.58 | 34.06 | 40.31
two-direction | 16.7M | 439G | 38.56 | 34.60 | 32.56 | 33.96 | 40.14
baseline | 16.7M | 439G | 38.57 | 34.67 | 32.58 | 34.15 | 40.28

Table 3: Quantitative comparison on classic image super-resolution with state-
of-the-art methods. The best and the second best results are in red and blue.

|  | Set5 | Set14 | BSDS100 | Urban100 | Manga109
---|---|---|---|---|---|---
Method | scale | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM
EDSR [37] | $\times 2$ | 38.11 | 0.9602 | 33.92 | 0.9195 | 32.32 | 0.9013 | 32.93 | 0.9351 | 39.10 | 0.9773
RCAN [76] | $\times 2$ | 38.27 | 0.9614 | 34.12 | 0.9216 | 32.41 | 0.9027 | 33.34 | 0.9384 | 39.44 | 0.9786
SAN [12] | $\times 2$ | 38.31 | 0.9620 | 34.07 | 0.9213 | 32.42 | 0.9028 | 33.10 | 0.9370 | 39.32 | 0.9792
HAN [51] | $\times 2$ | 38.27 | 0.9614 | 34.16 | 0.9217 | 32.41 | 0.9027 | 33.35 | 0.9385 | 39.46 | 0.9785
IGNN [79] |  $\times$2 | 38.24 | 0.9613 | 34.07 | 0.9217 | 32.41 | 0.9025 | 33.23 | 0.9383 | 39.35 | 0.9786
CSNLN [48] | $\times 2$ | 38.28 | 0.9616 | 34.12 | 0.9223 | 32.40 | 0.9024 | 33.25 | 0.9386 | 39.37 | 0.9785
NLSA [47] | $\times 2$ | 38.34 | 0.9618 | 34.08 | 0.9231 | 32.43 | 0.9027 | 33.42 | 0.9394 | 39.59 | 0.9789
ELAN [75] | $\times 2$ | 38.36 | 0.9620 | 34.20 | 0.9228 | 32.45 | 0.9030 | 33.44 | 0.9391 | 39.62 | 0.9793
IPT [8] | $\times 2$ | 38.37 | - | 34.43 | - | 32.48 | - | 33.76 | - | - | -
SwinIR [36] | $\times 2$ | 38.42 | 0.9623 | 34.46 | 0.9250 | 32.53 | 0.9041 | 33.81 | 0.9427 | 39.92 | 0.9797
SRFormer [80] | $\times 2$ | 38.51 | 0.9627 | 34.44 | 0.9253 | 32.57 | 0.9046 | 34.09 | 0.9449 | 40.07 | 0.9802
MambaIR | $\times 2$ | 38.57 | 0.9627 | 34.67 | 0.9261 | 32.58 | 0.9048 | 34.15 | 0.9446 | 40.28 | 0.9806
MambaIR+ | $\times 2$ | 38.60 | 0.9628 | 34.69 | 0.9260 | 32.60 | 0.9048 | 34.17 | 0.9443 | 40.33 | 0.9806
EDSR [37] |  $\times$3 | 34.65 | 0.9280 | 30.52 | 0.8462 | 29.25 | 0.8093 | 28.80 | 0.8653 | 34.17 | 0.9476
RCAN [77] |  $\times$3 | 34.74 | 0.9299 | 30.65 | 0.8482 | 29.32 | 0.8111 | 29.09 | 0.8702 | 34.44 | 0.9499
SAN [12] |  $\times$3 | 34.75 | 0.9300 | 30.59 | 0.8476 | 29.33 | 0.8112 | 28.93 | 0.8671 | 34.30 | 0.9494
HAN [51] |  $\times$3 | 34.75 | 0.9299 | 30.67 | 0.8483 | 29.32 | 0.8110 | 29.10 | 0.8705 | 34.48 | 0.9500
IGNN [79] |  $\times$3 | 34.72 | 0.9298 | 30.66 | 0.8484 | 29.31 | 0.8105 | 29.03 | 0.8696 | 34.39 | 0.9496
CSNLN [48] |  $\times$3 | 34.74 | 0.9300 | 30.66 | 0.8482 | 29.33 | 0.8105 | 29.13 | 0.8712 | 34.45 | 0.9502
NLSA [47] |  $\times$3 | 34.85 | 0.9306 | 30.70 | 0.8485 | 29.34 | 0.8117 | 29.25 | 0.8726 | 34.57 | 0.9508
ELAN [75] |  $\times$3 | 34.90 | 0.9313 | 30.80 | 0.8504 | 29.38 | 0.8124 | 29.32 | 0.8745 | 34.73 | 0.9517
IPT [8] |  $\times$3 | 34.81 | - | 30.85 | - | 29.38 | - | 29.49 | - | - | -
SwinIR [36] |  $\times$3 | 34.97 | 0.9318 | 30.93 | 0.8534 | 29.46 | 0.8145 | 29.75 | 0.8826 | 35.12 | 0.9537
SRformer [80] |  $\times$3 | 35.02 | 0.9323 | 30.94 | 0.8540 | 29.48 | 0.8156 | 30.04 | 0.8865 | 35.26 | 0.9543
MambaIR |  $\times$3 | 35.08 | 0.9323 | 30.99 | 0.8536 | 29.51 | 0.8157 | 29.93 | 0.8841 | 35.43 | 0.9546
MambaIR+ |  $\times$3 | 35.13 | 0.9326 | 31.06 | 0.8541 | 29.53 | 0.8162 | 29.98 | 0.8838 | 35.55 | 0.9549
EDSR [37] | $\times 4$ | 32.46 | 0.8968 | 28.80 | 0.7876 | 27.71 | 0.7420 | 26.64 | 0.8033 | 31.02 | 0.9148
RCAN [76] | $\times 4$ | 32.63 | 0.9002 | 28.87 | 0.7889 | 27.77 | 0.7436 | 26.82 | 0.8087 | 31.22 | 0.9173
SAN [12] | $\times 4$ | 32.64 | 0.9003 | 28.92 | 0.7888 | 27.78 | 0.7436 | 26.79 | 0.8068 | 31.18 | 0.9169
HAN [51] | $\times 4$ | 32.64 | 0.9002 | 28.90 | 0.7890 | 27.80 | 0.7442 | 26.85 | 0.8094 | 31.42 | 0.9177
IGNN [79] |  $\times$4 | 32.57 | 0.8998 | 28.85 | 0.7891 | 27.77 | 0.7434 | 26.84 | 0.8090 | 31.28 | 0.9182
CSNLN [48] | $\times 4$ | 32.68 | 0.9004 | 28.95 | 0.7888 | 27.80 | 0.7439 | 27.22 | 0.8168 | 31.43 | 0.9201
NLSA [47] | $\times 4$ | 32.59 | 0.9000 | 28.87 | 0.7891 | 27.78 | 0.7444 | 26.96 | 0.8109 | 31.27 | 0.9184
ELAN [75] | $\times 4$ | 32.75 | 0.9022 | 28.96 | 0.7914 | 27.83 | 0.7459 | 27.13 | 0.8167 | 31.68 | 0.9226
IPT [8] | $\times 4$ | 32.64 | - | 29.01 | - | 27.82 | - | 27.26 | - | - | -
SwinIR [36] | $\times 4$ | 32.92 | 0.9044 | 29.09 | 0.7950 | 27.92 | 0.7489 | 27.45 | 0.8254 | 32.03 | 0.9260
SRFormer [80] | $\times 4$ | 32.93 | 0.9041 | 29.08 | 0.7953 | 27.94 | 0.7502 | 27.68 | 0.8311 | 32.21 | 0.9271
MambaIR | $\times 4$ | 33.03 | 0.9046 | 29.20 | 0.7961 | 27.98 | 0.7503 | 27.68 | 0.8287 | 32.32 | 0.9272
MambaIR+ | $\times 4$ | 31.13 | 0.9054 | 29.25 | 0.7971 | 28.01 | 0.7510 | 27.80 | 0.8303 | 32.48 | 0.9281

Effects of Different Scan Modes in VSSM. To allow Mamba to process 2D images,
the feature map needs to be flattened before being iterated by the state-space
equation. Therefore, the unfolding strategy is particularly important. In this
work, we follow [38] which uses scans in four different directions to generate
scanned sequences. Here, we ablate different scan modes to study the effects,
the results are shown in Tab. 2. Compared with one-direction(top-left to
bottom-right) and two-direction(top-left to bottom-right, bottom-right to top-
left), using four directions of scanning allows the anchor pixel to perceive a
wider range of neighborhoods, thus achieving better results. Surprisingly,
using more scanning directions brings an almost negligible increase in the
number of parameters and computational complexity. This benefit may facilitate
future exploration of more effective scanning mechanisms for the Mamba-based
image restoration networks. We also include other ablation experiments, such
as the layer number of RSSBs, please see supplementary material for more
analysis.

###  4.3 Comparison on Image Super-Resolution

Classic Image Super-Resolution. Tab. 3 shows the quantitative results between
MambaIR and state-of-the-art super-resolution methods. Thanks to the
significant global receptive field, our proposed MambaIR achieves the best
performance on almost all five benchmark datasets for all scale factors. For
example, our Mamba-based baseline outperforms the Transformer-based benchmark
model SwinIR by 0.41dB on Manga109 for $\times 2$ scale, demonstrating the
prospect of Mamba for image restoration. We also give visual comparisons in
Fig. 4, and it can be seen that our method can facilitate the reconstruction
of sharp edges and natural textures.

Table 4: Model size comparisons. We compare the complexity on the $\times 4$
classic SR model of different methods, the output size is set to $3\times
640\times 640$ for MACs.

Method |  EDSR [37] |  RCAN [76] |  SRFBN [35] |  HAN [51] |  CSNLN [48] |  SwinIR [36] |  MambaIR (Ours)
---|---|---|---|---|---|---|---
Params (M) | 43.1 | 15.6 | 3.6 | 16.1 | 7.2 | 11.9 | 16.7
Mult-Adds $(\mathrm{G})$ | 1,286 | 407 | 498 | 420 | 103,640 | 336 | 439
PSNR on Urban100 (dB) | 26.64 | 26.82 | 26.60 | 26.85 | 27.22 | 27.45 | 27.68
PSNR on Manga109 (dB) | 31.02 | 31.22 | 31.15 | 31.42 | 31.43 | 32.03 | 32.32

Figure 5: Computational complexity comparison with different input scales. We
set the standard attention [57] which has a global receptive field as
baseline, and denote it as "Full-attn". We adjust the model to ensure the GPU
usage is roughly similar at the beginning, and then scale the input resolution
from $48\times 48$ to $84\times 84$.

Model Complexity Comparison. Tab. 4 provides the comparison of different
methods on parameters and MACs. Our MambaIR has comparable parameters and MACs
compared with SwinIR, but can achieve a global receptive field and better PSNR
performance. We further give comparison results of computational complexity
with input sizes in Fig. 5. As one can see, our method is far more efficient
than the full-attention baseline [57] and exhibits linear complexity with
input resolution which is similar to the efficient attention techniques such
as SwinIR. These observations above suggest that out MambaIR has similar scale
properties as shifted window attention, while possessing a global receptive
field similar to standard full attention.

Lightweight Image Super-Resolution. To demonstrate the scalability of our
method, we train the Mamba-light model and compare it with state-of-the-art
lightweight image SR methods. Following previous works [80, 41], we also
report the number of parameters (#param) and MACs (upscaling a low-resolution
image to $1280\times 720$ resolution). Tab. 5 shows the results. It can be
seen that our MambaIR-light outperforms SwinIR-light [36] by up to 0.17dB PSNR
on the $\times 4$ scale Manga109 dataset with similar parameters and MACs. The
performance results demonstrate the scalability and efficiency of our method.

Table 5: Quantitative comparison on lightweight image super-resolution with
state-of-the-art methods.

|  |  |  | Set5 | Set14 | BSDS100 | Urban100 | Manga109
---|---|---|---|---|---|---|---|---
Method | scale | #param | MACs | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM
CARN [2] |  $\times$2 | 1,592K | 222.8G | 37.76 | 0.9590 | 33.52 | 0.9166 | 32.09 | 0.8978 | 31.92 | 0.9256 | 38.36 | 0.9765
IMDN [25] |  $\times$2 | 694K | 158.8G | 38.00 | 0.9605 | 33.63 | 0.9177 | 32.19 | 0.8996 | 32.17 | 0.9283 | 38.88 | 0.9774
LAPAR-A [33] |  $\times$2 | 548K | 171.0G | 38.01 | 0.9605 | 33.62 | 0.9183 | 32.19 | 0.8999 | 32.10 | 0.9283 | 38.67 | 0.9772
LatticeNet [41] |  $\times$2 | 756K | 169.5G | 38.15 | 0.9610 | 33.78 | 0.9193 | 32.25 | 0.9005 | 32.43 | 0.9302 | - | -
SwinIR-light [36] |  $\times$2 | 878K | 195.6G | 38.14 | 0.9611 | 33.86 | 0.9206 | 32.31 | 0.9012 | 32.76 | 0.9340 | 39.12 | 0.9783
SRFormer-light [80] |  $\times$2 | 853K | 236G | 38.23 | 0.9613 | 33.94 | 0.9209 | 32.36 | 0.9019 | 32.91 | 0.9353 | 39.28 | 0.9785
Ours |  $\times$2 | 859K | 198.1G | 38.16 | 0.9610 | 34.00 | 0.9212 | 32.34 | 0.9017 | 32.92 | 0.9356 | 39.31 | 0.9779
CARN [2] |  $\times$3 | 1,592K | 118.8G | 34.29 | 0.9255 | 30.29 | 0.8407 | 29.06 | 0.8034 | 28.06 | 0.8493 | 33.50 | 0.9440
IMDN [25] |  $\times$3 | 703K | 71.5G | 34.36 | 0.9270 | 30.32 | 0.8417 | 29.09 | 0.8046 | 28.17 | 0.8519 | 33.61 | 0.9445
LAPAR-A [33] |  $\times$3 | 544K | 114.0G | 34.36 | 0.9267 | 30.34 | 0.8421 | 29.11 | 0.8054 | 28.15 | 0.8523 | 33.51 | 0.9441
LatticeNet [41] |  $\times$3 | 765K | 76.3G | 34.53 | 0.9281 | 30.39 | 0.8424 | 29.15 | 0.8059 | 28.33 | 0.8538 | - | -
SwinIR-light [36] |  $\times$3 | 886K | 87.2G | 34.62 | 0.9289 | 30.54 | 0.8463 | 29.20 | 0.8082 | 28.66 | 0.8624 | 33.98 | 0.9478
SRFormer-light [80] |  $\times$3 | 861K | 105G | 34.67 | 0.9296 | 30.57 | 0.8469 | 29.26 | 0.8099 | 28.81 | 0.8655 | 34.19 | 0.9489
Ours |  $\times$3 | 867K | 88.7G | 34.72 | 0.9296 | 30.63 | 0.8475 | 29.29 | 0.8099 | 29.00 | 0.8689 | 34.39 | 0.9495
CARN [2] |  $\times$4 | 1,592K | 90.9G | 32.13 | 0.8937 | 28.60 | 0.7806 | 27.58 | 0.7349 | 26.07 | 0.7837 | 30.47 | 0.9084
IMDN [25] |  $\times$4 | 715K | 40.9G | 32.21 | 0.8948 | 28.58 | 0.7811 | 27.56 | 0.7353 | 26.04 | 0.7838 | 30.45 | 0.9075
LAPAR-A [33] |  $\times$4 | 659K | 94.0G | 32.15 | 0.8944 | 28.61 | 0.7818 | 27.61 | 0.7366 | 26.14 | 0.7871 | 30.42 | 0.9074
LatticeNet [41] |  $\times$4 | 777K | 43.6G | 32.30 | 0.8962 | 28.68 | 0.7830 | 27.62 | 0.7367 | 26.25 | 0.7873 | - | -
SwinIR-light [36] |  $\times$4 | 897K | 49.6G | 32.44 | 0.8976 | 28.77 | 0.7858 | 27.69 | 0.7406 | 26.47 | 0.7980 | 30.92 | 0.9151
SRFormer-light [80] |  $\times$4 | 873K | 62.8G | 32.51 | 0.8988 | 28.82 | 0.7872 | 27.73 | 0.7422 | 26.67 | 0.8032 | 31.17 | 0.9165
Ours |  $\times$4 | 879K | 50.6G | 32.51 | 0.8993 | 28.85 | 0.7876 | 27.75 | 0.7423 | 26.75 | 0.8051 | 31.26 | 0.9175

Real-world Image Super-resolution. We also investigate the performance of the
network for real-world image restoration. We follow the training protocol in
[9] to train our MambaIR-real model. Since there are no ground-truth images
for this task, only the visual comparison is given in Fig. 6. Compared with
the other methods, our MambaIR exhibits a notable advancement in resolving
fine details and texture preservation, demonstrating the robustness of our
method.

Figure 6: Qualitative comparison with RealSR [27], ESRGAN [60], BSRGAN [69],
Real-ESRGAN [59], and SwinIR [36] on real image super-resolution with scale
$\times$4.

###  4.4 Comparison on Image Denoising

Gaussian Color Image Denoising. The results of gaussian color image denoising
are shown in Tab. 6. Following [67, 70], the compared noise levels include 15,
25 and 50. As one can see, our model achieves better performance than all
compared methods. In particular, it surpasses the SwinIR [36] by up to 0.25dB
with $\sigma$=50 on the Urban100 dataset. We also give a visual comparison in
Fig. 7. Thanks to the global receptive field, our MambaIR can achieve better
structure preservation, leading to clearer edges and natural shapes.

Table 6: Quantitative comparison on gaussian color image denoising with state-
of-the-art methods.

Method | BSD68 | Kodak24 | McMaster | Urban100
---|---|---|---|---
$\sigma$=15 |  $\sigma$=25 |  $\sigma$=50 |  $\sigma$=15 |  $\sigma$=25 |  $\sigma$=50 |  $\sigma$=15 |  $\sigma$=25 |  $\sigma$=50 |  $\sigma$=15 |  $\sigma$=25 |  $\sigma$=50
IRCNN [72] | 33.86 | 31.16 | 27.86 | 34.69 | 32.18 | 28.93 | 34.58 | 32.18 | 28.91 | 33.78 | 31.20 | 27.70
FFDNet [73] | 33.87 | 31.21 | 27.96 | 34.63 | 32.13 | 28.98 | 34.66 | 32.35 | 29.18 | 33.83 | 31.40 | 28.05
DnCNN [71] | 33.90 | 31.24 | 27.95 | 34.60 | 32.14 | 28.95 | 33.45 | 31.52 | 28.62 | 32.98 | 30.81 | 27.59
DRUNet [68] | 34.30 | 31.69 | 28.51 | 35.31 | 32.89 | 29.86 | 35.40 | 33.14 | 30.08 | 34.81 | 32.60 | 29.61
SwinIR [36] | 34.42 | 31.78 | 28.56 | 35.34 | 32.89 | 29.79 | 35.61 | 33.20 | 30.22 | 35.13 | 32.90 | 29.82
Restormer [63] | 34.40 | 31.79 | 28.60 | 35.47 | 33.04 | 30.01 | 35.61 | 33.34 | 30.30 | 35.13 | 32.96 | 30.02
MambaIR | 34.43 | 31.80 | 28.61 | 35.34 | 32.91 | 29.85 | 35.62 | 33.35 | 30.31 | 35.17 | 32.99 | 30.07
MambaIR+ | 34.44 | 31.82 | 28.64 | 35.35 | 32.92 | 29.87 | 35.63 | 33.36 | 30.32 | 35.17 | 32.99 | 30.06

Figure 7: Qualitative comparison of our MambaIR with other methods on color
image denoising task with noise level level $\sigma$=50.

Real Image Denoising. We further turn to the real image denoising task to
evaluate the robustness of MambaIR when facing real-world degradation.
Following [63], we adopt the progressive training strategy for fair
comparison. The results, shown in Tab. 7, suggest that our method achieves
comparable performance with existing state-of-the-art models Restormer [61]
and outperforms other methods such as Uformer [61] by 0.12dB PSNR on SIDD
dataset. The promising results indicate the ability of our method in real
image denoising.

Table 7: Quantitative comparison on real image denosing with state-of-the-art
methods.

| DeamNet [53] | MPRNet [64] | DAGL [49] | Uformer [61] | Restormer [63] | Ours
---|---|---|---|---|---|---
Dataset | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM
SIDD | 39.47 | 0.957 | 39.71 | 0.958 | 38.94 | 0.953 | 39.77 | 0.959 | 40.02 | 0.960 | 39.89 | 0.960
DND | 39.63 | 0.953 | 39.80 | 0.954 | 39.77 | 0.956 | 39.96 | 0.956 | 40.03 | 0.956 | 40.04 | 0.956

##  5 Conclusion

In this work, we explore for the first time the power of the recent advanced
state space model, i.e., Mamba, for image restoration, to help resolve the
dilemma of trade-off between efficient computation and global effective
receptive field. Specifically, we introduce the local enhancement to mitigate
the neighborhood pixel forgetting problem from the flattening strategy and
propose channel attention to reduce channel redundancy. Extensive experiments
on multiple restoration tasks demonstrate our MambaIR serves as a simple but
effective state-space model for image restoration.

## References

  * [1] Abdelhamed, A., Lin, S., Brown, M.S.: A high-quality denoising dataset for smartphone cameras. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1692–1700 (2018)
  * [2] Ahn, N., Kang, B., Sohn, K.A.: Fast, accurate, and lightweight super-resolution with cascading residual network. In: Proceedings of the European conference on computer vision (ECCV). pp. 252–268 (2018)
  * [3] Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE transactions on pattern analysis and machine intelligence 33(5), 898–916 (2010)
  * [4] Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding (2012)
  * [5] Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: European conference on computer vision. pp. 213–229. Springer (2020)
  * [6] Cavigelli, L., Hager, P., Benini, L.: Cas-cnn: A deep convolutional neural network for image compression artifact suppression. In: 2017 International Joint Conference on Neural Networks (IJCNN). pp. 752–759. IEEE (2017)
  * [7] Charbonnier, P., Blanc-Feraud, L., Aubert, G., Barlaud, M.: Two deterministic half-quadratic regularization algorithms for computed imaging. In: Proceedings of 1st international conference on image processing. vol. 2, pp. 168–172. IEEE (1994)
  * [8] Chen, H., Wang, Y., Guo, T., Xu, C., Deng, Y., Liu, Z., Ma, S., Xu, C., Xu, C., Gao, W.: Pre-trained image processing transformer. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 12299–12310 (2021)
  * [9] Chen, X., Wang, X., Zhou, J., Qiao, Y., Dong, C.: Activating more pixels in image super-resolution transformer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 22367–22377 (2023)
  * [10] Chen, Z., Zhang, Y., Gu, J., Kong, L., Yang, X.: Recursive generalization transformer for image super-resolution. arXiv preprint arXiv:2303.06373 (2023)
  * [11] Chen, Z., Zhang, Y., Gu, J., Kong, L., Yang, X., Yu, F.: Dual aggregation transformer for image super-resolution. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 12312–12321 (2023)
  * [12] Dai, T., Cai, J., Zhang, Y., Xia, S.T., Zhang, L.: Second-order attention network for single image super-resolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 11065–11074 (2019)
  * [13] Dong, C., Deng, Y., Loy, C.C., Tang, X.: Compression artifacts reduction by a deep convolutional network. In: Proceedings of the IEEE international conference on computer vision. pp. 576–584 (2015)
  * [14] Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part IV 13. pp. 184–199. Springer (2014)
  * [15] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
  * [16] Franzen, R.: Kodak lossless true color image suite (2021), http://r0k.us/graphics/kodak/
  * [17] Fu, D.Y., Dao, T., Saab, K.K., Thomas, A.W., Rudra, A., Ré, C.: Hungry hungry hippos: Towards language modeling with state space models. arXiv preprint arXiv:2212.14052 (2022)
  * [18] Fu, X., Zha, Z.J., Wu, F., Ding, X., Paisley, J.: Jpeg artifacts reduction via deep convolutional sparse coding. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 2501–2510 (2019)
  * [19] Gu, A., Dao, T.: Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752 (2023)
  * [20] Gu, A., Dao, T., Ermon, S., Rudra, A., Ré, C.: Hippo: Recurrent memory with optimal polynomial projections. Advances in neural information processing systems 33, 1474–1487 (2020)
  * [21] Gu, A., Goel, K., Ré, C.: Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.00396 (2021)
  * [22] Gu, A., Johnson, I., Goel, K., Saab, K., Dao, T., Rudra, A., Ré, C.: Combining recurrent, convolutional, and continuous-time models with linear state space layers. Advances in neural information processing systems 34, 572–585 (2021)
  * [23] Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 7132–7141 (2018)
  * [24] Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 5197–5206 (2015)
  * [25] Hui, Z., Gao, X., Yang, Y., Wang, X.: Lightweight image super-resolution with information multi-distillation network. In: Proceedings of the 27th acm international conference on multimedia. pp. 2024–2032 (2019)
  * [26] Islam, M.M., Hasan, M., Athrey, K.S., Braskich, T., Bertasius, G.: Efficient movie scene detection using state-space transformers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 18749–18758 (2023)
  * [27] Ji, X., Cao, Y., Tai, Y., Wang, C., Li, J., Huang, F.: Real-world super-resolution via kernel estimation and noise injection. In: proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. pp. 466–467 (2020)
  * [28] Kalman, R.E.: A new approach to linear filtering and prediction problems (1960)
  * [29] Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1646–1654 (2016)
  * [30] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  * [31] Lai, W.S., Huang, J.B., Ahuja, N., Yang, M.H.: Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 624–632 (2017)
  * [32] Li, W., Lu, X., Qian, S., Lu, J., Zhang, X., Jia, J.: On efficient transformer-based image pre-training for low-level vision. arXiv preprint arXiv:2112.10175 (2021)
  * [33] Li, W., Zhou, K., Qi, L., Jiang, N., Lu, J., Jia, J.: Lapar: Linearly-assembled pixel-adaptive regression network for single image super-resolution and beyond. Advances in Neural Information Processing Systems 33, 20343–20355 (2020)
  * [34] Li, Y., Fan, Y., Xiang, X., Demandolx, D., Ranjan, R., Timofte, R., Van Gool, L.: Efficient and explicit modelling of image hierarchies for image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 18278–18289 (2023)
  * [35] Li, Z., Yang, J., Liu, Z., Yang, X., Jeon, G., Wu, W.: Feedback network for image super-resolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 3867–3876 (2019)
  * [36] Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., Timofte, R.: Swinir: Image restoration using swin transformer. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 1833–1844 (2021)
  * [37] Lim, B., Son, S., Kim, H., Nah, S., Mu Lee, K.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops. pp. 136–144 (2017)
  * [38] Liu, Y., Tian, Y., Zhao, Y., Yu, H., Xie, L., Wang, Y., Ye, Q., Liu, Y.: Vmamba: Visual state space model. arXiv preprint arXiv:2401.10166 (2024)
  * [39] Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin transformer: Hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 10012–10022 (2021)
  * [40] Luo, W., Li, Y., Urtasun, R., Zemel, R.: Understanding the effective receptive field in deep convolutional neural networks. Advances in neural information processing systems 29 (2016)
  * [41] Luo, X., Xie, Y., Zhang, Y., Qu, Y., Li, C., Fu, Y.: Latticenet: Towards lightweight image super-resolution with lattice block. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXII 16. pp. 272–289. Springer (2020)
  * [42] Ma, J., Li, F., Wang, B.: U-mamba: Enhancing long-range dependency for biomedical image segmentation. arXiv preprint arXiv:2401.04722 (2024)
  * [43] Ma, K., Duanmu, Z., Wu, Q., Wang, Z., Yong, H., Li, H., Zhang, L.: Waterloo exploration database: New challenges for image quality assessment models. IEEE Transactions on Image Processing 26(2), 1004–1016 (2016)
  * [44] Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001. vol. 2, pp. 416–423. IEEE (2001)
  * [45] Matsui, Y., Ito, K., Aramaki, Y., Fujimoto, A., Ogawa, T., Yamasaki, T., Aizawa, K.: Sketch-based manga retrieval using manga109 dataset. Multimedia Tools and Applications 76, 21811–21838 (2017)
  * [46] Mehta, H., Gupta, A., Cutkosky, A., Neyshabur, B.: Long range language modeling via gated state spaces. arXiv preprint arXiv:2206.13947 (2022)
  * [47] Mei, Y., Fan, Y., Zhou, Y.: Image super-resolution with non-local sparse attention. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3517–3526 (2021)
  * [48] Mei, Y., Fan, Y., Zhou, Y., Huang, L., Huang, T.S., Shi, H.: Image super-resolution with cross-scale non-local attention and exhaustive self-exemplars mining. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 5690–5699 (2020)
  * [49] Mou, C., Zhang, J., Wu, Z.: Dynamic attentive graph learning for image restoration. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 4328–4337 (2021)
  * [50] Nguyen, E., Goel, K., Gu, A., Downs, G., Shah, P., Dao, T., Baccus, S., Ré, C.: S4nd: Modeling images and videos as multidimensional signals with state spaces. Advances in neural information processing systems 35, 2846–2861 (2022)
  * [51] Niu, B., Wen, W., Ren, W., Zhang, X., Yang, L., Wang, S., Zhang, K., Cao, X., Shen, H.: Single image super-resolution via a holistic attention network. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XII 16. pp. 191–207. Springer (2020)
  * [52] Plotz, T., Roth, S.: Benchmarking denoising algorithms with real photographs. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1586–1595 (2017)
  * [53] Ren, C., He, X., Wang, C., Zhao, Z.: Adaptive consistency prior based deep network for image denoising. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 8596–8606 (2021)
  * [54] Shazeer, N.: Glu variants improve transformer. arXiv preprint arXiv:2002.05202 (2020)
  * [55] Smith, J.T., Warrington, A., Linderman, S.W.: Simplified state space layers for sequence modeling. arXiv preprint arXiv:2208.04933 (2022)
  * [56] Timofte, R., Agustsson, E., Van Gool, L., Yang, M.H., Zhang, L.: Ntire 2017 challenge on single image super-resolution: Methods and results. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops. pp. 114–125 (2017)
  * [57] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017)
  * [58] Wang, J., Zhu, W., Wang, P., Yu, X., Liu, L., Omar, M., Hamid, R.: Selective structured state-spaces for long-form video understanding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6387–6397 (2023)
  * [59] Wang, X., Xie, L., Dong, C., Shan, Y.: Real-esrgan: Training real-world blind super-resolution with pure synthetic data. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 1905–1914 (2021)
  * [60] Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Qiao, Y., Loy, C.C.: Esrgan: Enhanced super-resolution generative adversarial networks. In: The European Conference on Computer Vision Workshops (ECCVW) (September 2018)
  * [61] Wang, Z., Cun, X., Bao, J., Zhou, W., Liu, J., Li, H.: Uformer: A general u-shaped transformer for image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 17683–17693 (June 2022)
  * [62] Wei, Y., Gu, S., Li, Y., Timofte, R., Jin, L., Song, H.: Unsupervised real-world image super resolution via domain-distance aware training. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13385–13394 (2021)
  * [63] Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.H.: Restormer: Efficient transformer for high-resolution image restoration. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 5728–5739 (2022)
  * [64] Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.H., Shao, L.: Multi-stage progressive image restoration. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 14821–14831 (2021)
  * [65] Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Curves and Surfaces: 7th International Conference, Avignon, France, June 24-30, 2010, Revised Selected Papers 7. pp. 711–730. Springer (2012)
  * [66] Zhang, J., Zhang, Y., Gu, J., Zhang, Y., Kong, L., Yuan, X.: Accurate image restoration with attention retractable transformer. In: ICLR (2023)
  * [67] Zhang, K., Li, Y., Zuo, W., Zhang, L., Van Gool, L., Timofte, R.: Plug-and-play image restoration with deep denoiser prior. IEEE Transactions on Pattern Analysis and Machine Intelligence 44(10), 6360–6376 (2021)
  * [68] Zhang, K., Li, Y., Zuo, W., Zhang, L., Van Gool, L., Timofte, R.: Plug-and-play image restoration with deep denoiser prior. IEEE Transactions on Pattern Analysis and Machine Intelligence 44(10), 6360–6376 (2021)
  * [69] Zhang, K., Liang, J., Van Gool, L., Timofte, R.: Designing a practical degradation model for deep blind image super-resolution. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 4791–4800 (2021)
  * [70] Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE transactions on image processing 26(7), 3142–3155 (2017)
  * [71] Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE transactions on image processing 26(7), 3142–3155 (2017)
  * [72] Zhang, K., Zuo, W., Gu, S., Zhang, L.: Learning deep cnn denoiser prior for image restoration. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3929–3938 (2017)
  * [73] Zhang, K., Zuo, W., Zhang, L.: Ffdnet: Toward a fast and flexible solution for cnn-based image denoising. IEEE Transactions on Image Processing 27(9), 4608–4622 (2018)
  * [74] Zhang, L., Wu, X., Buades, A., Li, X.: Color demosaicking by local directional interpolation and nonlocal adaptive thresholding. Journal of Electronic imaging 20(2), 023016–023016 (2011)
  * [75] Zhang, X., Zeng, H., Guo, S., Zhang, L.: Efficient long-range attention network for image super-resolution. In: European Conference on Computer Vision. pp. 649–667. Springer (2022)
  * [76] Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: ECCV (2018)
  * [77] Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European conference on computer vision (ECCV). pp. 286–301 (2018)
  * [78] Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2472–2481 (2018)
  * [79] Zhou, S., Zhang, J., Zuo, W., Loy, C.C.: Cross-scale internal graph neural network for image super-resolution. In: Advances in Neural Information Processing Systems (2020)
  * [80] Zhou, Y., Li, Z., Guo, C.L., Bai, S., Cheng, M.M., Hou, Q.: Srformer: Permuted self-attention for single image super-resolution. arXiv preprint arXiv:2303.09735 (2023)
  * [81] Zhu, L., Liao, B., Zhang, Q., Wang, X., Liu, W., Wang, X.: Vision mamba: Efficient visual representation learning with bidirectional state space model. arXiv preprint arXiv:2401.09417 (2024)


equidistant points on the continuous line between $\mbox{$\bf r$}^{a}$ and
$\mbox{$\bf r$}^{b}$ the closest grid point is incremented by
$w_{\alpha\beta}^{l}/k$. We do not care if sometimes a grid point gets several
or even all contributions (which happens if $a_{\mathrm{grid}}$ is large) or
none (which happens especially if the grid is too fine). All data presented in
other parts of this work have been obtained using Eq. (140) and Eq. (142) with
$k=20$ and a grid spacing $a_{\mathrm{grid}}\approx 0.2$. Different variants
are compared in Fig. 9 where we focus on the rescaled ICF $\beta
V\bar{c}_{\mathrm{N}}(q)$. We compare results obtained with

  * •

method “qA” using Eq. (136) and Eq. (138),

  * •

method “rA” using Eq. (141) and Eq. (142),

  * •

method “rB” using Eq. (141) and Eq. (143)

for different grid constants $a_{\mathrm{grid}}=L/n_{L}$ as indicated. Method
qA ($n_{L}=128$, solid line) required a month of computation on a local
workstation cluster with 64 nodes while all other examples were computed
within a couple of hours on the same cluster. Most importantly, all methods
are seen to yield similar results for small $q$ and the differences are minor
for wavevectors up to $q\approx 1$. The observed differences between the exact
relation Eq. (138) and the coarse-graining approximation are naturally
expected for $q$ of order unity and larger if a grid spacing
$a_{\mathrm{grid}}$ much smaller than the typical interaction range between
particles is used.

## References

  * McConnell (2021) A. J. McConnell, _Applications of Tensor Analysis_ (Hassell Street Press, 2021).
  * Schultz-Piszachich (1977) W. Schultz-Piszachich, _Mathematik 11: Tensoralgebra und -analysis_ (Verlag Harri Deutsch, Thun und Frankfurt/Main, 1977).
  * Lambourne (2010) R. Lambourne, _Relativity, Gravitation, and Cosmology_ (Cambridge University Press, Cambridge, 2010).
  * Tadmor et al. (2012) E. B. Tadmor, R. E. Miller, and R. S. Elliot, _Continuum Mechanics and Thermodynamics_ (Cambridge University Press, Cambridge, 2012).
  * Nedderman (1992) R. Nedderman, _Statics and Kinematics of Granular Materials_ (Cambridge University Press, Cambridge, 1992).
  * Argon and Kuo (1979) A. Argon and H. Kuo, Materials Science and Engineering 39, 101 (1979).
  * Falk and Langer (1998) M. L. Falk and J. S. Langer, Phys. Rev. E 57, 7192 (1998).
  * Picard et al. (2004) G. Picard, A. Ajdari, F. Lequeux, and L. Bocquet, Eur. Phys. J. E 15, 371 (2004).
  * Voigtmann (2014) T. Voigtmann, Current opinion in colloid & interface science 19, 549 (2014).
  * Fielding (2014) S. Fielding, Reports on Progress in Physics 77, 102601 (2014).
  * Rodney et al. (2011) D. Rodney, A. Tanguy, and D. Vandembroucq, Modelling Simul. Mater. Sci. Eng. 19, 083001 (2011).
  * Nicolas et al. (2018) A. Nicolas, E. Ferrero, K. Martens, and J.-L. Barrat, Rev. Mod. Phys. 90, 045006 (2018).
  * Landau and Lifshitz (1959) L. D. Landau and E. M. Lifshitz, _Theory of Elasticity_ (Pergamon Press, New York, 1959).
  * Hansen and McDonald (2006) J. P. Hansen and I. R. McDonald, _Theory of simple liquids_ (Academic Press, New York, 2006), 3nd edition.
  * Rubinstein and Colby (2003) M. Rubinstein and R. H. Colby, _Polymer Physics_ (Oxford University Press, Oxford, 2003).
  * Donth (2001) J. K. G. Donth, _The Glass Transition: Relaxation dynamics in liquids and disordered materials_ (Springer, Berlin-Heidelberg, 2001).
  * foo (a) Isotropy implies mathematically homogeneity Schultz-Piszachich (1977).
  * Maier et al. (2017) M. Maier, A. Zippelius, and M. Fuchs, Phys. Rev. Lett. 119, 265701 (2017).
  * Maier et al. (2018) M. Maier, A. Zippelius, and M. Fuchs, J. Chem. Phys. 149, 084502 (2018).
  * Vogel et al. (2019) F. Vogel, A. Zippelius, and M. Fuchs, Europhys. Lett. 125, 68003 (2019).
  * Klochko et al. (2018) L. Klochko, J. Baschnagel, J. P. Wittmer, and A. N. Semenov, Soft Matter 14, 6835 (2018).
  * Lemaître (2015) A. Lemaître, J. Chem. Phys. 143, 164515 (2015).
  * Lemaître (2018) A. Lemaître, J. Chem. Phys. 149, 104107 (2018).
  * Chowdhury et al. (2016) S. Chowdhury, S. Abraham, T. Hudson, and P. Harrowell, J. Chem. Phys. 144, 124508 (2016).
  * Klochko et al. (2022) L. Klochko, J. Baschnagel, J. Wittmer, H. Meyer, O. Benzerara, and A. N. Semenov, J. Chem. Phys. 156, 164505 (2022).
  * Steffen et al. (2022) D. Steffen, L. Schneider, M. Müller, and J. Rottler, J. Chem. Phys. 157, 064501 (2022).
  * Bocquet et al. (2009) L. Bocquet, A. Colin, and A. Ajdari, Phys. Rev. Lett. 103, 036001 (2009).
  * Nicolas et al. (2014) A. Nicolas, J. Rottler, and J.-L. Barrat, EPJE 37, 1 (2014).
  * Flenner and Szamel (2015) E. Flenner and G. Szamel, Phys. Rev. Lett. 114, 025501 (2015).
  * Desmond and Weeks (2015) K. W. Desmond and E. R. Weeks, Phys. Ref. Lett. 115, 098302 (2015).
  * Hassani et al. (2018) M. Hassani, E. Zirdehi, K. Kok, P. Schall, M. Fuchs, and F. Varnik, Europhysics Letters 124, 18003 (2018).
  * Liu et al. (2021) C. Liu, G. Biroli, D. R. Reichman, and G. Szamel, Phys. Rev. E 104, 054606 (2021).
  * Chacko et al. (2021) R. N. Chacko, F. P. Landes, G. Biroli, O. D. A. J. Liu, and D. R. Reichman, Phys. Rev. Lett. 127, 048002 (2021).
  * Eshelby (1957) J. Eshelby, Proc. R. Soc. Lond. A 241, 376 (1957).
  * Eshelby (1959) J. Eshelby, Proc. R. Soc. Lond. A 252, 561 (1959).
  * Lemaître (2014) A. Lemaître, Phys. Rev. Lett. 113, 245702 (2014).
  * Lemaître (2017) A. Lemaître, Phys. Rev. E 96, 052101 (2017).
  * Schouten (2015) J. A. Schouten, _Tensor Analysis for Physicists_ (Dover Publications, Oxford, 2015).
  * Abramowitz and Stegun (1964) M. Abramowitz and I. A. Stegun, _Handbook of Mathematical Functions_ (Dover, New York, 1964).
  * Press et al. (1992) W. Press, S. Teukolsky, W. Vetterling, and B. Flannery, _Numerical Recipes in FORTRAN: the art of scientific computing_ (Cambridge University Press, Cambridge, 1992).
  * foo (b) Discrete time is denoted by $t$, continuous time by $\tau$.
  * Forster (1995) D. Forster, _Hydrodynamic Fluctuations, Broken Symmertry, and Correlation Functions_ (Perseus Books, New York, 1995).
  * foo (c) See, e.g., the wikipedia entries on quadrupoles and general multipolar expansions as used, say, in electrostatics. For the planar harmonic basis functions $\cos(p\theta)$ or $\sin(p\theta)$ relevant for isotropic systems in $d=2$ dimensions a monopole corresponds to $p=0$, a dipole to $p=1$, a quadrupole to $p=2$ and an octupole to $p=4$. We remind that for isotropic systems in $d=3$ both the correlation function fields and the associated response fields must be expanded instead in terms of spherical harmonic functions. Multipole expansions are related to the underlying isotropic symmetry and even though source terms often may not share this symmetry, one can expand quite generally the response field in terms of irreducible representations (basis functions) of the symmetry group.
  * George et al. (2021a) G. George, L. Klochko, A. Semenov, J. Baschnagel, and J. P. Wittmer, EPJE 44, 13 (2021a).
  * Allen and Tildesley (2017) M. P. Allen and D. J. Tildesley, _Computer Simulation of Liquids, 2nd Edition_ (Oxford University Press, Oxford, 2017).
  * Ninarello et al. (2017) A. Ninarello, L. Berthier, and D. Coslovich, Phys. Rev. X 7, 021039 (2017).
  * Lutsko (1988) J. F. Lutsko, J. Appl. Phys 64, 1152 (1988).
  * Lutsko (1989) J. F. Lutsko, J. Appl. Phys 65, 2991 (1989).
  * Wittmer et al. (2013a) J. P. Wittmer, H. Xu, P. Polińska, F. Weysser, and J. Baschnagel, J. Chem. Phys. 138, 12A533 (2013a).
  * Wittmer et al. (2013b) J. P. Wittmer, H. Xu, P. Polińska, C. Gillig, J. Helfferich, F. Weysser, and J. Baschnagel, Eur. Phys. J. E 36, 131 (2013b).
  * Wittmer et al. (2015a) J. P. Wittmer, H. Xu, and J. Baschnagel, Phys. Rev. E 91, 022107 (2015a).
  * Wittmer et al. (2015b) J. P. Wittmer, H. Xu, O. Benzerara, and J. Baschnagel, Mol. Phys. 113, 2881 (2015b).
  * Maloney and Lemaître (2004) C. Maloney and A. Lemaître, Phys. Rev. Lett. 113, 245702 (2004).
  * foo (d) In agreement with the discussion in Sec. V.3, “$\tau\to\infty$” should be read “$\tau$ is much larger than $\tau_{1}$ but many orders of magnitude smaller than $\tau_{2}$”.
  * Doi and Edwards (1986) M. Doi and S. F. Edwards, _The Theory of Polymer Dynamics_ (Clarendon Press, Oxford, 1986).
  * Ferry (1980) J. D. Ferry, _Viscoelastic properties of polymers_ (John Wiley & Sons, New York, 1980).
  * Provencher (1982) S. Provencher, Computer Physics Communications 27, 229 (1982).
  * Balucani and Zoppi (1995) U. Balucani and M. Zoppi, _Dynamics of the Liquid State_ (Oxford University Press, Oxford, 1995).
  * Evans and Morriss (1990) D. Evans and G. Morriss, _Statistical mechanics of non-equilibrium liquids_ (Academic Press, London, 1990).
  * Rubinstein and Rubinstein (1998) I. Rubinstein and L. Rubinstein, _Partial differential equations in classical mathematical physics_ (Cambridge University Press, Cambridge, 1998).
  * foo (e) It is assumed that temperature variations can be neglected. This assumption is valid by construction for our MC simulations. The isothermicity condition remains also true if the head conductivity is very fast compared to all other relaxation processes.
  * Chaikin and Lubensky (1995) P. M. Chaikin and T. C. Lubensky, _Principles of condensed matter physics_ (Cambridge University Press, 1995).
  * Semenov et al. (2012) A. Semenov, J. Farago, and H. Meyer, J. Chem. Phys. 136, 244905 (2012).
  * Ruscher et al. (2017) C. Ruscher, A. Semenov, J. Baschnagel, and J. Farago, J. Chem. Phys. 146, 144502 (2017).
  * foo (f) The generic $1/r^{2}$-law (cf. Fig. 6) for two-dimensional, isotropic and achiral systems was recently also revealed in the elastic-like response of a liquid probed by local strain responses to a localized plastic event (particle rearrangements) in a polydisperse model system Chacko et al. (2021).
  * Vest et al. (2015) J.-P. Vest, G. Tarjus, and P. Viot, J. Chem. Phys. 143, 084505 (2015).
  * Turci et al. (2017) F. Turci, G. Tarjus, and C. P. Royall, Phys. Rev. Lett. 118, 215501 (2017).
  * Grosberg and Joanny (2015) A. Grosberg and J. Joanny, Phys. Rev. E 92, 032118 (2015).
  * Elismaili et al. (2020) M. Elismaili, S. Hamze, H. Xu, and D. Gonzalez-Rodriguez, Eur. Phys. J. E 43, 18 (2020).
  * Wittmer et al. (2022) J. Wittmer, A. Semenov, and J. Baschnagel, EPJE 45, 65 (2022).
  * Klochko et al. (2019) L. Klochko, J. Baschnagel, J. P. Wittmer, and A. N. Semenov, J. Chem. Phys. 151, 054504 (2019).
  * George et al. (2021b) G. George, L. Klochko, A. N. Semenov, J. Baschnagel, and J. P. Wittmer, EPJE 44, 54 (2021b).
  * Klochko et al. (2021) L. Klochko, J. Baschnagel, J. P. Wittmer, and A. N. Semenov, Soft Matter 17, 7867 (2021).
  * Irving and Kirkwood (1950) J. Irving and J. Kirkwood, J. Chem. Phys. 18, 817 (1950).
  * Kirkwood and Buff (1949) J. Kirkwood and F. Buff, J. Chem. Phys. 17, 338 (1949).
  * Tadmor and Miller (2011) E. B. Tadmor and R. E. Miller, _Modeling Materials_ (Cambridge University Press, Cambridge, 2011).
  * Noll (1955) W. Noll, J. Rational Mech. Anal. 4, 627 (1955).
  * Varnik et al. (2000) F. Varnik, J. Baschnagel, and K. Binder, J. Chem. Phys. 113, 444 (2000).
  * Goldhirsch and Goldenberg (2002) I. Goldhirsch and C. Goldenberg, Eur. Phys. J E 9, 245 (2002).
  * Vanegas et al. (2014) J. M. Vanegas, A. Torres-Sánchez, and M. Arroyo, J. Chem. Theory Comput. 10, 691 (2014).
  * Behbahani et al. (2021) A. Behbahani, L. Schneideri, A. Rissanou, A. Chazirakis, P. Bocova, P. Jana, W. Li, M. Doxastakis, O. Polińska, C. Burkhart, et al., Macromolecules 54, 2740 (2021).