# Comparing neural network training performance between Elixir and Python

Lucas C. Tavano111E-mail<EMAIL_ADDRESS>Lucas K. Amin222E-mail:
<EMAIL_ADDRESS>and Adolfo Gustavo Serra-Seca-Neto333E-mail:
<EMAIL_ADDRESS>
(October 2022)

###### Abstract

With a wide range of libraries focused on the machine learning market, such as
TensorFlow, NumPy, Pandas, Keras, and others, Python has made a name for
itself as one of the main programming languages. In February 2021, José Valim
and Sean Moriarity published the first version of the Numerical Elixir (Nx)
library, a library for tensor operations written in Elixir. Nx aims to allow
the language be a good choice for GPU-intensive operations. This work aims to
compare the results of Python and Elixir on training convolutional neural
networks (CNN) using MNIST and CIFAR-10 datasets, concluding that Python
achieved overall better results, and that Elixir is already a viable
alternative.

##  1 Introduction

Programming languages have optimized their data structures and libraries for
processing n-dimensional tensors seeking performance and simplicity.
Currently, the market for massively parallel data processing, such as machine
learning, is led by the Python language, followed by R, JavaScript, and C++
[1].

In this scenario, the Elixir language ecosystem has received new tools such as
the Nx library, a new compiler to perform operations in GPU, the ability to
work with float16 variables, among other improvements. These new features
enables the Elixir language to participate in the machine learning market.
This work aims to compare the programming languages Elixir and Python in the
context of processing large-volume tensors, comparing execution time, process
parallelization, and machine resources use like Central Processing Unit (CPU),
Random Access Memory (RAM), Graphics Processing Unit (GPU), and Video RAM
(VRAM) in similar algorithms.

##  2 Conceptual and Empirical Foundations

This chapter presents theoretical background on tensors, neural networks,
datasets, and the Elixir Nx and the Python NumPy libraries. It also discusses
the difference between functional and object-oriented paradigms in the context
of this project and how this can impact the comparison between Elixir and
Python results.

###  2.1 Datasets

A dataset is a large amount of data related to the same context [2]. In this
project, we will work with two datasets: MNIST and CIFAR-10, which will be
used to train CNN’s capable of classifying objects in images.

####  2.1.1 What is the MNIST dataset

MNIST is a labeled set of images of handwritten digits containing 70 thousand
entries; 60 thousand represent the training set, and 10 thousand represent the
test set. Each image in this collection has a resolution of 28x28 pixels;
thus, it is represented in code by a 28 by 28 matrix. Each value contained in
this matrix ranges from 0 to 255, 0 represents black, and 255 is white.

Figure 1: Example of MNIST set images plotted graphically with the Matplotlib
tool

####  2.1.2 What is the CIFAR-10 dataset

CIFAR-10, as MNIST, represents a large dataset of images; however, in the case
of CIFAR-10, the image set is related to objects and animals. The present
classes in the CIFAR-10 dataset are airplane, automobile, bird, cat, deer,
dog, frog, horse, ship, and truck. It consists of 66,000 images, split 90% for
training and 10% for testing. Each image comprises a 32x32x3 matrix,
representing a 32 by 32 RGB image.

Figure 2: Example of CIFAR-10 set images plotted graphically with the
Matplotlib tool

###  2.2 Neural Networks

It can be represented by a multi-layered directional graph, where each node is
referred as a neuron and is responsible for an mathematical operation.

Artificial neural networks are systems inspired by biological neural networks
constructed for data processing. It can be represented by a multi-layered
directional graph, where each node is referred as a neuron and is responsible
for an mathematical operation [5]. A neural network is designed to be trained
on a specific dataset, performing changes on its node operations with the
objective of searching the neuron weight set that has the lowest error
compared to the desired labeled result on the dataset [5].

####  2.2.1 Convolutional Neural Networks

A Convolutional Neural Network is a specific type of neural network where each
node is responsible for a convolution operation on the received input. An
image vector is used as input and processed through the network into a final
score represented on the output layer [5].

###  2.3 Trained Model Structures

The CNNs used to compare the performance metrics between Python and Elixir are
present in the Axon library examples designed for model creation and training.

####  2.3.1 MNIST Model Strucute

This CNN is a straightforward network with two dense layers and a dropout
between them444Available at: https://github.com/elixir-
nx/axon/blob/main/examples/vision/mnist.exs, as exemplified:

⬇

1Axon.input({nil, 784}, "input")

2 |> Axon.dense(128, activation: :relu)

3 |> Axon.dropout(rate: 0.5)

4 |> Axon.dense(10, activation: :softmax)

####  2.3.2 CIFAR-10 Model Strucute

It consists of a CNN on two convolutional layers followed by batch
normalization and max-pooling layers and two dense layers with a dropout
between them to avoid overfitting 555Available at: https://github.com/elixir-
nx/axon/blob/main/examples/vision/cifar10.exs, as exemplified:

⬇

1Axon.input({nil, 3, 32, 32}, "input")

2 |> Axon.conv(32, kernel_size: {3, 3}, activation: :relu)

3 |> Axon.batch_norm()

4 |> Axon.max_pool(kernel_size: {2, 2})

5 |> Axon.conv(64, kernel_size: {3, 3}, activation: :relu)

6 |> Axon.batch_norm()

7 |> Axon.max_pool(kernel_size: {2, 2})

8 |> Axon.flatten()

9 |> Axon.dense(64, activation: :relu)

10 |> Axon.dropout(rate: 0.5)

11 |> Axon.dense(10, activation: :softmax)

##  3 Project

This section presents the three sub-projects that we created:

  * •

An Elixir sub-project able to train CNNs able classify MNIST and CIFAR-10
datasets;

  * •

A Python sub-project able to train CNNs able classify MNIST and CIFAR-10
datasets;

  * •

An Elixir sub-project able to extract performance benchmarks during the CNNs
train.

Figure 3 is a visual representation of how the sub-projects interact and how
they compose the needed stakeholders for this project. Also, we will explain
the performance benchmark step in Section 4.

Figure 3: High-level diagram exemplified the 3 projects interact.

###  3.1 Elixir CNN trainer

This sub-project consists of a code created with Mix666More information at:
https://elixirmix.com/ using Elixir at version 1.12.3 and Erlang/OTP 24.0.6.
This sub-project source code is available on Github777Available at:
https://github.com/sallaumen/elixir_neural_network_labs. This sub-project
mainly uses three libraries from the Nx888Available at:
https://github.com/elixir-nx ecosystem: Scidata, Axon, Exla.

####  3.1.1 Scidata

According to its documentation999Available at: https://github.com/elixir-
nx/scidata, this library is responsible for providing datasets in a simple way
for Elixir projects. At this moment, it supports 11 different dataset types,
of which two of them, MNIST and CIFAR-10, are used for this sub-project
implementation.

####  3.1.2 Axon

According to its documentation101010Available at: https://github.com/elixir-
nx/axon, this library is an Application Programming Interface (API) for Nx. At
our project, Axon is used mainly for two functionalities:

  * •

Model Creation API: Axon provides a way to create a data structure for a CNN
model;

  * •

Training API: It provides a simple interface for training and evaluation.

####  3.1.3 EXLA

This library111111Available at: https://github.com/elixir-nx/nx/tree/main/exla
is the Google’s Accelerated Linear Algebra (XLA) compiler for Nx, acting both
as a backend for Nx tensors and as an Nx.Defn121212Nx.Defn a new way to define
methods in Elixir added to the language when using Nx compiler [3]. It is also
the responsible library that allows Nx.Defn code to be compiled and executed
at the GPU.

###  3.2 Python CNN trainer

We developed the Python model131313Available at:
https://github.com/sallaumen/python_neural_network_labs using Keras, a machine
learning library built for model construction, training, and evaluation [4].
We built this sub-project as follows:

  * •

Dataset preparation: MNIST and CIFAR10 datasets are loaded similarly, using
the dataset module from Keras, reshaped and normalized, so the RGB values are
between 0 and 1.

  * •

Model construction: Both models built for MNIST and CIFAR10 are built with the
Sequential method from the models module, using convolutional and dense
layers.

  * •

Model compilation: We built the model using sparse categorical cross-entropy
as a loss function, SGD as an optimizer, and accuracy as a metric.

  * •

Model training: We trained the model with three epochs and batch size set as
default to 32.

As the purpose of this work is to analyze performance and qualitative metrics,
we will not evaluate the accuracy of this algorithm.

###  3.3 UnixStats: Performance benchmark data exporter

UnixStats is another Elixir sub-project developed for this study. It is a
terminal-only software able to measure the use of CPU, RAM, GPU, and Graphic
RAM from the executing machine. To run it, the user needs to provide three
inputs:

  * •

The "measured process" represents the name of the process that UnixStats will
be measuring, which for Elixir was "beam.smp" and for Python, it was
"python3.8". If the process is not running currently, UnixStats admits the
Graphic RAM value as zero;

  * •

The "visualization type" can be ":pretty" or ":csv". The chosen format defines
how the measured data will be presented to the software user. The image 4
exemplifies how it looks at the terminal;

  * •

The "monitoring time" represents how long UnixStats should keep measuring the
machine.

Figure 4: Example of how UnixStats ":pretty" format is printed at the Linux
terminal.

##  4 Experiment

This section describes how the experiment was prepared, executed and how data
analysis was possible using the measured benchmarks.

###  4.1 Preparation

The environment used to train the neural network in Elixir and Python is the
same, a personal computer using Linux Ubuntu operational system, with no
graphical interface, configured with the needed drivers, as described at 4.2,
in a machine containing the following hardware:

  * •

Motherboard: ASUSTeK P8H61-M LX3 Plus R2.0

  * •

CPU: Intel Core i5 3570 3.40GHz

  * •

Graphics card: NVIDIA GeForce RTX 3060 Ti

  * •

RAM: 16GB DDR3 1333MHz

  * •

Power source potency: 850W

###  4.2 GPU training environment preparation

A fast way to train a CNN model is by using GPU and taking advantage of its
processing power. Therefore, the used OS had to be prepared to run Elixir and
Python CNN model training datasets. To be able to use the GPU, the following
driver, Compute Unified Device Architecture (CUDA), and NVIDIA CUDA® Deep
Neural Network library (cuDNN) were installed:

  * •

NVIDIA video driver version: 470.103.01

  * •

NVIDIA CUDA version: 11.4

  * •

NVIDIA cuDNN version: 11.2

Finally the preparation of the local environment to run the projects made in
Elixir and Python was completed.

###  4.3 Execution

  1. 1.

We prepared the machine described above;

  2. 2.

Initialized the OS where only a terminal screen gets started;

  3. 3.

Using the software tmux, we divided the terminal in 2 independent sides;

  4. 4.

We initialized the software UnixStats at the left terminal;

  5. 5.

Then we used the right terminal to execute the CNN trainings in Elixir and
Python with both analyzed datasets;

  6. 6.

We’ve trained CNN models twenty times, five for each combination of the
studied languages: Elixir and Python, and datasets: MNIST and CIFAR-10, while
UnixStats software was used to measure each execution performance benchmark;

  7. 7.

For each run measured, UnixStats generated one file in Comma-separated Values
(CSV) format;

  8. 8.

All CSVs then got imported Microsoft Excel software;

  9. 9.

Since every measured execution had a variable initial time due to the dataset
download time, we had to do data normalization to minimize the download
variation time effect at our measured data;

  10. 10.

For each group of five samples, we’ve created a sixth sample with the use
average of CPU, RAM, GPU, and Video RAM;

  11. 11.

At Last, with the samples from the experiments done and the average data, we
made graphs to enable the performance comparison aimed at this study, one for
each combination group

##  5 Results

In this section, we compared the CNN’s training datasets using the Elixir and
the Python languages. The metrics analyzed were the training time, and how
much used of CPU, RAM, GPU, and Graphic RAM during the experiments.

###  5.1 Evaluation

Below are the general performance graphs of training neural networks trained
for datasets MNIST and CIFAR-10 in Elixir and Python. The comparison and
analysis of these data will be further analyzed in this section.

General performance graphs of the two programming languages when training
their neural networks to solve the MNIST dataset model.

Figure 5: Graph of the system usage during model training MNIST in Elixir.
Figure 6: Graph of system usage during model training MNIST in Python. Figure
7: Graph of system usage during model training CIFAR-10 in Elixir. Figure 8:
Graph of system usage during model training CIFAR-10 in Python.

###  5.2 Training time

In this session, the overall training time will be analyzed in table 1,
composed of three columns. The "Language" column refers to the programming
language used, and the column "Dataset" contains the dataset used. Lastly,
"Training time" has the total time used to train the neural network model
capable of classifying images from the given dataset.

Table 1: Table with training times of datasets MNIST and CIFAR-10 in Elixir and Python. Language | Dataset | Training time (s)
---|---|---
Elixir | MNIST | 20,4
Python | MNIST | 15,2
Elixir | CIFAR-10 | 28,5
Python | CIFAR-10 | 23,0

Table 1 shows that the MNIST dataset had 34.21% greater time in Elixir than
the same dataset trained in Python. While for CIFAR-10, the training time in
Elixir was 23.91% greater than in Python.

###  5.3 CPU usage

In this session, we will present CPU usage while training the CNN, and for
that, graphs 9 and 10 will be analyzed. While training in Elixir, CPU usage
variation is 0.55% when trained in the MNIST dataset and 0.36% for CIFAR-10.
While training in Python, CPU usage variation is 1.59% when trained in the
MNIST dataset and 0.91% for CIFAR-10. It is possible to conclude, due to the
low variance of this resource, that CPU seems not to be too relevant when
training in both languages.

Figure 9: Graph comparing CPU usage while training using the MNIST dataset.
Figure 10: Graph comparing CPU usage while training using the CIFAR-10
dataset.

###  5.4 RAM usage

In this session, we will present RAM usage while training the CNN, and for
that, graphs 11 and 12 will be analyzed. While training in Elixir, the
variation of RAM usage is 7.2% on MNIST and 15.9% on CIFAR-10. While training
in Python, the variation of RAM usage is 12.52% on MNIST and 23.95% on
CIFAR-10.

It is possible to conclude that RAM is an essential resource during training
in both languages, which was expected once the sets of images of both datasets
are stored in memory during the execution. In addition, it is also possible to
notice that in Elixir, there is less RAM usage than in Python, using an
average of 42.4% less of this resource for training on MNIST and 33.6% less of
this resource for CIFAR-10. Since the data size of datasets in both languages
is the same, we can conclude that Elixir has better memory management in the
analyzed scenarios.

Figure 11: Graph comparing RAM usage while training using the MNIST dataset.
Figure 12: Graph comparing RAM usage while training using the CIFAR-10
dataset.

###  5.5 GPU usage

In this session, we will analyze the GPU usage for training, and graphs 13 and
14 will be examined. While training in Elixir, the variation of GPU usage is
54% on MNIST and 78% for CIFAR-10. Analyzing each step, GPU usage peaks at 2.2
seconds, reaching 54%, dropping to 0 for 1 second, then the usage goes up
again, keeping up about 10% from 4.1 seconds to 24.1 seconds. While training
in Python, the variation of GPU usage is 19% on MNIST and 25% for CIFAR-10.
Analyzing each step, GPU usage peaks at 1.4 seconds (VALIDATE), reaching 16%
GPU usage, dropping to 0 for 1 second, then the usage rises again, maintaining
10% from 3.4 seconds to 18.1 seconds.

With this, we can draw preliminary conclusions from this analysis. The first
point is that both languages follow a pattern of using approximately 10% of
GPU during model training. Furthermore, we can also conclude that in this
context that training the model in Python was faster than in Elixir by
approximately 25%, finishing 6 seconds faster.

Figure 13: Graph comparing GPU usage while training using the MNIST dataset.
Figure 14: Graph comparing GPU usage while training using the CIFAR-10
dataset.

###  5.6 Video RAM usage

In this section, we will analyze Video RAM use during training using graphs 15
and 16. While training in Elixir, the variation in VRAM is 6.70% for MNIST and
16.48% for CIFAR-10. While training in Python, the variation in Video RAM is
13.42% for MNIST and 16.78% for CIFAR-10. By this initial analysis, it is
possible to notice that for MNIST dataset, Elixir used less GPU memory, and
for CIFAR-10 both languages used almost the same.

Also, it is possible to notice that both CNN trained in Elixir never
deallocate the VRAM, this is caused by the compiler Accelerated Linear Algebra
(XLA)[6], which keeps this memory allocated until the process that it is
associated ends, and since Elixir runs at the Erlang virtual machine:
Bogdan/Björn’s Erlang Abstract Machine (BEAM), it only deallocates the VRAM
when this virtual machine gets shutdown, which at our experiments we never
done.

This behavior was also explained to us by the author of "Genetic Algorithms in
Elixir", Sean Moriarity, when we made a post at Elixir Forum commenting about
this behavior which until then we thought it was a bug141414Elixir Forum post
available at: https://elixirforum.com/t/possible-graphic-ram-deallocation-
issue-noticed-when-using-nx-with-exla/47629.

This issue does not happen with the Python trainer, since it runs as a script
instead of in a virtual machine, making XLA deallocate the graphic RAM almost
instantly after finishing the training, noticeable at the graphs 15 and 16 at
the "Python model training’s end".

Figure 15: Graph comparing GPU RAM usage while training using the MNIST
dataset. Figure 16: Graph comparing GPU usage while training using the
CIFAR-10 dataset.

##  6 Conclusion

In this article, we concluded that Elixir took more time in both datasets
analyzed compared to Python while demanding more Graphic RAM on peaks. We
observed that RAM usage is an exception to the other metrics as Elixir
consumed on average 42% and 33% less RAM than Python, showing that Elixir has
better memory management for these test cases.

As for usability, we observed that Python has much more documentation,
material, and available libraries for dataset preparation, model creation, and
training. This was expected as Python is already much more consolidated in the
machine learning market. However, the Nx library was a significant improvement
to Elixir, allowing the language to be used for neural network creations much
more efficiently than before Nx.

##  7 Acknowledgments

We are grateful to José Valim, Sean Moriarity, and every other developer
working on the Numerical Elixir ecosystem. It has been incredible for the
Elixir language, and it is incredible to see how Nx has been improving since
its announcement in February 2021.

## References

  * [1] Miltiadis Allamanis, Earl T Barr, Premkumar Devanbu, and Charles Sutton.  A survey of machine learning for big code and naturalness.  ACM Computing Surveys (CSUR), 51(4):1–37, 2018.
  * [2] Google Cloud.  Introduction to datasets, 2022.
  * [3] EXLA.  Exla hexdocs, 2022.
  * [4] Keras.  Keras, 2022.
  * [5] Keiron O’Shea and Ryan Nash.  An introduction to convolutional neural networks.  arXiv preprint arXiv:1511.08458, 2015.
  * [6] Amit Sabne.  Xla : Compiling machine learning for peak performance, 2020.


11institutetext:  Parks College, Department of Chemistry, Saint Louis
University - Saint Louis, MO 63103, USA
Department of Mathematics, University of Leicester - Leicester LE1 7RH, UK
Department of Mathematical Analysis, Ghent University - Galglaan 2, B-9000
Gent, Belgium
Laboratory for Chemical Technology, Ghent University - Krijgslaan 281 (S5),
B-9000 Gent, Belgium

Kinetic theory Nonequilibrium and irreversible thermodynamics

# Reciprocal Relations Between Kinetic Curves

G.S. Yablonsky Long Term Structural Methusalem Funding by the Flemish
Government - grant number BOF09/01M0040911 A.N. Gorban Support from the
University of Leicester and Isaac Newton Institute for Mathematical Sciences22
D. Constales Support from BOF/GOA 01GA0405 of Ghent University33 V.V. Galvita
44 G.B. Marin (a)4(a)411223344

###### Abstract

We study coupled irreversible processes. For linear or linearized kinetics
with microreversibility, $\dot{x}=Kx$, the kinetic operator $K$ is symmetric
in the entropic inner product. This form of Onsager’s reciprocal relations
implies that the shift in time, $\exp(Kt)$, is also a symmetric operator. This
generates the reciprocity relations between the kinetic curves. For example,
for the Master equation, if we start the process from the $i$th pure state and
measure the probability $p_{j}(t)$ of the $j$th state ($j\neq i$), and,
similarly, measure $p_{i}(t)$ for the process, which starts at the $j$th pure
state, then the ratio of these two probabilities $p_{j}(t)/p_{i}(t)$ is
constant in time and coincides with the ratio of the equilibrium
probabilities. We study similar and more general reciprocal relations between
the kinetic curves. The experimental evidence provided as an example is from
the reversible water gas shift reaction over iron oxide catalyst. The
experimental data are obtained using Temporal Analysis of Products (TAP)
pulse-response studies. These offer excellent confirmation within the
experimental error.

###### pacs:

05.20.Dd

###### pacs:

05.70.Ln

##  1 Introduction

###  1.1 A bit of history

In 1931, L. Onsager [1, 2] gave the backgrounds and generalizations to the
reciprocal relations introduced in 19th century by Lord Kelvin and H. v.
Helmholtz. In his seminal papers, L. Onsager mentioned also the close
connection between these relations and detailed balancing of elementary
processes: at equilibrium, each elementary transaction should be equilibrated
by its inverse transaction. This principle of detailed balance was known long
before for the Boltzmann equation [3]. A. Einstein used this principle for the
linear kinetics of emission and absorption of radiation [4]. In 1901, R.
Wegscheider published an analysis of detailed balance for chemical kinetics
[5].

The connections between the detailed balancing and Onsager’s reciprocal
relations were clarified in detail by N. G. v. Kampen [6]. They were also
extended for various types of coordinate transformations which may include
time derivatives and integration in time [7]. Recently, [8], the reciprocal
relations were derived for nonlinear coupled transport processes between
reservoirs coupled at mesoscopic contact points. Now, an elegant geometric
framework is elaborated for Onsager’s relations and their generalizations [9].

Onsager’s relations are widely used for extraction of kinetic information
about reciprocal processes from experiments and for the validation of such
information (see, for example, [10]): one can measure how process A affects
process B and extract the reciprocal information, how B affects A.

The reciprocal relations were tested experimentally for many systems. In 1960,
D.G. Miller wrote a remarkable review on experimental verification of the
Onsager reciprocal relations which is often referred to even now [11].
Analyzing many different cases of irreversible phenomena (thermoelectricity,
electrokinetics, isothermal diffusion, etc), Miller found that these
reciprocal relations are valid. However, regarding the chemical reactions,
Miller’s point was : “The experimental studies of this phenomenon … have been
inconclusive, and the question is still open from an experimental point.”

According to Onsager’s work [1], the fluxes in chemical kinetics are time
derivatives of the concentrations and potentials are expressed through the
chemical potentials. The fluxes (near equilibrium) are linear functions of
potentials and the reciprocal relations state that the coefficient matrix of
these functions is symmetric. It is impossible to measure these coefficients
directly. To find them one has to solve the inverse problem of chemical
kinetics. This problem is often ill-posed.

Such a difficulty, appearance of ill-posed problems in the verification of the
reciprocal relations, is typical because these relations connect the kinetic
coefficients. Sometimes it is possible to find them directly in separate
experiments but if it is impossible then the inverse problem arises with all
the typical difficulties.

In our work we, in particular, demonstrate how it is possible to verify the
reciprocal relations without the differentiation of the empiric kinetic curves
and solving the inverse problems, and present the experimental results which
demonstrate these relations for one reaction kinetic system. For this purpose,
we have to formulate the reciprocal relations directly between the measurable
quantities.

These reciprocal relations between kinetic curves use the symmetry of the
propagator in the special entropic inner product. A dual experiment is defined
for each ideal kinetic experiment. For this dual experiment, both the initial
data and the observables are different (they exchange their positions), but
the results of the measurement is essentially the same function of time.

###  1.2 The structure of the paper

We start from the classical Onsager relations and reformulate them as
conditions on the kinetic operator $K$ for linear or linearized kinetic
equations $\dot{x}=Kx$. This operator should be symmetric in the entropic
inner product, whereas the matrix $L$ that transforms forces into fluxes
should be symmetric in the standard inner product, i.e. $L_{ij}=L_{ji}$. The
form of reciprocal relations with special inner product is well known in
chemical and Boltzmann kinetics [13, 14]. They are usually proved directly
from the detailed balance conditions. Such relations are also universal just
as the classical relations are.

Real functions of symmetric operators are also symmetric. In particular, the
propagator $\exp(Kt)$ is symmetric. Therefore, we can formulate the reciprocal
relation between kinetic curves. These relations do not include fluxes and
time derivatives, hence, they are more robust. We formulate them as the
symmetry relations between the observables and initial data (the observables-
initial data symmetry).

A particular case of this symmetry for a network of monomolecular chemical
reactions or for the Master equation, which describe systems with detailed
balance, seems rather unexpected. Let us consider two situations for a linear
reaction network.

  1. 1.

The process starts at the state “everything is in $A_{q}$”, and we measure the
concentration of $A_{r}$. The result is $c^{a}_{r}(t)$ (“how much $A_{r}$ is
produced from the initial $A_{q}$”).

  2. 2.

The process starts at the state “everything is in $A_{r}$”, and we measure the
concentration of $A_{q}$. The result is $c^{b}_{q}(t)$ (“how much $A_{q}$ is
produced from the initial $A_{r}$”) (the dual experiment).

The results of the dual experiments are connected by the identity

$\frac{c^{a}_{r}(t)}{c^{\rm eq}_{r}}\equiv\frac{c^{b}_{q}(t)}{c^{\rm
eq}_{q}}\,,$

where $c$ are concentrations and $c^{\rm eq}$ are equilibrium concentrations.

The symmetry with respect to the observables-initial data exchange gives the
general rule for production of the reciprocal relations between kinetic
curves.

Many real processes in chemical engineering and biochemistry include
irreversible reactions, i.e. the reactions with a negligible (zero) rate of
the reverse reaction. For these processes, the micro-reversibility conditions
and the backgrounds of classical Onsager relations are not applicable
directly. Nevertheless, they may be considered as limits of systems with
micro-reversibility when some of the rate constants for inverse reactions tend
to zero. We introduce the correspondent weak form of detailed balance,
formulate the necessary and sufficient algebraic conditions for this form of
detailed balance and formulate the observables-initial data symmetry for these
systems.

The experimental evidence of the observables-initial data symmetry is
presented for the reversible water gas shift reaction over iron oxide
catalyst. The experimental data are obtained using Temporal Analysis of
Products (TAP) pulse-response studies. These offer excellent confirmation
within experimental error.

##  2 Two forms of the reciprocal relations: forces, fluxes and entropic inner
product

Let us consider linear kinetic equations or kinetic equations linearized near
an equilibrium $x^{\rm eq}$ (sometimes, it may be convenient to move the
origin to $x^{\rm eq}$):

$\dot{x}=Kx\,.$ (1)

In the original form of Onsager’s relations, the vector of fluxes $J$ and the
vector of thermodynamic forces $X$ are connected by a symmetric matrix,
$J=LX$, $L_{ij}=L_{ji}$. The vector $X$ is the gradient of the corresponding
thermodynamic potential: $X_{i}=\partial\Phi/\partial x_{i}$. For isolated
systems, $\Phi$ is the entropy. For other conditions, other thermodynamic
potentials are used. For example, for the constant volume $V$ and temperature
$T$ conditions, $\Phi$ is $-F/T$ and for the constant pressure $P$ and
temperature conditions, $\Phi$ is $-G/T$, where $F$ is the Helmholtz energy
(free energy) and $G$ is the Gibbs energy (free enthalpy). These free entropy
functions are also known as the Massieu–Planck potentials [12]. Usually, they
are concave.

For the finite-dimensional systems, like chemical kinetics or the Master
equation, the dynamics satisfy linear (linearized) kinetic equation
$\dot{x}=Kx$, where

$K_{ij}=\sum_{l}L_{il}\left.\frac{\partial^{2}\Phi}{\partial x_{l}\partial
x_{j}}\right|_{x^{\rm eq}}\mbox{ i.e. }K=L(D^{2}\Phi)_{x^{\rm eq}}.$

This matrix is not symmetric but the product $(D^{2}\Phi)_{x^{\rm eq}}K$
$=(D^{2}\Phi)_{x^{\rm eq}}L(D^{2}\Phi)_{x^{\rm eq}}$ is already symmetric,
hence, $K$ is symmetric (self-adjoint) in the entropic scalar product

$\langle a\,|\,Kb\rangle_{\Phi}\equiv\langle Ka\,|\,b\rangle_{\Phi}\,,$ (2)

where

$\langle
a\,|\,b\rangle_{\Phi}=-\sum_{ij}a_{i}\left.\frac{\partial^{2}\Phi}{\partial
x_{l}\partial x_{j}}\right|_{x^{\rm eq}}b_{j}\,.$ (3)

Further on, we use the angular brackets for the entropic inner product (3) and
its generalizations and omit the subscript $\Phi$.

For the spatially distributed systems with transport processes, the variables
$x_{i}$ are functions of the space coordinates $\xi$, the equations of
divergence form appear, $\partial_{t}x_{i}=-\nabla_{\xi}\cdot J_{i}$,
thermodynamic forces include also gradients in space variables,
$X_{i}=\nabla_{\xi}\partial\Phi/\partial x_{i}$ and the operator $K$ has the
form

$K_{ij}=\sum_{l}L_{il}\left.\frac{\partial^{2}\Phi}{\partial x_{l}\partial
x_{j}}\right|_{x^{\rm eq}}\Delta_{\xi}\,\mbox{ i.e. }K=L(D^{2}\Phi)_{x^{\rm
eq}}\Delta_{\xi}\,,$

where $\Delta_{\xi}$ is the Laplace operator. This operator $K$ is self-
adjoint in the inner product which is just the integral in space of (3). The
generalizations to inhomogeneous equilibria, non-isotropic and non-euclidian
spaces are also routine but lead to more cumbersome formulas.

Symmetric operators have many important properties. Their spectrum is real,
for a function of a real variable $f$ with real values it is possible to
define $f(K)$ through the spectral decomposition of $K$, and this $f(K)$ is
also symmetric in the same inner product. This property is the cornerstone for
further consideration.

##  3 Symmetry between observables and initial data

The exponential of a symmetric operator is also symmetric, hence, Onsager’s
relations (2) immediately imply

$\langle a\,|\,\exp(Kt)\,b\rangle\equiv\langle b\,|\,\exp(Kt)\,a\rangle\,.$
(4)

The expression $x(t)=\exp(Kt)\,b$ gives a solution to the kinetic equations
(1) with initial conditions $x(0)=b$. The expression $\langle
a\,|\,x(t)\rangle$ is the result of a measurement: formally, for each vector
$a$ we can introduce a “device” (an observer), which measures the scalar
product of vector $a$ on a current state $x$.

The left hand side of (4) represents the result of such an experiment: we
prepare an initial state $x(0)=b$, start the process from this state and
measure $\langle a\,|\,x(t)\rangle$. In the right hand side, the initial
condition $b$ and the observer $a$ exchange their positions and roles: we
start from the initial condition $x(0)=a$ and measure $\langle
b\,|\,x(t)\rangle$. The result is the same function of time $t$.

This exchange of the observer and the initial state transforms an ideal
experiment into another ideal experiment (we call them dual experiments). The
left and the right hand sides of (4) represent different experimental
situations but with the same results of the measurements.

This observation produces many consequences. As a first class of examples, we
present the time–reversible Markov chains [15], or the same class of kinetic
equations, the monomolecular reactions with detailed balance (see any detailed
textbook in chemical kinetics, for example, [13]).

Here a terminological comment is necessary. The term “reversible” has three
different senses in thermodynamics and kinetics.

  * •

First of all, processes with entropy growth are irreversible. In this sense,
all processes under consideration are irreversible.

  * •

Secondly, processes with microreversibility, which satisfy the detailed
balance and Onsager relations, are time–reversible (or, for short, one often
calls them “reversible”). We always call them time–reversible to avoid
confusion.

  * •

In the third sense, reversibility is the existence of inverse processes: if
transition $A\to B$ exists then transition $B\to A$ exists too. This condition
is significantly weaker than microreversibility.

“Time–reversibility” of irreversible processes sounds paradoxical and requires
comments. The most direct interpretation of “time–reversing” is to go back in
time: we take a solution to dynamic equations $x(t)$ and check whether $x(-t)$
is also a solution. For the microscopic dynamics (the Newton or Schrödinger
equations) we expect that $x(-t)$ is also a solution to the dynamic equations.
Nonequilibrium statistical physics combines this idea with the description of
macroscopic or mesoscopic kinetics by an ensemble of elementary processes:
collisions, reactions or jumps. The microscopic “reversing of time” turns at
this level into the “reversing of arrows”: reaction
$\sum_{i}\alpha_{i}A\to\sum_{j}\beta_{j}B_{j}$ transforms into
$\sum_{j}\beta_{j}B_{j}\to\sum_{i}\alpha_{i}A$ and conversely. The equilibrium
ensemble should be invariant with respect to this transformation. This leads
us immediately to the concept of detailed balance: each process is
equilibrated by its reverse process. “Time–reversible kinetic process” stands
for “irreversible process with the time–reversible underlying microdynamics”.

We consider a general network of linear reactions. This network is represented
as a directed graph (digraph) [13]: vertices correspond to components $A_{i}$
($i=1,2,\ldots,n$), edges correspond to reactions $A_{i}\to A_{j}$ ($i\neq
j$). For each vertex, $A_{i}$, a positive real variable $c_{i}$
(concentration) is defined. For each reaction, $A_{i}\to A_{j}$ a nonnegative
continuous bounded function, the reaction rate constant $k_{ji}>0$ is given.
The kinetic equations have the standard Master equation form

$\frac{{\mathrm{d}}c_{i}}{{\mathrm{d}}t}=\sum_{j,\,j\neq
i}(k_{ij}c_{j}-k_{ji}c_{i})\,.$ (5)

The principle of detailed balance (“time–reversibility”) means that there
exists such a positive vector $c^{\rm eq}_{i}>0$ that for all $i,j$ ($j\neq
i$)

$k_{ij}c^{\rm eq}_{j}=k_{ji}c^{\rm eq}_{i}\,.$ (6)

The following conditions are necessary and sufficient for existence of such an
equilibrium $c^{\rm eq}_{i}>0$:

  * •

Reversibility (in the third sense): if $k_{ji}>0$ then $k_{ij}>0$;

  * •

For any cycle $A_{i_{1}}\to A_{i_{2}}\to\ldots\to A_{i_{q}}\to A_{i_{1}}$ the
product of constants of reactions is equal to the product of constants of
reverse reactions,

$\prod_{j=1}^{q}k_{i_{j+1}i_{j}}=\prod_{j=1}^{q}k_{i_{j}i_{j+1}}\,,$ (7)

where $i_{q+1}=i_{1}$. This is the Wegscheider identity [5].

It is sufficient to consider in conditions (7) a finite number of basic cycles
[13].

The free entropy function for the Master equation (5) is the (minus) relative
entropy

$Y=-\sum_{i}c_{i}\ln\left(\frac{c_{i}}{c^{\rm eq}_{i}}\right)\,.$ (8)

In this form, the function $-RTY$ was used already by L. Onsager [1] under the
name “free energy”. The entropic inner product for the free entropy (8) is

$\langle a\,|\,b\rangle=\sum_{i}\frac{a_{i}b_{i}}{c^{\rm eq}_{i}}\,.$ (9)

Let $c^{a}(t)$ be a solution of kinetic equations (5) with initial conditions
$c^{a}(0)=a$. Then the reciprocity relations (4) for linear systems with
detailed balance take the form

$\sum_{i}\frac{b_{i}c^{a}_{i}(t)}{c^{\rm
eq}_{i}}=\sum_{i}\frac{a_{i}c^{b}_{i}(t)}{c^{\rm eq}_{i}}\,.$ (10)

Let us use for $a$ and $b$ the vectors of the standard basis in
$\mathbb{R}^{n}$: $a_{i}=\delta_{iq}$, $b_{i}=\delta_{ir}$, $q\neq r$. This
choice results in the useful particular form of (10). We compare two
experimental situations, $c^{a}_{i}(0)=\delta_{iq}$ (the process starts at the
state “everything is in $A_{q}$”) and $c^{b}_{i}(0)=\delta_{ir}$ (the process
starts at the state “everything is in $A_{r}$”); for the first situation we
measure $c^{a}_{r}(t)$ (“how much $A_{r}$ is produced from the initial
$A_{q}$”), for the second one we measure $c^{b}_{q}(t)$ (“how much $A_{q}$ is
produced from the initial $A_{r}$”). The reciprocal relations (10) give

$\frac{c^{a}_{r}(t)}{c^{\rm eq}_{r}}=\frac{c^{b}_{q}(t)}{c^{\rm eq}_{q}}\,.$
(11)

More examples of such relations for chemical kinetics are presented in [16].
It is much more straightforward to check experimentally these relations
between kinetic curves than the initial Onsager relations between kinetic
coefficients. We give an example of such an experiment below. For processes
distributed in space, instead of concentrations of $A$ and $B$ some of their
Fourier or wavelet coefficients appear.

##  4 Weak form of detailed balance

For many real systems some of the elementary reactions are practically
irreversible. Hence the first condition of detailed balance, the reversibility
(if $k_{ji}>0$ then $k_{ij}>0$) may be violated. Nevertheless, these systems
may be considered as limits of systems with detailed balance when some of the
constants tend to zero. For such limits, the condition (7) persists, and for
any cycle the product of constants of direct reactions is equal to the product
of constants of reverse reactions.

This is a weak form of detailed balance without the obligatory existence of a
positive equilibrium. In this section, we consider the systems, which satisfy
this weak condition, the weakly time–reversible systems.

For a linear system, the following condition is necessary and sufficient for
its weak time–reversibility: In any cycle $A_{i_{1}}\to A_{i_{2}}\to\ldots\to
A_{i_{q}}\to A_{i_{1}}$ with strictly positive constants $k_{i_{j+1}i_{j}}>0$
(here $i_{q+1}=i_{1}$) all the reactions are reversible ($k_{i_{j}i_{j+1}}>0$)
and the identity (7) holds.

The components $A_{q}$ and $A_{r}$ ($q\neq r$) are strongly connected if there
exist oriented paths both from $A_{q}$ to $A_{r}$ and from $A_{r}$ to $A_{q}$
(each oriented edge corresponds to a reaction with nonzero reaction rate
constant). It is convenient to consider an empty path from $A_{i}$ to itself
as an oriented path.

For strongly connected components of a weakly time–reversible system, all
reactions in any directed path between them are reversible. This is a
structural condition of the weak time–reversibility.

Under this structural condition, the classes of strongly connected components
form a partition of the set of components: these classes either coincide or do
not intersect and each component belongs to one of them. Each cycle belongs to
one class.

Let $A_{q}$ and $A_{r}$ be strongly connected. Let us select an arbitrary
oriented path $p$ between $A_{q}$ and $A_{r}$: $A_{q}\leftrightarrow
A_{i_{1}}\leftrightarrow A_{i_{2}}\leftrightarrow\ldots\leftrightarrow
A_{i_{l}}\leftrightarrow A_{r}$. For the product of direct reaction rate
constants in this path we use $K_{p}^{+}$ and for the product of reverse
reaction rate constants we use $K_{p}^{-}$. The ratio
$K_{rq}=K_{p}^{+}/K_{p}^{-}$ does not depend on the path $p$ and characterizes
the pair $A_{r},A_{q}$, because of the Wegscheider identity (7). This is the
quantitative criterion of the weak time–reversibility.

The constant $K_{rq}$ is an analogue to the equilibrium constant. Indeed, for
the systems with positive equilibrium and detailed balance, $K_{rq}c^{\rm
eq}_{q}=c^{\rm eq}_{r}$ and $K_{rq}=c^{\rm eq}_{r}/c^{\rm eq}_{q}$.

For weakly time–reversible system, the reciprocal relations between kinetic
curves can be formulated for any strongly connected pair $A_{q}$ and $A_{r}$.
Exactly for the same pair of kinetic curves, as in (11), we obtain

$\frac{c^{a}_{r}(t)}{c^{b}_{q}(t)}=K_{rq}\,.$ (12)

This formula describes two experiments: (i) we start the system at $t=0$ from
the pure $A_{q}$ and measure $c_{r}(t)$, then (ii) we start at $t=0$ from the
pure $A_{r}$ and measure $c_{q}(t)$. The ratio of these two kinetic curves,
$c_{r}(t)/c_{q}(t)$ does not depend on $t$ and is equal to the generalized
equilibrium constant $K_{rq}$.

The weak form of the Wegscheider identity for general (nonlinear) kinetic
systems is also possible. Let us consider the reaction system:

$\alpha_{r1}A_{1}+\ldots+\alpha_{rn}A_{n}\to\beta_{r1}A_{1}+\ldots+\beta_{rn}A_{n}\,,$
(13)

which satisfies the mass action law:
$\dot{c}=\sum_{r}\gamma_{r}k_{r}\prod_{i}c_{i}^{\alpha_{i}}\,,$ where
$k_{r}>0$, $\gamma_{ri}=\beta_{ri}-\alpha_{ri}$ is the stoichiometric vector
of the $r$th reaction, and the reverse reactions with positive constants are
included in the list (13) separately.

Let us consider linear relations between vectors $\\{\gamma_{r}\\}$:

$\sum_{r}\lambda_{r}\gamma_{r}=0\mbox{ and }\lambda_{r}\neq 0\mbox{ for some
}r\,.$ (14)

If all the reactions are reversible then the principle of detailed balance
gives us the identity [13]:

$\prod_{r}(k_{r}^{+})^{\lambda_{r}}=\prod_{r}(k_{r}^{-})^{\lambda_{r}}$ (15)

for any linear relation (14). For reversible reactions, we can take
$\lambda_{r}\geq 0$ in (15) for all $r$: if we substitute the reactions with
$\lambda_{r}<0$ by their reverse reactions then $\gamma_{r}$ and $\lambda_{r}$
change signs. It is sufficient to consider only the cone $\Lambda_{+}$ of non-
negative relations (14) ($\lambda_{r}\geq 0$) and take in (15) the direction
vectors of its extreme rays. Let $k_{r}^{-}=0$ for some $r$. The weak form of
the identity (15) is:

For any extreme ray of $\Lambda_{+}$ with a direction vector $\lambda_{r}\geq
0$ the reactions which correspond to the positive coefficients $\lambda_{r}>0$
are reversible ($k_{r}^{-}>0$) and their constants satisfy the identity (15).

##  5 Nonlinear Examples

It seems impossible to find a general relation between kinetic curves for
general nonlinear kinetics far from equilibrium. Nevertheless, simple examples
encourage us to look for a nontrivial theory for some classes of nonlinear
systems. In this Section, we give two examples of nonlinear elementary
reactions which demonstrate the equilibrium relations between nonequilibrium
kinetic curves [16].

###  5.1 $2A\leftrightarrow B$

The linear conservation law is $c_{A}+2c_{B}=const$. Let us take two initial
states with the same value $c_{A}+2c_{B}=1$: ($a$) $c_{A}(0)=1,\,c_{B}(0)=0$
and ($b$) $c_{A}(0)=0,\,c_{B}(0)=1/2$. We will mark the corresponding
solutions by the upper indexes $a,b$. The mass action law gives:

$\dot{c}_{A}=-2k^{+}c_{A}^{2}+k^{-}(1-c_{A})\,,\;c_{B}=(1-c_{A})/2\,.$ (16)

The analytic solution easily gives

$\frac{c_{B}^{a}(t)}{c_{A}^{a}(t)c_{A}^{b}(t)}=\frac{k^{+}}{k^{-}}=K^{\mathrm{eq}}=\frac{c_{B}^{\mathrm{eq}}}{(c_{A}^{\mathrm{eq}})^{2}}\,,$
(17)

the denominator involves the $A$ concentrations of both trajectories, $c^{a}$
(started from $c_{A}(0)=1,\,c_{B}(0)=0$) and $c^{b}$ (started from
$c_{A}(0)=0,\,c_{B}(0)=1/2$). A ratio is equal to the equilibrium constant at
every time $t>0$. This identity between the non-stationary kinetic curves
reproduces the equilibrium ratio.

###  5.2 $2A\leftrightarrow 2B$

The linear conservation law is $c_{A}+c_{B}=const$. Let us take two initial
states with the same value $c_{A}+c_{B}=1$: ($a$) $c_{A}(0)=1,\,c_{B}(0)=0$
and ($b$) $c_{A}(0)=0,\,c_{B}(0)=1$. The kinetic equation is

$\dot{c}_{A}=-2k^{+}c_{A}^{2}+k^{-}(1-c_{A})^{2}\,,\;c_{B}={1-c_{A}}\,.$ (18)

It can be solved analytically. For this solution,

$\frac{c_{B}^{a}(t)c_{B}^{b}(t)}{c_{A}^{a}(t)c_{A}^{b}(t)}=\frac{k^{+}}{k^{-}}=K^{\mathrm{eq}}=\frac{(c_{B}^{\mathrm{eq}})^{2}}{(c_{A}^{\mathrm{eq}})^{2}}\,,$
(19)

both the numerator and denominator include trajectories for both initial
states, $a$ and $b$. This identity between the kinetic curves also reproduces
the equilibrium ratio.

##  6 Experimental evidences

In this work, we investigate the validity of the reciprocal relations using
the TAP (Temporal Analysis of Products) technique proposed by Gleaves in 1988
[17]. It has been successfully applied in many areas of chemical kinetics and
engineering for non-steady-state kinetic characterization [18]. The studied
reaction is a part of the reversible water gas shift reaction over iron oxide
catalyst. The overall reaction is
$\mbox{H}_{2}\mbox{O}+\mbox{CO}\leftrightarrow\mbox{H}_{2}+\mbox{CO}_{2}$.

###  6.1 Experimental set-up

The TAP reactor system used in this work is made of quartz and is of the size
33 mm bed-length and 4.75 mm inner diameter. The products and the unreacted
reactants coming out of the reactor are monitored by a UTI 100C quadrupole
mass spectrometer (QMS). The number of molecules admitted during pulse
experiments amounts to $10^{15}$ molecules/pulse.

To ensure uniformity of the catalyst along the bed, we use a thin–zone TAP
reactor (TZTR), the width of the catalyst zone being 2mm. Experiments were
performed over 40 mg of Fe2O3 catalyst. The catalyst was packed in between two
inert zones of quartz particles of the same size ($250<d_{p}<500\mu m$). The
temperature of the reactor was measured by a thermocouple positioned in the
center of the catalyst bed. Several single pulse experiments were performed by
pulsing CO or CO2 at the temperature of 780K. In all the experiments, the
reaction mixture was prepared with Ar as one of the components, so that the
inlet amount of the components can be determined from the Ar response.

###  6.2 Application to the measurements

Figure 1: Fourier domain result values for the “B from A/A from B” ratio (20),
vs. frequency $f$ in Hz (so that $\omega=2\pi f$, $s=i\omega$); real and
imaginary part. The error bars were obtained from 10,000 resampled
measurements.

In a thin–zone TAP-reactor, the diffusion occurring in the inert zones
flanking the thin reactive zone must be accounted for. The Knudsen regime in
these zones guarantees a linear behaviour, so that the resulting outlet fluxes
can be expressed in terms of convolutions. Switching to the Laplace domain
greatly facilitates the analysis, and we can prove in general that the fixed
proportion property is equivalent to the following equality in terms of the
exit fluxes $F_{B_{A}}$ of gas $B$ given a unit inlet pulse of gas $A$ and
$F_{A_{B}}$, of $A$ given a unit inlet pulse of $B$, see [19]:

$K_{\mathop{\rm
eq}}={\frac{(\cosh\sqrt{s\tau_{1,A}})(\sqrt{\tau_{3,B}}\sinh\sqrt{s\tau_{3,B}})}{(\cosh\sqrt{s\tau_{1,B}})(\sqrt{\tau_{3,A}}\sinh\sqrt{s\tau_{3,A}})}}{\frac{{\cal
L}F_{B_{A}}(s)}{{\cal L}F_{A_{B}}(s)}}$ (20)

identically in the Laplace variable $s$, where
$\tau_{i,G}=\epsilon_{i}L_{i}^{2}/D_{G}$, $\epsilon_{i}$ denoting the packing
density of the $i$-th zone, $L_{i}$ its length, and $D_{G}$ the diffusivity of
gas $G$. To apply this in practice, we set $s=i\omega$ and switch to the
Fourier domain.

Performing these corrections, with $A$ denoting CO and $B$ CO2, the results of
Fig. 1 in the Fourier domain are obtained. The real and imaginary parts of the
right-hand side in (20) are graphed, with error bars corresponding to three
times the standard deviation estimated from resampling 10,000 times the exit
flux measurements using their principal error components. Ideally, all
imaginary values should be zero; we see that zero does lie within all the
confidence intervals. We also see that the smallest error in the real parts
occurs for the second frequency, 2.2 Hz. This confidence interval lies snugly
within the others, offering confirmation that (within experimental error) the
same value for all frequencies is obtained.

##  7 Conclusion

The shift in time operator is symmetric in the entropic inner product. Its
symmetry allows us to formulate the symmetry relations between the observables
and initial data. These relations could be validated without differentiation
of empiric curves and are, in that sense, more robust and closer to the direct
measurements. For the Markov processes and chemical kinetics, the symmetry
relations between the observables and initial data have an elegant form of the
symmetry between “$A$ produced from $B$” and “$B$ produced from $A$”: their
ratio is equal to the equilibrium constant and does not change in time (11),
(12). For processes distributed in space, instead of concentrations of $A$ and
$B$ some of their Fourier or wavelet coefficients appear.

The symmetry relations between the observables and initial data have a rich
variety of realizations, which makes the direct experimental verification
possible. On the other hand, this symmetry provides the possibility to extract
information about the experimental data through the dual experiments. These
relations are applicable to all systems with microreversibility.

## References

  * [1] Onsager L. Phys. Rev.37 1931 405–426.
  * [2] Onsager L. Phys. Rev.38 1931 2265–2279.
  * [3] Boltzmann L. Lectures on gas theoryU. of California Press, Berkeley, CA 1964.
  * [4] Einstein A. Verhandlungen der Deutschen Physikalischen Gesellschaft1813/14(1916) 318–323.
  * [5] Wegscheider R. Monatshefte für Chemie / Chemical Monthly32 8(1911), 849–906. DOI: 10.1007/BF01517735
  * [6] Van Kampen N. G. Physica 671973 1–22.
  * [7] Stöckel H. Fortschritte der Physik/Progress of Physics311983 165–184.
  * [8] Astumian R. D. Phys. Rev. Lett.1012008046802.
  * [9] Grmela M. Physica A 3092002 304–328.
  * [10] Ozera M. Provaznik I. J. Theor. Biology2332005 237–243.
  * [11] Miller D. G. Chem. Rev. 601960 15–37.
  * [12] Callen H. B. Thermodynamics and an Introduction to Thermostatistics (2nd ed.). John Wiley and Sons, New York 1985.
  * [13] Yablonskii G. S., Bykov V. I., Gorban A. N. Elokhin V. I. Kinetic Models of Catalytic Reactions, Series: Comprehensive Chemical Kinetics 32 Elsevier, Amsterdam, The Netherlands 1991.
  * [14] Gorban A. N. Karlin I. V. Invariant manifolds for physical and chemical kinetics, vol. 660 of Lect. Notes Phys Springer, Berlin-Heidelberg-New York 2005.
  * [15] Hasegawa H. Progr. Theor. Phys.551976 90–105.
  * [16] Yablonsky G. S., Constales D. Marin G. B. Equilibrium relationships for non-equilibrium chemical dependencies, Chem. Eng. Sci.662011 111–114.
  * [17] Gleaves J. T., Ebner J. R. Kuechler T. C. Catal. Rev.- Sci. Eng.301988 49–116.
  * [18] Yablonsky G. S., Olea M. Marin G. B. J. Catal.2162003 120–134.
  * [19] Constales D., Yablonsky G. S., Marin G. B. Gleaves J. T. Chem. Eng. Sci.592004 3725–3736.