# Interactive Semantic Map Representation for Skill-based Visual Object
Navigation

Tatiana Zemskova1, Aleksei Staroverov1, 2, 3, Kirill Muravyev3,
Dmitry Yudin1, 2, Aleksandr Panov1, 2
1Moscow Institute of Physics and Technology, 2Artificial Intelligence Research
Institute,
3Federal Research Center for Computer Science and Control of Russian Academy
of Sciences
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>

###### Abstract

Visual object navigation using learning methods is one of the key tasks in
mobile robotics. This paper introduces a new representation of a scene
semantic map formed during the embodied agent interaction with the indoor
environment. It is based on a neural network method that adjusts the weights
of the segmentation model with backpropagation of the predicted fusion loss
values during inference on a regular (backward) or delayed (forward) image
sequence. We have implemented this representation into a full-fledged
navigation approach called SkillTron, which can select robot skills from end-
to-end policies based on reinforcement learning and classic map-based planning
methods. The proposed approach makes it possible to form both intermediate
goals for robot exploration and the final goal for object navigation. We
conducted intensive experiments with the proposed approach in the Habitat
environment, which showed a significant superiority in navigation quality
metrics compared to state-of-the-art approaches. The developed code and used
custom datasets are publicly available at github.com/AIRI-Institute/skill-
fusion.

##  1 Introduction

Accurate visual navigation to target objects in unfamiliar environments is
essential for on-board mobile robot systems. In this case, the choice of
embodied agent actions can be carried out by various methods: classical
modular map-based motion planning algorithms Muravyev et al. (2021), end-to-
end neural network models based on images of on-board cameras and/or their
segmentation (Gordon et al., 2019). Classical algorithms use separate learning
modules to build a map, explore the environment, or select the final goal.
End-to-end models are typically trained using reinforcement learning methods,
which is valuable in conditions of incomplete information about the
environment.

All visual navigation methods use a semantic representation of the surrounding
scene, usually formed using various learned image segmentation models. The
representation of a one-shot observed map in the form of a multi-channel
segmentation mask can be used directly or to build an accumulated map
representation in the form of 2D (so-called birds-eye-views) (Staroverov et
al., 2023), 2.5D (Ewen et al., 2022) or 3D Yang et al. (2017) semantic maps,
where each map point contains a class label of the found object. The
possibility of increasing the efficiency of visual navigation methods based on
semantic maps is studied in special photorealistic simulators. For indoor
navigation, some of the most popular environments are Habitat (Yadav et al.,
2023c), AI2-THOR (Kolve et al., 2017), OmniGibson (Li et al., 2022). For the
study of outdoor navigation, there are environments Carla (Dosovitskiy et al.,
2017), AirSim (Shah et al., 2017), etc. There are also universal simulators
for indoor and outdoor environments, such as Isaac Sim (NVIDIA, 2021). This
work considers only simulation environments for the indoor navigation to given
objects.

Modern works (Gadre et al., 2022; Staroverov et al., 2023), as well as
solutions of such indoor navigation competitions as the Habitat
Challenge111https://aihabitat.org, show that the semantic representation of
the map plays a key role in increasing the success of achieving the goal and
reducing the length of the resulting trajectories. The usual approach is in
which semantic segmentation results are mapped using heuristic projection
algorithms and noisy information about the localization of the agent (Chaplot
et al., 2020). Some approaches (Ramakrishnan et al., 2022) use different rule-
based methods to filter and refine segmentation results, but usually, such
approaches are not correctly linked to the algorithms for planning map
trajectories. The results on the success of achieving the target objects (no
more than 60% in the Habitat Challenge 2022 and 2023
competitions222https://aihabitat.org/challenge/2022/) show that new integrated
approaches to the presentation of the semantic map are needed, which would
take into account the specifics of the moving embodied agent.

We propose a full-fledged navigation approach called SkillTron using
adjustment of the segmentation model with backpropagation of the predicted
fusion loss values during inference on a regular (backward) or delayed
(forward) image sequence. SkillTron can select robot skills from end-to-end
policies based on reinforcement learning and classic map-based planning
methods.

Figure 1: Proposed two-level representation of a semantic map. The one-shot
map in the form of a segmentation mask is built during the interaction of the
embodied agent with the environment. We consider regular (backward) or delayed
(forward) image sequences as a result of such interaction. The accumulated map
is a multi-channel semantic birds-eye-view map formed using an original
filtering algorithm

Our main contributions are:

  * •

We proposed a two-level representation of a semantic map (see Figure 1), in
which a one-shot map in the form of a segmentation mask is built during the
interaction of the embodied agent with the environment, and the accumulated
map is formed using an original filtering algorithm for the semantic birds-
eye-view map.

  * •

We have developed a navigation approach, called SkillTron, using a proposed
interactive semantic map representation with robot skill selection from end-
to-end policies based on reinforcement learning and classic map-based planning
methods. It allows us to form both intermediate goals for robot exploration
and the final goal for object navigation.

  * •

To study the proposed approaches, we collected several custom datasets in the
Habitat Indoor Environment. This allowed us to train and test the interactive
semantic segmentation model and approach for building the accumulated map.
During experiments, we showed a significant superiority of the proposed
SkillTron visual object navigation method in terms of quality metrics compared
to state-of-the-art approaches.

##  2 Related Works

###  2.1 Navigation

Like most navigational tasks, the ObjectNav task can be addressed using SLAM
and deterministic planners. As a result, the agent constructs an occupancy map
and a collision-free path to the goal. Despite the unknown coordinates of the
goal object, methods like Frontier-based exploration (FBE) (Yamauchi, 1997)
are frequently employed. A frontier is the boundary between the explored free
and unexplored spaces. Frontier-based exploration essentially samples points
on this frontier as goals to explore space. If the agent sees a goal type of
object during this exploration, it navigates to it directly. A significant
breakthrough of the learning approaches in navigation tasks was the DDPPO
method (Wijmans et al., 2019), which used Proximal Policy Optimization
(Schulman et al., 2017) at its core. Without mapping or planning modules,
DDPPO at the PointNav task could perform 2.5 billion steps in the environment
and solve the task at human-level performance. However, DDPPO demonstrated
that at the ObjectNav task, the pure end-to-end RL algorithms that use vanilla
visual and recurrent modules perform poorly due to overfitting and sample
inefficiency. The authors of the Auxiliary task RL method (Ye et al., 2021)
partially solved this by adding auxiliary learning tasks and an exploration
reward during the training phase. Another promising approach to solving the
ObjectNav task is to mix analytic and learned components and operate on
explicit spatial maps of the environment. Such a combination of classical and
learned methods was employed in the SemExp (Chaplot et al., 2020), CoW (Gadre
et al., 2022), SkillFusion (Staroverov et al., 2023) methods. Usually, authors
use a deterministic map module and divide a policy into a global one that, by
planning on a map, outputs a short-term subgoal and a local policy that
pursues that subgoal. Inspired by those works, we also built a two-level
policy, but our low level consists of several independent skills and high-
level switches between them depending on what the agent needs to do at a given
time.

###  2.2 Interactive Computer Vision.

An embodied agent navigating through the environment can interactively update
the scene’s semantic representation based on new sensor data. This allows the
agent to improve the semantic understanding of the scene. The recent emergence
of environments for embodied agents, e.g., Habitat (Yadav et al., 2023c),
AI2-THOR (Kolve et al., 2017), OmniGibson (Li et al., 2022), enabling the
simulation of agent navigation and interaction with different objects, led to
the development of interactive segmentation methods.

An agent can predict its future actions to improve perception quality for the
next observation. In this case, the interactive segmentation would be a
special case of the Next Best View selection task aiming to identify the next
most informative sensor position for computer vision tasks. The recent works
propose different methods to assess the informativeness of a view. The choice
of the next best view can rely on the confidence score of a frozen object
detector (Ding et al., 2023), segmentation quality (Yang et al., 2019;
Chaudhary et al., 2023) or statistical criteria derived from the image itself
(Hoseini et al., 2022). The interactive computer vision methods use both
learned policies (Ding et al., 2023; Yang et al., 2019; Chaudhary et al.,
2023) and predefined policies, e.g., the output of voting system (Hoseini et
al., 2022), for the next best view selection.

Other interactive segmentation methods aim to improve the quality of the
semantic representation of an environment rather than focusing solely on image
recognition. (Li and Bansal, 2023) improve the agent’s performance on the
Vision-Language Navigation Task by adding an auxiliary task to predict the
view semantics for the next step. (Asgharivaskasi and Atanasov, 2023) propose
to learn an exploration policy to decrease the uncertainty of different
semantic classes by considering the motion cost.

We can highlight methods that improve the understanding of the semantics of a
scene based on active exploration. The embodied agents use active exploration
to facilitate the adaptation to complex and unfamiliar environments. Agents
can query human expert help (Singh et al., 2022) or create pseudo-labels in
testing environments using multiple points of view (Fang et al., 2020).
Zurbrügg et al. (2022), Jing and Kong (2023) introduce learnable policies that
consider the uncertainty of semantic maps for collecting data to fine-tune
semantic segmentation models of embodied agents.

Another approach for adapting models to a test environment is fine-tuning the
model during inference. Recent works Interactron (Kotar and Mottaghi, 2022)
and SegmATRon (Zemskova et al., 2023) propose the use of an adaptive loss
function, predicted from a frame sequence to refine object detection (Kotar
and Mottaghi, 2022) and semantic segmentation (Zemskova et al., 2023). The
adaptive loss function is used during both training and inference. Our work
modifies the SegmATRon approach to create an interactive semantic map
representation. We investigate different methods for selecting consecutive RGB
observations to improve the quality of the semantic map representation.

##  3 Object Goal Navigation Task

In the context of the indoor Object Goal task as described in the literature
(Batra et al., 2020), the aim is to navigate towards an instance of a
specified object category $C\in\\{c_{1},c_{2},...,c_{n}\\}$ (for instance, a
chair) within an unfamiliar environment. The agent receives an observation
${S}=(S_{RGBD},S_{GPS+Compass},C)$ at each step. The action space is discrete
and encompasses four types of actions: callstop (to terminate the episode),
forward by $0.25m$, turnleft, and turnright by an angle $\alpha$ (in our
experiments this angle can be equal to $30^{\circ}$ or $15^{\circ}$). The
choice of such discrete actions is typical for indoor simulators such as
Habitat (Yadav et al., 2023c).

After the agent executes the callstop action, the agent’s performance is
assessed using three primary metrics: 1) Success, where an episode is deemed
successful if the agent executes the callstop command within 1.0m of any
object of the goal type; 2) Success weighted by (the inverse normalized) Path
Length (SPL), where success is weighted by the efficiency of the agent’s path
to the nearest object from the starting point; 3) SoftSPL, where binary
success is substituted by progress toward the goal.

##  4 Method

###  4.1 SkillTron Navigation Approach

Figure 2: A scheme of the proposed SkillTRon visual navigation approach, which
generates robot actions during interaction with the indoor environment. We use
a fusion of robot skills, which are formed using a classic modular pipeline
and end-to-end RL-based policies. Both of these pipelines utilize different
layers of semantic map representation: the first uses an accumulated multi-
layer map and the second works with a one-shot environment map.

Our proposed visual navigation approach (see Figure 2) employs two skills for
solving ObjectGoal Navigation task: Exploration and Goal Reaching. Each skill
is implemented with both map-based and learning-based approach. Our pipeline
switches from exploration to goal reaching skill as soon as a target object is
appeared in the agent’s view. First, the learning-based goal reacher is used
to guide the agent to the target object. After the target object is mapped,
the agent reaches it using classic path planner. To avoid segmentation
outliers and eliminate noises, we implement semantic map filtering with
erosion, dilation, accumulation and fading.

Map-based skills. In our research, we utilize the Potential Function for
ObjectGoal Navigation (PONI (Ramakrishnan et al., 2022)) method as a map-based
exploration skill. The PONI method navigates the environment using a multi-
layered map, which comprises the explored space layer, the obstacle layer, and
the layers for 16 semantic classes. This map is constructed during the
exploration process using depth observations and semantic segmentation masks.
For exploration purposes, PONI establishes intermediate goals using a
learning-based potential function on the multi-layered map. This potential
function is composed of two components: the area potential and the object
potential. The area potential indicates the extent of the area that can be
explored from a point on the map, while the object potential represents the
proximity to the goal object. The potential function neural network is based
on an UNet-like map encoder (Ronneberger et al., 2015), and includes two UNet
decoders for the area and object components. In the context of autonomous
navigation, when a goal object is projected onto the semantic map, a path to
the nearest cell of the object is constructed utilizing the A-Star path
planning algorithm (Hart et al., 1968; Staroverov et al., 2023). The robot
then navigates along this planned path using a straightforward path follower.
The navigation episode concludes when the robot’s distance to the target
entity falls below a specific threshold. In our experimental setup, this
threshold was set to 0.84.

Learning-based skills. In relation to the learning-based skills, we employ a
learning-based approach within the framework of a Markov Decision Process
(MDP), formally represented as $<S,A,T,R,\gamma>$ . Here, $S$ represents the
set of states, $A$ signifies the set of possible actions,
$T(s_{t+1}|s_{t},a_{t})$ is the transition function, $R$ is the reward
function, and $\gamma$ is the discount factor. In our proposed model, the
states $s_{t}$ are not fully observable. The agent only has access to partial
information about the state at each time step, represented as the observation
$o_{t}$. We make an assumption that the agent is equipped with an approximator
$f$ that estimates the state $s_{t}$ based on the history of observations:
$s_{t}\approx f(o_{t},o_{t-1},\dots)$. In practical terms, this approximator
is implemented as a component of the agent’s neural network responsible for
decision-making. For the training of these skills, we have employed a
Decentralized Distributed Proximal Policy Optimization (DDPPO (Wijmans et al.,
2019)) and a SkillFusion (Staroverov et al., 2023) approach. Its
distinctiveness lies in the fact that the exploration skill is trained
concurrently with the point navigation and flee tasks to develop a dynamic
robust RNN encoder, and it utilizes a frozen CLIP model (Radford et al., 2021)
as a robust visual image encoder. A unique feature of the Goal Reacher skill
is its use of a ‘not sure’ action, which the agent can execute when it first
identifies the goal, but upon approaching it, realizes that it was a false
positive and needs to revert to exploration.

Skill Fusion Decider. At each step, both map-based and RL approaches are
updated from the observations. And the navigational actions are taken from one
of the approaches, depending from the skill fusion decider. At the exploration
stage, our skill fusion decider switches between map-based and RL-based
approaches comparing the RL critic score with a pre-defined threshold $\tau$.
If the critic score exceedes the threshold, the agent is guided by the RL
policy. Otherwise, the agent is guided by PONI. At the goal reaching stage,
the skill fusion decider switches from RL-based to map-based approach as soon
as the target object is mapped. If the target object disappeared from the map
due to filtering, our pipeline switches from goal reaching back to
exploration. Also, our pipeline switches from goal reaching to exploration if
the goal object disappeared from the agent’s view and is not mapped.

Figure 3: Image Sequence Semantic Segmentation Network: SegmATRon (B) and
SegmATRon (F).

###  4.2 Interactive segmentation

Transformer model. As a baseline method for interactive image segmentation we
consider the SegmATRon model (Zemskova et al., 2023) fusing the information
from several frames through mechanism of a hybrid multicomponent fusion loss
function. The detailed illustration of the SegmATRon is presented in Figure 3.
The SegmATron model consists of two modules: a semantic segmentation model
(modification of the OneFormer (Jain et al., 2023)) and a fusion module (DETR
Transformer Decoder (Carion et al., 2020)). For each frame of the input image
sequence the semantic segmentation model outputs image features, mask features
and predicted logits. The sequence of outputs is passed to the Fusion Module
(see Figure 3). The Fusion module predicts the learned fusion loss which is
used to update parameters of OneFormer. Then, the updated OneFormer makes
another prediction for the first frame in the sequence. The predicted masks
and logits are considered as final semantic segmentation of the first frame in
the sequence.

Adaptive Fusion Loss Function. The adaptive fusion loss function
$\mathcal{L}_{fusion}(\phi,\theta,\mathbf{F})$ is parameterized by Fusion
Module parameters $\phi$ and depends on parameters $\theta$ of the OneFormer
model and a sequence of frames $\mathbf{F}$. The parameters $\theta$ are
updated by backpropagation through adaptive gradients. During the training
process the Fusion module parameters $\phi$ and the OneFormer parameters
$\theta$ are optimized jointly. The goal is to minimize multicomponent
segmentation loss $\mathcal{L}_{segm}(\theta,\mathbf{F})$ over all ground-
truth sequences $\mathbf{R}_{all}$.

$\min_{\theta,\phi}{\sum_{\mathbf{F}\in\mathbf{R}_{all}}\mathcal{L}_{segm}(\theta-\alpha\nabla_{\theta}\mathcal{L}_{fusion}(\phi,\theta,\mathbf{F}),\mathbf{F})}.$
(1)

The segmentation loss function is the original OneFormer loss function (Jain
et al., 2023) without the contrastive loss term. Thus,

$\mathcal{L}_{segm}=\lambda_{cls}\mathcal{L}_{cls}+\lambda_{bce}\mathcal{L}_{bce}+\lambda_{dice}\mathcal{L}_{dice},$
(2)

where, $\mathcal{L}_{cls}$ – cross-entropy loss for class prediction, binary
cross-entropy ($\mathcal{L}_{bce}$) and dice loss ($\mathcal{L}_{dice}$) are
controlling mask predictions.

Navigation Image Sequences. The SegmATRon model is trained on offline data of
image sequences that were collected in a simulation environment. However,
during navigation, sequences of images appear online, and the frames content
depends on the actions chosen by the agent. Therefore, the SegmATRon can use a
buffer storing frame sequences which is updating interactively. Two parameters
controls the buffer properties: the number of images and the sequence order.
In our experiments we consider buffers of different lengths: $2$ images, $3$
images and $5$ images. These sizes correspond to $1$, $2$ and $4$ additional
frames (steps) used to refine semantic segmentation. In addition, we conduct
experiments with different image sequence orders in the frame buffer. We call
the image sequence order "backward" when a sequence of frames
$\\{I_{t},I_{t-1},...,I_{t-n}\\}$ is used for prediction at step $t$, where
$n$ is the number of additional frames that the SegmATRon (B) uses. In the
case of "forward" image sequence order, at time $t+n$ the SegmATRon (F) makes
a prediction for frame $I_{t}$, using "future" frames relatively to frame
$I_{t}$, i.e. $\\{I_{t},I_{t+1},...,I_{t+n}\\}$. Thus, the semantic map is
updated with a delay, but the SegmATRon (F) model can use frames in which the
goal objects are better viewed. When a goal is observed, the agent navigates
towards it, so the "forward" images could have more information about the goal
than "backward" images.

###  4.3 Semantic map accumulation and filtering

Learning-based semantic segmentation predictions sometimes have noises,
especially in far objects. Projecting these noises onto the map causes
semantic mapping outliers and leads to reaching a false goal and unsuccessful
finish. To prevent this, we implement semantic map filtering consistting of
erosion, dilation, map accumulation and fading. A scheme of semantic map
filtering is shown in Figure 4.

At each step $t$, a local semantic map $L_{t}$ is created using pose and depth
observation with the SegmATRon-predicted semantic mask. First, an erosion is
applied to the local map to filter out semantic segmentation outliers. Next,
to return the initial size to the target objects, dilation is applied as a
convolution. After that, the local map is fused with the global map $M_{t-1}$
to obtain the updated global map $M_{t}$. The fusion is implemented as
accumulation and fading. The global map values in the cells containing a goal
object in the local map are increased by 1 (accumulation), and the global map
values in the cells not containing a goal object are multiplied by a decay
coefficient $\alpha<1$ (fading). The global map values in the cells not
covered by a local map are not changed. A cell is considered a target object
cell if its value in the global map exceedes some pre-defined threshold $T$. A
formal description of the semantic map filtering is shown below:

$L_{t}=dilation_{k}(erosion_{k}(L_{t})),$ (3)
$M_{t}=(M_{t-1}+L_{t})\cdot(L_{t}+\alpha(1-L_{t})C_{t}+(1-L_{t})(1-C_{t})),$
(4)

where $C_{t}$ is the coverage mask of the local map $L_{t}$.

Figure 4: Scheme of the proposed semantic map filtering.

##  5 Experiments

###  5.1 Experimental Setup

Navigation. We validate our navigation pipeline in the Habitat environment. We
use validation scenes from the Habitat Matterport3D semantics (HM3DSem) v0.2
dataset (Yadav et al., 2022) and select 108 episodes with the following
distribution of goal categories: potted plant ($18$), chair ($15$), chair
($25$), tv ($17$), bed ($17$), toilet ($16$). The validation scenes were never
seen during training phase. We use the environment configuration from Habitat
Challenge 2023 (Yadav et al., 2023b) with slight changes. The use of adaptive
gradients during inference requires more computational sources for SegmATRon
method. Therefore, we increase the maximum amount of time per episode up to
$1500$ seconds and fix the maximum number of steps to $500$. We conduct
experiments on a server with $1$ Nvidia Tesla V100 GPU. We repeat each
validation navigation experiment $5$ times and report the mean value of
standard navigation metrics: Success rate, SPL and SoftSPL as they defined in
Habitat Challenge 2023 (Yadav et al., 2023b).

Semantic maps representation. We assess the quality of semantic map
representation for different methods using the same observation dataset. We
collect this dataset recording observations during navigation with SkillTron
approach with SegmATRon (B) (1 Step) semantic method. At each step, we save
the agent’s position, the tilt of its head, the depth map, GT semantics, RGB
observation and SegmATRon (B) (1 Step) predictions of semantic mask. Then, we
make predictions on the saved image sequence using different versions of the
SegmATRon approach and various sets of hyper-parameters for semantic map
construction. This approach allows us to distinguish between the navigation
performance and the quality of semantic map representation. We evaluate the
quality of semantic map representation using metrics of Closeness-sigmoid, IoU
described in the Section 5.2.

Interactive segmentation model training. We train SegmATRon on offline data
collected in HM3DSem v0.2 dataset (Yadav et al., 2022). We collect a training
dataset in $1160$ random points of train scenes and a validation dataset in
$144$ random points of validation scenes. We make sure to include some random
viewpoints of goal categories in our datasets. The image sequences for the
SegmATRon models training are obtained by considering all possible combination
of $4$ actions made from starting random points. We consider the following set
of actions: turn left, turn right, look up, look down, and move backward. By
moving backward the agent can observe a scene from more distant point of view.
The tilt angle for look up and look down action is fixed to 30°. In our
experiments we consider turn angles of 15°and 30°. For both values of turn
angle we collect datasets starting from the same root points. Since during the
training and validation process the image sequences are chosen randomly from
available combinations, we use the same sequence of weights for SegmATRon (B)
and SegmATRon (F) approaches.

In our experiments we use a custom mapping of 1624 original categories HM3DSem
v0.2 dataset (Yadav et al., 2022) to 150 categories of ADE20k (Zhou et al.,
2019). Thus, we can efficiently fine-tune our segmentation models starting
from weights trained on the rich semantics of ADE20k (Zhou et al., 2019)
without pseudo-labeling. We consider this mapping as ground-thruth during the
training and validation process of SegmATRon. Additionally, we use this
mapping to compute ground-truth semantic maps to assess the quality of
semantic map representation.

Baselines. As a baseline method for our approach to semantic map
representation we consider the SkillFusion (Staroverov et al., 2023) method -
the winner of Habitat Challenge 2023 (Yadav et al., 2023b). We distinguish the
role of interactive semantic map representation by considering the Single
Frame baseline for our approach: the OneFormer (Swin-L backbone) model (Jain
et al., 2023) fine-tuned on the datasets collected for SegmATRon training.

###  5.2 Semantic maps

We estimate the quality of semantic maps using two metrics: Closeness-Sigmoid
and Intersection over Union (IoU). The Closeness-Sigmoid metric is calcluated
as an average distance from the predicted semantic map cells to the closest
cell of the ground truth semantic map, followed by a sigmoid function. The
resulting metric has value range between 0 and 1. If there are no predicted
map cells, the metric is considered 1. Also, we estimate False Positive Rate
(FPR) and False Negative Rate (FNR) metrics. The FPR value is the part of
episodes where the predicted map contains a spurious goal object, and the
ground truth map contains no goal objects. The FNR value is the part of
episodes where the ground-truth semantic map contains goal objects and the
predicted semantic map is empty.

First, we choose the optimal filtering hyperparameters: decay coefficient
$\alpha$ and threshold $T$. For this purpose, we test four pairs $(\alpha,T)$
with SegmATRon (B) (1 Step). The results of the tests are shown in Table 1.
According to all the metric values, we choose $\alpha=0.9$ and $T=2$ for our
SegmATRon experiments. Next, we test different SegmATRon approaches with the
chosen filtering hyperparameters. The results of the tests are shown in the
Table 2. According to the both Closeness-Sigmoid and IoU metrics, the best
approach is the SegmATRon (B) with 4 steps - this approach reaches Closeness-
Sigmoid $0.175$ and IoU $0.446$. The results with two-step SegmATRon (F) are
relatively close - Closeness-Sigmoid $0.184$ and IoU $0.415$.

Table 1: Semantic map quality with respect to filtering parameters Decay coef. $\alpha$ |  Threshold $T$ | Closeness-sigmoid | IoU | FPR | FNR
---|---|---|---|---|---
0.8 | 1 | 0.366 | 0.365 | 0.23 | 0.02
0.9 | 1 | 0.392 | 0.358 | 0.24 | 0.01
0.8 | 2 | 0.240 | 0.371 | 0.12 | 0.03
0.9 | 2 | 0.272 | 0.395 | 0.13 | 0.02
Table 2: Semantic map quality with different SegmATRon approaches Semantic method | Number of steps | Closeness-sigmoid | IoU | FPR | FNR
---|---|---|---|---|---
SegmATRon (B) | 1 | 0.272 | 0.395 | 0.13 | 0.02
SegmATRon (B) | 2 | 0.192 | 0.434 | 0.10 | 0.01
SegmATRon (B) | 4 | 0.175 | 0.446 | 0.09 | 0.03
SegmATRon (F) | 1 | 0.217 | 0.418 | 0.11 | 0.02
SegmATRon (F) | 2 | 0.184 | 0.415 | 0.08 | 0.02
SegmATRon (F) | 4 | 0.213 | 0.371 | 0.09 | 0.05
Table 3: Performance of SkillTron as compared to the baselines on the HM3DSem v0.2 dataset. Method | Exploration | Goalreacher | Semantic method | Success | SPL | SoftSPL
---|---|---|---|---|---|---
Host_74441_Team333We report metrics from public leaderboard of Habitat Challenge 2023 Test Standard Phase (Yadav et al., 2023a), (Yadav et al., 2023b) due to absence of public code implementation. | - | - | - | 0.12 | 0.05 | 0.27
ICanFly333We report metrics from public leaderboard of Habitat Challenge 2023 Test Standard Phase (Yadav et al., 2023a), (Yadav et al., 2023b) due to absence of public code implementation. | - | - | - | 0.43 | 0.26 | 0.37
SkillFusion | PONI+RL | Classic+RL | OneFormer | 0.55 | 0.26 | 0.34
SkillTron | PONI+RL | Classic+RL | OneFormer (finetuned) | 0.57 | 0.28 | 0.35
SkillTron | PONI+RL | Classic+RL | SegmATRon (B) (2 Steps) | 0.59 | 0.28 | 0.36
Table 4: Ablation study. Number of steps (Additional Frames) and Turn Angle Values. Method | Semantic method | Number of steps | Turn Angle | Success | SPL | SoftSPL
---|---|---|---|---|---|---
SkillTron | OneFormer | - | 30° | 0.57 | 0.28 | 0.35
SkillTron | SegmATRon (B) | 1 | 30° | 0.58 | 0.29 | 0.35
SkillTron | SegmATRon (B) | 2 | 30° | 0.59 | 0.28 | 0.36
SkillTron | SegmATRon (B) | 4 | 30° | 0.57 | 0.27 | 0.35
SkillTron | SegmATRon (B) | 1 | 15° | 0.51 | 0.25 | 0.33
SkillTron | SegmATRon (B) | 2 | 15° | 0.49 | 0.24 | 0.31
SkillTron | SegmATRon (B) | 4 | 15° | 0.50 | 0.25 | 0.32

###  5.3 Navigation with different semantic maps

Comparison with different baselines. The SkillTron approach significantly
outperforms various state-of-the-art methods listed on the public leaderboard
of Habitat Challenge 2023 Test Standard Phase (Yadav et al., 2023a), (Yadav et
al., 2023b) (see Table 3). There is no public code available for the ICanFly
and Host_74441_Team approaches, therefore we report the navigation metrics
based on its performance on the Habitat Challenge 2023 Test Standard dataset
(Yadav et al., 2023b). As one can see from Table 3, the SkillTron surpasses
the state-of-the-art approach SkillFusion (Staroverov et al., 2023) by a
considerable margin of $4\%$ of success rate, $2\%$ of SPL and $2\%$ of
SoftSPL metrics. The interactive semantic map representation plays an
important role in the performance of SkillTron method. Our best interactive
segmentation network uses two additional frames and backward image sequence.
The SkillTron method with interactive semantic map representation shows the
increase of $2\%$ of the Success and $1\%$ of SoftSPL metrics compared to the
baseline SkillTron using OneFormer as the semantic segmentation network.

Ablation on number of steps (Additional Frames). Long image sequences carry
more information about the environment. We vary the number of additional
frames used to predict a segmentation mask. We consider the backward image
sequence order for these experiments and turn angle of $30$°. The Table 4
shows the navigation metrics for SegmATRon (B) with different number of steps
(additional frames). The SegmATRon (B) (2 Steps) demonstrates the increase of
success rate and SoftSPL metrics comparing to the SegmATRon (B) (1 Steps).
However, the SegmATRon (B) (4 Steps) shows decrease of navigation metrics
comparing to the SegmATRon (B) (2 Steps). These results are in agreement with
the dependence of semantic maps quality metrics on the number of steps (see
Section 5.2) for $1$ and $2$ additional frames. The $4$ additional frames may
not be optimal for navigation, since the processing of longer image sequences
slows the navigation pipeline.

Ablation on the turn angle values. The continuity of view during navigation is
determined by agent’s turn angle. We conduct an ablation study to investigate
if a smaller turn angle would increase the performance of the Fusion module of
the interactive segmentation network. We consider 15 °- a half of turn angle
in the configuration of discrete action space of Habitat Challenge 2023 (Yadav
et al., 2023b). We retrain learning-based skills in the new action space and
the interactive segmentation network. The 15°turn angle is less effective for
navigation than 30 °(see Table 4) for all considered number of steps. It can
be partially since the smaller turn angle requires more steps during the
exploration phase of object goal navigation.

##  6 Conclusion

In this paper, we proposed a new visual navigation approach called SkillTron,
which uses a two-level interactive semantic map representation, as well as
fusing exploration and goal reacher skills of the robot. To build a one-shot
map level, we studied in detail the neural network method, which corrects the
weights of the segmentation model based on the predicted values of fusion loss
during inference on a regular (backward) or delayed (forward) image sequence.
We showed that the backward interaction mode provided a more accurate
construction of a 2D accumulated semantic map, which was then used for
navigation. We demonstrated that the proposed combination of an RL-based
navigation pipeline and a classic modular approach using learnable modules
outperformed existing state-of-the-art approaches in indoor environments from
the Habitat simulator.

As limitations, it should be noted that the selected semantic segmentation
network is resource-demanding, as well as the fact that proposed visual
navigation approach has only been tested in simulation environments. Further
directions for the development of the proposed approaches could be the study
of transferring the method of visual navigation to a real robot, the use of
other more compact basic models of semantic segmentation and image sequence
fusion to form a one-shot representation of a semantic map. Considering the
interactive segmentation as a separate robot skill with learned action policy
is also of interest for future work.

## References

  * Asgharivaskasi and Atanasov [2023] Arash Asgharivaskasi and Nikolay Atanasov.  Semantic octree mapping and shannon mutual information computation for robot exploration.  _IEEE Transactions on Robotics_ , 2023.
  * Batra et al. [2020] Dhruv Batra, Aaron Gokaslan, Aniruddha Kembhavi, Oleksandr Maksymets, Roozbeh Mottaghi, Manolis Savva, Alexander Toshev, and Erik Wijmans.  ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objects.  In _arXiv:2006.13171_ , 2020.
  * Carion et al. [2020] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko.  End-to-end object detection with transformers.  In _European conference on computer vision_ , pages 213–229. Springer, 2020.
  * Chaplot et al. [2020] Devendra Singh Chaplot, Dhiraj Gandhi, Abhinav Gupta, and Ruslan Salakhutdinov.  Object goal navigation using goal-oriented semantic exploration, 2020\.  URL https://arxiv.org/abs/2007.00643.
  * Chaudhary et al. [2023] Gaurav Chaudhary, Laxmidhar Behera, and Tushar Sandhan.  Active perception system for enhanced visual signal recovery using deep reinforcement learning.  In _ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 1–5. IEEE, 2023.
  * Ding et al. [2023] Wenhao Ding, Nathalie Majcherczyk, Mohit Deshpande, Xuewei Qi, Ding Zhao, Rajasimman Madhivanan, and Arnie Sen.  Learning to view: Decision transformers for active object detection.  _arXiv preprint arXiv:2301.09544_ , 2023.
  * Dosovitskiy et al. [2017] Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun.  CARLA: An open urban driving simulator.  In _Proceedings of the 1st Annual Conference on Robot Learning_ , pages 1–16, 2017.
  * Ewen et al. [2022] Parker Ewen, Adam Li, Yuxin Chen, Steven Hong, and Ram Vasudevan.  These maps are made for walking: Real-time terrain property estimation for mobile robots.  _IEEE Robotics and Automation Letters_ , 7(3):7083–7090, 2022.  doi: 10.1109/LRA.2022.3180439.
  * Fang et al. [2020] Zhaoyuan Fang, Ayush Jain, Gabriel Sarch, Adam W. Harley, and Katerina Fragkiadaki.  Move to see better: Towards self-supervised amodal object detection, 2020\.
  * Gadre et al. [2022] Samir Yitzhak Gadre, Mitchell Wortsman, Gabriel Ilharco, Ludwig Schmidt, and Shuran Song.  Cows on pasture: Baselines and benchmarks for language-driven zero-shot object navigation, 2022.
  * Gordon et al. [2019] Daniel Gordon, Abhishek Kadian, Devi Parikh, Judy Hoffman, and Dhruv Batra.  Splitnet: Sim2sim and task2task transfer for embodied visual navigation, 2019.  URL https://arxiv.org/abs/1905.07512.
  * Hart et al. [1968] Peter E Hart, Nils J Nilsson, and Bertram Raphael.  A formal basis for the heuristic determination of minimum cost paths.  _IEEE transactions on Systems Science and Cybernetics_ , 4(2):100–107, 1968.
  * Hoseini et al. [2022] Pourya Hoseini, Shuvo Kumar Paul, Mircea Nicolescu, and Monica Nicolescu.  A one-shot next best view system for active object recognition.  _Applied Intelligence_ , 52(5):5290–5309, 2022\.
  * Jain et al. [2023] Jitesh Jain, Jiachen Li, Mang Tik Chiu, Ali Hassani, Nikita Orlov, and Humphrey Shi.  Oneformer: One transformer to rule universal image segmentation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 2989–2998, 2023.
  * Jing and Kong [2023] Ya Jing and Tao Kong.  Learning to explore informative trajectories and samples for embodied perception.  _arXiv preprint arXiv:2303.10936_ , 2023.
  * Kolve et al. [2017] Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli VanderBilt, Luca Weihs, Alvaro Herrasti, Matt Deitke, Kiana Ehsani, Daniel Gordon, Yuke Zhu, et al.  Ai2-thor: An interactive 3d environment for visual ai.  _arXiv preprint arXiv:1712.05474_ , 2017.
  * Kotar and Mottaghi [2022] Klemen Kotar and Roozbeh Mottaghi.  Interactron: Embodied adaptive object detection.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 14860–14869, 2022.
  * Li et al. [2022] Chengshu Li, Ruohan Zhang, Josiah Wong, Cem Gokmen, Sanjana Srivastava, Roberto Martín-Martín, Chen Wang, Gabrael Levine, Michael Lingelbach, Jiankai Sun, Mona Anvari, Minjune Hwang, Manasi Sharma, Arman Aydin, Dhruva Bansal, Samuel Hunter, Kyu-Young Kim, Alan Lou, Caleb R Matthews, Ivan Villa-Renteria, Jerry Huayang Tang, Claire Tang, Fei Xia, Silvio Savarese, Hyowon Gweon, Karen Liu, Jiajun Wu, and Li Fei-Fei.  BEHAVIOR-1k: A benchmark for embodied AI with 1,000 everyday activities and realistic simulation.  In _6th Annual Conference on Robot Learning_ , 2022.  URL https://openreview.net/forum?id=_8DoIe8G3t.
  * Li and Bansal [2023] Jialu Li and Mohit Bansal.  Improving vision-and-language navigation by generating future-view image semantics.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10803–10812, 2023.
  * Muravyev et al. [2021] Kirill Muravyev, Andrey Bokovoy, and Konstantin Yakovlev.  Enhancing exploration algorithms for navigation with visual slam.  In _Russian Conference on Artificial Intelligence_ , pages 197–212. Springer, 2021.
  * NVIDIA [2021] NVIDIA.  Nvidia isaac sim.  https://developer.nvidia.com/isaac-sim, 2021.  [Online; accessed 28-September-2023].
  * Radford et al. [2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever.  Learning transferable visual models from natural language supervision, 2021.  URL https://arxiv.org/abs/2103.00020.
  * Ramakrishnan et al. [2022] Santhosh Kumar Ramakrishnan, Devendra Singh Chaplot, Ziad Al-Halah, Jitendra Malik, and Kristen Grauman.  Poni: Potential functions for objectgoal navigation with interaction-free learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 18890–18900, 2022.
  * Ronneberger et al. [2015] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation.  In _Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18_ , pages 234–241. Springer, 2015.
  * Schulman et al. [2017] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov.  Proximal policy optimization algorithms, 2017.
  * Shah et al. [2017] Shital Shah, Debadeepta Dey, Chris Lovett, and Ashish Kapoor.  Airsim: High-fidelity visual and physical simulation for autonomous vehicles.  In _Field and Service Robotics_ , 2017.  URL https://arxiv.org/abs/1705.05065.
  * Singh et al. [2022] Kunal Pratap Singh, Luca Weihs, Alvaro Herrasti, Aniruddha Kembhavi, and Roozbeh Mottaghi.  Ask4help: Learning to leverage an expert for embodied tasks.  In _NeurIPS_ , 2022.
  * Staroverov et al. [2023] Aleksei Staroverov, Kirill Muravyev, Konstantin Yakovlev, and Aleksandr I Panov.  Skill fusion in hybrid robotic framework for visual object goal navigation.  _Robotics_ , 12(4):104, 2023.
  * Wijmans et al. [2019] Erik Wijmans, Abhishek Kadian, Ari Morcos, Stefan Lee, Irfan Essa, Devi Parikh, Manolis Savva, and Dhruv Batra.  Dd-ppo: Learning near-perfect pointgoal navigators from 2.5 billion frames, 2019.  URL https://arxiv.org/abs/1911.00357.
  * Yadav et al. [2022] Karmesh Yadav, Ram Ramrakhya, Santhosh Kumar Ramakrishnan, Theo Gervet, John Turner, Aaron Gokaslan, Noah Maestre, Angel Xuan Chang, Dhruv Batra, Manolis Savva, et al.  Habitat-matterport 3d semantics dataset.  _arXiv preprint arXiv:2210.05633_ , 2022.  URL https://arxiv.org/abs/2210.05633.
  * Yadav et al. [2023a] Karmesh Yadav, Jacob Krantz, Ram Ramrakhya, Santhosh Kumar Ramakrishnan, Jimmy Yang, Austin Wang, John Turner, Aaron Gokaslan, Vincent-Pierre Berges, Roozbeh Mootaghi, Oleksandr Maksymets, Angel X Chang, Manolis Savva, Alexander Clegg, Devendra Singh Chaplot, and Dhruv Batra.  Habitat challenge 2023 public leaderboard.  https://eval.ai/web/challenges/challenge-page/1992/leaderboard/4705, 2023a.  [Online; accessed 28-September-2023].
  * Yadav et al. [2023b] Karmesh Yadav, Jacob Krantz, Ram Ramrakhya, Santhosh Kumar Ramakrishnan, Jimmy Yang, Austin Wang, John Turner, Aaron Gokaslan, Vincent-Pierre Berges, Roozbeh Mootaghi, Oleksandr Maksymets, Angel X Chang, Manolis Savva, Alexander Clegg, Devendra Singh Chaplot, and Dhruv Batra.  Habitat challenge 2023.  2023b.
  * Yadav et al. [2023c] Karmesh Yadav, Ram Ramrakhya, Santhosh Kumar Ramakrishnan, Theo Gervet, John Turner, Aaron Gokaslan, Noah Maestre, Angel Xuan Chang, Dhruv Batra, Manolis Savva, et al.  Habitat-matterport 3d semantics dataset.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 4927–4936, 2023c.
  * Yamauchi [1997] Brian Yamauchi.  A frontier-based approach for autonomous exploration.  In _Proceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA’97.’Towards New Computational Principles for Robotics and Automation’_ , pages 146–151. IEEE, 1997\.
  * Yang et al. [2019] Jianwei Yang, Zhile Ren, Mingze Xu, Xinlei Chen, David J Crandall, Devi Parikh, and Dhruv Batra.  Embodied amodal recognition: Learning to move to perceive objects.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 2040–2050, 2019.
  * Yang et al. [2017] Shichao Yang, Yulan Huang, and Sebastian Scherer.  Semantic 3d occupancy mapping through efficient high order crfs.  In _2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 590–597. IEEE, 2017.
  * Ye et al. [2021] Joel Ye, Dhruv Batra, Abhishek Das, and Erik Wijmans.  Auxiliary tasks and exploration enable objectgoal navigation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 16117–16126, 2021.
  * Zemskova et al. [2023] Tatiana Zemskova, Margarita Kichik, Dmitry Yudin, Aleksei Staroverov, and Aleksandr Panov.  Segmatron: Embodied adaptive semantic segmentation for indoor environment.  _arXiv preprint arXiv:2310.12031_ , 2023.
  * Zhou et al. [2019] Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fidler, Adela Barriuso, and Antonio Torralba.  Semantic understanding of scenes through the ade20k dataset.  _International Journal of Computer Vision_ , 127(3):302–321, 2019.
  * Zurbrügg et al. [2022] René Zurbrügg, Hermann Blum, Cesar Cadena, Roland Siegwart, and Lukas Schmid.  Embodied active domain adaptation for semantic segmentation via informative path planning.  _IEEE Robotics and Automation Letters_ , 7(4):8691–8698, 2022.


Fields and strings on non commutative q-deformed spaces

Poula Tadros

Department of Applied Physics, Aalto University School of Science, FI-00076
Aalto, Finland.

<EMAIL_ADDRESS>
###### Abstract

We study scalar field and string theory on non commutative q-deformed spaces.
We define a product of functions on a non commutative algebra of functions
resulting from the q-deformation analog to the Moyal product for canonically
non commutative spaces. We then give the general procedure to define scalar
field and classical string theories on the mentioned spaces, we argue that the
resulting theories will have enlarged sets of both spacetime and internal
symmetries which can be used to study gravitational effects due to the
q-deformation.

##  1 Introduction

Non commutative geometry was introduced in string theory in [1] where it was
shown that the coordinates of the endpoints of strings on D-branes in presence
of Neveu-Schwartz field is non commutative. In field theory it was even older
where Yang-Mills theory on non commutative torus was introduced [2].

The main motivation to introduce non commutative space times is field theory
is explained in [3,4]. In quantum mechanics Heisenberg uncertainty principle
states that at small distance scales there is a large uncertainty in momentum
measurement i.e. energy can reach very high values in small space distance
(close to the Planck scale), but according to the general theory of
relativity, high energy in sufficiently small distance scale creates a black
hole preventing measurement of position to be fully certain i.e. there is
uncertainty in position measurement in small scales, this can only be achieved
by introducing non commutativity in space time. Notice that this implies non
locality in the theory.

Since the introduction of non commutativity in field and string theories a lot
of progress has been made in all directions including classical and quantum
field theories, theories of gravity and string theory. However, the non
commutativity used is the canonical non commutativity which does not capture
the mathematical structure of the given field or string theory and it is clear
that it was imposed by hand. In this article we use another type of non
commutativity, the q-deformation, to study classical scalar field theory and
the consequences on string theory. In section 2, we review the most popular
types of non commutativity on space times and motivate the choice of
q-deformation as the non commutativity of choice. In section 3, we define a
product of functions on q-deformed spaces similar to the Moyal product on
canonically non commutative spaces and show the procedure failed for Lie-type
non commutativity. In section 4, we study scalar field theory on q-deformed
space time. In section 5, we study string theory on the same space time. In
section 6, we discuss the symmetries of the non commutative theories, we show
that there are more symmetries in the non commutative theories than the
corresponding commutative ones, then use this to argue that we can define
theories with dynamical spacetimes by quantizing the spacetime symmetry group
of the theory.

##  2 Types of non commutativity

This section is dedicated to review three types of non commutativity of space
times

###  2.1 Canonical non commutativity

It is the simplest type and is the one used in physics literature, it was
introduced in [5], it is defined by imposing the following commutation
relations on the space time

$[x^{\mu},x^{\nu}]=i\theta^{\mu\nu},$

where $x^{\mu}$ are the space time coordinates and $\theta^{\mu\nu}$ is a
constant, anti symmetric matrix.

Canonical non commutativity corresponds to smearing of the space time, it can
be easily seen from solutions of polynomial equations on the commutative space
compared to its non commutative counterpart.

As an example consider the two dimensional Euclidean space with coordinates
$x$ and $y$ with the commutation relation $[x,y]=k$, where $k$ is a positive
constant, and consider the polynomial equation $(x-y)^{2}=0$.
In the usual commutative space, the solution to the above equation is $x=y$
which is a straight line with slope = 1 and passing through the origin.
However, in the corresponding non commutative space the equation can be
written as $x^{2}-2yx+y^{2}=k$ whose solutions are two parallel straight lines
separated by a distance proportional to $k$, when $k=0$ the two straight lines
coincide and we recover the solution on the commutative space.

Note that this procedure is valid regardless of whether or not there are
additional mathematical structures on the space, the smearing is carried out
the same way. That is why we need more complicated non commutativity to use in
physics.

###  2.2 Lie-type non commutativity

In this case the coordinates has a Lie algebra structure i.e. the commutation
relations can capture a Lie algebra structures if defined on the space time
for example like in field theories [6]. The commutation relations are given by

$[x^{\mu},x^{\nu}]=if^{\mu\nu}_{\rho}x^{\rho},$

where $f^{\mu\nu}_{\rho}$ are the structure constants of the defined Lie
algebra. However, this type is not useful because Lie structures are rigid
i.e. any small deformation of a Lie algebra is isomorphic to the Lie algebra.
This leads to difficulties in defining products of functions on the resulting
non commutative space as we will see in the next section.

###  2.3 q-deformations

This type was introduced to solve the rigidity problem for Lie algebras. The
main idea is to replace Lie group with a flexible structure which is called
quantum groups, for more details on the theory of quantum groups see [7,8].

The commutation relations are given by

$x^{\mu}x^{\nu}=\frac{1}{q}R^{\mu\nu}_{\sigma\tau}x^{\sigma}x^{\tau},$

where $q$ is a parameter and $R^{\mu\nu}_{\sigma\tau}$ is the R-matrix of the
quantum group defined on the space.

In this space a Lie algebra is replaced by a non commutative Hopf algebra with
deformation parameter q. Hopf algebras are considered deformations of the
universal enveloping algebra of the Lie group. The resulting space is deformed
according to the Lie group on the space and on the parameter q, this is the
simplest way to deform a space time while capturing the full structure of the
space, other more complicated approaches can be studied such as deforming with
more than one parameter [9,10] but it is beyond the scope of the article.

##  3 Moyal-like product on q-deformed spaces

Here, we define a non commutative product of functions on q-deformed spaces
i.e. non commutative spaces in the R-matrix formalism.

###  3.1 Moyal product on canonically non commutative spaces

We begin with reviewing the original Moyal product. On Canonically non
commutative spaces, the algebra of functions is replaced by a non commutative
$C*$ algebra, the Moyal product is the product of functions on the non
commutative algebra. Its formula can be derived as follows:
Consider two functions $f(x)$ and $g(x)$, their Fourier transforms are

$f(x)=\int\frac{d^{D}k}{(2\pi)^{D}}\bar{f(k)}e^{ik_{i}x^{i}},$
$g(x)=\int\frac{d^{D}k^{\prime}}{(2\pi)^{D}}\bar{g(k)}e^{ik_{j}x^{j}},$

The product on the non commutative space is

$f(x)\star
g(x)=\int\frac{d^{D}kd^{D}k^{\prime}}{(2\pi)^{2D}}\bar{f(k)}\bar{g(k)}e^{ik_{i}x^{i}}e^{ik_{j}x^{j}}.$

Using Baker-Campbell-Hausdorff formula we get

$f(x)\star
g(x)=\int\frac{d^{D}kd^{D}k^{\prime}}{(2\pi)^{2D}}\bar{f(k)}\bar{g(k)}e^{ik_{i}x^{i}}e^{ik_{j}x^{j}}e^{i/2k_{i}k_{j}\theta^{ij}}$
$=\int\frac{d^{D}kd^{D}k^{\prime}}{(2\pi)^{2D}}\bar{f(k)}\bar{g(k)}e^{ik_{i}x^{i}}e^{ik_{j}x^{j}}(1+\sum_{n=1}^{\infty}(\frac{i}{2})^{n}\frac{1}{n!}(k_{i}k_{j}\theta^{ij})^{n})$
$=f(x)g(x)+\sum_{n=1}^{\infty}(\frac{i}{2})^{n}\frac{1}{n!}\theta^{i_{1}j_{1}}...\theta^{i_{n}j_{n}}\partial_{i_{1}}\partial_{i_{2}}...\partial_{i_{n}}f\partial_{j_{1}}...\partial_{j_{n}}g.$

###  3.2 Product of functions on q-deformed space

We follow the same procedure to define a product on q-deformed spaces. The non
commutativity is given by

$x^{\mu}x^{\nu}=\frac{1}{q}R_{\sigma\tau}^{\mu\nu}x^{\sigma}x^{\tau}.$

It can be written as a commutation relation as

$[x^{\mu},x^{\nu}]=Q_{\sigma\tau}^{\mu\nu}x^{\sigma}x^{\tau},$

where
$Q_{\sigma\tau}^{\mu\nu}=\frac{1}{q}R_{\sigma\tau}^{\mu\nu}-\delta_{\tau}^{\mu}\delta_{\sigma}^{\nu}.$
Note that at $q=1$ we have $Q_{\sigma\tau}^{\mu\nu}=0$ and we recover
commutativity.

Now again two functions $f(x)$ and $g(x)$. the product of their Fourier
transforms is

$f(x)\star
g(x)=\int\frac{d^{D}kd^{D}k^{\prime}}{(2\pi)^{2D}}\bar{f(k)}\bar{g(k)}e^{ik_{i}x^{i}}e^{ik_{j}x^{j}},$

and using Baker-Campbell-Hausdorff formula we get

$f(x)\star
g(x)=\int\frac{d^{D}kd^{D}k^{\prime}}{(2\pi)^{2D}}\bar{f(k)}\bar{g(k)}e^{ik_{i}x^{i}}e^{ik_{j}x^{j}}$
$e^{x^{i}+x^{j}+1/2[x^{i},x^{j}]+1/12[x^{i},[x^{i},x^{j}]]-1/12[x^{j},[x^{i},x^{j}]]+...},$

after some calculations we have

$[x^{i},[x^{i},x^{j}]]=(Q^{ij}_{nl}Q_{ab}^{il}+Q_{cb}^{ij}Q_{na}^{im})x^{n}x^{a}x^{b},$
$[x^{i},[x^{i},x^{j}]]=(Q^{ij}_{nl}Q_{ab}^{jl}+Q_{cb}^{ij}Q_{na}^{jm})x^{n}x^{a}x^{b},$

Substituting we get

$f(x)\star
g(x)=\int\frac{d^{D}kd^{D}k^{\prime}}{(2\pi)^{2D}}\bar{f(k)}\bar{g(k)}e^{ik_{i}x^{i}}e^{ik_{j}x^{j}}$
$e^{x^{i}k_{i}+x^{j}k^{\prime}_{j}+1/2Q^{ij}_{mn}x^{m}x^{n}k_{i}k^{\prime}_{j}+1/12k_{i}k^{\prime}_{j}(Q^{ij}_{nl}Q_{ab}^{il}-Q_{cb}^{ij}Q_{na}^{im}-Q^{ij}_{nl}Q_{ab}^{jl}+Q_{cb}^{ij}Q_{na}^{jm})x^{n}x^{a}x^{b}+...}.$

In string theory it is reasonable to assume that the parameter $q$ is close to
1 since the operator $Q$ is related to the string length which is assumed to
be very small. In field theory the assumption is reasonable as well since in
this case $Q$ is related to the area of the space time where general
relativity breaks i.e. quantum gravity scale which is assumed to be very
small. Thus, the series converges and all the exponentials are well defined.
If we ignore the higher orders in the last exponent we get a formula similar
to the Moyal product:

$f(x)\star
g(x)=\int\frac{d^{D}kd^{D}k^{\prime}}{(2\pi)^{2D}}\bar{f(k)}\bar{g(k^{\prime})}e^{ik_{i}x^{i}}e^{ik^{\prime}_{j}x^{j}}e^{1/2Q^{ij}_{mn}x^{m}x^{n}k_{i}k^{\prime}_{j}}$
$=\int\frac{d^{D}kd^{D}k^{\prime}}{(2\pi)^{2D}}\bar{f(k)}\bar{g(k)}e^{ik_{i}x^{i}}e^{ik^{\prime}_{j}x^{j}}(1+\sum_{n=1}^{\infty}(1/2)^{n}\frac{1}{n!}(Q^{ij}_{mn}x^{m}x^{n}k_{i}k^{\prime}_{j})^{n})$
$=f(x)g(x)+\sum_{p=1}^{\infty}(1/2)^{p}\frac{1}{p!}x^{m_{1}}x^{l_{1}}Q_{m_{1}l_{1}}^{i_{1}j_{1}}...x^{m_{p}}x^{l_{p}}Q_{m_{p}l_{p}}^{i_{p}j_{p}}\partial_{i_{1}}\partial_{i_{2}}...\partial_{i_{p}}f(x)\partial_{j_{1}}...\partial_{j_{p}}g(x).$

The formula captures the mathematical structures on the space time in the form
of the $Q$ operator and the deformation is subsequently transformed into these
structures leading to additional symmetries at least in string theory.

From this procedure we can see that Lie-type non commutativity presents
difficulties in defining the product since the product would be of the form

$f(x)\star
g(x)=\int\frac{d^{D}kd^{D}k^{\prime}}{(2\pi)^{2D}}\bar{f(k)}\bar{g(k)}e^{ik_{i}x^{i}}e^{ik_{j}x^{j}}$
$e^{1/2(f^{ij}_{m}x^{m}k_{i}k^{\prime}_{j})+1/12k_{i}k^{\prime}_{j}(f^{ij}_{nl}f_{ab}^{il}-f_{cb}^{ij}f_{na}^{im})+...}.$

The series in the exponential generally diverges and the product can not be
defined.

##  4 Scalar field theory on q-deformed space

In this section we study massive scalar field theory on a flat non dynamical
q-deformed non commutative space time. While there are attempts to define
field theories on deformed spaces, the used non commutativity is usually the
canonical type or the use of quantum groups is limited to defining
differential structure on a specific examples of spaces [11-14], conformal
field theory also was studied on deformed spaces [15], however the
deformations considered are introduced by hand and does not introduce non
commutativity i.e. the deformed manifold is another manifold with no
additional structure.

The Lagrangian of the theory is

$\mathcal{L}=\partial_{\mu}\phi\partial^{\mu}\phi-m^{2}\phi^{2},$

where $\phi$ is the scalar field which we will assume is infinitely
differentiable, and $m$ is the mass.

Now we perform the deformation quantization:
The first step is to perform the q-quantization of the symmetry group in this
case $U(1)$. To do this we write its universal enveloping algebra which is a
$C*$ algebra of functions generated by the function
$z\in\mathcal{U}(U(1))\rightarrow e^{(iz)}\in\mathbf{C}$. We notice that the
algebra is commutative then its deformations are equivalent to itself, i.e. no
contribution from the symmetry group to non commutativity and the product on
the non commutative space is equivalent to the product on the original
manifold.

The second step is to replace the manifold on which the field theory is
defined with a non commutative locally compact topological space. On this
manifold the derivatives are q-deformed into Jackson derivatives.

The new q-deformed Lagrangian will be

$\mathcal{L}_{q}=D_{q\mu}\phi D_{q}^{\mu}\phi-m^{2}\phi^{2},$

Now we relate the theory on the non commutative topological space to the
theory on the commutative manifold (i.e. transforming the non commutative
theory back to the commutative manifold) using the formula

$D_{q\mu}(f(x))=\partial_{\mu}f+\sum_{k=1}^{\infty}\frac{(q-1)^{k}}{(k+1)!}x_{\mu}^{k}f^{(k+1)}(x),$

where $f^{(k)}$ is the k th ordinary derivative of f.

The resulting Lagrangian on the commutative manifold is

$\mathcal{L}_{q}=\partial_{\mu}\phi\partial^{\mu}\phi-m^{2}\phi^{2}+2\partial_{\mu}\phi\sum_{k=1}^{\infty}\frac{(q-1)^{k}}{(k+1)!}x^{\mu
k}\phi^{(k+1)}$
$+\sum_{l,m=1}^{\infty}\frac{(q-1)^{(l+m)}}{(m+1)!(l+1)!}\phi^{(l+1)}x_{\mu}^{k}x^{\mu
l}\phi^{(m+1)}.$

The first two terms of the Lagrangian is the non commutative original theory
and the rest are the contributions of non commutativity from replacing the non
commutative topological space with the original commutative manifold. The
theory is q-deformed i.e. if $q=1$ then we recover the original theory.

The additional terms are non local as expected and contain an infinite series
of higher (ordinary) derivatives of the field $\phi$.

##  5 String theory on q-deformed space

String theory follows the same q-quantization procedure as field theory but
with richer geometry since the fundamental object is one dimensional. Here we
establish the connection between the $Q$ operator defined above and the length
of the string, then give the general procedure of defining a string theory on
q-deformed space.

The uncertainty in position in case of q-deformed spaces can be calculated to
be

$\Delta x^{i}\Delta x^{j}\geq\frac{1}{2}<x^{\mu}Q_{\mu\nu}^{ij}x^{\nu}>,$

where $<x^{\mu}Q_{\mu\nu}^{ij}x^{\nu}>$ is the expectation value of the
quadratic form of the operator $Q$.

Following the same argument as [2], we find that the length of the string
squared is proportional to the above expectation value

$<x^{\mu}Q_{\mu\nu}^{ij}x^{\nu}>\ \ \propto l_{s}^{2}.$

This implies that the string’s length depends on the geometry of the non
commutative space i.e. depends on the string theory in question, and is
determined by the R-matrix of the quantized group.

The procedure on a static spacetime is as follows:

  1. 1.

Determine the symmetry group of the theory and find the corresponding quantum
group.

  2. 2.

Use the product presented in section 3 instead of the usual product and
Jackson’s derivative instead of the usual derivative.

  3. 3.

Use the corresponding formulae to relate back to the original manifold as we
did in section 4, this usually leads to infinite series of higher derivatives
in the Lagrangian.

##  6 Symmetries and theories on dynamical spacetimes

The first step of q-quantization is to replace the symmetry group with a
quantum group which is a deformation of its universal enveloping algebra. This
gives more symmetries than the commutative theory by definition.

In field and string theory, symmetries are classified into spacetime
symmetries and internal symmetries, spacetime symmetries relates directly to
the ambient manifold on which the field/string theory is defined while
internal symmetries are additional structure on the manifold. While on static
spacetime (disregarding gravity) only the internal symmetry group is to be
q-deformed, the spacetime symmetry group must contribute to the R-matrix if
dynamic spacetimes are to be studied, the deformations of the spacetime
symmetry should lead to effects on the gravitational aspects of the theory
like changes in curvature, singularities, etc. Similar studies of non
commutativity’s effects on gravity are found in [] but uses the canonical non
commutativity, using q-deformations to study gravity is a subject of future
research.

##  7 Conclusion and outlook

The results presented in this paper showed that a product of functions on a
q-deformed space at least for small deformations exists and is well defined,
we give an explicit formula in the paper. We also showed that field and string
theory can be defined on q-deformed manifolds but having enlarged set of
symmetries and extra features depending on the theory and the manifold in
question.

A possible direction of future research is to study the enlarged set of
symmetries due to q-deformations as well as their mathematical and the
phenomenological implications. Another direction is to study more complicated
field/string theories and find ways to define higher spin fields on such
spaces.

## Acknowledgments

We would like to thank Dr.Ivan Kolar for the useful discussions on the topic.

## References

  1. [1]

Seiberg, N. and Witten, E. (1999) “String theory and noncommutative geometry,”
Journal of High Energy Physics, 1999(09), pp. 032–032.

  2. [2]

Szabo, R. (2003) “Quantum field theory on noncommutative spaces,” Physics
Reports, 378(4), pp. 207–299.

  3. [3]

Doplicher, S., Fredenhagen, K. and Roberts, J.E. (1995) “The quantum structure
of spacetime at the Planck scale and Quantum Fields,” Communications in
Mathematical Physics, 172(1), pp. 187–220.

  4. [4]

Ahluwalia, D.V. (1994) “Quantum measurement, gravitation, and locality,”
Physics Letters B, 339(4), pp. 301–303.

  5. [5]

C. S. Chu and P. M. Ho, Noncommutative open string and D-brane, Nucl. Phys. B
550, 151 (1999) [hep-th/9812219].

  6. [6]

B. Jurco, S. Schraml, P. Schupp and J. Wess, Enveloping algebra valued gauge
transformations for non-Abelian gauge groups on non-commutative spaces, Eur.
Phys. J. C17, 521 (2000) [hep-th/0006246].

  7. [7]

Chaichian, M. and Demichev, A.P. Introduction to quantum groups. Singapore:
World Scientific (1996).

  8. [8]

A. Klimyk and K. Schmudgen, Quantum Groups and Their Representations, Springer
(1997).

  9. [9]

Hu, N.H. and Pei, Y.F. (2008) “Notes on 2-parameter Quantum Groups I,” Science
in China Series A: Mathematics, 51(6), pp. 1101–1110.

  10. [10]

Hu, N. and Pei, Y. (2012) “Notes on two-parameter quantum groups, (II),”
Communications in Algebra, 40(9), pp. 3202–3220.

  11. [11]

Wulkenhaar, R. (2006) “Field theories on deformed spaces,” Journal of Geometry
and Physics, 56(1), pp. 108–141.

  12. [12]

Grosse, H., Madore, J. and Steinacker, H. (2001) “Field theory on the
Q-deformed fuzzy sphere I,” Journal of Geometry and Physics, 38(3-4), pp.
308–342.

  13. [13]

Grosse, H., Madore, J. and Steinacker, H. (2002) “Field theory on the
Q-deformed Fuzzy Sphere II: Quantization,” Journal of Geometry and Physics,
43(2-3), pp. 205–240.

  14. [14]

BARDEK, V., DOREŠIĆ, M. and MELJANAC, S. (1994) “An example of Q-deformed
field theory,” International Journal of Modern Physics A, 09(23), pp.
4185–4194.

  15. [15]

Minahan, J., Naseer, U. and Thull, C. (2021) “Conformal field theories on
deformed spheres, anomalies, and supersymmetry,” SciPost Physics, 10(3).


# What sunspots are whispering about covid-19?

Mariam M. Morchiladze<EMAIL_ADDRESS>Tamila K. Silagadze
<EMAIL_ADDRESS>Zurab K. Silagadze<EMAIL_ADDRESS>Tbilisi
State Medical University, Tbilisi, Georgia  Novosibirsk State University and
Budker Institute of Nuclear Physics, 630 090, Novosibirsk, Russia.

###### Abstract

Several studies point to the antimicrobial effects of ELF electromagnetic
fields. Such fields have accompanied life from the very beginning, and it is
possible that they played a significant role in its emergence and evolution.
However, the literature on the biological effects of ELF electromagnetic
fields is controversial, and we still lack an understanding of the complex
mechanisms that make such effects, observed in many experiments, possible. The
Covid-19 pandemic has shown how fragile we are in the face of powerful
processes operating in the biosphere. We believe that understanding the role
of ELF electromagnetic fields in regulating the biosphere is important in our
fight against Covid-19, and research in this direction should be intensified.

###### keywords:

Covid-19 pandemic; Biological effects of ELF electromagnetic fields; Schumann
resonances.

##  1 Introduction

The covid-2019 pandemic is an extraordinary event in the history of our
civilization. Billions of people have never been restricted in their homes
before. Self-isolation is conducive to esoteric reflection, and we begun to
ruminate about Gaia.

Life on earth, once arisen, never completely disappeared, although it survived
at least five major extinctions. Isn’t that a small (or big) miracle?

Gaia hypothesis suggests that Nature’s mercy is neither an accident nor a
benevolent deity’s work, but instead is the inevitable result of interactions
between organisms and their environment [1]. In poetic terms, the Earth with
its ecosystem is a gigantic organism that harmonizes itself with the help of
many invisible feedbacks.

Unfortunately, human activity causes environmental degradation, which in its
destructive effect on the ecosystem rapidly approaches the level of the
asteroid impact that end the dinosaur era.

We better stop the destabilization of Gaia, because otherwise either we
succeed in this reckless enterprise and destroy the ecosystem and our own
existence, or Gaia recognizes our malicious character and invisible feedbacks
eliminate us, as a destabilizing element. However, in order to harmonize with
Gaia, we must improve our understanding of these invisible feedbacks.

##  2 Solar activity and pandemics

At first glance (and at the second too) there can be no connection between
pandemics and solar activity. However, this is exactly what Alexander
Chizhevsky discovered [2] many years ago.

Independently, Hope-Simpson observed the same correlation between influenza
pandemics and sunspot maximums [3]. Hoyle and Wickramasinghe confirmed these
findings [4] indicating that the two phenomena have kept in step over some 17
solar cycles.

Interestingly, the previous two Corona virus epidemics, the severe acute
respiratory syndrome SARS-CoV and the Middle-East respiratory syndrome MERS-
CoV both occurred at double peaks in the sunspot cycle [5].

A more general result states that most pandemics in the past occurred near the
sunspot extrema (maxima or minima) [6, 7]. In a sense, the present covid-19
pandemic was predicted based on this idea [8, 9].

Of course, such an outlandish idea should be taken with a grain of salt, and
not everybody believes in it [10]. However, we think this strange idea cannot
be dismissed easily. The correlation in the data is so obvious that different
people have noticed it. We already mentioned Chizhevsky and Hope-Simpson. It
seems that the first person who linked the sunspot cycle to the malaria
epidemics, as early as in 1881, was C. Meldrum [11]. In 1936, Gill noted that
the association of malaria epidemics with the epochs of maximum and minimum of
sunspots is extremely close [11].

Solar activity can affect biological organisms in various ways. Of particular
interest is the influence mediated by geomagnetic and extra-low frequency
(ELF) electromagnetic fields [12, 13].

The Schumann resonance with base frequency of about 8 Hz is a global
electromagnetic resonance excited by natural lightning activity within the
Earth-ionosphere waveguide [14]. Life evolved on Earth under the constant
presence of Schumann resonances, and thus, we cannot exclude that ELF
electromagnetic fields played a role in biological evolution.

Human activity has become a source of widespread electromagnetic pollution,
which raises concerns about the possible dangerous health effects [15].
Although studies on the possible biological effects of ELF and other
artificial electromagnetic fields remain largely controversial, there is a
growing evidence that ELF fields cause numerous types of changes in cells [16,
17, 18].

##  3 Biological effects of ELF electromagnetic fields

Modern life is not conceived without electricity with its power lines and
appliances, without telecommunication devices. A byproduct of this technical
revolution is an ever-growing number of sources of artificial electromagnetic
fields, both in ELF and radio frequency range, and this circumstance, as
already mentioned, elicits public health concerns [15].

The scope of interactions of electromagnetic fields (EMF) with living matter
is rich and diverse, requiring simultaneously an unbiased, open-minded,
careful and cautious approach when studying the influence of EMF on biological
processes.

Despite a large body of literature devoted to biological effects of ELF
magnetic fields (in contrast to ELF electric fields, magnetic fields easily
can penetrate biological tissues), no coherent picture has emerged so far
regarding the plausibility of such effects, or regarding the interaction
mechanisms.

Epidemiological studies that have focused on the potential health hazards of
EMFs are largely controversial. About half of the studies found such effects,
but the other half failed to find them [19]. The reason for these conflicting
results is unclear.

Nonetheless, ample and compelling evidence had been accumulated indicating
that ELF electromagnetic fields has important effects on cell functioning [20,
21, 22, 23, 24, 25, 26]. The nature of these effects is not entirely clear.
The problem is that such effects are observed for very weak magnetic fields,
so weak that any such effect is expected to be masked by thermal noise [27].

Perhaps, the extreme sensitivity of living organisms to weak electromagnetic
fields is not completely unexpected from the point of view of evolutionary
biology, since life arose and evolved on Earth with the constant presence of
natural ELF electromagnetic fields, especially Schumann resonances [28, 29].

Remarkably, there is a striking similarity between natural ELF signals and
human brain electroencephalograms [30]. Amazingly, many species exhibit,
irrespective of the size and complexity of their brain, essentially similar
low-frequency electrical activity [31], and it is possible that the dominant
frequencies of brain waves may be an evolutionary result of the presence and
influence of the resonant ELF electromagnetic background of Schumann [32].

An interesting hypothesis is that ELF background fields played an important
role in the evolution of biological systems and are used by them as a means of
stochastic synchronization for various biorhythms [29]. The Schumann resonance
frequencies are mainly controlled by the Earth’s radius, which has remained
constant over billions of years. Therefore, these frequencies can play a
special role for the regulatory pathways of living organisms, the Schumann
resonances providing a synchronization reference signal, a Zeitgeber (time
giver) [33].

This hypothesis, while attractive, has a serious drawback: it remains a
mystery how living cells can detect a Schumann resonance signal that is so
weak (the magnetic component is only several $pT$) compared to the ubiquitous
thermal noise. The clue maybe is provided by spatially and temporally coherent
interactions of Schumann resonances with a large ensemble of components of the
system [34]. For example, the human body contains about $10^{14}$ cells and
$10^{10}$ cortical neurons.

If the ELF electromagnetic fields, and in particular the Schumann resonances,
really play a regulatory role in biological processes, then the effect of
solar activity on living organisms will not look so mysterious, since solar
activity changes the geomagnetic field and can lead to geomagnetic storms, as
well as to changes in the parameters of the ionosphere, and, consequently, to
a change in the parameters of Schumann resonances and ELF radiation
background.

There are some indications that abrupt changes in geomagnetic and solar
activity, as well as geomagnetic storms, can act as stressors that alter the
regulatory processes of organisms, blood pressure, immune, neurological,
cardiac and some other important life-supporting processes in living organisms
[35]. There are studies that indicate that geomagnetic disturbances can
exacerbate existing diseases, can lead to cardiac arrhythmias, cardiovascular
disease, a significant increase in hospitalization rates for mental disorders,
depression, suicide attempts, homicide and traffic accidents (see [35] and
references therein).

There are several hypotheses that could explain this strange connection
between solar activity and geomagnetic disturbances and mental health.
According to the visceral theory of sleep [36], the same cortical neurons that
process exteroceptive information in the waking state switch to processing
information from various internal organs during sleep. At the same time, a
violation of the switching process, when visceral information is mistakenly
interpreted by the brain as exteroceptive, can manifest itself as a mental
disorder [37]. If these hypotheses are correct, and if geomagnetic
disturbances can influence the brain’s switching mechanisms, then the
unexpected link between solar activity and mental disorders could be
explained.

##  4 Concluding remarks

Scientific progress has greatly expanded the ability of humankind to cause
large-scale changes in the environment. Unfortunately, we do not always
understand the subtle feedback loops that operate in the biosphere to predict
all consequences of such changes. An amusing example of the unexpected outcome
of a large-scale human intervention in nature is provided in [38].

For unknown reasons, fish in the Gulf of Mexico position themselves over
buried oil pipelines off the shore of Texas, orienting themselves directly
above the buried pipeline at a height of 1-3 meters above the seabed and
perpendicular to the axis of the pipeline. Presumably they are responding to
some electromagnetic stimuli, such as remnant magnetism in pipeline sections,
voltage gradient induced by corrosion protection devices, or transient signals
induced into the pipeline by remote lightnings or solar wind induced magnetic
storms [38].

The research of biological effects was intensified at about 1967 as part of an
evaluation of the environmental impact of a proposed ELF military antenna
(Project Sanguine) [39]. Unfortunately, the presence of military and
commercial components in this research makes it politically very sensitive
[28]. Nonetheless, in light of the results so far available, it would be too
irresponsible to dismiss such effects as being implausible [34].

On the practical side, if ELF fields cause biological effects, whatever the
unknown mechanism of this interaction, we can try to use these effects in our
fight against SARS-CoV-2 and similar infections. It is known that bioelectric
signals generated by the metabolic activity of cells are in an ELF range,
therefore by interfering with these signals by external low intensity ELF
electromagnetic fields we can suppress microbial or bacterial activities [40,
41, 42].

We believe that under the burden of the Covid-19 pandemic, research in this
direction should be intensified. Studies of the antimicrobial effects of ELF
electromagnetic fields are not expected to be too expensive. If successful, it
promises a non-invasive, inexpensive, safe and fast technique to fight
infections [41, 42].

Is Covid-19 pandemic related to the deepest sunspot minimum for a century we
are experiencing now? During sunspot minimum solar magnetic field gets weaker
and, as a result, galactic cosmic rays flux entering the earth increases.
There are some indications that an increase in the flux of cosmic rays can
lead to an increase in lightning activity on earth [43, 44] and thus change
the natural ELF electromagnetic background. As noted in [28], changing the
electromagnetic background poses a twofold challenge to us: weakening the
immune system due to constant stress and more severe illnesses, since
electromagnetic fields can stimulate bacterial growth and increase their
resistance to antibiotics.

Increased cosmic rays can lead also to appearance of novel virion strains due
to induced mutation and genetic recombination events [9, 56], especially if
viruses spread even beyond the tropopause (new bacteria have been found in the
stratosphere and even on the exterior of the International Space Station,
orbiting at a altitude of 400 km) [56].

Interestingly, the idea that the flux of galactic cosmic rays can affect the
ELF electromagnetic background can be tested using the Forbush effect [45].
During solar flares, the flux of galactic cosmic rays decreases rapidly (over
a day or less) due to modification of the near-Earth interplanetary magnetic
field. This so-called Forbush decrease is transient and is followed by a
gradual recovery over several days [46, 47, 48].

Based on the measurements in the Kola peninsula, it was demonstrated that in
all ten events of significant Forbush-decreases, the intensity of the ELF-
atmospherics decreased (down to their complete disappearance) [45]. It was
hypothesized that this phenomenon is caused by a decrease in the intensity of
discharges of a special type (sprites and jets) as a result of a decrease in
atmospheric ionization at altitudes of 10-30 km during the Forbush decrease in
the flux of galactic cosmic rays [45].

Cosmic ray forcing of the climate acts simultaneously and with the same sign
throughout the entire globe and operates on all time scales from days to
hundreds of millions of years [49]. For this reason, even a relatively small
forcing can lead to a large climatic response over time [49]. To unravel the
anthropogenic contribution to the current climate change and assess its
danger, which is now the subject of much public concern and controversy, we
need to understand physical mechanisms underlying the influence of solar and
cosmic ray variability on climate and their impact on the biosphere.

It has recently been shown that bats, like many other animals with highly
developed magnetosensory skills, use magnetic field for orientation and can
sense even very weak magnetic fields [50, 51, 52]. Perhaps, this
magnetoreception is influenced by ELF electromagnetic background [53, 54].
Another source of possible influence is the change in cloudiness due to the
increased flux of galactic cosmic rays, as bats have been shown to calibrate
their magnetic compasses with sunset cues [55].

So, one can imagine the following scenario 111Suggested by an anonymous
reviewer. Changes in the ELF electromagnetic background, caused by the
increased flux of cosmic rays due to unusually deep sunspot minimum, can cause
abnormal movements of the population of bats and affect the time of their
arrival and departure. Delayed arrival or departure and longer travel times
can increase the population of bats in some areas, thereby increasing
competition for limited food supplies, and can also increase the likelihood of
interspecies transmission of the virus. Besides, increased level of
irradiation increases genetic recombination rates, as was demonstrated in
laboratories during the 1950s and 1960s [6]. Finally, under the influence of
these circumstances, the new coronavirus successfully recombined and caused
the Covid-19 pandemic.

We end this article with a funny observation from [56]. The Italian word
influencia means influence, meaning the influence of the stars in the case of
influenza illness. This etiology reflects the belief of our ancestors that
events in the sky and events on Earth are interconnected. It may well be that
they were right.

## Acknowledgments

The authors thank Olga Chashchina and an anonymous reviewer for critical
comments that helped to improve the manuscript.

## Conflict of interest statement

The author declares that the research was conducted in the absence of any
commercial or financial relationships that could be construed as a potential
conflict of interest.

## References

## References

  * [1] J. W. Kirchner, The Gaia Hypothesis: Fact, Theory, and Wishful Thinking, Climatic Change 52 (2002), 391–408. DOI: 10.1023/A:1014237331082
  * [2] A. L. Chizhevsky, The Terrestrial Echo of Solar Storms (Mysl: Moscow, 1976). First published in 1936.
  * [3] R. E. Hope-Simpson, Sunspots and flu: a correlation, Nature 275 (1978), 86. DOI:10.1038/275086a0
  * [4] F. Hoyle and N.C. Wickramasinghe, Sunspots and influenza, Nature 343 (1990), 304. DOI: 10.1038/343304a0
  * [5] J. Qu and N.C. Wickramasinghe, SARS, MERS and the sunspot cycle, Current Science 113 (2017), 1501–1502.
  * [6] J. Qu , Is sunspot activity a factor in influenza pandemics? Rev. Med. Virol. 26 (2016), 309–313. DOI: 10.1002/rmv.1887
  * [7] N.C. Wickramasinghe at al., Sunspot Cycle Minima and Pandemics: The Case for Vigilance? Astrobiol. Outreach 5 (2017), 159. DOI: 10.4172/2332-2519.1000159
  * [8] N.C. Wickramasinghe and J. Qu, Are We Approaching a New Influenza Pandemic? Virol. Curr. Res. 2 ( 2018), 107.
  * [9] N.C. Wickramasinghe at al., Space weather and pandemic warnings? Current Science 117 (2019), 1554.
  * [10] S. Towers, Sunspot activity and influenza pandemics: a statistical assessment of the purported association, Epidemiology & Infection 145 (2017), 2640–2655. DOI:10.1017/S095026881700173X
  * [11] C.A. Gill, Some points in the epidemiology of malaria arising out of the study of the malaria epidemic in Ceylon in 1934–1935, Trans. Roy. Soc. Trop. Med. Hyg. 29 (1936), 427–480. DOI:10.1016/S0035-9203(36)90001-9
  * [12] N.J. Cherry, Schumann Resonances, a plausible biophysical mechanism for the human health effects of Solar/Geomagnetic Activity, Natural Hazards 26 (2002), 279–331. DOI:10.1023/A:1015637127504
  * [13] V. Zaporozhan and A. Ponomarenko, Mechanisms of Geomagnetic Field Influence on Gene Expression Using Influenza as a Model System: Basics of Physical Epidemiology, Int. J. Environ. Res. Public Health 7 (2010), 938–965. DOI:10.3390/ijerph7030938
  * [14] A. Nickolaenko and M. Hayakawa, Schumann Resonance for Tyros (Springer: Tokyo, 2014).
  * [15] P. Bandara and D.O. Carpenter, Planetary electromagnetic pollution: it is time to assess its impact, Lancet Planet. Health 2 (2018), e512–e514. DOI:10.1016/S2542-5196(18) 30221-3.
  * [16] M.T. Santini, G. Rainaldi and P.L. Indovina, Cellular effects of extremely low frequency (ELF) electromagnetic fields, Int. J. Rad. Bio. 85 (2009), 294–313. DOI:10.1080/09553000902781097
  * [17] N.G. Ptitsyna, G. Villoresi, L.I. Dorman, N. Iucci and M.I. Tyasto, Natural and man-made low-frequency magnetic fields as a potential health hazard, Phys. Usp. 41 (1998), 687–709. DOI:10.1070/PU1998v041n07ABEH000419
  * [18] E.S. Babayev, N.B. Crosby, V.N. Obridko and M.J. Rycroft, Potential effects of solar and geomagnetic variability on terrestrial biological systems, In G. Maris and C. Demetrescu (Eds.), Advances in Solar and Solar-Terrsetrial Physics (Research Signpost: Kerala, India, 2012), pp. 329–376.
  * [19] Y. Touitou and B. Selmaoui, The effects of extremely low-frequency magnetic fields on melatonin and cortisol, two marker rhythms of the circadian system, Dialogues Clin. Neurosci. 14(4) (2012), 381–399. PMID:23393415
  * [20] R.H.W. Funk, T. Monsees and N. Özkucur, Electromagnetic effects – From cell biology to medicine, Progress in Histochemistry and Cytochemistry 43 (2009), 177–264. DOI:10.1016/j.proghi.2008.07.001
  * [21] V.N. Binhi and A.V. Savin, Effects of weak magnetic fields on biological systems: physical aspects, Phys. Usp. 46 (2003), 259–291.
DOI:10.1070/PU2003v046n03ABEH001283

  * [22] G.J. Beers, Biological Effects of Weak Electromagnetic Fields From 0 Hz to 200 MHz: A Survey of the Literature With Special Emphasis on Possible Magnetic Resonance Effects, Magn. Reson. Imaging 7 (1989), 309–331. DOI:10.1016/0730-725x(89)90556-0
  * [23] M.A. Persinger, H.W. Ludwig and K.-P. Ossenkopp, Psychophysiological Effects of Extremely Low Frequency Electromagnetic Fields: A Review, Percept. Motor Skills bf 36 (1973), 1131–1159.
DOI:10.2466/pms.1973.36.3c.1131

  * [24] N.G. Ptitsyna, G. Villoresi, L.I. Dorman, N. Iucci and M.I. Tyasto, Natural and man-made low-frequency magnetic fields as a potential health hazard, Phys. Usp. bf 41 (1998), 687–709.
DOI: 10.1070/PU1998v041n07ABEH000419

  * [25] A. Karimi, F. Ghadiri Moghaddam and M. Valipour, Insights in the biology of extremely low-frequency magnetic fields exposure on human health, Mol. Biol. Rep. (2020). DOI:10.1007/s11033-020-05563-8
  * [26] S.J. Palmer, M.J. Rycroft and M. Cermack, Solar and geomagnetic activity, extremely low frequency magnetic and electric fields and human health at the Earth’s surface, Surv. Geophys (2006) 27 (2007), 557–595. DOI:10.1007/s10712-006-9010-7
  * [27] R.K. Adair, Constraints on biological effects of weak extremely-low-frequency electromagnetic fields, Phys. Rev. A 43 (1991), 1039–1048. DOI:10.1103/physreva.43.1039
  * [28] R.O. Becker and G. Selden, The Body Electric: Electromagnetism and the Foundation of Life (Morrow: New York, 1985).
  * [29] F.E. Cole and E.R. Graf, Precambrian ELF and Abiogenesis, in M.A. Persinger (Ed.), ELF and VLF Electromagnetic Field Effects (Springer: Boston, 1974), pp. 243–274. DOI:10.1007/978-1-4684-9004-6_9
  * [30] H. König and F. Ankermüller, Über den Einfluß besonders niederfrequenter elektrischer Vorgänge in der Atmosphäre auf den Menschen, Naturwissenschaften 47 (1960), 486–490. DOI:10.1007/BF00631044
  * [31] C. Price, E. Williams, G. Elhalel and D. Sentman, Natural ELF fields in the atmosphere and in living organisms, Int. J. Biometeorol. (2020) DOI:10.1007/s00484-020-01864-6
  * [32] L.K. Direnfeld, The Genesis of the EEG and its Relation to Electromagnetic Radiation, J. Bioelectricity 2 (1983), 111–121.
DOI:10.3109/15368378309009845

  * [33] N. Cherry, Schumann Resonances, a plausible biophysical mechanism for the human health effects of Solar/Geomagnetic Activity, Natural Hazards 26 (2002), 279–331. DOI:10.1023/A:1015637127504
  * [34] J.G. Roederer, Are magnetic storms hazardous to your health? Eos Trans. Am. Geophys. Union 76N44 (1995), 441–442. DOI:10.1029/95eo00273
  * [35] A. Alabdulgader at al., Long-Term Study of Heart Rate Variability Responses to Changes in the Solar and Geomagnetic Environment, Sci. Rep. 8 (2018), 2663. DOI:10.1038/s41598-018-20932-x
  * [36] I.N. Pigarev, The Visceral Theory of Sleep, Neurosci. Behav. Physi. 44 (2014), 421–434. DOI:10.1007/s11055-014-9928-z
  * [37] M.M. Morchiladze, T.K. Silagadze and Z.K. Silagadze, Visceral theory of sleep and origins of mental disorders, Med. Hypotheses 120 (2018), 22–27. DOI:10.1016/j.mehy.2018.07.023
  * [38] B.T. Arnason, L.A. Hart and C.E. O’Connell-Rodwell, The Properties of Geophysical Fields and Their Effects on Elephants and Other Animals, J. Comp. Psychol. 116 (2002), 123–132. DOI:10.1037//0735-7036.116.2.123
  * [39] A.A. Marino and R.O. Becker, Biological effects of extremely low frequency electric and magnetic fields: A review, Physiol. Chem. Phys. 9 (1977), 131–147.
  * [40] A. Yadollahpour, M. Jalilifar and S. Rashidi, Antimicrobial Effects of Electromagnetic Fields: A Review of Current Techniques and Mechanisms of Action, J. Pure Appl. Microbio. 8 (2014), 4031-4043.
  * [41] A. Fadel, M. Hala, A. Nashwa, M. Hanan and H. Hamida, Control the activity of Rift Valley Fever Virus by Electric Field waves at resonance frequency (In vivo & In vitro) studies, IOSR J. Appl. Phys. 6 (2014), 7–17. DOI:10.9790/4861-06130717
  * [42] M.A. Fadel, S.A. Mohamed, A.M. Abdelbacki and A.H. El-Sharkawy, Inhibition of Salmonella typhi growth using extremely low frequency electromagnetic (ELF-EM) waves at resonance frequency, J. Appl. Microbio. 117 (2014), 358–365. DOI:10.1111/jam.12527
  * [43] L. Zhang, B. Tinsley and L. Zhou, Low Latitude Lightning Activity Responses to Cosmic Ray Forbush Decreases, Geophys. Res. Lett. 47 (2020), e2020GL087024. DOI:10.1029/2020GL087024
  * [44] S. Kumar, D. Singh, R.P. Singh, A.K. Singh and A.K. Kamra, Lightning Discharges, Cosmic Rays and Climate, Surv. Geophys. 39 (2018), 861–899. DOI:10.1007/s10712-018-9469-z
  * [45] M.I. Beloglazov, L.A. Pershakov and G.P. Beloglazova, About the change of atmospherics intensity in ELF-range during Forbush-decreases of galactic cosmic rays, Proc. XXIX Annual Seminar “Physics of Auroral Phenomena”, Apatity, Kola Science Centre, Russian Academy of Science, 2006, pp. 277–280.
  * [46] D. Venkatesan and A.G. Ananth, Forbush decreases in cosmic rays, Bull. Astr. Soc. India 19 (1991), 1-8.
  * [47] J.A. Lockwood, Forbush decreases in the cosmic radiation, Space Sci. Rev. 12 (1971), 658–715. DOI:10.1007/BF00173346
  * [48] H.V. Cane, Coronal Mass Ejections and Forbush Decreases, Space Sci. Rev. 93 (2000), 55–77. DOI:10.1023/A:1026532125747
  * [49] J. Kirkby, Cosmic rays and climate, Surv. Geophys. 28 (2007), 333-375. DOI:10.1007/s10712-008-9030-6
  * [50] L-X. Tian, Y-X. Pan, W. Metzner, J-S. Zhang and B-F. Zhang, Bats Respond to Very Weak Magnetic Fields, PLoS ONE 10 (2015), e0123205. DOI:10.1371/journal.pone.0123205
  * [51] R.A. Holland, K. Thorup, M. Vonhof, W.W. Cochran and M. Wikelski, Bat orientation using Earth’s magnetic field, Nature 444 (2006), 653–702. DOI:10.1038/444702a
  * [52] Y. Wang, Y. Pan, S. Parsons, M.M. Walker and S. Zhang, Bats respond to polarity of a magnetic field, Proc. R. Soc. B 274 (2007), 2901–2905. DOI:10.1098/rspb.2007.0904
  * [53] H. Burda, S. Begall, J. Červený, J. Neef and P. Némec, Extremely low-frequency electromagnetic fields disrupt magnetic alignment of ruminants, Proc. Natl. Acad. Sci. USA 106 (2009), 5708–5713.
DOI:10.1073/pnas.0811194106

  * [54] N.A. Belova and D. Acosta-Avalos, The Effect of Extremely Low Frequency Alternating Magnetic Field on the Behavior of Animals in the Presence of the Geomagnetic Field, J. Biophys. 2015 (2015), 423838. DOI:10.1155/2015/423838
  * [55] R.A. Holland, I. Borissov and B.M. Siemers, A nocturnal mammal, the greater mouse-eared bat, calibrates a magnetic compass by the sun, Proc. Natl. Acad. Sci. USA. 107 (2010), 6941–6945.
DOI::10.1073/pnas.0912477107

  * [56] J. Qu and N.C. Wickramasinghe, The world should establish an early warning system for new viral infectious diseases by space-weather monitoring, DOI:10.1002/mco2.20