# Autonomous Driving using Residual Sensor Fusion and Deep Reinforcement
Learning

Amin Jalal Aghdasian, Amirhossein Heydarian Ardakani, Kianoush Aqabakee,
Farzaneh Abdollahi
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Department of Electrical Engineering
Amirkabir University of Technology (Tehran Polytechnic)
Tehran, Iran

###### Abstract

This paper proposes a novel approach by integrating sensor fusion with deep
reinforcement learning, specifically the Soft Actor-Critic (SAC) algorithm, to
develop an optimal control policy for self-driving cars. Our system employs a
two-branch fusion method for vehicle image and tracking sensor data,
leveraging the strengths of residual structures and identity mapping to
enhance agent training. Through comprehensive comparisons, we demonstrate the
efficacy of information fusion and establish the superiority of our selected
algorithm over alternative approaches. Our work advances the field of
autonomous driving and demonstrates the potential of reinforcement learning in
enabling intelligent vehicle decision-making.

###### Index Terms:

Autonomous Driving, Sensor Fusion, Deep Reinforcement Learning, Soft Actor-
Critic, CARLA

##  I Introduction

In recent years, Autonomous Driving (AD) has become a tangible reality, and as
this technology progresses, it is imperative to ensure the ongoing safety and
reliability of this transportation mode. With the potential to significantly
decrease accidents stemming from human error, AD emerges as a promising
solution for enhancing road safety. According to data from the National
Highway Traffic Safety Administration (NHTSA), a staggering 94% of all traffic
accidents can be attributed to human error or its associated factors. AD
systems come equipped with advanced safety features like collision avoidance
systems and sophisticated sensors capable of preemptively averting accidents.
These safety mechanisms consistently monitor the vehicle surroundings and can
react much more swiftly than a human driver when confronted with unexpected
situations [1].

AD relies on a variety of control strategies, including classic controllers
such as PID [2], MPC [3], LQR [4], and Artificial Intelligence (AI)
controllers such as imitation learning based control [5] and RL based control.
PID control is simple but requires tuning, while MPC handles complex
scenarios. LQR control optimizes a cost function efficiently, but it requires
accurate knowledge of system dynamics and may need to be redesigned for
significant changes [6]. Model-free approaches offer distinct advantages over
model-based methods in various contexts. They shine in complex and dynamic
environments where precise modeling is challenging, providing adaptability to
uncertain or evolving dynamics. Their ease of implementation and reduced need
for manual tuning make them appealing for practical applications. Model-free
reinforcement learning, in particular, stands out for its ability to
continuously learn and adapt, achieving impressive performance across diverse
domains. These methods are well-suited for scenarios where modeling is
impractical or resource-intensive and excel in handling novel or unforeseen
situations without the constraints of fixed assumptions. Imitation learning
utilizes expert demonstrations but may encounter errors when faced with unseen
events or situations. In contrast, RL control offers adaptability, continuous
learning, and the ability to handle diverse conditions [7].

Deep reinforcement learning (DRL) represents a subset of machine learning
methodologies in which an agent acquires decision-making capabilities through
interactions with its environment [8]. Within AD, RL has found applications in
developing driving policies [9]. These algorithms have successfully tackled
complex problems such as Markov Decision Problems (MDPs). However, as
highlighted earlier, it can be challenging for the algorithm to learn from all
possible states and determine the optimal driving path. [10] employs a Deep
Q-Network (DQN) and Long-Short-Term Memory (LSTM) architecture, along with a
novel observation input, to enable autonomous agents to navigate intricate
scenarios autonomously and [7] employs DQN and Deep Deterministic Policy
Gradient (DDPG) algorithms to train autonomous vehicle control models within a
realistic simulator. DQN and DDPG achieve the desired control objectives, with
DDPG demonstrating superior performance since it is suitable for continuous
tasks. In [11], It was developed two approaches: pre-training Soft Actor-
Critic (SAC) with Learning from Demonstrations (LfD) and an online combination
of SAC, LfD, and Learning from Interventions (LfI). Given the recent advances
in DRL algorithms for self-driving cars, enhancing their performance can be
achieved by incorporating more complex methods for data utilization in the
network structures.

Sensor fusion enables AD systems to perceive and understand the surrounding
environment accurately. This technology integrates data from diverse sensors,
aligning them to generate different representations of the vehicle’s
surroundings [12, 13, 14]. This fusion of information not only enhances the
robustness and reliability of the perception system in various driving
scenarios [15, 16] but also helps in mitigating the limitations of individual
sensors, thus improving the overall performance of autonomous vehicles [17,
18]. In the context of RL, sensor fusion becomes even more crucial. RL-based
control systems learn to make decisions by interacting with the environment,
and the quality of these decisions depends on perceived information quality
[19, 7]. With the integrated and comprehensive environmental models provided
by sensor fusion, RL algorithms can better understand the state of the
environment, resulting in more accurate predictions and safer decision-making
[20]. Moreover, incorporating multi-modal fusion into DRL frameworks has
demonstrated significant potential in enhancing decision-making and
reliability in complex environments [21, 22].

This study presents a novel vehicular control methodology that leverages
sensor data fusion and reinforcement learning. Our approach incorporates a
sensor fusion module consisting of a dual branch architecture, effectively
combining information from the vehicle image and tracking sensors. We employed
residual blocks within this module to enhance feature extraction and
representation learning. By integrating the Soft Actor-Critic (SAC) with the
sensor fusion technique, our proposed method unifies the outputs of image
sensors with the vehicle speed and position, resulting in an optimal structure
for generating control commands and faster convergence. To evaluate the
effectiveness of our algorithm, we conducted comprehensive training and
validation procedures using the CARLA simulation platform. This platform
offers a realistic environment to assess our algorithm capabilities and
simulate real-world driving scenarios. Furthermore, this paper includes an
extensive comparative analysis, highlighting the potential advantages of our
approach compared to existing state-of-the-art methods in the field.

The paper is structured into five sections. Section 1 provides an
introduction, offering insights into autonomous driving approaches. Section 2
delves into the background information. Section 3 outlines the proposed
methodology, highlighting the essential aspects. The results and discussion
section, Section 4, presents the research findings. Finally, Section 5, the
conclusion, summarizes the paper and suggests potential future research
directions.

##  II Soft Actor-Critic (SAC) [23]

Soft Actor-Critic (SAC) represents an off-policy DRL algorithm that delivers
efficient learning while preserving the advantages associated with entropy
maximization and stability [23, 24, 25]. SAC bridges between stochastic policy
optimization and DDPG [26] methods. While not a direct successor to Twin
Delayed DDPG [27], it utilizes the clipped double-Q technique and benefits
from target policy smoothing due to its inherent stochasticity. SAC emphasizes
entropy regularization, wherein the policy seeks a balance between expected
return and policy randomness. This approach enhances exploration, aiding later
learning stages, and guards against premature convergence to suboptimal
solutions.

To delve into SAC, we must initially introduce the context of entropy-
regularized reinforcement learning. In this framework, value functions are
defined by slightly different equations. The entropy $H$ is determined as
follows:

$H(P)=E_{x\sim P}[-\log P(x)]$ (1)

Where $x$ is a random variable with a probability mass or density function P.
In entropy-regularized reinforcement learning, the agent receives an
additional reward at each time step proportionate to the policy entropy for
that specific time step. This adjustment transforms the RL goal, which is the
reward maximization problem, into:

$\small\pi^{*}=\arg\max_{\pi}\left[E_{\tau\sim\pi}\left[\sum_{t=0}^{\infty}\gamma^{t}\left(R(s_{t},a_{t},s_{t+1})+\alpha
H(\pi(\cdot\mid s_{t}))\right)\right]\right]$ (2)

In (2), a transition is denoted by $(s_{t},a_{t},r_{t},s_{t+1},a_{t+1})$,
including state, action, and reward. $\alpha$ represents the trade-off
coefficient, $\gamma$ is discount factor and $\pi(\cdot\mid s_{t})$ is policy
in state $s_{t}$ . We consider an infinite-horizon discounted. Under this
assumption, we introduce modified value functions. Specifically, the action-
state value function $Q^{\pi}$ is adapted to incorporate entropy bonuses at
each time step:

$\begin{split}\leavevmode\resizebox{422.77661pt}{}{$Q^{\pi}(s_{t},a_{t})=E\left[R(s_{t},a_{t},s_{t+1})+\gamma(Q^{\pi}(s_{t+1},a_{t+1})+\alpha
H(\pi(\cdot\mid s_{t+1})))\right]$}\\\
\leavevmode\resizebox{422.77661pt}{}{$=E\left[R(s_{t},a_{t},s_{t+1})+\gamma(Q^{\pi}(s_{t+1}a_{t+1})-\alpha\log(\pi(a_{t+1}\mid
s_{t+1})))\right]$}\end{split}$ (3)

This approach considers two parameterized soft Q-functions, $Q(s_{t},a_{t})$,
and a policy represented as $\pi(a_{t}|s_{t})$. Each component relies on
specific parameter sets, denoted as $\phi_{1}$, $\phi_{2}$, and $\theta$,
respectively. For instance, the Q-functions can be modeled as expressive
neural networks. Actions are resampled as $\tilde{a}_{t+1}\sim\pi(\cdot\mid
s_{t+1})$ under the current policy instead of being drawn from the replay
buffer. Therefor, it is essential to distinguish newly sampled actions. The
parameters of the soft Q-function can then undergo training aimed at
minimizing the soft Bellman residual:

$J_{Q_{i}}(\theta)=E_{(s_{t},a_{t})\sim
D}\left[\frac{1}{2}\left(Q_{\phi}(s_{t},a_{t})-\hat{Q}(s_{t},\tilde{a}_{t})\right)^{2}\right]$
(4)

where the target $Q$ is represented as:

$\small\hat{Q}(s_{t},\tilde{a}_{t})=r_{t}+\gamma\left(Q(s_{t+1},\tilde{a}_{t+1})-\alpha\log(\tilde{a}_{t+1}\mid
s_{t+1})\right)$ (5)

Equation (4) can be optimized using gradient descent:

$\small\hat{\nabla}_{\phi}J_{Q}(\phi)=\nabla_{\phi}Q_{\phi}(s_{t},a_{t})\left(Q_{\phi}(s_{t},a_{t})-\hat{Q}(s_{t},a_{t})\right)$
(6)

The policy parameters can be acquired by minimizing the Kullback-Leibler
divergence [23]. By reparameterizing the expectation, we can now define the
policy objective as:

$\small J_{\pi}(\theta)=E_{s_{t}\sim
D}\left[\log\pi_{\theta}(\tilde{a}_{\theta}(s_{t})\mid
s_{t})-Q_{\phi}(s_{t},\tilde{a}_{\theta}(s_{t}))\right]$ (7)

Then, we will approximate the gradient of (8) as follows:

$\small\hat{\nabla}_{\theta}J_{\pi}(\theta)=\nabla_{\theta}\log(\pi_{\theta}(\tilde{a}_{t}\mid
s_{t}))\\\ +\left(\nabla_{\tilde{a}_{t}}\log(\pi_{\theta}(\tilde{a}_{t}\mid
s_{t}))-\nabla_{\tilde{a}_{t}}Q(s_{t},\tilde{a}_{t})\right)\nabla_{\theta}\tilde{a}_{\theta}(s_{t})$
(8)

Through iterative interactions and data collection, the Q-function and policy
networks will reach a state of convergence, which enables the agent to obtain
the maximum reward in each episode.

##  III Methodology

###  III-A Problem Formulation

The autonomous driving problem can be delineated into several primary
objectives. These objectives include arriving at the designated destination
while ensuring collision avoidance and navigation within the road lane with
minimal error. Each state $s_{t}$ encompasses the data acquired from the car
front camera and the tracking sensor. These inputs are combined in a
structured manner, as described in the subsequent section, to generate the
desired control command. Car navigation relies on two controls: longitudinal
and lateral. Consequently, the DRL agent produces two control commands:
acceleration and steering angle. The car acceleration range is [0,1] and the
steering range is [-1,1]. Furthermore, the episode return $R_{t}$ for the
problem is established by (9), ensuring the car navigates between the
designated boundaries towards the specified destination.

$R_{t}=\sum_{t}\left|v_{t}cos(\phi_{t})\right|-\left|v_{t}sin(\phi_{t})\right|-\left|v_{t}\right|\left|d_{t}\right|$
(9)

Where the angle $\phi_{t}$ represents car orientation relative to the road,
the cosine term denotes the speed aligned with the road length, and the sine
term represents the speed perpendicular to the road. By formulating the return
function as (9), the RL agent maximizes car velocity $v_{t}$ by increasing the
first term and decreasing the second one. Furthermore, the third term aims to
minimize vehicle deviation from the center of the lane, as indicated by the
distance $d_{t}$. Fig. 1 visually represents the reward parameters of the
vehicle on the road.

Figure 1: Reward function parameters based on vehicle orientation.

The episode terminates in the event of a collision or the car deviating from
its designated lane, resulting in a reward of $-200$. Conversely, upon
reaching the goal destination, a reward of $100$ is recorded for the episode.
Consequently, the agent aims to navigate to the destination without collisions
to maximize the overall return.

###  III-B Sensor Data Fusion

The fusion module within our RL agent enhances its decision-making
capabilities and overall performance by integrating various data sources and
unifying information. This module plays a vital role in both the actor and
critic components, effectively improving car navigation by identifying the
optimal approach to information fusion. A visual representation of the fusion
module structure is presented in Fig. 2.

Figure 2: Sensor fusion module.

The fusion module within the system employs a dual-branch approach for
integrating the information derived from the image and the tracking sensors.
The image branch encompasses multiple residual blocks and a fully connected
layer. The residual block extracts image features using multiple convolutional
layers $\mathcal{F}_{W_{i}}$ and a shortcut path (10). In order to ensure
compatibility of dimensions, a projection $W_{s}$ is additionally employed
using a convolution layer in the direction of the shortcut path. Incorporating
skip connections within these residual blocks facilitates the direct
propagation of information, thereby mitigating the vanishing gradient problem
[28]. Moreover, a fully connected layer transforms the features extracted from
the image into the desired dimensional representation.

$features=\mathcal{F}_{W_{i}}(image)+W_{s}image$ (10)

In the other branch, the data obtained from the tracking sensor undergoes
resizing via a fully connected layer. The fused information is transmitted
throughout the remaining network structure by establishing a connection
between these two branches.

###  III-C Proposed Method Architecture

Our proposed method represents a combination of sensor fusion and SAC
architecture for controlling autonomous vehicles. SAC is chosen in this
research due to its vital attributes in stability, exploration, and robustness
[23]. The fusion structure proposed in the previous section is implemented at
the outset of each function approximator (Q-network 1, Q-network 2, and the
actor). This involves concatenating camera and tracking sensor data features,
utilizing them as input to the model while carefully considering the
importance of each data component in each function approximator to achieve
optimal results.

Figure 3: Proposed DRL control scheme

As illustrated in Fig. 3, the actor model is trained based on the approximated
action-value function, and after a sufficient number of training episodes, we
can attain the desired level of performance. The reason for merging multiple
inputs with the proposed fusion structure is to extract features at the
earlier layers and combine these representations at the subsequent layers to
create a more comprehensive understanding of the environment. Having a greater
variety of sensor types implies increased data dimensions collected from the
environment. Higher dimensionality provides a richer source of information for
making ultimate decisions. Nevertheless, selecting the most concise data
representation that exhibits the highest correlation with the desired output
remains critical. We used data concatenation and mapped it to a reduced-
dimensional space in subsequent layers with mentioned specifications. However,
this approach presents some challenges. We encounter issues such as vanishing
gradients and slow convergence during feature extraction and selection
processes. To navigate these challenges effectively, we added the residual
structures and SAC algorithm. The pseudocode of the proposed approach is shown
in the Alg. 1.

1 Input: Initialize networks parameters $\bm{\theta}$, $\bm{\phi_{1}}$,
$\bm{\phi_{2}}$ and replay buffer $\mathcal{D}$

2Initialize target networks parameters $\bm{\phi_{\text{target
1}}}\leftarrow\bm{\phi_{1}}$, $\bm{\phi_{\text{target
2}}}\leftarrow\bm{\phi_{2}}$

3for  _each episode_ do

4 for  _each CARLA step_ do

5 $a_{t}\leftarrow\pi_{\theta}(a_{t}|s_{t})$

6 $s_{t+1}\leftarrow p(s_{t+1}|s_{t},a_{t})$

7 $D\leftarrow D\cup\\{(s_{t},a_{t},r(s_{t},a_{t}),s_{t+1})\\}$

8 Reset environment$\leftarrow$ if ${s_{t+1}}$ is terminal state

9 end for

10 for  _each gradient step_ do

11 Randomly sample batch from transitions $\mathcal{D}$

12 Compute targets for Q-functions:

13
$y(r,s^{\prime})=r+\gamma\min_{i=1,2}\left[Q_{\phi_{\text{target},i}}(s^{\prime},{\tilde{a}}^{\prime})-\alpha\log(\pi_{\theta}({\tilde{a}}^{\prime}|s^{{}^{\prime}}))\right]$

14 Update Q-functions by one step of gradient descent for $i=1,2$:

15
$\nabla_{\phi_{i}}\frac{1}{|B|}\sum_{\left(s,a,r,s{}^{\prime}\right)\epsilon
B}\left(Q_{\phi_{i}}(s,a)-y(r,s^{\prime})\right)^{2}$

16 Update policy by one step of gradient ascent:

17 $\nabla_{\theta}\frac{1}{|B|}\sum_{s\epsilon
B}(\min_{i=1,2}Q_{\phi_{i}}(s,\tilde{a}_{\theta}(s))$

18
$\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad-\alpha\log(\pi_{\theta}(\tilde{a}_{\theta}(s)\mid
s)))$

19 Update target networks:

20 $\phi_{target,i}\leftarrow\rho\phi_{target,i}+\left(1-\rho\right)\phi_{i}\
for\ i=1,2$

21

22 end for

23

24 end for

Algorithm 1 Pseudocode of the proposed approach

##  IV Results and Discussion

We selected CARLA [29], an open-source simulator designed for AD research, for
this study. CARLA provides a Python library that facilitates interaction with
the environment, enabling dynamic modifications of the spawning and removal of
cars and objects within the simulated environment. To assess the efficacy of
the proposed method against alternative algorithms, SAC algorithm was
subjected to three distinct training scenarios in CARLA. These scenarios
encompassed using the sensor fusion block, solely relying on camera imagery,
and tracking sensor data under similar situations. Furthermore, we employed
the DDPG algorithm used in previous studies, following a similar training
procedure to establish the system superiority compared to previous works [7].

###  IV-A Training

We designed three different training configurations to assess the impact of
sensor fusion on the RL agent performance. In the first configuration, we
utilized sensor fusion, combining both image and tracking sensor data, to
inform the agent decision-making process. In the second configuration, we
isolated the use of only image data, while in the third configuration, we
relied solely on tracking sensor data for decision-making. These
configurations were designed to highlight the versatility and adaptability of
our algorithm under varying data input scenarios. The same training process
has been repeated for the DDPG algorithm to compare the performance of the
proposed algorithm with previous works [7].

To create episodic tasks for training, we randomly choose two points on the
town roadway and create a path from the start point to the end point. Fig. 4
shows an example of a random path generated on the map. By training on
different town maps and weather conditions, our DRL agent will become more
general and robust, capable of handling various driving tasks and
uncertainties.

Figure 4: Random path generation in CARLA simulator.

The camera image is mapped to 100 features using the proposed residual
structure. Then, a fully connected layer maps 16 features from the tracking
sensor to be concatenated with image features. In Q-networks, we have 116
input features for our agent; by adding two actions, throttle and steering
wheel angle, 118 features will be delivered. We trained the agent for 5000
episodes with Adam optimizer and a learning rate value of 0.0001. Also, the
buffer size is considered 1e6 to cover a wide range of explorations. The
average reward is calculated in each episode, as mentioned before. The
training and testing of the proposed approach were performed on a hardware
with a Core i9 5 GHz CPU, Nvidia Geforce RTX 3080 Ti GPU with 32 GB of RAM.

Figure 5: Smoothed plot of average reward per episode for six training
configurations.

As shown in Fig. 5, the proposed fusion structure leads to faster convergence
and higher average reward with minor deviation in both SAC and DDPG algorithms
compared to using only one sensor data. Also, SAC can unleash the potential of
the proposed structure better than DDPG due to the use of the entropy function
to enhance optimization and better exploration strategy.

###  IV-B Evaluation

The validation for proposed approaches employs the Root Mean Square Error
(RMSE) metric, designed to evaluate the error between each algorithm and the
ground truth provided by the CARLA Simulator. This approach ensures that the
performance analysis of algorithms adheres to a uniform criterion. In order to
compare the trained agents, 25 different routes are designated on the map. The
agents are navigated along these routes, facilitating the calculation of the
RMSE between the actual route and an ideal route derived from the
interpolation of the waypoints. Table I shows the RMSE yielded when the agent
navigates the route for 25 iterations. Additionally, the table reports the
maximum and minimum error recorded along the route, as well as the error
standard deviation (std) of the agent driving from the initial to the terminal
point.

TABLE I: calculated root mean square error in evaluation phase Method | Mean | Min | Max | std
---|---|---|---|---
SAC Fusion-based | 0.039 | 0.006 | 0.079 | 0.011
SAC Image-based | 0.591 | 0.189 | 1.073 | 0.141
SAC Sensor-based | 0.526 | 0.175 | 0.919 | 0.12
DDPG Fusion-based | 0.119 | 0.08 | 0.158 | 0.012
DDPG Image-based | 0.752 | 0.24 | 1.379 | 0.173
DDPG Sensor-based | 0.678 | 0.318 | 1.055 | 0.135

Our evaluation results demonstrate the significant improvement achieved
through sensor fusion. We observed a remarkable improvement in the overall
error when employing sensor fusion, indicating that the agent made more
informed and successful decisions. On the other hand, employing solely the
image as the agent input shows superior performance compared to relying on the
tracking sensor. This disparity arises from the agent ability to determine
road segments and anticipate forthcoming actions, particularly during turning
maneuvers. Conversely, the information derived from the tracking sensor only
provides the agent with the present state of the vehicle. A comparison between
SAC and DDPG also shows the relatively lower error shown by SAC agent. This
advantage comes from the effective employment of a stochastic policy, careful
consideration of entropy function, and an improved exploration process, all
aimed at attaining the optimal policy. These findings underscore the crucial
role of sensor fusion in enhancing the overall performance and robustness of
our RL algorithm, making it a valuable approach for a wide range of
applications where both image and tracking sensor data are available.

##  V Conclusion

In this study, we have introduced an intelligent approach for car navigation
by integrating sensor fusion into deep reinforcement learning. To enhance the
agent learning performance and ascertain an optimal information integration
method, we developed a fusion block that merges data acquired from the vehicle
camera and tracking sensor. Our proposed methodology was evaluated within the
CARLA simulation platform, which provides realistic simulations and offers
high diversity and repeatability in autonomous driving.

To substantiate the efficacy of our method in contrast to prior researches, we
conducted a comprehensive evaluation across three distinct scenarios,
encompassing tracking sensor data, image data, and the fusion of both. The
results obtained from these evaluations underscore the significant impact of
employing sensor fusion along with SAC algorithm to enhance the performance of
DRL algorithms in car navigation. Future works for this research include the
real-world implementation of the algorithm, addressing uncertainties inherent
in practical settings, and increasing the variety of sensors employed in the
navigation system. These explorations hold the promise of advancing the state-
of-the-art in autonomous vehicle navigation.

## References

  * [1] O. Zheng, M. Abdel-Aty, Z. Wang, S. Ding, D. Wang, and Y. Huang, “Avoid: Autonomous vehicle operation incident dataset across the globe,” arXiv preprint arXiv:2303.12889, 2023.
  * [2] W. Farag, “Complex trajectory tracking using pid control for autonomous driving,” International Journal of Intelligent Transportation Systems Research, vol. 18, no. 2, pp. 356–366, 2020.
  * [3] K. Honda, H. Okuda, T. Suzuki, and A. Ito, “Mpc builder for autonomous drive: Automatic generation of mpcs for motion planning and control,” in 2023 IEEE Intelligent Vehicles Symposium (IV), pp. 1–8, IEEE, 2023.
  * [4] J. Chen, W. Zhan, and M. Tomizuka, “Autonomous driving motion planning with constrained iterative lqr,” IEEE Transactions on Intelligent Vehicles, vol. 4, no. 2, pp. 244–254, 2019.
  * [5] E. Bronstein, M. Palatucci, D. Notz, B. White, A. Kuefler, Y. Lu, S. Paul, P. Nikdel, P. Mougin, H. Chen, et al., “Hierarchical model-based imitation learning for planning in autonomous driving,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 8652–8659, IEEE, 2022.
  * [6] J. H. Yang, W. Y. Choi, S.-H. Lee, and C. C. Chung, “Autonomous lane keeping control system based on road lane model using deep convolutional neural networks,” in 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pp. 3393–3398, IEEE, 2019.
  * [7] Ó. Pérez-Gil, R. Barea, E. López-Guillén, L. M. Bergasa, C. Gomez-Huelamo, R. Gutiérrez, and A. Diaz-Diaz, “Deep reinforcement learning based control for autonomous vehicles in carla,” Multimedia Tools and Applications, vol. 81, no. 3, pp. 3553–3576, 2022.
  * [8] A. K. Shakya, G. Pillai, and S. Chakrabarty, “Reinforcement learning algorithms: A brief survey,” Expert Systems with Applications, vol. 231, p. 120495, 2023.
  * [9] K. B. Naveed, Z. Qiao, and J. M. Dolan, “Trajectory planning for autonomous vehicles using hierarchical reinforcement learning,” in 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pp. 601–606, IEEE, 2021.
  * [10] M. Ahmed, C. P. Lim, and S. Nahavandi, “A deep q-network reinforcement learning-based model for autonomous driving,” in 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 739–744, IEEE, 2021.
  * [11] M. Savari and Y. Choe, “Online virtual training in soft actor-critic for autonomous driving,” in 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1–8, IEEE, 2021.
  * [12] X. Zhang, Y. Gong, J. Lu, J. Wu, Z. Li, D. Jin, and J. Li, “Multi-modal fusion technology based on vehicle information: A survey,” IEEE Transactions on Intelligent Vehicles, vol. 8, no. 6, pp. 3605 – 3619, 2023.
  * [13] J. Du, X. Huang, M. Xing, and T. Zhang, “Improved 3d semantic segmentation model based on rgb image and lidar point cloud fusion for automantic driving,” International Journal of Automotive Technology, vol. 24, no. 3, pp. 787–797, 2023.
  * [14] A. T. Candan and H. Kalkan, “U-net-based rgb and lidar image fusion for road segmentation,” Signal, Image and Video Processing, vol. 17, pp. 1–7, 2023\.
  * [15] S. Gite, K. Kotecha, and G. Ghinea, “Context–aware assistive driving: an overview of techniques for mitigating the risks of driver in real-time driving environment,” International Journal of Pervasive Computing and Communications, vol. 19, no. 3, pp. 325–342, 2023.
  * [16] W. Hou, W. Li, and P. Li, “Fault diagnosis of the autonomous driving perception system based on information fusion,” Sensors, vol. 23, no. 11, p. 5110, 2023.
  * [17] H. Liu, C. Wu, and H. Wang, “Real time object detection using lidar and camera fusion for autonomous driving,” Scientific Reports, vol. 13, no. 1, p. 8056, 2023.
  * [18] Q. Zhang, J. Liu, and X. Jiang, “Lane detection algorithm in curves based on multi-sensor fusion,” Sensors, vol. 23, no. 12, p. 5751, 2023.
  * [19] N. Chukamphaeng, K. Pasupa, M. Antenreiter, and P. Auer, “Learning to drive with deep reinforcement learning,” in 2021 13th International Conference on Knowledge and Smart Technology (KST), pp. 147–152, IEEE, 2021\.
  * [20] J. Dong, S. Chen, Y. Li, R. Du, A. Steinfeld, and S. Labi, “Space-weighted information fusion using deep reinforcement learning: The context of tactical control of lane-changing autonomous vehicles and connectivity range assessment,” Transportation Research Part C: Emerging Technologies, vol. 128, p. 103192, 2021.
  * [21] Y. H. Khalil and H. T. Mouftah, “Exploiting multi-modal fusion for urban autonomous driving using latent deep reinforcement learning,” IEEE Transactions on Vehicular Technology, vol. 72, no. 3, pp. 2921–2935, 2022.
  * [22] W. Liu, Z. Xiang, H. Fang, K. Huo, and Z. Wang, “A multi-task fusion strategy-based decision-making and planning method for autonomous driving vehicles,” Sensors, vol. 23, no. 16, p. 7021, 2023.
  * [23] T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,” in International conference on machine learning, pp. 1861–1870, PMLR, 2018\.
  * [24] T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, V. Kumar, H. Zhu, A. Gupta, P. Abbeel, et al., “Soft actor-critic algorithms and applications,” arXiv preprint arXiv:1812.05905, 2018.
  * [25] T. Haarnoja, S. Ha, A. Zhou, J. Tan, G. Tucker, and S. Levine, “Learning to walk via deep reinforcement learning,” arXiv preprint arXiv:1812.11103, 2018.
  * [26] D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. Riedmiller, “Deterministic policy gradient algorithms,” in International conference on machine learning, pp. 387–395, Pmlr, 2014.
  * [27] S. Fujimoto, H. Hoof, and D. Meger, “Addressing function approximation error in actor-critic methods,” in International conference on machine learning, pp. 1587–1596, PMLR, 2018.
  * [28] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016.
  * [29] A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, “Carla: An open urban driving simulator,” in Conference on robot learning, pp. 1–16, PMLR, 2017.


# TransformNet: Self-supervised representation learning through predicting
geometric transformations

Sayed Hashim∗ Muhammad Ali
Mohamed Bin Zayed University of Artificial Intelligence, UAE
{sayed.hashim<EMAIL_ADDRESS>

First authors

###### Abstract

Deep neural networks need a big amount of training data, while in the real
world there is a scarcity of data available for training purposes. To resolve
this issue unsupervised methods are used for training with limited data. In
this report, we describe the unsupervised semantic feature learning approach
for recognition of the geometric transformation applied to the input data. The
basic concept of our approach is that if someone is unaware of the objects in
the images, he/she would not be able to quantitatively predict the geometric
transformation that was applied to them. This self supervised scheme is based
on pretext task and the downstream task. The pretext classification task to
quantify the geometric transformations should force the CNN to learn high-
level salient features of objects useful for image classification. In our
baseline model, we define image rotations by multiples of 90 degrees. The CNN
trained on this pretext task will be used for the classification of images in
the CIFAR-10 dataset as a downstream task. we run the baseline method using
various models, including ResNet, DenseNet, VGG-16, and NIN with a varied
number of rotations in feature extracting and fine-tuning settings. In
extension of this baseline model we experiment with transformations other than
rotation in pretext task. We compare performance of selected models in various
settings with different transformations applied to images,various data
augmentation techniques as well as using different optimizers. This series of
different type of experiments will help us demonstrate the recognition
accuracy of our self-supervised model when applied to a downstream task of
classification.

Code: https://github.com/hashimsayed0/TransformNet

##  1 Introduction

Deep neural networks, especially convolutional neural networks (CNNs) lead to
big accomplishments in the computer vision field. If manually labelled data
like ImageNet is available, we can train CNNs models by using back
propagation. As a result, CNN’s give a state-of-the-art performance on many
tasks like image classification and object detection [10]. For real-world
problems, a very limited amount of data is available for training purposes, so
expensive efforts concerning time and resources are required to provide these
labelled training data. This problem leads to a big increase in the interest
of researchers to learn deep feature representations in an unsupervised
fashion to solve emerging visual understanding tasks with insufficient
labelled data [18].

Self-supervised learning technique can be scaled to real-world problems and it
provides a reliable methodology for learning visual representations [10]. In
the first step the underlying structural information of data as well as
pretext tasks is utilized for feature extraction which can be used to train
neural networks[1]. Pretext tasks helps in creating supervisory signals
without much expensive efforts. In the second step as downstream tasks
existing data sets of images can be used for model training[3]. We believe
that in order for a model to successfully predict the rotations, it needs to
understand the true concept of object by its various properties including its
position, pose and location. Our work describes a self-supervised learning
approach and we suggest image representations learning by training CNN’s to
recognize the geometric transformation when applied to an input image. We
propose to learn high level semantic image features by training CNNs to
recognize the geometric transformations applied to them. As the pretext task,
we train the network to predict the degrees of rotation and shear applied to
the unlabelled input images.

The pretext classification task to quantify the geometric transformations
applied on images should force CNN to learn high-level salient features of
objects useful for image classification [10]. These features may include
location of objects in the image, type of objects and their orientation in the
image. After learning the orientation in the image, model relates this
orientation to the actual transformation applied on the image. So this
technique proves to be simple yet very power technique which provides
supervisory signal in SSL-domain.

We believe that working in this domain which purely focuses on geometric
transformation based self-supervised techniques is necessary due to 2 reasons.
(1) Geometric transformations have proved to be simple yet powerful
supervisory signals in unsupervised representation learning. (2) Many
successful works have used geometric transformations from different paradigms
such as autoencoding and classification. To enable a detailed and in-depth
understanding of SSL with geometric transformations we can experiment with
various state of the art models using various tricks like data augmentation to
improve variability of data.

For these reasons, we start with baseline model. In the baseline model, we
define transformations as the image rotations by multiples of 90 degrees. The
CNN trained on this pretext task is used for object recognition in images in
the CIFAR-10 dataset as a downstream task. This helps in measuring the
accuracy of our self-supervised model when applied to a particular task of
classification. After successfully implementing the baseline models where
pretext task is comprised of rotation prediction by network we extend the
model in two domains.

In one domain we train the models using various levels of augmentations. These
augmentations are applied on top of rotations to improve the generalization
ability of the model along with inherent advantage of enhancement in
diversified data. Performance of model learning in pretext task using this
augmented data is tested in downstream tasks in unfrozen as well as frozen
settings to get a fair comparison.

Second domain is the architectural selection and hyper parameters tuning: In
this domain we test different models to compare the results quantitatively and
qualitatively. These models include VGG16, NIN, DensNet, ResNetV1 and
ResnetV2.Comparison of these models with different depth level of convolution
blocks i.e 1,2,3,4 and 5 give us better understanding of the performance. In
order to thoroughly evaluate and analayse the performance of our model we
conduct series of experiments to predict the transformations applied to image.
This series of different type of experiments will help us demonstrate the
recognition accuracy of our self-supervised model when applied to a downstream
task of classification.

We organize the rest of the paper as follows. Section 2 shows work
distribution and contribution. In section 3 we review the related work along
with a brief comparison of our work. In section 4 we briefly discuss data
exploration and prepossessing. We present our suggested model in section 5
describing in detail the methodology adopted. we describe the experiments in
section 6 using different models with reference to data augmentation
techniques,network depth level,hyper-parameters tuning and optimizer
selection. Section 7 contains information about results and data sets. In this
section we present quantitative and qualitative outcomes for better analysis
and conclusion. Section 8 concludes with brief summary of our work.

##  2 Related works

###  2.1 Self supervised learning

In some methods of self supervisory learning, the supervisory signal is
generated from the data itself e.g in a few methods part of data is recovered
in pretext task and the network learns during the process [9]. Examples of a
few such tasks include image completion [16], image colourization [19] and
channel prediction [20]. Other types of the technique include those which
takes underlying concept information of the image to ultimately match the
constraints. These techniques include counting [15], rotation [10] and
instance discrimination [8].

One of the works propose an idea to train neural networks by using the
prediction of the relative positions of two randomly sampled patches [7]. In
one other experiment, the concept of the jigsaw puzzle solution by the network
is used for training a convolutional neural network [14]. Another work
introduces auto-encoding transformations to exploit the representation of
features after different transformations [18]. To sum it up, these methods
help in training the network using self-supervised objectives and thus reduces
the dependency on manually labelled data.

Many classical hand-crafted features including SIFT [13] and RIFT [12] for
computer vision show insensitivity to some types of transformations. Data
augmentation can be used for transformation invariance. We use the data
augmentations in the final phase of our project, where we use two types of
transformation sets ;10 transformation set and 5 transformation set in
unsupervised task. These data augmentations not only help increase the data
but also it increase the generalizability of the model .This further helped us
obtain rotational invariance against transformations.

###  2.2 Feature Decoupling

Some authors proposed the idea of using rotation feature decoupling in self-
supervised learning [9]. In this method, instead of working on a pretext task,
the main theme is to focus on the attributes of the learned representations by
the network as well as their generalization ability. This helps to improve
performance in those tasks where we need rotation invariance. The novel scheme
is presented which comprised of two tasks, rotation prediction and individual
instance discrimination tasks. After predicting image rotation tasks on our
baseline model in a midterm, we utilize different types of augmentations in
the final phase of our project.

###  2.3 Auto-Encoders

The basic methodology relies on a assumption that in order to rebuild the data
feature representation must have enough information. Huge variety of encoders
have been proposed including, the variations auto-encoder which explicitly
introduces probabilistic assumption about distribution of features extracted
from data. Denoising auto-encoder [27] initiates learning more robust
representation by reconstructing original inputs from noise-corrupted inputs.
In this technique a novel methodology of unsupervised representation learning
using Auto-Encoding Transformation (AET) is used as against the Auto-Encoding
Data (AED)approach. In this technique after application of random
transformation, AET tries to predict it using encoded features as accurately
as possible at the output end. The central idea in this scheme is that as long
as the unsupervised features can encode the fundamental information about
original and transformed images, network can predict the transformation.

##  3 Data exploration and preprocessing

Dataset: We used CIFAR-10 dataset [11] for our pretext rotation prediction
task and downstream object recognition task. The CIFAR-10 dataset contains
60,000 32x32 colour pictures belonging to ten classes, with 6000 pictures in
one class. It consists of 50,000 training images and 10,000 test images. The
test batch comprises of 1000 images from each class. The training batches hold
the rest of the images in random order. The training batches contain 5000
images from each class.

Preprocessing: To obtain the data set for pretext task, we constructed data
sets with relevance to our experiments. In phase I of our project, we applied
rotation to each image in CIFAR-10 dataset and obtained 120,000, 240,000 and
480,000 images in the case of 2, 4 and 8 rotations respectively. In phase II,
we applied geometric transformations to each image, resulting in 600,000 and
300,000 images, for 10 and 5 transformations respectively.

In phase II, we performed data preprocessing in 2 ways. (1) In the first
method, we applied each transformation separately to each image, i.e., each
transformed image contained only one of the four transformations, rotation,
shearing, scaling and translation. (2) In the second method, we applied
transformations randomly to each image, resulting in 5 or 10 transformed
copies of each image. This means multiple transformations mentioned above
could be applied on the same image. We found that first method worked better,
so we went ahead with it.

##  4 Methodology

###  4.1 Overview

This work aims to learn meaningful features based on CNN in an unsupervised
way. For that purpose, a CNN model $F(.|)$ is trained to quantitatively
predict the geometric transformation that are applied to images that the
network receives as input.

Precisely, consider a set of $K$ geometric transformations $T$ defined as
$T=\\{t(.|y)\\}_{y=1}^{K}$ where $t(.|y)$ is a geometric transformation
applied to an image and results in a transformed image $X^{y}$ given as
$X^{y}=t(X|y)$. The CNN learns a function $F(.|y)$ that can predict the
geometric transformation $t(.|y)$ that is applied to the input.

Here, provided a set of N images, the training objective that the CNN learns
to solve is:

$\min_{\theta}\frac{1}{N}\sum_{i=1}^{N}loss\left(X_{i},\theta\right)$ (1)

whereby $loss$ is defined as:

$loss\left(X_{i},\theta\right)=-\frac{1}{K}\sum_{y=1}^{K}\log\left(F^{y}\left(t\left(X_{i}\mid
y\right)\mid\theta\right)\right)$ (2)

###  4.2 Rotation

The geometric transformations $T$ must force the CNN to learn semantic
features. In the process of learning to predict the geometric transformations
applied to the image, the CNN would also have to learn the features in the
image. Therefore, to get better at this task, the CNN is forced to learn the
features of the images. Precisely, to be able to predict the rotation applied
to an image, the CNN model should learn to localize the subjects and objects
in the image, identify their orientation, direction and type of object, and
then connect the orientation of the object with the main orientation shown by
different types of object within the images [10].

One such geometric transformation is rotation [10]. To predict the amount of
rotation that is applied to an image, the CNN would have to learn the semantic
features of this image. In this way, rotation provides a powerful supervisory
signal. In this work, we choose $T$ as a set of rotations applied to an image,
as shown in Figure 1. We experimented with different set of rotations as
described in the experiments section, and found that $T=\\{0$$,180$$\\}$
provides best results. In our upcoming work, we will experiment with more
geometric transformations.

Figure 1: Figure depicting the proposed self supervised task for meaningful
learning of features [10]. Given two plausible geometric transformations,
rotations of zero and 180 degrees, a CNN model F(.) is trained to identify the
rotation applied to the image that it receives as input. $F^{y}(X^{y})$ is the
prediction by model $F(.)$ that states the probability of rotation $y$ when it
receives as input an image which was transformed by the rotation $y$

[width=1]images/illustration.png

A benefit of applying rotations by multiples of ninety degrees instead of
other geometric transformations, is that rotations can be applied using flip
as well as transpose operations that don’t expose any easily identifiable low-
level visual artifacts that can help the CNN learn easy features that have no
practical value for the visual perception tasks [10]. For example, choosing
scale as transformation would provide the CNN with clues about the scale
factor without having to learn any semantic features in the image.

Moreover, images captured by humans usually are in up-standing position, which
give no ambiguity for the rotation task except for round objects.

The above mentioned reasons show that rotation is a well defined simple yet
powerful task for self supervision.

###  4.3 Other affine transformations

In the second part of the project, we explored more affine transformations,
specifically, rotation, translation, shearing and scaling. We applied 5 and 10
transformations in different settings to each image. Specifically, we
improvised the pretext task as the prediction of all the transformations
applied to the input image, as shown in Figure 2

Figure 2: Figure depicting the proposed self supervised task for meaningful
learning of features. After applying various transformations to input image
$X$ and obtaining $t(X)$, a CNN model is trained to identify the
transformations applied to the image

[width=1]images/final_arch.png

####  4.3.1 Translation

It involves shifting images left, right, up, or down. Learning to recognise
this transformation can force the CNN to not learn positional bias in the data
[17]. After the original picture is translated in a direction, the remaining
area can be filled with either a constant pixel value such as 0s or 255s, or
it can also be filled with random or Gaussian noise. This padding will help in
preserving the spatial dimensions of the image after the augmentation. In our
method, we fill the remaining space with 0s (black colour).

####  4.3.2 Shearing

Shearing is linear transformation that displaces each point in a fixed
orientation, by an amount proportional to its signed distance from the line
which is parallel to that orientation and passes through the origin [5]. In
the process of learning to predict the shearing applied to images, the network
is forced to learn angles of relevant features and thus represent the images
better. In our experiments, we used shearing factor of 0.3 and -0.3.

####  4.3.3 Scaling

Scaling is a linear transformation that increases (enlarges) or diminishes
(shrinks) objects by a scale factor which is the same in all orientations [4].
To be able to recognise if an image is scaled, the network would have to learn
the size of relevant features in the image. If the value of scaling factor is
set to less than 1, we decrease the size of the objects in the image [1]. If
the value scaling factor is set to more than 1, we increase size of the object
in the image. We used scaling factors of 0.7 and 1.3.

0.195 [width=0.9height=0.9]images/transformations/original.png 0.195
[width=0.9height=0.9]images/transformations/rotation.png 0.195
[width=0.9height=0.9]images/transformations/shear.png 0.195
[width=0.9height=0.9]images/transformations/scale.png 0.195
[width=0.9height=0.9]images/transformations/translation.png

Figure 3: Original Figure 4: Rotation Figure 5: Shearing Figure 6: Scaling
Figure 7: Translation Figure 8: Illustration of various transformations

###  4.4 Data augmentation techniques

Here, we will describe some of the data augmentation techniques that we used
in an experiment in the second phase of our project.

Horizontal flipping: It involves flipping an image about the horizontal axis
and is a lot more common than flipping the vertical axis [17]. It is easy to
implement and has shown to be useful on datasets such as CIFAR-10 and
ImageNet.

Random brightness: The RGB values are easily manipulated using trivial matrix
operations in order to increase or reduce the brightness of the image. In our
method for data augmentation, brightness of each image is changed to a random
value within a specified range.

Random zooming: In this method for data augmentation, images are randomly
zoomed in or out in a specified range [2]. When zoomed in, new pixel values
are either added around the image or pixel values are interpolated
respectively [6].

##  5 Experiments

###  5.1 Phase I: Rotation prediction

In the experiments in the first phase of our project, we implemented multiple
models and trained them on the pretext task of rotation prediction . We used
RMSprop with a batch size of 128, rho of 0.9, and learning rate (lr) of 0.001.
We dropped lr by a factor of five after thirty, sixty and eighty epochs. We
trained the models for 100 epochs and fed it all four rotated copies of each
image [3].

We conducted extensive experiments to test the performance of various models
with varied number of rotations. To investigate how much the depth of the
models affect the standard of features that are learned, we trained models
with different number of convolutional blocks in the architectures. On top of
that we also assess the models’ performance when the initial blocks are frozen
and not trained in the downstream task versus when they are not frozen and are
trained to fine tune in the downstream task. While the former measures the
models’ feature extracting accuracy, the latter measures the fine tuning
accuracy.

We trained classifiers above the feature maps produced by each convolutional
block of each model for 50 epochs each for frozen and unfrozen setting. These
classifiers are trained in a supervised manner on CIFAR-10 object recognition
task. They comprise of 3 fully connected layers; 2 hidden layers comprise of
200 feature channels each and are followed by batch-normalisation and relu
units.

###  5.2 Phase II

In the second phase of our project, we conducted three sets of experiments to
test the effect of more transformations, data augmentation and optimizers
respectively.

####  5.2.1 Affine transformation prediction

In this set of experiments, we implemented the reformulated problem of affine
transformation prediction. We trained the models with 5 and 10
transformations. All the other settings were same as those in the first phase
of our project.

####  5.2.2 Effect of data augmentation

Another experiment we conducted was to identify the effect of data
augmentation in training in the downstream task of object recognition on
CIFAR-10. For this purpose, we chose the best performing model, VGG16 with 2
convolutional blocks. Then, we experimented with no, weak (less) and strong
(more) augmentations. Weak augmentation included random zooming in range (0.5,
1), width shift in range (-2, 2), height shift up to 0.1 and horizontal flip.
Strong augmentation included those techniques in weak augmentation as well as
random rotation up to 45 degrees and random brightness change in range (0.5,
1).

####  5.2.3 Search for the best optimizer

In order to choose the best optimizer, we conducted an experiment by training
the best performing model, VGG16 with 2 convolutional blocks, in the
downstream task of object recognition task on CIFAR-10 with 3 different
optimizers, namely Adam, RMSprop and SGD.

##  6 Results

###  6.1 Phase I

The experiments produced interesting results, as shown in Table 4. VGG16 model
trained with 2 convolutional blocks for the rotation prediction task with 2
rotations (0 and 180 degrees) for the pretext produced best results in the
downstream task of object recognition on CIFAR-10 with almost 80 % accuracy
when the layers trained in pretext task are not retrained in the downstream
task and 70% accuracy when those layers are retrained. Figure 9 shows the
change in accuracy as training progressed during the downstream task for the
model mentioned above.

Overall, the rotation prediction pretext task where the input images were
rotated 2 times (0 and 180 degrees) performed better in the downstream task.
Also, using less convolutional blocks generally performed better. Using more
convolutional blocks produces a gradual decrease in object recognition
accuracy, which we believe is because the feature learnt in these layers start
to become more specific on the rotation prediction task. Furthermore, we see
that increased depth of models lead to better performance in object
recognition with regards to the feature maps produced by earlier layers. We
believe this is because a deeper model enables the features of layers early on
to be less peculiar to the rotation prediction task.

Table 1: Classification accuracy in the downstream task of object recognition
on CIFAR-10 test set. Models used are mentioned along with the number of used
convolutional blocks in the models. Accuracy is reported with varying number
of transformations in the pretext task. Frozen refers to the setting where the
model is trained on transformation prediction pretext task, then the layers
are kept frozen and not trained when non-linear layers are added on top of
them which are trained during the training of downstream task of object
recognition. Unfrozen refers to the setting where the layers trained in the
pretext task are retrained during training of downstream task

Table 2: Phase I: Rotation prediction

Model - block | Rotation - 2 | Rotation - 4 | Rotation - 8
---|---|---|---
| Unfrozen | Frozen | Unfrozen | Frozen | Unfrozen | Frozen
ResNet50 - 2 | 0.7179 | 0.6447 | 0.6145 | 0.5809 | 0.6064 | 0.5924
ResNet50 - 3 | 0.7084 | 0.5807 | 0.6563 | 0.5897 | 0.6306 | 0.6406
ResNet50 - 4 | 0.6927 | 0.3156 | 0.6446 | 0.3604 | 0.6284 | 0.3466
ResNet50 - 5 | 0.672 | 0.1692 | 0.6455 | 0.1655 | 0.6024 | 0.1678
DenseNet201 - 2 | 0.712 | 0.5444 | 0.6639 | 0.5249 | 0.6112 | 0.5094
DenseNet201 - 4 | 0.7341 | 0.1824 | 0.7147 | 0.3899 | 0.6704 | 0.3888
VGG16 - 2 | 0.8083 | 0.6957 | 0.7397 | 0.6647 | 0.7062 | 0.6344
VGG16 - 5 | 0.1024 | 0.0972 | 0.0964 | 0.0964 | 0.0824 | 0.0976
NIN - 2 | 0.7153 | 0.502 | 0.649 | 0.5036 | 0.6148 | 0.4906
ResNet152V2 - 2 | 0.6979 | 0.6788 | 0.622 | 0.6559 | 0.5796 | 0.652

Table 3: Phase II: Affine transformation prediction

Model - block | Transform - 5 | Transform - 10
---|---|---
| Unfrozen | Frozen | Unfrozen | Frozen
NIN - 1 | 0.7357 | 0.5196 | 0.7319 | 0.5538
VGG16 - 1 | 0.7672 | 0.6319 | 0.7605 | 0.6338
ResNet50 - 2 | 0.7633 | 0.5006 | 0.7443 | 0.4934
ResNet152V2 - 2 | 0.7312 | 0.4853 | 0.7188 | 0.4419
NIN - 2 | 0.7494 | 0.4385 | 0.7537 | 0.4742
DenseNet - 2 | 0.7401 | 0.475 | 0.7371 | 0.4818
VGG16 -2 | 0.8216 | 0.5177 | 0.8291 | 0.5952
ResNet50 - 3 | 0.6693 | 0.3729 | 0.6467 | 0.3893
ResNet152V2 - 3 | 0.7111 | 0.332 | 0.649 | 0.3398
DenseNet201 - 3 | 0.7256 | 0.3731 | 0.7193 | 0.3929
VGG16 - 3 | 0.8323 | 0.3596 | 0.8068 | 0.3855
ResNet50 - 4 | 0.6404 | 0.252 | 0.6108 | 0.2837
ResNet152V2 - 4 | 0.6885 | 0.2321 | 0.6251 | 0.2352
DenseNet201 - 4 | 0.7054 | 0.2462 | 0.6985 | 0.2546
VGG16 - 4 | 0.8048 | 0.2334 | 0.7698 | 0.2807
ResNet50 - 5 | 0.6312 | 0.1628 | 0.6013 | 0.1885
ResNet152V2 - 5 | 0.6785 | 0.1932 | 0.6269 | 0.1987
DenseNet201 - 5 | 0.7069 | 0.2447 | 0.689 | 0.2532
VGG16 - 5 | 0.7947 | 0.1688 | 0.7648 | 0.2289

Table 4: Phase II: Effect of data augmentation

VGG16 | Unfrozen | Frozen
---|---|---
Conv2 | Rot-2 | Rot-4 | Rot-8 | Rot-2 | Rot-4 | Rot-8
Strong | 0.706 | 0.7275 | 0.7171 | 0.5479 | 0.5828 | 0.5569
Weak | 0.7818 | 0.7946 | 0.7768 | 0.6033 | 0.6236 | 0.607
None | 0.8421 | 0.8544 | 0.8488 | 0.7249 | 0.7541 | 0.7467

Table 5: Phase II: Search for the best optimizer

VGG16 | Unfrozen | Frozen
---|---|---
Conv2 | Rot-2 | Rot-4 | Rot-8 | Rot-2 | Rot-4 | Rot-8
SGD | 0.7524 | 0.5399 | 0.7631 | 0.7038 | 0.7409 | 0.7242
RMSprop | 0.8409 | 0.8541 | 0.8466 | 0.7229 | 0.7569 | 0.7412
Adam | 0.8404 | 0.8556 | 0.8487 | 0.7301 | 0.7644 | 0.7477

Figure 9: Change in accuracy as training progressed during object recognition
downstream task for VGG16 model trained with 2 convolutional blocks for the
rotation prediction task with 2, 4 and 8 rotations. Here, fine tuning refers
to unfrozen setting and feature extracting refers to frozen setting.

[height=9cm, width=12cm]images/train_vgg-2.png

###  6.2 Phase II

All sets of experiments in the second phase presented fascinating results as
shown in Table 4. Generally, transformation prediction task produced better
models in the unfrozen setting, where the layers trained in the pretext task
are retrained during training of downstream task. VGG16 model with 3
convolutional blocks performed the best in this setting, achieving accuracy of
83.23%. But in frozen settings, here the layers trained in the pretext task
are frozen and not retrained during training of downstream task, rotation
prediction task produced better models.

When it comes to the variation in convolutional blocks, trends similar to
those in rotation prediction task were observed here as well. Figure 10 shows
that VGG16 model trained with 3 convolutional block performs best. The reason
for that could be that 3 convolutional blocks provides the network the
capacity to learn hierarchical features and at the same time does not make it
learn features specific to the pretext task. This trend of 3 convolutional
blocks performing well can be seen across models.

Figure 10: Change in accuracy as training progressed during object recognition
downstream task for VGG16 model trained in the pretext transformation
prediction task with 5 transformations

[height=9cm, width=12cm]images/val_trans_vgg.png

The second set of experiments in phase II indicated that data augmentation
could not help improve accuracy in the downstream task, as shown in Table 4.
In fact, applying augmentations led to decrease in model accuracy. This could
be due to the model not having enough capacity, or because the models in
downstream task are fine tuned instead of training from scratch. The search
for the best optimizer showed that Adam optimized the models best, with
RMSprop not much far off, as shown in Table 5.

##  7 Conclusion

We began with the aim of solving the problem of scarcity of training data. We
come across many works that showed that geometric transformations are powerful
supervisory signals for self supervised learning [10, 18]. This encouraged us
to explore this domain. We then formulated the problem as geometric
transformation prediction, and experimented with rotations in phase I. We then
expanded to more geometric transformations such as shearing, scaling and
translation.

We experimented with multiple models for the self-supervised feature learning
task of training a CNN model to predict the image rotation as well as other
transformations applied to its input images. It could be seen that tasks of
rotation and transformation prediction forces the CNN model trained on it to
learn meaningful features useful for object recognition task as demonstrated.
We ran extensive series of experiment to measure various CNN model’s
performance on the mentioned self-supervised tasks and evaluated their
performance on the downstream task of object recognition on CIFAR-10. It is
observed that transformation prediction task produced better models in the
unfrozen setting.

VGG16 model with 3 convolutional blocks exceeded all other models with Top-1
accuracy of 83.23% in unfrozen setting. In frozen setting, rotation prediction
task performed well. Similarly application of complex data augmentation
techniques did not help improve the accuracy. As inherently Adam combines the
best properties of RMSProp and other optimizers, it optimized well in
comparison to other optimizers. So on the basis of qualitative and
quantitative analysis we can say that using VGG16 with 3-conv blocks , as well
as using Adam optimizer in our model give us much better results in comparison
to other models.

We can conclude by emphasizing that our model which comprised of SSL with
geometric transformations is forced by pretext task to learn good
representations so that they contain sufficient information about visual
structures of transformed images. We further demonstrate that a huge number of
transformations can be easily included into this framework and the experiment
results show us sufficient increase in performance when we transfer our
unsupervised learned features on our downstream task of CIFAR-10
classification.

## References

  * [1] 2d transformation in computer graphics | set 1 (scaling of objects) - geeksforgeeks, 2021.
  * [2] Five popular data augmentation techniques in deep learning, 2021.
  * [3] Github: Waretler/selfsupervision, 2021.
  * [4] Scaling (geometry) - wikipedia, 2021.
  * [5] Shear mapping - wikipedia, 2021.
  * [6] J. Brownlee.  How to configure image data augmentation in keras, 2021.
  * [7] C. Doersch, A. Gupta, and A. A. Efros.  Unsupervised visual representation learning by context prediction.  CoRR, abs/1505.05192, 2015.
  * [8] A. Dosovitskiy, J. T. Springenberg, M. A. Riedmiller, and T. Brox.  Discriminative unsupervised feature learning with convolutional neural networks.  CoRR, abs/1406.6909, 2014.
  * [9] Z. Feng, C. Xu, and D. Tao.  Self-supervised representation learning by rotation feature decoupling.  2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  * [10] S. Gidaris, P. Singh, and N. Komodakis.  Unsupervised representation learning by predicting image rotations, 2021\.
  * [11] A. Krizhevsky.  Learning multiple layers of features from tiny images.  University of Toronto, 05 2012.
  * [12] S. Lazebnik, C. Schmid, and J. Ponce.  Semi-local affine parts for object recognition.  In Proceedings of the British Machine Vision Conference, pages 98.1–98.10. BMVA Press, 2004.  doi:10.5244/C.18.98.
  * [13] D. G. Lowe.  Distinctive image features from scale-invariant keypoints.  Int. J. Comput. Vision, 60(2):91–110, Nov. 2004.
  * [14] M. Noroozi and P. Favaro.  Unsupervised learning of visual representations by solving jigsaw puzzles.  CoRR, abs/1603.09246, 2016.
  * [15] M. Noroozi, H. Pirsiavash, and P. Favaro.  Representation learning by learning to count.  CoRR, abs/1708.06734, 2017.
  * [16] D. Pathak, P. Krähenbühl, J. Donahue, T. Darrell, and A. A. Efros.  Context encoders: Feature learning by inpainting.  CoRR, abs/1604.07379, 2016.
  * [17] C. Shorten and T. M. Khoshgoftaar.  A survey on image data augmentation for deep learning.  Journal of Big Data, 6(1), 2019.
  * [18] L. Zhang, G.-J. Qi, L. Wang, and J. Luo.  Aet vs. aed: Unsupervised representation learning by auto-encoding transformations rather than data, 2021.
  * [19] R. Zhang, P. Isola, and A. Efros.  Colorful image colorization.  In J. Matas, N. Sebe, M. Welling, and B. Leibe, editors, Computer Vision - 14th European Conference, ECCV 2016, Proceedings, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pages 649–666. Springer-Verlag, Jan. 2016.
  * [20] R. Zhang, P. Isola, and A. A. Efros.  Split-brain autoencoders: Unsupervised learning by cross-channel prediction.  In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 645–654, 2017.