# Recent Advances in Vision Transformer: A Survey and Outlook of Recent Work

Khawar Islam Khawar Islam is with FloppyDISK AI ResearchManuscript Submission
June 14, 2022; revised August 00, 0000.

###### Abstract

Vision Transformers (ViTs) are becoming more popular and dominating technique
for various vision tasks, compare to Convolutional Neural Networks (CNNs). As
a demanding technique in computer vision, ViTs have been successfully solved
various vision problems while focusing on long-range relationships. In this
paper, we begin by introducing the fundamental concepts and background of the
self-attention mechanism. Next, we provide a comprehensive overview of recent
top-performing ViT methods describing in terms of strength and weakness,
computational cost as well as training and testing dataset. We thoroughly
compare the performance of various ViT algorithms and most representative CNN
methods on popular benchmark datasets. Finally, we explore some limitations
with insightful observations and provide further research direction.

##  I Introduction

Transformer [1] was introduced in the field of NLP on translating English-to-
German language task. This transformer is solely based on attention mechanism,
instead of convolution layers. It comprises of an encoder and decoder modules.
Both modules are consists of attention layers with feed-forward network.
Before, inserting into a transformer, every word in sentence embedded into
vector $d_{model}=512$ dim. Recently, [2] applied existing architecture of
transformer on computer vision task and achieved top performing results.
Nowadays, ViTs have emerged and gave more promising results, can outperforms
CNN. From an architectural point of view, the backbone of ViTs entirely made
of self-attention mechanism and achieved remarkable success in visual tasks.
There are two major concepts that significantly contributed towards the
evolution of ViT models. (a)Self-attention mechanism where ViTs captured long-
range of token dependencies in a global context, same as traditional recurrent
neural networks. (b) The second important benefit is to train on large-scale
unlabelled datasets and fine-tune on other tasks with small datasets.

Figure 1: Additional steps of ViT. Figure 2: Overview of transformer encoder
block in vision transformer along with multi-head self-attention module.

##  II Overview of Vision Transformer

Graphical overview of additional ViT steps are presented in Figure. 1. It
requires several additional steps to utilize NLP transformer for computer
vision tasks. The overall architecture of ViT can be presented into five main
points:

  * •

Split image into non-overlap/overlap patches ($16\times 16$, $32\times 32$,
etc.)

  * •

Flatten patches and generate lower-dimensional linear embeddings from the
flattened patches referred to (Patch Embedding).

  * •

Add positional embedding and class token.

  * •

Feed the sequence of patches into the transformer layer and obtain the output
(label) through a class token.

  * •

Pass class token values to MLP head to obtain final output prediction.

#### Step 1.

Consider an input image of size $112\times 112$, we firstly generate $16\times
16$ non-overlap/overlap patches. Thus, we can generate $49$ patches and
straightforward feed into the linear projection layer. Remember that, the
number of color channels in each patch is $3$. To obtain a long vector
representation of each patch, the patches is fed into the projection layer and
these representations have been presented in the Figure.
LABEL:fig:INITIALSTEPTANSFORMER as

#### Patch Embedding.

The total number of overlap/non-overlap patches are $49$, patch size with a
number of channels is $16\times 16\times 3$. The size of the long vector of
each patch is $768$. Overall, patch embedding matrix is $49$x$196$. Further,
the class tokens have been added to the sequence of embedded patches and also
added position Embedding. Without positional encoding, the transformer cannot
retain the information and accuracy will be decreased to approx. 3%. Now, the
patch embeddings size becomes $50$ due to the additional class token. Finally,
the patch embeddings with positional encoding and class token feed into the
transformer layer and obtain the learned representations of the class token.
Therefore, output from the transformer encoder layer is $1$x$768$ which is
passed to the MLP block to obtain the final prediction.

#### Transformer Encoder Layer.

Especially in ViTs, the most important component is the transformer encoder
that contains MHSA and MLP block. The encoder layer receives combined
embeddings (patch embeddings, positional embeddings, and class tokens) of
shape $50$ ($49$ patches and $1$ [cls] token)$\times 768$($16\times 16\times
3$) as input. For all layers, the inputs and output of matrix shape $50$x$768$
from the previous layer. In ViT-Base architecture, there are $12$ heads (also
known as layers). Before feeding input into the MHA block, the input is being
normalized through the normalization layer in Fig. 2. In MHA, the inputs are
converted into $50\times 2304$($768\times 3$) shape using a Linear layer to
obtain the Query, Key, and Value matrix. Then, reshaped these three QKV
matrices into $50\times 3\times 768$ where each of the matrices of shape
$50\times 768$ presents the Query, Key, and Value matrices. Further, these
Query, K, and Value matrices are reshaped again to $12\times 50\times 64$.
Once, the Query, Key, and Value matrices are achieved, the attention operation
inside the MHA module is performed using the below equation:

$\mathrm{Attention}(\mathbf{Q},\mathbf{K},\mathbf{V})=\mathrm{softmax}(\frac{\mathbf{Q}\cdot\mathbf{K}^{\top}}{\sqrt{d_{k}}})\cdot\mathbf{V},$
(1)

Once we obtain the outputs from the MHSA block, these outputs are passed to
skip connection as an input to obtain the final output. Before obtaining the
final output, the output is fed into the normalization layer and passed to the
MLP block. Originally, the MLP block was comprised of linear layers and a GELU
activation function. Due to the significant progress in ViT, a locality
mechanism [3] is introduced in MLP to capture local features. Furthermore,
depth-wise convolution is embedded in the MLP block after the first FC layer
to reduce parameters and achieve superior results. Finally, the outcome of the
MLP block feeds into skip connection to obtain final outcome from an encoder
layer.

##  III ViTs for Image Classification

Recent development in ViTs have achieved valuable results in the field of
classification tasks, several researchers have investigated further models for
image classification tasks in different fields such as in medical images,
generative images, etc (See in Table. LABEL:tab:VIT_Methods). Where
transformers learn a useful representation from natural images. After a huge
success of the top-performing transformers in NLP (e.g BERT, GPT, etc). [2]
applied transformer for CV tasks and obtained SOTA results compare to
conventional CNN and required less computation than CNN. To adjust the image
as a sentence, the image is reshaped into 2D flattened patches. Then,
learnable embeddings were added to embedded patches same as BERT class tokens.
Finally, trainable positional embeddings are added into patch representation
to preserve positional information. Transformers are totally dependent on the
self-attention mechanism without the use of a convolutional layer. This self-
attention technique helps the transformer to understand the relationship
between image patches. In theory, an MLP performs better than a CNN model. But
data has been a large obstacle with regard to the performance of MLP models.
The timeline of ViT methods for image classification task is mentioned in Fig
3)

In addition to this, a distillation token has been added to the transformer,
which interacts with image component tokens and classification vectors. In CNN
architectures, we can easily increase performance by stacking more
convolutional layers but transformers are different which are quickly
saturated when architecture becomes deeper. The reason is that as the
transformer enters the deep layer, the attention map becomes more and more
similar. Based on this, [4] introduced a re-attention module, which
regenerated the attention map with a little computational cost in order to
enhance the diversity between layers. Then, re-attention module trained on a
32-layer of ViT [2] transformer, the model achieved top-1 accuracy on Image-
Net with the increment 1.6% performance. [5] focused on the optimization part
in transformers and optimized deeper transformers namely CaiT. It is similar
to encoder and decoder where [6] explicitly split transformer layers involved
self-attention between patches, from class-attention layers which were
dedicated to extracting the content of the processed patches into a single
vector so that it could be fed to a linear classifier.

Prior to existing transformers [5, 2, 7], [8] focused on training ViT from
scratch and re-designed the tokens-to-token process which is helpful to
simultaneously model images based on local-structure information and global
correlation. Moreover, the above method significantly reduced the depth and
hidden layer dimension. [9] introduced dual-branch ViT to extract multi-scale
feature representations and developed a cross-attention-based token-fusion
mechanism, which is linear in terms of memory and computation to combine
features at different scales.

TABLE I: A major contribution of vision transformers in different domains of
computer vision applications.

| Reference
---
Applications | Highlights
[10] | Object detection |  | Fast training convergence, Multi-head cross-Atten
---
[11] |  | Generative video
---
| Tackling quadratic cost, Next-frame prediction
---
[12] |  | Object re-identification
---
| Pure vision transformer, Person and vehicle re-ID
---
[13] |  | Image restoration
---
| Local-enhanced window, Skip-connection schemes
---
[14] |  |  Action recognition
---
| Inter-frame attention, Mutual-attention fusion
---
[15] |  | 3D object detection
---
| Point clouds backbone, Local and global context
---
[16] |  | Medical image segmentation
---
| Convolution for features Long-range association
---
[17] |  | Object goal navigation
---
| Object instances in scene, Spatial locations, regions
---
[18] | Tracking |  | Template, search region, Ego-context block
---
[19] |  | Visual recognition
---
| Light-weight attention, Two-stage architecture
---
[19] |  | Lane shape prediction
---
| Non-local interactions, Directly regressed output
---
[20] |  | Semantic segmentation
---
| Sequence-to-sequence, Different decoder designs
---
[21] |  | Generative adversarial
---
| Bipartite structure, High-resolution synthesis
---
[10] |  | Pixel-wise prediction
---
| Gates attention approach, Multi-scale information
---
[22] |  | Deepfake detection
---
| Mixed architecture, Local and global details
---
[23] |  | HR image generation
---
| Global positioning, Double attention
---
[24] |  | 3D point cloud
---
| Light-weight local-Attenation, 136 times faster inference
---
[25] |  | 6D object pose estimation
---
| Pixel-wise appearance, Point-wise geometric
---
[26] |  | 3D dance generation
---
| Cascading kinematics, Phantom Dance dataset
---
[27] |  | Makeup transfer and removal
---
| Semantic correspondence, Semantic alignment
---
[28] |  | Scene flow estimation
---
| Irregular point cloud, Rich contextual information
---
[29] |  | Place recognition
---
| Super light-weight network, Long-range contextualdata
---
[30] |  | Image Compression
---
| Lightweight image encoder Long-term information
---
[31] |  | Fine-grained recognition
---
| Part selection module, Integrate raw Atten weights
---
[32] |  | Audio and speech
---
| Audio Spectrogram, Generative pre-training
---
[33] |  | Crowd Counting
---
| Multi-scale receptive fields, Extract semantic features
---
[34] |  | Video Restoration
---
| Joint motion estimation, Parallel feature warping
---
[35] |  | Video Grounding
---
| Explore-and-match model, Multiple sentences at once
---
[36] |  | Video Recognition
---
| Multi-resolution context, Lateral connections
---

##  IV Transformers for Segmentation

Segmentation methods have been greatly increased due to the large-margin
success of deep learning methods. It always remains an essential and important
topic in the computer vision community. It includes three fundamental
segmentation types such as (semantic, instance, and panoptic segmentation).
Recently, various transformer methods have shown outstanding results in the
domain of segmentation.

###  IV-A Semantic Segmentation

[20] introduced a sequence-to-sequence approach and replaced the conv-encoder
with a pure transformer. Three different decoders are designed to perform
pixel-wise segmentation using progressive sampling, multi-level feature
aggregation, and a naive up-sampling approach. [37] presented a convolution-
free end-to-end trainable encoder and decoder that captures contextual
information. The encoding portion is built on standard ViT and depends on the
encoding of patches. Further, a point-wise linear decoder is applied and
decoded with a mask transformer. [38] presented SegFormer, a simple yet
powerful method with lightweight MLP decoders. An encoder is based on a
hierarchical structure that gives multi-scale features and does not require
any positional encoding scheme. SegFormer gets rid of complex decoders
combining local attention and global attention to gender representation. [39]
proposed a fully transformer network which relies on pyramid group transformer
encoder to progress learned hierarchical features, while reducing the
computational cost of standard ViT. Also, feature pyramid transformer fused
spatial-level information and semantic level of the pyramid group transformer.

###  IV-B Medical Image Segmentation

Medical image segmentation is an important task for the development of
healthcare frameworks, particularly for disease diagnosis (e.g biomedical,
multi-organ, skin lesion, pannuke and etc). The majority of medical image
segmentation problems, the U-NET111U-NET architecture is a de-facto standard
for Medical Image Segmentation architectures have become the major standard
and achieved top-performing results. Different methods incorporate CNN
approach to the self-attention approach to learning long-range dependencies
among patches and the convolutional layer is used for extracting feature maps.
[40] proposed a hybrid TransUNet for learning high-resolution spatial features
from CNN and long-range dependencies from Transformers. TransUNet obtained
top-performing performance compared to several FCN-based methods included CNN
and self-attention approaches. Transformers are data-hungry and require
massive pre-training datasets to achieve comparable results to CNN. Especially
for training on small-scale datasets, [41] introduced a gated position-
sensitive axial attention approach that worked on smaller datasets. Further,
they proposed a local-global training technique for transformers. The global
part worked on the actual resolution of an image, and a local part operated on
image patches.

For 3D medical image segmentation, [42] introduced a volumetric transformer
and proposed a unique architecture UNETR. They redefined 3D segmentation as 1D
sequence-to-sequence task and utilized a attention mechanism in an encoder to
obtain contextual knowledge from patches. Extracted features from encoder is
combined with a decoder using residual learning (e.g., like ResNet) at
multiple resolutions to predict the segmentation outputs. Similarly to multi-
scale architecture, [43] proposed a multi-scale attention technique to capture
long-range relation by worked on multi-resolution images. To extract features,
CNN-based pyramid architecture is employed. An adaptive-partitioning technique
is applied to maintain informative relations and accessed different receptive
fields. Recently, various transformer studies have been utilized self-
attention mechanism similar to ViT [2]. As a backbone, it is appealing study
to employ hierarchical structure in ViT. Swin Transformer [44] is a backbone
network for vision tasks based on shifted windows technique. [45] proposed a
Swin-Unet, an encoder utilized self-attention technique from local to global
and decoder up-sampled the input global features to correspond pixel-level
segmentation prediction. Different from previous work, [46] presented Segtran
for endless receptive fields to contextualize features for transformer.
Segtran is based on Squeeze-and-Expansion an approach that design for solving
images tasks. Segtran can easily see the fine details and local images.

Figure 3: Timeline of existing vision transformer studies including (pure-
attention, convolutional with attention, multi-layer perceptron, vision
transformer with multi-layer perception, and self-supervised ViTs). As
presented in the figure, the progress has started from the last quarter of
$2020$. ViT [2] and DeiT [5] is available to the community for $2020$. In
$2021$, a lot of vision transformers have been emerged enhancing training
strategies, attention mechanism,s and positional encoding techniques. Major
contributions to the Image-Net dataset for classification task is shown in
this figure. Figure 4: Point Transformer can serve as the backbone for various
3D point cloud tasks. [47]

##  V Transformers for 3D Point Clouds

3D Point cloud aims to collect data in a space with rich geometric
information, shape, and scale knowledge. Each data in point cloud data is
represented as a 3D object or shape and contains Cartesian coordinates (X, Y,
Z). The point collect data is collected from 3D sensors and devices or
photogrammetry software. Recently, self-attention-based point clouds methods
have been gained significant importance due to long-range dependencies,
especially in this year. Different point cloud approaches have been proposed
and achieved SOTA results compare to deep learning methods.

Figure 5: Top-performing method for point cloud registration (e.g.,
classification, part segmentation, and semantic segmentation.)

Recently, [47] designed an impressive layer to serve as a backbone of dense
prediction tasks and scene understanding. A proposed layer is invariant to
permutation and cardinality and is therefore inherently suitable for the
processing of point cloud data. By utilizing the proposed layer, they
constructed a point transformer that is reliable to process point cloud data.
Capturing long-range relationships is difficult in CNN networks, transformer
solves the long dependencies problem through a self-attention mechanism. The
transformer based SOTA method is highlighted in 5. Further, [15] presented a
feature learning-based backbone to capture global and local context. The main
focus on the local region of the same instance then extends the attention
process to other regions. Finally, attended to points from other instances
globally, influencing both local and global relationships. [48] developed a
DPCT transformer network that can easily capture the long-range relationship
and contextual knowledge using point-wise and channel-wise relationships. A
unique multi-scale and multi-level point pyramid transformer [49] are designed
to capture the cross-level and cross-scale context-aware features.

##  VI Transformer for Person Re-Identification

The goal of Re-ID is to retrieve a probe person from gallery images over
various non-overlapping surveillance cameras [50]. With the progressive
development of DDNs and the rising requirement of high-tech video surveillance
cameras, it has gained great interest in the CV and PR community. Re-ID has
been broadly considered a topic in research as a particular person retrieval
problem over non-overlapping CCTV cameras. Providing a probe image, the target
of the Re-ID task is to identify whether this human has seemed in another
place at a different time span captured by a different CCTV camera, or even
the same camera at a different time instant. The probe person can be
represented by an image, a video sequence, and even a text description.

Extracting discriminative features is a major and key problem in Re-ID. While
CNNs based approaches have obtained remarkable success, they only process
information via fixed-size grid mechanism at one time and lose important
information details due to convolution and down-sampling operators (e.g.
pooling and stride). Several transformer-based Re-ID methods have been
designed to overcome these problems, [12] presented a strong baseline that
utilized pure transformer and designed a jigsaw patches module, containing
shift and patch shuffle operation, which promotes perturbation-invariant and
robust discriminative features. Further, they introduced an SIE module that
encoded side information via learnable embeddings. Xuehu et a. [51] proposed a
pure transformer-based Re-ID method to obtain rich features and extract in-
depth video representation. Particularly, to jointly transform raw videos into
spatial and temporal domains, a trigeminal feature extractor is proposed.
Inspired by transformer architecture, three self-view transformers are
designed to utilize token’s dependencies and local information in ST domains.
For extensive video representations, a cross-view method is developed to
combine the multi-view features.

Figure 6: Timeline of self-attention-based mechanism methods for person re-
identification.

Different from Re-ID, video re-ID consists of continuous frames from
surveillance cameras providing more rich spatio-temporal information that can
be more effective and carefully note the movement of identity, view angle-
changes, and appearance. [52] presented a simple transformer-based re-ID
framework for the video re-ID task. Particularly, the constrained attention
technique and global attention learning module are designed to utilize ST
information from videos. To minimize the possibility of over-fitting, they
introduced a synthesized data pre-training strategy to a better-initialized
framework. In re-ID, extracting part-level features is a critical task to
design a robust framework. [53] introduced an alignment technique in
transformer and presented a transformer to automatically find person and non-
person parts at a single time at the patch level. Recently, in part-level
works, [54] combined the output of global and local token of ViTs with a PCB-
like technique to improve re-ID performance. They further incorporated a
block-wise fine-tuning scheme to regularize the fine-tuned of a pre-trained
backbone of ViTs. [55, 56, 57, 58] applied a transformer for image matching
and metric learning and proposed a simplified decoder for efficient image
matching with an emphasis on similarity computation and mapping.

Combining CNNs and transformers is the new direction in the computer vision
community and achieves more impressive results. Guowen e al. [59] utilized
CNNs and transformers to design a re-ID framework and proposed the DSA method
to enhance the multi-scale features and combine hierarchical features from the
CNN backbone. To integrate low-level information, transformer feature
calibration is employed to obtain prominent knowledge and feed into each level
of hierarchical features. Similarly, part-aware transformer [60] adopted CNN
as a backbone network and proposed an occluded re-ID method. Encoder-decoder
architecture is presented to utilize different portions of the human body.
Part diversity and part discriminability are designed to obtain part-
prototypes well with identity labels. [61] proposed disentangled
representation network which handled occluded re-ID using global reasoning of
local features without needing alignment of person image or further guidance.
It’s automatically measured image similarity of a person’s body parts under
the supervision of semantic knowledge of objects.

###  VI-A Future Prospects

Although recent years have observed remarkable progress in ViTs, there is
still a lot of research that is hidden and unexplored. In this section, we
highlight several new research directions for future work.

Most of the proposed ViT variants are highly computational expensive, require
significant hardware, and are time-consuming. The first direction is to apply
CNN pruning principle to prune ViTs according to their learnable scores. This
technique will be helpful to reduce the complexity and memory footprint of
ViT. In terms of the inference speed of ViT, pruning useless features, and
preserving important features so that the final model can run more
efficiently. Thus, deployment of the model becomes easier in real-time devices
(e.g., smartphone, surveillance system) while maintaining the low resource
cost. Therefore, it is a new research direction to obtain top-performing
results while reducing the number of parameters and FLOPS.

There are several unique qualities are present in pure ViT models like self-
attention, global context, and generalization. While CNN is a widely used
technique and popular for extracting pixel-level features (local features) and
shared weights. Interestingly, the majority of studies have been investigated
that a combination of CNN and ViT is more worthy instead of using
straightforward pure ViTs. The main reason behind this, pure ViTs have lack
shared weights, local context, and receptive fields. On other hand, CNN only
works on pixel-level features, compressed all information through filters and
does not consider the distanced pixel-relationship. Thus, it is an interesting
topic to develop convolutional ViT or develop pure ViT without CNN, this
question is still remains unsolved and more study required in this domain.

ViT architectures have achieved superior performance compared to CNN in
difficult tasks such as dense prediction and tiny object detection. These ViT
models can easily able to learn an internal representation of the visual data
which can surpass human capability. To some extent, an internal representation
of the visual data remains secret in the architecture and keeps this technique
as a black box. To explain how it works from an internal side, it is better to
develop a new visualization layer that aims to understand how it works and be
able to interpret its results. Thus, it is an interesting research area to
develop an explainable vision transformer.

Positional encoding is an important part of a transformer to capture a
sequence of input tokens. However, positional encoding is not well
investigated and studied, always remains a controversial topic which encoding
scheme is best for it (e.g., conditional encoding or absolute encoding).
Currently, the evidence available in the [62] relative positional encoding
scheme is able to achieve top-performing results and focus on positional
directivity. This question is for the future whether we need more positional
encoding schemes or not.

##  VII Conclusion

ViTs are gaining more popularity and showing more impressive results in the
domain of CV due to the advantage of the self-attention mechanism and creating
a long-range relationship. In this chapter, we first describe the fundamental
concepts of vision transformers and review the proposed ViT methods in recent
years. Secondly, we highlight the influential work and draw a timeline of ViTs
in different domains (e.g., classification, segmentation, point cloud, and
person re-identification). Lastly, we comparatively study different ViTs and
CNN methods in terms of their accuracy on an image-Net dataset. Overall, ViT
methods demonstrate excellent performance, and combining the self-attention
mechanism with CNN shows outstanding results. Still, the training strategies
and much-hidden work of ViTs have not been discovered yet and fully
investigated.

## References

  * [1] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in _NeurIPS_ , 2017.
  * [2] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly _et al._ , “An image is worth 16x16 words: Transformers for image recognition at scale,” _ICLR_ , 2021.
  * [3] Y. Li, K. Zhang, J. Cao, R. Timofte, and L. Van Gool, “Localvit: Bringing locality to vision transformers,” _arXiv_ , 2021.
  * [4] D. Zhou, B. Kang, X. Jin, L. Yang, X. Lian, Q. Hou, and J. Feng, “Deepvit: Towards deeper vision transformer,” _arXiv_ , 2021.
  * [5] H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. Jégou, “Training data-efficient image transformers & distillation through attention,” _ICML_ , 2020.
  * [6] H. Touvron, M. Cord, A. Sablayrolles, G. Synnaeve, and H. Jégou, “Going deeper with image transformers,” _ICCV_ , 2021.
  * [7] N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in _ECCV_ , 2020.
  * [8] L. Yuan, Y. Chen, T. Wang, W. Yu, Y. Shi, F. E. Tay, J. Feng, and S. Yan, “Tokens-to-token vit: Training vision transformers from scratch on imagenet,” _ICCV_ , 2021.
  * [9] C.-F. Chen, Q. Fan, and R. Panda, “Crossvit: Cross-attention multi-scale vision transformer for image classification,” _ICCV_ , 2021\.
  * [10] G. Yang, H. Tang, M. Ding, N. Sebe, and E. Ricci, “Transformer-based attention networks for continuous pixel-wise prediction,” _ICCV_ , 2021.
  * [11] Y.-F. Wu, J. Yoon, and S. Ahn, “Generative video transformer: Can objects be the words?” in _ICML_ , 2021.
  * [12] S. He, H. Luo, P. Wang, F. Wang, H. Li, and W. Jiang, “Transreid: Transformer-based object re-identification,” _arXiv_ , 2021\.
  * [13] Z. Wang, X. Cun, J. Bao, and J. Liu, “Uformer: A general u-shaped transformer for image restoration,” _arXiv_ , 2021.
  * [14] X. Li, Y. Hou, P. Wang, Z. Gao, M. Xu, and W. Li, “Trear: Transformer-based rgb-d egocentric action recognition,” _IEEE Transactions on Cognitive and Developmental Systems_ , 2021.
  * [15] X. Pan, Z. Xia, S. Song, L. E. Li, and G. Huang, “3d object detection with pointformer,” in _CVPR_ , 2021.
  * [16] Y. Gao, M. Zhou, and D. Metaxas, “Utnet: A hybrid transformer architecture for medical image segmentation,” _MICCAI_ , 2021.
  * [17] H. Du, X. Yu, and L. Zheng, “Vtnet: Visual transformer network for object goal navigation,” in _In ICLR_ , 2020.
  * [18] X. Chen, B. Yan, J. Zhu, D. Wang, X. Yang, and H. Lu, “Transformer tracking,” in _CVPR_ , 2021.
  * [19] L. Yuan, Q. Hou, Z. Jiang, J. Feng, and S. Yan, “Volo: Vision outlooker for visual recognition,” _arXiv_ , 2021.
  * [20] S. Zheng, J. Lu, H. Zhao, X. Zhu, Z. Luo, Y. Wang, Y. Fu, J. Feng, T. Xiang, P. H. Torr _et al._ , “Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers,” in _CVPR_ , 2021.
  * [21] D. A. Hudson and C. L. Zitnick, “Generative adversarial transformers,” _ICML_ , 2021.
  * [22] D. Coccomini, N. Messina, C. Gennaro, and F. Falchi, “Combining efficientnet and vision transformers for video deepfake detection,” _arXiv_ , 2022.
  * [23] B. Zhang, S. Gu, B. Zhang, J. Bao, D. Chen, F. Wen, Y. Wang, and B. Guo, “Styleswin: Transformer-based gan for high-resolution image generation,” _arXiv_ , 2021.
  * [24] C. Park, Y. Jeong, M. Cho, and J. Park, “Fast point transformer,” _arXiv_ , 2021.
  * [25] L. Zou, Z. Huang, N. Gu, and G. Wang, “6d-vit: Category-level 6d object pose estimation via transformer-based instance representation learning,” _arXiv_ , 2021.
  * [26] B. Li, Y. Zhao, Z. Shi, and L. Sheng, “Danceformer: Music conditioned 3d dance generation with parametric motion transformer,” _AAAI_ , 2022.
  * [27] Z. Sun, Y. Chen, and S. Xiong, “Ssat: A symmetric semantic-aware transformer network for makeup transfer and removal,” _AAAI_ , 2022.
  * [28] B. Li, C. Zheng, S. Giancola, and B. Ghanem, “Sctn: Sparse convolution-transformer network for scene flow estimation,” _AAAI_ , 2022\.
  * [29] Z. Fan, Z. Song, H. Liu, Z. Lu, J. He, and X. Du, “Svt-net: Super light-weight sparse voxel transformer for large scale place recognition,” _AAAI_ , 2022\.
  * [30] Y. Bai, X. Yang, X. Liu, J. Jiang, Y. Wang, X. Ji, and W. Gao, “Towards end-to-end image compression and analysis with transformers,” _AAAI_ , 2022\.
  * [31] J. He, J.-N. Chen, S. Liu, A. Kortylewski, C. Yang, Y. Bai, C. Wang, and A. Yuille, “Transfg: A transformer architecture for fine-grained recognition,” _AAAI_ , 2022.
  * [32] Y. Gong, C.-I. J. Lai, Y.-A. Chung, and J. Glass, “Ssast: Self-supervised audio spectrogram transformer,” _AAAI_ , 2022.
  * [33] Y. Tian, X. Chu, and H. Wang, “Cctrans: Simplifying and improving crowd counting with transformer,” _AAAI_ , 2022.
  * [34] J. Liang, J. Cao, Y. Fan, K. Zhang, R. Ranjan, Y. Li, R. Timofte, and L. Van Gool, “Vrt: A video restoration transformer,” _AAAI_ , 2022.
  * [35] S. Woo, J. Park, I. Koo, S. Lee, M. Jeong, and C. Kim, “Explore and match: End-to-end video grounding with transformer,” _AAAI_ , 2022.
  * [36] S. Yan, X. Xiong, A. Arnab, Z. Lu, M. Zhang, C. Sun, and C. Schmid, “Multiview transformers for video recognition,” _arXiv_ , 2022.
  * [37] R. Strudel, R. Garcia, I. Laptev, and C. Schmid, “Segmenter: Transformer for semantic segmentation,” _ICCV_ , 2021.
  * [38] E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “Segformer: Simple and efficient design for semantic segmentation with transformers,” _arXiv_ , 2021.
  * [39] S. Wu, T. Wu, F. Lin, S. Tian, and G. Guo, “Fully transformer networks for semantic image segmentation,” _arXiv_ , 2021.
  * [40] J. Chen, Y. Lu, Q. Yu, X. Luo, E. Adeli, Y. Wang, L. Lu, A. L. Yuille, and Y. Zhou, “Transunet: Transformers make strong encoders for medical image segmentation,” _arXiv_ , 2021.
  * [41] J. M. J. Valanarasu, P. Oza, I. Hacihaliloglu, and V. M. Patel, “Medical transformer: Gated axial-attention for medical image segmentation,” _arXiv_ , 2021.
  * [42] A. Hatamizadeh, D. Yang, H. Roth, and D. Xu, “Unetr: Transformers for 3d medical image segmentation,” _WACV_ , 2022.
  * [43] Z. Zhang, B. Sun, and W. Zhang, “Pyramid medical transformer for medical image segmentation,” _arXiv_ , 2021.
  * [44] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” _ICCV_ , 2021.
  * [45] H. Cao, Y. Wang, J. Chen, D. Jiang, X. Zhang, Q. Tian, and M. Wang, “Swin-unet: Unet-like pure transformer for medical image segmentation,” _arXiv_ , 2021.
  * [46] S. Li, X. Sui, X. Luo, X. Xu, Y. Liu, and R. S. M. Goh, “Medical image segmentation using squeeze-and-expansion transformers,” _IJCAI_ , 2021.
  * [47] H. Zhao, L. Jiang, J. Jia, P. Torr, and V. Koltun, “Point transformer,” _ICCV_ , 2021.
  * [48] X.-F. Han, Y.-F. Jin, H.-X. Cheng, and G.-Q. Xiao, “Dual transformer for point cloud analysis,” _arXiv_ , 2021.
  * [49] X.-F. Han, Y.-J. Kuang, and G.-Q. Xiao, “Point cloud learning with transformer,” _arXiv_ , 2021.
  * [50] K. Islam, “Person search: New paradigm of person re-identification: A survey and outlook of recent works,” _Image and Vision Computing_ , vol. 101, p. 103970, 2020.
  * [51] X. Liu, P. Zhang, C. Yu, H. Lu, X. Qian, and X. Yang, “A video is worth three views: Trigeminal transformers for video-based person re-identification,” _arXiv_ , 2021.
  * [52] T. Zhang, L. Wei, L. Xie, Z. Zhuang, Y. Zhang, B. Li, and Q. Tian, “Spatiotemporal transformer for video-based person re-identification,” _arXiv_ , 2021.
  * [53] K. Zhu, H. Guo, S. Zhang, Y. Wang, G. Huang, H. Qiao, J. Liu, J. Wang, and M. Tang, “Aaformer: Auto-aligned transformer for person re-identification,” _arXiv_ , 2021.
  * [54] C. Sharma, S. R. Kapil, and D. Chapman, “Person re-identification with a locally aware transformer,” _arXiv_ , 2021.
  * [55] S. Liao and L. Shao, “Transformer-based deep image matching for generalizable person re-identification,” _arXiv_ , 2021.
  * [56] K. Islam, S. Lee, D. Han, and H. Moon, “Face recognition using shallow age-invariant data,” in _2021 36th International Conference on Image and Vision Computing New Zealand (IVCNZ)_. IEEE, 2021, pp. 1–6.
  * [57] K. Islam, M. Z. Zaheer, and A. Mahmood, “Face pyramid vision transformer-supplementary,” 2022.
  * [58] ——, “Face pyramid vision transformer,” _arXiv preprint arXiv:2210.11974_ , 2022.
  * [59] G. Zhang, P. Zhang, J. Qi, and H. Lu, “Hat: Hierarchical aggregation transformers for person re-identification,” _arXiv_ , 2021\.
  * [60] Y. Li, J. He, T. Zhang, X. Liu, Y. Zhang, and F. Wu, “Diverse part discovery: Occluded person re-identification with part-aware transformer,” in _CVPR_ , 2021.
  * [61] M. Jia, X. Cheng, S. Lu, and J. Zhang, “Learning disentangled representation implicitly via transformer for occluded person re-identification,” _arXiv_ , 2021.
  * [62] K. Wu, H. Peng, M. Chen, J. Fu, and H. Chao, “Rethinking and improving relative position encoding for vision transformer,” in _ICCV_ , 2021.


# Changing Software Engineers’ Self-Efficacy with Bootcamps: A Research
Proposal

Danilo Monteiro Ribeiro<EMAIL_ADDRESS>ZUP InnovationSão PauloSão
PauloBrazil ,  Alberto Souza<EMAIL_ADDRESS>ZUP InnovationSão
PauloSão PauloBrazil ,  Victor Santiago<EMAIL_ADDRESS>ZUP
InnovationSão PauloSão PauloBrazil ,  Danilo Lucena  ZUP InnovationSão
PauloSão PauloBrazil<EMAIL_ADDRESS>,  Geraldo Gomes
<EMAIL_ADDRESS>ZUP InnovationSão PauloSão PauloBrazil and  Gustavo
Pinto<EMAIL_ADDRESS>ZUP InnovationSão PauloSão PauloBrazil

###### Abstract.

In several areas of knowledge, self-efficacy is related to the performance of
individuals, including in Software Engineering. However, it is not clear how
self-efficacy can be modified in training conducted by the industry.
Furthermore, we still do not understand how self-efficacy can impact an
individual’s team and career in the industry. This lack of understanding can
negatively impact how companies and individuals perceive the importance of
self-efficacy in the field. Therefore, We present a research proposal that
aims to understand the relationship between self-efficacy and training in
Software Engineering. Moreover, we look to understand the role of self-
efficacy at Software Development industry. We propose a longitudinal case
study with software engineers at Zup Innovation that participating of our
bootcamp training. We expect to collect data to support our assumptions that
self-efficacy can be related to training in Software Engineering. The other
assumption is that self-efficacy at the beginning of training is higher than
the middle, and that self-efficacy at the end of training is higher than the
middle. We expect that the study proposed in this article will motivate a
discussion about self-efficacy and the importance of training employers in the
industry of software development.

Self-efficacy, Training, Research Proposal, Industry Training, coding bootcamp

##  1\. Introduction

Self-efficacy as a belief in one’s capabilities to organize and execute the
courses of action required to produce a given attainment (Bandura, 1977a).
More concretely, self-efficacy is how much an individual  _believes_ that s/he
is capable of performing an specific task. For example, when a developer
assesses that s/he is able to finish all her/him task by the end of the week,
based on her/him experience in conducting similar tasks, this developer is
exercising her self-efficacy. Self-efficacy determines the effort that the
individual expended to cope with a task, even if it has impediments and
difficulties (Bandura, 1977a). An individual who has low self-efficacy tends
to avoid difficult tasks and put less effort into completing a task (Artino,
2012).For example, if a developer believes he is not able to act on a certain
part of the code, he may not pick up activities related to that part. Or, if
he doesn’t believe he’s able to learn some technology in time to apply it to
the project, he may be less effortful about doing this task.

Software engineers are also subject to the self-efficacy effect. For instance,
it was observed that individuals may have high or low self-efficacy (Hazzan
and Seger, 2010). If software engineers have low self-efficacy but high
skills, their performance could be compromised because they might not believe
in their potential ability to perform certain tasks and thus they can strive
less (Bandura, 2010), which could impact their professional development and
career progress. For instance, engineers may not apply for more senior
positions because they do not feel they are capable for performing the duties.
On the other hand, if engineers have high self-efficacy but low skills, their
performance can be compromised. For example, the weekly sprint could be
mistakenly estimated, adding more tasks that they are able to conduct during
the sprint (Vancouver et al., 2002; Pajares, 1996; Schunk and Pajares, 2009)

Some authors have been investigating the students perception about their self-
efficacy. Moores and Chang (Moores and Chang, 2009) conducted a study with
software engineering students and found that overconfidence in self-efficacy
is negatively related to performance. The authors also found that when there
was an intervention (for example, if a mentor gives a task performing
feedback), self-efficacy was positively related to performance. Another study
found that algorithms self-efficacy can be improve by pedagogic interventions
(Toma and Vahrenhold, 2018). One important characteristics of self-efficacy is
that one could calibrate it. For example, Brannick et al. (Brannick et al.,
2005) found that mentoring can calibrate self-efficacy of psychology’s
students.

For instance, by know that an engineer has low self-efficacy, this engineer
could use some practices and tools to increase it. Although there are a couple
of studies that investigate self-efficacy in a classroom setting, little is
know about the role of self-efficacy in modern software development bootcamps.

Coding bootcamps (or bootcamps for short) can be defined as a training program
focused on teaching programming-related concepts that are demanded by software
producing companies to developers in the initial levels of their software
development career (Waguespack et al., 2018).

In other areas, such as health, bootcamps are already used to develop more
qualified professionals with better experiences to work on a daily
basis(Esterl Jr et al., 2006). Coding bootcamps are, however, becoming
commonplace in the software development industry. Developers participate in
bootcamps due to several reasons, including: they believe that bootcamp focus
on the most important aspects of coding, the application of learned knowledge
to solve real-world problems to and the potential opportunity to find a job in
a tech company (House, [n. d.]). In a survey with 105 bootcamps and 133k
bootcamp graduates, it was noticed that a $\sim$5% in the number of students
participating in bootcamps (from 2018 to 2019); in the same period, however,
online bootcamps grew 31.44% (Gallagher, [n. d.]). As a consequence, bootcamps
have become an important source for companies looking to hire technical
talents. Some organizations have developed their own bootcamps seeking to
improve their internal workforce (Navas, [n. d.]).

The goal of this paper is to present a proposal to understand the role of
self-efficacy in coding bootcamps. We present and evaluate an approach to
assess and calibrate self-efficacy in bootcamps. This bootcamps were organized
by a Brazilian tech company, with about 3k employees.

##  2\. Background

###  2.1. Self-efficacy Theory

Self-efficacy is proposed as a construct of Social Cognitive Theory (Bandura,
1977b) and can be defined as the belief in one’s own ability to accomplish
something successfully (Bandura, 1977a).

Self-efficacy can change the perception of reality and how individuals behave
(Bandura, 2010), the main assumption of self-efficacy is that people generally
will only attempt things they believe they can accomplish and will not attempt
things they believe they will fail (Bandura, 1977a). This is because
individuals generally do not put effort to do a task that they believe they
cannot do (Bandura, 1977a). When individuals have a strong sense of efficacy,
they trust that they can accomplish even the most difficult tasks.
Furthermore, they can see the task as challenges to be mastered, rather than
threats to be avoided (Bandura and Self-efficacy In, 1994).

Bandura affirms that there are four sources of self-efficacy. The first and
most effective way to impact self-efficacy is by mastering experiences
(Bandura and Self-efficacy In, 1994). The process of mastering an experience
is based on doing a task and succeeding in doing it. This occurs because
individuals tend to believe that they are more capable to do a task if they
have performed well a similar task before.

A famous example of mastery experience is babysitting. Women who have
experience taking care of kids before becoming mothers are more confident with
their abilities to take care their kids (Froman and Owen, 1989). Therefore, to
master at something, an individual has to practice the task. However, Bandura
(Bandura and Self-efficacy In, 1994) presents some important points. For
example, if the tasks are easy or very similar to the ones already performed,
self-efficacy may be poorly developed by individuals. Moreover, the task needs
to be difficult but not impossible. The individual needs to approach
difficulty tasks and work through obstacles to improve self-efficacy (Bandura,
2010). Heslin (Heslin, 1999) gives another tip to improve mastery experience:
breaking down a difficult task into small steps, which are relatively easy, to
ensure a high level of initial success, then progressively increase the
difficulty of the task so that the participant still feels challenged. At
last, Heslin suggests providing feedback through workshops, training programs,
internships, and clinical experiences to improve mastery experiences.

Another factor is vicarious experience. Vicarious experience can occur when an
individual observed others’ successes and failures who are similar to him/her.
Bandura (Bandura, 1977a) mentioned that watching someone like you (e.g., a
colleague in a similar level of experience) doing and successfully
accomplishing something you would like to do (e.g., a task during the spring)
can increase _your_ self-efficacy. Conversely, when you are observing someone
like you to fail, _your_ self-efficacy tends to be negatively impacted.
However, failure is not always harmful; when they feel confident, individuals
can avoid repeating the errors they observed others doing (Brown et al.,
2013). Heslin (Heslin, 1999) commented that to improve self-efficacy, a
manager can do workshops and training sessions because when individuals are
watching others in a training session, the manager can provide observational
experiences.

The third factor affecting self-efficacy is verbal persuasion. When
individuals are persuaded verbally, they may become more confident that they
can do the task (Bandura, 1977a). Having others verbally supporting the
attainment of a task helps to support a person’s belief in himself or herself
(Bandura, 1977a). Verbal persuasion is frequently used by sports coaches to
improve self-efficacy (Brown et al., 2013). Verbal persuasion builds self-
efficacy when managers encourage and praise their competence and ability to
improve their performance (Heslin, 1999).

The fourth and last factor is physical and emotional states that occur when
someone contemplates doing something (Bandura, 1977a). For example, anxiety,
fear, and worry can affect self-efficacy negatively and can lead an individual
to believe that she cannot perform a task (Pajares, 1996).

###  2.2. Self-efficacy in Software Engineering

Some studies investigated self-efficacy in Software Engineering. Tsai and
Cheng (Tsai and Cheng, 2010) conducted an industrial research that found self-
efficacy is related positively to intention to knowledge sharing and knowledge
sharing. The authors hypothesized that self-efficacy could improve knowledge
sharing at software teams. Another study investigated that the students’
programming self-efficacy beliefs had a strong positive impact on the effort
and persistence of Software Engineering students. Furthermore, it was found
that self-efficacy is negatively related to seeking help (Kanaparan et al.,
2017). Arya et al.(Arya et al., 2012) found that self-efficacy is positively
but not very significantly related to organizational commitment. Fu (Fu et
al., 2010) found that professional self-efficacy is positively related to the
career commitment of Software Engineers. Hazzan and Seger (Hazzan and Seger,
2010) identified in their study that high self-efficacy practitioners tend to
be: ”more cooperative, have a greater sense of morale working with their team
members, feel that their relationships with co-workers are closer, get better
managerial support, report higher needs in achievement, dominance,
affiliation, and difference and have better attitudes towards change”.

Other studies are aiming to understand how to improve self-efficacy. For
example, Dunlap (Dunlap, 2005) observed that the use of Problem-Based Learning
could improve the self-efficacy of Software Engineering students. Steinmacher
et al. (Steinmacher et al., 2015) found that an online coach called FLOSScoach
had a positive influence on open-source newcomers’ self-efficacy, making
newcomers more confident and comfortable during the project contribution
process. Srisupawong et al. (Srisupawong et al., 2018) revealed that
perceptions of autonomy, meaningfulness, and involvement are positively
associate with strong self-efficacy. Furthermore, the students’ perceptions of
vicarious experiences and perceptions of social persuasions demonstrated a
positive relationship with self-efficacy. Perceived physiological and
affective states demonstrated a negative influence self-efficacy of computer
science students.

###  2.3. Bootcamps

Early uses of bootcamp dates from the intensive physical training that sought
to make prisoners physically active and encourage discipline in the 19th
century (Anderson et al., 1999). Later, in training camps, American sailors
received a set of intense activities during the American-Spanish war
(bootcampmilitaryfitnessinstitute, [n. d.]).

Bootcamps are also widely used in health education. The objective is to
accelerate the learning process about a given topic following a practical and
dynamic way (Esterl Jr et al., 2006). In general, students have little or no
experience previous experience in the subject covered. From the bootcamp
onwards, they are expected to master the skills studied, helping them to
mitigate eventual skills gaps.

Some studies investigated the relationship between bootcamp and self-efficacy.
Smith et al (Smith et al., 2019) investigated students in their final year of
an entry-to-practice master of occupational therapy program. They found that
students’ self-efficacy for assessing, training, spotting, and documenting
manual and power wheelchair skills improved by between 28% and 35%. Devon et
al. developed a pediatric bootcamp where medical students participated in
practical sessions on their day-to-day work. They observed that the subjects’
self-efficacy increased in all investigated items (Devon et al., 2019).

##  3\. Our Bootcamp context

Our organization has intense training as one of its main pillars. We believe
that focus on the training process should be part of every developer’s
learning journey. Our bootcamp can be seen as a career acceleration program
and seeks to improve one’s path in technology, through an intense and
deliberate training perspective. During the last year, we conducted a total of
11 bootcamps. The goal of the bootcamp is to prepare the bootcamp participants
(and future company’s employees) to situations that may occur in the teams to
which they will be allocated, arming them with knowledge and tools to solve
the most common problems. Thus, our bootcamp training is based on real
situations that are needed to perform daily coding activities in our
organization. Our bootcamp takes from 12 to 16 weeks, and have between 10 and
80 participants (depends on the needs of the organization). Next we further
detail our bootcamp protocol.

###  3.1. Bootcamp participants

We create open calls for junior developers all around the country. Although
the needed skills vary from bootcamp to bootcamp, overall anyone with basic
programming skills (e.g., Java or Kotlin programming; Hibernate and SQL for
managing databases; Spring MVC for structuring web applications; Git for
distributed code sharing) could apply to participate in the bootcamp. We also
require that the participants should be available to work 40h peer week.
During the 11 bootcamps we conducted, we received more than 12k applications,
where around 450 participants participated in our bootcamp. Most of these
participants are junior developers (some with no previous professional
experience).

###  3.2. Bootcamp mentors

A team of experts in the technologies used in the company was recruited to
create our bootcamp protocol. These technical mentors (so called mentors) were
also responsible for teaching, mentoring and evaluating the participants. We
have two mentors for each bootcamp. Mentors are also well-known figures in the
tech radar in Brazil, including renowned speakers and event organizers related
to these technologies. Some even have books among the bestsellers about the
technology used in the bootcamp. All mentors are full time dedicated to
bootcamp execution. Besides from the technical mentors, the bootcamp
participants are also closely followed by a program manager. The program
manager monitors the development of training participants, ensuring the best
experience during the program, supporting the mentor team.

###  3.3. Bootcamp sessions

The bootcamp had synchronous and asynchronous sessions. The training occurs
mostly on synchronous sessions. The mentors give one hour of synchronous
theoretical class per day and two more synchronous sessions of two hours each
during the week. At those sessions, the mentors given feedback and answer
questions. For example, prior to the session, the mentors look at the
participants’ source code, raise points for improvements, and highlight good
coding patterns. Mentors showcase these code snippets during the sessions to
the participants so that they can discuss and understand the rationale behind
the code changes. The mentors are available to clarify any doubts during
working hours, in a virtual room. There are also coding sessions such as dojo
and pair programming that take place during the training. To help participants
further explore their learning path, we also provide to them several
asynchronous programming courses. These courses are made by a third-party
company, which the company bought a license that allow the participants to
access the courses. The bootcamp approaches several technical subjects,
including: Software development (Java, Kotlin, Spring Boot, SQL and
Micronaut), Software design (SOLID), Software architecture (Distributed
Systems and HTTP), Software testing and Software Observability (Prometheus,
Grafana, RED and USE metrics). All topics were chosen due to the company
needs. The full list of technical subjects is available online at:
https://bit.ly/3a9I7cB

###  3.4. Bootcamp evaluation

At the beginning of each week, we plan to deliver a self-efficacy
questionnaire to the bootcamp participants; since our bootcamps have on
average 12 week, we expect to deliver 12 self-efficacy questionnaires per
bootcamp. The goal of these self-efficacy questionnaires is to assess how much
the participants believe they are capable to perform the software development
activities. We provide more information about these self-efficacy
questionnaires in Section 5.1.

For each technical subject, there is a set of key items raised by the experts
that the participant should be able to perform at the end of the course. These
items are on a scale from 0 to 10 (where 0 means Not to all confident and 10
means Extremely confident). The participant will answers a questionnaire
(example111https://encurtador.com.br/noPZ6) with these items at the beginning
and at the end of each technical subject. In addition, at the end of the
technical subject, the participant also has a set of exercises about it.

After answering the exercises, the participants can compare their responses
with the mentors’ responses; The participants answers are stored in a
database. Therefore, when all participants finish a topic, mentors have access
to the participant’s responses and prepare feedback sessions with the most
common mistakes, as well as examples of responses that were well constructed.

The mentors also perform 1:1 feedback with participants, where they use the
responses sent to question situations, praise responses, and propose
improvements.

##  4\. Research Questions

The goal of this study is to understand the role of self-efficacy in the
coding bootcamps. We plan to conduct this investigation in a software
producing company in Brazil, the same one that performs the bootcamps.

###  4.1. Research Questions

In this work, we plan to investigate two research question. First, we want to
understand how software engineer’s self-efficacy and the lack of self-efficacy
calibration can impact their work. Therefore, we pose the following research
question (RQ):

RQ1: How self-efficacy can impact the career and teamwork of software engineer
who participated in an industrial bootcamp?

Next, we want to understand how the participants perceive their self-efficacy
being impacted during the training. Our hypothesis is that: at the beginning
of the training, participants’ self-efficacy may be uncalibrated with their
performance. For instance, since our bootcamps focused on training junior
software engineers, these participants are more likely to have little
knowledge in the software development stack we trained them on. We believe
that as new skills are introduced during the training process, the
participants might have less perception of their self-efficacy. However, when
participants challenge themselves to perform tasks, and receive good feedback
they might be more calibrate their self-efficacy with their real skills.

It is important to note that the (re-)calibration process can be more
challenging in the online bootcamp scenario because participants have less
contact with their mentors, and therefore they have to assess their own self-
efficacy. The goal of our next research question is to provide means to help
mentors to assess the participant’s self-efficacy without have constant
contact with them.

Thus, our second research question is the following:

RQ2: How to calibrate software engineers’ self-efficacy in a industrial
bootcamp?

By answering these questions, we hope we could improve software engineering
training, in particular, in the context of bootcamps, which are becoming even
more prevalent in the software development industry. As a possible outcome of
this research, we plan make recommendations for future industrial training and
academic courses alike about approaches for measuring self-efficacy— and how
we could better calibrate it.

Moreover, by exploring this gap, we will make recommendations for future
academic courses and industrial training about measuring self-efficacy and how
we can calibrate it. Finally, we want to propose a structure to monitor and
measure the self-efficacy of individuals in bootcamps carried out by the
industry.

##  5\. Methodological Framework

This investigation will be conducted in the context of a large (3k+ employees)
Brazilian software producing company: Zup Innovation. Itaú Unibanco recently
purchased Zup innovation. Itaú Unibanco is the largest private bank in Brazil,
the largest financial institution in Latin America, and one of the largest in
the world222https://www.itau.com.br/institucional/sobre-o-itau/. The empirical
method we plan to use for this proposal is the case study. A case study can be
defined as an empirical inquiry that investigates a contemporary phenomenon
within its real-life context (Easterbrook et al., 2008; Runeson et al., 2012)
We intend to use a prospective longitudinal design, when the participants are
followed over a period of time. In our study, the participants will be
evaluated during 1) the bootcamp and 2) at the beginning of their career at
the company.

This research will collect qualitative and quantitative data at different
times. Runeson and Host (Runeson and Höst, 2009) state that the researcher
must clarify their research objectives and questions when planning a case
study. The authors also state that the purpose of the case study may be to
explore, describe or explain a given phenomenon. In this research, the
phenomenon is self-efficacy, and the objective is to understand how self-
efficacy impacts software development. Moreover, understand how self-efficacy
is affected by training in the industrial context. Thus, the case of the study
is how self-efficacy works and shifts from bootcamps conducted by the software
industry.

In the view of Yin (Yin, 2009), to develop a case study, it is also necessary
to have a well-defined unit of analysis. In this work, two units of analysis
will be used: The class in which the students will participate in the training
and the individuals who will participate in the training. The class was chosen
because we are going to group the results in the class that they are performed
to understand the context that class is going through. Individuals were chosen
because self-efficacy is an individual construct.

Given this context, our study can be characterized as a multiple case study
with the embedded approach (Yin, 2009). Yin (Yin, 2009) also comments that
multiple case studies present more convincing evidence and more robust overall
results, as single case studies may be limited by the unique conditions
surrounding the case.

To conduct this research, we plan to conduct ten bootcamps, with approximately
50 participants per bootcamp. As the reader observed in Section 3, we have
extensive experience in conducting bootcamps. All subjects will pass through
an on-line recruitment process design by bootcamp’s organizers. The
organization is aimed to hire all participants that successfully complete the
bootcamp. It is possible that some participants may leave (or even be fired)
during the process. For those that complete the bootcamp, they will be
allocated to a software development team at the organization.

###  5.1. Data Collection

We plan to gather quantitative and qualitative data. The quantitative data
will be based on a family of questionnaires. The qualitative data will come
from observations and interviews with the participants to get a more in-depth
understanding about self-efficacy in industry’s training.

####  5.1.1. Quantitative

We plan to deploy a family of questionnaires. First, we will gather
preliminary information from online questionnaires, like participants’
demographics, programming experience and education level, and then we will
conduct semi-structured interviews to get a more in-depth discussion about
self-efficacy and training in the industry.

Upon entering the training program, all participants should answer a
quantitative self-efficacy questionnaire. This questionnaire was inspired by
the one developed by Steinhorst et al. (Steinhorst et al., 2020). It
approaches four dimensions of self-efficacy: (i) Tracing Program Flow, (ii)
Using Structures and Patterns for Problem-Solving, (iii) Persistence,
Debugging, and Problem-Solving Competences, and (iv) Controlling Program Flow.
We remove the first dimension (Tracing Program Flow) because it was out of the
scope of the organization goals to explore it. We also plan to apply this
questionnaire at the end of the bootcamp process to understand eventual
changes in the participants’ perception. This questionnaire is called as
_basic self-efficacy_.

In addition, we also designed another quantitative self-efficacy questionnaire
focused on software developer roles. In this second questionnaire we evaluate:
(i) Performance (How much the individual feels able to perform activities and
perform well), (ii) Technical Mastery (how much the individual believes he
knows relating to his technical level), (iii) and Teamwork (how much that the
individual believes he/she can contribute to teamwork and influence others to
do their best). Unlike Steinhorst’s et al. questionnaire (Steinhorst et al.,
2020), in this questionnaire we assess the skills and behaviors that the
company perceive as valuable for software engineers. This questionnaire will
be evaluated weekly to follow the evolution of the participants during the
bootcamp. This questionnarie is called as _Role guide self-efficacy_. The
English version of this questionnaire can be seen here:
https://encurtador.com.br/hFLX0

Finally, for each subject taught during the bootcamp training, we will apply a
self-efficacy assessment questionnaire about that subject. After completing
this questionnaire, participants will have to answer an assessment developed
and evaluated by the mentors. This questionnaire is called as _course self-
efficacy_

####  5.1.2. Qualitative

Throughout the process, the mentors have to assess the students’ ability to
perform as a Software Engineer. This assessment can be made through pair
programming and coding dojo sessions. These sessions are (video and audio)
recorded. By the triangulation of these data, we intend to measure the real
ability of individuals to compare with their self-efficacy and their
calibration in different times of training.

Finally, participants will go through semi-structured qualitative interviews
during the bootcamp and after their allocation to software development teams
to understand their perception of self-efficacy and its calibration. This
questionnaire can be view here 333https://bit.ly/3iNO5Vo

The career success of individuals in the company does not depend on their
participation in the questionnaire. Before collecting the data, the research
team is introduced, clarifies what the research is about, and explains the
importance of being honest in the answers to the questionnaires. The bootcamp
participants signed a consent form in their first week. Each week, along with
the questionnaire, a brief description of the research, the confidentiality of
information, and the importance of honestly answering the questionnaire are
sent.

##  6\. Initial results

At this section, we present some initial results. Since we are going to
measure self-efficacy, it is very important to have a questionnaire that
actually reflects the individuals’ self-efficacy. To validate the
questionnaire, psychometric techniques were used based on the recommendations
proposed by Hair et al. (Hair, 2009). Although these techniques already exist
and are used in Software Engineering, they are recently gaining popularity in
the field for increasing the reliability of research (Graziotin et al., 2021).

The _Role guide self-efficacy_ questionnaire used in the bootcamp was applied
to 108 company employees, and an exploratory factor analysis was carried out.
Kaisor-Meyer-Olkin (KMO) and Bartlett’s sphericity test were used to measure
the sampling adequacy. The KMO test indicates the proportion of variance in
your variables that might be caused by underlying factors. Values close to 1.0
generally indicate that a factor analysis may be useful (IBM, [n. d.]; Hair,
2009). Bartlett’s test of sphericity tests investigate if the correlation
matrix is an identity matrix, which would indicate that your variables are
unrelated. Values than 0.05 indicate that a factor analysis may be useful
(IBM, [n. d.]; Hair, 2009).

The results showed that the KMO value was 0.947, and the significance of
Bartlett’s sphericity was 0.000. (v2 = 2915,422, df = 1,128, $\rho$ = 0.000),
indicating that the good samples adequacy.

After, principal component factor analysis was performed using Oblimax
rotation with Kaiser Normalization. The main objective is analyzing the items
to reduce a large dimension of data to a relatively smaller number of
dimension. Here, we are looking to found the same three factors previously
theoretically proposed for the questionnaire. (Performance, Technical Mastery
and Teamwork). Factor analysis yielded a 3-factor solution with an explained
variance of 73.81%.

Furthermore, We also used Cronbach’s alpha to measure our scale reliability.
We found $\beta$ = 0.953 at performance dimension, we found $\beta$ = 0.957 at
domain dimension and for culture dimension, we found $\beta$ = 0.858.
According to Streiner (Streiner, 2003), values above 0.7 are considered with
good reliability. All these results indicate a good quality of the scales
created in our bootcamp. In future works, we intend to present the whole
process of scale’s validation with more detail. Moreover, to increase the
validity of the questionnaire, we intend to apply Confirmatory factor analysis
to our scale too.

##  7\. Limitations and threats to validity

Similar to all research, there are limitations to how this study was designed.
First, we choose a case study and all investigation will be conducted on Zup
Innovation. The context can limit some results.

Regarding the participants, all of them will undergo a selection process led
by human resources. We can specify attributes, but all selection of
individuals is done outside the search. Therefore, we cannot guarantee that
any aspect about the characteristics of the selected people or even that the
selection process will be the same for all classes/participants.

Another important point is that the questionnaires are self-perceived,
therefore, the honesty of the participants in their answers is necessary. We
make clear the need to be honest with the questionnaire in the authorization
and consent form. In addition, we held a presentation for participants
explaining what research is all about. It is noteworthy that the organization
already hires all people.

##  8\. Conclusions

As mentioned above, self-efficacy can be related to performance and can be
impacted by training. Therefore, this work aims to clarify the importance of
developing self-efficacy with training in Software Engineering to the
industry. The main contribution of this idea study proposed will be the
identification of the relationship between training (bootcamp) and self-
efficacy. Moreover, we pretend to understand how self-efficacy change during
the weeks of training and the impact of self-efficacy in software engineering.
We believe that organizations should train and develop individuals’ skills and
their self-efficacy to perform better.

Other relationships can be investigated in future works. For example, the
relationship between self-efficacy and performance in an industrial context.
This relationship already was investigated in academic setting (Davazdahemami
et al., 2018) in Software Engineering and is critical.

Finally, this research should be an important step towards the consolidation
of self-efficacy and training in industrial context in Software Engineering
research.

## References

  * (1)
  * Anderson et al. (1999) James F Anderson, Laronistine Dyson, and Jerald C Burns. 1999\.  _Boot camps: An intermediate sanction_.  University Press of America.
  * Artino (2012) Anthony R Artino. 2012\.  Academic self-efficacy: from educational theory to instructional practice.  _Perspectives on medical education_ 1, 2 (2012), 76–85.
  * Arya et al. (2012) Bhavana Arya, Mridula Sharma, and Shriparna Singh. 2012\.  Moderating effect of gender role orientation on the relationship between organizational commitment and self efficacy.  _International Journal of Scientific and Research Publications_ 2, 12 (2012), 1–5.
  * Bandura (1977a) Albert Bandura. 1977a.  Self-efficacy: toward a unifying theory of behavioral change.  _Psychological review_ 84, 2 (1977), 191.
  * Bandura (1977b) A Bandura. 1977b.  Social learning theory.
  * Bandura (2010) Albert Bandura. 2010\.  Self-efficacy.  _The Corsini encyclopedia of psychology_ (2010), 1–3.
  * Bandura and Self-efficacy In (1994) A Bandura and VS Self-efficacy In. 1994.  Ramachaudran.  _Encyclopedia of human behavior_ 4 (1994), 71–81.
  * bootcampmilitaryfitnessinstitute ([n. d.]) bootcampmilitaryfitnessinstitute. [n. d.].  The history of bootcamp.  https://bootcampmilitaryfitnessinstitute.com/what-are-boot-camps/history-of-boot-camps/.  https://bootcampmilitaryfitnessinstitute.com/what-are-boot-camps/history-of-boot-camps/
  * Brannick et al. (2005) Michael T Brannick, Donald E Miles, and Jennifer L Kisamore. 2005.  Calibration between student mastery and self-efficacy.  _Studies in Higher Education_ 30, 4 (2005), 473–483.
  * Brown et al. (2013) Lisa J Brown, John M Malouff, and Nicola S Schutte. 2013\.  Self-efficacy theory.  _Retrieved May_ 31 (2013), 2015.
  * Davazdahemami et al. (2018) Behrooz Davazdahemami, Andy Luse, Kevin P Scheibe, and Anthony M Townsend. 2018.  Training, Self-Efficacy, and Performance; a Replication Study.  _AIS Transactions on Replication Research_ 4, 1 (2018), 3\.
  * Devon et al. (2019) Erin Pete Devon, Rebecca Tenney-Soeiro, Jeanine Ronan, and Dorene F Balmer. 2019.  A pediatric preintern boot camp: program development and evaluation informed by a conceptual framework.  _Academic pediatrics_ 19, 2 (2019), 165–169.
  * Dunlap (2005) Joanna C Dunlap. 2005\.  Problem-based learning and self-efficacy: How a capstone course prepares students for a profession.  _Educational Technology Research and Development_ 53, 1 (2005), 65–83.
  * Easterbrook et al. (2008) Steve Easterbrook, Janice Singer, Margaret-Anne Storey, and Daniela Damian. 2008.  Selecting empirical methods for software engineering research.  In _Guide to advanced empirical software engineering_. Springer, 285–311.
  * Esterl Jr et al. (2006) Robert M Esterl Jr, David L Henzi, and Stephen M Cohn. 2006\.  Senior medical student “boot camp”: can result in increased self-confidence before starting surgery internships.  _Current Surgery_ 63, 4 (2006), 264–268.
  * Froman and Owen (1989) Robin D Froman and Steven V Owen. 1989.  Infant care self-efficacy.  _Scholarly Inquiry for Nursing Practice_ 3, 3 (1989), 199–211.
  * Fu et al. (2010) Jen-Ruei Fu et al. 2010\.  Is information technology career unique? Exploring differences in career commitment and its determinants among IT and non-IT employees.  _International journal of electronic business management_ 8, 4 (2010), 272–281.
  * Gallagher ([n. d.]) James Gallagher. [n. d.].  State of the Coding Bootcamp Market Report 2020.  https://careerkarma.com/blog/bootcamp-market-report-2020.
  * Graziotin et al. (2021) Daniel Graziotin, Per Lenberg, Robert Feldt, and Stefan Wagner. 2021.  Psychometrics in Behavioral Software Engineering: A Methodological Introduction with Guidelines.  _ACM Trans. Softw. Eng. Methodol._ 31, 1, Article 7 (Sept. 2021), 36 pages.  https://doi.org/10.1145/3469888
  * Hair (2009) Joseph F Hair. 2009\.  _Multivariate Data Analysis_.
  * Hazzan and Seger (2010) Orit Hazzan and Tali Seger. 2010.  Recruiting software practitioners: the importance of self-efficacy.  (2010).
  * Heslin (1999) Peter A Heslin. 1999\.  Boosting empowerment by developing self-efficacy.  _Asia Pacific Journal of Human Resources_ 37, 1 (1999), 52–64.
  * House ([n. d.]) Fire House. [n. d.].  What is a coding bootcamp?  http://thefirehoseproject.com/developer-guide/1.
  * IBM ([n. d.]) IBM. [n. d.].  KMO and Bartlett’s Test.  https://url.gratis/f8bo3I.
  * Kanaparan et al. (2017) Geetha Kanaparan, Rowena Cullen, and David DM Mason. 2017\.  Self-efficacy and Behavioural Engagement in Introductory Programming Courses.. In _PACIS_. 209\.
  * Moores and Chang (2009) Trevor T Moores and Jerry Cha-Jan Chang. 2009.  Self-efficacy, overconfidence, and the negative effect on subsequent performance: A field study.  _Information & Management_ 46, 2 (2009), 69–76.
  * Navas ([n. d.]) Shinaz Navas. [n. d.].  The future of the coding bootcamp industry-from Springboard, Le Wagon and Career Karma.  {https://www.linkedin.com/pulse/future-coding-bootcamp-industry-from-springboard-le-shinaz-navas/}
  * Pajares (1996) Frank Pajares. 1996\.  Self-efficacy beliefs and mathematical problem-solving of gifted students.  _Contemporary educational psychology_ 21, 4 (1996), 325–344.
  * Runeson and Höst (2009) Per Runeson and Martin Höst. 2009.  Guidelines for conducting and reporting case study research in software engineering.  _Empirical software engineering_ 14, 2 (2009), 131–164.
  * Runeson et al. (2012) P Runeson, M Höst, A Rainer, and B Regnell. 2012\.  Case study research in software engineering-guidelines and examples Wiley.
  * Schunk and Pajares (2009) Dale H Schunk and Frank Pajares. 2009.  Self-efficacy theory.  _Handbook of motivation at school_ (2009), 35–53.
  * Smith et al. (2019) Emma M Smith, Krista L Best, and William C Miller. 2019\.  A condensed wheelchair skills training ‘bootcamp’improves students’ self-efficacy for assessing, training, spotting, and documenting manual and power wheelchair skills.  _Disability and Rehabilitation: Assistive Technology_ (2019).
  * Srisupawong et al. (2018) Yuwarat Srisupawong, Ravinder Koul, Jariya Neanchaleay, Elizabeth Murphy, and Emmanuel Jean Francois. 2018.  The relationship between sources of self-efficacy in classroom environments and the strength of computer self-efficacy beliefs.  _Education and Information Technologies_ 23, 2 (2018), 681–703.
  * Steinhorst et al. (2020) Phil Steinhorst, Andrew Petersen, and Jan Vahrenhold. 2020\.  Revisiting self-efficacy in introductory programming. In _Proceedings of the 2020 ACM Conference on International Computing Education Research_. 158–169.
  * Steinmacher et al. (2015) Igor Steinmacher, Igor Wiese, Tayana Uchoa Conte, and Marco Aurelio Gerosa. 2015.  Increasing the self-efficacy of newcomers to Open Source Software projects. In _2015 29th Brazilian Symposium on Software Engineering_. IEEE, 160–169.
  * Streiner (2003) David L Streiner. 2003\.  Being inconsistent about consistency: When coefficient alpha does and doesn’t matter.  _Journal of personality assessment_ 80, 3 (2003), 217–222.
  * Toma and Vahrenhold (2018) Laura Toma and Jan Vahrenhold. 2018.  Self-Efficacy, Cognitive Load, and Emotional Reactions in Collaborative Algorithms Labs-A Case Study. In _Proceedings of the 2018 ACM Conference on International Computing Education Research_. 1–10.
  * Tsai and Cheng (2010) Ming-Ten Tsai and Nai-Chang Cheng. 2010.  Programmer perceptions of knowledge-sharing behavior under social cognitive theory.  _Expert Systems with Applications_ 37, 12 (2010), 8479–8485.
  * Vancouver et al. (2002) Jeffrey B Vancouver, Charles M Thompson, E Casey Tischner, and Dan J Putka. 2002.  Two studies examining the negative effect of self-efficacy on performance.  _Journal of applied psychology_ 87, 3 (2002), 506\.
  * Waguespack et al. (2018) Leslie Waguespack, Jeffry Stephen Babb, and David Yates. 2018\.  Triangulating Coding Bootcamps in IS Education: Bootleg Education or Disruptive Innovation?  _Information Systems Education Journal_ 16, 6 (2018), 48\.
  * Yin (2009) Robert K Yin. 2009\.  _Case study research: Design and methods_. Vol. 5.  sage.