acm short = ACM , long = Association for Computing Machinery , sort = A ,

# Benchmarking Algorithms for Automatic License Plate Recognition

Marcel Del Castillo Velarde and Gissel Velarde Contact Author: Marcel Del
Castillo Velarde<EMAIL_ADDRESS>Report: November
2021

###### Abstract

We evaluated a lightweight Convolutional Neural Network (CNN) called LPRNet
[1] for automatic License Plate Recognition (LPR). We evaluated the algorithm
on two datasets, one composed of real license plate images and the other of
synthetic license plate images. In addition, we compared its performance
against Tesseract [2], an Optical Character Recognition engine. We measured
performance based on recognition accuracy and Levenshtein Distance. LPRNet is
an end-to-end framework and demonstrated robust performance on both datasets,
delivering 90 and 89 percent recognition accuracy on test sets of 1000 real
and synthetic license plate images, respectively. Tesseract was not trained
using real license plate images and performed well only on the synthetic
dataset after pre-processing steps delivering 93 percent recognition accuracy.
Finally, Pareto analysis for frequency analysis of misclassified characters
allowed us to find in detail which characters were the most conflicting ones
according to the percentage of accumulated error. Depending on the region,
license plate images possess particular characteristics. Once properly
trained, LPRNet can be used to recognize characters from a specific region and
dataset. Future work can focus on applying transfer learning to utilize the
features learned by LPRNet and fine-tune it given a smaller, newer dataset of
license plates.

###### Index Terms:

License Plate Recognition, Computer Vision, Convolutional Neural Network,
Optical Character Recognition, Pareto analysis

##  I Introduction

License Plate Recognition (LPR) aims at accurately retrieving a digital string
of characters given a License Plate image. The retrieved digitalized License
Plate can be stored in a database to perform several tasks automatically for
different applications such as Vehicle Traffic Management, Parking Tolls,
Digital Surveillance Systems, and Access Control to Buildings. LPR is a
solution that allows automatic vehicle control, reducing costs on human labor.
Moreover, in some populated countries, the number of vehicles on the streets
grows faster than the number of inhabitants. For example, between 1990 and
2015, the vehicle growth rate was 3.5 faster than the growth rate of the
Mexican population [3]. In Bolivia, the number of vehicles grew by 38 percent
between 2014 and 2019 [4]. Therefore, automatic LPR systems are considered to
help automize transportation, its management, and security.

In this study, we evaluated the performance of a deep network called LPRNet
[1]. We performed tests on two license plates datasets. In addition, we
compared its performance against Tesseract [2]. While LPRNet was designed to
recognize characters from license plates, Tesseract is generally used to
recognize characters from documents. Since we had a synthetic dataset
available for tests, we selected Tesseract as a baseline for comparison, at
least on the synthetic images. We were also interested in testing if LPRNet
was able to maintain its performance when trained on a different dataset,
given that license plates on each country have unique characteristics. We
assumed that a robust system should recognize characters independently of the
dataset in use if adequately trained. Besides, we expected LPRNet to be
similarly good either on a real and a synthetic dataset. We evaluated
recognition accuracy using a test dataset containing $1\,000$ real license
plates images [1] and a subset from a dataset of synthetic license plates
images [5], with $10\,000$ images for training and $1\,000$ images for
testing.

The contribution of this study is the benchmarking of LPRNet [1] and Tesseract
[2] for automatic license plate recognition on two different datasets and the
proposal of a framework based on Pareto Analysis [6] for conflicting
characters detection. The article is organized as follows, section II presents
the problem statement, section III the proposed solution with both analyzed
algorithms, section III-C describes the datasets, pre-processing, and
evaluation metrics, section IV presents the experiments and results. Finally,
we conclude in section V.

##  II Problem Statement

License Plate Recognition is an application of computer vision for intelligent
transportation. Deep learning architectures are currently used to retrieve
characters from license plates. Previous models included a segmentation step
before recognizing each character, combined with heuristics from the dataset
in question [7]. More recently, end-to-end architectures automatically allow
learning properties from raw images for their predictions. End-to-end
eliminates the need for intermediate pre-processing steps such as
segmentation, thresholding, or the use of heuristics. The advantage of an
automatic end-to-end system over a system that relies on heuristics is that by
training the model on the dataset in use, the system should be able to
discover relevant features on its own.

##  III The proposed solution

We selected LPRNet for our experiments because its implementation allowed us
to train the network from scratch using a selected dataset. Since the
alternative dataset we had at disposal for testing was a dataset of synthetic
images, we compared LPRNet’s performance against Tesseract, which is generally
used for Optical Character Recognition from documents. Although Tesseract
might not be a natural selection for our application, it allowed us to
benchmark LPRNet on the alternative synthetic dataset. If the evaluated
algorithms prove to adapt to different datasets, we could expect them to work
on a newly collected dataset, given that license plates posses unique
characteristics from their particular region.

###  III-A LPRNet

Convolutional Neural Networks (CNNs) are deep learning models delivering
state-of-the-art results in computer vision [8]. These models simulate neural
activation and hierarchical brain processing. In perceptual tasks, CNNs
demonstrate to learn relevant representations that are robust to data
variability. CNNs learn suitable filters or feature maps on each processing
layer, given input images for recognition.

LPRNet is a fast CNN that inputs RGB images of 94x24 pixels and outputs a
sequence of characters [1]. After all convolutional layers, it follows batch
normalization and ReLU activation. LPRNet’s architecture is designed to be
lightweight and has two main components, a Backbone and Small Basic Blocks.
LPRNet uses Connectionist Temporal Classification (CTC) to control the loss
for training as end-to-end learning instead of a segmentation process. The
Backbone architecture of the network can be seen in Table I, and the
architecture of the Small Basic Blocks can be seen in Table II. This network
was trained with Adam optimizer, batch size 32, gradient noise scale of 0.001,
and learning rate of 0.001 that was dropped by a factor of 10 every 100
thousand iterations for a total of 250 thousand iterations [1].

Layer Type |  Parameters
---|---
Input |  94x24 Pixels RGB image
Convolution |  Filters 64, 3x3, stride 1
Max Pooling |  Filters 64, 3x3, stride 1
Small Basic Block |  Filters 128, 3x3, stride 1
Max Pooling |  Filters 64, 3x3, stride (2,1)
Small Basic Block |  Filters 256, 3x3, stride 1
Small Basic Block |  Filters 256, 3x3, stride 1
Max Pooling |  Filters 64, 3x3, stride (2,1)
Dropout |  0.5 ratio
Convolution |  Number of class number, 1x13, stride 1
TABLE I: The LPRNet backbone structure from [1]. Layer Type |  Parameters
---|---
Input |  $C_{in}$ x H x W features maps
Convolution |  $C_{out}$ /4, 1x1, stride 1
Convolution |  $C_{out}$ /4, 3x1, stride H=1, padding H=1
Convolution |  $C_{out}$ /4, 1x3, stride W=1, padding W=1
Convolution |  $C_{out}$ 1x1, stride 1
Output |  $C_{out}$ x H x W features maps
TABLE II: Small Basic Block structure, from [1].

###  III-B Tesseract

Tesseract [2] is an open source for Optical Character Recognition that expects
binary images and outputs a digital string of characters per input. First,
Tesseract uses Adaptive Thresholding to binarize images, followed by Connected
Component Analysis to find lines and words which are organized as units. Then,
if a word is recognized, it aims at finding edges of characters that match
their training set. Tesseract was originally trained using $60\,160$ samples
of 94 characters, with 20 samples per class in 8 fonts and 4 presentations:
normal, italic, bold, and bold-italic [2].

###  III-C Data Sets

We used two datasets. The first dataset consists of Chinese license plates
dataset with $1\,000$ RGB real images sized 94x24 pixels, JPG format [1]. The
images are characterized by being taken at different angles and in real
situations; see Fig. 1a. The second dataset consists of a subset from the
Synthetic Turkey license plates [5]. We selected the images at random. We used
$10\,000$ images for training and $1000$ images for testing. The images are
sized 1025x218 pixels in PNG format. The license plates of this dataset are
synthetic images designed in a single frontal view, see Fig. 1b.

(a) a

(b) b

Figure 1: Examples of images per dataset [1], [5]. 1a, Chinese dataset (real)
and 1b, Turkey dataset (synthetic) .

###  III-D Data Pre-Processing

Turkey dataset images were rescaled to 94x24 pixels to match the dimensions of
the Chinese dataset. We used an antialiasing method, which uses a high-quality
resolution reduction filter [9]. In addition, PNG images were converted to JPG
format.

Figure 2: Extracting a region of interest and cropping a synthetic license
plate from Turkey dataset [5].

###  III-E Pre-Processing for Tesseract

Initially, Tesseract delivered poor results on the synthetic dataset.
Therefore, we extracted the interest region as seen in Fig. 2. Once the
interest region was extracted, Tesseract was able to recognize characters
accurately from the synthetic Turkey dataset. However, it was not able to
recognize characters from the Chinese dataset. Therefore, we applied image
manipulation techniques such as thresholding and binarization before sending
the images to Tesseract, but they did not yield significant changes.

###  III-F Evaluation metrics

We used two metrics for evaluation. First, we evaluated accuracy expressed as:

$Accuracy=\frac{TP}{TP+TN_{1}+TN_{2}}$ (1)

where True Positive (TP) is the correct classification of the license plate,
True Negative $TN_{1}$ represents a misclassification when the lengths of the
strings between label and prediction are different, and $TN_{2}$ represents a
misclassification when the string lengths are equal. This measure only
considers the fraction of fully correctly classified license plates over
correctly classified and misclassified license plates. However, it could be
that a misclassified license plate contains more than one erroneous character.

Therefore, we also used the Levenshtein distance between the ground truth
labels and the algorithms’ outputs. Levenshtein distance measures the smallest
number of insertions, deletions, or substitutions needed to transform one
sequence of characters to another [10]. See an example in Table III.

License Plate |  Classification |  Levenshtein Distance
---|---|---
|  16WW522 |  2
|  55L3928 |  1
TABLE III: Examples of Levenshtein Distance. License plates, classification
and corresponding Levenshtein Distance.

##  IV Results

Algorithm |  Pre-Processing |  Dataset |  Mean Accuracy |  Mean Levenshtein distance
---|---|---|---|---
LPRNet |  - |  Chinese Real images . Train set: $11\,696$ samples. Test set: $1\,000$ samples. |  0.900 |  0.047
LPRNet |  Resize,RGB to BGR |  Turkey Synthetic images. Train set: $10\,000$ samples. Test set: $1\,000$ samples. |  0.886 |  0.030
Tesseract |  Resize, Crop |  Turkey Synthetic images. Test set $1\,000$ samples. |  0.933 |  0.054
TABLE IV: Benchmarking LPRNET and Tesseract on real Chinese and synthetic
Turkey License Plates datasets. LPRNet’s mean accuracy on both datasets is 89
percent. Tesseract achieves 93 percent accuracy on the synthetic dataset.

We evaluated LPRNet and Tesseract on the synthetic Turkey license plates
dataset, as seen in Table IV. Only LPRNet was evaluated on the real Chinese
license plates dataset because we could not train Tesseract on this dataset.
We evaluated license plate recognition considering mean accuracy and mean
Levenshtein distance. In all cases, we used a test set of $1\,000$ images. We
tested the pre-trained LPRNet network. This network was pre-trained on a
dataset with $11\,696$ Chinese license plates [1]. On a test set of $1\,000$
images, LPRNet delivered a mean accuracy of 90 percent and a low mean
Levenshtein of 0.047. For the Turkey dataset, images were resized and then
converted from RGB to BGR. We re-trained the network with $10\,000$ Turkey
license plates, obtaining 88.6 percent mean accuracy and mean Levenshtein of
0.3 on a test set of $1\,000$ images.

Tesseract was not trained. On the Chinese test set of $1\,000$ images,
Tesseract could not recognize characters. Preliminary tests on the Turkey
dataset did not deliver good results either. Therefore, we performed the pre-
processing tasks described in section III-E. We cropped all license plates
such that images contained only characters. Then, Tesseract delivered up to 93
percent mean accuracy and a low mean Levenshtein distance.

We wondered if Tesseract could deliver similar performance for the Chinese
license plates dataset if re-trained on this dataset. However, training
Tesseract was out of our possibilities. LPRNet proved robust and appropriate
for use on different datasets if properly trained. It is known that for deep
learning networks, the more samples for training, the better the results [11].
Our training set for the synthetic dataset was smaller than the training
dataset of the real images. It seems possible that with a larger dataset for
training, the performance of LPRNet could improve. Given that dataset
collection is a costly process, transfer learning could be applied when using
LPRNet on a new and much smaller dataset.

Fig. 3, Fig. 4 and Fig. 5 show a subsample of randomly selected plates that
were misclassified by each algorithm. Table V summarizes the number of
misclassified license plates on each dataset. In some cases, the length of the
output string is the same length as that of the target string. In other cases,
the length of the output string has a different length than that of the target
string. In the next section, we evaluated misclassified characters of outputs
with the same length.

Algorithm |  Test set |  Same length |  Diferent length |  Total Misclassified
---|---|---|---|---
LPRNet |  Chinese |  40 |  60 |  100
LPRNet |  Turkish |  26 |  88 |  114
Tesseract |  Turkish |  49 |  18 |  67
TABLE V: Number of misclassified license plates depending on the length.
---
Figure 3: Sample of 5 plates out of 100 misclassified by LPRNet on Chinese dataset.  Plate |  LPRNet misclassification
---|---
|  01BNU35
|  02L4S78
|  08RFM60
|  38GVK61
|  39GLF53
Figure 4: Sample of 5 plates out of 114 misclassified by LPRNet on Turkish dataset. Plate |  Tesseract misclassification
---|---
|  3107646
|  5107011
|  56Ul992
|  5717125
|  70Tl627
Figure 5: Sample of 5 plates out of 67 misclassified by Tesseract on Turkish
plates.

###  IV-A Pareto analysis

We used Pareto analysis for both algorithms (Tesseract and LPRNet) to identify
misclassified license plates, where the predicted output had the same length
as the target string. Pareto analysis obtains a percentage of frequency of
misclassified characters and arranges these frequencies in descent order, from
most conflicting to least [6]. This analysis helped us identify which
characters were the most conflicting characters. We observed:

  * •

False Positive Characters, and

  * •

False Negative Characters.

False Positive Characters are predicted characters misclassified by the
algorithm; that is, the algorithm predicted a ‘P’ but the target was not a
‘P’. False Negative Characters are target characters misclassified by the
algorithm or characters that should have been predicted, but were not
recognized. For example, the target character was an ‘A’, but the algorithm
predicted another character. Therefore, for each experiment, we created two
Pareto charts identifying the most conflicting characters, the percentage of
error for each character and its accumulated error, see Fig. 6 to Fig. 11.

Since we could not train Tesseract on Chinese dataset, we performed a Pareto
analysis only for LPRNet on the Chinese dataset, row 1 in Table IV. The
analysis was performed on a set of 40 out of 1000 license plates misclassified
such that the output string of LPRNet had the same length as the target
license plate string. Erroneous outputs with strings of different lengths were
not considered in this analysis. Pareto charts for False Positive Characters
and False Negative Characters are shown in Fig. 6 and Fig. 7, respectively. We
observed that ‘0’ and ‘5’ are the most frequent False Positive Characters,
while ‘P’, ‘S’, and ‘5’ are the most frequent False Negative Characters. In
the synthetic Turkey dataset, we analyzed 26 out of 1000 license plates with
the same length as the target strings. Similarly, LPRNet outputs ‘0’ as a
frequent False Positive Character as well as ‘H’. The three most frequent
False Negative Characters were ‘A’, ‘I’ and ‘0’, see Fig. 8 and Fig. 9.

Finally, Fig. 10 and Fig. 11 are Pareto charts for Tesseract on a set of 49
misclassified license plates (same length), row 3 in Table IV. Like for
LPRNet, ‘0’ is the most frequent False Positive Character, with 17
occurrences, while ‘I’ and ‘O’ are the most frequent False Negative
Characters. In general, distinguishing ‘0’ from ‘O’ seems to be the most
challenging pair of characters.

##  V Conclusion

LPRNet demonstrated robustness in recognizing characters from both license
plate datasets, either from real images or synthetic images, given that the
network is properly trained with the corresponding dataset. Tesseract was not
trained on real license plate images and performed well on the synthetic
dataset only. We wondered if Tesseract could also perform well on the real
images from the Chinese dataset, if adequately trained on it. However,
training Tesseract was out of our reach. Pareto analysis helped us identify
the misclassified character frequencies automatically. For both algorithms,
distinguishing ‘0’ from ‘O’ proved to be difficult. Without Pareto analysis,
it would be time consuming to discover which are the conflicting characters.
LPRNet, a lightweight deep learning network, demonstrated to learn relevant
filters for a given dataset. We observe that LPRNet’s reported performance can
be achieved with at least $10\,000$ license plates images. Collecting this
amount of sample images could be a long and costly process. Future work could
investigate applying transfer learning to utilize the features learned by
LPRNet, and fine-tune the network on a new and smaller dataset from a
particular region. The application of smart technologies for automatic license
plate recognition constitute a step towards better transportation, its
management, and security.

## References

  * [1] S. Zherzdev and A. Gruzdev, “LPRNnet: License plate recognition via deep neural networks,” arXiv preprint arXiv:1806.10447, 2018.
  * [2] R. Smith, “An overview of the tesseract ocr engine,” in Ninth international conference on document analysis and recognition (ICDAR 2007), vol. 2, pp. 629–633, IEEE, 2007.
  * [3] IMCO, “Indice de movilidad urbana 2018: Barrios Mejor Connectdaos para cuidades más equitativas.” https://imco.org.mx/indice-movilidad-urbana-2018-barrios-mejor-conectados-ciudades-mas-equitativas/, 2019.  Accessed: 31-7-2021.
  * [4] Instituto Nacional de Estadística de Bolvia, “Parque Automotor.” https://www.ine.gob.bo/index.php/estadisticas-economicas/transportes/parque-automotor-cuadros-estadisticos/, 2019.  Accessed: 23-7-2021.
  * [5] T. Üstünkök, “Synthetic turkish license plates.” https://www.kaggle.com/tustunkok/synthetic-turkish-license-plates, 2019\.  Accessed: 2-7-2021.
  * [6] H. Gutiérrez Pulido and V. Salazar, Control estadístico de calidad y seis sigma.  MCGRALL-HILL, 2004.
  * [7] S. Montazzolli and C. Jung, “Real-time brazilian license plate detection and recognition using deep convolutional neural networks,” in 2017 30th SIBGRAPI conference on graphics, patterns and images (SIBGRAPI), pp. 55–62, IEEE, 2017.
  * [8] M. Z. Alom, T. M. Taha, C. Yakopcic, S. Westberg, P. Sidike, M. S. Nasrin, M. Hasan, B. C. Van Essen, A. A. Awwal, and V. K. Asari, “A state-of-the-art survey on deep learning theory and architectures,” Electronics, vol. 8, no. 3, p. 292, 2019.
  * [9] A. Clark, “Pillow (pil fork) documentation,” 2015.
  * [10] V. I. Levenshtein et al., “Binary codes capable of correcting deletions, insertions, and reversals,” in Soviet physics doklady, vol. 10, pp. 707–710, Soviet Union, 1966.
  * [11] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” nature, vol. 521, no. 7553, pp. 436–444, 2015.

Figure 6: False Positive Characters. Pareto chart for LPRNet. Set: 40 of 1000
Chinese license plates misclassified (same length).  Figure 7: False Negative
Characters. Pareto chart for LPRNet. Set: 40 of 1000 Chinese license plates
misclassified (same length).  Figure 8: False Positive Characters. Pareto
chart for LPRNet. Set: 26 of 1000 synthetic Turkey license plates
misclassified (same length).  Figure 9: False Negative Characters. Pareto
chart for LPRNet. Set: 26 of 1000 synthetic Turkey license plates
misclassified (same length).  Figure 10: False Positive Characters. Pareto
chart for Tesseract. Set: 49 of 1000 synthetic Turkey license plates
misclassified (same length).  Figure 11: False Negative Characters. Pareto
chart for Tesseract. Set: 49 of 1000 synthetic Turkey license plates
misclassified (same length).


# The legacy of Bletchley Park on UK mathematics

Daniel Shiu

##  1\. Introduction

The second world war saw a major influx of mathematical talent into the areas
of cryptanalysis and cryptography. This was particularly true at the UK’s
Government Codes and Cypher School (GCCS) at Bletchley Park. The success of
introducing mathematical thinking into activities previously dominated by
linguists is well-studied, but the reciprocal question of how the cryptologic
effort affected the field of mathematics has been less investigated. Although
their cryptologic achievements are not as celebrated as those of Turing, Tutte
and Welchman, Bletchley Park’s effort was supplemented by more eminent
mathematicians, and those who would achieve eminence and provide leadership
and direction for mathematical research in the United Kingdom. Amongst their
number were Ian Cassels, Sandy Green, Philip Hall, Max Newman and Henry
Whitehead. This paper considers how the experience of these and other
mathematicians at Bletchley Park may have informed and influenced the
mathematics that was produced in their post-war careers.

Three main themes will be pursued. Firstly, how the successful use of Bayesian
statistics at GCCS increased the use of this perspective by post-war
statisticians over the classical frequentist interpretation that was prevalent
in the UK before the war. Secondly, how Bletchley Park alumni recognised the
value that computation could add to mathematical study and that this
contributed to the development of strong UK research areas in computational
mathematics. Thirdly, how the mathematical community at Bletchley Park
influenced younger mathematicians in their choice of study and research as
well as laying the foundations for collaborations within academic mathematics.

##  2\. The promotion of Bayesian statistics

Our first theme has already been noted and the reader is commended to read the
excellent accounts of Zabell [41],[42]. Zabell notes that the Bayesian
interpretation of statistics was in abeyance in the United Kingdom leading up
to the war. Because statistics was relatively under-represented at Bletchley
Park, mathematicians less trained in statistics, such as Good and Turing,
approached the subject with fresh eyes and were able to demonstrate that the
Bayesian perspective was remarkably effective when applied to cryptanalytic
problems. After the war, Turing was more interested in the development of
computers, but Good realised that the isnights of Bayesian statistics could
have a broader application than cryptanalysis. Zabell goes on to relate how
Good became a prominent advocate for the Bayesian viewpoint starting with his
1950 book _Probability and the Weighing of Evidence_ [11], and continuing with
writings such as _Rational decisions_ [12],_The estimation of probabilities:
An essay on modern Bayesian methods_ [17], _Good thinking: The foundations of
probability and its applications_ [15], and _The Bayes/non-Bayes compromise: A
brief review_ [16].

It is worth emphasising the importance of his Bletchley Park experiences in
Good’s first book. In the preface, Good readily acknowledges his Bletchley
Park colleagues (without the GCCS context):

> Dr. A. M. Turing, Professor M. H. A. Newman and Mr. D. Michie were good
> enough to read the first draft (written in 1946) and I am most grateful for
> their numerous suggestions.

More than this though, Michie actively encouraged its publication when Good’s
own belief in the book was at a low ebb. As Good recalls in an interview with
David Banks [2]

> A second reason for moving from Manchester was that my book Probability and
> the Weighing of Evidence (completed in 1948) [11] had been declined by the
> Cambridge University Press. Who the referee was I don’t know, but it could
> have been Jeffreys, since Frank Smithies told me he saw the manuscript on
> Jeffreys’s desk. I was discouraged and did nothing about the manuscript for
> several months at least. Then Donald Michie, a good friend of mine right
> from the Newmanry days, said, Why don’t you send it in somewhere and let
> someone else do the work if you’re not working on it? I said, Look, the
> manuscript’s in Manchester and I’m in London. He replied, Take a train to
> Manchester and come back the same day on another train. I didn’t have enough
> faith in myself without his moral support. And I did what he advised, and
> submitted the manuscript to Charles Griffin, who published it in 1950. (Most
> publication was slow at the time.) I think it wasn’t such a bad little book.
> Somebody phoned me recently and called it a classic.

Although now well-established (with over 1500 citations), the initial
reception for the book explains some of Good’s doubts and the importance of
Michie’s intervention. Reviews varied from the dismissive [8]

> It is fair to say that statisticians in general will not accept Dr Good’s
> approach to statistical problems

to the qualified [7]

> This book is interesting reading whether or not one agrees with the author.
> It is a significant contribution to a field in which contributions are badly
> needed.

It is also worth noting that although the big names of British statistics at
the time were not present at Bletchley Park, two other cryptanalysts became
statisticians of note. Although they did not have Good’s evangelical zeal for
the Bayesian interpretation, we can deduce that the statistical community was
seeded with other ex-Bletchley Park cryptanalysts who knew of its classified
value. The first is the US statistician Solomon Kullback, known best for
Kullback-Leibler divergence. Kullback was seconded to Bletchley Park for three
months and was exposed to the work on Enigma and TUNNY [10] and Good reports
[14] that Kullback was a user of weighted evidence ideas. Another is E. H.
Simpson of Simpson’s Paradox fame. In a 2010 paper [32] Simpson explains how
in addition to its use on the TUNNY and Enigma challenges, Bayes theorem was
also applied to less celebrated work on the Japanese masked JN25 codebook. The
exchange of ideas between the different areas is unclear. Simpson states [32]:

> Because of the compartmenting of all the various teams at Bletchley Park, I
> knew nothing of their [Turing, Alexander, Good] work at the time.

Conversely, in his paper on Turing’s wartime work on statistics, in the
section on repeat rates, Good notes [13]

> E. H. Simpson and I both obtained the notion from Turing.

Whether or not others’ knowledge of the cryptanalytic efficacy of Bayes
influenced the opinion of the statistical community, it is clear that Good’s
mission to promote Bayesian thinking was successful. His obituary in the
London Times records [35]

> To statisticians, Good is one of the founding fathers of Bayesian statistics
> […] This approach – the Bayesian paradigm, as it is now called – was little
> used before Good’s work but was given an important boost by his 1950 book
> and his extensive subsequent writings, and is firmly established today.

##  3\. An age of experimental computational mathematics

We now turn to the effect on mathematics of computational machinery and how
Bletchley Park alumni were central to this evolution. We will not dwell long
on the design and development of computers, though Bletchley Park personnel
were again pivotal in this story. There were three significant computer
construction projects in the immediate post-war period: the Automatic
Computing Engine project, designed by Turing and supported by him 1945-47; the
Manchester Computing Machine Laboratory established by Max Newman and where he
was joined Jack Good and David Rees (and Turing in 1948); the Electronic Delay
and Storage Automatic Calculator at Cambridge’s mathematical laboratory. These
projects initially produced devices (Pilot ACE, Manchester Baby, EDSAC) that
are considered prototypes for later “commercial computers” (DEUCE, Ferranti Mk
1, Atlas). Nevertheless, they already produced computations of significance
for mathematics.

The first generation machines were not used by a broad research community, and
many programmes were proof-of-concept demonstrations of known mathematical
results. Alongside investigations of Mersenne primes, solution of differential
equations and chess programmes, the most ambitious use of the Manchester Mk 1
was probably research by Alan Turing. Prior to the war, Turing had become
interested in the computations of zeroes of the Riemann $\zeta$-function [37],
in particular whether the famous Riemann hypothesis could be disproved in such
a way. His pre-war proposals were mechanical in nature, with a £40 grant from
the Royal Society enabling him to construct some of the gears necessary for
the computation. The project took a back burner while Turing worked at
Bletchley, but the Manchester Mark 1 gave him an opportunity to revisit the
idea with electronic computation. The results are recorded in his 1953 paper
_Some calculations of the Riemann zeta-function_ [38]

> In June 1950 the Manchester University Mark 1 Electronic Computer was used
> to do some calculations concerned with the distribution of the zeros of the
> Riemann zeta-function. It was intended in fact to determine whether there
> are any zeros not on the critical line in certain particular intervals.

Although Turing’s paper expresses only limited satisfaction with the results,
it is a feat that few others could have accomplished and the outstanding
mathematical result of first generation machinery.

Early computers were unreliable and required expert oversight. The absence of
not only of “high level” programming languages, but even a recognisable
operating system left a high bar for those wishing to engage with the new
capabilities. A certain level of faith in the outcome was necessary to invest
the effort, but this faith was considerably easier for those who had seen the
successful use of electronic calculation on the problems of Bletchley Park.
One influential voice was J. W. S. Cassels, who although he did not programme
the early computers himself, encouraged their use. In his overview paper
_Computer-aided Serendipity_ [5], he notes

> I am not a computer expert, but I am fortunate in having friends who are,
> and in having been a witness of early developments. The first approaches to
> electronic computing were made during the war with specialised devices for
> ballistic or cryptographic purposes

> […]

> There was no such profession as computer science in those days. Several of
> my fellow research students in number theory found the challenges of the new
> discipline congenial and moved into it. I might add that I have always
> encouraged my research pupils to learn about computers.

One of Cassels’s students, Bryan Birch, was an eager adopter of computer
assistance. The famous Birch-Swinnerton-Dyer conjecture (one of the Clay
Institute’s Millennium Problems) motivated by computations on EDSAC2 in the
sixties is a notable example. Moreover, Birch supervised a very large number
of doctoral students and laid the foundations for a national strength in
computational number theory that endures to this day in both academic and
classified communities. Birch was an early researcher at the Atlas Computer
Lab in Harwell near Oxford. Here mathematics researchers were able to perform
experiments using a second generation computer, taking advantage of both
hardware developments in the form of transistors as well as usability features
such as the Atlas Supervisor (arguably the first modern operating system) and
high level language for programming.

Unsurprisingly, the expertise of Bletchley Park alumni was attractive to the
Atlas Lab and Jack Good and Donald Michie were recruited in short order. Good
recalls [2]

> For a few months, the Atlas was the fastest computer in the world, but IBM
> overtook it. The laboratory was sixteen miles away from Oxford, along a road
> dangerous for an inexperienced driver, so I didn’t visit the lab as often as
> I would have liked. I did most of my work at the college, some of which was
> on the “underware” for a “five-year plan” for chess programming. One day I
> came into the laboratory and found my office taken over…

The most impressive story of Atlas and a Bletchley Park alumnus is the work of
A. O. L. Atkin. In his _Tribute to Oliver Atkin_ [4], Birch is unequivocal in
drawing the connection between Atkin’s work at then Newmanry at Bletchley Park
and his subsequent successes.

> So by the time he was 20, Oliver had had more experience of a group of
> extremely intelligent people working together than most people ever see in
> their lifetimes; one can hardly imagine a more exciting mathematical
> environment. Oliver had learnt what mathematics was about, and could see how
> important computation could become.

> […]

> He wanted to know what these congruences looked like, and after Bletchley
> Park he thought he could use computers to find out. His opportunity occurred
> when the ATLAS laboratory advertised a research fellowship in 1961. [I
> should explain that the ATLAS computer laboratory was attached to the Atomic
> Energy Research Establishment at Chilton (Harwell) about twelve miles south
> of Oxford.] Oliver applied, and of course he was appointed; his duties were
> to use one of the world’s most advanced computers to do research in topics
> of his choice. At ATLAS, he was in his element, though there were those who
> didn’t like it when he walked down the corridor singing Wagner fortissimo

The early number theoretic successes of work at Atlas and the approach of the
researchers is documented in the proceedings of 1969 Atlas Symposium
_Computers in Number Theory_ [1]. In their preface Atkin and Birch note

> The papers illustrate all aspects of the use of computers in number theory:
> as an essential part of a proof, as an aid to discovery […], and negatively
> as a possible ally in doing what has not yet been done. The attitude
> sometimes maintained a few years ago, that a computer is a disreputable
> device in the context of “pure” mathematics, was noticeably absent at the
> symposium.

As well as papers by Atkin and Birch, Jack Good provided two papers to the
symposium. One on congruential pseudo-random number generators has a decidedly
cryptographic feel, the other was a computational investigation of the Mertens
conjecture. This is a number theoretic conjecture even more powerful than the
Riemann hypothesis, that by the time of the conference was already felt to be
false by the number theory community. Good and his former GCHQ colleague Bob
Churchhouse gather data to formulate a statistical argument for the falsity of
the conjecture whilst simultaneously making a heuristic case for the Riemann
hypothesis. In [18], inkeeping with the above remarks of Atkin and Birch, they
write

> Thus Conjecture A is not merely to be expected a priori, by mathematical
> common sense, but it is well supported by the numerical data. As we said
> before, we believe therefore that there is a good “reason” for believing the
> Riemann hypothesis, apart from the calculation of the first 2,000,000 zeros.

The computational perspective on Mertens conjecture proved powerful and
Odlyzko and te Riele were able to produce a computer-aided counterexample in
1985 [28].

Atkin’s time at Atlas only lasted until 1970 and the lab itself only until
1975. Atkin continued to be a prolific computer and correspondent, though
sparse in his writing for publication. It is worth emphasising how important
some of his contributions were in the 1980s when elliptic curves began to play
a major role in cryptography. The Schoof-Elkies-Atkin point counting method
and Atkin-Morain primality certification are both important methods for public
key cryptography. The results further underscore the symbiotic development of
mathematics and cryptography. The circle is completed by the observation that
the original “non-secret encryption” result by Cliff Cocks was informed by a
seminar by Cocks’s supervisor, Birch, on new computational methods for
factorisation. This ushered in a cryptographic era of public key cryptography
to which Atkin contributed strongly.

##  4\. Persistent connections

In this last section, we consider how the working and social connections
established at Bletchley Park continued as academic mathematical connections.
Many of the Bletchley Park mathematicians had already known each other, indeed
these pre-existing connections were often the reason for their recruitment.
Welchman [39] in particular reached recruited colleagues, students under him,
those who had shared his undergraduate days and even his old school. Not all
of the mathematicians knew each other however and the informal style of
Bletchley Park fostered the exchange of ideas and friendships between
undergraduates and established academics.

The evidence of this extension is manifested firstly in Bletchley Park alumni
who completed PhDs in mathematics after the war under the supervision of other
alumni. The PhD was not necessarily a pre-requisite for academic study prior
to the war (for example although David Rees studied for a PhD under Welchman
and Hall, he never completed the degree). After the war, the qualification
became more emphasised in the UK, bringing it more into line with America and
Europe. As such there was less formal process around pairing students with
supervisors, and if the pair already knew each of other the study fell into
place more naturally. We give some examples.

The cryptanalytic successes of Bill Tutte on TUNNY are now well-known.
Mathematically he is also famous for bringing about a renaissance in the study
of graph theory. It is worth noting that prior to Bletchley Park, Tutte had
studied and graduated as a chemist (though one with a strong talent for
solving mathematical problems), and only transferred to a study of mathematics
in 1940. Very soon after, he joined Bletchley Park where he worked on the FISH
ciphers, including being a member both of the Newmanry and one of the five
initial members of the research section. After the war Bill Tutte was elected
a Fellow of Trinity College Cambridge, in recognition of his work at Bletchley
Park. He noted [40]

> It seemed to me that the election might be criticized as a breach of
> security, but no harm came of it.

He set about obtaining a PhD in mathematics and his choice of supervisor was
his colleague from the Newmanry and research section, Shaun Wylie. He had also
fixed on graph theory as the topic. This was unusual as it was far from
Wylie’s speciality of homotopy. Supposedly a condition of the supervision was
that Tutte provide his own problems. Tutte adds [40]

> in those days, graph theory had a low reputation in the mathematical world.
> so it is not surprising that Shaun Wylie, as my Ph.D. supervisor, advised me
> to drop graph theory and take up something respectable, such as differential
> equations. If one assumes that graph theory was my métier, it was just as
> well that I had the prestige of a Fellow of Trinity.

Tutte was prolific as a PhD student, quickly producing four papers each of
which was hugely influential, with the Tutte 1-factor theorem and Tutte
polynomial rapidly becoming eponymous. His thesis went further still,
establishing the subject now known as matroid theory.

Whereas Tutte had very clear ideas of the problems that he wished to tackle
even though they were outside the sphere of his friend and supervisor, Peter
Hilton’s path was less planned. He relates how he came to study topology [33]

> two of my colleagues [at Bletchley Park] were Henry Whitehead and Max
> Newman. Due to the peculiar circumstances of the war, I was on an equal
> footing with them. In fact, I was simply a young man who had taken a wartime
> degree and they were eminent mathematicians. I became, in particular, very,
> very friendly with Henry Whitehead, on first-name, beer-drinking terms.
> After the war, Henry Whitehead invited me to come back to Oxford and be his
> first post-war research student. I said, “But I don’t know anything about
> topology.” And he said, “Oh, don’t worry, Peter, you’ll like it.” So in
> fact, I didn’t even know what the subject was. I went back to Oxford, and I
> studied topology. Whatever Henry Whitehead had specialized in, I would have
> studied. His personality was so attractive, that it was clear that it was
> going to be great fun to work with him. It turned out to be not only fun but
> extremely exacting and demanding - it was a marvellous experience. I really
> took up topology because it was Whitehead’s field.

Whitehead’s prediction was absolutely correct. Hilton went on to a very
successful research in topology as well as authoring multiple authoritative
textbooks in this and other areas.

Another of Whitehead’s students, Gordon Preston was even younger than Hilton
(Hilton joined Bletchley Park in 1942 at the age of 18; Preston in 1944 at the
age of 19). Whereas Hilton’s four terms at Oxford allowed him to claim a
wartime BA, Preston undertook another two years of undergraduate study. These
included popular lectures given by Whitehead and Preston went on to become his
graduate student [29],

> For the first year I worked with him on algebraic topology. He then went off
> to the States for a year and my new supervisor was E C (Edward) Thompson
> and, with him, I changed my subject to algebraic geometry, with a strong
> emphasis on commutative ring theory and with not so much geometry.

Preston’s mathematical interests were influenced work of other Bletchley Park
colleagues[29]

> My semigroup influence came because of the friends I had met at Bletchley
> Park. I used to go to the National Physical Laboratory at Teddington - this
> was while I was an undergraduate again from 1946 to 1948 - to keep in touch
> with the development of the pilot ACE (Automatic Computing Engine) that Alan
> Turing had been drafted there to develop. I was interested in the
> mathematical papers of my other Bletchley friends and, in particular, this
> was how I came to read David Rees’s papers. I have no records, and my memory
> may be playing me false, but I believe the first paper I read on semigroups
> was his paper ‘On semi-groups’ from the Proceeding of the Cambridge
> Philosophical Society, 1940, together with the small technical note that
> followed it, “Note on semi-groups”, ibid., 1941.

He also notes that a significant informal resource for semigroups were
lectures by Philip Hall at Cambridge; again Bletchley Park colleagues are
involved [29]

> My introduction to them came when Sandy Green, probably in 1951, lent me a
> copy of his notes on the lectures given one year by Philip Hall. I made a
> hand-written copy of them - this was before the days of the ubiquitous
> photocopier - which I still have. This was exciting and tremendous stuff.
> Semigroups were used quite naturally in Hall’s lectures, at any rate for
> that year.

Whitehead had returned to the UK when Preston submitted his thesis and was
internal examiner and David Rees as external examiner [29]

> My D. Phil. examiners were David Rees and Whitehead. David gave me a list of
> detailed comments which Henry Whitehead would not let him ask me about in
> the oral examination - Henry was the chairman of examiners - because Henry
> had to rush off to captain a cricket team playing that afternoon. Henry
> asked me a number of questions himself, including some about my chapter on
> semigroups; and in my answers to his questions which were critical questions
> I was able apparently to show that he had failed to grasp properly the
> concepts involved. After this I was invited to come and watch the cricket
> game, which I took as a sign that I had passed my viva.

The area of semigroups in UK mathematics is dominated by Bletchley Park
mathematicians. At its heart we find the eminent mathematician Philip Hall. In
contrast to Whitehead’s gregarious nature, Hall was more reserved111“When [the
mathematician Olga] Taussky accused Hall of being the worst recluse in
Cambridge, he replied ‘No, Turing is worse’.”[20] However, accounts of his
post-war lecturing show that he presented a compelling mathematical vision
[20]

> During the next five years he gave lectures in which he explored not groups
> so much as aspects of all the other algebras. He was full of vigour and
> brimming with ideas. When he was not presenting his own results, he
> illuminated others with his own interpretation, always going further than
> they had gone. Students felt that they were hearing today what the rest of
> the world would only hear tomorrow.’

Two junior Bletchley Park colleagues “Sandy” Green (who arrived at Bletchley
Park in 1944 at the age of 18) and Derek Taunt (who started in 1941 at the age
of 23) went on to study under Hall. Green notes [21]

> At Cambridge I was much impressed by lectures given by the algebraist Philip
> Hall; and in a different way by D E Littlewood.

Taunt’s college obituary notes [30]

> Derek remained at Bletchley until VE day, when suddenly there was nothing
> left to do. He was posted to Teddington to work on aerodynamics, but once
> the Japanese war was over he came back to Cambridge to pick up where he had
> left off. Hardy had retired and Derek chose instead to work with the
> distinguished group theorist Philip Hall, who became a life-long friend.

We summarise the supervisor-student relationships of this section in the
following diagram.

WylieTutteWhiteheadHiltonPrestonWelchmanHallReesGreenTaunt Figure 1. Bletchley
Park supervision graph

Another measure of collaboration is co-authorship of research and textbooks.
Mathematics papers tend more to single authorship than other scientific areas
and this was even more true before the modern era of cheap travel and rapid
communications. There are however some notable joint works by Bletchley Park
alumni. These papers should be judged both substantial and influential with
Google scholar recording dozens of citations for each. The most cited by far
(798 citations) is the definitive introductory homology textbook _Homology
theory: An introduction to algebraic topology_ [23], written by Peter Hilton
and Shaun Wylie. Glowingly reviewed on its publication, it became the standard
text for final year undergraduate courses in topology and early graduate
students. It is now regarded as a classic.

Another work by Hilton, this time with his supervisor Whitehead _Note on the
Whitehead product_ [24] was accepted to the Annals of Mathematics (regularly
ranked as the leading mathematics journal in the world).

Hilton also co-authored a paper with David Rees _Natural maps of extension
functors and a theorem of RG Swan_ [22].

There was also a significant collaboration on a family of semi-groups by David
Rees and Sandy Green _On semi-groups in which $xr=x$_ [19].

The other co-authorship of note is between Max Newman and Alan Turing. Prior
to their work at Bletchley Park, Newman and Turing had already collaborated on
computational models in their 1942 paper _A formal theorem in Church’s theory
of types_ [27]. Subsequent to their war work we have noted that Newman drew
Turing to Manchester to work on the Manchester baby. Public interest in the
Manchester work was piqued by talk of artificial intelligence. As part of
stating the case more clearly Newman and Turing took part in a BBC radio
broadcast _Can automatic calculating machines be said to think?_ [36] the
discussion was moderated by Richard Braithwaite (a professor of philosophy)
and an opposing view was put by Geoffrey Jefferson (a professor of
neurosurgery). The broadcast was transcribed and published in _Machine
Intelligence_ (OUP) in 1999.

We summarise the academic joint publications of Bletchley Park alumni in
diagrammatic form

HiltonWylieGreenReesWhiteheadTuringNewman Figure 2. Bletchley Park co-
authoring graph

##  5\. Summary

We have provided evidence that Bletchley Park experiences of mathematicians
went on to significantly affect their subsequent researches and publications
and, as a result, shaped the direction of post-war mathematics in the United
Kingdom. The effects are most prominent in Jack Good’s promotion of Bayesian
statistics based on his experiences with techniques used in cryptanalysis and
the encouragement of his colleagues, in early computational number theory
results by Turing, Good and Atkin, and in the continuation of associations
begun at Bletchley Park within academia.

## References

  * [1] Arthur Oliver Lonsdale Atkin, Bryan John Birch, et al.  Computers in Number Theory.  In Science Research Council Atlas Symposium 1969: Oxford). New York, Academic Press, 1971.
  * [2] D. L. Banks.  A conversation with I. J. Good.  Statistical Science, 11(1):1–19, 1996.
  * [3] Mavis Batey.  Dilly: The man who broke Enigmas.  Biteback Publishing, 2017.
  * [4] Bryan Birch.  A Tribute to Oliver Atkin.  Workshop on Elliptic Curves and Computation, 2010.
  * [5] J. W. S. Cassels.  Computer-aided serendipity.  Rendiconti del Seminario Matematico della Università di Padova, 93:187–197, 1995.
  * [6] Bruce W. Christopher.  https://www.bwchristopher.com/Family1/ps07/ps07_024.html.
  * [7] A. H. Copeland.  quoted in https://mathshistory.st-andrews.ac.uk/Biographies/Good/.
  * [8] F. N. David.  Probability and the Weighing of Evidence.  Biometrika, 38:485, 1951.
  * [9] Ralph Erskine and Michael Smith.  Action This Day.  Transworld Publishers, 2001.
  * [10] R. D. Farley and H. F. Schorreck.  Oral History Interview NSA OH-17-82.  1982\.
  * [11] I. J. Good.  Probability and the Weighing of Evidence.  Charles Griffin, 1950.
  * [12] I. J. Good.  Rational decisions.  Journal of the Royal Statistical Society: Series B (Methodological), 14(1):107–114, 1952.
  * [13] I. J. Good.  Studies in the history of probability and statistics. XXXVII AM Turing’s statistical work in World War II.  Biometrika, pages 393–396, 1979.
  * [14] I. J. Good.  The contributions of Jeffreys to Bayesian statistics.  Bayesian analysis in econometrics and statistics: Essays in honor of Harold Jeffreys, pages 21–34, 1980.
  * [15] I. J. Good.  Good thinking: The foundations of probability and its applications.  U of Minnesota Press, 1983.
  * [16] I. J. Good.  The Bayes/non-Bayes compromise: A brief review.  Journal of the American Statistical Association, 87(419):597–606, 1992.
  * [17] I. J. Good, I. Hacking, R. C. Jeffrey, and H. Törnebohm.  The estimation of probabilities: An essay on modern Bayesian methods.  Synthese, 16(2), 1966.
  * [18] Irving J Good and RF Churchhouse.  The Riemann hypothesis and pseudorandom features of the Möbius sequence.  Mathematics of Computation, 22(104):857–861, 1968.
  * [19] James A Green and David Rees.  On semi-groups in which $xr=x$.  In Mathematical Proceedings of the Cambridge Philosophical Society, volume 48, pages 35–40. Cambridge University Press, 1952.
  * [20] James Alexander Green, J. E. Roseblade, and John Griggs Thompson.  Philip Hall, 11 April 1904 - 30 December 1982.  Biographical Memoirs of Fellows of the Royal Society, 30:250–279, 1984.
  * [21] Sandy Green.  Autobiography for the Royal Society.  2004\.
  * [22] Peter J Hilton and David Rees.  Natural maps of extension functors and a theorem of RG Swan.  In Mathematical proceedings of the Cambridge philosophical society, volume 57, pages 489–502. Cambridge University Press, 1961.
  * [23] Peter John Hilton and Shaun Wylie.  Homology theory: An introduction to algebraic topology.  CUP Archive, 1967.
  * [24] PJ Hilton and JHC Whitehead.  Note on the Whitehead product.  Annals of Mathematics, pages 429–442, 1953.
  * [25] Francis Harry Hinsley and Alan Stripp.  Codebreakers: the inside story of Bletchley Park.  Oxford University Press, USA, 2001.
  * [26] Kerry Howard.  Dear Code Breaker: The Letters of Margaret Rock (Bletchley Park Code Breaker) & John Rock (Parachute and Glider Forces Pioneer).  BookTower Publishing, 2013.
  * [27] MHA Newman and Alan M. Turing.  A formal theorem in Church’s theory of types.  The Journal of Symbolic Logic, 7(1):28–33, 1942.
  * [28] Andrew M Odlyzko and HJJ te Riele.  Disproof of the Mertens conjecture.  1985\.
  * [29] G. B. Preston.  Personal reminiscences of the early history of semigroups.  Monash Conference on Semigroup Theory, Melbourne, pages 16–30, 1990\.
  * [30] J. Roseblade.  Obituary: Derek Taunt.  Jesus College Cambridge Annual Report, 2004.
  * [31] Bob Roseveare.  Roseveare family history 1440-2005.  Brinkmann pub., 2005.
  * [32] E. H. Simpson.  Edward Simpson: Bayes at Bletchley Park.  Significance, 7(2):76–80, 2010.
  * [33] L. A. Steen and Alexanders G. L.  Mathematical People, chapter Peter Hilton, pages 133–150.  Birkhauser, 1985.
  * [34] The Daily Telegraph.  Obituary: Keith Batey, September 2nd, 2010.
  * [35] The Times.  Professor Jack Good: mathematician and wartime codebreaker , April 16th, 2009.
  * [36] Alan Turing, Richard Braithwaite, Geoffrey Jefferson, and Max Newman.  Can automatic calculating machines be said to think?(1952).  B. Jack Copeland, page 487, 2004.
  * [37] Alan M Turing.  A Method for the Calculation of the Zeta-Function.  Proceedings of the London Mathematical Society, 2(1):180–197, 1945\.
  * [38] Alan M Turing.  Some calculations of the Riemann zeta-function.  Proceedings of the London Mathematical Society, 3(1):99–117, 1953\.
  * [39] Gordon Welchman.  The hut six story: breaking the enigma codes.  M&M Baldwin, 1982.
  * [40] D. H. Younger.  William Thomas Tutte. 14 May 1917–2 May 2002.  Biogr. Mems. Fell. R. Soc., 58:283–297, 2012.
  * [41] S. Zabell.  Breaking Teleprinter Ciphers at Bletchley Park: An edition of I.J. Good, D. Michie and G. Timms: General Report on Tunny with Emphasis on Statistical Methods (1945), chapter Statistics at Bletchley Park, pages lxxv–ci.  Wiley, 2015.
  * [42] S. Zabell.  The Secret Life of I. J. Good.  Statistical Science, 38(2):285 – 302, 2023.

##  Appendix A List of mathematicians at Bletchley Park

In the course of researching this paper, various sources have listed names of
mathematicians at Bletchley Park that do not appear on the most easily
accessed lists. The definition of mathematician is problematic in this case.
Some of the younger mathematicians had barely begun their undergraduate
careers (Bill Tutte had formally transferred to graduate study of mathematics
only a few months earlier). Several people recruited due to their mathematics
background remained at GCHQ and never returned to their studies. The PhD was
not yet a _de facto_ requirement for an academic career and some notable names
went on to academic careers without this qualification (David Rees for
example). The following then is a list of people at Bletchley Park that can be
confirmed to have enrolled in some university mathematics course before their
classified work.

  * •

Hugh Alexander

  * •

Michael Ashcroft [29]

  * •

A. O. L. (Oliver) Atkin

  * •

Dennis Babbage

  * •

Keith Batey [34]

  * •

Howard Campaigne (US) [29]

  * •

J. W. S. (Ian) Cassels

  * •

Michael Chamberlain [39]

  * •

L. N. (John) Chown [6]

  * •

Joan Clarke

  * •

Michael Crum [29]

  * •

Harold Fletcher [39]

  * •

Joe Gillis [29]

  * •

I. J. (Jack) Good

  * •

J. A. (Sandy) Green

  * •

Philip Hall

  * •

John Herivel

  * •

Peter Hilton

  * •

Walter Jacobs (US) [25]

  * •

John Jeffreys

  * •

Solomon Kullback (US)

  * •

Kenneth LeCouteur [25]

  * •

Tim Molien [25]

  * •

Max Newman

  * •

Rolf Noskwith [9]

  * •

Gordon Preston

  * •

David Rees

  * •

Margaret Rock [26]

  * •

Bob Roseveare [31]

  * •

Bernard Scott [2]

  * •

Edward Simpson

  * •

Abraham Sinkov (US)

  * •

Derek Taunt

  * •

Alan Turing

  * •

Bill Tutte

  * •

Peter Twinn [3]

  * •

Houston Wallace [39]

  * •

Philip Watson [29]

  * •

Gordon Welchmann

  * •

J. H. C. (Henry) Whitehead

  * •

Jimmy Whitworth [32]

  * •

Shaun Wylie

  * •

Leslie Yoxall


11institutetext: Software Engineering Research Group, Computer Science, Lund
University
<EMAIL_ADDRESS>
Authors’ Instructions

# Investigating applicability of inner source on small development teams

Maria Krantz  Johan Linåker  Martin Höst

###### Abstract

The phenomenon of adopting open source software development practices in a
corporate environment is known by many names, one being inner source. The
objective of this study is to investigate how an organization consisting of
small development teams can benefit from adopting inner source and assess the
level of applicability. The research has been conducted as a case study at a
software development company. Data collection was carried out through
interviews and a series of focus group meetings, and then analysed by mapping
it to an available framework. The analysis shows that the organization
possesses potential, and also identified a number of challenges and benefits
of special importance to the case company. To address these challenges, the
case study synthesized the organizational and infrastructural needs of the
organization in a requirements specification, describing a technical
infrastructure, and a suitable organizational context and work process.

###### Keywords:

Inner Source, Life cycle, Programming teams, Software process models, Reusable
software

##  1 Introduction

Many open source software products have been successful in recent years, which
have led to an increased interest from the industry to investigate how the
development practices could be introduced in a corporate environment and take
advantage of the benefits seen in open source projects. Such practises include
e.g. universal access to project artefacts [8], early and frequent releases,
and “community” peer-review [3].

Mistrik et al. [10] address how closed development organizations could benefit
from open source practices as an area where further research is needed. Though
studies conducted so far are quite limited, several success stories [20], [3],
[1], [8], [12] can be found of large corporations adopting open source
development.

The phenomenon of adopting these development practices in a corporate
environment has in research been called _inner source_ [18], [2], _community
source_ [19], _corporate open source_ [3], [4] and _progressive open source_
[1]. In this report we have chosen use the term inner source, as described by
Stol et al. [18].

The changes required when adopting inner source in a corporate environment led
Gurbani et al. [4] to suggest two different methods to effectively manage
inner source assets; an infrastructure-based model and a project-based model.

In the infrastructure-based model, the corporation provides the critical
infrastructure that allows interested developers to host individual software
projects on the infrastructure, much like
SourceForge111http://www.sourceforge.com/ or Github222http://github.com does
with open source projects. Platforms like these, also known as forges [12],can
be resembled as a component libraries where each project represents a
component of different abstractions, e.g. modules, frameworks or executables.
Developers can browse between the components and use or contribute to those
they wish. The reuse of software can be considered opportunistic or ad hoc and
there is no limitation on the number of projects to be shared within the
organization. Success stories include cases from SAP [12], IBM [15], HP [1]
[Melian08] and Nokia [8].

In the project-based approach the software is managed in a project, instead of
as a long-term infrastructure. Gurbani et al. [4] describe how an advanced
technology group, or a research group funded by other business divisions in a
corporation takes over a critical resource and makes it available across the
organization. This team is often referred to as the ”core team” and is
responsible for the project and the decision making. Philips Healthcare [20]
and Alcatel-Lucent [4] are two documented cases where this variant has been
adapted.

In order to asses the applicability of inner source on an organization, Stol
[18] developed a framework. This framework is based on reviewed literature and
a case study of a software company referred to as “newCorp”. Though the
framework focuses on project-based models, it is based on success factors and
guidelines described in both project-based and infrastructure-based case
studies. The framework consists of 17 elements divided into four categories;
Software product, Development practices, Tools and infrastructure, and
Organization and community. The elements can be found in the left column of
Table 1.

Adopting inner source requires significant effort and change management, which
is one reason why it may be of interest to start on a smaller scale before
investing globally. However, this requires an understanding of how inner
source can be implemented on smaller teams and what parts that can be
implemented and evaluated.

This study is focused towards the latter and aims to contribute theoretically
by using a new way of assessing the applicability of inner source, identifying
key benefits and challenges as well as synthesising a solution that addresses
the organizations needs.

The outline of this paper is as follows. In Section 2 the research methodology
is presented and the results are presented in Section 3. The results are
discussed and further analysed in Section 4, expected implications of an
introduction of inner source is also discussed. The validity of the conducted
research is presented in Section 5 and the research results are summarized in
Section 6.

##  2 Methodology

This research is of a problem-solving nature and conducted as a case study
with an exploratory strategy approach [14] [21] [13]. The case company is
experiencing problems in regards to its reuse of code and overall efficiency.
The hypothesis is that the concept of inner source, as described earlier, can
help the organization manage these issues.

The organization was observed in order to further define the problem. Then it
needed to be assessed whether inner source would fit the organizations or not,
and what the challenges would be. Based on the findings, the parts of inner
source suitable for the organizations needs were synthesized in a requirement
specification describing a technical infrastructure together with an
organizational context and work process.

This improving approach can be compared to that of action research. However,
this study has focused on the initial parts and proposed a solution. This is
yet to be implemented and evaluated. I.e. the complete change process is yet
to be observed. Due to limitations in time for the researchers and
organizational conditions of the case company, this is left for future
research.

###  2.1 The case company

An international software development firm, hereby known as “the case
company”, has been chosen for this study. The case company has a division
based in a local office in Sweden which specializes in rapid software
development and deployment of projects where the customers seek a combination
of high quality and a fast release.

The scope is limited to the local division, though a network of corresponding
divisions is established globally. The division of interest is divided into
two teams with similar set-up and structure. Each team consists of 20-25
engineers including developers, testers and project managers.

Reuse today within the division is insufficient, which was one reason that the
organization was interested in the topic. Knowledge of modules and functions
developed within the projects are spread orally and physically in an
unstructured manner which causes redundant work and loss of information.

###  2.2 Case study steps

The data collection and analysis was carried out as outlined in Figure 1, and
described below.

Figure 1: Overview of data collection and analysis

###  2.3 Situation analysis

A situation analysis was conducted with the objective to describe and somewhat
explain the current situation at the company. The main goal was to gain an
understanding of how work is conducted within the organization and can be seen
as an observational part of the research.

Qualitative data was collected by studying documentation at the case company
and by interviewing a sample group. The criteria for selecting people in this
phase were that they should: there should be representatives from different
areas, with different work tasks, to get a good variation of answers; have
some work experience, within this or other companies; and be available for
interviews.

In total 9 individuals were interviewed which included 4 project
managers/technical leaders/senior back-end developers, 2 senior front-end
developer, 1 junior front-end developer, 1 junior back-end developer, and 1
service manager.

The interviews were semi-structured with about 20 questions prepared in
advance. The interviews covered areas such as the experience, roles and
responsibilities of the interviewee, the interviewees’ view on how work is
carried out today, the interviewees’ opinion about the ideas of inner source
at the case company, and the interviewees’ previous experience of new
technology introduction. The opinion about the possibility of introducing
inner source at the case company was of course an important part of the
interviews.

All interviews were recorded with the permission of the participants. During
the interviews, one of the authors acted as an observer, focusing on taking
supporting notes. The notes were clarified directly after the interviews and
written down in an interview summary.

The interview data was then analysed using an editorial approach (e.g. [14]),
meaning that the categories and statements for characterizing the reasoning in
the interviews were not to a large extent predefined.

The framework presented by Stol [16], with some modifications to suit the
company, were then used to evaluate the compatibility of the company to adopt
inner source.

###  2.4 Specification of requirements on technical and practical support

To define the technical infrastructure, related context and practices, a
requirement specification was chosen because it is a natural approach within
the software industry to describe a desired solution. The requirements
specification is not presented here, although an overview of the domain can be
found in Section 3.2, and more information is provided in [6].

Several methods were used in order to elicit requirements from all levels of
interest, e.g. stakeholder analysis, additional interviews, and a series of
focus groups.

####  2.4.1 Stakeholder analysis.

A stakeholder analysis (e.g. [7]) was used to map all of the stakeholders and
elicit their different areas of interest. It is important that everyone with a
stake in the product gets to contribute their view, goals and wishes
concerning both functional and non-functional requirements in order for the
final product to get a corporate wide approval.

Stakeholders were identified amongst developers, project- and service
managers, team managers and corporate representatives. The analysis was based
on material from the interviews held in the situation analysis, complemented
by the focus group meetings and a longer interview with the case companys’
former CTO.

####  2.4.2 Focus groups.

As it became clear early on in the case study that stakeholders had different
opinions and priorities, this technique was considered appropriate. The
incentive was to create an understanding between stakeholders in addition to
identify problems and gather ideas and opinions in a structured manner [7].
The other objective of the focus groups [5] was to elicit requirements for a
substantial part for the proposed solution. Three areas with different themes
were therefore identified on which the focus groups were based upon: Reuse of
code and knowledge; Tools and functionality; Time, sales strategy and
incentives.

With these themes the authors regarded to have covered all relevant aspects of
the product. Several subtopics were then identified around which the
discussions were held.

The focus groups were carried out in 1.5 hour sessions. Focus groups 1 and 2
were both split into two sessions, while focus group 3 was carried out in one
single session. The sessions were moderated by one of the two first authors,
whilst the other documented by audio recording and taking supporting notes.

Each session had a brief list of subtopics where participants were allowed to
briefly describe bad experiences and focus more on the ideal usage and
functions. Post-it notes were used by the participants to record their
opinions, where considered appropriate by the authors. These were then
collected by the moderator and presented for a joint discussion. The
discussions also included different aspects of risk, cost and benefits of the
proposals. Where different opinions were present, a collective prioritization
of the ideas was conducted and motivation to the priorities encouraged by the
moderator. The sessions concluded with a summary by the moderator.

##  3 Results

In this section the results are presented.

###  3.1 Situation analysis

As recognized before, reuse today within the division is seen as insufficient.
It mainly happens by a ”mouth-to-mouth” spread of what has been developed
before and where it can be found. That is, there is not sufficient
systematized knowledge available on what software is available for reuse.
Certain modules and functions, which are commonly used, risk being re-
developed. For example, one interviewee stated

> ”We could benefit a lot from having our own demo site or basic platform,
> including common modules, that projects can be based on.”

and

> ”I use standard modules that are needed in projects that I sometimes know
> that someone else has done in another project or that I have done myself in
> another project and thereby I can use it. In other cases, we are not aware
> of it. Especially when a new developer enters [a project] who has not been
> around for so long and do not know what is available.”

This also raises potential for a common framework that can be used as a
standard template in many of the projects. An apparent need for a platform
facilitating reuse exist, since redundant work is conducted. Some solutions
are however considered too customer specific in order to be reusable in other
projects. Two other important aspects are time and budget. These two factors
are tightly knit together. The time set for documentation is seldom used for
this specific purpose. Transfer of knowledge in general is a subject that
needs to be incorporated in the day-to-day work process in every project.
There is little or no time between projects for project feedback and knowledge
transfer. Time estimations are tight in order to win customer deals and
chargeable coverage is of high priority, leaving limited time for internal
improvements.

Concerning standardized tools, a common set of collaborative tools are in
place, which also is positive from an inner source perspective, including an
application lifecycle management tool
(TeamForge333http://www.collab.net/products/teamforge) which is under
evaluation. There is also an open discussions ongoing in the case company, and
willingness to change exist, even if time is a restraining factor for internal
improvements. It is possible to identify classes and functions for reuse, but
the extraction may be very time consuming. Concerning maintenance, there are
little or no time between projects and development projects are generally
transferred to maintenance projects after acceptance test.

There are some aspects that require extra effort if the case company choose to
work with inner source. Modularization of code is needed, which may require
training. Also, requirements are project-specific and are mainly set at the
start of the project, but also constantly evolving in each sprint. However,
the most important issue is probably about code ownership. The customer is the
owner of the code, which may result in constraints on what can be reused.
Another issue is related to communication. Developers sit closely together and
are unlikely to benefit from “open” communication. Communication with
customers is desired to be closer and steered away from e-mailing.

Table 1: Summary chart of findings from interviews in relation to inner source
practises. Elements based on framework by Stol [16]

Element |  Findings from interviews
---|---
Software product
Runnable software |  Classes and functions can be identified from previous projects, but the extraction may be very time consuming.
Needed by several project groups |  There is potential for a common framework that can be used in several projects. An apparent need for a platform facilitating reuse exist, since redundant work is conducted.
Maturity state of the software |  Constantly evolving techniques and modules for customer-specific solutions.
Utility vs simplicity |  Some solutions may be too specific for the project in order to reuse.
Modularity |  Modularization of code is needed, which may require training.
Development practices
Requirement elicitation |  Requirements are project-specific and are mainly set at the start of the project, but also constantly evolving in each sprint.
Implementation and quality control |  Agile, sprint-driven development, planned per sprint. The level of competence in and knowledge of the process used varies. Senior developers review junior developers informally. Because of insufficient unit testing, quality can sometimes be an issue. Testing is to some extent ”bazaar-like” and peer-testing is performed as much as possible.
Release management |  Frequent releases, after each sprint. Customers are provided with prototypes.
Maintenance |  Little or no time between projects. Development projects are generally transferred to maintenance projects after acceptance test.
Tools & Infrastructure
Standardized tools |  Common set of collaborative tools are in place, though older projects remain using older tools. Freedom to select tools locally.
Infrastructure for open access |  Projects are archived in a traditional folder structure. A project platform for distributed development has been initiated and is under evaluation.
Organisation & Community
Work coordination |  Developers are assigned tasks. In order to control that the correct tasks are prioritized, developers may switch between projects. Better overview desirable.
Communication |  Developers sit closely together and are unlikely to benefit from ”open” communication. Communication with customers is desired to be closer and steered away from e-mailing.
Leadership and decision making |  Discussions are considered open and inputs appreciated. Evangelists and/or core team needed to take responsibility for a common framework.
Motivation and incentives |  A lack of time is the biggest concern. Attractiveness of the tool also considered a critical success factor.
Open culture |  Open discussions and willingness to change exist, though time a restraining factor for internal improvements.
Management support |  Budget constraints a concern. Chargeable occupancy important. Plan for incorporating activities related to reuse in the sales strategy needed.
Additional factors
Project feedback and knowledge sharing |  More feedback on a division level desirable to improve knowledge sharing across projects.
Project initiation |  Dependent on a few individuals because of expertise needed to set up projects.
Code ownership |  The customer is the owner of the code, which may result in constraints on what can be reused.

###  3.2 Overview of technical infrastructure

This section presents the results from the specification of requirements and
the definition of technical support for the case company. The domain for the
technical solution consists of the forge, the users of the forge, and the
system administrator within the studied division of the case company. The
systems for the running customer projects, as well as the documentation of old
projects, are outside the domain. The forge includes a component library with
a component project view for each shared component, see Figure 2.

Figure 2: Context Diagram

####  3.2.1 Components.

All components are stored in a component library. A search function allows the
user to find a component of interest. There are two types of components that
can be shared, project-based components and open components.

Project-based components require administration by a core team that is
responsible for the maintenance and development of the component. All users
can access the components but the core team may restrict the user’s rights to
make any changes. Different types of components that have been identified as
project-based components are Product, Framework and Templates.

  * •

Product: A basic solution that is ready to be sold to the customer, or to be
customized. This type is a long term idea and though the forge provides the
infrastructure to share this component, special requirements to support this
type of component will not be further considered in this project.

  * •

Framework: A framework for the most commonly used modules. The framework would
be used to have an initial set of modules that are reusable and can easily be
implemented in a new project. A core team decides what goes into the framework
and makes sure the framework is up to date and of the required quality.

  * •

Templates: Documentation templates developed by a core team in order to
facilitate the documentation process.

Open components do not need any administration or anyone responsible for the
development of the component. No quality or generalization requirements exist
to share these components and the creator is not responsible for any
maintenance or further development. Modules and Classes, and Knowledge Base
are types of components identified as open components.

  * •

Modules and classes: Modules and classes that have been used in projects. The
size and complexity of these components may vary and may be more or less
suitable to reuse.

  * •

Knowledge bas: User guides for commonly performed tasks, tutorials, lessons
learned from previous projects, common errors etc. that could be helpful in
future projects.

####  3.2.2 Users.

The identified user types are all employees with access to the forge. They can
be divided into three groups:

  * •

General user: Developers reusing and contributing to components. The general
users have full rights to open components and limited rights to project-based
components.

  * •

Core team member: Project-based components have at least one core team member.
The core team members have full access to its components and are responsible
for maintenance, support and further development of that component.

  * •

System administrator: The system administrators is responsible for the
technical support and maintenance of the forge.

####  3.2.3 Component project view.

The component project view is unique and scalable, dependent of whether the
component is open or project-based. Generally, users should be able to browse
between the different versions of the component and read what changes that has
been made by whom. Once the user has chosen a version of the component, a
snapshot should be available for download. The user can then start
investigating the component and see if it matches his expectations and
requirements. If the user wishes to contribute, e.g report a bug or suggest a
new feature this should be possible through an issue tracker or a by
submitting a comment. The user can also choose to make the change by him or
herself by accessing the code through the configuration management interface.

For project-based components specifically, it is up to the core team members
to decide and configure how they want the component to be managed and accessed
to other users. The creator of the component is automatically assigned the
role as a core team member. This individual can add additional users to the
core team when appropriate. The component can be free for everyone to see and
download or may be restricted to an individual or groups of users. This is to
allow different levels of core team responsibility depending on the
criticality of the component.

Depending on the core team’s preferences and the conditions of the component,
different processes can be used. A general option in open source development
and closely linked to project-based inner source is that users have free
access to the communal component project view where they can share
information, communicate and access the repository but with reading rights
exclusively. They are free to make change/feature requests and bug reports,
but all changes and features made by themselves are sent in as deltas via the
configuration management interface. These deltas can then be reviewed by the
core team and either be sent back or implemented into the component
repository. With this way of working, core team members have full control of
what goes into the component and that re-factorizations needed are done
properly.

###  3.3 Work process

The level to which inner source is adopted depends on the organization and can
be done in various ways. This Section aims to provide an organizational
context to the technical solution and discuss how to adopt inner source
practices within the studied division, based on the situation analysis, the
requirement elicitation and findings from the literature.

####  3.3.1 Development practices.

As described in Section 3.2, the domain does not include the customer
projects. Hence, the introduction of inner source, as proposed here, will have
a low impact on the development practices used. Collaborative development is
used to some extent and under improvement with the introduction of TeamForge
(TF), which would benefit the implementation process of the forge.

It was revealed in the situation analysis, that the quality of the code is an
issue from time to time. Hence, it is relevant to take advantage of the
quality benefits associated with inner source. Clear visualization of ratings
and issues as well as test- and review results for each components is
therefore an important aspect. That way, quality can be improved both directly
on the specific component and indirectly by allowing developers to gain skills
through the identification of errors. A drawback of ratings, anticipated by
the team, is that it can be misleading. Users who have tested or reviewed the
component may have done this to different extents why their conception of the
component may vary. This is why additional information such as tagging,
rating, comments and descriptions are considered important so that whole
experiences are reflected and potential users can get a fair comprehension of
the component.

Alignment between the different projects is important to the team. A framework
is considered to improve alignment of the development from project to project
through a communal set of components, optimizing the initiation process and
facilitating for developers to enter a new project.

####  3.3.2 Organization and community.

To complement the technical aspect of adopting inner source in the division,
organizational aspects as well as community building are discussed in this
section. Though the general organizational structure does not need to change,
some effort on all levels is required to adapt to the new conditions and
create business value from the initiative.

With the solution proposed, a volunteer approach to the work coordination
related to open and project-based components will be used, in contrast to
assigned tasks. To encourage contributions, sharing open components does not
require any further responsibility from the creator, nor a leadership or
decision making structure. The motivation is to limit the responsibility of
the creator, decrease dependency on individual developers and enable assets to
be highly dynamic.

Regarding the project-based components, there is a need for coordination and
management since these components are of a more business critical character.
Depending on the complexity of the component, the amount of resources needed
by the core team may vary. For a critical asset such as a framework, the core
team members need to have deep technical knowledge as well as an understanding
of the business- and delivery models.

An incentive and motivational structure was identified in literature as
essential for users to be attracted to the forge. From a management
perspective, there is a wish to acknowledge the competent developer and the
platform could be used as a tool for doing so. However, it is required to put
some thought into what is being measured, to prevent that rewards, if any, are
not misleading, nor cause negative implications for contributing.
Additionally, it demands of the technical solution to provide these measures
on an individual developer level in a simple manner for the manager to
extract.

The individual developer may be motivated to contribute by the rewards and
acknowledgement of management, but motivation is also a highly cultural matter
that needs to be incorporated in the working environment. Developers as well
as managers and technical leaders, encouraging each other to contribute and to
use the forge, foster awareness and integration of the solution in the day-to-
day work. The need for an “evangelist” has been described in both literature
[1] [15] and the situation analysis, Section 3.1 as essential for the success
of an inner source initiative. This important role is hence to be chosen
carefully and early on in the initiation process in order to push the projects
forward.

As described by Wesselius, [20], one of the limiting external factors of inner
source development is overall profitability. That is, that the group should
not optimize its own profits at the expense of the company’s total
profitability. By constantly retaining an awareness of what exists on the
forge and the quality of it, individuals with responsibility for sales can
adapt their estimates to presumptive customers. This cross-divisional
dependency calls for a communication and discussion between the different
internal stakeholders. Planning and development of the assets on the forge is
of communal interest since it benefits the whole company.

##  4 Expected implications

It is found that the investigated case has good potential for adopting inner
source. The division has plenty to gain by adopting the open source practises
on which inner source is based, e.g.

  * •

Improved reuse of code and solutions to complex problems [8], [1], [15], [3],
[4], [20], [16] [11]

  * •

Improved quality of code and general level of knowledge amongst developers
[12], [15], [9], [3], [4]

  * •

Creation of a framework to standardize and shorten initiation process of new
projects [1], [9]

  * •

Better visibility and spread of information and knowledge [8], [1], [15],
[16], [11]

  * •

Higher margins for tender processes [9], [3] [4], [20], [16], [11]

Emphasis should be on these benefits in the planning and evaluation processes
to keep focus on what the organization want to achieve with the system.

In order to receive these benefits and prosper from them, several challenges
has to be overcome. Stol et al. [18] have identified numerous challenges, and
there are some of special concern to the case company, e.g. uncertainty about
the quality of a component, awareness of the library content, finding the
right component, motivating developers to contribute, commitment in terms of
devoting time for the developers to contribute, and costs of creating modular
and generic components.

##  5 Validity

The interviews were analysed based on the compatibility framework developed by
Stol in [16]. The framework has not been evaluated before by others than the
author, but was developed based on an extensive literature study and case
study within the subject performed by Stol et al. [17] [18]. For this case
study it was considered a valuable tool for structuring the findings from the
interviews from an inner source perspective.

Since this is an individual case study, it can not be established if the
technical solution and recommendations are applicable to other case companies.
Additionally, the solution proposed has yet not been implemented, nor
evaluated. The authors consider the solution to be an option for similar size
of development teams, being a small company or part of a larger company that
want to experiment with the adoption of inner source on a smaller scale before
making significant investments. Evaluation of the solution would be needed in
order to investigate what challenges the solution truly impose as well as the
benefits similar organizations can expect to gain.

The main measures taken to improve the validity (e.g. [14]) in the case study
can be summarized as follows.

_Prolonged involvement_ was achieved since the two main authors spend most of
their working time at the premises of the case company during a time period of
about 4 months. The fact that the researches spent so much time there and that
they were able to build a network of interested engineers in the organization
made it possible to have relevant discussions about the results, and also to
get access to required data.

_Peer debriefing_ , meaning that fellow researchers comment on the results was
achieved by having the third author reviewing the findings and research
methodology during the research without being actively involved in the day-to-
day data collection.

_Member checking_ , in this case meaning that engineers at the case company
can review findings were achieved by having regular discussions about the
results with the members. In particular there were one person who acted as
contact person and main discussant part during the study.

_Audit trails_ were achieved by recording all interviews and taking extensive
notes during data collection phases.

##  6 Conclusion

Several potential benefits and expected challenges for the case company have
been identified. One of the main advantages of inner source, as perceived by
the case company, is the possibilities it offers for reuse of code and other
artefacts.

In order to address the challenges seen in introducing inner source, this case
study has proposed a technical infrastructure, presented in form of a
requirement specification, together with an adapted working process and
organizational context. The infrastructure forms a collaborative platform
where knowledge and code can be shared in form of components, and where people
can interact according to the principles of inner source.

Two types of components were identified in order to address the types of data
which the case company wishes to share internally. Project-based components
which are, to some extent, to be seen as business critical and demands
supervision by a core team. This can be related to the concepts of project-
based inner source as identified by Gurbani et al. [4]. The other type, open
components, relates in some parts to the concept of infrastructural inner
source. It can also be compared to a combination of a knowledge base and a
code snippet library. This type of component can in general be anything of
general interest and creator is not obligated to any support or maintenance of
it.

During this research the framework of Stol [16] was used as a framework in the
analysis. We can conclude that this framework was useful for us during this
purpose, and we believe that it includes relevant factors. It can also be
noticed that the concept of project-based and open components [4] is useful
when formulating the support needed in the organization.

The division studied within the case company possesses potential for the
application of inner source and if successfully applied, it can bring several
rewards to the organisation by optimizing its resources. An eventual future
implementation is yet to be studied.

Since this study has focused on one case, it cannot be generalized to other
organization by default. Many of the findings though, can be of value to other
cases where smaller teams and organizations are investigating the
opportunities to introduce inner source, which is an area for future research.

#### Acknowledgments.

The authors would like to express their gratitude to all the people at the
case company for supporting the research and participating in the study. This
work was partly funded by the Industrial Excellence Center EASE - Embedded
Applications Software Engineering, (http://ease.cs.lth.se).

## References

  * [1] Jamie Dinkelacker, Pankaj K. Garg, Rob Miller, and Dean Nelson.  Progressive open source.  In ICSE ’02: Proceedings of the 24th International Conference on Software Engineering, pages 177–184, New York, NY, USA, 2002. ACM Press.
  * [2] Gary Gaughan, Brian Fitzgerald, and Maha Shaikh.  An examination of the use of open source software processes as a global software development solution for commercial software engineering.  In EUROMICRO - Software Engineering and Advanced Applications (SEAA), pages 20–27, 2009.
  * [3] Vijay K. Gurbani, Anita Garvert, and James D. Herbsleb.  A case study of a corporate open source development model.  In ICSE ’06: Proceedings of the 28th International Conference on Software Engineering, pages 472–481, Shanghai, China, 2006.
  * [4] Vijay K. Gurbani, Anita Garvert, and James D. Herbsleb.  Managing a corporate open source software asset.  Community of the ACM (Association for computing machinery, 53(2):155–159, 2010.
  * [5] Jyrki Kontio, Laura Lehtola, and Johanna Bragge.  Using the focus group method in software engineering: Obtaining practitioner and user experiences.  In International Symposium on Empirical Software Engineering, pages 271–280, Redondo Beach, CA, USA, 2004.
  * [6] Maria Krantz and Johan Linåker.  Inner source: Application within small-sized development teams.  Master’s thesis, Lund University, 2012.
  * [7] Soren Lauesen.  Software requirements: styles and techniques.  Addison-Wesley, Pearson Education Limited, Harlow, England, 2002.
  * [8] Juho Lindman, Matti Rossi, and Pentti Marttiin.  Applying open source development practices inside a company.  In The 4th international conference on Open Source Systems, pages 381–387, Milan, Italy, 2008.
  * [9] Ken Martin and Bill Hoffman.  An open source approach to developing software in a small organization.  IEEE Software, 24(1):46–53, jan 2007.
  * [10] Ivan Mistrík, John Grundy, André Hoek, and Jim Whitehead.  Collaborative software engineering: Challenges and prospects.  In Ivan Mistrík, John Grundy, André Hoek, and Jim Whitehead, editors, Collaborative Software Engineering, pages 389–402. Springer Berlin Heidelberg, 2010.
  * [11] Lorraine Morgan, Joseph Feller, and Patrick Finnegan.  Exploring inner source as a form of intraorganisational open innovation.  In The 19th European Conference on Information Systems - ICT and sustainable service development, Helsinki, Finland, 2011.
  * [12] Dirk Riehle, John Ellenberger, Tamir Menahem, Boris Mikhailovski, Yuri Natchetoi, Barak Naveh, and Thomas Odenwald.  Open collaboration within corporations using software forges.  IEEE Software, 26(2):52–58, 2009.
  * [13] Colin Robson.  Real World Research.  Blackwell Publishers, 2002.
  * [14] Per Runeson, Martin Höst, Austen Rainer, and Björn Regnell.  Case Study Research in Software Engineering: Guidelines and Examples.  John Wiley & Sons, 2012.
  * [15] Danny Sabbah.  The open internet - open source, open standards and the effects on collaborative software developmen.  In 11th International workshop on High Performance Transaction Systems, Pacific Grove, CA, USA, 2005.
  * [16] Klaas-Jan Stol.  Supporting Product Development with Software from the Bazaar.  PhD thesis, University of Limerick, 2011.
  * [17] Klaas-Jan Stol and Muhammad Ali Babar.  Challenges in using open source software in product development: a review of the literature.  In Proceedings of the 3rd International Workshop on Emerging Trends in Free/Libre/Open Source Software Research and Development, FLOSS ’10, pages 17–22, New York, NY, USA, 2010. ACM.
  * [18] Klaas-Jan Stol, Muhammad Ali Babar, Paris Avgeriou, and Brian Fitzgerald.  A comparative study of challenges in integrating open source software and inner source software.  Information & Software Technology, 53(12):1319–1336, 2011.
  * [19] Darryl K Taft.  Community source development appeals in tough times.  [Mar. 15, 2012].
  * [20] Jacco H. Wesselius.  The bazaar inside the cathedral: Business models for internal markets.  IEEE Software, 25(3):60–66, 2008.
  * [21] Robert K. Yin.  Case study research: Design and methods.  Sage Publications Inc., 3rd edition, 2002.