# Breaking Free Transformer Models: Task-specific Context Attribution Promises
Improved Generalizability Without Fine-tuning Pre-trained LLMs

Stepan Tytarenko,
Fordham University, New York
<EMAIL_ADDRESS>Dr. Mohammad Ruhul Amin,
Fordham University, New York
<EMAIL_ADDRESS>
###### Abstract

> Fine-tuning large pre-trained language models (LLMs) on particular datasets
> is a commonly employed strategy in Natural Language Processing (NLP)
> classification tasks. However, this approach usually results in a loss of
> models’ generalizability. In this paper, we present a framework that allows
> for maintaining generalizability, and enhances the performance on the
> downstream task by utilizing task-specific context attribution. We show that
> a linear transformation of the text representation from any transformer
> model using the task-specific concept operator results in a projection onto
> the latent concept space, referred to as context attribution in this paper.
> The specific concept operator is optimized during the supervised learning
> stage via novel loss functions. The proposed framework demonstrates that
> context attribution of the text representation for each task objective can
> improve the capacity of the discriminator function and thus achieve better
> performance for the classification task. Experimental results on three
> datasets, namely HateXplain, IMDB reviews, and Social Media Attributions,
> illustrate that the proposed model attains superior accuracy and
> generalizability. Specifically, for the non-fine-tuned BERT on the
> HateXplain dataset, we observe 8% improvement in accuracy and 10%
> improvement in F1-score. Whereas for the IMDB dataset, fine-tuned state-of-
> the-art XLNet is outperformed by 1% for both accuracy and F1-score.
> Furthermore, in an out-of-domain cross-dataset test, DistilBERT fine-tuned
> on the IMDB dataset in conjunction with the proposed model improves the
> F1-score on the HateXplain dataset by 7%. For the Social Media Attributions
> dataset of YouTube comments, we observe 5.2% increase in F1-metric. The
> proposed framework is implemented with PyTorch and provided open-source on
> GitHub111https://github.com/StepanTita/space-model.

## Introduction

Currently, the domain of Language Models is one of the most rapidly developing
areas of machine learning. Transformer architecture (?) has proven itself as a
state-of-the-art approach towards the absolute majority of Natural Language
Processing (NLP) domains (?). A particular strength of the language models is
their generalizability (?), (?).
By pre-training the model on a big chunk of semi-structured data and then
fine-tuning with task-specific labeled data, we may obtain state-of-the-art
performance in the problems of classification, regression, language
translation, and more. However, the important note here is that the most
crucial pre-training stage is usually costly, and repeating it for every new
task is computationally inefficient. At the same time, the fine-tuning only
downstream task specific head of pre-trained models is time efficient and
requires much less labeled data, preserving models’ generalizability. On the
other hand, this part of the pipeline might be a bottleneck to the process.
Usually, single fine-tuning does not produce results on par with the complete
model adaptation via training (?). Explaining and adapting the results to the
various downstream tasks is also tricky. To avoid any confusion in this paper,
we are going to use term ”training” or ”model adaptation” referring to the
process of full model retraining (adapting all of the weights), while for the
head-only adaptation we are going to use term ”fine-tuning”. Fine-tuning is
usually easier, faster and requires less data.

As a potential solution to the problem, we propose a novel method of model
fine-tuning. We call this approach the Space Model. The whole idea is to
replace the classification head of the transformer model with a set of
conceptual operators, projecting the contextual embeddings of the model to the
set of concept spaces referred to as context attributions. The Space model is
an additional model framework that plays the role of the pipeline’s original
downstream task head. In this work, we limit ourselves to the review of the
classification capabilities of the proposed approach, but generally, this is
not a limitation to the technique in any way.

The model is designed in a way that a set of concepts describes different
classes; such a set is called the “context attribution”. It is worth noting
that we do not limit these concepts in terms of overlapping. Some context
attributions might overlap if that makes sense in terms of the problem solved.
This paper reviews one such task where overlapping context attributions are
entirely appropriate. What we would like to avoid is allowing multiple
concepts to converge to the same representation. For that type of
regularization, we introduce an additional loss called Intra-Space loss. Its
goal is to make sure concepts in the context attribution are disjoint.

As was stated previously, the Space Model is an external framework with a set
of operators on top of the transformer model. Generally speaking, this is not
limited to the transformer architecture either. Potentially, any technique
that can produce embeddings may be used as the Space model’s base model, such
as Word2Vec (?), Glove (?), or RNN (?), (?). Further in this paper, whenever
we refer to the base model, we mean the model that produces the embeddings for
the space model. Some of the base models tested in this paper include BERT
(?), DistilBERT (?), and XLNet (?).

The benchmarking and evaluation of the proposed solution are done with various
configurations of the base models and across multiple datasets. We test
performance for the specific task, fine-tuning the Space model for that
particular downstream task, and we also test the performance of the model on
the task that is related to the original semantically; however, it uses
different data. The baseline for the comparison is mainly the performance of
the original base model fine-tuned for the downstream task. During the
experiments, we prove that besides an evident performance boost, the Space
model also stabilizes the training process and generalizes better for the
semantically close tasks. We also prove that the space model can achieve a
significant performance boost even when using a smaller number of parameters
than the base model fine-tuned on the downstream task.

The datasets used for benchmarking are HateXplain (?), IMDB reviews sentiment
dataset (?), and Social Media Attributions dataset of YouTube comments,
related to Chennai water crisis (?). The main reason for choosing
corresponding datasets is that HateXplain is considered a very complex
dataset, with imbalanced data, and labels “offensive” and “hateful” are
conceptually very close. On the other hand, IMDB sentiment reviews are a
semantically close dataset to the former one and are reasonably easily
interpretable. Such a relation is essential since we would like to test the
generalizability of the proposed technique. Besides, in the Social Media
Attributions paper, the authors apply a very similar approach to the one
proposed in this paper, however, with additional manual labeling of the
concepts and multiple runs. We would like to show that our approach achieves
superior performance without additional manual labeling and via a single pass.

The impact and novelty of this paper include:

  * •

A novel framework for Language Model fine-tuning, which outperforms the
baseline score of the base models such as BERT, DistilBERT, and XLNet

    * –

For the non-pretrained BERT (only trained context attribution operator) we
observe 8% improvement in accuracy and 10% in F1-score on HateXplain data

    * –

For the IMDB with XLNet base model, we observe an improvement of around 1%
after full model adaptation compared to fully trained vanilla XLNet

    * –

Space-model with the base model as DistilBERT non-pretrained (only trained
context attribution operator) on IMDB dataset, in a zero-shot manner
outperforms basic DistilBERT in the same manner (only head fine-tuning) by 7%
on F1-score

    * –

Compared to the results from the Social Attribution paper (with manual
supervision), we observe an improvement of 5.2% with our model without
additional supervision

  * •

A novel loss function that improves the generalization and stabilization of
the training process, improving the zero-shot capacity of the transformers

## Related work

The task of effective fine-tuning is one of the main tasks in the modern NLP.
Cheaper and faster results of great quality are very appealing and a current
trend in the domain. However, we are sourcing the inspiration for our
framework not only from the latest NLP findings. The core idea has a root in a
Psychological Belief Attribution Theory (?), (?). The theory revolves around
the idea of attribution of certain concepts with corresponding behavior
patterns. The concepts (sometimes also referred to as factors) may be external
and internal. These factors are usually related to personal beliefs, and they
affect the decisions and behavior of an individual. Researchers have also
classified people based on these factors (e.g., pessimistic attribution,
optimistic attribution, hostile attribution). We try to apply the same idea to
language modeling, attributing certain concepts with class labels. In general,
the idea of measuring and researching the belief attribution of language
models is not novel. The authors of (?) have not only proved that certain
language models possess the beliefs, but they have also provided metrics to
measure such beliefs and a tool to update these beliefs, as well as
visualization of beliefs graph.

It is very natural that semi-supervised solutions are mentioned when it comes
to fine-tuning with the least resources. These also mainly include ensembling
to achieve regularization when working with unsupervised data. One of the
first such approaches addressing this issue is the COREG (?). The technique
uses two k-nearest-neighbor regressors with different distance metrics to
label the data. The distance metric, in that case, would serve as the
confidence label. This approach uses a fundamental idea that some features in
some spaces are aligned with similar class labels and are further apart from
the different class labels. This is an essential fact that is reused in the
Space model.

Another later technique involves minimal supervision for the labeling of the
concept space, and then, based on this concept space, the model can
autonomously label the unlabelled data (?). The key idea here is the knowledge
extraction from the manually labeled concept space. It is claimed in the work
that labeling a set of concepts and then running an algorithm on a set of
documents to label them based on these supervised concepts is a superior
technique. Our main takeaway from there is that we can extract knowledge from
the supervised concept space for unlabelled data. Furthermore, what we would
like to propose is testing if this concept space can help us make a prediction
at the inference stage rather than during labeling.

Social Media attributions in the Context of Water Crisis paper (?) is
accomplishing a task very close to the one we are dealing with. However,
unlike our approach, same as the previous one, their technique requires
supervised sub-concept labeling. Besides, they measure the similarities
between sub-concepts and the attention of the Language Model by feeding the
sentence to the model multiple times, each time with a new sub-concept.
However, they are using the similarity measure to find the concept sub-space
that best describes the given sentence and make the decision based on that.
Our approach does this all in one pass and in an automated manner. We do not
require manual labeling of the concept sub-spaces. We expect to learn them
during the fine-tuning phase.

In the paper on Interacting Conceptual Spaces (?), the authors create all of
the necessary mathematical background required to formulate the knowledge
extraction process from the concept space. They converge the optimization task
to the convex relations and prove that by means of optimizing the conceptual
space and merging multiple concepts (or even spaces) together, one can extract
new knowledge practical for the downstream task. They also provide an algebra
language on how concepts are organized and interact and what it means
mathematically when several concepts (or concept spaces) are combined. They
put the conceptual representations in different compacts and explore the
vectors’ behavior there. This is one of the ideas we are adopting in our
paper, which we believe helps regularize the network. The concept spaces are
encapsulated into a compact hypercube with the side 2. This is achieved due to
the utilization of the $tanh$ activation, which we will review in more detail
in the methodology section.

## Methodology

We are going to use the transformer architecture to extract the contextual
embeddings from the input text. However, the methodology is not limited to
transformers and may be reused with any architecture producing some kind of
embeddings. In this specific research, we are focusing on the BERT family
models (and some variations such as XLNet).

### Context Attribution

Context attribution - is a projection of a collection of contextual embeddings
(vectors) or, simply, a matrix. Projection is done via a concept operator.
When we train the model, we ensure that concept operators project disjoint
concepts far away from similar concepts. We project the sentence in multiple
context attributions and then find the similarity between the original
sentence and the conceptual projections. This similarity tells how to classify
the instance correctly. Note that in the actual implementation, we do not do
the pairwise comparisons of the similarities or any other type of processing.
Instead, we concatenate obtained projections into a single tensor and feed it
to the classification layer. Thus, instead of manually defining the
classification criteria, we specify it as a set of trainable parameters.

### Contextual word embeddings

As it was already stated, the only assumption we impose on the base model is
that it can create (contextual) embeddings from the input. Let $N_{s}$ be the
sequence length, $d$ be the dimensionality of the contextualized embedding of
the model. $E=[e_{1},e_{2},...,e_{N_{s}}]\in R^{N_{S}\times d}$,
$E_{N_{s}\times d}\in R^{N_{S}\times d}$ is the contexual embedding matrix.

In our research, we assume that the BERT-like models produce this embedding.
So $N_{s}$ would be defined in the range between 256 and 512 (as a maximum
sequence length used by the BERT architecture), and $d$ would be 768 for all
of the base models and 1024 for the XLNet large.

### Conceptual projections

For each of the classes in the classification problem, we assume a single
concept space operator. This concept space operator transforms (projects) the
contextual embeddings to the context attribution and produces new conceptual
embeddings. The obtained representation of the embeddings is also called a
latent representation. This representation’s dimensionality is defined as the
latent space (target space for the projection). Let $m$ be the dimensionality
of the latent space. We first define the projection operator as a matrix with
trainable parameters: $P_{d\times m}\in R^{d\times m}$. Thus obtained
projection matrix (context attribution) $C_{N_{s}\times m}\in R^{N_{s}\times
m}$.

Basically, context attribution is a new representation of the embeddings in
the latent space, where the transformation operator is trained during fine-
tuning. However, since we want to obtain proximity of the contextualized
embedding to the context attribution, we introduce previously defined $tanh$
operation as a similarity measure.

$C_{N_{s}\times m}=tanh(E_{N_{s}\times d}\times P_{d\times m})$ (1)

$tanh$ is applied element-wise. In that case, our conceptual matrix is a
representation of how close a certain sentence is to the concept from the
target context attribution (1 is very close, and -1 is from an orthogonal
attribution). As an example, when we feed the word “terrible” to the context
attribution that was predefined as “positive”, we expect to see -1 in the
conceptual representation and 1 for a word like “great”.

The training objective of the model is, by taking into account multiple
projections of the input embeddings, to find the projection that is most
aligned with the sentence content. The similarity measure we are using is a
slight modification of the cosine similarity, where normalizing the value by
the vectors’ norms is replaced with the $tanh$.

$tanh(x)=\frac{e^{x}-e^{-x}}{e^{x}+e^{-x}}$ (2)

Similarly to the original paper introducing LSTM (?) , we use $tanh$ to
control the flow of the information in the network. It squashes the range,
centers the values around zero, and introduces non-linearity. This has also
proven to be an excellent regularization technique, which reduces the
instability, improves models’ generalizability, and improves the results. This
aspect is discussed in more detail in the results section.

Figure 1: 3D projection of the space embeddings for the 2-class
classification. After projecting the sentence onto different concept spaces,
we expect these projections to be orthogonal if the classes are completely
divergent. For the case between positive and negative sentiment, we expect
that positive class projection would be orthogonal to the negative class
projection.

As a good side-effect of the $tanh$ we add additional non-linearity and
squashing effect to the model. Thus, no additional normalization of values is
required. Besides, we shrink our problem to the compact (hypercube with side
2, from -1 to 1). In Figure 1, one can find a benefit from such an approach.
We can now easily interpret the outcome of the binary classification model.
The visualization provided is the contextual embedding of the BERT model into
3-dimensional context attribution space for the IMDB classification task (this
is done for test examples, so the model is not overfitted, and what we clearly
see is the orthogonality of the negative and positive sentiment concepts).

According to our definition, every target class has a unique context
attribution for itself. So for $n$ classes classification problem:

$C_{N_{s}\times m}^{i}=tanh(E_{N_{s}\times d}\times P_{d\times m}^{i})$ (3)

for $i\leq n$. Where $C^{i}$ and $P^{i}$ are namely $i$-th context attribution
and concept projection operator.

### Classification

After we have projected the embeddings to all of the context attributions, we
need to perform classification. In that case, since every projection is a set
of vectors (where each vector is a conceptual embedding with latent size $m$)
we would find the centroid of this representation for each context attribution
and then concatenate these representations. This concatenated representation
is then fed to the single linear layer for classification. This basically
identifies the proximity of the embedding to the corresponding context
attribution. Let $k_{i}$ represent $i$-th context attribution centroid, and
$c_{i,j}$ $j$-th conceptual embedding vector ($j$-column) of the $i$-th
context attribution $C_{N_{s}\times m}^{i}$.

$k_{i}=\frac{1}{N_{s}}\cdot\sum_{j=0}^{N_{s}}c_{i,j}$ (4)

### Loss function

The loss that we are optimizing is primarily the Cross-Entropy loss. To ensure
that the conceptual embeddings don’t converge to the same embedding inside the
conceptual space, we introduce an intra-space loss. This also adds additional
regularization and improves generalization. This is proved during the
experiments. Controlling the weight of this loss compared to the cross entropy
loss is another hyperparameter fine-tuning task. The intra-space loss is
basically an inverse of the variance of the vectors inside the context
attribution.

$\sigma^{2}=\sum_{i=1}^{m}\frac{1}{m}\cdot(c_{i}-\hat{c})^{2}$ (5)

where $c_{i}$ is $i$-th embedding ($i$-th column of the context attribution
matrix) $C_{N_{s}\times m}$ and $\hat{c}$ is the mean vector of conceptual
embedding matrix (column-wise).

## Results

We evaluate our framework using 3 base models in 3 benchmarks with 3 different
datasets. With benchmarks, we want to measure:

  * •

Performance of the proposed Space Model

  * •

Generalization property of the novel technique

  * •

How does it compare with existing context attribution solutions which involve
a manual process

To achieve that, we are going to use 3 datasets, namely HateXplain, IMDB
reviews sentiment dataset, and Social Media Attributions dataset of YouTube
comments related to the Chennai water crisis. IMDB sentiment reviews is a
semantically close dataset to the HateXplain and is reasonably easily
interpretable. Such a relation is essential since we would like to test the
generalizability of the proposed technique. Besides, in the Social Media
Attributions paper, the authors do manual labeling of the concepts and measure
similarity with the so-called Social Media Attributions; we would like to show
that our approach achieves superior performance without additional manual
labeling and via a single pass.

The experiments are structured in a way that we have basic experiments with
smaller models and simpler tasks. Additionally, we conducted experiments to
compare the Space Model to the state-of-the-art model of the IMDB dataset. We
also investigate and analyze various properties of the Space Model and explore
some of the hyperparameters’ usage, with their respectful effect on the model
performance. We explore the generalization property of the model by cross-
testing it on the unseen dataset.

Dataset | Model | Train Params | Accuracy | F1-score (macro) | Recall | Intra-Space weight
---|---|---|---|---|---|---
IMDB (training) | DistilBERT | 592130 | 0.7852 | 0.7819 | 0.6614 | N/A
| Space Model | 197122 | 0.7917 | 0.7916 | 0.7728 | 0.001
| Space Model | 197122 | 0.8322 | 0.8320 | 0.8663 | 0
HateXplain (zero-shot testing) | DistilBERT | 592130 | 0.6013 | 0.4450 | 0.0869 | N/A
| Space Model | 197122 | 0.5821 | 0.5187 | 0.2698 | 0.001
| Space Model | 197122 | 0.5977 | 0.5040 | 0.2007 | 0
Table 1: Comparative table of the results of the Space Model in different
configurations with DistilBERT on the IMDB dataset and HateXplain dataset

### Preprocessing and settings

BERT is a standard model that we use as a reference and a baseline. We only
fully train the weights of this model once when compared with the Chennai
water crisis data. For all of the other experiments, we preserve all of the
generalizability and do not spend time on training. XLNet is the current
state-of-the-art transformer for multiple benchmarks; in this specific work,
we focus on the IMDB sentiment analysis dataset. By using this model and
comparing the results with it, we want to prove that attaching the Space-model
head to virtually any current state-of-the-art transformer would significantly
boost performance.

We are not conducting any data preprocessing for either of the datasets. We
use cased models for all of the expriments except for the Social Media
Attributions comparison. For the space model, the key idea is the contextual
embedding generation. The entity doing this in our framework is called a base
model; virtually any transformer model can play this role. We use cased
DistliBERT, cased BERT, and cased XLNet.

Figure 2: 3D projection of the space embeddings for the 3-class classification
(HateXplain). For the 3-class, similar to the 2-class, we expect to have 3
orthogonal projections. Here, we observe that if we review this image in
multiple projections - some projections are clearly orthogonal, and some are
more aligned. This is the effect that we have discussed previously, that
contextual attributions might have overlapping concepts.

We use the base model configuration for all of the experiments except for the
state-of-the-art establishment (12 layers for BERT and XLNet and 6 layers for
DistilBERT). For the state-of-the-art performance, we trained large (24
layers) XLNet, which was used for reporting the results in the original paper.
We use the Adam optimizer with a learning rate of $2\cdot 10^{-4}$ for all
experiments, except for the state-of-the-art establishment, since the original
paper states that $10^{-5}$ was used to achieve the best results. Maximum
sequence length and batch size is 256 for all of the basic experiments and is
replaced with 512 and 4, respectively, for the XLNet large state-of-the-art
results.

Since the original paper recommends using 32 as the batch size for the IMDB
benchmark for the best results, and we could not fit that to the GPU memory,
we used 8 gradient accumulation steps and adjusted the number of training
steps accordingly. Even though the result does not precisely reproduce the
original outcome, it is close, and the evident performance boost from the
space model is transparent.

For the number of latent spaces, we use three for most experiments since this
is enough to outperform significantly and is easy to visualize. As discussed
previously, when we project the contextual embeddings onto the context
attribution, we expect these projections to be orthogonal if the classes are
different. That is what we observe in Figure 1 and Figure 2.

For the comparison with the Social Media Attributions, use the latent size of
64. For the state-of-the-art results using XLNet, we use 128 as the latent
space size. We use a single Nvidia A5000 GPU for our training. Our model with
various configurations may take from 30 seconds per epoch with DistilBERT to
25 minutes with XLNet large. A standard number of fine-tuning epochs is set to
5; however, for the XLNet large state-of-the-art results, we used only one
epoch of training with one epoch of head fine-tuning to prevent overfitting.

### Evaluation Metrics

Since we are evaluating the model between multiple benchmarks simultaneously,
we want to adjust to both a perfectly balanced IMDB dataset and a less
balanced HateXplain dataset. So, we report accuracy and f1-macro score. Our
loss throughout the experiments is Cross-Entropy loss, sometimes combined with
intra-space loss for better regularization. We also report the weight of the
Intra-space loss in the experiments. This is usually set to a very low number
to avoid dominance over the cross-entropy loss.

### Experimental Results

#### Fine-tuning Space Model

First, we ran a set of experiments on the IMDB benchmark dataset with the
DistilBERT model as a base model (Table 1). We observe that the Space model is
superior for both accuracy and f1-macro score. We also explore the number of
trained parameters. With 3-time fewer parameters, the performance boost is
already around 5% for both metrics. We also observe that with around 128 times
fewer trainable parameters, the space model performs better by almost 2%.

Metric | Space Model | BERT
---|---|---
Accuracy | 0.5296 | 0.4485
F1-score (macro) | 0.4304 | 0.3314
Precision | 0.5431 | 0.4471
Recall | 0.5296 | 0.4485
Table 2: BERT HateXplain (3-class) evaluation

Then, we compare these results with the experiments for a much more complex
HateXplain benchmark. The choice of the datasets is non-arbitrary in that
case. We want data to have the evident polarization between classes, which is
aligned cross-datasets, to prove the zero-short generalization component of
our approach. BERT, DistilBERT, and XLNet are all evaluated with this
benchmark against the Space Model in a 3-class and 2-class setting.

#### Generalizability

We take corresponding models and evaluate them on the HateXplain benchmark in
a zero-shot manner (Table 1). For the sentiment analysis, negative labels are
encoded as 0 and positive as 1; for the HateXplain, we encode Hateful and
offensive labels as 0 and normal labels as 1. Here, we see that the Space
model with intra-space loss is a top model in terms of f1-score, while the
accuracy is the highest for the DistilBERT. However, accounting for the
dataset imbalance, we see that DistilBERT is worse in terms of f1 by at least
6-7% and almost 4 times worse in terms of recall. Next, we compare the BERT
model with the Space model and base BERT on the HateXplain benchmark with 3
classes. Here, we only train the classification head for BERT and contextual
attribution operators for the Space model (as discussed previously). The
results in Table 2 clearly show that the Space Model is superior in all of the
metrics by at least 8%.

We then fine-tune the classification head and the space model with XLNet and
BERT base models for the same HateXplain benchmark (Table 5) and observe that
for BERT, the performance gap with identical training settings and identical
base model is more than 16% on the f1-macro score. In comparison, for XLNet,
this gap is around 6%.

To further prove the effect of the performance boost using the space model, we
do the full training of the XLNet, a state-of-the-art model for the IMDB
benchmark (Table 3). With almost identical settings to the original paper, we
obtain a 0.9386 f1-score, while training the space model with the exact same
settings gives us 0.9487 (all of the other metrics are also superior for the
space model, except for the recall, which is again very different with
precision for the vanilla model, and very close for the space model). We
observe that the space model surpasses the state-of-the-art models in the
tasks and is much more tolerant to the imbalanced data. The precision-recall
trade-off is evident in most of the experiments. To prove this point further,
we conducted the ablation study and researched how the space model stabilizes
performance during training.

Metric | Space Model | XLNet
---|---|---
Accuracy | 0.9488 | 0.9387
F1-score (macro) | 0.9487 | 0.9386
Precision | 0.9463 | 0.9106
Recall | 0.9516 | 0.9731
Table 3: State-of-the-art XLNet on IMDB

#### Social Media Attribution

We compare our Space Model with the Social Media Attribution and observe 5.2%
F1-score improvement on our own reproducing experiment and almost 2% F1-score
improvement compared to the best-reported score from the original paper. The
best result there was obtained on the adapted Indian BERT, while we used base
uncased BERT without any adaptation, so this performance boost is not
exhaustive.

Metric | Space-model | BERT (uncased)
---|---|---
Accuracy | 0.8309 | 0.8220
F1-score (macro) | 0.8006 | 0.7484
Precision | 0.7126 | 0.8876
Recall | 0.7337 | 0.4674
Table 4: Social Media Attribution BERT-uncased Metric | Train Params | Accuracy | F1-score (macro) | Precision | Recall
---|---|---|---|---|---
Space-model (XLNet) | 4622 | 0.8798 | 0.8797 | 0.8764 | 0.8824
XLNet-base-cased | 1538 | 0.8160 | 0.8156 | 0.8421 | 0.7750
Space-model (BERT) | 4622 | 0.8110 | 0.8108 | 0.8227 | 0.7899
BERT-base-cased | 1538 | 0.6588 | 0.6555 | 0.6919 | 0.5649
Table 5: BERT and XLNet Comparison on HateXplain Dataset (2-class)

### Regularization effect on fine-tuning

Figure 3: DistilBERT (upper part) vs DistilBERT and Space Model (lower part)
stabilization comparison

During the experiments, we observed that the space model has a much better
ratio of recall/precision, which means that it handles imbalanced data much
more efficiently. Another observation is that adding just a space model
stabilizes the results during training, not allowing the performance to vary a
lot between iterations. Find the visualization of the ablation study in Figure
3. Additionally, intra-space loss adds more regularization and stabilization
and ensures that the concepts in the context attribution will not converge to
a single vector.

## Conclusion and discussion

In conclusion, this research focused on the novel methodology towards
conceptual embedding for classification with Language models. As an outcome of
this research, we have conducted a set of experiments to empirically prove the
efficiency of the proposed technique. We have also created the implementation
of the proposed framework via PyTorch and provided an open-source GitHub
repository to incivate and simplify future collaboration and exploration. We
believe that the potential of this approach is yet to be discovered, and the
goal of this paper was to provide some baseline ideas and understanding.

We anticipate improvements by adding more complex transformations after the
conceptual projection phase. We also believe that this technique should in no
way be limited to classification problems only. The formulation of the
regression problem is quite straightforward but needs to be additionally
researched. With that, we also expect that 1-to-1 correspondence of the
context attribution to the target class is an artificial limitation that we
hold in this paper for the simplicity of interpretation. However, if domain
knowledge suggests that having multiple context attributions (more than the
number of classes) for the task makes sense - then this should also be an
option. We would also like to explore further the potential of the
interpretation capabilities of the framework and how we can use it to extract
knowledge from the model.

## References

  * [Bem 1972] Bem, D. J.  1972\.  Self-perception theory11development of self-perception theory was supported primarily by a grant from the national science foundation (gs 1452) awarded to the author during his tenure at carnegie-mellon university.  volume 6 of Advances in Experimental Social Psychology. Academic Press.  1–62.
  * [Blackledge and Atapour-Abarghouei 2021] Blackledge, C., and Atapour-Abarghouei, A.  2021\.  Transforming fake news: Robust generalisable news classification using transformers.  In 2021 IEEE International Conference on Big Data (Big Data), 3960–3968.
  * [Bolt et al. 2019] Bolt, J.; Coecke, B.; Genovese, F.; Lewis, M.; Marsden, D.; and Piedeleu, R.  2019\.  Interacting Conceptual Spaces I: Grammatical Composition of Concepts.  Cham: Springer International Publishing.  151–181.
  * [Chenthamarakshan et al. 2011] Chenthamarakshan, V.; Melville, P.; Sindhwani, V.; and Lawrence, R.  2011\.  Concept labeling: Building text classifiers with minimal supervision.  1225–1230.
  * [Devlin et al. 2019] Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K.  2019\.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In Burstein, J.; Doran, C.; and Solorio, T., eds., Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 4171–4186.  Minneapolis, Minnesota: Association for Computational Linguistics.
  * [Ghosh et al. 2016] Ghosh, S.; Vinyals, O.; Strope, B.; Roy, S.; Dean, T.; and Heck, L.  2016\.  Contextual lstm (clstm) models for large scale nlp tasks.  ArXiv abs/1602.06291.
  * [Hase et al. 2023] Hase, P.; Diab, M.; Celikyilmaz, A.; Li, X.; Kozareva, Z.; Stoyanov, V.; Bansal, M.; and Iyer, S.  2023\.  Methods for measuring, updating, and visualizing factual beliefs in language models.  In Vlachos, A., and Augenstein, I., eds., Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, 2714–2731.  Dubrovnik, Croatia: Association for Computational Linguistics.
  * [Hochreiter and Schmidhuber 1997] Hochreiter, S., and Schmidhuber, J.  1997\.  Long short-term memory.  Neural Computation 9(8):1735–1780.
  * [Maas et al. 2011] Maas, A. L.; Daly, R. E.; Pham, P. T.; Huang, D.; Ng, A. Y.; and Potts, C.  2011\.  Learning word vectors for sentiment analysis.  In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, 142–150.  Portland, Oregon, USA: Association for Computational Linguistics.
  * [Maia et al. 2021] Maia, M.; Sales, J. E.; Freitas, A.; Handschuh, S.; and Endres, M.  2021\.  A comparative study of deep neural network models on multi-label text classification in finance.  In 2021 IEEE 15th International Conference on Semantic Computing (ICSC), 183–190.
  * [Mathew et al. 2021] Mathew, B.; Saha, P.; Yimam, S. M.; Biemann, C.; Goyal, P.; and Mukherjee, A.  2021\.  Hatexplain: A benchmark dataset for explainable hate speech detection.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 14867–14875.
  * [Mikolov et al. 2013] Mikolov, T.; Chen, K.; Corrado, G.; and Dean, J.  2013\.  Efficient estimation of word representations in vector space.  CoRR abs/1301.3781.
  * [Pennington, Socher, and Manning 2014] Pennington, J.; Socher, R.; and Manning, C.  2014\.  GloVe: Global vectors for word representation.  In Moschitti, A.; Pang, B.; and Daelemans, W., eds., Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1532–1543.  Doha, Qatar: Association for Computational Linguistics.
  * [Peters, Ruder, and Smith 2019] Peters, M. E.; Ruder, S.; and Smith, N. A.  2019\.  To tune or not to tune? adapting pretrained representations to diverse tasks.  In Augenstein, I.; Gella, S.; Ruder, S.; Kann, K.; Can, B.; Welbl, J.; Conneau, A.; Ren, X.; and Rei, M., eds., Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), 7–14.  Florence, Italy: Association for Computational Linguistics.
  * [Rumelhart, Hinton, and Williams 1986] Rumelhart, D. E.; Hinton, G. E.; and Williams, R. J.  1986\.  Learning Internal Representations by Error Propagation.  Cambridge, MA, USA: MIT Press.  318–362.
  * [Sanh et al. 2019] Sanh, V.; Debut, L.; Chaumond, J.; and Wolf, T.  2019\.  Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.  ArXiv abs/1910.01108.
  * [Sarkar et al. 2020] Sarkar, R.; Mahinder, S.; Sarkar, H.; and KhudaBukhsh, A.  2020\.  Social media attributions in the context of water crisis.  In Webber, B.; Cohn, T.; He, Y.; and Liu, Y., eds., Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1402–1412.  Online: Association for Computational Linguistics.
  * [Spilka, Shaver, and Kirkpatrick 1985] Spilka, B.; Shaver, P.; and Kirkpatrick, L. A.  1985\.  A general attribution theory for the psychology of religion.  Journal for the Scientific Study of Religion 24(1):1–20.
  * [Swamy, Jamatia, and Gambäck 2019] Swamy, S. D.; Jamatia, A.; and Gambäck, B.  2019\.  Studying generalisability across abusive language detection datasets.  In Bansal, M., and Villavicencio, A., eds., Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), 940–950.  Hong Kong, China: Association for Computational Linguistics.
  * [Vaswani et al. 2017] Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L. u.; and Polosukhin, I.  2017\.  Attention is all you need.  In Guyon, I.; Luxburg, U. V.; Bengio, S.; Wallach, H.; Fergus, R.; Vishwanathan, S.; and Garnett, R., eds., Advances in Neural Information Processing Systems, volume 30.  Curran Associates, Inc.
  * [Yang et al. 2019] Yang, Z.; Dai, Z.; Yang, Y.; Carbonell, J.; Salakhutdinov, R. R.; and Le, Q. V.  2019\.  Xlnet: Generalized autoregressive pretraining for language understanding.  In Wallach, H.; Larochelle, H.; Beygelzimer, A.; d'Alché-Buc, F.; Fox, E.; and Garnett, R., eds., Advances in Neural Information Processing Systems, volume 32.  Curran Associates, Inc.
  * [Zhou and Li 2005] Zhou, Z.-H., and Li, M.  2005\.  Semi-supervised regression with co-training.  In Proceedings of the 19th International Joint Conference on Artificial Intelligence, IJCAI’05, 908–913.  San Francisco, CA, USA: Morgan Kaufmann Publishers Inc.


# Multi-modal microscopy imaging with the OpenFlexure Delta Stage

Samuel McDermott<EMAIL_ADDRESS>Department of Physics, University of
Cambridge, UK  Filip Ayazi  Department of Physics, University of Cambridge, UK
Joel Collins  Department of Physics, University of Bath, UK  Joe Knapper
Department of Physics, University of Bath, UK  Julian Stirling  Department of
Physics, University of Bath, UK  Richard Bowman  Department of Physics,
University of Bath, UK  Pietro Cicuta  Department of Physics, University of
Cambridge, UK

###### Abstract

Microscopes are vital pieces of equipment in much of biological research and
medical diagnostics. However, access to a microscope can represent a
bottleneck in research, especially in lower-income countries. ‘Smart’ computer
controlled motorized microscopes, which can perform automated routines or
acquire images in a range of modalities are even more expensive and
inaccessible. Developing low-cost, open-source, smart microscopes enables more
researchers to conceive and execute optimized or more complex experiments.
Here we present the OpenFlexure Delta Stage, a 3D-printed microscope designed
for researchers. Powered by the OpenFlexure software stack, it is capable of
performing automated experiments. The design files and assembly instructions
are freely available under an open licence. Its intuitive and modular
design—along with detailed documentation—allows researchers to implement a
variety of imaging modes with ease. The versatility of this microscope is
demonstrated by imaging biological and non-biological samples (red blood cells
with Plasmodium parasites and colloidal particles in brightfield, epi-
fluorescence, darkfield, Rheinberg and differential phase contrast. We present
the design strategy and choice of tools to develop devices accessible to
researchers from lower-income countries, as well as the advantages of an open-
source project in this context. This microscope, having been open-source since
its conception, has already been built and tested by researchers around the
world, promoting a community of expertise and an environment of
reproducibility in science.

##  1 Introduction

One of the most common scientific instruments, particularly in biological
research, is the optical microscope. Access to a microscope capable of
multiple imaging modalities and automation is often a limiting factor even in
well funded laboratories, where an advanced unit might be shared across many
users. It is particularly acute in laboratories of lower income countries,
limiting the scope and ambition of research projects. The OpenFlexure Delta
Stage (OFDS), shown in Figure 1, aims to provide a low-cost microscope
enabling a range of imaging modalities, in an open format that can be easily
modified, and with parts that are mostly 3D printed enabling local
manufacturing and repair.

Figure 1: The OpenFlexure Delta Stage’s compact and well-tested design is
suitable for a laboratory bench or microbiological safety cabinet. Here it is
shown imaging a blood smear on a microscope slide. It is majority 3D printed,
and powered using a Raspberry Pi. Its ‘delta’ geometry stage allows the sample
to be moved independently of the optics in $x$, $y$, and $z$.

The OFDS stems as a variant of the OpenFlexure Microscope (OFM), an open
source project that has evolved over many years, recently overviewed in [1].
OFDS uses the same flexure properties provided by 3D printed plastics, but a
different orientation of the actuators compared to OFM means that now the
stage can move in 3 dimensions independently of the optics. In contrast to the
OpenFlexure Microscope’s ‘Cartesian’ geometry, where the stage moves in $x$
and $y$ and the optics move in $z$, the stage in the OFDS moves independently
of the other optics modules, in each of $x,\,y$ and $z$. This makes it
particularly suitable for heavier optics and more complex optical systems. A
early version of this 3D printed ‘delta robot’ design [2] was previously used
for optical projection tomography ‘OptiJ’ [3], but this is the first time it
has been developed into a self-contained microscope.

This paper demonstrates the versatility and simplicity of using the OFDS to
implement a wide range of imaging modes, as outlined in Table 1. These
microscopes are a self contained units, they can be built using common tools
and parts, have comprehensive build instructions and are controlled using
intuitive software. This makes the unit suitable both as an instrument in
itself, for a variety of imaging applications, or for integration with other
experimental instruments. Low-cost, open-source, optical systems such as the
UC2 [4], Microscopi [5], and OctoPi [6] designs are becoming a popular way of
increasing accessibility to optical experiments, lowering the initial barrier
for users to tailor their microscopy needs to their experiments. The open-
source nature of the OFDS has already led to adaptors to link different
modules and systems, such as using the OFDS with the UC2 system [7].

Throughout the design phase we have worked closely with biologists and
physicists with a range of budgets and experience to create a microscope that
would have significant functionalities whilst keeping in mind ease of use. The
OFDS is compact enough to fit on a lab bench, such that it can become an
everyday tool for quickly checking a sample, digital capture of images, or can
be built up as multiple units to enable a high throughput workflow.

Table 1: The OpenFlexure Delta Stage is capable of multiple imaging modes. The
key imaging modes laid out in this table are described in this paper.

Brightfield Epi-fluorescence Darkfield Rheinberg Differential Phase Contrast
Description Section 2.1 Section 2.3 Section 2.5 Section 2.5 Section 2.5
Illumination module Transmission illumination Reflection illumination LED
array LED array LED array Optics module Transmission optics module Reflection
optics module Transmission optics module Transmission optics module
Transmission optics module Cross section figure Figure 2 (a) Figure 2 (b)
Figure 2 (c) Figure 2 (c) Figure 2 (c) Sample images Figure 4 Figure 6 Figure
8 (b) Figure 8 (c) Figure 8 (d)

Figure 2: The OpenFlexure Delta Stage has three modular setups for performing
different imaging modes. These cross-sections show the three main optics
setups. (a) Brightfield (transmission) imaging. (b) Reflection imaging. (c)
LED array imaging. ( L1: Condenser lens, L2: ‘Tube length correction’ lens,
BS: Beamsplitter, F1: Emission filter, F2: Excitation filter) Scale bar 2cm.

##  2 Optics

The optics of the OFDS are designed to be modular. There are three optical
setups for imaging, illustrated with cross sectional diagrams in Figure 2: (a)
Brightfield (transmission); (b) Reflection; (c) Computational illumination
with a LED array.

Figure 3: The transmission mode optics module is constructed from 3D printed
parts and common optical components. This computer generated render shows the
assembly of the transmission (a) illumination and (b) optics modules.

###  2.1 Brightfield (transmission) illumination

####  2.1.1 Optics module

The optics module (Figure 3(a)) is a single printed part that holds an
objective lens, a ‘tube length correction’ lens and the imaging sensor in
position. It is attached to the main body with a wedge-shaped mount, allowing
initial coarse adjustment of its vertical position. The objective lens is a
standard RMS threaded objective, which is screwed into the top of the optics
module. This type of lens provides high-quality imaging, suitable for
scientific use, and is readily available from various manufacturers in a range
of magnifications ($4\times-100\times$) and numerical apertures (and prices).
It is recommended to use Plan or Semi-Plan corrected lenses up to $40\times$
and Plan corrected or better at $100\times$ magnification.

The optics module is designed for use with $160\text{\,}\mathrm{mm}$ tube
length objective lenses, however the actual length of the optics module is
less than this, due to space constraints. A ‘tube length correction’ lens is
push-fitted into the optics module, at a position such that it forms an image
on the sensor at a shorter distance. The magnification was lowered such that
the field of view of a standard field number eyepiece (17mm) would match the
smaller imaging sensor of the Raspberry Pi Camera v2 (diagonal 4.6mm). An
achromatic doublet lens (focal length: 50 mm, outer diameter: 12.7 mm, e.g.
Thorlabs AC127-050-A) is used to minimize the chromatic aberrations which can
occur when using white illumination. By using push fits and screw threads for
the lenses, alignment is made as simple as possible, further reducing
chromatic and other optical aberrations.

The camera (described further in Section 4.1.2) mounts with screws to the
bottom of the optics module, with the exposed circuit board protected by a
cover.

####  2.1.2 Illumination module

For transmission illumination (Figure 3(b)) the single-print condenser module
focuses a white LED onto the sample using a $f=$5\text{\,}\mathrm{mm}$$ PMMA
condenser lens. It is positioned on a mount which uses a friction dovetail to
hold the module in position. The mount can be positioned in $x$ and $y$ to
align the illumination before being secured into position on the main body.
The same mount can be used for different types of illumination module, for
example a LED array illumination module can be attached to a frame that
connects to the mount.

####  2.1.3 Light path

The OFDS uses a modified version of the OpenFlexure Microscope’s brightfield
imaging setup [1], adjusting the relative rotations of the camera to the
stage. Figure 2(a) shows the cross section of the brightfield setup.
Illumination from the white LED is collected by the condenser lens and
projected onto the sample. The objective lens and ‘tube length correction’
lens image the sample onto the imaging sensor.

####  2.1.4 Sample images – brightfield

We have tested the microscope for use with automating blood smear imaging,
with a particular focus on blood-borne malaria. Images of malaria infected
human red blood cells, smeared and fixed, were captured using the brightfield
setup of the OFDS with a $100\times$, 1.25NA RMS objective lens.

###  2.2 Sample preparation and brightfield imaging

Smears of malaria infected cells were prepared following the standard
procedure for lab cultures. Briefly, a culture of Plasmodium falciparum
malaria parasites (3D7, $3\text{\,}\mathrm{\char 37\relax}$ HCT,
$5\text{\,}\mathrm{\char 37\relax}$ parasitemia, mixed stages) were maintained
in human red blood cells and RPMI culture medium, following [8, 9]. The blood
was from healthy donors, purchased from NHS Blood and Transplant. They were
cultured by adding gas (1% O2, 3%CO2) and storing in an incubator at
$37\text{\,}\mathrm{\SIUnitSymbolDegree}$. In ideal conditions, the
parasitemia would multiply by 5 every 48 hours. A
$200\text{\,}\mathrm{\SIUnitSymbolMicro L}$ sample of the culture was removed
and centrifuged for 20 seconds. $150\text{\,}\mathrm{\SIUnitSymbolMicro L}$ of
the supernatant was removed and the blood pellet was re-suspended.
$5\text{\,}\mathrm{\SIUnitSymbolMicro L}$ was dropped onto a glass microscope
slide and smeared into a thin smear using a second glass slide, following
standard thin film smear preparation[10]. The smear was dried by hairdryer,
and fixed by covering the slide completely with methanol for
$30\text{\,}\mathrm{s}$. The methanol was removed and the slide dried again.

$10\text{\,}\mathrm{\char 37\relax}$ Giemsa (VWR Giemsa’s stain improved
solution R66, Gurr™, diluted in Evian bottled water, filtered through
$20\text{\,}\mathrm{\SIUnitSymbolMicro m}$ filter) was poured onto the slide
and left from $10\text{\,}\mathrm{min}$ at room temperature. The Giemsa was
then removed by reverse osmosis under a stream of tap water. Excess water was
removed and the slide dried.

Figure 4: A typical use case of the OpenFlexure Delta Stage is for brightfield
imaging of malaria infected red blood cells. These micrographs taken using the
OpenFlexure Delta Stage show malaria infected red blood cells, with insets
showing zoomed-in regions with the malaria parasites. They were stained using
Giemsa, which makes the red blood cells appear blue and the malaria parasites
appear purple. The Raspberry Pi v2 camera (Section 4.1.2) captures color
images, which is necessary for stained or dyed histological samples. These
images were taken using a $100\times$, 1.25NA RMS oil objective lens. Images
cropped for visibility, scale bars $20\text{\,}\mathrm{\SIUnitSymbolMicro m}$.

The slide was imaged using the brightfield configuration of the OFDS with a
$100\times$, 1.25NA RMS objective lens. The camera was configured to have an
even response across the field of view with the correct exposure time and
white balance settings using the OpenFlexure server software[11]. A small drop
of immersion oil was placed on the objective lens and the sample was placed on
the stage with the blood smear side facing the objective. The sample was moved
into a good field of view and focused using the auto focus routine. Images
from the sample can be seen in Figure 4.

Figure 5: The reflection mode modules uses a modified version of the
transmission optics modules. They are constructed from 3D printed parts,
custom-cut optical filters, a LZ1 LED and common optical parts. This computer
generated render shows the assembly of the reflection (a) optics and (b)
illumination modules.

###  2.3 Epi-fluorescence (reflection) illumination

####  2.3.1 Optics module

The design of the reflection illumination optics module is similar to the
transmission illumination variant. However, between the ‘tube length
correction’ lens and the camera, a square hole is cut out of the side, and
there are two screw holes either side (Figure 5 (a)). The filter cube attached
to the reflection illumination module fits inside this hole, and it is secured
in place with the two screws.

####  2.3.2 Illumination module

The illumination module consists of a filter cube, a condenser module and the
reflection illumination holder, (Figure 5(b)). The filter cube has two slots
for custom cut glass filters. At the bottom of the filter cube is a slot for
the emission filter. At $45\text{\,}\mathrm{\SIUnitSymbolDegree}$ to the base
there is a slot for the beamsplitter. The filter cube couples onto the
reflection illumination holder with a dovetail, and the combined unit slides
into the reflection illumination optics module and is attached with two
screws. In the top of the reflection illumination holder is a slot for a
custom cut glass excitation filter. A filter cube, as opposed to a cheaper
fluorescent dark field setup, was chosen as it is more optically efficient,
and does not photo-bleach the entire sample.

The LED (LED Engin LZ1 with starboard MCPCB) is screwed onto the back of the
reflection illumination condenser module. This more powerful mono-chromatic
LED was chosen as it is brighter, to increase the signal from the
fluorophores. Inside the reflection illumination module is a pinhole. A 5 mm
focal length PMMA condenser lens focuses the light from the LED onto the back
focal plane of the objective lens. The condenser module is attached to a slip
plate on the reflection illumination holder so that it can be moved to the
correct position.

####  2.3.3 Light path

The reflection mode module is used for reflection and epi-fluorescence
imaging. Figure 2 (b) shows the cross section of the reflection imaging set
up. The more powerful mono-chromatic LED is attached to the illumination
module. A pinhole is in front of it and the condenser lens projects the
illumination from the LED, through the filter cube and onto the back focal
plane of the objective lens. The excitation filter ensures that the
illumination wavelength matches the excitation wavelength of the sample’s
fluorophores. The dichroic beamsplitter reflects the illumination towards the
objective. The objective lens focuses the illumination onto the sample. The
fluorophores are excited and emit illumination of a higher wavelength in all
directions. The objective lens captures some of that illumination and projects
it onto the camera sensor.

####  2.3.4 Sample images – epi-fluorescence

Two samples were used to test two different epi-fluorescence wavelengths.
Fluorescent stained malaria infected human red blood cells were imaged using
epi-fluorescence. SYBR Green I (ex = $497\text{\,}\mathrm{nm}$, em =
$520\text{\,}\mathrm{nm}$) was used as the staining agent. This stain binds to
double-stranded DNA, such as those present in the malaria parasite. A
StarLight calibration slide (Bangs Laboratory SL1GB), consisting of
$\approx$$6\text{\,}\mathrm{\SIUnitSymbolMicro m}$ fluorescent microspheres
(ex = $360\text{\,}\mathrm{nm}$, em = $450\text{\,}\mathrm{nm}$) was also
used.

Figure 6: The OpenFlexure Delta Stage is capable of imaging samples with quite
low fluorescent intensity. (a) Epi-fluorescence micrographs of SYBR-green
stained malaria infected red blood cells show the location of the red blood
cells and the brighter regions show the locations of the malaria parasites. It
is possible to see the characteristic ring shape of the trophozoite stage of
the malaria erythrocytic cycle. (b) Epi-fluorescence micrographs show
$6\text{\,}\mathrm{\SIUnitSymbolMicro m}$ fluorescent dyed
$\approx$$6\text{\,}\mathrm{\SIUnitSymbolMicro m}$microspheres. They are
illuminated with the UV LED and emit blue light. Images cropped and contrast
enhanced for visibility, scale bars $20\text{\,}\mathrm{\SIUnitSymbolMicro
m}$.

###  2.4 Sample preparation and epi-fluorescence imaging

The sample of malaria parasites in human red blood cells was smeared and fixed
as above. The slide was covered completely with 10 mM Tris pH8 for
$10\text{\,}\mathrm{s}$. The excess TRIS was removed, but the slide was not
dried. The slide was then covered completely with 1:3000 dilution of SYBR
Green I (Thermofisher, supplied as $10,000\times$ concentrate in DMSO, Ex: 498
nm, Em: 520 nm) in 10 mM Tris pH8. The slide was enclosed in a dark box for 10
minutes. The excess SYBR Green 1 solution was removed by reverse osmosis under
a stream of tap water. The slide was dried using a hairdryer.

The malaria sample slide was imaged using the epi-fluorescence configuration
of the OFDS with a $100\times$, 1.25NA RMS objective lens. A blue LED
(LZ1-10B202-0000, $\lambda=460$ nm) was used, with a short pass excitation
filter with 510 nm transition (Comar optics 510 IK 50), long pass dichroic
beamsplitter with 506 nm transition at
$45\text{\,}\mathrm{\SIUnitSymbolDegree}$ (Comar optics 550 IY 16), and long
pass emission filter with 510 nm transition (Comar optics 510 IY 50). The
images can be seen in Figure 6 (a).

The microsphere sample slide was also imaged using the epi-fluorescence
configuration of the OFDS with a $100\times$, 1.25NA RMS objective lens. A UV
LED (LZ1-10UV0R-0000, $\lambda=365-370$ nm) was used as illumination. A band
pass excitation filter with peak 360 nm (Comar optics 360 GB 50), a long pass
dichroic beamsplitter with 410 nm transition at
$45\text{\,}\mathrm{\SIUnitSymbolDegree}$ (Comar optics 454 IY 125) and a long
pass emission filter with transition at 435 nm (Comar optics 435 GY 50) were
used in the filter cube. The images can be seen in Figure 6 (b).

###  2.5 Computational illumination with LED array

Using a LED array for illumination has been shown to be a low cost way to add
new imaging modes to a standard microscope setup [12, 13]. Figure 2(c) shows
the cross-section of the LED array setup for the OFDS. The LED pixels that
fall within the numerical aperture of the objective are shown in blue. The
specification and control of the LED array is given in Section 4.1.1. The LED
array is positioned such that the imaging NA is one LED pixel smaller than the
LED array. This imaging NA is shown as a circle in Figure 7(a).

The LED array allows for a spatial illumination source. Using an extension
that adds functions to the control software (Section 4.2.3) [14] it is
possible to control each of the 64 RGB LEDs individually. As the LED array is
a flat surface, the LEDs closest to the objective lens (in the centre of the
array) will appear brighter than those further away (at the corners). To
correct for this, the LED uses a look up table (LUT) based on the height of
the LED array and the LED pixel density to make an illumination profile that
simulates a curved surface. This means that the intensity of the illumination
of any pixel at the pupil of the objective lens will be equal.

Figure 7: An 8-by-8 LED array (Section 4.1.1) is used to provide complex
illumination patterns, which can enhance contrast and generate phase images.
This diagram shows examples of LED illumination patterns for different types
of imaging. Each square in the grid represents one LED, and the circle
represents an example Numerical Aperture of the objective lens calculated
geometrically for a $40\times$, 0.65NA objective , as shown in Figure 2. The
LED array is positioned as close as reasonably possible to the sample
($\approx 5$mm) in order that the NA of the objective falls within the LED
array. Each LED ‘pixel’ is 2 mm $\times$ 2 mm with a viewing angle of
$160\text{\,}\mathrm{\SIUnitSymbolDegree}$ and consists of 3 LEDs (red, green
and blue) which can be individually controlled to produce a single color. (a)
For brightfield imaging, only the pixels within the NA are illuminated, but by
illuminating less than the full set, the illumination NA can be reduced. (b)
For Darkfield imaging, the LEDS outside the NA are illuminated. (c) For
Rheinberg imaging, the ‘Darkfield’ LEDs are illuminated in one color and the
‘Brightfield’ in another. (d) For Differential Phase Contrast imaging, three
LED patterns are used, and an image captured for each. This is automated
through the phase contrast extension [14]. Figure 8: Sample images taken with
the different imaging modes using the LED array. (a) Brightfield image of
$9.56\text{\,}\mathrm{\SIUnitSymbolMicro m}$ SiO2 microparticles (RI: 1.43)
suspended in Milli-Q water (RI: 1.33) using the central $2\times 2$ LEDs.
($63\times$, 0.90NA) (b) Darkfield image of
$9.56\text{\,}\mathrm{\SIUnitSymbolMicro m}$ SiO2 microparticles. ($63\times$,
0.90NA, Imaged using green LEDs, Contrast enhanced). (c) Differential Phase
Contrast image of $9.56\text{\,}\mathrm{\SIUnitSymbolMicro m}$ SiO2
microparticles ($63\times$, 0.90NA) showing the phase gradient along the
horizontal direction. (d) Rheinberg image of tissue paper ($40\times$,
0.65NA). All scale bars $20\text{\,}\mathrm{\SIUnitSymbolMicro m}$.

For brightfield imaging with the LED array, all the pixels within imaging NA
circle are lit as shown in Figure 7 showing that the illumination NA is equal
to the imaging NA. The illumination NA can be reduced by reducing the radius
of the illuminated pixels. As the illumination NA is reduced, the coherence is
increased and the depth sectioning is reduced, as shown in Figure 8(a).

In darkfield microscopy [12], the unscattered illumination is excluded from
the sensor. If the LEDs outside of the imaging NA are illuminated, then there
will be no direct light entering the objective, and the image will be dark.
When a sample is loaded in position, it scatters the light from the LEDs into
the objective and forms a darkfield image as shown in Figure 8(b). The
darkfield LEDs were chosen to be only green as they have the highest
brightness (while minimising chromatic aberrations) and the Bayer filter
pattern of the camera sensor is more sensitive to green.

Rheinberg imaging [15] follows the same principle as darkfield imaging, but
because it is possible to change the colors of the LEDs, the LEDs that would
form the brightfield image (within the imaging NA) can be one color, and the
LEDs that would form the darkfield image (outside the imaging NA) can be a
different color, as shown in Figure 7(c). The images produced by Rheinberg
imaging are a form of optical staining. The background of the image will be in
the brightfield color, and the darkfield color will provide enhanced contrast
effects. An example image using this technique is shown in Figure 8.

Differential Phase Contrast (DPC) can be achieved using the brightfield LEDs
of the LED array [13]. The principle of DPC is that the gradient of phase can
be calculated from two intensity images when the sample is illuminated from
opposite angles. Figure 7(d) show the LED patterns when illuminated from the
left ($I_{\text{L}}$) and right ($I_{\text{R}}$). $I_{\text{TOT}}$ is the
brightfield image ($I_{\text{L}}+I_{\text{R}}$). The DPC image is calculated
as

$I_{\text{DPC}}=\frac{I_{\text{L}}-I_{\text{R}}}{I_{\text{TOT}}}.$ (1)

This DPC image is related to the object’s phase gradient along the axis of
asymmetry[16], creating a ‘shadow’ effect along the axis. DPC images are
automated on the OFDS using the phase contrast extension [14]. Figure 8 shows
a sample image captured using the DPC technique.

With the use of extensions, as described in Section 4.2.3, many more
illumination routines such as 3D differential phase contrast [13], color-
multiplexed differential phase contrast [17], or Fourier ptychography [18] are
possible.

##  3 Stage and body design

The OFDS is majority 3D printed. The main body, which consists of a 3D moving
stage, is printed as one piece. The base of the stage houses the electronic
components.

###  3.1 Stage mechanism

One of the most important aspects of a microscope’s design is a stable stage.
A high resolution microscope with a stage that drifts or shakes will produce
low quality images. The microscope’s stage is inspired by a delta robot design
[2]. The sample stage is mechanically controlled, and can move the sample in
three dimensions. This design is particularly advantageous as the optics
remain stationary.

The kinematics of the OFDS are based around the delta robot design. The delta
robot is a type of parallel arm robot. As shown as a cross-section in Figure
9, it consists of a fixed base and a moving stage. The moving platform is
situated directly above the fixed platform. The kinematic chain consists of a
driving arm and a driven arm. There are three of these, situated at
$120\text{\,}\mathrm{\SIUnitSymbolDegree}$ around the platforms. The driven
arm is a parallelogram, taking advantage of the well characterised flexure
mechanism of the 3D printed plastic [19]. As per that tested design, the
thickness of the flexure is set to three layers of plastic, with a flexure
length of 1.5mm. A compromise of deflection range and stiffness, this design
allows approximately $6\text{\,}\mathrm{\SIUnitSymbolDegree}$ of motion
without exceeding the elastic limit. By using parallelograms, the movement of
the stage is restricted to solely translation. This means that although there
may be some parasitic z-motion for x and y movements, it does not tilt or
rotate, undesirable attributes for a microscope stage.

Figure 9: The stage is 3D-printed as a single part, and uses the flexure
properties of the plastic to enable fine positioning of the stage. This cross
section shows this mechanism used to control the actuators and move the stage
with its pin-joint structure shown in red.

The three driving arms are linear actuators, controlled by three stepper
motors. The actuator design, shown in Figure 9, is adapted from the
OpenFlexure Microscope [1, 19]. The stepper motor drives the 3D printed
actuator mechanism in a vertical motion. As the actuator is connected to the
parallelogram structures by flexures, they are driven by the vertical movement
of the actuator.

Figure 10: The Cartesian stage motions are created from the rotations of a
combination of the actuator gears. For example, to move the stage in the
positive $x$-direction, gear ‘b’ rotates clockwise and gear ‘a’ rotates anti-
clockwise.

There has been some work done in calculating the kinematics of delta robot
mechanics [20]. However, as the driven arms are constrained to only bend
through small angles ($6\text{\,}\mathrm{\SIUnitSymbolDegree}$), the small
angle approximation means that the kinematics can be approximated using linear
relations. It would be possible for users to modify the driven arms to
increase the range of motion, but this will come at a reduction in stability
and the small angle approximation will no longer be applied. As shown in
Figure 10, the movement of each of the parallelograms can be mapped from their
delta-bot kinematics to Cartesian kinematics, which is more intuitive for
controlling the stage. The software converts the Cartesian kinematics into the
delta kinematics using the following relation (in addition to $x,y,z$
scaling):

$\begin{bmatrix}x\\\ y\\\
z\end{bmatrix}=\begin{bmatrix}-\cos{30}&\cos{30}&0\\\ \cos{60}&\cos{60}&-1\\\
1&1&1\end{bmatrix}\begin{bmatrix}a\\\ b\\\ c\\\ \end{bmatrix}$ (2)

The geometric proportions of the stage mechanism have been optimised to allow
for a stable stage with the necessary movement accuracy, travel, and size. The
horizontal travel is set by the stage height, as shown in Figure 9. This is a
compromise between the overall height of the structure–affecting stiffness and
print reliability and travel–which increases with a taller stage. The vertical
travel is set by the lever length–the distance between the central pivot and
the point where the legs attach to the actuator levers. The length of the
actuating lever sets the number of motor stpdf across the range of travel.
Starting with the actuators in their central positions, the range of the
stage’s motion is [9mm, 12mm, 5mm]. Further information about the parameters
in the design of the stage can be found in the online documentation [21].

It is possible to mount different samples onto the stage using stage adapters
which screw onto the top of the stage. Sample clips are used to hold
microscope slides with fixed samples, whereas $35\text{\,}\mathrm{mm}$ and
$55\text{\,}\mathrm{mm}$ Petri dish adapters enables the imaging of live
samples.

###  3.2 Base

The base is at the bottom of the microscope. It is designed to keep the
electronic components securely in place and provide a stable platform for the
main body of the microscope. The Raspberry Pi sits in the bottom of the base,
on four raised 3D printed standoffs. There are holes in the side of the base
to allow connections and there is a cutout underneath to access the SD card.
$20\text{\,}\mathrm{mm}$ M2.5 standoffs then secure the Raspberry Pi to the
base. The Sangaboard then sits on top of these standoffs and is secured with
M2.5 screws. Again, there are holes for access to the Sangaboard connectors.

The main body of the microscope fits tightly inside the base and its feet sit
on elevated platforms. It is secured to the base by three screws connected to
each actuator column. It is recommended that rubber feet are attached to the
bottom of the base, or that it sits on a layer of foam to reduce environmental
vibrations.

##  4 Electronics

The OFDS forms the basis of a digital ‘smart’ microscope, capable of
automating the imaging tasks. To operate, it requires some basic electronics
and electrical components.

###  4.1 Electronic components

####  4.1.1 Illumination

##### Transmission LED

A white through hole 5 mm 3.2V LED (Nichia NSPL500DS) is used for transmission
illumination. This LED was chosen because it has a spectrum that closely
matches that of a filament bulb, but any 5 mm LED can be used. A
$100\text{\,}\mathrm{\SIUnitSymbolOhm}$ resistor is soldered to one of the
legs of the LED and it is connected to the 5V and ground connectors of the
Raspberry Pi’s GPIO. This means that whenever the Raspberry Pi is turned on,
the illumination will turn on.

##### Epi-fluorescence LED

LED Engin LZ1 LEDs with the star MCPCB are used for illumination. This was
chosen as it provides the high flux required for fluorescence illumination in
a convenient form factor and with wide availability. They are available in 5
colors (Table 2) which are suitable for a range of fluorophores and for epi-
illumination. They are attached to an aluminium backplate for heat
dissipation. They require a bench top constant-current power supply, although
it may be possible in the future to specify a reliable LED driver.

Table 2: The LZ1 LEDs were chosen for their brightness and their range of colors. Their wavelengths are applicable for a wide variety of reflection illumination samples. Color | Wavelength (nm) | Manufacturer number
---|---|---
UV | 365-370 | LZ1-10UV0R-0000
Blue | 460 | LZ1-10B202-0000
Green | 523 | LZ1-10G102-0000
Red | 623 | LZ1-10R102-0000
Cool white | 5500K | LZ1-10CW02-0055

##### LED array

Figure 11: The electronics and illumination holder are modified for the LED
array, but it is still possible to use the same main body and base as in the
other imaging modalities. This computer generated render of the OpenFlexure
Delta Stage shows, with the base cutaway, how the extra components are
assembled.

The LED array (Adafruit 3444) consists of 64 DotStar (APA102-2020) LEDs spaced
3.5 mm over an eight by eight grid. This commercially available LED array has
a small enough pitch suitable for the scale of the OFDS with a convenient
control interface. Each LED ‘pixel’ is 2 mm $\times$ 2mm with a viewing angle
of $160\text{\,}\mathrm{\SIUnitSymbolDegree}$ and consists of 3 LEDs (red,
green and blue) and can be addressed individually. The array is connected to
the Raspberry Pi GPIO pins via a 3V to 5V quad logic level shifter (74AHCT125)
mounted on a perma-proto board. It is mounted to the illumination arm using a
3D printed mount. In future, a condenser lens should be added to the mount, as
demonstrated in [13], in order that the pixels at the edge of the LED array
can have more of their intensity captured by the objective lens. The LEDs are
controlled using the Adafruit Circuit Python DotStar driver library [22]. An
interface to the OpenFlexure Microscope Server’s interface is provided with an
extension [14], which presents a GUI to the user, allowing them to turn on and
off individual and groups of LEDs, as well as run different imaging routines.
These routines synchronise the illumination of LEDs with the camera, allowing
for complex imaging techniques such as differential phase contrast imaging.
The specification of the LEDs is shown in Table 3.

Table 3: Specification of the RGB LEDs in LED array (Adafruit 3444). Color | Wavelength (nm) | Brightness at 20mA (mcd)
---|---|---
Red | 620-625 | 300-330
Green | 520-525 | 420-460
Blue | 465-470 | 160-180

####  4.1.2 Camera

The camera at the bottom of the optics module is a Raspberry Pi Camera Module
v2 [23]. This uses a well characterised Sony IMX219 sensor, and connects to
the Raspberry Pi with a ribbon cable. It is an 8 megapixel ($3280\times
2464$), back illuminated CMOS color image sensor.

The optics module creates an image of the sample directly onto the sensor, so
the camera’s attached lens is not required and it can be unscrewed using a
plastic tool. However, as the sensor has been optimised in optical design and
firmware for its original short focal lens, it now generates images with a
non-uniform background and color response. This effect has been considered
recently, and can be calibrated and corrected on the fly using software [11].

####  4.1.3 Motor controller and motors

The OFDS has a 3 axis stage. Due to the complexity of the geometry, it is not
possible to be controlled by hand and so requires 3 stepper motors to actuate
the stage’s levers to enable precise movements. 28BYJ-48 stepper motors were
chosen as they are low-cost, low-power and light. They are unipolar and
require a convenient 5V power supply. Due to the reduction gearing, they are
slower than typical Nema-17 motors, but have sufficient speed for the
microscope stage movement. Additionally they can suffer from backlash, but
this is mitigated in the control software. They are accurate enough that it is
possible to move and focus objects using a step size of half a micron when
using a $100\times$ objective. They are controlled by a Sangaboard (v0.2 or
v0.3) [24]. This is an open-source, custom 3-axis motor controller board built
around an Arduino architecture. It connects to the Raspberry Pi via USB and
has three connections for each motor. It is necessary to power this controller
board from a separate 5V power supply so as not to overdraw current from the
Raspberry Pi.

###  4.2 Computing

Figure 12: The computing parts are housed tidily in the base of the
OpenFlexure Delta Stage. This computer generated render demonstrates a cutaway
of the base of the OFDS, showing the positions of the Raspberry Pi and
Sangaboard and their access holes.

####  4.2.1 Raspberry Pi

In the base of the OFDS is a Raspberry Pi, as shown in Figure 12. This
controls all the electronic components and can be used as a data store for the
images and videos. A Raspberry Pi 4 is recommended over previous models, due
to its more powerful processor, large RAM and still modest cost. The software
used is the OpenFlexure Microscope software (2.10)[25, 26]. This python-based
software uses a server-client architecture. This means that the OFDS can be
controlled either locally on the Raspberry Pi, or remotely over a http
connection. This is distributed as a pre-built Raspberry Pi OS SD card image
for ease of use and to provide users with a familiar interactive operating
system. The motors are controlled using a Sangaboard v0.3 [24] which is
positioned above the Raspberry Pi on 20 mm standoffs.

The Raspberry Pi Camera v2 is connected directly to the Raspberry Pi using a
ribbon cable into the camera port. The Raspberry Pi Camera v2 was chosen
because of its excellent integration with the Raspberry Pi. The OpenFlexure
Microscope software communicates with the camera using the picamerax library
[27]. This library takes advantage of the camera’s GPU based imaging pipeline
to provide additional functionality such as auto gain and exposure, white
balance and flat field correction with minimal processing time. If the OFDS is
being used as a stand alone device, a USB keyboard and mouse can be attached
and a display can be connected via HDMI.

####  4.2.2 Core software

The core software of the microscope, as described in [25], is a python based
flask app. The software controls the hardware (camera and motors), performs
basic imaging processing (camera calibration), launches a server and presents
a web API. Users can connect to the microscope via this API, to view a live
stream from the cameras, control the hardware and capture images. This means
it is possible to control the microscope from on-board the Raspberry Pi, or
from an external client computer over the internet. For most users, this is
done using from a web browser or OpenFlexure Connect, an electron app. For
more advanced users, the on-device scripting extension[28], the OpenFlexure
Python Client[29] and the OpenFlexure MATLAB client[30] allow scripts to send
commands directly to the OFDS. This flexibility suits the OFDS when it is
placed in Microbiological Safety Cabinets (MSCs) or incubators, so that the
researchers can control the device remotely. It is also possible to run
complex routines on the microscope, such as autofocusing, tiling scans,
triggering imaging and stage movement events based on image analysis.

####  4.2.3 Scripting automation through ‘Extensions’

It is possible to add “extensions” to the OpenFlexure server software. This
allows users to create additional functionality and to control additional
hardware. It is a minimal amount of python code that is executed on the
server, as if it were part of the core software. Extensions can create their
own web APIs and GUIs to enable users to activate certain additional
functionality.

The software to control the LED array was written as an extension [14]. Using
the extension functionality and the Adafruit drivers [22], the extension
presents a simple interface to run routines such as dark field imaging and
phase imaging.

##  5 Engineering considerations

The OFDS was carefully designed to meet our objective that the device is
convenient for researchers to replicate, at a reasonable cost, while including
features that are useful for general experimentation.

###  5.1 Design considerations

The optical design of the microscope is conventional, prioritising quality and
convenience over extreme cost minimisation. The brightfield version described
here costs $\approx$£244 ($336). The device is instantly recognisable as a
microscope, providing an intuitive experience for new users. The most
expensive parts of the OFDS are the optics and the electronics. By specifying
RMS objectives, we enable researchers to reuse old objective lenses, buy them
second-hand or purchase them for a low cost. The electronics chosen are also
convenient to source, well documented, and affordably priced. The hardware
chosen to assemble the device are commonly available machine screws, nuts and
standoffs. Although the device can be used for different purposes, it has been
designed so that the choices are not overwhelming for the end user. All
imaging modes use the same main body, base, and illumination dovetail. This
simplifies the design and makes it easier for users to change modes. The
design has been refined over many iterations, as we have learned from end
users about their typical use cases and issues, with careful consideration
made to quality of life improvements.

###  5.2 Manufacturing considerations

The device is designed to be manufactured by the end user. It was very
important therefore to make sure that it can be printed and assembled by
someone without a industrial workshop or a large amount of manufacturing
experience. 3D printing itself has opened up a new world of distributed
manufacturing, and the designs of the 3D printed parts were chosen to be able
to be printed on a range of 3D printers. They require no supports, meaning
clean-up after printing is easy. There are also not a large number of parts,
making manufacturing easier. For example, the main body is printed as one
single piece, making it easier to replicate. The device does not require
expensive or complicated tools for assembly, instead it can be put together
using hand tools. We have listened to the community about issues when
assembling the device, and made improvements to improve the experience.

###  5.3 Development infrastructure and continuous deployment

The 3D models are designed using OpenSCAD [31] and version control is managed
using git and GitLab. Because the OpenSCAD code is parameterised, it is
intuitive to adapt and change the dimensions of the designs to suit different
requirements. Using the principles outlined in [32], GitLab’s continuous
integration and continuous development (CI/CD) tools are used to automatically
compile the OpenSCAD code into STL files, and store them alongside the
assembly instructions for download from the server.

###  5.4 Documentation

Documentation is key to the success and replication of designs. GitBuilding
[33] is used to generate detailed and high quality instructions for printed
and assembling the OFDS, together with a bill of materials, photos and links
to download the STLs [21].

###  5.5 Open Source Hardware philosophy

At the heart of the OpenFlexure philosophy is the open-source nature of our
projects. The code for both the software and hardware has been available
online since the project’s inception and its design has been shaped because of
its open source nature. It now has an active community of users, who use it
for their own scientific experiments and instrument designs. Users from around
the world have already manufactured and used the OFDS, and have suggested
improvements and features they would like to use through the GitLab repository
[34], our forum [35] and social media. This iterative feedback process has
meant the OFDS has a track record of a reliable and replicable design, and so
it should be actively supported into the future. Being open-source has helped
it to become useful for more use cases than the maintainers could ever develop
and test.

##  6 Conclusion

In this paper, the overall design of the OFDS has been discussed and
rationalised. The construction and operation of the different optics modes
have been explained, and some sample images of each of the different modes
have been presented. The mechanics and kinematics of the ‘delta robot’ stage
have been described and justified. The efficacy of the open-source nature of
the project has been shown, alongside its development pipeline and CI/CD
tools. The appeal of the OFDS for biologists and physicists has also been
outlined, with some use cases.

Future work will include the development of more extensions to control the LED
array, to enable even more imaging applications. The optics module will also
be improved to allow easier objective lens and filter cube switching. The
community will continue to be listened to and improvements made, based on real
world applications of the device.

## Funding

This work was funded by the Engineering and Physical Sciences Research Council
[EP/R013969/1, EP/R011443/1]; Royal Society [URF\R1\180153, RGF\EA\181034]

## Data Availability

All materials and data are available at: https://gitlab.com/openflexure. All
the original data is on a Zenodo archive at:
https://doi.org/10.5281/zenodo.6225993. The archived versions of the software
and hardware described in this paper are available on Zenodo at: [36] and
[37].

## Disclosures

The authors declare no conflicts of interest.

## References

  * [1] Joel T. Collins, Joe Knapper, Julian Stirling, Joram Mduda, Catherine Mkindi, Valeriana Mayagaya, Grace A. Mwakajinga, Paul T. Nyakyi, Valerian L. Sanga, Dave Carbery, Leah White, Sara Dale, Zhen Jieh Lim, Jeremy J. Baumberg, Pietro Cicuta, Samuel McDermott, Boyko Vodenicharski, and Richard Bowman.  Robotic microscopy for everyone: the OpenFlexure microscope.  Biomedical Optics Express, 11(5):2447, 2020.
  * [2] Johann Rocholl.  Rostock (delta robot 3D printer).  https://www.thingiverse.com/thing:s17175, 2012.
  * [3] Pedro P. Vallejo Ramirez, Joseph Zammit, Oliver Vanderpoorten, Fergus Riche, Francois Xavier Blé, Xiao Hong Zhou, Bogdan Spiridon, Christopher Valentine, Simeon E. Spasov, Pelumi W. Oluwasanya, Gemma Goodfellow, Marcus J. Fantham, Omid Siddiqui, Farah Alimagham, Miranda Robbins, Andrew Stretton, Dimitrios Simatos, Oliver Hadeler, Eric J. Rees, Florian Ströhl, Romain F. Laine, and Clemens F. Kaminski.  OptiJ: Open-source optical projection tomography of large organ samples.  Scientific Reports, 9(1):1–9, 2019.
  * [4] Benedict Diederich, René Lachmann, Swen Carlstedt, Barbora Marsikova, Haoran Wang, Xavier Uwurukundo, Alexander S. Mosig, and Rainer Heintzmann.  A versatile and customizable low-cost 3D-printed open standard for microscopic imaging.  Nature Communications, 11(1):5979, 12 2020.
  * [5] Matthew Wincott, Richard M. Parton, Andrew Jefferson, Ian M. Dobbie, Martin J. Booth, and Ilan Davis.  Democratising ’Microscopi’: A 3D printed automated XYZT fluorescence imaging system for teaching, outreach and fieldwork.  Wellcome Open Research, 6, 2021.
  * [6] Hongquan Li, Hazel Soto-Montoya, Maxime Voisin, Lucas Fuentes Valenzuela, and Manu Prakash.  Octopi: Open configurable high-throughput imaging platform for infectious disease diagnosis in the field.  bioRxiv, 2019.
  * [7] Michael Eisenstein.  Microscopy made to order.  Nature Methods, 18(11):1277–1281, 2021.
  * [8] William Trager and James B. Jensen.  Human Malaria Parasites in Continuous Culture.  Science, 193(4254):673–675, 8 1976.
  * [9] Alessandro Esposito, Jean-Baptiste Choimet, Jeremy N. Skepper, Jakob M.A. Mauritz, Virgilio L. Lew, Clemens F. Kaminski, and Teresa Tiffert.  Quantitative Imaging of Human Red Blood Cells Infected with Plasmodium falciparum.  Biophysical Journal, 99(3):953–960, 8 2010.
  * [10] World Health Organization.  Basic Malaria Microscopy: Learner’s Guide.  World Health Organization, 2010.
  * [11] Richard W. Bowman, Boyko Vodenicharski, Joel T. Collins, and Julian Stirling.  Flat-Field and Colour Correction for the Raspberry Pi Camera Module.  Journal of Open Hardware, 4(1):1–9, 2020.
  * [12] Guoan Zheng, Christopher Kolner, and Changhuei Yang.  Microscopy refocusing and dark-field imaging by using a simple LED array.  Optics Letters, 36(20):3987, 2011.
  * [13] Lei Tian, Jingyan Wang, and Laura Waller.  3D differential phase-contrast microscopy with computational illumination using an LED array.  Optics Letters, 39(5):1326, 2014.
  * [14] LED grid illumination extension [Repository].  https://gitlab.com/openflexure/microscope-extensions/led-grid-illumination-extension, 2021\.
  * [15] Julius H. Rheinberg.  On an Addition to the Methods of Microscopical Research, by a New Way of Optically Producing Colour-Contrast Between an Object and its Background, or Between Definite Parts of the Object Itself.  Journal of the Royal Microscopical Society, 16(4):373–388, 1896\.
  * [16] D. K. Hamilton and C. J.R. Sheppard.  Differential phase contrast in scanning optical microscopy.  Journal of Microscopy, 133(1):27–39, 1984.
  * [17] Zachary F. Phillips, Michael Chen, and Laura Waller.  Single-shot quantitative phase microscopy with color-multiplexed differential phase contrast (cDPC).  PLoS ONE, 12(2), 2017.
  * [18] Guoan Zheng, Roarke Horstmeyer, and Changhuei Yang.  Wide-field, high-resolution Fourier ptychographic microscopy.  Nature Photonics, 7(9):739–745, 2013.
  * [19] James P. Sharkey, Darryl C.W. Foo, Alexandre Kabla, Jeremy J. Baumberg, and Richard W. Bowman.  A one-piece 3D printed flexure translation stage for open-source microscopy.  Review of Scientific Instruments, 87(2), 2016.
  * [20] Yu Li, Deyong Shang, and Yue Liu.  Kinematic modeling and error analysis of Delta robot considering parallelism error.  International Journal of Advanced Robotic Systems, 16(5):1–9, 2019\.
  * [21] OpenFlexure Delta Stage [Documentation].  https://openflexure.org/projects/deltastage/, 2022.
  * [22] Adafruit CircuitPython DotStar [Repository].  https://github.com/adafruit/Adafruit_CircuitPython_DotStar, 2021\.
  * [23] Raspberry Pi Camera Module 2.  https://www.raspberrypi.com/products/camera-module-v2/, 2021.
  * [24] Sangaboard [Repository].  https://gitlab.com/bath_open_instrumentation_group/sangaboard, 2021\.
  * [25] Joel T. Collins, Joe Knapper, Samuel J. McDermott, Filip Ayazi, Kaspar E. Bumke, Julian Stirling, and Richard W. Bowman.  Simplifying the OpenFlexure microscope software with the web of things.  Royal Society Open Science, 8(11), 2021.
  * [26] OpenFlexure Microscope Server [Repository].  https://gitlab.com/openflexure/openflexure-microscope-server, 2021\.
  * [27] picamerax.  https://github.com/labthings/picamerax, 2021.
  * [28] OpenFlexure Scripting Extension [Repository].  https://gitlab.com/openflexure/microscope-extensions/ofm-scripting, 2021\.
  * [29] OpenFlexure Microscope Python Client [Repository].  https://gitlab.com/openflexure/openflexure-microscope-pyclient, 2020\.
  * [30] OpenFlexure Microscope MATLAB Client toolbox [Repository].  https://gitlab.com/openflexure/openflexure-microscope-matlab-client, 2021\.
  * [31] OpenSCAD.  https://github.com/openscad/openscad/, 2021.
  * [32] Julian Stirling, Kaspar Bumke, Joel Collins, Vimal Dhokia, and Richard Bowman.  HardOps: utilising the software development toolchain for hardware design.  International Journal of Computer Integrated Manufacturing, 00(00):1–13, 2 2022.
  * [33] GitBuilding.  https://gitbuilding.io, 2021.
  * [34] OpenFlexure Microscope Extensions [Repository].  https://gitlab.com/openflexure/microscope-extensions, 2021.
  * [35] OpenFlexure forum.  https://openflexure.discourse.group/, 2021.
  * [36] Joel T. Collins, Julian Stirling, Kaspar Bumke, and Richard Bowman.  OpenFlexure Software Stack archive.  https://doi.org/10.5281/zenodo.5541934, 2021.
  * [37] Samuel McDermott, Richard Bowman, Julian Stirling, Joel T. Collins, and Joe Knapper.  OpenFlexure Delta Stage 3D Files archive.  https://doi.org/10.5281/zenodo.6048630, 2022.