# Co-training and Co-distillation for
Quality Improvement and Compression of Language Models

Hayeon Lee1 Rui Hou1 Jongpil Kim1
Davis Liang2 Hongbo Zhang1 Sung Ju Hwang3 Alexander Min1
Meta AI1 Abridge AI2 KAIST3
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Work
done while interning at Meta AI.

###### Abstract

Knowledge Distillation (KD) compresses computationally expensive pre-trained
language models (PLMs) by transferring their knowledge to smaller models,
allowing their use in resource-constrained or real-time settings. However,
most smaller models fail to surpass the performance of the original larger
model, resulting in sacrificing performance to improve inference speed. To
address this issue, we propose Co-Training and Co-Distillation (CTCD), a novel
framework that improves performance and inference speed together by co-
training two models while mutually distilling knowledge. The CTCD framework
successfully achieves this based on two significant findings: 1) Distilling
knowledge from the smaller model to the larger model during co-training
improves the performance of the larger model. 2) The enhanced performance of
the larger model further boosts the performance of the smaller model. The CTCD
framework shows promise as it can be combined with existing techniques like
architecture design or data augmentation, replacing one-way KD methods, to
achieve further performance improvement. Extensive ablation studies
demonstrate the effectiveness of CTCD, and the small model distilled by CTCD
outperforms the original larger model by a significant margin of 1.66 on the
GLUE benchmark.

##  1 Introduction

In recent years, the high computational cost of pre-trained language models
(PLMs) Radford et al. (2019); Yang et al. (2019); Dai et al. (2019); Shoeybi
et al. (2019); Li et al. (2020); Brown et al. (2020) become a constraint in
serving resource-limited or real-time applications. Knowledge Distillation
(KD) (Hinton et al., 2015; Romero et al., 2015) is a popular model compression
technique to tackle this issue, with a smaller model (student) learning
(distilling) from a larger model (teacher).

The ideal scenario would involve compressing the teacher to match the size of
the small student without any performance drop. However, despite intense
research in the area of KD (Turc et al., 2019; Tsai et al., 2019; Tang et al.,
2019; Jiao et al., 2020; Sanh et al., 2019a; Sun et al., 2019; Wang et al.,
2020b, a), none of the existing methods have successfully avoided performance
degradation during the distillation process. Some approaches have attempted to
mitigate the performance gap by incorporating external factors. For instance,
Jiao et al. (2020) incorporates data augmentation, while Wang et al. (2020b,
a) focuses on designing student architecture. However, these approaches have
limitations as they only provide supplementary techniques rather than
addressing the fundamental issue of performance loss in KD. This raises an
interesting question: Can we compress a model without scarifying the
performance through KD?

Figure 1: Task-agnostic KD method The proposed framework are task-agnostic KD
methods that perform KD during the pre-training stage. This allows the
distilled model to be deployed and fine-tuned on various downstream tasks.

In this work, we propose a novel framework called Co-Training and Co-
Distillation (CTCD) for improving the performance of a language model while
compressing it through KD. CTCD involves jointly training the teacher and the
student models, allowing them to transfer knowledge to each other
bidirectionally, from the teacher to the student and vice versa. Our work
uncovers two key findings within the CTCD framework: Firstly, we demonstrate
that transferring knowledge from the smaller model to the larger model during
co-training significantly improves the performance of the larger model. In
other words, by employing knowledge distillation (KD), we enhance the
performance of the teacher model compared to standalone training. This is
significantly different from conventional one-way KD, where the teacher model
cannot benefit from the distillation process since it is no longer trained or
distilled. Secondly, the improved performance of the larger model leads to
further enhancements in the performance of the smaller model through KD from
the teacher model to the student model. These novel findings enable the
smaller model to surpass the independently trained larger model while
maintaining its inference efficiency within the CTCD framework. Moreover, the
CTCD framework can be combined orthogonally with existing techniques for
external factors such as student architecture design Wang et al. (2020b, a) or
data augmentation Jiao et al. (2020). We achieve this by replacing traditional
one-way KD methods with the proposed CTCD framework, which holds promise for
significant performance improvements in language modeling tasks.

In addition to our CTCD framework, we introduce a Community KD that utilizes
CTCD framework with the pre-trained teacher model by combining CTCD with
conventional KD. Community KD distills knowledge from two students to each
other, as well as from the pre-trained teacher to each student, during the co-
training of the students. We show that distilling from the other student as
well as the pre-trained teacher is better than only one-way distillation from
the pre-trained teacher, as is done in conventional KD. Note that the
inference cost on a task is the same as in conventional KD, as we take one of
the students distilled by Community KD when deploying it to the task.

We validate the effectiveness and efficiency of our CTCD on the GLUE benchmark
Wang et al. (2019), which contains different language understanding tasks. As
shown in Figure 1, We focus on task-agnostic KD scenarios, where KD occurs
during the pre-training stage. Then, we fine-tune the distilled PLMs on each
downstream task and evaluate their performance. This is more challenging
compared to task-specific KD, as it requires substantial computational
resources to train PLMs on a large text corpus, and the distilled knowledge
should be transferable across diverse downstream tasks. Our extensive ablation
study revealed that the larger model benefits from the distillation of the
smaller model, and the performance improvement of the larger model further
enhances the performance of the smaller model. In our experiments, the student
model compressed by CTCD framework obtained 1.66 higher gain than the original
large model trained using a stand-alone method, demonstrating that our
approach can improve model quality and inference efficiency concurrently.

In summary, our contributions are as follows:

  * •

We propose a novel knowledge distillation framework called Co-Training and Co-
Distillation (CTCD) framework to improve the performance of models while
compressing them through KD.

  * •

Through our experiments, we demonstrate that distilling knowledge from the
smaller model to the larger model during co-training improves the performance
of the larger model.

  * •

Additionally, we highlight that the enhanced performance of the larger model
further boosts the performance of the smaller model, resulting in improved
performance for both models.

  * •

We provide valuable insights into adjusting loss weights and the length of the
training phase for the effective application of the CTCD framework through
extensive ablation study.

Figure 2: Co-Training and Co-Distillation (CTCD) During training, the teacher
and student models can learn more effectively by comparing their prediction
outputs not only against the ground truth but also against each other’s
predictions. We refer to the former as a "hard" loss and the latter as a
"soft" loss. For example, the student soft loss captures the distance between
the student’s prediction and the teacher’s prediction, and vice versa. This
co-training and co-distillation approach improve the performance of the
teacher model, which in turn benefits the performance of the student model.

##  2 Related Work

#### One-way Knowledge Distillation

Knowledge distillation (KD) (Hinton et al., 2015) is a model compression
technique in which a smaller student model distills knowledge from a larger,
pre-trained teacher model. Recently, many researchers have explored KD to
address a high computational complexity of a language model resulting from its
increasing size. This includes task-agnostic KD methods for the pre-training
stage (Sanh et al., 2019a; Wang et al., 2020b, a), task-specific KD method for
the fine-tuning stage (Sun et al., 2019), and both of pre-training and fine-
tuning stages (Jiao et al., 2020). Additionally, Wang et al. (2020b, a) have
redesigned the architectures of student language models. However, the one-way
KD can lead to a loss of knowledge, resulting in smaller models that generally
have difficulty matching the performance of larger models, leading to
performance degradation. In contrast, our CTCD can improve the quality of both
teacher and student models, making it possible for the student to achieve the
same quality as the original teacher model.

#### Reversed Knowledge Distillation

Recently, researchers (Yuan et al., 2020; Qin et al., 2022; Lee et al., 2023)
have demonstrated that reversed Knowledge Distillation (reversed KD), which
transfers knowledge from a smaller or poorer model to a larger model, can
improve the performance of the student model. In particular, Qin et al.
(2022); Lee et al. (2023) investigated the application of reversed KD in the
PLMs, showing that a larger model can benefit from a poorer and pre-trained
model for a specific downstream task. Inspired by the success of reversed KD,
we design a co-distillation framework that includes reversed KD to improve the
performance of the teacher model by distilling knowledge from the smaller
student model. Unlike existing reversed KD methods, which are limited to
improving the performance of the larger model, our proposed co-distillation
framework can achieve both performance improvement and model compression, by
showing a better-quality teacher leads to a better-quality student.

##  3 Co-training and Co-distillation

We first introduce the concepts of co-training and co-distillation briefly:

#### Co-training

trains two (different-sized) models (e.g., a teacher and student) concurrently
with the goal of achieving similar model quality.

Figure 3: Community KD Different from conventional KD, where each student
learns from the prediction of the pre-trained teacher only, the proposed
approach learns each student from both prediction of the pre-trained teacher
and prediction of another student during co-training. Note that since we take
one of the pre-trained students to adapt it to downstream tasks, the inference
cost is the same as the student training with the conventional KD.

#### Co-distillation

transfers knowledge in both directions between two models (e.g., a teacher and
student), during co-training.

Figure 2 illustrates how co-trained models learn together by comparing their
prediction outputs against predictions from each other and to the hard labels
(or “ground truth”). We refer to the former as a “soft” loss and the latter as
a “hard” loss. For instance, the soft loss of the student model measures the
accuracy of the student’s prediction by considering the teacher’s prediction
as a soft label, and vice versa.

#### Task Formulation

Suppose that we are given a classification task with $K$ classes. For each
training instance $x$ and its ground truth label $y$, we denote that the
ground truth distribution over the labels is $q(k|x)$ ($q(k)$ for simplicity)
where for each label $k\in\\{1...K\\}$, $q(y)=1$ and $q(k)=0$ for all $k\neq
y$. For each $x$, the teacher model $t_{\bm{\phi}}$ parameterized by
$\bm{\phi}$ and the student model $s_{\bm{\theta}}$ parameterized by
$\bm{\theta}$ predict the probability of each label $k$ as
$p^{\tau}_{\bm{\phi}}(k|x)$ and $p^{\tau}_{\bm{\theta}}(k|x)$, respectively as
follows:

$\begin{gathered}p^{\tau}_{\bm{\phi}}(k|x)=f(\bm{z}^{t})=\frac{\exp(z_{k}^{t}/\tau)}{\sum{{}^{K}_{i=1}}\exp(z^{t}_{i}/\tau)}\end{gathered}$
$\begin{gathered}p^{\tau}_{\bm{\theta}}(k|x)=f(\bm{z}^{s})=\frac{\exp(z^{s}_{k}/\tau)}{\sum{{}^{K}_{i=1}}\exp(z^{s}_{i}/\tau)}\end{gathered}$

where $f$ is the softmax function,
$\bm{z}^{t}=\\{z_{i}^{t}\\}_{i=1}^{K}=t_{\bm{\phi}}(x)$ is the output logit of
the teacher model, $\bm{z}^{s}=\\{z_{i}^{s}\\}_{i=1}^{K}=s_{\bm{\theta}}(x)$
is the output logit of the student model, and $\tau$ is the temperature to
soften $p_{\bm{\phi}}(k)$ and $p_{\bm{\theta}}(k)$.

The proposed objective $\mathcal{L}_{CTCD}(\bm{\theta},\bm{\phi})$ consists of
a normal KD objective $\mathcal{L}_{KD:t\rightarrow s}$ to distill knowledge
from the teacher model to the student model and a reversed KD objective
$\mathcal{L}_{ReKD:s\rightarrow t}$ to distill knowledge from the student
model to the teacher model, during their co-training.

#### Teacher $\rightarrow$ Student

The normal KD objective $\mathcal{L}_{KD:t\rightarrow
s}(\bm{\theta},\bm{\phi})$ aims to train the student model by minimizing a
weighted sum of the cross-entropy loss $H(q,p_{\bm{\theta}})$ between the
ground truth $q$ and student prediction $p_{\bm{\theta}}$ and Kullback-Leibler
divergence (KL divergence) $D(p^{\tau}_{\bm{\phi}},p^{\tau}_{\bm{\theta}})$
between the predictions of the student and the teacher as follows:

$\begin{gathered}\mathcal{L}_{KD}(\bm{\theta},\bm{\phi})=\alpha_{h}\cdot
H(q,p_{\bm{\theta}})+\alpha_{s}\cdot
D(p^{\tau}_{\bm{\phi}},p^{\tau}_{\bm{\theta}})\end{gathered}$ (1)

where

$\begin{gathered}H(q,p_{\bm{\theta}})=-\sum_{k=1}^{K}q(k)\log(p_{\bm{\theta}}(k)),\\\
D(p^{\tau}_{\bm{\phi}},p^{\tau}_{\bm{\theta}})=\sum_{k=1}^{K}p^{\tau}_{\bm{\phi}}(k)\cdot\log\frac{p^{\tau}_{\bm{\phi}}(k)}{p^{\tau}_{\bm{\theta}}(k)},\end{gathered}$

$\alpha_{h}$ and $\alpha_{s}$ are weighting hyper-parameter values for the
cross-entropy loss and KL divergence, respectively. We regard the cross-
entropy loss $H(q,p_{\bm{\theta}})$ as the hard loss for the student model, KL
divergence $D(p^{\tau}_{\bm{\phi}},p^{\tau}_{\bm{\theta}})$ as the soft loss
for the student model, and following BERT (Devlin et al., 2019),
$H(q,p_{\bm{\theta}})$ denotes the Masked Language Modeling loss (MLM loss).
In the KD objective, we consider the teacher parameters $\bm{\phi}$ as
constant since we only train the student parameters $\bm{\theta}$ while the
teacher model $t_{\bm{\phi}}$ is fixed:

$\begin{gathered}\mathcal{L}_{KD:t\rightarrow
s}(\bm{\theta},\text{StopG}(\bm{\phi}))\end{gathered}$ (2)

where $\text{StopG}(x)$ denotes that we do not compute the gradient of $x$. In
the conventional KD method, Equation (2) is the final objective to learn the
student model only with the pre-trained teacher model.

#### Student $\rightarrow$ Teacher

Different from such a one-way KD method, we introduce the reversed KD
objective $\mathcal{L}_{ReKD:s\rightarrow
t}(\text{StopG}(\bm{\theta}),\bm{\phi})$ to train the teacher model
$t_{\bm{\phi}}$ as follows:

$\begin{gathered}\mathcal{L}_{ReKD:s\rightarrow
t}(\text{StopG}(\bm{\theta}),\bm{\phi})=\\\ \beta_{h}\cdot
H(q,p_{\bm{\phi}})+\beta_{s}\cdot
D(p^{\tau}_{\bm{\theta}},p^{\tau}_{\bm{\phi}})\end{gathered}$ (3)

where $\beta_{h}$ and $\beta_{s}$ are weighting hyper-parameter values of the
hard loss $H(q,p_{\bm{\phi}})$ and soft loss
$D(p^{\tau}_{\bm{\theta}},p^{\tau}_{\bm{\phi}})$ for the teacher model,
respectively. By minimizing KL divergence
$D(p^{\tau}_{\bm{\theta}},p^{\tau}_{\bm{\phi}})$ between the predictions of
the student model ($p^{\tau}_{\bm{\theta}}$) and the teacher model
($p^{\tau}_{\bm{\phi}}$), the teacher model learns from the student model. In
the reversed KD objective, we only train the teacher model by applying
$\text{StopG}(x)$ to the gradient of the student parameters $\bm{\theta}$.

#### Co-training

With the Equations (2) and (3), we get the final objective
$\mathcal{L}_{CTCD}(\bm{\theta},\bm{\phi})$ as follows:

$\begin{gathered}\bm{\theta}^{*},\bm{\phi}^{*}=\operatorname*{argmin}_{\bm{\theta},\bm{\phi}}\mathcal{L}_{CTCD}(\bm{\theta},\bm{\phi})=\\\
\mathcal{L}_{KD}(\bm{\theta},\text{StopG}(\bm{\phi}))+\mathcal{L}_{ReKD}(\text{StopG}(\bm{\theta}),\bm{\phi})\end{gathered}$
(4)

#### Adapting to Downstream Task

After model co-training/-distillation, the trained smaller (student) model
$s_{\bm{\theta}^{*}}$ with trained parameter $\bm{\theta}^{*}$ can be deployed
for multiple downstream tasks to improve inference efficiency. To fine-tune
the model for a specific downstream task, we adapt the trained parameter
$\bm{\theta}^{*}$ using the dataset for that task.

##  4 Community KD

Furthermore, we introduce an advanced CTCD application named Community KD that
can utilize CTCD framework with the pre-trained teacher model, as shown in
Figure 3. It consists of a pre-trained teacher $t_{\bm{\phi}^{*}}$ with pre-
trained parameters $\bm{\phi}^{*}$ and two students $s_{\bm{\theta_{1}}}$ and
$s_{\bm{\theta_{2}}}$ parameterized by $\bm{\theta_{1}}$ and
$\bm{\theta_{2}}$, respectively. During the co-training of two students, each
student learns from the hard labels, soft labels generated from the pre-
trained teacher predictions, and soft labels generated from other student
predictions. In other words, we conduct one-way knowledge distillation from
the pre-trained teacher to each student by minimizing KL divergence between
the teacher prediction and predictions of each student
$D(p^{\tau}_{\bm{\theta_{1}}},p^{\tau}_{\bm{\phi}^{*}})$ and
$D(p^{\tau}_{\bm{\theta_{2}}},p^{\tau}_{\bm{\phi}^{*}})$ and co-distillation
between students in both directions by minimizing
$\mathcal{L}_{CTCD}(\bm{\theta_{1}},\bm{\theta_{2}})$. The final objective
$\mathcal{L}_{CM}(\bm{\theta_{1}},\bm{\theta_{2}},\text{StopG}(\bm{\phi}^{*}))$
is as follows:

$\begin{gathered}\bm{\theta_{1}}^{*},\bm{\theta_{2}}^{*}=\operatorname*{argmin}_{\bm{\theta_{1}},\bm{\theta_{2}}}\mathcal{L}_{CM}(\bm{\theta_{1}},\bm{\theta_{2}},\text{StopG}(\bm{\phi}^{*}))=\\\
\mathcal{L}_{CTCD}(\bm{\theta_{1}},\bm{\theta_{2}})+D(p^{\tau}_{\bm{\theta_{1}}},p^{\tau}_{\bm{\phi}^{*}})+D(p^{\tau}_{\bm{\theta_{2}}},p^{\tau}_{\bm{\phi}^{*}})\end{gathered}$
(5)

We select one of the two students distilled by Community KD
($\bm{\theta}^{*}=\bm{\theta_{1}}^{*}$ or
$\bm{\theta}^{*}=\bm{\theta_{2}}^{*}$) and fine-tune the selected single
student $s_{\bm{\theta}^{*}}$ for downstream tasks, resulting that the
inference cost does not increase compared with the conventional KD method.

|  | Performance | GAP w/ Teacher
---|---|---|---
Original Teacher | 78.06 | -
Student | One-way Distil. (10 epoch) | 77.46 | -0.60
CTCD (10 epoch) | 77.94 | -0.12
One-way Distil. (20 epoch) | 78.39 | +0.33
CTCD (20 epoch) | 79.12 | +1.66

Table 1: Average performance on dev sets of GLUE benchmark The student
distilled by CTCD significantly outperforms the original teacher trained using
the stand-alone method, achieving a higher gain of 1.66.

##  5 Experiment

We present a comprehensive analysis of the proposed CTCD method through
empirical experiments. In Section 5.1, we validate our CTCD method by
comparing the performance of small models distilled by CTCD to the original
large model on the GLUE benchmark (Wang et al., 2019). In Section 5.2, we
analyze the impact of co-distillation by adjusting loss weights for the soft
losses of a student and a teacher. In Section 5.3, we study the impact of
training length on CTCD method, allowing us to determine the optimal training
length for CTCD method. In Section 5.4, we demonstrate the efficacy of
Community KD by comparing the performance of a model distilled by the
Community KD to a model distilled by the one-way KD.

#### Implementation details

We use a learning rate of 5e-4, linear warm-up of 5%, AdamW optimizer
(Loshchilov and Hutter, 2019), and batch size of 128 with A100 GPUs for pre-
training. We train the teacher and student models from scratch for 20 epochs
in Section 5.1. To analyze the effectiveness of CTCD method, we train the
teacher and student models for 10 epochs and 20 epochs in Section 5.2 and
Section 5.3, respectively. In Section 5.4, we train the models for 3 epochs
after parameter remapping, which is the same as in conventional one-way KD
method (Sanh et al., 2019a).

#### Training Time

For the one-way distillation, we need 1 GPU day to train the teacher for 10
epochs and 1.3 GPU days to distill knowledge from the pre-trained teacher to
the student for another 10 epochs, which consumes a total of 2.3 days. For
CTCD, it takes 3 GPU days to train both teacher and student models from
scratch for 10 epochs. We use the automatic mixed precision (AMP) of PyTorch
(Paszke et al., 2019) to accelerate training for all our models.

#### Dataset

To validate our CTCD method, we use a reduced dataset (30M) created by
uniformly sampling 1 out of every 4 sentences from the original pre-training
dataset (BookCorpus (Zhu et al., 2015) \+ Wikipedia (Foundation, )) used in
the conventional one-way KD method (Sanh et al., 2019a). We evaluate our
distilled models on the dev sets of the GLUE benchmark Wang et al. (2019),
which consists of nine sentence-level classification tasks. In Section 5.4, we
use the original pre-training dataset to train Community KD method.

Figure 4: Average performance on dev sets of GLUE benchmark after training 10
epochs Learning from the student improves the performance of the teacher by an
average of 1.88. Such improvement in teacher performance leads to improvement
in student quality from 77.46 to 77.94.

#### Model Architecture

We use a 6-layer BERT (Devlin et al., 2019) model as the teacher and a 4-layer
BERT model as the student to analyze the effectiveness and efficiency of our
CTCD method. In Section 5.4, we use a pre-trained BERT-base model as the
teacher and a 6-layer BERT model as the student.

(a) Teacher Performance

(b) Student Performance

Figure 5: Adjusting Loss Weight We investigate the impact of the distillation
for the teacher model and student model by adjusting loss weights
($\alpha_{h},\alpha_{s},\beta_{h},\beta_{s}$) for hard loss and soft loss. (a)
We (co-)train the teacher model distilling knowledge from the student by
fixing $\alpha_{h}:\alpha_{s}=1:1$ and varying $\beta_{h}:\beta_{s}$ on the
large text corpus. (b) We (co-)train the student model distilling knowledge
from the teacher by fixing $\beta_{h}:\beta_{s}=1:1$ and varying
$\alpha_{h}:\alpha_{s}$ on the large text corpus. Then we report the average
performance of each pre-trained model after fine-tuning it on downstream tasks
(dev sets) of GLUE benchmark.  Figure 6: Length of Training We pre-trained
student models under two different training lengths 10/20 epochs while
distilling knowledge from teacher models via ours or the conventional KD
method. Then we adapt pre-trained student models on CoLA task. With enough
longer training (20 epoch), the student model distilled by ours significantly
outperforms the student model distilled by the conventional KD method, with a
higher gain of 5.22.

###  5.1 Could Knowledge Distillation Help Improve Performance?

In Table 1 and Figure 4, we show the average performance of models trained
using different methods on a large text corpus and fine-tuned against the GLUE
benchmark. Stand-alone trains a model without using any knowledge
distillation. Co-training & One-way distillation trains teacher and student
models together from scratch, with knowledge only flowing from the teacher to
the student. Co-training & Co-distillation (Ours) is CTCD method, which trains
both teacher and student models together from scratch and distills knowledge
between each other in both directions. For distillation methods, we set the
weights of the hard losses for the teacher and student to 1. The weights of
the soft losses are chosen from the set $\\{0.5,1,2,4\\}$, and the results are
reported with the best-performing weights.

#### 1) Overall Results

As shown in Table 1, the student distilled by CTCD method significantly
outperforms the original teacher trained using the stand-alone method on the
average performance of the GLUE benchmark, achieving a higher gain of 1.66.
Furthermore, the student model distilled by our CTCD method outperforms the
student distilled by the one-way distillation method on the average
performance of the GLUE benchmark. After training for 10 and 20 epochs, the
student distilled by our CTCD method consistently has higher gains than the
student distilled by one-way distillation, as 77.46 vs. 77.94 and 78.39 vs.
79.12, respectively.

#### 2) Does learning from a small and weak student provide a performance
benefit to the teacher?

As shown in Figure 4, the distillation of knowledge from the student model to
the teacher model has been shown to significantly improve the quality of the
teacher model, with an average increase of 1.88 compared to teacher training
methods that do not incorporate such distillation process (such as Stand-alone
and Co-training & one-way distillation).

#### 3) Is the teacher’s performance improvement reflected in the student’s
performance improvement?

Furthermore, we find that a better teacher leads to further performance
improvement of the student from 77.46 to 77.94. The results demonstrate that
the distillation process successfully improves the performance of both the
teacher and student. The student trained with our CTCD method achieves better
performance than students trained with the Stand-alone or Co-training & one-
way distillation methods, with an average improvement of 0.88 and 0.47,
respectively.

###  5.2 In-depth Study 1: Adjusting Loss Weights

In this Section, we investigate the impact of distillation on the performance
of both the student and the teacher by adjusting loss weights
($\alpha_{h},\alpha_{s},\beta_{h},\beta_{s}$) for hard loss and soft loss. For
example, setting $\alpha_{h}:\alpha_{s}=1:4$ emphasizes learning from the
teacher’s knowledge (i.e., the soft loss for the student) 4$\times$ more than
the ground truth label distribution (i.e., the hard loss for the student),
during co-training. This allows us to better understand the effect of
distillation on each model and optimize their performance.

#### Teacher side

In Figure 5(a), we co-train the teacher model using distillation to transfer
knowledge from the student model. We fix the weighting values for the losses
of the student to $\alpha_{h}:\alpha_{s}=1:1$ and vary the weighting values
for the losses of the teacher, $\beta_{h}:\beta_{s}$, while training on a
large text corpus. We then evaluate the average performance of the pre-trained
teacher models on downstream tasks from the dev sets of GLUE benchmark.

Our results show that co-training the teacher with distillation outperforms
training the teacher alone, regardless of the weighting values for the soft
loss, by obtaining higher gains of 0.36, 0.77, 1.80, and 1.02 for
$\beta_{h}:\beta_{s}=1:1$, $1:2$, $1:4$, and $4:4$, respectively.
Additionally, we find that giving more weight to the soft loss of the teacher
during training
($\alpha_{h}:\alpha_{s}:\beta_{h}:\mathbf{\beta_{s}}=1:1:1:\mathbf{1}\rightarrow
1:1:1:\mathbf{2}\rightarrow 1:1:1:\mathbf{4}$) leads to improved performance,
with an average score of $78.42\rightarrow 78.83\rightarrow 79.86$.
Furthermore, we observe that emphasizing only the soft loss of the teacher
($1:1:1:4$) yields better performance than emphasizing both the hard and soft
losses of the teacher ($1:1:4:4$), with an average score of $79.86$ vs.
$79.09$.

Downstream Task |  | MNLI | QQP | QNLI | SST-2 | CoLA | STSB | MRPC | RTE | Average
---|---|---|---|---|---|---|---|---|---|---
Metric | AMP | Acc. | Acc. | Acc. | Acc. | Matthew. | Pearson. | Spear. | F1 | Acc. | Acc. | Acc.
Dataset Size |  | 392.7k | 363.8k | 104.7k | 67.3k | 8.5k | 5.7k | 3.7k | 2.5k |
Teacher | BERT (109M) |  | 84.17 | 90.89 | 90.68 | 91.86 | 57.54 | 88.84 | 88.56 | 89.31 | 85.04 | 65.34 | 83.23
Student (67M) | One-way KD Sanh et al. (2019b) | FP32 | 81.93 | 90.05 | 87.72 | 90.94 | 52.03 | 86.28 | 86.07 | 87.94 | 82.59 | 57.76 | 80.33
Ours: Student 1 | FP16 | 81.88 | 90.17 | 88.24 | 91.51 | 54.82 | 86.70 | 86.49 | 89.76 | 85.29 | 59.21 | 81.40
Ours: Student 2 | FP16 | 81.34 | 89.75 | 88.37 | 90.71 | 56.08 | 86.42 | 86.44 | 89.80 | 85.29 | 59.20 | 81.34

Table 2: Efficacy of Community KD The pre-trained BERT and 6-layer BERT is the
teacher model and student architecture, respectively, for both ours and the
conventional one-way KD method. We fine-tune the distilled students on dev
sets of GLUE benchmark. We observe that learning from the soft knowledge of
different student model improves performance over the conventional one-way KD
method on most downstream tasks.

#### Student side

We find that the student model’s performance is not sensitive to the weighting
values for the hard and soft losses of the student, ($\alpha_{h}:\alpha_{s}$).
Regardless of the chosen values, co-training the student with distillation
consistently improves its performance compared to training the student alone.
For instance, when we emphasize the soft loss of the student by increasing the
weighting value for ($\alpha_{s}$) as $1:\mathbf{1}:1:1\rightarrow
1:\mathbf{2}:1:1\rightarrow 1:\mathbf{4}:1:1$, we observe similar levels of
performance for the student model.

###  5.3 In-depth Study 2: Length of Training

We studied the impact of co-training length on the effectiveness of the CTCD
method (see Figure 6). We find that longer training leads to improved
performance, as demonstrated by our experiments using two different training
lengths: 10 epochs and 20 epochs. After pre-training the student models with
these different lengths, we adapted them to the CoLA downstream tasks and
evaluated their performance using Matthew Correlation.

#### Results

By increasing the (co-)training length from 10 epochs to 20 epochs, CTCD
(Ours) significantly improves the performance of the student model from 35.02
to 46.23, with a gain of 11.21. This outperforms the conventional KD method,
which only achieves a gain of 0.07 from 40.94 to 41.01, despite a longer
training time. The conventional KD method relies on a pre-trained teacher
model to train the student model, which allows for fast convergence but limits
the learning of the student model. In contrast, the CTCD method allows for
additional performance gains for both the teacher and student models by
enabling them to learn and grow together during co-training. This can provide
further benefits to the student model’s performance with longer co-training.

###  5.4 Efficacy of Community KD

We compare the proposed Community KD with the conventional one-way KD method
Sanh et al. (2019a). To ensure a fair comparison, we use the pre-trained BERT-
base as the teacher model for both methods and the 6-layer BERT as the
student, which is the same architecture used in the conventional one-way KD
method. As described in Section 4, we train two student models concurrently
and they learn from the pre-trained BERT, the ground truth labels, and each
other’s knowledge. Note that since we fine-tune one of the two students
distilled by Community KD for downstream tasks, the inference cost is the same
as the conventional one-way KD method. In Table 2, we report the results of
BERT and the conventional one-way KD method using checkpoints provided by
Hugging Face (HuggingFace, ) and both students (Ours: Student 1 and Ours:
Student 2) on the dev sets of the GLUE benchmark. We apply Automatic Mixed
Precision (AMP) (Paszke et al., 2019) to Community KD, which typically speeds
up training but may hurt performance.

#### Results

The results presented in Table 2 shows that Community KD, leads to improved
performance on downstream tasks such as QQP, QNLI, SST-2, CoLA, STSB, MRPC,
and RTE, even when applying quantization techniques. Specifically, the average
performance gain of the student model distilled using our Community KD method
is 1.04 (1.2%) higher than that of the student model distilled by the
conventional one-way KD method. This suggests that incorporating knowledge
distillation from both a student model and a pre-trained teacher model is more
effective than only using knowledge distillation from the pre-trained teacher
model.

##  6 Limitations & Future Work

#### Limitations

The proposed method co-train models from scratch and may require a longer pre-
training time than the conventional KD method. However, as we described in
Section 5.3, when the student model is trained long enough with its teacher,
it can outperform the models trained with the conventional KD on the
downstream task. The proposed co-training method may increase the overall
training cost compared with one-way distillation, and it may become a
performance bottleneck depending on training resource constraints. However,
note that CTCD can improve model quality while having the same inference cost
as the one-way distillation on downstream tasks.

#### Future Work

1) Architecture Sharing. Models can share some of their architectures by
reusing the output of such architectures and updating them together during
back-propagation. This may help reduce the additional computing and memory
overhead incurred by model co-training, while improving the model quality,
especially for the student model. 2) Integration of Student Architecture
Design and Data Augmentation. Future research can focus on effectively
combining the CTCD framework with student architecture design and data
augmentation techniques. This integration provides a promising alternative to
traditional one-way knowledge distillation methods, leading to significant
improvements in language modeling tasks.

##  7 Conclusion

The size and complexity of pre-trained language models (PLMs) can hinder their
practicality for online downstream tasks. To address this, we introduced a
novel framework called co-training and co-distillation (CTCD). By training
models of different sizes together and extracting inter-model knowledge in
both directions, the proposed CTCD framework improves both model efficiency
and performance. The proposed framework overcomes the trade-off between
efficiency and performance in traditional one-way knowledge distillation
methods. Notably, our compressed models achieved an impressive gain of 1.66 on
the GLUE benchmark, outperforming large models trained using standalone
methods.

## References

  * Brown et al. (2020) Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020.  Language models are few-shot learners.  In _NeurIPS_.
  * Dai et al. (2019) Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc Le, and Ruslan Salakhutdinov. 2019.  Transformer-XL: Attentive language models beyond a fixed-length context.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 2978–2988, Florence, Italy. Association for Computational Linguistics.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers)_ , pages 4171–4186.
  * (4) Wikimedia Foundation.  Wikimedia downloads.
  * Hinton et al. (2015) Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al. 2015.  Distilling the knowledge in a neural network.  _arXiv preprint arXiv:1503.02531_ , 2(7).
  * (6) HuggingFace.  Huggingface.
  * Jiao et al. (2020) Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. 2020.  TinyBERT: Distilling BERT for natural language understanding.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 4163–4174, Online. Association for Computational Linguistics.
  * Lee et al. (2023) Hayeon Lee, Rui Hou, Jongpil Kim, Davis Liang, Sung Ju Hwang, and Alexander Min. 2023.  A study on knowledge distillation from weak teacher for scaling up pre-trained language models.  _arXiv preprint arXiv:2305.18239_.
  * Li et al. (2020) Chunyuan Li, Xiang Gao, Yuan Li, Xiujun Li, Baolin Peng, Yizhe Zhang, and Jianfeng Gao. 2020.  Optimus: Organizing sentences via pre-trained modeling of a latent space.  In _EMNLP_.
  * Loshchilov and Hutter (2019) Ilya Loshchilov and Frank Hutter. 2019.  Decoupled weight decay regularization.  In _7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019_. OpenReview.net.
  * Paszke et al. (2019) Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019.  Pytorch: An imperative style, high-performance deep learning library.  In _Advances in Neural Information Processing Systems 32_ , pages 8024–8035. Curran Associates, Inc.
  * Qin et al. (2022) Yujia Qin, Yankai Lin, Jing Yi, Jiajie Zhang, Xu Han, Zhengyan Zhang, Yusheng Su, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. 2022.  Knowledge inheritance for pre-trained language models.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 3921–3937, Seattle, United States. Association for Computational Linguistics.
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019.  Language models are unsupervised multitask learners.  _OpenAI blog_ , 1(8):9.
  * Romero et al. (2015) Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. 2015.  Fitnets: Hints for thin deep nets.  In _In Proceedings of ICLR_.
  * Sanh et al. (2019a) Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019a.  Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.  _arXiv preprint arXiv:1910.01108_.
  * Sanh et al. (2019b) Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019b.  Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.  _arXiv preprint arXiv:1910.01108_.
  * Shoeybi et al. (2019) Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. 2019.  Megatron-lm: Training multi-billion parameter language models using model parallelism.  _arXiv preprint arXiv:1909.08053_.
  * Sun et al. (2019) Siqi Sun, Yu Cheng, Zhe Gan, and Jingjing Liu. 2019.  Patient knowledge distillation for BERT model compression.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 4323–4332, Hong Kong, China. Association for Computational Linguistics.
  * Tang et al. (2019) Raphael Tang, Yao Lu, Linqing Liu, Lili Mou, Olga Vechtomova, and Jimmy Lin. 2019\.  Distilling task-specific knowledge from bert into simple neural networks.  _arXiv preprint arXiv:1903.12136_.
  * Tsai et al. (2019) Henry Tsai, Jason Riesa, Melvin Johnson, Naveen Arivazhagan, Xin Li, and Amelia Archer. 2019.  Small and practical BERT models for sequence labeling.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3632–3636, Hong Kong, China. Association for Computational Linguistics.
  * Turc et al. (2019) Iulia Turc, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  Well-read students learn better: The impact of student initialization on knowledge distillation.  _arXiv preprint arXiv:1908.08962_ , 13.
  * Wang et al. (2019) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019.  GLUE: A multi-task benchmark and analysis platform for natural language understanding.  In _International Conference on Learning Representations_.
  * Wang et al. (2020a) Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, and Furu Wei. 2020a.  Minilmv2: Multi-head self-attention relation distillation for compressing pretrained transformers.  _arXiv preprint arXiv:2012.15828_.
  * Wang et al. (2020b) Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. 2020b.  Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers.  _Advances in Neural Information Processing Systems_ , 33:5776–5788.
  * Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019.  Xlnet: Generalized autoregressive pretraining for language understanding.  In _Advances in Neural Information Processing Systems_ , volume 32. Curran Associates, Inc.
  * Yuan et al. (2020) Li Yuan, Francis EH Tay, Guilin Li, Tao Wang, and Jiashi Feng. 2020.  Revisiting knowledge distillation via label smoothing regularization.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 3903–3911.
  * Zhu et al. (2015) Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015.  Aligning books and movies: Towards story-like visual explanations by watching movies and reading books.  In _Proceedings of the IEEE international conference on computer vision_ , pages 19–27.


# Teaching Design of Experiments using Hasse diagrams

Hans-Michael Kaltenbach
Department of Biosystems Science and Engineering, ETH Zurich
<EMAIL_ADDRESS>
https://orcid.org/0000-0003-2500-1221

###### Abstract

Hasse diagrams provide a principled means for visualizing the structure of
statistical designs constructed by crossing and nesting of experimental
factors. They have long been applied for automated construction of linear
models and their associated linear subspaces for complex designs. Here, we
argue that they could also provide a central component for planning and
teaching introductory or service courses in experimental design.

Specifically, we show how Hasse diagrams allow constructing most elementary
designs and finding many of their properties, such as degrees of freedom,
error strata, experimental units and denominators for $F$-tests. Linear
(mixed) models for analysis directly correspond to the diagrams, which
facilitates both defining a model and specifying it in statistical software.
We demonstrate how instructors can seamlessly use Hasse diagrams to construct
designs by combining simple unit- and treatment structures, identify pseudo-
replication, and discuss a design’s randomization, unit-treatment versus
treatment-treatment interactions, or complete confounding. These features
commend Hasse diagrams as a powerful tool for unifying ideas and concepts.

Keywords: Experimental Design; Education; Hasse Diagram; Linear Model
Specification

##  1 INTRODUCTION

Most statistical designs encountered in introductory courses—such as
(fractional) factorial, randomized complete block, Latin square, and split-
unit designs—can be created by crossing and nesting of factors. Hasse diagrams
provide a visual representation of such designs (Lohr, 2006) and have long
been used in the statistical literature, in particular for discussing the
analysis of orthogonal designs (Tjur, 1984; Speed & Bailey, 1987; Andersson,
1990), for defining algorithms to automate the analysis of a given design
(Großmann, 2014; Bate & Chatfield, 2016a, b; Goos & Gilmour, 2012), and to
describe the linear subspaces of a design (Bailey, 2008). However, they are
used in only two textbooks (Oehlert, 2000; Bailey, 2008) as a supplementary
tool for teaching experimental design, and have found no application as a
central component to unify ideas and concepts for teaching.

Here, our goal is to reinforce the case made almost 25 years ago by Lohr
(1995) that Hasse diagrams are “useful in teaching others about design of
experiments: because they are visual rather than analytic, they give students
a supplementary perspective on different designs.” In particular, we show how
they provide instructors with a unifying tool to develop many of the classical
designs by combining treatment and unit factors in different ways via crossing
and nesting, and invite discussion of successful “design patterns” that can be
combined to create more complex designs. This approach emphasizes finding a
suitable design by looking at the relation between factors rather than relying
on a table of known designs and picking the most suitable one. It is thus
similar in spirit to the ”no-name approach” by Lorenzen & Anderson (1993) and
the focus on the data layout by Cobb (2008). In addition, the visual character
of the diagrams encourages exploring different design alternatives. We use
Hasse diagrams as a main teaching tool in a service course on introductory
experimental design to biology / biotechnology MSc students in our department.

With Bailey (2008), we explicitly distinguish an experiment’s _unit structure_
from its _treatment structure_ , use separate Hasse diagrams for the two
structures, and create the diagram of an experimental design by connecting
each treatment factor to the unit factor on which it is randomized. This idea
has attracted surprisingly little attention in expositions of experimental
design geared towards non-statisticians, an early exception being Cox (1958).
It directly clarifies the concept of an experimental unit (Hurlbert, 2013),
explicitly distinguishes between a unit-by-treatment from a treatment-by-
treatment interaction (Cox, 1984; de Gonzalez & Cox, 2007), and shows that the
same unit and treatment structures can be combined in different ways, leading
to very different experimental designs. It also allows easier recognition of
pseudo-replication (Hurlbert, 1984) and (inadvertent) split-unit designs (Goos
& Gilmour, 2012).

The emphasis on the factor structure of a design is complemented by the fact
that a corresponding linear (mixed) model and its specification for a
statistical software like R (R Core Team, 2019) can be derived directly from a
design’s diagram. This enables non-specialists to analyze data from a complex
designed experiment once the Hasse diagrams have been constructed. The
diagrams also allow extracting the ANOVA skeleton table, degrees of freedom
for each experimental factor, and correct denominator factor for $F$-tests,
which can all be used as sanity checks of the software output.

In the following, we always assume a balanced design with equal number of
replicates. While Hasse diagrams can also describe unbalanced designs, most of
the simple algorithms for calculating degrees of freedom or for deriving a
linear model do not work in this case. We do not claim originality for any
specific part of the following exposition; rather, we tried to collect and
organize useful ideas for applying Hasse diagrams as a unifying component for
teaching classical experimental designs, their properties, and the
corresponding linear (mixed) models, and hope to encourage instructors to
integrate them into their own courses. We provide generic diagrams for most
designs that can easily be adapted to an instructor’s preferred specific
examples; to also provide a concrete example, we use Yates’ oat varieties
data-set for illustrating a blocked split-unit design (Yates, 1935).

##  2 A FIRST EXAMPLE

###  2.1 Constructing Hasse Diagrams

To define an experiment design, we construct three Hasse diagrams for (i) the
treatment structure describing the treatment factors and their relation; (ii)
the unit structure describing the organization of the experimental material by
unit factors and their relation; and (iii) the experiment structure resulting
from randomizing each treatment factor on a unit factor. Each Hasse diagram is
a visual representation of the partial order induced by nesting of factors and
we represent each experimental factor by a vertex and use edges for
representing the nesting relation.

A Hasse diagram always contains a top node M, representing a one-level factor
for a general mean, and every other factor is nested in M. If a factor Y is
nested in a factor X, then each level of Y occurs together with only one level
of X. We then draw Y below X and connect the two vertexes with an edge. The
diagrams are thus “read” from top to bottom and a factor reached by following
an edge corresponds to a more fine-grained partition of the experimental
material into groups receiving the same treatment or belonging to the same
block, for example. If two factors X and Y are crossed, then each level of X
occurs with each factor of Y and we write X and Y next to each other without a
connecting edge. They provide independent partitions of the experimental
material and give rise to an interaction factor X:Y nested in both X and Y.
Notationally, we denote treatment factors in bold and unit factors in italics,
write random factors in parentheses, and assume a single response factor,
which we mark by underlining. We often augment a diagram by denoting the
number of factor levels by superscripts.

###  2.2 A Simple Factorial Design

To illustrate the constructions, we consider a factorial design with two
treatment factors A and B with $a$ and $b$ levels, respectively, in a balanced
completely randomized design with $n$ experimental units per treatment
combination and an experiment size of $N=n\cdot a\cdot b$. Extensions of this
example to factorials with more than two treatment factors and designs with
nested treatment factors are straightforward and we omit the details here.

The treatment structure diagram for our example is given in Figure 1(a). Our
two treatment factors A and B are crossed such that each level of A occurs
with each level of B, and both are nested in the general mean M. The resulting
interaction factor A:B is nested in both A and B.

The unit structure is shown in Figure 1(b) and consists of a general mean and
a single unit factor E with $N$ levels. This factor is therefore the only
possible experimental unit. It is good practice to use a descriptive name for
this factor in a concrete example, e.g. Mouse, Plot, or Sample.

To construct the experiment design in Figure 1(c), we first identify the two
general means from the treatment and unit structures. We then draw an edge
from each treatment factor to the unit factor on which it is randomized. This
determines the experimental unit for each treatment factor, and nests it
within its treatment factor(s). For our completely randomized design, we
randomize both treatments A and B on the only available unit factor E, which
also makes it the experimental unit factor for A:B.

The diagrams give the transitive closure of the partial order on factors given
by nesting. Consequently, we omit the direct edges from A to E and from B to
E, since they are implied by nesting E in A:B, which in turn is nested in both
A and B. We also remove the edge from M to E for the same reason. We might
emphasize the balance in the design by writing $n\cdot a\cdot b$ instead of
$N$ in the diagram, indicating that each treatment combination is replicated
$n$ times.

$\mathbf{M}^{1}$$\mathbf{A}^{a}$$\mathbf{B}^{b}$$\mathbf{A:B}^{ab}$(a)$\mathit{M}^{1}$$(\underline{\mathit{E}})^{N}$(b)$\mathbf{M}^{1}_{1}$$\mathbf{A}^{a}_{a-1}$$\mathbf{B}^{b}_{b-1}$$\mathbf{A:B}^{ab}_{(a-1)(b-1)}$$(\underline{\mathit{E}})^{N}_{N-ab}$(c)
Figure 1: (a): Treatment structure diagram. Hasse diagram of generic treatment
structure with two crossed factors and their interaction. (b): Unit structure
diagram. Simple unit structure with single unit factor. (c): Experiment
structure diagram. Randomizing each treatment factor on the same unit factor
yields a two-factor factorial in a completely randomized design.

###  2.3 Degrees of Freedom

For any orthogonal design, we can calculate the degrees of freedom of a factor
from the experiment diagram by subtracting from its number of levels the
degrees of freedom of any factor in which it is nested (Tjur, 1984; Lohr,
2006). Since the general mean M has one level and is not nested in another
factor, it has one degree of freedom. The interaction A:B has $ab$ levels,
from which we subtract one degree of freedom of M, and $a-1$ respectively
$b-1$ degrees of freedom of A and B. This yields the familiar $(a-1)(b-1)$
degrees of freedom. For the unit factor E, we find $(n-1)ab$ degrees of
freedom for a balanced design with $n$-fold replication and $N=nab$.

We may find that a factor Y nested in X has zero degrees of freedom. It is
then completely confounded with X and parameters and effects cannot be
estimated individually. In our example, we might use a single experimental
unit for each factor level combination (i.e., set $n=1$). The residual factor
E then has zero degrees of freedom and is completely confounded with the
interaction A:B. In other words, no replication exists for the treatment
combinations and we can then only compute a pooled estimate of the combined
variation of the interaction and the residual effects. This corresponds to
merging A:B and E into a single factor in the diagram.

Another example is a one-way ANOVA design as shown in Figure 2(d) with only a
single treatment factor level. The treatment is then confounded with the
general mean, and we essentially have a simple sampling design where all units
are assigned the same treatment and their responses are recorded.

###  2.4 ANOVA Table, Error Terms, and $F$-Tests

Using the Hasse diagram for the experiment design, we readily find the
skeleton ANOVA table for an analysis. It contains one line for each factor in
the diagram, where the general mean is excluded as usual and the lowest unit
factor provides the residuals. For each line, the degrees of freedom are
determined using the algorithm in Sect. 2.3, and the sum of squares and
expected mean squares can be determined using similar algorithms, see for
example Lohr (2006). Each random unit factor gives rise to an error stratum in
the table, and treatment factors randomized on this unit factor are tested
within that stratum.

The diagram facilitates identification of the error mean squares for the
denominator of an $F$-test: it is provided by the closest random factor below
the factor tested that does not involve a new fixed factor. If this factor is
not unique, then no exact $F$-test exists. In our example, E is the random
factor nearest to each treatment factor and the $F$-test for testing the A
main effect, for example, is based on the ratio
$\text{MS}(A)\,/\,\text{MS}(E)$ with $a-1$ numerator and $(n-1)ab$ denominator
degrees of freedom.

###  2.5 Linear Model

Each factor provides a set of parameters for the linear model, with one
parameter per factor level. For our example, the general mean is associated
with a single parameter $\mu$, the treatment factors A, B, and A:B have $a$
parameters $\alpha_{i}$, $b$ parameters $\beta_{j}$, and $ab$ parameters
$(\alpha\beta)_{ij}$, respectively. The (random) residual unit factor E has
$nab$ levels corresponding to i.i.d. random variables $e_{ijk}$ with $k=1\dots
n$; the corresponding population parameter is the variance component
$\sigma^{2}=\text{Var}(e_{ijk})$. The full linear model for the example is
thus

$y_{ijk}\;=\;\mu+\alpha_{i}+\beta_{j}+(\alpha\beta)_{ij}+e_{ijk}\;,$ (1)

where $i=1\dots a$, $j=1\dots b$, and $k=1\dots n$.

This model is over-parametrized, and each factor’s subscript denotes how many
of its parameters can be independently estimated. For example, A is nested in
M and its parameters $\alpha_{i}$ are thus deviations from the general mean
$\mu$ with expected response $\mu+\alpha_{i}$ for level $i$ of A. If $\mu$ is
given, we can only estimate $a-1$ out of the $a$ parameters of A. This problem
is resolved by imposing one additional relation either between the parameters
or between their estimators, and Nelder (1994) provides a pointed discussion
of the difference. With $ab$ fixed effect parameters, the variance
$\sigma^{2}$ is estimated based on $(n-1)ab$ degrees of freedom. For $n=1$,
merging A:B with E yields the additive model
$y_{ij}=\mu+\alpha_{i}+\beta_{j}+e^{*}_{ij}$ where
$e^{*}_{ij}=(\alpha\beta)_{ij}+e_{ij}$ with assumed $(\alpha\beta)_{ij}=0$ and
$k\equiv 1$ omitted.

###  2.6 Symbolic Model Specification

We can specify a linear model symbolically using the formulae proposed by
Wilkinson & Rogers (1973) and extended by Tjur (1984). This model description
uses the general operator + to add factors and : to denote interaction. Common
patterns are abbreviated, such as A*B=A+B+A:B for two fully crossed factors,
or A/B=A+A:B for B is nested in A. The error strata are defined by the random
unit factors, and the corresponding specification is written by Tjur (1984)
using brackets: A+[Blk] describes a single treatment factor A crossed with a
random blocking factor Blk.

Formulating the model is then often a matter of writing out the diagram. For
the example model (1), the symbolic description is 1+A+B+A:B (abbreviated
A*B). It is equivalent to A/(A:B)+B/(A:B) and to (A+B)/(A:B), which also
describe the diagram. The additive model without interaction is A+B.

We use the almost identical model specification for R for our examples. The
general mean is then denoted 1, but often omitted and then added implicitly.
Also omitted is the lowest unit factor, corresponding to the lowest set of
residuals. Error strata given by [$\cdots$] are implemented by the
Error($\cdots$) part of a model specification.

###  2.7 Marginality Principle

With only rare exceptions, the removal of a factor from a model should adhere
to the _marginality principle_ and is admissible only when all higher-order
interactions containing this factor are also removed (Nelder, 1994). In our
example of a two-factor treatment design (Fig. 2(a)), removal of the
interaction term A:B observes the marginality principle (Fig. 2(b)).

In contrast, removal of the main treatment factor B while retaining the
interaction A:B violates this principle. The factors A and B are no longer
crossed as intended by the treatment structure, but B now appears nested in A
in the diagram (Fig. 2(c)).

Removal of the interaction factor together with one main effect factor (e.g.,
B) yields a model with a single treatment factor, and this model reduction
adheres again to the marginality principle (Fig. 2(d)). The resulting design
resembles a one-way ANOVA and simplifies to a $t$-test situation with the
familiar $2n-2$ error degrees of freedom if $a=2$.

$\mathbf{M}^{1}_{1}$$\mathbf{A}^{a}_{a-1}$$\mathbf{B}^{b}_{b-1}$$\mathbf{A\\!:\\!B}^{ab}_{(a-1)(b-1)}$$(\underline{\mathit{E}})^{nab}_{(n-1)ab}$(a)$\mathbf{M}^{1}_{1}$$\mathbf{A}^{a}_{a-1}$$\mathbf{B}^{b}_{b-1}$$(\underline{\mathit{E}})^{nab}_{nab-
a-b+1}$(b)$\mathbf{M}^{1}_{1}$$\mathbf{A}^{a}_{a-1}$$\mathbf{A\\!:\\!B}^{ab}_{a(b-1)}$$(\underline{\mathit{E}})^{nab}_{(n-1)ab}$(c)$\mathbf{M}^{1}_{1}$$\mathbf{A}^{a}_{a-1}$$(\underline{\mathit{E}})^{na}_{(n-1)a}$(d)
Figure 2: Marginality principle. (a): the full model consists of two crossed
factors and their interaction. (b): removing the interaction is compatible
with the marginality principle. (c): removing the B main effect and keeping
the interaction violates the principle and leads to a design with B nested in
A. (d): removing both the B main effect and the interaction restores
marginality.

##  3 UNIT STRUCTURES: BLOCKING AND REPLICATION

While treatment structures are predominantly based on crossed factors, both
crossing and nesting lead to commonly used unit structures. Multiple unit
factors provide different possibilities for randomization, and identical unit-
and treatment structures then lead to different experimental designs. In
addition, unit-by-treatment interactions emerge and need to be addressed in
the analysis or by treating them as negligible and removing them from the
diagrams.

###  3.1 Nested Unit Factors

We first consider a simple treatment structure with a single factor A with $a$
levels and combine it with a unit structure with two nested factors B ($b$
levels) and E ($nb$ levels), with responses recorded for the lower factor E
(Fig. 3(a,b)). This situation might correspond to an agricultural experiment
with subplots nested in plots or to an animal experiment with mice nested in
litters, for example. We now have two options for constructing the experiment
design by randomizing the treatment factor on either the upper or lower unit
factor.

$\mathbf{M}^{1}$$\mathbf{A}^{a}$(a)$\mathit{M}^{1}$$\mathit{(B)}^{b}$$\mathit{(\underline{E})^{nb}}$(b)$\mathbf{M}^{1}_{1}$$\mathbf{A}^{a}_{a-1}$$\mathit{(B)}^{b=ka}_{(k-1)a}$$\mathit{(\underline{E})}^{nka}_{(n-1)ka}$(c)$\mathbf{M}^{1}_{1}$$\mathit{(B)}^{b}_{b-1}$$\mathbf{A}^{a}_{a-1}$$(\mathit{B}\\!:\\!\mathbf{A})^{ab}_{(a-1)(b-1)}$$\mathit{(\underline{E})}^{ab}_{0}$(d)$\mathbf{M}^{1}_{1}$$\mathit{(B)}^{b}_{b-1}$$\mathbf{A}^{a}_{a-1}$$(\mathit{B}\\!:\\!\mathbf{A})^{ab}_{(a-1)(b-1)}$$\mathit{(E)}^{nab}_{(n-1)ab}$$\mathit{(\underline{S})}^{snab}_{(s-1)nab}$(e)$\mathbf{M}^{1}_{1}$$\mathbf{A}^{a}_{a-1}$$\mathit{(Rep)}^{r}_{r-1}$$\mathbf{B}^{b}_{b-1}$$\mathbf{A\\!:\\!B}^{ab}_{(a-1)(b-1)}$$(\underline{\mathit{E}})^{rab}_{(n-1)rab}$(f)$\mathbf{M}^{1}_{1}$$\mathit{(B1)}^{k}$$\mathit{(B2)}^{mk}$$\mathbf{A}^{a}$$(\underline{\mathit{E}})^{amk}$(g)
Figure 3: (a): one-factor treatment structure. (b): unit structure with two
nested factors. (c): A randomized on B with responses measured on E gives a
design with sub-sampling. (d): randomized complete block design (RCBD) with A
randomized on E and crossed with B. (e): generalized RCBD with treatment
levels replicated per block and multiple samples per experimental unit. (f):
replicating a treatment design by crossing with a unit factor. (g): design
with two nested blocks resembles an RCBD with additional block structure.

####  3.1.1 Sub-sampling and Decomposing Variation

If we randomize the treatment factor A on the upper unit factor B as in Figure
3(c), then B is the experimental unit for A with $b=ka$ levels and provides
the denominator for the $F$-test with $(a-1)$ and $(k-1)a$ degrees of freedom.
Since E is still the response unit, individual measurements are thus pseudo-
replications (Hurlbert, 1984).

The ANOVA table for this design has two error strata. If $k>1$ and $n>1$, we
have replication on both unit factors and can estimate their two variance
components $\sigma^{2}_{B}$ and $\sigma^{2}_{E}$. The relevant variance for
testing treatment effects is proportional to $\sigma_{B}^{2}+\sigma_{E}^{2}/n$
and the design is helpful only if the $\sigma_{E}^{2}$ is not small compared
to $\sigma_{B}^{2}$, and thus precision can be gained by averaging several
measurements for each experimental unit.

In R notation for ANOVA, the right-hand side of the model specification is
A+Error(B) and we note that the term outside Error() corresponds to the
treatment structure (with the general mean omitted as usual) while the term
inside Error() corresponds to the unit structure (with the lowest factor
omitted as usual). This observation simplifies the model specification in the
frequent case that all fixed factors are in the treatment structure and all
random factors are in the unit structure and there is no interaction between
them. For a linear mixed model using the lme4 package, the right-hand side is
A+(1|B) and using the Kenward-Roger approximation (Kenward & Roger, 1997)
ensures agreement between model and diagram degrees of freedom.

The chain of unit factors can be elongated, for example, to separate variation
of the experimental material, samples, individual sample preparations, and
individual measurements. The contributions of each factor can only be
estimated if the factor is replicated and are otherwise completely confounded
with the next-higher factor in the chain. If no replication is present in the
chain, then the overall variation is conceptually divided into individual
contributions, but the analysis has to be based on a single estimate of the
overall variation. Thus, it becomes clear that the lowest unit factor in any
design contains all variation not explicitly captured by other factors.

####  3.1.2 (Generalized) Randomized Complete Block Designs

If we randomize the treatment factor on the lower unit factor E as in Figure
3(d), we automatically cross A with B and achieve a block design. Crossing a
unit and a treatment factor imposes restrictions on the randomization of
treatment factor levels on experimental units, which we need to account for in
the analysis.

The two crossed factors give rise to a unit-by-treatment interaction factor
A:B. It only occurs in the experiment design diagram and is not present in
either treatment or unit structure; it is a random factor because one of its
constituent factors is random. The interaction is mathematically very similar
to a treatment-by-treatment interaction, but its interpretation can be very
different (Cox, 1984; de Gonzalez & Cox, 2007).

Following the diagram, the treatment factor is tested against the unit-by-
treatment interaction. If each treatment level occurs once per block, we
derive the standard randomized complete block design (RCBD) shown in Figure
3(d) with zero residual degrees of freedom. The treatment factor can only be
tested if we assume that the unit-by-treatment interaction is negligible and
we pool the interaction and the residual factor E to provide the error mean
squares. Because we often have substantial freedom in choosing a blocking
factor, subject-matter knowledge should ensure the no-interaction assumption
is reasonable.

We can extend the design to a generalized randomized complete block design
(GRCBD) by allowing multiple experimental units per block to receive the same
treatment and/or by sampling each experimental unit multiple times (Addelman,
1969). The full diagram is shown in Figure 3(e), where E represents the
experimental units and S represents multiple samples per unit. We re-derive
the RCBD with $n=s=1$. For $s=1,n>1$, we get a design where multiple units per
block receive the same treatment, and we have one response per unit. For
$s>1,n=1$, we get an RCBD, where each unit is measured multiple times. Again,
a negligible unit-treatment interaction allows us to remove the factor A:B
from the diagram.

The (G)RCBD with non-negligible unit-by-treatment interaction cannot be
analyzed using the standard R commands for ANOVA because it involves the
interaction of a fixed and a random factor. An analysis of a GRCBD without
sub-sampling, for example, then requires a linear mixed model with
specification A+(1|B)+(1|A:B). Both RCBD and GRCBD also invite discussion of
finding an $F$-test for the random block effect of B and its possible
interpretation (Samuels et al., 1991). Since the interaction factor involves A
as a new fixed factor, the correct error term for testing B is provided by E.

The idea of crossing a unit factor with a treatment factor can be extended to
crossing it with the entire treatment structure. This yields a simple design
pattern for replicating treatment designs. An example is shown in Figure 3(f),
where a two-factor factorial design is crossed with a unit factor Rep to
account for potential differences in replicates. For example, the logistics of
the experiment might require performing replicates on multiple days, and
calibration drift of measurement devices might lead to day-to-day differences.
Assuming negligible interaction between replicate and treatments, we can
remove the Rep:A:B interaction from this design to yield an (G)RCBD. If Rep
additionally removes substantial variation from the experimental units E, this
strategy also yields efficient blocking.

####  3.1.3 Blocks as Fixed Factors

A different analysis emerges if we consider an (G)RCBD with a fixed rather
than random block factor B. Then, the analysis is similar to a two-factor
completely randomized design with a single error stratum, with A, B, and A:B
now tested against E, and the model is specified simply by A*B including the
interaction, and by A+B for an additive model. The ANOVA table contains an
$F$-test for the blocking factor B, which invites discussion about its
interpretation and comparison to a random blocking factor. An example would be
testing different drug treatments on men and women, with sex considered as a
fixed unit factor with two levels, and E providing replication within each
sex. In the notion of Cox (1984), B is then an _intrinsic_ rather than a _non-
specific_ unit factor, and the unit-by-treatment interaction might be of
direct relevance to the interpretation of the experimental results and usually
cannot be ignored.

####  3.1.4 Nested Blocks

The idea of the RCBD can be expanded in another direction, by considering
several nested blocking factors, all crossed with the treatment factor. Such a
design is shown in Figure 3(g) for two nested blocking factors B1 and B2, with
$k$ and $mk$ levels, respectively. Again, we assume all unit-by-treatment
interactions to be negligible.

For treatment comparisons, this design effectively looks like a RCBD with a
single blocking factor with $mk$ levels, but both factors contribute to the
reduction in residual variance, and both enforce restrictions on the
assignment of treatment factor levels to experimental units. The two blocking
contributions can be separately estimated if $m>1$ and are confounded for
$m=1$. For example, in an animal experiment, the levels of B1 might be cages,
while levels of B2 are litters and the unit structure describes using $m$
litters per cage. The experiment might require keeping only one litter per
cage, which confounds the litter-to-litter and cage-to-cage variations. The
model is specified as A+Error(B1/B2) for a classical ANOVA, and as A+(1|B1/B2)
for a mixed model analysis.

###  3.2 Crossed Unit Factors

Unit factors can also be crossed to allow simultaneous control for multiple
sources of variation. We consider two crossed unit factors called R (for row)
and C (for column) to emphasize the rectangular row-column form of the
blocking. Each intersection of a row and column (a cell) corresponds to a
level of the interaction R:C, and several units might be nested in each cell.
We consider a single treatment factor A with $a$ levels, but more complex
treatment structures are possible.

####  3.2.1 All Treatments per Cell

A first design uses $r$ rows and $c$ columns and crosses their interaction
with a simple treatment structure as shown in Figure 5(a), with $n$ replicates
of each treatment level per cell. For simplicity, we again assume that unit-
by-treatment interactions are negligible; the model is then given by
A+Error(R*C) and A+(1|R)+(1|C)+(1|R:C), respectively. The corresponding data
layout is shown in Figure 4.

Figure 4: Randomized data layout for design in Figure 5(a) with $r=c=t=3$ and
$n=1$. Treatments are crossed with interaction of row and column block factor,
yielding independent randomization of treatment levels to units within row-
column intersections.

The rectangular blocking appears as a simple blocking with $narc$ levels, and
the design is thus similar to a (G)RCBD, but imposes additional structure in
the blocking and restrictions on the treatment randomization.

A different design emerges if we randomize the treatments directly on the
interaction factor, such that each cell gets assigned a single treatment
level. A balanced assignment requires that $r$ and $c$ are multiples of $a$,
and pseudo-replication occurs for $n>1$. An important example is the Latin
square shown in Figure 5(b) which uses $r=c=a$ and $n=1$, and thus has $a^{2}$
experimental units, such that each treatment level occurs exactly once per row
and column. The factor E in Figure 5(b) then provides the error term for
testing the treatment factor and all two-way interactions have to be assumed
negligible in this design.

An interesting situation occurs if we cannot assume that two-way interactions
are negligible, as shown in Figure 5(c). Following edges from A downwards, we
find that there is no unique factor for providing the denominator mean squares
for an $F$-test of the treatment. Indeed, as is well known, no exact $F$-test
exists for this situation, and we need to resort to finding an approximation
of the denominator mean squares and degrees of freedom, such as the weighted
averages given by Satterthwaite (1946). Note again that the two-way
interactions are qualitatively different: while R:C is a combination of
blocking levels (corresponding to a cell), both R:A and C:A are unit-treatment
interactions.

$\mathbf{M}^{1}$$\mathit{(C)}^{c}$$\mathit{(R)}^{r}$$\mathbf{A}^{a}$$(\mathit{R\\!:\\!C})^{rc^{2}}$$(\underline{\mathit{E}})^{narc^{2}}$(a)$\mathbf{M}^{1}_{1}$$\mathit{(C)}^{a}_{a-1}$$\mathit{(R)}^{a}_{a-1}$$\mathbf{A}^{a}_{a-1}$$(\underline{\mathit{E}})^{a^{2}}_{(a-1)(a-2)}$(b)$\mathbf{M}^{1}$$\mathit{(C)}^{ca}$$\mathit{(R)}^{ra}$$\mathbf{A}^{a}$$\mathit{(R\\!:\\!C)}^{rca^{2}}$$\mathit{(R\\!:}\mathbf{A}\mathit{)}^{ra^{2}}$$\mathit{(C\\!:}\mathbf{A}\mathit{)}^{ca^{2}}$$\mathit{(R\\!:\\!C\\!:}\mathbf{A}\mathit{)}^{rca^{2}}$(c)
Figure 5: (a): two crossed unit factors provide a block with internal
structure for the treatment factor. (b): the classical Latin square design.
(c): with non-negligible block-by-treatment interactions, no exact $F$-test
exists for the treatment factor.

####  3.2.2 Replicating Latin Square Designs

While the Latin square is the most common design based on two crossed unit
factors, it might suffer from insufficient error degrees of freedom: for
$a=2$, $a=3$, and $a=4$ treatment levels, these are $0$, $2$, and $6$,
respectively. It is therefore often necessary to replicate a Latin square
design and we illustrate the use of Hasse diagrams for discussing different
strategies for this replication. In each case, the Hasse diagram allows us to
quickly calculate the resulting error degrees of freedom and to compare them
between the designs. Moreover, the specification of an appropriate linear
model is again quickly derived from the diagrams.

First, we might consider replicating rows while keeping the columns, using
$ra$ row factor levels instead of $a$, for example. If we do not impose any
restrictions on the assignment of treatments to rows other than overall
balancedness, consecutive sets of $a$ rows do not need to form Latin squares,
and we arrive at a design known as _Latin rectangle_. It corresponds to Figure
5(a) with $c=1$ and negligible interaction R:C; the data layout for this
design is shown in Figure 7(a). The linear model is thus given in R notation
by A+Error(R+C) for an ANOVA, and A+(1|R)+(1|C) for a mixed model.

Another option is using $r$ replicates of $a$ rows again, but to additionally
require that each replicate forms an independent Latin square. We achieve this
by introducing a new unit factor Rep to account for the replication and arrive
at the design shown in Figure 6(a) with R nested in Rep; the data layout is
shown in Figure 7(b). Since the treatment factor A is crossed with the new
unit factor Rep, each treatment level must occur once in each row per
replicate, and this design corresponds to $r$ independently randomized Latin
squares, where the column factor levels are identical in each replicate, but
the row factor levels are different. The model specifications are
A+Error(Rep/R+C) (ANOVA) and A+(1|Rep/R)+(1|C) (LMM), respectively.

By nesting both R and C in Rep, we find yet another useful design that uses
different row and column factor levels in each replicate of the Latin square
(cf. diagram in Fig. 6(b) and data layout in Fig. 7(c)) with models
A+Error(Rep/(R+C)) (ANOVA) and A+(1|Rep)+(1|Rep:R)+(1|Rep:C) (LMM). Finally,
by crossing Rep with the row and column factor we keep the same rows and
columns in each replicate and re-randomize the treatment levels independently
on each square. The diagram is shown in Figure 6(c) and the data layout in
Figure 7(d) with models A+Error(Rep+R+C) (ANOVA) and A+(1|Rep)+(1|R)+(1|C)
(LMM).

$\mathbf{M}^{1}_{1}$$\mathit{(C)}^{a}_{a-1}$$\mathit{(Rep)}^{r}_{r-1}$$\mathit{(R)}^{ra}_{r(a-1)}$$\mathbf{A}^{a}_{a-1}$$(\underline{\mathit{E}})^{ra^{2}}_{(ra-2)(a-1)}$(a)$\mathbf{M}^{1}_{1}$$\mathit{(Rep)}^{r}_{r-1}$$\mathit{(R)}^{ra}_{r(a-1)}$$\mathit{(C)}^{ra}_{r(a-1)}$$\mathbf{A}^{a}_{a-1}$$(\underline{\mathit{E}})^{ra^{2}}_{(r(a-1)-1)(a-1)}$(b)$\mathbf{M}^{1}_{1}$$\mathit{(C)}^{a}_{a-1}$$\mathit{(R)}^{a}_{a-1}$$\mathit{(Rep)}^{r}_{r-1}$$\mathbf{A}^{a}_{a-1}$$(\underline{\mathit{E}})^{ra^{2}}_{(a-1)(r(a+1)-3)}$(c)
Figure 6: Replication of Latin square design. (a): $r$ sets of $a$ rows, and
identical columns per replicate. (b): new rows and columns in each replicate.
(c): identical rows and columns in each replicate. Figure 7: Data layout for
replicating Latin square designs. (a): Latin rectangle with two replicates of
three rows; replicates do not form Latin squares. (b): two independent
replicates of rows and identical columns, each replicate is a Latin square
(cf. diagram Fig. 6(a)). (c): two independent replicates of rows and columns,
each replicate is a Latin square (cf. diagram Fig. 6(b)). (d): two replicates
of Latin square with identical rows and columns, but independent randomization
of treatments (cf. diagram Fig. 6(c)).

##  4 SPLIT-UNIT DESIGNS

As a last illustration, we consider a classical agricultural example provided
in Yates (1935), with three oat varieties tested at four nitrogen levels each.
Oat varieties are randomized on larger plots of land, while the four nitrogen
levels are randomized on subplots within each plot, and the experiment is
replicated in six blocks. The diagrams for this example are given in Figure 8.
The treatment structure is a simple factorial design with two factors, and the
unit structure consists of a simple chain of three nested factors. However,
the treatment factors are now randomized on different unit factors, resulting
in a split-unit design for the experiment structure. Note how some of the
previous designs occur as part of the experiment diagram, including blocking a
full treatment structure, sub-sampling, and crossing a treatment with nested
blocks.

The diagram makes explicit that this design has two experimental unit factors
with Plot providing the experimental unit for Variety, while Subplot provides
the experimental unit for both Nitrogen and the interaction Variety:Nitrogen.
As a consequence, contrasts on Variety main effects profit from variance
reduction via Block, but still encounter the combined variance of Plot and
Subplot and the low replication on Plot; the nested Subplot resembles pseudo-
replication. On the other hand, contrasts on Nitrogen main effects and the
interaction profit from reduced variance due to the blocking Plot and Block
(resembling the design of Fig. 5(a)) and from higher replication on Subplot.

In order to improve estimation and testing on the whole-unit factor Variety,
we need to improve on the blocking by Block or increase the replication on
Plot. Both measures also positively impact estimation and testing involving
the two other treatment factors. The linear model is specified by
Variety*Nitrogen+Error(Block/Plot) respectively by
Variety*Nitrogen+(1|Block/Plot). The model specification again decomposes
nicely into two very simple parts describing the treatment and the unit
structures, respectively. A very common alternative specification is
Variety*Nitrogen+Error(Variety/Nitrogen) and exploits that the unit structure
is implicit in combination of levels of the treatment factors. It is however
an illogical model formula and a source of much confusion for students.

Using Hasse diagrams, it is straightforward to expand the idea of a split-unit
design to more “splits”, to designs such as split-blocks (or criss-cross), and
to split-unit designs with more complex whole- or sub-unit designs, such as
using a Latin square for the whole-unit factor.

$\mathbf{U}^{1}$$\mathbf{Variety}^{3}$$\mathbf{Nitrogen}^{4}$$\mathbf{Variety\\!:\\!Nitrogen}^{12}$(a)$\mathit{U}^{1}$$\mathit{(Block)^{6}}$$\mathit{(Plot)^{18}}$$\mathit{(\underline{Subplot})^{72}}$(b)$\mathbf{U}^{1}_{1}$$\mathbf{Variety}^{3}_{2}$$\mathbf{Nitrogen}^{4}_{3}$$\mathbf{Variety\\!:\\!Nitrogen}^{12}_{6}$$\mathit{(Block)^{6}_{5}}$$\mathit{(Plot)}^{18}_{10}$$\mathit{(\underline{Subplot})}^{72}_{45}$(c)
Figure 8: Hasse diagrams of a split-unit design with oat varieties randomized
on plots, and nitrogen amounts to subplots within plots, the whole design
replicated in blocks. (a): treatment structure. (b): unit structure. (c):
experiment structure.

##  5 CONCLUSIONS

The teaching of introductory experimental design to undergraduates and non-
statisticians remains an important task for the statistical community, and it
has been argued that such course provides additional benefits for students
with comparatively little mathematical background (Blades et al., 2015). In
our own teaching of experimental design as second statistics course for
biotechnology master students, we found that Hasse diagrams enable a more
direct approach to establishing the classical “good” designs and allow
recognition of design patterns shared between designs. They encourage free
composition of designs from unit and treatment structures; the model
specification for a resulting design can be derived directly from the
experiment diagram and used with statistical software. The skeleton ANOVA
table with factors, error strata, degrees of freedom, sum of squares, and
expected mean squares can be derived directly from the experiment diagram
(Lohr, 2006). This assists in teaching properties of a design and additionally
allows students to check software output against a diagram to ensure
compatibility of design and analysis.

While Hasse diagrams are suitable for many designs created by nesting and
crossing factors, their main power lies with balanced and orthogonal designs.
They are, however, not applicable for designs with more complex confounding,
such as balanced incomplete block designs or Youden squares, but these designs
can be developed once the RCBD and Latin squares are mastered. Hasse diagrams
are also not well suited for explaining fractional factorial designs, although
this could be done by introducing pseudo-factors as in Bailey (2008). Again,
these can be developed once full factorial designs are understood.

We did not discuss more complex treatment structures such as higher-order
factorial treatment designs, but their Hasse diagrams are easily derived. With
more than two treatment factors, interactions of the same order are now
crossed and give rise to new factors (namely the next-higher order
interactions) in the diagram. This results in exponentially many factors in
the treatment structure and methods for reducing this number include removal
of higher-order interactions and fractions of the design. For the important
special case of a $2^{k}$ factorial, the Hasse diagram immediately shows that
each treatment factor has precisely one degree of freedom, and thus fractional
factorials can be constructed by exploiting either the well-known additive
$0/1$ or the multiplicative $-1/+1$ algebra to encode factor levels.

We also did not discuss random treatment factors. These are straightforward to
capture in the diagrams, but necessarily require analysis by linear mixed
models instead of a classical ANOVA.

Some more recent developments extend the ideas of Hasse diagrams to larger
classes of design problems. Although typically not relevant for introductory
courses, they might stimulate similar uses of diagrams for more advanced
courses and are certainly interesting for many consultation projects. Bate and
co-workers (Bate & Chatfield, 2016a, b) propose diagrams that allow more
complex response structures and are tailored towards linear mixed models with
covariance structures beyond simple variance components. Independently, the
work by Brien and co-workers extends the description from a single experiment
to a chain of “tiered” experiments (Brien, 1983; Brien & Payne, 1999; Brien &
Demétrio, 2009) which use multiple randomization. In reality, many experiments
are of course multi-tiered, such as treatments randomly allocated to
experimental units, and multiple measurement devices are then randomly
assigned to record the response values. It is then easy to “break” the overall
randomization when assigning all units with the same treatment level to the
same measurement device, for example.

## References

  * (1)
  * Addelman (1969) Addelman, S. (1969), ‘The generalized randomized block design’, The American Statistician 23(4), 35–36.
  * Andersson (1990) Andersson, S. A. (1990), ‘The Lattice Structure of Orthogonal Linear Models and Orthogonal Variance Component Models’, Scandinavian Journal of Statistics 17, 287–319.
  * Bailey (2008) Bailey, R. A. (2008), Design of Comparative Experiments, Cambridge University Press.
  * Bate & Chatfield (2016a) Bate, S. T. & Chatfield, M. J. (2016a), ‘Identifying the Structure of the Experimental Design’, Journal of Quality Technology 48(4), 343–364.
  * Bate & Chatfield (2016b) Bate, S. T. & Chatfield, M. J. (2016b), ‘Using the Structure of the Experimental Design and the Randomization to Construct a Mixed Model’, Journal of Quality Technology 48(4), 365–387.
  * Blades et al. (2015) Blades, N. J., Schaalje, G. B. & Christensen, W. F. (2015), ‘The Second Course in Statistics: Design and Analysis of Experiments?’, American Statistician 69(4), 326–333.
  * Brien (1983) Brien, A. C. J. (1983), ‘Analysis of Variance Tables Based on Experimental Structure’, Biometrics 39(1), 53–59.
  * Brien & Demétrio (2009) Brien, C. J. & Demétrio, C. G. B. (2009), ‘Formulating mixed models for experiments, including longitudinal experiments’, Journal of Agricultural, Biological, and Environmental Statistics 14(3), 253–280.
  * Brien & Payne (1999) Brien, C. J. & Payne, R. W. (1999), ‘Tiers, structure formulae and the analysis of complicated experiments’, Journal of the Royal Statistical Society: … 48(1), 41–52.
  * Cobb (2008) Cobb, G. (2008), Introduction to Design and Analysis of Experiments, Wiley, New York.
  * Cox (1958) Cox, D. R. (1958), Planning of Experiments, Wiley-Blackwell.
  * Cox (1984) Cox, D. R. (1984), ‘Interaction’, International Statistical Review 52, 1–31.
  * de Gonzalez & Cox (2007) de Gonzalez, A. & Cox, D. R. (2007), ‘Interpretation of interaction: a review’, The Annals of Applied Statistics 1(2), 371–385.
  * Goos & Gilmour (2012) Goos, P. & Gilmour, S. G. (2012), ‘A general strategy for analyzing data from split-plot and multistratum experimental designs’, Technometrics 54(4), 340–354.
  * Großmann (2014) Großmann, H. (2014), ‘Automating the analysis of variance of orthogonal designs’, Computational Statistics and Data Analysis 70, 1–18.
  * Hurlbert (1984) Hurlbert, S. (1984), ‘Pseudoreplication and the Design of Ecological Field Experiments’, Ecological Monographs 54(2), 187–211.
  * Hurlbert (2013) Hurlbert, S. H. (2013), ‘Affirmation of the classical terminology for experimental design via a critique of Casella’s Statistical Design’, Agronomy Journal 105(2), 412–418.
  * Kenward & Roger (1997) Kenward, M. G. & Roger, J. H. (1997), ‘Small Sample Inference for Fixed Effects from Restricted Maximum Likelihood’, Biometrics 53(3), 983–997.
  * Lohr (1995) Lohr, S. L. (1995), ‘Hasse diagrams in statistical consulting and teaching’, The American Statistician 49(4), 376–381.
  * Lohr (2006) Lohr, S. L. (2006), Hasse Diagrams, in ‘Encyclopedia of Statistical Sciences’, John Wiley & Sons, Inc.
  * Lorenzen & Anderson (1993) Lorenzen, T. J. & Anderson, V. L. (1993), Design of Experiments: A No-Name Approach, Marcel Dekker, New York.
  * Nelder (1994) Nelder, J. A. (1994), ‘The statistics of linear models: back to basics’, Statistics and Computing 4(4), 221–234.
  * Oehlert (2000) Oehlert, G. W. (2000), A First Course in Design and Analysis of Experiments, W. H. Freeman.
  * R Core Team (2019) R Core Team (2019), ‘R: A Language and Environment for Statistical Computing’.
  * Samuels et al. (1991) Samuels, M. L., Casella, G. & McCabe, G. P. (1991), ‘Interpreting Blocks and Random Factors’, Journal of the American Statistical Association 86(415), 798–808.
  * Satterthwaite (1946) Satterthwaite, F. E. (1946), ‘An approximate distribution of estimates of variance components’, Biometrika Bulletin 2(6), 110–114.
  * Speed & Bailey (1987) Speed, T. P. & Bailey, R. A. (1987), ‘Factorial dispersion models’, International Statistical Review 55(3), 261–277.
  * Tjur (1984) Tjur, T. (1984), ‘Analysis of variance models in orthogonal designs’, International Statistical Review 52(1), 33–65.
  * Wilkinson & Rogers (1973) Wilkinson, G. N. & Rogers, C. E. (1973), ‘Symbolic description of factorial models for analysis of variance’, Journal of the Royal Statistical Society C 22(3), 392–399.
  * Yates (1935) Yates, F. (1935), ‘Complex Experiments’, Journal of the Royal Statistical Society 2(2), 181–247.