11institutetext: Department of Computer Science, ETH Zürich
11email<EMAIL_ADDRESS>22institutetext:
Department of Neonatology,
University Children’s Hospital Regensburg (KUNO), Germany

# Interpretable Prediction of Pulmonary Hypertension in Newborns using
Echocardiograms

Hanna Ragnarsdottir 11 Laura Manduchi(✉) 11 Holger Michel 22 Fabian Laumer 11
Sven Wellmann 22 Ece Ozkan(✉) 11 Julia Vogt 11

###### Abstract

Pulmonary hypertension (PH) in newborns and infants is a complex condition
associated with several pulmonary, cardiac, and systemic diseases contributing
to morbidity and mortality. Therefore, accurate and early detection of PH is
crucial for successful management. Using echocardiography, the primary
diagnostic tool in pediatrics, human assessment is both time-consuming and
expertise-demanding, raising the need for an automated approach. In this work,
we present an interpretable multi-view video-based deep learning approach to
predict PH for a cohort of 194 newborns using echocardiograms. We use spatio-
temporal convolutional architectures for the prediction of PH from each view,
and aggregate the predictions of the different views using majority voting. To
the best of our knowledge, this is the first work for an automated assessment
of PH in newborns using echocardiograms. Our results show a mean F1-score of
0.84 for severity prediction and 0.92 for binary detection using 10-fold
cross-validation. We complement our predictions with saliency maps and show
that the learned model focuses on clinically relevant cardiac structures,
motivating its usage in clinical practice.

###### Keywords:

Echocardiography Pediatrics Computer Assisted Diagnosis (CAD) Interpretable
Machine Learning.

##  1 Introduction

Pulmonary hypertension (PH) is a complex condition that can affect newborns
and children as well as adults and is formally defined as an increased mean
pulmonary artery pressure (PAP) at rest [12]. The level of PAP in newborns is
frequently high and it is expected to decrease after birth to reach a level
comparable to healthy adult values [4]. When the normal cardiopulmonary
transition fails to occur, the newborns are affected by persistent pulmonary
hypertension of the newborn (PPHN), which is associated with bronchopulmonary
dysplasia in older premature infants and various chronic pulmonary, cardiac,
and systemic diseases for newborns at term contributing to morbidity and
mortality [9, 12, 25].

Pulmonary hypertension is diagnosed primarily with echocardiography, which is
recommended as a non-invasive screening tool for PH using its two modes: 2D
echocardiography videos (ECHOs) and Doppler echocardiograms [19, 21]. It is
one of the most common and growing diagnostic tools due to its low-cost,
portable, and non-invasive technology, which makes it an ideal choice for
pediatrics [19]. Screening of PH typically involves estimating PAP with
Doppler echocardiography, however, the measurements may frequently be
inaccurate, thus, is not the ultimate predictive tool to assess and manage PH
[10]. Since elevated PAP can result in abnormalities in the shape and
structure of the heart, subjective evaluation on ECHOs is often performed as
well, i.e. to detect the abnormalities in the shape of the septal wall and
changes in the right-ventricular area [11, 27]. However, the aforementioned
procedures for PH estimation are time-consuming and expertise-demanding, which
may delay care to a more advanced stage of illness, potentially decreasing the
chance of survival [2].

####  1.0.1 Related Work.

Although several machine learning methods have been proposed to automatically
estimate PH in adults using different input modalities, such as chest X-rays
[29, 17], ECGs [18, 20], heart sounds recorded by sensors [14], CTs [26], and
MRIs [8, 3], not much effort has been directed towards the automatic
assessment of PH using echocardiography. The only exception is the work of
Zhang et al. [27]. This method, however, focuses on PH in adults and has
several limitations. First of all, it works on static frames of the ECHO
videos and does not exploit the spatio-temporal patterns in the sequence.
Second, it only uses a single view of the heart (apical 4-chamber view),
although the literature has shown that considering multiple views improves
accuracy for the manual assessment of PH [22]. Moreover, previous works on
automatic PH detection focus on binary classification, which limits the
clinical usability, because, the determination of the severity of PH from
ECHOs is crucial for correct treatment, but is a challenge for cardiologists
[7, 10]. Finally, the applicability of deep learning methods, such as [27], is
still limited in the medical domain due to their black-box nature that makes
their internal mechanisms and their results opaque.

####  1.0.2 Contribution.

In this work, we propose a robust and interpretable deep learning approach –
_i ntepretable prediction of pulmonary hypertension in newborns_ (IP-PHN) to
predict and classify the severity of PH by utilising spatio-temporal patterns
of the ECHOs from multiple views. To the best of our knowledge, this is the
first work on multi-view video-based automated assessment of PH in newborns.
To increase its clinical usability, we complement our predictions with
saliency maps that highlight how the learned model focuses on clinically
relevant cardiac structures. We show that these learned localization maps
align with how clinicians subjectively assess PH. To ensure the
reproducibility of this work, the code of IP-PHN was made publicly available
under https://anonymous.4open.science/r/echo_classification-DE4E/.

##  2 Materials and Methods

###  2.1 Dataset

The dataset in this work consists of 2D transthoracic echocardiography videos
(ECHOs) in 5 different standard views. Retrospectively, 194 newborns were
examined and 536 ECHOs were performed in a single centre by a pediatric
cardiologist between the years 2019-2020. All ECHOs were performed with the GE
Logic S8 ultrasound machine using the transducer S4-10. A single ECHO contains
a sequence of ultrasound images of the patient’s heart at a specific view. The
five views include a parasternal long-axis view (PLAX), apical four-chamber
view (A4C), and three parasternal short-axis views; at the level of papillary
muscles (PSAX-P), at the level of semilunar valves (PSAX-S), and on the apical
short-axis view (PSAX-A). The ECHOs operate on 25 fps, and the average video
length is 5 seconds, whereby each ECHO consists of around 10 heartbeats.

The ground-truth for each ECHO was manually annotated by a pediatric
cardiologist based on the visual evaluation. The ground truth labels
differentiate between, none (65%), mild (16%), and moderate to severe (19%)
PH. Furthermore, for each ECHO its corresponding view is also annotated. A
detailed overview of the data is provided in Supplementary Material. The study
was approved by the local ethics committee. In addition, all data is
pseudonymized.

###  2.2 Preprocessing and Data Augmentation

As the first step in pre-processing the available data, we crop and mask the
ECHOs to eliminate information (such as additional text or signals) outside
the scanning sector and resize them to $224$x$224$ pixels using bilinear
interpolation. We then apply histogram equalization to distribute the pixel
intensities to the full range of gray-scale values and normalize them.

During training, two types of image transformations are applied: intensity
transformations so that the learned model is invariant to intensity
variations, and spatial transformations to increase resilience against
different zoom settings of the US machine and/or placements of the transducer.
In particular, we apply the following random transformations to each sequence:
sharpness and brightness adjustment, gamma correction, addition of salt and
pepper or Gaussian noise, variation of the background with different amounts
of speckle noise, rotation up to $15^{\circ}$, translation up to 0.1x, scaling
down to 0.8x, zooming up to 1.2x.

###  2.3 Proposed Method

We introduce an end-to-end deep learning approach, IP-PHN, to automatically
assess PH severity without the need for tedious manual measurements of the
ventricles used in standard clinical workflow. We assume that for each patient
we have access to multiple ECHOs showing the heart from different views. Even
though a single view is commonly used for subjective assessment of PH, recent
works have shown that using multiple views is beneficial for the assessment
[9, 22].

The proposed framework is depicted in Fig. 1. We first process each view
separately. In particular, we employ a 3D-CNN architecture with residual
connections and spatio-temporal convolutions across frames [13] using a
ResNet-18 as the model backbone [5] (see Fig. 1(a)). In contrast to previous
work [27], our approach integrates spatial as well as temporal information
into the learning process. This mitigates the frame-level variations that can
occur due to external changes, such as the position or the contact of the
transducer, or in the cardiac function itself. To overcome the scarcity of the
annotated data, common in the medical domain, from each ECHO we extract $n$
shorter video sequences by randomly choosing a frame as their starting frame
followed by $k-1$ consecutive frames, with total $k$ frames, covering on
average one heartbeat. We then aggregate sequence-level predictions
$\\{y_{\text{view},i}\\}_{i=1,\ldots,n}$ through majority voting, i.e. by
selecting the most frequently predicted label, to a view-level prediction
$y_{\text{view}}$. The view-level confidence is then defined as
$C=|y^{*}_{\text{view}}|/n$, where $|y^{*}_{\text{view}}|$ is the count of the
most frequently predicted label from the list of predictions for the $n$
sequences of a given ECHO per view.

Figure 1: Overview of our proposed method, IP-PHN, to automatically assess PH
severity of a patient using (a) single view and (b) multi-view approach with
majority voting utilising spatio-temporal patterns of ECHOs. Spatio-temporal
saliency maps (c) are provided from each view to increase clinical usability.

To increase the robustness of the method further we employ a multi-view
approach by combining the models trained on each available view (as in Fig.
1(b)). The final subject-level prediction, $y_{\text{f}}$, is then achieved by
majority voting of the view-level predictions. In the case of a tie, the
prediction of the model(s) with higher confidence is selected. We tried
different approaches for view aggregation, but simple majority voting
performed the best. Note that our proposed method uses view annotations to
differentiate the distinct modalities. Following recent work on view
classification [27], IP-PHN can easily be extended to incorporate ECHOs
without annotation.

####  2.3.1 Interpretability.

To increase the accountability and clinical usability of our proposed method,
we complement our predictions with spatio-temporal saliency maps from each
view (as in Fig. 1(c)). The automatic localisation of relevant pixels in the
video sequence for the model’s prediction provides an interpretable
explanation that mimics the clinical workflow. Among different methods [24,
28, 23], we chose to use Grad-CAM (Gradient-weighted Class Activation
Mapping), which exploits the gradients of any target concept flowing into a
given convolutional layer to produce a coarse localization map highlighting
the important regions in the image for predicting the label [23]. Recent works
[15, 1] have shown how pixel-space gradient visualizations, such as Guided
Backpropagation and Guided Grad-CAM, could be rather insensitive to model and
data, making them similar to simple edge detectors. Grad-CAM is one of few
saliency methods that pass the insensitivity check, making it our saliency
method of choice [1].

However, Grad-CAM was originally proposed for 2D-CNNs trained on images. We,
therefore, extend Grad-CAM to 3D-CNNs processing spatio-temporal video inputs.
This allows us to identify the spatio-temporal regions on the video sequence
that the network finds most informative for its prediction, which are the
regions in both spatial and time domains.

##  3 Experiments and Results

###  3.1 Implementation Details

Our method was developed using the Python programming language with the
PyTorch deep learning library. Experiments were run on a cluster containing
different NVIDIA GeForce graphic cards: GTX 1080, GTX 1080 Ti, RTX 2080 Ti
with 2048 MB RAM per processor core.

For each view, we extracted $n=10$ video sequences and each sequence was
composed of $k=12$ consecutive frames, covering on average one heartbeat in
every sequence. To deal with class imbalance we employed a weighted random
sampler, which samples elements from the dataset using their inverse class
weight as their sample weight, ensuring that for every epoch the model sees
approximately equal number of samples from each class. Additionally, during
training, we continuously augmented each sample with a probability of 90%.
Each model was trained for around $150$ epochs per view minimizing the
(categorical) cross-entropy loss with the Adam [16] optimiser. Both the
learning rate and weight decay were set to $0.001$, while the batch size was
set to $8$ video sequences.

###  3.2 Experimental Setup

For all experiments, a stratified 10-fold cross-validation was performed, such
that the data was randomly split 10 times into 20% validation set and 80%
training set. Note that, the splitting into training and validation sets was
done on a patient basis. As classification metrics, we evaluated the area
under the receiver operation characteristic (AUROC, one-vs-one), balanced
accuracy, weighted F1-score, weighted precision, and weighted recall. The
multi-view AUROC was computed from the output probabilities of the most
confident model selected by the majority voting. We also report the average
confidence of the models in the Supplementary Material. Results were averaged
over the folds, and the mean and standard deviation are reported.

###  3.3 Results and Discussions

We hereby provide an empirical assessment of IP-PHN for PH severity
prediction. We report in Table 1(a) the quantitative performance from each of
the three major views (A4C, PLAX, PSAX-P) (see Fig. 1(a)), as well as from the
multi-view approach (see Fig. 1(b)) obtained through majority voting using
different combinations of views. In particular, we combined (i) the $3$ major
views (MV-3) and (ii) all views for a total of $5$ different views (MV-All).

Table 1: Quantitative performance of IP-PHN for (a) PH severity prediction and (b) binary PH detection using both single ECHO views (A4C, PLAX, and, PSAX-P) and multiple views through majority voting (MV-3 and MV-All). The best results for each task have been highlighted in bold.  | Views | AUROC | F1-Score | Precision | Recall |  Balanced Accuracy
---|---|---|---|---|---|---
(a) | A4C | 0.77$\pm 0.03$ | 0.72$\pm 0.05$ | 0.75$\pm 0.05$ | 0.72$\pm 0.05$ | 0.65$\pm 0.06$
PLAX | 0.85$\pm 0.04$ | 0.78$\pm 0.05$ | 0.82$\pm 0.06$ | 0.79$\pm 0.06$ | 0.72$\pm 0.05$
PSAX-P | 0.85$\pm 0.04$ | 0.81$\pm 0.05$ | 0.83$\pm 0.06$ | 0.82$\pm 0.04$ | 0.73$\pm 0.06$
MV-3 | 0.84$\pm 0.08$ | 0.83$\pm 0.05$ | 0.86$\pm 0.04$ | 0.83$\pm 0.05$ | 0.76$\pm 0.07$
MV-All | 0.86$\mathbf{\pm 0.09}$ | 0.84$\mathbf{\pm 0.06}$ | 0.86$\mathbf{\pm 0.05}$ | 0.85$\mathbf{\pm 0.05}$ | 0.78$\mathbf{\pm 0.07}$
(b) | A4C | 0.83$\pm 0.05$ | 0.81$\pm 0.04$ | 0.84$\pm 0.03$ | 0.81$\pm 0.04$ | 0.81$\pm 0.04$
PLAX | 0.90$\pm 0.07$ | 0.86$\pm 0.09$ | 0.88$\pm 0.07$ | 0.86$\pm 0.09$ | 0.86$\pm 0.08$
PSAX-P | 0.95$\mathbf{\pm 0.04}$ | 0.92$\mathbf{\pm 0.03}$ | 0.93$\mathbf{\pm 0.03}$ | 0.92$\mathbf{\pm 0.03}$ | 0.94$\mathbf{\pm 0.03}$
MV-3 | 0.90$\pm 0.03$ | 0.87$\pm 0.04$ | 0.88$\pm 0.03$ | 0.87$\pm 0.04$ | 0.87$\pm 0.04$
MV-All | 0.90$\pm 0.03$ | 0.89$\pm 0.02$ | 0.90$\pm 0.02$ | 0.89$\pm 0.02$ | 0.89$\pm 0.02$

Among the single-view methods, the parasternal short-axis view at the level of
papillary muscles (PSAX-P) shows the best performance in PH severity
prediction and achieves an F1-score of 0.81 and Balanced Accuracy of 0.73,
followed by the parasternal long-axis view (PLAX). We observe that both PSAX-P
and PLAX are fairly confident in their prediction, with an average confidence
of correct predictions of 0.92 and 0.90 respectively, while it decreases to
0.83 and 0.84 for wrong predictions. Although the apical four-chamber view
(A4C) is one of the most commonly used views for cardiovascular disease
diagnosis, our evaluation shows that it is not as discriminative as PSAX-P and
PLAX, yielding an F1-score of 0.72 and Balanced Accuracy of 0.65, which are
clearly lower than the other two views. This is also in line with the neonatal
echocardiography teaching manual [9], where it is stated that subjective
assessment of PH from the A4C view in a 2D ECHO is usually only possible for
moderate to severe PH cases, and quantitative evaluation is difficult.

The PH severity prediction problem is challenging, not only due to the hard
task at hand but also because of the data imbalance. In this case, the
robustness and accuracy can be increased by utilising more views. By combining
results from the PSAX-P, PLAX, and A4C views using majority voting, the
F1-score improves from from $0.81$ to $0.83$, while the Balanced Accuracy
improves from $0.73$ to $0.76$. When the other two short-axis views are
furthermore included, we get an F1-score of $0.84$ and a Balanced Accuracy of
$0.78$. The majority voting is not only helpful to enhance the performance,
but it is also useful in case a single view has an unsatisfactory quality for
a given subject, a common scenario in many real-world applications.

Moreover, we performed an additional ablation, where we simplified the problem
setting to a binary classification. We then discriminated between no PH (65%
of the data) and PH, combining mild, moderate, and severe cases, in line with
previous work [27]. Even though the assessment of the severity of PH is
crucial for a correct treatment, as the morbidity rate significantly increases
for higher degrees of PH [6, 11], the clinicians might also be interested in
simply discriminating between healthy and unhealthy patients as an initial
screening procedure. In such a case, the data imbalance would be less
significant. We report the binary PH detection results in Table 1(b). PSAX-P
view is still the most discriminative one, which yields improved accuracy
compared to the severity prediction, with an F1-score of $0.92$ and Balanced
Accuracy of $0.94$. Given the substantial prediction accuracy of the PSAX-P
view alone for binary PH detection, including more views to the aggregated
model does not result in increased performances.

The existing method [27] for binary PH detection in adults does not exploit
the spatio-temporal patterns. Thus, as a comparison to our proposed spatio-
temporal approach, we also evaluated the spatial-only approach which proved to
be inferior in both, severity prediction and binary detection tasks. The
results are reported in Supplementary Material and can be used as a baseline
comparison.

####  3.3.1 Interpretability.

To increase the clinical usability, our method contains a post-hoc analysis of
the single-view spatio-temporal convolutions. For each ECHO view, we highlight
the pixels that are the most relevant for the assessment of PH severity. In
Fig. 2 we show the original ECHO frames with different levels of PH (left
column) combined with saliency maps using Grad-CAM (right column)
corresponding to the significant views, in (a) PSAX-P and in (b) PLAX.

According to the neonatal echocardiography teaching manual [9], the ideal view
for subjective evaluation of the intraventricular septum (IVS) morphology and
left ventricle (LV) shape is PSAX-P. In mild to moderate PH the IVS becomes
flat during systole and in moderate to severe PH the septum bows into the LV,
such that the LV becomes D-shaped, or crescentic. We show in Fig. 2(a) that
our PSAX-P severity prediction model focuses on the same clinically relevant
features as are recommended for diagnosis, that is the LV and IVS.

Subjective evaluation of the IVS morphology is also possible from the PLAX
view [9]. Furthermore, quantitative assessments are frequently performed on
this view, including measurements of the aortic valve (AV) annulus diameter,
and left atrial-to-aortic root diameter ratio (LA:Ao) by extracting the M-mode
as demonstrated with the yellow line in Fig. 2(b). When exploring the saliency
map of the PLAX severity model, we see in Fig. 2(b) that the model focuses on
the area around the LA, AV and Ao, and IVS. This suggests that the model is
able to consider both the relevant quantitative features and the subjective
ones.

Note that, for simplicity, we show the visualisation results of a single frame
per patient in Fig. 2. In a clinical setting, the visualizations can be viewed
as a video containing spatio-temporal explanation. In Supplementary Material,
we show more examples of how the focus changes along the frames of a sequence.

(a) PSAX-P

(b) PLAX

Figure 2: ECHO frames of subjects with no, mild and severe PH (left), as well
as the IP-PHN saliency maps (right), for (a) the PSAX-P view, and (b) the PLAX
view. The yellow line shows how the M-mode for the LA:Ao measurement is
extracted. The highlighted pixels feature crucial cardiac structures.

##  4 Conclusion

In this work we developed an automated and streamlined approach, IP-PHN, to
assist clinicians in the assessment of PH in newborns. We achieved optimal
performance for the severity prediction of PH when using a multi-view approach
with spatio-temporal convolutional architectures using majority voting of
several views. The severity estimation of PH from ECHOs is critically
important as it determines the urgency of a treatment [6, 11], but it remains
a challenge for cardiologists [7, 10]. Thus, IP-PHN may have a considerable
clinical impact in increasing the accuracy and steadiness of ECHO examinations
by reducing the number of late or missed diagnoses of PH. Furthermore, it may
assist less trained specialists and thereby reduce the workload of highly
trained experts. Finally, by highlighting the input features that are crucial
for the PH assessment, the proposed approach provides interpretable
explanations for the clinicians, which in turn make the system accountable.

## References

  * [1] Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., Kim, B.: Sanity checks for saliency maps. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. p. 9525–9536. NIPS’18, Curran Associates Inc., Red Hook, NY, USA (2018)
  * [2] Barst, R.J., McGoon, M.D., Elliott, C.G., Foreman, A.J., Miller, D.P., Ivy, D.D.: Survival in childhood pulmonary arterial hypertension. Circulation 125(1), 113–122 (Jan 2012)
  * [3] Bello, G.A., Dawes, T.J.W., Duan, J., Biffi, C., de Marvao, A., Howard, L.S.G.E., Gibbs, J.S.R., Wilkins, M.R., Cook, S.A., Rueckert, D., O’Regan, D.P.: Deep-learning cardiac motion analysis for human survival prediction. Nature Machine Intelligence 1(2), 95–104 (Feb 2019)
  * [4] de Boode, W.P., , Singh, Y., Molnar, Z., Schubert, U., Savoia, M., Sehgal, A., Levy, P.T., McNamara, P.J., El-Khuffash, A.: Application of neonatologist performed echocardiography in the assessment and management of persistent pulmonary hypertension of the newborn. Pediatric Research 84(S1), 68–77 (Jul 2018)
  * [5] Carreira, J., Zisserman, A.: Quo vadis, action recognition? a new model and the kinetics dataset. pp. 4724–4733 (07 2017)
  * [6] Corris, P., Degano, B.: Severe pulmonary arterial hypertension: treatment options and the bridge to transplantation. European Respiratory Review 23(134), 488–497 (Dec 2014)
  * [7] Dasgupta, S., Richardson, J.C., Aly, A.M., Jain, S.K.: Role of functional echocardiographic parameters in the diagnosis of bronchopulmonary dysplasia-associated pulmonary hypertension. Journal of Perinatology 42(1), 19–30 (Mar 2021)
  * [8] Dawes, T.J.W., de Marvao, A., Shi, W., Fletcher, T., Watson, G.M.J., Wharton, J., Rhodes, C.J., Howard, L.S.G.E., Gibbs, J.S.R., Rueckert, D., Cook, S.A., Wilkins, M.R., O’Regan, D.P.: Machine learning of three-dimensional right ventricular motion enables outcome prediction in pulmonary hypertension: A cardiac MR imaging study. Radiology 283(2), 381–390 (May 2017)
  * [9] EL-Khuffash, A.: Neonatal echocardiography teaching manual (01 2014)
  * [10] Fisher, M.R., Forfia, P.R., Chamera, E., Housten-Harris, T., Champion, H.C., Girgis, R.E., Corretti, M.C., Hassoun, P.M.: Accuracy of doppler echocardiography in the hemodynamic assessment of pulmonary hypertension. American Journal of Respiratory and Critical Care Medicine 179(7), 615–621 (Apr 2009)
  * [11] Galiè, N., Humbert, M., Vachiery, J.L., Gibbs, S., Lang, I., Torbicki, A., Simonneau, G., Peacock, A., Noordegraaf, A.V., Beghetti, M., Ghofrani, A., Sanchez, M.A.G., Hansmann, G., Klepetko, W., Lancellotti, P., Matucci, M., McDonagh, T., Pierard, L.A., Trindade, P.T., Zompatori, M., Hoeper, M.: 2015 ESC/ERS guidelines for the diagnosis and treatment of pulmonary hypertension. European Respiratory Journal 46(4), 903–975 (Aug 2015)
  * [12] Hansmann, G.: Pulmonary hypertension in infants, children, and young adults. Journal of the American College of Cardiology 69(20), 2551–2569 (2017)
  * [13] Hara, K., Kataoka, H., Satoh, Y.: Learning spatio-temporal features with 3D residual networks for action recognition. pp. 3154–3160 (10 2017)
  * [14] Kaddoura, T., Vadlamudi, K., Kumar, S., Bobhate, P., Guo, L., Jain, S., Elgendi, M., Coe, J.Y., Kim, D., Taylor, D., Tymchak, W., Schuurmans, D., Zemp, R.J., Adatia, I.: Acoustic diagnosis of pulmonary hypertension: automated speech- recognition-inspired classification algorithm outperforms physicians. Scientific Reports 6(1) (Sep 2016)
  * [15] Kindermans, P.J., Hooker, S., Adebayo, J., Alber, M., Schütt, K.T., Dähne, S., Erhan, D., Kim, B.: The (Un)reliability of Saliency Methods, pp. 267–280. Springer International Publishing, Cham (2019)
  * [16] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  * [17] Kusunose, K., Hirata, Y., Tsuji, T., Kotoku, J., Sata, M.: Deep learning to predict elevated pulmonary artery pressure in patients with suspected pulmonary hypertension using standard chest X-ray. Scientific Reports 10(1) (Nov 2020)
  * [18] Kwon, J.M., Kim, K.H., Medina-Inojosa, J., Jeon, K.H., Park, J., Oh, B.H.: Artificial intelligence for early prediction of pulmonary hypertension using electrocardiography. The Journal of Heart and Lung Transplantation 39(8), 805–814 (Aug 2020)
  * [19] Lang, R., Badano, L., Mor-Avi, V., Afilalo, J., Armstrong, A., Ernande, L., Flachskampf, F., Foster, E., Goldstein, S., Kuznetsova, T., Lancellotti, P., Muraru, D., Picard, M.H., Rietzschel, E.R., Rudski, L., Spencer, K., Tsang, W., Voigt, J.: Recommendations for cardiac chamber quantification by echocardiography in adults: an update from the american society of echocardiography and the european association of cardiovascular imaging. European heart journal cardiovascular Imaging 16 3, 233–70 (2015)
  * [20] Mori, H., Inai, K., Sugiyama, H., Muragaki, Y.: Diagnosing atrial septal defect from electrocardiogram with deep learning. Pediatric Cardiology 42(6), 1379–1387 (Apr 2021)
  * [21] Ni, J.R., Yan, P.J., Liu, S.D., Hu, Y., Yang, K.H., Song, B., Lei, J.Q.: Diagnostic accuracy of transthoracic echocardiography for pulmonary hypertension: a systematic review and meta-analysis. BMJ Open 9(12), e033084 (Dec 2019)
  * [22] Schneider, M., Pistritto, A.M., Gerges, C., Gerges, M., Binder, C., Lang, I.M., Maurer, G., Binder, T., Goliasch, G.: Multi-view approach for the diagnosis of pulmonary hypertension using transthoracic echocardiography. The International Journal of Cardiovascular Imaging 34, 695 – 700 (2017)
  * [23] Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: Visual explanations from deep networks via gradient-based localization. In: 2017 IEEE International Conference on Computer Vision. pp. 618–626 (2017)
  * [24] Springenberg, J.T., Dosovitskiy, A., Brox, T., Riedmiller, M.A.: Striving for simplicity: The all convolutional net. CoRR abs/1412.6806 (2015)
  * [25] Steinhorn, R.H.: Neonatal pulmonary hypertension. Pediatric Critical Care Medicine 11, S79–S84 (2010)
  * [26] Vainio, T., Mäkelä, T., Savolainen, S., Kangasniemi, M.: Performance of a 3d convolutional neural network in the detection of hypoperfusion at CT pulmonary angiography in patients with chronic pulmonary embolism: a feasibility study. European Radiology Experimental 5(1) (Sep 2021)
  * [27] Zhang, J., Gajjala, S., Agrawal, P., Tison, G.H., Hallock, L.A., Beussink-Nelson, L., Lassen, M.H., Fan, E., Aras, M.A., Jordan, C., Fleischmann, K.E., Melisko, M., Qasim, A., Shah, S.J., Bajcsy, R., Deo, R.C.: Fully automated echocardiogram interpretation in clinical practice. Circulation 138(16), 1623–1635 (Oct 2018)
  * [28] Zhou, B., Khosla, A., Lapedriza, À., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 2921–2929 (2016)
  * [29] Zou, X.L., Ren, Y., Feng, D.Y., He, X.Q., Guo, Y.F., Yang, H.L., Li, X., Fang, J., Li, Q., Ye, J.J., Han, L.Q., Zhang, T.T.: A promising approach for screening pulmonary hypertension based on frontal chest radiographs using deep learning: A retrospective study. PLOS ONE 15(7), e0236378 (Jul 2020)

##  Appendix 0.A Supplementary Material

Table 2: Characteristics of the dataset. Feature | Value
---|---
PH (#None (%) / #Mild(%) / #Severe(%)) | 126(65%) / 32(16%) / 36(19%)
Age (days) (Mean $\pm$ SD) | 56 $\pm$ 160
Maturity in birth (days) (Mean $\pm$ SD) | 230 $\pm$ 46
Patient’s weight (kg) (Mean $\pm$ SD) | 2.9 $\pm$ 1.5
Manufacturer (Ultrasound Machine / Transducer) | GE Logic S8 / S4-10 at 6 MHz
Spatial size of original 2D images (pixels) | 1440 x 866
Video length (frames) | 122 $\pm$ 2
Video FPS | 25 fps
Figure 3: Spatio-temporal Grad-CAM saliency maps (bottom) imposed on the
original frames (top) for frames corresponding to systole, mid, diastole, mid
in a PLAX ECHO. Table 3: Extended results from both spatial and spatio-
temporal approaches for (a) PH severity prediction and (b) binary PH
detection. MV-3 refers to majority voting of A4C, PLAX, and, PSAX-P views. MV-
All refers to majority voting of all five views. The best results for each
task have been highlighted in bold. In both cases, these are spatio-temporal
approaches. For the spatial approach for binary PH detection, we achieve AUROC
of 0.87 on the $\textrm{A4C}^{*}$ view, compared to state-of-the-art with 0.85
when evaluating a similar task in adults. (a)
---
| View | AUROC | F1-Score | Precision | Recall |  Balanced Accuracy | Confidence
Spatial | A4C | 0.79$\pm 0.04$ | 0.75$\pm 0.05$ | 0.77$\pm 0.04$ | 0.75$\pm 0.06$ | 0.67$\pm 0.06$ | 0.84$\pm 0.03$
PLAX | 0.84$\pm 0.04$ | 0.76$\pm 0.04$ | 0.78$\pm 0.05$ | 0.77$\pm 0.04$ | 0.70$\pm 0.05$ | 0.85$\pm 0.03$
PSAX-P | 0.83$\pm 0.03$ | 0.81$\pm 0.02$ | 0.82$\pm 0.03$ | 0.81$\pm 0.03$ | 0.74$\pm 0.03$ | 0.83$\pm 0.03$
PSAX-S | 0.74$\pm 0.07$ | 0.68$\pm 0.06$ | 0.70$\pm 0.07$ | 0.70$\pm 0.08$ | 0.62$\pm 0.06$ | 0.83$\pm 0.03$
PSAX-A | 0.80$\pm 0.03$ | 0.75$\pm 0.04$ | 0.76$\pm 0.04$ | 0.76$\pm 0.05$ | 0.66$\pm 0.04$ | 0.84$\pm 0.03$
MV-3 | 0.84 $\pm 0.03$ | 0.81 $\pm 0.03$ | 0.83 $\pm 0.04$ | 0.82 $\pm 0.03$ | 0.74 $\pm 0.06$ | 0.85 $\pm 0.02$
MV-All | 0.84$\pm 0.03$ | 0.82 $\pm 0.03$ | 0.83 $\pm 0.04$ | 0.83 $\pm 0.03$ | 0.73 $\pm 0.04$ | 0.85 $\pm 0.01$
Spatio-temporal | A4C | 0.77$\pm 0.03$ | 0.72$\pm 0.05$ | 0.75$\pm 0.05$ | 0.72$\pm 0.05$ | 0.65$\pm 0.06$ | 0.88$\pm 0.02$
PLAX | 0.85$\pm 0.04$ | 0.78$\pm 0.05$ | 0.82$\pm 0.06$ | 0.79$\pm 0.06$ | 0.72$\pm 0.05$ | 0.89$\pm 0.03$
PSAX-P | 0.85$\pm 0.04$ | 0.81$\pm 0.05$ | 0.83$\pm 0.06$ | 0.82$\pm 0.04$ | 0.73$\pm 0.06$ | 0.90$\pm 0.03$
PSAX-S | 0.73$\pm 0.07$ | 0.68$\pm 0.08$ | 0.69$\pm 0.09$ | 0.69$\pm 0.08$ | 0.62$\pm 0.07$ | 0.85$\pm 0.04$
PSAX-A | 0.77$\pm 0.07$ | 0.74$\pm 0.06$ | 0.77$\pm 0.04$ | 0.74$\pm 0.06$ | 0.67$\pm 0.06$ | 0.84$\pm 0.04$
MV-3 | 0.84$\pm 0.08$ | 0.83$\pm 0.05$ | 0.86$\pm 0.04$ | 0.83$\pm 0.05$ | 0.76$\pm 0.07$ | 0.91$\pm 0.02$
MV-All | 0.86$\mathbf{\pm 0.09}$ | 0.84$\mathbf{\pm 0.06}$ | 0.86$\mathbf{\pm 0.05}$ | 0.85$\mathbf{\pm 0.05}$ | 0.78$\mathbf{\pm 0.07}$ | 0.90$\mathbf{\pm 0.02}$
(b)
| View | AUROC | F1-Score | Precision | Recall |  Balanced Accuracy | Confidence
Spatial | $\textrm{A4C}^{*}$ | 0.87$\pm 0.04$ | 0.83$\pm 0.04$ | 0.85$\pm 0.03$ | 0.83$\pm 0.04$ | 0.83$\pm 0.03$ | 0.87$\pm 0.03$
PLAX | 0.92$\pm 0.05$ | 0.88$\pm 0.04$ | 0.89$\pm 0.04$ | 0.88$\pm 0.04$ | 0.88$\pm 0.04$ | 0.89$\pm 0.02$
PSAX-P | 0.93$\pm 0.04$ | 0.91$\pm 0.03$ | 0.92$\pm 0.03$ | 0.91$\pm 0.03$ | 0.92$\pm 0.03$ | 0.87$\pm 0.03$
PSAX-S | 0.83$\pm 0.03$ | 0.81$\pm 0.03$ | 0.83$\pm 0.02$ | 0.81$\pm 0.03$ | 0.81$\pm 0.03$ | 0.86$\pm 0.04$
PSAX-A | 0.86$\pm 0.04$ | 0.85$\pm 0.03$ | 0.85$\pm 0.03$ | 0.85$\pm 0.03$ | 0.84$\pm 0.03$ | 0.87$\pm 0.02$
MV-3 | 0.91 $\pm 0.02$ | 0.88 $\pm 0.02$ | 0.88 $\pm 0.02$ | 0.88 $\pm 0.02$ | 0.88 $\pm 0.02$ | 0.88 $\pm 0.01$
MV-All | 0.92 $\pm 0.02$ | 0.90 $\pm 0.02$ | 0.91 $\pm 0.01$ | 0.90 $\pm 0.02$ | 0.90 $\pm 0.01$ | 0.87 $\pm 0.02$
Spatio-temporal | A4C | 0.83$\pm 0.05$ | 0.81$\pm 0.04$ | 0.84$\pm 0.03$ | 0.81$\pm 0.04$ | 0.81$\pm 0.04$ | 0.91$\pm 0.03$
PLAX | 0.90$\pm 0.07$ | 0.86$\pm 0.09$ | 0.88$\pm 0.07$ | 0.86$\pm 0.09$ | 0.86$\pm 0.08$ | 0.91$\pm 0.02$
PSAX-P | 0.95$\mathbf{\pm 0.04}$ | 0.92$\mathbf{\pm 0.03}$ | 0.93$\mathbf{\pm 0.03}$ | 0.92$\mathbf{\pm 0.03}$ | 0.94$\mathbf{\pm 0.03}$ | 0.90$\mathbf{\pm 0.03}$
PSAX-S | 0.79$\pm 0.04$ | 0.81$\pm 0.03$ | 0.82$\pm 0.04$ | 0.81$\pm 0.03$ | 0.80$\pm 0.04$ | 0.90$\pm 0.02$
PSAX-A | 0.88$\pm 0.05$ | 0.87$\pm 0.03$ | 0.88$\pm 0.03$ | 0.87$\pm 0.03$ | 0.87$\pm 0.04$ | 0.89$\pm 0.03$
MV-3 | 0.90$\pm 0.03$ | 0.87$\pm 0.04$ | 0.88$\pm 0.03$ | 0.87$\pm 0.04$ | 0.87$\pm 0.04$ | 0.92$\pm 0.01$
MV-All | 0.90$\pm 0.03$ | 0.89$\pm 0.02$ | 0.90$\pm 0.02$ | 0.89$\pm 0.02$ | 0.89$\pm 0.02$ | 0.91$\pm 0.01$


$\displaystyle d(\Psi^{\prime})$
$\displaystyle=\alpha(p,r)^{2}(\sqrt{2}r,0,0,0)+\beta(p,r)^{2}(\sqrt{2}r,0,0,0)\
,$ (320)

where we used that $\alpha(p,r)^{2}+\beta(p,r)^{2}=1$. It follows that

$\displaystyle\mathsf{Var}_{\Psi^{\prime}}$
$\displaystyle=\operatorname{tr}(\Gamma(\Psi^{\prime}))/2$ (321)
$\displaystyle=2\alpha(p,r)^{2}+\beta(p,r)^{2}(1+\cosh(2z))\ .$ (322)

We also have

$\displaystyle\langle\Psi^{\prime},H\Psi^{\prime}\rangle$
$\displaystyle=2\alpha(p,r)^{2}+\beta(p,r)^{2}(1+\cosh(2z))+2r^{2}+2$ (323)

because

$\displaystyle\langle\Psi^{\prime},H\Psi^{\prime}\rangle$
$\displaystyle=\sum_{j=1}^{2}\langle\Psi^{\prime},(Q_{j}^{2}+P_{j}^{2}+I)\Psi^{\prime}\rangle$
(324)
$\displaystyle=\sum_{j=1}^{2}\left(\langle\Psi^{\prime},((Q_{j}-d_{Q_{j}}I)^{2}+(P_{j}-d_{P_{j}}I)^{2}+I),\Psi^{\prime}\rangle+d_{Q_{j}}^{2}+d_{P_{j}}^{2}\right)$
(325)
$\displaystyle=\mathsf{Var}_{\Psi}^{\prime}+2\|\Psi^{\prime}\|^{2}+\sum_{j=1}^{2}(d_{Q_{j}}^{2}+d_{P_{j}}^{2})\
.$ (326)

It is straightforward to check that $\lim_{p\rightarrow 0}\beta(p,1/p)=1$,
hence the claim follows. ∎

## References

  * [1] Richard P. Feynman.  Simulating physics with computers.  International Journal of Theoretical Physics, 21(6):467–488, Jun 1982.
  * [2] Leslie G. Valiant.  Quantum computers that can be simulated classically in polynomial time.  In Proceedings of the Thirty-Third Annual ACM Symposium on Theory of Computing, STOC ’01, page 114–123, New York, USA, 2001.
  * [3] Richard Jozsa and Akimasa Miyake.  Matchgates and classical simulation of quantum circuits.  Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 464(2100):3089–3106, Jul 2008.
  * [4] Barbara M. Terhal and David P. DiVincenzo.  Classical simulation of noninteracting-fermion quantum circuits.  Phys. Rev. A, 65:032325, Mar 2002.
  * [5] Emanuel Knill.  Fermionic linear optics and matchgates.  Technical Report LAUR-01-4472, Los Alamos National Laboratory, 2001.
  * [6] Scott Aaronson and Daniel Gottesman.  Improved simulation of stabilizer circuits.  Phys. Rev. A, 70:052328, Nov 2004.
  * [7] Julien Niset, Jaromír Fiurášek, and Nicolas J. Cerf.  No-go theorem for Gaussian quantum error correction.  Phys. Rev. Lett., 102:120501, Mar 2009.
  * [8] Christophe Vuillot, Hamed Asasi, Yang Wang, Leonid P. Pryadko, and Barbara M. Terhal.  Quantum error correction with the toric Gottesman-Kitaev-Preskill code.  Phys. Rev. A, 99:032344, Mar 2019.
  * [9] Daniel Gottesman, Alexei Kitaev, and John Preskill.  Encoding a qubit in an oscillator.  Physical Review A, 64(1), Jun 2001.
  * [10] Kyungjoo Noh, S. M. Girvin, and Liang Jiang.  Encoding an oscillator into many oscillators.  Phys. Rev. Lett., 125:080503, Aug 2020.
  * [11] Marcel Bergmann and Peter van Loock.  Quantum error correction against photon loss using multicomponent cat states.  Phys. Rev. A, 94:042332, Oct 2016.
  * [12] Linshu Li, Chang-Ling Zou, Victor V. Albert, Sreraman Muralidharan, S. M. Girvin, and Liang Jiang.  Cat codes with optimal decoherence suppression for a lossy bosonic channel.  Phys. Rev. Lett., 119:030502, Jul 2017.
  * [13] Zaki Leghtas, Gerhard Kirchmair, Brian Vlastakis, Robert J. Schoelkopf, Michel H. Devoret, and Mazyar Mirrahimi.  Hardware-efficient autonomous quantum memory protection.  Phys. Rev. Lett., 111:120501, Sep 2013.
  * [14] Jacob Hastrup and Ulrik Lund Andersen.  All-optical cat-code quantum error correction.  Phys. Rev. Res., 4:043065, Oct 2022.
  * [15] Victor Veitch, Nathan Wiebe, Christopher Ferrie, and Joseph Emerson.  Efficient simulation scheme for a class of quantum optics experiments with non-negative Wigner representation.  New Journal of Physics, 15(1):013037, Jan 2013.
  * [16] Andrea Mari and Jens Eisert.  Positive Wigner functions render classical simulation of quantum computation efficient.  Phys. Rev. Lett., 109:230503, Dec 2012.
  * [17] Victor Veitch, Christopher Ferrie, David Gross, and Joseph Emerson.  Negative quasi-probability as a resource for quantum computation.  New Journal of Physics, 14(11):113011, Nov 2012.
  * [18] Ulysse Chabaud, Giulia Ferrini, Frédéric Grosshans, and Damian Markham.  Classical simulation of Gaussian quantum circuits with non-Gaussian input states.  Phys. Rev. Res., 3:033018, Jul 2021.
  * [19] Victor V. Albert, Kyungjoo Noh, Kasper Duivenvoorden, Dylan J. Young, R. T. Brierley, Philip Reinhold, Christophe Vuillot, Linshu Li, Chao Shen, S. M. Girvin, Barbara M. Terhal, and Liang Jiang.  Performance and structure of single-mode bosonic codes.  Phys. Rev. A, 97:032346, Mar 2018.
  * [20] Sergey Bravyi and David Gosset.  Improved classical simulation of quantum circuits dominated by Clifford gates.  Physical Review Letters, 116(25):250501, Jun 2016.
  * [21] Beatriz Dias and Robert Koenig.  Classical simulation of non-gaussian fermionic circuits.  arXiv:2307.12912, 2023.
  * [22] Joshua Cudby and Sergii Strelchuk.  Gaussian decomposition of magic states for matchgate computations.  arXiv:2307.12654, 2023.
  * [23] Sergey Bravyi, Dan Browne, Padraic Calpin, Earl Campbell, David Gosset, and Mark Howard.  Simulation of quantum circuits by low-rank stabilizer decompositions.  Quantum, 3:181, Sep 2019.
  * [24] Oliver Reardon-Smith, Michał Oszmaniec, and Kamil Korzekwa.  Improved simulation of quantum circuits dominated by free fermionic operations.  arXiv:2307.12702, Jul 2023.
  * [25] Hakop Pashayan, Stephen D. Bartlett, and David Gross.  From estimation of quantum probabilities to simulation of quantum circuits.  Quantum, 4:223, Jan 2020.
  * [26] Hakop Pashayan, Oliver Reardon-Smith, Kamil Korzekwa, and Stephen D. Bartlett.  Fast estimation of outcome probabilities for quantum circuits.  PRX Quantum, 3(2):020361, Jun 2022.
  * [27] Gheorghe S. Paraoanu and Horia Scutaru.  Fidelity for multimode thermal squeezed states.  Phys. Rev. A, 61:022306, Jan 2000.
  * [28] John Williamson.  On the algebraic problem concerning the normal forms of linear dynamical systems.  American Journal of Mathematics, 58(1):141–163, 1936.
  * [29] Vladimir I. Arnold.  Mathematical methods of classical mechanics, volume 60.  Springer Science & Business Media, 2013.
  * [30] Arvind, B Dutta, N Mukunda, and R Simon.  The real symplectic groups in quantum mechanics and optics.  Pramana, 45(6):471–497, Dec 1995.
  * [31] Raul Garcia-Patron Sanchez.  Quantum information with optical continuous variables: from Bell tests to key distribution.  Université libre de Bruxelles, 2007.
  * [32] Leonardo Banchi, Samuel L. Braunstein, and Stefano Pirandola.  Quantum fidelity for arbitrary Gaussian states.  Phys. Rev. Lett., 115:260501, Dec 2015.
  * [33] Sergey Bravyi and David Gosset.  Complexity of quantum impurity problems.  Communications in Mathematical Physics, 356(2):451–500, Dec 2017\.
  * [34] Andreas Winter.  Energy-constrained diamond norm with applications to the uniform continuity of continuous variable channel capacities.  arXiv:1712.10267, 2017.
  * [35] Maksim E. Shirokov.  On extension of quantum channels and operations to the space of relatively bounded operators.  Lobachevskii Journal of Mathematics, 41(4):714–727, Apr 2020.


# College Spread of COVID-19 in Ohio

Akinkunle Akinola and Kimberlyn Brooks  Data Science Program
Bowling Green State University
Bowling Green, OH, USA
<EMAIL_ADDRESS>Akoh Atadoga, Vibhuti Chandna, and Robert C. Green
II  Department of Computer Science
Bowling Green State University
Bowling Green, OH, USA
<EMAIL_ADDRESS>
###### Abstract

Cumulative COVID-19 case counts by county in Ohio were gathered and combined
with population data from the Census Bureau and student enrollment by county
from the Integrated Postsecondary Education Data System (IPEDS) for colleges
or universities that compete in NCAA sports. Monthly median percent increases,
monthly average percent increases, and average cases per 100k of COVID cases
between counties with colleges and counties without colleges were calculated.
The Wilcoxon test was used to determine if the samples were similar, and the
analysis found the differences were statistically significant. Metro and non-
metro groupings were added to the average cases per 100k to further subdivide
the data set. Analysis found no statistically significant differences.

###### Index Terms:

COVID-19, college, university, county, Ohio, test

##  I Introduction

In the spring of 2020, a global pandemic began to spread across the United
States. Many colleges and universities quickly moved their courses online and
sent students home to finish out the semester, including those in the state of
Ohio. As it became more evident that COVID-19 was not going to be easily
subdued, institutions had to come up with alternate plans for how to safely
deliver educational experiences to students while protecting the health and
well-being of students, faculty, and staff. Armed with a variety of
instructional modes and guidelines from the Centers for Disease Control and
Prevention (CDC)[9], colleges and universities began making plans for students
to return for a very different type of experience in the fall of 2020[4][6].
As the pandemic continued through 2021, colleges and universities searched for
ways to safely bring students back to campus for an in-person experience. This
study analyzes community spread rates to understand the effects of these
mitigation plans by comparing the median percent change, the average percent
change, and the average cases per 100,000 people of COVID-19 in Ohio counties
with and without NCAA colleges.

In order to perform the analysis, several sources of data needed to be
considered. First and foremost COVID-19 case count data both at the daily case
count level and the cumulative case count level by county was required. Daily
case count data was obtained from the Ohio Department of Health[12] and was
used to calculate the cumulative case counts. College data was gathered from
the Integrated Postsecondary Education Data System (IPEDS)[8] and was used to
determine whether the county contained a NCAA competing college. The National
Center for Health Statistics[13] data was used to determine whether a county
is classified as a metro or non-metro region by the Centers for Disease
Control. US Census Bureau[1] data was also added to scale the data for the
average cases per 100,000 data set.

In the rest of this article, a detailed distinction is made between the
distribution of COVID-19 in college and non-college counties in Ohio. Section
II explains the datasets used and describes the age distribution of COVID-19
in Ohio. Describing data analysis and hypothesis testing, Section III walks
through the methodology. The results of the data visualization are addressed
in Section IV and statistical inference from testing of hypotheses is given.
The research conducted and its drawbacks are outlined in Section V and the
future directions are provided.

##  II Background

Throughout the pandemic, various studies have been conducted related to
college campuses and the spread of COVID-19, with mixed results. Particularly
concerning for colleges and universities was the positivity rate among persons
18-24. The positivity rate for this group was 14%, which was the highest of
any age bracket in September of 2020 [17]. College campuses are at risk to
develop an high incidence of COVID-19 and become superspreaders for
neighboring communities [27]. Meanwhile, more than 80% of our university
county sample did not experience a significant case increase in Fall 2020 and
there were no significant relationships between opening approaches and
community transmission in both mask-required and non-mask-required states
[16]. Referring to the same semester, fall of 2020,

> County-level COVID-19 incidence decreased in much of the United States in
> late summer 2020. Comparing the 21 days before and after instruction start
> dates, university counties with in-person instruction experienced a 56%
> increase in incidence and 30% increase in hotspot occurrence as well as
> increases in COVID-19-related testing and test percentage positivity.
> Results from the unmatched analysis were consistent with those from the
> matched analysis. If percentage positivity had been stable or declining
> across the observation period, then efforts on the part of many colleges and
> universities to conduct or require testing before students’ return to campus
> and their ongoing surveillance efforts might explain an increase in case
> counts, as a result of increased case discovery. However, the concurrent
> increases in percentage positivity and in incidence in these counties
> suggest that higher levels of transmission, in addition to increased case
> discovery, occurred in these communities[16].

Yet, the New York Times reported that, “When many campuses reopened in the
fall, outbreaks raced through dorms, infected hundreds of thousands of
students and employees, and spread to the wider community. After students
returned for the spring term this year, increased testing, social distancing
rules and an improving national outlook helped curb the virus on many
campuses”[2]. The Center for Disease Control and Prevention (CDC) established
and continually updated guidance for Institutions of Higher Education (IHE) to
prevent transmission of COVID-19, although the guidance was considered
supplemental and not intended to replace any federal, state, tribal, local, or
territorial health and safety laws, rules, and regulations with which IHEs
must comply [3]. The guidance included masking policies, isolation and
quarantine recommendations, and testing/screening strategies. Even though
there was guidance provided by the CDC and funding provided by the Coronavirus
Aid, Relief, and Economic Security Act (CARES), college and university plans
for instruction varied widely for the fall of 2020 and beyond. Six factors
were statistically significant in determining whether instruction would be in-
person, hybrid, or online. The six factors were: the financial strength of the
institution (using average faculty salary as a surrogate), the selectivity
(admission rate), whether the institution was public or private, the political
affiliation of the state in which the institution was located, whether the
institution was religiously affiliated, and the percentage of humanities
degrees offered[25]. Similarly, the proportion of COVID-19 cases, deaths, and
mask mandates had no association with colleges’ decisions to open their
campuses. Rather, reopening decisions were driven largely by state and county
politics as well as by budgetary concerns[24]. Ohio institutions vary widely
with respect to the variables considered, so it is no surprise that the
opening plans of the colleges and universities in Ohio varied. Instructional
modes were also modified in response to positivity rates on campuses. Since
the mode of instruction was a fluid variable, it was not considered as part of
the following analysis.

##  III COVID-19 in Ohio

An exploratory data analysis on the data sets obtained from the Ohio
Department of Health[12] was covered. The Python pandas library was used to
process and extract information from the available data set and appropriate
graphs were created for better visualizations using matplotlib, and Seaborn
libraries of Python.

###  III-A COVID-19 Spread in Ohio over time

In Fig. 1, the x-axis represents the months from 2020-01 to 2022-02, and the
y-axis represents the number of cases by onset date as of January 2020. On
March 9, Governor Mike DeWine reported Ohio’s first 3 cases in Cuyahoga
County, and by December 1, the number of total cases had increased to 430,093
in Ohio[10].

Figure 1: COVID-19 cases by onset date

###  III-B Age and Gender distribution of COVID-19 cases in Ohio

In the case of the COVID-19 pandemic, the importance of presenting
epidemiological data by age and sex groups has been emphasized. Without this
knowledge, the public is unable to make fully informed decisions regarding
their own risk of disease, and reactions to public policy cannot be clearly
tailored. Fig. 2 analyzes the spread of COVID-19 in Ohio to understand which
age group is most affected. It appears that age group 20-29 is affected the
most i.e. 19.7% as compared to elderly who are more prone to disease, only
account for 12.5%. The 12.3% are in between 0-19 years, 29% in 30-49 years,
and 27% in the 50-69 age group. Fig. 3 shows the gender distribution of
COVID-19 cases across different age groups. Women account for more COVID-19
positive cases as compared to men, independent of age, in the state of Ohio.

Figure 2: COVID-19 cases in Ohio distribution by age group

Figure 3: Monthly median percent change in COVID-19 cases in Ohio

From the preliminary data analysis and visualization, it can be seen that
thousands of new coronavirus cases continue to emerge in the age group 20-29,
most of which represents the college student population. Governor Mike DeWine
also mentioned that the sudden increase in the number of cases is connected to
students returning to college campuses for the fall semester[11].

###  III-C Other Pertinent Data

Noting that the sudden increase in number of COVID-19 cases has been
attributed to the opening of colleges and students returning to campuses, an
impetus was provided to compare the COVID-19 spread in Ohio counties with and
without NCAA affiliated colleges or universities. For this, the data of
colleges including the county Federal Information Processing Standards (FIPS)
code was obtained from the Integrated Postsecondary Education Data System
(IPEDS)[8]. Data from the New York Times COVID-19 data repository[7] contained
COVID-19 case counts by date, county, and the FIPS code. The 2020 county
population was retrieved from the Census Bureau[1]. Lastly, data was obtained
from the National Center for Health Statistics (NCHS)[13] to obtain the Urban
Rural Classification Scheme for counties.

##  IV Methodology

###  IV-A Data set cleaning

There are many existing datasets tracking COVID-19 cases worldwide. Since this
study is specific to Ohio, the Ohio Department of Health (ODH) COVID-19[12]
was selected as the basis of the data used. The data on the ODH website is
also updated regularly, which provided the ability to update the analysis over
time. The dataset contains county, sex, age range, onset date, admission date,
date of death, case count, hospitalized count, and death due to illness count
by county of residence. The subject of interest in this study is case counts
by age range and the monthly increase in case count by age range. A dataframe
was constructed transforming the daily case counts into cumulative case counts
by date and county. Since the fields sex, admission data, date of death,
hospitalized count and death due to illness count were not needed, they were
removed.

In order to make comparisons between counties that contained NCAA colleges and
those that did not, data from the Integrated Postsecondary Education Data
System (IPEDS)[8] database for 2020 was merged by county into the cumulative
case counts by data and county (working) dataframe. The last classification
that was added to the working dataframe is whether the county is classified as
a metro or non-metro county. The National Century for Health Statistics
(NCHS)[13] from the Center for Disease Control (CDC) dataset for the year 2013
was merged to the working dataframe which distinguished the classification of
each Ohio county. The last data source needed to complete the study was the
United States Census Bureau[8]. The 2020 population by county was merged into
the working dataframe.

Before conducting the analysis, the data was trimmed to only include the first
day of each month and the percent monthly change was calculated. Since the
initial months with 0 cases showed a NaN in the percent change column, the NaN
values were changed to 0. An additional column was added to the working
dataframe to calculate the cases per 100k. The average and median percent
change were calculated according to the classification of whether the county
contained an NCAA college or not. The average and median percent change were
also calculated by using four classifications, metro county with and without
an NCAA college and non-metro county with and without an NCAA college.
Outliers were found for the months of April and May of 2020 using the
interquartile ranges and were omitted from the dataframe before the
statistical analysis. No outliers were detected in the average cases per 100k
field, so all data was considered for the analysis.

###  IV-B Visualization

After cleaning and processing the data set, multiple visualizations in forms
of graphs were developed. Variables in the data set like the number of cases
recorded per day, the amount of patients who were hospitalized, and the death
counts due to COVID-19 for different age groups and gender were analyzed.
After which the number of COVID-19 cases for every county was scaled to cases
per thousand.

###  IV-C Graphs

Matplotlib and Seaborn were used to make multiple bar plots graphs to
visualize the results. The following bar plots were constructed:

  1. 1.

Fig. 1 showing the number of cases in Ohio by onset date

  2. 2.

Fig. 2 showing the case count for different age groups

  3. 3.

Fig. 3 showing the median rate of change of COVID-19 cases by month

  4. 4.

Fig. 4 showing the box plot of cases per 100k versus college and metro
groupings

  5. 5.

Fig. 5 showing the average case counts per 100k by metro/non-metro, with NCAA
college/without NCAA college

###  IV-D Normality Test

Understanding the importance of normal distribution is crucial as it is a
fundamental assumption of many statistical approaches. When a statistical
analysis is being conducted, it is vital to verify the most used distribution
in statistical theory and applications. So, validating the assumption of
normality is essential for the analysis[5]. Amongst various normality tests,
Shapiro Wilk, Jaque and Kolmogorov-Smirnov tests were used to check if the
average cases per 100 thousand, average percentage change, and median percent
change are normally distributed to be able to choose between parametric or
non-parametric approach. Shapiro Wilk is among the most widely used of
regression tests, with its value in the range of 0 and 1, which determines the
normality/non-normality status of the data. The SW values contribute immensely
to the P-value when the test is carried out[33]. On the other hand, Jarque and
Bera’s (JB ) test has optimum asymptotic power properties and good finite
sample performance[34]. The JB tests for normality were obtained using the
Lagrange multiplier procedure. Its statistics follow a chi-square distribution
with 2 degrees of freedom. The hypothesis is as follows: The null hypothesis
indicates that the skewness is 0 and kurtosis is 3. The null hypothesis of
normality fails to reject if there are small values of skewness and kurtosis
values close to or equal to 3[33]. In addition to the SW and JB tests,
Kolmogorov-Smirnov, a highly regarded empirical distribution function (EDF)
test used in testing the normality of data, is aimed to establish a comparison
between EDF and the cumulative distribution function (CDF) of normal
distribution[33]. It is recommended that the Shapiro Wilk test is the best to
use whenever there is a sign that the distribution is asymmetric. JB’s
simplicity makes it useful for validating data’s normality state[34].

Data |  Test |  Significance |  P-value |  Test Statistic |  Conclusion
---|---|---|---|---|---
Avg cases per 100k |  Shapiro |  0.05 |  0.00 |  0.90 |  Reject Null
Avg cases per 100k |  Jarque |  0.05 |  0.19 |  3.31 |  Reject Null
Avg cases per 100k |  Kolmogorov-smirnov |  0.05 |  3.18e-158 |  0.90 |  Reject Null
Avg pct change |  Shapiro |  0.05 |  0.00 |  0.81 |  Reject Null
Avg pct change |  Jarque |  0.05 |  0.00 |  11.71 |  Reject Null
Avg pct change |  Kolmogorov-smirnov |  0.05 |  1.07e-10 |  0.50 |  Reject Null
Median pct change |  Shapiro |  0.05 |  0.00 |  0.82 |  Reject Null
Median pct change |  Jarque |  0.05 |  0.00 |  18.11 |  Reject Null
Median pct change |  Kolmogorov-smirnov |  0.05 |  1.07e-10 |  0.50 |  Reject Null
TABLE I: Results of normality tests

It was seen that they are not normally distributed, which infers that non-
parametric tests will be adopted for this analysis.

###  IV-E Non-parametric test

In cases where data are skewed and not distributed normally, the non-
parametric experiment is mostly adopted[14]. We considered two non-parametric
tests known as the Kruskal Wallis test and the Wilcoxon Signed-Rank test since
the data in this case is skewed.

The Kruskal Wallis test was used to compare the continuous outcome in greater
than two independent samples which were the cases per 100 thousands of metro
and non-metro cities while Wilcoxon Signed-Rank was used to compare the paired
samples of median percentage change of NCAA colleges and non- NCAA colleges as
well as the comparison of their average percent changes.

Data | Significance | P-value | Test Statistic | Conclusion
---|---|---|---|---
Avg cases per 100k | 0.05 | 0.80 | 0.99 | Fail to Reject Null
TABLE II: Results of Kruskal Wallis test

In Wilcoxon Signed-Rank test, the null hypothesis H0 says that the average
percent change in cases in counties with colleges is less than or equal the
average percent change in non-college counties [15]. Since our prior theory
was that the rate of change of cases in college counties was lower than it was
in non-college counties, a 1-sided test was performed. With the P-value being
0.249, the Wilcoxon test gives no evidence against the null hypothesis of
higher average percentage change of cases in college counties. However, the
null hypothesis is rejected in the comparison between the median percent
changes and the average cases per 100 thousand with respective p-values of
0.019 and 0.000.

Data | Significance | P-value | Test Statistic | Conclusion
---|---|---|---|---
Avg cases per 100k | 0.05 | 0.00 | 269.00 | Reject Null
Avg pct change | 0.05 | 0.25 | 135.00 | Fail to reject Null
Median pct change | 0.05 | 0.02 | 175.00 | Reject Null
TABLE III: Results of Wilcoxon Signed-Rank tests

Figure 4: Box plot of Average cases per 100k vs College and metro grouping

A further breakdown of the county data was done into metro and non-metro
regions resulting in four groups- metro counties with and without NCAA
colleges, and non-metro counties with and without NCAA colleges. This four
group data was unbalanced because there was an uneven distribution of
observations corresponding to each of the four groups- there were over double
the number of observations for non-metro counties without NCAA colleges as
compared to the metro counties with/without NCAA colleges, and there were
almost four times the number of observations compared to the metro counties
with NCAA colleges. Hence, a Kruskal-Wallis analysis was performed on these
four groupings.

The motivation of this test was to compare the four groups whether they are
statistically different. This comparison yields a p-value of 0.80 which is
obviously greater than the significant 0.05. The conclusion in this case is
that the four groups are the same by failing to reject the null hypothesis.
This implies that the mean of the cases per 100k are the same for the four
groups and it is safe to say that whether the college is in the metro or non-
metro, it does not impact COVID-19 cases.

Figure 5: Average cases per 100k by college and metro grouping

##  V Conclusion

Since the COVID-19 pandemic is still currently ongoing and measures are still
being taken to reduce the spread of the disease, the data set is incomplete
and the analysis is far from final. The analysis this paper addresses is
initial in its findings: In the state of Ohio, the median percent change and
the average cases per 100k in COVID-19 cases through February 1, 2022 is
greater in counties with colleges than those without colleges. The average
percent change in college counties is however less in counties with colleges.
When further subdividing the cases per 100k by month and whether an NCAA
college is present into metro and non-metro counties, a statistical comparison
among the mean of the four groups resulted in the acceptance of the hypothesis
that the means were the same.

A reflection on the mitigation strategies that colleges and universities used
during the global pandemic is crucial for future planning. Infectious diseases
is ranked sixth in the World Economic Forum Global Risks Perception Survey
2021-2022 [28]. It is also noted that 26.4% of the countries surveyed indicate
that infectious disease will become a critical threat to the world in 0-2
years [28].

> Since there are four endemic coronaviruses that circulate globally in
> humans, coronaviruses must have emerged and spread pandemically in the era
> prior to the recognition of viruses as human pathogens. The severe acute
> respiratory syndrome (SARS) coronavirus (SARS-CoV) emerged from an animal
> host, likely a civet cat, in 2002–2003, to cause a near-pandemic before
> disappearing in response to public health control measures. The related
> Middle East respiratory syndrome (MERS) coronavirus (MERS-CoV) emerged into
> humans from dromedary camels in 2012, but has since been transmitted
> inefficiently among humans [30]. COVID-19, recognized in late 2019, is but
> the latest example of an unexpected, novel, and devastating pandemic
> disease. One can conclude from this recent experience that we have entered a
> pandemic era [29] [31].

Certainly, planning for future pandemic events should be top of mind,
particularly since ”as climate change and urbanization progress, pandemics are
likely to occur [32].

###  V-A Limitations

There are many limitations to the analysis performed and many opportunities
for further study:

  * •

The pandemic is currently ongoing and the data is incomplete.

  * •

Reporting structure (which county the student case was reported) for student
cases and number of student cases at each institution was not included.

  * •

An assumption was made that any increase is student cases, regardless of
reporting structure, would increase the amount of community spread of
COVID-19.

  * •

Only institutions that compete in NCAA sports were considered, but differences
between them exist, i.e. small private college, large public university, etc.

  * •

Plans for students returning to campuses and modes of instruction varied
widely[6] and were also not considered in this study.

111Source code available at: Ohio COVID-19 Data Analysis

## References

  * [1] “County Population Totals: 2010-2019”. [Online]. Available: https://www.census.gov/data/datasets/time-series/demo/popest/2010s-counties-total.html. [Accessed: 2020-12-04].
  * [2] “Tracking Coronavirus Cases at U.S. Colleges and Universities”. [Online]. Available: https://www.nytimes.com/interactive/2021/us/college-covid-tracker.html. [Accessed: 2022-02-26].
  * [3] “Guidance for Institutions of Higher Education (IHEs)”. [Online]. Available: https://www.cdc.gov/coronavirus/2019ncov/community/collegesuniversities/considerations.html. [Accessed: 2022-02-26].
  * [4] “College Reopening: The Outlook for In-Person Classes - The New …”. [Online]. Available: https://www.nytimes.com/2020/08/24/us/college
-university-reopening-coronavirus.html. [Accessed: 2020-12-04].
  * [5] “Comparisons of various types of normality tests”, B. W. Yap & C. H. Sim (2011) , Journal of Statistical Computation and Simulation, 81:12, 2141-2155, DOI: 10.1080/00949655.2010.520163
  * [6] “Here’s Our List of Colleges’ Reopening Models”. [Online]. Available: https://www.chronicle.com/article/heres-a-list-of-colleges-plans-for-
reopening-in-the-fall/. [Accessed: 2020-12-04].

  * [7] “Nytimes/covid-19-DATA GitHub”. [Online]. Available: https://github.com/nytimes/covid-19-data/blob/master/live/us-counties.csv. [Accessed: 2020-12-04].
  * [8] “IPEDS Data Center”. [Online]. Available: https://nces.ed.gov/ipeds
/datacenter/login.aspx?gotoReportId=7. [Accessed: 2020-12-04].

  * [9] “Testing, Screening, and Outbreak Response for Institutions of …”. [Online]. Available: https://www.cdc.gov/coronavirus/2019
-ncov/community/colleges-universities/ihe-testing.html. [Accessed: 2020-12-04].
  * [10] “COVID-19 pandemic in Ohio - Wikipedia”. [Online]. Available: https://
en.wikipedia.org/wiki/COVID-19_pandemic_in_Ohio. [Accessed: 2020-12-04].

  * [11] “Governor believes increase in Ohio COVID-19 cases could be …”. [Online]. Available: https://www.cleveland19.com/2020/09/01/governor
-believes-increase-ohio-covid-cases-could-be-linked-return-schools-
colleges/. [Accessed: 2020-12-04].

  * [12] “COVID-19 Dashboard”. [Online]. Available: https://coronavirus.ohio
.gov/wps/portal/gov/covid-19/dashboards. [Accessed: 2020-12-04].

  * [13] “NCHS data”. [Online]. Available: https://www.cdc.gov/nchs/data/data_acces_files/NCHSURCodes2013.xlsx. [Accessed: 2021-10-24].
  * [14] “Non-paramteric test”. [Online]. Available: https://byjus.com/maths/non-parametric-test/. [Accessed: 2022-02-26].
  * [15] “Wilcoxon test”. [Online]. Available: https://www.stat.auckland.ac.nz/w̃ild/ChanceEnc/Ch10.wilcoxon.pdf. [Accessed: 2022-02-26].
  * [16] Chang, Chi-Ning, Hsiang-Yu Chien, and Laura Malagon-Palacios. “College Reopening and Community Spread of COVID-19 in the United States.” Public Health (2022).
  * [17] Oster, Alexandra M., et al. “Transmission dynamics by age group in COVID-19 hotspot counties—United States, April–September 2020.” Morbidity and Mortality Weekly Report 69.41 (2020): 1494.
  * [18] Li, Yang, et al. “Association of university reopening policies with new confirmed COVID-19 cases in the United States.” MedRxiv (2021): 2020-12.
  * [19] Salvatore, Phillip P., et al. “Recent increase in COVID-19 cases reported among adults aged 18–22 years—United States, May 31–September 5, 2020.” Morbidity and Mortality Weekly Report 69.39 (2020): 1419.
  * [20] Lehnig, Chris L., Eyal Oren, and Naveen K. Vaidya. “Effectiveness of alternative semester break schedules on reducing COVID-19 incidence on college campuses.” Scientific Reports 12.1 (2022): 1-8.
  * [21] Mangrum, Daniel, and Paul Niekamp. “JUE Insight: College student travel contributed to local COVID-19 spread.” Journal of Urban Economics (2020): 103311.
  * [22] Leidner, Andrew J., et al. “Opening of large institutions of higher education and county-level COVID-19 incidence—United States, July 6–September 17, 2020.” Morbidity and Mortality Weekly Report 70.1 (2021): 14.
  * [23] Goldhaber, Dan, et al. “To what extent does in-person schooling contribute to the spread of COVID-19.” Evidence from Michigan and Washington 70 (2021).
  * [24] Felson, Jacob, and Amy Adamczyk. “Online or in person? Examining college decisions to reopen during the COVID-19 pandemic in Fall 2020.” Socius 7 (2021): 2378023120988203.
  * [25] Weitz, Rob, Viswa Viswanathan, and David Rosenthal. “TheTrilemma of 2020: Understanding Higher Education’s Fall 2020 Reopening Decision Amidst the COVID Crisis.” American Business Review 24.2 (2021): 3.
  * [26] “H.R.748 - 116th Congress (2019-2020): CARES Act.” Congress.gov, Library of Congress, 27 March 2020, https://www.congress.gov/bill/116th-congress/house-bill/748.
  * [27] Lu, Hannah, et al. “Are college campuses superspreaders? A data-driven modeling study.” Computer Methods in Biomechanics and Biomedical Engineering 24.10 (2021): 1136-1145.
  * [28] “World Economic Forum Global Risks Perception Survey 2021-2022.”. [Online]. Available: https://www.weforum.org/reports/global-risks-report-2022/. [Accessed: 2022-06-12].
  * [29] Morens, David M., and Anthony S. Fauci. “Emerging pandemic diseases: how we got to COVID-19.” Cell 182.5 (2020): 1077-1092.
  * [30] Cui, Jie, Fang Li, and Zheng-Li Shi. “Origin and evolution of pathogenic coronaviruses.” Nature reviews microbiology 17.3 (2019): 181-192.
  * [31] Morens, David M., Peter Daszak, and Jeffery K. Taubenberger. “Escaping Pandora’s box—another novel coronavirus.” New England Journal of Medicine 382.14 (2020): 1293-1295.
  * [32] Meyer, Kenneth L. “Confronting the pandemic superthreat of climate change and urbanization.” Orbis 63 (2019): 565-581.
  * [33] B. W. Yap & C. H. Sim (2011) Comparisons of various types of normality tests, Journal of Statistical Computation and Simulation, 81:12, 2141-2155, DOI: 10.1080/00949655.2010.520163.
  * [34] Jarque, C. M., & Bera, A. K. (1987). A Test for Normality of Observations and Regression Residuals. International Statistical Review / Revue Internationale de Statistique, 55(2), 163–172. https://doi.org/10.2307/1403192.