temporality of the behaviour indicative of the chronicity of behaviour and
trends and fashions (e.g. the greater use of LSD in the 1960s and 1970s, the
rise of ecstasy in the 1980s, some persons being one-off experimenters with
recreational drugs, and others using recreational substances on a daily
basis).

We defined substance use in terms of behaviour rather than legality, as
legislation in the field is variable. Our data were gathered before ‘ legal
highs’ emerged as a health concern [96] so we did not differentiate, for
example, synthetic cannabinoids and cathinone-based stimulants; these
substances have been since widely made illegal. We were nevertheless able to
accurately classify users of these substances (reciprocally, our data were
gathered before cannabis decriminalisation in parts of North America, but
again, we were able to accurately classify cannabis users). We included
control participants who had never used these substances, those who had used
them in the distant past, up to and including persons who had used the drug in
the past day, avoiding the procrustean data-gathering and classifying methods
which may occlude an accurate picture of drug use behaviour and risk [97].
Such rich data and the complex methods used for analysis necessitated a large
and substantial sample.

The study was atheoretical regarding the morality of the behaviour, and did
not medicalise or pathologise participants, optimising engagement by persons
with heterogeneous drug-use histories. Our study used a rigorous range of
data-mining methods beyond those typically used in studies examining the
association of drug use and personality in the psychological and psychiatric
literature, revealing that decision tree methods were most commonly effective
for classifying drug users. We found that high N, low A, and low C are the
most common personality correlates of drug use, these traits being sometimes
seen in combination as an indication of higher-order stability and behavioural
conformity, and, inverted, are associated with externalisation of distress
[98, 99, 100].

Low stability is also a marker of negative urgency [47] whereby persons act
rashly when distressed. Our research points to the importance of individuals
acquiring emotional self-management skills anteceding distress as a means to
reduce self-medicating drug-using behaviour, and the risk to health that
injudicious or chronic drug use may cause.

## Supporting Information

##### S1 Appendix.

Mean for groups of users and non-users. In this section we present mean
T-$score_{sample}$ for groups of users and non-users for decade, year, month,
and week based user definitions respectively. Column $p$-value assesses the
significance of differences of mean scores for groups of users and non-users:
it is the probability of observing by chance the same or greater differences
for mean scores if both groups have the same mean. Rows ‘#’ contain number of
users and non-users for the drugs.

Table 22:  Mean T-$score_{sample}$ and 95% CI for it for groups of users and non-users with decade based definition of users Factor | Users | Non-users |  $p$-value
---|---|---|---
| Mean T-score | 95% CI for mean | Mean T-score | 95% CI for mean |
Alcohol
# | 1817 | 68 |
N | 50.13 | 49.67, 50.59 | 48.19 | 45.77, 50.61 | 0.116
E | 50.06 | 49.60, 50.52 | 50.04 | 47.61, 52.42 | 0.988
O | 50.04 | 49.58, 50.51 | 48.81 | 46.45, 51.17 | 0.318
A | 49.93 | 49.47, 50.39 | 52.51 | 50.26, 54.77 | 0.036
C | 49.94 | 49.48, 50.40 | 53.31 | 51.05, 55.56 | 0.006
Amphetamines
# | 679 | 1206 |
N | 51.71 | 50.95, 52.46 | 49.14 | 48.58, 49.69 | 0.001
E | 49.71 | 48.89,50.53 | 50.26 | 49.72, 50.80 | 0.251
O | 53.05 | 52.34, 53.77 | 48.28 | 47.72, 48.84 | 0.001
A | 48.39 | 47.60, 49.18 | 50.94 | 50.39, 51.48 | 0.001
C | 47.04 | 46.29, 47.80 | 51.76 | 51.22, 52.30 | 0.001
Amyl nitrite
# | 370 | 1515 |
N | 50.78 | 49.78, 51.79 | 49.89 | 49.37, 50.39 | 0.122
E | 50.97 | 49.95, 51.99 | 49.84 | 49.33, 50.35 | 0.052
O | 51.45 | 50.47, 52.43 | 49.65 | 49.14, 50.15 | 0.002
A | 48.69 | 47.65, 49.72 | 50.35 | 49.84, 50.85 | 0.004
C | 48.08 | 46.10, 49.07 | 50.54 | 50.04, 51.05 | 0.001
Benzodiazepines
# | 769 | 1116 |
N | 52.83 | 52.12, 53.54 | 48.15 | 47.59, 48.72 | 0.001
E | 49.07 | 48.31, 49.83 | 50.74 | 50.19, 51.30 | 0.001
O | 52.66 | 51.97, 53.34 | 48.17 | 47.59, 48.75 | 0.001
A | 48.28 | 47.53, 49.02 | 51.22 | 50.67, 51.78 | 0.001
C | 47.70 | 46.98, 48.41 | 51.69 | 51.12, 52.25 | 0.001
Cannabis
# | 1265 | 620 |
N | 51.08 | 50.52, 51.65 | 47.98 | 47.25, 48.71 | 0.001
E | 49.75 | 49.17, 50.33 | 50.70 | 49.98, 51.41 | 0.053
O | 52.48 | 51.96, 52.99 | 44.95 | 44.20,45.70 | 0.001
A | 48.84 | 48.28,49.40 | 52.42 | 51.69,53.15 | 0.001
C | 48.15 | 47.60,48.70 | 53.92 | 53.27,54.65 | 0.001
Chocolate
# | 1850 | 35 |
N | 50.06 | 49.60, 50.51 | 50.29 | 46.36, 54.21 | 0.894
E | 50.05 | 49.59, 50.51 | 50.80 | 47.71, 53.89 | 0.660
O | 50.05 | 49.59, 50.51 | 47.37 | 44.42, 50.32 | 0.117
A | 50.05 | 49.59, 50.50 | 48.66 | 44.61, 52.70 | 0.416
C | 50.03 | 49.58, 50.49 | 51.57 | 47.87, 55.27 | 0.366
Cocaine
# | 687 | 1198 |
N | 51.85 | 51.09, 52.60 | 49.04 | 48.48, 49.59 | 0.001
E | 50.33 | 49.55, 51.12 | 49.91 | 49.35, 50.46 | 0.374
O | 52.60 | 51.89, 53.30 | 48.51 | 47.94, 49.08 | 0.001
A | 47.71 | 46.93, 48.50 | 51.34 | 50.81,51.88 | 0.001
C | 47.49 | 46.76, 48.22 | 51.53 | 50.98, 52.09 | 0.001
Caffeine
# | 1848 | 37 |
N | 50.06 | 49.61, 50.52 | 50.08 | 46.59,53.57 | 0.991
E | 50.13 | 49.67, 50.59 | 46.76 | 43.82, 49.69 | 0.043
O | 50.11 | 49.65, 50.57 | 44.59 | 41.67,47.52 | 0.001
A | 49.99 | 49.54, 50.45 | 51.11 | 47.71, 54.51 | 0.504
C | 49.99 | 49.53, 50.44 | 53.59 | 50.79, 56.40 | 0.029
Crack
# | 191 | 1694 |
N | 53.08 | 51.64, 54.52 | 49.72 | 49.25, 50.20 | 0.001
E | 48.80 | 47.33, 50.27 | 50.20 | 49.73, 50.68 | 0.066
O | 52.91 | 51.60, 54.21 | 49.67 | 49.19, 50.15 | 0.001
A | 47.02 | 45.46, 48.58 | 50.36 | 49.89, 50.83 | 0.001
C | 46.19 | 44.70, 47.68 | 50.50 | 50.03, 50.96 | 0.001
Ecstasy
# | 751 | 1134 |
N | 51.30 | 50.58, 52.02 | 49.24 | 48.67, 49.82 | 0.001
E | 50.56 | 49.80, 51.31 | 49.73 | 49.17, 50.30 | 0.081
O | 53.62 | 52.96, 54.28 | 47.60 | 47.03, 48.17 | 0.001
A | 48.49 | 47.75, 49.23 | 51.03 | 50.47, 51.60 | 0.001
C | 47.30 | 46.59, 48.00 | 51.89 | 51.33, 52.45 | 0.001
Heroin
# | 212 | 1673 |
N | 54.60 | 53.29, 55.92 | 49.49 | 49.01, 49.96 | 0.001
E | 48.42 | 46.94, 49.90 | 50.27 | 49.80, 50.74 | 0.011
O | 54.25 | 53.04, 55.47 | 49.46 | 48.98, 49.94 | 0.001
A | 45.53 | 44.00, 47.06 | 50.59 | 50.12, 51.05 | 0.001
C | 45.91 | 44.55, 47.26 | 50.59 | 50.11, 51.06 | 0.001
Ketamine
# | 350 | 1535 |
N | 51.42 | 50.40, 52.43 | 49.75 | 49.25, 50.26 | 0.005
E | 50.36 | 49.23, 51.48 | 49.99 | 49.50, 50.49 | 0.542
O | 53.87 | 52.90, 54.84 | 49.12 | 48.62, 49.62 | 0.001
A | 47.80 | 46.67, 48.94 | 50.53 | 50.04, 51.01 | 0.001
C | 46.86 | 45.81, 47.92 | 50.79 | 50.30, 51.28 | 0.001
Legal highs
# | 762 | 1123 |
N | 51.49 | 50.77, 52.22 | 49.09 | 48.52, 49.66 | 0.001
E | 49.71 | 48.94, 50.47 | 50.30 | 49.75, 50.86 | 0.206
O | 54.30 | 53.67, 54.92 | 47.08 | 46.51, 47.65 | 0.001
A | 48.61 | 47.86, 49.36 | 50.98 | 50.42, 51.53 | 0.001
C | 46.98 | 46.27, 47.72 | 52.14 | 51.60, 52.68 | 0.001
LSD
# | 557 | 1328 |
N | 50.87 | 50.05, 51.69 | 49.72 | 49.18, 50.26 | 0.023
E | 50.07 | 49.16, 50.98 | 50.06 | 49.54, 50.58 | 0.986
O | 55.25 | 54.54, 55.96 | 47.80 | 47.27, 48.33 | 0.001
A | 48.44 | 47.57, 49.31 | 50.68 | 50.16, 51.21 | 0.001
C | 47.54 | 46.71, 48.36 | 51.12 | 50.59, 51.65 | 0.001
Methadone
# | 417 | 1468 |
N | 53.41 | 52.47, 54.35 | 49.11 | 48.61, 49.62 | 0.001
E | 47.97 | 46.88, 49.05 | 50.66 | 50.17, 51.15 | 0.001
O | 53.77 | 52.86, 54.69 | 48.93 | 48.42, 49.44 | 0.001
A | 47.07 | 46.03, 48.10 | 50.86 | 50.37, 51.35 | 0.001
C | 46.21 | 45.22, 47.20 | 51.15 | 50.66, 51.64 | 0.001
Magic Mushrooms
# | 694 | 1191 |
N | 50.73 | 49.98,51.47 | 49.67 | 49.10, 50.24 | 0.027
E | 50.15 | 49.35,50.96 | 50.01 | 49.47, 50.55 | 0.765
O | 54.36 | 53.71,55.01 | 47.46 | 46.90, 48.02 | 0.001
A | 48.55 | 47.78,49.32 | 50.88 | 50.33, 51.43 | 0.001
C | 47.54 | 46.81,48.27 | 51.53 | 50.97, 52.08 | 0.001
Nicotine
# | 1264 | 621 |
N | 50.97 | 50.41, 51.52 | 48.22 | 47.45, 48.99 | 0.001
E | 49.98 | 49.41, 50.54 | 50.24 | 49.48, 50.99 | 0.599
O | 51.47 | 50.92, 52.01 | 47.01 | 46.26, 47.77 | 0.001
A | 49.20 | 48.65, 49.75 | 51.69 | 50.92, 52.46 | 0.001
C | 48.64 | 48.08, 49.20 | 52.95 | 52.24, 53.67 | 0.001
VSA
# | 230 | 1655 |
N | 52.88 | 51.57, 54.20 | 49.67 | 49.19,50.15 | 0.001
E | 48.96 | 47.45, 50.47 | 50.22 | 49.74,50.69 | 0.075
O | 54.20 | 53.00, 55.41 | 49.42 | 48.93,49.90 | 0.001
A | 47.30 | 45.92, 48.68 | 50.40 | 49.92,50.87 | 0.001
C | 45.22 | 43.88, 46.56 | 50.73 | 50.26,51.20 | 0.001

Table 23:  Mean T-$score_{sample}$ and 95% CI for it for groups of users and non-users with year based definition Factor | Users | Non-users |  $p$-value
---|---|---|---
| Mean T-score | 95% CI for mean | Mean T-score | 95% CI for mean |
Alcohol
# | 1749 | 136 |
N | 49.96 | 49.49, 50.43 | 50.55 | 48.87, 52.24 | 0.5012
E | 50.12 | 49.64, 50.59 | 48.52 | 46.89, 50.14 | 0.0642
O | 50.07 | 49.60, 50.54 | 49.11 | 47.46, 50.76 | 0.2712
A | 49.95 | 49.48, 50.42 | 50.70 | 49.03, 52.37 | 0.3921
C | 49.94 | 49.47, 50.41 | 50.73 | 49.03, 52.44 | 0.3782
Amphetamines
# | 436 | 1449 |
N | 52.40 | 51.43, 53.36 | 49.28 | 48.77, 49.78 | 0.0001
E | 49.48 | 48.45, 50.50 | 50.16 | 49.66, 50.66 | 0.2431
O | 53.83 | 52.92, 54.74 | 48.85 | 48.34, 49.35 | 0.0001
A | 47.47 | 46.45, 48.48 | 50.76 | 50.27, 51.26 | 0.0001
C | 45.91 | 44.94, 46.88 | 51.23 | 50.74, 51.72 | 0.0001
Amyl nitrite
# | 133 | 1752 |
N | 51.57 | 49.85, 53.29 | 49.88 | 49.41, 50.35 | 0.0633
E | 50.25 | 48.44, 52.06 | 49.98 | 49.51, 50.45 | 0.7777
O | 51.97 | 50.36, 53.59 | 49.85 | 49.38, 50.32 | 0.0136
A | 46.35 | 44.64, 48.07 | 50.28 | 49.81, 50.74 | 0.0001
C | 46.92 | 45.25, 48.60 | 50.23 | 49.77, 50.70 | 0.0002
Benzodiazepines
# | 535 | 1350 |
N | 53.69 | 52.84, 54.54 | 48.54 | 48.03, 49.05 | 0.0001
E | 48.91 | 47.97, 49.85 | 50.43 | 49.92, 50.94 | 0.0053
O | 53.00 | 52.18, 53.83 | 48.81 | 48.28, 49.34 | 0.0001
A | 47.55 | 46.64, 48.46 | 50.97 | 50.46, 51.48 | 0.0001
C | 47.06 | 46.19, 47.94 | 51.16 | 50.65, 51.68 | 0.0001
Cannabis
# | 999 | 886 |
N | 51.10 | 50.46, 51.74 | 48.76 | 48.14, 49.38 | 0.0001
E | 49.73 | 49.08, 50.39 | 50.30 | 49.69, 50.91 | 0.2177
O | 53.68 | 53.13, 54.23 | 45.85 | 45.22, 46.48 | 0.0001
A | 48.77 | 48.13, 49.40 | 51.39 | 50.76, 52.02 | 0.0001
C | 47.32 | 46.70, 47.95 | 53.02 | 52.42, 53.62 | 0.0001
Chocolate
# | 1840 | 45 |
N | 50.01 | 49.56, 50.47 | 49.45 | 46.04, 52.86 | 0.7445
E | 49.99 | 49.53, 50.45 | 50.49 | 47.75, 53.24 | 0.7156
O | 50.04 | 49.58, 50.50 | 48.27 | 45.59, 50.94 | 0.1945
A | 50.02 | 49.56, 50.48 | 49.17 | 45.72, 52.62 | 0.6259
C | 49.97 | 49.51, 50.42 | 51.41 | 47.81, 55.01 | 0.4254
Cocaine
# | 417 | 1468 |
N | 52.16 | 51.19, 53.13 | 49.39 | 48.88, 49.89 | 0.0001
E | 50.93 | 49.88, 51.98 | 49.74 | 49.24, 50.23 | 0.0439
O | 52.79 | 51.87, 53.71 | 49.21 | 48.70, 49.72 | 0.0001
A | 46.84 | 45.81, 47.86 | 50.90 | 50.41, 51.39 | 0.0001
C | 46.81 | 45.86, 47.75 | 50.91 | 50.40, 51.41 | 0.0001
Caffeine
# | 1824 | 61 |
N | 49.99 | 49.53, 50.45 | 50.36 | 47.75, 52.96 | 0.7827
E | 50.10 | 49.64, 50.56 | 46.92 | 44.34, 49.50 | 0.0181
O | 50.10 | 49.64, 50.56 | 47.07 | 44.45, 49.69 | 0.0265
A | 49.96 | 49.50, 50.42 | 51.15 | 48.61, 53.69 | 0.3605
C | 49.91 | 49.45, 50.37 | 52.79 | 50.14, 55.43 | 0.0362
Crack
# | 79 | 1806 |
N | 54.06 | 51.78, 56.35 | 49.82 | 49.36, 50.28 | 0.0005
E | 49.24 | 47.20, 51.28 | 50.03 | 49.57, 50.50 | 0.4548
O | 52.90 | 50.72, 55.08 | 49.87 | 49.41, 50.33 | 0.0084
A | 46.65 | 44.17, 49.12 | 50.15 | 49.69, 50.61 | 0.0070
C | 45.08 | 42.92, 47.25 | 50.22 | 49.76, 50.67 | 0.0001
Ecstasy
# | 517 | 1368 |
N | 50.71 | 49.82, 51.61 | 49.73 | 49.21, 50.25 | 0.0624
E | 51.51 | 50.60, 52.42 | 49.43 | 48.91, 49.95 | 0.0001
O | 54.10 | 53.32, 54.89 | 48.45 | 47.92, 48.97 | 0.0001
A | 48.55 | 47.64, 49.45 | 50.55 | 50.03, 51.07 | 0.0002
C | 47.02 | 46.17, 47.87 | 51.13 | 50.61, 51.65 | 0.0001
Heroin
# | 118 | 1767 |
N | 55.37 | 53.63, 57.12 | 49.64 | 49.18, 50.10 | 0.0001
E | 47.68 | 45.68, 49.69 | 50.15 | 49.69, 50.62 | 0.0192
O | 53.45 | 51.80, 55.11 | 49.77 | 49.30, 50.24 | 0.0001
A | 44.37 | 42.25, 46.50 | 50.38 | 49.92, 50.83 | 0.0001
C | 45.45 | 43.52, 47.39 | 50.30 | 49.84, 50.77 | 0.0001
Ketamine
# | 208 | 1677 |
N | 51.52 | 50.20, 52.84 | 49.81 | 49.33, 50.29 | 0.0175
E | 51.02 | 49.48, 52.57 | 49.87 | 49.40, 50.34 | 0.1619
O | 54.18 | 52.88, 55.49 | 49.48 | 49.00, 49.96 | 0.0001
A | 47.90 | 46.42, 49.38 | 50.26 | 49.79, 50.73 | 0.0030
C | 46.34 | 44.91, 47.76 | 50.45 | 49.98, 50.93 | 0.0001
Legal highs
# | 564 | 1321 |
N | 51.27 | 50.41, 52.12 | 49.46 | 48.93, 49.99 | 0.0004
E | 50.02 | 49.11, 50.92 | 49.99 | 49.48, 50.51 | 0.9659
O | 54.49 | 53.76, 55.22 | 48.08 | 47.55, 48.62 | 0.0001
A | 48.09 | 47.23, 48.96 | 50.81 | 50.29, 51.34 | 0.0001
C | 46.56 | 45.71, 47.41 | 51.47 | 50.95, 51.98 | 0.0001
LSD
# | 380 | 1505 |
N | 49.98 | 48.97, 50.99 | 50.00 | 49.50, 50.51 | 0.9691
E | 50.72 | 49.63, 51.81 | 49.82 | 49.32, 50.31 | 0.1403
O | 56.29 | 55.48, 57.11 | 48.41 | 47.91, 48.91 | 0.0001
A | 49.05 | 47.98, 50.12 | 50.24 | 49.74, 50.74 | 0.0484
C | 47.74 | 46.71, 48.77 | 50.57 | 50.07, 51.07 | 0.0001
Methadone
# | 320 | 1565 |
N | 53.74 | 52.65, 54.84 | 49.23 | 48.75, 49.72 | 0.0001
E | 47.75 | 46.46, 49.03 | 50.46 | 49.99, 50.94 | 0.0001
O | 53.81 | 52.76, 54.86 | 49.22 | 48.73, 49.71 | 0.0001
A | 46.53 | 45.33, 47.73 | 50.71 | 50.23, 51.19 | 0.0001
C | 46.01 | 44.87, 47.15 | 50.82 | 50.33, 51.30 | 0.0001
Magic Mushrooms
# | 434 | 1451 |
N | 50.33 | 49.40, 51.26 | 49.90 | 49.38, 50.42 | 0.4311
E | 50.71 | 49.66, 51.76 | 49.79 | 49.29, 50.28 | 0.1179
O | 55.72 | 54.96, 56.49 | 48.29 | 47.78, 48.80 | 0.0001
A | 48.51 | 47.48, 49.53 | 50.45 | 49.95, 50.95 | 0.0009
C | 47.30 | 46.36, 48.24 | 50.81 | 50.30, 51.32 | 0.0001
Nicotine
# | 1060 | 825 |
N | 51.16 | 50.55, 51.76 | 48.51 | 47.85, 49.18 | 0.0001
E | 49.80 | 49.17, 50.42 | 50.26 | 49.61, 50.91 | 0.3094
O | 51.91 | 51.31, 52.51 | 47.55 | 46.89, 48.20 | 0.0001
A | 49.04 | 48.42, 49.65 | 51.24 | 50.58, 51.90 | 0.0001
C | 47.98 | 47.38, 48.57 | 52.60 | 51.95, 53.25 | 0.0001
VSA
# | 95 | 1790 |
N | 53.81 | 51.67, 55.95 | 49.80 | 49.34, 50.26 | 0.0004
E | 49.55 | 47.32, 51.79 | 50.02 | 49.56, 50.49 | 0.6832
O | 53.59 | 51.58, 55.60 | 49.81 | 49.35, 50.27 | 0.0004
A | 47.48 | 45.35, 49.61 | 50.13 | 49.67, 50.60 | 0.0174
C | 45.31 | 43.18, 47.44 | 50.25 | 49.79, 50.71 | 0.0001
Table 24:  Mean T-$score_{sample}$ and 95% CI for it for groups of users and non-users with month based problem Factor | Users | Non-users |  $p$-value
---|---|---|---
| Mean T-score | 95% CI for mean | Mean T-score | 95% CI for mean |
Alcohol
# | 1551 | 334 |
N | 49.85 | 49.35, 50.35 | 50.69 | 49.61, 51.78 | 0.1669
E | 50.62 | 50.12, 51.11 | 47.14 | 46.11, 48.17 | 0.0001
O | 50.22 | 49.73, 50.72 | 48.96 | 47.87, 50.06 | 0.0401
A | 50.10 | 49.60, 50.60 | 49.55 | 48.50, 50.60 | 0.3575
C | 50.15 | 49.66, 50.65 | 49.28 | 48.22, 50.35 | 0.1460
Amphetamines
# | 238 | 1647 |
N | 52.78 | 51.49, 54.07 | 49.60 | 49.12, 50.08 | 0.0001
E | 49.07 | 47.63, 50.51 | 50.13 | 49.66, 50.61 | 0.1665
O | 52.94 | 51.69, 54.20 | 49.57 | 49.09, 50.06 | 0.0001
A | 46.57 | 45.21, 47.92 | 50.50 | 50.02, 50.97 | 0.0001
C | 45.06 | 43.75, 46.37 | 50.71 | 50.24, 51.19 | 0.0001
Amyl nitrite
# | 41 | 1844 |
N | 49.36 | 46.12, 52.61 | 50.01 | 49.56, 50.47 | 0.6911
E | 49.83 | 46.38, 53.28 | 50.00 | 49.55, 50.46 | 0.9218
O | 50.40 | 47.06, 53.74 | 49.99 | 49.53, 50.45 | 0.8083
A | 45.43 | 41.87, 49.00 | 50.10 | 49.65, 50.56 | 0.0122
C | 47.31 | 44.60, 50.01 | 50.06 | 49.60, 50.52 | 0.0490
Benzodiazepines
# | 299 | 1586 |
N | 55.26 | 54.11, 56.41 | 49.01 | 48.53, 49.48 | 0.0001
E | 48.06 | 46.79, 49.33 | 50.37 | 49.89, 50.84 | 0.0010
O | 52.68 | 51.59, 53.77 | 49.49 | 49.00, 49.99 | 0.0001
A | 46.72 | 45.47, 47.97 | 50.62 | 50.14, 51.10 | 0.0001
C | 46.54 | 45.40, 47.69 | 50.65 | 50.17, 51.14 | 0.0001
Cannabis
# | 788 | 1097 |
N | 50.72 | 49.99, 51.45 | 49.48 | 48.91, 50.06 | 0.7142
E | 50.06 | 49.31, 50.80 | 49.96 | 49.40, 50.52 | 0.5738
O | 54.34 | 53.74, 54.95 | 46.88 | 46.30, 47.46 | 0.3233
A | 48.66 | 47.93, 49.38 | 50.97 | 50.39, 51.54 | 0.5477
C | 47.26 | 46.55, 47.97 | 51.97 | 51.41, 52.53 | 0.3800
Chocolate
# | 1786 | 99 |
N | 49.98 | 49.52, 50.44 | 50.40 | 48.19, 52.60 | 0.0002
E | 50.03 | 49.57, 50.49 | 49.43 | 47.38, 51.49 | 0.1295
O | 50.05 | 49.58, 50.52 | 49.13 | 47.36, 50.90 | 0.0008
A | 50.04 | 49.57, 50.50 | 49.36 | 47.19, 51.53 | 0.0001
C | 49.95 | 49.49, 50.41 | 50.90 | 48.82, 52.97 | 0.0002
Cocaine
# | 159 | 1726 |
N | 52.87 | 51.29, 54.46 | 49.74 | 49.27, 50.21 | 0.3787
E | 51.34 | 49.50, 53.18 | 49.88 | 49.41, 50.34 | 0.0078
O | 52.53 | 51.00, 54.06 | 49.77 | 49.29, 50.24 | 0.0560
A | 45.75 | 44.05, 47.46 | 50.39 | 49.93, 50.86 | 0.7537
C | 47.23 | 45.71, 48.75 | 50.25 | 49.78, 50.73 | 0.4728
Caffeine
# | 1764 | 121 |
N | 50.05 | 49.59, 50.52 | 49.21 | 47.37, 51.05 | 0.0092
E | 50.16 | 49.69, 50.63 | 47.68 | 45.93, 49.43 | 0.8390
O | 50.12 | 49.65, 50.58 | 48.29 | 46.47, 50.11 | 0.0001
A | 49.98 | 49.51, 50.45 | 50.27 | 48.50, 52.04 | 0.0001
C | 49.95 | 49.49, 50.42 | 50.67 | 48.77, 52.56 | 0.0001
Crack
# | 20 | 1865 |
N | 57.86 | 52.26, 63.46 | 49.92 | 49.46, 50.37 | 0.1931
E | 45.97 | 41.30, 50.64 | 50.04 | 49.59, 50.50 | 0.0852
O | 50.89 | 47.21, 54.57 | 49.99 | 49.54, 50.45 | 0.6165
A | 42.98 | 36.34, 49.62 | 50.08 | 49.62, 50.53 | 0.0380
C | 45.14 | 40.37, 49.91 | 50.05 | 49.60, 50.51 | 0.0448
Ecstasy
# | 240 | 1645 |
N | 49.53 | 48.18, 50.89 | 50.07 | 49.59, 50.55 | 0.4649
E | 52.24 | 50.83, 53.65 | 49.67 | 49.20, 50.15 | 0.0008
O | 54.41 | 53.23, 55.59 | 49.36 | 48.88, 49.84 | 0.0001
A | 48.10 | 46.75, 49.45 | 50.28 | 49.80, 50.76 | 0.0029
C | 47.27 | 45.96, 48.59 | 50.40 | 49.92, 50.88 | 0.0001
Heroin
# | 53 | 1832 |
N | 56.70 | 54.05, 59.34 | 49.81 | 49.35, 50.26 | 0.0001
E | 45.58 | 42.06, 49.10 | 50.13 | 49.67, 50.58 | 0.0130
O | 52.48 | 49.63, 55.34 | 49.93 | 49.47, 50.39 | 0.0823
A | 42.18 | 39.00, 45.35 | 50.23 | 49.77, 50.68 | 0.0001
C | 43.36 | 40.35, 46.37 | 50.19 | 49.74, 50.65 | 0.0001
Ketamine
# | 79 | 1806 |
N | 51.29 | 49.14, 53.45 | 49.94 | 49.48, 50.41 | 0.2270
E | 49.62 | 46.74, 52.49 | 50.02 | 49.56, 50.47 | 0.7851
O | 54.79 | 52.59, 56.98 | 49.79 | 49.33, 50.25 | 0.0001
A | 46.90 | 44.15, 49.66 | 50.14 | 49.68, 50.59 | 0.0237
C | 45.03 | 42.50, 47.56 | 50.22 | 49.76, 50.67 | 0.0001
Legal highs
# | 241 | 1644 |
N | 52.02 | 50.68, 53.36 | 49.70 | 49.23, 50.18 | 0.0015
E | 49.10 | 47.59, 50.61 | 50.13 | 49.66, 50.60 | 0.2002
O | 54.37 | 53.22, 55.53 | 49.36 | 48.88, 49.84 | 0.0001
A | 46.83 | 45.50, 48.16 | 50.46 | 49.99, 50.94 | 0.0001
C | 45.30 | 44.01, 46.60 | 50.69 | 50.21, 51.16 | 0.0001
LSD
# | 166 | 1719 |
N | 50.55 | 48.97, 52.12 | 49.95 | 49.48, 50.42 | 0.4717
E | 51.28 | 49.53, 53.04 | 49.88 | 49.41, 50.34 | 0.1279
O | 57.28 | 56.16, 58.41 | 49.30 | 48.83, 49.77 | 0.0001
A | 48.92 | 47.35, 50.48 | 50.10 | 49.63, 50.58 | 0.1533
C | 47.10 | 45.59, 48.60 | 50.28 | 49.81, 50.75 | 0.0001
Methadone
# | 171 | 1714 |
N | 54.53 | 53.00, 56.06 | 49.55 | 49.08, 50.02 | 0.0001
E | 46.86 | 45.07, 48.65 | 50.31 | 49.85, 50.78 | 0.0003
O | 52.89 | 51.37, 54.40 | 49.71 | 49.24, 50.18 | 0.0001
A | 46.18 | 44.50, 47.87 | 50.38 | 49.92, 50.85 | 0.0001
C | 45.44 | 43.83, 47.05 | 50.45 | 49.99, 50.92 | 0.0001
Magic Mushrooms
# | 159 | 1726 |
N | 49.91 | 48.41, 51.42 | 50.01 | 49.53, 50.48 | 0.9064
E | 50.31 | 48.50, 52.12 | 49.97 | 49.51, 50.44 | 0.7205
O | 56.92 | 55.77, 58.07 | 49.36 | 48.89, 49.83 | 0.0001
A | 48.57 | 46.94, 50.19 | 50.13 | 49.66, 50.60 | 0.0699
C | 46.85 | 45.25, 48.45 | 50.29 | 49.82, 50.76 | 0.0001
Nicotine
# | 875 | 1010 |
N | 51.11 | 50.43, 51.79 | 49.04 | 48.44, 49.64 | 0.0001
E | 49.98 | 49.29, 50.66 | 50.02 | 49.42, 50.62 | 0.9237
O | 51.86 | 51.19, 52.52 | 48.39 | 47.79, 48.99 | 0.0001
A | 49.02 | 48.34, 49.71 | 50.85 | 50.25, 51.44 | 0.0001
C | 47.69 | 47.03, 48.34 | 52.01 | 51.41, 52.60 | 0.0001
VSA
# | 34 | 1851 |
N | 51.34 | 47.48, 55.20 | 49.98 | 49.52, 50.43 | 0.4793
E | 51.80 | 48.00, 55.60 | 49.97 | 49.51, 50.42 | 0.3378
O | 54.65 | 51.65, 57.66 | 49.91 | 49.46, 50.37 | 0.0032
A | 45.91 | 42.12, 49.71 | 50.08 | 49.62, 50.53 | 0.0336
C | 47.22 | 43.63, 50.81 | 50.05 | 49.60, 50.51 | 0.1209
Table 25:  Mean T-$score_{sample}$ and 95% CI for it for groups of users and non-users with week based definition Factor | Users | Non-users |  $p$-value
---|---|---|---
| Mean T-score | 95% CI for mean | Mean T-score | 95% CI for mean |
Alcohol
# | 1264 | 621 |
N | 49.82 | 49.28, 50.36 | 50.37 | 49.56, 51.19 | 0.2673
E | 50.90 | 50.35, 51.44 | 48.17 | 47.38, 48.97 | 0.0001
O | 50.08 | 49.54, 50.63 | 49.83 | 49.03, 50.62 | 0.6023
A | 50.05 | 49.50, 50.60 | 49.89 | 49.09, 50.69 | 0.7454
C | 50.19 | 49.64, 50.74 | 49.61 | 48.81, 50.42 | 0.2441
Amphetamines
# | 163 | 1722 |
N | 52.86 | 51.27, 54.45 | 49.73 | 49.26, 50.20 | 0.0003
E | 48.50 | 46.78, 50.21 | 50.14 | 49.68, 50.61 | 0.0695
O | 52.87 | 51.27, 54.47 | 49.73 | 49.26, 50.20 | 0.0003
A | 46.77 | 45.09, 48.44 | 50.31 | 49.84, 50.77 | 0.0001
C | 45.29 | 43.72, 46.85 | 50.45 | 49.98, 50.91 | 0.0001
Amyl nitrite
# | 17 | 1868 |
N | 49.64 | 43.98, 55.29 | 50.00 | 49.55, 50.46 | 0.8922
E | 45.85 | 39.05, 52.64 | 50.04 | 49.59, 50.49 | 0.2106
O | 49.02 | 43.02, 55.02 | 50.01 | 49.56, 50.46 | 0.7320
A | 44.36 | 38.95, 49.77 | 50.05 | 49.60, 50.50 | 0.0409
C | 44.81 | 39.74, 49.89 | 50.05 | 49.59, 50.50 | 0.0447
Benzodiazepines
# | 179 | 1706 |
N | 56.56 | 55.08, 58.05 | 49.31 | 48.85, 49.77 | 0.0001
E | 46.15 | 44.61, 47.70 | 50.40 | 49.93, 50.87 | 0.0001
O | 52.18 | 50.72, 53.64 | 49.77 | 49.30, 50.25 | 0.0022
A | 46.57 | 44.94, 48.21 | 50.36 | 49.89, 50.83 | 0.0001
C | 46.20 | 44.75, 47.65 | 50.40 | 49.93, 50.87 | 0.0001
Cannabis
# | 648 | 1237 |
N | 50.62 | 49.82, 51.42 | 49.68 | 49.13, 50.22 | 0.0555
E | 50.17 | 49.36, 50.97 | 49.91 | 49.37, 50.46 | 0.6056
O | 54.70 | 54.05, 55.35 | 47.54 | 46.99, 48.09 | 0.0001
A | 48.78 | 48.00, 49.56 | 50.64 | 50.09, 51.19 | 0.0001
C | 47.45 | 46.67, 48.23 | 51.34 | 50.80, 51.88 | 0.0001
Chocolate
# | 1490 | 395 |
N | 49.95 | 49.44, 50.45 | 50.20 | 49.18, 51.23 | 0.6601
E | 50.22 | 49.72, 50.73 | 49.16 | 48.14, 50.17 | 0.0648
O | 49.89 | 49.37, 50.40 | 50.42 | 49.48, 51.36 | 0.3270
A | 50.20 | 49.70, 50.70 | 49.23 | 48.19, 50.27 | 0.0988
C | 50.13 | 49.62, 50.63 | 49.51 | 48.50, 50.53 | 0.2864
Cocaine
# | 60 | 1825 |
N | 53.24 | 50.50, 55.99 | 49.89 | 49.44, 50.35 | 0.0191
E | 52.03 | 49.22, 54.84 | 49.93 | 49.48, 50.39 | 0.1464
O | 51.40 | 48.83, 53.97 | 49.95 | 49.49, 50.41 | 0.2727
A | 43.73 | 40.79, 46.68 | 50.21 | 49.75, 50.66 | 0.0001
C | 46.72 | 44.32, 49.11 | 50.11 | 49.65, 50.57 | 0.0072
Caffeine
# | 1658 | 227 |
N | 50.07 | 49.59, 50.55 | 49.46 | 48.10, 50.81 | 0.3998
E | 50.20 | 49.72, 50.68 | 48.54 | 47.27, 49.81 | 0.0166
O | 50.04 | 49.56, 50.52 | 49.71 | 48.40, 51.03 | 0.6450
A | 49.93 | 49.45, 50.41 | 50.48 | 49.15, 51.81 | 0.4462
C | 49.95 | 49.47, 50.43 | 50.37 | 49.01, 51.72 | 0.5690
Crack
# | 11 | 1874 |
N | 55.26 | 45.53, 64.99 | 49.97 | 49.52, 50.42 | 0.2540
E | 46.46 | 38.43, 54.50 | 50.02 | 49.57, 50.47 | 0.3480
O | 48.01 | 43.74, 52.28 | 50.01 | 49.56, 50.47 | 0.3242
A | 39.61 | 30.12, 49.11 | 50.06 | 49.61, 50.51 | 0.0343
C | 44.28 | 36.73, 51.83 | 50.03 | 49.58, 50.49 | 0.1211
Ecstasy
# | 84 | 1801 |
N | 50.28 | 47.99, 52.58 | 49.99 | 49.53, 50.45 | 0.8031
E | 53.37 | 50.71, 56.03 | 49.84 | 49.39, 50.30 | 0.0110
O | 56.15 | 54.15, 58.15 | 49.71 | 49.25, 50.17 | 0.0001
A | 48.56 | 46.38, 50.75 | 50.07 | 49.61, 50.53 | 0.1837
C | 46.98 | 44.78, 49.18 | 50.14 | 49.68, 50.60 | 0.0065
Heroin
# | 29 | 1856 |
N | 58.66 | 55.84, 61.47 | 49.86 | 49.41, 50.32 | 0.0001
E | 44.77 | 39.80, 49.73 | 50.08 | 49.63, 50.53 | 0.0376
O | 52.41 | 48.00, 56.81 | 49.96 | 49.51, 50.42 | 0.2681
A | 41.48 | 38.25, 44.70 | 50.13 | 49.68, 50.59 | 0.0001
C | 43.04 | 38.95, 47.12 | 50.11 | 49.66, 50.56 | 0.0015
Ketamine
# | 37 | 1848 |
N | 50.47 | 46.78, 54.16 | 49.99 | 49.54, 50.45 | 0.7952
E | 47.23 | 42.01, 52.46 | 50.06 | 49.61, 50.51 | 0.2822
O | 54.39 | 50.87, 57.90 | 49.91 | 49.46, 50.37 | 0.0148
A | 44.50 | 40.15, 48.85 | 50.11 | 49.66, 50.56 | 0.0133
C | 44.99 | 41.13, 48.85 | 50.10 | 49.65, 50.55 | 0.0113
Legal highs
# | 131 | 1754 |
N | 53.13 | 51.32, 54.94 | 49.77 | 49.30, 50.23 | 0.0005
E | 47.12 | 45.06, 49.18 | 50.22 | 49.76, 50.67 | 0.0044
O | 53.16 | 51.53, 54.79 | 49.76 | 49.30, 50.23 | 0.0001
A | 46.25 | 44.62, 47.87 | 50.28 | 49.81, 50.75 | 0.0001
C | 44.50 | 42.88, 46.11 | 50.41 | 49.95, 50.88 | 0.0001
LSD
# | 69 | 1816 |
N | 50.28 | 47.82, 52.73 | 49.99 | 49.53, 50.45 | 0.8196
E | 52.70 | 49.90, 55.51 | 49.90 | 49.44, 50.35 | 0.0531
O | 57.57 | 55.78, 59.35 | 49.71 | 49.25, 50.17 | 0.0001
A | 50.03 | 47.88, 52.17 | 50.00 | 49.54, 50.46 | 0.9793
C | 46.98 | 44.90, 49.06 | 50.11 | 49.65, 50.58 | 0.0045
Methadone
# | 121 | 1764 |
N | 54.99 | 53.14, 56.84 | 49.66 | 49.20, 50.12 | 0.0001
E | 45.27 | 43.10, 47.43 | 50.32 | 49.87, 50.78 | 0.0001
O | 51.87 | 50.02, 53.72 | 49.87 | 49.41, 50.34 | 0.0406
A | 46.00 | 43.84, 48.15 | 50.27 | 49.82, 50.73 | 0.0002
C | 45.74 | 43.89, 47.59 | 50.29 | 49.83, 50.76 | 0.0001
Magic Mushrooms
# | 44 | 1841 |
N | 49.79 | 46.82, 52.75 | 50.01 | 49.55, 50.46 | 0.8844
E | 53.71 | 50.31, 57.12 | 49.91 | 49.46, 50.37 | 0.0309
O | 57.89 | 55.73, 60.05 | 49.81 | 49.35, 50.27 | 0.0001
A | 50.14 | 46.71, 53.57 | 50.00 | 49.54, 50.45 | 0.9352
C | 48.03 | 45.33, 50.74 | 50.05 | 49.59, 50.51 | 0.1459
Nicotine
# | 767 | 1118 |
N | 51.32 | 50.59, 52.04 | 49.10 | 48.52, 49.67 | 0.0001
E | 49.91 | 49.17, 50.65 | 50.06 | 49.49, 50.63 | 0.7485
O | 51.57 | 50.85, 52.28 | 48.92 | 48.35, 49.50 | 0.0001
A | 49.04 | 48.31, 49.78 | 50.66 | 50.09, 51.23 | 0.0007
C | 47.69 | 46.99, 48.38 | 51.59 | 51.01, 52.16 | 0.0001
VSA
# | 21 | 1864 |
N | 50.92 | 46.02, 55.82 | 49.99 | 49.54, 50.44 | 0.6974
E | 52.60 | 47.66, 57.53 | 49.97 | 49.52, 50.42 | 0.2828
O | 56.30 | 53.63, 58.97 | 49.93 | 49.47, 50.38 | 0.0001
A | 46.36 | 41.38, 51.34 | 50.04 | 49.59, 50.49 | 0.1407
C | 49.99 | 45.49, 54.49 | 50.00 | 49.55, 50.45 | 0.9955

##### S2 Appendix.

PCCs between drug consumptions. In this section we show PCCs between drug
consumptions for decade and year based user/non user separation.

Table 26: PCCs between drug consumptions with decade based user/non user
separation

Drug |  Alcohol |  Amphetamines |  Amyl nitrite |  Benzodiazepines |  Cannabis |  Chocolate |  Cocaine |  Caffeine |  Crack |  Ecstasy |  Heroin |  Ketamine |  Legal highs |  LSD |  Methadone |  MMushrooms |  Nicotine |  VSA
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
Alcohol |  | $0.074^{1}$ | $0.074^{1}$ | $0.051^{2}$ | $0.119^{1}$ | $0.099^{1}$ | $0.111^{1}$ | $0.157^{1}$ | $0.027^{4}$ | $0.105^{1}$ | $0.033^{4}$ | $0.078^{2}$ | $0.061^{2}$ | $0.069^{2}$ | $-0.007^{4}$ | $0.071^{1}$ | $0.113^{1}$ | $0.046^{3}$
Amphetamines | $0.074^{1}$ |  | $0.372^{1}$ | $0.463^{1}$ | $0.469^{1}$ | $0.013^{4}$ | $0.580^{1}$ | $0.106^{1}$ | $0.323^{1}$ | $0.597^{1}$ | $0.359^{1}$ | $0.412^{1}$ | $0.481^{1}$ | $0.490^{1}$ | $0.415^{1}$ | $0.481^{1}$ | $0.343^{1}$ | $0.304^{1}$
Amyl nitrite | $0.074^{1}$ | $0.372^{1}$ |  | $0.226^{1}$ | $0.292^{1}$ | $0.028^{4}$ | $0.381^{1}$ | $0.060^{2}$ | $0.144^{1}$ | $0.392^{1}$ | $0.137^{1}$ | $0.345^{1}$ | $0.268^{1}$ | $0.213^{1}$ | $0.084^{1}$ | $0.271^{1}$ | $0.196^{1}$ | $0.130^{1}$
Benzodiazepines | $0.051^{2}$ | $0.463^{1}$ | $0.226^{1}$ |  | $0.354^{1}$ | $0.006^{4}$ | $0.428^{1}$ | $0.055^{2}$ | $0.326^{1}$ | $0.383^{1}$ | $0.395^{1}$ | $0.303^{1}$ | $0.348^{1}$ | $0.352^{1}$ | $0.468^{1}$ | $0.366^{1}$ | $0.260^{1}$ | $0.294^{1}$
Cannabis | $0.119^{1}$ | $0.469^{1}$ | $0.292^{1}$ | $0.354^{1}$ |  | $0.046^{3}$ | $0.453^{1}$ | $0.113^{1}$ | $0.216^{1}$ | $0.521^{1}$ | $0.217^{1}$ | $0.302^{1}$ | $0.526^{1}$ | $0.421^{1}$ | $0.299^{1}$ | $0.497^{1}$ | $0.533^{1}$ | $0.237^{1}$
Chocolate | $0.099^{1}$ | $0.013^{4}$ | $0.028^{4}$ | $0.006^{4}$ | $0.046^{3}$ |  | $0.006^{4}$ | $0.122^{1}$ | $0.032^{4}$ | $0.040^{4}$ | $-0.026^{4}$ | $0.035^{4}$ | $0.017^{4}$ | $0.029^{4}$ | $0.007^{4}$ | $0.024^{4}$ | $0.037^{4}$ | $-0.021^{4}$
Cocaine | $0.111^{1}$ | $0.580^{1}$ | $0.381^{1}$ | $0.428^{1}$ | $0.453^{1}$ | $0.006^{4}$ |  | $0.099^{1}$ | $0.396^{1}$ | $0.633^{1}$ | $0.414^{1}$ | $0.454^{1}$ | $0.445^{1}$ | $0.442^{1}$ | $0.354^{1}$ | $0.480^{1}$ | $0.362^{1}$ | $0.277^{1}$
Caffeine | $0.157^{1}$ | $0.106^{1}$ | $0.060^{2}$ | $0.055^{3}$ | $0.113^{1}$ | $0.122^{1}$ | $0.099^{1}$ |  | $0.035^{4}$ | $0.107^{1}$ | $0.026^{4}$ | $0.058^{3}$ | $0.085^{1}$ | $0.075^{1}$ | $0.039^{4}$ | $0.100^{1}$ | $0.145^{3}$ | $0.053^{3}$
Crack | $0.027^{4}$ | $0.323^{1}$ | $0.144^{1}$ | $0.326^{1}$ | $0.216^{1}$ | $0.032^{4}$ | $0.396^{1}$ | $0.035^{4}$ |  | $0.280^{1}$ | $0.509^{1}$ | $0.255^{1}$ | $0.203^{1}$ | $0.268^{1}$ | $0.367^{1}$ | $0.276^{1}$ | $0.191^{1}$ | $0.278^{1}$
Ecstasy | $0.105^{1}$ | $0.597^{1}$ | $0.392^{1}$ | $0.383^{1}$ | $0.521^{1}$ | $0.040^{4}$ | $0.633^{1}$ | $0.107^{1}$ | $0.280^{1}$ |  | $0.301^{1}$ | $0.511^{1}$ | $0.586^{1}$ | $0.599^{1}$ | $0.315^{1}$ | $0.599^{1}$ | $0.370^{1}$ | $0.289^{1}$
Heroin | $0.033^{4}$ | $0.359^{1}$ | $0.137^{1}$ | $0.395^{1}$ | $0.217^{1}$ | $-0.026^{4}$ | $0.414^{1}$ | $0.026^{4}$ | $0.509^{1}$ | $0.301^{1}$ |  | $0.274^{1}$ | $0.237^{1}$ | $0.347^{1}$ | $0.494^{1}$ | $0.306^{1}$ | $0.185^{1}$ | $0.293^{1}$
Ketamine | $0.078^{1}$ | $0.412^{1}$ | $0.345^{1}$ | $0.303^{1}$ | $0.302^{1}$ | $0.035^{4}$ | $0.454^{1}$ | $0.058^{3}$ | $0.255^{1}$ | $0.511^{1}$ | $0.274^{1}$ |  | $0.393^{1}$ | $0.462^{1}$ | $0.246^{1}$ | $0.436^{1}$ | $0.243^{1}$ | $0.192^{1}$
Legal highs | $0.061^{2}$ | $0.481^{1}$ | $0.268^{1}$ | $0.348^{1}$ | $0.526^{1}$ | $0.017^{4}$ | $0.445^{1}$ | $0.085^{1}$ | $0.203^{1}$ | $0.586^{1}$ | $0.237^{1}$ | $0.393^{1}$ |  | $0.519^{1}$ | $0.334^{1}$ | $0.575^{1}$ | $0.364^{1}$ | $0.314^{1}$
LSD | $0.069^{2}$ | $0.490^{1}$ | $0.213^{1}$ | $0.352^{1}$ | $0.421^{1}$ | $0.029^{4}$ | $0.442^{1}$ | $0.075^{1}$ | $0.268^{1}$ | $0.599^{1}$ | $0.347^{1}$ | $0.462^{1}$ | $0.519^{1}$ |  | $0.343^{1}$ | $0.680^{1}$ | $0.289^{1}$ | $0.299^{1}$
Methadone | $-0.007^{4}$ | $0.415^{1}$ | $0.084^{1}$ | $0.468^{1}$ | $0.299^{1}$ | $0.007^{4}$ | $0.354^{1}$ | $0.039^{4}$ | $0.367^{1}$ | $0.315^{1}$ | $0.494^{1}$ | $0.246^{1}$ | $0.334^{1}$ | $0.343^{1}$ |  | $0.343^{1}$ | $0.234^{1}$ | $0.277^{1}$
MMushrooms | $0.071^{1}$ | $0.481^{1}$ | $0.271^{1}$ | $0.366^{1}$ | $0.497^{1}$ | $0.024^{4}$ | $0.480^{1}$ | $0.100^{1}$ | $0.276^{1}$ | $0.599^{1}$ | $0.306^{1}$ | $0.436^{1}$ | $0.575^{1}$ | $0.680^{1}$ | $0.343^{1}$ |  | $0.324^{1}$ | $0.253^{1}$
Nicotine | $0.113^{1}$ | $0.343^{1}$ | $0.196^{1}$ | $0.260^{1}$ | $0.533^{1}$ | $0.037^{4}$ | $0.362^{1}$ | $0.145^{1}$ | $0.191^{1}$ | $0.370^{1}$ | $0.185^{1}$ | $0.243^{1}$ | $0.364^{1}$ | $0.289^{1}$ | $0.234^{1}$ | $0.324^{1}$ |  | $0.221^{1}$
VSA | $0.046^{3}$ | $0.304^{1}$ | $0.130^{1}$ | $0.294^{1}$ | $0.237^{1}$ | $-0.021^{4}$ | $0.277^{1}$ | $0.053^{3}$ | $0.278^{1}$ | $0.289^{1}$ | $0.293^{1}$ | $0.192^{1}$ | $0.314^{1}$ | $0.299^{1}$ | $0.277^{1}$ | $0.253^{1}$ | $0.221^{1}$ |

Note: ${}^{1}p$-value$<0.001$, ${}^{2}p$-value$<0.01$, ${}^{3}p$-value$<0.05$,
${}^{4}p$-value$>0.05$. _p_ -value is the probability to observe by chance the
same or greater correlation for uncorrelated variables.

Table 27: PCCs between drug consumptions with year based user definition

Drug |  Alcohol |  Amphetamines |  Amyl nitrite |  Benzodiazepines |  Cannabis |  Chocolate |  Cocaine |  Caffeine |  Crack |  Ecstasy |  Heroin |  Ketamine |  Legal highs |  LSD |  Methadone |  MMushrooms |  Nicotine |  VSA
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
Alcohol |  | $0.046^{4}$ | $0.061^{3}$ | $0.048^{2}$ | $0.078^{3}$ | $0.077^{1}$ | $0.124^{1}$ | $0.111^{1}$ | $0.058^{1}$ | $0.107^{1}$ | $0.030^{1}$ | $0.072^{4}$ | $0.093^{1}$ | $0.084^{1}$ | $-0.005^{1}$ | $0.075^{4}$ | $0.081^{1}$ | $0.055^{3}$
Amphetamines | $0.046^{3}$ |  | $0.222^{1}$ | $0.436^{1}$ | $0.421^{1}$ | $0.003^{4}$ | $0.453^{1}$ | $0.072^{2}$ | $0.193^{1}$ | $0.461^{1}$ | $0.305^{1}$ | $0.325^{1}$ | $0.471^{1}$ | $0.392^{1}$ | $0.382^{1}$ | $0.375^{1}$ | $0.311^{1}$ | $0.173^{1}$
Amyl nitrite | $0.061^{2}$ | $0.222^{1}$ |  | $0.199^{1}$ | $0.185^{1}$ | $0.016^{4}$ | $0.262^{1}$ | $0.050^{3}$ | $0.077^{1}$ | $0.276^{1}$ | $0.100^{1}$ | $0.280^{1}$ | $0.277^{1}$ | $0.120^{1}$ | $0.091^{1}$ | $0.159^{1}$ | $0.139^{1}$ | $0.107^{1}$
Benzodiazepines | $0.048^{3}$ | $0.436^{1}$ | $0.199^{1}$ |  | $0.334^{1}$ | $-0.009^{4}$ | $0.365^{1}$ | $0.062^{2}$ | $0.232^{1}$ | $0.304^{1}$ | $0.318^{1}$ | $0.263^{1}$ | $0.318^{1}$ | $0.212^{1}$ | $0.464^{1}$ | $0.271^{1}$ | $0.261^{1}$ | $0.183^{1}$
Cannabis | $0.078^{1}$ | $0.421^{1}$ | $0.185^{1}$ | $0.334^{1}$ |  | $0.020^{4}$ | $0.392^{1}$ | $0.074^{2}$ | $0.165^{1}$ | $0.484^{1}$ | $0.199^{1}$ | $0.277^{1}$ | $0.516^{1}$ | $0.433^{1}$ | $0.301^{1}$ | $0.470^{1}$ | $0.517^{1}$ | $0.164^{1}$
Chocolate | $0.077^{1}$ | $0.003^{4}$ | $0.016^{4}$ | $-0.009^{4}$ | $0.020^{4}$ |  | $0.008^{4}$ | $0.089^{1}$ | $-0.037^{4}$ | $0.057^{3}$ | $-0.003^{4}$ | $0.000^{4}$ | $0.026^{4}$ | $0.044^{4}$ | $0.006^{4}$ | $0.019^{4}$ | $0.009^{4}$ | $-0.012^{4}$
Cocaine | $0.124^{1}$ | $0.453^{1}$ | $0.262^{1}$ | $0.365^{1}$ | $0.392^{1}$ | $0.008^{4}$ |  | $0.069^{2}$ | $0.322^{1}$ | $0.535^{1}$ | $0.358^{1}$ | $0.379^{1}$ | $0.394^{1}$ | $0.302^{1}$ | $0.314^{1}$ | $0.346^{1}$ | $0.329^{1}$ | $0.187^{1}$
Caffeine | $0.111^{1}$ | $0.072^{2}$ | $0.050^{3}$ | $0.062^{2}$ | $0.074^{2}$ | $0.089^{1}$ | $0.069^{2}$ |  | $0.023^{4}$ | $0.059^{3}$ | $0.023^{4}$ | $0.026^{4}$ | $0.067^{3}$ | $0.032^{4}$ | $0.027^{4}$ | $0.050^{3}$ | $0.105^{1}$ | $0.042^{4}$
Crack | $0.058^{3}$ | $0.193^{1}$ | $0.077^{1}$ | $0.232^{1}$ | $0.165^{1}$ | $-0.037^{4}$ | $0.322^{1}$ | $0.023^{4}$ |  | $0.156^{1}$ | $0.350^{1}$ | $0.180^{1}$ | $0.147^{1}$ | $0.139^{1}$ | $0.265^{1}$ | $0.181^{1}$ | $0.126^{1}$ | $0.145^{1}$
Ecstasy | $0.107^{1}$ | $0.461^{1}$ | $0.276^{1}$ | $0.304^{1}$ | $0.484^{1}$ | $0.057^{3}$ | $0.535^{1}$ | $0.059^{1}$ | $0.156^{1}$ |  | $0.190^{1}$ | $0.455^{1}$ | $0.502^{1}$ | $0.509^{1}$ | $0.245^{1}$ | $0.480^{1}$ | $0.343^{1}$ | $0.174^{1}$
Heroin | $0.030^{4}$ | $0.305^{1}$ | $0.100^{1}$ | $0.318^{1}$ | $0.199^{1}$ | $-0.003^{4}$ | $0.358^{1}$ | $0.023^{4}$ | $0.350^{1}$ | $0.190^{1}$ |  | $0.217^{1}$ | $0.185^{1}$ | $0.170^{1}$ | $0.385^{1}$ | $0.171^{1}$ | $0.149^{1}$ | $0.121^{1}$
Ketamine | $0.072^{2}$ | $0.325^{1}$ | $0.280^{1}$ | $0.263^{1}$ | $0.277^{1}$ | $0.000^{4}$ | $0.379^{1}$ | $0.026^{4}$ | $0.180^{1}$ | $0.455^{1}$ | $0.217^{1}$ |  | $0.373^{1}$ | $0.351^{1}$ | $0.202^{1}$ | $0.362^{1}$ | $0.222^{1}$ | $0.151^{1}$
Legal highs | $0.093^{1}$ | $0.471^{1}$ | $0.277^{1}$ | $0.318^{1}$ | $0.516^{1}$ | $0.026^{4}$ | $0.394^{1}$ | $0.067^{2}$ | $0.147^{1}$ | $0.502^{1}$ | $0.185^{1}$ | $0.373^{1}$ |  | $0.434^{1}$ | $0.309^{1}$ | $0.485^{1}$ | $0.348^{1}$ | $0.220^{1}$
LSD | $0.084^{1}$ | $0.392^{1}$ | $0.120^{1}$ | $0.212^{1}$ | $0.433^{1}$ | $0.044^{4}$ | $0.302^{1}$ | $0.032^{4}$ | $0.139^{1}$ | $0.509^{1}$ | $0.170^{1}$ | $0.351^{1}$ | $0.434^{1}$ |  | $0.234^{1}$ | $0.627^{1}$ | $0.267^{1}$ | $0.174^{1}$
Methadone | $-0.005^{4}$ | $0.382^{1}$ | $0.091^{1}$ | $0.464^{1}$ | $0.301^{1}$ | $0.006^{4}$ | $0.314^{1}$ | $0.027^{4}$ | $0.265^{1}$ | $0.245^{1}$ | $0.385^{1}$ | $0.202^{1}$ | $0.309^{1}$ | $0.234^{1}$ |  | $0.253^{1}$ | $0.211^{1}$ | $0.167^{1}$
MMushrooms | $0.075^{2}$ | $0.375^{1}$ | $0.159^{1}$ | $0.271^{1}$ | $0.470^{1}$ | $0.019^{4}$ | $0.346^{1}$ | $0.050^{3}$ | $0.181^{1}$ | $0.480^{1}$ | $0.171^{1}$ | $0.362^{1}$ | $0.485^{1}$ | $0.627^{1}$ | $0.253^{1}$ |  | $0.282^{1}$ | $0.174^{1}$
Nicotine | $0.081^{1}$ | $0.311^{1}$ | $0.139^{1}$ | $0.261^{1}$ | $0.517^{1}$ | $0.009^{4}$ | $0.329^{1}$ | $0.105^{1}$ | $0.126^{1}$ | $0.343^{1}$ | $0.149^{1}$ | $0.222^{1}$ | $0.348^{1}$ | $0.267^{1}$ | $0.211^{1}$ | $0.282^{1}$ |  | $0.145^{1}$
VSA | $0.055^{3}$ | $0.173^{1}$ | $0.107^{1}$ | $0.183^{1}$ | $0.164^{1}$ | $-0.012^{4}$ | $0.187^{1}$ | $0.042^{4}$ | $0.145^{1}$ | $0.174^{1}$ | $0.121^{1}$ | $0.151^{1}$ | $0.220^{1}$ | $0.174^{1}$ | $0.167^{1}$ | $0.174^{1}$ | $0.145^{1}$ |

Note: ${}^{1}p$-value$<0.001$, ${}^{2}p$-value$<0.01$, ${}^{3}p$-value$<0.05$,
${}^{4}p$-value$>0.05$. _p_ -value is the probability to observe by chance the
same or greater correlation for uncorrelated variables.

## References

  * 1. Kleiman MA, Caulkins JP, Hawken A. Drugs and Drug Policy: What Everyone Needs to Know. Oxford University Press; 2011.
  * 2. McGinnis JM, Foege WH. Actual causes of death in the United States. Journal of the American Medical Association. 1993; 270(18):2207–2212.
  * 3. Sutina AR, Evans MK, Zonderman AB. Personality traits and illicit substances: the moderation role of poverty. Drug and Alcohol Dependence. 2013; 131:247–251.
  * 4. Cleveland MJ, Feinberg ME, Bontempo DE, Greenberg MT. The role of risk and protective factors in substance use across adolescence. Journal of Adolescent Health. 2008; 43(2):157–164.
  * 5. Ventura CA, de Souza J, Hayashida M, Ferreira PS. Risk factors for involvement with illegal drugs: opinion of family members or significant others. Journal of Substance Use. 2014; 20(2):136–142.
  * 6. WHO. Prevention of mental disorders: Effective interventions and policy options: Summary report. Geneva: World Health Organization; 2004. Available: http://www.who.int/mental_health/evidence/en/prevention_of_mental_disorders_sr.pdf.
  * 7. Dubey C, Arora M, Gupta S, Kumar B. Five factor correlates: A comparison of substance abusers and non-substance abusers. Journal of the Indian Academy of Applied Psychology. 2010; 36(1):107–114.
  * 8. Bogg T, Roberts BW. Conscientiousness and health-related behaviors: A meta-analysis of the leading behavioral contributors to mortality. Psychological Bulletin. 2004; 130(6):887.
  * 9. Beaglehole R, Bonita R, Horton R, Adams C, Alleyne G, Asaria P, Baugh V, Bekedam H, Billo N, Casswell S, et al. Priority actions for the non-communicable disease crisis. The Lancet. 2011; 377(9775):1438–1447.
  * 10. Bickel WK, Johnson MW, Koffarnus MN, MacKillop J, Murphy G James. The behavioral economics of substance use disorders: reinforcement pathologies and their repair. Annual review of clinical psychology. 2014; 10:641–677.
  * 11. Costa PT, MacCrae RR. Revised NEO-Personality Inventory (NEO PI-R) and NEO Five-Factor Inventory (NEO FFI): Professional manual. Odessa, FL: Psychological Assessment Resources; 1992.
  * 12. Roncero C, Daigre C, Barral C, Ros-Cucurull E, Grau-López L, Rodríguez-Cintas L, Tarifa N, Casas M, Valero S. Neuroticism associated with cocaine-induced psychosis in cocaine-dependent patients: a cross-sectional observational study. PloS one. 2014; 9(9):e106,111.
  * 13. Vollrath M, Torgersen S. Who takes health risks? a probe into eight personality types. Personality and Individual Differences. 2002; 32(7):1185–1197.
  * 14. Flory K, Lynam D, Milich R, Leukefeld C, Clayton R. The relations among personality, symptoms of alcohol and marijuana abuse, and symptoms of comorbid psychopathology: results from a community sample. Experimental and Clinical Psychopharmacology. 2002; 10(4):425–434.
  * 15. Andreassen CS, Griffiths MD, Gjertsen SR, Krossbakken E, Kvam S, Pallesen S. The relationships between behavioral addictions and the five-factor model of personality. Journal of Behavioral Addictions. 2013; 2(2): 90–99.
  * 16. Terracciano A, Lóckenhoff CE, Crum RM, Bienvenu OJ, Costa PT. Five-Factor Model personality profiles of drug users. Bmc Psychiatry. 2008; 8(1):22.
  * 17. Turiano NA, Whiteman SD, Hampson SE, Roberts BW, Mroczek DK. Personality and substance use in midlife: Conscientiousness as a moderator and the effects of trait change. Journal of research in personality. 2012; 46(3):295–305.
  * 18. Stewart SH, Devine H. Relations between personality and drinking motives in young adults. Personality and Individual Differences. 2000; 29(3):495–511.
  * 19. Haider AH, Edwin DH, MacKenzie EJ, Bosse MJ, Castillo RC, Travison TG, Group LS, et al. The use of the NEO-Five Factor inventory to assess personality in trauma patients: a two-year prospective study. Journal of Orthopaedic trauma. 2002; 16(9):660–667.
  * 20. Belcher AM, Volkow ND, Moeller FG, Ferré S, Personality traits and vulnerability or resilience to substance use disorders, Trends in cognitive sciences. 2016; 18(4):2
  * 21. Kopstein AN, Crum RM, Celentano DD, Martin SS. Sensation seeking needs among 8th and 11th graders: characteristics associated with cigarette and marijuana use. Drug and alcohol dependence. 2001; 62(3):195–203.
  * 22. Yasnitskiy L, Gratsilev V, Kulyashova J, Cherepanov F. Possibilities of artificial intellect in detection of predisposition to drug addiction. Perm University Herald Series “Philosophy Psychology Sociology”. 2015; 1(21):61–73.
  * 23. Valero S, Daigre C, Rodríguez-Cintas L, Barral C, Gomà-i-Freixanet M, Ferrer M, Casas M, Roncero C. Neuroticism and impulsivity: Their hierarchical organization in the personality characterization of drug-dependent patients from a decision tree learning perspective. Comprehensive Psychiatry. 2014; 55(5):1227–1233.
  * 24. Bulut F, Bucak IÖ. An urgent precaution system to detect students at risk of substance abuse through classification algorithms. Turkish Journal of Electrical Engineering & Computer Sciences. 2014; 22(3):690–707.
  * 25. Rioux C, Castellanos-Ryan N, Parent S, Ségu JR, The interaction between temperament and the family environment in adolescent substance use and externalizing behaviors: Support for diathesis–stress or differential susceptibility? Developmental Review. 2016; 40: 117–150.
  * 26. Weissman DG, Schriber RA, Fassbender C, Atherton O, Krafft C, Robins RW, Hastings PD, Guyerb AE, Earlier adolescent substance use onset predicts stronger connectivity between reward and cognitive control brain networks. Developmental Cognitive Neuroscience. 2015; 16:121–129.
  * 27. Fehrman E, Egan V. Drug consumption, collected online March 2011 to March 2012, English-speaking countries. ICPSR36536-v1. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2016-09-09. Deposited by Mirkes E. http://doi.org/10.3886/ICPSR36536.v1
  * 28. McCrae RR, Costa PT. A contemplated revision of the NEO Five-Factor Inventory. Personality and Individual Differences. 2004; 36(3):587–596.
  * 29. Stanford MS, Mathias CW, Dougherty DM, Lake SL, Anderson NE, Patton JH. Fifty years of the Barratt Impulsiveness Scale: An update and review. Personality and Individual Differences. 2009; 47(5):385–395.
  * 30. Zuckerman M. Behavioral expressions and biosocial bases of sensation seeking. New York: Cambridge University Press; 1994.
  * 31. Bruinsma K, Taren DL. Chocolate: food or drug? Journal of the American Dietetic Association. 1999; 99(10):1249–1256.
  * 32. Fehrman E, Muhammad AK, Mirkes EM, Egan V, Gorban AN. The Five Factor Model of personality and evaluation of drug consumption risk. ArXiv preprint, version 1 https://arxiv.org/abs/1506.06297v1. 2015.
  * 33. Egan V, Deary I, Austin E. The NEO-FFI: Emerging British norms and an item-level analysis suggest N, A and C are more reliable than O and E. Personality and Individual Differences. 2000; 29(5):907–920.
  * 34. Gurrera RJ, Nestor PG, O’Donnell BF. Personality traits in schizophrenia: comparison with a community sample. The Journal of Nervous and Mental Disease. 2000; 188(1):31–35.
  * 35. Terentjev PV. Biometrische Untersuchungen ’́Uber Die Morpho-Logischen Merkmale Von Rana Ridibunda Pall:(Amphibia, Salientia). Biometrika. 1931; 1:23–51. Available: http://www.jstor.org/stable/2333629
  * 36. Mitteroecker P, Bookstein F. The conceptual and statistical relationship between modularity and morphological integration. Systematic biology. 2007; 56(5):818-836.
  * 37. Berg RL. The ecological significance of correlation pleiades. Evolution. 1960; 1:171-180. Available: http://www.jstor.org/stable/2405824
  * 38. Armbruster WS, Di Stilio VS, Tuxill JD, Flores TC, Runk JL. Covariance and decoupling of floral and vegetative traits in nine Neotropical plants: a re-evaluation of Berg’s correlation-pleiades concept. American Journal of Botany. 1999; 86(1):39–55.
  * 39. Krishnapuram R, Keller JM. A possibilistic approach to clustering. IEEE transactions on fuzzy systems. 1993; 1(2):98–110.
  * 40. Bezdek, J. Pattern recognition with fuzzy objective function algorithms. New York, NY: Plenum Press; 1981.
  * 41. Xu R, Wunsch D. Clustering. Hoboken, New Jersey: John Wiley & Sons Inc.; 2008.
  * 42. Omote H, Sugiyama K. Method for drawing intersecting clustered graphs and its application to web ontology language. In Proceedings of the 2006 Asia-Pacific Symposium on Information Visualisation-Volume 60 2006 Jan 1 (pp. 89-92). Australian Computer Society, Inc. Available: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.294.2209&rep=rep1&type=pdf
  * 43. Hoare J, Moon D. (eds.) Drug misuse declared: findings from the 2009/10. British Crime Survey Home Office Statistical Bulletin 13/10; 2010. Available: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/116321/hosb1310.pdf
  * 44. Egan V. Individual differences and antisocial behaviour. In: Furnham A, Stumm S, Petredies K, editors. Handbook of Individual Differences. Oxford: Blackwell-Wiley; 2011. pp. 512–537.
  * 45. McCrae RR, Costa PT. The NEO Personality Inventory: Using the Five-Factor model in counseling. Journal of Counseling & Development. 1991; 69(4):367–372.
  * 46. Jakobwitz S, Egan V. The dark triad and normal personality traits. Personality and Individual Differences. 2006; 40(2):331–339.
  * 47. Settles RE, Fischer S, Cyders MA, Combs JL, Gunn RL, Smith GT. Negative urgency: a personality predictor of externalizing behavior characterized by neuroticism, low conscientiousness, and disagreeableness. Journal of Abnormal Psychology. 2012; 121(1):160–172.
  * 48. Snowden RJ, Gray NS. Impulsivity and psychopathy: Associations between the barrett impulsivity scale and the psychopathy checklist revised. Psychiatry Research. 2011; 187(3):414–417.
  * 49. García-Montes JM, Zaldívar-Basurto F, López-Ríos F, Molina-Moreno A. The role of personality variables in drug abuse in a Spanish university population. International journal of mental health and addiction. 2009; 7(3):475–487.
  * 50. Fossati P, Ergis AM, Allilaire JF. Problem-solving abilities in unipolar depressed patients: comparison of performance on the modified version of the Wisconsin and the California sorting tests. Psychiatry Research. 2001; 104(2):145–156.
  * 51. McDaniel SR, Mahan JE. An examination of the Impss scale as a valid and reliable alternative to the SSS-V in optimum stimulation level research. Personality and Individual Differences. 2008; 44(7):1528–1538.
  * 52. Ragan DT, Beaver KM. Chronic Offenders: A Life-Course Analysis of Marijuana Users. Youth & Society. 2010; 42:174–198.
  * 53. HomeOfficeUK. Drug misuse:findings from the 2013 to 2014 CSEW 2nd ed. 2014. Available: https://www.gov.uk/government/statistics/drug-misuse-findings-from-the-2013-to-2014-csew.
  * 54. Fridberg DJ, Vollmer JM, O’Donnell BF, Skosnik PD. Cannabis users differ from non-users on measures of personality and schizotypy. Psychiatry Research. 2011; 186(1):46–52.
  * 55. Lee SY, Poon WY, Bentler PM. A two-stage estimation of structural equation models with continuous and polytomous variables. British Journal of Mathematical and Statistical Psychology. 1995; 48(2):339–358.
  * 56. Martinson EO, Hamdan MA. Maximum likelihood and some other asymptotically efficient estimators of correlation in two way contingency tables. Journal of Statistical Computation and Simulation. 1971; 1(1):45–54.
  * 57. Linting M, van der Kooij A. Nonlinear Principal Components Analysis with CATPCA: A tutorial. Journal of Personality Assessment. 2012; 94(1):12–25.
  * 58. Gorban AN, Zinovyev AY. Principal graphs and manifolds. In: Olivas ES, Guerrero JDM, Sober MM, Benedito JRM, López AJS, editors. Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques. Hershey – New York. IGI Global; 2009. pp. 28–59.
  * 59. Gorban AN, Zinovyev AY. Principal manifolds and graphs in practice: from molecular biology to dynamical systems. International journal of neural systems. 2010; 20(03):219–232.
  * 60. Gorban AN, Kégl B, Wunsch DC, Zinovyev AY (eds). Principal Manifolds for Data Visualisation and Dimension Reduction. LNCSE, Vol. 58, Berlin-Heidelberg-New York. Springer; 2008.
  * 61. Pearson K. On lines and planes of closest fit to system of points in space. Philosophical magazine. 1901; 2(6):559–572.
  * 62. Guttman L. Some necessary conditions for common-factor analysis. Psychometrika. 1954; 19(2):149–161.
  * 63. Kaiser HF. The application of electronic computers to factor analysis. Educational and Psychological Measurement. 1960; 20:141–151.
  * 64. Gujarati DN. Basic econometrics, 4th ed. New York: McGraw-Hill; 2003.
  * 65. McCabe GP. Principal variables. Technometrics. 1984; 26(2):137–144.
  * 66. Naikal N, Yang AY, Sastry SS. Informative feature selection for object recognition via Sparse PCA. In: Computer Vision (ICCV), 2011 IEEE International Conference on, IEEE; 2011. pp. 818–825. doi: 10.1109/ICCV.2011.6126321
  * 67. Clarkson KL. Nearest-neighbor searching and metric space dimensions. In: Shakhnarovich G, Darrell T, Indyk P, editors. Nearest-neighbor methods for Learning and Vision: Theory and Practice. MIT Press; 2006. pp. 15–59.
  * 68. Fisher RA. The use of multiple measurements in taxonomic problems. Annals of Eugenics. 1936; 7(2):179–188.
  * 69. Hastie T, Tibshirani R. Discriminant adaptive nearest neighbor classification. Pattern Analysis and Machine Intelligence, IEEE Transactions on. 1996; 18(6):607–616.
  * 70. Li Q, Racine JS. Nonparametric econometrics: theory and practice. Princeton University Press; 2007.
  * 71. Breiman L, Friedman J, Stone CJ, Olshen RA. Classification and regression trees. Belmont, Calif: Wadsworth International Group; 1984.
  * 72. Quinlan JR. Simplifying decision trees. International Journal of Man-Machine Studies. 1987; 27(3):221–234.
  * 73. Rokach L, Maimon O. Decision trees. In: Rokach L, Maimon O, editors. Data Mining and Knowledge Discovery Handbook. Berlin: Springer; 2010. pp. 165–192.
  * 74. Sofeikov KI, Tyukin IY, Gorban AN, Mirkes EM, Prokhorov DV, Romanenko IV. Learning optimization for decision tree classification of non-categorical data with information gain impurity criterion. In: Neural Networks (IJCNN); 2014\. Interna. Joint Confe. on, IEEE; 2014. pp. 3548–3555.
  * 75. Gelfand SB, Ravishankar CS, Delp EJ. An iterative growing and pruning algorithm for classification tree design. IEEE Transaction on Pattern Analysis and Machine Intelligence. 1991; 13(2):163–174.
  * 76. Dietterich T, Kearns M, Mansour Y. Applying the weak learning framework to understand and improve C4.5. In: ICML, Proc. of the 13th Int. Conf. on Machine Learning. San Francisco: Morgan Kaufmann; 1996. pp. 96–104.
  * 77. Kearns M, Mansour Y. On the boosting ability of top-down decision tree learning algorithms. Journal of Computer and System Sciences. 1999; 58(1):109–128.
  * 78. Dinov ID. Expectation Maximization and Mixture Modeling Tutorial. UCLA, Statistics Online Computational Resource; 2008. Available: http://escholarship.org/uc/item/1rb70972.
  * 79. Buhmann MD. Radial basis functions: theory and implementations, vol 12. Cambridge: University Press; 2003.
  * 80. Scott DW. Multivariate Density Estimation: Theory, Practice, and Visualization, 1st edn. New York: Wiley; 1992.
  * 81. Hosmer Jr DW, Lemeshow S. Applied logistic regression. John Wiley & Sons; 2004.
  * 82. Russell S, Norvig P. Artificial intelligence: a modern approach; 1995.
  * 83. Breiman L. Random Forests. Machine Learning. 2001; 45(1):5–32.
  * 84. Biau G. Analysis of a random forests model. The Journal of Machine Learning Research. 2012; 13(1):1063–1095.
  * 85. Hastie T, Tibshirani R, Friedman J. The elements of statistical learning. New York: Springer; 2009.
  * 86. Liaw A, Wiener MR. Classification and Regression by randomForest. R news. 2002; 2(3):18–22.
  * 87. Williams G. Data Mining with Rattle and R: The Art of Excavating Data for Knowledge Discovery. Springer Science & Business Media; 2011.
  * 88. Arlot S, Celisse A. A survey of cross-validation procedures for model selection. Statistics surveys. 2010; 4:40–79.
  * 89. Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. 1995; 57(1):289–300.
  * 90. Mitchell TM. Machine learning. 1997. Burr Ridge, IL: McGraw Hill 45; 1997.
  * 91. Mirkes EM, Alexandrakis I, Slater K, Tuli R, Gorban AN. Computational diagnosis and risk evaluation for canine lymphoma. Computers in Biology and Medicine. 2014a; 53:279–290.
  * 92. Mirkes EM, Alexandrakis I, Slater K, Tuli R, Gorban AN. Computational diagnosis of canine lymphoma. Journal of Physics: Conference Series. 2014; 490(1):012135. Available: http://stacks.iop.org/1742-6596/490/i=1/a=012135.
  * 93. Wilmoth DR. Intelligence and past use of recreational drugs. Intelligence. 2012; 40(1):15–22.
  * 94. Jones SP, Heaven PCL, Psychosocial correlates of adolescent drug-taking behaviour. Journal of Adolescence. 1998; 21(2): 127–134.
  * 95. Whiteside SP, Lynam DR. The five factor model and impulsivity: Using a structural model of personality to understand impulsivity. Personality and Individual Differences. 2001; 30:669–689.
  * 96. Gibbons S. ‘Legal Highs’–novel and emerging psychoactive drugs: a chemical overview for the toxicologist. Clinical Toxicology. 2012; 50:15–24.
  * 97. Nutt D, King LA, Saulsbury W, Blakemore C. Development of a rational scale to assess the harm of drugs of potential misuse. The Lancet. 2007; 369:1047–1053.
  * 98. Digman JM. Higher-order factors of the Big Five. Journal of Personality and Social Psychology. 1997; 73:1246–1256.
  * 99. DeYoung CG, Peterson JB, Higgins DM. Higher-order factors of the Big Five predict conformity: Are there neuroses of health?. Personality and Individual Differences. 2002; 33:533–552.
  * 100. DeYoung CG, Peterson JB, Séguin JR, Tremblay RE. Externalizing behavior and the higher order factors of the Big Five. Journal of Abnormal Psychology. 2008; 117:947–953.


# Convex-hull Estimation using XPSNR for Versatile Video Coding

###### Abstract

As adaptive streaming becomes crucial for delivering high-quality video
content across diverse network conditions, accurate metrics to assess
perceptual quality are essential. This paper explores using the eXtended Peak
Signal-to-Noise Ratio (XPSNR) metric as an alternative to the popular Video
Multimethod Assessment Fusion (VMAF) metric for determining optimized bitrate-
resolution pairs in the context of _Versatile Video Coding_(VVC). Our study is
rooted in the observation that XPSNR shows a superior correlation with
subjective quality scores for VVC-coded Ultra-High Definition (UHD) content
compared to VMAF. We predict the average XPSNR of VVC-coded bitstreams using
spatiotemporal complexity features of the video and the target encoding
configuration and then determine the convex-hull online. On average, the
proposed convex-hull using XPSNR (VEXUS) achieves an overall quality
improvement of $5.84\text{\,}\mathrm{dB}\text{p}$ PSNR and
$0.62\text{\,}\mathrm{dB}\text{p}$ XPSNR while maintaining the same bitrate,
compared to the default UHD encoding using the VVenC encoder, accompanied by
an encoding time reduction of $44.43\text{\,}\mathrm{\char 37\relax}\text{p}$
and a decoding time reduction of $65.46\text{\,}\mathrm{\char
37\relax}\text{p}$. This shift towards XPSNR as a guiding metric shall enhance
the effectiveness of adaptive streaming algorithms, ensuring an optimal
balance between bitrate efficiency and perceptual fidelity with advanced video
coding standards.

Index Terms—  Convex-hull, video streaming, complexity reduction, XPSNR, VVC.

##  1 Introduction

Modern video standards come with ever-increasing complexity. The newest,
_Versatile Video Coding_ (VVC) [1], was expected to be up to ten times more
complex to encode than its predecessor, _High Efficiency Video Coding_ (HEVC)
[2]. Practical implementations, like the open source Versatile Video Encoder
(VVenC) [3], can efficiently deal with scaling the compression efficiency
versus the runtime [4] using various presets offering a trade-off between
runtime and compression efficiency. The convex hull is where the encoding
point achieves Pareto efficiency. Online convex-hull estimation methods emerge
as a critical necessity in this landscape, providing a dynamic and adaptive
means to optimize bitrate and resolution selections. By dynamically adjusting
the bitrate-resolution pairs in response to the video content complexity and
coding algorithms, these methods play a pivotal role in achieving an optimal
trade-off between computational efficiency and visual fidelity in the face of
the increased intricacies associated with advanced codecs like VVC [5, 6].

(a) 0024

(b) 0047

Fig. 1: Rate-distortion (RD) and rate-decoding time curves of representative
sequences of Inter-4K dataset [7] encoded at 540p, 1080p, and 2160p
resolutions using VVenC at faster preset, and decoded using VTM decoder. Fig.
2: Convex-hull estimation and encoding architecture using VEXUS.

In content-adaptive encoding, especially for adaptive streaming applications,
pursuing optimal bitrate-resolution pairs is crucial for balancing efficient
video delivery and perceptual quality [8]. The Pareto-front (referred to as
_convex hull_ in the adaptive streaming literature) estimation methods used to
determine the optimized bitrate-resolution pairs for individual video titles
have been studied intensively in recent literature, motivated by the
observation that specific encoding resolutions yield better quality than
others in a certain bitrate range, as shown in Fig. 1. By considering the
intricacies of content complexity, these methods ensure that encoding is
precisely tuned to the requirements of each video, minimizing the bitrate and
maximizing visual fidelity. Moreover, by selecting the optimized encoding
resolution for each bitrate, we reduce the computational workload during
decoding, as lower-resolution videos require less processing power and memory
than their higher-resolution counterparts [9, 10], as shown in Fig. 1. One of
the convex-hull estimation methods is _Bruteforce_ approach, where, for a set
of $\tilde{r}$ resolutions and $\tilde{q}$ quantization parameters (QP), video
is compressed $\tilde{r}\times\tilde{q}$ times, and then the best resolution-
QP pair is selected for each target bitrate [8, 11]. Notably, the convex-hull
construction is repeated for every scene. Some other techniques involve
spatiotemporal complexity-adaptive encoding resolution selection, employing
machine learning models to predict optimal encoding configurations [12, 13,
14, 15]. As the demand for high-quality and low-bitrate encoding intensifies,
these cutting-edge convex-hull estimation methods stand at the forefront,
promising more efficient and quality-driven video streaming experiences.

Traditional convex-hull estimation methods use Video Multimethod Assessment
Fusion (VMAF) [16] as the perceptual quality metric [8, 13, 17, 15]. However,
as observed in [18], Peak Signal to Noise Ratio (PSNR) [19] and VMAF [16]
measures fail to model the subjective quality of VVC-coded bitstreams
accurately. Moreover, it was observed that Structural Similarity Index (SSIM)
[20], Multi-Scale Structural Similarity Index (MS-SSIM), and eXtended Peak
Signal-to-Noise Ratio (XPSNR) [21] can predict the subjective codec ranking
reported in [22] with acceptable accuracy [23]. Hence, XPSNR appears to be a
superior perceptual quality metric to VMAF when applied to VVC-coded ultra
high definition (UHD) bitstreams [18]. XPSNR goes beyond traditional PSNR
metrics by incorporating additional perceptual considerations, providing a
more nuanced and human-like video quality assessment. XPSNR excels in aligning
with the complexities introduced by VVC encoding, making it a more reliable
and comprehensive metric for assessing the true perceptual quality of VVC-
coded bitstreams compared to the more conventional VMAF. However, despite the
efficiency gained by the faster calculation of XPSNR compared to VMAF [21],
the computational demands of encoding videos at all supported resolutions and
QPs, coupled with the subsequent XPSNR calculations, pose significant time and
energy challenges.

Encoding and evaluating an exhaustive set of configurations becomes
increasingly burdensome as the number of supported resolutions grows. To
mitigate this computational bottleneck, we propose a method, VEXUS, which
predicts optimized convex-hulls (using XPSNR as the quality metric) for VVC
encoding without requiring exhaustive encoding and quality evaluation. It
includes introducing XPSNR and QP prediction models that help estimate
optimized bitrate-resolution-QP triples for a given video scene.

##  2 VEXUS architecture

VEXUS streamlines the VVC encoding process to achieve optimal bitrate-quality
trade-offs by introducing a comprehensive convex-hull estimation scheme with
three distinct phases as shown in Fig. 2:

  1. 1

Spatiotemporal complexity feature extraction;

  2. 2

XPSNR-optimized resolution prediction;

  3. 3

Optimized QP prediction.

VEXUS begins with spatiotemporal feature extraction, where intricate details
and motion dynamics are analyzed to capture the inherent complexity of the
input video scene. Subsequently, in the optimized resolution estimation phase,
VEXUS leverages the extracted features and a set of supported resolutions
($\mathcal{R}$) and a set of target bitrates ($\mathcal{B}$) to determine the
most suitable encoding resolution. Finally, in the optimized QP estimation
phase, VEXUS refines its encoding strategy by incorporating target bitrates
and previously estimated resolutions to predict the optimized QP values for
each bitrate-resolution pair. Through this iterative approach, VEXUS outputs
bitrate-resolution-QP triples tailored to each target bitrate, ensuring
efficient bitrate allocation and preserving visual quality across diverse
resolutions in VVC encoding.

###  2.1 Spatiotemporal complexity feature extraction

We use seven DCT-energy-based features: the average texture energy
($E_{\text{Y}}$), the average gradient of the luma texture energy ($h$), the
average luma brightness ($L_{\text{Y}}$), Average chroma texture energy of U
and V channels ($E_{\text{U}}$ and $E_{\text{V}}$), and the average chroma
intensity of U and V channels ($L_{\text{U}}$ and $L_{\text{V}}$) [24] as the
content complexity features of video scenes.

###  2.2 XPSNR-optimized resolution prediction

The objective of selecting the optimized resolution based on bitrate and video
complexity features is decomposed into two parts:

#### Modeling:

The core of the XPSNR measure is a psycho-visually inspired, simplified
spatiotemporal sensitivity model specified locally for each disjoint block
$B_{k}$ at index $k$, of each input picture $P$ of bit depth $d$, and size
$W\times H$ pixels, as [21]:

(1) (2)

where $w_{k}$ is the visual sensitivity weight associated with the $N\times N$
sized $B_{k}\in P$ and calculated from the visual activity measure of the
block $\hat{a}_{k}$ and an average overall activity $\hat{a}_{P}$. $N$ is
calculated as:

(3)

The definitions of $a_{\text{min}}$, $h_{\text{s}}$, and $h_{\text{t}}$ are
provided in [23] and omitted here for brevity, and $N^{2}$ is the number of
picture samples in each block. A low computational complexity is reached
because the spatial high-pass operator $h_{\text{s}}$, and temporal high-pass
operator $h_{\text{t}}$ use very simple fixed-point operations, and the square
root in the calculation of $w_{k}$ cancels the squaring operations in Eq. 2.

Towards this realization, XPSNR of a video scene encoded at resolution $r$ and
bitrate $b$, _i.e._ , $x_{(r,b)}$ is modeled as a function of video complexity
features {$E_{\text{Y}}$, $h$, $L_{\text{Y}}$, $E_{\text{U}}$, $L_{\text{U}}$,
$E_{\text{V}}$, $L_{\text{V}}$}, $b$, and normalized resolution height
$r`=\frac{r}{2160}$:

(4)

$L_{\text{Y}}$, $L_{\text{U}}$, and $L_{\text{V}}$ features consider
variations in luminance and chrominance within localized regions.
$E_{\text{Y}}$, $E_{\text{U}}$, and $E_{\text{V}}$ features account for
variations in texture across different frame regions, providing insights into
how well the compression or reconstruction method preserves textural details.
$h$ introduces a temporal dimension, where dynamic changes in texture over
time influencing perceived quality are captured [25, 15].

#### Resolution optimization:

In the context of resolution optimization, the goal is to enhance the XPSNR of
encoded video scenes by predicting the optimized encoding resolution. This
optimization process aims to select the resolution that maximizes the
predicted XPSNR, thereby improving the visual quality of the encoded content.
The optimization function is:

$\displaystyle\hat{r}=\operatorname*{arg\,max}_{r\in\mathcal{R};r<r_{\text{max}}}\hat{x}_{(r,b)},$
(5)

where $\hat{x}_{(r,b)}$ is the predicted XPSNR of the video scene encoded at
resolution $r$ and bitrate $b$. By iteratively evaluating the XPSNR prediction
for different bitrate-resolution combinations, the optimization function
identifies the resolution that yields the highest predicted XPSNR, ensuring an
optimal balance between bitrate utilization and visual fidelity in the encoded
video scenes.

###  2.3 Optimized QP prediction

For applications such as streaming, avoiding exceeding the maximum bitrates
specified in the HLS/DASH manifests [26] during the encoding process is
essential. Failure to adhere to these limits can lead to buffer overflows or
underflows in video players. Therefore, optimized QP is predicted for a given
target bitrate for capped variable bitrate encoding.

#### Modeling:

The optimized QP is modeled as a function of spatiotemporal features, target
bitrate $b$, and normalized resolution height $r`$ as:

(6)

By considering spatiotemporal features, the encoding can adaptively adjust the
QP parameter to efficiently allocate bits to different scenes within the video
[27]. This ensures that the encoding gives more bits to preserve essential
details in complex scenes while allocating fewer bits to simpler regions,
optimizing the overall bitrate distribution. Target bitrate and resolution
features serve as quality constraints and objectives for encoding. By
incorporating these features into the prediction model, the encoding process
can dynamically adjust the QP parameter to achieve the desired bitrate while
maintaining the specified resolution.

#### Optimization:

The optimization function is:

$\displaystyle\hat{q}_{(r,b)}$
$\displaystyle=\operatorname*{arg\,min}_{q\in[q_{\text{min}},q_{\text{max}}]}\mid\hat{b}_{(r,q)}-b\mid,$
(7)

where $\hat{q}_{(r,b)}$ is the predicted QP of the scene encoded at resolution
$r$ and bitrate $b$. The optimization function aims to predict the QP,
minimizing the discrepancy between the predicted and target bitrate for a
given resolution. This function seeks to find the optimized QP value within
the specified range $[q_{\text{min}},q_{\text{max}}]$ that best aligns the
predicted bitrate $\hat{b}_{r,q}$ with the target bitrate $b$. The chosen QP
value is expected to result in an encoded scene with a bitrate that closely
matches the specified target bitrate, thereby optimizing the encoding process
for bitrate control.

###  2.4 Encoding

The encoding uses the predicted bitrate-resolution-QP configurations for a
given input scene. In VVenC [3], the QP is specified using the qp option. The
maxrate (easy mode) or MaxBitrate (expert mode) option is used to specify the
upper bound of bitrate variability. The lower bound of the bitrate variability
is indirectly achieved through the XPSNR-based perceptual QP adaptation in
combination with the qp option [28]. This means that while the maximum bitrate
is explicitly set, the encoder adjusts the QP values dynamically based on the
perceptual quality model to avoid exceeding the lower bitrate threshold. This
adaptive adjustment ensures that the bitrate remains within the acceptable
range without requiring explicit specification.

##  3 Experimental design

###  3.1 Dataset preparation

We used 1000 videos of the Inter-4K dataset [7] to validate the performance of
the encoding methods. We encoded the sequences at UHD (2160p) 60fps using
VVenC v1.10 [3] using preset 0 (faster). We extracted the spatiotemporal
features, {$E_{\text{Y}}$, $E_{\text{U}}$, $E_{\text{V}}$, $L_{\text{Y}}$,
$L_{\text{U}}$, $L_{\text{V}}$, $h$} using VCA v2.0 [24]. We ran constant
quality encoding by varying _qp_ values from $q_{\text{min}}$ to
$q_{\text{max}}$ for each resolution in $\mathcal{R}$. We computed full-
reference PSNR and XPSNR quality metrics after the compressed video was
upscaled to the original resolution (2160p), as shown in Fig. 3. We used the
bi-cubic filtering algorithm for upscaling and downscaling (from 2160p) of
videos.

###  3.2 Model training

#### XPSNR prediction:

We trained the XPSNR prediction models using multiple regressors, including
extra-trees, XGBoost, and random forests, and we observed that the XGBoost
regressor [29] performed the best consistently using our feature set. We
performed five-fold cross-validation, and the results are averaged from all
folds. Since these values are similar, we assume that the model generalizes
well. We ensured the training set did not include segments from the same
scenes in the test set. We performed a grid search to explore different
combinations of hyperparameter values for the XGBoost model and selected
_max_depth_ =10, and _n_estimators_ =400 that maximized performance.

#### QP prediction:

Since machine learning models are associated with an error during prediction,
and we need our QP to be monotonic, our QP prediction framework employs a
cascading approach to predict bitrate for a given QP. This method involves
training distinct XGboost regression models for minimum and maximum QP values
($q_{\text{min}}$ and $q_{\text{max}}$, respectively), enabling the prediction
of the maximum ($\hat{b}_{\text{max}}$) and minimum bitrate
($\hat{b}_{\text{min}}$). Once the points ($q_{\text{min}}$,
$\hat{b}_{\text{max}}$) and ($q_{\text{max}}$, $\hat{b}_{\text{min}}$) are
estimated, the optimized QP $\hat{q}$ for a target bitrate $b$ is determined
using linear regression [30], as follows:

(8)

The equation captures the non-linear relationship between bitrate and QP by
employing a logarithmic mapping of the bitrate values. This enables finer
granularity in QP adjustment, particularly for target bitrates that fall
within the lower or higher ends of the predicted bitrate range. Moreover,
considering both the maximum and minimum predicted bitrates ensures stability
and reliability in QP prediction, even in scenarios where the target bitrate
varies significantly from the predicted bitrate range.

Fig. 3: Calculation of the groundtruth PSNR, XPSNR, and bitrate to train the
prediction models. This example shows the ground truth calculation of a video
encoded at 1080p with qp 30.  Table 1: Experimental parameters used in this
paper.

_Parameter_ | _Values_
---|---
$\mathcal{R}$ | { 360, 540, 720, 1080, 1440, 2160 }
$\mathcal{B}$ | 0.145 | 0.300 | 0.600 | 0.900 | 1.600 | 2.400
| 3.400 | 4.500 | 5.800 | 8.100 | 11.600 | 16.800
$r_{\text{max}}$ | 720 | 1080 | 2160
Target encoder | VVenC (faster)
Target decoder | VTM

(a) 0076 ($E_{\text{Y}}$=43.49, $h$=21.70, $L_{\text{Y}}$=122.92)

(b) 0094 ($E_{\text{Y}}$=148.67, $h$=14.00, $L_{\text{Y}}$=100.01)

(c) 0153 ($E_{\text{Y}}$=50.83, $h$=9.80, $L_{\text{Y}}$=93.54)

(d) 0263 ($E_{\text{Y}}$=122.64, $h$=38.48, $L_{\text{Y}}$=116.31)

Fig. 4: RD curves, encoding and decoding times of representative video
sequences using Default (green line), FixedLadder (blue line), Bruteforce
(black line), and VEXUS (red line) encodings.

###  3.3 Evaluation setup

We run all experiments on a dual-processor server with Intel Xeon Gold 5218R
(80 cores, frequency at 2.10 GHz), where each VVenC encoding instance uses
four CPU threads. We use $q_{\text{min}}$ and $q_{\text{max}}$ as 10 and 50,
respectively. We run VCA using four CPU threads. The experimental parameters
used to evaluate VEXUS are shown in Table 1.

#### Reference methods:

We compare VEXUS with the following benchmarks:

  1. 1.

_Default_ : This method employs a fixed resolution encoding, _i.e._ , all
bitstreams are encoded at the exact resolution as the input video.

  2. 2.

_FixedLadder_ : This method employs a fixed set of bitrate-resolution pairs.
This paper uses the HLS bitrate ladder specified in the Apple authoring
specifications [31] as the fixed set of bitrate-resolution pairs.

  3. 3.

_Bruteforce_ : This method determines optimized resolution, which yields the
highest XPSNR for a given target bitrate after an exhaustive encoding process
at all supported resolutions and QPs [8, 11].

#### Performance metrics:

We compare the overall quality in PSNR [19] and XPSNR [21] and the achieved
bitrate for every target bitrate of each test sequence. Bjøntegaard Delta
values [32] BD-PSNR and BD-XPSNR refer to the average increase in PSNR and
XPSNR of the bitstreams compared with the _Default_ encoding with the same
bitrate, respectively. Bjøntegaard Delta rate $BDR_{\text{P}}$ and
$BDR_{\text{X}}$ refer to the average increase in bitrate of the bitstreams
compared with the _Default_ encoding with the same PSNR and XPSNR,
respectively. We also evaluate the average relative difference in encoding
time ($\Delta T_{\text{E}}$) and decoding time ($\Delta T_{\text{D}}$)
compared with the _Default_ encoding.

##  4 Evaluation results

#### Speed and accuracy:

We evaluate the pre-processing latency ($\tau_{\text{p}}$) in encoding
introduced by the spatiotemporal complexity feature extraction and the model
inference to predict the optimized resolutions and QPs. We extract the
features at an average rate of 178 fps over the entire dataset (2160p
resolution). The time to predict the resolution for each target bitrate is
$4\text{\,}\mathrm{ms}\text{p}$. The overall latency is negligible as
spatiotemporal complexity feature extraction, and the XPSNR-optimized
resolution prediction can be executed concurrently in real applications. The
average mean absolute error (MAE) of the XPSNR prediction model is
$0.17\text{\,}\mathrm{dB}\text{p}$ with a standard deviation of
$0.22\text{\,}\mathrm{dB}\text{p}$, while the MAE of the QP prediction model
is 1.32, with a standard deviation of 1.86.

#### Rate-distortion performance:

Fig. 4 shows the RD curves of the representative videos (of various video
content complexities) in the test dataset. Notably, the Bruteforce method
consistently yields the highest XPSNR, showcasing its exhaustive search
approach. Meanwhile, the RD curve of VEXUS closely mirrors the Bruteforce
method, indicating the effectiveness of its predictive modeling in
approximating optimized resolutions and QPs. Furthermore, Table 2 shows
$BDR_{\text{P}}$, $BDR_{\text{X}}$, BD-PSNR, and BD-XPSNR results of the
considered methods compared to the _Default_ encoding. VEXUS
($r_{\text{max}}$=2160) yields $BDR_{\text{P}}$, $BDR_{\text{X}}$, BD-PSNR,
and BD-XPSNR of $-25.66\text{\,}\mathrm{\char 37\relax}\text{p}$,
$-18.18\text{\,}\mathrm{\char 37\relax}\text{p}$,
$5.84\text{\,}\mathrm{dB}\text{p}$, and $0.62\text{\,}\mathrm{dB}\text{p}$. It
is observed that the coding efficiency decreases as $r_{\text{max}}$
decreases. To further validate the efficiency of VEXUS, we evaluated the
perceptual quality using the VMAF metric in addition to PSNR and XPSNR. BD-
VMAF of _FixedLadder_ , and VEXUS ($r_{\text{max}}$=2160) are
$-2.17\text{\,}\text{p}$ and $9.39\text{\,}\text{p}$, which confirms the
robustness of VEXUS.

Table 2: Average results of the encoding schemes compared to Default encoding.

Method | $r_{\text{max}}$ |  $BDR_{\text{P}}$ | $BDR_{\text{X}}$ | BD-PSNR | BD-XPSNR | $\Delta T_{\text{E}}$ | $\Delta T_{\text{D}}$
---|---|---|---|---|---|---|---
| [pixels] | [%] | [%] | [dB] | [dB] | [%] | [%]
FixedLadder | - | -11.86 | 1.35 | 1.53 | -0.24 | -52.39 | -65.77
Bruteforce | 720 | -13.17 | -6.29 | 4.65 | 0.51 | 418.96 | -78.82
1080 | -19.49 | -7.93 | 6.21 | 0.83 | 596.23 | -72.60
2160 | -27.26 | -20.27 | 7.45 | 1.02 | 1312.37 | -65.12
VEXUS | 720 | -11.24 | -3.17 | 2.93 | 0.22 | -73.97 | -79.06
1080 | -17.89 | -6.01 | 4.30 | 0.55 | -64.71 | -74.27
2160 | -25.66 | -18.80 | 5.84 | 0.62 | -44.43 | -65.46

#### Latency:

As resolutions decrease for a given target bitrate in VEXUS, the encoding time
tends to decrease. This is because lower resolutions require less
computational effort for encoding than UHD resolution. Similarly, decoding
time is reduced for lower resolutions as decoding operations become less
complex. Therefore, Fig. 4 demonstrates that selecting lower resolutions for a
given target bitrate not only improves encoding efficiency but also lowers
both encoding and decoding time compared to the _Default_ encoding, making it
a favorable approach for optimizing VVC encoding. As shown in Table 2,
Bruteforce approach yields the highest encoding time, as we need to encode at
all supported resolutions and QPs to determine the convex-hull. In VEXUS, the
highest encoding time is observed when $r_{\text{max}}$=2160 due to selecting
higher encoding resolutions optimized for maximum XPSNR. As $r_{\text{max}}$
decreases, the overall encoding and decoding time decreases, and the coding
efficiency is lowered. This trade-off between quality and coding efficiency is
based on the target audience, delivery platform, and available resources.

##  5 Conclusions

This paper introduced XPSNR as an alternative metric to the traditional VMAF
metric to estimate convex-hull for VVC encoding. Our investigation stemmed
from a keen observation: XPSNR demonstrated a better correlation with
subjective quality scores for VVC-coded UHD content. Leveraging this insight,
we introduced an approach where XPSNR is predicted for VVC-coded bitstreams
using spatiotemporal complexity features of the video and the target encoding
configuration. Furthermore, we propose VEXUS, where the convex-hull is
estimated online using the predicted XPSNR. On average, VEXUS yields a
substantial improvement of $5.84\text{\,}\mathrm{dB}\text{p}$ in PSNR and
$0.62\text{\,}\mathrm{dB}\text{p}$ in XPSNR for the same bitrates compared to
the conventional UHD encoding with the VVenC encoder, followed by a
$44.43\text{\,}\mathrm{\char 37\relax}\text{p}$ reduction in overall encoding
time, and a $65.46\text{\,}\mathrm{\char 37\relax}\text{p}$ reduction in
overall decoding time using VTM decoder. As the demand for superior streaming
experiences grows, our findings emphasize the pivotal role of XPSNR in
achieving optimized bitrate-resolution pairs for VVC-coded content.

One promising avenue of future research is investigating how the insights of
convex-hull can be leveraged within the encoder’s rate control mechanisms and
mode decisions. By incorporating knowledge of the convex hull estimation into
rate control algorithms, the encoder could dynamically adjust encoding
parameters to better allocate bitrate resources, particularly in scenarios
where low resolutions are utilized to enhance quality at lower bitrates.

## References

  * [1] B. Bross _et al._ , “Overview of the Versatile Video Coding (VVC) Standard and its Applications,” in _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 31, no. 10, 2021, pp. 3736–3764.
  * [2] G. J. Sullivan _et al._ , “Overview of the High Efficiency Video Coding (HEVC) Standard,” in _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 22, no. 12, 2012, pp. 1649–1668.
  * [3] A. Wieckowski _et al._ , “VVenC: An Open And Optimized VVC Encoder Implementation,” in _2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)_, pp. 1–2.
  * [4] J. Brandenburg _et al._ , “Pareto-optimized coding configurations for VVenC, a fast and efficient VVC encoder,” in _2021 IEEE 23rd International Workshop on Multimedia Signal Processing (MMSP)_ , 2021, pp. 1–6.
  * [5] R. Kaafarani _et al._ , “Evaluation Of Bitrate Ladders For Versatile Video Coder,” in _2021 International Conference on Visual Communications and Image Processing (VCIP)_ , 2021, pp. 1–5.
  * [6] V. V. Menon _et al._ , “Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Streaming,” in _2023 IEEE International Conference on Visual Communications and Image Processing (VCIP)_ , 2023, pp. 1–5.
  * [7] A. Stergiou and R. Poppe, “AdaPool: Exponential Adaptive Pooling for Information-Retaining Downsampling,” in _IEEE Transactions on Image Processing_ , vol. 32, 2023, pp. 251–266.
  * [8] J. De Cock _et al._ , “Complexity-based consistent-quality encoding in the cloud,” in _2016 IEEE International Conference on Image Processing (ICIP)_ , 2016, pp. 1484–1488.
  * [9] V. V. Menon _et al._ , “Video Super-Resolution for Optimized Bitrate and Green Online Streaming,” in _2024 Picture Coding Symposium (PCS)_ , 2024.
  * [10] Z. Azimi _et al._ , “Decoding Complexity-Aware Bitrate-Ladder Estimation for Adaptive VVC Streaming,” in _32nd European Signal Processing Conference (EUSIPCO)_ , 2024.
  * [11] C. Chen _et al._ , “Optimized Transcoding for Large Scale Adaptive Streaming Using Playback Statistics,” in _2018 25th IEEE International Conference on Image Processing (ICIP)_ , 2018, pp. 3269–3273.
  * [12] A. V. Katsenou _et al._ , “Content-gnostic Bitrate Ladder Prediction for Adaptive Video Streaming,” in _2019 Picture Coding Symposium (PCS)_ , 2019, pp. 1–5.
  * [13] A. Zabrovskiy _et al._ , “FAUST: Fast Per-Scene Encoding Using Entropy-Based Scene Detection and Machine Learning,” in _2021 30th Conference of Open Innovations Association FRUCT_ , 2021, pp. 292–302.
  * [14] M. Bhat _et al._ , “Combining Video Quality Metrics To Select Perceptually Accurate Resolution In A Wide Quality Range: A Case Study,” in _2021 IEEE International Conference on Image Processing (ICIP)_ , 2021, pp. 2164–2168.
  * [15] V. V. Menon _et al._ , “JND-Aware Two-Pass Per-Title Encoding Scheme for Adaptive Live Streaming,” in _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 34, no. 2, 2024, pp. 1281–1294.
  * [16] Z. Li _et al._ , “VMAF: The journey continues,” _Netflix Technology Blog_ , vol. 25, 2018.
  * [17] F. Nasiri _et al._ , “Ensemble Learning for Efficient VVC Bitrate Ladder Prediction,” in _2022 10th European Workshop on Visual Information Processing (EUVIP)_ , 2022, pp. 1–6.
  * [18] C. R. Helmrich _et al._ , “Information on and analysis of the VVC encoders in the SDR UHD verification test,” in _WG 05 MPEG Joint Video Coding Team(s) with ITU-T SG 16, document JVET-T0103_ , Oct. 2020.
  * [19] Alliance For Telecommunications Industry Solutions, “Objective Video Quality Measurement Using A Peak-Signal-To-Noise-Ratio (PSNR) Full Reference Technique,” _T1.TR.74-2001_ , 2001.
  * [20] Z. Wang _et al._ , “Image quality assessment: from error visibility to structural similarity,” in _IEEE Transactions on Image Processing_ , vol. 13, no. 4, 2004, pp. 600–612.
  * [21] C. R. Helmrich _et al._ , “XPSNR: A Low-Complexity Extension of The Perceptually Weighted Peak Signal-To-Noise Ratio For High-Resolution Video Quality Assessment,” in _IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2020, pp. 2727–2731.
  * [22] M. Wien and V. Baroncini, “Report on VVC compression performance verification testing in the SDR UHD Random Access Category,” in _WG 05 MPEG Joint Video Coding Team(s) with ITU-T SG 16, document JVET-T0097_ , Oct. 2020.
  * [23] C. R. Helmrich _et al._ , “A study of the extended perceptually weighted peak signal-to-noise ratio (XPSNR) for video compression with different resolutions and bit depths,” in _ITU Journal: ICT Discoveries_ , vol. 3, May 2020. [Online]. Available: http://handle.itu.int/11.1002/pub/8153d78b-en
  * [24] V. V. Menon _et al._ , “Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming,” in _First International ACM Green Multimedia Systems Workshop (GMSys ’23)_ , 2023.
  * [25] V. V. Menon _et al._ , “Video Quality Assessment with Texture Information Fusion for Streaming Applications,” in _Proceedings of the 3rd Mile-High Video Conference_ , 2024, p. 1–6.
  * [26] I. Sodagar, “The MPEG-DASH Standard for Multimedia Streaming Over the Internet,” _IEEE MultiMedia_ , vol. 18, no. 4, pp. 62–67, 2011.
  * [27] V. V. Menon _et al._ , “All-Intra Rate Control Using Low Complexity Video Features for Versatile Video Coding,” in _2023 IEEE International Conference on Image Processing (ICIP)_ , 2023, pp. 2760–2764.
  * [28] C. R. Helmrich _et al._ , “Fast constant-quality video encoding using VVenC with rate capping based on pre-analysis statistics,” in _2024 IEEE International Conference on Image Processing (ICIP)_ , 2024.
  * [29] T. Chen and C. Guestrin, “XGBoost: A Scalable Tree Boosting System,” in _Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ , Aug. 2016, pp. 785–794.
  * [30] A. Premkumar _et al._ , “Quality-Aware Dynamic Resolution Adaptation Framework for Adaptive Video Streaming,” in _Proceedings of the 15th ACM Multimedia Systems Conference_ , 2024, p. 292–298.
  * [31] Apple Inc., “HLS Authoring Specification for Apple Devices.” [Online]. Available: https://developer.apple.com/documentation/http-live-streaming/hls-authoring-specification-for-apple-devices
  * [32] HSTP-VID-WPOM, “Working practices using objective metrics for evaluation of video coding efficiency experiments,” in _International Telecommunication Union_ , 2020.