Temperature dependence of the lower critical field of the noncentrosymmetric
superconductor $\alpha$-BiPd

J. Juraszek1, M. Konczykowski2, D. Kaczorowski1, and T. Cichorek1

1Institute of Low Temperature and Structure Research,
Polish Academy of Sciences, 50-422 Wrocław, Poland

2Laboratoire des Solides Irradiés, CEA/DRF/IRAMIS, École Polytechnique,
CNRS, Institut Polytechnique de Paris, Palaiseau, F-91128, France

##  I Abstract

Temperature variation of the lower critical field in the noncentrosymmetric
superconductor $\alpha$-BiPd was probed by local magnetization measurements
using Hall micromagnetometry, performed down to 0.3 K in a magnetic field
applied along the crystallographic $b$ axis. Below a critical temperature
$T_{c}$ $\simeq$ 3.8 K, a conventional $H_{c1}(T)$ dependence was found,
typical for a single-band $s$-wave BCS superconductor. The obtained data imply
an absence of spin-triplet component in the superconducting wavefunction and
marginal multiband effects in this material, which contradicts some literature
reports.

##  II Introduction

A sufficiently large antisymmetric spin-orbit coupling in superconductors
without inversion symmetry opens the possibility of a novel pairing state
which is a mixture of spin-singlet and spin-triplet components. These parity-
breaking materials have attracted much theoretical and experimental interest,
as they make it a promising route to realize an intrinsic topological
superconductivity [1, 2]. Indeed, a singlet-triplet mixing driven by Rashba
splitting is expected to give rise to zero-energy Majorana modes, which have
potential applications in quantum computing. Moreover, this mixing leads to
two-gap physics that significance is determined by the pairing interactions as
well as the spin-orbit coupling. However, evidence of a triplet component in
the order parameter of noncentrosymmetric superconductors is still elusive,
and presence of multiple conduction and valence bands raises an additional
question about multigap superconductivity that makes an insight into this
material system even more complex.

The compound $\alpha$-BiPd is believed to be the first material identified as
superconducting [3] and noncentrosymmetric [4]. It has a monoclinic crystal
structure (space group $P2_{1}$) of its own type (see critical discussion in
Ref. [5]), characterized by very weak chemical bonds between Bi atoms along
the $b$ axis [6]. The superconductivity emerges below a critical temperature
$T_{c}$ $\simeq$ 3.8 K at ambient conditions and bears a renewed interest due
to strong antisymmetric spin-orbit interaction that generates large Rashba
splitting. The latter effect was clearly evidenced experimentally by means of
angle-resolved photoemission spectroscopy (ARPES) [7, 8, 9, 10, 11] and
scanning tunneling spectroscopy [8, 12], and confirmed by ab-initio electronic
band structure calculations [5, 7, 8, 9, 10, 12, 13, 14]. Furthermore, the
spin-resolved ARPES spectra revealed the presence of Dirac surface states [7]
that were also inferred from the de Haas - van Alphen oscillations [13, 14].
Odd-parity order parameter in $\alpha$-BiPd was hypothesized from the 209Bi
nuclear quadrupole resonance study, where strongly reduced coherence peak just
below $T_{c}$ in the spin-lattice relaxation rate was found [15].
Unconventional Cooper paring was also anticipated from the point-contact
Andreev reflection spectra [16] and the penetration depth data [17]. In both
experiments, however, the presence of two gaps was assumed with a BCS-type
behavior in the $ac$ plane and a power-law temperature variation for the out-
of-plane direction. Multiband aspect of superconductivity in $\alpha$-BiPd has
been put forward in a few other studies showing, e.g., an upward curvature in
the temperature dependence of the upper critical field $H_{c2}(T)$ close to
zero-field $T_{c}$ [16, 18, 19]. Nevertheless, numerous further experiments,
including the extensive investigations of $H_{c2}(T)$ down to 30 mK, have
characterized $\alpha$-BiPd as a single-gap nodeless dominantly $s$-wave
superconductor [20]. In addition to this, the order-of-magnitude discrepancy
in the reported values of the upper critical field $H_{c2}$ [12, 16, 20, 21,
22] raises a need for verification of two-band scenario [17]. And yet, the
recently observed $\pi$ phase shift in oscillations of $T_{c}$ as a function
of the applied magnetic flux threading through a mesoscopic polycrystalline
$\alpha$-BiPd rings has been reported as a firm evidence for the spin-triplet
pairing state [23].

Here, we critically addressed the issue of multiband superconductivity in
$\alpha$-BiPd via examining the temperature dependence of the lower critical
field $H_{c1}(T)$. The $H_{c1}(T)$ behavior was inspected by means of local
magnetization measurements performed down to 0.3 K. As the experimental
technique we used Hall micromagnetomery, which minimizes disadvantages caused
by (i) existence of various surface barriers that inhibit penetration of
magnetic field, hence leading to overestimation of $H_{c1}$, and (ii)
distortion of magnetic field around finite-size sample that brings about
underestimation of $H_{c1}$. Our results obtained for single-crystalline
$\alpha$-BiPd in magnetic fields applied along the crystallographic $b$ axis
indicate a single-gap superconductivity with conventional $s$-wave pairing.

##  III Results and Discussion

###  III.1 Electrical Resistivity and Specific Heat

Figure 1(a) presents the temperature dependence of the electrical resistivity
$\rho(T)$ of $\alpha$-BiPd (sample 1) measured with current flowing within the
crystallographic $a$-$c$ plane. At 4.2 K, the resistivity is as small as
0.36(4) $\mu\Omega$ cm, and increases with increasing temperature up to about
40.5(4.5) $\mu\Omega$ cm at 300 K (inset). The residual resistivity ratio RRR
= $\rho(300K)/\rho(4.2K)$ = 113 is similar to that reported in the literature
[17, 20, 21]. Another indication of good quality of our single crystals is a
relatively large value of $T_{c}$ = 3.85(2) K and a narrow width of the
superconducting transition $\Delta T_{c}$ $\simeq$ 0.29 K (estimated from the
10%–90% criterion).

The low-temperature dependencies of the specific heat $C(T)$ of $\alpha$-BiPd
(sample 2), measured in zero and finite magnetic fields, are depicted in
Figure 1(b). A pronounced superconducting anomaly found in $H$ = 0 (red
points) is significantly depressed in a rather weak external field of 50 mT
(grey points). For 100 mT, we note an absence of superconducting anomaly down
to $T$ = 0.4 K, in concert with the upper critical field $\mu_{0}$$H_{c2}(0)$
$\simeq$ 89 mT along the [010] direction reported in Ref. [20]. From these
normal-state data, we estimated the electronic $C_{e}$ = $\gamma$$T$ and
phonon $C_{ph}$ contributions, where $\gamma$ = 3.9(2) mJmol-1K-2 is the
Sommerfeld coefficient. The inset of Figure 1(b) presents the electronic
specific heat of $\alpha$-BiPd in the vicinity of $T_{c}$, as $C_{e}$/T vs
temperature. Using an equal-entropy construction, one finds for this crystal
$T_{c}$ = 3.70(5) K, $\Delta T_{c}$ $\simeq$ 0.09 K, and a specific heat jump
$\Delta C_{e}$/$\gamma T_{c}$ $\simeq$ 1.72. The superconducting discontinuity
is somewhat larger than that predicted by the BCS theory (= 1.426), but very
close to that reported in Ref. [12]. On the contrary, it notably differs from
the reduced specific heat jump given in Ref. [21], where it was discussed in
the context of a multiband superconductivity. It is also worth noting that the
gap size $\Delta$(0) = 1.94$k_{\rm{B}}T_{c}$ found for our crystal is much
larger than $\Delta$(0) = 1.35$k_{B}T_{c}$ derived from the 209Bi nuclear
quadrupole resonance measurements [15]. In the latter study, the suppressed
and broad coherence peak below $T_{c}$ in the spin-lattice relaxation rate was
suggested to be a signature of spin-triplet component in the superconducting
state of $\alpha$-BiPd.

###  III.2 Magnetization Measurements

For three other $\alpha$-BiPd samples used in our $H_{c1}$ study, we have
determined their temperature dependencies of the magnetization $M(T)$ in the
vicinity of superconducting transition at $\mu_{0}$$H$ = 1 mT, as depicted in
the insets of Figures 2(a-c). Whereas a drop of $M(T)$ signals the onset of a
superconducting phase transition at virtually the same $T_{c}$onset of 3.8 K,
a somewhat sharper transition $\Delta T_{c}$ = 0.05(1) K was found for sample
B with $T_{c}$ = 3.77(2) K. For sample C, we have measured both the zero-field
cooled (ZFC) and field-cooled (FC) magnetization, as shown in the lower inset
of Figure 2(c). When we assume the ZFC data to represent the complete
diamagnetic shielding below 3.6 K, we can conclude from the FC results that
about 36% volume of the $\alpha$-BiPd sample was in the Meissner-Ochsenfeld
state at $\mu_{0}H$ = 1 mT below this temperature. The main panels of Figures
2(a) and 2(b) present the initial parts of the magnetization curves of two
$\alpha$-BiPd samples measured using a Hall micromagnetometry at different
temperatures down to 0.3 K ($\simeq$ 0.08$T_{c}$). For comparison, in Figure
2(c) there are shown the $M(H)$ curves measured for sample C using a
commercial SQUID magnetometer. These bulk magnetization measurements were
performed down to 2 K ($\simeq$ 0.53$T_{c}$) only.

In Figures 3(a) and 3(b) we show the local magnetization $\Delta M(H)$ curves
obtained at different temperatures after subtraction of the Meissner
(shielding) slope for samples A and B, respectively. (See Figure S1 for
similar results for sample C.) When an applied field reaches a critical value,
called the field of first flux penetration $H_{\rm{p}}$, the first Abrikosov
vortex enters the sample and more vortices penetrate it with the further
increasing magnetic field. Thus, $H_{\rm{p}}$ corresponds to the field at
which a rapid rise of $\Delta M(H)$ sets in, as indicated by black triangles.
An $H_{c1}$ value at given temperature can be directly estimated from the
geometric conversion factor. According to Ref. [24], for a cuboid with
dimensions $2m\times 2n\times 2k$ placed in a magnetic field oriented along
the $k$ direction, the appropriate approximation of $H_{c1}$ is given by

$H_{c1}=\frac{H_{\rm{p}}}{1+\chi N},$ (1)

where $\chi$ is the dimensionless intrinsic magnetic susceptibility of the
material in the superconducting state which can be taken to be equal to $-$1,
and $N$ = [$1+\frac{3}{4}\frac{k}{m}\left(1+\frac{m}{n}\right)]^{-1}$ stands
for the effective demagnetization factor (see Supporting Table 1). In our
experiments, 2$m$ = 40 $\mu$m and 2$n$ = 40 $\mu$m correspond to the width and
length of the Hall sensor active area, respectively, and 2$k$ is the thickness
of the measured $\alpha$-BiPd specimen along the crystallographic $b$ axis.

In the Ginzburg-Landau theory, the lower critical field is given by

$H_{c1}(T)\,=\,\frac{\phi_{0}}{4\pi\mu_{0}\lambda^{2}(T)}\left(\rm{ln}\kappa(\textit{T})+0.5\right),$
(2)

where $\kappa$($T$) = $\lambda$($T$)/$\xi$($T$) is the Ginzburg-Landau
parameter and $\xi$($T$) is the coherence length. Since the dimensionless
parameter $\kappa(T)$ is effectively constant under the logarithm, the
behavior of $H_{c1}$$(T)$ $\propto$ $\lambda^{-2}(T)$ provides a powerful way
to deduce the temperature dependence of the superfluid density, and hence the
symmetry of Cooper pairing as well as the presence of multiband effects.

Figure 4 displays the $H_{c1}(T)$ dependencies derived for $\alpha$-BiPd in
$H$ $\parallel$ [010] using Eq. 1 and the $\Delta M(H)$ data shown in Figure
3. At $T$ = 0.3 K, the values of $\mu_{0}H_{c1}$ are equal to 20.9(2) and
20.5(2) mT for samples A and B, respectively. Small difference between these
values likely arises from uncertainty in the geometrical factor, and their
average $\mu_{0}H_{c1}$ = 20.7(2) mT is in good agreement with the lower limit
of the literature data, which span from 20 to 28 mT [12, 18, 19]. Relatively
large spread of about of 40% in the hitherto reported vales of
$\mu_{0}H_{c1}$(0) can be attributed mainly to unspecified geometric
conversion factors and some ambiguity in the determination of $H_{\rm{p}}$.
Besides, it should be noted that the majority of the previous estimations of
$H_{c1}$(0) was based on experiments performed at relatively high temperatures
$T$ $\geq$ 2 K, i.e., above about 0.5$T_{c}$.

The $H_{c1}(T)$ dependencies for $H$ $\parallel$ [010] presented in Figure 4
point at a clear picture of $\alpha$-BiPd being a single-gap $s$-wave
superconductor with a somewhat enhanced electron-phonon coupling. Following
the elaboration of the alpha-model derived from the BCS theory of
superconductivity [25], this is illustrated by the solid lines calculated with
the superconducting gap $\Delta(0)$=2.0$k_{B}T_{c}$ which is slightly larger
that that given by the conventional BCS relation
$\Delta(0)$=1.764$k_{B}T_{c}$. Most importantly, the $H_{c1}(T)$ curve lack a
singularity down to at least 0.08$T_{c}$ that would hint at multiband effects
[26]. The single-band parabolic behavior of the lower critical field in
$\alpha$-BiPd is additionally highlighted in the inset of Figure 4, where the
reduced $H_{c1}$ values are plotted as a function of ($T/T_{c}$)2.

The comprehensive investigation of the upper critical field of $\alpha$-BiPd
revealed $\mu_{0}H_{c2}$(0) $\simeq$ 75 mT for $H$ $\parallel$ [010] [20],
which is by more than an order of magnitude smaller than the values reported
earlier [16, 21, 22]. This zero-temperature value of the upper critical field
combined with our estimation of $\mu_{0}H_{c1}$(0) $\simeq$ 20.7 mT yields the
out-of-plane thermodynamic critical field $\mu_{0}H_{c}$ $\simeq$
($H_{c1}H_{c2}$)1/2 $\simeq$ 39.4 mT that is in good agreement with that
determined from the specific-heat measurements [20]. Remarkably, it implies a
very narrow interval for an external field penetrating the sample as non-
overlapping vortices. In other words, the relation 2$H_{c}\simeq H_{c2}$ found
for $H$ $\parallel$ [010] calls for reinterpretation of the field-dependent
experiments, which were argued to provide the evidence for multiple
superconducting energy gaps in $\alpha$-BiPd [16, 17, 27].

Contrary to common belief, a large number of bands crossing the Fermi level
(approximately 13 in the case of $\alpha$-BiPd [7, 12, 14, 20]) does not
necessarily promote multiband superconductivity. This is because a variety of
interband interactions tend to smear out specific characteristics of singular
gaps, and thus give rise to an effective single-gap behaviour [28]. Therefore,
in view of the $H_{c1}(T)$ results shown in Figure 4, the preceding
observations of the upward curvature of $H_{c2}(T)$ for $\alpha$-BiPd [16, 18,
19, 20] may reflect other effects caused by extrinsic mechanisms. In
particular, it was theoretically proposed that such an unusual $H_{c2}(T)$
behavior can be a consequence of spatial dependence of the coherence length
associated with a modulation of disorder characteristics, such as the
diffusion coefficient or the mean-free path, that frequently happens in
superconducting layered structures due to, e.g., fluctuations in the crystal
axes orientation [29, 30]. The latter scenario seems probable to occur in
$\alpha$-BiPd that exhibits the layered crystal structure with an easy-
cleavage (010) plane which favors an occurrence of step edges on the surface
[12], and thus a locally different field distribution can mimic fluctuations
in the crystal axes orientation.

The conventional behavior of $H_{c1}(T)$ along the $b$ axis established in our
study does not support the scenario of spin-triplet component in the
superfluid density of bulk $\alpha$-BiPd samples suggested in the literature
[16, 17, 23]. In particular, the well-defined single-band $s$-wave picture is
in striking contrast to the results of the Little-Parks experiment on
mesoscopic polycrystalline $\alpha$-BiPd rings which have been ascribed to
half-integer magnetic flux quantization, an expected feature of spin-triplet
pairing [23]. Furthermore, the fact that the period of Little-Parks
oscillations remains unchanged in fields much higher than those corresponding
to the $H_{c2}$ values for bulk samples raises doubts about a mixture of
singlet-triplet pairing in $\alpha$-BiPd. It should be mentioned, however,
that the observation of half-quantum flux was reported before for mesoscopic
rings made of another Pd-Bi superconductor $\beta$-Bi2Pd with $\mu_{0}H_{c2}$
$\simeq$ 0.7 T [31]. Also for this compound, the symmetry of the
superconducting order parameter remains unsolved. While Hall-probe
magnetometry and scanning tunneling spectroscopy strongly supported single-gap
$s$-wave superconductivity in bulk specimens [32], other tunneling
spectroscopy studies reported realization of Majorana-bound states at the
center of the vortices in epitaxial thin films [33].

##  IV Conclusion

In conclusion, we have determined the temperature dependence of the lower
critical field of $\alpha$-BiPd down to $\sim$0.08$\textsl{T}_{c}$ along the
monoclinic $b$ axis. The $H_{c1}(T)$ results indicate that this
noncentrosymmetric compound should be considered a single-gap $s$-wave
superconductor. We found no signature of an unconventional contribution to the
superconducting gap. We also did not observe any hint at multiple
superconducting gaps. The zero-temperature value of $H_{c1}$ being by only a
factor of $\sim$3.7 smaller than $H_{c2}(0)$ points out the need for a
reinterpretation of the field-dependent thermodynamic data hitherto reported
for $\alpha$-BiPd, which have been used as an argument for the claim of
multiple gaps. In particular, we suggest that the observed upward curvature of
$H_{c2}(T)$ in the vicinity of $T_{c}$ reflects an extrinsic effect closely
associated with a layered crystal structure of the compound.

##  V Experimental Section

Single crystals of $\alpha$-BiPd were grown by a modified Bridgman method,
following a recipe described in the literature [21]. Powder and single-crystal
X-ray diffraction as well as a back-reflection Laue method (see Supporting
Figure 1) were used to prove their noncentrosymmetric monoclinic crystal
structure (space group $P2_{1}$), and energy dispersive X-ray spectroscopy was
applied to verify their stoichiometric, highly homogeneous chemical
composition. Further quality check of the $\alpha$-BiPd crystals was made by
means of electrical resistivity and heat capacity measurements performed using
a Quantum Design PPMS-9 platform. Inter alia, the results revealed for each
single-crystalline specimen a sharp superconducting transition at $T_{c}$
$\simeq$ 3.75 K, in agreement with the other works [15, 16, 17, 18, 19, 20] .
The specific heat was measured in the temperature range between 0.4 and 4 K in
magnetic fields up to 100 mT. The zero-field electrical resistivity was
measured in the temperature interval 2–300 K using a standard ac four-probe
technique and electric leads attached to a bar-shaped specimen with silver
epoxy paste.

For low-temperature magnetization measurements, we used three single crystals
grown in the same batch. Samples A and B, were investigated using a Hall
micromagnetometry. In our local magnetization experiments performed on a 3He
cryostat operating down to 0.3 K, each sensor has the active area of 40 $\mu$m
$\times$ 40 $\mu$m and the distance between a sample and a Hall sensor amounts
to 1500 Å. A field-sweep rate as small as 50 $\mu$T/min was applied to ensure
a high accuracy. After each sweep up to 20 mT, the sample was warmed above
$T_{c}$ and cooled again in zero applied field. An identical sensor was used
to monitor an applied field $H$ that was oriented perpendicular to the $a$-$c$
plane of the crystal. More details about our Hall-micromagnetometry setup can
be found elsewhere [26]. For the magnetization measurement on sample C, we
used a Quantum Design superconducting quantum interference device magnetometer
(MPMS-XL) operating on a 4He cryostat. Unfortunately, a brittleness of
crystals and a layered structure of the compound make challenging the
preparation of small and thin samples suitable for the magnetization
measurements in the $a$-$c$ plane.

Acknowledgements

D.K. thanks the National Science Centre (Poland) for the Research Grant
2021/40/Q/ST5/00066. J.J. was supported by the Foundation for Polish Science
(FNP), program START 036.2021.

## References

  * [1] M. Sato, Y. Ando,  _Rep. Prog. Phys._ 2017, _80,_ 076501\.
  * [2] M. Smidman, M. B. Salamon, H. Q. Yuan, D. F. Agterberg,  _Rep. Prog. Phys._ 2017, _80,_ 036501\.
  * [3] N. E. Alekseevskii,  _Zh. Eksp. Teor. Fiz._ 1952, _23,_ 484\.
  * [4] D. M. Kheiker, G. S. Zhdanov, N. N. Zhuravlev,  _Zh. Eksp. Teor. Fiz._ 1953, _25,_ 621\.
  * [5] A. Yaresko, A. P. Schnyder, H. M. Benia, C.-M. Yim, G. Levy, A. Damascelli, C. R. Ast, D. C. Peets, P. Wahl,  _Phys. Rev. B_ 2018, _97,_ 075108\.
  * [6] Y. Bhatt, K. Schubert,  _J. Less-Common Met._ 1979, _64,_ P17.
  * [7] M. Neupane, N. Alidoust, M. M. Hosen, J.-X. Zhu, K. Dimitri, S.-Y. Xu, N. Dhakal, R. Sankar, I. Belopolski, D. S. Sanchez, T.-R. Chang, H.-T. Jeng, K. Miyamoto, T. Okuda, H. Lin, A. Bansil, D. Kaczorowski, F. Chou, M. Z. Hasan, T. Durakiewicz,  _Nat. Commun._ 2016, _7,_ 13315\.
  * [8] H. M. Benia, E. Rampi, C. Trainer, C. M. Yim, A. Maldonado, D. C. Peets, A. Stöhr, U. Starke, K. Kern, A. Yaresko, G. Levy, A. Damascelli, C. R. Ast, A. P. Schnyder, P. Wahl,  _Phys. Rev. B_ 2016, _94,_ 121407\.
  * [9] S. Thirupathaiah, S. Ghosh, R. Jha, E. D. L. Rienks, K. Dolui, V. V. Ravi Kishore, B. Büchner, T. Das, V. P. S. Awana, D. D. Sarma, J. Fink,  _Phys. Rev. Lett._ 2016, _117,_ 177001\.
  * [10] H. Lohani, P. Mishra, A. Gupta, V. P. S. Awana, B. R. Sekhar,  _Phys. C_ 2017, _534,_ 13\.
  * [11] A. Pramanik, R. P. Pandeya, D. V. Vyalikh, A. Generalov, P. Moras, A. K. Kundu, P. M. Sheverdyaeva, C. Carbone, B. Joshi, A. Thamizhavel, S. Ramakrishnan, K. Maiti,  _Phys. Rev. B_ 2021, _103,_ 155401\.
  * [12] Z. Sun, M. Enayat, A. Maldonado, C. Lithgow, E. Yelland, D. C. Peets, A. Yaresko, A. P. Schnyder, P. Wahl,  _Nat. Commun._ 2015, _6,_ 6633\.
  * [13] M. A. Khan, D. E. Graf, I. Vekhter, D. A. Browne, J. F. DiTusa, W. A. Phelan, D. P. Young,  _Phys. Rev. B_ 2019, _99,_ 020507\.
  * [14] J. Klotz, T. A. Butcher, T. Förster, J. Hornung, I. Sheikin, P. Wisniewski, A. Jesche, J. Wosnitza, D. Kaczorowski,  _Phys. Rev. B_ 2020, _101,_ 235139\.
  * [15] K. Matano, S. Maeda, H. Sawaoka, Y. Muro, T. Takabatake, B. Joshi, S. Ramakrishnan, K. Kawashima, J. Akimitsu, G.-q. Zheng,  _J. Phys. Soc. Jpn._ 2013, _82,_ 084711\.
  * [16] M. Mondal, B. Joshi, S. Kumar, A. Kamlapure, S. C. Ganguli, A. Thamizhavel, S. S. Mandal, S. Ramakrishnan, P. Raychaudhuri,  _Phys. Rev. B_ 2012, _86,_ 094520\.
  * [17] L. Jiao, J. L. Zhang, Y. Chen, Z. F. Weng, Y. M. Shao, J. Y. Feng, X. Lu, B. Joshi, A. Thamizhavel, S. Ramakrishnan, H. Q. Yuan,  _Phys. Rev. B_ 2014, _89,_ 060507\.
  * [18] R. Jha, R. Goyal, P. Neha, V. K. Maurya, A. K. Srivastava, A. Gupta, S. Patnaik, V. P. S. Awana,  _Supercond. Sci. Technol._ 2015, _29,_ 025008\.
  * [19] M. Kannan, S. Arumugam, R. Thiyagarajan, G. Kalaiselvan, K. Manikandan, P. Neha, S. Patnaik,  _Phys. Status Solidi RRL_ 2019, _13,_ 1900344\.
  * [20] D. C. Peets, A. Maldonado, M. Enayat, Z. Sun, P. Wahl, A. P. Schnyder,  _Phys. Rev. B_ 2016, _93,_ 174504\.
  * [21] B. Joshi, A. Thamizhavel, S. Ramakrishnan,  _Phys. Rev. B_ 2011, _84,_ 064518\.
  * [22] K. Okawa, M. Kanou, T. Katagiri, H. Kashiwaya, S. Kashiwaya, T. Sasagawa,  _Phys. Procedia_ 2013, _245,_ 101\.
  * [23] X. Xu, Y. Li, C. L. Chien,  _Phys. Rev. Lett._ 2020, _124,_ 167001\.
  * [24] K. Joshi, N. Nusran, M. Tanatar, K. Cho, W. Meier, S. Bud’ko, P. Canfield, R. Prozorov,  _Phys. Rev. Applied_ 2019, _11,_ 014035\.
  * [25] D. C. Johnston,  _Supercond. Sci. Technol._ 2013, _26,_ 115011\.
  * [26] J. Juraszek, R. Wawryk, Z. Henkie, M. Konczykowski, T. Cichorek,  _Phys. Rev. Lett._ 2020, _124,_ 027001\.
  * [27] X. B. Yan, Y. Xu, L. P. He, J. K. Dong, H. Cho, D. C. Peets, J.-G. Park, S. Y. Li,  _Supercond. Sci. Technol._ 2016, _29,_ 065001\.
  * [28] M. Zehetmayer,  _Supercond. Sci. Technol._ 2013, _26,_ 043001\.
  * [29] A. A. Kopasov, D. A. Savinov, A. S. Mel’nikov,  _Phys. Rev. B_ 2017, _95,_ 104520\.
  * [30] A. Antonov, A. Ikonnikov, D. Masterov, A. Mikhaylov, S. Morozov, Y. N.Nozdrin, S. Pavlov, A. Parafin, D. Tetel’baum, S. Ustavschikov, V. Vasiliev, P. Yunin, D. Savinov,  _Phys. C_ 2020, _568,_ 1353581\.
  * [31] Y. Li, X. Xu, M.-H. Lee, M.-W. Chu, C. L. Chien,  _Science_ 2019, _366,_ 238\.
  * [32] J. Kačmarčík, Z. Pribulová, T. Samuely, P. Szabó, V. Cambel, J. Šoltýs, E. Herrera, H. Suderow, A. Correa-Orellana, D. Prabhakaran, P. Samuely,  _Phys. Rev. B_ 2016, _93,_ 144502\.
  * [33] Y.-F. Lv, W.-L. Wang, Y.-M. Zhang, H. Ding, W. Li, L. Wang, K. He, C.-L. Song, X.-C. Ma, Q.-K. Xue,  _Sci. Bull._ 2017, _62,_ 852\.

Figure 1: Characteristics of the $\alpha$-BiPd single crystals. (a) Low-
temperature electrical resistivity in the $a$-$c$ plane showing a sudden drop
to zero at $T_{c}$ = 3.85(2) K (sample 1). Inset: The $\rho$($T$) dependence
in the temperature range 2 – 300 K. (b) Low-temperature specific heat taken in
zero and finite magnetic fields applied perpendicular to the $a$-$c$ plane
(sample 2). The dashed and dotted lines represent the electronic $C_{e}$ and
lattice $C_{ph}$ contributions, respectively, in the 100 mT data. Inset: The
ratio $C_{e}$/$T$ versus $T$ in the vicinity of $T_{c}$ = 3.70(5) K and an
equal-entropy construction. Figure 2: Low-temperature magnetization of
$\alpha$-BiPd for $H$ $\parallel$ [010]. (a) Local magnetization isotherms
taken at different temperatures down to 0.3 K for sample A. The dashed line
marks a shielding slope, which is slightly smaller from unity due to a flux
leakage around the sample (cf. Experimental Section). Inset: The ZFC
magnetization measured in $\mu_{0}H$ = 1 mT. (b) Similar $M(H)$ and $M(T)$
results for sample B. (c) The bulk $M(H)$ data for sample C obtained down to 2
K. Upper inset: Zoom into the $M(H)$ curves measured at temperatures close to
zero-field $T_{c}$. Lower inset: The ZFC and FC curves taken in $\mu_{0}H$ = 1
mT. Figure 3: Field of first flux penetration in $\alpha$-BiPd for
$H$$\,\parallel$ [010]. (a) Local magnetization in sample A obtained after
subtraction of the Meissner slope from the $M(H)$ data shown in Figure 2(a).
For clarity, each isotherm was shifted vertically. The triangles indicate the
field of first flux penetration $H_{\rm{p}}$. Inset: The normalized $M(H)$
dependencies measured at temperatures close to zero-field $T_{c}$. (b) Similar
results obtained for sample B. Figure 4: The lower critical field of
$\alpha$-BiPd along the $b$ axis. (a) Temperature dependencies of $H_{c1}$
determined for samples A and B. The solid lines represent $H_{c1}(T)$
calculated with $\Delta(0)$=2.0$k_{B}T_{c}$. Above 2 K, error bars are the
same size or smaller than the symbols. Inset: Normalized lower critical field
as a function of the square of the normalized temperature.

Supporting Information

Supporting Figure 1: Back-reflection Laue photographs of the $\alpha$-BiPd single crystal. Figure shows the diffractogram along the monoclinic $b$ axis. Table 1:  Dimensions of three $\alpha$-BiPd samples used in this study for $H$ $\parallel$ [010]: $m$ and $n$ are the sample length and width in the a-c plane and $k$ is the dimension along the $b$ axis. For sample A and B, $m$ and $n$ correspond to the width and length of the Hall sensor active area. $N$ is the effective demagnetization factor and $H_{c1}$(0) is the lower critical field in the limit $T$ = 0. Sample | 2$m$ [mm] | 2$n$ [mm] | 2$k$ [mm] | $N$ |  $\mu_{0}H_{c1}$(0) [mT]
---|---|---|---|---|---
A | 0.04 | 0.04 | 0.06 | 0.31 | 20.9
B | 0.04 | 0.04 | 0.05 | 0.34 | 20.7
C | 0.76 | 0.97 | 0.15 | 0.79* | 48.4

*Note that $N$ = 0.74 for sample C can be alternatively estimated from the slope of the initial magnetization curves shown in Fig. 2(c).

Supporting Figure 2: Field of first flux penetration in sample C of
$\alpha$-BiPd for $H$$\,\parallel$ [010]. Bulk magnetization in sample C
obtained after subtraction of the Meissner slope from the $M(H)$ data shown in
Figure 2(c). For clarity, each isotherm was shifted vertically. The triangles
indicate the field of first flux penetration $H_{\rm{p}}$.

Supporting Figure 3: The temperature dependence of $H_{c1}$ of $\alpha$-BiPd
for $H$ $\parallel$ [010]. For all samples investigated, the $H_{c1}$($T$)
data follow the single-band $s$-wave dependence calculated with
$\Delta(0)$=2.0$k_{B}T_{c}$, as shown by the solid lines. Note larger values
of $H_{c1}$ for sample C compared to those estimated using a Hall
micromagnetometry. Inset: The $H_{c1}$/$H_{c1}(0)$ data versus the square of
the normalized temperature.


Copyright for this paper by its authors. Use permitted under Creative Commons
License Attribution 4.0 International (CC BY 4.0).

NeSy 22: 16th International Workshop on Neural-Symbolic Learning and
Reasoning, 28-30 September, London, UK

[orcid=0000-0001-6772-1943<EMAIL_ADDRESS>] [1]

[orcid=0000-0001-6398-0921, ]

[orcid=0000-0003-0063-6378, ]

[orcid=0000-0002-3469-4923<EMAIL_ADDRESS>url=https://jannahastings.github.io/, ] [1]

[1]Corresponding author.

# ESC-Rules: Explainable, Semantically Constrained Rule Sets

Martin Glauer  Institute for Intelligent Interacting Systems, Otto-von-
Guericke University Magdeburg, Germany  Robert West  Susan Michie  Janna
Hastings  Department of Behavioural Science and Health, University College
London, UK  Department of Clinical, Educational and Health Psychology,
University College London, UK

(2022)

###### Abstract

We describe a novel approach to explainable prediction of a continuous
variable based on learning fuzzy weighted rules. Our model trains a set of
weighted rules to maximise prediction accuracy and minimise an ontology-based
’semantic loss’ function including user-specified constraints on the rules
that should be learned in order to maximise the explainability of the
resulting rule set from a user perspective. This system fuses quantitative
sub-symbolic learning with symbolic learning and constraints based on domain
knowledge. We illustrate our system on a case study in predicting the outcomes
of behavioural interventions for smoking cessation, and show that it
outperforms other interpretable approaches, achieving performance close to
that of a deep learning model, while offering transparent explainability that
is an essential requirement for decision-makers in the health domain.

###### keywords:

explainable machine learning rules learning semantic loss evidence synthesis
behaviour change

##  1 Introduction

The rate evidence is generated outstrips the rate at which it can be
synthesised, necessitating automated approaches [1]. The Human Behaviour-
Change Project (HBCP) is an interdisciplinary collaboration between
behavioural scientists, computer scientists and information systems architects
that aims to build an end-to-end automated system for evidence synthesis in
behavioural science [2]. For this objective, explanations of the predictions
are as important as the accuracy of the system, since the intended users are
practitioners and policy-makers who will use the insights gained from the
evidence in order to make recommendations thus require suitable transparency
and accountability [3].

Deep neural networks typically operate as black boxes without giving intrinsic
insights into why specific predictions have been made.111Although, methods to
give explanations for such networks are advancing [4, 5]. Thus, there is a
need for “glass-box” explainable machine learning frameworks for making
predictions and recommendations that can transparently provide complete
explanations in a form that matches the semantic expectations of the users [6,
7].

We aimed to develop an explainable system for the prediction of behaviour
change intervention outcomes, based on a corpus of annotated literature
together with features from an ontology - the Behaviour Change Intervention
ontology [8] \- and their logical relationships. Straightforward application
of semantic approaches is not well suited for the quantitative task of
predicting intervention outcomes, and moreover traditional symbolic learning
approaches such as rules or decision trees lead to explanations that are
overly complex and not ranked by their quantitative impact on the outcome
variable. A deep neural network approach had better quantitative performance,
but was not acceptable for our users due to the lack of transparency and
explainability. Thus, we aimed to develop a ‘best of both worlds’ hybrid
predictive approach that combined aspects of the symbolic and neural
approaches.

Rules-based systems are inherently explainable because the features appear
transparently in the rules. Our approach builds on systems that are able to
learn rules from data. One of the earliest learning rule neural network
systems was the Knowledge Based Artificial Neural Network - KBANN [9]. This
approach translates domain knowledge into rules which are encoded into the
structure of a neural network, for which weights are then learned. The
approach that we developed has furthermore been inspired by traditional
‘neuro-fuzzy’ systems. It is, in particular, based on the principles of
Tagaki-Sugeno controllers [10]. These controllers have been developed to
account for the fact that expert knowledge, albeit valuable, is often too
vague to be turned into rigid logical rules, thus necessitating fuzzy and
weighted approaches to rules learning.

##  2 Explainable Semantically Constrained Rule Sets

We describe our approach in terms of feature preparation, architecture and
optimisation, and semantic penalties. The system is implemented in Python
using PyTorch. Source code is available at
https://github.com/HumanBehaviourChangeProject/semantic-prediction.

###  2.1 Feature preparation

A precondition for our rules-based approach is that all input data features
are binarized. Thus, categorical variables in the dataset are exploded into
separate columns per value, and continuous (quantitative) variables are
binarized by selecting ranges using one of several different approaches
depending on the meanings of the values:

  * •

Separation into meaningful semantic categories, e.g. for our case study we
transform mean age values into child, young adult, older adult, and elderly,
delineated with a fuzzy membership operator since the boundaries between
categories are not rigid.

  * •

Fixed-width categories, delineated with a fuzzy membership operator. For
example, we divide number of times tobacco smoked into groups of width 5
corresponding to <5, <10, <15, … <50\. Note that this formulation creates an
ordering because if the value is e.g. 6, then all of <10, <15, …, <50 will be
set on, and if the value is 46, only <50 will be set on.

  * •

Categories selected based on quantiles in the dataset (i.e. a fixed proportion
of the available data in each grouping, rather than fixed range of values),
again using the <x formulation to maintain ordering.

  * •

Fixed numeric values, for flagging exact values with a specific meaning, for
example 100% female within the population percentage female continuous
variable.

  * •

Data-driven clustering can be used to determine clusters of data associated
with specific value ranges.

###  2.2 Architecture and Optimisation

The general design of the architecture was inspired by the construction of
Takagi-Sugeno controllers. But, while that approach assumes that a set of
rules is given as a prior, the goal of our system is to automatically derive
the rules from the dataset. We assume that rules are represented as
conjunctive formulae of all features in the dataset and their negations. A
weight is then attached to each rule and each feature in the rule as well as
their negated counterparts:

$(w_{i,1},A_{1})\wedge\cdots\wedge(w_{i,n},A_{n})\wedge(w_{i,n+1},\neg
A_{1})\wedge\cdots\wedge(w_{i,2n},\neg A_{n})\Rightarrow r_{i}$ (1)

A weight $w_{i,j}\in\mathbb{R}$ is attached each literal $A_{j}$. The sigmoid
$\sigma(w_{i,j})$ of these weights denotes the degree to which literal $A_{j}$
impacts the $i$-th rule. Each rule is also attached with a rule weight
$r_{i}\in\mathbb{R}$ that denotes the impact of this rule on the prediction.
For the sake of readability, we will further refer to the literals
$A_{1},...,A_{n},\neg A_{1},...,\neg A_{n}$ (i.e. features and negated
features) as just $B_{1},...,B_{2n}$. The basic idea of our approach is to
hide the parts of the conjunction that are not relevant to a particular rule.
That is, variables with a small weight should not influence the rule as much
as ones with larger weights. Given a fuzzy evaluation function that assigns a
fuzzy membership value to each feature
$\mu:\\{B_{1},...,B_{2n}\\}\rightarrow[0,1]$, we therefore calculate the
weighted conjunctive contribution of a literal $B_{j}$ in rule $i$ as a linear
combination between $\mu(B_{j})$ and 1. This allows us to finally define the
fit of a rule as
$\text{fit}_{i}=\max_{j=1}^{2n}(\sigma(w_{i,j})\mu(B_{j})+(1-\sigma(w_{i,j})))$.
Similar to the Takagi-Sugeno controller approach, this fit is an indicator of
how much the right-hand side of the rule should impact the final prediction of
the system. Our system uses this fit as a simple scaling factor
$y_{\text{pred}}=\sum^{m}_{i=1}\text{fit}_{i}\cdot r_{i}$ to aggregate
predictions from multiple rules.

###  2.3 Semantic penalties

While the system described so far does learn rules, those rules are not
meaningfully explainable. This is mainly due to two effects: A) The sigmoids
of the weights often take arbitrary values from [0,1]. B) The left-hand sides
of the rules are often very long, which makes it difficult to analyse the
system easily. To address these points, we introduce additional regularisation
terms that penalise such behaviour. The first one
$\lambda_{\text{long}}=\sum^{m}_{i=1}\max\left(\sum_{j=1}^{n}\sigma(w_{i,j}),\theta\right)$
penalises rules with length exceeding the threshold $\theta$. The second term
$\lambda_{\text{fuzzy}}=\sum^{m}_{i=1}\sum_{j=1}^{n}(1-\sigma(w_{i,j}))\sigma(w_{i,j})$
uses a quadratic equation for non-crisp literals. This penalty is only non-
zero for those feature weights that are non-crisp.

These penalties improve the learned rules. However, the rules learned are
still somewhat arbitrary and lack semantic coherence. Thus, as a final measure
we add semantic ontology-based penalties that aim to increase the coherence
and sense of the rules that are learned using the background domain knowledge
as embodied in an ontology. For our use case we derive the rules from the
Behaviour Change Intervention Ontology [8], using several semantic features.

Implied features

The ontology contains a class hierarchy, which is used in two different ways.
First, to pre-complete the data table: if a lower level feature is on, we
ensure that the higher level feature that subsumes it is also set on in the
input dataset. Hierarchical implications and other implications deriving from
other types of relationships between entities in the ontology are also used to
reduce rule length and enhance rule interpretability. Intuitively, a rule that
has two features, such as
’$\text{Somatic}(X)\wedge\text{PharmacologicalSupport}(X)\Rightarrow r$’,
where one feature implies the other, can be reduced to a rule with one
feature, e.g. ’$\text{PharmacologicalSupport}(X)\Rightarrow r$’, because of
the implication given by the ontological relationship between the entities, as
every intervention with pharmacological support is also one with a somatic
delivery mode. This results in shorter, more readable rules. Therefore, we
introduce an additional penalty
$\lambda_{\text{implied}}=\sum^{m}_{i=1}\sum_{(j,j^{\prime})\in\mathcal{I}}\min(\sigma(w_{i,j}),\sigma(w_{i,j^{\prime}}))$
for the co-occurence of implied features in each rule.

Mutually exclusive features

The ontology also contains axioms regarding negative dependencies between
features. For example, an intervention that has the feature ‘buproprion’ (a
pharmaceutical that is administered in form of a pill) cannot have the feature
‘not pill’. These dependencies can, for instance, be expressed as disjointness
axioms. Rules that include features that contradict each other semantically
are penalised using the same mechanism that has been employed for the
implications. This penalty $\lambda_{\text{exclusive}}$ is also applied to
logically contradictory features within rules, e.g. $A\wedge\neg A\rightarrow
r$.

These penalties ensure the logical soundness and usability of the final rules.
Of course, it would be possible to use purely symbolic techniques to achieve
similar behaviour. Using penalties does however have the added benefit that
the neural network can learn which parts of a conflicting rule can be dropped
with minimal impact on predictive performance, and thereby find a better fit
for the data.

Additional penalties increase the complexity of the training task
considerably, as a multitude of different goals have to be optimised. We
therefore use an additional scaling factor $\alpha$ that fades these penalties
in as the training progresses to later epochs. The final loss function is
therefore calculated using the binary cross entropy (bce):

$\text{loss}(y_{\text{pred}},y_{\text{target}})=\text{bce}(y_{\text{pred}},y_{\text{target}})+\alpha\left(\lambda_{\text{long}}+\lambda_{\text{fuzzy}}+\lambda_{\text{impl}}+\lambda_{\text{exclusive}}\right)$

The resulting system learns human-readable rules from data. These rules are
constrained to be simple and semantically meaningful (and explainable) through
the semantic regularisation terms.

##  3 Evaluation

###  3.1 Dataset

The dataset consists of features extracted from randomised controlled trials
of smoking cessation interventions by manual annotation as described in [11].
Features - annotated using ontology classes - cover several different aspects
of intervention trial reports, including population attributes (e.g. mean
age), delivery attributes such as who delivered it (e.g. nurse) and how it was
delivered (e.g. face to face), and types of intervention (e.g. pharmacological
support or goal setting). In the corpus, these features are associated with
the text specifying the presence of the feature and any associated value, as
well as the wider surrounding contextual text, and the whole composite is
encoded in JSON. From the JSON, additional processing is used to extract
tabular data with a column for each feature and a row for each intervention
arm, and some additional data cleaning ensures that numeric values are
appropriately parsed and that units are regularised. The resulting table has
77 feature columns and 1198 rows corresponding to intervention arms.

###  3.2 Model and Results

We evaluated our system against four other approaches. Firstly, we compare to
the model that was previously developed within the HBCP [12], which we call
‘NLPG’ (short for NLP+Graph, as it learns from a combination of the textual
feature contexts and the annotations encoded as a graph). This model was the
previous best approach for this problem and has been applied as-is as it works
directly from the data encoded in the JSON corpus. The remainder of the
approaches work from the tabular feature dataset described above, which
includes additional data cleaning steps such as ensuring consistent units for
numeric features. Thus, for additional points of comparison, we also applied
three different models from different families of machine learning approach.

decision trees

A random forest of 100 trees with a maximal depth of 4 layers and 7 leaf
nodes.

deep

A feed-forward neural network with 77 input neurons and 3 hidden layers with
154, 77 and 38 neurons, trained for 100 epochs using the Adam optimizer.

mixed-linear

As baseline we use a mixed-effect linear regression model with a random effect
for study and fixed effects for all other variables.

Figure 1: Comparisons of the results of different models. Left: Box plot
showing the distribution and mean of the absolute errors in each model.
Centre: Training/validation loss and penalties. Penalties are faded it at the
beginning of the training. Right: Numbers of rules used in predictions with
no, some or all penalties.

Our model was initialised with a set of 100 rules. Singleton rules for each
(non-negated) feature were initialised based on the weights from a simple
linear regression, and the remaining 23 were initialised randomly using two
uniform distributions: $\mathcal{U}(-1,1)$ for feature weights and
$\mathcal{U}(-10,10)$ for rule weights. The resulting model was trained for at
least 300 epochs, after which the training was stopped as soon as the loss
(including penalties) did not further improve on the validation set. The final
evaluation was then conducted on a test set that had not been seen before.

Figure 1 shows the performances of each model in terms of the absolute errors
of the predictions on the unseen test set data. Our model outperforms the
original approach for prediction, as well as decision trees and the mixed
linear model. It is still slightly outperformed by the pure deep neural
network, however, that approach is not explainable. Additional evaluations
have shown that training on a smaller subset of the dataset leads to less
robust results.

The final state of our system can be used to extract human-readable rules that
apply for each prediction. Some examples of rules that are used in predictions
are illustrated in Listing 1.

"Pharmaceutical company competing interest"[1.0] => raise predicted outcome by
0.2623 (fit: 0.9999) "1.2 Problem solving"[1.0] => raise predicted outcome by
3.2550 (fit: 0.9999) "Mean age (adult)"[1.0] & "doctor"[0.46218] &
"Pill"[0.3893] => raise predicted outcome by 4.6709 (fit: 0.5382)
"doctor"[0.4442] & "Biochemical verification"[1.0] => lower predicted outcome
by 2.7093 (fit: 0.5563) "aggregate patient role"[0.6734] & "Abstinence:
Continuous"[1.0] & not "1.4 Action planning"[1.0] & not "Somatic"[0.7504] &
not "Proportion identifying as female gender" (<= 35)[0.7507] => lower
predicted outcome by 2.9024 (fit: 0.3272)

Listing 1:  Excerpt of a learned rule base that has been applied to a set of
input values. Feature names are in quotations, followed by their attached
weight. The denoted increases and decreases are already scaled according to
the fit of the corresponding rule.

As can be seen, the rules that are learned vary in length from simple weights
for individual features, to complex interactions between features, and vary in
their impact on the outcome value, which may be positive or negative.

The rules and their fits are completely transparent, allowing experts to
inspect the functioning of the system and add additional semantic constraints
where needed to improve the overall performance of the system, leading to an
iterative process which may also have the benefit of improving the associated
ontology from which the semantic constraints are drawn.

##  4 Related Work

Our work is related to several ongoing research areas within neuro-symbolic
computing, which we discuss in turn.

###  4.1 Learning fuzzy logic operations in deep neural networks

Our approach has aspects of a fuzzy inference system, which are less general
than neural networks but more explainable. Most fuzzy neural systems are
restricted to just a few logical operations, usually and and or. They present
an activation function that can learn additional logical operations and allow
learning of complex logical expressions with accuracy comparable to a standard
deep neural network with tanh activation functions. A deep learning
architecture has been proposed for learning fuzzy logic expressions in [13].
More generally, Logic Tensor Networks [14] are able to represent arbitrary
first-order formulae and learn the semantic interpretation of constants,
predicate and function symbols during training by minimising a specific
semantic loss function. The semantics for logical connectives are based on
existing fuzzy semantics - usually based on Łukasiewicz logic. Another notable
approach to rule learning are differentiable logic machines[15]. These
networks allow the expression and learning of logical formulae and use a
hiding mechanism similar to the one presented in this work. They are however
more suitable for purely symbolic applications and less for the regression
task that was our use-case.

###  4.2 Logically constraining deep neural networks

There is a significant body of work on constraining neural networks with
decision rules (reviewed in [16]) or with logical constraints (reviewed in
[17]). A recent addition to this family of approaches is Deep Neural Networks
with Controllable Rule Representations (DeepCTRL) [18], an approach that
incorporates a rule encoder into a deep neural network model together with a
rule-based objective. It allows specification of rules for inputs and outputs,
with rule ‘strength’ adjustable at inference time via a parameter (not
requiring retraining). The rules constrain the model search space to reduce
under-specification and improve generalisability, which is similar to the
effect of the semantic ontology-based constraints in our approach. However,
their rules are not based on an ontology thus must be manually specified.

###  4.3 Learning weighted rule sets

The RuleFit algorithm [19] learns sparse linear models that include
automatically detected interaction effects in the form of rules. Similarly to
our approach, in this approach base variables take values in (0,1) with
categorical variables one-hot encoded and numerical variables discretised into
ranges. However, this approach uses pre-specified rule lengths rather than
optimising the rule length together with the rule weights as we do. In
RuleFit, the sizes of the rules are governed by a random variable selected
from an exponential distribution so that smaller rules are favoured. Thus,
most of the rules will be simple capturing main effects while some will be
larger capturing interaction effects. Moreover, it does not incorporate
semantic constraints on the rules as our system does.

A two-layer neural network for learning rules for binary classification rather
than regression, Decision Rules-net (DR-net) [20] includes a first layer
consisting of a set of neurons that each map to decision rules, and a second
layer that performs a disjunctive combination of the rules from the first
layer. Input features are binarized in a similar fashion to our system, and
then the first layer operates as a conjunction over features in the input
using a binary step activation function. This operation is not differentiable,
thus it is approximated with the straight-through estimator using gradient
clipping. Learnable weights are associated with each decision rule in the
first layer of the network, with positive weights corresponding to a positive
association of the input feature, negative weights with a negative
association, and a zero weight corresponding to exclusion of the input
feature. Stochastic gradient descent is used for training. Sparsity-based
regularisation implements a tradeoff between accuracy of predictions and
simplicity of explanations in terms of the length of the rules. However, they
include no additional semantic penalties.

##  5 Conclusions

We presented a system for rule learning on sparse data for a regression
problem, and evaluated the performance on a use-case from the domain of
behaviour change interventions. Our system out-performed other explainable
methods, and was not far from the predictive power of a black-box deep neural
network. Our system is able to generate human-readable rules that can be used
to easily explain predictions. Moreover, it is possible to enhance the system
through semantic constraints that can either be defined by experts or
extracted from an ontology. This ensures that the system is able to generate
rules that not only fit the dataset but also reflect domain knowledge, are
semantically correct, explainable, and do not contain unnecessarily complex
(and likely overfitted) conditions.

In the future, we aim to conduct a more in-depth evaluation study with users
and domain experts, in particular with respect to the explainability of the
rules. One particular aspect which we would like to explore more deeply with
the domain experts is the explainability associated with the negated features,
i.e. the explanatory value of absences of features, where these appear in
rules. We will also conduct a more in-depth comparison to newer neuro-symboli
approaches. Another aspect we plan to explore is whether there is a
distinction or preference among domain experts for rules that contain
interaction terms from different semantic sub-domains (e.g. intervention,
population, setting) or those that show interactions within a semantic domain
(e.g. different types of intervention in combination).

We also plan to apply the system on different datasets drawn from a wider
range of behavioural domains, including physical activity.

###### Acknowledgements.

Thanks to all members of the Human Behaviour-Change Project team and in
particular to Alison Wright and James Thomas. The HBCP is funded by a Wellcome
Trust collaborative award led by Susan Michie.

## References

  * Elliott et al. [2021] J. Elliott, R. Lawrence, J. C. Minx, O. T. Oladapo, P. Ravaud, B. Tendal Jeppesen, J. Thomas, T. Turner, P. O. Vandvik, J. M. Grimshaw,  Decision makers need constantly updated evidence synthesis,  Nature 600 (2021) 383–385. URL: https://www.nature.com/articles/d41586-021-03690-1. doi:10.1038/d41586-021-03690-1, bandiera_abtest: a Cg_type: Comment Number: 7889 Publisher: Nature Publishing Group Subject_term: Research management.
  * Michie et al. [2020] S. Michie, J. Thomas, P. Mac Aonghusa, R. West, M. Johnston, M. P. Kelly, J. Shawe-Taylor, J. Hastings, F. Bonin, A. O’Mara-Eves,  The Human Behaviour-Change Project: An artificial intelligence system to answer questions about changing behaviour,  Wellcome Open Research 5 (2020) 122. URL: https://wellcomeopenresearch.org/articles/5-122/v1. doi:10.12688/wellcomeopenres.15900.1.
  * Kelly et al. [2019] C. J. Kelly, A. Karthikesalingam, M. Suleyman, G. Corrado, D. King,  Key challenges for delivering clinical impact with artificial intelligence,  BMC medicine 17 (2019) 195. doi:10.1186/s12916-019-1426-2.
  * Moradi and Samwald [2021] M. Moradi, M. Samwald,  Post-hoc explanation of black-box classifiers using confident itemsets,  Expert Systems with Applications 165 (2021) 113941. URL: https://www.sciencedirect.com/science/article/pii/S0957417420307302. doi:10.1016/j.eswa.2020.113941.
  * Madsen et al. [2021] A. Madsen, S. Reddy, S. Chandar,  Post-hoc Interpretability for Neural NLP: A Survey,  arXiv:2108.04840 [cs] (2021). URL: http://arxiv.org/abs/2108.04840, arXiv: 2108.04840.
  * Holzinger et al. [2022a] A. Holzinger, M. Dehmer, F. Emmert-Streib, R. Cucchiara, I. Augenstein, J. D. Ser, W. Samek, I. Jurisica, N. Díaz-Rodríguez,  Information fusion as an integrative cross-cutting enabler to achieve robust, explainable, and trustworthy medical artificial intelligence,  Information Fusion 79 (2022a) 263–278. URL: https://www.sciencedirect.com/science/article/pii/S1566253521002050. doi:10.1016/j.inffus.2021.10.007.
  * Holzinger et al. [2022b] A. Holzinger, R. Goebel, R. Fong, T. Moon, K.-R. Müller, W. Samek,  xxAI - Beyond Explainable Artificial Intelligence,  in: A. Holzinger, R. Goebel, R. Fong, T. Moon, K.-R. Müller, W. Samek (Eds.), xxAI - Beyond Explainable AI: International Workshop, Held in Conjunction with ICML 2020, July 18, 2020, Vienna, Austria, Revised and Extended Papers, Lecture Notes in Computer Science, Springer International Publishing, Cham, 2022b, pp. 3–10. URL: https://doi.org/10.1007/978-3-031-04083-2_1. doi:10.1007/978-3-031-04083-2_1.
  * Michie et al. [2020] S. Michie, R. West, A. N. Finnerty, E. Norris, A. J. Wright, M. M. Marques, M. Johnston, M. P. Kelly, J. Thomas, J. Hastings,  Representation of behaviour change interventions and their evaluation: Development of the Upper Level of the Behaviour Change Intervention Ontology,  Wellcome Open Research 5 (2020) 123. URL: https://wellcomeopenresearch.org/articles/5-123/v1. doi:10.12688/wellcomeopenres.15902.1.
  * Towell and Shavlik [1994] G. G. Towell, J. W. Shavlik,  Knowledge-based artificial neural networks,  Artificial Intelligence 70 (1994) 119–165. URL: https://www.sciencedirect.com/science/article/pii/0004370294901058. doi:10.1016/0004-3702(94)90105-8.
  * Takagi and Sugeno [1985] T. Takagi, M. Sugeno,  Fuzzy identification of systems and its applications to modeling and control,  IEEE transactions on systems, man, and cybernetics (1985) 116–132.
  * Bonin et al. [2020] F. Bonin, M. Gleize, A. Finnerty, C. Moore, C. Jochim, E. Norris, Y. Hou, A. J. Wright, D. Ganguly, E. Hayes, S. Zink, A. Pascale, P. Mac Aonghusa, S. Michie,  HBCP Corpus: A New Resource for the Analysis of Behavioural Change Intervention Reports,  in: Proceedings of the 12th Language Resources and Evaluation Conference, European Language Resources Association, Marseille, France, 2020, pp. 1967–1975. URL: https://aclanthology.org/2020.lrec-1.242.
  * Ganguly et al. [2021] D. Ganguly, M. Gleize, Y. Hou, C. Jochim, F. Bonin, A. Pascale, P. Tommasi, P. Mac Aonghusa, R. West, M. Johnston, et al.,  Outcome prediction from behaviour change intervention evaluations using a combination of node and word embedding,  in: AMIA Annual Symposium Proceedings, volume 2021, American Medical Informatics Association, 2021, p. 486.
  * Godfrey and Gashler [2017] L. B. Godfrey, M. S. Gashler,  A parameterized activation function for learning fuzzy logic operations in deep neural networks,  arXiv:1708.08557 [cs] (2017). URL: http://arxiv.org/abs/1708.08557, arXiv: 1708.08557.
  * Serafini and Garcez [2016] L. Serafini, A. d. Garcez,  Logic tensor networks: Deep learning and logical reasoning from data and knowledge,  arXiv preprint arXiv:1606.04422 (2016).
  * Zimmer et al. [2021] M. Zimmer, X. Feng, C. Glanois, Z. Jiang, J. Zhang, P. Weng, L. Dong, H. Jianye, L. Wulong, Differentiable logic machines, 2021\. URL: https://arxiv.org/abs/2102.11529. doi:10.48550/ARXIV.2102.11529.
  * Okajima and Sadamasa [2019] Y. Okajima, K. Sadamasa,  Deep Neural Networks Constrained by Decision Rules,  Proceedings of the AAAI Conference on Artificial Intelligence 33 (2019) 2496–2505. URL: https://ojs.aaai.org/index.php/AAAI/article/view/4095. doi:10.1609/aaai.v33i01.33012496, number: 01.
  * Giunchiglia et al. [2022] E. Giunchiglia, M. C. Stoian, T. Lukasiewicz,  Deep Learning with Logical Constraints,  arXiv:2205.00523 [cs] (2022). URL: http://arxiv.org/abs/2205.00523, arXiv: 2205.00523.
  * Seo et al. [2021] S. Seo, S. O. Arik, J. Yoon, X. Zhang, K. Sohn, T. Pfister,  Controlling Neural Networks with Rule Representations,  2021\. URL: https://openreview.net/forum?id=owQmPJ9q9u.
  * Friedman and Popescu [2008] J. H. Friedman, B. E. Popescu,  Predictive learning via rule ensembles,  The Annals of Applied Statistics 2 (2008). URL: http://arxiv.org/abs/0811.1679. doi:10.1214/07-AOAS148, arXiv: 0811.1679.
  * Qiao et al. [2021] L. Qiao, W. Wang, B. Lin,  Learning Accurate and Interpretable Decision Rule Sets from Neural Networks,  arXiv:2103.02826 [cs] (2021). URL: http://arxiv.org/abs/2103.02826, arXiv: 2103.02826.