* Sattigeri, Hoffman, Chenthamarakshan, and Varshney Sattigeri, P., Hoffman, S. C., Chenthamarakshan, V., and Varshney, K. R. (2019).  Fairness gan: Generating datasets with fairness properties using a generative adversarial network.  IBM Journal of Research and Development, 63(4/5), 3–1.
  * Schaefer, Boche, Khisti, and Poor Schaefer, R. F., Boche, H., Khisti, A., and Poor, H. V. (2017).  Information Theoretic Security and Privacy of Information Systems.  Cambridge University Press.
  * Schölkopf Schölkopf, B. (2019).  Causality for machine learning..
  * Schwab Schwab, K. (2021).  ‘this is bigger than just timnit’: How google tried to silence a critic and ignited a movement.  https://www.fastcompany.com/90608471/timnit-gebru-google-ai-ethics-equitable-tech-movement.
  * Selbst, Boyd, Friedler, Venkatasubramanian, and Vertesi Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., and Vertesi, J. (2019).  Fairness and abstraction in sociotechnical systems.  In FAT*, pp. 59–68.
  * Shaul Shaul, B. (2015).  Honestly looks to combat cyberbullying on ios, android.  https://www.adweek.com/performance-marketing/honestly-looks-to-combat-cyberbullying-on-ios-android/.
  * Shokri, Stronati, Song, and Shmatikov Shokri, R., Stronati, M., Song, C., and Shmatikov, V. (2017).  Membership inference attacks against machine learning models.  In 2017 IEEE Symposium on SP, pp. 3–18. IEEE.
  * Shu, Cui, Wang, Lee, and Liu Shu, K., Cui, L., Wang, S., Lee, D., and Liu, H. (2019).  dEFEND: Explainable fake news detection.  In KDD, pp. 395–405.
  * Shu, Mahudeswaran, Wang, Lee, and Liu Shu, K., Mahudeswaran, D., Wang, S., Lee, D., and Liu, H. (2020).  Fakenewsnet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media.  Big Data, 8(3), 171–188.
  * Shu, Wang, and Liu Shu, K., Wang, S., and Liu, H. (2018).  Understanding user profiles on social media for fake news detection.  In 2018 IEEE MIPR, pp. 430–435. IEEE.
  * Shu, Wang, and Liu Shu, K., Wang, S., and Liu, H. (2019).  Beyond news contents: The role of social context for fake news detection.  In WSDM, pp. 312–320.
  * Shu, Zheng, Li, Mukherjee, Awadallah, Ruston, and Liu Shu, K., Zheng, G., Li, Y., Mukherjee, S., Awadallah, A. H., Ruston, S., and Liu, H. (2020).  Leveraging multi-source weak social supervision for early detection of fake news..
  * Siegel Siegel, E. (2020)  https://www.kdnuggets.com/how-machine-learning-works-for-social-good.html/.
  * Silva, Rich, and Hall Silva, Y. N., Rich, C., and Hall, D. (2016).  Bullyblocker: Towards the identification of cyberbullying in social networking sites.  In ASONAM, pp. 1377–1379. IEEE.
  * Simonyan, Vedaldi, and Zisserman Simonyan, K., Vedaldi, A., and Zisserman, A. (2013).  Deep inside convolutional networks: Visualising image classification models and saliency maps..
  * Singh, Vatsa, and Ratha Singh, R., Vatsa, M., and Ratha, N. (2021).  Trustworthy AI.  In 8th ACM IKDD CODS and 26th COMAD, pp. 449–453.
  * Slack, Hilgard, Jia, Singh, and Lakkaraju Slack, D., Hilgard, S., Jia, E., Singh, S., and Lakkaraju, H. (2020).  Fooling lime and shap: Adversarial attacks on post hoc explanation methods.  In AIES, pp. 180–186.
  * Smith, Mahdavi, Carvalho, Fisher, Russell, and Tippett Smith, P. K., Mahdavi, J., Carvalho, M., Fisher, S., Russell, S., and Tippett, N. (2008).  Cyberbullying: Its nature and impact in secondary school pupils.  Journal of Child Psychology and Psychiatry, 49(4), 376–385.
  * Ståhl, Falkman, Karlsson, and Mathiason Ståhl, N., Falkman, G., Karlsson, A., and Mathiason, G. (2020).  Evaluation of uncertainty quantification in deep learning.  In IPMU, pp. 556–568. Springer.
  * Stoica, Riederer, and Chaintreau Stoica, A.-A., Riederer, C., and Chaintreau, A. (2018).  Algorithmic glass ceiling in social networks: The effects of social recommendations on network diversity.  In The Web Conference, pp. 923–932.
  * Sühr, Hilgard, and Lakkaraju Sühr, T., Hilgard, S., and Lakkaraju, H. (2020).  Does fair ranking improve minority outcomes? understanding the interplay of human and algorithmic biases in online hiring..
  * Suter, Miladinovic, Schölkopf, and Bauer Suter, R., Miladinovic, D., Schölkopf, B., and Bauer, S. (2019).  Robustly disentangled causal mechanisms: Validating deep representations for interventional robustness.  In ICML, pp. 6056–6065. PMLR.
  * Sweeney Sweeney, L. (2002).  k-anonymity: A model for protecting privacy.  IJUFKS, 10(05), 557–570.
  * Thiebes, Lins, and Sunyaev Thiebes, S., Lins, S., and Sunyaev, A. (2020).  Trustworthy artificial intelligence..
  * Tjoa and Guan Tjoa, E., and Guan, C. (2020).  A survey on explainable artificial intelligence (xai): Toward medical xai..
  * Tong, Du, and Wu Tong, A., Du, D.-Z., and Wu, W. (2018).  On misinformation containment in online social networks.  In NeurIPS, pp. 341–351.
  * Tramèr, Zhang, Juels, Reiter, and Ristenpart Tramèr, F., Zhang, F., Juels, A., Reiter, M. K., and Ristenpart, T. (2016).  Stealing machine learning models via prediction apis.  In USENIX Security, pp. 601–618.
  * Ungar, De Veaux, and Rosengarten Ungar, L. H., De Veaux, R. D., and Rosengarten, E. (1996).  Estimating prediction intervals for artificial neural networks.  In Proc. of the 9th Yale WALS.
  * Varodayan and Khisti Varodayan, D., and Khisti, A. (2011).  Smart meter privacy using a rechargeable battery: Minimizing the rate of information leakage.  In ICASSP, pp. 1932–1935. IEEE.
  * Varshney Varshney, K. R. (2019).  Trustworthy machine learning and artificial intelligence.  XRDS: Crossroads, The ACM Magazine for Students, 25(3), 26–29.
  * Varshney and Alemzadeh Varshney, K. R., and Alemzadeh, H. (2017).  On the safety of machine learning: Cyber-physical systems, decision sciences, and data products.  Big Data, 5(3), 246–255.
  * Vishwamitra, Zhang, Tong, Hu, Luo, Kowalski, and Mazer Vishwamitra, N., Zhang, X., Tong, J., Hu, H., Luo, F., Kowalski, R., and Mazer, J. (2017).  Mcdefender: Toward effective cyberbullying defense in mobile online social networks.  In CODASPY, pp. 37–42.
  * Vosoughi, Roy, and Aral Vosoughi, S., Roy, D., and Aral, S. (2018).  The spread of true and false news online.  Science, 359(6380), 1146–1151.
  * Wachter, Mittelstadt, and Russell Wachter, S., Mittelstadt, B., and Russell, C. (2017).  Counterfactual explanations without opening the black box: Automated decisions and the gdpr.  Harv. JL & Tech., 31, 841.
  * Wan, Xu, Pinson, Dong, and Wong Wan, C., Xu, Z., Pinson, P., Dong, Z. Y., and Wong, K. P. (2013).  Probabilistic forecasting of wind power generation using extreme learning machine.  IEEE Transactions on Power Systems, 29(3), 1033–1044.
  * Wang, Zhang, Bao, Zhu, Cao, and Yu Wang, J., Zhang, J., Bao, W., Zhu, X., Cao, B., and Yu, P. S. (2018a).  Not just privacy: Improving performance of private deep learning in mobile cloud.  In KDD, pp. 2407–2416.
  * Wang, Golbandi, Bendersky, Metzler, and Najork Wang, X., Golbandi, N., Bendersky, M., Metzler, D., and Najork, M. (2018b).  Position bias estimation for unbiased learning to rank in personal search.  In WSDM, pp. 610–618.
  * Wang, Liang, Charlin, and Blei Wang, Y., Liang, D., Charlin, L., and Blei, D. M. (2018c).  The deconfounded recommender: A causal inference approach to recommendation..
  * Wei, Ramamurthy, and Calmon Wei, D., Ramamurthy, K. N., and Calmon, F. d. P. (2020).  Optimized score transformation for fair classification.  In AISTATS.
  * Weller Weller, A. (2017).  Challenges for transparency..
  * Wikipedia Wikipedia (2021a)  https://en.wikipedia.org/w/index.php?title=Robustness_(computer_science)&oldid=1009774103.  Page Version ID: 1009774103.
  * Wikipedia Wikipedia (2021b)  https://en.wikipedia.org/w/index.php?title=Facebook\%E2\%80\%93Cambridge_Analytica_data_scandal&oldid=1035933869.  Page Version ID: 1035933869.
  * Wikipedia Wikipedia (2021c)  https://en.wikipedia.org/w/index.php?title=Cambridge_Analytica&oldid=1036071985.  Page Version ID: 1036071985.
  * Wold, Esbensen, and Geladi Wold, S., Esbensen, K., and Geladi, P. (1987).  Principal component analysis.  Chemometrics and intelligent laboratory systems, 2(1-3), 37–52.
  * Xu, Wu, Yuan, Zhang, and Wu Xu, D., Wu, Y., Yuan, S., Zhang, L., and Wu, X. (2019).  Achieving causal fairness through generative adversarial networks..  In IJCAI, pp. 1452–1458.
  * Xu, Jun, Zhu, and Bellmore Xu, J.-M., Jun, K.-S., Zhu, X., and Bellmore, A. (2012).  Learning from bullying traces in social media.  In NAACL HLT, pp. 656–666. ACL.
  * Yang and Feng Yang, Z., and Feng, J. (2020).  A causal inference method for reducing gender bias in word embedding relations..  In AAAI, pp. 9434–9441.
  * Yang, Yang, Dyer, He, Smola, and Hovy Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., and Hovy, E. (2016).  Hierarchical attention networks for document classification.  In NAACL HLT, pp. 1480–1489.
  * Yao, Chu, Li, Li, Gao, and Zhang Yao, L., Chu, Z., Li, S., Li, Y., Gao, J., and Zhang, A. (2020).  A survey on causal inference..
  * Yeo Yeo, C. (2020).  What is transparency in AI?.  https://medium.com/fair-bytes/what-is-transparency-in-ai-bd08b2e901ac.
  * Young, Magassa, and Friedman Young, M., Magassa, L., and Friedman, B. (2019).  Toward inclusive tech policy design: a method for underrepresented voices to strengthen tech policy documents.  Ethics and Information Technology, 21(2), 89–103.
  * Yuan, He, Zhu, and Li Yuan, X., He, P., Zhu, Q., and Li, X. (2019).  Adversarial examples: Attacks and defenses for deep learning.  IEEE Transactions on Neural Networks and Learning Systems, 30(9), 2805–2824.
  * Zhang, Wu, Wong, Xu, Dong, and Iu Zhang, G., Wu, Y., Wong, K. P., Xu, Z., Dong, Z. Y., and Iu, H. H.-C. (2014).  An advanced approach for construction of optimal wind power prediction intervals.  IEEE Transactions on Power Systems, 30(5), 2706–2715.
  * Zhang and Wu Zhang, L., and Wu, X. (2017).  Anti-discrimination learning: a causal modeling-based framework.  JDSA, 4(1), 1–16.
  * Zhang, Wu, and Wu Zhang, L., Wu, Y., and Wu, X. (2016).  A causal framework for discovering and removing direct and indirect discrimination..
  * Zio Zio, E. (2006).  A study of the bootstrap method for estimating the accuracy of artificial neural networks in predicting nuclear transient processes.  IEEE Transactions on Nuclear Science, 53(3), 1460–1478.


# Lab 4: Design of the Single Balanced Mixer (Using HSMS-281x Diode) and
Measurements

Varun Mannam, Student Member IEEE

###### Abstract

We designed the Mixer operating at 2.4 GHz for WIFI-band using ADS software
and fabricated using Rogers 4350B substrate. We designed the single-balanced
mixer using the diode (HSMS-281x model). We measured the isolation across all
ports of the mixer. We measured the conversion loss of the mixer. We performed
the non-linear measurements, which includes IM3 vs frequency. The performance
metrics are compared with the data-sheet values and explained if any
discrepancy exists.

##  I INTRODUCTION

The mixer is a critical component of RF transceiver design. The following
figure shows the RF receiver block diagram which has the mixer acts as down-
converter from RF (high frequency) to IF (low-frequency).

Figure 1: RF receiver block diagram

The mixer is the non-linear device which produces the non-linear frequencies
for up-conversion/down-conversion.The mixer circuit generates the following
frequencies: $fIF=n*fLO\pm m*fRF$ (m and n are ALL integers). The Mixer is
evolved from single simple diode where RF and LO signal is given to anode and
result IF is at cathode port is taken. Now we used balanced structures (like:
hybrid) in some combinations. here in-phase and out-of phase signals will be
produced using hybrid, where the out of phase signals can be canceled by using
some components. The signal is spread across multiple inter-mods in frequency
domain. The initial single-ended mixer is designed and added few components to
cancel out-of phase inter-mods.The phase-correlation power-diving of 0 and 180
deg are used. In this paper we are designing the single-balanced mixer which
is the hybrid junction with 2-diode. Here the balance means the measure of
cancellation of single-tone inter-mods b/w RF and LO. This mixer provides
Isolation b/w RF and LO ports, 50 percent reduction in inter-mods and higher
conversion efficiency.

The Paper mainly discuss in 2-parts, where as in part-I discuss mainly about
the design of Mixer which includes the fabrication and part-II discuss mainly
about the measurements on the designed mixer. We measured few parameters of
the designed Mixer. Those are linear and non-linear measurements. In our lab,
professor gave us a HSMS-2841x diode model and attached this to the designed
board which is operating at 2.4 GHz. This board has RF ports for inputs (RF
and LO) and output (IF).

The design of Mixer involves multiple steps mainly, diode IV curves, Large-
signal S-parameters of the diode, Rat-race coupler, IF filter and match
network for diode to rat-race coupler. Each of the step is explained in the
later sections in detail. The Fabrication is done with LPKF machine using
Rogers 4350B substrate. The fabrication process is explained separately in the
later sections in-detail.

From this data sheet of the HSMS-281x, the diode designed is operating at 2.45
GHz using SOT-23 package. The linear model of the diode is taken from Avago
Technologies application note 1124.

In linear operation, conversion gain is independent of RF signal power. i.e
1dB decrease in RF power results, 1dB decrease in IF power, results same
conversion loss, but at high RF power, this effects is not same. The IF output
is decreases more than conversion loss of the mixer. This stage is called
Compression in mixer. At this stage, the RF power is used as a function of
switches along with LO powers. The linear measurements are the conversion loss
of the mixer. We measured the isolation of the mixer across all ports. The
non-linear measurements are IM3, 1-dB compression point. These measurements
are performed using 2-tone test which we did similar to our previous labs.

We used a couple of attenuators in our circuit to protect the Mixer and test
equipment. The spectrum analyzer handles maximum power of 1watt. We added one
more 10dB attenuators with lower power rating. Accounting the cable losses in
all measurements are important. We designed the Mixer using HSMS-281x diode
and measured the RF performance. The performance metrics are compared with the
data-sheet values and explained if any discrepancy exists.

##  II Design of the Mixer

As mentioned in the Introduction, design of mixer includes the following steps
mainly Diode-model, I-V characteristics, Large-signal S-parameters, Rat-race
coupler, IF filter design and match network for diode and IF stage. In this
section each step of diode design is explained in-detail.

###  II-A Diode Model

In this section, we created the diode from die model of HSMS-281x data-sheet
and package model from Avago-SMT respectively. We created the spice model of
the die using ADS software. The spice model parameters are given below.

Parameter (Units) | Value
---|---
$B_{v}$ (V) | 25
$C_{j}0$ (pF) | 1.1
$E_{g}$ (eV) | 0.69
$I_{b}v$ (A) | 1e-5
$I_{s}$ (A) | 4.8e-9
N | 1.08
$R_{s}$ (Ohm) | 10
M | 0.5
$V_{j}$ (V) | 0.65
XTI | 2
TABLE I: SPICE parameters of the diode

The following figure shows the ADS model of the diode.

Figure 2: HSMS-281x diode model in ADS

We used the Avago SMT package Model to model the parasitics of the diode which
uses the inductor and capacitors. we are using SOT-23 package which includes
the two-diodes in the package, out of which only 1-diode is activated at
present. The following figure shows the parasitic values of $C_{P}$,$C_{C}$
and $C_{L}$ values.

Figure 3: HSMS-281x diode package in ADS

The parasitic values used in the package model are $L_{L}$ is 0 nH, $C_{L}$ is
0 pF, $C_{P}$ is 0.08 pF, Coupling capacitor $C_{C}$ is 0.06 pF and bond-wire
inductance $(L_{B})$ is 1.0 nH.

###  II-B I-V characteristics

In this section, we performed the DC-simulation of the diode which mainly says
the IV characteristics of the diode. The given ADS template have this step
which plots the I-V characteristics. The following figure shows the dc-
simulation setup of the diode using ADS.

Figure 4: DC simulation of diode in ADS

The following figure shows the IV characteristics of the diode from ADS. The
IV curves dives into the 3- regions, first one is the saturation current below
$v_{D}$ of the diode where as the second one the diode current is dominated by
input voltage from the effect of junction resistance ($R_{j}$) and the third
one is diode current by linear voltage by the Resistance ($R_{s}$). These
characteristics can be seen in log-log plot of IV curves.

Figure 5: I-V curves of the diode in ADS

The diode has series resistance ($Rs$) of 10 ohms.

The diode equivalent linear model is shown below.Here the $R_{s}$ is the
series resistance, $C_{j}$ is the junction capacitance and $R_{j}$ is the
junction resistance is given by the following equation.

$R_{j}=8.33*(1e-5)*N*T/(i_{b}+i_{s})$ (1)

Here $i_{b}$ is the bias-current, $i_{s}$ is the saturation current, N is the
ideality factor and T is the temperature.

Figure 6: Diode model equivalent circuit

After completing this section we understood the the diode resistance of series
and junction values are critical for the operation. The IV curves of the diode
are mainly due to these $R_{j}$ and $R_{s}$ values.

###  II-C Large-signal S-parameters

In this section, we applied single tone at the diode input and finds the input
impedance of the diode at various power levels of the tone. This step is
required to know the mismatch of the diode and rat-race coupler section. The
following figure shows the schematic of the Large-signal S-parameters (LSSP)
of the diode.

Figure 7: LSSP schematic of the diode in ADS

From the figure, it is clear that we vary the power level of the input RF tone
and compute the $Z_{in}$ value for all power levels. The following figure
shows the $Z_{in}$ value for various power levels. Here we select the LO power
is 10 dBm, so used $Z_{in}$ corresponding to 10 dBm RF tone power which gives
the $Z_{in}$ of<EMAIL_ADDRESS>deg at 2.4 GHZ frequency. This $Z_{in}$ is the
normalized one with $Z_{0}$ of 50 Ohms.

Figure 8: $Z_{in}$ values of the diode in ADS

In the same way we compute the $Z_{in}$ of the diode in reverse bias case,
since in our model the diode are placed in opposite directions. The $Z_{in}$
for opposite direction of the diode is 0.621-j*0.354. This $Z_{in}$ is the
normalized one with $Z_{0}$ of 50 Ohms.

###  II-D Matched network for diode

From the previous steps, we got the load impedance ($Z_{in}$) which gives the
matched circuit need to design for rat-race coupler to the diode. Now we need
to design the matched network for the source $Z_{0}$ and and load-impedance
$Z_{in}$. The ADS software with smith chart utility has the the flexibility to
design the matched network.

In this step, we designed load-impedance matched network for diode in forward
bias. The load-impedance is 19.3-j*31.5 ohms. The following figure shows the
load-matched network.

Figure 9: Load-matching network from smith chart tool

Using the Smith Chart Utility, we got the schematics with equivalent
electrical lengths. By using the Line-cal tool in ADS, we got the equivalent
physical length of the transmission lines. We used open stubs which is easy to
design, since these don’t need to connect to via.

Figure 10: Load-matching network with stub physical length

The Line-Cal tool gives us the length and width of the transmission lines for
the specified impedance and Transmission lines length. In the same way, we
computed the electrical and physical lengths of the diode matched network in
reverse bias. The following figure shows the matching network for diode in
opposite direction.

Figure 11: Load-matching network for reverse direction of diode

The following figure shows the equivalent physical lengths of the matched
network.

Figure 12: Load-matching network with physical length for reverse direction of
diode

Till this section, we finished the diode model and matched network of the
diode.

###  II-E Rat-race Coupler

The important component in the mixer design is the hybrid of 90 deg or 180
deg. Here we are using 180 deg hybrid which is the rat-race coupler. The rat-
race coupler is having length of 1.5*$\lambda$ of length. Out of which 3 ports
are separated by $\lambda/4$ and other one is separated by three times
$\lambda/4$. The following figure shows the general rat-race coupler design.

Figure 13: Rat-race coupler

From the figure, the port P1 is the RF input, port P2 is the sum port, port P3
is the LO input and port P4 is the IF output port. The S-parameter matrix of
the rat-race coupler is given below.

Figure 14: S-matrix of the rat-race coupler

In this section, we designed the same rat-race coupler using ADS software.
Design steps: Instead of $\lambda/4$ lines, we used the curves of the angel of
$60^{\circ}$. Out of the 2-designs provided by the professor we are using
design 1 to make the rat-race coupler. The following figure shows the design 1
of the rat-race coupler.

Figure 15: chosen Rat-race design 1

As mentioned earlier, we are using curve of $60^{\circ}$ with same electrical
length instead of $\lambda/4$ lines to make the design looks like a circle. We
added another $\lambda/4$ division so that the diode can be placed symmetric
from $3*\lambda/4$ section. The "MCurves" are connected using matched Tee
sections. To make the diode and LO ports vertical , we used another
$30^{\circ}$ curve before connecting the ports.

Figure 16: Rat-race schematic

Now we connected the matched network and diode equivalent model which are
obtained in the earlier steps. By doing this we can suppress the LO signal.
The diodes one end is connected to the rat-race coupler where as other end is
connected to the ground. The ground is modeled as "VIAGND" component in ADS.
The following figure shows the rat-race hybrid with the diodes.

Figure 17: Rat-race schematic with diodes

From the figure, the diode are placed in opposite directions. We added the
matched network of the diodes before connecting them to the rat-race coupler.
The following figure shows the S-parameters of the rat-race coupler.

Figure 18: S-parameters of the rat-race with diodes

Here the RF is at port1, LO is at port2 and IF at port3. clearly the LO is the
high power signal than RF and results at the IF output. It is required to get
only IF frequency at the IF. So we pass the IF signal with inter-mod products
through the low-pass filter (IF filter) to suppress other frequency
components.

###  II-F IF filter design

From the previous section, it is clear that we have to suppress all other non-
linear inter-mod products expect at IF frequency. So we defined the IF as
below.

Definition: Passband ripple is less than 0.1dB, passband frequency 300 MHz,
stop-band attenuation is less than 80dB, stop-band frequency will contain IM
products of all orders (here we took at 1.2GHz to keep the filter order less
and wide transition band).

In this section, we used the butterworth filter which is maximally flat in the
pass band with order of 6 using filter design smart component. We tuned the
lumped components little bit so that it will be exact match with real
component values using Tuning option in ADS. The following circuit shows the
IF filter used in the design. We used a coupling capacitor for blocking dc
component from IF.

Figure 19: IF filter schematic

We made the physical realization of the each component using their
S-parameters in ADS. For example, the following figure shows the physical
realization of 11nH inductor.

Figure 20: IF filter schematic

The symbol is created for this schematic, so that we can use later. We did the
same for all lumped elements. We used the matched "Tee" sections to connect
these lumped elements. We used the tee section each of width is 43 mils. For
inductors we are using 0603CS package.

Figure 21: IF filter with magic tees

The following figure shows the S-parameters of the IF filter. Clearly from
this figure, the LO and RF signal are attenuated by more than 80dB.

Figure 22: S-parameters of the IF filter

The zoomed version of IF filter is shown below. (till 2.5GHz)

Figure 23: S-parameters of the IF filter

###  II-G Complete Mixer

In this section, we added the rat-race coupler with the IF filter. The
following figure shows the ADS schematic of the rat-race coupler with IF
filter.

Figure 24: Mixer schematic

The following figure shows the IF spectrum of the mixer circuit.

Figure 25: Output products of the mixer circuit from Matlab

From the above figure, it is clear that RF to OF loss is less than 8dB, LO
signal power is given by -75 dBm and RF signal at IF port is at -104 dBm.

Figure 26: Output products of the mixer circuit from ADS

The S-parameters of the mixer circuit is given in the following figures.

Figure 27: RF to IF S-parameters Figure 28: LO to IF S-parameters Figure 29:
LO to RF S-parameters

Clearly, from the S-parameters, the conversion loss is given by -4.43 dB which
is RF to LO, RF to IF @RF frequency is given by -91.2 dB, the LO to IF
isolation is given by -6.241 dB, and the LO to RF isolation is given by -84.92
dB. From this it is clear that the LO matching is not proper at the IF output.
The matched network of the all ports are explained in the later sections.

###  II-H Layout design

In this section, we replace all the schematics with appropriate layout
circuits. The complete layout of the schematic is shown below.

Figure 30: Layout of the mixer circuit

The capacitors used in the design are 0.5 pF, 10 pF, 15 pF and 82 pF (coupling
capacitor $C_{c}$). The inductors used in the design are 11 nH, 30 nH and 33
nH. In the schematic, additional transmission line of 200 mils is added to
make sure we have enough transmission line to place the ports of the
transistors at all the ports. We added viaholes for short circuit. We dig the
holes for short-circuit in the design.

##  III Fabrication

The layout using Gerber file is fabricated using the LPKF machine. We used the
Rogers 4350B substrate with effective resistance of 3.754, thickness is 60mil,
1oz copper (1.4 mil), loss tangent of 0.0031 and conductivity of 5.96e7. We
selected the appropriate settings for the milling. The following figure shows
after fabrication from LPKF machine.

Figure 31: Fabricated Mixer with LPKF machine

###  III-A Lumped components and diodes

We soldered the capacitors, inductors and diodes on the fabricated design. We
added the 3-ports for RF input, LO input and IF output with 3.5mm SMA female
connectors.

The following figure shows after adding lumped elements and diodes.

Figure 32: Mixer after soldering the components

##  IV Measurements

The following are the performance metrics of the mixer.

  1. 1.

Conversion loss: $C_{L}$= $P_{rf}$-$P_{if}$ which is opposite of the
conversion gain ($C_{G}$)

  2. 2.

Isolation of ports: isolation across all ports

  3. 3.

1-dB compression point: impact on switching of RF power

  4. 4.

Noise figure: proportional to Conversion loss

  5. 5.

single tone IM distortion: unwanted harmonics are called IMD

  6. 6.

Multi-tone IM distortion: using 2-tone test,measure IIP3 value.

Out of which here we measured the conversion loss, isolation and IIP3 values
of the designed mixer. The following figure shows the typical setup for mixer
measurements.

Figure 33: Mixer measurement setup

###  IV-A Calibration

Before measuring the loss parameters, it is better to calibrate the cables and
know the cables loss. Before inserting the Mixer in the measurement, perform
the port1 and port2 loss which includes the splitter. We used the Keysight
vector signal generator(s) and signal analyzers for the measurement. The loss
of the splitter along with cable loss is measured as -5.2 dB. The LO cable
loss is 0.93 dB, RF cable loss is 0.94 dB and IF cable loss is 0.1 dB and
splitter loss is 4.16 dB.

###  IV-B Measurement setup for conversion loss

Now insert the mixer in between MXG and VSA. We selected the RF frequency as
2.4GHz with power level of -25 dBm and LO signal of 10 dBm at 2.235 GHz, which
provides the IF value of 165 MHz. we measured the IF value at 165 MHz, which
results as -34.4 dBm provides the conversion loss of 9.3dB. The following
table gives the conversion loss at various power-levels.

RF Pin (dBm) | IF Out Power (dBm)
---|---
-25.1 | -34.4
-15.1 | -24.3
-5.1 | -14.14
4.9 | -4.7
5.9 | -3.8
6.9 | -3.15
7.9 | -2.5
TABLE II: IF Power vs RF Pin

The following figure shows the conversion loss vs RF power. Here we plotted
interpolated line from first RF power to show the conversion loss in increases
as the RF power increases which is nothing but compression at higher RF
powers. The ip1 dB is 7dBm.

Figure 34: Conversion loss vs RF input power (dB)

###  IV-C Measurement setup for Isolation

In this step, we measured the isolation across all ports. The RF to IF
isolation is measured such that apply the tone at RF port and measured the
tone at IF port with RF frequency. Example, apply a tone of -25.1 dBm at RF
input @2.4 GHz and measure the IF @2.4 GHz (in this case, no LO signal is
applied). The RF to IF isolation is measured as 17.6 dB.

RF Pin (dBm) | IF Out Power (dBm @2.4 GHz)
---|---
-25.1 | -47.2
-15.1 | -32.72
TABLE III: RF vs IF isolation

The LO to IF isolation is measured such that apply the tone at LO port and
measured the tone at IF port with LO frequency. Example, apply a tone of 4.9
dBm at LO input @2.235 GHz and measure the IF @2.235 GHz (in this case, no RF
signal is applied). The LO to IF isolation is measured as 20.3 dB.

In the same way, the LO to RF isolation is measured such that apply the tone
at LO port and measured the tone at RF port with LO frequency. Example, apply
a tone of 4.9 dBm at LO input @2.235 GHz and measure the RF port @2.235 GHz
(in this case, IF port is terminated with matched load). The LO to RF
isolation is measured as 6.4 dB.

##  V Non-linear measurements

In this section we perform the non-linear measurements of the designed mixer
using two-tone test. In this section, we discussed mainly about the third
order inter-modulation power. The measurement setup is same as our-previous
lab setup.

###  V-A Losses measured

We measured the total loss with and without splitter. We got the loss due to
each component. The splitter gave the loss of 4.1 dB, the RF cable has loss of
0.94 dB, LO cable has loss of 0.93 dB and IF cable has loss of 0.1 dB. These
losses are same in the above mentioned section.

###  V-B IM3 vs RF power

We used a two tone (2-tone) inputs at same power levels and measured the Pout
at inter-mods using the PXA. For RF input power, we used 2.4 GHz for generator
1 and 2.45 GHz for generator 2. The inter-mod product is observed at 2.5 GHz.
Initially we increased the input power in steps of 5 dBm, but later input
power is increased in steps of 1 dBm. To find the IM3 point, the Pout at f1
and IM3 at 2f2-f1 power levels should meet. We interpolated this data for more
number of power levels.
The Third-order inter-modulations curve is derived based on few observations
and interpolated throughout the Pin (dBm). The following table shows the IM3
powers vs RF input powers.

Pin (dBm) | IM3 Power (dBm)
---|---
-5.1 | -71.7
-0.1 | -56.5
4.9 | –42.2
5.9 | -40.2
6.9 | -38.4
7.9 | -36.7
TABLE IV: IM3 Power vs Pin

The following figure shows IIP3 of the mixer by interpolating the RF input
data, RF output data and IM3 data.

Figure 35: Third-order inter-modulations vs Pin

We observed the OP1dB and IM3 power are intersecting at input power of 23.5
dBm as IIP3 value and 14.1 dBm as OIP3 values.

##  VI Matching of the Mixer

In this section, we found the match of the mixer circuit from all ports using
ADS simulation. After the mixer designed, we did the S-parameters of the RF,
LO and IF ports to find the refections from each of the port. The following
figure shows the return loss of the RF port ($S_{11}$).

Figure 36: Return loss of the RF port

The following figure shows the return loss of the LO port ($S_{22}$).

Figure 37: Return loss of the LO port

The following figure shows the return loss of the IF port ($S_{33}$).

Figure 38: Return loss of the IF port

##  VII CONCLUSIONS

We designed the single balanced mixer from scratch and understood each
component in the design which effects the mixer performance. We fabricated the
mixer using LPKF machine with Rogers 4350B substrate with Effective
permittivity is 3.745. We performed the mixer linear and non-linear
measurements. We understood how to measure the conversion loss of the mixer
using the VSA. We verified the measured values are closely matching with the
data sheet values. This gave us confidence to design and measure any active
component in the next labs. Coming to conversion loss, we measured it which is
9.3 dB. For non-linear measurements, we used two-tone tests, same as with our
previous labs. The compression point and IM3 points are measured and verified
using data sheets, here the IIP3 is 17.9 dBm.

## References

  * [1] HSMS-281x datasheet.pdf.
  * [2] Avago Technologies Application Note 1124.pdf.
  * [3] Class notes and material provided by Professor
  * [4] Johnson Technologies Capacitors (JTI0603SseriesCAPS 2011) datasheet.pdf
  * [5] Coilraft (CCI) RF Library (0603CS inductors) datasheet.pdf
  * [6] Asif Ahmed. A Study and Design of a Rat-Race Coupler Based Microwave Mixer. Journal of Electrical and Electronic Engineering. Vol. 3, No. 5, 2015, pp. 121-126. doi: 10.11648/j.jeee.20150305.14


Copyright for this paper by its authors. Use permitted under Creative Commons
License Attribution 4.0 International (CC BY 4.0).

PASIR’22: First Workshop on Proactive and Agent-Supported Information
Retrieval at CIKM 2022, October 21, 2022, Atlanta, GA

<EMAIL_ADDRESS>]

# Towards Proactive Information Retrieval in Noisy Text with Wikipedia
Concepts

Tabish Ahmed  Centre for Artificial Intelligence, University College London,
UK.  Sahan Bulathwela

(2022)

###### Abstract

Extracting useful information from the user history to clearly understand
informational needs is a crucial feature of a proactive information retrieval
system. Regarding understanding information and relevance, Wikipedia can
provide the background knowledge that an intelligent system needs. This work
explores how exploiting the context of a query using Wikipedia concepts can
improve proactive information retrieval on noisy text. We formulate two models
that use entity linking to associate Wikipedia topics with the relevance
model. Our experiments around a podcast segment retrieval task demonstrate
that there is a clear signal of relevance in Wikipedia concepts while a
ranking model can improve precision by incorporating them. We also find
Wikifying the background context of a query can help disambiguate the meaning
of the query, further helping proactive information retrieval.

##  1 Introduction

The informational needs of people are highly contextual and can depend on many
different factors such as their current knowledge state, interests and goals
[1, 2, 3]. However, an effective information retrieval companion should
minimise the human effort required in i) expressing a human information need
and ii) navigating a lengthy result set. Using topical representations of the
user history (e.g. [4]) can immensely help formulating zero shot queries and
refining short user queries that enable proactive information retrieval (IR).
While the world has digital textual information in abundance, it can often be
noisy (e.g. extracted through Automatic Speech Recognition (ASR), PDF text
extraction etc.), leading to state-of-the-art neural models being highly
sensitive to the noise producing sub-optimal results [5]. This demands
denoising steps to refine both query and document representation.

In this paper, we argue that Wikipedia, an openly available encyclopedia, can
be a humanly intuitive knowledge base [6] that has the potential to provide
the world view many _noisy_ information Retrieval systems need. In the midst
of noisy text, we point out that it can help refine user queries (both
explicit and implicit) while compiling large result sets into structured
narratives that human users can navigate with optimal effort. We use a dataset
with noisy text where the query description is used as a proxy of the user’s
context used in a proactive information retrieval system. We demonstrate that
positive results can be obtained in comparison to relevant baselines when
ranking documents.

##  2 Related Work

Retrieving relevant documents to a personal information need involves mining
textual features that can be used to match them to user queries. While
previous works have explored Wikipedia-based concept modelling for query
expansion [7, 8] exclusively, our work deviates from these works as we focus
on i) simultaneously adding Wiki features to the documents, ii) ranking
documents with noisy texts and iii) using the query description (proxy for
context extracted from user history) for query disambiguation. Latest neural
ranking models dominate the state-of-the-art in IR albeit neural models
struggle in the presence of noise in text [5] making much simpler,
computationally efficient IR models such as BM25, DPH and PL2 much more
suitable for noisy data [9]. Prior evidence has demonstrated the success of
simpler models such as BM25 with the exception of failure on long documents
[10]. However, more sophisticated probabilistic models such as DPH [11],
motivated by the Divergence from Randomness (DFR) framework based on a
hypergeometric distribution and Popper normalisation instead of Laplace
normalization, have outperformed the BM25 model when it comes to noisy text
[9].

This emerging need for retrieving from noisy-text has led to recent
competitions such as the podcast segment retrieval challenge [12] that has
paved way to improving the state-of-the-art. This competition is relevant to
our work as we also tackle proactive IR in noisy text. While the best
performing model within the competition uses a neural approach, it uses token
embeddings rather than document embeddings [13]. The leading model uses a re-
ranking model trained on an orthogonal dataset adding to computational costs
significantly. The next best, Dublin City University model (DCU) uses a
variety of features ranging from text, WordNet based synonyms and entities for
expanding the query [14]. As the DCU model uses automatically extracted
entities (via Named Entity Recognition) to expand the query, we consider this
work as the most relevant model to our proposal. But, we also annotate the
segments with Wikipedia concepts/entities further differentiating our work
from theirs. User queries, inherently are short, averaging two to three key
words in general [15]. Exploiting auxiliary information available to recover
the context of the search is essential in such scenarios. In recent
competitions such as the podcast segment retrieval challenge, the queries used
were accompanied by a small description that provided the contextual
information about the query itself. It was evidenced that the leading models
in this challenge had to use the description to extract features and sometimes
use the identified keywords to make queries to external search engines to
further expand the queries [9]. However, in a real world system, such
information is not available. Hence a proactive IR system has to rely on the
user interactions from the past to build the context [16]. In this work, we
use the query context as a proxy representation extracted by a proactive IR
system.

_Proactive information retrieval_ is a field in IR that gained a lot of
attention recently. Supporting user information queries with minimal/zero
effort will enable users to use IR systems more effectively. However, a key
part of facilitating information retrieval tasks proactively is building
systems that can capture the relevant signal from the user that indicate the
informational needs of them. Prior works in this domain have identified
several ways to harvest user intent. The simplest approach in this direction
is to explicitly ask questions [17] or use the demographic information of the
user [18] to profile them. Rather than explicitly disrupting the user
experience, many works harvest implicit user actions such as clicks,
impressions and previous searches [19, 18]. Especially in the social media
domain, the interactions of users are used to create concept-based models [20]
that can be used to infer user preferences and create zero effort queries for
new content. The extracted preference features are usually used to expand
information queries [8, 7], synthesise zero-effort queries [20] or personalise
search results [21]. In this work, we argue in support of using concept-based
user models as a form of zero-effort query for informational recommendation
and learning scenarios, especially in the presence of noisy document
representations.

###  2.1 Concept-based User Modelling and Wikipedia

In the field of document retrieval and personalisation, keyword extraction is
a heavily used form of identifying _concepts_ from textual documents. When
working with short text such as social media posts, the common method is to
use the words in the posts as words and build a user profile over these words
[22]. While this approach provides fine grained features, the feature space
can be too vast and pose challenges in increasing recall. To address this
issue, other systems use topic detection using unsupervised learning
approaches such as LDA [23]. Such unsupervised techniques are complex to tune
and are not guaranteed to give humanly intuitive topics. While expert
annotation (e.g. in education [24]) has been one of the approaches to obtain
humanly intuitive, representative topics from textual content, this approach
is not scalable. Wikification, a more recent approach, looks promising towards
automatically extracting explainable concepts from text. Wikification
identifies Wikipedia concepts present in a document by connecting natural text
to Wikipedia articles via entity linking [25]. In this work, we argue for the
usefulness of Wikification in creating precise, humanly intuitive concept
representations at scale. Our experiments also demonstrate how Wikipedia
concepts can be used to i) disambiguate query keyword and ii) at the absence
of them, create entity features that can find relevant documents for users
proactively.

##  3 Wikipedia to Support Proactive Search-based Systems

Through this work, we aim to explore how the knowledge contained in Wikipedia
can be utilised to better a proactive search-based system. In order to tread
towards this direction, we aim to answer multiple research questions in this
work as a promising step. We hypothesise that associating Wikipedia concepts
to a natural language based IR system can improve a proactive IR system by i)
bettering disambiguation of the meaning of user queries, ii) by exploiting the
semantic meaning of document words while being less sensitive to noisy words
in the document transcripts and iii) by creating opportunities for the system
to understand the learner interactions more meaningfully.

RQ1

Do Wikipedia concepts carry a signal that indicates relevance of documents to
queries?

RQ2

Can Wikipedia annotations improve noisy information retrieval?

Through this work, we conduct a few preliminary experiments to validate these
research questions. We choose the segment retrieval task from the Podcast
Track at TREC 2020 [12] for these experiments as ”search” is a key component
of proactive information retrieval. We use this task specifically as the query
description is provided as part of the query, providing the context of the
query. In many proactive IR systems, the query context is mined using the user
interaction history. In this case, the description can be treated as a context
representation mined from the users’ prior interactions.

###  3.1 Information Retrieval Models

We restrict our model choices to a probabilistic retrieval method BM25 and a
hypergeometric weighing model DPH. The reasons for restricting to
probabilistic models are i) its superior performance in noisy IR [9], ii) the
simplicity to introducing Wikipedia concept features, iii) the data and
computational efficiency in training models and iv) the practical feasibility
to real world systems. The models we utilise in this these experiments use the
query $q$, query description $d$ and the segment $s$.

####  3.1.1 Baseline Models

We use BM25 and DPH models as probabilistic baseline models for this work.
These two models use the textual tokens of the query $q$ and the segment $s$
to compute the relevance score $rel$ as per equation 1 where txt identifies
the text token features..

$\centering rel(q,s)=f(q_{\texttt{txt}},s_{\texttt{txt}})\@add@centering$ (1)

As described in section 2, we implement and test the DCU run 2 model [14]
(referred to as DCU model hereafter) that expands the query with entities
extracted from the description as per equation 2 that expands equation 1. The
entities are extracted using Named Entity Recognition (NER).

$\centering
rel(q,{\color[rgb]{0,0,1}d},s)=f(q_{\texttt{txt}}{\color[rgb]{0,0,1}+d_{\texttt{ent}}},s_{\texttt{txt}})\@add@centering$
(2)

where ent represents the entities extracted from the query description $d$.

###  3.2 Proposed Models

The proposed models in this work uses Wikipedia concepts to enrich the
representations used in the IR model. We hypothesise that adding Wikipedia
concepts can improve precision of the model as exact entities can be matched
between the query + context and the segments. We propose two models that use
Wikipedia topics both in query and segment expansion.

##### Wiki_rel

, This model expands the query and the document by adding the Wikipedia
concepts extracted using both the query $q$ and the query description $d$ as
per equation 4. This model replaces the entities in the DCU model by Wikipedia
concepts.

$\centering rel(q,d,s)=f(q_{\texttt{txt}}{\color[rgb]{1,.5,0}+\
q_{\texttt{wiki}}}{\color[rgb]{1,.5,0}+\
d_{\texttt{wiki}}},s_{\texttt{txt}}{\color[rgb]{1,.5,0}+\
s_{\texttt{wiki}}})\@add@centering$ (3)

where $\cdot_{\texttt{wiki}}$ represents the concepts extracted from the query
$q$, description $d$ and segment $s$ .

##### Ent_Wiki_rel

, This model expands the DCU model by enriching the query, description and the
segment with Wikipedia concepts. This formulation is presented in equation 4.

$\centering
rel(q,d,s)=f(q_{\texttt{txt}}{\color[rgb]{0,0,1}+d_{\texttt{ent}}}{\color[rgb]{1,.5,0}+\
q_{\texttt{wiki}}}{\color[rgb]{1,.5,0}+\
d_{\texttt{wiki}}},s_{\texttt{txt}}{\color[rgb]{1,.5,0}+\
s_{\texttt{wiki}}})\@add@centering$ (4)

The two proposed models help us validate RQ2 by i) Wiki_rel model validating
if _replacing_ named-entities with Wikipedia features improves the ranking
models and ii) Wiki_Ent_rel model validating if _adding_ Wikipedia features
improves the model.

###  3.3 Data

As per section 3, we use the Podcast segment retrieval task to demonstrate
usefulness of Wikipedia features in proactive information retrieval. The
Spotify podcast Dataset contains approx 100,000 transcripts from around 60,000
hours of audio data culminating in the largest corpus of transcribed speech
data. The episodes in the dataset were randomly sampled from 105,360 English
podcast episodes published between January 1, 2019 to March 1, 2020 on Spotify
with 10% of the podcasts from professional creators with high production
values and the other 90% coming from amateurs [26]. The data is also provided
with training and testing topics (queries) along with human relevance
judgements. There are 8 topics in the training dataset and another 50 topics
in the test set.

To keep the computational costs low, we run the experiments outlined in
section 3.4 exclusively using the training data in the podcast dataset. We
first transform the dataset into two minute overlapping segments as described
in [9]. Then, we take all the relevant segments as positive documents for the
different queries. As there is a substantial number of segments that are
irrelevant (as the whole dataset contains 3.5 Mn segments), we down sample
this data and get a negative segment set of $\approx 14000$ segments that are
not relevant to any of the labelled queries. This set of positive and negative
segments make up the smaller scale dataset that we use in our empirical
experiments. We refer to this dataset as Podcast Small dataset. We use the
Wikifier [25] to associate Wikipedia concepts to both the queries (query +
description) and the podcast segments as the models in section 3.1 require.

###  3.4 Experiments

The empirical experiments we run in this work aim to answer RQ 1 and 2. We use
the Podcast Small dataset for these experiments. In a proactive information
retrieval setting, the user queries are inferred from users’ historical
interactions. We treat the query description provided in the dataset in the
place of the features extracted from the user history. We hypothesise that the
query description, can be used in two ways, i) as the user context that can be
used in a zero effort query to make up the context and rank relevant documents
from the corpus and/or ii) as a mean to disambiguate the true meaning of a
short query provided by the user without prompting the user to provide further
clarifications.

To answer RQ1, we aim to see if there is a significant alignment between the
Wikipedia concepts present in the query and a relevant document in contrast to
a non-relevant document. To measure this quantitatively, we measure the
Jaccard similarity coefficient $\rho$ between the query and the document using
the sets of Wikipedia concepts. Then, for each query, we take the sets of
values $\rho_{\text{rel}}$ and $\rho_{\text{non rel}}$ and compare the
difference of medians using a one-tailed Mann–Whitney U test
($H_{alt.}:\rho_{\text{rel}}>\rho_{\text{non rel}}$).

To answer RQ2, we take the annotated Wikipedia concepts of the query and the
podcast segments as additional features. Then we develop a new set of
information retrieval algorithms outlined in section 3.1 to account for the
alignment of Wikipedia concepts present. Then we use the same Podcast Small
dataset to evaluate if the ranking of relevant documents is different between
the baselines and the proposed models. To validate how Wikifying the context
can be used to disambiguate a query, we ran Wikification on each query using
i) the query words exclusively and ii) the query words + the description
words, to observe if more precise identification of Wikipedia concepts is
carried out.

In order to run the evaluation experiments needed for RQ2, we utilise Python
Terrier library [27]. The text processing and NER enrichment needed for the
DCU model is done using the Spacy library. The Wikipedia annotations are
sourced using the Wikifier service [25].

####  3.4.1 Evaluation

To evaluate if the Jaccard similarity coefficients in relation to RQ1 are
significantly different, we use the test statistic of the hypothesis test. We
present the performance of the ranking models related to validating RQ2 using
NDCG, NDCG@30 and Precision@10 as these are the same evaluation metrics used
in prior work [9]. NDCG uses graded relevance instead of binary relevance/non-
relevance to focus on highly relevant documents, implying that retrieving a
highly relevant document is much more important than retrieving a less
important document. Whereas precision measures the number of relevant
documents retrieved over the total relevant documents. Similar to the prior
work, we calculate NDCG for the entire set of relevant segments available and
up-till rank 30. Precision is computed by using a cut-off of top-10 ranked
segments.

##  4 Experimental Results

The results of the experiments are reported in this section. Figure 1 shows
the difference of medians of Jaccard similarity coefficients, while Table 1
summarises the results obtained from the hypothesis test conducted to verify
RQ1. Table 2 outlines the ranking performance obtained in the experiments run
in order to verify RQ2.

Figure 1: Jaccard similarity distribution of relevant vs. non-relevant segments for each query in the podcast dataset. Table 1: The difference of medians test for each query reported with the statistical significance. | Number of Documents | Median Jaccard Similarity | MannWhitney U test
---|---|---|---
Query ID | Relevant | Non-Relevant | Relevant | Non-Relevant | Difference | Test Statistic | p value
1 | 70 | 14109 | 0.027 | 0.007 | 0.020 | 25680 | 1.03E-49
2 | 63 | 14116 | 0.023 | 0.011 | 0.012 | 47466 | 8.84E-46
3 | 78 | 14101 | 0.043 | 0.023 | 0.019 | 264688 | 1.22E-16
4 | 78 | 14101 | 0.034 | 0.027 | 0.007 | 393033 | 1.41E-06
5 | 80 | 14099 | 0.039 | 0.029 | 0.010 | 455015 | 1.42E-03
6 | 37 | 14142 | 0.027 | 0.010 | 0.017 | 35748 | 8.37E-48
7 | 77 | 14102 | 0.016 | 0.011 | 0.005 | 56322 | 2.80E-44
8 | 80 | 14099 | 0.032 | 0.023 | 0.010 | 271961 | 6.31E-16
Table 2: Ranking performance of the models. The best performance in bold and the second best in italic face. | Features | Metrics
---|---|---
Model | Query | NER | Wiki | NDCG | NDCG at 30 | Precision at 10
| Text | Entities | Concepts |  |  |
_Baselines_
DPH | $\surd$ | $\times$ | $\times$ | 0.48 | 0.30 | 0.32
BM25 | $\surd$ | $\times$ | $\times$ | 0.48 | 0.28 | 0.31
DCU | $\surd$ | $\surd$ | $\times$ | 0.51 | 0.32 | 0.30
_New Proposals_
Wiki_rel | $\surd$ | $\times$ | $\surd$ | 0.49 | 0.29 | 0.31
Ent_Wiki_rel | $\surd$ | $\surd$ | $\surd$ | 0.51 | 0.30 | 0.36

###  4.1 Query Keyword Disambiguation using the Context

We hyptothesise that the context gathered by the user can also be used to
disambiguate the meaning of keywords in a search query. To test this, we take
7 queries from the training dataset that belongs to categories _topical_ and
_known item_ (”story about riding a bird” is ignored as it is a query that
belongs to the _refinding_ category). We enrich these queries using
Wikification in two settings, i) query only vs. ii) query + description, to
investigate if the identified Wiki concepts are different. Table 3 reports how
the salient entities in the query keywords are detected differently in the two
settings.

Table 3: The Quality of Query Word Disambiguation/Refinement when using the query exclusively vs. when using the description. Query | Query Only | Query + Description
---|---|---
coronavirus spread | wiki/Coronavirus | wiki/Novel_coronavirus
greta thunberg cross atlantic | - | wiki/Greta_Thunberg
black hole image | wiki/Black_hole | wiki/Black_hole
daniel ek interview | - | wiki/Daniel_Ek
michelle obama becoming | - | wiki/Michelle_Obama
| wiki/Becoming_(philosophy) | wiki/Becoming_(book)
anna delvey | wiki/Indian_anna | wiki/Anna_Sorokin
facebook stock prediction | wiki/Facebook | wiki/Facebook
| wiki/Stock | wiki/Stock

##  5 Discussion

The results obtained in the preliminary experiments conducted are very
promising. Majority of the results suggest that Wikipedia concepts can help
the IR system.

###  5.1 The Usefulness of Wikipedia Features (RQ1)

Figure 1 clearly shows that the Jaccard similarity coefficient between the
relevant segments $\rho_{\text{rel}}$ and the non-relevant documents
$\rho_{\text{non rel}}$ are different from each other. It is even evident that
the confidence intervals don’t even overlap in some of the queries in the
training data. The test statistics presented in table 1 further confirms this
observation. In a one-tailed hypothesis test, all the queries demonstrate
statistically significant differences of medians where the median Jaccard
similarity of the Wikipedia concepts of the relevant segments and the query is
greater than that of the non-relevant segments. These results enable us to
conclude that the the overlap of Wikipedia concepts between queries and
relevant documents is an influential signal that can be incorporated in
relevance computation. The results indicate that there is promise in using
Wikipedia concepts as a feature in IR systems where noise is present.

###  5.2 Utility of Wikification in Information Retrieval (RQ2)

The usefulness of Wikipedia concepts in relevance prediction is validated in
the results presented in table 2. The results in the table show that replacing
the NER-based entities with Wikipedia concepts is not as fruitful as
augmenting them while the entities are retained. It is observed that both the
proposed models outperform the entity based DCU model in precision at 10. This
aligns with the our understanding of the effect of Wikipedia concepts. NER,
while detecting specific persons, organisations and other entities in the
text, only adds the textual representations of these entities to the query to
expand it. Therefore, the newly added queries are not capable of entity
disambiguation (when the same term occurs in a segment meaning something else.
e.g. Apple, the fruit vs. the company) and coreference resolution (when a
different textual form is used to mean the same entity. e.g. United Nations
vs. UN). On the contrary, enriching both the query and the document with
Wikipedia entities enable accounting for both of these effects, making the
relevance calculation more precise. The _Ent_Wiki_rel_ model that contains
both the NER-based entities and Wikipedia concepts, manages to improve the
precision at 10 while retaining the NDCG performance. This shows that keeping
the entities in the query is important to retain the NDCG gains found in the
DCU model.

In this scenario, we use the Wikipedia concepts and entities extracted from
the description to improve the search ranking. While we treat the description
in this scenario as a representation extracted from the user history, many
concept-based user models described in section 2 support extracting features
such as keywords [21] and Wikipedia concepts [1] from content.

##### Query Keyword Disambiguation

The results in table 3 also give us insights into the potential of using query
context for keyword disambiguation. It is seen that in multiple queries,
certain concepts were not even detected when the query was used exclusively
for Wikificaiton (e.g. Greta Thunberg, Michelle Obama etc.). However, with the
presence of context (that can be extracted from historic encounter of concepts
of a user [4]), the algorithm managed to identify these concepts in the query
increasing recall. Table 3 also shows instances where the precision of
identification is improved in the presence of the context. The more precise
_Novel Coronavirus_ was detected in the first query while the system
understood that the query was about Michelle Obama’s book, _Becoming_ in the
fifth query, not the philosophical concept. These results show how many
scenarios where short query keywords can be clarified automatically by
referring to the context rather than needlessly prompting the human user.

###  5.3 Wikipedia Concepts and the User Experience

The previous section discussed one opportunity of Wikification, automatic
query word disambiguation. Such features will substantially decrease the
number of user prompts in a system leading to a smoother experience. Wikipedia
concepts also bring the advantage of being grounded in a taxonomy that is
intuitive to humans. The human intuitive nature of Wikipedia concepts allows
the system to provide more transparent result sets that the user can navigate
more efficiently. Figure 2 shows two recent systems demonstrating the
potential of using Wikipedia based representations to improve human-in-the-
loop information retrieval.

Figure 2: (i) X5Learn [28] and IFacetSum [29] showing that using Wikipedia
concepts to represent documents can support proactive information retrieval by
summarising the key concepts/entities in the result collection, helping the
user to make more informed decisions on their next steps.

The left system, X5Learn [28] shows how different Wikipedia concepts are
related to different segments of videos. The Jaccard-based relevance can be
used to visualise the degree of concept overlap between the segments and the
query indicating relevance (connecting to the hypothesis of RQ1). Also, using
Wikipedia concepts rather than free-form keywords or NER-based entities allows
the system to provide explanations of the concepts users encounter. This can
be done using the knowledge base behind the taxonomy (in this case, Wikipedia
as per figure 2 (left)) Many users seeking information do not fully understand
the knowledge landscape they navigate and can use learning opportunities to
get more familiar with the topics.

IFacetSum, a multi-faceted mix-initiative interface shown in figure 2(right)
is an example of how the result sets can be summarised using the entities and
concepts in the document. As shown in the demonstration, the detected entities
and concepts allow the user to understand the contents of the relevant
documents quickly. Their user study also found that the users didn’t utilise
the statements and summaries as much as they used the concepts and entities
[29]. However, this system lacks the capabilities of explaining the topics as
it is not backed by a knowledge base pointing to the advantages of using a
taxonomy that has auxiliary information that can be used. In a nutshell, many
recent visualisation approaches use concepts to represent documents. Having
Wikipedia as a topic taxonomy enables these systems to make use of the
advancements of Wikification that is advancing rapidly at present. It also
allows taking use of other information in Wikipedia (e.g. semantic relatedness
between topics, hierarchy of concepts etc.) to model the concepts in a much
more sophisticated manner. This in turn, helps bridging the system seamlessly
to the end user, creating cleaner interaction footprints that transform into
the input for proactive search.

##  6 Conclusion

Through this work, we explore the potential of using Wikipedia concepts to
drive the proactive search experience of an IR system working on noisy
documents. We explored the usefulness of Wikipedia concepts focusing on 2
research questions. We treated the query description in a podcast segment
retrieval task dataset as the user context in a proactive IR system and ran a
series of experiments. Our first experiment showed that the overlap of
Wikipedia concepts between the relevant segments and the query are
significantly larger than with the non-relevant segments. A subsequent
experiment that added Wikipedia concept features to the search query and the
document showed that the relevance model that uses Wikipedia concepts can
improve precision of the ranking model while retaining the NDCG score in
comparison to a counterpart that uses entities instead of Wikipedia concepts.
A deeper analysis on the Wikification of the queries also showed that the
meaning of the query keywords can be enhanced by using the query context. We
also demonstrated using two recent works on how Wikipedia taxonomy can help an
IR system to better connect with the end user and enable them to carry out
structured IR tasks by proactively helping them navigate the information in
the result sets. It is seen that having Wikipedia concepts allows presenting
the result set in a humanly intuitive form that encourages the user to
efficiently refine their query in the subsequent steps. All the observations
from the analysed systems indicate that Wikipedia concepts have a key role to
play in creating proactive information retrieval systems that can facilitate a
higher degree human-in-the-loop operation. In the future work, we aim to use
more sophisticated representations of Wikipedia concepts and advanced
relevance models to run large scale experiments on the full podcast dataset
with 3.5 Mn segments. Understanding how to use Wikipedia concepts effectively
(e.g. by ranking them) is a top priority.

###### Acknowledgements.

This research was partially conducted as part of the X5GON project funded from
the EU’s Horizon 2020 research programme grant No 761758. This work is also
supported by the European Commission-funded project “Humane AI: Toward AI
Systems That Augment and Empower Humans by Understanding Us, our Society and
the World Around Us” (grant 820437), EU Erasmus+ project ”European Network for
Catalysing Open Resources in Education” (project ref: 621586-EPP-1-2020-1-NO-
EPPKA2-KA), and the AT2030 Programme. The AT2030 programme is funded by aid
from the UK government and led by the Global Disability Innovation Hub.

## References

  * Bulathwela et al. [2020] S. Bulathwela, M. Pérez-Ortiz, E. Yilmaz, J. Shawe-Taylor,  Towards an integrative educational recommender for lifelong learners,  in: AAAI Conference on Artificial Intelligence, AAAI 20, 2020.
  * Jiang et al. [2019] W. Jiang, Z. A. Pardos, Q. Wei,  Goal-based course recommendation,  in: Proceedings of International Conference on Learning Analytics & Knowledge, 2019.
  * Bulathwela et al. [2022] S. Bulathwela, M. Pérez-Ortiz, E. Yilmaz, J. Shawe-Taylor,  Power to the learner: Towards human-intuitive and integrative recommendations with open educational resources,  Sustainability 14 (2022) 11682.
  * Bulathwela et al. [2020] S. Bulathwela, M. Pérez-Ortiz, E. Yilmaz, J. Shawe-Taylor,  Truelearn: A family of bayesian algorithms to match lifelong learners to open educational resources,  in: AAAI Conference on Artificial Intelligence, AAAI 20, 2020.
  * Sidiropoulos et al. [2022] G. Sidiropoulos, S. Vakulenko, E. Kanoulas,  On the impact of speech recognition errors in passage retrieval for spoken question answering,  arXiv preprint arXiv:2209.12944 (2022).
  * Bulathwela et al. [2021] S. Bulathwela, M. Pérez-Ortiz, C. Halloway, J. Shawe-Taylor,  Could ai democratise education? socio-technical imaginaries of an edtech revolution,  in: In Proc. of the NeurIPS Workshop on Machine Learning for the Developing World (ML4D), 2021.
  * Azad and Deepak [2019] H. K. Azad, A. Deepak,  A new approach for query expansion using wikipedia and wordnet,  Information sciences 492 (2019) 147–163.
  * Nasir et al. [2019] J. A. Nasir, I. Varlamis, S. Ishfaq,  A knowledge-based semantic framework for query expansion,  Information processing & management 56 (2019) 1605–1617.
  * Jones et al. [2021] R. Jones, B. Carterette, A. Clifton, M. Eskevich, G. J. Jones, J. Karlgren, A. Pappu, S. Reddy, Y. Yu,  Trec 2020 podcasts track overview,  arXiv preprint arXiv:2103.15953 (2021).
  * Lv and Zhai [2011] Y. Lv, C. Zhai,  When documents are very long, bm25 fails!,  in: Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval, 2011, pp. 1103–1104.
  * Amati [2006] G. Amati,  Frequentist and bayesian approach to information retrieval,  in: European Conference on Information Retrieval, Springer, 2006, pp. 13–24.
  * Yu et al. [2020] Y. Yu, J. Karlgren, H. Bonab, A. Clifton, M. I. Tanveer, R. Jones,  Spotify at the trec 2020 podcasts track: Segment retrieval,  in: Proceedings of the Twenty-Ninth Text REtrieval Conference (TREC 2020), 2020.
  * Galušcáková et al. [2020] P. Galušcáková, S. Nair, D. W. Oard,  Combine and re-rank: The university of maryland at the trec 2020 podcasts track (2020).
  * Moriya and Jones [2020] Y. Moriya, G. J. Jones,  Dcu-adapt at the trec 2020 podcasts track.,  in: TREC, 2020.
  * Arampatzis and Kamps [2008] A. Arampatzis, J. Kamps,  A study of query length,  in: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, 2008, pp. 811–812.
  * Sen [2021] P. Sen, Proactive information retrieval, Ph.D. thesis, Dublin City University, 2021.
  * Torbati et al. [2020] G. H. Torbati, A. Yates, G. Weikum,  Personalized entity search by sparse and scrutable user profiles,  in: Proceedings of the 2020 Conference on Human Information Interaction and Retrieval, 2020, pp. 427–431.
  * Yang et al. [2016] L. Yang, Q. Guo, Y. Song, S. Meng, M. Shokouhi, K. McDonald, W. B. Croft,  Modeling user interests for zero-query ranking,  in: European Conference on Information Retrieval, Springer, 2016, pp. 171–184.
  * Bulathwela et al. [2021] S. Bulathwela, M. Perez-Ortiz, E. Novak, E. Yilmaz, J. Shawe-Taylor,  Peek: A large dataset of learner engagement with educational videos,  in: Proc. of RecSys Workshop on Online Recommender Systems and User Modeling (ORSUM’21), 2021. URL: https://arxiv.org/abs/2109.03154.
  * Zarrinkalam et al. [2020] F. Zarrinkalam, S. Faralli, G. Piao, E. Bagheri,  Extracting, mining and predicting users’ interests from social media (2020).
  * Syed and Collins-Thompson [2017] R. Syed, K. Collins-Thompson,  Retrieval algorithms optimized for human learning,  in: Proc. of Int. Conf. on Research and Development in Information Retrieval (SIGIR), 2017.
  * Piao and Breslin [2016] G. Piao, J. G. Breslin,  Analyzing mooc entries of professionals on linkedin for user modeling and personalized mooc recommendations,  in: Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization, UMAP ’16, 2016.
  * Zarrinkalam et al. [2017] F. Zarrinkalam, H. Fani, E. Bagheri, M. Kahani,  Predicting users’ future interests on twitter,  in: European Conference on Information Retrieval, Springer, 2017, pp. 464–476.
  * Corbett and Anderson [1994] A. T. Corbett, J. R. Anderson,  Knowledge tracing: Modeling the acquisition of procedural knowledge,  User Modeling and User-Adapted Interaction 4 (1994).
  * Brank et al. [2017] J. Brank, G. Leban, M. Grobelnik,  Annotating documents with relevant wikipedia concepts,  in: Proc. of Slovenian KDD Conf. on Data Mining and Data Warehouses (SiKDD), 2017.
  * Clifton et al. [2020] A. Clifton, A. Pappu, S. Reddy, Y. Yu, J. Karlgren, B. Carterette, R. Jones,  The spotify podcast dataset,  arXiv preprint arXiv:2004.04270 (2020).
  * Macdonald and Tonellotto [2020] C. Macdonald, N. Tonellotto,  Declarative experimentation in information retrieval using PyTerrier,  in: Proceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval, ACM, 2020. doi:10.1145/3409256.3409829.
  * Bulathwela et al. [2020] S. Bulathwela, S. Kreitmayer, M. Pérez-Ortiz,  What’s in it for me? augmenting recommended learning resources with navigable annotations,  in: Proceedings of the 25th International Conference on Intelligent User Interfaces Companion, IUI 20, 2020\.
  * Hirsch et al. [2021] E. Hirsch, A. Eirew, O. Shapira, A. Caciularu, A. Cattan, O. Ernst, R. Pasunuru, H. Ronen, M. Bansal, I. Dagan,  iFacetSum: Coreference-based interactive faceted summarization for multi-document exploration,  in: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, 2021, pp. 283–297. URL: https://aclanthology.org/2021.emnlp-demo.33. doi:10.18653/v1/2021.emnlp-demo.33.