# Simulating Users in Interactive Web Table Retrieval

Björn Engelmann  TH Köln - University of Applied SciencesGermany
<EMAIL_ADDRESS>,  Timo Breuer  TH Köln - University of Applied
SciencesGermany<EMAIL_ADDRESS>and  Philipp Schaer  TH Köln -
University of Applied SciencesGermany<EMAIL_ADDRESS>
(2023)

###### Abstract.

Considering the multimodal signals of search items is beneficial for retrieval
effectiveness. Especially in web table retrieval (WTR) experiments, accounting
for multimodal properties of tables boosts effectiveness. However, it still
remains an open question how the single modalities affect user experience in
particular. Previous work analyzed WTR performance in ad-hoc retrieval
benchmarks, which neglects interactive search behavior and limits the
conclusion about the implications for real-world user environments.

To this end, this work presents an in-depth evaluation of simulated
interactive WTR search sessions as a more cost-efficient and reproducible
alternative to real user studies. As a first of its kind, we introduce
interactive query reformulation strategies based on Doc2Query, incorporating
cognitive states of simulated user knowledge. Our evaluations include two
perspectives on user effectiveness by considering different cost paradigms,
namely query-wise and time-oriented measures of effort. Our multi-perspective
evaluation scheme reveals new insights about query strategies, the impact of
modalities, and different user types in simulated WTR search sessions.

User Simulation, Interactive Table Retrieval, Multimodality, Query Generation

††journalyear: 2023††copyright: acmlicensed††conference: Proceedings of the
32nd ACM International Conference on Information and Knowledge Management;
October 21–25, 2023; Birmingham, United Kingdom††booktitle: Proceedings of the
32nd ACM International Conference on Information and Knowledge Management
(CIKM ’23), October 21–25, 2023, Birmingham, United Kingdom††price:
15.00††doi: 10.1145/3583780.3615187††isbn: 979-8-4007-0124-5/23/10††ccs:
Information systems Users and interactive retrieval††ccs: Information systems
Query reformulation††ccs: Human-centered computing

##  1\. Introduction

Web table retrieval is defined as finding relevant tables from a corpus for a
given information need (typically expressed as a query). This task is closely
related to the classical retrieval of documents or passages. For tables
extracted from websites, however, the modalities, i.e., the contextual
information, are of great importance. This can be, e.g., the page title from
which the table originates. While user simulation is an established practice
in classical retrieval (Carterette et al., 2015a; Zhang et al., 2017a; Balog
et al., 2022), there is no research on web table retrieval so far. However,
this is particularly interesting since the impact of table modalities on users
needs to be clarified. In this paper, we combine insights from user simulation
and table retrieval on the one hand and present new approaches to make
interactive retrieval dynamic on the other hand. This dynamic component is
based on the growing knowledge state of a user, where new queries depend on
examined documents.

In sum, our contributions are as follows. We present a 1) query generation
approach based on Doc2Query for interactive user simulation. This is its first
use for this kind of user simulation. Furthermore, we provide 2) an in-depth
investigation of modality effects for table retrieval. We analyze the
information gain over an interactive course by modeling user behavior in
various fashions. In our evaluation, we 3) contrast two cost paradigms. The
information gain is evaluated on the one hand — classically throughout
successive queries — and, on the other hand, depending on the effort invested.
Thus, we gain realistic insights into our user sessions and a comprehensive
picture of the simulation. This work is also the first study of its kind to
run 4) an extensive simulation on an entire test collection, including 60
topics. We also 5) make all resources used publicly
available.111https://github.com/irgroup/simiir-wtr These include query
variants of Doc2Query and by GPT-3.5 for every topic in the dataset, and the
source code used.

##  2\. Related Work

Many approaches in WTR have been presented, from classical retrieval methods
based on term matching (Yan et al., 2017), mapping tables or queries into
semantic spaces (Zhang and Balog, 2019), to modern techniques based on large
language models (Trabelsi et al., 2022). Furthermore, it was investigated how
the structure of the table and the context in which it appears can be
exploited to improve the document representation for retrieval (Wang et al.,
2015; Shraga et al., 2020). Different datasets were used to compare these
approaches. The Wiki Tables Test Corpus consists of 1.6M tables extracted from
Wikipedia, for which 60 queries are available, along with relevance
assessments (Zhang and Balog, 2018). An extension is presented with the Web
Table Retrieval Test collection (Chen et al., 2021). Here, tables, with their
context, are added from the Common Crawl dataset. Tables are thus present in
different modalities, such as page titles, entities, text before/after, and
the table itself. Modality relevance assessments were created by
crowdsourcing.

Recently, Zobel (Zobel, 2022) highlighted the “gap” between system/oriented
measures and real user effectiveness. System-oriented measures are proxies
that do not necessarily reflect the actual user effectiveness in real search
situations. User variance is at least as important as system variance in
retrieval evaluations (Bailey et al., 2015). For this reason, user simulations
provide a promising solution to draw better conclusions about real-world
implications in a cost-efficient way (Balog et al., 2021).

There exist several simulation framework modeling user behavior that generally
agree on a core sequence of user interactions shared by all models (Thomas et
al., 2014; Maxwell and Azzopardi, 2016b; Pääkkönen et al., 2017; Carterette et
al., 2015b; Zhang et al., 2017b). The user interaction sequence includes the
1) query formulation induced by the (topical) information need, 2) scanning of
the retrieved list that is usually done by screening snippet texts, 3)
selecting and clicking appealing items, 4) reading documents and judging about
their relevance, and finally, 5) inspecting other items in the result list and
deciding about query reformulations or 6) abandoning the search session
entirely.

There exist different ways to simulate queries if no real user logs are
available (Azzopardi et al., 2007; Günther and Hagen, 2021; Baskaya et al.,
2013; Penha et al., 2022). For instance, by generating queries from topic
texts with principled rules (Baskaya et al., 2013), making use of language
models based on relevant documents to simulate queries for known-item search
(Jordan et al., 2006; Azzopardi et al., 2007), using query suggestions by
Google (Günther and Hagen, 2021), or fine-tuning Large Language Models with
the help of topic texts and keyword queries as targets to generate new queries
at inference time (Penha et al., 2022). There are different ways to model
browsing the snippets and making click decisions, e.g., by using editorial
relevance labels to simulate click decisions based on probabilistic modeling
(Hofmann et al., 2013) or parameterizing click models with the help of click
logs (Chuklin et al., 2015). Likewise, it is also possible to make click
decisions based on language modeling approaches (Maxwell and Azzopardi,
2016a). In a similar way, relevance decisions about the entire document can be
based on probabilistic modeling with editorial relevance labels or with
language models. In this regard, it is also possible to incorporate reading
time (Smucker and Clarke, 2012) and simulate different types of stopping
behavior (Maxwell and Azzopardi, 2018).

##  3\. Methodology

Our approach focuses on user simulation and a feedback procedure. Since our
analysis is concerned with tracking information gained over a sequential
interactive course, we use a classical term matching-based approach, BM25,
instead of neural methods that would maximize the effectiveness of ad-hoc
retrieval. BM25 has already been used in various works for table retrieval and
provides a well-established foundation. We use the WTR dataset (Chen et al.,
2021) and index across all modalities in our work. Both indexing and multi-
field retrieval are implemented using PyTerrier (Macdonald et al., 2021). For
retrieval, we follow the methodology of (Chen et al., 2021). Regarding the
user simulation, we make use of the simulation toolkit SIMIIR 2.0 (Maxwell and
Azzopardi, 2016b; Zerhoudi et al., 2022).

We ground our user simulations on an adaption of the Complex Searcher Model
(CSM) (Maxwell and Azzopardi, 2016b) that follows the interaction sequences
outlined in the previous Section 2. CSM implicitly assumes that the simulated
users scan result lists and make click decisions based on the attractiveness
of the search items’ titles and snippets before judging about their relevance.
In our experiments, we replace snippets with the different modalities that
represent the table in the result list. If not specified otherwise, we assume
that the users browse result lists with ten tables per search engine result
page (SERP), click on a table if its modality is relevant, use the page title
as the default modality, and the single table only adds up to the total
information gain if it was not seen before. Most importantly, we put a special
focus on the analysis of different modalities and how earlier seen tables
affect generating query reformulations.

###  3.1. Query simulations

As the WTR dataset by Chen et al. does not provide any topic-related
information besides the keyword-based queries, it is challenging to simulate
queries without additional context information about the topics. Thus, we
implement a query generation strategy that makes use of instruction-tuned LLMs
and principled prompting. More specifically, we prompt the LLM to generate
keyword queries with topic-adapted instructions. In our experiments, we query
OpenAI’s API and parse the outputs of GPT-3.5 (more specifically,
gpt-3.5-turbo-0301 (OpenAI, 2023)) based on the adaption of the following
prompting template: Please generate 100 keyword queries about <query>., where
<query> is replaced with the query string of the particular topic. In total,
we generate companion datasets with 100 query variants for each of the 60
topics that are publicly shared with the community for follow-up studies. In
our simulation experiments, we treat the query sequence made by the LLM as
query reformulations. Following this approach, we intend to generate queries
that are topically related but do not consider the context of earlier seen
search results.

Doc2Query Query generation with GPT-3.5 has the disadvantage that a fixed
number of queries must be generated in advance, which do not refer to the
domain-specific language of the data set and are independent of the search
results. For this reason, we present an approach that generates new queries
based on a list of retrieval results (similar to pseudo relevance feedback).
This approach is particularly suitable for user simulations in interactive
information retrieval since each search iteration integrates the user’s newly
acquired knowledge. We model this growing knowledge using keywords generated
from seen tables. We generate these keywords using the Doc2Query approach
(Nogueira et al., 2019). Here, queries are predicted for a record (tables in
our case) that are possible questions answered by the document. The knowledge
state $KS_{i}$ of a user after the $i$-th iteration (i.e., $i$-th query) is
defined by the union of the terms that were generated from the seen tables
using Doc2Query (Equation 1).

(1) $KS_{i}=\bigcup_{j\in\\{1,\dots,i\\}}\phi(\theta(Q_{j})).$ (2)
$\phi(D)=\bigcup_{d\in D}\\{t\in d2q(d)\,|\,idf(t)<0.5\land t\notin S\\}.$

$\theta(Q)$ is the set of documents returned by a retrieval system for a given
query $Q$. We filter all stop words and terms whose inverse document frequency
(idf) is less than 0.5. The function $d2q(d)$ returns a set of terms retrieved
from document $d$ using the Doc2Query approach. These query terms are then
used for subsequent iterations by adding a term from the knowledge state to
the initial query $Q_{0}$. In this way, query variations are created that
differ only in one term, are dependent on the search results, and represent
the simulated user’s knowledge.

Feedback based Doc2Query Since the query simulation strategy is based only on
tables without relevance judgments — but our simulated user makes relevance
judgments in each iteration — we present a further approach that integrates
these signals. To account for user feedback in the form of relevance
judgments, we define another knowledge state $KS_{i}^{rel}$, based only on
terms obtained from documents that the simulated user has marked as relevant.

(3) $Q_{i+1}=\left\\{\begin{array}[]{ll}Q_{0}\cup t^{rel}:t^{rel}\in
KS_{i}^{rel}&KS_{i}^{rel}\neq\emptyset\\\ Q_{0}\cup t\;\;\;\;:t\;\;\;\>\in
KS_{i}&else.\par\end{array}\right.$

This way, terms from relevant documents are used for the query variation. If
no new terms from relevant documents are available, the knowledge state
$KS_{i}$ is used instead (Equation 3).

###  3.2. Evaluation

We conduct a multi-perspective cost analysis, including the query- and time-
wise evaluation of the information gain. As proposed in earlier work, we
evaluate the simulated sessions by the Session-based DCG (sDCG) (Järvelin et
al., 2008), which discounts the cumulative gain document- and also query-wise.
sDCG exclusively considers query formulations as costs and thus models the
stopping decision by a log-harmonic probability distribution over queries and
documents.

We stress that users’ search behavior typically covers more interactions that
could result in costs, e.g., inspecting snippets, reading full texts, and
making judgments about relevance. To this end, we complement the sDCG
evaluations by considering all simulated actions in the simulated session logs
to account for a more comprehensive perspective on the “effort vs. effect”
ratio. In our evaluations, we determine the effort by the passed time units
(measured in seconds using the default configurations of SIMIIR) and compare
them against the effect, i.e., information gain, based on the cumulated
relevance scores of unseen documents.

##  4\. Experimental results

Figure 1. Simulated web table retrieval sessions. Column-wise, the plots show
the outcomes of four different evaluation levels. Row-wise, the plots can be
compared by two evaluation paradigms. The blue lines represent the default
parameters for all plots.

The following section describes the experimental results covering the
comparison of querying strategies (cf. 4.1), selection strategies based on
different types of modalities (cf. 4.2), and the general browsing behavior
(cf. 4.3). All of the results are visualized in Figure 1. The upper row
contains plots based on a query-wise evaluation and the sDCG measure. In
contrast, the bottom row contains plots based on a more detailed resolution of
effort and the resulting effects, i.e., information gain. Column-wise, the
plots can be categorized into the four evaluation levels concerned with query
strategy (first column), the modalities (second column), the click
probabilities (third column), and browsing depths (fourth column). With this
type of representation, the effects of the different evaluation levels can be
compared horizontally, and the two cost paradigms can be compared vertically.

###  4.1. Comparison of Query Strategies

The first column of plots in Figure 1 compares simulated sessions with
different types of query simulation strategies as introduced in Subsection
3.1. Both plots show the increase in the average information gain over 60
topics with an increasing number of queries per topic. As can be seen, the
feedback-based Doc2Query approach yields more effective sessions. By including
key terms of relevant tables in later query reformulations, the simulated
users pick up the terminology and better specify their information needs. In
comparison, the GPT-based queries are less effective, which can be explained
by the more generic way the queries were generated. Likewise, the Doc2Query-
based query reformulations that consider all of the earlier retrieved tables
for the reformulation result in lower effectiveness as they lack relevance
feedback. There are no substantial differences between GPT and Doc2Query when
no feedback is used. In conclusion, the instruction-tuned and task-agnostic
GPT model achieves similar effectiveness as the task-specific Doc2Query.

By comparing both evaluation methods based on sDCG and the time-wise
information gain, we see similar results. However, our experiments show the
limitations of using a predefined number of query reformulations that can be
seen by the stagnating information gain for the GPT-based queries in the
bottom plot. As the simulator runs out of queries, the search session ends,
and there is no gain of information. In contrast, the Doc2Query-based
generation method can be used for simulations with an arbitrary number of
query reformulations. By these results, we conclude that the feedback-based
Doc2Query method is suitable for reasonable user simulations, and we use it in
all subsequent evaluations.

###  4.2. Comparison of Modalities

Since the feedback-based Doc2Query strategy was shown to be the most
effective, the evaluations of the modality effects for this query expansion
method follow in the second column of Figure 1. For this purpose, the tables’
four modalities (page title, before, text after, entity), an Oracle, and a
user with random behavior are examined. Using the relevance scores of all
modalities, we simulate users who only decide to click on a snippet based on
its relevance and thus examine the complete table. Furthermore, we add a user
as an upper bound, the oracle, who already knows when viewing the snippet
whether the table is relevant and only then clicks on the snippet.
Additionally, there is a random behavior user who clicks on each snippet with
a probability of 50% ($P(C)=0.5$).

Under both cost paradigms, the entity modality performs worst as a relevance
proxy. Furthermore, compared to the Oracle, a single modality does not serve
as a good proxy for relevance. We suggest that relevance is a multidimensional
concept and should not be represented by a single modality. The random
behavior performs better than the modalities considering just the query-wise
costs. However, compared to the time-wise perspective, this behavior incurs
higher costs and thus no longer performs better. In particular, at the
beginning of the simulation, the random behavior performs worse than a user
who uses the relevance signal of the page title. We explain this effect by the
fact that relevance proxies prevent the user from clicking on non-relevant
tables at the beginning. On average, the user with random behavior clicks on
half of the snippets in the SERPs and thus produces higher costs. In the
course of the simulation, this becomes an advantage since he also finds
relevant tables that a negative relevance signal of the snippets would hide.
Our random user has a higher propensity to explore, while other users focus
more on exploitation.

###  4.3. Comparison of Browsing Strategies

The previous results indicate that random search behavior yields considerably
effective outcomes, letting us conclude that a more in-depth evaluation
concerning the exploitation/explorations tradeoffs is required. To this end,
the third column of Figure 1 compares browsing strategies based on different
click probabilities. We define the click probabilities by the snippet’s
relevance, as the random and straightforward click decisions are too
simplistic and only reflect either fully exploratory- or exploitation-focused
strategies.

The straightforward selection behavior implied that a table is clicked if its
modality is relevant. At this evaluation level, we lower the click probability
to simulate less strict exploitation-focused browsing strategies by
combinations of $P(C|\mathrm{rel})\in\\{0.6,0.7,0.8,0.9\\}$ with
$P(C|\neg\mathrm{rel})=0.3$, where $P(C|\mathrm{rel})$ and
$P(C|\neg\mathrm{rel})$ denote the click probabilities for relevant or non-
relevant modalities, respectively. For better comparability, we include the
random ($P(C|\mathrm{rel})=0.5$, $P(C|\neg\mathrm{rel})=0.5$) and
straightforward ($P(C|\mathrm{rel})=1.0$, $P(C|\neg\mathrm{rel})=0.0$)
selection behaviors.

As can be seen, the straightforward selection behavior is still the least
effective strategy, especially when evaluated by sDCG or as the search session
tends to get longer. In contrast, most of the browsing strategies that relax
the straightforward selection by slightly lower click probabilities perform
best, which suggests that it is more effective to balance the strategies
between exploitation- and exploration-focused browsing.

In conclusion, a single modality cannot comprise the entire notion of
relevance that generally has to be understood as a multi-dimensional
phenomenon. Random behavior is more effective, especially in the long run,
which can be explained by serendipity effects that occur by chance. However,
it is more effective to apply a hybrid browsing strategy that emphasizes
exploitation but which is also exploratory to some extent.

We leave it as future work to analyze the combinations of modalities to
simulate click decisions and to find the sweet spots for weighting
exploitation and exploration. Finally, the fourth column of Figure 1 shows the
sDCG and time-wise information gain for different browsing depths. As
expected, users can either formulate more queries or browse more documents to
increase their information gain, and consequently, the sDCG curves with higher
browsing depths lay above those with lower depths. However, these differences
are less apparent as more interactions are considered as costs as shown in the
bottom plot. Naturally, browsing more documents also requires additional
effort and time, which is not included in the sDCG-based evaluations that
solely model costs by the number of queries. Still, a higher browsing depth
results in better overall effectiveness, but their advantages are less present
early in the search sessions.

##  5\. Conclusion

Through this work, we introduce the first study of its kind that simulates
interactive search sessions for web table retrieval. Furthermore, we
introduced a query simulation method based on Doc2Query that can simulate an
arbitrary number of queries by considering simulated relevance feedback. In
this regard, we applied two different evaluation approaches based on query-
and more comprehensive time-wise evaluations. Our results suggest that query-
wise evaluations could be too simplistic as they only model queries as costs,
and including other costs, such as scanning snippets and making click and
relevance decisions, reveal a different picture of how different search
strategies perform. Our modality-focused evaluations showed that there are
differences between search effectiveness, and using a single modality as a
snippet substitute is not recommended. Relevance is multi-dimensional and
difficult to represent by a single modality. Future work should explore
different combinations of modalities and analyze to which extent these could
be used as proxies of the table’s overall relevance in real user studies.

## ACKNOWLEDGEMENTS

This work was supported by Klaus Tschira Stiftung (JoIE - 00.003.2020) and
Deutsche Forschungsgemeinschaft (RESIRE - 509543643).

## References

  * (1)
  * Azzopardi et al. (2007) Leif Azzopardi, Maarten de Rijke, and Krisztian Balog. 2007.  Building Simulated Queries for Known-Item Topics: An Analysis Using Six European Languages. In _SIGIR 2007: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Amsterdam, the Netherlands, July 23-27, 2007_ , Wessel Kraaij, Arjen P. de Vries, Charles L. A. Clarke, Norbert Fuhr, and Noriko Kando (Eds.). ACM, 455–462.  https://doi.org/10.1145/1277741.1277820 https://doi.org/10.1145/1277741.1277820.
  * Bailey et al. (2015) Peter Bailey, Alistair Moffat, Falk Scholer, and Paul Thomas. 2015.  User Variability and IR System Evaluation. In _Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, Santiago, Chile, August 9-13, 2015_ , Ricardo Baeza-Yates, Mounia Lalmas, Alistair Moffat, and Berthier A. Ribeiro-Neto (Eds.). ACM, 625–634.  https://doi.org/10.1145/2766462.2767728 https://doi.org/10.1145/2766462.2767728.
  * Balog et al. (2021) Krisztian Balog, David Maxwell, Paul Thomas, and Shuo Zhang. 2021.  Report on the 1st Simulation for Information Retrieval Workshop (Sim4IR 2021) at SIGIR 2021.  _SIGIR Forum_ 55, 2 (2021), 10:1–10:16.
  * Balog et al. (2022) Krisztian Balog, David Maxwell, Paul Thomas, and Shuo Zhang. 2022.  Report on the 1st Simulation for Information Retrieval Workshop (Sim4IR 2021) at SIGIR 2021.  _SIGIR Forum_ 55, 2, Article 10 (mar 2022), 16 pages.  https://doi.org/10.1145/3527546.3527559
  * Baskaya et al. (2013) Feza Baskaya, Heikki Keskustalo, and Kalervo Järvelin. 2013.  Modeling Behavioral Factors in Interactive Information Retrieval. In _22nd ACM International Conference on Information and Knowledge Management, CIKM’13, San Francisco, CA, USA, October 27 - November 1, 2013_ , Qi He, Arun Iyengar, Wolfgang Nejdl, Jian Pei, and Rajeev Rastogi (Eds.). ACM, 2297–2302.  https://doi.org/10.1145/2505515.2505660 https://doi.org/10.1145/2505515.2505660.
  * Carterette et al. (2015a) Ben Carterette, Ashraf Bah, and Mustafa Zengin. 2015a.  Dynamic Test Collections for Retrieval Evaluation. In _Proceedings of the 2015 International Conference on The Theory of Information Retrieval_ (Northampton, Massachusetts, USA) _(ICTIR ’15)_. Association for Computing Machinery, New York, NY, USA, 91–100.  https://doi.org/10.1145/2808194.2809470
  * Carterette et al. (2015b) Ben Carterette, Ashraf Bah, and Mustafa Zengin. 2015b.  Dynamic Test Collections for Retrieval Evaluation. In _Proceedings of the 2015 International Conference on the Theory of Information Retrieval, ICTIR 2015, Northampton, Massachusetts, USA, September 27-30, 2015_ , James Allan, W. Bruce Croft, Arjen P. de Vries, and Chengxiang Zhai (Eds.). ACM, 91–100.  https://doi.org/10.1145/2808194.2809470 https://doi.org/10.1145/2808194.2809470.
  * Chen et al. (2021) Zhiyu Chen, Shuo Zhang, and Brian D. Davison. 2021.  WTR: A Test Collection for Web Table Retrieval. In _SIGIR_. ACM, 2514–2520.
  * Chuklin et al. (2015) Aleksandr Chuklin, Ilya Markov, and Maarten de Rijke. 2015.  _Click Models for Web Search_.  Morgan & Claypool Publishers.  https://doi.org/10.2200/S00654ED1V01Y201507ICR043 https://doi.org/10.2200/S00654ED1V01Y201507ICR043.
  * Günther and Hagen (2021) Sebastian Günther and Matthias Hagen. 2021.  Assessing Query Suggestions for Search Session Simulation.  _Sim4IR: The SIGIR 2021 Workshop on Simulation for Information Retrieval Evaluation_ (2021).  http://ceur-ws.org/Vol-2911/paper6.pdf.
  * Hofmann et al. (2013) Katja Hofmann, Anne Schuth, Shimon Whiteson, and Maarten de Rijke. 2013.  Reusing Historical Interaction Data for Faster Online Learning to Rank for IR. In _Sixth ACM International Conference on Web Search and Data Mining, WSDM 2013, Rome, Italy, February 4-8, 2013_ , Stefano Leonardi, Alessandro Panconesi, Paolo Ferragina, and Aristides Gionis (Eds.). ACM, 183–192.  https://doi.org/10.1145/2433396.2433419 https://doi.org/10.1145/2433396.2433419.
  * Järvelin et al. (2008) Kalervo Järvelin, Susan L. Price, Lois M. L. Delcambre, and Marianne Lykke Nielsen. 2008.  Discounted Cumulated Gain Based Evaluation of Multiple-Query IR Sessions. In _Advances in Information Retrieval , 30th European Conference on IR Research, ECIR 2008, Glasgow, UK, March 30-April 3, 2008. Proceedings_ _(Lecture Notes in Computer Science, Vol. 4956)_ , Craig Macdonald, Iadh Ounis, Vassilis Plachouras, Ian Ruthven, and Ryen W. White (Eds.). Springer, 4–15.  https://doi.org/10.1007/978-3-540-78646-7_4 https://doi.org/10.1007/978-3-540-78646-7_4.
  * Jordan et al. (2006) Chris Jordan, Carolyn R. Watters, and Qigang Gao. 2006.  Using Controlled Query Generation to Evaluate Blind Relevance Feedback Algorithms. In _JCDL_. ACM, 286–295.
  * Macdonald et al. (2021) Craig Macdonald, Nicola Tonellotto, Sean MacAvaney, and Iadh Ounis. 2021.  PyTerrier: Declarative Experimentation in Python from BM25 to Dense Retrieval. In _CIKM_. ACM, 4526–4533.
  * Maxwell and Azzopardi (2016a) David Maxwell and Leif Azzopardi. 2016a.  Agents, Simulated Users and Humans: An Analysis of Performance and Behaviour. In _Proceedings of the 25th ACM International on Conference on Information and Knowledge Management_ _(CIKM ’16)_. Association for Computing Machinery, New York, NY, USA, 731–740.  https://doi.org/10.1145/2983323.2983805 https://dl.acm.org/doi/10.1145/2983323.2983805.
  * Maxwell and Azzopardi (2016b) David Maxwell and Leif Azzopardi. 2016b.  Simulating Interactive Information Retrieval: SimIIR: A Framework for the Simulation of Interaction. In _SIGIR_. ACM, 1141–1144.
  * Maxwell and Azzopardi (2018) David Maxwell and Leif Azzopardi. 2018.  Information Scent, Searching and Stopping - Modelling SERP Level Stopping Behaviour. In _Advances in Information Retrieval - 40th European Conference on IR Research, ECIR 2018, Grenoble, France, March 26-29, 2018, Proceedings_ _(Lecture Notes in Computer Science, Vol. 10772)_ , Gabriella Pasi, Benjamin Piwowarski, Leif Azzopardi, and Allan Hanbury (Eds.). Springer, 210–222.  https://doi.org/10.1007/978-3-319-76941-7_16 https://doi.org/10.1007/978-3-319-76941-7_16.
  * Nogueira et al. (2019) Rodrigo Frassetto Nogueira, Wei Yang, Jimmy Lin, and Kyunghyun Cho. 2019.  Document Expansion by Query Prediction.  _CoRR_ abs/1904.08375 (2019).
  * OpenAI (2023) OpenAI. 2023.  Model Index for Researchers.  https://platform.openai.com/docs/model-index-for-researchers.
  * Pääkkönen et al. (2017) Teemu Pääkkönen, Jaana Kekäläinen, Heikki Keskustalo, Leif Azzopardi, David Maxwell, and Kalervo Järvelin. 2017.  Validating Simulated Interaction for Retrieval Evaluation.  20, 4 (2017), 338–362.  https://doi.org/10.1007/s10791-017-9301-2 https://doi.org/10.1007/s10791-017-9301-2.
  * Penha et al. (2022) Gustavo Penha, Arthur Câmara, and Claudia Hauff. 2022.  Evaluating the Robustness of Retrieval Pipelines with Query Variation Generators. In _ECIR (1)_ _(Lecture Notes in Computer Science, Vol. 13185)_. Springer, 397–412.
  * Shraga et al. (2020) Roee Shraga, Haggai Roitman, Guy Feigenblat, and Mustafa Canim. 2020.  Web Table Retrieval Using Multimodal Deep Learning. In _SIGIR_. ACM, 1399–1408.
  * Smucker and Clarke (2012) Mark D. Smucker and Charles L. A. Clarke. 2012.  Time-Based Calibration of Effectiveness Measures. In _The 35th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’12, Portland, OR, USA, August 12-16, 2012_ , William R. Hersh, Jamie Callan, Yoelle Maarek, and Mark Sanderson (Eds.). ACM, 95–104.  https://doi.org/10.1145/2348283.2348300 https://doi.org/10.1145/2348283.2348300.
  * Thomas et al. (2014) Paul Thomas, Alistair Moffat, Peter Bailey, and Falk Scholer. 2014.  Modeling Decision Points in User Search Behavior. In _Fifth Information Interaction in Context Symposium, IIiX ’14, Regensburg, Germany, August 26-29, 2014_ , David Elsweiler, Bernd Ludwig, Leif Azzopardi, and Max L. Wilson (Eds.). ACM, 239–242.  https://doi.org/10.1145/2637002.2637032 https://doi.org/10.1145/2637002.2637032.
  * Trabelsi et al. (2022) Mohamed Trabelsi, Zhiyu Chen, Shuo Zhang, Brian D. Davison, and Jeff Heflin. 2022.  StruBERT: Structure-aware BERT for Table Search and Matching. In _Proceedings of the ACM Web Conference 2022_. 442–451.  https://doi.org/10.1145/3485447.3511972 arXiv:2203.14278 [cs]  http://arxiv.org/abs/2203.14278.
  * Wang et al. (2015) Hong Wang, Anqi Liu, Jing Wang, Brian D. Ziebart, Clement T. Yu, and Warren Shen. 2015.  Context Retrieval for Web Tables. In _ICTIR_. ACM, 251–260.
  * Yan et al. (2017) Zhao Yan, Duyu Tang, Nan Duan, Junwei Bao, Yuanhua Lv, Ming Zhou, and Zhoujun Li. 2017.  Content-Based Table Retrieval for Web Queries.  http://arxiv.org/abs/1706.02427.  arXiv:1706.02427 [cs]
  * Zerhoudi et al. (2022) Saber Zerhoudi, Sebastian Günther, Kim Plassmeier, Timo Borst, Christin Seifert, Matthias Hagen, and Michael Granitzer. 2022.  The SimIIR 2.0 Framework: User Types, Markov Model-Based Interaction Simulation, and Advanced Query Generation. In _CIKM_. ACM, 4661–4666.
  * Zhang and Balog (2018) Shuo Zhang and Krisztian Balog. 2018.  Ad Hoc Table Retrieval Using Semantic Similarity. In _Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW ’18_. 1553–1562.  https://doi.org/10.1145/3178876.3186067 arXiv:1802.06159 [cs]  http://arxiv.org/abs/1802.06159.
  * Zhang and Balog (2019) Shuo Zhang and Krisztian Balog. 2019.  Web Table Extraction, Retrieval and Augmentation. In _SIGIR_. ACM, 1409–1410.
  * Zhang et al. (2017a) Yinan Zhang, Xueqing Liu, and ChengXiang Zhai. 2017a.  Information Retrieval Evaluation as Search Simulation: A General Formal Framework for IR Evaluation. In _Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval_ (Amsterdam, The Netherlands) _(ICTIR ’17)_. Association for Computing Machinery, New York, NY, USA, 193–200.  https://doi.org/10.1145/3121050.3121070
  * Zhang et al. (2017b) Yinan Zhang, Xueqing Liu, and ChengXiang Zhai. 2017b.  Information Retrieval Evaluation as Search Simulation: A General Formal Framework for IR Evaluation. In _ICTIR_. ACM, 193–200.
  * Zobel (2022) Justin Zobel. 2022.  When Measurement Misleads: The Limits of Batch Assessment of Retrieval Systems.  _ACM SIGIR Forum_ 56, 1 (2022), 20.

  *[WTR]: web table retrieval
  *[CSM]: Complex Searcher Model
  *[SERP]: search engine result page
  *[LLM]: Large Language Model
  *[idf]: inverse document frequency
  *[sDCG]: Session-based DCG


# Comment on “Nonlinear photoluminescence spectra from a quantum dot-cavity
system revisited” by Yao _et al._ and “Photoluminescence from Microcavities
Strongly Coupled to Single Quantum Dots” by Ridolfo _et al._

F. P. Laussy  School of Physics and Astronomy. University of Southampton,
Southampton, SO171BJ, United Kingdom.  E. del Valle  School of Physics and
Astronomy. University of Southampton, Southampton, SO171BJ, United Kingdom.
C. Tejedor  Física Teórica de la Materia Condensada, Universidad Autónoma de
Madrid, 28049, Spain.

###### Abstract

We refute the criticisms of our work on strong-coupling in the presence of an
incoherent pumping.

###### pacs:

42.50.Ct, 78.67.Hc, 42.55.Sa, 32.70.Jz

Our description of strong-coupling in semiconductors Laussy et al. (2008,
2009); del Valle et al. (2009) has been recently criticized by two groups:
Ridolfo _et al._ Ridolfo et al. (2009) and Yao _et al._ Yao et al. (2009).

Both groups of authors make the same statements: they claim that our master
equation is flawed, on the ground that its domain of convergence is bounded,
they both propose to use exclusively a thermal bath for the reservoir of
excitations of the cavity instead of our most general case (Yao _et al._
require a thermal bath also for the excitonic reservoir whereas Ridolfo _et
al._ allow independent pumping and decay coefficients for the Quantum Dot
(QD)) and they all claim a better agreement than our model with experimental
data.

We show in this comment that their criticisms are erroneous and that the
alternatives they propose to fulfill them are well known particular cases
that, in their approximations, are also erroneous.

Both groups focus on our boson model only, and in fact only on our Letter on
the topic Laussy et al. (2008). Many of their statements are already addressed
in our full boson text Laussy et al. (2009) and more crucially in its fermion
counterpart del Valle et al. (2009). Throughout, we made clear that the boson
model is adequate either in the limit of small pumping, or in the limit of
bosonic 0D system. The interest of the boson case is in its analytical
solutions, that allow to explain transparently fundamental features of
pumping, such as the effect of the effective quantum state on the spectral
shapes. We will show that the analytical solutions for the fermion models from
Ridolfo et al. (2009) and Yao et al. (2009) are in fact valid only in trivial
cases (namely, the boson limit or the uncoupled limit).

In the following, we shall clarify, on the one hand, that our approach does
not suffer from any inconsistency or pathology, and on the other hand, that in
the particular cases where the reservoirs of excitations are thermal baths,
our model also applies (by enforcing the ad-hoc constrain in the equation) and
show the errors made by the approximations of Refs. Ridolfo et al. (2009); Yao
et al. (2009).

##  I Validity of our master equation

Our model describes the linear regime (of vanishing excitations) analytically
and the nonlinear regime semi-analytically. Its master equation reads:

$\displaystyle\frac{d\rho}{dt}$ $\displaystyle=$ $\displaystyle
i[\rho,H_{a}+H_{\sigma}+H_{a\sigma}]$ (1a) $\displaystyle+$
$\displaystyle\frac{\gamma_{a}}{2}(2a\rho{a^{\dagger}}-{a^{\dagger}}a\rho-\rho{a^{\dagger}}a)$
(1b) $\displaystyle+$ $\displaystyle\frac{P_{a}}{2}(2{a^{\dagger}}\rho
a-a{a^{\dagger}}\rho-\rho a{a^{\dagger}})$ (1c) $\displaystyle+$
$\displaystyle\frac{\gamma_{\sigma}}{2}(2\sigma\rho{\sigma^{\dagger}}-{\sigma^{\dagger}}\sigma\rho-\rho{\sigma^{\dagger}}\sigma)$
(1d) $\displaystyle+$
$\displaystyle\frac{P_{\sigma}}{2}(2{\sigma^{\dagger}}\rho\sigma-\sigma{\sigma^{\dagger}}\rho-\rho\sigma{\sigma^{\dagger}})\,.$
(1e)

where $a$ is the cavity mode and $\sigma$ the exciton in the QD. The cavity is
always a bosonic mode. In Ref. Laussy et al. (2008) and Laussy et al. (2009),
$\sigma$ is also a Bose operator, that we note $b$ for clarity, while in Ref.
del Valle et al. (2009), $\sigma$ is a fermion operator, describing a two-
level system 111We have been aware of and considering the Fermi dynamics from
the start, as one can see from the initial version arXiv:0711.1894v1 of our
manuscript Laussy et al. (2008) although it eventually retained only the boson
results, postponing the general but non-analytical case to Ref. del Valle et
al. (2009). All our statements in the linear and boson models, even in
November 2007, were confronted with the fermion case for validity, in contrast
with our critics, as is shown in this comment.. This choice of $\sigma$ as a
Bose operator addresses two important cases: $i$) the limit of vanishing
pumpings (even if the QD is indeed a fermion emitter) and $ii$) the case where
the excitons follow bose statistics. The latter case could be realized in
large QDs, that recover the physics of quantum wells where excitons are known
to behave as good bosons Kavokin et al. (2007).

The main reproach of Ridolfo et al. (2009); Yao et al. (2009) is that in Eq.
(1), the effective pumping rates $P_{a,\sigma}$ can vary independently of the
effective decay rates $\gamma_{a,\sigma}$ 222They both also make the reproach
that $\sigma$ is limited to Bose statistics but this is because they are
unaware—or do not want to consider—our main line of work, for which we can
only point them to Ref. del Valle et al. (2009).. They observe that this can
lead to some divergences beyond some critical values of pumping, as we have
ourselves discussed before, and they conclude that the model is flawed. They
propose instead to use thermal reservoirs, that do not exhibit such
divergences, at all values of pumping.

A thermal reservoir for a bosonic mode $a$ (with frequency $\omega_{a}$ and
$H_{a}=\omega_{a}{a^{\dagger}}a$) at temperature $T$ leads to the effective
rate of excitation:

$P_{a}=\kappa_{a}\bar{n}_{T}\,.$ (2)

with $\bar{n}_{T}$ given by the reservoir Bose-Einstein distribution. It
vanishes at $T=0$. In thermal equilibrium, the system is loosing excitations
at a larger rate of:

$\gamma_{a}=\kappa_{a}(1+\bar{n}_{T})=\kappa_{a}+P_{a}\,.$ (3)

The parameter $\kappa_{a}$ is the _spontaneous emission_ (SE) rate at $T=0$.
The steady state thermal equilibrium reads

$n_{a}=\langle{a^{\dagger}}a\rangle=\frac{P_{a}}{\gamma_{a}-P_{a}}=\frac{P_{a}}{\kappa_{a}}=\bar{n}_{T}\,.$
(4)

At very high temperatures, as the effective income of particles approaches the
outcome, $P_{a}\approx\gamma_{a}$, the number of particles remains finite,
since $P_{a}<\gamma_{a}$. As long as $\gamma_{a}\neq 0$, any combination of
parameters $\gamma_{a},P_{a}$ corresponds to a given thermal bath (with
$\kappa_{a}=\gamma_{a}-P_{a}$ and $T>0$).

The linewidth of the optical spectrum of emission is:

$\Gamma_{a}\equiv\gamma_{a}-P_{a}=\kappa_{a}\,.$ (5)

and is independent of temperature (i.e., of the population of the mode), since
it is always equal to the spontaneous emission decay rate $\kappa_{a}$.

It is clear from the above results, that a bosonic thermal bath cannot provide
gain and does not exhibit any line-narrowing in its luminescence spectrum. A
thermal bath is a medium of _loss_ as $P_{a}<\gamma_{a}$ by definition.

A thermal reservoir is, however, a particular case. In out-of-equilibrium
conditions, especially under externally applied pumping, one can expect
deviations from the thermal paradigm.

In contrast to the thermal case, a _gain_ medium can be derived with bosonic
baths out-of-equilibrium. This is discussed in textbooks, e.g., in Chapter 7
of Gardiner and Zoller’s text Gardiner and Zoller (2000). A linear gain can be
obtained with an “inverted” harmonic oscillator maintained at a negative
temperature $-T^{\prime}$. The effect in the master equation is that the
effective parameters are now given by a relation opposite to Eqs. (2-3):

$\gamma_{a}=G_{a}\bar{m}_{-T^{\prime}}\,,\quad
P_{a}=G_{a}(1+\bar{m}_{-T^{\prime}})=G_{a}+\gamma_{a}\,.$ (6)

In this way, $G_{a}$ is the gain or input of particles into the mode at zero
temperature. Obviously, given that now $P_{a}>\gamma_{a}$, there is no
stationary solution to the master or rate equations. Since the absence of this
stationary solution is the source of confusion in Refs Ridolfo et al. (2009);
Yao et al. (2009), we quote here Gardiner and Zoller’s comment, p°216:

> _If $\gamma<\kappa$_ [that is $\gamma_{a}<P_{a}$ with our notation]_, there
> is no stationary situation, and the amplifier gives a signal that increases
> without limit. Essentially, the power being fed into the cavity cannot
> escape fast enough. (Of course the idea of an inverted medium which
> maintains its inversion independent of power output is not exactly valid,
> and depletion effects will then need to be considered. The system is then
> essentially a laser)._

Obviously, an ever-growing population will always be stopped by some external
physical effect (the sample will have burnt, the reservoir will be depleted,
etc…). There is however nothing pathologic in this behaviour. In particular,
this does not invalidate the results for values below the critical pumping
rates. In the case where $\gamma_{c}>P_{c}$ ($c=a,b$), there is a physical
solution to the dynamical master equation, starting at $t=0$: at all
(_finite_) times, there is a valid master equation, with positive trace,
normed to unity, etc… However, indeed, the system diverges with time. There is
no unphysical behaviour or flaw of some sort. Not all dynamical systems have a
steady state, some because they are oscillatory, others because they increase
without bounds.

In our work Laussy et al. (2008, 2009); del Valle et al. (2009), we have
naturally considered the configurations which admit a steady state. We have
even given analytical solutions for their domain of convergence, supported by
a clear physical picture 333The limit of validity of the boson master equation
is given by a total outcome of excitation larger than the income
$\gamma_{a}+\gamma_{b}>P_{a}+P_{b}$, that is, $\Gamma_{a}+\Gamma_{b}>0$ (in
the notations of Laussy et al. (2009)) If one of the oscillators has a net
gain ($\Gamma_{b}<0$), there is an extra condition for convergence that has a
clear physical motive: not only mode $a$ is constrained to be neatly lossy,
$\Gamma_{a}>0$, but also the income through the “inverted” mode $b$, given by
$P_{b}$, must be smaller than its total outcome, which is given by
$\gamma_{b}$ plus the effective (Purcell) decay through cavity emission. That
is, $P_{b}<\gamma_{b}+\frac{4(g^{\mathrm{eff}})^{2}}{\Gamma_{a}}$ where
$g^{\mathrm{eff}}=g/\sqrt{1+[2\Delta/(\Gamma_{a}+\Gamma_{b})]^{2}}$, and $=g$
at resonance..

In a microcavity QED system (a QD in a microcavity), which is a complicated
solid state system, open to many sources of excitations Gies et al. (2007);
Winger et al. (2009), one should also include gain effects in the most general
case. Granted all together, the microscopic coefficients are likely terms of
the form:

$\displaystyle\gamma_{a}=\kappa_{a}(1+\bar{n}_{T})+G_{a}\bar{m}_{-T^{\prime}}\,,$
(7a) $\displaystyle
P_{a}=\kappa_{a}\bar{n}_{T}+G_{a}(1+\bar{m}_{-T^{\prime}})\,,$ (7b)

that is, including loss media and gain media. Net losses in the case of a
cavity mode comes, among other reasons, from the fact that the photons can
escape the cavity through the imperfect mirrors. Net gain could come from
surrounding off-resonance or weakly coupled QDs, high energy QD levels or the
wetting layer. A gain medium can be obtained by the very configuration
realized with self-assembled QDs in a microcavity. Gardiner and Zoller, p°140
(ibid) study a bath of uncorrelated two-level emitters that are kept in
average in the excited state. Quoting them again:

> _Note that there is no restriction on $N^{+}_{a}$ and $N^{-}_{a}$_ [that is,
> in their notations, the number of emitters in the excited and ground states,
> respectively, that provide the pump and decay terms for the bosonic
> mode]_—this means that $N^{+}_{a}>N^{-}_{a}$ is permissible. Physically such
> a population inversion could only be maintained by some kind of pumping, as
> indeed happens in a laser._

and as indeed could happen in an incoherently pumped microcavity. The previous
discussion on gain-media that applies for the cavity mode can also be extended
for bosonic emitters (QDs, in our case). In this case, for example, the
electron-hole pairs that form excitons inside the QD decay from the wetting
layer (at higher energy) by, e.g., emitting phonons with the corresponding
energy difference. Such phonons will not be reabsorbed to bring back the
electron-hole pair to the wetting layer, leading to a net source of particles.
Net losses take place through the spontaneous decay into leaky modes.

In general, there is thus no reason to restrict the master equation (1)
designed to account effectively for the largest possible amount of physical
effects, to a given type of reservoirs (namely, thermal baths). Our study aims
at the greatest level of applicability and generality and provides the tools
for the understanding of any case. Therefore, better than trying to apply at
the theoretical level any criteria (other than convergence) to choose
$\gamma_{a}$, $P_{a}$, we prefer to consider them independent. So far, the
dynamics of lines (1b-1c) found its most important domain of applicability
with atom lasers and polariton lasers Holland et al. (1996); Ĭmamoḡlu and Ram
(1996); Porras and Tejedor (2003); Rubo et al. (2003); Laussy et al. (2004a);
Schwendimann and Quattropani (2006, 2008); Doan et al. (2008), that is,
systems where a condensate (or coherent state) is formed by scattering of
bosons into the final state from another state rather than by emission. In
both cases, scattering or emission, the process is stimulated. In this case
the income and outcome of particles is a complicated function of the
distribution of excitons (or polaritons) in the higher $k$-states. See, e.g.,
Ref Ĭmamoḡlu and Ram (1996). In a pulsed experiment, it is typically time
dependent (see, e.g., Fig. 2 of Ref. Laussy et al. (2004b)). Line narrowing is
a natural feature of this dynamics in such systems.

It remains, of course, possible that a particular experiment corresponds to a
thermal bath of excitation. In this case, a fitting analysis with our model
should indicate this constrain in correlations between the $\gamma$ and $P$
coefficients. As a result of this analysis, one will then understand that the
given system refers to the particular cases of Eqs. (2) and (3).

The above discussion concerns the bosonic mode. We now turn to the fermion
mode.

The thermal equilibrium case, at temperature $T$, gives the counterpart of the
boson case (given above):

$\displaystyle\gamma_{\sigma}$
$\displaystyle=\kappa_{\sigma}(1+\bar{n}_{T})=\kappa_{\sigma}+P_{\sigma}\,,$
(8a) $\displaystyle P_{\sigma}$ $\displaystyle=\kappa_{\sigma}\bar{n}_{T}\,,$
(8b)

where $\kappa_{\sigma}$ is the Einstein $A$-coefficient and $P_{\sigma}$ is
the Einstein $B$-coefficient. The steady state is the _Fermi-Dirac
distribution_ :

$n_{\sigma}=\frac{P_{\sigma}}{\gamma_{\sigma}+P_{\sigma}}=\frac{P_{\sigma}}{\kappa_{\sigma}+2P_{\sigma}}=\frac{\bar{n}_{T}}{2\bar{n}_{T}+1}=\Big{(}e^{\frac{\omega_{\sigma}}{k_{B}T}}+1\Big{)}^{-1}\,.$
(9)

The maximum occupation for the emitter is $1/2$, at infinite temperature. It
is, therefore, not possible to invert the two-level system with a thermal
bath, (where, again, $\gamma_{\sigma}>P_{\sigma}$), which is a well known
result. In the fermion case, even a gain-medium does not lead to a divergence
thanks to the intrinsic saturation of a two-level system. The emission
spectrum is also a Lorentzian, with effective linewidth:

$\Gamma_{\sigma}\equiv\gamma_{\sigma}+P_{\sigma}=\kappa_{\sigma}+2P_{\sigma}$
(10)

which broadens from the decay rate at zero temperature, $\kappa_{\sigma}$ when
the temperature increases.

This elementary discussion illustrates the notorious fact that thermal
reservoirs are unable to achieve population inversion of a two-level system
Briegel and Englert (1993). Therefore, the choices of excitation reservoirs of
Yao _et al._ Yao et al. (2009), forbids lasing in such systems, which is
already contradicted by experiments Nomura et al. (2009).

The issue of achieving gain with a master equation for a fermion emitter has
been extensively discussed in the lasing literature, in particular in the one-
atom laser case Agarwal and Dutta Gupta (1990); Cirac et al. (1991); Mu and
Savage (1992); Horak et al. (1995); Briegel et al. (1996); Löffler et al.
(1997); Meyer and Briegel (1998); Benson and Yamamoto (1999); Koganov and
Shuker (2000); Florescu et al. (2004); Karlovich and Kilin (2008); Li et al.
(2009), and it is typically described theoretically also by considering an
effective negative-temperature thermal bath. A popular notation to consider
gain and dissipation is Briegel et al. (1996):

$P_{\sigma}=\Gamma_{\sigma}s\,,\quad\gamma_{\sigma}=\Gamma_{\sigma}(1-s)\,.$
(11)

With $\Gamma_{\sigma}>0$ (the linewidth broadening) and $0\leq s\leq 1$,
including both the situations with net losses ($s<1/2$) and gain ($s>1/2$). In
this case:

$n_{\sigma}=\frac{P_{\sigma}}{\gamma_{\sigma}+P_{\sigma}}=s\,.$ (12)

This parameterization is not linked to any particular experimental realization
but is designed to separate the physical effects that lead to line broadening,
$\Gamma_{\sigma}$, from those that change the population, $s$. Apart from
that, it is equivalent to consider directly the effect of varying the
effective decay and pumping parameters, $\gamma_{\sigma}$ and $P_{\sigma}$, as
we have done in Refs del Valle et al. (2009) and Gonzalez-Tudela et al.
(2009).

The Jaynes-Cummings model couples the fermionic and bosonic modes. It has been
extensively studied in the case of thermal cavity bath and some gain-medium
for the emitter (which is also the case of Ref. Ridolfo et al. (2009)). We
studied it in its most general form, using the master equation (1). Again, if
a particular constrain arises from a given realization of the reservoirs of
excitations, such as (2-3), (6), (7), (8) or some other case, this would
appear in our unconstrained case from correlations following these trends. We
expect, as we previously discussed, that a successful statistical analysis
would indeed inform about underlying microscopic details of the excitation
scheme.

##  II Validity of the proposed substitutes to our work

In the case of Ridolfo _et al._ Ridolfo et al. (2009), only the photonic mode
was excited thermally while the excitonic pump was still allowing an
unconstrained pumping and decay. Yao _et al._ Yao et al. (2009), on the other
hand, require both modes to be excited by thermal baths.

Thermal baths in the linear (boson) model reduce to results identical to the
spontaneous emission of an initial state that is a mixture of excitons and
photons in the ratio of population $n_{a}(t=0)/n_{b}(t=0)=P_{a}/P_{b}$. The
thermal character of the bath merely prevents renormalization of the
linewidths and of the Rabi frequencies. The ratio $P_{a}/P_{b}$ still
determines the possibility to resolve the line-splitting. This fundamental
consequence of the effective quantum state is independent of any choice of the
reservoirs. It is a general result that we have amply discussed before Laussy
et al. (2008, 2009); del Valle et al. (2009) and that is “rediscovered” by
Ridolfo _et al._

Ridolfo _et al._ Ridolfo et al. (2009) otherwise have mistakes in their
formulas, that certainly bias their analysis. For instance, their parameters
$n_{a}$ and $C$ should read, in their notations: (cf. their Eq. (7) & (8))

$\displaystyle n_{a}$ $\displaystyle=$
$\displaystyle\frac{P_{a}}{\gamma_{a}}+\frac{g^{2}}{\gamma_{a}}\frac{(\gamma_{a}+\gamma_{x})(\gamma_{a}P_{x}-\gamma_{x}P_{a})}{g^{2}(\gamma_{a}+\gamma_{x})^{2}+\gamma_{a}\gamma_{x}|\tilde{\omega}_{a}^{*}-\tilde{\omega}_{x}|^{2}}\,,$
(13a) $\displaystyle C$ $\displaystyle=$
$\displaystyle\frac{g}{\tilde{\omega}_{a}^{*}-\tilde{\omega}_{x}}(n_{a}-n_{x})\,.$
(13b)

Also, their spectra are normalized to $\sqrt{2\pi}n_{a}$, again apparently as
an error since they compare them directly to ours which are normalized to
unity. These mistakes do not seem to be a misprint, given that the authors
state in their paper:

> _Although at low pump intensities, our approach and that of Ref. [Laussy et
> al.] essentially represent models of a linear Bose-like dynamics of two
> coupled harmonic oscillators, nontrivial differences can be appreciated._

There should be no difference in the limit of vanishing pump. In fact, once
corrected as above, their formulas and lineshapes do converge to our results
at low pumps.

In the _non-vanishing_ case, of course, the thermal reservoir gives a
different result than unconstrained parameters of our Eq. (1), even in the
linear regime (that is, when $n_{\sigma}\ll 1$, although $n_{a}$ is not
compulsorily also vanishing). The illustration of this fact was attempted in
Fig. (1a) of Ref. Ridolfo et al. (2009), although here also the plot is wrong.
With non-vanishing cavity pumping, the linear regime can be maintained only if
the modes are almost uncoupled. This is shown in our corrected version (Fig.
1).

Figure 1: Cavity emission for the parameters of Fig. 1(a) of Ridolfo _et al._
Ridolfo et al. (2009): $\Delta=-3.6g$, $\gamma_{\sigma}=1.48g$, $P_{a}=0.49g$,
$P_{\sigma}=0.0078g$. Like in their figure—but with the correct formulas
(13)—we compare: in solid-blue, the case where $\gamma_{a}=1.96g$ and in
dashed-purple $\kappa_{a}=1.96g$ ($\gamma_{a}=\kappa_{a}+P_{a}$, thermal
bath). Both cases are indeed in the linear regime ($n_{\sigma}\ll 1$) since
the fermion model del Valle et al. (2009) gives the same results. The system
is however almost decoupled due to the large detuning. The emission is thus
basically that of the bare cavity (Lorentzian). Figure 2: Converged spectra
with the full fermion model del Valle et al. (2009) [in solid black] for the
choice of pumping reservoirs and parameters of Yao _et al._ , along with the
approximate spectra proposed by these authors [in dashed red]. Their
approximation is correct only at the smallest values of pumping, where it also
recovers the boson results of Refs. Laussy et al. (2008, 2009). Figure 3:
Converged populations [in solid black] for the cavity,
$\langle{a^{\dagger}}a\rangle$ [thin] and the QD,
$\langle\sigma^{+}\sigma^{-}\rangle$ [thick], for the choice of pumping
reservoirs and parameters of Yao _et al._ , along with the approximate values
proposed by these authors [in dashed red]. Again, their approximation is
correct only at the smallest values of pumping, where it also recovers the
boson results of Refs. Laussy et al. (2008, 2009). The breakdown of their
approximation is further manifest by the inversion of population of the QD
(indicated by the arrow), which is notoriously impossible with thermal
reservoirs. Our model shows the expected saturation bounded by 1/2.

We now turn to the approach of Yao _et al._ Yao et al. (2009). They miss a
factor $\Gamma_{c}$ in the second term of the denominator of their Eq. (4),
although in their case this is a misprint only since their figures match the
formulas that follow from their approximations. These approximations, however,
are incorrect.

As we discussed above, it is straightforward in our approach to constrain the
coefficients, following a given choice of the reservoirs of excitations. If we
choose the thermal baths advocated by Yao _et al._ , we are able with our
approach del Valle et al. (2009) to apply the full fermion model with no
truncation in the number of excitations, for the parameters of these authors.
We find that their approximation is a poor one, as seen in Fig. 2, where we
superimpose the exact (numerical) result, in solid black, to their approximate
(analytical) formula, in dashed red. As could be expected, the agreement is
good only at very low pumping (linear regime) and very high pumping (uncoupled
regime). It is incorrect in the most relevant region of the transition where
the doublet collapses (as has been reported before, e.g., Fig. 13 of our Ref.
del Valle et al. (2009)).

Their approximation is also basically flawed in that it allows an inversion of
population for the two-level system, although it is excited by thermal
reservoirs, as seen in the magnified version of their Fig. 3(a), that we
reproduce in dashed lines in our Fig. 3, along with the converged solutions
from our model del Valle et al. (2009) (with their parameters and choices of
reservoirs). Beyond the poor quantitative agreement when pumping is non-
vanishing, at the point indicated by the arrow and above, the QD is inverted,
which indicates a pathology of their approximation.

Their implication that cavity pumping is determinant to achieve lasing is in
contradiction with well-known and established facts of the one-atom laser
theory. See for instance our text Ref. del Valle et al. (2009) where lasing is
achieved without cavity pumping, thanks to the gain-medium that our general
model allows. On the other hand, thermal reservoirs of Yao _et al._ forbids
lasing, regardless of the magnitude of cavity pumping (note that with their
parameters, very high cavity populations are already achieved, but they have
thermal statistics, with second-order correlator $g^{(2)}$ that increases
rapidly towards 2 with pumping).

Finally, we want to stress that their Fig. 2(b), that supposedly represents
our model, does not make any meaningful comparison, since, fitting some data
with their model [that we have just shown is wrong, but even if it was
correct], they proceed to plot our _boson_ model with _their_ fitting
parameters. It is obvious that, the two formulas being different, the best-
fitting parameters for one of them will yield poor agreement on the same data
for the other. Beside, they should have used our _fermion_ model, since they
consider a supposedly two-level emitter in a nonlinear regime. Fitting them
independently, on the one hand, and comparing them on statistical grounds on
the other hand, rather than settling for some aesthetic of the agreement, is
the correct course of action. Fitting with the nonlinear fermion model is not
a trivial task. With E. Cancellieri and A. Gonzalez-Tudela, we have recently
obtained results in this direction, to be published shortly.

In conclusion, we have shown that our work Laussy et al. (2008, 2009); del
Valle et al. (2009) is correct and that the critics addressed against it
Ridolfo et al. (2009); Yao et al. (2009) are unsubstantiated on the one hand,
and the proposed substitutes are incorrect on the other hand. These authors do
not derive any master equation. They settle for thermal reservoirs, which
derivation is a standard textbook material. This is a particular case of our
work that they apply incorrectly or beyond its limits of validity. We have
already provided the valid limit for the nonlinear regime del Valle et al.
(2009). There remain many open questions in the field. Some can be settled by
statistical analysis of experimental data with our model, Eq. (1), which
correlations between the fitting parameters can teach about underlying
microscopic mechanisms (such as the nature of the bath of excitations, among
other). To this intent, we invite experimentalists to make their raw data
available to everybody 444http://sciencecommons.org/about/towards.

## References

  * Laussy et al. (2008) F. P. Laussy, E. del Valle, and C. Tejedor, Phys. Rev. Lett. 101, 083601 (2008).
  * Laussy et al. (2009) F. P. Laussy, E. del Valle, and C. Tejedor, Phys. Rev. B 79, 235325 (2009).
  * del Valle et al. (2009) E. del Valle, F. P. Laussy, and C. Tejedor, Phys. Rev. B 79, 235326 (2009).
  * Ridolfo et al. (2009) A. Ridolfo, O. D. Stefano, S. Portolan, and S. Savasta, arXiv:0906.1455 (2009).
  * Yao et al. (2009) P. Yao, P. Pathak, S. Hughes, S. Munch, S. Reitzenstein, P. Franeck, A. Loffler, T. Heindel, S. Höfling, L. Worschech, et al., arXiv:0909.564 (2009).
  * Kavokin et al. (2007) A. Kavokin, J. J. Baumberg, G. Malpuech, and F. P. Laussy, _Microcavities_ (Oxford University Press, 2007).
  * Gardiner and Zoller (2000) G. W. Gardiner and P. Zoller, _Quantum Noise_ (Springer-Verlag, Berlin, 2000), 2nd ed.
  * Gies et al. (2007) C. Gies, J. Wiersig, M. Lorke, and F. Jahnke, Phys. Rev. A 75, 013803 (2007).
  * Winger et al. (2009) M. Winger, T. Volz, G. Tarel, S. Portolan, A. Badolato, K. Hennessy, E. Hu, A. Beveratos, J. Finley, V. Savona, et al., arXiv:0907.1881 (2009).
  * Holland et al. (1996) M. Holland, K. Burnett, C. Gardiner, J. I. Cirac, and P. Zoller, Phys. Rev. A 54, R1757 (1996).
  * Ĭmamoḡlu and Ram (1996) A. Ĭmamoḡlu and R. J. Ram, Physics Letter A 214, 193 (1996).
  * Porras and Tejedor (2003) D. Porras and C. Tejedor, Phys. Rev. B 67, 161310(R) (2003).
  * Rubo et al. (2003) Y. G. Rubo, F. P. Laussy, G. Malpuech, A. Kavokin, and P. Bigenwald, Phys. Rev. Lett. 91, 156403 (2003).
  * Laussy et al. (2004a) F. P. Laussy, G. Malpuech, A. Kavokin, and P. Bigenwald, Phys. Rev. Lett. 93, 016402 (2004a).
  * Schwendimann and Quattropani (2006) P. Schwendimann and A. Quattropani, Phys. Rev. B 74, 045324 (2006).
  * Schwendimann and Quattropani (2008) P. Schwendimann and A. Quattropani, Phys. Rev. B 77, 085317 (2008).
  * Doan et al. (2008) T. D. Doan, H. T. Cao, D. B. T. Thoai, and H. Haug, Phys. Rev. B 78, 205306 (2008).
  * Laussy et al. (2004b) F. P. Laussy, G. Malpuech, A. V. Kavokin, and P. Bigenwald, J. Phys.: Condens. Matter 16, S3665 (2004b).
  * Briegel and Englert (1993) H.-J. Briegel and B.-G. Englert, Phys. Rev. A 47, 3311 (1993).
  * Nomura et al. (2009) M. Nomura, N. Kumagai, S. Iwamoto, Y. Ota, and Y. Arakawa, arXiv:0906.4181 (2009).
  * Agarwal and Dutta Gupta (1990) G. S. Agarwal and S. Dutta Gupta, Phys. Rev. A 42, 1737 (1990).
  * Cirac et al. (1991) J. I. Cirac, H. Ritsch, and P. Zoller, Phys. Rev. A 44, 4541 (1991).
  * Mu and Savage (1992) Y. Mu and C. M. Savage, Phys. Rev. A 46, 5944 (1992).
  * Horak et al. (1995) P. Horak, K. M. Gheri, and H. Ritsch, Phys. Rev. A p. 3257 (1995).
  * Briegel et al. (1996) H.-J. Briegel, G. M. Meyer, and B.-G. Englert, Europhys. Lett. 33, 515 (1996).
  * Löffler et al. (1997) M. Löffler, G. M. Meyer, and H. Walther, Phys. Rev. A 55, 3923 (1997).
  * Meyer and Briegel (1998) G. M. Meyer and H.-J. Briegel, Phys. Rev. A 58, 3210 (1998).
  * Benson and Yamamoto (1999) O. Benson and Y. Yamamoto, Phys. Rev. A 59, 4756 (1999).
  * Koganov and Shuker (2000) G. A. Koganov and R. Shuker, Phys. Rev. A 63, 015802 (2000).
  * Florescu et al. (2004) L. Florescu, S. John, T. Quang, and R. Wang, Phys. Rev. A 69, 013816 (2004).
  * Karlovich and Kilin (2008) T. B. Karlovich and S. Y. Kilin, Laser Phys. 18, 783 (2008).
  * Li et al. (2009) G. Li, M. Luo, and Z. Ficek, Phys. Rev. A 79, 053847 (2009).
  * Gonzalez-Tudela et al. (2009) A. Gonzalez-Tudela, E. del Valle, C. Tejedor, D. Sanvitto, and F. P. Laussy, arXiv:0907.1302 (2009).