Observers Programme. She does not participate in, or contribute to, the
definition of the Guaranteed Time Programme of the CHEOPS mission through
which observations described in this paper have been taken, nor to any aspect
of target selection for the programme. This work was granted access to the HPC
resources of MesoPSL financed by the Region Ile de France and the project
Equip@Meso (reference ANR-10-EQPX-29-01) of the programme Investissements
d’Avenir supervised by the Agence Nationale pour la Recherche. This work was
also partially supported by a grant from the Simons Foundation (PI Queloz,
grant number 327127). IRI acknowledges support from the Spanish Ministry of
Science and Innovation and the European Regional Development Fund through
grant PGC2018-098153-B- C33, as well as the support of the Generalitat de
Catalunya/CERCA programme. GyMSz acknowledges the support of the Hungarian
National Research, Development and Innovation Office (NKFIH) grant K-125015, a
a PRODEX Experiment Agreement No. 4000137122, the Lendület LP2018-7/2021 grant
of the Hungarian Academy of Science and the support of the city of
Szombathely. NAW acknowledges UKSA grant ST/R004838/1.

## References

  * Aigrain et al. (2004) Aigrain, S., Favata, F., & Gilmore, G. 2004, A&A, 414, 1139
  * Aigrain et al. (2012) Aigrain, S., Pont, F., & Zucker, S. 2012, MNRAS, 419, 3147
  * Alvan et al. (2014) Alvan, L., Brun, A. S., & Mathis, S. 2014, A&A, 565, A42
  * Appourchaux et al. (2010) Appourchaux, T., Belkacem, K., Broomhall, A. M., et al. 2010, A&A Rev., 18, 197
  * Appourchaux et al. (2018) Appourchaux, T. et al. 2018, A&A, 617, A108
  * Asplund et al. (2000) Asplund, M., Nordlund, Å., Trampedach, R., Allende Prieto, C., & Stein, R. F. 2000, A&A, 359, 729
  * Bailer-Jones et al. (2021) Bailer-Jones, C. A. L., Rybizki, J., Fouesneau, M., Demleitner, M., & Andrae, R. 2021, AJ, 161, 147
  * Ballot et al. (2021) Ballot, J., Roudier, T., Malherbe, J. M., & Frank, Z. 2021, A&A, 652, A103
  * Basri et al. (2010) Basri, G., Walkowicz, L. M., Batalha, N., et al. 2010, ApJ, 713, L155
  * Bastien et al. (2013) Bastien, F. A., Stassun, K. G., Basri, G., & Pepper, J. 2013, Nature, 500, 427
  * Bastien et al. (2016) Bastien, F. A., Stassun, K. G., Basri, G., & Pepper, J. 2016, ApJ, 818, 43
  * Bastien et al. (2014) Bastien, F. A., Stassun, K. G., Pepper, J., et al. 2014, AJ, 147, 29
  * Beeck et al. (2013) Beeck, B., Cameron, R. H., Reiners, A., & Schüssler, M. 2013, A&A, 558, A49
  * Beeck et al. (2015) Beeck, B., Schüssler, M., Cameron, R. H., & Reiners, A. 2015, A&A, 581, A43
  * Benz et al. (2021) Benz, W., Broeg, C., Fortier, A., et al. 2021, Experimental Astronomy, 51, 109
  * Blackwell & Shallis (1977) Blackwell, D. E. & Shallis, M. J. 1977, MNRAS, 180, 177
  * Bonfanti et al. (2021) Bonfanti, A., Delrez, L., Hooton, M. J., et al. 2021, A&A, 646, A157
  * Bonfanti et al. (2016) Bonfanti, A., Ortolani, S., & Nascimbeni, V. 2016, A&A, 585, A5
  * Bonfanti et al. (2015) Bonfanti, A., Ortolani, S., Piotto, G., & Nascimbeni, V. 2015, A&A, 575, A18
  * Boumier & Dame (1993) Boumier, P. & Dame, L. 1993, Exp. Astron., 4, 87
  * Brun & Browning (2017) Brun, A. S. & Browning, M. K. 2017, Living Reviews in Solar Physics, 14, 4
  * Bugnet et al. (2018) Bugnet, L., García, R. A., Davies, G. R., et al. 2018, A&A, 620, A38
  * Castelli & Kurucz (2003) Castelli, F. & Kurucz, R. L. 2003, in IAU Symposium, Vol. 210, Modelling of Stellar Atmospheres, ed. N. Piskunov, W. W. Weiss, & D. F. Gray, A20
  * Cegla et al. (2014) Cegla, H. M., Stassun, K. G., Watson, C. A., Bastien, F. A., & Pepper, J. 2014, ApJ, 780, 104
  * Cegla et al. (2019) Cegla, H. M., Watson, C. A., Shelyag, S., Mathioudakis, M., & Moutari, S. 2019, ApJ, 879, 55
  * Chaplin et al. (2019) Chaplin, W. J., Cegla, H. M., Watson, C. A., Davies, G. R., & Ball, W. H. 2019, AJ, 157, 163
  * Collier Cameron et al. (2019) Collier Cameron, A., Mortier, A., Phillips, D., et al. 2019, MNRAS, 487, 1082
  * Dravins (1987) Dravins, D. 1987, A&A, 172, 200
  * Dravins (1988) Dravins, D. 1988, in IAU Symposium, Vol. 132, The Impact of Very High S/N Spectroscopy on Stellar Physics, ed. G. Cayrel de Strobel & M. Spite, 239
  * Dravins et al. (1981) Dravins, D., Lindegren, L., & Nordlund, A. 1981, A&A, 96, 345
  * Dravins et al. (2021) Dravins, D., Ludwig, H.-G., & Freytag, B. 2021, A&A, 649, A17
  * Dravins & Nordlund (1990a) Dravins, D. & Nordlund, A. 1990a, A&A, 228, 184
  * Dravins & Nordlund (1990b) Dravins, D. & Nordlund, A. 1990b, A&A, 228, 203
  * Dumusque et al. (2021) Dumusque, X., Cretignier, M., Sosnowska, D., et al. 2021, A&A, 648, A103
  * Dumusque et al. (2011) Dumusque, X., Udry, S., Lovis, C., Santos, N. C., & Monteiro, M. J. P. F. G. 2011, A&A, 525, A140
  * Elsworth et al. (1994) Elsworth, Y., Howe, R., Isaak, G. R., et al. 1994, Monthly Notices of the Royal Astronomical Society, 269, 529
  * Findeisen et al. (2011) Findeisen, K., Hillenbrand, L., & Soderblom, D. 2011, The Astronomical Journal, 142, 23
  * Foreman-Mackey et al. (2013) Foreman-Mackey, D. et al. 2013, PASP, 125, 306
  * Fröhlich et al. (1997) Fröhlich, C., Andersen, B. N., Appourchaux, T., et al. 1997, Sol. Phys., 170, 1
  * Fröhlich et al. (1995) Fröhlich, C., Romero, J., Roth, H., et al. 1995, Sol. Phys., 162, 101
  * Futyan et al. (2020) Futyan, D., Fortier, A., Beck, M., et al. 2020, A&A, 635, A23
  * Gabriel et al. (1995) Gabriel, A. H. et al. 1995, Sol. Phys., 162, 61
  * Gaia Collaboration et al. (2021) Gaia Collaboration, Brown, A. G. A., Vallenari, A., et al. 2021, A&A, 649, A1
  * Garcia et al. (2005) Garcia, R. et al. 2005, A&A, 442, 385
  * García & Ballot (2019) García, R. A. & Ballot, J. 2019, Living Reviews in Solar Physics, 16, 4
  * Gray (2009) Gray, D. F. 2009, ApJ, 697, 1032
  * Hall (2008) Hall, J. C. 2008, Living Reviews in Solar Physics, 5, 2
  * Harvey (1988) Harvey, J. W. 1988, in IAU Symposium, Vol. 123, Advances in Helio- and Asteroseismology, ed. J. Christensen-Dalsgaard & S. Frandsen, 497
  * Hatzes et al. (2011) Hatzes, A. P., Fridlund, M., Nachmani, G., et al. 2011, ApJ, 743, 75
  * Houdek & Dupret (2015) Houdek, G. & Dupret, M.-A. 2015, Living Reviews in Solar Physics, 12, 8
  * Hoyer et al. (2020) Hoyer, S., Guterman, P., Demangeon, O., et al. 2020, A&A, 635, A24
  * Huber et al. (2011) Huber, D., Bedding, T. R., Stello, D., et al. 2011, ApJ, 743, 143
  * Jiménez et al. (2002) Jiménez, A., Roca Cortés, T., & Jiménez-Reyes, S. J. 2002, Sol. Phys., 209, 247
  * Kallinger et al. (2014) Kallinger, T., De Ridder, J., Hekker, S., et al. 2014, A&A, 570, A41
  * Kallinger et al. (2016) Kallinger, T., Hekker, S., Garcia, R. A., Huber, D., & Matthews, J. M. 2016, Science Advances, 2, 1500654
  * Kjeldsen & Bedding (1995) Kjeldsen, H. & Bedding, T. R. 1995, A&A, 293, 87
  * Kurucz (1993) Kurucz, R. L. 1993, SYNTHE spectrum synthesis programs and line data
  * Lindegren et al. (2021) Lindegren, L., Bastian, U., Biermann, M., et al. 2021, A&A, 649, A4
  * Lovis & Fischer (2010) Lovis, C. & Fischer, D. 2010, in Exoplanets, ed. S. Seager, 27–53
  * Ludwig (2006) Ludwig, H. G. 2006, A&A, 445, 661
  * Ludwig & Steffen (2013) Ludwig, H. G. & Steffen, M. 2013, Memorie della Societa Astronomica Italiana Supplementi, 24, 53
  * Magic et al. (2013) Magic, Z., Collet, R., Asplund, M., et al. 2013, A&A, 560, A8
  * Marigo et al. (2017) Marigo, P., Girardi, L., Bressan, A., et al. 2017, ApJ, 835, 77
  * Martin et al. (2005) Martin, D. C., Fanson, J., Schiminovich, D., et al. 2005, ApJ, 619, L1
  * Mathur et al. (2011) Mathur, S., Hekker, S., Trampedach, R., et al. 2011, ApJ, 741, 119
  * Maxted et al. (2022) Maxted, P. F. L., Ehrenreich, D., Wilson, T. G., et al. 2022, MNRAS, 514, 77
  * Meunier & Lagrange (2020) Meunier, N. & Lagrange, A. M. 2020, A&A, 642, A157
  * Meunier et al. (2015) Meunier, N., Lagrange, A. M., Borgniet, S., & Rieutord, M. 2015, A&A, 583, A118
  * Michel et al. (2008) Michel, E., Baglin, A., Auvergne, M., et al. 2008, Science, 322, 558
  * Muller et al. (2018) Muller, R., Hanslmeier, A., Utz, D., & Ichimoto, K. 2018, A&A, 616, A87
  * Nardiello et al. (2019) Nardiello, D., Borsato, L., Piotto, G., et al. 2019, MNRAS, 490, 3806
  * Nardiello et al. (2021) Nardiello, D., Deleuil, M., Mantovan, G., et al. 2021, MNRAS, 505, 3767
  * Nardiello et al. (2020) Nardiello, D., Piotto, G., Deleuil, M., et al. 2020, MNRAS, 495, 4924
  * Newville et al. (2014) Newville, M., Stensitzki, T., Allen, D. B., & Ingargiola, A. 2014, LMFIT: Non-Linear Least-Square Minimization and Curve-Fitting for Python
  * Nordlund (1982) Nordlund, A. 1982, A&A, 107, 1
  * Nordlund (1985) Nordlund, A. 1985, Sol. Phys., 100, 209
  * Nordlund & Dravins (1990) Nordlund, A. & Dravins, D. 1990, A&A, 228, 155
  * Nordlund & Galsgaard (1995) Nordlund, A. & Galsgaard, K. 1995, A 3D MHD code for Parallel Computers, Tech. rep.
  * Nordlund et al. (2009) Nordlund, Å., Stein, R. F., & Asplund, M. 2009, Living Reviews in Solar Physics, 6, 2
  * Noyes et al. (1984) Noyes, R. W., Hartmann, L. W., Baliunas, S. L., Duncan, D. K., & Vaughan, A. H. 1984, ApJ, 279, 763
  * Oshagh et al. (2017) Oshagh, M., Santos, N. C., Figueira, P., et al. 2017, A&A, 606, A107
  * Pallé et al. (1999) Pallé, P. L., Roca Cortés, T., Jiménez, A., GOLF Team, & Virgo Team. 1999, in Astronomical Society of the Pacific Conference Series, Vol. 173, Stellar Structure: Theory and Test of Connective Energy Transport, ed. A. Gimenez, E. F. Guinan, & B. Montesinos, 297
  * Pande et al. (2018) Pande, D., Bedding, T. R., Huber, D., & Kjeldsen, H. 2018, MNRAS, 480, 467
  * Pepe et al. (2021) Pepe, F., Cristiani, S., Rebolo, R., et al. 2021, A&A, 645, A96
  * Pepe et al. (2013) Pepe, F., Cristiani, S., Rebolo, R., et al. 2013, The Messenger, 153, 6
  * Perryman (2018) Perryman, M. 2018, The Exoplanet Handbook
  * Philidet et al. (2020) Philidet, J., Belkacem, K., Ludwig, H. G., Samadi, R., & Barban, C. 2020, A&A, 644, A171
  * Piskunov & Valenti (2017) Piskunov, N. & Valenti, J. A. 2017, A&A, 597, A16
  * Piskunov et al. (1995) Piskunov, N. E., Kupka, F., Ryabchikova, T. A., Weiss, W. W., & Jeffery, C. S. 1995, A&AS, 112, 525
  * Ricker et al. (2015) Ricker, G. R., Winn, J. N., Vanderspek, R., et al. 2015, Journal of Astronomical Telescopes, Instruments, and Systems, 1, 014003
  * Rodríguez et al. (2016) Rodríguez, E., Rodríguez-López, C., López-González, M. J., et al. 2016, MNRAS, 457, 1851
  * Rodríguez Díaz et al. (2022) Rodríguez Díaz, L. F., Bigot, L., Aguirre Børsen-Koch, V., et al. 2022, MNRAS, 514, 1741
  * Saar et al. (2003) Saar, S. H., Hatzes, A., Cochran, W., & Paulson, D. 2003, in The Future of Cool-Star Astrophysics: 12th Cambridge Workshop on Cool Stars, Stellar Systems, and the Sun, ed. A. Brown, G. M. Harper, & T. R. Ayres, Vol. 12, 694–698
  * Salabert et al. (2016) Salabert, D., García, R. A., Beck, P. G., et al. 2016, A&A, 596, A31
  * Salaris & Cassisi (2017) Salaris, M. & Cassisi, S. 2017, Royal Society Open Science, 4, 170192
  * Salmon et al. (2021) Salmon, S. J. A. J., Van Grootel, V., Buldgen, G., Dupret, M. A., & Eggenberger, P. 2021, A&A, 646, A7
  * Samadi et al. (2013) Samadi, R., Belkacem, K., Ludwig, H. G., et al. 2013, A&A, 559, A40
  * Santos et al. (2000) Santos, N. C., Mayor, M., Naef, D., et al. 2000, A&A, 361, 265
  * Santos et al. (2013) Santos, N. C., Sousa, S. G., Mortier, A., et al. 2013, A&A, 556, A150
  * Scargle (1982) Scargle, J. D. 1982, ApJ, 263, 835
  * Schanche et al. (2020) Schanche, N., Hébrard, G., Collier Cameron, A., et al. 2020, MNRAS, 499, 428
  * Schuster (1898) Schuster, A. 1898, Terrestrial Magnetism, 3, 13
  * Scuflaire et al. (2008) Scuflaire, R., Théado, S., Montalbán, J., et al. 2008, Ap&SS, 316, 83
  * Seleznyov et al. (2011) Seleznyov, A. D., Solanki, S. K., & Krivova, N. A. 2011, A&A, 532, A108
  * Skrutskie et al. (2006) Skrutskie, M. F., Cutri, R. M., Stiening, R., et al. 2006, AJ, 131, 1163
  * Sneden (1973) Sneden, C. A. 1973, PhD thesis, THE UNIVERSITY OF TEXAS AT AUSTIN.
  * Soto & Jenkins (2018) Soto, M. G. & Jenkins, J. S. 2018, A&A, 615, A76
  * Sousa (2014) Sousa, S. G. 2014, [arXiv:1407.5817] [arXiv:1407.5817]
  * Sousa et al. (2021) Sousa, S. G., Adibekyan, V., Delgado-Mena, E., et al. 2021, A&A, 656, A53
  * Sousa et al. (2015) Sousa, S. G., Santos, N. C., Adibekyan, V., Delgado-Mena, E., & Israelian, G. 2015, A&A, 577, A67
  * Sousa et al. (2007) Sousa, S. G., Santos, N. C., Israelian, G., Mayor, M., & Monteiro, M. J. P. F. G. 2007, A&A, 469, 783
  * Sousa et al. (2008) Sousa, S. G., Santos, N. C., Mayor, M., et al. 2008, A&A, 487, 373
  * Stein & Nordlund (2001) Stein, R. F. & Nordlund, Å. 2001, ApJ, 546, 585
  * Stein & Nordlund (2003) Stein, R. F. & Nordlund, Å. 2003, in Astronomical Society of the Pacific Conference Series, Vol. 288, Stellar Atmosphere Modeling, ed. I. Hubeny, D. Mihalas, & K. Werner, 519
  * Sulis et al. (2020a) Sulis, S., Lendl, M., Hofmeister, S., et al. 2020a, A&A, 636, A70
  * Sulis et al. (2017) Sulis, S., Mary, D., & Bigot, L. 2017, IEEE TSP, 65, 2136
  * Sulis et al. (2020b) Sulis, S., Mary, D., & Bigot, L. 2020b, A&A, 635, A146
  * Svensson & Ludwig (2005) Svensson, F. & Ludwig, H. G. 2005, in ESA Special Publication, Vol. 560, 13th Cambridge Workshop on Cool Stars, Stellar Systems and the Sun, ed. F. Favata, G. A. J. Hussain, & B. Battrick, 979
  * Tayar et al. (2019) Tayar, J., Stassun, K. G., & Corsaro, E. 2019, ApJ, 883, 195
  * Torres et al. (2012) Torres, G., Fischer, D. A., Sozzetti, A., et al. 2012, ApJ, 757, 161
  * Trampedach et al. (2013) Trampedach, R., Asplund, M., Collet, R., Nordlund, Å., & Stein, R. F. 2013, ApJ, 769, 18
  * Trampedach et al. (1998) Trampedach, R., Christensen-Dalsgaard, J., Nordlund, A., & Stein, R. F. 1998, in The First MONS Workshop: Science with a Small Space Telescope, ed. H. Kjeldsen & T. R. Bedding, 59
  * Tremblay et al. (2013) Tremblay, P. E., Ludwig, H. G., Freytag, B., Steffen, M., & Caffau, E. 2013, A&A, 557, A7
  * Valenti & Piskunov (1996) Valenti, J. A. & Piskunov, N. 1996, A&AS, 118, 595
  * Wright et al. (2010) Wright, E. L., Eisenhardt, P. R. M., Mainzer, A. K., et al. 2010, AJ, 140, 1868
  * Wright (2005) Wright, J. T. 2005, PASP, 117, 657
  * Zechmeister & Kürster (2009) Zechmeister, M. & Kürster, M. 2009, A&A, 496, 577

##  Appendix A Rotation period estimates based on TESS observations

To measure the stellar rotation period of HD 88595 and HD 67458, we use the
TESS light curves introduced in Sect. 2.3. We analyze the light curves using
the Generalized Lomb-Scargle periodogram (Zechmeister & Kürster 2009), after
having masked out the most obvious outliers and after applying a linear
detrending in time to each sector.

###  A.1 HD 88595

This star has been observed in sectors 9 and 35. For the latter we detect a
clear peak around $\sim 3$ days, while for the former we get a broader comb of
aliases between 1 and 10 days. The joint analysis of the two sectors returns a
period of $3.1151\pm 0.0003$ days and a photometric amplitude of $83\pm 4$
ppm. The detected period would be consistent with rotation and with
$v\sin{i}=7.2$ km s-1 assuming that the inclination of the rotation axis is
$\sim 16^{\circ}$, meaning the star is seen nearly pole on. In such a
scenario, the active regions on the stellar surface are always visible (unless
they have nearly equatorial latitudes), and this would explain why the star
has a low photometric signal despite being a fast rotator.

###  A.2 HD 67458

This star has been observed in sectors 7, 8, and 34.

The light curve obtained in sector 8 is the most problematic. The data release
note212121https://archive.stsci.edu/missions/tess/doc/tess_drn/tess_sector_34_drn50_v02.pdf
reports noisy quaternions, and the periodogram is difficult to interpret. For
all these reasons we exclude this sector from our analysis.

The periodogram of sector 7 suggests a periodicity of $5.80\pm 0.03$ days,
while for sector 34 we get a period of $10.57\pm 0.06$ days. In both cases the
photometric amplitude is $\sim 100$ ppm. The joint analysis of the two sectors
show significant peaks at these two periods in the periodogram. Since the two
periods are roughly in a 1:2 proportion, we thus argue that the period
detected in sector 7 is the first harmonic of the true rotation period. The
detected period of $10.57\pm 0.06$ days would be consistent with rotation and
with $v\sin{i}=2.179$ km s-1 assuming an inclination of $\sim 26^{\circ}$,
meaning this star is also seen nearly pole-on.

##  Appendix B Exoplanets in the habitable zone

As a reference in the paper, we evaluate the planetary timescales (transit and
in-egress durations, orbital period) and amplitudes (transit depth, RV semi-
amplitude) of an Earth-like planet ($1$ $M_{\oplus}$, $1$ $R_{\oplus}$) that
would orbit in the HZ of HD 67458 and HD 88595. For that purpose, we assume an
impact parameter $b=0$, an eccentricity $e=0$, and the inclination to be
$i=90^{\circ}$. We use the stellar parameters given in Table 4 and the
classical equations given in Perryman (2018).

We compute the transit depth as

$\delta=\Big{(}\frac{R_{p}}{R_{\star}}\Big{)}^{2},$

and obtain $\delta=80.6$ ppm for HD 67458, and $\delta=32.18$ ppm for HD
88596.

We compute the orbital separation of the planet in the middle of the HZ as

$a=\sqrt{\frac{L_{\star}}{L_{\odot}}},$

with $L_{\star}=4\pi\sigma R_{\star}^{2}T_{\mathrm{eff}}^{4}$ the stellar
luminosity in SI from the Stefan-Boltzmann law, $\sigma=5.67\times 10^{-8}$ kg
s-3 K-4 the Stefan-Boltzmann constant, and $L_{\odot}=3.828\times 10^{26}$ W
the solar luminosity according to IAU. We obtain $a=1.04$ AU for HD 67458 and
$a=1.86$ AU for HD 88595. Using Kepler’s 3rd law, we derive the corresponding
orbital periods and find $P=402.3$ days for HD 67458 and $P=802.2$ days for HD
88595.

We express the total transit duration as

$T_{T}\approx\frac{P}{\pi}\leavevmode\nobreak\ \frac{(R_{\star}+R_{p})}{a},$

and obtain $T_{T}=14.12$ hours for HD 67458, and $T_{T}=24.8$ hours for HD
88595.

The duration of full transit is computed as

$T_{F}\approx\frac{P}{\pi}\leavevmode\nobreak\ \frac{(R_{\star}-R_{p})}{a},$

which gives a duration of the transit ingress expressed as

$T_{in}=\frac{T_{T}-T_{F}}{2}.$

We obtain $T_{in}=7.5$ min for HD 67458, and $T_{in}=8.4$ min for HD 88595.

Finally, the RV semi-amplitude of an Earth-mass planet is expressed in m/s as
(Lovis & Fischer 2010)

$K=28.4329\leavevmode\nobreak\
\Big{(}\frac{M_{p}}{M_{J}}\Big{)}\Big{(}\frac{M_{\star}+M_{p}}{M_{\odot}}\Big{)}^{-0.5}\Big{(}\frac{a}{1\leavevmode\nobreak\
{\rm AU}}\Big{)}^{-0.5},$

with $M_{J}$ the Jupiter mass. We obtain $K=9$ cm/s for HD 67458 and $K=5.6$
cm/s for HD 88595.

##  Appendix C Solar granulation as seen from different instruments

Stellar activity is wavelength-dependent. To evaluate this dependence with
solar observations, we compare in this section the periodograms of VIRGO/SoHO
solar irradiance (see Sec. 2.4) with HMI/SDO photometric observations (see
description in Sulis et al. 2020a). We also compare the GOLF/SoHO
spectrophotometric data (see Sec. 2.4) with HARPS-N spectroscopic
observations222222http://cdsarc.u-strasbg.fr/viz-bin/cat/J/A+A/648/A103
(Collier Cameron et al. 2019, Dumusque et al. 2021).

For each dataset, we compute the averaged periodogram based on $1$-day
regularly sampled time series of VIRGO (year:1996), HMI (year:2008), and GOLF
observations (year:1996). Since HARPS-N observations (years:2015-2018) are
taken from the ground, they are irregularly sampled and their durations are
around $6$-$8$-hours. For this reason, the Lomb-Scargle periodogram (Scargle
1982) is used to compute $P_{L}$ in Eq. (3) for HARPS-N data, while the
classical periodogram (Schuster 1898) is used for the other datasets.

The resulting averaged periodograms are shown in Fig. 18. We observe a very
good match between the slopes of the different periodograms in the frequency
region dominated by the stellar granulation signal. While the amplitudes of
this signal are wavelength-dependent (see Fig 3 for example), the flicker
index, which is based on the periodogram’s slope, is not (at least, in first
approximation). This makes this indicator a useful diagnostic tool for
granulation studies.

Figure 18: Comparison of averaged periodograms from VIRGO and HMI observations
(left), and from GOLF and HARPS-N observations (right). Note the instrumental
artifact affecting the high frequencies ($\nu\sim 7000\leavevmode\nobreak\
\mu$Hz) in HMI data.

##  Appendix D F8 scaling relation from 3D stellar atmosphere models

Since one of the scaling relations provided by Rodríguez Díaz et al. (2022)
was for the standard deviation ($\sigma_{HD}$), we calculated a new scaling
relation for F8, so we can compare directly with our two CHEOPS targets.

Table 10 summarizes the F8 values for the models used and Fig 19 shows the F8
values for the 3D stellar atmosphere models at solar metallicity and with
${\rm log}g\geq 3.0$, together with a fit for the scaling relation, which is
of the form:

$\log_{10}{\rm F8}_{HD}=a\log_{10}{\nu_{\rm max}}+b.$

We find $a=-0.530\pm 0.040$ and $b=3.506\pm 0.112$ leading to the relation
F8${}_{HD}\propto\nu_{\rm max}^{-0.530}$ that we used, instead on the RMS
based relation $\sigma_{HD}\sim\nu_{\rm max}^{-0.567}$ initially published in
Rodríguez Díaz et al. (2022).

Table 10: Stellar and physical parameters of the 3D stellar models at solar metallicity used to determine the scaling relation of F8. From left to right: model name, target effective temperature (target $T_{\rm eff}$, in K) corresponding to the $T_{\rm eff}$ that we aimed to achieve for each model, mean $T_{\rm eff}$ (K) from the time series, surface gravity in logarithm scale (${\rm log}g$) (cm/s2), metallicity [Fe/H], $\nu_{\rm max}$ ($\mu$Hz), standard deviation of the brightness fluctuations ($\sigma_{HD}$) (ppm), and F8HD (ppm). Model | Target $T_{\rm eff}$ | $T_{\rm eff}$ | ${\rm log}g$ | [Fe/H] | $\nu_{\rm max}$ | $\sigma_{HD}$ | F8HD
---|---|---|---|---|---|---|---
t50g30m00 | 5000 | 4960 | 3.00 | 0.0 | 121.516 | 219 | 286
t47g32m00 | 4750 | 4727 | 3.25 | 0.0 | 221.351 | 199 | 220
t55g35m00 | 5500 | 5516 | 3.50 | 0.0 | 364.387 | 136 | 145
t50g35m00 | 5000 | 4958 | 3.50 | 0.0 | 384.345 | 142 | 153
t65g40m00 | 6500 | 6413 | 4.00 | 0.0 | 1068.672 | 88 | 83
t60g40m00 | 6000 | 5962 | 4.00 | 0.0 | 1108.355 | 83 | 88
t55g40m00 | 5500 | 5462 | 4.00 | 0.0 | 1157.975 | 81 | 85
t5777g44m00 | 5777 | 5759 | 4.44 | 0.0 | 3106 | 38 | 46

Figure 19: F8 values obtained for 3D stellar atmosphere models at solar
metallicity compared to $\nu\rm_{max}$, and the derived scaling relation.

##  Appendix E Influence of observational parameters on the flicker index

The flicker index is linked to several observational settings that affect its
estimation. In this section, we investigate how the length of the subseries,
the temporal sampling, and the level of high-frequency noise impact the
inferred flicker indexes.

The flicker index is estimated from the averaged periodogram (3), which is
computed based on short-duration subseries. To evaluate the influence of the
different parameters, we use VIRGO solar observations (see Sec. 2.4).

Fig. 20 shows the flicker index as a function of the number of $1$-day
subseries ($L$) for different parameters. In all panels, the reference setting
if $\Delta t=1$ min, $T=1$ day and $\sigma_{w}<30$ ppm (VIRGO high-frequency
noise level). The inferred flicker indexes converge toward a fixed estimate
with the increase of $L$ (see red horizontal lines). We note this “asymptotic”
estimate $\alpha_{g,\infty}$ in the following.

The first column shows the influence of the time sampling on the flicker
index. From top to bottom, the subseries are sampled at $\Delta t=1$ min, $5$
min, and $10$ min. For a solar-like star, we see that a too long temporal
sampling, as $\Delta t\geq 5$ min, affects the flicker index value since the
high-frequency cut-off is cropped. The convergence of the inferred index
toward $\alpha_{g,\infty}$ is fast ($L\leq 10$), for all $\Delta t$, and no
specific bias is observed at low $L$ (the asymptotic index $\alpha_{g,\infty}$
is within the $1\sigma$ errorbars). We note that, while $\Delta t$ is not
expected to affect the flicker indexes obtained from CHEOPS and the future
PLATO space data (since they both have $\Delta t<5$ min), it can be critical
for ground-based RV observations where the exposure time is, in general, $\geq
5$ min.

The second column shows the influence of the subseries duration on the
inferred flicker index. From top to bottom, the subseries have duration of $5$
days, $1$ day, and $8$ hours. Again, we observe a fast convergence of the
flicker index toward $\alpha_{g,\infty}$ for all $T$. It indicates that the
relatively short length of our CHEOPS observations ($8$ hours) are enough to
infer accurate flicker index. However, the low number of visits ($3$ to $4$)
may slightly bias our inferred values. We note that long observations (e.g.,
$T=5$ days) does not improve the flicker index convergence toward
$\alpha_{g,\infty}$.

The last column shows the influence of the high-frequency noise level on the
inferred flicker index. From top to bottom, we added to VIRGO observation a
WGN of standard deviation $\sigma_{w}=0$ ppm (our reference), $85$ ppm
($\approx$ CHEOPS high-frequency noise level), and $96$ ppm ($\approx$ TESS
high-frequency noise level). As discussed in Sec. 5.2, the index values
logically decrease with the level of this high-frequency noise (that was
largely dominated by photon noise for our two stars observed by CHEOPS).

Figure 20: Inferred flicker indexes as a function of the number of subseries
($L$) used to compute the averaged periodogram in Eq. (3) based on VIRGO solar
data. Left column: influence of the time sampling, $\Delta t=1,5,10$ min (from
top to bottom). Middle column: influence of the duration of the subseries,
$T=5$ day, $1$ day, $8$-hours (from top to bottom). Right column: influence of
the high-frequency noise level, $\sigma_{W}<30$ ppm (VIRGO data, top),
$\sigma_{W}=82$ ppm (level measured on CHEOPS HD 67458 periodogram),
$\sigma_{W}=92$ ppm (approximately the level measured on TESS HD 67458
periodogram). The red horizontal line indicates the flicker index value for
large $L$ ($\alpha_{g,\infty}$).

##  Appendix F Flicker indexes of Kepler and CHEOPS bright targets

In Table 11, we show the results from the flicker analyses on some targets
described in the present study. Kepler targets comes from Sulis et al.
(2020a). The table shows the cut-off frequencies ($f_{c}$ and $f_{g}$) and
flicker indexes ($a_{g}$) inferred from the MCMC analyses for a time series
affected by a high-frequency noise level $\sigma_{W}$ (see Sec. 5.1). The
errorbars indicated in the table are the more conservative ones. The flicker
index interpolated at the high-frequency noise level of $30$ ppm is indicated
by $\widehat{\alpha_{g}}(\sigma_{30})$. Stellar parameters for Kepler targets
($M_{\star}$, $R_{\star}$, $T_{\mathrm{eff}}$, ${\rm log}g$, mag) are from the
Kepler_stellar17.csv.gz catalog available at
https://archive.stsci.edu/kepler/catalogs.html.

Table 11: Results from the flicker analyses on some targets described in the present study. This table is available in its entirety in machine-readable form ($315$ targets). We note that values $\widehat{\alpha_{g}}(\sigma_{30})<0.2$ have been replaced by “-1” in the online table.  Target | Mission | $f_{c}$ | $f_{g}$ | $\sigma_{w}$ | $a_{g}$ | $\widehat{\alpha_{g}}(\sigma_{30})$ | $M_{\star}$ | $R_{\star}$ | $T_{eff}$ | $\log{g}$ | mag
---|---|---|---|---|---|---|---|---|---|---|---
|  |  [$\mu$Hz] |  [$\mu$Hz] | [ppm] |  |  |  [$M_{\odot}$] |  [$R_{\odot}$] | [K] | [cgs] |
HD 67458 | CHEOPS | $3638$ | $480$ | $81$ | $0.526\pm 0.120$ | $0.888$ | $0.94\pm 0.04$ | $1.02\pm 0.02$ | $5833\pm 62$ | $4.37\pm 0.10$ | $6.8$
HD 88595 | CHEOPS | $1036$ | $351$ | $77$ | $0.608\pm 0.493$ | $2.079$ | $1.35\pm 0.06$ | $1.62\pm 0.02$ | $6205\pm 35$ | $3.99\pm 0.06$ | $6.5$
KIC 1430163 | Kepler | $824$ | $378$ | $115$ | $0.61\pm 0.05$ | $1.35$ | $1.29\pm 0.08$ | $1.46\pm 0.06$ | $6586\pm 85$ | $4.22\pm 0.01$ | $9.6$
KIC 2837475 | Kepler | $837$ | $350$ | $83$ | $0.99\pm 0.24$ | $1.60$ | $1.34\pm 0.07$ | $1.58\pm 0.05$ | $6642\pm 92$ | $4.16\pm 0.01$ | $8.5$
KIC 3424541 | Kepler | $448$ | $251$ | $142$ | $1.36\pm 0.14$ | $2.36$ | $1.37\pm 0.21$ | $2.45\pm 0.16$ | $6100\pm 96$ | $3.80\pm 0.02$ | $9.7$
KIC 3427720 | Kepler | $2029$ | $607$ | $95$ | $0.34\pm 0.06$ | $0.78$ | $1.03\pm 0.08$ | $1.09\pm 0.04$ | $6045\pm 81$ | $4.38\pm 0.01$ | $9.1$
KIC 3456181 | Kepler | $642$ | $392$ | $124$ | $0.60\pm 0.47$ | $1.15$ | $1.27\pm 0.18$ | $2.03\pm 0.15$ | $6372\pm 77$ | $3.93\pm 0.01$ | $9.7$
KIC 3656476 | Kepler | $1415$ | $385$ | $116$ | $0.65\pm 0.03$ | $1.37$ | $1.03\pm 0.10$ | $1.30\pm 0.06$ | $5666\pm 76$ | $4.22\pm 0.01$ | $9.5$
KIC 3733735 | Kepler | $1320$ | $360$ | $84$ | $0.86\pm 0.04$ | $1.52$ | $1.30\pm 0.07$ | $1.38\pm 0.05$ | $6676\pm 80$ | $4.27\pm 0.01$ | $8.4$
KIC 3735871 | Kepler | $2176$ | $357$ | $110$ | $0.27\pm 0.01$ | $0.65$ | $1.05\pm 0.08$ | $1.09\pm 0.04$ | $6108\pm 92$ | $4.39\pm 0.02$ | $9.7$

##  Appendix G Bisector inverse span and curvature optimization

We explore the impact of optimizing the BIS and curvature region definitions
by searching for the regions of the bisector most sensitive to changes in net
RV. This analysis is performed only on the binned data in Sec. 8.1 as the CCF
shape could potentially be impacted by the p-modes in the individual data. For
the optimal BIS and curvature range exploration, we employ two approaches: in
the first instance we employ an MCMC (implemented with EMCEE; Foreman-Mackey
et al. 2013) to explore the top and bottom (middle) definitions, and in the
second case we search a fixed grid.

For the MCMC approach, we start 100 walkers in a tight Gaussian ball near the
standard definitions and let the walkers explore the parameters space for 1000
steps (excluding a burn-in of 100 steps). We set uniform priors to avoid the
continuum and CCF core (excluding the top 10% and bottom 95% of the CCF) and
to ensure the various regions are separate. We aim to minimize the residuals
between the measured RV and that predicted by a linear fit with the given
indicator. We note that this assumes a linear relationship between RV and
indicators or flux, which may well not be the case; it also does not account
for any potential time lags between RV and shape indicator. As such, this
approach may not identify the regions of the bisector most sensitive to
granulation, but it is a good first parameter space to search.

In the second approach, we loop over various top and bottom (middle) region
definitions in steps of 5% of the CCF depth, requiring the range of a given
region to be at least as large as one step size and that the regions do not
overlap. Similar to the MCMC approach, we limit the search to 10-95% of the
CCF depth. The “optimal” region definitions were then those that maximized the
correlation (as determined by the Spearman’s Rank test) between the BIS or
curvature and RV. This approach has the advantage that we maximally search all
possible region definitions (and do not need to worry about convergence), but
the disadvantage of having a fixed step size. If the relationship between BIS
or curvature and RV is linear, then this is equivalent to the previous
approach.

We consider instances where both approaches agree to be the most reliable. We
find there are often multiple top, bottom, and middle definitions that could
be considered optimal and the sign of the correlation can change depending on
either the region definition or the night of observations. We note that a
change in correlation sign can happen simply from moving to different regions
within the bisector and should not be cause for alarm. Nonetheless the
inconsistencies across nights indicate that we likely do not identify the
regions of the CCF bisector most sensitive to granulation (and may be
sensitive primarily to instrumental or telluric noise and/or low-number
statistics). Nonetheless, we outline the individual results below.

###  G.1 BIS results

The CCF bisector for HD 67458 is fairly straight with a slight blue-ward bend
near the top, and does not change significantly between nights (see Fig. 21).
Both nights 1 and 2 are optimized by probing a ‘top’ region very high up the
bisector ($\sim$15-20% the CCF depth) and a ‘bottom’ region defined just below
the blue-ward bend in the CCF ($\sim$20-40%). For night 3, this changes
slightly to probe the bottom of this bend ($\sim$25-35%) and near the middle
of the bisector ($\sim$35-55%). Oddly, for nights 1 and 3 we find a similar
correlation, with a Spearman’s rank correlation coefficient of $\sim$0.9, but
for night 2 the correlation is opposite in sign and weaker, $\sim$ -0.6 to
-0.8 depending on the exact range used. The reason for these differences is
unclear, but given the correlations seen with S/N in Sec. 8 we may need future
observations under more ideal observing conditions to reach any firm
conclusions. Alternatively, an optimized CCF mask or a line-by-line approach
may be needed as the current CCF template smears out too much of the bisector
curvature; this is the subject of a forthcoming analysis for a follow-up
paper.

As noted in Sec. 8, the bisectors for HD 88595 change significantly in shape
between the two nights observed (see Fig. 21). The bisector on night 1 is more
vertical, with less blue-shift near the CCF core compared to night 2; this is
in agreement with the fact that the FWHM is larger and contrast shallower for
night 1 (see Fig. 15). The broader CCF in night 2 might be linked to the lower
RV precision we find for this night. The backwards bend in the bisector is far
more prominent for this star, indicative of the more vigorous convection with
larger contrasts between granules and inter-granular lanes; this bend occurs
at a similar depth in both nights. Since binning to help mitigate the effects
of the p-modes only leaves 6 points per night, efforts to find the bisector
ranges most sensitive to granulation via maximizing the correlation with RV
should be taken with extreme caution. For night 1, the MCMC approach points
toward a ‘top’ region of $\sim$10-30% (above the bend in the bisector) and a
‘bottom’ region of $\sim$70-75% and a Spearman’s rank correlation coefficient
near -0.9, while the nested loop approach points toward $\sim$20-40% (top) and
$\sim$30-60% (bottom) with a correlation coefficient closer to -1.
Interestingly, ranges of 55-60% and 60-65% can lead to a positive correlation
coefficient near 1. For night 2, both approaches find optimal regions near
35-45% and 70-80% for the top and bottom, respectively, and a positive
correlation coefficient near 1. With the nested loop approach we find a
similar correlation sign and strength with a narrow region defined very near
the CCF core (85-90% and 90-95%) that might simply be tracing centroid shift
rather than any true asymmetry change. The differences between lines is
probably being driven by whatever is driving the larger shape changes between
the nights, potentially linked to observing conditions since a broader CCF
also leaders to poorer RV precision.

###  G.2 Bisector curvature results

For HD 67458, on night 1 we find strong correlations between RV and bisector
curvature regions (with a Spearman’s rank coefficient near 1) if the top,
middle, and bottom regions are approximately 15-20%, 35-50%, and 55-60% of the
CCF line depth. For night 2, the strongest Spearman’s rank correlation
coefficient is near -0.8, with regions of 10-15%, 35-40%, and 40-60% (found
with the nested loop approach – the maximum correlation found with the MCMC
approach was near 0.7 for regions near 30-35%, 60-70%, and 75-80%). Similar to
the BIS results, night 3 behaved somewhat similar to night 1 with a positive
correlation coefficient close to 0.9, for regions near 20-30%, 35-75%, and
80-95%. Interestingly, the optimal regions for nights 1 and 2 probe the top
two thirds of the CCF, while for night 3 nearly the whole bisector is probed;
this is somewhat similar to the BIS results where nights 1 and 2 favor higher
up regions in the bisector as compared to night 3.

For HD 88595, there are numerous combinations of top, middle, and bottom
region definitions that yield similar results, likely owing to the low number
of data points for each night of the binned data. For night 1, the optimal
combinations found with the MCMC approach were narrowly defined top regions at
the blue-ward bend of the bisector ($\sim$25-30% of the CCF depth), a large
middle region spanning most of the bisector ($\sim$40-80%), and a narrow
region near the bottom ($\sim$80-90%); this combination yielded Spearman’s
rank correlation coefficients near -0.9. The nested loop approach yielded 272
different combinations that were equally strongly correlated, both positively
and negatively, between RV and curvature; combinations that probed the top two
thirds were negatively correlated, while combinations probing the bottom two
thirds were positively correlated. For night 2, the MCMC approach led to
optimal regions defined with a narrow regions above the blue-ward bisector
bend ($\sim$10-20%), a region just below the bend ($\sim$35-55%), and a region
near the bottom of the bisector ($\sim$70-95%), with a correlation coefficient
near -0.9. The nested loop approach yielded 26 combinations of similar
strength; 25 of which led to anticorrelations near -1 (the exception, with a
positive correlation, was regions 40-55%, 65-75%, and 75-80%).

On one hand, since we can optimize the BIS and bisector curvature regions to
reveal strong correlations with RV, it is possible that we might be probing
the impact of granulation. However, since the bisectors do not seem to behave
in a coherent way across the nights, it is difficult to confirm the driving
force behind the changes we see. Forthcoming work will include a refined
treatment of the p-modes and an optimization to mitigate the effects of the
current CCF template masks smearing out the individual line shapes and
curvatures. This work would also benefit from further observations,
potentially at higher resolution.

Figure 21: Bisectors of the binned CCFs for HD 67458 (left) and HD 88595
(right) for all nights (night 1 indicated by ‘n1’ etc.). See Section 8.1 for
binning details.


# AINS: Affordable Indoor Navigation Solution via Line Color Identification
Using Mono-Camera for Autonomous Vehicles
††thanks: Identify applicable funding agency here. If none, delete this.

1st Nizamuddin Maitlo∗ Department of Computer Science
Sukkur IBA University
Sukkur, Pakistan
Corresponding Author<EMAIL_ADDRESS>2nd Nooruddin Noonari
Department of Computer Science
IBA Community College, Naushahro Feroze
Sindh, Pakistan
Corresponding Author<EMAIL_ADDRESS>3rd Kaleem Arshid  DITEN
University of Genova
Genova, Italy
<EMAIL_ADDRESS>4th Naveed Ahmed  IBA Community College Naushahro
Feroze
Sukkur IBA University
Sindh, Pakistan
<EMAIL_ADDRESS>5th Sathishkumar Duraisamy  Department of Mechanical
Engineering
Kathir College of Engineering
Tamilnadu, India.
<EMAIL_ADDRESS>
###### Abstract

Recently, researchers have been exploring various ways to improve the
effectiveness and efficiency of autonomous vehicles by researching new
methods, especially for indoor scenarios. Autonomous Vehicles in indoor
navigation systems possess many challenges especially the limited accuracy of
GPS in indoor scenarios. Several, robust methods have been explored for
autonomous vehicles in indoor scenarios to solve this problem, but the
ineffectiveness of the proposed methods is the high deployment cost.

To address the above-mentioned problems we have presented A low-cost indoor
navigation method for autonomous vehicles called Affordable Indoor Navigation
Solution (AINS) which is based on based on Monocular Camera. Our proposed
solution is mainly based on a mono camera without relying on various huge or
power-inefficient sensors to find the path, such as range finders and other
navigation sensors. Our proposed method shows that we can deploy autonomous
vehicles’ indoor navigation systems while taking into consideration the cost.
We can observe that the results shown by our solution are better than existing
solutions and we can reduce the estimated error and time consumption.

###### Index Terms:

Autonomous Vehicles, path detection, Indoor Navigation System, object
segmentation.

##  I Introduction

With the advancement of computer vision technology now we can give the power
of vision to machines and vehicles. The research on autonomous vehicles is
becoming more prominent to replace the traditional way of driving to improve
the quality of travel and reduce the chances of human error like fatigue, and
tiredness. Automation is widely welcomed in all aspects of life to replace
conventional transportation methods for industrial usage, self-driving vehicle
indoor navigation system is a popular research topic in the field of
automation [1][2].

Despite the popularity of indoor navigation systems researchers are facing
challenges while developing autonomous vehicle navigation methods due to the
low precision of GPS. Researchers are exploring many methods and there is a
potential area that can be further explored to enhance the robustness of the
indoor navigation system for autonomous vehicles [3][4].

There are several methods implemented in self-driving vehicles to achieve the
required task such as laser-based simultaneous localization and mapping (SLAM)
and visual SLAM [5]. Both approaches come equipped with the pros and cons such
as the cost of the inertial sensors-based navigation-based vehicle method [6].
This method is not feasible for those with limited budgets and who are looking
to gain prosperity. There is a wide range of methods required to implement in
order to navigate a vehicle in an indoor scenario, such as object tracking,
angle prediction, steering module, path planning, etc.

There are lots of methods that already exist for indoor vehicle navigation
depending on different kinds of hardware and software. These hardware and
software components can be cheap and expensive, but the results of low-price
components are below average. Ultrasonic or infrared sensors can be utilized
for the navigation of autonomous vehicles but more than one sensor is required
and a very specific environment to self-navigate. The environmental noise is a
problem for these sensors because these sensors can be easily affected by
noise resulting in the durability and performance of the sensors [7][8].

In [9][10], a method, which can track the moving object and map the
environment using a sequence of images is introduced. Their method used the
Kalman filter algorithm, which has to detect the feature points, but first
whole space is mapped as well and the system initially starts with known
features and then the upcoming objects are detected using certain features. On
the other hand [11], also presented a vehicle motion model using a sequence of
images and three degrees of freedom IMU with a digital camera. The low-quality
IMU can be affected easily by noise, which will tremendously affect vehicle
motion.

To address the above-mentioned problems we have presented A low-cost indoor
navigation method for autonomous vehicles called Affordable Indoor Navigation
Solution (AINS) which is based on based on Monocular Camera. In this paper, we
have proposed a robust method based on image processing in order to achieve
indoor vehicle navigation. The popular method of using color lines to navigate
a vehicle in the indoor scenario has been studied widely in the few past
years. The vehicle can be navigated using several line methods but in our
proposed system we will be using image processing methods such as color object
tracking [12]. This method aims to keep the self-driving vehicle on the path
and deliver the package after traveling a certain path. There are a few things
that are important to achieve the desired goal. The vehicle has to be very
stable and precise with its movements, proper rotation along the path, reduce
environmental effects, and reach its destination at a very specific timeline.

Accordingly, the further article is organized as follows. The following
section II briefly describes the literature review. Section III briefly
describes the proposed methodology of the indoor navigation system of
autonomous vehicles. Section IV results and discussion contains the results
and outcomes of the proposed methods. In section V article is concluded at the
end.

##  II Indoor Navigation System

Figure 1: Indoor Navigation System

In Figure 1 indoor navigation system working mechanism is elaborated as you
can see the camera mounted on the vehicle continuously scans the path to
identify a line by processing the input stream and searching for the specified
color line and once the line is identified the angle in the line is calculated
to make precise moves by vehicle. One path is determined as the vehicle
progresses forward.

##  III Literature Review

Currently, researchers are exploring various ways to improve the effectiveness
and efficiency of self-driving vehicles by proposing new techniques,
specifically for indoor scenarios. Self-driving vehicles in indoor navigation
systems face challenges such as the limited accuracy of GPS. These methods are
as follows.

In this paper [13], a pioneering approach to autonomous vehicle navigation in
indoor settings are introduced, utilizing omni-directional vision and circular
landmarks affixed to the ceiling. The use of upward-looking omni-directional
images minimizes occlusion and noise, enhancing location estimation. Analytic
formulas demonstrate that the circular landmark’s perspective shape in the
omni-directional image can be accurately approximated by an ellipse, enabling
precise vehicle localization for effective navigation, as validated through
simulations and real-world experiments.

This study [14], advocates for smartphones as a promising solution to
ubiquitous positioning issues, emphasizing the need for minimal impact on
device battery life. Introducing SmartSLAM, a novel indoor positioning system,
the research proposes an intelligent filtering approach that dynamically
adapts sensor fusion algorithms based on system certainty levels. This
innovation effectively reduces computational load, enhances positioning
accuracy, prolongs battery life, and allocates CPU resources more efficiently
for foreground processes.

The following research [15], tackles the challenging problem of indoor
positioning in GPS-denied environments, crucial for applications like
augmented reality and autonomous systems. Introducing a novel tandem
architecture of deep network-based systems, the approach leverages scene
images obtained through photogrammetry, employing an EfficientNet-based CNN
for scene classification and a MobileNet-based CNN for regression. The system
achieves remarkable precision in both Cartesian position and quaternion
information for camera localization, marking a significant advancement in
indoor positioning technology.

This paper [16] addresses the surging applications of unmanned aerial vehicles
(UAVs) across diverse domains, emphasizing the growing significance of indoor
use cases such as rescue operations and warehouse inventory tracking. The
study introduces a novel scheme employing deep neural networks, specifically a
DCNN-GA architecture, for autonomous UAV navigation in indoor corridors,
overcoming challenges of accurate localization and obstacle avoidance.
Notably, the proposed approach utilizes genetic algorithms for hyperparameter
tuning, demonstrating superior performance compared to state-of-the-art
ImageNet models in terms of minimal loss and heightened efficiency.

This study [17] introduces a real-time monocular vision-based range
measurement method tailored for Simultaneous Localization and Mapping (SLAM)
in GPS-denied environments, specifically designed for an Autonomous Micro
Aerial Vehicle (MAV) with limited payload capacity. The navigation strategy
relies on corner-based feature points extracted from a monocular camera,
presenting a solution for GPS-denied manmade environments. The experimental
validation involves a case study mission demonstrating vision-based
pathfinding through a conventional maze of corridors in a sizable building.

This study [18] proposed a unique monocular camera-based indoor navigation and
ranging strategy for autonomous self-driving vehicles. They are utilizing the
orthogonality of the indoor environment architecture by introducing a new
technique using a mono camera for Visual-SLAM to estimate the range and state
of the vehicle. The navigation approach presupposes an artificial indoor or
indoor-like environment with an unknown layout, denied the GPS information,
and characterized by features based on energy and straight architectural
lines. The proposed algorithms were tested on a fully autonomous microaerial
vehicle (MAV) equipped with advanced onboard image processing and SLAM
capabilities. Results from the experiments indicate that the system’s
performance is constrained solely by the camera’s capabilities and the
environmental complexity.

This study [19] presents an autonomous integrated indoor navigation system for
ground vehicles, leveraging a fusion of inertial sensors, LiDAR, wireless
local area network (WLAN) signal strength, odometry, and predefined floor
maps. Addressing the challenge of self-alignment and position initialization
in the absence of GNSS, the system employs an extended Kalman filter for tilt
angle estimation and a subimage matching algorithm for initial position and
heading estimation using 2-D LiDAR scans. The approach is validated in a real
office environment, demonstrating effective self-alignment, initialization,
and submeter-level positioning accuracy.

This Study [20] introduces a real-time path-planning algorithm for indoor UAV
navigation, specifically designed for contact inspection tasks in diverse
environments. Utilizing only the point cloud of the building as input, the
algorithm comprises a pre-processing step for segmentation and discretization,
enabling swift execution of the real-time path planning algorithm.
Demonstrated in various buildings, the method consistently achieves route
calculations in 8–9 milliseconds, meeting execution time requirements and
establishing its reliability for UAV route calculation in indoor settings.

##  IV Methodology

###  IV-A Transitional Behaviour of Framework

The proposed framework AINS flow-work is shown in Figure 2 which elaborates on
the steps taken by the navigation method. Firstly the method takes an input
video stream from the camera mounted on the vehicle. Secondly, the input video
stream is further prepossessed and fed to the path detection method. Once the
path color is detected the angle is calculated for possible turns in the path
minimizing the possible turn errors. lastly, the navigation module takes
place, and the final path is detected.

Figure 2: Transitional Behaviour of Framework

###  IV-B Purposed Framework (AINS)

We have proposed a vision-based navigation system called Affordable Indoor
Navigation Solution (AINS) for our prototype vehicles as shown in Figure 3. In
AINS fixed RGB camera is equipped with a vehicle. AINS consists of different
components including motors, microcontroller units, and motion controllers.
Our proposed AINS proved that with an RGB camera, we can automatically detect
a path and then follow that path correctly. In the proposed framework, we have
experimentally shown that our system detects path and navigate vehicle camera
using the mono camera. In the AINS framework, an RGB camera is mounted to the
front side of the vehicle facing the path. The resolution of the captured
image is 640×480 pixels. This experiment is based on real-time performance,
the vision application will detect the path and then the vehicle will run
along the path to finish the maneuver.

Figure 3: The Proposed Framework

###  IV-C The Methodical Stages of Our Approach

####  IV-C1 Image Feature Extraction

The proposed AINS framework has first and foremost step is the Image framework
extraction, where we have used the Gaussian Filtering technique. We have used
Gaussian filtering first to remove the noise from the image and then used the
image for further process, the following (1) is the Gaussian Filtering formula
for Image Feature Extraction.

$G(x,y)=\frac{12\pi^{2}}{e^{x^{2}+\frac{y^{2}}{22}}}$ (1)

In the Gaussian Filtering, x and x represent the point coordinates system and
they behave as Gaussian distribution. These x and y point coordinates control
the noise-removing process. After the de-noising step, we used the following
(2) to smooth the image.

$f^{\prime}(x,y)=G(x,y)*f(x,y)$ (2)

Where:

  * •

$G(x,y)$ represents the Gaussian filter,

  * •

$f(x,y)$ represents the original image, and

  * •

$f^{\prime}(x,y)$ represents the filtered image.

####  IV-C2 Spotting Anomalies on the Path

In our AINS proposed framework, we have performed an experiment based on HSV
color space and further used it to identify the path colors. The camera first
detects the image then it identifies the color in RGB space, after identifying
the path colors, the image colors are transferred from RGB form to HSV form.
Among other color models, the HSV color model is ideal for providing a more
human way to describe color, because hue, saturation, and value components are
closely related to how the eye perceives color. In the HSV space, hue
represents the hue, saturation represents the grayscale of the color space,
and value represents the brightness of the image [[21][22]. Once image
transformation is completed, our solution identifies all pixels from the HSV
color space and we present those HSV pixels in the Histogram displayed in
Figure 4. The graph shows that the distribution of the HSV pixels represents
the all viable count of pixels in the path color to be tracked.

Figure 4: Histogram of H,S,V

Furthermore, for displaying pixels as colors space HSV as probability
distribution we have used the probability density function (PDF), as PDF has
mean and standard deviation so we have shown the distribution of each pixel in
Figure 5. In this graph, the standard deviation is used to measure the upper
and lower bounds of the pixel values, and also ignore outliers, and pixels
that are not connected with paths.

Figure 5: PDF distribution of Pixels TABLE I: Ranges of HSV Values for Identifying Yellow Color Criteria | Hue Range | Saturation/Value Range
---|---|---
Color(Yellow) | $H\in(0.11,0.22)$ | $S,V\in(0.4,1.0)$

Calculating the HSV value manually is discouraged due to the potential need
for multiple attempts to accurately determine the specific color range.
Therefore, Table I establishes the range of path colors based on statistical
analysis of HSV color pixel data. Furthermore, for edge detection for paths we
have used the contour method, this method first analyzes the structure of
binary images and follows the border. This extends the border following
algorithm, which distinguishes between the outer borders and the internal hole
borders of a binary image [23].

For detecting path-centred points [24][25] we have implemented Image moment
and shape center calculation is done using spatial moments calculation of the
contour with the following equation (3).

$m_{ji}=x,y\cdot(x_{i}\cdot y_{i})$ (3)

The (x, y) array represents pixel values in the image, with 0 denoting black
and 1 denoting white. Here, x corresponds to the row position, and y
represents the column position within the image. Central moments are then
computed as follows (4):

$\mu_{ji}=x,y\cdot(x_{i}-x)\cdot(y_{i}-y)$ (4)

where $(x,y)$ represents the center of mass.

$\displaystyle x$ $\displaystyle=\frac{m_{10}}{m_{00}}$ (5) $\displaystyle y$
$\displaystyle=\frac{m_{01}}{m_{00}}$ (6)

Contour moments are expressed similarly, but their calculation is based on
Green’s formula.

$CL\,dx+M\,dy=\iint_{D}\left(\frac{\partial M}{\partial x}-\frac{\partial
L}{\partial y}\right)\,dxdy$ (7)

Consider a contour C, which bounds a region D. If functions L and M are
defined for points (x, y) within an open region encompassing D and possess
continuous partial derivatives, and if the direction of integration along C is
clockwise, then:

####  IV-C3 Navigation and Measurement of Angle

The angle between the reference vector and the central moments of the path is
determined by using trigonometric functions. This angle can be represented as
the function tan($\theta$), where $\theta$ signifies the angle.

$\theta=\arctan\left(\frac{y}{x}\right)$ (8)

To convert the angle from radians to degrees, you can multiply the given angle
by 180 and then divide it by $\pi$. This transformation is represented as:

Angle in degrees =( (Angle in radians × 180)/$\pi$ )

The transform range function is employed to further adjust the angle, as
depicted in Equation (8), where values of x within the range [a, b] are
transformed into values of y within the range [c, d].

$y=x-ad-cb-a+c$ (9)

The angle calculation relies on the characteristics of the vehicle’s wheel
motors. A straightforward approach is suggested for fine-tuning the angle and
steering the car using the equations (7) and (8). Below is the pseudocode for
this method.

Figure 6: Psudocode

This method Figure 6 involves providing the reference vector and the origin of
the path. Within the function, the initial angle on the unit circle is
determined, which is then modified to align with the minimum and maximum
rotation angles of the vehicle. Ultimately, by utilizing trigonometric
functions and the transform range function, the rotation angle is computed for
steering the vehicle.

##  V Experiment and Results

###  V-A Experimental Configuration

The proposed solution experiment was performed on the Ubuntu Operating System,
and the system configuration contains 16 GB RAM, We have used VSCode with C++
release mode, and the communication method was performed on UART and CAN.

###  V-B Results

In our proposed method AINS, the camera is fixed in front of the vehicle, so
the path can be clearly visible. The actual path is detected using HSV and
contour approach, but such input images have more than one contours, so first
we remove noise and contours with the selection of the biggest or widest
contours in the input image as described in Figure 7.

Figure 7: (a) Contour detection is performed after applying an adaptive
threshold, resulting in an image that includes an ambiguous object and a path.
(b) Subsequent filtering of the contour image isolates the edges of the path.

Once the contour is removed and the noise is reduced, we apply the angle
calculation method to calculate the angle between vector and centroid points
for the referenced path used by self-driving or autonomous cars. The prototype
of the vision-based vehicle has been tested across a range of indoor
environments. As a result, in particularly complex settings, some applications
may experience minor impacts. However, these issues should be manageable as
long as the HSV color range is appropriately calibrated, and the camera is
correctly positioned. The ultimate outcome is illustrated in Figure 8.

Figure 8: An image illustrating the navigation method is shown, featuring the
angle between the reference vector (indicated in blue text) and the centroid
of the path (highlighted by a red semi-circle). The centroid of the path is
depicted at the center of the crossed lines.

Based on the experiments conducted, an alternative method for vehicle
navigation, as opposed to utilizing a fixed angle, involves dividing the image
into three regions: left, right, and center. In theory, using image region
information seems like a plausible approach for navigation. However, this
method presents a significant challenge. Maintaining a constant angle for
vehicle navigation can be problematic since there’s a high likelihood that the
camera may not detect the path due to improper rotation. This underscores the
need for dynamic angle calculations instead of relying solely on image region
information.

A comparison between sensor-based and vision-based vehicles is provided in
Table II.

TABLE II: Sensor-Based vs. Vision-Based Methods |  Distance (m) |  Estimated Error (%) |  Time Consumption (min:sec)
---|---|---|---
Sensor-Based |  100 |  4.6 |  3:51
Vision-Based |  100 |  1.2 |  2:44

As evident in Table II, the performance of the vision-based vehicle
significantly outperforms that of the sensor-based vehicle. Both vehicles are
operated at a constant speed of 22 kilometers per hour, and error calculations
are made during vehicle maneuvers, considering factors such as rotation
stability and environmental effects.

The vision-based approach proves to be more dependable compared to the sensor-
based method. This is primarily because sensors require precise placement
along the path, whereas a camera can detect the path as long as it remains
within the camera’s field of view.

In summary, our method entails detecting the vehicle’s path through image
processing techniques and subsequently calculating the centroid of the path.
The angle between this centroid and the reference vector is then computed to
steer the vehicle effectively.

##  VI Discussion

In summary, the AINS is a robust and cost-effective indoor navigation method
for autonomous vehicles is proposed. The method. In the First step, the method
is fed by an input video stream by a vehicle camera. In the second step, the
path is detected after prepossessing the video. The method detects the path
and turns based on the color and angle of the path. In the final step, the
navigation is performed according to the path detected by the method.

##  VII Conclusion

In recent times, researchers have been actively exploring innovative
approaches to enhance the effectiveness and efficiency of autonomous vehicles,
particularly in indoor settings. Autonomous vehicle navigation indoors
presents numerous challenges, particularly due to the limited accuracy of GPS
in such environments. While various robust methods have been examined to
tackle this issue, many of these solutions suffer from high deployment costs.

In response to the challenges outlined above, we introduce an economical
indoor navigation solution for autonomous vehicles called the Affordable
Indoor Navigation Solution (AINS), which relies primarily on a monocular
camera. Our proposed approach is centered around a single camera, eliminating
the need for multiple, large, or power-intensive sensors like range finders
and other navigation sensors. AINS demonstrates that it’s possible to deploy
indoor navigation systems for autonomous vehicles without incurring exorbitant
costs.

Our method exhibits superior results compared to existing solutions, offering
reductions in estimated error and time consumption

##  VIII Future Work

We are planning to extend this work to robots enhancing the efficiency of
algorithms with deep learning techniques so we can achieve better and higher
proficiency in indoor scenarios, like deploying robots in libraries and other
similar environment settings.

## References

  * [1] Morales, Eduardo Sánchez, et al. ”High precision indoor navigation for autonomous vehicles.” 2019 International Conference on Indoor Positioning and Indoor Navigation (IPIN). IEEE, 2019.
  * [2] Hussein, A., Al-Kaff, A., de la Escalera, A., & Armingol, J. M. (2015, November). Autonomous indoor navigation of low-cost quadcopters. In 2015 IEEE international conference on service operations and logistics, and informatics (SOLI) (pp. 133-138). IEEE.
  * [3] Kayalvizhi, S., et al. ”A Comprehensive Study on Supermarket Indoor Navigation for Visually Impaired using Computer Vision Techniques.” 2022 OPJU International Technology Conference on Emerging Technologies for Sustainable Development (OTCON). IEEE, 2023.
  * [4] Zang, S., Ding, M., Smith, D., Tyler, P., Rakotoarivelo, T., & Kaafar, M. A. (2019). The impact of adverse weather conditions on autonomous vehicles: How rain, snow, fog, and hail affect the performance of a self-driving car. IEEE vehicular technology magazine, 14(2), 103-111.
  * [5] Bresson, Guillaume, et al. ”Simultaneous localization and mapping: A survey of current trends in autonomous driving.” IEEE Transactions on Intelligent Vehicles 2.3 (2017): 194-220.
  * [6] Cheng, J., Zhang, L., Chen, Q., Hu, X., & Cai, J. (2022). A review of visual SLAM methods for autonomous driving vehicles. Engineering Applications of Artificial Intelligence, 114, 104992.
  * [7] Run, Ray-Shine, and Zhi-Yu Xiao. ”Indoor autonomous vehicle navigation—a feasibility study based on infrared technology.” Applied System Innovation 1.1 (2018): 4.
  * [8] Run, R. S., & Xiao, Z. Y. (2018). Indoor autonomous vehicle navigation—a feasibility study based on infrared technology. Applied System Innovation, 1(1), 4.
  * [9] Chai, W., Chen, C., Edwan, E., Zhang, J., & Loffeld, O. (2012, March). INS/Wi-Fi based indoor navigation using adaptive Kalman filtering and vehicle constraints. In 2012 9th Workshop on Positioning, Navigation and Communication (pp. 36-41). IEEE.
  * [10] Chiang, K. W., Huang, C. H., Chang, H. W., Lin, C. X., Tsai, M. L., Zeng, J. C., & Hung, M. C. (2023). Semantic proximity update of GNSS/INS/VINS for Seamless Vehicular Navigation using Smartphone sensors. IEEE Internet of Things Journal.
  * [11] Cheng, J., Zhang, L., Chen, Q., Hu, X., & Cai, J. (2022). A review of visual SLAM methods for autonomous driving vehicles. Engineering Applications of Artificial Intelligence, 114, 104992.
  * [12] A. D. Luca, M. Ferri, G. Oriolo and R. P. Giordano, “Visual Servoing with Exploitation of Redundancy: An Experimental Study,” in IEEE International Conference on Robotics and Automation (2008), pp. 3231–3237.
  * [13] Wu, Chih-Jen, and Wen-Hsiang Tsai. ”Location estimation for indoor autonomous vehicle navigation by omni-directional vision using circular landmarks on ceilings.” Robotics and Autonomous Systems 57.5 (2009): 546-555.
  * [14] Faragher, R. M., & Harle, R. K. (2013, September). SmartSLAM-an efficient smartphone indoor positioning system exploiting machine learning and opportunistic sensing. In Proceedings of the 26th International Technical Meeting of The Satellite Division of the Institute of Navigation (ION GNSS+ 2013) (pp. 1006-1019).
  * [15] Ghofrani, A., Toroghi, R. M., & Tabatabaie, S. M. (2020, January). ICPS-net: an end-to-end RGB-based indoor camera positioning system using deep convolutional neural networks. In Twelfth International Conference on Machine Vision (ICMV 2019) (Vol. 11433, pp. 572-578). SPIE.
  * [16] Chhikara, Prateek, et al. ”DCNN-GA: A deep neural net architecture for navigation of UAV in indoor environment.” IEEE Internet of Things Journal 8.6 (2020): 4448-4460.
  * [17] Celik, K., Chung, S. J., & Somani, A. (2008, May). Mono-vision corner SLAM for indoor navigation. In 2008 IEEE International Conference on Electro/Information Technology (pp. 343-348). IEEE.
  * [18] Celik, Koray, and Arun K. Somani. ”Monocular vision SLAM for indoor aerial vehicles.” Journal of electrical and computer engineering 2013 (2013): 4-4.
  * [19] Atia, M. M., Liu, S., Nematallah, H., Karamat, T. B., & Noureldin, A. (2015). Integrated indoor navigation system for ground vehicles with automatic 3-D alignment and position initialization. IEEE Transactions on Vehicular Technology, 64(4), 1279-1292.
  * [20] Gonzalez de Santos, L. M., Frias Nores, E., Martinez Sanchez, J., & Gonzalez Jorge, H. (2021). Indoor path-planning algorithm for UAV-based contact inspection. Sensors, 21(2), 642.
  * [21] R. S. Run and Z. Y. Xiao, “Indoor Autonomous Vehicle Navigation—A Feasibility Study Based on Infrared Technology,” Appl. Syst. Innov. 1(1), 4–4 (2018).
  * [22] R. Nock and F. Nielsen, “Statistical Region Merging,” IEEE Trans. Pattern Anal. Mach. Intell. 26(11), 1452–1458 (2004).
  * [23] H. D. Cheng, X. H. Jiang, Y. Sun and J. Wang, “Color image segmentation: Advances and prospects,” Pattern Recognit. 34(12), 2259-2281 (2001).
  * [24] S. Suzuki and K. Abe, “Topological Structural Analysis of Digitized Binary Images by Border Following,” Comput. Vis. Graph. Image Process 30(1), 32-46 (1985).
  * [25] J. Flusser and T. Suk, “Rotation Moment Invariants for Recognition of Symmetric Objects,” IEEE Trans. Image Process. 15(12), 3784–3790 (2006).

IEEE conference templates contain guidance text for composing and formatting
conference papers. Please ensure that all template text is removed from your
conference paper prior to submission to the conference. Failure to remove the
template text from your paper may result in your paper not being published.


# Human Evaluation of Text-to-Image Models on a Multi-Task Benchmark

Vitali Petsiuk
Boston University
<EMAIL_ADDRESS>
Alexander Siemenn
MIT
<EMAIL_ADDRESS>
Saisamrit Surbehera
Columbia University
<EMAIL_ADDRESS>
Zad Chin
Harvard University
<EMAIL_ADDRESS>
Keith Tyser
Boston University
<EMAIL_ADDRESS>
Gregory Hunter
Columbia University
<EMAIL_ADDRESS>
Arvind Raghavan
Columbia University
<EMAIL_ADDRESS>
Yann Hicke
Cornell University
<EMAIL_ADDRESS>
Bryan Plummer
Boston University
<EMAIL_ADDRESS>
Ori Kerret
Ven Commerce
<EMAIL_ADDRESS>
Tonio Buonassisi
MIT
<EMAIL_ADDRESS>
Kate Saenko
BU, MIT-IBM Watson AI Lab
<EMAIL_ADDRESS>
Armando Solar-Lezama
MIT
<EMAIL_ADDRESS>
Iddo Drori
MIT, Columbia University, Boston University
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

###### Abstract

We provide a new multi-task benchmark for evaluating text-to-image models. We
perform a human evaluation comparing the most common open-source (Stable
Diffusion) and commercial (DALL-E 2) models. Twenty computer science AI
graduate students evaluated the two models, on three tasks, at three
difficulty levels, across ten prompts each, providing 3,600 ratings. Text-to-
image generation has seen rapid progress to the point that many recent models
have demonstrated their ability to create realistic high-resolution images for
various prompts. However, current text-to-image methods and the broader body
of research in vision-language understanding still struggle with intricate
text prompts that contain many objects with multiple attributes and
relationships. We introduce a new text-to-image benchmark that contains a
suite of thirty-two tasks over multiple applications that capture a model’s
ability to handle different features of a text prompt. For example, asking a
model to generate a varying number of the same object to measure its ability
to count or providing a text prompt with several objects that each have a
different attribute to identify its ability to match objects and attributes
correctly. Rather than subjectively evaluating text-to-image results on a set
of prompts, our new multi-task benchmark consists of challenge tasks at three
difficulty levels (easy, medium, and hard) and human ratings for each
generated image.

##  1 Introduction

Spurred by large-scale pretraining on billions of image-text pairs, vision-
language models have seen rapid progress in recent years. Large-scale models
like CLIP [8] and Flamingo [1] have reported remarkable performance on dozens
of benchmarks using a single model, even when using few or no task-specific
training samples. Generating high-resolution images given a text prompt has
improved in quality to such an extent with models like Stable Diffusion [10],
Imagen [12], and DALL-E 2 [9] that their influence has affected popular
culture as illustrated in their use to generate magazine
covers111https://www.cosmopolitan.com/lifestyle/a40314356. There has been much
recent progress in improving text-to-image models, allowing the synthesis of
objects within novel contexts [11] such as different backgrounds,
illumination, and poses. However, these methods still need help generating
images in complex scenes or where compositionality is essential. A critical
bottleneck in further progress is the need for rigorous evaluation protocols,
as current evaluation methods focus on prompts that do not fully account for
the diverse settings these models must support [12].

We propose a new text-to-image generation benchmark covering thirty-two tasks
over multiple applications, each targeting a different capability of text-to-
image generation models as shown in Table 2. For example, we may ask a model
to produce varying numbers of an object to identify its ability to count or to
generate an image with an object of a specified shape. We divide each task
into three difficulty levels: easy, medium, and hard. For example, suppose the
task is to synthesize different numbers of objects. In that case, the task may
be easy: generating 1-3 objects, medium - generating 4-10 objects, and hard -
generating more than ten objects. Next, we provide ten different instances for
each task difficulty level. These instances are specific prompts that
implement the tasks. We score text-to-image models on each of the thirty
instances (ten for each of the three difficulty levels) for each of the fifty
tasks and applications. Specifically, we run our benchmark on DALL-E 2 [9],
and Stable Diffusion [10].

We can quantify and compare any new text-to-image generation model with our
new benchmark. In this work, we perform a human evaluation of three tasks.
Note, however, that many of the tasks may also be evaluated automatically,
e.g. by a neural network. For instance, incorporating spatial-aware methods
ensures that prompts with spatial relationships and compositional elements are
correctly generated. Using OCR mechanisms ensures that quoted text is legible
and accurate.

Our key contributions are: (i) developing challenge tasks for state-of-the-art
text-to-image generative models, (ii) defining human evaluation procedures ,
and (iii) Performing a human evaluation for a subset of tasks with 3,600 human
ratings, comparing the performance of two of the most common open source
(Stable Diffusion) and commercial (DALL-E 2) models.

##  2 Related Work

Text-to-image models may be roughly split into two types: autoregressive
transformer-based models [14] and diffusion-based models [12]. Prior state-of-
the-art [2, 4, 5, 11] handles specific limitations of text-to-image models,
such as generating an image within the context or modifying object attributes
automatically. A comprehensive and quantitative multi-task benchmark for text-
to-image synthesis does not exist that covers a diverse set of tasks with
varying difficulty levels. Our goal is to develop a benchmark that will become
the gold standard in the field for evaluating text-to-image models that will
endure time.

Text-to-image models are commonly evaluated by the Inception Score (IS) and
the Fréchet Inception Distance (FID). Both of these metrics are based on
Inception v3 classifier. These measures, therefore, are designed for the
unconditional setting and are primarily trained on single-object images. We
have seen several approaches which rectify these shortcomings. Imagen [12]
introduced DrawBench, a benchmark with 11 categories with approximately 200
prompts total. Human raters (25 participants) were asked to choose a better
set of generated images from two models regarding image fidelity and image-
text alignment. Categories are: colors counting, conflicting, DALL-E 2,
description, misspellings, positional, rare words, Reddit, text.

DALL-Eval [3] proposed PaintSkills to test skills of the generative models —
specified object generation, counting, color, and spatial relations. It
utilizes the Unity engine to test these tasks using predefined sets of
objects, a subset of MS-COCO [6] objects, colors, and spatial relations. We
propose a more comprehensive benchmark of tasks at a finer level of detail,
with three difficulty levels. Localized Narratives [7] is a multi-modal image
captioning approach that can be adapted to benchmarking images. Text captions
are first generated by human annotators whose cursor movement and voice
commentary hover their cursor over the image to provide richness and accuracy.
PartiPrompts, a holistic benchmark of 1,600 English prompts [13], compared to
Localized Narratives, is better in probing model capabilities on open-domain
text-to-image generation. The 1,600 prompts span 12 different categories and
11 challenge aspects. The 12 categories are artifacts, animals, indoor scenes,
produce and plants, abstract, arts, food & beverage, vehicles, illustrations,
outdoor scenes, people, and world knowledge, while the 11 challenge aspect are
basic, fine-grained detail, properties & positioning, linguistic structures,
perspective, quantity, writing & symbols, complex, imagination, style & format
and simple detail. The image quality and the alignment of the generated image
with the input text are evaluated.

##  3 Methods

We create a comprehensive multi-task text-to-image generation benchmark of
thirty-two diverse tasks, as shown in Table 2. The motivation behind selecting
benchmark tasks is to cover a wide range of downstream applications that
benefit from high-quality text-to-image models, including: (1) graphic design
(generating new designs for websites), (2) e-Commerce (generating personalized
ads), (3) architectural planning and design (generating new renderings of
buildings, interior design, and creative home design), (4) real-estate
listings (generating furnished versions of unfurnished apartment and house
photos for advertisement), (5) education (generating personalized digital
learning interfaces with customized enhancements), (6) user interface and user
experience (generating design templates for mobile and desktop applications),
(7) stop-motion video (generating frames for short animations), (8) cosmetics
(generating realistic images showcasing products), (9) stock photos
(generating large amounts of stock images for general audiences), (10) product
design (quickly prototyping digital and physical products), (11) illustrations
(generating professional artwork for custom purposes), (12) synthetic data
(generating synthetic data for boosting training samples size), (13) social
media (generating memes and shareable content), (14) image recommendation
(generating recommendations based on user preferences), (15) gaming (using
natural language to create complex scenes for video games), (16) proteomics
(designing new proteins visualizing existing structures), (17) material
science (designing new crystals).

We use the benchmark to compare different models, comparing Stable Diffusion
[10] 222Publicly available https://beta.dreamstudio.ai/ and DALL-E 2 [9], and
identify their limitations. Our evaluation protocol consists of human ratings
between 1 (worst) and 5 (best) of tasks at three difficulty levels. The three
difficulty levels are currently assigned heuristically based on the experience
with failure modes of generative models. Examples of cases in which human
evaluation is required are: (i) concepts that are difficult to define, such as
successfully combining objects that are rarely co-occurring in the real world;
(ii) complex tasks, such as images that require common sense; and (iii) cases
where human expertise is essential such as generating images without racial or
gender bias. We obtained 3,600 human ratings: twenty graduate students, two
models, three difficulty levels, and ten prompts each. The images were all
generated with identical default model parameters and evaluated on the same
scale, scoring between 1 (worst) and 5 (best).

Difficulty | Easy | Medium | Hard | Average
---|---|---|---|---
Task / Model | SD | DALL-E 2 | SD | DALL-E 2 | SD | DALL-E 2 | SD | DALL-E 2
Counting | 74.8 | 91.8 | 52.2 | 51.4 | 36.1 | 54.0 | 54.4 | 65.7
Faces | 72.5 | 93.5 | 74.0 | 74.3 | 64.2 | 77.2 | 70.2 | 81.7
Shapes | 70.8 | 67.1 | 56.8 | 46.0 | 45.1 | 57.3 | 57.6 | 56.8
Table 1: Normalized ratings (%) for human evaluation of Stable Diffusion and DALL-E 2 across the three tasks of counting, shapes, and faces. DALL-E 2 outperforms Stable Diffusion on Counting and Faces tasks, and Stable Diffusion shows a minor advantage on the shapes task. On 6 out of 9 sub-tasks, DALL-E 2 produces better images.  Id | Task
---|---
1 | Generating a specified number of objects
2 | Generating objects with specified spatial positioning
3 | Combining objects very rarely co-occurring in the world
4 | Generating images obeying physical rendering rules of shadows, reflections, and acoustics
5 | Generating objects with specified colors
6 | Generating conflicting interactions between objects
7 | Understanding complex and long text prompts describing objects
8 | Understanding misspelled prompts
9 | Handling absurd requests
10 | Understanding rare words
11 | Incorporating quoted text with correct spelling
12 | Understanding negation and counter-examples
13 | Understanding anaphora and phrases that refer to other parts of the prompt
14 | Aligning text as specified in the prompt
15 | Generating common-sense images
16 | Removing objects without needing manual annotation
17 | Removing content that is not child-safe
18 | Editing the color of objects without marking them manually
19 | Replacing objects without marking them manually
20 | Generating objects with abstract shapes
21 | Objects obeying physics rules
22 | Generating images without racial or gender bias
23 | Understanding comparative concepts like fewer and more
24 | Photo-realistic faces
25 | Understanding prompts regarding weather
26 | Handling multi-lingual prompts
27 | Duplicating objects perfectly
28 | Generating multiple camera viewpoints of the same scene
29 | Generating realistic faces with a specific emotion
30 | Generating well-known faces
31 | Generating a thumbnail summary for text and video
32 | Changing dimensions of an image without losing information
Table 2: Multi-task text-to-image benchmark and applications: we propose a
series of tasks on which text-to-image models could be evaluated. The tasks
are selected to cover a multitude of downstream applications.

##  4 Results

We survey twenty students that evaluate the performance of Stable Diffusion
and DALL-E 2. Each student evaluated the three tasks at three levels of
difficulty and ten prompts each, providing a rating between 1 (worst) and 5
(best). We collected a total of 3,600 scores. The results are summarized in
Table 1, and a detailed breakdown of the evaluations by prompts are available
in Table 3 of the Appendix. Our human evaluation shows that DALL-E 2 (65.7%)
performs better on the counting task than Stable Diffusion (54.4%). On the
Faces tasks, both models perform very well, and DALL-E 2 (81.7%) performs
better than Stable Diffusion (70.2%), and on the Shapes task, both models
perform equally well (56.8% compared to 57.6%). Performance gracefully
degrades as the tasks are more difficult, except for DALL-E 2, which performs
slightly better on the hard than on the medium Shapes task.

##  5 Conclusion

We propose a new quantitative way of evaluating text-to-image generative
models using a benchmark covering many model competencies and applications.
Initial human evaluation on a subset of benchmark tasks shows a slight
advantage of DALL-E 2 over Stable Diffusion. Our proposed benchmark allows for
testing individual competencies and limitations of the different generative
models. Understanding the limitations is critical for picking the suitable
model for each task and application, advancing the quality of generative
models, and aligning their performance with human goals.

## References

  * Alayrac et al. [2022] Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, et al.  Flamingo: a visual language model for few-shot learning.  _arXiv preprint arXiv:2204.14198_ , 2022.
  * Bar-Tal et al. [2022] Omer Bar-Tal, Dolev Ofri-Amar, Rafail Fridman, Yoni Kasten, and Tali Dekel.  Text2live: Text-driven layered image and video editing.  _arXiv preprint arXiv:2204.02491_ , 2022.
  * Cho et al. [2022] Jaemin Cho, Abhay Zala, and Mohit Bansal.  Dall-eval: Probing the reasoning skills and social biases of text-to-image generative transformers.  _arXiv preprint arXiv:2202.04053_ , 2022.
  * Gal et al. [2022] Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or.  An image is worth one word: Personalizing text-to-image generation using textual inversion.  _arXiv preprint arXiv:2208.01618_ , 2022.
  * Hertz et al. [2022] Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or.  Prompt-to-prompt image editing with cross attention control.  _arXiv preprint arXiv:2208.01626_ , 2022.
  * Lin et al. [2014] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.  Microsoft coco: Common objects in context.  In _European conference on computer vision_ , pages 740–755. Springer, 2014.
  * Pont-Tuset et al. [2020] Jordi Pont-Tuset, Jasper Uijlings, Soravit Changpinyo, Radu Soricut, and Vittorio Ferrari.  Connecting vision and language with localized narratives.  In _European conference on computer vision_ , pages 647–664. Springer, 2020.
  * Radford et al. [2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In _International Conference on Machine Learning_ , pages 8748–8763. PMLR, 2021.
  * Ramesh et al. [2022] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen.  Hierarchical text-conditional image generation with clip latents.  _arXiv preprint arXiv:2204.06125_ , 2022.
  * Rombach et al. [2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.  High-resolution image synthesis with latent diffusion models.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10684–10695, 2022.
  * Ruiz et al. [2022] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman.  DreamBooth: Fine tuning text-to-image diffusion models for subject-driven generation.  _arXiv preprint arXiv:2208.12242_ , 2022.
  * Saharia et al. [2022] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes, et al.  Photorealistic text-to-image diffusion models with deep language understanding.  _arXiv preprint arXiv:2205.11487_ , 2022.
  * Yu et al. [2022a] Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al.  Scaling autoregressive models for content-rich text-to-image generation.  _arXiv preprint arXiv:2206.10789_ , 2022a.
  * Yu et al. [2022b] Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al.  Scaling autoregressive models for content-rich text-to-image generation.  _arXiv preprint arXiv:2206.10789_ , 2022b.

##  Appendix A Appendix

Difficulty | Easy | Medium | Hard
---|---|---|---
Prompt / Model | SD | DALL-E 2 | SD | DALL-E 2 | SD | DALL-E 2
1 | 96 | 100 | 69 | 32 | 30 | 57
2 | 93 | 97 | 62 | 62 | 28 | 52
3 | 49 | 97 | 34 | 36 | 22 | 61
4 | 87 | 99 | 38 | 46 | 37 | 44
5 | 51 | 90 | 77 | 64 | 46 | 57
6 | 97 | 50 | 54 | 45 | 35 | 82
7 | 92 | 93 | 51 | 79 | 47 | 48
8 | 67 | 97 | 38 | 39 | 37 | 45
9 | 43 | 98 | 43 | 50 | 30 | 55
10 | 73 | 97 | 56 | 61 | 49 | 39
1 | 69 | 96 | 79 | 86 | 66 | 73
2 | 82 | 93 | 83 | 95 | 70 | 69
3 | 76 | 96 | 76 | 75 | 68 | 80
4 | 97 | 98 | 70 | 87 | 91 | 91
5 | 23 | 92 | 81 | 86 | 58 | 90
6 | 75 | 96 | 71 | 77 | 74 | 77
7 | 60 | 92 | 91 | 20 | 53 | 95
8 | 58 | 84 | 76 | 66 | 41 | 81
9 | 96 | 94 | 74 | 77 | 59 | 51
10 | 89 | 94 | 39 | 74 | 62 | 65
1 | 93 | 92 | 73 | 30 | 36 | 41
2 | 94 | 84 | 49 | 36 | 67 | 64
3 | 89 | 77 | 32 | 32 | 44 | 62
4 | 85 | 49 | 48 | 38 | 44 | 50
5 | 49 | 38 | 69 | 42 | 30 | 66
6 | 89 | 45 | 45 | 83 | 47 | 71
7 | 52 | 80 | 65 | 52 | 34 | 70
8 | 37 | 32 | 32 | 42 | 49 | 47
9 | 79 | 90 | 84 | 37 | 51 | 37
10 | 41 | 84 | 71 | 68 | 49 | 65
Average: | 72.70 | 84.13 | 61.00 | 57.23 | 48.47 | 62.83
Table 3: Normalized ratings (%) for human evaluation of Stable Diffusion and
DALL-E 2 across the three tasks of counting, shapes, and faces for each of the
ten prompts. Figure 1: Evaluation of image generation on a counting task at
various difficulties. Each panel contains tasks with different difficulties.
The columns correspond to the text prompt, and the rows correspond to the
model used to generate the image: (i) Stable Diffusion and (ii) DALL-E 2. The
prompts used to generate the images are classified into three different
difficulties: (i) easy difficulty tasks consisting of generating 1–3 objects,
e.g., two cars, three people; (ii) medium difficulty tasks consisting of
generating 4–10 objects, including uncommon combinations of quantities and
objects, e.g., six bowling pins, seven fire hydrants; (iii) hard difficulty
tasks consisting of 10 or more objects with other numerical concepts in the
prompt, e.g., twelve boats arranged in three rows. Figure 2: Evaluation of
image generation on a shape task at various difficulties. Each panel contains
tasks at different difficulties where the columns correspond to the text
prompt, and the rows correspond to the model used to generate the image: (i)
Stable Diffusion and (ii) DALL-E 2 The prompts used to generate the images are
classified into three different difficulties: (i) easy difficulty tasks
consisting of generating simple shapes, e.g., circle, star; (ii) medium
difficulty tasks consisting of generating entities in the form of a shape,
e.g., hexagonal maze, octagonal TV; (iii) hard difficulty tasks consisting of
multiple entities, each of a specified shape, e.g., square-shaped water bottle
next to a semicircular orange. Figure 3: The prompts used to generate the
images are classified into three different difficulties: generating photo-
realistic faces given (i) easy: 1 to 2 features for an individual; (ii)
medium: different 1 to 3 features for each individual in a group of 1 to 2
people with easy angle and posture; (iii) hard: given more than two features
for each individual in a group of more than two people with challenging angle,
posture, lighting or occlusion. Asterisk $(*)$ indicates a failure to generate
an image because of the model’s content filters.