Reynolds number effects on transition, turbulence intensity and axial-velocity decay rate of turbulent round jets

Ramanathan Varadharajan

Laboratory of Physical Chemistry & Soft Matter,
Wageningen University & Soft Matter,
6708WE Wageningen, The Netherlands.
E-mail<EMAIL_ADDRESS>
Numerical simulations of turbulent round jets, using explicit-filtered LES technique, are performed, for three different Reynolds numbers(Re = 3600, 88000, 400000), to understand the Reynolds number effect on subsonic jets with Mach number 0.9. Eigth-order compact schemes are used for spatial derivative estimation. Second order Runge-Kutta method is used for Time evolution of flow. One-parameter fourth-order compact filter is used for low-pass explicit filtering of transported variables. Decreased centreline axial velocity decay rate and reduced turbulence intensities are observed as jet Reynolds number increases. Jet spread-rate is observed to be independent of the Reynolds number. An early potential-core collapse and transition to turbulent regime is observed in jets with high Reynolds number, for same inflow turbulence seeding. Role of increased smaller length scales with increase in Reynolds number is analyzed. It is observed that behavior of turbulent round jets is similar to that of plane jets, irrespective of gain in dimensional freedom. Detailed discussions of the observations made from numerical experiments are presented.

Round jet, Reynolds number effects, Subsonic flow, Large eddy simulation, Turbulence intensity, Transition .

§ INTRODUCTION

Turbulent round jets have been extensively studied, both experimentally and numerically, for their rich physics and applications.
Applications include but not limited to, turbulent mixing for efficient combustion, industrial whistles, turbulent noise reduction in aircraft jets, etc.,
Thus the evolution of Mean flow and turbulence characteristics of a round jet are extensively researched and documented in literature.
Notable works involve experiments performed by Panchapakesan,[15] & Hussein et al,[13] to understand mean-flow and turbulence in jets.
Jonathan B. Freund, performed Direct numerical simulations of subsonic round jet at 0.9 Mach Number [7] to determine the noise sources in a turbulent jet.

Fellouh et al.[6],  proved the Reynolds dependence in mixing transition in near and intermediate-field regions of round jet using hot-wire measurements.
Effect of Reynolds number and initial conditions on turbulent mixing was studied by Boersma[3],Hussain[12], George[9], Dimotakis[5], Xu et al.[24] and Uddin et al.[20]. A good review of works can be found in Ball et al.[1].
Presence of Reynolds number effects on jet parameters for plane jets was initially reported by Xu et al.[25].
Xu's experiments on plane jets shows decrease in centerline decay rates, spread rate with increase in Reynolds number till a critical point is achieved.

In this paper, an extension of the work for round jets is presented, as most of the reported studies on Reynolds number effects were performed on non-circular jets, because of their high mixing efficiency.
It should be noted that the mixing efficiency of circular jets are not insignificant.
It will be shown that for round jets, the dependence on Reynolds number is similar to that of plane jets.
Article also documents the first attempt to study Reynolds number effects on turbulence using explicit-filtered LES technique.
Numerical Methods are explained in Section: <ref>. An account on the jet inflow conditions used are provided in Section: <ref>. Computational domain used for various simulations are described in Section: <ref>. Section: <ref> presents the results and discussions.

§ NUMERICAL METHOD

The flows considered is governed by the Navier-Stokes equations for compressible flow

\begin{equation}
\centering
\frac{\partial \rho}{\partial t} + \frac{\partial \rho u_{i}}{\partial x_{i}} = 0 ,
\end{equation}

\begin{equation}
\centering
\frac{\partial \rho u_{i}}{\partial t} + \frac{\partial \rho u_{i} u_{j}}{\partial x_{j}} = - \frac{\partial p}{\partial x_{i}} + \frac{\partial \tau_{ij}}{\partial x_{j}} ,
\end{equation}

\begin{equation}
\centering
\frac{\partial \rho E}{\partial t} + \frac{\partial}{\partial x_{j}}[(\rho E + p)u_{i}] =  \frac{\partial q_{i}}{\partial x_{i}} + \frac{\partial u_{j} \tau_{ij}}{\partial x_{i}}.
\end{equation}

Here $\rho$ is the density, $u_{i}$ are Cartesian velocity components, $p$ is pressure, $E$ is the energy and $ \rho E = (\rho u_{k} u_{k})/2 + p/(\gamma -1)$. $q_{i}$ is the heat flux.  The Prandtl Number was set to a constant value of 0.71. Shear stress

\begin{equation}
\centering
\tau_{ij} = \mu \Bigg{(}{\frac{\partial u_{i}}{\partial x_{j}} + \frac{\partial u_{j}}{\partial x_{i}} -\frac{2}{3}\bigg{[}\frac{\partial u_{k}}{\partial x_{k}}\bigg{]}\delta_{ij}}\Bigg{)},
\end{equation}

where $\mu$, the dynamic viscosity, was calculated from Sutherland's law

\begin{equation}
\centering
\mu = \frac{1.458 \times T^{1.5}}{(T + 110.0)}.
\end{equation}

Since the flows considered are turbulent, the numerical solution was found as a large eddy simulation using the explicit filtering method. Detailed information of the numerical procedure are provided in R. Varadharajan[21] and S. Ganesh[8].  Essential requirements for this method are that a high resolution numerical method be used along with a high resolution low pass spatial filter applied to transported variables after every time step.  This approach to LES has been used successfully, for several types of flows, by at least two other groups [23] [4].  Here, a Cartesian grid was used with a 8th-order compact difference formula for spatial derivatives, split into a forward and a backward step extending the method of Hixon and Turkel[10]  [11].  Time-stepping was by a 2nd-order Runge-Kutta(RK2) scheme.  A one-parameter fourth-order compact filter [14] was applied with filter paramter $\alpha = 0.475$ in the stream-wise direction and 0.498 in the cross-stream directions.

Partially non-reflecting, characteristic boundary conditions developed by K. W. Thompson[18] [19] and adapted for compressible fluid flow, by S.K. Lele [16], were specified at boundaries.   A constant time-step estimated from a maximum CFL number of 0.075 was used.

The numerical method described above, including boundary treatments, combining non-reflecting conditions, stretched grid buffer zone, and anchoring inflow variables to target values was developed and tested for jet aeroacoustics studies, see S. Ganesh[8]. And later successfully tested to understand jet instability modes on Hartmann whistle, see R.Varadharajan et al.[22]

Schematic of computational domain on the plane $z = 0$, in three-dimensional computational space. Sponge-layer is implemented to avoid spurious reflections from boundaries as shown in figure.

M$=0.9$, Re$=3600$

M$=0.9$, Re$=3600$

M$=0.9$, Re$=88000$

M$=0.9$, Re$=88000$

M$=0.9$, Re$=400000$

M$=0.9$, Re$=400000$

[Decay of centerline axial velocity & streamwise variation of jet half-width radius.]Decay of centerline axial velocity & streamwise variation of jet half-width radius.

[Second moments of velocity]Second moments of velocity:  [black,fill=black] (0,0) circle (.5ex); - Hussein et al.
(1994); [black,fill=white] (0,0) circle (.5ex); - Panchapakesan & Lumley (1993); (<ref>) - Re = 88000, Grid 1;
(<ref>) - Re = 88000, Grid 2; (<ref>) - Re = 400000, Grid 1;
(<ref>) - Re = 400000, Grid 2; (<ref>) - Re = 400000, Grid 3.

[TKE spectrum of J2 jets]Re = 88000

[TKE spectrum of J2 jets]Re = 400000

[Turbulent kinetic energy spectrum]Turbulent kinetic energy spectrum: Re = 88000: [(<ref>)- Grid 1; (<ref>) - Grid 2.]
Re = 400000: [(<ref>)- Grid 1; (<ref>) - Grid 2; (<ref>) - Grid 3.]; (<ref>) - $5/3$ slope line

[Instantaneous contours of u-velocity]Instantaneous contours of u-velocity at $xy$ plane,  $z=0$. $Re = 4 \times 10^{5}$, showing potential-core collapse at $x = 14.5r_0$.

Details of round jet simulations

2*Simulation     2cjet Details     2cParameters

Reynolds Number     Grid (No.of.Div. in jet)      $B_u$     $x_{jc}/r_0$

J1G2a     3600     $386 \times 201 \times 201$ (10)     5.96      42.4

J2G1     88000     $226 \times 171 \times 171$ (08)     6.37      20.2

J2G2     88000     $386 \times 201 \times 201$ (10)     6.28     19.8

J3G1     400000     $226 \times 171 \times 171$ (08)     7.40     14.8

J3G2     400000     $386 \times 201 \times 201$ (10)     6.60     14.5

J3G3     400000     $291 \times 241 \times 241$ (12)     6.55     14.2

Lumley, 1993     11000     N.A.(Experiment)     6.06     –

Freund, 2001     3600     $640 \times 250 \times 160$(55)     5.80     14.0

§ JET INFLOW CONDITIONS

Round jet with thin shear layer is provided as inflow. Velocity profile of jet is defined by top-hat profile
given below,

\begin{equation}
U = \frac{U_j}{2} + \frac{U_j}{2} \ \tanh\bigg{(}\frac{r_0 - r}{2 \theta} \bigg{)}
\end{equation}

The above specified nozzle inflow condition represents jet with a thin laminar annular shear layer with radius $r_0$ and jet velocity $U_j$.
Momentum thickness, $\theta = r_0/20$, is used for simulations.
Mean inlet pressure is constant and maintained same as the ambient chamber pressure, $p_0 = 1823.85 \ Pa$. Inlet
density is determined using the Crocco-Busemann relationship. Ambient density, $\rho_0 = 0.022 \ kg/m^{3}$
and inflow density, $\rho_{in} = 0.02564 \ kg/m^{3}$ are used.
Mach Number of simulated jets are set to constant value of 0.9.
Random fluctuations are added to the stream-wise components at inflow to induce turbulence. The fluctuations are calculated as follows,

\begin{equation}
u' (\phi, t) = A \Bigg{[} \sum_{i=1}^{6} \sin (i\phi+\alpha_i+\psi_it) + \sum_{i=7}^{8} \sin(\psi_it)\Bigg{]}_{(x = 0)}
\end{equation}

\begin{equation}
A = \frac{U_j}{1000} \ e^{\big{(}-128(\frac{r - R}{R})^{2}\big{)}}
\end{equation}

where $\phi$ is the azimuthal angle. $\alpha_i$ and $\psi_i$ are random numbers to generate random phase and frequencies.
R = 0.875$r_0$ is used, providing with a gaussian function centered around R near nozzle lip. The perturbation
$u'$ is added at each step of RK2. The peak rms of the stream-wise velocity fluctuations are under 2% for
all cases.

§ COMPUTATIONAL DOMAIN

Computation domain used for simulations, are cartesian grids with x-axis along streamwise direction, y,z axis along cross-stream direction and origin
located at jet centerline, is schematically represented in Figure <ref>. Three different grids with similar configuration are used as
computational domain for all numerical simulations.
6 Simulations of Mach 0.9, Re = 3600, 88000, $4 \times 10^5$ round jets, in three different grids, are performed to study the effect of Reynolds number.

Numerical grid with 16 uniform divisions in jet diameter, both in y and z direction, is called as Grid 1 (G1) throughout the article.
Grid 1 is used to simulate two cases of jet with Re $= 88000, 4 \times 10^5;$ Ḟrom $ y = \pm 4r_0$ and $z = \pm 4r_0$, $2$% stretching in local $\Delta$y is used.
Buffer region with $5$ % stretching is used above $x = 75r_0$.

Numerical grid with 20 uniform divisions in jet diameter is called as Grid 2 (G2) throughout the article.
Grid 2 is used to simulate three cases of jet with Re $= 3600$ (Grid 2a is used), $88000, 4 \times 10^5;$ Ḟrom $ y = \pm4r_0$ and $z = \pm4r_0$, $2$ % stretching in local $\Delta$y is used. Buffer region with $5$ % stretching is used above $x = 75r_0$.
Grid 2a is a variant of Grid 2, where the compression in the stream-wise direction is started at $x = 40r_0$ (which was the expected breakdown length for $Re = 3600$ case).

Numerical grid with 24 uniform divisions in jet diameter is called as Grid 3 (G3) throughout the article.
Grid 3 is used to simulate single case of jet with Re $= 4 \times 10^5;$ Ḟrom $ y = \pm 4r_0$ and $z = \pm 4r_0$, $2$ % stretching in local $\Delta$y is used.
Buffer region with $5$ % stretching is used above $x = 75r_0$.

All three grids are compressed from $x =6r_0$ and stretched after $x = 12r_0$ approximately.
Grid is compressed to capture finer length scales in the near break down region.
The region of compression is chosen based on expected break down point from previous numerical experiments.
Compression and Stretching parameters $r_c = 0.985$ and $r_s = 1.015$ represent the common ratio of compression and stretching respectively for all three grids.
Grids described above are all developed as a function of jet radius $r_0$.

§ SIMULATION RESULTS

§.§ Mean flow

In the numerical computations, there is an ambiguity in the definition of effective nozzle location as the computational domain doesn't include the nozzle.
Therefore, virtual origin values of these simulations are matched to compare the simulation results with experimental and other computational works.
Virtual origin values are obtained from the jet half-width radius. A linear fit for half-width radius is found in the self-similar region.
This linear fit is extrapolated to find the virtual origin of the jets.

Self-Similarity [17][2] states, downstream of jet breakdown and potential core collapse, the centerline velocity decays according to,

\begin{align}
\frac{U_c}{U_j} = 2B_u \frac{r_0}{x-x_0}
\end{align}

where $B_u$ is a measure of center-line axial-velocity decay rate. For the cases of simulations performed center-line axial velocity decay and
Half-width radius are presented in Figure [<ref>].
From Table <ref>, it can be observed that jets with higher Reynolds number has higer values of $B_u$. This shows that increase in Reynolds number causes decreased axial-velocity decay rate for turbulent round jets. It should also be noted that there isn't any significant variation of spread rate for different Reynolds numbers(Can be observed from jet halfwidth plots($r_{0.5}/r_0$ $vs.$ $x/r_0$) in Figure <ref>). In Figure <ref>(e,f) the slight deviation can be attributed to use of Cylindrical geometry by Bogey & Bailey[4].

§.§ Turbulence

Downstream of the jet breakdown, jets become self similar. In the self similar region, the
second moments of velocity are used to characterize the turbulence. Figure [<ref>] shows that post-transition turbulence intensities decreases as Reynolds number increases.
It can be observed that peak turbulence intensities are 33% lower for Re = 400000 when compared to Re = 88000 jets.
This is similar to the observations of Xu et al.[25] for turbulent plane jets.

For all the simulations performed the Reynolds stress behavior is found to lie
mostly within the profiles measured by Panchapakesan & Lumley [15] and
Hussein et al. [13]. Slight deviation observed in near centerline region for some of the
simulation is because the profiles were averaged in same range $ x \ge 20r_0 $, for all simulations, where some of the
jets simulated are in transition or near-transition zone. Turbulence intensity profiles of J1G2a are not presented as the turbulence regime simulated for the case is small, and averaging in that span will not produce accurate results.

§.§ Energy spectra

Turbulent kinetic energy spectra, for simulations performed, are presented in Figure: [<ref>]. Energy spectra is broad banded with
the smallest scales having approximately $10^{6}$ times less energy than greatest scale for all simulations. Turbulent kinetic
energy of the simulated jets peaks around Strouhal Number, St $ = 0.3 - 0.4$. Figure: [<ref> & <ref>]
shows that on grid refinement finer scales are captured accurately. Figure: [<ref>] shows energy content of
a fixed length-scale (Strouhal Number) increases with increase in Reynolds number.
This is observed for smaller length-scales or higher strouhal numbers (St $\ge$ 0.5), whereas the
energy content in the highest length-scales or lower strouhal numbers remain independent of Reynolds number.

§.§ Flow organisation

Axial velocity contour of simulated jet is presented in Figure: [<ref>]. Except for the
differences in transition length, flow organisation remains the same for all simulations. Potential core
collapse is observed at a range of $x = 14r_0$ - $20r_0$ on an average for the simulations performed, with exception to Re $=3600$ case. (where
the potential core collapse is observed at $x = 39r_0$).
It is observed that high Reynolds number jets have early transition (potential core collapse) for the same level of inflow turbulence.

§ SUMMARY

An early break-down to turbulence, decreased axial-velocity decay rates and decreased levels of turbulence intensities are observed with increase in Reynolds numbers for turbulent round jets.
Eventhough more accurate numerical simulations for a range of Reynolds number are required to understand the effects, it could be observed that increased smaller length scales play a significant role in determining decay rates.
Jet spread-rate though observed to remain almost independent of Reynolds number, it could be an artifact of cartesian grids inability to represent the physics.
Simulations on a cylindrical grid, might reveal dependence of spread-rate on Reynolds number.

§ ACKNOWLEDGEMENTS

Author acknowledges the fruitful discussions with Dr. Joseph Mathew, Indian Institute of Science, Bangalore. Author is thankful to Dr. Subramanian Ganesh, General Electric, India for providing the basic computational algorithm. Computations were carried during author's stay at Indian Institute of Science, India, as a part of his graduate research on jet instability modes of Hartmann whistle. Analysis and findings published in this article were made after author moved to his new position at Wageningen University, The Netherlands.

[1]
Ball, C., Fellouah, H., and Pollard, A. (2012).
“The flow field in turbulent round free jets.” Progress in
Aerospace Sciences, 50, 1–26.

[2]
Benzi, R., Ciliberto, S., Tripiccione, R., Baudet, C., Massaioli, F., and
Succi, S. (1993).
“Extended self-similarity in turbulent flows.” Physical
review E, 48(1), R29.

[3]
Boersma, B., Brethouwer, G., and Nieuwstadt, F. (1998).
“A numerical investigation on the effect of the inflow conditions on
the self-similar region of a round jet.” Physics of Fluids
(1994-present), 10(4), 899–909.

[4]
Bogey, C. and Bailly, C. (2006).
“Computation of a high Reynolds number jet and its radiated noise
using large eddy simulation based on explicit filtering.” Computers
& Fluids, 35(10), 1344 – 1358.

[5]
Dimotakis, P. E. (2000).
“The mixing transition in turbulent flows.” Journal of Fluid
Mechanics, 409, 69–98.

[6]
Fellouah, H., Ball, C., and Pollard, A. (2009).
“Reynolds number effects within the development region of a
turbulent round free jet.” International Journal of Heat and Mass
Transfer, 52(17), 3943–3954.

[7]
Freund, J. B. (2001).
“Noise sources in a low-reynolds-number turbulent jet at mach
0.9.” Journal of Fluid Mechanics, 438, 277–305.

[8]
Ganesh, S. (2014).
“Large eddy simulation of free and impinging subsonic jets and their
sound fields.” Ph.D. thesis, Indian Institute of Science, Department of
Aerospace Engineering, Indian Institute of Science, Department of Aerospace

[9]
George, W. K. (1989).
“The self-preservation of turbulent flows and its relation to
initial conditions and coherent structures.” Advances in turbulence,

[10]
Hixon, R. and Turkel, E. (1998).
High-accuracy compact MacCormack-type schemes for computational
National Aeronautics and Space Administration, Lewis Research Center.

[11]
Hixon, R. and Turkel, E. (2000).
“Compact implicit maccormack-type schemes with high accuracy.”  Journal of Computational Physics, 158(1), 51–70.

[12]
Hussain, A. F. (1986).
“Coherent structures and turbulence.” Journal of Fluid
Mechanics, 173, 303–356.

[13]
Hussein, H. J., Capp, S. P., and George, W. K. (1994).
“Velocity measurements in a high-reynolds-number,
momentum-conserving, axisymmetric, turbulent jet.” Journal of Fluid
Mechanics, 258, 31–75.

[14]
Lele, S. K. (1992).
“Compact finite difference schemes with spectral-like resolution.”  Journal of computational physics, 103(1), 16–42.

[15]
Panchapakesan, N. and Lumley, J. (1993).
“Turbulence measurements in axisymmetric jets of air and helium.
part 1. air jet.” Journal of Fluid Mechanics, 246, 197–223.

[16]
Poinsot, T. J. and Lele, S. (1992).
“Boundary conditions for direct simulations of compressible viscous
flows.” Journal of computational physics, 101(1), 104–129.

[17]
Pope, S. B. (2001).
“Turbulent flows.

[18]
Thompson, K. W. (1987).
“Time dependent boundary conditions for hyperbolic systems.” Journal of computational physics, 68(1), 1–24.

[19]
Thompson, K. W. (1990).
“Time-dependent boundary conditions for hyperbolic systems, ii.”  Journal of Computational Physics, 89(2), 439–461.

[20]
Uddin, M. and Pollard, A. (2007).
“Self-similarity of coflowing jets: the virtual origin.” Physics of Fluids (1994-present), 19(6), 068103.

[21]
Varadharajan, R.
“Les study of free jets and jets impinging on cuboidal cavity.”  Ph.D. thesis, Ph.D. thesis.

[22]
Varadharajan, R., Kamin, M., Ganesh, S., and Mathew, J. (2016).
“On the jet instability modes of hartmann whistle.” Journal
of Sound and Vibration, (Submitted).

[23]
Visbal, M. R. and Rizzetta, D. P. (2014).
“Large-Eddy Simulation on Curvilinear Grids Using Compact
Differencing and Filtering Schemes.” ASME J. Fluids Engg.,
124(December 2002).

[24]
Xu, G. and Antonia, R. (2002).
“Effect of different initial conditions on a turbulent round free
jet.” Experiments in Fluids, 33(5), 677–683.

[25]
Xu, M., Pollard, A., Mi, J., Secretain, F., and Sadeghi, H. (2013).
“Effects of reynolds number on some properties of a turbulent jet
from a long square pipe.” Physics of Fluids (1994-present), 25(3),

[hide axis]
[color=red,solid,line width= 1.0 pt,forget plot]
[color=blue,solid,line width= 1.0 pt,forget plot]
[color=cyan,solid,line width= 1.0 pt,forget plot]
[color=green, solid, line width= 1.0 pt, forget plot]

[hide axis]
line width= 1.0 pt,
forget plot
densely dashed,
line width=1.0pt,
forget plot
line width=1.0pt,
forget plot
loosely dashed,
line width=1.0pt,
forget plot
line width=1.0pt,
forget plot


11institutetext: The University of Texas at Austin, TX USA
11email: {smodak11, npatt, isil<EMAIL_ADDRESS>
#  Synapse: Learning Preferential Concepts from Visual Demonstrations

Sadanand Modak 11 0000-0001-7618-9274 Noah Patton 11 0009-0002-7028-518X Isil
Dillig 11 0000-0001-8006-1230 Joydeep Biswas 11 0000-0002-1211-1731

###### Abstract

This paper addresses the problem of _preference learning_ , which aims to
learn user-specific preferences (e.g., _“good parking spot”_ , _“convenient
drop-off location”_) from visual input. Despite its similarity to learning
_factual_ concepts (e.g., “red cube”), preference learning is a fundamentally
harder problem due to its subjective nature and the paucity of person-specific
training data. We address this problem using a new framework called Synapse,
which is a neuro-symbolic approach designed to efficiently learn preferential
concepts from limited demonstrations. Synapse represents preferences as neuro-
symbolic programs in a domain-specific language (DSL) that operates over
images, and leverages a novel combination of visual parsing, large language
models, and program synthesis to learn programs representing individual
preferences. We evaluate Synapse through extensive experimentation including a
user case study focusing on mobility-related concepts in mobile robotics and
autonomous driving. Our evaluation demonstrates that Synapse significantly
outperforms existing baselines as well as its own ablations. The code and
other details can be found on the project website
https://amrl.cs.utexas.edu/synapse.

###### Keywords:

Concept learning Neuro-symbolic programming Program Synthesis Visual Reasoning

##  1 Introduction

Figure 1: Illustration of Synapse for identifying good contingency locations.
(a) Synapse learns a neuro-symbolic program that represents the preferential
concept based on user demonstrations, which include both a NL explanation and
a physical demonstration. The learning algorithm consists of three steps,
namely updating the concept library, synthesizing a program sketch, and
performing parameter synthesis; (b) The new sketch gets synthesized based on
the previous sketch and the current natural language description; (c)
Parameter synthesis determines suitable values for numeric parameters in the
program sketch; (d) Synapse evaluates the program on a new query image to
return a preference (in this case, boolean) mask over the input image.

Imagine trying to come up with a definition of _“a good taxi drop-off
location”_. One person may consider a spot to be a good drop-off location
depending on whether it is close to the door of a building, while someone else
might want it in the shade. Such concepts vary from person to person and
inherently depend on their preferences. We call them _preferential concepts_ ,
and we are interested in the problem of _preference learning_ from visual
input. Learning preferences is important because we want systems that are
customizable and that can adapt to end-users. This problem is quite related to
the task of visual concept learning, wherein much of the work focuses on
learning concepts such as _having the color red_ or _being to the left of
another object_ [5, 52, 40, 43, 35, 23, 33, 61, 13, 41, 44, 31, 49, 26, 55,
25]. All such prior work assumes there is a _ground-truth_ for the concept,
_i.e_., the definition of the concept does not differ among people, and as a
consequence, sufficiently many examples are available, and can be objectively
evaluated. We refer to such concepts as _factual concepts_. While most prior
work that learns visual concepts exploits the availability of large datasets
such as CLEVR [29], those methods cannot be applied to preference learning
because it is a data-impoverished setting by its very nature: a single
individual can put up with providing only so much data. This limitation is
also present in most of the preference learning work in the reinforcement
learning literature as well, where human preferences are represented as neural
networks [59, 10] or latent reward models [60, 2, 14, 48, 7, 15]. Furthermore,
because preferences are inherently individual, they can depend on entirely
different concepts, such as in the drop-off location example above (_i.e_.,
based on _proximity to door_ as opposed to _being in the shade_). This
requires learning novel visual concepts in a _hierarchical_ manner. Lastly,
coming up with a complete definition of a preferential concept at once is
itself a hard problem: it is much easier for someone to _show_ examples that
satisfy their intuition as humans tend to build their notion of a preferential
concept over time. Thus, preference learning calls for an approach that can
handle _incremental learning_ from visual demonstrations.

To address these challenges, we present Synapse, a novel framework that learns
human preferences in a data-efficient manner. In contrast to prior preference
learning approaches [64, 59, 10, 19, 60, 2, 14, 48, 7, 15] which take in weak
reward signals to learn preferences, we use a more direct form of a preference
signal, which consists of a physical demonstration including visual data, and
a natural language (NL) explanation for the preference. We use NL input from
the human to identify new concepts to be learned as well as how to compose
them. However, in addition to learning new concepts or composing existing
ones, preferences also have a quantitative aspect. For example, to be a good
drop-off spot, you should be close to a door, but exactly how close is a
personal preference. This is where the demonstrations come into play and allow
us to infer quantitative aspects of the preference that are hard to capture
via natural language alone. Finally, to allow _incremental_ , _data-efficient_
learning, Synapse expresses preferential concepts as programs in _neuro-
symbolic_ domain specific language (DSL) operating over images, and learns
these programs based on demonstrations. Such a programmatic representation
also facilitates _life-long learning_ , allowing incremental changes to the
learnt program as new demonstrations arrive.

Fig. 1 shows a schematic of our proposed Synapse framework. Given a user
demonstration (_i.e_., the physical demonstration and NL input), the general
workflow of Synapse has three main components: First, Synapse leverages the
user’s NL explanation, along with Synapse’s existing _concept library_ , to
ground the visual _concepts_ needed to represent the user’s preference. If the
NL explanation contains auxiliary concepts that are not part of Synapse’s
existing concept library, Synapse may query the user for additional
demonstrations of the auxiliary concept, which are then used to update
Synapse’s concept library. Once the library contains all required concepts,
Synapse uses the NL explanation to generate a so-called _sketch_ which is a
program in our DSL with missing values for numeric parameters. Finally,
Synapse uses constrained optimization techniques (based on _maximum
satisfiability_ [24]) to find values of the numeric paramaters that are
maximally consistent with the user’s physical demonstrations.

To demonstrate the effectiveness of our framework, we evaluate it on three
mobility-related visual preferential concepts which find their use in mobile
robotics and autonomous driving: a) CONTINGENCY: What is a good spot for a
robot to pull over to in case of contingency?, b) DROPOFF: What is a good
location for an autonomous car to stop and drop-off the customer?, and c)
PARKING: Where can the autonomous car be parked?. We carry out extensive
experiments, including a user study, spanning multiple baselines. Empirical
results show that Synapse outperforms the baselines by a significant margin,
especially when evaluated on out-of-distribution data — even when Synapse is
trained on an order of magnitude fewer examples than baseline Neural-Network
(NN) approaches. A case study with multi-user preferences demonstrates that
our method can learn personalized preferences unique to each user with just a
few demonstrations.

In summary, this paper contributes:

  1. 1.

Synapse, a neuro-symbolic framework to learn and evaluate preferences

  2. 2.

A new method for hierarchical lifelong learning from visual demonstrations and
natural language

  3. 3.

A comprehensive experimental evaluation of the proposed approach, including a
multi-user case study and comparison against baselines and ablations

##  2 Related work

Figure 2: A taxonomy of (visual) concept learning and VQA

Our framework positions itself in the larger field of concept learning and
visual question answering (VQA), a broad taxonomy depicted in Fig. 2. While
there exist Reinforcement-Learning based methods for preference learning, most
of them fall in the imitation learning setting, where human preferences are
represented via neural policies [59, 10] or latent reward models [60, 2, 14,
48, 7, 15]. Further, they do not deal with natural language, but rather take
some form of weak preference signal as input. In the following discussion, we
focus on work that is most closely-related to Synapse.

####  2.0.1 Language Model Programs (LMPs).

Generating executable programs from natural language is not a new idea. Many
earlier works [5, 52, 63, 40, 35, 23, 13, 41, 44] use custom semantic parsers
to perform specific tasks. However, with the advent of Large Language Models
(LLMs) such as GPT-4 [1], LMPs have gained significant attention [27, 22, 53,
54, 50, 26, 16, 37, 25, 56] due to the extensive knowledge that these
foundation models possess. Code-as-Policies [36] pioneered the effort in this
direction and demonstrated that LLMs can generate simple Python programs for
tasks ranging from drawing shapes to tabletop manipulation. This approach uses
recursive prompting as a strategy to get rid of invalid function calls.
Voyager [55] builds an LLM-powered embodied agent that can learn a diverse set
of skills in a lifelong manner in the game of Minecraft.

####  2.0.2 Neuro-symbolic visual learning and reasoning.

Neuro-symbolic approaches couple the interpretability of rule-based symbolic
AI with the flexibility of neural networks. This idea of composing neural
networks into classical symbolic programs to learn a particular concept for
VQA dates back to Neural Module Networks (NMN) [5]. NMNs compose neural
modules into an end-to-end differentiable network, which are then jointly
trained. While methods like LLaVa [38] and GPT-4V aim to tackle visual-
language problems in a completely neural way, works such as VisProg [22] and
ViperGPT [53] build on top of these and take the zero-shot route by composing
the available pretrained models. One work [52] uses a trained semantic parser
to first extract useful feature definitions from a few statements describing
the concept, which are then evaluated for each datapoint to build feature
vectors on which standard classification can be done. NS-VQA [63] is another
approach that uses a separately trained visual parser to generate a structured
representation of objects in the image and a semantic parser trained to parse
the question into a predefined DSL format, which is followed by Pythonic
program execution to generate the answer. NS-CL [40] uses the same framework
as NS-VQA [63], but instead of answering questions given the trained modules,
it represents concepts as neural operators and tries to learn them given the
question-answer pairs. DCL [13] goes one step further and extracts keyframes
from video and tracks object trajectories to build latent feature vectors that
describe dynamic concepts such as collision. Similarly, a host of other
methods have been developed [43, 35, 23, 33, 41, 61, 44, 31, 49, 26, 55, 25],
of which a few [35, 31] also use learning from demonstrations such as GUI
feedback in a mobile app to guide interactive learning.

####  2.0.3 Program synthesis.

There is a rich literature on synthesizing programs from user-provided
specifications in the programming languages community [21, 4, 51, 45, 30, 3,
9, 20, 28, 17, 12, 58, 57, 18, 11]. Closer to our application domain, LDIPS
[24] is a recent approach that makes use of the dimensionality of variables to
sythesize Action Selection Policies (ASPs) given user demonstrations. From the
viewpoint of visual reasoning and concept learning, [43] tries to solve Visual
Discrimination Puzzles (VDP) by first constructing scene graphs for all images
in the puzzle, and then synthesizing a discriminator expressed in first-order
logic by performing a full-blown discrete search. However, this can quickly
become inefficient as problem size scales. To tackle this, our method uses
natural language informed sketch generation and performs synthesis over the
space of parameters.

##  3 Method

We define a _preference task_ $\mathcal{T}{}:=\langle O,Q,P\rangle$ as a tuple
consisting of an observation space $O$, a query space $Q$, and a preference
space $P$. A preference evaluator $\pi$ accepts an observation and a query,
and returns a preference value: $\pi:O\times Q\rightarrow P$. The goal of
_preference learning_ is to synthesize a suitable evaluator $\pi$ that
accurately predicts a person’s visual preferences given an image. As stated
earlier, a distinguishing feature of preference learning is that it must be
performed using _small_ amounts of training data due to the subjective nature
of preferences. To enable data-efficient learning, our proposed Synapse
approach represents the preference evaluator $\pi$ as a neuro-symbolic program
in a DSL and synthesizes $\pi$ from a small number of user demonstrations,
where each demonstration includes a sequence of images along with a natural
language explanation for the user’s preference. In the following discussion,
we first present our DSL for representing preferences and then describe our
learning algorithm.

###  3.1 Representing Preferences

We represent preference evaluation functions $\pi$ (or preferences for short)
in the neuro-symbolic DSL shown in Fig. 3. This DSL is parameterized over a
so-called _concept library_ $\mathcal{C}$, which includes both predicates
$\mathcal{C}_{b}$ as well as non-boolean functions $\mathcal{C}_{f}$. The
concept library includes built-in operators and predicates (e.g.,
$+,\times,\leq,\ldots$), pre-trained neural networks (e.g., for object
classification and terrain detection), zero-shot visual language models
(VLMs), as well as previously learned concepts and functions (expressed in the
same DSL). We sometimes use the notation $\pi_{\mathcal{C}}$ to denote
programs using concept library $\mathcal{C}$ and omit the subscript
$\mathcal{C}$ where it is clear from context.

At a high level, a program $\pi$ consists of (nested) if-then-else statements
and is therefore conceptually similar to a decision tree. Each leaf of this
decision tree is a preference (e.g., _good, bad, neutral_) drawn from the
preference space $P$, which is assumed to be a finite set. Internal nodes of
the decision tree are neuro-symbolic conditions $\phi$, which include boolean
combinations of predicates of the form $p(t_{1},\ldots,t_{n})$ where each
$t_{i}$ is a neuro-symbolic term and $p$ is a predicate drawn from
$\mathcal{C}_{b}$, which could be a built-in relation (e.g., $\leq$), result
of a neural classifier, or a previously-learned concept (e.g., _close-to_).

Inputs
q∈Qo∈O

Constants
v∈{Int, Real, …}p∈P

Terms
t := q ∣ o ∣ v∣ f(t_1, …, t_n) where  f ∈$\mathcal{C}$_f

Conditions
ϕ:=p_c(t_1, …, t_n) where  p_c ∈$\mathcal{C}$_b∣ ¬ϕ ∣ ϕ∧ϕ ∣ ϕ∨ϕ

Programs
π:=p ∣ if (ϕ) then π else π’

Figure 3: The Synapse neuro-symbolic DSL parametrized over concept library
$\mathcal{C}$, which consists of a set of predicates $\mathcal{C}_{b}$ and
functions $\mathcal{C}_{f}$.

Input: _a set of previously seen demonstrations $\mathcal{D}$, the new
demonstration $d_{n}$, a potentially empty previous sketch $\hat{\pi}_{o}$, a
previous concept library $\mathcal{C}$_

Output: _The new demonstrations set $\mathcal{D}^{\prime}$, a neuro-symbolic
preference evaluator $\pi$ parameterized by the new concept library
$\mathcal{C}^{\prime}$, the sketch $\hat{\pi}$ used to generate $\pi$_

1: $\mathsf{Learn}(\mathcal{D},d_{n},\hat{\pi}_{o},\mathcal{C})$

2: # Update concept library with new natural language utterance

3:
$\mathcal{C}^{\prime}\leftarrow\mathsf{UpdateConceptLibrary}(d_{n}.e,\mathcal{C})$

4: # Get the updated sketch from the new demonstration and previous sketch

5:
$\hat{\pi}\leftarrow\mathsf{SketchSynth}(d_{n},\hat{\pi}_{o},\mathcal{C}^{\prime})$

6: # Fill the holes in $\hat{\pi}$ based off of the demonstrations
$\mathcal{D}^{\prime}$

7: $\pi\leftarrow\mathsf{ParamSynth}(\hat{\pi},\mathcal{D}\cup\\{d_{n}\\})$

8: return $\mathcal{D}\cup\\{d_{n}\\},\pi,\mathcal{C}^{\prime},\hat{\pi}$

Algorithm 1 Synapse-Learn

Input: _A new natural language explanation $e$ and the previous concept
library $\mathcal{C}$_

Output: _A new concept library $\mathcal{C}^{\prime}$_

1: $\mathsf{UpdateConceptLibrary}(e,\mathcal{C})$

2: $\mathcal{C}^{\prime}\leftarrow\mathcal{C}$ # Initialize new concept
library with old concept library

3: # Extract new visual groundings from natural language and

4: # add them to concept library

5: $g\leftarrow\mathsf{ExtractEntities}(e,\mathcal{C})$

6: $\mathcal{C}^{\prime}\leftarrow\mathcal{C}^{\prime}\cup g$

7: # Extract new predicates from $e$

8: $\mathrm{Preds}\leftarrow\mathsf{ExtractPredicates}(e,\mathcal{C})$

9: # Recursively update concepts with user feedback

10: for $\mathrm{pred}\ \in\mathrm{Preds}\ \textbf{where}\ pred\ \notin\
\mathcal{C}$

11: $\mathcal{D}\leftarrow\emptyset\ \
\color[rgb]{0,0.3,0.0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.3,0.0}{\\#\text{{Empty
initial demonstration
set}}}\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}$

12: $\mathcal{C}^{\prime\prime}\leftarrow\mathcal{C}^{\prime}$

13: $\hat{\pi}\leftarrow\mathrm{None}$

14: do

15: $d\leftarrow\mathsf{QueryUserForDemonstration}(\mathrm{pred})$

16:
$\mathcal{D},\pi,\mathcal{C}^{\prime\prime},\hat{\pi}\leftarrow\mathsf{Learn}(\mathcal{D},d,\hat{\pi},\mathcal{C}^{\prime\prime})$

17: while $d$

18: $\mathcal{C}^{\prime}\leftarrow\mathcal{C}^{\prime\prime}$

19: return $\mathcal{C}^{\prime}$

Algorithm 2 Update Concept Library

Input: _A program sketch $\hat{\pi}$, a set of demonstrations $\mathcal{D}$_

Output: _A complete program $\pi$_

1: $\mathsf{ParamSynth}(\hat{\pi},\mathcal{D})$

2: $\varphi\leftarrow$true # Initialize condition

3: for $d\in D$

4: # Perform partial evaluation on the sketch and demonstration

5: # to get a simplified sketch $\hat{\pi}$ and expected result $r$

6: $(\hat{\pi}^{\prime},r)\leftarrow\mathsf{PartialEval}(\hat{\pi},d)$

7: # Merge with condition

8: $\varphi\leftarrow\varphi\ \land\ [\\![\hat{\pi}^{\prime}]\\!]^{r}$

9: # Include negation for each parameter $\in P$

10: for $i\in P$

11: if $(i\neq r)\ \varphi\leftarrow\varphi\
\land\neg[\\![\hat{\pi}^{\prime}]\\!]^{i}$

12: # Use solver to fill holes over symbolic features

13: $\pi\leftarrow\mathsf{Model}(\varphi)$

14: return $\pi$

Algorithm 3 Parameter Synthesis

###  3.2 Learning Preferences

We now discuss our learning algorithm, Synapse-Learn, for synthesizing
preference evaluation functions from a set of demonstrations $\mathcal{D}$. As
Synapse-Learn is meant to be used in a life-long-learning setting, we present
it as an incremental algorithm that takes one additional demonstration in each
invocation and returns an updated preference evaluation function. As mentioned
earlier, we represent each demonstration $d$ as a pair $(t,e)$ where $t$ is a
physical demonstration consisting of a sequence of images and LiDAR point
clouds and $e$ is a natural language explanation for the preference. Given a
demonstration $d$, we write $d.t$ and $d.e$ to denote its physical
demonstration and explanation component respectively.

In addition to the new demonstration $d_{n}$, Synapse-Learn takes three
additional arguments, namely the previous set of demonstrations $\mathcal{D}$,
the previously learnt preference evaluation function $\pi_{o}$ (None for the
first invocation), and the current concept library $\mathcal{C}$, which is
initialized to contain only a set of built-in concepts. Synapse-Learn uses the
old program $\pi_{o}$ to bootstrap the learning process, and the previous
demonstrations are required to ensure that the updated program is consistent
with _all_ demonstrations provided thus far.

At a high level, the learning procedure consists of three steps, which are
explained in more detail in the remainder of this section:

  1. 1.

Updating the concept library: Synapse first checks whether the existing
concept library $\mathcal{C}$ is sufficient for successfully learning the
desired preference evaluation function. For example, if the natural language
explanation uses the term “far away” but the concept library does not contain
a suitable definition, Synapse-Learn interactively queries the user for
clarification and updates its concept library as needed.

  2. 2.

Synthesizing a program sketch: If the concept library is sufficient for
representing the preference, Synapse-Learn proceeds to synthesize a so-called
_program sketch_ , which is a program with missing constants to be
synthesized. We differentiate between program sketches and complete programs
because the user’s natural language explanation is often sufficient to
understand the general structure of the preference evaluation function but not
its numeric parameters, which can only be accurately learned from the physical
demonstrations. Thus, Synapse-Learn generates the program sketch based only on
the natural language explanation.

  3. 3.

Parameter synthesis: The final phase of the learning algorithm utilizes all
physical demonstrations provided thus far to synthesize the unknown numeric
parameters of the sketch using a constraint-solving approach. For example, if
the user’s NL explanation mentions “not too close to the sidewalk”, the
physical demonstrations are needed to understand what the user considers “too
close”. For this reason, Synapse-Learn utilizes a separate parameter synthesis
procedure to determine suitable numeric parameters from the physical
demonstrations.

####  3.2.1 Concept Library Update.

Synapse analyzes the user’s natural language explanation $e$ to extract
concepts of interest. We differentiate between two types of concepts: (1)
entities (e.g., car, door, sidewalk) and (2) predicates (e.g., far, near).
Because Synapse uses an open-vocabulary visual language model to find entities
of interest in the current observation, new entity concepts do not require
interacting with the user. On the other hand, if the natural language
explanation contains new predicates that are not part of the existing concept
library, Synapse needs to query the user to provide suitable demonstrations.

Algorithm 2 summarizes this discussion as an algorithm. As shown in lines 5-6,
the ExtractEntities procedure grounds the entities used in the NL description
and cross-references them against existing entities in the concept library.
Any new entities are added to the concept library without requiring user
interaction, as we assume that any entity can be extracted from the
observation using a modern VLM. Lines 8-17, on the other hand, extract new
_predicates_ from the natural language description. Since the semantics of
these predicates are not known a priori (unless they are already in the
concept library), we must query the user to learn their semantics. Thus, the
QueryUserForDemonstration procedure obtains new demonstrations, which are then
used to synthesize the implementation of the new predicate through recursive
invocation of Synapse-Learn at line 15. Thus, when the UpdateConceptLibrary
procedure terminates, the new concept library $\mathcal{C}^{\prime}$ contains
all entities and predicates of interest.

####  3.2.2 Program Sketch Synthesis.

Once Synapse has all the required concepts as part of its library, it uses a
large language model to synthesize a suitable program sketch based on the
natural language description and concept library. In particular, it first
prompts the LLM to translate the NL explanation $e$ to a pair $(\Phi,p)$ where
$\Phi$ is a formula (in conjunctive normal form) over the predicates in the
concept library and $p$ is the user’s preference. Then, in a second step,
Synapse prompts the LLM to update the previous sketch $\hat{\pi}$ to a new one
$\hat{\pi}^{\prime}$ such that $\hat{\pi}^{\prime}$ returns $p$ when $\Phi$
evaluates to True. We found this two-stage process of first converting the NL
explanation to a CNF formula and then prompting the LLM to repair the old
sketch to work better in practice compared to prompting the LLM directly with
all inputs (see Section 4).

####  3.2.3 Parameter Synthesis.

As mentioned earlier, a program sketch contain unknown numeric parameters that
arise from the ambiguity of natural language (e.g., what does “close” mean in
terms of distances between objects?) Thus, the last step of the Synapse
pipeline utilizes the user’s physical demonstrations to synthesize numeric
parameters in the sketch. Our parameter synthesis algorithm is summarized in
Algorithm 3 and constructs a logical formula $\varphi$ whose models are
guaranteed to be consistent with all demonstrations. The algorithm constructs
this formula $\varphi$, which is initialized to true at line 2, as follows:
First, for each physical demonstration $d$, it partially evaluates $\hat{\pi}$
by concretely evaluating all expressions without unknowns. For example, if the
sketch contains the predicate distanceTo(car), we can use the observation from
$d$ to compute the actual distance between the subject and the car. This
partial evaluation (line 6) yields a much simpler sketch containing only
unknowns to be synthesized but no other variables. Now, given a sketch
$\hat{\pi}$, let $\llbracket\hat{\pi}\rrbracket^{i}$ denote the condition
under which $\hat{\pi}$ returns preference $p_{i}\in P$, and suppose that the
current demonstration $d$ illustrates preference class $p_{r}$. Since we would
like the synthesized program to return $p_{r}$ for demonstration $d$,
$\llbracket\hat{\pi}\rrbracket^{r}$ should evaluate to true, and for all other
preference classes $p_{i}$ where $i\neq r$,
$\llbracket\hat{\pi}\rrbracket^{i}$ should evaluate to false. Thus, the loop
in lines 10-12 iteratively strengthens formula $\varphi$ by conjoining it with
$\llbracket\hat{\pi}\rrbracket^{r}$ and the negation of
$\llbracket\hat{\pi}\rrbracket^{i}$ for any $i$ distinct from $r$. Finally, we
use an off-the-shelf constraint solver to obtain a model of the resulting
formula.111In general, the demonstration may be noisy, meaning that $\varphi$
could be unsatisfiable. Since this is quite often the case, our implemetation
uses a MaxSMT solver [8] to find a solution that maximizes the number of
satisfied clauses in the formula. By substituing the unknowns in the sketch
with the model returned by the solver, Synapse obtains a program that is
maximally consistent with the user’s demonstrations.

##  4 Evaluation

To test the effectiveness of Synapse, we evaluate it on three mobility-related
preferential concepts relevant to mobile robotics and autonomous driving
domains: a) CONTINGENCY: _What is a good spot for a robot to pull over to in
case of an emergency?_ , b) DROPOFF: _What is a good location for an
autonomous taxi to stop and drop-off a customer?_ , and c) PARKING: _What is a
good location for parking an autonomous car?_. The following subsection gives
some details on how we ground Synapse to learn such mobility-related concepts.
Further information is provided in supplementary material.

###  4.1 Implementation Details

Inputs. Human demonstrations include the robot trajectories of the user
driving the robot to the desired location using a joystick based on the user’s
preference, and a natural language description to explain the rationale for
choosing that location. We use RGB-camera images as well as the LIDAR sensor
data for the entire trajectory to construct a symbolic representation of the
scene.

Models. We use Grounded-SAM [46, 39, 32] zero-shot VLM for object detection
and a custom terrain model with SegFormer architecture [62] trained to segment
terrains from sensor data. We use both of these models to get segmentation
masks of the neural concepts (_i.e_., objects and terrains) currently in the
concept library. We use GPT-4 [1] as the language model for sketch synthesis
in Synapse.

Concept Library Initialization. We impart some basic capabilties to the
learning framework by seeding the concept library with a total of six
functions based on neural models (_i.e_., terrain model and VLM) as well as
some domain-specific predicates. For example, terrain_at returns the terrain
type at a specific pixel location on a provided image, while
project_map_to_pixel transforms real-world 3D coordinates to pixel
coordinates.

####  4.1.1 Querying the user.

As mentioned earlier in Sec. 3.2.1, Synapse can interactively query the user
to clarify new concepts that are present in the user’s NL explanation but not
in the current concept library. In principle, Synapse can query the user for
both physical demonstrations and NL explanations. To reduce the burden on the
user, Synapse, by default, only queries the user for NL explanations of
auxiliary concepts and performs synthesis of auxiliary concepts using NL
explanations alone.

###  4.2 Results

To evaluate Synapse, we create a dataset of 815 labeled images taken from the
UT Austin campus area, where the labels mark the locations on the images that
are consistent with the intended user preference for each of the three tasks.
We split the dataset into three sets: train, in-distribution test, and out-of-
distribution test sets. The train and in-distribution sets belong to the same
part of the UT Austin campus region, while the out-of-distribution set belongs
to a different part of the region. Table 1 shows the comparison against
various baselines. We use mean Intersection-Over-Union (mIOU) as the metric
and evaluate the following baselines: (1) pure neural models based on
SegFormer [62] architecture with pretrained weights, with and without depth
input, fine-tuned on our custom dataset; (2) GPT4 [1] with vision
capabilities, and (3) VisProg [22]. The ‘+’ for the latter two indicate
additional prompting that provides additional information about the underlying
reasoning behind the preferential concept. More details about the dataset and
how different baselines are trained and/or evaluated can be found in the
supplementary material.

We find that Synapse outperforms all baselines, and improves on the closest
baseline by a significant margin on out-of-distribution test data — 74.07
_vs_. 57.42 for CONTINGENCY, 80.72 _vs_. 55.04 for DROPOFF, and 62.75 _vs_.
52.90 for PARKING. Further, even though Synapse is trained on an order of
magnitude fewer samples (for instance, 29 demonstration for CONTINGENCY) than
neural baselines (for instance, 224 images for CONTINGENCY), it performs at-
par, if not better, on the train dataset.

Table 1: Mean IOU (%) results for the three concepts. The train set represents the training set for neural baselines – Synapse only needs 29 demonstrations (from the train area), and the visual-language baselines have not been fine-tuned. | CONTINGENCY | DROPOFF | PARKING
---|---|---|---
| train | in-test | out-test | train | in-test | out-test | train | out-test
Synapse | 77.64 | 76.29 | 74.07 | 79.32 | 80.18 | 80.72 | 68.60 | 62.76
SF-RGB-b0 | 70.49 | 62.75 | 57.42 | 72.99 | 68.13 | 52.21 | 57.68 | 49.66
SF-RGB-b5 | 74.59 | 70.48 | 56.00 | 77.26 | 72.83 | 55.04 | 68.63 | 52.91
SF-RGBD-b0 | 72.17 | 67.23 | 54.75 | 74.33 | 69.80 | 54.90 | 60.90 | 50.69
SF-RGBD-b5 | 76.48 | 67.81 | 56.11 | 77.69 | 70.70 | 52.39 | 71.06 | 49.99
GPT4V | 26.56 | 29.71 | 30.45 | 30.54 | 32.97 | 37.65 | 38.41 | 40.18
GPT4V+ | 28.73 | 28.96 | 33.92 | 39.38 | 38.34 | 39.14 | 41.38 | 39.77
VisProg | 45.62 | 45.63 | 44.98 | 46.29 | 47.49 | 48.07 | 39.43 | 40.99
VisProg+ | 38.94 | 39.21 | 41.83 | 39.17 | 39.44 | 43.14 | 38.88 | 38.99

###  4.3 Ablations

We investigate three types of ablations: (1) NN-ablations, in which we compare
the performance of neural baselines against Synapse when trained on the _same_
number of samples (_i.e_., 29), (2) LLM-based ablations, where we see how
using different LLMs affect the performance of Synapse, and (3) framework
ablations where we test different design choices. We investigate ablations
based on these framework features: (1) _feat1:_ whether it queries the user
for auxiliary concepts, (2) _feat2:_ whether it performs lifelong learning by
building on its concept library, and (3) _feat3:_ whether it has got _direct
access_ to the full concept library learned by Synapse. More details are
provided in the supplementary material.

Table 2 show how the ablation methods differ based on these features as well
their performance when evaluated on the CONTINGENCY dataset. It can be seen
that the neural ablations perform poorly since they are only exposed to so few
training samples that they aren’t able to generalize well to the full dataset.
Changing the program synthesis process of Synapse (_i.e_., doing direct
synthesis instead of two-step) or the LLM which in turn affects the accuracy
of the program sketch being synthesized, also has a significant impact on the
performance. Examples showing the difference in inference outputs for a few of
the baselines and the ablations is shown in Fig. 4.

Table 2: The results for the ablation studies. It highlights the importance of different elements of Synapse. All methods trained on 29 images (for neural) or 29 demonstrations (for Synapse-based). Evaluation is on the full CONTINGENCY dataset. | feat1 | feat2 | feat3 | LLM | mIOU (%)
---|---|---|---|---|---
Synapse | ✓ | ✓ | ✗ | GPT-4 | 76.11
Synapse-SynthDirect | ✗ | ✗ | ✗ | GPT-4 | 60.74
Synapse-SynthDirect+ | ✗ | ✗ | ✓ | GPT-4 | 68.86
Synapse-SynthCaP | ✗ | ✓ | ✗ | GPT-4 | 64.11
Synapse-CodeLLama | ✓ | ✓ | ✗ | CodeLLama [47] | 69.88
Synapse-StarCoder | ✓ | ✓ | ✗ | StarCoder [34] | 63.62
Synapse-PaLM2 | ✓ | ✓ | ✗ | PaLM2 [6] | 71.62
SF-RGB-b0 | - | - | - | - | 54.58
SF-RGB-b5 | - | - | - | - | 56.30
SF-RGBD-b0 | - | - | - | - | 55.84
SF-RGBD-b5 | - | - | - | - | 63.81
Figure 4: An illustrative comparison between Synapse, baselines, and ablations
for CONTINGENCY. Color coding shows the overlap of the predictions with the
_ground-truth_.

###  4.4 Case study with multi-user preferences

We also perform a case study222Reviewed by IRB and the determination was ‘Not
Human Research’ to evaluate the alignment and customizability characteristics
of Synapse with inputs from different users. We ask three participants to
provide demonstrations for their preferences in the CONTINGENCY task. We also
each participant to label a set of 15 images with their _ground-truth_
preference to be used for evaluation, which are not seen by the algorithm
during training. The results are summarized in Tab. 3. For each user, the
highest performance is attained by the program that learned from the same
user’s demonstrations, which indicates good alignment. The results also
indicate that the preference programs trained on one user’s input performs
poorly on a different user’s evaluation set, indicating that there are indeed
user-specific preferences captured by Synapse.

Table 3: The results (mean IOU (%)) for case study. Rows represent the learned programs for the particular user, and columns represent the _ground-truth_ for that user. | user-1 | user-2 | user-3
---|---|---|---
user-1 | 76.85 | 66.29 | 67.74
user-2 | 65.66 | 73.75 | 69.28
user-3 | 65.46 | 68.97 | 73.82

###  4.5 Reordering of demonstrations

We investigate if Synapse is susceptible to performance degradation if the
order of demonstrations is altered. For this evaluation, we provide the
demonstrations in a randomized order for the CONTINGENCY concept and then
evaluate the learned program. The variations in the mean IOU are shown in Fig.
5. It is clearly visible that after about 10-15 demonstrations, the effect of
re-ordering diminishes which shows the robustness of Synapse.

Figure 5: Plot showing susceptibility of Synapse to reordering of
demonstrations. _Gray_ area represents the mean IOU (%) variation as Synapse
sees more demonstrations.

##  5 Discussion of Limitations

Our experimental results demonstrate that Synapse significantly outperforms
baselines when it comes to generalizing to new data, which is crucial for
achieving transferability of learned models. However, Synapse has three
potential limitations that we discuss below.

First, Synapse relies substantially on how good the underlying neural modules
are and their capabilities, specifically, the zero-shot LLMs and VLMs. In our
experiments, we observe that a careful selection of parameters and verbose
prompting is needed to achieve best performance. Additionally, due to the poor
performance of existing VLMs for terrain detection, our implementation of the
Synapse framework uses a custom terrain model.

Second, Synapse relies on the quality of the user’s physical demonstrations
for accurate parameter synthesis. In practice, the demonstrations may be noisy
and imperfect, and Synapse tries to compensate for slight inconsistencies in
the user demonstration by using a constrained optimization approach based on
MaxSMT. While poor-quality demonstrations can still negatively impact overall
accuracy, our experiments show that Synapse is effective for demonstrations
provided by participants in our user study and that it is not particularly
sensitive to the other of demonstrations. Third, another limitation of
operating on real-world data with Synapse is the incomplete depth information
– Synapse approximates depth at all locations by interpolation, however, that
introduces noise into the quantitative evaluation. Better approaches for scene
completion [42] would reduce such noise.

##  6 Conclusion

We presented Synapse, a data-efficient, neuro-symbolic framework for learning
preferential concepts from a small number of human demonstrations. The
framework utilises a novel combination of visual parsing, large language
models, and program synthesis to represent preferences as interpretable
programs that can be synthesized from demonstrations. We experimentally showed
that Synapse achieves strong generalization on new data and that it
outperforms the baselines by a large margin ($\approx 15\%$ mIOU). Further, we
demonstrated that Synapse is able to align well with user preferences and that
it is robust to the order in which demonstrations are given. Finally, we
demonstrated the importance of key design choices underlying Synapse through
ablation studies.

## Acknowledgement

This work is supported in part by NSF awards OIA-2219236, CCF-1762299,
CCF-1918889, CNS-1908304, CCF-1901376, CNS-2120696, CCF- 2210831, and
CCF-2319471, as well as Amazon and JP Morgan. We thank the members of
Autonomous Mobile Robotics Laboratory (AMRL) at UT Austin for participating in
the user case study. We are also grateful to the amazing Hugging Face
ecosystem for simplifying the use of state-of-the-art neural models.

## References

  * [1] Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)
  * [2] Akrour, R., Schoenauer, M., Sebag, M., Souplet, J.C.: Programming by feedback. In: International Conference on Machine Learning. pp. 1503–1511. No. 32, JMLR. org (2014)
  * [3] Alur, R., Bodík, R., Dallal, E., Fisman, D., Garg, P., Juniwal, G., Kress-Gazit, H., Madhusudan, P., Martin, M.M.K., Raghothaman, M., Saha, S., Seshia, S.A., Singh, R., Solar-Lezama, A., Torlak, E., Udupa, A.: Syntax-guided synthesis. In: Dependable Software Systems Engineering, pp. 1–25 (2015)
  * [4] Alur, R., Černý, P., Radhakrishna, A.: Synthesis through unification. In: Kroening, D., Păsăreanu, C.S. (eds.) Computer Aided Verification. pp. 163–179. Springer International Publishing, Cham (2015)
  * [5] Andreas, J., Rohrbach, M., Darrell, T., Klein, D.: Neural module networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 39–48 (2016)
  * [6] Anil, R., Dai, A.M., Firat, O., et al.: PaLM 2 Technical Report (2023)
  * [7] Bestick, A., Pandya, R., Bajcsy, R., Dragan, A.D.: Learning human ergonomic preferences for handovers. In: 2018 IEEE international conference on robotics and automation (ICRA). pp. 3257–3264. IEEE (2018)
  * [8] Bjørner, N., Phan, A.D., Fleckenstein, L.: $\nu$z-an optimizing smt solver. In: Tools and Algorithms for the Construction and Analysis of Systems: 21st International Conference, TACAS 2015, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2015, London, UK, April 11-18, 2015, Proceedings 21. pp. 194–199. Springer (2015)
  * [9] Bornholt, J., Torlak, E., Grossman, D., Ceze, L.: Optimizing synthesis with metasketches. In: ACM SIGPLAN Notices. vol. 51, pp. 775–788. ACM (2016)
  * [10] Busa-Fekete, R., Szörényi, B., Weng, P., Cheng, W., Hüllermeier, E.: Preference-based evolutionary direct policy search. In: ICRA Workshop on autonomous learning. vol. 2 (2013)
  * [11] Chasins, S.E., Mueller, M., Bodik, R.: Rousillon: Scraping distributed hierarchical web data. In: Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. p. 963–975. UIST ’18, Association for Computing Machinery, New York, NY, USA (2018). https://doi.org/10.1145/3242587.3242661, https://doi.org/10.1145/3242587.3242661
  * [12] Chen, Q., Lamoreaux, A., Wang, X., Durrett, G., Bastani, O., Dillig, I.: Web question answering with neurosymbolic program synthesis. In: Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation. p. 328–343. PLDI 2021, Association for Computing Machinery, New York, NY, USA (2021). https://doi.org/10.1145/3453483.3454047, https://doi.org/10.1145/3453483.3454047
  * [13] Chen, Z., Mao, J., Wu, J., Wong, K.Y.K., Tenenbaum, J.B., Gan, C.: Grounding physical concepts of objects and events through dynamic visual reasoning. arXiv preprint arXiv:2103.16564 (2021)
  * [14] Christiano, P.F., Leike, J., Brown, T., Martic, M., Legg, S., Amodei, D.: Deep reinforcement learning from human preferences. Advances in neural information processing systems 30 (2017)
  * [15] Cui, Y., Niekum, S.: Active reward learning from critiques. In: 2018 IEEE international conference on robotics and automation (ICRA). pp. 6907–6914. IEEE (2018)
  * [16] Ding, Y., Zhang, X., Paxton, C., Zhang, S.: Task and motion planning with large language models for object rearrangement. arXiv preprint arXiv:2303.06247 (2023)
  * [17] Feng, Y., Martins, R., Bastani, O., Dillig, I.: Program Synthesis Using Conflict-Driven Learning. In: Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation. p. 420–435. PLDI 2018, Association for Computing Machinery, New York, NY, USA (2018). https://doi.org/10.1145/3192366.3192382, https://doi.org/10.1145/3192366.3192382
  * [18] Feser, J.K., Chaudhuri, S., Dillig, I.: Synthesizing data structure transformations from input-output examples. ACM SIGPLAN Notices 50(6), 229–239 (2015)
  * [19] Fürnkranz, J., Hüllermeier, E., Cheng, W., Park, S.H.: Preference-based reinforcement learning: a formal framework and a policy iteration algorithm. Machine learning 89, 123–156 (2012)
  * [20] Gulwani, S.: Automating string processing in spreadsheets using input-output examples. In: Proc. of POPL. pp. 317–330 (2011)
  * [21] Gulwani, S., Polozov, O., Singh, R.: Program synthesis. vol. 4, pp. 1–119 (2017)
  * [22] Gupta, T., Kembhavi, A.: Visual programming: Compositional visual reasoning without training. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 14953–14962 (2023)
  * [23] Han, C., Mao, J., Gan, C., Tenenbaum, J., Wu, J.: Visual concept-metaconcept learning. Advances in Neural Information Processing Systems 32 (2019)
  * [24] Holtz, J., Guha, A., Biswas, J.: Robot action selection learning via layered dimension informed program synthesis. In: Conference on Robot Learning. pp. 1471–1480. PMLR (2021)
  * [25] Hsu, J., Mao, J., Tenenbaum, J.B., Wu, J.: What’s left? concept grounding with logic-enhanced foundation models. arXiv preprint arXiv:2310.16035 (2023)
  * [26] Hsu, J., Mao, J., Wu, J.: Ns3d: Neuro-symbolic grounding of 3d objects and relations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 2614–2623 (2023)
  * [27] Hu, Z., Lucchetti, F., Schlesinger, C., Saxena, Y., Freeman, A., Modak, S., Guha, A., Biswas, J.: Deploying and evaluating llms to program service mobile robots. arXiv preprint arXiv:2311.11183 (2023)
  * [28] Jha, S., Gulwani, S., Seshia, S.A., Tiwari, A.: Oracle-guided component-based program synthesis. In: 2010 ACM/IEEE 32nd International Conference on Software Engineering. vol. 1, pp. 215–224. IEEE (2010)
  * [29] Johnson, J., Hariharan, B., Van Der Maaten, L., Fei-Fei, L., Lawrence Zitnick, C., Girshick, R.: Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2901–2910 (2017)
  * [30] Kalyan, A., Mohta, A., Polozov, O., Batra, D., Jain, P., Gulwani, S.: Neural-guided deductive search for real-time program synthesis from examples (2018). https://doi.org/10.48550/ARXIV.1804.01186, https://arxiv.org/abs/1804.01186
  * [31] Kane, B., Gervits, F., Scheutz, M., Marge, M.: A system for robot concept learning through situated dialogue. In: Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue. pp. 659–662 (2022)
  * [32] Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., Dollár, P., Girshick, R.: Segment anything. arXiv:2304.02643 (2023)
  * [33] Koh, P.W., Nguyen, T., Tang, Y.S., Mussmann, S., Pierson, E., Kim, B., Liang, P.: Concept bottleneck models. In: International conference on machine learning. pp. 5338–5348. PMLR (2020)
  * [34] Li, R., Allal, L.B., Zi, Y., et al.: StarCoder: may the source be with you! (2023)
  * [35] Li, T.J.J., Radensky, M., Jia, J., Singarajah, K., Mitchell, T.M., Myers, B.A.: Interactive task and concept learning from natural language instructions and gui demonstrations. arXiv preprint arXiv:1909.00031 (2019)
  * [36] Liang, J., Huang, W., Xia, F., Xu, P., Hausman, K., Ichter, B., Florence, P., Zeng, A.: Code as policies: Language model programs for embodied control. In: 2023 IEEE International Conference on Robotics and Automation (ICRA). pp. 9493–9500. IEEE (2023)
  * [37] Liu, B., Jiang, Y., Zhang, X., Liu, Q., Zhang, S., Biswas, J., Stone, P.: Llm+ p: Empowering large language models with optimal planning proficiency. arXiv preprint arXiv:2304.11477 (2023)
  * [38] Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. arXiv preprint arXiv:2304.08485 (2023)
  * [39] Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., Li, C., Yang, J., Su, H., Zhu, J., et al.: Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499 (2023)
  * [40] Mao, J., Gan, C., Kohli, P., Tenenbaum, J.B., Wu, J.: The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. arXiv preprint arXiv:1904.12584 (2019)
  * [41] Mei, L., Mao, J., Wang, Z., Gan, C., Tenenbaum, J.B.: Falcon: fast visual concept learning by integrating images, linguistic descriptions, and conceptual relations. arXiv preprint arXiv:2203.16639 (2022)
  * [42] Meng, X., Hatch, N., Lambert, A., Li, A., Wagener, N., Schmittle, M., Lee, J., Yuan, W., Chen, Z., Deng, S., et al.: Terrainnet: Visual modeling of complex terrain for high-speed, off-road navigation. arXiv preprint arXiv:2303.15771 (2023)
  * [43] Murali, A., Sehgal, A., Krogmeier, P., Madhusudan, P.: Composing neural learning and symbolic reasoning with an application to visual discrimination. arXiv preprint arXiv:1907.05878 (2019)
  * [44] Namasivayam, K., Singh, H., Bindal, V., Tuli, A., Agrawal, V., Jain, R., Singla, P., Paul, R.: Learning neuro-symbolic programs for language guided robot manipulation. In: 2023 IEEE International Conference on Robotics and Automation (ICRA). pp. 7973–7980. IEEE (2023)
  * [45] Patton, N., Rahmani, K., Missula, M., Biswas, J., Dillig, I.: Programming-by-demonstration for long-horizon robot tasks. Proc. ACM Program. Lang. 8(POPL) (jan 2024). https://doi.org/10.1145/3632860, https://doi.org/10.1145/3632860
  * [46] Ren, T., Liu, S., Zeng, A., Lin, J., Li, K., Cao, H., Chen, J., Huang, X., Chen, Y., Yan, F., Zeng, Z., Zhang, H., Li, F., Yang, J., Li, H., Jiang, Q., Zhang, L.: Grounded sam: Assembling open-world models for diverse visual tasks (2024)
  * [47] Rozière, B., Gehring, J., Gloeckle, F., et al.: Code Llama: Open Foundation Models for Code (2023)
  * [48] Sadigh, D., Dragan, A.D., Sastry, S., Seshia, S.A.: Active preference-based learning of reward functions (2017)
  * [49] Silver, T., Athalye, A., Tenenbaum, J.B., Lozano-Perez, T., Kaelbling, L.P.: Learning neuro-symbolic skills for bilevel planning. arXiv preprint arXiv:2206.10680 (2022)
  * [50] Singh, I., Blukis, V., Mousavian, A., Goyal, A., Xu, D., Tremblay, J., Fox, D., Thomason, J., Garg, A.: Progprompt: Generating situated robot task plans using large language models. In: 2023 IEEE International Conference on Robotics and Automation (ICRA). pp. 11523–11530. IEEE (2023)
  * [51] Solar-Lezama, A.: Program synthesis by sketching. Citeseer (2008)
  * [52] Srivastava, S., Labutov, I., Mitchell, T.: Joint concept learning and semantic parsing from natural language explanations. In: Proceedings of the 2017 conference on empirical methods in natural language processing. pp. 1527–1536 (2017)
  * [53] Surís, D., Menon, S., Vondrick, C.: Vipergpt: Visual inference via python execution for reasoning. arXiv preprint arXiv:2303.08128 (2023)
  * [54] Wake, N., Kanehira, A., Sasabuchi, K., Takamatsu, J., Ikeuchi, K.: Gpt-4v (ision) for robotics: Multimodal task planning from human demonstration. arXiv preprint arXiv:2311.12015 (2023)
  * [55] Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., Anandkumar, A.: Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291 (2023)
  * [56] Wang, R., Mao, J., Hsu, J., Zhao, H., Wu, J., Gao, Y.: Programmatically grounded, compositionally generalizable robotic manipulation. arXiv preprint arXiv:2304.13826 (2023)
  * [57] Wang, Y., Shah, R., Criswell, A., Pan, R., Dillig, I.: Data migration using datalog program synthesis. VLDB (2020)
  * [58] Wang, Y., Wang, X., Dillig, I.: Relational program synthesis. PACMPL 2(OOPSLA), 155:1–155:27 (2018)
  * [59] Wilson, A., Fern, A., Tadepalli, P.: A bayesian approach for policy learning from trajectory preference queries. Advances in neural information processing systems 25 (2012)
  * [60] Wirth, C., Fürnkranz, J., Neumann, G.: Model-free preference-based reinforcement learning. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 30 (2016)
  * [61] Xie, A., Finn, C.: Lifelong robotic reinforcement learning by retaining experiences. In: Conference on Lifelong Learning Agents. pp. 838–855. PMLR (2022)
  * [62] Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J.M., Luo, P.: Segformer: Simple and efficient design for semantic segmentation with transformers. Advances in Neural Information Processing Systems 34, 12077–12090 (2021)
  * [63] Yi, K., Wu, J., Gan, C., Torralba, A., Kohli, P., Tenenbaum, J.: Neural-symbolic vqa: Disentangling reasoning from vision and language understanding. Advances in neural information processing systems 31 (2018)
  * [64] Zhang, R., Bansal, D., Hao, Y., Hiranaka, A., Gao, J., Wang, C., Martín-Martín, R., Fei-Fei, L., Wu, J.: A dual representation framework for robot learning with human guidance. In: Conference on Robot Learning. pp. 738–750. PMLR (2023)

## Appendix

##  Appendix 0.A Implementation Details

####  0.A.0.1 Inputs.

For learning the program, we collected 29 demonstrations where each
demonstration had a trajectory of robot poses, the associated RGB camera
image, and pointcloud data from Ouster-128 LiDAR sensor. A high-resolution
LiDAR was needed to achieve as accurate a mapping as possible from 2D to 3D. A
few of the natural language prompts from the user are listed below:

1. It is safe since it is on a sidewalk, and is far from any person and the pole, and it is not on grass. 2. This looks good since it is on a sidewalk, and is far from the approaching person and the pole. 3. This is not safe since it is not far from bushes, even though its on a sidewalk.

####  0.A.0.2 Models.

The details are as follows:

  1. a.

Grounded-SAM: Using a set of hyperparameters for all objects does not always
yield good results due to the limitations of current VLMs, _i.e_., their
accuracy and sensitivity differs a lot between different classes of objects.
We use Grounded-SAM [46] with the object-specific hyperparameters as shown in
Table 4 to perform zero-shot object detection and segmentation on images.

  2. b.

Terrain segmentation: Our experiments showed that present VLMs do not do so
well on terrain segmentation, which was a domain-specific essential capability
to be able to represent the preferential concept well enough. Thus, we
finetuned the SegFormer-b5 [62] model, with pretrained weights from the
HuggingFace transformers library, on our custom UT campus dataset. The details
about the models can be found on the project website.

  3. c.

We use GPT-4 [1] as our language model for the sketch synthesis module. We set
a temperature of 0.0 and seed it for enhancing reproducibility. Prompts for
doing different tasks in our framework (_i.e_., grounding, synthesis _etc_.)
can be found in the codebase. Note that they have certain placeholders like
$<!\ldots!>$ and $<dyn!\ldots!dyn>$ which refer to other prompt instances or
are replaced dynamically in the code based on generated outputs.

Table 4: Object-specific hyperparameters for VLM | box-threshold | text-threshold | nms-threshold
---|---|---|---
barricade | 0.5 | 0.5 | 0.3
board | 0.3 | 0.3 | 0.5
bush | 0.4 | 0.4 | 0.4
car | 0.3 | 0.3 | 0.3
entrance | 0.3 | 0.3 | 0.2
person | 0.25 | 0.25 | 0.6
pole | 0.4 | 0.4 | 0.5
staircase | 0.25 | 0.25 | 0.4
tree | 0.4 | 0.4 | 0.45
wall | 0.5 | 0.5 | 0.4
Figure 6: Concept library seeded with some basic predicates

####  0.A.0.3 Concept Library Initialization.

We impart some basic neural and domain-specific capabilities to the learning
framework by seeding it with the functions shown in Fig. 6.

Table 5: Results for CONTINGENCY concept | iou_pos (%) | iou_neg (%) | miou (%)
---|---|---|---
| train | in-test | out-test | train | in-test | out-test | train | in-test | out-test
Synapse | 57.22 | 54.46 | 50.18 | 98.06 | 98.11 | 97.96 | 77.64 | 76.29 | 74.07
SF-RGB-b0 | 43.54 | 28.46 | 17.77 | 97.44 | 97.03 | 97.07 | 70.49 | 62.75 | 57.42
SF-RGB-b5 | 51.63 | 43.87 | 19.04 | 97.54 | 97.08 | 92.96 | 74.59 | 70.48 | 56.00
SF-RGBD-b0 | 46.71 | 37.07 | 13.49 | 97.62 | 97.39 | 96.00 | 72.17 | 67.23 | 54.75
SF-RGBD-b5 | 55.10 | 38.48 | 15.71 | 97.85 | 97.13 | 96.50 | 76.48 | 67.81 | 56.11
GPT4V | 01.73 | 01.91 | 02.74 | 51.38 | 57.51 | 58.15 | 26.56 | 29.71 | 30.45
GPT4V+ | 02.29 | 02.18 | 02.59 | 55.16 | 55.74 | 65.24 | 28.73 | 28.96 | 33.92
VisProg | 04.99 | 01.25 | 05.04 | 86.24 | 90.01 | 84.92 | 45.62 | 45.63 | 44.98
VisProg+ | 08.13 | 06.36 | 07.15 | 69.75 | 72.06 | 76.51 | 38.94 | 39.21 | 41.83
Table 6: Results for DROPOFF concept | iou_pos (%) | iou_neg (%) | miou (%)
---|---|---|---
| train | in-test | out-test | train | in-test | out-test | train | in-test | out-test
Synapse | 60.64 | 62.05 | 63.13 | 97.99 | 98.31 | 98.31 | 79.32 | 80.18 | 80.72
SF-RGB-b0 | 48.59 | 38.93 | 08.12 | 97.39 | 97.32 | 96.30 | 72.99 | 68.13 | 52.21
SF-RGB-b5 | 56.73 | 48.00 | 15.94 | 97.78 | 97.66 | 94.13 | 77.26 | 72.83 | 55.04
SF-RGBD-b0 | 51.09 | 42.17 | 13.89 | 97.57 | 97.43 | 95.90 | 74.33 | 69.80 | 54.90
SF-RGBD-b5 | 57.43 | 43.86 | 08.93 | 97.95 | 97.54 | 95.84 | 77.69 | 70.70 | 52.39
GPT4V | 02.40 | 02.25 | 03.09 | 58.67 | 63.69 | 72.20 | 30.54 | 32.97 | 37.65
GPT4V+ | 04.43 | 01.90 | 03.02 | 74.33 | 74.77 | 75.25 | 39.38 | 38.34 | 39.14
VisProg | 01.48 | 01.94 | 06.58 | 91.10 | 93.03 | 89.56 | 46.29 | 47.49 | 48.07
VisProg+ | 08.62 | 06.86 | 08.70 | 69.71 | 72.01 | 77.57 | 39.17 | 39.44 | 43.14
Table 7: Results for PARKING concept | iou_pos (%) | iou_neg (%) | miou (%)
---|---|---|---
| train | out-test | train | out-test | train | out-test
Synapse | 39.14 | 27.24 | 98.06 | 98.27 | 68.60 | 62.76
SF-RGB-b0 | 21.28 | 04.91 | 94.07 | 94.40 | 57.68 | 49.66
SF-RGB-b5 | 38.99 | 08.13 | 98.27 | 97.68 | 68.63 | 52.91
SF-RGBD-b0 | 24.80 | 04.67 | 97.00 | 96.70 | 60.90 | 50.69
SF-RGBD-b5 | 43.97 | 03.50 | 98.14 | 96.48 | 71.06 | 49.99
GPT4V | 01.10 | 01.11 | 75.72 | 79.25 | 38.41 | 40.18
GPT4V+ | 01.56 | 00.64 | 81.19 | 78.90 | 41.38 | 39.77
VisProg | 04.72 | 01.69 | 74.13 | 80.29 | 39.43 | 40.99
VisProg+ | 03.31 | 03.08 | 74.44 | 74.89 | 38.88 | 38.99

##  Appendix 0.B Detailed Results & Explanations

###  0.B.1 Baselines

We curate a custom dataset on UT campus region, consisting of 815 pixel-level
labeled images (375 each for CONTINGENCY and DROPOFF, and 65 for PARKING). We
evaluate the following baselines:

  1. enumi

SegFormer b0 and b5 models, both with or without depth input. For taking in
depth, we only modify the input layer, and still retain all other pretrained
weights. We take measures such as early stopping to prevent overfitting. The
hyperparameters used to finetune each of these models can be found on their
model cards in HuggingFace (links to which can be found on the project
website).

  2. enumi

GPT4-vision: Due to token limitations, we query GPT4-vision to output a
$20\times 20$ output class array given the image. For reporting the IOU for
GPT4-vision, we downsample the ground-truth to the same size for the
comparison to be fair. The ‘+’ variant essentially means that we prompt it
with additional information about the user’s ground-truth, _i.e_., we provide
it the program that Synapse has learned, in natural language. All prompts can
be found in the codebase.

  3. enumi

VisProg and VisProg+ come from a related approach to VQA [22]. We again follow
the same methodology of prompting as for GPT4-vision. Here too, no finetuning
is done and the evaluation is zero-shot. All prompts can be found in the
codebase.

Figure 7: Illustration of reasoning about failures using the learned program

Tables 5, 6 and 7 show the full evaluation for the three concepts, with the
IOU for each class. Due to unavoidable data imbalance on the pixel-level
(_i.e_., $\approx$ 90% negative pixels against 10% positive class pixels), it
is much easier to learn the negative class.

###  0.B.2 Ablations

We test the following ablations for our framework:

  1. enumi

Framework ablations: We test three alternative ways to generate the program
sketch from given natural language input from the user:

    1. a.

Synapse-SynthDirect: given only the basic predicates, we adopt a one-step
approach to synthesis, where we ask LLM to update the program sketch based on
the new NL input and previous program sketch, while utilising only the basic
predicates, _i.e_., it has no way to hierarchically build and retain higher-
level predicates, as well as it does not do any CNF extraction.

    2. b.

Synapse-SynthDirect+: we provide the higher-level concepts learned by our main
framework and then given these already learned higher-level predicates, we
again adopt a one-step approach to synthesis, where we ask LLM to update the
program sketch based on the new NL input and previous program sketch, _i.e_.,
it still does not do any CNF extraction. Note, however, this is _only_
possible for a post-learning ablation study, as in an actual learning
framework, we do not have apriori access to the higher-level learned
predicates.

    3. c.

Synapse-SynthCaP: we disallow the framework from querying the user for
auxiliary demonstrations to learn auxiliary concepts, _i.e_., the framework is
forced to generate code (using the concept library) for the auxiliary concepts
solely based on the information available, which essentially is the _name_ of
that particular concept. This is similar to the recursion performed in Code-
as-Policies [36].

  2. enumi

NN-ablations: We finetune the four SegFormer models on the same number of
samples as Synapse, which is 29 for the CONTINGENCY concept.

  3. enumi

LLM-ablations: We test the performance of our overall framework using other
language models available: (1) CodeLLama [47], (2) StarCoder [34], and (3)
PaLM2 [6]. We see that it is essential for the LLM to be strong enough for the
performance of Synapse to be good.

The results reaffirm the design choices made in Synapse.

###  0.B.3 User-study

The user case study involved each of the three participants using Synapse to
have it learn their notion about the preferential concept under consideration.
We chose CONTINGENCY for the study. Each of the three participants gave 10-15
demonstrations after which each of them labeled 15 images with their _ground-
truth_.

It was observed that the learned programs differed on two levels: (1) symbolic
concepts, and (2) quantitative parameters. While many of the symbolic concepts
were similar across users, there were a few such as is_too_far or is_next_to
that differed. We then evaluated each learned program on the ground-truth
labels of all the users, which showed good one-one correspondence between
learned programs and ground-truth of users, indicating good alignment.

##  Appendix 0.C Error Analysis using Synapse

An added benefit of the neuro-symbolic representation of the preferential
concept with Synapse is that it facilitates reasoning about failures. Fig. 7
shows one such example where we can easily reason about the false positives
and false negatives comparing the prediction with the individual component
outputs of the program.


# A revised comparison of distant and nearby solar twins

Charles R. Cowley  Department of Astronomy, University of Michigan, 1085 S.
University, Ann Arbor, MI 481090-1107
orcid:0000-0001-9837-3662  Robert E. Stencel  Chamberlin Observatory,
University of Denver, 2930 E Warren Ave., Denver, CO 80210, USA,
orcid:0000-0001-8217-9435
e-mail<EMAIL_ADDRESS>
###### Abstract

Properties of solar twins (Lehmann et al., 2023) at kiloparsec distances from
the local standard of rest (LSR) are compared to solar twins within 100 pc of
the Sun. They have velocity distributions closely similar to those of the
nearby twins, in addition to closely matching $T_{\rm eff}$, $\log{(g)}$ and
$[Fe/H]$. The new twins are at slightly higher galactic latitudes, and are
somewhat closer to the Galactic center. Additionally, they may be
significantly older than nearby solar twins.

stars: abundances – stars: solar-type –Galaxy: abundances – Galaxy: solar
neighborhood

##  1 Distant solar twins

Solar twins have been discussed extensively in the literature (see Bedell, et
al., 2018, henceforth, BD18). The BD18 stars are all within 100 pc of the Sun.
In this note we discuss potential distant twins or analogues as investigated
by Lehmann et al. (2022, 2023) and Liu et al. (2022). The authors designate
their work SDST (Survey for Distant Solar Twins). Twins and analogues are
defined in their work by limits on effective temperature, surface gravity, and
[Fe/H], 111We use the notation $[Y/X]$ = $\log(Y/X)_{\rm star}-\log(Y/X)_{\rm
Sun}$ with kiloparsec distances–far greater than for twins previously
investigated.

A goal of SDST was to provide observations that might detect variations in the
fine structure constant over kiloparsec distances. Murphy et al. (2022) give
an introduction to the problem of fine structure variation and the proposed
investigation method using sunlike spectra.

SDST explored one sharply limited direction of the Galaxy, an area of only
3.02 square degrees, in the direction of Galactic longitude $330.0$ deg,
latitude 16.55 deg. The survey yielded 877 candidate stars with Gaia DR3
distances $483\leq d\leq 5425$ (mean=1933) pc. Twenty of the 877 stars were
designated as twins and 299 as solar analogues.

##  2 Sightline

Figure 1 is a plot of $Z$-values of the 877 SDST stars vs. $\rho$ =
$\sqrt{(}X^{2}+Y^{2})$, where the origin of the coordinates is located at the
mean sun position. The line of sight is out of the plane, the most distant of
the 877 stars is Z = 1495 pc. The $\rho$ values of the 20 solar twins are
indicated by the blue arrows. This sightline encroaches on the so-called thick
disk. The idea of a thick galactic disk was proposed by Abadi et al. (2003) –
see also Prantzos et al.  (2023) – to describe ‘accretion debris’ from
galactic mergers. A scale height of 1.5 kpc would be seen at a distance of 4.5
kpc, for objects observed at galactic latitude 16 degrees. The Lehmann sample
includes a few objects between 4.5 and 5.5 kpc distance and thus, could
include thick disk components with potentially altered composition.

Figure 1:  Z-values vs. $\sqrt{(}X^{2}+Y^{2})$ for 877 SDST stars (black
filled circles). The red clump in the lower left corner are the 79 BD18 stars,
mostly with velocities less than 63 km/sec. The orange line through the SDST
points shows their allignment which departs very slightly for the most distant
stars. See text for further discussion.

The vertical scatter in Figure 1 is partially due to small differences in the
line of sight to the SDST stars, and partially due to errors in the Gaia
parallaxes ($\varpi$ and $\Delta\varpi$). It is beyond the scope of this paper
to explore the latter in detail (see Luri et al., 2018). A naive
interpretation of the distance errors, however, between 1/($\varpi$ \+
$\Delta\varpi$) and 1/($\varpi$ \- $\Delta\varpi$) would lead to far greater
scatter than is seen in Figure 1.

##  3 Kinematics: velocities

We have computed the velocities of the 20 twins and all 877 of the SDST stars,
and compared them with the kinematical properties of sets of solar twins
discussed by Cowley & Yüce (2022, henceforth, CY22) notably, 79 stars with
precision differential abundances from BD18, which we consider definitive
twins for the local solar neighborhood. Because the SDST twins are in a
different region of the Galaxy, and at significantly greater distances than
those of the BD18 sample, we focus on the velocities relative to the LSR; the
distribution of stars in physical space is distinct by definition (see Figure
2).

Figure 2: Velocities with respect to the LSR of solar twins, near (BD18, red)
and distant (SDST, black) from the solar neighborhood, showing little
difference in the distribution of the two sets of stars. The circle has a
radius of 63 km sec-1. It is distorted because of the different $U$ and $V$
scales.

We also consider the velocities in the Z-direction with the help of Figure 3.

Figure 3: Velocities with respect to the LSR in the Z and XY directions. The
similarity of the BD18 (red) and SDST (black) distributions is clear.

Finally, it is of interest to examine the distributions of velocity in the
phase space of kinetic energy (per unit mass) vs. $L_{Z}$, the Z-component of
the angular momentum (per unit mass). The solid line is part of a parabola
that describes the locus of circular orbits near the Sun. Both the BD18 and
SDST points adhere closely to that line.

Figure 4: BD18 and SDST velocities in phase space.

We conclude that the velocity kinematics of the 20 SDST distant twins are
closely similar to BD18 objects.

##  4 ages

Both the BD18 and SDST ages were computed using the q2 algorithm of Ramírez et
al. (2014). The mean age differences are of interest. For 12 twins from the
BD18 stars, CY22 found a mean age of 4.86 Gyr (see their Table 2). In CY22,
four other sources of 12 twins were discussed, with mean ages computed for
this paper. These ages are: 4.01 (Delgado Mena et al., 2019), 4.42 (Liu et
al., 2022) 4.78 (Brewer et al., 2016), and 3.96 Gyr (Nissen, 2015). The
average of these five means is 4.41 Gyr. We take 1.5 Gyr as a working,
conservative uncertainty of an individual twin’s age–roughly a standard
deviation (see Delgado Mena et al. (2019) and CY22). Dividing these N - 1
values by $\sqrt{(}11)=3.32$, these give an uncertainty of 0.45 Gyr for the
mean. Altogether, we may say that solar twins in the solar neighborhood have a
mean age of $4.43\pm 0.45$ Gyr.

The mean distance of the 20 distant twins from the Galactic center is $6.72\pm
0.65$ kpc. They have a mean age of 8.35 Gyr and if we again use 1.5 Gyr as a
typical uncertainty, the uncertainty of this mean is $1.5\sqrt{(}19)=0.34$
Gyr. So the distant twins have a mean age of $8.35\pm 0.34$ Gyr. The mean age
of the near twins plus 2 sigma is 5.25 Gyr while the mean age of the distant
twins minus 2 sigma is 7.69, leaving a substantial gap.

Judging by conservative estimates for the random errors, the distant twins are
significantly ($>2\sigma$) older than the nearby ones. However, the typically
larger systematic errors could change that conclusion.

##  5 Summary

The distant solar twins of the SDST studies have very similar velocity
distributions to those of the classic BD18 stars in the solar neighborhood,
supporting their validity as solar twins, and supporting their use in the
investigation of possible variations in the fine structure constant. An age
difference in the near and distant twins is of interest because of a possible
connection with galactic chemical evolution, and/or violation of the Vogt-
Russell theorem (Russell , 1931).

## References

  * Abadi et al. (2003) Abadi, M., Navarro, J., Steinmetz, M., Eke, V. 2003 ApJ 591, 499. doi:10.1086/375512
  * Bedell, et al. (2018) Bedell M., Bean J. L., Meléndez J., et al., 2018 ApJ 865, 68. doi:10.3847/1538-4357/aad908(BD18)
  * Brewer et al. (2016) Brewer, J. M., Fischer, D. A., Valenti, J. A., et al. 2016 ApJS225 32. doi:10.3847/0067-0049/225/2/32 (BR)
  * Cowley & Yüce (2022) Cowley C. R., Yüce K., 2022 MNRAS 512, 3684. doi:10.1093/mnras/stac637 (CY22)
  * Delgado Mena et al. (2019) Delgado Mena E., Moya A., Adibekyan V., et al., 2019, A&A, 624, A78. doi:10.1051/0004-6361/201834783
  * Lehmann et al. (2022) Lehmann, C., Murphy, M. T., Liu, F., et al. 2022 MNRAS512, 11. doi:10.1093/mnras/stac421 (SDST I)
  * Lehmann et al. (2023) Lehmann, C., Murphy, M. T., Liu, F., et al. 2023 MNRAS521, 148. doi:10.1093/mnras/stad381 (SDST III)
  * Liu et al. (2022) Liu, F., Murphy, M. T., Lehmann, C., et al. 2022 MNRAS517, 5569. doi:10.1093/mnras/stac3033
  * Luri et al. (2018) Luri, X., Brown, A. G. A., Sarro, L. M., et al. 2018 A&A616, A9. doi:10.1051/0004-6361/201832964
  * Murphy et al. (2022) Murphy, M. T., Berke, D. A., Liu, F., et al. 2022 Science 378, 634. doi:10.1126/science.abi9232
  * Nissen (2015) Nissen, P. E. 2015 A&A579, A52. doi:10.1051/0004- 6361/201526269
  * Prantzos et al. (2023) Prantzos, N., Abia, C., Chen, T. et al. 2023 https://arxiv.org/pdf/2305.13431.pdf
  * Ramírez et al. (2014) Ramírez, I., Meléndez, J., Bean, J., et al. 2014 A&A572, A48. doi:10.1051/0004-6361/201424244
  * Russell (1931) Russell, H.N. 1931 MNRAS 91, 951, doi.org/10.1093/mnras/91.9.951.
  * Spina et al. (2018) Spina L., Meléndez J., Karakas A. I., et al., 2018 MNRAS 474, 2580\. doi:10.1093/mnras/stx2938