11institutetext:  University of Wisconsin-Madison1 22institutetext: Carnegie
Mellon University2

(eccv) Package eccv Warning: Running heads incorrectly suppressed - ECCV
requires running heads. Please load document class ‘llncs’ with ‘runningheads’
option (eccv) Package eccv Warning: Package ‘hyperref’ is loaded with option
‘pagebackref’, which is *not* recommended for camera-ready version

# Audio-Synchronized Visual Animation

Lin Zhang1 Shentong Mo2 Yijing Zhang1 Pedro Morgado1

###### Abstract

Current visual generation methods can produce high-quality videos guided by
texts. However, effectively controlling object dynamics remains a challenge.
This work explores audio as a cue to generate temporally-synchronized image
animations. We introduce Audio-Synch-ronized Visual Animation (ASVA), a task
animating a static image to demonstrate motion dynamics, temporally guided by
audio clips across multiple classes. To this end, we present AVSync15, a
dataset curated from VGGSound with videos featuring synchronized audio-visual
events across 15 categories. We also present a diffusion model, AVSyncD,
capable of generating dynamic animations guided by audios. Extensive
evaluations validate AVSync15 as a reliable benchmark for synchronized
generation and demonstrate our model’s superior performance. We further
explore AVSyncD’s potential in a variety of audio-synchronized generation
tasks, from generating full videos without a base image to controlling object
motions with various sounds. We hope our established benchmark can open new
avenues for controllable visual generation. More videos on project webpage
https://lzhangbj.github.io/projects/asva/asva.html.

###### Keywords:

Audio Video Synchronization Generation

##  1 Introduction

Generative modeling has witnessed remarkable progress recently, largely due to
the emergence of robust architectures like diffusion models [17, 36, 32].
Conditional generation and, in particular, text-to-image generation [31, 32],
has been the focal point given its application potential and availability of
high-quality training data [34, 2]. This success has also led to a revived
interest in video generation, such as text-to-video [43, 20, 35, 5]. While
text conditioning has been thoroughly investigated, the unique advantages of
audio-visual synchrony for video generation remain underexplored. Unlike text,
which provides control over global semantics, audio offers both semantic
control on video content and precise control at each moment of the generated
video.

Most existing works on audio-to-visual generation are however either limited
to semantic control [21, 19, 37, 38], or constrained on singular scenarios
such as human talking face [9, 49, 28, 47, 50, 26, 10, 11]. The former focuses
on ambient audio datasets lacking synchronization cues with environmental
sounds (ranining, fire crakcling, wind) [21] or music [39], where shifting the
audio temporally does not lead to major changes in visual content. The input
audio in such cases thus substitutes text to provide only global semantics.
The latter, talking faces, while synchronized, are extremely limited in
generation diversity and control.

Figure 1: Given an audio and an image (green box), we produce animations
beyond image stylization with complex but natural dynamics, synchronized with
audio at each frame. Results are produced by our AVSyncD model trained on the
proposed AVSync15.

To bridge this gap, we introduce Audio-Synchronized Visual Animation, ASVA, a
task which aims to animate objects depicted in natural static images into a
video, with clear _motion dynamics_ that are _semantically aligned_ and
_temporally synchronized_ with the input audio. ASVA requires a sophisticated
understanding of the audio’s temporal structure, as well as of how objects
move in synchrony with sound. Prior attempts on synchronized generation with
diverse audios fall short in generated visual quality [22] and accurate
synchronization control [46, 19] due to two challenges: (1) the lack of high-
quality training data and benchmark for learning audio-synchronized visual
dynamics; (2) the development of effective methods capable of generating
highly synchronized video motions. Successfully addressing these challenges
will expand the scope of current video generation methods to enable more
finegrained semantic and temporal control over the generation process via
synchronized audio conditioning.

We address the first challenge by constructing a high-quality diverse dataset
with strong correlations between audio and object motions at each moment in
time. In an ideal dataset, sound sources should be easily identifiable in the
scene. Every visual motion in the video should highly correlate with the audio
semantically and temporally, and vice versa. Moreover, the visual content
should be of high quality for generation. However, existing audio-visual
datasets either are too noisy [13, 7], containing a large number of
unassociated audio-visual pairs [29], or predominantly feature ambient sound
categories that lack meaningful synchronized object dynamics [21, 39]. We thus
curate a high-quality dataset from VGGSound [7] by deploying an efficient two-
step curation pipeline. In the first step, we use a variety of signal
processing techniques and foundation models to automatically filter out videos
with poor semantic alignment or temporal synchronization, as well as those
depicting static scenes or with too fast camera motions. Afterwards, to ensure
the highest possible quality as a benchmark, we narrow down the dataset to
sound categories with strong audio-visual synchonization cues, and manually
verify the quality of each video. We end up obtaining AVSync15 with 15 dynamic
sound classes, each with 100 examples rich and accurate in semantics, object
dynamics, and audio-visual synchronization. An overview of AVSync15 classes is
in Figs. 2(a) and 2(b) and its comparison with prior audio-video generation
datasets is in supplementary.

The second challenge pertains to the generation of audio-synchronized motions,
which requires a detailed understanding of audio-visual correlations and
object dynamics. Take, for instance, a video featuring a dog. The model is
expected to not only synchronize the dog’s mouth motion with the barking
sound, but also accurately depicting subtleties such as the dog’s head pose
before and after barking, to generate realistic video. In a more challenging
scenario, the model should discern which object to animate to preserve
semantic consistency depending on the input sound. For example, when a violin
also appears in the video and input is violin playing sound, the dog should
not bark. However, existing audio-conditioned visual generation frameworks
[14, 37] primarily focus on semantic control, often encoding the audio into a
single global semantic feature and thereby neglecting the audio’s temporal
domain. Even recent attempts at audio-synchronized video generation [22, 19,
46] have not fully released the potential of audio for fine-grained temporal
control, as they either rely on crude audio representations such as audio
amplitude [22], or learn from weakly synchronized [21] or noisy datasets [7],
or ignore learning motion dynamics in the generation process [22, 21, 19, 46].
To this end, we introduce Audio-Video Synchronized Diffusion (AVSyncD), a
framework improving a pretrained image latent diffusion model [32] for
enhanced audio guidance and motion generation. We employ the pretrained
ImageBind [14] model to encode audio into time-aware semantic tokens, which
are then fused into image latent features at each frame. This allows for
precise audio guidance on video semantics and synchronization. To capture
complex video motions, we add temporal attention layers to the diffusion
model. Finally, to ensure faithful animation of the input image, we
incorporate temporal convolutions and attention layers that always reference
the input image, i.e., first-frame lookups.

With the carefully designed dataset and architecture, we are able to obtain a
well-trained model specialized for ASVA, and produce animations with more
visually pleasing and audio-synchronized motions than pior works (Figs. 1 and
4). We provide thorough experiments to validate the effectiveness of the
proposed AVSync15 and AVSyncD. We also demonstrate how to deploy AVSyncD for
controllable generation, including amplifying audio guidance and generating
object dynamics with various audios (Secs. 5.3 and 5.4).

##  2 Related Work

###  2.1 Controllable Visual Generation

Many conditional visual generation models based on diffusion process [17, 36]
have emerged recently. Benefiting from more efficient architecture, large-
scale training data [34], and aligned semantic space [30], Latent Diffusion
Model (LDM) [32] has achieved great success to generate realistic images
conditioned on text. This inspired researchers to explore various diffusion-
based visual generation tasks, such as text-to-video [20, 3, 43, 44, 5, 42],
audio-to-image [14, 37], and audio-to-video [38, 19, 46, 22]. The
architectures can be training-free [20, 43, 22], fully-trained [5, 42], or
trained partially, which augments a pretrained LDM by carefully adding some
trainable layers [3, 19, 44]. Extensive works have also attempted to control
the semantics of the generated content [25, 15, 48, 24], while how to apply
control in the temporal dimension remains under-explored.

In this work, we developed an image animation model AVSyncD to control
generation semantically and temporally guided by audio. AVSyncD augments pre-
trained StableDiffusion [32] with trainable temporal layers and audio
conditioning mechanism, preserving training efficiency and generalizing well.

###  2.2 Audio-to-Video Generation

Traditionally, audio has been used as a temporal cue for talking face
generation [49, 28, 47, 50, 26], where face and lip actions should be
synchronized with audio at each frame. Many works also rely on complex inputs
such as 3D meshes and human poses [47, 26]. Although accurately synchronized,
this line of research is extremely limited in scenarios and cannot generate
videos for diverse audios.

A series of work attempted to expand the class of audio by encoding sound into
a global semantic condition for video generation [37, 14, 38], however often
overlooked the temporal aspect inherent in audio. Some recent works, although
divided audio features into time-aware segments as inputs [21, 33], failed to
achieve promising visual quality or synchronize video motions with audios.
AADiff [22] is a training-free method re-weighting the text-image cross-
attention map in LDM using audio amplitude at each frame, however can only
control styles of each frame. TPoS [19] learns segmented audio features
aligned with CLIP [30] using sophisticated modules and training losses, and
feeds them into a pretrained text-to-image model [32] for video generation.
TempoToken [46] also learns segmented audio features with a pretrained audio
encoder BEATs [8], and fuses them into a pretrained text-to-video model [42].
However, primarily focused on monotonous sound classes in Landscapes [21] or
noisy audio-visual data in VGGSound [7], these methods are limited to
generating video semantics without capturing the natural and synchronized
dynamics of video content. Frozen generation architectures also prevents them
from generating natural motions.

Addressing these limitations, we introduce AVSync15, a high-quality dataset
specifically designed for ASVA. AVSync15 stands out from previous efforts by
focusing on synchronization cues between audio and visual dynamics, allowing
for generating motions beyond mere visual effects. Our proposed model,
AVSyncD, uses frozen segmented audio features from ImageBind [14], and
improves the architecture of pretrained text-to-image model [32] for video
generation. Trained on AVSync15, our AVSyncD can generalize to many
applications to control video motions guided by audios, on which previous
methods performed poorly.

##  3 Audio-Synchronized Visual Animation

Formally, the Audio-Synchronized Visual Animation (ASVA) task can be posed as
follows. Given an audio clip ${\bm{a}}$ of length $T$ seconds and an image
${\bm{x}}_{1}$, the goal is to generate the future video sequence
${\bm{x}}_{2},\ldots,{\bm{x}}_{rT}$ (or ${\bm{x}}_{2:rT}$ for short), where
$r$ is the desired frame rate. Despite the simple formulation, this is a
challenging task as the generated video sequence should (1) be of high visual
quality, (2) be semantically aligned with the image ${\bm{x}}_{1}$ and audio
${\bm{a}}$, (3) exhibit temporal coherence and (4) natural object motions
temporally synchronized with the audio ${\bm{a}}$. To facilitate research in
ASVA, we introduce a new benchmark that includes a curated high-quality
dataset and a suite of evaluation metrics designed to capture the various
components of audio-synchronized generation.

###  3.1 AVSync15: A High-Quality Dataset for Audio Synchronized Video
Generation

Existing large-scale audio-visual datasets like VGGSound [7, 6] and AudioSet
[13] often contain amateur videos from platforms like YouTube. These videos,
while diverse, can pose challenges for audio-synchronized video generation
tasks due to rapid scene changes, camera motion, noisy audio, or out-of-frame
sound sources. Prior work [23, 21] has addressed this by focusing on simpler
videos, such as those depicting fire crackling or weather patterns. However,
such videos often lack strong synchronization cues between audio and visual
motion, making them unsuitable for ASVA. To facilitate research in this area,
we introduce a high-quality dataset specifically designed for audio-
synchronized video generation, ensuring a close synchronization between audio
and visuals. More specifically, our selection criteria to create the dataset
were: (1) High Correlation: Significant visual changes should be closely
associated with audio at each timestamp, and vice versa. (2) Dynamic Content:
We included content rich in temporal changes, excluding ambient or monotonous
classes. (3) Quality and Relevance: Both video and audio needed to be clean,
stable, and semantically aligned.

(a)

(b)

Figure 2: (a): Overview of 15 categories in AVSync15. Categories are listed
below x-axis on the right plot. (b): Categoriy-wise averages of av-sync score
$\phi$, IA, and IT on AVSync15 and equivalently sized subsets of VGGSS and
AVSync-AC. Error bars for VGGSS and AVSync-AC are obtained from 3 random
splits.

#### Preliminary Curation

We construct our dataset from VGGSound [7], a large-scale dataset with 309
diverse audio classes. Similar to VGGSoundSync [6], we start by narrowing to
149 classes with potential clear audio-visual synchronized events, removing
ambient classes. We refer to this intermediate dataset as VGGSS. From VGGSS,
we deploy a sequence of automatic cleaning steps and a final manual selection
stage to identify appropriate videos. We summarize procedures below and
provide details in supplementary.

#### Automatic Curation

We first use PySceneDetect [1] to cut videos with sharp scene changes into
different scenes, which are still likely to contain low-quality short clips.
To maximize usage, we split each scene into 3-second clips with 0.5-second
strides, and discard unsuitable clips based on the following metrics:

  * Raw Pixel Difference: We calculate average pixel distances between consecutive frames and remove clips with both small and large values, likely depicting frame either static or with excessive motion.

  * Image Semantics Difference: Complementing the raw pixel analysis above, we also compute distances on CLIP [30] image features, removing videos with small semantics changes such as zoom in/out or large semantic content transitions.

  * Audio Waveform Amplitude: We exclude clips whose maximum audio waveform amplitude is low, indicating weak audio cues.

  * Semantic Alignment: We compute the average image-audio (IA) and image-text (IT) alignment scores [30] in a video using ImageBind [14], removing clips with low scores to ensure cross-modal semantic alignment.

  * Audio-Video Synchronization: To measure audio-visual synchronization, we follow VGGSoundSync [6] to contrastively train an audio-visual synchronization classifier on VGGSS, ending up with a comparable $40.85\%$ test accuracy. The model outputs an unbounded av-sync score $\phi_{{\bm{a}}_{i},{\bm{v}}_{j}}$ for an audio-video pair $({\bm{a}}_{i},{\bm{v}}_{j})$. During training, we compute $\phi$ for a synchronized pair $({\bm{a}}_{i},{\bm{v}}_{i})$ and its temporally-shifted pairs from the same instance. Contrastive loss is then applied on these shifted pairs to maximize the synchronization probability:

$P_{\text{Sync}}({\bm{a}}_{i},{\bm{v}}_{i})=\frac{\exp(\phi_{{\bm{a}}_{i},{\bm{v}}_{i}}/\tau)}{\sum_{j}\exp(\phi_{{\bm{a}}_{i},{\bm{v}}_{j}}/\tau)}$
(1)

to distinguish the synchronized pair from shifted ones. We use
$P_{\text{Sync}}$ as a synchronization indicator to remove low-scoring clips.
When computing $P_{\text{sync}}$, we discard the temperature parameter $\tau$
used to improve training efficacy. We detail the synchronization classifier
and $P_{\text{sync}}$ in supplementary.

We empirically determine the thresholds of each metric by prioritizing
quality, acknowledging that some acceptable clips might be discarded. After
automatic curation, we merge all 3-second clips from each video, and remove
categories with less than 100 examples to address category imbalance,
resulting in a dataset with 76 categories and 39,902 examples. We refer to
this dataset as AVSync-AC (Audio-Video Sync with Automatic Curation).

#### Manual Curation

We further select 15 diverse categories with clear audio-visual motion cues
from AVSync-AC for manual refinement. The categories range from animals and
human actions to triggered tools and musical instruments. Manual curation once
again seeks to identify appropriate videos for ASVA with the criteria above:
high correlation, dynamic content, and quality and relevance.

#### Dataset Summary

The final dataset, AVSync15, contains 90 training and 10 testing videos per
category, each 2$\sim$10 seconds long. We provide an overview of AVSync15 in
Fig. 2(a). To verify the effectiveness of our curation pipeline, we randomly
sample three 1500-video splits on the selected 15 categories from VGGSS and
AVSync-AC, and quantitatively compare them with AVSync15 in Fig. 2(b) and Tab.
2(b). We also compare AVSync15 with existing audio-visual datasets in
supplementary, highlighting its advantages for ASVA.

###  3.2 Evaluation Metrics

ASVA is a multi-faceted generation task, requiring generated content to have
high visual quality at both image and video level. At the image level, we
follow previous conventions [4, 12] to use (1) Fréchet Inception Distance
(FID) [16] to measure the quality of individual frames; (2) IA [14]/IT [45] to
measure image-audio/image-text semantics alignment of CLIP [30] features,
respectively. At the video level, we first use Fréchet Video Distance (FVD)
[40] to assess video quality. Then, to measure audio-video synchronization, we
re-use the trained synchronization classifier to compute the following
metrics:

#### RelSync

During testing, we have only the groundtruth synchronized audio-video pair
(${\bm{a}},{\bm{v}}$) and the generated video $\bm{\hat{v}}$, lacking reliable
shifted audio-video pairs to compute Eq. 1. Therefore, we contrast the
generated audio-video pair (${\bm{a}},\bm{\hat{v}}$) by using the groundtruth
pair (${\bm{a}},{\bm{v}}$) as a reference:

$\text{RelSync}({\bm{a}},\bm{\hat{v}},{\bm{v}})=\frac{\exp(\phi_{{\bm{a}},\bm{\hat{v}}})}{\exp(\phi_{{\bm{a}},{\bm{v}}})+\exp(\phi_{{\bm{a}},\bm{\hat{v}}})}$
(2)

which produces a synchronization score between input audio and the generated
video. Note that the metric becomes more reliable when a better synchronized
reference audio-video pair (${\bm{a}},{\bm{v}}$) with clearer synchronization
cues is adopted.

#### AlignSync

The synchronization classifier is only trained on semantically-aligned and
temporally-shifted audio-video pairs sampled from the same instance (See
supplementary). RelSync is thus implicitly conditioned on semantics alignment,
i.e., P(Sync|Align). To jointly measure semantics aligment and
synchronization, we first approximate $P_{\text{Align}}$ similarly as RelSync:

$P_{\text{Align}}({\bm{a}},\bm{\hat{v}},{\bm{v}}_{1})=\frac{1}{b-1}\sum_{i=2...b}\frac{\exp(\text{IA}_{{\bm{a}},\bm{\hat{v}}_{i}})}{\exp(\text{IA}_{{\bm{a}},\bm{\hat{v}}_{i}})+\exp(\text{IA}_{{\bm{a}},{\bm{v}}_{1}})}$
(3)

where ${\bm{v}}_{1}$ is the first frame input for animation, and $b$ is the
number of generated frames. The generated first frame $\bm{\hat{v}}_{1}$ is
eliminated because it is often a replicate of input ${\bm{v}}_{1}$. By
multiplying $P_{\text{Align}}$ with RelSync, we obtain the joint score:

$\text{AlignSync}({\bm{a}},\bm{\hat{v}},{\bm{v}})=P_{\text{Align}}({\bm{a}},\bm{\hat{v}},{\bm{v}}_{1})\cdot\text{RelSync}({\bm{a}},\bm{\hat{v}},{\bm{v}})$
(4)

which jointly measures semantic alignment and temporal synchronization between
${\bm{a}}$ and $\bm{\hat{v}}$, by referring to the groundtruth pair
(${\bm{a}},{\bm{v}}$).

These automated metrics are not always aligned with human preference. We
therefore conduct a user study detailed in Sec. 5.2, asking human raters to
compare videos generated by multiple models and select the best one.

##  4 Audio-Video Synchronized Diffusion

###  4.1 Preliminary: Text-to-Image Latent Diffusion

Text-to-image latent diffusion models (LDMs [32]) encode images ${\bm{x}}$
into a lower-dimensional latent space ${\bm{z}}={\mathcal{E}}({\bm{x}})$ using
a pre-trained perceptual auto-encoder ${\mathcal{E}}(\cdot)$, and learn the
conditional distribution $p({\bm{z}}|{\bm{\tau}})$ given a CLIP-encoded text
prompt ${\bm{\tau}}$. It gradually denoises latents ${\bm{z}}^{k}$, obtained
by corrupting the image latent ${\bm{z}}$ with Gaussian noise $\epsilon$, over
$k$ time steps. A denoising UNet $\epsilon_{\theta}(\cdot)$ parameterized by
$\theta$ is deployed to estimate the added noise $\epsilon$ by minimizing

${\mathcal{L}}_{\textit{LDM}}={\mathbb{E}}_{{\bm{z}},\epsilon\sim{\mathcal{N}}(0,1),k}\left[\|\epsilon-\epsilon_{\theta}({\bm{z}}^{k},k,{\bm{\tau}})\|_{2}^{2}\right]$
(5)

During inference, LDMs start from a random gaussian noise map ${\bm{z}}^{K}$,
and iterate over $K$ reverse diffusion steps [17, 36] for cleaner latent
${\bm{z}}^{k-1}={\bm{z}}^{k}-\epsilon_{\theta}({\bm{z}}^{k},k,{\bm{\tau}})$,
until ${\bm{z}}^{0}$ is obtained. LDMs then decode the latent into image
${\bm{x}}^{0}={\mathcal{D}}({\bm{z}}^{0})$ using the pre-trained decoder
${\mathcal{D}}(\cdot)$. For simplicity, we refer to the images by their
latents ${\bm{z}}$ rather than ${\bm{x}}$ throughout the rest of the paper.

###  4.2 Proposed Architecture

In this work, we extend the capabilities of LDMs for ASVA, by focusing on
learning video dynamics and temporal synchronization, unlike existing
approaches [14, 37, 38] which primarily use audio to guide global semantics.
We propose the Audio-Video Synchronized Diffusion model (AVSyncD), which
builds upon a pretrained image LDM and integrates synchronized audio control
and temporal layers for improved video consistency. The major component is a
UNet $\epsilon_{\theta}$ aiming to denoise a video instead of image. The UNet
is trained on synchronized audio-video pairs, with the first frame
${\bm{z}}_{1}$, the corresponding audio ${\bm{a}}$, and the CLIP encoded audio
category name ${\bm{\tau}}$ as input conditions. The LDM denoising objective
in Eq. 5 is applied to the remaining frames ${\bm{z}}_{2:rT}$ to be predicted.
The architecture of AVSyncD is shown in Fig. 3 and discussed below.

#### Initial-frame Conditioning

To condition LDM on input image, we feed initial frame’s latent ${\bm{z}}_{1}$
without noise into the UNet at every diffusion timestep $k$. For all
subsequent frames, we adhere to the original LDM, using independently sampled
noisy latents ${\bm{z}}_{2:rT}^{k}$ as inputs and predicting the added noise
$\epsilon^{k}$.

#### Text Conditioning

We retained the text cross attention layers in original LDMs [32] by
conditioning on the audio category as a prompt. It however brings
insignificant gains due to limited text diversity in training, see
supplementary.

#### Audio Conditioning

To facilitate audio-synchronized generation, we condition the generation on
ImageBind audio embeddings [14]. ImageBind computes a audio classification
token, ${\bm{a}}^{g}$, representing global semantics, and local patch tokens,
${\bm{a}}_{f,t}$, across $T_{a}$ timestamps. Original ImageBind only uses
${\bm{a}}^{g}$ for contrastive learning and discards ${\bm{a}}_{f,t}$.
However, we found these frozen local tokens as efficient synchronization cues.
We split the patch tokens temporally into $rT$ segments, corresponding to the
same timestamps as the frames ${\bm{z}}_{1:rT}$ , and append the global token
to each. Each frame ${\bm{z}}_{t}$ then learns both semantics and
synchronization guidance from its audio segment ${\bm{a}}_{t}$ via cross-
attention [41].

#### Spatial Convolution

The original LDM’s pretrained spatial convolutional layers were frozen and
used without modification.

#### First-frame Temporal Convolution

Each spatial convolution block was augmented with a 1D temporal convolution
layer (kernel size 3) to capture temporal dependencies. To better adhere to
the starting image, ${\bm{z}}_{1}$, we adjusted the receptive field at frame
$t$ to include frames $(1,t-1,t)$ as opposed to $(t-1,t,t+1)$. These temporal
convolutions with _first-frame lookup_ were applied to three components in the
UNet, i.e., the input/output conv layers and all ResNet convs.

#### First-frame Spatial Attention

We also leveraged the pretrained LDM’s spatial self-attentions. Following
[20], we modified the frozen spatial attention layers to cross-attend to the
first frame rather than self-attend to the current frame, by computing key-
values from the first frame and queries from the current one.

#### Temporal Attention

To effectively model long-range visual dependencies, we incorporated temporal
attention layers [3]. Each frame index $t$ was converted into a sinusoidal
positional embedding, added to the corresponding frame’s latents after a
learnable projection [41]. Each frame’s local representation at spatial
position ($h,w$), ${\bm{z}}_{hwt}$, was then updated by attending to all
frames at the same position
$({\bm{z}}_{hw1},{\bm{z}}_{hw2},\ldots,{\bm{z}}_{hw(rT)})$ through a standard
attention mechanism.

Figure 3: AVSyncD overview. Left: We use ImageBind to encode audio into
semantically aware time-dependent tokens $({\bm{a}}_{t})_{t=1}^{rT}$ and CLIP
to encode the audio category into text embedding ${\bm{\tau}}$. In addition,
the model receives the latent of the first frame ${\bm{z}}_{1}$, and
iteratively denoises noisy latents of the subsequent frames
${\bm{z}}_{2:rT}^{k}$ via reverse diffusion. The denoising UNet, based on LDMs
[32], consists of a sequence of downsampling, bottleneck and up-sampling
blocks, with structure detailed on the right. Right: Anatomy of a UNet block
for frame ${\bm{z}}_{t}$. LDM’s original spatial conv, spatial attention and
text cross attention layers are frozen, while its spatial self-attention
layers are adjusted to first-frame spatial attentions, cross-attending to
${\bm{z}}_{1}$ instead. To learn video dynamics, we introduce temporal
attention layers, and first-frame lookup temporal convolutions applied to
input, output, and ResNet layers. We also train audio cross attentions for
audio conditioning and synchronization. Trainable layers are marked with
[10pt]8pt.

#### Classifier-free Audio Guidance

Classifier-free guidance [18] is a technique used to control the impact of
input prompts on the generated outputs. We extended it to amplify audio
guidance for improved synchronization. To accomplish this, we trained the
model for both audio-conditioned and unconditioned generation, by randomly
replacing ${\bm{a}}$ with a null audio embedding, ${\bm{a}}_{\emptyset}$, with
a 20% likelihood. ${\bm{a}}_{\emptyset}$ was computed by encoding an all-zero
audio melspectrogram via ImageBind. During inference, a factor $\eta\geq 1$
controls the audio guidance:

${\bm{z}}_{2:rT}^{k-1}=(1-\eta)\cdot\epsilon_{\theta}({\bm{z}}_{2:rT}^{k},k;{\bm{z}}_{1},{\bm{a}}_{\emptyset},{\bm{\tau}})+\eta\cdot\epsilon_{\theta}({\bm{z}}_{2:rT}^{k},k;{\bm{z}}_{1},{\bm{a}},{\bm{\tau}})$
(6)

where $\eta$ guides the denoising process towards latents congruent with
audio-conditioned generation and away from those of audio-unconditional
generation.

Moreover, for comparison, we also trained an image-to-video variant of
AVSyncD, named I2VD, by removing audio conditioning.

##  5 Experiments

We provide more experiment details in supplementary, including implementation
details, experiment results, and sounded videos for comparison.

###  5.1 Implementation

#### Dataset

We experimented on 3 datasets. (1) AVSync15: Our high-quality dataset curated
from VGGSound [7]. It is a class-balanced dataset, with 15 motion categories,
1350 training videos, and 150 test videos. We also assessed our data curation
pipeline via experiments on uncurated VGGSS and automatically curated AVSync-
AC. (2) Landscapes [21]: A conventional dataset for audio-to-video generation,
with 9 environmental sound classes. We followed the split in [33, 46] with 900
clips for training and 100 for testing. Since Landscapes is full of ambient
sounds without synchronized video motion, we mainly use it to evaluate visual
quality. (3) TheGreatestHits [27]: An audio-video dataset recording humans
probing environments with a drumstick, with 733 videos for training and 244
for testing. The videos are clean, synchronized with audio at the moments of
probing, but still contain a large portion of moments with static background.
It also offers limited diversity, featuring a singular motion of probing.

#### Data Preprocess

We sampled 2-second synchronized audio-video pairs for experiments. Videos
were sampled at 6 fps with 12 frames, and resized to 256$\times$256 on
AVSync15/Landscapes or 128$\times$256 on TheGreatestHits. Following ImageBind
[14], audios were sampled at 16kHz and converted into 128-d spectrograms.

#### Baselines

We first adopted a simple baseline, Static, by repeating the input frame into
a video, then compared to several state-of-the-art audio-to-video methods. (1)
Semantic audio-to-video generation (CoDi [38]): Image, text, and audio are
encoded into a shared CLIP [30] space and summed, then cross attend to a video
diffusion model trained on large-scale datasets. (2) Image-to-video generation
(VideoCrafter [5]): A video diffusion model superior to CoDi on video
generation but without audio inputs. It animates images by fusing CLIP-encoded
image and text features into the model via modality-dependent cross attention
layers. (3) Synchronized audio-to-video generation (TPoS [19], AADiff [22],
TempoToken [46]): Audio is encoded into time-dependent segments and fused into
frozen text-to-image [32] or text-to-video [42] models. We re-implemented
AADiff, and used TPoS and TempoToken’s pretrained checkpoints on VGGSound and
Landscapes. More details provided in supplementary.

#### Training & Evaluation

We adopted the pretrained Stable Diffusion-V1.5 [32] as diffusion model and
ImageBind [14] as audio encoder. All models were primarily trained using Adam
optimizer with a batch size of 64, a learning rate of 0.0001, with 37k/24k/16k
iterations on AVSync15/Landscapes/TheGreatestHits, respectively. Evaluation
metrics include FID, IA, IT, FVD, AlignSync, RelSync, and a user study, as
detailed in Sec. 3.2. At test time, we evaluated on 3 videos uniformly sampled
from each example. More details provided in supplementary.

###  5.2 Main Results

(a) Comparison on AVSync15. AVSyncD produces synchronized dynamic motions,
such as muzzle flash, chicken and dog’s head poses, and swirling in toilet.

(b) Comparison on Landscapes. Without synchronization cues in audio, AVSyncD
still produces evolving visual effects and semantics, such as flames, waves,
and lightning.

(c) Generation by AVSyncD on TheGreatestHits. AVSyncD generates hitting
actions at appropriate moments.

Figure 4:  Qualitative results on three datasets. Sounded videos are provided
in supplementary. Table 1:  Quantitative results. User study shows votes on 3
metrics: image quality, frame consistency, and synchronization. Inputs are
combinations of image, text, audio.

Input | Model | FID$\downarrow$ | IA$\uparrow$ | IT$\uparrow$ | FVD$\downarrow$ | AlignSync$\uparrow$ | RelSync$\uparrow$ | User Study
---|---|---|---|---|---|---|---|---
IQ$\uparrow$ | FC$\uparrow$ | Sync$\uparrow$
T+A | TPoS [19] | $13.5$ | $23.38$ | $24.83$ | $2671.0$ | $19.52$ | $42.50$ | - | - | -
TempoToken [46] | $12.2$ | $18.84$ | $17.45$ | $4466.4$ | $19.74$ | $44.05$ | - | - | -
I+T | VideoCrafter [5] | $11.8$ | - | $29.87$ | $840.7$ | $21.28$ | $43.16$ | $38$ | $20$ | $12$
I2VD | $12.1$ | - | $30.35$ | $398.2$ | $21.80$ | $43.92$ | $62$ | $90$ | $91$
I+T+A | CoDi [38] | $14.5$ | $28.15$ | $23.42$ | $1522.6$ | $19.54$ | $41.51$ | - | - | -
TPoS [19] | $11.9$ | $38.36$ | $\bf 30.73$ | $1227.8$ | $19.67$ | $39.62$ | - | - | -
AADiff [22] | $18.8$ | $34.23$ | $28.97$ | $978.0$ | $22.11$ | $45.48$ | $37$ | $4$ | $5$
AVSyncD $\eta=1$ | $12.1$ | $38.36$ | $30.34$ | $382.7$ | $22.25$ | $44.81$ | - | - | -
AVSyncD $\eta=4$ | $\bf 11.7$ | $\bf 38.53$ | $30.45$ | $\bf 349.1$ | $22.62$ | $45.52$ | $\bf 163$ | $\bf 186$ | $\bf 192$
AVSyncD $\eta=8$ | $\bf 11.7$ | $37.99$ | $30.27$ | $420.7$ | $\bf 22.74$ | $\bf 45.88$ | - | - | -
Static | - | $39.76$ | $30.39$ | $1220.4$ | $21.83$ | $43.66$ |  |  |
Groundtruth | - | $40.06$ | $30.31$ | - | $25.04$ | $50.00$ |  |  |

(a) Performance on AVSync15.

Input | Model | FID$\downarrow$ | IA$\uparrow$ | IT$\uparrow$ | FVD$\downarrow$ | AignSync$\uparrow$ | RelSync$\uparrow$
---|---|---|---|---|---|---|---
T+A | TPoS [19] | $16.5$ | $15.61$ | $\bf 26.70$ | $2081.3$ | $23.12$ | $48.15$
TempoToken [46] | $16.4$ | $22.58$ | $22.87$ | $2480.0$ | $24.21$ | $48.65$
I+T | I2VD | $16.7$ | - | $22.56$ | $539.5$ | $24.74$ | $49.89$
I+T+A | CoDi [38] | $20.5$ | $22.63$ | $24.23$ | $982.9$ | $22.63$ | $45.48$
TPoS [19] | $\bf 16.2$ | $\bf 23.52$ | $23.20$ | $789.6$ | $23.51$ | $47.05$
AADiff [22] | $70.7$ | $22.07$ | $22.92$ | $1186.3$ | $\bf 26.77$ | $\bf 53.93$
AVSyncD $\eta=1$ | $16.5$ | $22.29$ | $22.81$ | $463.1$ | $24.81$ | $49.96$
AVSyncD $\eta=4$ | $\bf 16.2$ | $22.49$ | $22.79$ | $\bf 415.2$ | $24.82$ | $49.93$
Static | - | $23.60$ | $22.21$ | $1177.5$ | $25.79$ | $51.59$
Groundtruth | - | $23.65$ | $22.08$ | - | $25.01$ | $50.00$

(b) Performance on Landscapes.

Input | Model | FID$\downarrow$ | IA$\uparrow$ | IT$\uparrow$ | FVD$\downarrow$ | AignSync$\uparrow$ | RelSync$\uparrow$
---|---|---|---|---|---|---|---
I+T | I2VD | $9.1$ | - | $\bf 13.42$ | $425.0$ | $22.05$ | $44.58$
I+T+A | AVSyncD $\eta=1$ | $9.0$ | $11.85$ | $13.18$ | $313.5$ | $22.59$ | $45.52$
AVSyncD $\eta=4$ | $\bf 8.7$ | $\bf 12.07$ | $13.31$ | $\bf 249.3$ | $\bf 22.83$ | $\bf 45.95$
Static | - | $13.33$ | $16.56$ | $348.9$ | $24.36$ | $48.73$
Groundtruth | - | $13.52$ | $16.49$ | - | $25.02$ | $50.00$

(c) Performance on TheGreatestHits.

#### Dataset Comparison

In Tab. 1(c), the Static baseline is a good indicator of dataset attributes.
It has similar IA/IT scores compared to Groundtruth, since Static is composed
of a subset of groundtruth frames, which are obviously semantically aligned.
FVD of Static on TheGreatestHits is relatively low, since the TheGreatestHits
contains frequent static groundtruth clips without any moving objects. More
importantly, the RelSync and AlignSync of Static gradually increase from
AVSync15, to TheGreatestHits, to Landscapes, with those on Landscapes even
surpassing the groundtruth. The fact that static videos perform well in terms
of audio synchronization on TheGreatestHits and Landscapes datasets testify
for the superiority of AVSync15 for audio-video synchronized generation.

#### Model Comparison

We compared to prior works on AVSync15 and Landscapes in Tabs. 1(a), 1(b),
4(a) and 4(b). In Tab. 1(a), CoDi achieves inferior results on almost all
metrics. TPoS(I+T+A) shows strong image quality (FID), but is worse in video
quality (FVD) and synchronization (RelSync). TempoToken, on the other hand, is
better at synchronization rather than visual quality, likely due to lack of
image input. AADiff is competitive on synchronization but extremely bad on
image quality (FID). This is expected as AADiff adjusts each frame using audio
amplitude, producing visual changes that highly correlate to audio changes
temporally but may be overwhelmed by noises, as shown in Fig. 4(a). On
Landscapes, its similar FVD to Static but abnormally higher AlignSync and
RelSync also suggest that it only applied minor modifications to the input
image due to lack of sound changes, as shown in Fig. 4(b). Without audio
input, VideoCrafter performs poorly on synchronization. It also has difficulty
faithfully adhering to the input frame, as in Fig. 4(a). Our AVSyncD, however,
achieves the best animation results on almost all metrics. On Landscapes,
AVSyncD also performs the best on FID and FVD, with other metrics being
similar to the groundtruth. We provide qualitative comparison in Fig. 4.

#### User Study

We invited 15 participants to compare 4 animation models with top overall
performance (VideoCrafter, AADiff, I2VD, AVSyncD) on AVSync15, based on 3
metrics in Tab. 1(c). The 4 models generated videos conditioned on the same
test examples (audio+image). Each test example was independently evaluated by
2 participants to select their most preferred generation (vote) on each
metric. In total, we evaluated all 150 test examples on AVSync15, collecting
150$\times$2=300 votes on each metric. Tab. 1(a) shows votes each model
received. AVSyncD received the most votes on all metrics.

###  5.3 Ablation Studies

#### Audio Conditioning

In Tab. 1(c), AVSyncD almost always outperforms I2VD on all metrics,
expecially AlignSync and RelSync. On Landscapes, however, AVSyncD does not
improve RelSync, further suggesting lack of synchronization cues on
Landscapes. These results validates AVSyncD can effectively utilize audio as
guidance to improve visual quality as well as synchronization.

#### Audio Guidance

Tab. 1(c) shows increasing the audio guidance factor $\eta$ from 1 to 4
improves FID, IA, and FVD on all three datasets. As expected, audio guidance
also improved AlignSync and RelSync significantly on AVSync15 and
TheGreatestHits, but not on the less synchronized Landscapes dataset. Prior
works [37, 22, 19] claimed that increasing audio amplitude can also lead to
stronger visual effects. We compare this approach with our audio guidance in
Fig. 5(a). Audio guidance offers a better control mechanism than audio
amplitude.

Table 2: Effect of (a) first-frame lookups (b) data curation, evaluated on
AVSync15.

FF-Conv | FF-Attn | FID$\downarrow$ | FVD$\downarrow$ | AignSync$\uparrow$
---|---|---|---|---
✗ | ✗ | $11.8$ | $383.3$ | $22.19$
✓ | ✗ | $\bf 11.6$ | $347.4$ | $22.24$
✓ | ✓ | $11.8$ | $\bf 325.6$ | $\bf 22.33$

(a)

Dataset | AC | MC | FID$\downarrow$ | FVD$\downarrow$ | AignSync$\uparrow$
---|---|---|---|---|---
VGGSS | ✗ | ✗ | $12.9$ | $1307.9$ | $21.50$
AVSync-AC | ✓ | ✗ | $12.0$ | $428.8$ | $22.09$
AVSync15 | ✓ | ✓ | $\bf 11.8$ | $\bf 325.6$ | $\bf 22.33$

(b)

#### Effect of First-frame (FF) Lookups

We validate the effectiveness of First-frame (FF) Lookups by replacing them
with the standard temporal convolutions or spatial self-attentions. Tab. 2(a)
shows their effectiveness.

#### Effect of Data Curation

We trained AVSyncD on randomly sampled subsets from VGGSS and AVSync-AC,
ensuring equal training data scale and balanced category distributions.
Results in Tab. 2(b) justified our data curation pipeline.

###  5.4 Applications and Extensions

Despite AVSync15 as a relatively small dataset, we found AVSyncD trained on it
can generalize well to many applications. See more results in supplementary.

(a)

(b)

(c)

Figure 5: (a): Effects of audio amplitude vs. classifier-free audio guidance.
top: original audio with $\eta=1$; mid: $100\times$ amplified audio with
$\eta=1$; bottom: original audio with $\eta=8$. (b): Animate generated images.
(c): Animation with internet images and audios.

(a) Controllable animation with (un)related audios.

(b) Animate target objects on internet images with audios. Cartridges ejection
effects when firing a gun are better viewed in videos in supplementary.

Figure 6: Controllable image animation with audios. Key frames are visualized.

#### Animate Generated Images

When lacking an image as input, we can use existing image generators to
generate the image, which AVSyncD can also animate. Fig. 5(b) shows animations
on images generated by StableDiffusion-V1.5 [32].

#### Animate Contents from Internet

AVSyncD can also generalize well to unseen images and audios from internet, as
shown in Fig. 5(c).

#### Control Animations with (Un)related Audios

We can control the motion of an image to follow desired audios, e.g., animate
a baby to not only cry, but also bark like a dog or roar like a lion, as seen
in Fig. 6(a). When there is no object related to the audio, the animations do
not demonstrate corresponding motion.

#### Animate Target Objects with Audios

When multiple objects of interests exist in the image, a scenario not existing
in training data, we can still use audios to only animate the related target
object, as shown in Fig. 6(b).

##  6 Conclusion

In this paper, we tackled the under-explored Audio-Synchronized Visual
Animation task, with an emphasis on generating video dynamics synchronized
with provided audios. Lacking appropriate datasets for the task, we
contributed a high-quality dataset AVSync15 and an evaluation benchmark
through a two-stage data curation pipeline. We further proposed Audio-Video
Synchronized Diffusion model, capable of animating an image with natural and
audio-synchronized motions. We hope our research inspires further work on
synchronized generation.

One limitation of our work is that it cannot generalize to all audios classes
in the world, due the scale of the proposed AVSync15. This however requires
several orders of magnitude larger clean training data, which is challenging
to collect. We thus leave it as a future work.

## References

  * [1] Pyscenedetect. https://www.scenedetect.com/
  * [2] Bain, M., Nagrani, A., Varol, G., Zisserman, A.: Frozen in time: A joint video and image encoder for end-to-end retrieval. In: IEEE International Conference on Computer Vision (2021)
  * [3] Blattmann, A., Rombach, R., Ling, H., Dockhorn, T., Kim, S.W., Fidler, S., Kreis, K.: Align your latents: High-resolution video synthesis with latent diffusion models. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2023)
  * [4] Brock, A., Donahue, J., Simonyan, K.: Large scale gan training for high fidelity natural image synthesis. In: ICLR (2019)
  * [5] Chen, H., Xia, M., He, Y., Zhang, Y., Cun, X., Yang, S., Xing, J., Liu, Y., Chen, Q., Wang, X., Weng, C., Shan, Y.: Videocrafter1: Open diffusion models for high-quality video generation (2023)
  * [6] Chen, H., Xie, W., Afouras, T., Nagrani, A., Vedaldi, A., Zisserman, A.: Audio-visual synchronization in the wild. In: Proceedings of the British Machine Vision Conference (BMVC) (2021)
  * [7] Chen, H., Xie, W., Vedaldi, A., Zisserman, A.: Vggsound: A large-scale audio-visual dataset. In: ICASSP (2020)
  * [8] Chen, S., Wu, Y., Wang, C., Liu, S., Tompkins, D., Chen, Z., Wei, F.: Beats: Audio pre-training with acoustic tokenizers (2022)
  * [9] Chung, J.S., Zisserman, A.: Lip reading in the wild. In: ACCV (2016)
  * [10] Chung, J.S., Zisserman, A.: Out of time: automated lip sync in the wild. In: ACCV Workshop (2016)
  * [11] Chung, S.W., Chung, J.S., Kang, H.G.: Perfect match: Improved cross-modal embeddings for audio-visual synchronisation. In: ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2019)
  * [12] Dhariwal, P., Nichol, A.: Diffusion models beat gans on image synthesis. In: NeurIPS (2021)
  * [13] Gemmeke, J.F., Ellis, D.P.W., Freedman, D., Jansen, A., Lawrence, W., Moore, R.C., Plakal, M., Ritter, M.: Audio set: An ontology and human-labeled dataset for audio events. In: Proc. IEEE ICASSP 2017 (2017)
  * [14] Girdhar, R., El-Nouby, A., Liu, Z., Singh, M., Alwala, K.V., Joulin, A., Misra, I.: Imagebind: One embedding space to bind them all. In: CVPR (2023)
  * [15] Hertz, A., Mokady, R., Tenenbaum, J., Aberman, K., Pritch, Y., Cohen-Or, D.: Prompt-to-prompt image editing with cross attention control. In: ICLR (2023)
  * [16] Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time-scale update rule converge to a local nash equilibrium. In: Advances in Neural Information Processing Systems (2017)
  * [17] Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. In: NeurIPS (2020)
  * [18] Ho, J., Salimans, T.: Classifier-free diffusion guidance. In: NeurIPS Workshop on Deep Generative Models and Downstream Applications (2022)
  * [19] Jeong, Y., Ryoo, W., Lee, S., Seo, D., Byeon, W., Kim, S., Kim, J.: The power of sound (tpos): Audio reactive video generation with stable diffusion. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 7822–7832 (2023)
  * [20] Khachatryan, L., Movsisyan, A., Tadevosyan, V., Henschel, R., Wang, Z., Navasardyan, S., Shi, H.: Text2video-zero: Text-to-image diffusion models are zero-shot video generators. In: ICCV (2023)
  * [21] Lee, S.H., Oh, G., Byeon, W., Bae, J., Kim, C., Ryoo, W.J., Yoon, S.H., Kim, J., Kim, S.: Sound-guided semantic video generation. In: ECCV (2022)
  * [22] Lee, S., Kong, C., Jeon, D., Kwak, N.: Aadiff: Audio-aligned video synthesis with text-to-image diffusion. In: CVPR Workshop on Content Generation (2023)
  * [23] Li, R., Yang, S., Ross, D.A., Kanazawa, A.: Learn to dance with aist++: Music conditioned 3d dance generation. In: ICCV (2021)
  * [24] Li, Y., Liu, H., Wu, Q., Mu, F., Yang, J., Gao, J., Li, C., Lee, Y.J.: Gligen: Open-set grounded text-to-image generation (2023)
  * [25] Mokady, R., Hertz, A., Aberman, K., Pritch, Y., Cohen-Or, D.: Null-text inversion for editing real images using guided diffusion models. In: CVPR (2023)
  * [26] Ng, E., Romero, J., Bagautdinov, T., Bai, S., Darrell, T., Kanazawa, A., Richard, A.: From audio to photoreal embodiment: Synthesizing humans in conversations. In: ArXiv (2024)
  * [27] Owens, A., Isola, P., McDermott, J., Torralba, A., Adelson, E.H., Freeman, W.T.: Visually indicated sounds. In: CVPR (2016)
  * [28] Park, S.J., Kim, M., Hong, J., Choi, J., Ro, Y.M.: Synctalkface: Talking face generation with precise lip-syncing via audio-lip memory. In: AAAI Conference on Artificial Intelligence (AAAI) (2022)
  * [29] Pedro Morgado, Ishan Misra, N.V.: Robust audio-visual instance discrimination. In: Computer Vision and Pattern Recognition (CVPR), IEEE/CVF Conf. on (2021)
  * [30] Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision. In: ICML (2021)
  * [31] Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., Chen, M.: Hierarchical text-conditional image generation with clip latents. In: arXiv (2022)
  * [32] Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: CVPR (2022)
  * [33] Ruan, L., Ma, Y., Yang, H., He, H., Liu, B., Fu, J., Yuan, N.J., Jin, Q., Guo, B.: Mm-diffusion: Learning multi-modal diffusion models for joint audio and video generation. In: CVPR (2023)
  * [34] Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., Schramowski, P., Kundurthy, S., Crowson, K., Schmidt, L., Kaczmarczyk, R., Jitsev, J.: Laion-5b: An open large-scale dataset for training next generation image-text models. In: NeurIPS (2022)
  * [35] Singer, U., Polyak, A., Hayes, T., Yin, X., An, J., Zhang, S., Hu, Q., Yang, H., Ashual, O., Gafni, O., Parikh, D., Gupta, S., Taigman, Y.: Make-a-video: Text-to-video generation without text-video data (2022)
  * [36] Song, J., Meng, C., Ermon, S.: Denoising diffusion implicit models. In: ICLR (2021)
  * [37] Sung-Bin, K., Senocak, A., Ha, H., Owens, A., Oh, T.H.: Sound to visual scene generation by audio-to-visual latent alignment. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023)
  * [38] Tang, Z., Yang, Z., Zhu, C., Zeng, M., Bansal, M.: Any-to-any generation via composable diffusion. In: Thirty-seventh Conference on Neural Information Processing Systems (2023), https://openreview.net/forum?id=2EDqbSCnmF
  * [39] Tsuchida, S., Fukayama, S., Hamasaki, M., Goto, M.: Aist dance video database: Multi-genre, multi-dancer, and multi-camera database for dance information processing. In: Proceedings of the 20th International Society for Music Information Retrieval Conference, ISMIR 2019. Delft, Netherlands (Nov 2019)
  * [40] Unterthiner, T., van Steenkiste, S., Kurach, K., Marinier, R., Michalski, M., Gelly, S.: Towards accurate generative models of video: A new metric & challenges. In: arXiv (2019)
  * [41] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: NeurIPS (2017)
  * [42] Wang, J., Yuan, H., Chen, D., Zhang, Y., Wang, X., Zhang, S.: Modelscope text-to-video technical report (2023)
  * [43] Wang, W., Xie, k., Liu, Z., Chen, H., Cao, Y., Wang, X., Shen, C.: Zero-shot video editing using off-the-shelf image diffusion models. arXiv preprint arXiv:2303.17599 (2023)
  * [44] Wu, R., Chen, L., Yang, T., Guo, C., Li, C., Zhang, X.: Lamp: Learn a motion pattern by few-shot tuning a text-to-image diffusion model. arXiv preprint arXiv:2310.10769 (2023)
  * [45] Xu, H., Ghosh, G., Huang, P.Y., Okhonko, D., Aghajanyan, A., Metze, F., Zettlemoyer, L., Feichtenhofer, C.: Videoclip: Contrastive pre-training for zero-shot video-text understanding. In: EMNLP (2021)
  * [46] Yariv, G., Gat, I., Benaim, S., Wolf, L., Schwartz, I., Adi, Y.: Diverse and aligned audio-to-video generation via text-to-video model adaptation (2023)
  * [47] Ye, Z., Jiang, Z., Ren, Y., Liu, J., He, J., Zhao, Z.: Geneface: Generalized and high-fidelity audio-driven 3d talking face synthesis. In: ICLR (2023)
  * [48] Zhang, L., Rao, A., Agrawala, M.: Adding conditional control to text-to-image diffusion models (2023)
  * [49] Zhou, H., Liu, Y., Liu, Z., Luo, P., Wang, X.: Talking face generation by adversarially disentangled audio-visual representation. In: AAAI Conference on Artificial Intelligence (AAAI) (2019)
  * [50] Zhou, H., Sun, Y., Wu, W., Loy, C.C., Wang, X., Liu, Z.: Pose-controllable talking face generation by implicitly modularized audio-visual representation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2021)


Comments on non-Abelian T-duals and their holographic description

Paul<EMAIL_ADDRESS>Ricardo
<EMAIL_ADDRESS>
Department of Physics, Swansea University, Swansea SA2 8PP, United Kingdom

###### Abstract

We present three different Type IIB AdS4 Supergravity solutions, derived using
the electrostatic-like problem formalism which preserves 8 Poincaré
Supercharges. Two of these solutions correspond to the Abelian and non-Abelian
T-duals of a Type IIA background, obtained from the dimensional reduction of
11D AdS${}_{4}\times S^{7}$ Supegravity. The third solution is a new
background with some special properties. We compute the Page charges in each
case, and provide brane set-up descriptions of the geometries. Also, we argue
that the non-Abelian T-dual can be understood as a zoom-in of a more general
background with a well defined holographic dual.

###### Contents

  1. 1 Introduction
  2. 2 AdS4 Type IIB Background
    1. 2.1 Abelian T-dual (ATD) Solution
    2. 2.2 Non-Abelian T-dual (NATD) Solution
  3. 3 On the Completion of Geometries
    1. 3.1 NATD as a zoom-in
    2. 3.2 Comments on the ATD
  4. 4 A New Supergravity Solution
  5. 5 Conclusions

##  1 Introduction

The AdS/CFT conjecture [1] allows us to give geometrical descriptions of
super-conformal field theories in different dimensions in terms of a given
brane configuration in Supergravity. For the case of $\mathcal{N}=4$
Supersymmetric theories, these gauge/gravity dualities had been established in
different dimensions, e.g. [2]-[5] for AdS2, [6]-[11] for AdS3,[12]-[15] for
AdS4, [16]-[18] for AdS5, [19]-[23] for AdS6 and [24]-[26] for AdS7 among
others.

When considering the Abelian T-duality (ATD) transformation of a Supergravity
solution with a certain holographically dual field theory, since it is a
symmetry of string theory, both geometries describe the same dynamics in
different ways. Therefore both geometries are holographically dual to the same
field theory.

The previous statement is no longer valid if one considers non-Abelian
T-duality (NATD) transformations. These transformations, firstly developed for
pure NS-NS fields in [27] and then extended for backgrounds containing R-R
fields that have a $SU(2)$ isometry subgroup in [28] (the extension beyond
$SU(2)$ was done in [29] and an explicit flux transformation was given in
[30]), do not preserve the symmetries of the compact manifold along which one
performs the transformations. Hence, the background obtained after the NATD
transformations describes different dynamics with respect to its seed, which
can be seen at the level of the string $\sigma$-model [31]. In this manner
NATD is not a symmetry of String Theory but rather a solution generating
method, see e.g. [32, 33].

It is then natural to ask what is the corresponding dual field theory of a
background obtained after NATD transformations. This was first addressed in
[34, 35], where it was shown that it is indeed possible to describe properties
of the NATD geometry in terms of those of its predecessor, e.g. observables
charged under the dualised isometries changed after the transformation while
non-charged (neutral) observables and the holographic central charge remain
the same. However, no precise proposal for a dual field theory to NATD
geometries was given until the work of [36] in AdS${}_{5}\times S^{5}$, where
the dual theory came in the form of a ‘long quiver’ CFT preserving
$\mathcal{N}=2$ supersymmetry.

In [37] the NATD of the AdS${}_{4}\times S^{7}$ reduction to Type IIA
Supergravity was shown to have a brane-configuration given by an unbounded
Hanany-Witten set-up [38], which suggests that the dual field theory is not
well defined. Using the formalism of [36, 39], a consistent way of completing
the field theory was then developed, such that the completed form can be
written in terms of a linear quiver. This leads to a new Type IIB background,
in which the NATD appears as a zoom-in on a region of this completed
background.

Here we continue with the discussion about the holographic description of
$\text{AdS}_{4}$ backgrounds obtained via NATD using the electrostatic-like
problem set-up developed in [15], providing an interpretation of NATDs as a
zoom-in on backgrounds that have a well defined holographic dual, along
similar lines to the $\text{AdS}_{6}$ case in [22].

The outline of the paper is as follows: in section 2 we present two solutions
to Type IIB Supergravity on AdS4, which correspond to the ATD and the NATD
found in [37], computing the respective Page charges in each case. In section
3 we argue how the NATD appears as a zoom-in of the backgrounds found in [15],
which were shown to be dual to linear quiver field theories. In section 4 we
study a new AdS4 Supergravity solution. We conclude in section 5 with a
summary of the paper and some final remarks.

##  2 AdS4 Type IIB Background

In [12], an infinite family of solutions to Type IIB Supergravity that are
dual to $\mathcal{N}=4$ super-conformal field theories were proposed. It was
then showed in [15] that the same family of solutions can be obtained in the
‘electrostatic’ formalism. In order to match the conformality and global
symmetries of the field theory, the Supergravity background geometry must
contain an AdS4 factor and a couple of two spheres $S^{2}_{1}$ and
$S^{2}_{2}$. The other NS and R background fields must preserve the isometries
of the geometry. The Type IIB AdS${}_{4}\times S^{2}\times S^{2}$ has the
following background

$\begin{gathered}ds_{10,st}^{2}=f_{1}(\sigma,\eta)\bigg{[}ds^{2}(\text{AdS}_{4})+f_{2}(\sigma,\eta)ds^{2}(S_{1}^{2})+f_{3}(\sigma,\eta)ds^{2}(S_{2}^{2})+f_{4}(\sigma,\eta)(d\sigma^{2}+d\eta^{2})\bigg{]},\\\
e^{-2\Phi}=f_{5}(\sigma,\eta),~{}~{}~{}~{}~{}B_{2}=f_{6}(\sigma,\eta)\text{Vol}(S_{1}^{2}),~{}~{}~{}C_{2}=f_{7}(\sigma,\eta)\text{Vol}(S_{2}^{2}),~{}~{}\tilde{C}_{4}=f_{8}(\sigma,\eta)\text{Vol}(\text{AdS}_{4}),\\\
f_{1}=\frac{\pi}{2}\sqrt{\frac{\sigma^{3}\partial_{\eta\sigma}^{2}V}{\partial_{\sigma}(\sigma\partial_{\eta}V)}},~{}~{}~{}~{}f_{2}=-\frac{\partial_{\eta}V\partial_{\sigma}(\sigma\partial_{\eta}V)}{\sigma\Lambda},~{}~{}~{}~{}f_{3}=\frac{\partial_{\sigma}(\sigma\partial_{\eta}V)}{\sigma\partial_{\eta\sigma}^{2}V},~{}~{}~{}~{}f_{4}=-\frac{\partial_{\sigma}(\sigma\partial_{\eta}V)}{\sigma^{2}\partial_{\eta}V},\\\
f_{5}=-16\frac{\Lambda\partial_{\eta}V}{\partial_{\eta\sigma}^{2}V},~{}~{}~{}~{}f_{6}=\frac{\pi}{2}\Bigg{(}\eta-\frac{\sigma\partial_{\eta}V\partial_{\eta}^{2}V}{\Lambda}\Bigg{)},~{}~{}~{}~{}f_{7}=-2\pi\Bigg{(}\partial_{\sigma}(\sigma
V)-\frac{\sigma\partial_{\eta}V\partial_{\eta}^{2}V}{\partial_{\eta\sigma}^{2}V}\Bigg{)},\\\
f_{8}=-\pi^{2}\sigma^{2}\bigg{(}3\partial_{\sigma}V+\frac{\sigma\partial_{\eta}V\partial_{\eta}^{2}V}{\partial_{\sigma}(\sigma\partial_{\eta}V)}\bigg{)},~{}~{}~{}~{}\Lambda=\partial_{\eta}V\partial_{\eta\sigma}^{2}V+\sigma\big{(}(\partial_{\eta\sigma}^{2}V)^{2}+(\partial_{\eta}^{2}V)^{2}\big{)},\end{gathered}$
(2.1)

with

$F_{1}=0,~{}~{}~{}~{}H=dB_{2},~{}~{}~{}~{}~{}F_{3}=dC_{2},~{}~{}~{}~{}~{}F_{5}=d\tilde{C}_{4}+*d\tilde{C}_{4}.$
(2.2)

Notice that all functions $f_{1},...,f_{8}$ can be determined in terms of a
single function $V(\sigma,\eta)$, and partial derivatives.

In order to be a solution of Type IIB Supergravity, $V$ must satisfy

$\partial_{\sigma}(\sigma^{2}\partial_{\sigma}V(\sigma,\eta))+\sigma^{2}\partial_{\eta}^{2}V(\sigma,\eta)=0.$
(2.3)

By using the redefinition

$V(\sigma,\eta)=\frac{\partial_{\eta}W(\sigma,\eta)}{\sigma},$ (2.4)

equation (2.3) becomes the 2D Laplace equation

$\partial_{\sigma}^{2}W(\sigma,\eta)+\partial_{\eta}^{2}W(\sigma,\eta)=0.$
(2.5)

Within this set-up, the boundary conditions imposed in [15] were analogous to
an electrostatic problem in which there are two parallel conducting planes and
a line of charge density given by $\mathcal{R}(\eta)$, which is usually called
the ‘Rank function’. This Rank function, which must be linear by sections in
order to have quantized Page charges, completely characterises the brane
configuration and therefore the field content of the dual quiver theory.

We are interested in solutions of the form (2.1) which do not necessarily
satisfy the same boundary conditions. In the variables $z,\bar{z}=\sigma\pm
i\eta$, any solution to the Laplace equation can be written as
$W=f(z)+\bar{f}(\bar{z})$. In what follows we will only consider the case in
which $f(z)$ and $\bar{f}(\bar{z})$ are regular, hence we can expand them in a
Taylor series. After imposing a reality condition for $W$, namely
$W=W^{\star}$, and going back to the variables $\sigma,\eta$, the most general
solution to equation (2.5) (ignoring boundary conditions) takes the form

$W=\alpha_{0}+\sum^{+\infty}_{n=1}\left[\alpha_{n}\sum^{\left\lfloor\frac{n}{2}\right\rfloor}_{k=0}\binom{n}{2k}(-1)^{k}\sigma^{n-2k}\eta^{2k}+\beta_{n}\sum^{\left\lfloor\frac{n-1}{2}\right\rfloor}_{k=0}\binom{n}{2k+1}(-1)^{k}\sigma^{n-2k-1}\eta^{2k+1}\right],$
(2.6)

from which we obtain the following general solution to equation (2.3)

$V=\sum^{+\infty}_{n=1}\left[\alpha_{n}\sum^{\left\lfloor\frac{n}{2}\right\rfloor}_{k=0}2k\binom{n}{2k}(-1)^{k}\sigma^{n-2k-1}\eta^{2k-1}+\beta_{n}\sum^{\left\lfloor\frac{n-1}{2}\right\rfloor}_{k=0}(2k+1)\binom{n}{2k+1}(-1)^{k}\sigma^{n-2k-2}\eta^{2k}\right].$
(2.7)

We will now show that different solutions appear by considering truncations of
the Taylor series.

###  2.1 Abelian T-dual (ATD) Solution

We first consider the case in which all the coefficients $\alpha_{n}$ and
$\beta_{n}$ are set to zero, except for $\alpha_{2}$ and $\alpha_{3}$. This
then leads to the following potential

$V_{\text{ATD}}(\sigma,\eta)=2\alpha_{2}\frac{\eta}{\sigma}-6\alpha_{3}\eta.$
(2.8)

Let us now set $\alpha_{2}=6\alpha_{3}$, $\alpha_{3}=M/12$ and use the change
of coordinates

$\sigma=2\cos^{2}\left(\frac{\mu}{2}\right),\ \ \ \ \eta=\frac{2}{\pi}r.$
(2.9)

The range of the coordinates are $\mu\in[0,\pi]$ and, as we will explain
below, it is convenient to choose $r\in[n\pi,(n+1)\pi]$, where the points
$r=n\pi$ and $r=(n+1)\pi$ are identified. With this parametrization, the
background metric reads

$ds^{2}=\pi\cos\left(\frac{\mu}{2}\right)\left[ds^{2}(\text{AdS}_{4})+\sin^{2}\left(\frac{\mu}{2}\right)ds^{2}(S^{2}_{1})+\cos^{2}\left(\frac{\mu}{2}\right)ds^{2}(S^{2}_{2})+d\mu^{2}+\frac{4}{\pi^{2}\sin^{2}(\mu)}dr^{2}\right]$
(2.10)

and the rest of the background fields are given by

$\displaystyle e^{-2\Phi}$
$\displaystyle=4M^{2}\tan^{2}\left(\frac{\mu}{2}\right),$ (2.11)
$\displaystyle B_{2}$ $\displaystyle=r\,\text{Vol}(S^{2}_{1}),$ (2.12)
$\displaystyle C_{2}$ $\displaystyle=2Mr\,\text{Vol}(S^{2}_{2}),$ (2.13)
$\displaystyle\tilde{C}_{4}$ $\displaystyle=6\pi
Mr\,\text{Vol}(\text{AdS}_{4}).$ (2.14)

The field strengths for the gauge forms are

$\displaystyle H_{3}$ $\displaystyle=dr\wedge\text{Vol}(S^{2}_{1}),$ (2.15)
$\displaystyle F_{3}$ $\displaystyle=2Mdr\wedge\text{Vol}(S^{2}_{2}),$ (2.16)
$\displaystyle F_{5}$ $\displaystyle=6\pi
Mdr\wedge\text{Vol}(\text{AdS}_{4})-\frac{3}{4}\pi^{2}M\sin^{3}(\mu)d\mu\wedge\text{Vol}(S_{1}^{2})\wedge\text{Vol}(S_{2}^{2}).$
(2.17)

By observation, one can easily see the presence of singularities at $\mu=\pi$
(from the D6 branes in Type IIA) and at $\mu=0$ (from the NS5 branes), as
discussed in [37], and hence the system is defined in the interval
$\mu\in[0,\pi]$ (corresponding to $\sigma\in[0,1]$). The dilaton diverges to
positive infinity as $\mu\rightarrow 0$ and to negative infinity as
$\mu\rightarrow\pi$. Hence, the above Supergravity description is not well
defined in these limits.

This solution corresponds to the Abelian T-dual of the Type IIA solution
obtained from a dimensional reduction of 11D Supergravity [37].

#### Page Charges

We now calculate the Page Charges, using the following definition

$Q_{D_{p}/NS5}=\frac{1}{(2\pi)^{7-p}\alpha^{\prime}}\int_{\Sigma_{8-p}}\widehat{F}_{8-p},$
(2.18)

with $\alpha^{\prime}=1$, $2\kappa_{10}^{2}T_{D_{p}}=(2\pi)^{7-p}$ and
$\widehat{F}_{p}=F_{p}\wedge e^{-B_{2}}$. The background of equation (2.1) has
the following Page fluxes

$\widehat{F}_{3}=F_{3},~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}\widehat{F}_{5}=F_{5}-B_{2}\wedge
F_{3}.$ (2.19)

When integrating the charges, it is convenient to choose $r\in[n\pi,(n+1)\pi]$
since this is the minimal building block of the brane set-up, i.e. is the
smallest size interval in which we get integer charges for all the gauge
forms.

For the NS5 charge, we integrate in the submanifold
$\mathcal{M}_{1}=(r,S^{2}_{1})$ for any value of $\mu$

$N_{NS5}=\frac{1}{(2\pi)^{2}}\int_{\mathcal{M}_{1}}H_{3}=1.$ (2.20)

For the D5 charge, the integration is performed in the three-cycle
$\mathcal{M}_{2}=(r,S^{2}_{2})$ for any value of $\mu$

$N_{D5}=\frac{1}{(2\pi)^{2}}\int_{\mathcal{M}_{2}}F_{3}=2M.$ (2.21)

Finally, for the D3 charge, we integrate in the cycle
$\mathcal{M}_{3}=(\mu,S^{2}_{1},S^{2}_{2})$ for any value of $r$

$\displaystyle N_{D3}$
$\displaystyle=-\frac{1}{(2\pi)^{4}}\int_{\mathcal{M}_{3}}\widehat{F}_{5}=M,$
(2.22)

where the negative charge suggests that the orientation of the cycle should be
swapped in this case.

If we had considered instead the interval $r\in[n\pi,(n+2)\pi]$, the NS5 and
D5 charges would have counted the branes between $[n\pi,(n+1)\pi]$ and
$[(n+1)\pi,(n+2)\pi]$, thus not giving the smallest possible building block.

The full brane configuration for this geometry is described in figure 1(a), in
which M D3-branes are stretched between NS5-branes located at $n\pi$ and
$(n+1)\pi$, with 2M D5-branes in each interval. The two NS5-branes represent
the same brane, as the points $n\pi$ and $(n+1)\pi$ are identified. The
corresponding balanced linear quiver diagram is then given in figure 1(b).

$\mu$$r$D5$\otimes$$\otimes$NS5$n\pi$NS5$(n+1)\pi$D3 (a) Brane set-up

$2M$$M$ (b) Linear quiver

Figure 1: The Hanany-Witten (NS5, D3, D5) brane set-up for the ATD solution in
(a), along with the corresponding balanced linear quiver diagram, (b). The
brane set-up consists of $M$ D3-branes stretched between NS5-branes located at
$n\pi$ and $(n+1)\pi$, with $2M$ D5-branes in each interval (taking $M=1$).
Since the points $r=n\pi$ and $r=(n+1)\pi$ are identified, the two NS5-branes
in the figure correspond to the same brane (represented using a dashed line).

###  2.2 Non-Abelian T-dual (NATD) Solution

Another well known solution can be obtained by considering the case in which
only $\beta_{3}$ and $\beta_{4}$ are non-vanishing, namely

$V_{\text{NATD}}=\beta_{3}\left(3\frac{\eta^{2}}{\sigma}-3\sigma\right)+\beta_{4}(4\sigma^{2}-12\eta^{2}).$
(2.23)

After fixing $\beta_{3}=8\beta_{4}$ and $\beta_{4}=M/96$, and using the change
of coordinates

$\sigma=2\cos^{2}\left(\frac{\mu}{2}\right),\ \ \ \ \eta=\frac{2}{\pi}r,$
(2.24)

we obtain the following metric

$ds^{2}=\pi\cos\left(\frac{\mu}{2}\right)\left[ds^{2}(\text{AdS}_{4})+\frac{4}{\pi^{2}}\frac{r^{2}}{\Delta}\sin^{2}\left(\frac{\mu}{2}\right)ds^{2}(S^{2}_{1})+\cos^{2}\left(\frac{\mu}{2}\right)ds^{2}(S^{2}_{2})+d\mu^{2}+\frac{4}{\pi^{2}\sin^{2}(\mu)}dr^{2}\right],$
(2.25)

where $\Delta=\Delta(\mu,r)$ is given by

$\Delta(\mu,r)=\sin^{2}\left(\frac{\mu}{2}\right)\sin^{2}(\mu)+\frac{4}{\pi^{2}}r^{2},$
(2.26)

with the rest of the background fields given by

$\displaystyle e^{-2\Phi}$
$\displaystyle=M^{2}\tan^{2}\left(\frac{\mu}{2}\right)\Delta,$ (2.27)
$\displaystyle B_{2}$
$\displaystyle=\frac{4}{\pi^{2}}\frac{r^{3}}{\Delta}\text{Vol}(S^{2}_{1}),$
(2.28) $\displaystyle C_{2}$
$\displaystyle=\frac{\pi}{4}M\left(\frac{4}{\pi^{2}}r^{2}-\cos(\mu)\left(\cos^{2}(\mu)-3\right)\right)\text{Vol}(S^{2}_{2}),$
(2.29) $\displaystyle\tilde{C}_{4}$
$\displaystyle=\frac{\pi^{2}}{8}M\left(\frac{24}{\pi^{2}}r^{2}-(\cos(2\mu)-4\cos(\mu))\right)\text{Vol}(\text{AdS}_{4}).$
(2.30)

The gauge fields lead to the following field strengths

$\displaystyle H_{3}$
$\displaystyle=\frac{r^{2}}{2\pi^{2}\Delta^{2}}\left[8\left(\frac{4}{\pi^{2}}r^{2}+3\sin^{2}\left(\frac{\mu}{2}\right)\sin^{2}(\mu)\right)dr-r\bigg{(}\sin(\mu)+4\sin(2\mu)-3\sin(3\mu)\bigg{)}d\mu\right]\wedge\text{Vol}(S^{2}_{1}),$
(2.31) $\displaystyle F_{3}$
$\displaystyle=\frac{\pi}{2}M\left[\frac{4}{\pi^{2}}r\,dr-\frac{3}{2}\sin^{3}(\mu)d\mu\right]\wedge\text{Vol}(S^{2}_{2}),$
(2.32)

together with the self-dual 5-form

$\displaystyle F_{5}$
$\displaystyle=M\frac{\pi^{2}}{8}d\left(\frac{24}{\pi^{2}}r^{2}-(\cos(2\mu)-4\cos(\mu))\right)\wedge\text{Vol}(\text{AdS}_{4})$
(2.33)
$\displaystyle\phantom{=}-M\frac{r^{2}\sin^{2}(\mu)}{\pi\Delta}\left(3r\sin(\mu)d\mu+2\sin^{2}\left(\frac{\mu}{2}\right)dr\right)\wedge\text{Vol}(S^{2}_{1})\wedge\text{Vol}(S^{2}_{2}).$

This solution reproduces the non-Abelian T-dual of the Type IIA solution, in
[37]. As in the ATD, we have $\mu\in[0,\pi]$ due to the presence of
singularities. Once again, the above description becomes weak when
$\mu\rightarrow 0$ and $\mu\rightarrow\pi$, as the dilaton diverges in these
limits.

#### Page Charges

Now we compute the Page charges using equations (2.18). Note that unlike the
ATD, in this case, the region of $r$ is non-compact. However the charges below
are calculated over the interval $r\in[n\pi,(n+1)\pi]$, with $n\in\mathbb{N}$,
since each one of these intervals is a minimal building block of the brane
set-up.

Before computing the charges, it is convenient to perform the following large
gauge transformation

$B_{2}\rightarrow B_{2}-n\pi\text{Vol}(S^{2}_{1}).$ (2.34)

For the NS5 charge, we integrate over the submanifold
$\Sigma_{1}=(r,S^{2}_{1})$ for $\mu=0$ and $\mu=\pi$

$N_{NS5}=\frac{1}{(2\pi)^{2}}\int_{\Sigma_{1}|_{\mu=0,\pi}}H_{3}=1.$ (2.35)

This suggests that the location of the NS5-branes correspond to the location
of the singularities of the background.
For the D5 charge, we integrate over the submanifold
$\Sigma_{2}=(r,S^{2}_{2})$ for any value of $\mu$

$N_{D5}=\frac{1}{(2\pi)^{2}}\int_{\Sigma_{2}}F_{3}=(2n+1)M.$ (2.36)

Finally, for the D3 charge, we integrate over the cycle
$\Sigma_{3}=(\mu,S^{2}_{1},S^{2}_{2})$ for any value of $r$

$N_{D3}=-\frac{1}{(2\pi)^{4}}\int_{\Sigma_{3}}\widehat{F}_{5}=Mn.$ (2.37)

The full brane configuration of the entire geometry (for the case $M=1$) is
described in figure 2, in which $Mn$ D3-branes are stretched between two
NS5-branes located at $n\pi$ and $(n+1)\pi$, with $(2n+1)M$ D5-branes in each
interval. The corresponding (overbalanced) linear quiver diagram is then given
in figure 3.

$\mu$$r$$\otimes$D5$\otimes$$\otimes$D5$\otimes$$\otimes$$...$$\otimes$D5$\otimes$$...$$\otimes$D5NS5$\pi$NS5$2\pi$NS5$3\pi$NS5$n\pi$NS5$(n+1)\pi$.
. ..... . .D3D3D3 Figure 2: The Hanany-Witten (NS5, D3, D5) brane set-up for
the NATD solution (taking $M=1$), in which $Mn$ D3-branes are stretched
between two NS5 branes located at $n\pi$ and $(n+1)\pi$, with $(2n+1)M$
D5-branes in each interval. $M$$3M$$5M$$7M$$M$$2M$$3M$. . . Figure 3: The
corresponding overbalanced linear quiver diagram for the NATD geometry.

##  3 On the Completion of Geometries

###  3.1 NATD as a zoom-in

In [22], by using the electrostatic problem formalism, evidence was presented
on how the non-Abelian T-dual of AdS6 in massive Type IIA backgrounds could be
thought of as a particular zoom-in of a more general solution, which possesses
a well-defined holographic dual. Here, we give a similar argument for the NATD
of AdS4 in Type IIA Supergravity.

In [15], the infinite family of backgrounds dual to balanced linear quivers is
derived from the following potential (we consider $\sigma>0$ for simplicity)

$V_{\text{LQ}}(\sigma,\eta)=\sum^{+\infty}_{k=0}\frac{a_{k}}{\sigma}\cos\left(\frac{k\pi\eta}{P}\right)e^{-\frac{k\pi\sigma}{P}},$
(3.1)

with $a_{k}$ the coefficients of the Fourier expansion of the respective Rank
function (which contains the ranks of colour and flavour groups). Expanding
this potential in a Taylor series gives

$V_{\text{LQ}}(\sigma,\eta)=\frac{1}{\sigma}\sum^{+\infty}_{k=0}a_{k}-\frac{\pi}{P}\sum^{+\infty}_{k=0}k\,a_{k}+\frac{\pi^{2}}{2P^{2}}\left(-\frac{\eta^{2}}{\sigma}+\sigma\right)\sum^{+\infty}_{k=0}k^{2}\,a_{k}+\frac{\pi^{3}}{6P^{3}}\left(3\eta^{2}-\sigma^{2}\right)\sum^{+\infty}_{k=0}k^{3}\,a_{k}+...$
(3.2)

By defining

$\beta_{1}=\sum^{+\infty}_{k=0}a_{k},\,\,\,\,\beta_{2}=-\frac{\pi}{P}\sum^{+\infty}_{k=0}k\,a_{k},\,\,\,\,\beta_{3}=-\frac{1}{3}\frac{\pi^{2}}{2P^{2}}\sum^{+\infty}_{k=0}k^{2}\,a_{k},\,\,\,\,\beta_{4}=-\frac{1}{4}\frac{\pi^{3}}{6P^{3}}\sum^{+\infty}_{k=0}k^{3}\,a_{k},$
(3.3)

one can check that the truncated expansion completely contains the potential
for the NATD ($\beta_{1}$ does not appear on any background fields nor in the
Rank function, and $\beta_{2}$ appears as a gauge transformation of $C_{2}$).
As for the $\text{AdS}_{6}$ case, this suggest that the $\text{AdS}_{4}$ NATD
corresponds to a zoom-in of a background for which we can completely
characterise the holographic dual. This means that even though the NATD
holographic dual is not properly defined, it can be completed into a 3d
$\mathcal{N}=4$ linear quiver.

###  3.2 Comments on the ATD

The ATD background can be found via a similar zoom-in in the family of
orthogonal functions to (3.1), namely

$V_{\perp}(\sigma,\eta)=\sum^{+\infty}_{k=0}\frac{a_{k}}{\sigma}\sin\left(\frac{k\pi\eta}{P}\right)e^{-\frac{k\pi\sigma}{P}}.$
(3.4)

Notice that, although this is a solution to the equation of motion, it does
not satisfy the boundary conditions of the electrostatic problem. This means
that it is not possible to use the same arguments to suggest that the ATD be
completed into a more general geometry that has a well defined holographic
dual. This is consistent with the fact that the ATD already has a precise
holographic dual: the same one as its seed background.

##  4 A New Supergravity Solution

In this section we investigate the background obtained when using a third
finite part of the infinite Taylor series expansion of $V(\sigma,\eta)$. Note
that, in addition to the previous two examples, it is the only other solution
for which the $\text{AdS}_{4}$ warp factor $f_{1}(\sigma,\eta)$ depends only
on a single coordinate. For a constant $f_{1}(\sigma,\eta)$, the string
background was shown to be integrable for the case of $\text{AdS}_{7}$ in [40]
(where the classical 2d dynamics of the $\sigma$-model can be written in a Lax
pair leading to an infinite number of conserved charges). However, in this
case, a constant $f_{1}(\sigma,\eta)$ is not possible. Hence, an
$\text{AdS}_{4}$ warp factor of a single coordinate is the next best option,
as integrability may occur for fixed values of the coordinate.

Now we present the new solution of Type IIB Supergravity that comes from
considering the following higher order truncation to equation (3.4)

$V_{\text{HO}}=\alpha_{4}\left(4\frac{\eta^{3}}{\sigma}-12\eta\sigma\right).$
(4.1)

Consider the change of coordinates

$\displaystyle\sigma$ $\displaystyle=\frac{2\sqrt{2}}{\pi}~{}r\sin(\theta),$
(4.2) $\displaystyle\eta$
$\displaystyle=\frac{2\sqrt{2}}{\pi}~{}r\cos(\theta).$ (4.3)

After fixing $\alpha_{4}=M/64$, we find

$ds^{2}=r\left(ds^{2}(\text{AdS}_{4})+\frac{\cos(2\theta)}{\tilde{\Delta}}ds^{2}(S^{2}_{1})+2\sin^{2}\left(\theta\right)ds^{2}(S^{2}_{2})+\frac{2}{\cos(2\theta)}\left(\frac{dr^{2}}{r^{2}}+d\theta^{2}\right)\right),$
(4.4)

where $\tilde{\Delta}=2\cos^{2}(\theta)+1$. In order to ensure that the metric
does not change signature, we must make the restriction
$\theta\in\left[0,\frac{\pi}{4}\right]$.

In addition, there is the Dilaton

$e^{-2\Phi}=\frac{9M^{2}}{\pi^{2}}r^{2}\cos(2\theta)\tilde{\Delta},$ (4.5)

and the NS and R forms

$\displaystyle B_{2}$
$\displaystyle=2\sqrt{2}\,\frac{r\cos(\theta)}{\tilde{\Delta}}\text{Vol}(S^{2}_{1}),$
(4.6) $\displaystyle C_{2}$
$\displaystyle=\frac{12}{\pi}Mr^{2}\sin^{3}(\theta)\cos(\theta)\text{Vol}(S^{2}_{2}),$
(4.7) $\displaystyle\tilde{C}_{4}$
$\displaystyle=\frac{6\sqrt{2}}{\pi}Mr^{3}\cos(\theta)\text{Vol}(\text{AdS}_{4}),$
(4.8)

which lead to the following field strengths

$\displaystyle H_{3}$
$\displaystyle=\frac{\sqrt{2}}{\tilde{\Delta}^{2}}\bigg{(}2\cos(\theta)\tilde{\Delta}\,dr+\,r\,\Big{(}\sin(3\theta)-\sin(\theta)\Big{)}\,d\theta\bigg{)}\wedge\text{Vol}(S^{2}_{1}),$
(4.9) $\displaystyle F_{3}$
$\displaystyle=\frac{M}{\pi}\Big{(}24\sin^{3}(\theta)\cos(\theta)\,rdr+12r^{2}(2\tilde{\Delta}-3)\sin^{2}(\theta)\,d\theta\Big{)}\wedge\text{Vol}(S^{2}_{2}),$
(4.10)

and also

$\displaystyle F_{5}$
$\displaystyle=d\left(\frac{6\sqrt{2}}{\pi}Mr^{3}\cos(\theta)\right)\wedge\text{Vol}(\text{AdS}_{4})$
(4.11)
$\displaystyle\phantom{=}-\frac{12\sqrt{2}}{\pi\tilde{\Delta}}M\,r^{2}\cos(2\theta)\sin^{2}(\theta)\Big{(}\sin(\theta)\,dr+3r\cos(\theta)\,d\theta\Big{)}\wedge\text{Vol}(S^{2}_{1})\wedge\text{Vol}(S^{2}_{2}).$

This background has the nice property of a dilatation, in which, under the
transformation $r\rightarrow\lambda r$ (where $\lambda$ is some constant),
everything simply scales independently by differing powers of $\lambda$,
leaving the forms of the background otherwise invariant. This is a slightly
weaker condition to the ATD case, in which the background metric and field
strengths are totally translation invariant. It is interesting to observe
however, that the solution presented here, appearing as a higher order
truncation of equation (3.4), comes from the same family of solutions as the
ATD.

#### Page Charges

In the previous two solutions, the interval of $\mu$ was fixed by
singularities, but we were free to choose the form of the interval of $r$ in
order to ensure quantisation of the Page charges. In the NATD case, this
included splitting $r\in[0,+\infty[$ into an infinite number of finite
‘minimal building block’ intervals. The same needs to be done here, noting the
presence of a singularity at $r=0$. Hence, choosing
$\theta\in[0,\frac{\pi}{4}]$ and $r\in[n\pi,(n+1)\pi]$, with $n\in\mathbb{N}$,
integer quantisation of the Page charges can be achieved.

For the NS5 charge, we integrate in the submanifold $\chi_{1}=(r,S^{2}_{1})$,
fixing $\theta=\pi/4$ to obtain integer quantisation (as taking $\theta=0$
leads to an irrational number of $N_{NS5}$ branes)

$N_{NS5}=\frac{1}{(2\pi)^{2}}\int_{\chi_{1}|_{\theta=\pi/4}}H_{3}=1.$ (4.12)

For the D5 charge, we first integrate in the submanifold
$\chi_{2}=(r,S^{2}_{2})$, fixing $\theta=\pi/4$ (as $\theta=0$ gives zero
charge)

$N_{D5}=\frac{1}{(2\pi)^{2}}\int_{\chi_{2}|_{\theta=\pi/4}}F_{3}=3M(2n+1).$
(4.13)

Finally, for the D3 charge, we integrate in the submanifold
$\chi_{3}=(\theta,S^{2}_{1},S^{2}_{2})$ at fixed $r=n\pi$

$N_{D3}=-\frac{1}{(2\pi)^{4}}\int_{\chi_{3}|_{r=n\pi}}\widehat{F}_{5}=2Mn^{3}.$
(4.14)

The brane configuration for this geometry is shown in figure 4, in which
$2Mn^{3}$ D3-branes are stretched between two NS5-branes located at $n\pi$ and
$(n+1)\pi$, with $3M(2n+1)$ D5-branes in each interval. The corresponding
(overbalanced) linear quiver diagram is then given in figure 5.

$\theta$$r$$\otimes$$\otimes$D5$\otimes$$\otimes$$...$$\otimes$D5$\otimes$$...$$\otimes$D5$\otimes$$...$$\otimes$D5NS5$\pi$NS5$2\pi$NS5$3\pi$NS5$n\pi$NS5$(n+1)\pi$....
. ..... . .D3D3D3 Figure 4: The Hanany-Witten (NS5, D3, D5) brane set-up for
the new solution, in which $2Mn^{3}$ D3-branes are stretched between two
NS5-branes located at $n\pi$ and $(n+1)\pi$, with $3M(2n+1)$ D5-branes in each
interval (taking $M=1$ in the figure). $3M$$9M$$15M$$21M$$2M$$16M$$54M$. . .
Figure 5: The corresponding overbalanced linear quiver diagram for the new
geometry.

##  5 Conclusions

In this paper we used the electrostatic problem formalism developed in [15] to
obtain three AdS${}_{4}\times S^{2}\times S^{2}$ backgrounds in Type IIB
Supergravity, derived from the polynomial solutions of the 2D Laplace equation
defining the geometry. However, they do not satisfy the boundary conditions
that lead to linear quiver field theories. Two of these solutions correspond
to the Abelian and non-Abelian T-duals of the Type IIA background obtained
from the dimensional reduction of AdS${}_{4}\times S^{7}$. We therefore re-
derive the NATD solution using a different approach to [37], characterising
the solutions by specific terms in the polynomial expansion of the potential.
Hence, the dual Field Theory should be more easily determined via this
approach. The third solution is a new Supergravity background, which in
addition to the ATD and NATD cases, form the three backgrounds with the
simplest possible $\text{AdS}_{4}$ warp factor - depending on only a single
coordinate. We then computed the Page charges for each background, and
described the Hanany-Witten brane set-up and linear quiver diagrams for all
three geometries. As in previous references, we find that the NATD brane set-
up is unbounded, making the dual field theory description obscure.

We then showed how the potential leading to the NATD solution emerges as a
truncation of the full solution of the electrostatic problem, which leads to
linear quiver field theories as holographic duals. This agrees with the
picture that NATD solutions can be completed into a background that has a
well-defined holographic dual. The ATD backgrounds are self-consistent on
their own, i.e. since they describe the same dynamics as their seed, they are
dual to the same field theory. The new background has a nice dilatation and
originates from the same general solution as the ATD case.

Interestingly, each term in the expansion of the potential is in itself a
solution, meaning any unique combination will lead to a different background -
with the ATD and NATD cases emerging naturally. The dynamics of the full
$\text{AdS}_{4}$ Type IIB Supergravity solution dual to the linear quivers is
then constructed from the separate constituent dynamics of each term in the
Taylor series.

It is worth noting that, although they are obtained as a truncation of
different families of solutions, the NATD and the new solution have a brane
set-up that are unbounded. It would be interesting to see whether all other
solutions that can be obtained as a truncation share this same property. In
the same line, if one were to find a solution coming from a higer order term
in the Taylor series expansion of the potential that leads to the linear
quiver, one could use the same arguments as for the NATD to argue that those
solutions can be completed. This is not true for the solutions that belong to
the same family as the ATD, such as the new background presented here. Since
these solutions come from a potential that does not satisfy the boundary
conditions of the electrostatic problem, it is not possible to argue that they
can be completed in a similar way, with the ATD being an exception to this
argument. It is then yet to be seen whether those solutions can be completed
into a background with a well-defined holographic dual.

## Acknowledgments

The authors thank Mohammad Akhond, Andrea Legramandi, Carlos Nuñez and Lucas
Schepers for useful discussions and guidance. The work of R.S. is supported by
STFC grant ST/W507878/1.

## References

  * [1] J. M. Maldacena, “The Large N limit of superconformal field theories and supergravity,” Adv. Theor. Math. Phys. 2, 231-252 (1998) [arXiv:hep-th/9711200 [hep-th]].
  * [2] Y. Lozano, C. Nunez, A. Ramirez and S. Speziali, “New AdS2 backgrounds and $\mathcal{N}$ = 4 conformal quantum mechanics,” JHEP 03 (2021), 277 [arXiv:2011.00005 [hep-th]].
  * [3] D. Corbino, “Warped $AdS_{2}$ and $SU(1,1|4)$ symmetry in Type IIB,” [arXiv:2004.12613 [hep-th]].
  * [4] Y. Lozano, C. Nunez, A. Ramirez and S. Speziali, “AdS2 duals to ADHM quivers with Wilson lines,” JHEP 03 (2021), 145 [arXiv:2011.13932 [hep-th]].
  * [5] Y. Lozano, C. Nunez and A. Ramirez, “$\text{AdS}_{2}\times\text{S}^{2}\times\text{CY}_{2}$ solutions in Type IIB with 8 supersymmetries,” JHEP 04 (2021), 110 [arXiv:2101.04682 [hep-th]].
  * [6] N. T. Macpherson, “Type II solutions on AdS${}_{3}\times$ S${}^{3}\times$ S3 with large superconformal symmetry,” JHEP 05 (2019), 089 [arXiv:1812.10172 [hep-th]].
  * [7] A. Legramandi and N. T. Macpherson, “AdS3 solutions with from $\mathcal{N}=(3,0)$ from S${}^{3}\times$S3 fibrations,” Fortsch. Phys. 68 (2020) no.3-4, 2000014 [arXiv:1912.10509 [hep-th]].
  * [8] A. Legramandi, G. Lo Monaco and N. T. Macpherson, “All $\mathcal{N}=(8,0)$ AdS3 solutions in 10 and 11 dimensions,” JHEP 05 (2021), 263 [arXiv:2012.10507 [hep-th]].
  * [9] Y. Lozano, N. T. Macpherson, C. Nunez and A. Ramirez, “AdS3 solutions in Massive IIA with small $\mathcal{N}=(4,0)$ supersymmetry,” JHEP 01 (2020), 129 [arXiv:1908.09851 [hep-th]].
  * [10] Y. Lozano, N. T. Macpherson, C. Nunez and A. Ramirez, “Two dimensional ${\cal N}=(0,4)$ quivers dual to AdS3 solutions in massive IIA,” JHEP 01, 140 (2020) [arXiv:1909.10510 [hep-th]].
  * [11] Y. Lozano, N. T. Macpherson, C. Nunez and A. Ramirez, “1/4 BPS solutions and the AdS3/CFT2 correspondence,” Phys. Rev. D 101, no.2, 026014 (2020) [arXiv:1909.09636 [hep-th]].
  * [12] E. D’Hoker, J. Estes and M. Gutperle, “Exact half-BPS Type IIB interface solutions. I. Local solution and supersymmetric Janus,” JHEP 06 (2007), 021 doi:10.1088/1126-6708/2007/06/021 [arXiv:0705.0022 [hep-th]].
  * [13] E. D’Hoker, J. Estes and M. Gutperle, “Exact half-BPS Type IIB interface solutions. II. Flux solutions and multi-Janus,” JHEP 06 (2007), 022 [arXiv:0705.0024 [hep-th]].
  * [14] B. Assel, C. Bachas, J. Estes and J. Gomis, “Holographic Duals of D=3 N=4 Superconformal Field Theories,” JHEP 08 (2011), 087 [arXiv:1106.4253 [hep-th]].
  * [15] M. Akhond, A. Legramandi and C. Nunez, “Electrostatic description of 3d $\mathcal{N}$ = 4 linear quivers,” JHEP 11 (2021), 205 [arXiv:2109.06193 [hep-th]].
  * [16] D. Gaiotto and J. Maldacena, “The Gravity duals of N=2 superconformal field theories,” JHEP 10 (2012), 189 [arXiv:0904.4466 [hep-th]].
  * [17] R. A. Reid-Edwards and B. Stefanski, jr., “On Type IIA geometries dual to N = 2 SCFTs,” Nucl. Phys. B 849 (2011), 549-572 [arXiv:1011.0216 [hep-th]].
  * [18] O. Aharony, L. Berdichevsky and M. Berkooz, “4d N=2 superconformal linear quivers with type IIA duals,” JHEP 08 (2012), 131 [arXiv:1206.5916 [hep-th]].
  * [19] E. D’Hoker, M. Gutperle, A. Karch and C. F. Uhlemann, “Warped $AdS_{6}\times S^{2}$ in Type IIB supergravity I: Local solutions,” JHEP 08 (2016), 046 [arXiv:1606.01254 [hep-th]].
  * [20] E. D’Hoker, M. Gutperle and C. F. Uhlemann, “Holographic duals for five-dimensional superconformal quantum field theories,” Phys. Rev. Lett. 118 (2017) no.10, 101601 [arXiv:1611.09411 [hep-th]].
  * [21] E. D’Hoker, M. Gutperle and C. F. Uhlemann, “Warped $AdS_{6}\times S^{2}$ in Type IIB supergravity II: Global solutions and five-brane webs,” JHEP 05 (2017), 131 [arXiv:1703.08186 [hep-th]].
  * [22] A. Legramandi and C. Nunez, “Electrostatic Description of Five-dimensional SCFTs,” [arXiv:2104.11240 [hep-th]].
  * [23] A. Legramandi and C. Nunez, “Holographic description of SCFT5 compactifications,” [arXiv:2109.11554 [hep-th]].
  * [24] F. Apruzzi, M. Fazzi, D. Rosa and A. Tomasiello, “All AdS7 solutions of type II supergravity,” JHEP 04 (2014), 064 [arXiv:1309.2949 [hep-th]].
  * [25] D. Gaiotto and A. Tomasiello, “Holography for (1,0) theories in six dimensions,” JHEP 12 (2014), 003 [arXiv:1404.0711 [hep-th]].
  * [26] C. Núñez, J. M. Penín, D. Roychowdhury and J. Van Gorsel, “The non-Integrability of Strings in Massive Type IIA and their Holographic duals,” JHEP 06 (2018), 078 [arXiv:1802.04269 [hep-th]].
  * [27] X. C. de la Ossa and F. Quevedo, “Duality symmetries from nonAbelian isometries in string theory,” Nucl. Phys. B 403, 377-394 (1993) [arXiv:hep-th/9210021 [hep-th]].
  * [28] K. Sfetsos and D. C. Thompson, “On non-abelian T-dual geometries with Ramond fluxes,” Nucl. Phys. B 846, 21-42 (2011) [arXiv:1012.1320 [hep-th]].
  * [29] Y. Lozano, E. O Colgain, K. Sfetsos and D. C. Thompson, “Non-abelian T-duality, Ramond Fields and Coset Geometries,” JHEP 06, 106 (2011) [arXiv:1104.5196 [hep-th]].
  * [30] Ö. Kelekci, Y. Lozano, N. T. Macpherson and E. Ó. Colgáin, “Supersymmetry and non-Abelian T-duality in type II supergravity,” Class. Quant. Grav. 32, no.3, 035014 (2015) [arXiv:1409.7406 [hep-th]].
  * [31] A. Giveon and M. Rocek, “On nonAbelian duality,” Nucl. Phys. B 421 (1994), 173-190 [arXiv:hep-th/9308154 [hep-th]].
  * [32] M. Gasperini, R. Ricci and G. Veneziano, “A Problem with nonAbelian duality?,” Phys. Lett. B 319, 438-444 (1993) [arXiv:hep-th/9308112 [hep-th]].
  * [33] S. Elitzur, A. Giveon, E. Rabinovici, A. Schwimmer and G. Veneziano, “Remarks on nonAbelian duality,” Nucl. Phys. B 435, 147-171 (1995) [arXiv:hep-th/9409011 [hep-th]].
  * [34] G. Itsios, C. Nunez, K. Sfetsos and D. C. Thompson, “On Non-Abelian T-Duality and new N=1 backgrounds,” Phys. Lett. B 721, 342-346 (2013) [arXiv:1212.4840 [hep-th]].
  * [35] G. Itsios, C. Nunez, K. Sfetsos and D. C. Thompson, “Non-Abelian T-duality and the AdS/CFT correspondence:new N=1 backgrounds,” Nucl. Phys. B 873, 1-64 (2013) [arXiv:1301.6755 [hep-th]].
  * [36] Y. Lozano and C. Núñez, “Field theory aspects of non-Abelian T-duality and $\mathcal{N}=$ 2 linear quivers,” JHEP 05, 107 (2016) [arXiv:1603.04440 [hep-th]].
  * [37] Y. Lozano, N. T. Macpherson, J. Montero and C. Nunez, “Three-dimensional $\mathcal{N}=4$ linear quivers and non-Abelian T-duals,” JHEP 11, 133 (2016) [arXiv:1609.09061 [hep-th]].
  * [38] A. Hanany and E. Witten, “Type IIB superstrings, BPS monopoles, and three-dimensional gauge dynamics,” Nucl. Phys. B 492 (1997), 152-190 [arXiv:hep-th/9611230 [hep-th]].
  * [39] B. Assel, C. Bachas, J. Estes and J. Gomis, “Holographic Duals of $D=3$ $\mathcal{N}=4$ Superconformal Field Theories”, JHEP 1108, 087 (2011) [arXiv:1106.4253 [hep-th]].
  * [40] K. Filippas, C. Núñez and J. Van Gorsel, “Integrability and holographic aspects of six-dimensional $\mathcal{N}=\left(1,\ 0\right)$ superconformal field theories,” JHEP 06, 069 (2019) doi:10.1007/JHEP06(2019)069 [arXiv:1901.08598 [hep-th]].