# ProCC: Progressive Cross-primitive Consistency for Open-World
Compositional Zero-Shot Learning

Fushuo Huo1, Wenchao Xu1, Song Guo1, Jingcai Guo1, Haozhao Wang2, and Ziming
Liu1
1Department of Computing, The Hong Kong Polytechnic University
2School of Computer Science and Technology, Huazhong University of Science and
Technology

###### Abstract

Open-World Compositional Zero-shot Learning (OW-CZSL) aims to recognize novel
compositions of state and object primitives in images with no priors on the
compositional space, which induces a tremendously large output space
containing all possible state-object compositions. Existing works either learn
the joint compositional state-object embedding or predict simple primitives
with separate classifiers. However, the former heavily relies on external word
embedding methods, and the latter ignores the interactions of interdependent
primitives, respectively. In this paper, we revisit the primitive prediction
approach and propose a novel method, termed Progressive Cross-primitive
Consistency (ProCC), to mimic the human learning process for OW-CZSL tasks.
Specifically, the cross-primitive consistency module explicitly learns to
model the interactions of state and object features with the trainable memory
units, which efficiently acquires cross-primitive visual attention and avoids
cross-primitive feasibility scores. Moreover, considering the partial-
supervision setting (pCZSL) as well as the imbalance issue of multiple tasks
prediction, we design a progressive training paradigm to enable the primitive
classifiers to interact to obtain discriminative information in an easy-to-
hard manner. Extensive experiments on three widely used benchmark datasets
demonstrate that our method outperforms other representative methods on both
OW-CZSL and pCZSL settings by large margins. 111The Codes will be released..

##  1 Introduction

Humans can extrapolate new concepts from previously learned knowledge. For
instance, if the people are taught what the _fried chip_ and _toasted bread_
are, most of them can recognize the _fried bread_ immediately. This ability is
known as _compositional generalization_ [1], which is one of the ultimate
targets for artificial intelligence. In the literature, such a task is
formulated as _Compositional Zero-Shot Learning_ (CZSL). Concretely, the
training set contains images with corresponding descriptions (primitives),
i.e., state and object. The model is expected to recognize unseen compositions
based on known primitives, which is non-trivial because object and state are
semantically tangled, i.e., objects in different states often have different
appearances, and states can vary greatly conditioned on different objects. The
major challenge behind the CZSL lies in how to model the interactions between
state and object primitives and extrapolate seen compositions to unseen ones.
Existing methods mainly focus on learning a shared embedding space for object-
state compositions [25, 32, 33, 17] or compositional attribute and object
classifiers [37, 31, 24, 47, 49].

However, the performances of these methods degrade to some extent [28, 29] as
for the open-world setting (OW-CZSL), where there are no priors on the unseen
compositions and the model must consider the whole possible compositions in
terms of all objects and states. To deal with such a problem, existing
mainstream methods utilize feasibility constraints on the composition
embedding [28, 29] or independently predict simple state and object primitives
[16]. While [28, 29] somewhat rely on different word embedding methods and
embedding dimensions also matter. The straightforward but effective method
[16] predicts the state and object primitives while ignoring the interaction
and consistency between two primitives. So external knowledge is introduced to
eliminate less feasible compositions, while it is cumbersome to select proper
external knowledge for the varying datasets.

Figure 1: The overall concept of our method. Following the principle of
‘forest before trees’ [10], human feedforward hierarchy underlies implicit
processing for initial vision at a glance, and feedback connections add
details to explicit vision with scrutiny. As for composition generalization
learning, humans first learn to recognize overall objects, then gradually
identify the scrutiny attribute of objects, i.e., state, and finally
reasonably compose the object and state primitive. Inspired by this, we aim to
progressively recognize the object and state primitives and guide the network
to discriminative information conditioned on learned knowledge via the CPC
module.

To address aforementioned problems, we propose Progressive Cross-primitive
Consistency (ProCC) network to recognize compositions in the open-world
setting and a more realistic setting (i.e., partial supervision), aiming at
attaining cross-primitive consistency during easy-hard recognition progress,
as shown in Figure 1. Specifically, following the route of the human learning
process, we first learn to classify objects, which is easier than recognizing
states because the same state varies greatly conditioned on objects and
related contexts, i.e., _ancient castle_ / _ancient coin_ , and different
states are sometimes less feasible composed with the same object, i.e., _old
dog_ / _ripe dog_. Then, with the learned knowledge of object primitive, we
sequentially classify state primitives conditioned on guided object features
via Cross-Primitive Consistency (CPC) module, excavating discriminative
information. Finally, we finetune the whole network conditioned on each other
prior knowledge. The ProCC network achieves the cross-primitive consistency by
adjusting the visual attention via the CPC module, without the aid of external
knowledge like Word2vec [30], Glove [35], Conceptnet [42] etc. Also, the
progressive training paradigm effectively model the interactions of primitives
via trained features in pCZSL and alleviates the imbalance (over/under-
fitting) learning of state and object classifiers compared with the jointly
training.

In summary, our contributions are four-fold:

1) We propose a novel Progressive Cross-primitive Consistency (ProCC) network,
mimicking the human learning progress of recognizing the state and object
compositions without external knowledge.

2) We present a Cross-Primitive Consistency (CPC) module consisting of cross-
primitive memory units to model the interactions of classifiers to focus the
discriminative visual attention conditioned on each other, guiding the model
to generalize to feasible compositions.

3) The progressive training paradigm alleviates the negative effect of invalid
cross-primitive interactions in pCZSL without other cumbersome pseudo-label
methods, as well as the imbalance issue of learning multiple tasks.

4) Comprehensive experimental results on three large-scale datasets for OW-
CZSL and pCZSL tasks demonstrate the effectiveness of our proposed approach,
which outperforms the state-of-the-art methods.

##  2 Related Work

Compositional zero-shot learning. Compositional Zero-shot Learning (CZSL) aims
to recognize the state and object from the images, and even the state-object
compositions are not ever seen in the training datasets. Different from
typical zero-shot learning [46, 13, 23, 5, 26, 6], which aims to utilize
attributed vectors or inherent semantic descriptions to recognize unseen
instances, The main challenge of CZSL is modeling the relation and affordance
of states and objects, generalizing this capability to unseen compositions.
Existing methods mainly deal with CZSL in two ways. The first way is inspired
by Biederman’s Recognition-ByComponents theory[3] and Hoffman’s part theory
[11]. For instance, Misra _et al._ [31] learns a transformation between
individual classifiers of states and objects. Other representative methods
learn a hierarchical decomposition and composition of the state and object
primitives [48], model objects to be symmetric under attribute transformations
[25], and learn independent prototypical representations of visual primitives
then propagated prototype via a compositional graph [39]. The second way tries
to learn the joint representation of the state-object compositions from given
images [25, 32, 33, 17]. Similarly, [25] enforces symmetries in the
representation of objects given their state transformations. Graph network is
also employed to enforce the compositional information transfer from seen to
unseen compositions. Nagarajan _et al._[33] regard attribute as the operator
and model each state as a linear transformation of objects. [17] exploit the
self-attention mechanism to discover the complex interdependency structure
between compositions and propagate it to unseen ones. Differently, causality-
based methods [2, 49] explore decomposable objects and state representations.

These methods perform well on the close-world CZSL, while suffering from
severe degradation for the open-world setting [28, 29, 16], where the output
space has not imposed any limit. Mancini _et al,_ [28] compute feasibility
scores (i.e., cosine similarity) between visual features and compositional
embeddings to reduce the output space. Then they further inject the
feasibility scores both at the loss level and within the graph connections
[29]. [16] follows the Visual Product [31] and predicts state and object
primitives independently with non-linear feature extractors. To refine the
relation between independent primitives, Conceptnet [42] is introduced as the
external knowledge. We revisit the Visual Product and achieve cross-primitive
consistency in an easy-hard learning manner, avoiding the external knowledge
in [16] and cumbersome word embeddings in [28, 29].

Multi-task consistency. Our method predicts state and object primitives and
therefore has a close relationship to Multi-Task Learning (MTL) [38, 44]. A
rich body of work [40, 51, 27, 22] study the relation and consistency between
tasks in MTL. Saha _et al._ [40] encodes the relationships between semantic
segmentation and depth prediction to improve model performance. Zamir _et al._
[51] learns with cross-task consistency and leads to more accurate predictions
and out-of-distribution generalization. [27] proposes the consistency loss to
jointly train multiple tasks to benefit each other, reducing the need for
labeled data. [22] further solve the partial-supervision MTL task via cross-
task consistency in joint space. However, different from [27, 22], which apply
strong cross-task consistency constraints to the decision level in terms of
dense prediction tasks, we apply soft cross-primitive consistency in the
feature level for sparse prediction tasks, avoiding over-fitting degradation.

Conditional model. To mitigate the domain diversity or informatively adjust
feature weight, some conditional model has been proposed [36, 8, 52, 7, 54].
For instance, Perez _et al._[36] develops an image feature modulation method
conditioned on input words for visual reasoning. Zhao _et al._ [8] predicts a
set of adaptive weights from conditional inputs to formulate specific filters,
which facilitates parameter learning across different conditions for few-shot
learning. The conditional model is also employed in the image restoration
tasks [52, 7], which learn the conditional vector to control the enhancement
degree. Recently, Zhou _et al._ [54] generated the input-conditional token
from given images to improve the generalization ability of the pre-trained
vision-language model. Inspired by them, the Cross-Primitive Consistency (CPC)
module of ProCC can also be viewed as the layers conditioned on state and
object primitives, which facilitates the model generalizing to new
compositions with high feasibility.

##  3 Approach

###  3.1 Problem formulation

Compositional Zero-Shot Learning (CZSL) aims to recognize the composition of
two primitive concepts, i.e., an state ($e.g.,tiny$) and an object (e.g.,
$dog$). Given $S$ and $O$ as two sets of states and objects, we compose a set
of possible state-object pairs, i.e., $C=S\times O=\left\\{{(s,o)\left|{s\in
S,o\in O}\right.}\right\\}$. Formally, given a training set
${D^{s}}=\\{(i,c)|i\in{I^{s}},c\in{C^{s}}\\}$, where $I^{s}$ is an training
image set, and $C^{s}$ is the corresponding state-object labels. The close
world CZSL follows the generalized ZSL [46] that the test sample comes from
either seen ($C^{s}$) or unseen ($C^{u}$) composition ($C^{s}\cup C^{u}$). For
the Open-World CZSL (OW-CZSL) setting [28], there assumes no prior on the set
of testing compositions. It means the model must consider the full
compositional space ($C$), which is much larger than $C^{s}\cup C^{u}$.
Consequently, the unseen compositions are $C_{u}=C\backslash C^{s}$. OW-CZSL
introduces a more practical setting while bringing more challenging problems:
1) It is hard to generalize from small seen compositions to large unseen
compositions. 2) There are a large number of less feasible compositions in the
full composition space ($C$), confusing the prediction models. Recently, [16]
proposes a new practical setting, i.e., only training with one of the state
and object annotations, named partial-supervision CZSL (pCZSL). Formally, for
the training set $C^{s}$, The relation of the partial label of state and
object primitives can be formulated as: $\\{(s,u)\\}\cup\\{(u,o)\\}={C^{s}}$,
where $u$ indicates unlabeled primitives. Consequently, the test set in pCZSL
has the full output composition space ($C$) like OW-CZSL, while the training
set in pCZSL does not have the composition knowledge about any state-object
pairs. Therefore, the joint training strategy may fail due to lacking the
explicit supervision to learn how states interact with objects and vice-versa.

Figure 2: The framework of ProCC. Features from the encoder ($\omega$) are
respectively fed to the object and state ($\varphi_{o}$ and $\varphi_{s}$)
classifiers, where the Cross-Primitive Consistency (CPC) modules aim to model
the cross primitives interactions. The progressive learning strategy is
proposed to handle the partial supervision and imbalance MTL issue. The
detailed training procedure can refer to Algorithm1.

###  3.2 Progressive Cross-primitive Consistency

Most CZSL methods [2, 25, 33, 37, 41, 28, 29, 32] explicitly modulate the
interactions of states and objects to improve the generalization ability.
While it is less effective for OW-CZSL and pCZSL due to large output space and
missing labels. Some methods [15, 16] follow the Visual Product [31] that
independently predict the state and object primitives, disregarding
compositional nature. Following the route of [15, 16, 31], we propose
Progressive Cross-primitive Consistency (ProCC) network while achieving cross-
primitive consistency. Also, like the human learning process, ProCC trains the
network in an easy-hard manner, which dynamically models interactions between
state and objects, alleviating the negative influence of no explicit
supervision on both states and objects in pCZSL and imbalance issues of MTL.
Figure 2 shows the framework of the proposed approach.

In the following subsections, we revisit the Visual Product and introduce a
cross-primitive consistency module and progressive learning strategy.

Revisit Visual Product. In a nutshell, given an image $i$, CZSL wants to model
the joint probability distribution $p(s,o|i)$. The visual product simplifies
this as:

$\displaystyle p(s,o|i)\approx p(s|i)\times p(o|i)$ (1)

In this way, Visual Product treats the states and objects independently only
from the visual cues, without side information (i.e., word embeddings).
Concretely, the image $i$ is firstly encoded to obtain the feature
representation $z$ as: $z=\omega(i)$, mapping images to a feature space $Z$,
i.e., $\omega\colon X\rightarrow Z$. Then the object ($\varphi_{o}\colon
Z\rightarrow\delta_{o}$) and state ($\varphi_{s}\colon
Z\rightarrow\delta_{s}$) classifiers assign $z$ to the vectors in the
probability simplex $\delta_{o}$ and $\delta_{s}$, spanning all object and
state classes. The loss function for object and state tasks is as follows:

$\displaystyle{\ell_{obj}}=\mathop{\min}\limits_{\theta}\sum\limits_{(i,(o,s))\in{C^{s}}}{{\ell_{ce}}({\varphi_{o}}(\omega(i)),o)}$
(2)

$\displaystyle{\ell_{state}}=\mathop{\min}\limits_{\theta}\sum\limits_{(i,(o,s))\in{C^{s}}}{{\ell_{ce}}({\varphi_{s}}(\omega(i)),s)}$
(3)

The final loss for the Visual Product is:

$\displaystyle{\ell_{visprod}}={\ell_{obj}}+{\ell_{state}}$ (4)

During inference, we predict the output composition as:

$\displaystyle{f_{\theta}}(i)=\arg\mathop{\max}\limits_{(s,o)\in
C}{\varphi_{s}}(\omega(i),s)\times{\varphi_{o}}(\omega(i),o)$ (5)

where $C$ represents all the state-object composition pairs in OW-CZSL. As the
search space is huge, Visual Product is more effective than previous methods,
which aim to produce discriminative state-object embeddings [15, 16].
Recently, [15, 16] expanded the visual product and equipped the classifiers
with multi-layer perceptrons (MLP) to excavate discriminative features. Also,
external knowledge [42] is employed in [16] to estimate the feasibility scores
of compositions. Here, we explicitly model the composition interactions via
Cross-Primitive Consistency (CPC) module during the training procedure,
without external knowledge. Also, considering the pCZSL setting and unbalanced
multi-task learning (i.e., recognizing state is more challenging than object),
where joint training may obtain sub-optimal results, the progressive learning
strategy is proposed to deal with it in an easy-hard manner.

Figure 3: The detailed framework of the Cross-Primitive Consistency (CPC).
Take the object-state CPC (CPCo→s) for example, the features from the object
classifier ($\varphi_{o-1}$ and $\varphi_{o-2}$) are modulated by Cross-
Primitive Memory (CPM) unit with the learnable parameters. Then respectively
fused with the state features to achieve consistency of state and object
primitives.

Cross-primitive consistency module. Visual Product methods independently
predict compositions via Equation 1, which ignores the fact that the
feasibility of state-object compositions is heavily conditioned on each other.
The more practical compositional probability can be modeled as follows:

$\displaystyle p(s,o|i)\approx p(s|i,f{{}_{o}}(i)))\times
p(o|i,f{{}_{s}}(i)))$ (6)

where $f{{}_{o}}(i)$ and $f{{}_{s}}(i)$ are intermediate features of the
object and state primitives. It is non-trivial to directly model the
relationship between objects and states due to the diverse semantic
entanglement and a large amount of possible compositions. We integrate the
feasibility reasoning into the trainable Cross-Primitive Consistency (CPC)
module, which facilitates interactions between two classifiers to explore
informative visual attention conditioned on feature representations of each
primitive. Specifically, The features extracted by the encoder ($\omega$) are
fed to multi-layer perceptron (MLP) (i.e., $\varphi_{o}$ and $\varphi_{s}$) to
enhance the feature representation. Take the object-state primitive
consistency ($C_{o\rightarrow s}$) module for example, as shown in Figure 3,
intermediate features from $\varphi_{o-1}$ and $\varphi_{o-2}$ are fed to
$\varphi_{s}$ to interact with state features. However, direct modulation
state features will induce information degradation because of the huge task
diversity. Inspired by the condition model [36, 8, 52, 7, 54], we propose a
learnable Cross-Primitive Memory (CPM) unit to learn the external cross-
primitive relationship. Then, similar to channel attention [12, 45], the
learnable CPM unit modulate corresponding features along with the residual
connection. The whole process is formulated as follows:

$\displaystyle\varphi_{o-l}^{m}=\sigma\left({{\rm{Conv}}_{\rm{{1d}}}^{k}\left({{\varphi_{o-l}}}\right)}\right),{\rm{}}l\in\left({1,2}\right)$
(7)
$\displaystyle\varphi_{s-l}^{,}={\varphi_{s-l}}\times\varphi_{o-l}^{m}+{\varphi_{s-l}},{\rm{}}l\in\left({1,2}\right)$
(8)

where ${{\rm{Conv}}_{\rm{1d}}^{k}}$ and $\sigma$ represent the $\rm{1d}$
convolution layer and softmax activation function. Kernel size ($k$) is equal
to 1/10 feature dimension to efficiently capture the long-range dependency.
Then the enhanced state features are fed to the next layer of $\varphi_{s}$
as:

$\displaystyle{\varphi_{s-(l+1)}}={f_{s-l}}(W_{s-l}^{T}\varphi_{s-l}^{,}+{b_{s-l}}),{\rm{}}l\in(1,2),$
(9)

where $W$ and $b$ are the weights and biases of MLP layers. Accordingly, the
conditioned cross-primitive interactions are injected to each other, reducing
the less feasible primitive predictions.

Figure 4: Visualizations of class activation maps of ProCC with (w/) and
without (w/o) CPC modules on the testing dataset of MIT-States. The
discriminative regions are marked with red rectangles.

Visual explanation. To further illustrate and explain the effect of the CPC
module, we visualize the attention learned from the classifier via Class
Activation Map (CAM) [53] in Figure 4. The standard CAM can be formulated as:

$\displaystyle{\rm{CA}}{{\rm{M}}_{c}}(x,y)=\sum\limits_{k}{\omega_{k}^{c}{f_{k}}(x,y)}$
(10)

where $\rm{CAM}_{c}$ means the class activation map that leads to the
classification of an image to class $c$. ${f_{k}}(x,y)$ and $\omega_{k}^{c}$
stand for the activation of unit $k$ in the last layer at spatial location
$(x,y)$ and the weight corresponding to class c for unit $k$. Here,
$\omega_{k}^{c}$ is the final layer of the MLP (i.e., ${\varphi_{o-3}}$ and
${\varphi_{s-3}}$), which has been modulated by the CPC modules. Figure 4
shows some visualization examples with (w/) and without (w/o) CPC module. As
the encoder is pre-trained for the object classification task, most CAMs for
the object classifier can locate and recognize the proper attention regions.
However, the CAMs for the state classifier vary greatly as state primitives
are conditioned on the object primitive and related contexts. For the “$tiny$
$dog$” and “$large$ $dog$” compositions, the CPC module drives the model to
focus on the discriminative regions that a dog with a small head compared with
other objects tends to classify to the “$tiny$” otherwise classify to
“$huge$”. For more abstract compositions, “$broken$ $bridge$” and “$ripe$
$banana$” compositions, the state primitives heavily depend on the object
primitives otherwise may induce less feasibility compositions. The state of
“$broken$” is mainly reflected in the curvatures of the bridge and the
“$ripe$” primitive of the banana displays the black spots on the surface.
Overall, the CPC module enables the efficient adjustment of visual attention
conditioned on mutual primitives.

Input: Training data ${D^{s}}=\\{(i,c)|i\in{I^{s}},c\in{C^{s}}\\}$, pre-
trained $\omega$, learning rate $\lambda_{1}$, $\lambda_{2}$, $\lambda_{3}$

Output: Optimal $\varphi_{o}$, $\varphi_{s}$, CPC: $\varphi_{o\to s}$,
$\varphi_{o\to s}$

1Initialize: $\varphi_{o}$, $\varphi_{s}$, $\varphi_{o\to s}$, $\varphi_{o\to
s}$;

2Stage 1: _// train $\varphi_{o}$_

3while  _not converged_ do

4 Sample a batch from $D^{s}$ as images ${(i_{k})}_{k=1}^{n}$ with their
object labels ${(o_{k})_{k=1}^{n}}$ ;

5

6 for  _samples in the batch_ do

7 Compute $\ell_{obj}$ via Equation 2.;

8 Update
${\varphi_{o}}\leftarrow{\varphi_{o}}+\lambda_{1}{\nabla_{{\varphi_{o}}}}{\ell_{obj}}$

9 end for

10

11 end while

12

13Stage 2: _// train $\varphi_{s}$ and $\varphi_{o\to s}$_

14while  _not converged_ do

15 Sample a batch from $D^{s}$ as images ${(i_{k})}_{k=1}^{n}$ with their
state labels ${(s_{k})_{k=1}^{n}}$ ;

16

17 for  _samples in the batch_ do

18 Compute $\ell_{state}$ via Equation 3.;

19 Update ${\varphi_{s\cup o\to s}}\leftarrow{\varphi_{s\cup o\to
s}}+\lambda_{2}{\nabla_{{\varphi_{s\cup o\to s}}}}{\ell_{state}}$

20 end for

21

22 end while

23

24Stage 3: _// finetune $\varphi_{o}$, $\varphi_{s}$, $\varphi_{s\to o}$, and
$\varphi_{o\to s}$_

25while  _not converged_ do

26 Sample a batch from $D^{s}$ as images ${(i_{k})}_{k=1}^{n}$ with their
object and state labels ${(o_{k},s_{k})_{k=1}^{n}}$ ;

27

28 for  _samples in the batch_ do

29 Compute $\ell_{visprod}$ via Equation 4.;

30 Update
${\varphi_{total}}\leftarrow{\varphi_{total}}+\lambda_{3}{\nabla_{{\varphi_{total}}}}{\ell_{visprod}}$

31 end for

32

33 end while

Algorithm 1 Training procedure of ProCC.

Progressive learning strategy. However, jointly training the state and object
classifiers may induce two issues: 1) When it comes to the more practical
setting, partial supervision Compositional Zero-Shot Learning (pCZSL), where
only the partial label, not both, is available [16]. The missing label makes
the joint training strategy invalid to model the interactions between the
state and object primitives. A naive way of learning from such partial
supervision is to update the parameters of state and object classifier only
based on the available labels, which lacks the interaction information across
primitives via the CPC module. Recent method [16] estimates the missing labels
via pseudo-labeling [20] as well as utilizes the external knowledge [42]. The
challenge of missing labels also exists in the standard MTL task that the
traditional updating rule will give inferior results due to the missing
annotations [44, 18, 34, 22]. Some typical solutions propose hard knowledge
distillation [18], alternative optimization strategy [34], and learning in the
joint pairwise task spaces [22]. However, compared with the MTL task, the
missing label issue matters more to the CZSL task, as the object and state
primitives are heavily tangled. 2) Also, classifying attributes is more
challenging than objects [41, 16]. This is due to the fact that image encoder
$\omega$ is pre-trained for object classification, not for the more semantic
state classification. Meanwhile, the state primitive is highly conditioned on
the object classification results. Therefore, joint training makes the object
classifier over-fitting or makes the state classifier under-fitting.

To enable the full interaction of state and object primitives and alleviate
the imbalance issue of MTL, we propose a progressive learning strategy,
mimicking the easy-hard learning process shown in Figure 1. Concretely, with
the features from the encoder ($\omega$), we first train the object classifier
$\varphi_{o}$ with given labels via $\ell_{obj}$, to obtain trained object
features from classifier ($\varphi_{o-l}$, $x\in(1,2)$). Then we sequentially
train the state classifier $\ell_{state}$ conditioned on well-trained object
features ($\varphi_{o-l}$) through the CPC module $\varphi_{o\rightarrow s}$,
to interact to adjust the visual attention. Finally, with the pre-trained
object and state classifier, we fine-tune the state and object classifiers as
well as CPC modules conditioned on the mutual information. We utilize this
training protocol both in the OW-CZSL and pCZSL settings. During the easy-hard
recognition progress, our method enables the interactions of cross primitives
in the pCZSL setting without external knowledge and alleviates imbalance issue
with the early stop strategy at each stage. The detailed training procedure of
our network can refer to Algorithm 1.

Method | C-GQA | MIT-States | UT-Zappos
---|---|---|---
Val | Test | Val | Test | Val | Test
HM | AUC | $S$ | $U$ | HM | AUC | HM | AUC | $S$ | $U$ | HM | AUC | HM | AUC | $S$ | $U$ | HM | AUC
TMN[37] | NA | NA | NA | NA | NA | NA | 2.1 | 0.2 | 12.6 | 0.9 | 1.2 | 0.1 | 21.2 | 9.2 | 55.9 | 18.1 | 21.7 | 8.4
AoP[33] | NA | NA | NA | NA | NA | NA | 3.2 | 0.3 | 16.6 | 5.7 | 4.7 | 0.7 | 23.4 | 10.1 | 50.9 | 34.2 | 29.4 | 13.7
LE+ [31] | 9.3 | 1.8 | 19.2 | 0.7 | 1.0 | 0.08 | 5.3 | 0.5 | 41.2 | 2.5 | 2.7 | 0.3 | 26.6 | 14.3 | 60.4 | 36.5 | 30.5 | 16.3
VisProd [31] | 10.5 | 2.0 | 24.8 | 1.7 | 2.8 | 0.33 | 7.2 | 1.0 | 20.9 | 5.8 | 5.6 | 0.7 | 28.8 | 15.4 | 54.6 | 42.8 | 36.9 | 19.7
SymNet [25] | 12.3 | 2.5 | 26.7 | 2.2 | 3.3 | 0.43 | 8.0 | 1.2 | 21.4 | 7.0 | 5.8 | 0.8 | 32.5 | 16.7 | 53.3 | 44.6 | 34.5 | 18.5
CGE [32] | 12.8 | 2.8 | 28.3 | 1.3 | 2.2 | 0.30 | 8.3 | 1.8 | 29.6 | 4.0 | 4.9 | 0.7 |  34.5 | 18.9 | 58.8 | 46.5 | 38.0 | 21.5
CompCos [28] | 12.0 | 2.4 | 28.4 | 1.8 | 2.8 | 0.39 |  8.4 | 1.5 | 25.4 | 10.0 |  8.9 |  1.6 | 32.5 | 18.1 |  59.3 | 46.8 | 36.9 | 21.3
Co-CGE [29] | 12.3 | 2.7 |  28.7 | 1.6 | 2.6 | 0.37 |  8.4 | 2.1 | 26.4 |  10.4 | 10.1 | 2.0 | 34.8 | 19.2 | 60.1 | 44.3 | 38.1 | 21.3
KGSP [16] |  13.2 |  2.9 | 26.6 |  2.1 |  3.4 |  0.44 | 7.9 | 1.4 | 23.4 | 7.0 | 6.7 | 1.0 | 33.2 |  19.8 | 58.0 |  47.2 |  39.1 |  22.9
Ours | 16.1 | 4.0 | 29.0 | 2.6 | 3.8 | 0.54 | 8.6 |  1.9 |  27.6 | 10.6 | 7.8 |  1.6 | 36.5 | 22.4 | 62.2 | 48.0 | 39.9 | 23.6
Table 1: The state-of-the-art comparisons of C-GQA, MIT-States, and UT-Zappos
datasets in the OW-CZSL setting. We report the best seen ($S$) and best unseen
($U$) accuracy and best harmonic mean (HM) on the test sub-dataset. $S$, $U$,
and HM metrics are also reported on the validation sub-dataset. The best and
second-best results are bold and underlined.

##  4 Experiments

Datasets. We conduct experiments on three widely-use datasets including UT-
Zappos [50], MIT-States [14], and C-GQA [31]. UT-Zappos is a dataset for the
shoes and has 50025 images. It contains 12 object classes and 16 state
classes, with 83 seen compositions and a total of 192 compositional spaces.
MIT-States has 53753 images with 115 state classes and 245 object classes. The
seen and all output compositions are 1,262 and 28,175, respectively. C-GQA is
the largest dataset that contains 186,577 images with 413 state classes and
674 object classes. It contains 5,592 seen compositions and a full output
space of 278,362 compositions, which makes it the most extensive for the OW-
CZSL.

Evaluation Metrics. For the OW-CZSL, we follow the splits of [28, 29, 16] and
evaluate based on the generalized settings, where the test samples are from
both seen and unseen compositions. Considering the performance of the model
with different bias factors for the unseen compositions, we vary the bias on
the seen composition ($C^{s}$) during the test phase and report the
performance as best seen ($S$), best unseen ($U$), best harmonic mean (HM),
and the Area Under the Curve (AUC). For the pCZSL, following [16], we remove
the label and calculate the metrics on the full output composition space
($C$). As we can not access the full-labeled seen compositions ($C^{s}$), we
do not subtract any bias on $C^{s}$. Therefore, we use the seen (S) and unseen
(U) as well as accuracy harmonic mean (HM) metrics.

Baselines. For the OW-CZSL setting, we compare ProCC with other OW-CZSL
methods, including Compositional Cosine Logits (CompCos) [28], Knowledge
Guided Simple Primitives (KGSP) [16], and Compositional Cosine Graph
Embeddings (Co-CGE) [29]. CZSL methods are also compared, including Label
Embed+ (LE+) [31], Attributes as Operators (AoP) [33], Task Modular Networks
(TMN) [37], SymNet [25], and Compositional Graph Embeddings (CGE) [32]. For
the pCZSL setting, ProCC are compared with KGSP [16] as well as standard
(OW-)CZSL methods like CGE [32], CompCos [28], and Co-CGE [29], which are
state-of-the-art (OW-)CZSL methods respectively. They all adopt the same
missing label protocol as us.

Implementation Details. Following the standard practices in the CZSL, we
utilize the pre-trained ResNet-18 [9] without updating parameters as the
feature encoder ($\omega$) to extract 512-dimensional feature vector.
Following [32, 16], each classifier is composed of Multi-Layer Perceptrons
(MLP) with three layers with dimensions 768, 512, and the number of output
classes, respectively, and comprise Layer Normalization[21] and Dropout[43].
To be consistent with other methods, we randomly augment input images with
random crop and horizontal flip. We use PyTorch to implement our network and
optimize it with Adam [19] with default settings. The batch size is 256, and
the learning rate is $5.0\times 10^{-5}$ for the first two stages and
$1.0\times 10^{-5}$ for the third stage. For the UT-Zappos, MIT-States, and
C-GQA datasets, the total training time is approximately 1, 3, and 5 hours for
40/80/20, 50/100/25, and 50/100/25 epochs for three stages, respectively, with
the early stop strategy.

###  4.1 Open-World CZSL (OW-CZSL)

The results of the Open-World Compositional Zero-Shot Learning (OW-CZSL)
setting are illustrated in Table 1. In general, closed-world CZSL methods
achieve inferior performance, especially in two large datasets (i.e., C-GQA
and MIT-States), due to the large cardinality of the output space. ProCC
outperforms previous methods on almost all metrics in terms of three datasets.
Concretely, as for the most comprehensive dataset, i.e., C-GQA, the proposed
method exceeds the previous SOTA methods, especially for best harmonic (HM)
metrics (3.4$\rightarrow$3.8: $\uparrow$12$\%$), which means that ProCC has
the better ability to recognize both the seen and unseen compositions. Also,
in the validation sub-dataset, Our method suppresses the best baseline (i.e.,
KGSP) by a large margin in two overall evaluation indexes (i.e., HM:
13.2$\rightarrow$16.1: $\uparrow$22$\%$; AUC: 2.9$\rightarrow$4.0:
$\uparrow$38$\%$). As for the MIT-States dataset, our method also has
comparative results. Notably, we achieve the best performance on $U$ metric,
which validates the generalization ability of ProCC. For UT-Zappos, it is
specially designed for shoes and is relatively simpler than others. ProCC
consistently outperforms others, i.e., $S$: 59.3$\rightarrow$62.2; $U$:
47.2$\rightarrow$48.0; HM: 39.1$\rightarrow$39.9; AUC: 22.9$\rightarrow$23.6.

Remarkably, previous methods, including TMN[37], AoP[33], SymNet[25],
CompCos[28], CGE[32], Co-CGE[29] also utilize word embedding methods to encode
the word expression, which already contains semantic knowledge of similar
objects and attributes for composition learning [41]. Recent advanced Visual
Product based method [16] employs more complex classifiers (with hidden layers
of 768 and 1024) than ours as well as uses external knowledge to eliminate the
less feasibility compositions. We predict the state and object primitives with
more lightweight classifiers and explicitly model the cross-primitive
interactions to learn the relationship between primitives without external
knowledge. Therefore, our method is a more effective method to deal with the
large output composition space in the OW-CZSL task.

Method | C-GQA | MIT-States | UT-Zappos
---|---|---|---
Val | Test | Val | Test | Val | Test
S | U | HM | S | U | HM | S | U | HM | S | U | HM | S | U | HM | S | U | HM
CGE[32] | 19.2 | 2.9 | 5.6 | 17.4 | 0.4 | 0.9 | 10.0 | 2.8 | 4.3 | 19.6 | 1.3 | 2.4 | 46.5 | 3.5 | 6.6 | 50.3 | 3.4 | 5.0
CompCos[28] | 18.2 | 3.0 | 5.2 | 24.3 | 0.4 | 0.7 | 11.1 | 2.9 | 4.6 | 10.8 | 2.0 | 3.6 | 50.2 | 3.9 | 7.3 | 52.4 | 4.1 | 7.6
Co-CGE[29] | 19.8 | 3.9 | 6.4 | 22.1 | 0.6 | 1.2 | 14.8 |  3.3 |  5.3 | 13.1 | 2.3 | 4.0 | 47.2 |  6.1 |  10.8 | 52.6 | 5.4 | 9.9
KGSP [16] |  20.1 |  4.8 |  8.3 |  22.3 |  0.9 |  1.7 |  15.7 | 3.2 |  5.3 | 13.5 |  2.6 |  4.4 |  49.4 | 5.9 | 9.7 |  53.8 |  6.9 |  12.3
Ours | 21.6 | 5.4 | 8.7 | 24.1 | 1.1 | 2.0 | 16.3 | 3.5 | 5.8 |  14.1 | 2.9 | 4.8 | 51.0 | 7.1 | 12.5 | 55.1 | 8.1 | 14.1
Table 2: The state-of-the-art comparisons of C-GQA, MIT-States, and UT-Zappos
datasets in the pCZSL setting. We report the seen (S) and unseen (U) accuracy,
best harmonic mean (HM), and area under the curve (AUC) on the testing
dataset. HM and AUC metrics are also reported on the validation dataset. The
best and second-best results are bold and underlined.

###  4.2 partial-supervision CZSL (pCZSL)

As for the more challenging setting, partial-supervision CZSL (pCZSL), the
challenges come from not only the huge output composition space but also the
missing labels. The results of ProCC and previous SOTAs are reported in Table
2. We can see that our method achieves state-of-the-art performance compared
with previous CZSL, OW-CZSL, and pCZSL methods. Concretely, for the largest
dataset, C-GQA, the performance of SOTAs on pCZSL severely degrades compared
with OW-CZSL, even for KGSP, which is equipped with the pseudo label and
external knowledge. Our method consistently exceeds them both on validation
and testing datasets. For the MIT-States dataset, our method surpasses the
second-best method in terms of HM metrics by a large margin (i.e., val:
5.3$\rightarrow$5.8: $\uparrow$9$\%$; test: 4.4$\rightarrow$4.8:
$\uparrow$9$\%$). For the simplest dataset, UT-Zappos, our method also has the
best performance. Note that we do not use any external knowledge like Word2vec
[30], Glove [35], Conceptnet [42], and other semi-supervised learning
techniques [20, 4] for the missing annotations. The superior performance
indicates even with the partial labels of the object and state primitives, and
our progressive learning strategy can also model the interactions of cross
primitives with the pre-trained classifiers.

Method | OW-CZSL | pCZSL
---|---|---
C-GQA | MIT-States | C-GQA | MIT-States
HM | AUC | HM | AUC | S | U | HM | S | U | HM
w/o CPC | 3.3 | 0.40 | 6.2 | 0.8 | 17.4 | 0.5 | 1.0 | 11.6 | 2.2 | 3.7
w/o Pro-L | 3.7 | 0.50 | 7.6 | 1.5 | 22.4 | 0.8 | 1.6 | 12.5 | 2.5 | 4.1
w/ Ex | 3.6 | 0.48 | 7.8 | 1.5 | 22.6 | 1.0 | 1.9 | 13.2 | 2.7 | 4.4
w/o Stage3 | 3.5 | 0.47 | 7.4 | 1.4 | 23.2 | 1.1 | 2.0 | 13.6 | 2.8 | 4.6
w/o C&P | 3.2 | 0.39 | 6.2 | 0.8 | 17.3 | 0.5 | 1.0 | 11.4 | 2.1 | 3.5
Ours | 3.8 | 0.54 | 7.8 | 1.6 | 24.1 | 1.1 | 2.0 | 13.8 | 2.9 | 4.7
Table 3: Ablation studies for both OW-CZSL and pCZSL on C-GQA and MIT-States
datasets. (w/o: without, CPC: Cross-Primitive Consistency, Pro-L: Progressive
Learning strategy, Ex: exchange the stage 1 and 2, C&P: CPC and Pro-L. )

###  4.3 Ablation Study

We ablate the two important components of the proposed method: the Cross-
Primitive Consistency (CPC) module and the Progressive learning strategy. We
adopt the same implementation strategy and conduct the OW-CZSL and pCZSL
experiments on the two largest datasets, i.e., C-GQA and MIT-States.

Effect of the Cross-Primitive Consistency module. In Table 3, without the CPC
module, the performance is severely degraded both on the OW-CZSL and pCZSL
settings. The reason is that lacking the interaction between cross primitives
makes the network degenerate to the base model, i.e., the Visual Product
baseline [31, 15, 16]. Meanwhile, the external knowledge used in [16] makes it
surpass the ablation configuration. Note that the CPC is extremely lightweight
with two trainable 1d convolution layers. In general, the CPC module greatly
improves the performance with negligible computation burden and also does not
import external information, which is practical for real-world applications.

Effect of the Progressive learning strategy. Another important aspect of the
ProCC is the progressive learning strategy. From Table 3, we can learn that
with the traditional end-end training strategy (w/o Pro-L), the performance of
ProCC degrades to some extent in the pCZSL setting (i.e., HM:
2.0$\rightarrow$1.6 (C-GQA) and 4.7$\rightarrow$4.1 (MIT-States)) and the
performance on OW-CZSL also slightly drops. As jointly training the whole
network under the pCZSL setting does not explicitly learn the relationship
between state and object primitives, which is the critical issue in the CZSL
task. While for the full supervision setting (i.e., OW-CZSL), imbalance tasks
lead to the over/under-fitting problem for each classifier if jointly training
two classifiers. Also, we exchange (w/ Ex) the training sequence (i.e., stage
2 $\rightarrow$ stage 1) and ablate the fine-tuning stage (w/o Stage 3). For
the configuration of w/ Ex, the performance of ProCC degrades on both
settings, especially for the pCZSL setting because the trained state features
somewhat induce noise due to the challenge of classifying state primitives.
For the configuration of w/ Stage 3, the performance on the OW-CZSL and pCZSL
settings both degrade to some extent as the CPCs→o and fine-tuning also matter
for the cross-primitive consistency and global optimum.

##  5 Conclusion

In this paper, we propose a method named Progressive Cross-primitive
Consistency (ProCC) network for both OW-CZSL and pCZSL tasks. The simple but
effective Cross-Primitive Consistency module drives the network learning to
predict feasible object and state primitives conditioned on mutual
information. Also, the progressive learning strategy significantly avoids the
negative influence of the pCZSL setting and imbalance issue of MTL in an easy-
hard learning manner with pre-trained classifiers. Comprehensive experiments
on OW-CSZL and pCZSL settings illustrate superior performance compared with
other state-of-the-art methods.

## References

  * [1] Yuval Atzmon, Jonathan Berant, Vahid Kezami, Amir Globerson, and Gal Chechik.  Learning to generalize to new compositions in image understanding.  arXiv e-prints, page arXiv:1608.07639, Aug. 2016.
  * [2] Yuval Atzmon, Felix Kreuk, Uri Shalit, and Gal Chechik.  A causal view of compositional zero-shot recognition.  In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, NeurIPS, volume 33, pages 1462–1473, 2020.
  * [3] Irving Biederman.  Recognition-by-components: a theory of human image understanding.  Psychological review, 94(2), 1987.
  * [4] Yves Grandvalet and Yoshua Bengio.  Semi-supervised learning by entropy minimization.  In L. Saul, Y. Weiss, and L. Bottou, editors, NeurIPS, volume 17, 2004.
  * [5] Jingcai Guo and Song Guo.  A novel perspective to zero-shot learning: Towards an alignment of manifold structures via semantic feature expansion.  IEEE TMM, 23:524–537, 2020.
  * [6] Jingcai Guo, Song Guo, Qihua Zhou, Ziming Liu, Xiaocheng Lu, and Fushuo Huo.  Graph knows unknowns: Reformulate zero-shot learning as sample-level graph recognition.  In AAAI, 2023.
  * [7] Jingwen He, Chao Dong, and Yu Qiao.  Modulating image restoration with continual levels via adaptive feature modification layers.  In CVPR, June 2019.
  * [8] Jingwen He, Yihao Liu, Yu Qiao, and Chao Dong.  Conditional sequential modulation for efficient global image retouching.  In ECCV, pages 679–695, 2020.
  * [9] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In CVPR, June 2016.
  * [10] Shaul Hochstein and Merav Ahissar.  View from the top: Hierarchies and reverse hierarchies in the visual system.  Neuron, 36(5):791–804, 2002.
  * [11] D.D. Hoffman and W.A. Richards.  Parts of recognition.  Cognition, 18(1):65–96, 1984.
  * [12] Jie Hu, Li Shen, and Gang Sun.  Squeeze-and-excitation networks.  In CVPR, June 2018.
  * [13] Dat Huynh and Ehsan Elhamifar.  Fine-grained generalized zero-shot learning via dense attribute-based attention.  In CVPR, June 2020.
  * [14] Phillip Isola, Joseph J. Lim, and Edward H. Adelson.  Discovering states and transformations in image collections.  In CVPR, June 2015.
  * [15] Shyamgopal Karthik, Massimiliano Mancini, and Zeynep Akata.  Revisiting visual product for compositional zero-shot learning.  In NeurIPS Workshop, 2021.
  * [16] Shyamgopal Karthik, Massimiliano Mancini, and Zeynep Akata.  Kg-sp: Knowledge guided simple primitives for open world compositional zero-shot learning.  In CVPR, pages 9336–9345, June 2022.
  * [17] Muhammad Gul Zain Ali Khan, Muhammad Ferjad Naeem, Luc Van Gool, Alain Pagani, Didier Stricker, and Muhammad Zeshan Afzal.  Learning Attention Propagation for Compositional Zero-Shot Learning.  arXiv e-prints, page arXiv:2210.11557, Oct. 2022.
  * [18] Dong-Jin Kim, Jinsoo Choi, Tae-Hyun Oh, Youngjin Yoon, and In So Kweon.  Disjoint multi-task learning between heterogeneous human-centric tasks.  In WACV, pages 1699–1708, 2018.
  * [19] Diederik P. Kingma and Jimmy Ba.  Adam: A method for stochastic optimization, 2015.
  * [20] Dong-Hyun Lee.  Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks.  In ICML Workshop, 2013.
  * [21] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton.  Layer Normalization.  arXiv e-prints, page arXiv:1607.06450, July 2016.
  * [22] Wei-Hong Li, Xialei Liu, and Hakan Bilen.  Learning multiple dense prediction tasks from partially annotated data.  In CVPR, pages 18879–18889, June 2022.
  * [23] Xiangyu Li, Zhe Xu, Kun Wei, and Cheng Deng.  Generalized zero-shot learning via disentangled representation.  AAAI, 35(3):1966–1974, May 2021.
  * [24] Xiangyu Li, Xu Yang, Kun Wei, Cheng Deng, and Muli Yang.  Siamese contrastive embedding network for compositional zero-shot learning.  In CVPR, pages 9326–9335, June 2022.
  * [25] Yong-Lu Li, Yue Xu, Xiaohan Mao, and Cewu Lu.  Symmetry and group in attribute-object compositions.  In CVPR, June 2020.
  * [26] Ziming Liu, Song Guo, Jingcai Guo, Yuanyuan Xu, and Fushuo Huo.  Towards unbiased multi-label zero-shot learning with pyramid and semantic attention.  IEEE TMM, 2022.
  * [27] Yao Lu, Soren Pirk, Jan Dlabal, Anthony Brohan, Ankita Pasad, Zhao Chen, Vincent Casser, Anelia Angelova, and Ariel Gordon.  Taskology: Utilizing task relations at scale.  In CVPR, pages 8700–8709, June 2021.
  * [28] Massimiliano Mancini, Muhammad Ferjad Naeem, Yongqin Xian, and Zeynep Akata.  Open world compositional zero-shot learning.  In CVPR, pages 5222–5230, June 2021.
  * [29] Massimiliano Mancini, Muhammad Ferjad Naeem, Yongqin Xian, and Zeynep Akata.  Learning graph embeddings for open world compositional zero-shot learning.  IEEE TPAMI, pages 1–1, 2022.
  * [30] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean.  Distributed representations of words and phrases and their compositionality.  In NeurIPS, volume 26, 2013.
  * [31] Ishan Misra, Abhinav Gupta, and Martial Hebert.  From red wine to red tomato: Composition with context.  In CVPR, July 2017.
  * [32] Muhammad Ferjad Naeem, Yongqin Xian, Federico Tombari, and Zeynep Akata.  Learning graph embeddings for compositional zero-shot learning.  In CVPR, pages 953–962, June 2021.
  * [33] Tushar Nagarajan and Kristen Grauman.  Attributes as operators: Factorizing unseen attribute-object compositions.  In ECCV, September 2018.
  * [34] Vladimir Nekrasov, Thanuja Dharmasiri, Andrew Spek, Tom Drummond, Chunhua Shen, and Ian Reid.  Real-time joint semantic segmentation and depth estimation using asymmetric annotations.  In ICRA, pages 7101–7107, 2019.
  * [35] Jeffrey Pennington, Richard Socher, and Christopher D Manning.  Glove: Global vectors for word representation.  In EMNLP, 2014.
  * [36] Ethan Perez, Florian Strub, Harm de Vries, Vincent Dumoulin, and Aaron Courville.  Film: Visual reasoning with a general conditioning layer.  AAAI, 32(1), 2018.
  * [37] Senthil Purushwalkam, Maximilian Nickel, Abhinav Gupta, and Marc’Aurelio Ranzato.  Task-driven modular networks for zero-shot compositional learning.  In ICCV, October 2019.
  * [38] Caruana Rich.  Multitask learning.  Machine learning, 28(1):41–75, 1987.
  * [39] Frank Ruis, Gertjan Burghouts, and Doina Bucur.  Independent prototype propagation for zero-shot compositionality.  In NeurIPS, volume 34, 2021.
  * [40] Suman Saha, Anton Obukhov, Danda Pani Paudel, Menelaos Kanakis, Yuhua Chen, Stamatios Georgoulis, and Luc Van Gool.  Learning to relate depth and semantics for unsupervised domain adaptation.  In CVPR, pages 8197–8207, June 2021.
  * [41] Nirat Saini, Khoi Pham, and Abhinav Shrivastava.  Disentangling visual embeddings for attributes and objects.  In CVPR, pages 13658–13667, June 2022.
  * [42] Robyn Speer, Joshua Chin, and Catherine Havasi.  Conceptnet 5.5: An open multilingual graph of general knowledge.  In AAAI, page 4444–4451, 2017.
  * [43] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov.  Dropout: A simple way to prevent neural networks from overfitting.  JMLR, 15:1929–1958, JUN 2014.
  * [44] Simon Vandenhende, Stamatios Georgoulis, Wouter Van Gansbeke, Marc Proesmans, Dengxin Dai, and Luc Van Gool.  Multi-task learning for dense prediction tasks: A survey.  IEEE TPAMI, 44(7):3614–3633, 2022.
  * [45] Qilong Wang, Banggu Wu, Pengfei Zhu, Peihua Li, Wangmeng Zuo, and Qinghua Hu.  Eca-net: Efficient channel attention for deep convolutional neural networks.  In CVPR, pages 11531–11539, 2020.
  * [46] Yongqin Xian, Christoph H. Lampert, Bernt Schiele, and Zeynep Akata.  Zero-shot learning—a comprehensive evaluation of the good, the bad and the ugly.  IEEE TPAMI, 41(9):2251–2265, 2019.
  * [47] Ziwei Xu, Guangzhi Wang, Yongkang Wong, and Mohan S. Kankanhalli.  Relation-aware compositional zero-shot learning for attribute-object pair recognition.  IEEE TMM, 24:3652–3664, 2022.
  * [48] Muli Yang, Cheng Deng, Junchi Yan, Xianglong Liu, and Dacheng Tao.  Learning unseen concepts via hierarchical decomposition and composition.  In CVPR, June 2020.
  * [49] Muli Yang, Chenghao Xu, Aming Wu, and Cheng Deng.  A decomposable causal view of compositional zero-shot learning.  IEEE TMM, pages 1–11, 2022.
  * [50] Aron Yu and Kristen Grauman.  Fine-grained visual comparisons with local learning.  In CVPR, June 2014.
  * [51] Amir R. Zamir, Alexander Sax, Nikhil Cheerla, Rohan Suri, Zhangjie Cao, Jitendra Malik, and Leonidas J. Guibas.  Robust learning through cross-task consistency.  In CVPR, June 2020.
  * [52] Fang Zhao, Jian Zhao, Shuicheng Yan, and Jiashi Feng.  Dynamic conditional networks for few-shot learning.  In ECCV, September 2018.
  * [53] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba.  Learning deep features for discriminative localization.  In CVPR, June 2016.
  * [54] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu.  Conditional prompt learning for vision-language models.  In CVPR, pages 16816–16825, June 2022.