# Aesthetics, Personalization and Recommendation: A survey on Deep Learning in
Fashion

Wei Gong<EMAIL_ADDRESS>University of Science and Technology of
ChinaNo.96, JinZhai Road Baohe DistrictHefeiAnhuiChina230026 and  Laila Khalid
<EMAIL_ADDRESS>University of Science and Technology of ChinaNo.96,
JinZhai Road Baohe DistrictHefeiAnhuiChina230026

(2020)

###### Abstract.

Machine learning is completely changing the trends in the fashion industry.
From big to small every brand is using machine learning techniques in order to
improve their revenue, increase customers and stay ahead of the trend. People
are into fashion and they want to know what looks best and how they can
improve their style and elevate their personality. And since systems are
already monitoring every sale and coming trends , why not utilize their power
in getting a recommendation regarding outfit. Using Deep learning technology
and infusing it with Computer Vision techniques one can do so by utilizing
Brain-inspired Deep Networks, and engaging into Neuroaesthetics, working with
GAN’s and Training them, playing around with Unstructured Data,and infusing
the transformer architecture are just some highlights which can be touched
with the Fashion domain. It’s all about designing a system that can tell us
information regarding the fashion aspect that can come in handy with the ever
growing demand. Personalization is a big factor that impacts the spending
choices of customers.The survey also shows remarkable approaches that encroach
the subject of achieving that by divulging deep into how visual data can be
interpreted and leveraged into different models and approaches. Aesthetics
play a vital role in clothing recommendation as users’ decision depends
largely on whether the clothing is in line with their aesthetics, however the
conventional image features cannot portray this directly. For that the survey
also highlights remarkable models like tensor factorization model, conditional
random field model among others to cater the need to acknowledge aesthetics as
an important factor in Apparel recommendation.These AI inspired deep models
can pinpoint exactly which certain style resonates best with their customers
and they can have an understanding of how the new designs will set in with the
community. With AI and machine learning your businesses can stay ahead of the
fashion trends and deliver exactly what your customers want and when they want
it.

Deep Learning, neural networks, Fashion, Aesthetics ,Recommendation,
Personalization

††copyright: acmcopyright††journalyear: 2020††doi: not available yet††journal:
JACM††journalvolume: 00††journalnumber: 0††article: 111††publicationmonth:
0††ccs: Computing methodologies Computer vision††ccs: Applied computing Online
shopping††ccs: Computing methodologies Neural networks

##  1\. Introduction

If we go over the past decade and see how deep learning has achieved
significant success in many popular Industries and areas. We observe how
perception tasks, including visual object recognition and text understanding
and speech recognition, have revolutionized different regions. There is no
comparison as to how successful deep learning has been. Still, suppose we want
to discuss deep learning in the real terms of the fashion industry. In that
case, we see a lot of opportunities and research areas that are still
available to work on. As we all know, fashion is an ever-evolving industry.
There are new trends that are setting in every second that is passing by.
Although clothing design is like one of the most creative realms in the
Contemporary World (Insight, [n.d.]), whether it’s because of the considerable
creative part of the design process or equivocal information about clothing,
the fact remains to be. Internet shopping has also grown incredibly in the
last few years, and fashion has created immense opportunities. Exciting
applications for image understanding , retrieval and tagging are surfacing,
and there are loads of different application areas that they can be used on.
For example, text analysis, image analysis, and similarity retrieval can be
utilized in fashion. So deep learning is an aspect that we can use to train
our computer to perform human-like tasks such as recognizing speech,
identifying images or making predictions. For example, the results described
in the apparel design and fashion industry allow users to translate the image
into the text that might as well be interpreted as a description of the
garment based on its sketch.

We also know that images are an essential aspect because they display content
and convey emotions like sadness, excitement, anger, etc. So useful image
classification is beneficial, and obviously, it’s been used in computer vision
in multimedia. Still, if you find research regarding fashion and specifically
in terms of aesthetic features or personalization, you will find only a few
specific directions. Discussing one of them that is available to describe
images inspired by art theories, which are, you know, intuitive,
discriminative, and easily understandable. So we know that the effective image
classification based on these features can achieve high accuracy compared with
the state-of-the-art. For that, we take an example in the paper (Wang, 2013)
where they develop an Emotion guided image gallery to demonstrate the proposed
feature collection. So the authors achieve mining the interpretable visual
features directly affecting human emotional perception from the viewpoint of
art theories.

Another example in another paper (Borràs et al., 2003) is where they discussed
that content-based image retrieval is done in terms of people’s appearance.
It’s a two-stage process that is composed of image segmentation and region-
based interpretation. The modelling of an image is due to an attributed graph
and a hybrid method that follows a split and merge strategy. There are a lot
of different stuff that is being worked on in this field of computer vision
specifically, and image retrieval from databases is usually a formula, in
terms of descriptions that combine the Salient features such as colour,
texture, shapes etc.

Today, more and more retail companies are trying to understand, to stay ahead
of the trend curve and because they want to reshape their business to stay
ahead while implementing tech forward approaches and solutions. And data
analysis brings diverse opportunities to companies, which allows them to reach
their customer goal and offer a smarter experience to them. But the thing is
that the lack of profound insights based on reliable statistics is the major
challenge of fashion retailers that they face. So for that, computer vision
technologies and deep learning can come in very handy. And as we all know,
computer vision is still an evolving technology, so we can speak about
specific optimization and cost reduction techniques that can come in handy,
for example, like how the information regarding what people wear, how
customers kind of match their garments and what or which or who influences
their taste is essential for fashion retailers. As we can see the Instagram
influencers, we see that many people follow them and try to copy their trends
and how they are inspiring a lot of followers. Image recognition technology
also helps business owners collect data, process it, and gain an actionable
insight for Effective Trend forecasting.

For that, in this particular article (ELEKS, [n.d.]b), we see that the
dashboard they developed allows seeing how frequently one specific type of
garment appears a day. Like what type of apparel is popular within a
particular age range or how people sort of match their attire. Like for
example, how a specific jacket is trending or why is it popular among
teenagers? Or why is a scarf popular amongst the elders. They developed the
graph that shows how certain prevalent types of garments would be over the
next season’s you know, which could broadly impact the new upcoming trend for
the fashion. This kind of analysis also aims to help fashion retailers and
brands plan sales and learn to avoid any surplus. The author suggests that in
the visual search domain with a focus mainly on image similarity for like,
e-commerce and Online shops and understanding images of clothing, it means a
lot more than just classifying them into different categories. Because if you
don’t get a meaningful description of the whole image you classify, then you
are losing a lot of information that could come in handy. In this way, one can
gain reliable and timely insights into fashion trends across any location.
What defines those trends is people’s unique choices, like how they choose
something and what goes with their personality. The element of personalization
is one of the biggest game-changers in this apparel recommendation. By
targeting this factor, businesses can attract more customers.

The thing that I like about this deep learning aspect is that it penetrates
the industry and, you know, activities where human creativity has
traditionally dominated. It adds a futuristic touch to fashion, art ,
architecture and music so on. Another paper’s (ELEKS, [n.d.]a) key finding is
that the representation of content and style in the convolutional neural
networks are separable. That is, you know if we can manipulate both
representations independently to produce new and perceptually meaningful
images. If you look, fashion is an entirely new direction for machine
learning. So to design clothes one should you know, basically have an
understanding of the mechanism of technique, like how certain styles go
famous, what things they are having that are attracting millions of followers
around and what causes the spread, you know the spread of the Fashion trends
and principles and evolution of patterns, so the task of designing or
predicting trends can be simplified. The paper under discussion where the
author suggests that now designing or predicting Trends can be simplified,
thanks to a new class of neural networks. These networks basically can
automatically allocate shapes, elements, and types of clothing and further
combine them. This allows a whole fresh feel of how you can manipulate the
patterns and see which patterns can influence more influence than the others.
Now aesthetics play a vital role in the user’s pick, and even though
personalization is tricky to play with, aesthetics are not. Because everyone
appreciates eye-pleasing items and if we can manipulate the role of aesthetics
in our fashion recommendation, we can hit the jackpot.

So there are many various aspects of fashion in which deep learning can
enhance and help us out. There are multiple domains for improving the current
elements and how we can help predict and revolutionize this industry. This
survey is organized in the following sections. Sec. 2 reviews the fashion
recommendation systems and approaches that come out on top and are the basis
for future work. Sec. 3 illustrates the positions for aesthetics in fashion,
all it’s analysis containing various approaches. Sec. 4 provides an overview
of personalization in fashion , different top approaches that have tasks
comprising Deep Neural Networks, GAN’s, and handling unstructured data. Sec. 5
demonstrates selected applications and future horizons that can be worked on.
Last but not least, concluding remarks are given in Sec. 6.

##  2\. Recommendation Systems

Well, if you indulge in object recognition, you will find that fashion sense
is a bit more subtle and sophisticated, you know, which can require specific
domain expertise in outfit composition. So, for example, if you refer to an
outfit as a set of clothes working together kind of typically for a desired
specific style or to find a good Outfit composition, what we need is not only
to follow the appropriate dressing course, but it can also have a creative
aspect in balancing the contrast of colours and different styles. And although
we have seen a relative number of researches that are mainly based on clothes
retrieval and recommendation but what we have seen is that none of them
consider the problem of fashion outfit composition. On the one hand, you know
a fashion concept is often subtle and subjective and is non-trivial to get you
to know consensus from ordinary labellers if they are not Fashion experts. On
the other hand, there may be a large number of attributes for describing
fashion.

###  2.1. End-to-End Deep Learning Approach on Set Data.

It is challenging to obtain exhaustive labels for training. So, as a result,
most of the existing studies are kind of, you know, limited to the simple
scenario of retrieving similar clothes or choosing individual clothes for a
given event. So the paper (Li et al., 2016) that is being reviewed proposes a
data-driven approach to train a model that can automatically compose a
suitable fashion outfit. This approach is motivated by the surge of the
increasing online fashion trends, including Pinterest and YouTube, and how
teenagers have been addicted to creating every new culture trend on these
sites.

####  2.1.1. Background

So basically what they have done is that they have developed a full automatic
composition system that is based upon a scorer by iteratively evaluating all
the possible outfit candidates. But this model had some challenges in which
they had to look out for possible solutions. For example, one of the
challenges that they Encountered was that complicated visual contents of the
fashion images? So, you know, there are potentially many kinds of different
attributes like color, textures, categories and spectrum’s etc and it is
impossible to label or even list all possible attributes. So there is this
hindrance and second one would be the rich context of fashion outfit for
example, clothing outfits can kind of sort of reflect current personality and
interest. So if one style is acceptable to a specific group or culture. It may
be offensive to the others. So to infer such information they have taken into
account not only the pixel information but also the context information in the
fashion output.

####  2.1.2. Proposed Approach

So basically for These challenges they proposed different solutions like for
the first challenge they have proposed an end-to-end system of encoding visual
features through a deep convolutional network which sort of, you know, takes a
fashion outfit as an input and processes it and then predicts the user
engagement levels. And for the Second Challenge what happens is that a
multimode Deep learning framework, which sort of leverages the context
information from the image itself and the experiment that they did through
that was that the multi-modal approach significantly outperforms the single
model. And provides the suitable and more reliable solution for the fashion
outfit for scoring tasks and thus the full composition tasks. So these are the
contributions that they are enlisting and they are basically proposing an end-
to-end trainable system to fuse signals from multi-level hybrid modalities
that includes image and metadata of the fashion items and they also collected
a large scale of database that are for the fashion outfit related research.
Lastly they propose a fashion outfit composition to the solution based on a
reliable sort of outfit quality predictor and predicting fashion is never
easy, but it is something that they have put forward because many interleaving
factors visible or hidden contribute to the process the combinatorial nature
of the problem also makes it very interesting and it’s a test tone for the
state-of-the-art machine learning systems.

Figure 1. The proposed fashion outfit scoring model

###  2.2. Implicit Feedback Based

As we know that the fashion domain has quite a lot of several intriguing
properties that can be personalized and which make personalization
recommendations even far more difficult than the traditional domains. So in
order to sort of avoid potential bias, like when using explicit user ratings,
which are also pretty much expensive to obtain.

####  2.2.1. Background

So this paper (Nguyen et al., 2014) basically suggests the work that
approaches fashion recommendations by sort of analyzing the implicit feedback
from users in an app. Basically the design criteria is that the system shall
be completely unobstructive and thus the recommendation system cannot , you
know, rely explicitly on the ratings rather It will be based on the rich
history and the interaction between the user and the app. In simple words it
relies on the implicit feedback that is you know, the user preference is to be
automatically inferred from the behavior.

Though there are still some challenges that can be gathered in this approach
that is the most notable interaction a user has with an item is a sign of
interest ,so the system therefore never receives a negative feedback and of
course, you know an item can be both clicked and loved so it is also multi-
faceted and then the different types of feedback will have to be combined into
a single numerical value as defined for an experiment. Set a preference score
for the recommendation algorithms. It is difficult to evaluate such a system
compared to explicit-rating-systems, because the system does not have a target
rating to compare its predictions to. So all in all the success basically
relies on the implicit feedback system that has a well-defined strategy for
inferring user preference from implicit feedback data and combining even types
into implicit scores and then evaluating these scores and recommendations by
using a suitable metric.

####  2.2.2. Proposed Approach

So basically the authors in order to build this recommendation system took the
first step and that was to generate implicit preference scores and to you know
translate data that is being captured by a user’s interaction with an item
into a specific number that can be called employees implicit preference score
and that can be also later used to rank it with the other items so that most
important factor in this was when they created such numbers to understand the
data available and their implications for user preference. So once you can
have the data analyzed suitable generalizations can then be furthermore
chosen. And then the second step was for defining the penalisation functions.
Important properties in the fashion domain that must be captured by the system
include seasons and trends, price sensitivity and popularity. In general, when
a user $u$ triggers an event $e,$ e.g. Clicks, we have a range of possible
scores to give this event. We use $S_{e}$ to denote this score, and let
$m_{e}$ and $M_{e}$ denote the minimum and maximum score possible for event
$e,$ respectively. We then use a penalisation function $p_{u}(x)$ taking a
feature value $x$ (e.g., related to an item’s price), and returns a number
between zero and one to adjust the score inside the possible range.

$S_{e}=M_{e}-\left(M_{e}-m_{e}\right)\cdot p_{u}(x)$

So as mentioned that you know fashion is all about the trend and timing, so
the recentness of an event is a natural feature for having the events
importance and therefore, penalise the items that the user has not , you know
considered recently. So for that they had a look at the number of days since
the user did the event in question. Let’s say we can denote that event by $x$.
And then compare this to the old event that the user has in the database and
that can be, you know denoted by $F_{u}.$

So this can be known later on, forced to create a linear penalization letting
$p_{u}(x)=x/F_{u},$ , but it wasn’t fitting well with the idea of Seasons. So
as an example what they did was that even if a store may be selling the warm
clothes from November to March , they wanted to focus on the recommendations
on summer clothes when the season changes so for that they had to, kind of
duplicate this behavior and choose a sigmoid function that you know, considers
the recentness in a way that could obscure the preference of users that have
been absent from the app for some time. So they used linear penalization
because you know, it could ensure that the difference in penalization between
the two most recent items is equal to the difference between the two old ones.

Figure 2. Screenshots from the application. From the left to right: editorial
information, product details, and a “love list”.

So for the price what they did was that, you know different users have
different price range because they tend to be price sensitive and if an item’s
price should also be taken into account then what they did was that the users
typical price range was used and that was that created a personalized score
and penalized that were not in the price layer range preferred by the user. So
this procedure was basically done in simple two steps. In the first step what
they did was they found the average of all the price items related to a user
and on second base they pretty much calculated the difference that was found
in the price of an item $i$ that triggered the event $e$ and the average and
then used that to penalize that item. Rregarding the third aspect that they
used was popularity. So for the popularity expect what they did was that they
considered popularity as a feature by, you know, having a comparison with
users Behavior to the rest of the population. So, you know, we can tell it
like that that if a user’s activities conform to the common standards that are
likely to be his or her taste then it is more unique giving significant clues
about the items to recommend. So basically they judged each user’s behavior by
looking at the overall popularity of the items. They pretty much interacted
with them and they use a linear pair punishment for items with different
popularities.

And lastly what they did was they combined all these different penalisation
and came over a sum of all models this sort of required setting different
weights for different factors. So simply what they did was in order to
validate their approach that there were scores built using features and that
was you know, Event for the fashion domain and secondly, they distributed the
scores over a full range of valid scores and had an average confirming the
hypothesis.

###  2.3. Based on Weak Appearance Feature

As we know that the technology regarding online shopping has been developing
rapidly and that online fitting and other clothing intelligent equipment have
been introduced in the fashion industry. A lot of different Advanced
algorithms have been developed and there are many more currently in the
process.

####  2.3.1. Background

For example the CRESA (Melo et al., 2015) combined textual attributes, visual
features and human visual attention to compose the clothes profile in the
recommendation. Recommendation that is based on the content is usually
applicable for multiple regions. So for new projects, let’s say if the user
has according to the individual browsing records, they can recommend results
have been proven to be explicit and accessible but the content-based
recommendation usually is improper when you kind of apply it in the industry.
And obviously this means that the new users that sign up would not be getting
any recommendations based on the browsing record.

####  2.3.2. Proposed Approach

Basically what this paper (Zhang et al., 2017) proposes is that the
classification process usually needs to consider the quarter sales clothing
styles and other factors. So as a result, they basically divided this into
four categories where the fashion level is a subjective method that usually
needs subjective evaluation on image characters through the expert group. So
knowledge background and psychological motivation of the edge experts is
involved. And as for the researchers of visual psychological characteristics,
there wasn’t a quantitative description method by which the objective
evaluated results can represent the subjective evaluation results. So what
this aims to find out is to have a set of objective indexes, which can be used
to access the fashion level. This was done by considering all the factors that
usually affect the evaluation of personal scoring. So this paper basically
regards the weak appearance feature as an important index that can influence
the fashion level. So there are many, as you know weak appearance features
related to the individual fashion level. But the three major categories that
can be known namely if we want to go over are makeup ,accessories and hair
colors. So this could include the blush, the lip color, eyebrow color ,hat,
any accessories on hand and neck etc. By utilizing all these features what
they do is that the SVM classification method is leveraged in this and they
evaluate based on whether the human body has weak appearance features. So
there is no effective way to sort of establish a fashion level database. But
the one established in this paper is a basis of the follow-up studies that can
be taken up by the future researchers. So basically the image database is of a
pretty much very important significance in all this training and testing of
algorithms.

Figure 3.  Fashion level classification framework based on weak appearance
feature.

For the extraction of weak feature index, the current face detection methods
usually have sort of two categories in which knowledge based ones and
statistics based ones are available. So in order to extract the weak facial
feature, they find the facial feature points and then they use the facial
recognition. This paper basically adopts the Adaptive boosting method for
facial feature positioning. So the idea behind is that they have to endure
large amounts of unsuccessful training samples making the algorithm learning
focus on the difficult training samples in the subsequent study and finally
they weight and add the number of weak classifiers selected by the algorithms
to Strong classifier.

Table 1. Customer fashion level classification. Fashion level | Description classification
---|---
First level | Wonderful
Second level | Great
Third level | Good
Fourth level | Common
Table 2. Weak appearance features catalogue. Category | Weak feature index
---|---
Make-up | Eyebrow, blush, lips, eye shadow
Accessories | Neck accessories, hand accessories, brooch, nail, hat
Hair color | Red, yellow, green, blue, brown, black, gray, white

So all in all what the paper does is that it uses the appearance week feature
to sort of characterize consumers’ fashion level and what it does is that it
draws the conclusion by, you know, comparing the science experiment and expert
evaluation. So both categories of evaluation are involved in this study.
Basically the fashion level of the users is what they determine which is based
on their makeup ,the accessories they are wearing and the hair color they
have. So if a person is into red hair color or you know, having a lot of
makeup on they can you know access their level that oh, okay so this person is
more into fashion. So based on their level they kind of you know just
recommend them the things that they like so for example, let’s say if a
certain person is into dark eye shades and dark lip color and you know, they
are having some sort of streaks in their hair and stuff like that. So these
May indicate a level that is higher in the fashion aspect and they will
obviously recommend the products accordingly.

###  2.4. Semantic Attribute Region Guided Approach

A lot of multiple semantic attributes built up a fashion product for example
sleeves, collars etc. So while making you know these decisions regarding the
clothes, a lot of preferences for different semantics attributes, like v neck
collar ,deep neck or pointed toes shoes, high heels etc, are looked over.
Semantic attributes can not only let you know how one generates a
comprehensive representation of products, but they can also help us make an
understanding of how the user preferences work. But unfortunately, there
aren’t any unique challenges that can be inherited in designing efficient
solutions in order to integrate semantic attribute information for the fact
that we want fashion recommendation.

####  2.4.1. Previous Methods

It is quite difficult to obtain semantic attribute features without the manual
attribute annotation and especially in large scale e-commerce. On the other
hand if the user preferences are basically classy or sophisticated while
traditional methods usually have to transform the item image into a vector
directly. So these two aspects make it very very difficult to explain
recommendations with current recommendation models. It is very hard on the
other hand with these aspects to generate an explainable recommendation with
the current recommendation models (Wu et al., 2019; Kang et al., 2017; Xu
Chen, 2018) that are currently being used in the industry.

####  2.4.2. Proposed Approach

So for that this the paper (Hou et al., 2019) basically proposes a novel
semantic attribute explainable recommendation system as a fine-grained
interpretable space name semantic attribute space is introduced in which each
Dimension corresponds to a semantic attribute. So basically they project the
users and items into this space. The users’ fine-grained preferences are being
able to generate explainable recommendations specifically if they first
develop a semantic extraction Network that can be used to extract the region
specific attribute representations. Then by this each item is then projected
to the semantic attribute space and then you can easily capture the diversity
of semantic attribute. The design aspects contain a fine-grained preferences
attention FPA module which basically does that it automatically matches and
the user preferences for each given attribute in the space and aggregate all
these attributes with different weights. So now each attribute has a weight of
it’s own so in the end what happens is that finally they optimize the SAERS
models in Bayesian personalized rank BPR framework, which not only
significantly improves and out performs several base lines on the visual
recommendation task, but it also sort of provides interpretable insights by
highlighting attribute semantics in a personalized manner. Basically, what
they have done is that previously as we know that these attempts were made to
capture users’ visual preferences, but in order to make institutional
explanations for the recommendations, they were pretty much very limited on
item level. So the paper basically takes a further step to discuss the user
preferences on Visual attribute level.

Table 3. List of semantic attributes used in method Category | Attribute: Class
---|---
Top | high neck: ruffle semi-high, turtle,…
| collar: rib collar, puritan collar,…
| lapel: notched, shawl, collarless,…
| neckline: V, square, round,…
| sleeves length: sleeveless, cap, short,…
| body length: high waist, long, regular,…
Bottom | skirt length: short, knee, midi, ankle,…
| trousers length: short, mid, 3/4, cropped,…
Shoes | heel height: flat, 1 in-7/4 in, under 1 in,…
| boots height: ankle, knee-high, mid-calf,…
| closure: lace-up, slip-on, zipper,…
| toe style: round, pointed, peep, open,…

With their semantic attribute explainable recommendations system. They
basically bridge the gap and utilize a new semantic attribute visual space in
which each Dimension represents an attribute that corresponds to the region
that basically different regions of the clothing items are usually split into
several semantics attributes via the extraction Network and then they are
later projected into the visual space. So later the users are projected
according to the Fine graded preferences for clothing attributes. So this all
makes it easily for them to obtain the fashion item projection in the semantic
feature space. And from there they can use the FPA to project users into the
same semantic feature space. Here FPA is the Fine grain preferences attention
where they jointly learned the item representation in both Global visual space
and semantic attribute visual space under a pairwise learning framework. And
with this they are able to generate the explainable recommendations.

###  2.5. Complementary Recommendations Using Adversarial Feature Transformer

Traditional procedures for complementary product hints depend on behavioral
and non-visible facts along with consumer co-perspectives or co-buys. However,
positive domain names along with style are often visible. Recommendation
algorithms are important to many business applications, specially for online
shopping. In domain names along with style, clients are seeking out apparel
hints that visually supplement their modern outfits, styles, and wardrobe.
Which the conventional strategies do now no longer cater to.

####  2.5.1. Previous Methods

Now we have seen that there are traditional content-based and collaborative
recommendation algorithms (Adomavicius and Tuzhilin, 2005; Lew et al., 2006).
But among these collaborative filtering approaches (Koren and Bell, 2015;
Melville et al., 2002) are the common ones that primarily rely on behavioral
and historical data such as you know, Co purchases , the views and past
purchases to suggest new items to customers. So this work basically on
providing complimentary item recommendations for a given query item based on
visual cues.

####  2.5.2. Proposed Approach

So basically what this paper (Huynh et al., 2018) does is that it proposes a
framework in which they harness visual clues in an unsupervised manner in
order to learn the distribution that exists between co-occurring complimentary
items in real world images. The model runs are nonlinear transformations
between two manifolds of source and Target complimentary item categories, for
example, a top and a bottom in an outfit. And training it on a large data set
they train generative Transformer Network directly on the feature
representation space by just casting it as an Adversarial Optimization
problem. Now such a conditional generative model can produce multiple novel
samples of complimentary items in the feature space for a given query item.Now
for that they develop an unsupervised learning approach for complementary
recommendation using adversarial feature transform CRAFT by learning the co-
occurrence of item pairs in real images. So the Assumption here is that the
co-occurrence frequency of item pairs is sort of a strong indication of
likelihood of their complementary relationship. So the paper advises a defined
and adversarial process to train a conditional generative Transformer Network
which can then learn the joint distribution of item pairs by observing samples
from the real distribution.

Now their approach is quite novel and unique in a certain way that they
utilize generative adversarial training with several advantages over
traditional generative adversarial network (GAN) (Goodfellow et al., 2014)
based image generation. Well, we know that the quality of visual image
generation using GANs has improved a lot but it still lacks the realism
required for many real world applications and fashion apparel is one of them.
And more importantly if we see that their goal of recommendation systems in
certain types of application is often not to generate synthetic images, but
they have to recommend real images from a catalog of items. Now we know that
an approach that generates synthetic images will still need to perform a
search and that will be typically done by searching in the feature space in
order to find the most visually similar image in the catalog. Now CRAFT
directly generates these features of the recommended items and bypasses the
need to generate synthetic images and enable a simpler and more efficient
algorithm. So by working in a feature space, what they do is that they can use
a simpler Network architecture that improves stability during the training
time and avoid common pitfalls such as model collapse (Che et al., 2016).

Figure 4. Generating recommendations using the proposed CRAFT network.

####  2.5.3. Network Architecture

The network architecture basically comprises several steps and first is the
selection of appropriate visual representations for the source and Target
images. Then what they do is that the encoding which are fixed feature
representations are generally derived from pre-trained CNN’s. Typically it is
advisable to use application specific feature representations, for example,
apparel feature embeddings for clothing recommendations, but a general
representation such as one trained on ImageNet (Deng et al., 2009) or MS-COCO
(Lin et al., 2014) offer nice efficient alternatives. So as shown in figure,
what basically is happening, is that the source and the target feature
encoders $E_{s}$ and $E_{t},$ respectively are fixed and are used to generate
feature vectors for training and inference. Now, the architecture resembles
traditional Grand designs with two main components , a conditional feature
transformer and a discriminator. The role of the feature transformer is to
transform the source feature $s_{q}$ into a complementary target feature
$\hat{t}_{q}.$ The input to the transformer also consists of a random noise
vector $z$ sampled uniformly from a unit sphere in a $d_{z}$ -dimensional
space. By design, the transformer is generative since it is able to sample
various features in the target domain.

As the transformer consists of several fully-connected layers in which each is
followed by batch normalization (Ioffe and Szegedy, 2015) and leaky ReLU
(Maas, 2013) activation layers. The discriminator is commensurate to the
transformer in capacity, consisting of the same number of layers. This helps
balance the power between the transformer and the discriminator in the two-
player game, leading to stable training and convergence.

From a query image, the query feature $f$ is extracted by the source encoder,
$E_{s},$ and multiple samples of transformed features
$\left\\{\hat{t}_{i}\right\\}$ are generated by sampling random vectors
$\left\\{z_{i}\right\\}.$ Now basically what it does is that it allows them to
generate a diverse set of complementary recommendations by sampling the
underlying conditional probability distribution function. And when they
performed a nearest neighbor search within a set of pre-indexed target
features extracted using the same target encoder, $E_{t},$ used during
training. Actual recommendation images were retrieved by a reverse lookup that
maps the selected features to the original target images.

Figure 5. Complementary recommendation for a common query item (dark jeans)

####  2.5.4. Performance evaluation

The feature transformer in CRAFT samples from a conditional distribution to
generate diverse and relevant item recommendations for a given query. The
recommendations generated by CRAFT are preferred by the domain experts over
those produced by competing approaches.

###  2.6. Neural Compatibility Modeling

It’s easy these days where fashion communities are online and we can
experience that a lot of fashion experts are publicly sharing their own
fashion tips by showing how their outfit compositions work , where each item a
top or a bottom usually has an image and context metadata title and category.
With such Rich information, fashion data offers us an opportunity to
investigate the code in clothing matching. Now we know that the colors,
materials and shape are some aspects that affect the compatibility of fashion
items and also each fashion item involves multiple modalities and also if we
notice that the composition relation between fashion items is rather sparse.
Now this makes Matrix factorization methods not applicable.

Figure 6. Illustration of the proposed scheme. They employed a dual
autoencoder network to learn the latent compatibility space, where they
jointly model the coherent relation between visual and contextual modalities
and the implicit preference among items via the Bayesian personalized ranking.

####  2.6.1. Previous Methods

The recent advancement in these Fashion aspects has been done, but the
previous models (Iwata et al., 2011; Hu et al., 2015; McAuley et al., 2015;
Liu et al., 2012) proposed were lacking in terms of how they wanted to
approach this subject.

####  2.6.2. Proposed Approach

So what this paper (Song et al., 2017) proposes is a content-based neural
scheme that models the compatibility between fashion items based on the
Bayesian personalized ranking BPR framework. Now this scheme jointly models
the coherent relation between modalities of items and their implicit matching
preference.So basically they propose focusing on modeling the sophisticated
compatibility between fashion items by seeking the nonlinear latent
compatibility space with neural networks. And they also were able to aggregate
the multimodal data of fashion items and exploit the inherent relationship
that basically exists between different modalities to comprehensively model
the compatibility between fashion items.

Now we know that it is not correct to directly measure the compatibility
between fashion items from a distinct space due to their heterogeneity. So for
that the author’s they assume that there exists a little compatibility space
that is able to bridge the gap between heterogeneous fashion items where
highly compatible fashion items share the similar style material which can
show high similarity or functionality should also show high similarity. For
example a wide casual T-shirt goes really well with black jeans, but it does
not go with a black suit while a pair of high boots prefer skinny jeans rather
than flared pants. So they further go along and assume that the subtle
compatibility factors lie in a highly nonlinear space that can be learned by
the advanced neural network models. So they employ the auto encoders networks
to learn the latent space which has been proven to be effective in the latent
space learning.(Wang et al., 2016)

To fully take advantage of the implicit relation between tops and bottoms,
basically what they did was that they naturally adopt the BPR framework and
assumed that bottoms from the positive set $\mathcal{B}_{i}^{+}$ are more
favorable to top $t_{i}$ than those unobserved neutral bottoms. According to
BPR, built a training set:

$\mathcal{D}_{S}:=\left\\{(i,j,k)\mid
t_{i}\in\mathcal{T},b_{j}\in\mathcal{B}_{i}^{+}\wedge
b_{k}\in\mathcal{B}\backslash\mathcal{B}_{i}^{+}\right\\}$

where the triple $(i,j,k)$ indicates that bottom $b_{j}$ is more compatible
than bottom $b_{k}$ with top $t_{i}$

Then according to(Rendle et al., 2012) , they got the following objective
function,

$\mathcal{L}_{bpr}=\sum_{(i,j,k)\in\mathcal{D}_{S}}-\ln\left(\sigma\left(m_{ij}-m_{ik}\right)\right)$

Taking the modality consistency into consideration, they got the following
objective function:

$\mathcal{L}=\mathcal{L}_{bpr}+\gamma\mathcal{L}_{mod}+\mu\mathcal{L}_{rec}+\frac{\lambda}{2}\|\Theta\|_{F}^{2}$

where

$\mathcal{L}_{\text{rec}}=\mathcal{L}_{\text{rec}}^{v}+\mathcal{L}_{\text{rec}}^{c}$

with

$\mathcal{L}_{\text{rec}}^{v}=\Sigma_{(i,j,k)\in\mathcal{D}_{S}}\left(l\left(\mathbf{v}_{i}^{t}\right)+l\left(\mathbf{v}_{j}^{b}\right)+\right.\left.l\left(\mathbf{v}_{k}^{b}\right)\right)$

and

$\mathcal{L}_{rec}^{c}=\Sigma_{(i,j,k)\in\mathcal{D}_{S}}\left(l\left(\mathbf{c}_{i}^{t}\right)+l\left(\mathbf{c}_{j}^{b}\right)+l\left(\mathbf{c}_{k}^{b}\right)\right)\cdot\mu,\gamma,\lambda$

are non-negative trade-off hyperparameters. $\Theta$ refers to the set of
network parameters (i.e., $\mathbf{W}_{k}$ and
$\left.\hat{\mathbf{W}}_{k}\right)$. The last regularizer term is designed to
avoid overfitting.

##  3\. Aesthetics and Fashion

The word aesthetic (of Philosophy, 2009) was basically introduced in the 18th
century where it has come to be used to designate among other things a kind of
object, a kind of judgment, a kind of attitude or experience and a kind of
value. Where aesthetic comes the concept of aesthetic descends usually from
the concept of taste. So in the 18th century, the theory of taste emerged in
part as a corrective to the rise of rationalism particularly as applied to
Beauty and the rise of egoism particularly as applied to virtue.

###  3.1. Mapping Association

So how do people usually describe clothing ,so there are words like informal,
casual ,formal ,party, where they are usually used. But the recent focus on
recognizing or extraction of the features that are available visually in
clothing is pretty much different.

####  3.1.1. Background

To accurately guess that, the authors in the paper (Jia et al., 2016) describe
a way to bridge the gap between visual features and the aesthetic words. So
what they basically do is that they formulate a novel three-level framework
visual features (VF) - image-scale space (ISS) - aesthetic words space (AWS)
and then they leverage the Art field image scale space which serves as an
intermediate layer. So firstly they proposed a stacked diagnosing auto encoder
Guided by correlative labels SDAEGCL, to map the visual features to the image
scale space and then with that accordingly what they do is that the semantic
distance is computed by the Wordnet similarity (Pedersen et al., 2004). They
map the most often using static words available and being used by people in
the online clothing shops to the image scale space. Now, what they do is that
they employ the upper body menswear images that they have downloaded from
several different online shops as their experimental data and they proposed a
3-level framework that can help to capture the relationship that is standing
between visual features and aesthetic words. It is quite important for people
to wear aesthetically and properly and specifically given a user input
occasion wedding ,shopping or dating ,a system should be able to suggest the
most suitable clothing that is from the user’s own clothing available. So
another paper (Liu et al., 2012) similar idea was mentioned where the two
criterion’s are explicitly considered for the system where it is paid heed to
wear properly and to wear aesthetically like for example that red T shirt
matches better with white pants than green pants and to basically narrow down
the semantic Gap that is between the low-level features of clothing and the
high-level occasion categories. From where these clothing attributes are
treated as latent variables in the support Vector machine based recommendation
model. But nevertheless the matching rules cannot reveal the aesthetic effects
holistically and lacked Interpretability.

Figure 7.  Examples of clothing images and their corresponding aesthetic
words.

####  3.1.2. Proposed Approach

So the paper (Jia et al., 2016) basically aims to bridge the gap between
visual features and aesthetic words of clothing where in order to capture the
intrinsic and holistic relationship between them they sort of introduce a
middle layer ,intermediate layer and form a novel three-level framework, which
is based on the proposed Theory by Kobayashi (Kobayashi, [n.d.]). Where two
dimensional space warm cool and hard soft aspects are applied in the art
design.

Basically the contribution of the papers is that they build an association
between clothing images and aesthetic words by proposing a three-level
framework. It basically does a novel notation of using the 2D continuous image
scale space as a layer that is intermediate with a very strong ability of
description thus it facilitates the deep and high-level understanding of
aesthetic effects. And secondly what it does is that the paper proposes a
stacked denoising auto-encoder Guided by correlative labels SDAEGCL to
implement mapping of visual features to the image scale space and that can
amend the random error existing in initial input and make full use of the
information of both labeled and unlabeled data and moreover we can also find
that the stack methods improve the representation capability of model by
adding more hidden layers.

So basically Kobayashi proposed 180 keywords into different 16 categories of
Aesthetics and defined their coordinate values in the image scale space. But
as in fashion, there are some words that are unrelated like alert ,robust,
sad, happy. These are not something that we usually use to describe clothing.
So first the authors sort of removed manually all these not often used words
and established a static word space $Y$ for clothing containing 527 words.

Now in order to illustrate how to map the aesthetic words $y_{i}\left(\forall
y_{i}\in Y\right)$ to the image-scale space $D.$ To determine the coordinate
value $D_{y_{i}}\left(wc_{y_{i}},hs_{y_{i}}\right)$ of an aesthetic word
$y_{i}\in Y,$ the authors basically first define the 180 keywords as keyword
${}_{j}(j=1,2,\cdots,180)$ and calculate the semantic distances between
$y_{i}$ and each keyword j using WordNet::Similarity . Then what they do is
that they basically pick 3 keywords with the shortest distances
$d_{i_{1}},d_{i_{2}}$ and $d_{i_{3}},$ marking the coordinate values of these
3 keywords as
$D_{i_{1}}\left(wc_{i_{1}},hs_{i_{1}}\right),D_{i_{2}}\left(wc_{i_{2}},hs_{i_{2}}\right)$
$D_{i_{3}}\left(wc_{i_{3}},hs_{i_{3}}\right).$ Afer that they take the
reciprocals of distances $rec_{i_{1}}$ rec ${}_{i_{2}},$ rec ${}_{i_{3}}$ as
weights (e.g. rec ${}_{i_{1}}=\frac{1}{d_{i_{1}}}$ ), the weighted arithmetic
mean 1 of $D_{i_{1}},D_{i_{2}}$ and $D_{i_{3}}$ can also be regarded as the
coordinate value $D_{y_{i}}\left(wc_{y_{i}},hs_{y_{i}}\right)$ of $y_{i}.$ The
formula is shown as follows:

$wc_{y_{i}}=\frac{\sum_{k=1}^{3}wc_{i_{k}}\cdot
rec_{i_{k}}}{\sum_{k=1}^{3}rec_{i_{k}}},hs_{y_{i}}=\frac{\sum_{k=1}^{3}hs_{i_{k}}\cdot
rec_{i_{k}}}{\sum_{k=1}^{3}rec_{i_{k}}}$

So by this way what they do is that for each $y_{i}\in Y,$ they basically
calculate its coordinate value $D_{y_{i}}$ in the image-scale space as
$\left(wc_{yi},hs_{yi}\right).$ To label an input clothing image $v$ with an
aesthetic word, they use the proposed SDAE-GCL to predict its coordinate value
$D_{v}\left(wc_{v},hs_{v}\right)$ in $D.$ Then, after that they find a word
$y_{v}\in Y$ whose corresponding coordinate value $D_{yv}$ has the shortest
Euclidean distance to the $D_{v}$. Thus, $y_{v}$ can be regarded as the
aesthetic word of image $v$

###  3.2. Brain-inspired Deep Network

Now we know that most existing methods sort of rely on conventional features
in order to represent an image. Such features that can be extracted by
convolutional neural networks are the scale-invariant feature, transform
algorithm, color histogram and so on but one important type of feature is the
aesthetic feature and as we have already discussed it before it plays an
important role in clothing and specially in clothing recommendation since
users largely depend on whether the clothing is in line with their aesthetics
or not.

####  3.2.1. Previous Methods

Now we have seen in some papers (Han et al., 2017; Hsiao and Grauman, 2017;
McAuley et al., 2015; Vasileva et al., 2018) in which there was a
recommendation for different fashion garments for an unfinished outfit. But
their goal was different from the one mentioned in this paper. That is
basically that they focused on clean per-garment catalog photos and the
recommendations were mostly restricted to retrieve garments from a specific
data set. Now the only feature in those recommendation systems was that they
were adding to the Garment. Most prior fashion work addresses recognition
problems, like matching street-to shop (Kalantidis et al., 2013; Kiapour et
al., 2015; Yan, 2012; Vittayakorn et al., 2015) But in this case, what they
are doing is that they are saying that some problems demand going beyond
seeking an existing garment and adding to it and for that, they said that
there are garments which are detrimental and it should be taken off. You know
like cuff the jeans above the ankle or how to adjust the presentation and
detail of them within a complete outfit to improve its style.

####  3.2.2. Background

So in order to bridge the gap there are a lot of different methods but we are
going to discuss another one (Yu et al., 2018) which introduces the intense
static information. Which is highly relevant with user’s preference into the
clothing recommendation system. So what they basically do, is that the
aesthetic feature extracted by the pre-training on network, which is a brain
inspired deep structured trained for the assessment task of Aesthetics. So for
that they consider the aesthetic preference which varies significantly from
user to user as different people have different sorts of reference in
Aesthetics. So they proposed a new tensor factorization model that
incorporates the static features in a very personalized manner. So what they
do is that they conduct different experiments and demonstrate that the
approach they are putting forward captures the static preference of the user.
It significantly outperforms the already available state-of-the-art
recommendation methods.What happens is that usually when we are shopping for
clothing on the web. We used to look through product images before making a
certain decision before buying that thing and product images usually provide a
lot of information including design, color schemes ,patterns structure and so
on. We can get an estimation of the thickness and quality of a product from
its images. As such product images play a lot of key roles in the clothing
recommendation task. So what the authors in this paper do is that they
leverage this information and enhance the performance of the existing clothing
recommendation systems.

Figure 8.  Brain-inspired Deep Network (BDN) architecture.

However, an important factor regarding aesthetics is that it has been
considered not much in previous researchers’ research. So basically what
happens is that while most user’s concern regarding clothing is that the
product should be good looking. What happens is that the author’s use the
static Network to extract relevant features that is between an aesthetic
network and a CNN. That are demonstrated and they proposed a brain inspired
deep Network, which is a deep structure trained for image aesthetic assessment
that inputs several raw features that are indicative of aesthetic feelings
like hue, saturation, color, duotones ,complementary color etc. And what is it
that extracts high-level aesthetics from these barely barely raw features.

####  3.2.3. Proposed Approach

So the paper works on BDN that is utilized to strike the holistic feature in
order to represent the static elements of a clothing. And as different people
prefer different aesthetic tastes. So to capture the diversity of the
aesthetic preference among different consumers and over different times. They
exploit tensor factorization as a basic model. Now, there are several ways to
decompose a tensor however, there are certain drawbacks in existing models
(Kolda and Bader, 2009; Rendle and Schmidt-Thieme, 2010; Sidiropoulos et al.,
2016) . So what they do is that they address the clothing recommendation task
better and propose a dynamic collaborative filtering DCF model that is trained
with coupled matrices to mitigate the sparsity problem. And then afterwards
they combined the models with Bayesian personalized ranking optimization
criteria and evaluated the proper performance on an Amazon clothing dataset.
So basically what they are doing is that they are proposing an novel DCF model
to portray the purchase events in three dimensions: user, items, and time and
then incorporate the aesthetic features into DCF and train it. And of course,
they are leveraging the novel aesthetic features in recommendation to capture
consumers specific aesthetic preference and they compare the effect with
several conventional features to demonstrate the necessity of the aesthetic
features.

So in order to illustrate the hybrid model that integrates image features into
the basic model the DCFA. They first introduced the basic tensor factorization
model DCF. So the basic model is the impact of time on aesthetic preference.
So what they do is that they proposed a context-aware model as the basic model
to account for the temporal factor. What they do is that they use P × Q × R
tensor a to indicate the purchase events among the users clothes and time
dimensions. So if a user P purchase an item Q in the time interval R

Then ${\mathrm{A}}_{pqr}$ is 1 otherwise, it will be 0. so for that the tensor
factorization has been widely used to predict all the missing entries 0
elements in $A$ which can be used for recommendation. So as the previous
models have some limitations what they do is that they proposed a new tensor
factorization method in which a user makes a purchase by deciding a product
and there are two primary factors. So the first one is that if the product
fits the users preference and the appearance is good looking or appealing to
that specific user. And if the time is correct that if it’s in the season and
fashionable, for example, of course winter clothing cannot be recommended or
aesthetically fine if it’s being recommended in the summer season, so for user
$p$, clothing $q$, and time interval $r$, they use the scores $S_{1}$ and
$S_{2}$ to indicate how the user likes the clothing and how the clothing fits
the time respertively. $S_{1}=1$ when the user likes the clothing and
$S_{1}=0$ otherwise. Similarly, $S_{2}=1$ if the clothing fits the time and
$S_{2}=0$ otherwise. The consumer will buy the clothing only if $S_{1}=1$ and
$S_{2}=1,\mathrm{so},\hat{\mathrm{A}}_{pqr}=S_{1}\&S_{2}.$ To make the formula
differentiable, they approximately formulated it as
$\hat{\mathrm{A}}_{pqr}=S_{1}\cdot S_{2}.$ And the presented $S_{1}$ and
$S_{2}$ in the form of matrix factorization:

$\begin{array}[]{l}S_{1}=\sum_{i=1}^{K_{1}}\mathbf{U}_{ip}\mathbf{V}_{iq}\\\
S_{2}=\sum_{j=1}^{K_{2}}\mathbf{T}_{jr}\mathbf{W}_{jq}\end{array}$

where $\mathrm{U}\in\mathbb{R}^{K_{1}\times
P},\mathrm{V}\in\mathbb{R}^{K_{1}\times
Q},\mathrm{T}\in\mathbb{R}^{K_{2}\times R},$ and
$\mathrm{W}\in\mathbb{R}^{K_{2}\times Q}.$ The prediction is then given by:

$\hat{\mathrm{A}}_{pqr}=\left(\mathrm{U}_{*p}^{\mathrm{T}}\mathrm{V}_{\cdot
q}\right)\left(\mathrm{T}_{*r}^{\mathrm{T}}\mathrm{W}_{*q}\right)$

We can see that in Equation that the latent features relating users and
clothes are independent with those relating clothes and time. Though $K_{1}$
-dimensional vector $\mathrm{V}_{*q}$ and $K_{2}$ -dimensional vector
$\mathrm{W}_{*}q$ are all latent features of clothing $q,\mathrm{V}_{*q}$
captures the information about users” preference intuitively whereas
$\mathrm{W}_{*}q$ captures the temporal information of the clothing. The model
is more expressive in capturing. The underline related patterns in purchases.
Moreover this model is efficient and easy to train compared with the Tucker
decomposition.

###  3.3. Minimalistic Approach

We know that the physical attributes of a product are very much influencing
the buying behavior. (Streamoid, [n.d.]) We also know that the aesthetic calls
intuitively while we shop. so it may not even be you know, the person might
not even be aware of making multiple decisions on every product, for example,
you know like the style but not the color of the product. Various aspects of
our life influence the style of how we dress. Every look that we wear tells a
different story about us. So basically it communicates a certain image
representation which is you know decoded by others within their own cultural
context. So it is sort of possible that the Aesthetics of a garment is similar
for all in a particular society.

####  3.3.1. Background

So when we look into a garment, what are the main things that we should or we
usually look into. Queries like. so can I wear it? , What occasion it would
suit and how does it make me feel? And also another precise preference is ,
you know included in this aspect and how does it reflect their own
personality. So these are just a few of the questions that we usually, ask
ourselves when we are out shopping and when we want to wear clothes that are
aesthetically pleasing. But as we have seen in this new modern era that
minimalism is getting into every aspect of life and people are tending to move
towards simpler versions, but aesthetically pleasing ones. As Coco Chanel has
said:

“before you leave the house look in the mirror and take one thing off”

So minimal outfit edits in an already used outfit they can use to change the
existing outfit and improve its fashionability. Whether it can be removing an
accessory selecting a blouse with a higher neckline or you know, just tucking
your shirt in or simply, you know, changing the pants to a darker color. So
these all small adjustments are accountable for a more stylish outfit that is
more aesthetically pleasing to a large group of people or to your own self as
well.

####  3.3.2. Proposed Approach

So motivated by these observations which made the authors of this particular
paper (Hsiao et al., 2019) go for the minimal edits for fashion outfit
improvement. So minimally editing an outfit and getting an algorithm must
impose alternations to the garments and accessories that are slight, yet
visibly improve the overall fashionability. So basically what they’re doing is
that a minimal edit need not strictly minimize the out amount of change rather
it incrementally adjust in an outfit as opposed to starting from scratch. So
basically, it can be a recommendation regarding which garment you need to you
know, replace or take off or you know to swap out or simply, you know, just
wear the same garment in a better way.

And also it is well known that clothing fashion is sort of just intuitive and
often a habitual trend in the style in which you know, an individual usually
dresses but it is sort of not clear which visual stimulus places higher or
lower significance or influence on the updation of clothing and fashion
trends. So another paper (Zou et al., 2016) that we have seen in which they
have employed machine learning techniques in order to analyze the influence
that the visual stimuli of different clothing fashion are having on the
fashion trends and specifically classification-based model was proposed by
them that quantified the influence of different visual stimuli in which each
stimuli influenced was quantified by, you know, it’s a corresponding accuracy
in fashion classification. So experimental results also, demonstrated that if
they were quantifying style color and texture so out of those three on
clothing fashion updates the style holds a higher influence than the color.
And the color holds a higher influence than the texture. So all of these are
very important in determining the Aesthetics as well.

Figure 9.  Overview of our Fashion $++$ framework. We first obtain latent
features from texture and shape encoders $E_{t}$ and $E_{s}$. Our editing
module $F^{++}$ operates on the latent texture feature $t$ and shape feature
s. After an edit, the shape generator $G_{s}$ first decodes the updated shape
feature $s^{++}$ back to a $2\mathrm{D}$ segmentation mask $\mathrm{m}^{++},$
and then we use it to region-wise broadcast the updated texture feature
$\mathrm{t}^{++}$ into a $2\mathrm{D}$ feature
$\operatorname{map}\mathbf{u}^{++}.$ This feature map and the updated
segmentation mask are passed to the texture generator $G_{t}$ to generate the
final updated outfit $x^{++}$.

So basically the main idea and approach for this model. Is that the activation
maximization method. That works on localized encodings from a deep image
generation Network. So what they basically do is that you give them an
original outfit and they map it’s composing pieces for example, you know, the
bag, boots, jeans. blouse to their respective codes. And then what they do is
that they use a discriminative fashionability model for the editing in which
it gradually updates the encodings in the direction that maximizes the outfit
score so when they do this, they are hence improving its style. And also the
update trajectory offers various ranges of edits starting from you know, the
least changed and going towards the item that is most fashionable from you
know, which users can choose a preferred endpoint. The approach basically says
that it provides its outputs in two formats:

  1. (1)

Retrieved garments from an inventory that would best achieve its
recommendation.

  2. (2)

And the second one is rendering of the same person in the newly adjusted look
generated from the edited outfits encoding

####  3.3.3. System Working

So basically, what they do is that they present an image generation framework,
which is comprised of outfit images into their garment regions and factorizes
shape/fit and texture in support of the later objectives. So the framework is
basically about coordination of all composing pieces defines and outfits look.
What they do is that they can control which parts like the pants or the skirts
or you know shirts and then aspects like the length of their sleeve, color,
the pattern and neckline to change and sort of, you know, keep the identity
and fashion irrelevant factors unchanged. So what they want to do is they want
to explicitly model their spatial locality and to perform minimal edits. So
what they needed to do was to control the piece’s textures as well as their
shapes. So basically what textures comprise in outfits is for example, like in
denim with solid patterns gives more casual look or like leather with red
colors, give more street style look. So with the same material color and
pattern of garment and how they are worn, you know, like tucked in or pulled
out and skinny or baggy pants and you know, what sort of cut they have v-neck
or turtleneck or you know boatneck. So the Garment will compliment a person’s
silhouette in different ways. So what they do is that they account for all of
these factors and devise an image generation framework that gives control over
individual pieces accessories body parts and also factorize the shapes from
the texture.

For computing an edit the main steps are: calculating the desired edit, and
generating the edited image. For calculation of an edit, they basically took
an activation maximization approach where they iteratively alter the outfit’s
feature such that it increases the activation of the fashionable label
according to $f$. Formally, let
$\mathbf{z}^{(0)}:=\left\\{\mathbf{t}_{0},\mathbf{s}_{0},\ldots,\mathbf{t}_{n-1},\mathbf{s}_{n-1}\right\\}$
be the set of all features in an outfit, and
$\tilde{\mathbf{z}}^{(0)}\subseteq\mathbf{z}^{(0)}$ be a subset of features
corresponding to the target regions or aspects that are being edited ( $e.g.,$
shirt region, shape of skirt, texture of pants). The updated outfit’s
representation is as follows:

$\tilde{\mathbf{z}}^{(k+1)}:=\tilde{\mathbf{z}}^{(k)}+\lambda\frac{\partial
p_{f}\left(y=1\mid\mathbf{z}^{(k)}\right)}{\partial\tilde{\mathbf{z}}^{(k)}},k=0,\ldots,K-1$

where $\tilde{\mathbf{z}}^{(k)}$ denotes the features after $k$ updates,
$\mathbf{z}^{(k)}$ denotes substituting only the target features in
$\mathbf{z}^{(0)}$ with $\tilde{\mathbf{z}}^{(k)}$ while keeping other
features unchanged, $p_{f}\left(y=1\mid\mathbf{z}^{(k)}\right)$ denotes the
probability of fashionability according to classifier $f$, and $\lambda$
denotes the update step size. Each gradient step yields an incremental
adjustment to the input outfit.

####  3.3.4. Performance evaluation

This Approach makes slight yet noticeable improvements better than baseline
methods in both quantitative evaluation and user studies and it effectively
communicates to users through image generation and supports all possible edits
from swapping, adding, removing garments to adjusting outfit presentations
through qualitative examples.

###  3.4. Neuroaesthetics

Mark Twain has said that the “Finest Clothing made is a person skin”, but of
course society demands something more than this. Now, we know that fashion has
a tremendous impact on our society and clothing is basically something that
reflects the person’s social status and thus puts pressure on how they are to
dress to, you know, fit a particular occasion. For this the authors of this
particular paper (Simo-Serra et al., 2015) analyze the fashion of clothing of
a large social website in which their main aim is to learn and predict how
fashionable a person looks on a photograph and suggest subtle improvements
that they can make in order to improve their image and appeal.

####  3.4.1. Previous Methods

Now the approach these authors have suggested is also somewhat related to
recent approaches (Dhar et al., 2011; Gygli et al., 2013; Isola et al., 2013;
Khosla et al., 2014) that were aimed at modeling the human perception of what
beauty actually is. So in papers these authors basically address the questions
of what makes a particular image memorable and interesting or you know popular
to viewers. So this line of work usually contains mining of large image data
sets in order to you know, find a relation of visual cues to popularity
scores. But in this paper what they do is that they tackle the problem of
predicting fashionability. So they are going a step further from the previous
work by identifying High-level semantic properties that cause a particular
aesthetic score which can be then conveyed to the user so that they can
improve their outfit or their look. And this work is very much closest to
(Khosla et al., 2013) which was able to infer whether our faces are memorable
or not and then upon that results modify it such that it becomes. Although
this is quite different as their domain is different and it is also different
in formulation.

####  3.4.2. Proposed Approach

So they are modeling the perception of fashionability. And for that what they
have done is that they have proposed a conditional random field model that
jointly reasons about several fashionability factors such as the type of
outfit and garments that an individual is wearing and the type of user and the
photograph setting for example, the scenery and fashionability score. And
based on that they give the recommendation to user in which they convey which
garments or scenery the individual should change in order to improve
fashionability.

This paper predicts how fashionable a person looks on a particular photograph.
So the fashionability is then affected by the clothes the subject is wearing
and also by a large number of other factors such as how appealing they are in
a scene that is containing that person and how that image was taken and how
appealing visually the person is ,their age and also the garment itself being
fashionable is not a perfect indicator of someone’s fashionability as people
typically judge how well the garments aligned with someone’s look, body,
characteristic or even personality. So the model proposed exploit several
domain inspired features which include beauty, age and mood inferred from the
image. And the scene and the type of photograph and if available metadata in
the form of where the user is from, how many online followers he/she has the
and the sentiment of comments by other users. For this they have to create
their own data set from different online sources. And if we see our daily
lives we can see how much of an impact fashion has in it. So this also proves
the growing interest in clothing related applications in Vision community.
Early work (Jammalamadaka et al., 2013; Simo-Serra et al., 2014; Yamaguchi et
al., 2013, 2012; Yang et al., 2015) that was focused was mainly on clothing
parsing in terms of diverse set of garments types.The paper’s objective was
basically to be able to predict fashionability of a given post, but they also
wanted to build a model that can understand fashion at a higher level. So for
that purpose what they did was they made a Conditional Random Field (CRF) to
learn the different outfits , types of peoples and settings. Now here the word
setting is basically something that describes the location where the picture
is taken and both at a scenic and geographic level. They use their own fashion
data set fashion144k Images and metadata to produce accurate predictions of
how fashionable a certain person is.

More formally, let $u\in\left\\{1,\cdots,N_{U}\right\\}$ be a random variable
capturing the type of user, $o\in\left\\{1,\cdots,N_{O}\right\\}$ the type of
outfit, and $s\in\left\\{1,\cdots,N_{S}\right\\}$ the setting. Further, we
denote $f\in\\{1,\cdots,10\\}$ as the fashionability of a post $\mathbf{x}$.
They represented the energy of the CRF as a sum of energies encoding unaries
for each variable as well as non-parametric pairwise potentials which
reflected the correlations between the different random variables. It is
defined as:

$\displaystyle E(u,o,s,f)$
$\displaystyle=E_{user}(u)+E_{out}(o)+E_{set}(s)+E_{fash}(f)$
$\displaystyle+E_{np}^{uf}(u,f)+E_{np}^{of}(o,f)+E_{np}^{sf}(s,f)$
$\displaystyle+E_{np}^{uo}(u,o)+E_{np}^{so}(s,o)+E_{np}^{us}(u,s)$

Figure 10. An overview of the CRF model and the features used by each of the
nodes.

####  3.4.3. Performance Output

An exciting property of this specific model was that it could be used for
outfit recommendation.What they basically did was they used to take a post as
an input and estimated the outfit that maximizes the fashionability while the
kept the other variables fixed. So basically what was happening was that they
were predicting what the user should be wearing in order to increase their
looks instead of their current outfit. And this can be just one example of the
flexibility of the approach. They proposed other thoughts such as what would
be the low fitting outfit and what would be the best place to go with the
current outfit or you know, what type of users this outfit fits the most, this
can be done with this same model.

##  4\. Personalisation in Fashion

One of the key aspects in fashion is personalization. So personalization is
basically something that is intended for a certain individual based on their
likes and dislikes and what they cater as good for them. And we know that
fashion industry included e-commerce worldwide is supposed to hit the 35
billion dollar Mark by 2020 this year and there’s a need for applications
which can help the user in making Intelligent Decisions on their day-to-day
purchases or a system that can recommend them a model or something that is
personalized to their liking.

###  4.1. Personalized Outfit Recommendation with Deep Neural Network

So for this purpose the use of deep neural networks for this challenge is
needed and we are going to discuss one of a system that is dubbed as
FashionNet (He and Hu, 2018) that consists of basically two components: a
feature Network for the feature extraction function and a matching Network for
the compatibility computation. The former one is achieved through a deep
convolutional Network and the second one for that they adopt a multi-layered
fully connected Network structure and design, and compare the three
alternative architectures for FashionNet and to achieve personalized
recommendations, what they do is that they develop a two stage training
strategy, which uses the fine-tuning technique to sort of transfer a general
compatibility model to the model that embeds personal preference.

####  4.1.1. Previous Methods

Now we know that existing recommender systems are heavily dependent on the
collaborative filtering techniques CF which basically uses historical ratings
given to the item by users as the sole source of information for their
learning expect and the performance is very much sensitive to the sparsity
level of user item metrics. The recent progress of deep neural networks
provides promising solution to the representation problem of image
content(Lecun et al., 1998; Krizhevsky et al., 2012; Chatfield et al., 2014;
Szegedy et al., 2015) .

####  4.1.2. Background

This specific paper explores the Deep use of neural networks for outfit
recommendation and specifically for the personalized outfit recommendation.
Now for this they encounter two key problems. The first one was modeling of
the compatibility among multiple fashion items and obviously the second one
was capturing users personal interest.

So for that the former one was solved by first mapping the item images to a
latent semantic space with convolutional neural network and for the second one
they adopt a multi-layer fully-connected network structure. And they also
studied alternative architectures that combine feature learning and
compatibility modeling. Different ways for the other problem. What they do is
that they encode user-specific information in terms of parameters of the
network. Although we know that each user may have his own unique personal
taste and they follow some general rules for making outfits. But besides that
the usual small number of training samples for individual users makes it very
much important to borrow training data from other users that share similar
tastes. So with these observations in mind, what they do is that they adopt a
two-stage strategy for the training of their model network; the first stage
basically learns a general compatibility model from outfits of users. And in
the later stage, what they do is that they fine-tune the general model with
the specific data that they get from the user in fine-tuning. It is an
important technique for training deep neural networks for applications that
have limited number of training samples.

####  4.1.3. Proposed Approach

So in their approach they basically assume that heterogeneous fashion items
can be grouped into n categories. Let’s take an example where the three most
social categories for fashion are usually shoes, tops and bottoms and outfit
is a collection of fashion items which are usually coming from different
categories. So an outfit can consist of a bottom, top and a pair of shoes. So
given some historical data what they did was that for any user outfit pair
they pretty much assigned a rating score as the score kind of reflected the
level of affection the user has for the outfit. So the higher the score then
obviously the more appealing the outfit is for the users and those outfits
that had the highest score were recommended to the users. So basically the
rating system was used and the rating $s$ for a user outfit pair is determined
by how well the items in the outfit go with each other. So if you know a pair
of red shirts and you know, let’s say black slacks or tight jeans and maybe
they go well instead of, you know, something with a yellow skirt and red
shirt. So we basically see the author’s design appropriate deep neural network
structure to model the interactions among these items and they achieve
Personalization by developing a two-stage training strategy and embed the user
specific preferences in the parameter of the network.

So what they basically do is that they explore three different network
architectures and naming them as fashionet A ,B and C and without the loss of
generality. They assume an outfit consists of three items: top, bottom and
pair of shoes. So in fashionNet A the images of the items are first
concatenated to create a new image with nine color channels, and the
compounded images are then forwarded to a widely used CNN model VGGNet. The
output layer is a fully connected layer with softmax function as its
activation function. So in this architecture the components of representation
learning and compatibility measure are fully integrated. The two steps are
carried out simultaneously right from the first convolution layer.

Figure 11.  Network architectures

Now in fashionNet B we see that they apply representation learning and
compatibility measures sequentially and the images are first of all mapped to
a feature representation through a feature Network. So the same CNN model is
used for items from different categories. To model the compatibility they
concatenate the features of all items and feed them to three fully connected
layers. So in this work what they show that this network structure also has
the capacity for approximating the underlying compatibility among multiple
features.

Now for fashionNet C , what they do is that both FashionNet A and B try to
directly model the compatibility among multiple items. They sort of come
across difficulties when trying to capture the High order relationships and
the data is significantly expanded when we concatenate all the items. Due to
the dimensionality issue a huge number of training samples may be required for
a good model to be learned and we know that users on the internet have
contributed so many outfit ideas. It is still minor compared to the number of
all possible outfits. So in order to overcome this problem what the authors
propose is that a prior restraint in fashionNet C. They assume that the
compatibility of a set of items is mainly determined by how well a pair of
these items go with each other. Then all the outfits from the final layers
regarding the probabilities that the item pairs are matched while are added
together to get a final score as for the whole outfit. The learning task is
formulated as a learn to rank problem.A training sample contains two outfits,
e.g. $\left\\{I_{t}^{+},I_{b}^{+},I_{s}^{+}\right\\}$ and
$\left\\{I_{t}^{-},I_{b}^{-},I_{s}^{-}\right\\},$ where the former is
preferable to the latter. A two-tower structure to train the networks and rank
loss is used to minimize this following equation.

$L=\frac{1}{M}\sum_{i=1}^{M}\log\left(1+\exp\left(-\left(s_{i}^{+}-s_{i}^{-}\right)\right)\right)$

In the training expect what happens is that for an individual user they
usually have a small number of training outfits. And furthermore, although
each user may have their own preference. There are some rules that should be
followed by most people for making an outfit. For example t-shirts and jeans
are usually paired up. With these observations. What they do is that they
design a two stage procedure to train the deep network for personalized outfit
recommendation. So the first stage is basically that they learn a general
model for compatibility. Here they discard the information of the user and mix
the outfit created by different users all together. And then they create a new
neutral outfit by mixing randomly selected fashion items. Now, this is
reasonable in order to assume that items in a user created outfit are more
compatible than those in neutral outfit. So for that ,training samples can be
made by pairing a user-generated outfit with a neutral one. So they initialize
the parameters in VGGNet that would be trained on imagenet and initialize the
other layers with random numbers drawn from gaussian distribution.

Then furthermore these are optimized for the whole network using the mixed
data set and in the second stage we see that the authors train using the
specific model for personalized recommendations so we can say that for each
user what they did was they first initialize the network with the certain
parameters that were obtained by the previous general training and then they
use each user’s own personal data to fine grain or fine tune the parameters.
We know that fine-tuning is very important in this aspect. It sort of helps
the data insufficiency problem in a lot of different applications. So for
fashionNet A they saw that they fine-tune the whole network in this stage and
for fashionNet B and C. There were two strategies used. The first one was to
fine-tune the whole network. So both the feature Network and the matching
network will have personalized parameters. Now this one resulted in different
feature representations of each item for different users. The second method
was to freeze the feature Network and only fine-tune the matching Network. So
the features will keep the same and the user-specific information will be
carried only by the matching Network and this will save a lot of computation
during testing and which is quite a favorable aspect in terms of practice.

####  4.1.4. Performance evaluation

In the end they found that the performance of FashionNet A was inferior to the
other two architectures namely FashionNet B and C. When all the possible
reasons for fashionNet B and C to obtain such an advantage was that the
representation learning incompatibility modeling was performed in them
separately so that they were able to use different network structures in order
to achieve different functionalities. So these kinds of networks are easier to
design and optimize in this case.

###  4.2. Generative Adversarial Training

For personalization another approach is the generative adversarial training.
So for that we go over another paper (Yu et al., 2019) in which they propose
an approach in which a convolutional network is first used to map the query
image into a latent Vector presentation. Now this latent representation all
together with another Vector which characterizes users style preference as an
input are taken into the generator Network in order to generate the target
image item.

####  4.2.1. Previous Methods

Although there are few works (Hu et al., 2015; Xu Chen, 2018) that have shown
the personalized model is more capable of picking outfits that suit or a model
to generate new items images for some category for a user that was
personalized. But no query item was provided in their settings. They did not
consider the compatibility between items.

Figure 12.  Network architecture for personalized fashion design. It contains
one generator and two discriminators. The generator uses an encoder-decoder
architecture. One of the discriminators is for real/fake supervision. And the
other one is for compatibility prediction

####  4.2.2. Proposed Method

Now, discriminator networks are built to guide the generation process. One of
them is the classic real fake discriminator. And the other is a matching
Network which simultaneously models the compatibility between fashion items
and also learns the preference representations.When the given inventory is
limited. It’s a possibility there. There are no good items enough to
complement the query and when we have the inventory that is too large then
generating the recommendation may face some efficiency problems. So this paper
basically suggests that existing items can be synthesized images of new items
that are compatible to a given one. So basically this solves the deficit
problem for small inventories and for large inventory when targeting real
items is necessary. We can adjust search items that are similar with the
synthesized ones. Which is pretty much more efficient in terms than the
exhaustive compatibility valuations, since similarity search can be very fast
with techniques like hashing.

Now aside from General compatibility they are also considering the personal
issue. Personalization comes in here, which is an important trend as we have
already discussed. Now given the same query item different persons would like
to choose different items which goes with their own personal style. So while
personalized recommendations have been prevalent in areas, like movies, songs
and book recommendations, but for fashion, they are still not user-specific.
So basically what this paper suggests is that the proposed system is
personalized using the generative adversarial training framework GAN’s.
Generative adversity networks have pretty much achieved a great success in
synthesizing realistic images for different applications. So they apply this
technique and they first use an encoder Network to map the query image into a
latent Vector representation. And then this representation together with
another vector that characterizes user style preference is taken into the
input as for the generator Network that generates the target item. So
basically the approach goes like this: the task of personalized fashion design
is basically to develop a fashion item for a specific individual given an
input query item. So there are two general requirements for this design that
they have: the first one is the realness requirement which practically means
that the design item should look realistic. And then the second thing comes is
the compatibility requirement that is basically that the design item should be
compatible with the query item.

###  4.3. Personalization in Unstructured Data

Now we know that a lot of challenges in e-commerce usually come up from the
fact that new products are continuously being added to the catalog. So the
challenge invoked is properly personalizing the customers experience
forecasting demand and planning the product range.

####  4.3.1. Background

The paper (Ângelo Cardoso et al., 2018) in discussion is about a global
e-commerce company that creates and curates clothing and beauty products for
fashion lovers. So over the years they have a lot of products and this amounts
to more than 1 million unique Styles. So for each product different divisions
within the company produce and consume different product attributes, so mostly
the attributes are manually curated and there could be cases in which
information is sometimes missing or wrongly labeled. However, sometimes
incomplete information still carries a lot of potential value for the
business; the ability to have a systematic and quantitative characterization
of a product is basically one of the key aspects for the company to make data-
driven decisions that can be used across a set of problems including
personalization. So the paper basically shows how to predict a consistent and
complete set of product attributes that will illustrate how this enables them
to personalize the customer experience by providing more relevant products.

Figure 13.  Schematic view of the multi-task attribute prediction network

####  4.3.2. Proposed Approach

So basically the model that they proposed attracts attribute values from
product images and textual descriptions. In terms of image processing what
they do is that fashion is predominantly a visual business and visual features
are at the core of many data science products. They use image features for
many of their applications. So in order to minimize the computational cost
what they did was they implemented a centralized visual feature generation
pipeline. That uses a pre-trained convolutional neural network to extract
product representation from images. Now for the text processing what they did
was that the CNN’s were originally applied to images which are treated as
matrices of pixel color values. And it’s a possibility to apply these
convolutions to other types of matrices as well and in particular paragraphs
of text. So similarly, they process images to produce product representations
they also used the same technique for text descriptions. In multi modal
Fusion, they say that the image and the text representations simply are
concatenated together within a neural network, which is trained to predict the
product attributes. This is pretty much straight forward because it’s a common
way to fuse the different inputs. That works well in practice.

Now the primary focus of the paper design was to find a solution that deals
with missing labels at scale. Because in the paper, they also argue that the
foundational piece to solve all of the problems is having consistent and
detailed information about each product, which is rarely available. So they
show this by having a quantitative understanding of the products. Can be used
to improve recommendations in a Hybrid recommender system approach.They say
that they could have chosen to build a separate model for each attribute, but
then they would have to maintain multiple models in production. And in terms
of independent models would also be oblivious to the correlations between
attribute values and they would also only work well for common attributes,
where there must be enough training data. Alternatively they said that they
could have built a single model to predict all attributes at once also, but
however few products are fully annotated and there would have not been enough
data to train such a model. So because of these reasons what they did was they
chose to cast attribute prediction as a multitask learning problem. This means
training a neural network for each attribute but sharing most of the
parameters between Networks.

####  4.3.3. Hybrid Approach

The hybrid approach incorporates several state-of-the-art advances in
recommender systems and not only incorporates new products, but also enhances
the recommendations that customers receive overall. Their approach creates an
embedding of products, i.e. a representation of all the products in their
catalogue in a high-dimensional vector space. In this vector space, products
with similar styles and attributes will be closer than unrelated ones. When
producing personalised recommendations, the algorithm also assigns a vector to
every customer. The items with the highest inner product with the customer
vector are the recommended ones. The position of products and customers in
this space is determined not only by the customer-product interactions, but
also by the augmented product attributes. This ensures that newly added
products are positioned correctly in the space and can be recommended to the
right customers.

###  4.4. POG: Personalized Outfit Generation

Another paper (Chen et al., 2019) proposes a personalized outfit generation
POG model. Basically what happens in this model is that they connect the user
preferences regarding individual items and then the outfits with transformer
architecture. So the extensive offline and online experiments they did
provided them with strong quantitative evidence that the method they proposed
found alternative methods regarding port compatibility and personalization
metrics. So basically what happens is that they can generate compatible and
personalized outfits based on user recent behavior. So specifically for this
they use a Transformer encoder decoder architecture that models both signals
from user preference and outfit compatibility. And this is interestingly one
of the first study to generate personalized outfits based on user historical
Behavior within encoder decoder framework. They also developed a platform
named IDA where POG. has been deployed in order to help out without
regeneration and recommendation at a very large scale application Ifashion.

####  4.4.1. Previous Methods

There are several methods for generating a fashion outfit that is likeable by
the user and usually these methods fall into basically two types. So the first
type is basically the one in which they focus on calculating a pairwise
compatibility metric (McAuley et al., 2015; Song et al., 2018; Veit et al.,
2015) . And the second type is in which they present modeling and outfit as a
set or an ordered sequence. And then there are models (Li et al., 2016) in
which they classify a given outfit as popular or unpopular or train a bi-
directional LSTM model (Han et al., 2017) sequentially generate outfits. Now
we can see that all these methods generally use a simple pooling of item
vectors in order to represent an outfit and they have to rely on the order of
the outfits item. So this is noted that these methods belonging to either
category hardly considers all the interactions between the items in an outfit.
And it is quite unreasonable to consider an outfit as an ordered sequence
because you know shuffling of items in the outfit itself should make no
difference on its compatibility.

####  4.4.2. Proposed Approach

So what they are trying to say is that they want to explicitly incorporate
this into their modeling architecture by which they require that each item
should have a different interaction weight with respect to other item in one
outfit and they have given example like a red shirt should have a higher
interaction with you know, blue jeans or black jeans, but a smaller weight
with a pair of white gloves.

So the model they propose in this is basically what they do, is that they
build a three-step process in which the first step has the items that are to
be embedded and in the second they build FOM which learns compatibilities of
items within an outfit and lastly the third stage once their training is
completed. They use the result to pretrained FOM to initialize POG Transformer
architecture. Representing these items using a multi model embedding model. So
for every fashion item f they compute a non linear feature embedding f . The
concept of fashion basically relies on Visual and textual information So
basically in previous models (Li et al., 2016; Han et al., 2017) what they did
was the authors used the image and text to learn the multimodal embeddings.
But in this scenario, what they do is that they use a multi-modal embedding
model that takes the following input for every item

  1. (1)

Dense vector encoding the white background picture of the item from a CNN
model,

  2. (2)

Dense vector encoding the title of the item obtained from a TextCNN network,
which has been pre-trained to predict an item’s leaf category based on its
title

  3. (3)

D ense vector encoding a collaborative filtering signal for the item using
Alibaba’s proprietary Behemoth Graph embedding platform. So this platform is
used for generating item embeddings based on the co-occurrence statistics of
items in recorded user click sessions in the taobao application.

Figure 14.  The architecture of POG, which is an encoder-decoder architecture
with a Per network and a Gen network. The outfit item is generated step by
step according to the user preference signal from the Per network and the
compatibility signal from the Gen network.

So the generation model works like this, it generates personalized and
compatible outfit by introducing user preference signals. Taking the advantage
of encoder-decoder structure, it translates an user’s historical behaviors to
a personalized outfit. Let $\mathcal{U}$ denote the set of all users and
$\mathcal{F}$ be the set of all outfits. They have used a sequence of user
behaviors $U=\left\\{u_{1},\ldots,u_{i},\ldots,u_{m}\right\\}$ to characterize
an user, where $u_{i}$ are the clicked items by the user.
$F=\left\\{f_{1},\ldots,f_{t},\ldots,f_{n}\right\\}$ is the clicked outfit
from the same user, where $f_{t}$ are the items in the outfit. At each time
step, it predicts the next outfit item given previous outfit items and user’s
click sequence on items $U.$ Thus for pair $(U,F)$ the objective function of
$\mathrm{POG}$ can be written as:

$\mathcal{L}_{(U,F)}=-\frac{1}{n}\sum_{t=1}^{n}\log\operatorname{Pr}\left(f_{t+1}\mid
f_{1},\ldots,f_{t},U;\Theta_{(U,F)}\right)$

where $\Theta_{(U,F)}$ denotes the model parameters.
$\operatorname{Pr}(\cdot)$ is the probability of seeing $f_{t+1}$ conditioned
on both previous outfit items and user clicked items.

In POG the encoder basically what it does is that it takes the user clicked
input items and then it gives a special token like [start]. And then the
decoder generates an outfit one item at a time. So at each step what happens
is that the model is basically autoregressively consuming the previously
generated items as input.The generation basically stops when a special token
[end] appears. So basically what happens is that there in the end an outfit is
given that is generated by composing the output items. So in the figure, you
can also see that the encoder is termed as PER Network and then the decoder is
as Gen Network. So the PER’s natural is basically that it provides a user
preference in terms of signal and then the Gen Network what it does is that it
generates outfits based on both personalization signal and compatibility
signal. So basically the general network is initialized using the
aforementioned pre trained FOM.

###  4.5. Item-to-Set Metric Learning Approach

Social media has been a great source for fashion recommendation and fashion
promotion. It provides us with an open and new data source for personalized
fashion analysis.

####  4.5.1. Background

So this paper (Zheng et al., 2020) basically studies the problem of
personalized fashion recommendation by gathering the data from different
social media. That is they recommend new outfits to social media users that
fit their fashion preferences. They present an item to set metric learning
framework that basically learns to compute similarity that exists between a
set of historical fashion items of a user to a new fashion item. For
extracting features from a multi model street view fashion item the author
basically proposes an embedding module that performs multi-modality feature
extraction and cross Modality gated fusion. By studying the problem of
personalized fashion recommendation with social media data that they are
seeking to recommend new fashion outfits based on the activities that are
being carried by the social network users.

####  4.5.2. Previous Methods

A lot of different studies (Kiapour et al., 2015; Hu et al., 2015; Huang et
al., 2015; Iwata et al., 2011; Jagadeesh et al., 2014) are done for clothing
retrieval and recommendation. But leveraging the user’s interaction on social
media for data for fashion recommendation is very much still challenging and
is quite less explored. And usually what we can gather from social media is
online activities like a street view selfie with additional word description.
So this gives that the granularity of such data is much coarser than you know,
that is unexplored. And most models (Li et al., 2016; Tangseng et al., 2018)
are not directly applicable to the task due to their lack of supervision.

####  4.5.3. Proposed Approach

So paper basically proposes a self supervisor approach for effective and
personalized fashion recommendation in which they divide into two categories
the pictures in which the selfie posts of users a set that reveals their
personal fashion preferences or outfit items that are to be recommended items.
So they proposed that to learn an item to set metric that measures
similarities between a set and items for personalized recommendation. They
minimize the item to set distance for the set and items of a user and while
making sure they maximize such distances for certain items of different users.
And benefiting from this framework they are able to perform personalized
recommendations without requiring any sort of additional supervision. Now we
know that metric learning is well studied in literature and learning such an
item to set metric is previously unexplored. And therefore pose new challenges
because we know that the user can have interest in more than one fashion style
and not the one that is being depicted in their picture. So the item to set
similarity cannot be captured by an over simplified average of multiple items
by similarities. Which therefore states that the nearest neighbor item to set
metric is difficult to learn as it is susceptible to noise and outliers.

So in highlight what their contribution is that they present a fashion
recommendation system built on personal social media data and their system
recommends personalized outfits for using few constraint street view selfie
post of the users. They also proposed a self supervise scheme in which they
enable the training of the system. The approach is based on a novel item to
set a metric learning framework that basically needs only the user selfie pose
as the supervision. For this they design a multi model embedding module that
better fuses the social media data for obstruction of fashion features.

Built upon the item-wise measurement $d\left(f_{i},f_{j}\right),$ they propose
an item-to-set similarity metric $D(S,f),$ which measures how dissimilar an
item $f$ is to a set of items $S=\left\\{f_{1},\cdots,f_{K}\right\\}.$ The
itemto-set metric aims to predict how similar a outfit candidate is to a set
of user selfies for personalized fashion recommendation.

To design a metric that better captures the multiple interests of a user while
facilitating robust training, the paper proposes a generalized item-to-set
distance. Specifically, given a set $S$ and a query $f$, they first assign an
importance weight $w_{i}$ to each item $f_{i}\in S$ before feature averaging
and distance computation. The importance weight is computed using an
importance estimator $w_{i}=K\left(f_{i};f,S\right)$ Such a item-to-set
distance is defined by:

$\displaystyle D(S,\boldsymbol{f})$
$\displaystyle=d\left(\sum_{i=1}^{K}\alpha_{i}f_{i},\boldsymbol{f}\right)$
$\displaystyle\alpha_{i}$
$\displaystyle=\frac{\exp\left(w_{i}\right)}{\sum_{j}\exp\left(w_{j}\right)}$

To reduce the influences of noise and outliers when computing the distance,
basically what they did was that they further considered an intra-set
importance weight:

$v\left(f_{i};S\right)=\operatorname{MLP}_{v}\left(\left[f_{i},\operatorname{stat}(S)\right]\right)$

where $\mathrm{MLP}_{v}$ outputs a scalar from an input vector, and
$\operatorname{stat}(S)$ is a vector that captures the statistics of the set
$S$ along all feature dimensionalities ${}^{2}.$ In this way, we compare each
item $f_{i}$ with the set $S$ to eliminate the outliers from the sets.

Now as we know that there are different individuals that focus on different
particular aspects of fashion items and the item to set metric itself should
be user specific. So for that issue what they did was that for the minimalist
fashion style users the items that distance was made more sensitive to the
amount of colors that are used but for users of the artsy style the item to
set distance should focus more on unusual prints and the complexity of
accessories. So they extended the similarity metric equation to a user
specific metric in which they performed a user specific space transformation
before the distance computation. In particular, given the set $S$, we compute
a scaling vector $t(S)$ which indicates the scaling factor at each feature
dimension:

$\boldsymbol{t}(S)=\operatorname{softmax}\left(\operatorname{MLP}_{t}(\operatorname{stat}(S))\right)$

Using the space transformation, they extended the item-to-set metric to a set-
specific metric. Specifically, they defined a user-specific item-to-set
metric:

$D_{us}(S,f)=d\left(t(S)\odot\left(\sum_{i=1}^{K}\alpha_{i}f_{i}\right),t(S)\odot
f\right)$

where $\odot$ represents vector elementwise multiplication. It filters out the
feature dimensions that a user focuses less on before the distance
computation. This procedure helps the recommendation system to be more user-
specific.

##  5\. Future Research

In the post coronavirus era one of the industries that is obviously
undoubtedly incorporating advanced technologies at much faster speed than ever
before is fashion. And thanks to AI and computer vision power tools, new and
engaging experiences are being born for both retailers and consumers. The
e-commerce customer experience is completely incorporated with AI Solutions
like online site navigation, search, retrieval ,target marketing, labeling,
personalized offers ,size fitting, recommendations and online fitting rooms
and also style recommendation analytics and much more. So by using computer
vision and AI the image pixels are automatically taken and then they generate
semantic data from them, which is very crucial for the e-commerce stores.

One of the things that is the basic thing is the discovery of the products
that the visual search should be easy enough for the Shoppers to find what
they are looking for and should also be benefiting the retailers as well so
that they can take the advantage of users behavior and then show them the
recommendations and can get more profit from this aspect as the stores are
getting more online this post covid era. So the AI technology enables fashion
brands to sort of gain insight as to which product features their customers
would like to prefer.

Now an interesting aspect (Countants, 2020) is that we can see that the
fashion industry is at over 3 trillion dollars that contributes to the healthy
portion of the global GDP and in the 21st century, we can see that AI or
machine learning or specifically deep learning in the fashion industry is
changing every expectation of this forward-looking business. So the use of AI
let alone in the fashion industry of 2020 has so entrenched that 44 percent of
the fashion retailers that are not using AI today are facing bankruptcy. So
you can take this as an example and as a result of this Global spending on AI
Technologies by fashion and retail industry is expected to reach 7.3 billion
each year by the year 2022 and that’s just in two years.

AI powered fashion designing can be based to get the preferred customer color
textures and other style preferences and then they can be used ahead in order
to design the apparel, the textile itself. Regarding The factoring process,
what they can do is that they can use AI tools to identify the Super fast
changing trends and supply the latest fashion accessories to the retail
shelves, which is pretty much faster than the traditional retailing system.
And a lot of leading fashion brands like Zara, Topshop and Achieve and are
already using this and they are pretty much quicker in providing instant
gratification to retail customers by recognizing seasonal demand and
Manufacturing the right supply of the latest clothing and obviously virtual
merchandising is something that has enabled technologies like augmented
reality and virtual reality and now are closing the gap that is between online
and in-store shopping. So this is also really popular regarding this system.
And this is something that can be worked in the recommendation systems. As a
lot of people would like to experience the virtual reality and augmented
reality aspect in terms of the clothes fitting and checking out the online
buying experience and making it more human-like.

##  6\. Conclusion

As the advancements in deep learning, CV and AI are getting stronger day by
day their usage in the fashion industry has also become a very popular topic.
From product personalization or better designing there are multiple ways in
which AI and machine learning Technologies are impacting the global fashion
industry and they are increasing the investment by Leading fashion brands in
these Technologies are a proof of their immense potential. They provide
enhanced customer service, Virtual merchandising, smart manufacturing process
and improved inventory management and need less Manpower through Automation
and provide reduction in returned products which also improves customer
satisfaction. And one of the biggest things is personalization, which is
pretty much the key of business success and thanks to deep learning
Technologies like AI and ML along with business analytics is enabling fashion
business to keep track of fashion trends and purchasing behavior of individual
customers. So now it may be a trend or it may be a season prediction. You can
do anything with these powerful tools and the fashion industry is magnified.
And this is a field that has the potential to grow and ever expand, so any
future research in this line that will be done would be something that paves
way ahead for more jaw dropping phenomenon.

## References

  * (1)
  * Adomavicius and Tuzhilin (2005) Gediminas Adomavicius and Alexander Tuzhilin. 2005.  Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions.  _Knowledge and Data Engineering, IEEE Transactions on_ 17 (07 2005), 734–749.  https://doi.org/10.1109/TKDE.2005.99
  * Borràs et al. (2003) Agnés Borràs, Francesc Tous, Josep Lladós, and María Vanrell. 2003.  High-Level Clothes Description Based on Colour-Texture and Structural Features. 108–116.  https://doi.org/10.1007/978-3-540-44871-6_13
  * Chatfield et al. (2014) Ken Chatfield, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014.  Return of the Devil in the Details: Delving Deep into Convolutional Nets.  _BMVC 2014 - Proceedings of the British Machine Vision Conference 2014_ (05 2014).  https://doi.org/10.5244/C.28.6
  * Che et al. (2016) Tong Che, Yanran Li, Athul Jacob, Y. Bengio, and Wenjie Li. 2016.  Mode Regularized Generative Adversarial Networks.  (12 2016).
  * Chen et al. (2019) Wen Chen, Binqiang Zhao, Pipei Huang, Jiaming Xu, Xin Guo, Cheng Guo, Fei Sun, Chao Li, Andreas Pfadler, and Huan Zhao. 2019.  POG: Personalized Outfit Generation for Fashion Recommendation at Alibaba iFashion. 2662–2670.  https://doi.org/10.1145/3292500.3330652
  * Countants (2020) Countants. 2020\.  _AI and Machine Learning For Fashion Industry — Global Trends and Benefits_.  https://medium.com/datadriveninvestor/ai-and-machine-learning-for-fashion-industry-global-trends-benefits-3fe11a17849e
  * Deng et al. (2009) Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei Fei Li. 2009\.  ImageNet: a Large-Scale Hierarchical Image Database.  _IEEE Conference on Computer Vision and Pattern Recognition_ , 248–255.  https://doi.org/10.1109/CVPR.2009.5206848
  * Dhar et al. (2011) Sagnik Dhar, Vicente Ordonez, and Tamara Berg. 2011\.  High level describable attributes for predicting aesthetics and interestingness.  _Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition_ , 1657–1664.  https://doi.org/10.1109/CVPR.2011.5995467
  * ELEKS ([n.d.]a) ELEKS. [n.d.]a.  _Designing Apparel with Neural Style Transfer_.  https://labs.eleks.com/2016/09/designing-apparel-neural-style-transfer.html
  * ELEKS ([n.d.]b) ELEKS. [n.d.]b.  _Fashion and Technology: How Deep Learning Can Create an Added Value in Retail_.  http://labs.eleks.com/2017/05/fashion-technology-deep-learning-can-create-added-value-retail.html
  * Goodfellow et al. (2014) Ian J. Goodfellow, Jean Pouget-Abadie, M. Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. 2014\.  Generative Adversarial Nets. In _NIPS_.
  * Gygli et al. (2013) Michael Gygli, Helmut Grabner, Hayko Riemenschneider, Fabian Nater, and Luc Van Gool. 2013\.  The Interestingness of Images.  _Proceedings of the IEEE International Conference on Computer Vision_ , 1633–1640.  https://doi.org/10.1109/ICCV.2013.205
  * Han et al. (2017) Xintong Han, Zuxuan Wu, Yu-Gang Jiang, and Larry Davis. 2017\.  Learning Fashion Compatibility with Bidirectional LSTMs.  (07 2017).  https://doi.org/10.1145/3123266.3123394
  * He and Hu (2018) Tong He and Yang Hu. 2018\.  FashionNet: Personalized Outfit Recommendation with Deep Neural Network.
  * Hou et al. (2019) Min Hou, Le Wu, Enhong Chen, Zhi Li, Vincent Zheng, and Qi Liu. 2019\.  Explainable Fashion Recommendation: A Semantic Attribute Region Guided Approach. 4681–4688.  https://doi.org/10.24963/ijcai.2019/650
  * Hsiao and Grauman (2017) Wei-Lin Hsiao and Kristen Grauman. 2017.  Creating Capsule Wardrobes from Fashion Images.  (12 2017).
  * Hsiao et al. (2019) Wei-Lin Hsiao, Isay Katsman, Chao-Yuan Wu, Devi Parikh, and Kristen Grauman. 2019.  Fashion++: Minimal Edits for Outfit Improvement.  arXiv:1904.09261 [cs.CV]
  * Hu et al. (2015) Yang Hu, Xi Yi, and Larry Davis. 2015.  Collaborative Fashion Recommendation: A Functional Tensor Factorization Approach. 129–138.  https://doi.org/10.1145/2733373.2806239
  * Huang et al. (2015) Junshi Huang, Rogerio Feris, Qiang Chen, and Shuicheng Yan. 2015\.  Cross-Domain Image Retrieval with a Dual Attribute-Aware Ranking Network.  (05 2015).  https://doi.org/10.1109/ICCV.2015.127
  * Huynh et al. (2018) Cong Phuoc Huynh, Arridhana Ciptadi, Ambrish Tyagi, and Amit Agrawal. 2018.  CRAFT: Complementary Recommendations Using Adversarial Feature Transformer.  arXiv:1804.10871 [cs.CV]
  * Insight ([n.d.]) First Insight. [n.d.].  _AI and Machine Learning for Fashion_.  https://www.firstinsight.com/knowledge-base/machine-learning-ai-for-retail-fashion
  * Ioffe and Szegedy (2015) Sergey Ioffe and Christian Szegedy. 2015.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift.  (02 2015).
  * Isola et al. (2013) Phillip Isola, Jianxiong Xiao, Devi Parikh, Antonio Torralba, and Aude Oliva. 2013.  What Makes a Photograph Memorable?  _IEEE transactions on pattern analysis and machine intelligence_ 36 (10 2013).  https://doi.org/10.1109/TPAMI.2013.200
  * Iwata et al. (2011) Tomoharu Iwata, Shinji Wanatabe, and Hiroshi Sawada. 2011\.  Fashion Coordinates Recommender System Using Photographs from Fashion Magazines. 2262–2267.  https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-377
  * Jagadeesh et al. (2014) Vignesh Jagadeesh, Robinson Piramuthu, Anurag Bhardwaj, Wei di, and Neel Sundaresan. 2014\.  Large Scale Visual Recommendations From Street Fashion Images.  _Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ (01 2014).  https://doi.org/10.1145/2623330.2623332
  * Jammalamadaka et al. (2013) Nataraj Jammalamadaka, Ayush Minocha, Digvijay Singh, and CV Jawahar. 2013.  Parsing Clothes in Unrestricted Images. 88.1–88.11.  https://doi.org/10.5244/C.27.88
  * Jia et al. (2016) Jia Jia, Jie Huang, G. Shen, T. He, Zhiyuan Liu, H. Luan, and Chao Yan. 2016\.  Learning to Appreciate the Aesthetic Effects of Clothing. In _AAAI_.
  * Kalantidis et al. (2013) Yannis Kalantidis, Lyndon Kennedy, and Li-Jia Li. 2013\.  Getting the Look: Clothing Recognition and Segmentation for Automatic Product Suggestions in Everyday Photos.  https://doi.org/10.1145/2461466.2461485
  * Kang et al. (2017) Wang-Cheng Kang, Chen Fang, Zhaowen Wang, and Julian McAuley. 2017.  Visually-Aware Fashion Recommendation and Design with Generative Image Models.  (11 2017).
  * Khosla et al. (2013) Aditya Khosla, Wilma Bainbridge, Antonio Torralba, and Aude Oliva. 2013.  Modifying the Memorability of Face Photographs.  _Proceedings of the IEEE International Conference on Computer Vision_ , 3200–3207.  https://doi.org/10.1109/ICCV.2013.397
  * Khosla et al. (2014) A. Khosla, A. D. Sarma, and R. Hamid. 2014.  What makes an image popular?. In _WWW_.
  * Kiapour et al. (2015) M. Kiapour, Xufeng Han, Svetlana Lazebnik, Alexander Berg, and Tamara Berg. 2015.  Where to Buy It: Matching Street Clothing Photos in Online Shops. 3343–3351.  https://doi.org/10.1109/ICCV.2015.382
  * Kobayashi ([n.d.]) S Kosdansha International Kobayashi. [n.d.].  _Art of color combinations_.
  * Kolda and Bader (2009) T. Kolda and B. Bader. 2009\.  Tensor Decompositions and Applications.  _SIAM Rev._ 51 (2009), 455–500.
  * Koren and Bell (2015) Yehuda Koren and Robert Bell. 2015.  _Advances in Collaborative Filtering_.  77–118.  https://doi.org/10.1007/978-1-4899-7637-6_3
  * Krizhevsky et al. (2012) Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. 2012\.  ImageNet Classification with Deep Convolutional Neural Networks.  _Neural Information Processing Systems_ 25 (01 2012).  https://doi.org/10.1145/3065386
  * Lecun et al. (1998) Yann Lecun, Leon Bottou, Y. Bengio, and Patrick Haffner. 1998\.  Gradient-Based Learning Applied to Document Recognition.  _Proc. IEEE_ 86 (12 1998), 2278 – 2324.  https://doi.org/10.1109/5.726791
  * Lew et al. (2006) Michael Lew, Nicu Sebe, Chaabane Djeraba, and Ramesh Jain. 2006\.  Content-based multimedia information retrieval: State of the art and challenges.  _TOMCCAP_ 2 (02 2006), 1–19.  https://doi.org/10.1145/1126004.1126005
  * Li et al. (2016) Yuncheng Li, LiangLiang Cao, Jiang Zhu, and Jiebo Luo. 2016\.  Mining Fashion Outfit Composition Using An End-to-End Deep Learning Approach on Set Data.  _IEEE Transactions on Multimedia_ PP (08 2016).  https://doi.org/10.1109/TMM.2017.2690144
  * Lin et al. (2014) Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Zitnick. 2014\.  Microsoft COCO: Common Objects in Context.  (05 2014).
  * Liu et al. (2012) Si Liu, Tam Nguyen, Jiashi Feng, Meng Wang, and Shuicheng Yan. 2012.  Hi, magic closet, tell me what to wear! 1333–1334.  https://doi.org/10.1145/2393347.2396470
  * Maas (2013) Andrew L. Maas. 2013\.  Rectifier Nonlinearities Improve Neural Network Acoustic Models.
  * McAuley et al. (2015) Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Hengel. 2015.  Image-Based Recommendations on Styles and Substitutes.  (06 2015).  https://doi.org/10.1145/2766462.2767755
  * Melo et al. (2015) Ernani Melo, Emilia Nogueira, and Denise Guliato. 2015\.  Content-Based Filtering Enhanced by Human Visual Attention Applied to Clothing Recommendation. 644–651.  https://doi.org/10.1109/ICTAI.2015.98
  * Melville et al. (2002) Prem Melville, Raymond Mooney, and Ramadass Nagarajan. 2002\.  Content-Boosted Collaborative Filtering for Improved Recommendations.  _Proceedings of the National Conference on Artificial Intelligence_ (05 2002).
  * Nguyen et al. (2014) Hai Nguyen, Martin Havig, Herman Schistad, Thomas Almenningen, Anders Kofod-Petersen, Helge Langseth, and Heri Ramampiaro. 2014.  Learning to Rank for Personalised Fashion Recommender Systems via Implicit Feedback.  https://doi.org/10.1007/978-3-319-13817-6_6
  * of Philosophy (2009) Stanford Encyclopedia of Philosophy. 2009\.  _The Concept of the Aesthetic_.  https://plato.stanford.edu/entries/aesthetic-concept/
  * Pedersen et al. (2004) Ted Pedersen, Siddharth Patwardhan, and Jason Michelizzi. 2004\.  WordNet::Similarity - Measuring the Relatedness of Concepts.  (04 2004).
  * Rendle et al. (2012) Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2012.  BPR: Bayesian Personalized Ranking from Implicit Feedback.  _Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, UAI 2009_ (05 2012).
  * Rendle and Schmidt-Thieme (2010) Steffen Rendle and Lars Schmidt-Thieme. 2010.  Pairwise Interaction Tensor Factorization for Personalized Tag Recommendation.  _WSDM 2010 - Proceedings of the 3rd ACM International Conference on Web Search and Data Mining_ , 81–90.  https://doi.org/10.1145/1718487.1718498
  * Sidiropoulos et al. (2016) N.D. Sidiropoulos, Lieven Lathauwer, Xiao Fu, Kejun Huang, Evangelos Papalexakis, and Christos Faloutsos. 2016.  Tensor Decomposition for Signal Processing and Machine Learning.  _IEEE Transactions on Signal Processing_ PP (07 2016).  https://doi.org/10.1109/TSP.2017.2690524
  * Simo-Serra et al. (2014) Edgar Simo-Serra, Sanja Fidler, Francesc Moreno-Noguer, and Raquel Urtasun. 2014.  A High Performance CRF Model for Clothes Parsing. 64–81.  https://doi.org/10.1007/978-3-319-16811-1_5
  * Simo-Serra et al. (2015) Edgar Simo-Serra, Sanja Fidler, Francesc Moreno-Noguer, and Raquel Urtasun. 2015.  Neuroaesthetics in fashion: Modeling the perception of fashionability. 869–877.  https://doi.org/10.1109/CVPR.2015.7298688
  * Song et al. (2018) X. Song, Fuli Feng, Xianjing Han, X. Yang, W. Liu, and L. Nie. 2018\.  Neural Compatibility Modeling with Attentive Knowledge Distillation.  _The 41st International ACM SIGIR Conference on Research &. Development in Information Retrieval_ (2018).
  * Song et al. (2017) Xuemeng Song, Fuli Feng, Jinhuan Liu, Zekun Li, Liqiang Nie, and Jun Ma. 2017\.  NeuroStylist: Neural Compatibility Modeling for Clothing Matching. 753–761.  https://doi.org/10.1145/3123266.3123314
  * Streamoid ([n.d.]) Streamoid. [n.d.].  _The Aesthetics of Fashion Part 2_.  https://blog.streamoid.com/the-aesthetics-of-fashion-part-2-66deaaf349dc
  * Szegedy et al. (2015) Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015.  Going deeper with convolutions.  _The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 1–9.  https://doi.org/10.1109/CVPR.2015.7298594
  * Tangseng et al. (2018) Pongsate Tangseng, Kota Yamaguchi, and Takayuki Okatani. 2018\.  Recommending Outfits from Personal Closet.
  * Vasileva et al. (2018) Mariya Vasileva, Bryan Plummer, Krishna Dusad, Shreya Rajpal, Ranjitha Kumar, and David Forsyth. 2018\.  Learning Type-Aware Embeddings for Fashion Compatibility.  (03 2018).
  * Veit et al. (2015) Andreas Veit, Balazs Kovacs, Sean Bell, Julian McAuley, Kavita Bala, and Serge Belongie. 2015\.  Learning Visual Clothing Style with Heterogeneous Dyadic Co-Occurrences.  (09 2015).  https://doi.org/10.1109/ICCV.2015.527
  * Vittayakorn et al. (2015) Sirion Vittayakorn, Kota Yamaguchi, Alexander Berg, and Tamara Berg. 2015.  Runway to Realway: Visual Analysis of Fashion.  _Proceedings - 2015 IEEE Winter Conference on Applications of Computer Vision, WACV 2015_ (02 2015), 951–958.  https://doi.org/10.1109/WACV.2015.131
  * Wang et al. (2016) Daixin Wang, Peng Cui, and Wenwu Zhu. 2016.  Structural Deep Network Embedding. 1225–1234.  https://doi.org/10.1145/2939672.2939753
  * Wang (2013) Xiaohui Wang. 2013\.  Interpretable Aesthetic Features for Affective Image Classification.  _Proceedings / ICIP … International Conference on Image Processing_ (09 2013), 3230–3234.  https://doi.org/10.1145/1188913.1188915
  * Wu et al. (2019) Le Wu, Lei Chen, Richang Hong, Yanjie Fu, Xing Xie, and Meng Wang. 2019\.  A Hierarchical Attention Model for Social Contextual Image Recommendation.  _IEEE Transactions on Knowledge and Data Engineering_ PP (04 2019), 1–1.  https://doi.org/10.1109/TKDE.2019.2913394
  * Xu Chen (2018) Hongteng Xu Yixin Cao Zheng Qin Hongyuan Zha Xu Chen, Yongfeng Zhang. 2018.  Visually Explainable Recommendation.  _CoRR_ abs/1801.10288 (2018).  arXiv:1801.10288 http://arxiv.org/abs/1801.10288
  * Yamaguchi et al. (2013) Kota Yamaguchi, M. Kiapour, and Tamara Berg. 2013\.  Paper Doll Parsing: Retrieving Similar Styles to Parse Clothing Items.  _Proceedings of the IEEE International Conference on Computer Vision_ , 3519–3526.  https://doi.org/10.1109/ICCV.2013.437
  * Yamaguchi et al. (2012) Kota Yamaguchi, M.H. Kiapour, L.E. Ortiz, and T.L. Berg. 2012\.  Parsing clothing in fashion photographs.  _Proceedings / CVPR, IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society Conference on Computer Vision and Pattern Recognition_ , 3570–3577.  https://doi.org/10.1109/CVPR.2012.6248101
  * Yan (2012) Shuicheng Yan. 2012\.  Street-to-shop: Cross-scenario clothing retrieval via parts alignment and auxiliary set. 3330–3337.
  * Yang et al. (2015) Wei Yang, Ping Luo, and Liang Lin. 2015.  Clothing Co-Parsing by Joint Image Segmentation and Labeling.  _Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition_ (02 2015).  https://doi.org/10.1109/CVPR.2014.407
  * Yu et al. (2019) Cong Yu, Yang Hu, Yan Chen, and Bing Zeng. 2019\.  Personalized Fashion Design. In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_.
  * Yu et al. (2018) Wenhui Yu, Huidi Zhang, Xiangnan He, Xu Chen, Li Xiong, and Zheng Qin. 2018.  Aesthetic-based Clothing Recommendation.  _CoRR_ abs/1809.05822 (2018).  arXiv:1809.05822 http://arxiv.org/abs/1809.05822
  * Zhang et al. (2017) Yan Zhang, Xiang Liu, Yunyu Shi, Yunqi Guo, Chaoqun Xu, Erwen Zhang, Jiaxun Tang, and Zhijun Fang. 2017\.  Fashion Evaluation Method for Clothing Recommendation Based on Weak Appearance Feature.  _Scientific Programming_ 2017 (10 2017), 1–12.  https://doi.org/10.1155/2017/8093057
  * Zheng et al. (2020) Haitian Zheng, Kefei Wu, Jong Park, Wei Zhu, and Jiebo Luo. 2020.  Personalized Fashion Recommendation from Personal Social Media Data: An Item-to-Set Metric Learning Approach.
  * Zou et al. (2016) Qin Zou, Zheng Zhang, Qian Wang, Qingquan Li, Long Chen, and Song Wang. 2016\.  Who Leads the Clothing Fashion: Style, Color, or Texture? A Computational Study.  (08 2016).