# An Analysis under a Unified Formulation of
Learning Algorithms with Output Constraints

Mooho Song
Seoul National University
<EMAIL_ADDRESS>
&Jay-Yoon Lee
Seoul National University
<EMAIL_ADDRESS>

###### Abstract

Neural networks (NN) perform well in diverse tasks, but sometimes produce
nonsensical results to humans. Most NN models solely learn from (input,
output) pairs, occasionally conflicting with human knowledge. Many studies
indicate injecting human knowledge by reducing output constraints during
training can improve model performance and reduce constraint violations. While
there have been several attempts to compare different existing algorithms
under the same programming framework, nonetheless, there has been no previous
work that categorizes learning algorithms with output constraints in a unified
manner. Our contributions are as follows: (1) We categorize the previous
studies based on three axes: type of constraint loss used (e.g. probabilistic
soft logic, REINFORCE), exploration strategy of constraint-violating examples,
and integration mechanism of learning signals from main task and constraint.
(2) We propose new algorithms to integrate the information of main task and
constraint injection, inspired by continual-learning algorithms. (3)
Furthermore, we propose the $H\beta$-score as a metric for considering the
main task metric and constraint violation simultaneously. To provide a
thorough analysis, we examine all the algorithms on three NLP tasks: natural
language inference (NLI), synthetic transduction examples (STE), and semantic
role labeling (SRL). We explore and reveal the key factors of various
algorithms associated with achieving high $H\beta$-scores.

An Analysis under a Unified Formulation of
Learning Algorithms with Output Constraints

Mooho Song Seoul National University<EMAIL_ADDRESS>Jay-Yoon Lee Seoul
National University<EMAIL_ADDRESS>

##  1 Introduction

The majority of neural networks (NN) models solely learn from data in the form
of (input, output) pairs, and such models can sometimes result in a conflict
with human knowledge. Previous work has shown that injecting human knowledge
into NN models in the form of reducing relevant constraint violations during
training time can improve the model performance as well as reducing constraint
violations Li et al. (2020); Nandwani et al. (2019); Mehta et al. (2018);
Rajaby Faghihi et al. (2023); Xu et al. (2018). The relation between
constraint and the task itself can be viewed as a relation between the sub-
task and the main task. The goal of the main task would be to acquire the most
accurate prediction possible, whereas the goal of the subtask is to simply
acquire constraint-satisfying output. The focus in injecting constraint is to
preserve or improve the main task performance while improving the constraint
satisfaction.

Various literature exists on constraint injection during training time Li et
al. (2020); Nandwani et al. (2019); Mehta et al. (2018), where the majority of
them formulates the loss function as an addition of loss related to constraint
to the existing supervised loss term. Research on how to formulate the loss
related to constraint and how much to incorporate in comparison to existing
supervised loss is scattered as these approaches vary across different studies
and applications.

The first goal of this work is to provide a unified analysis of existing
methods from previous studies under a single mathematical formulation. Early
efforts to compare different constraint injection methods Rajaby Faghihi et
al. (2023) do exist, however, their focus was on comparing performances of
different algorithms, as presented in previous work. On the other hand, our
study aims to formalize previous literature from a new unified perspective, to
understand key success factors in existing algorithms. For example, while
primal-dual algorithm Nandwani et al. (2019) have shown positive results with
the idea of dynamic weight update on constraint-loss, it was only tested under
the single loss type of Probabilistic Soft Logic (PSL) Bröcheler et al.
(2010). This makes it unclear whether the positive results were contributions
mostly coming from PSL or from the novel dynamic weight update algorithm. As
the same weight update mechanism can be applied to different loss types, such
as REINFORCE loss, it is worthwhile to investigate mixing and matching
different components of injecting constraint under a unified formulation.
While numerous studies have focused on injecting constraint during training
time, to the best of our knowledge, there has been no research consolidating
these studies into a unified mathematical formulation to compare their
characteristics component by component.

The second goal of this work is to propose new effective learning algorithms
that integrate constraints within the suggested unified formulation. A common
approach to learning with constraints involves handling a constraint loss
term, $\lambda\times\mathcal{C}$, where $\mathcal{C}$ denotes the constraint
loss and $\lambda$ is a fixed scalar representing the weight of $\mathcal{C}$.
By adding $\lambda\times\mathcal{C}$ to the pre-existing supervised loss term.
Nandwani et al. introduced an algorithm that dynamically controls $\lambda$,
starting training with $\lambda$ at 0 and progressively adjusting its value
during the learning process. This algorithm is characterized by the gradual
increase of the weight $\lambda$, updating it solely based on the degree of
constraints. While the work of Nandwani et al. is distinguished from existing
methods that use a fixed hyperparameter $\lambda$, there has not been
sufficient research for integrating the learning signals form supervised data
and constraint information beyond this work.

Is it always necessary to have the value of $\lambda$ monotonically increasing
for training? Is there a way to update the value of $\lambda$ considering both
supervised learning and constraint injection? Inspired by continual learning
methods, this paper proposes a new approach that considers both supervised
loss and constraint loss during gradient updates. This approach takes into
account the progress of both tasks: supervised learning and constraint
injection tasks. It offers a new viewpoint for injecting constraint on
simultaneously learning these two tasks. Experiments demonstrate that our new
approach achieves the highest-level of performance other learning algorithms
in various scenarios.

##  2 Unified formulation of previous work

In this section, we categorize the previous studies on injecting constraints
during training time based on three dimensions: type of mathematical
expression used for constraint loss (§2.1), exploration strategy of
constraint-violating examples (§2.2), and mechanism for integrating losses
from the main task and the constraint injection task (§2.3). A common approach
in machine learning is to define a loss function and employ optimization
algorithms to update model parameters in the direction of minimizing that
loss. When the labeled data $\\{(x_{i},y_{i})\\}^{N}_{i=1}$ is given, the goal
of typical supervised learning is to solve the following optimization problem:

$\displaystyle\underset{\theta}{\min}\;\cfrac{1}{N}\overset{N}{\underset{i=1}{\sum}}\mathcal{L}(x_{i},y_{i};\theta),\;\;\text{or
simply}\;\;\underset{\theta}{\min}\;\mathcal{L}(\theta)$ (1)

, where $\mathcal{L}(x,y;\theta)$ is the standard supervised loss function for
the task we are learning.

Most of the existing constraint injection methods, while differing in specific
formulations, inject the constraint information by expanding the loss function
in a following manner:

$\displaystyle\mathcal{T}(\theta)=\lambda_{1}\mathcal{L}(\theta)+\lambda_{2}\cdot\mathcal{C}(\theta)$
(2)

, where $\mathcal{C}(\theta)$ is a loss related to the constraint, and
$\lambda_{i}$’s are fixed weights, We can further generalize the equation (2)
as follow:

$\displaystyle\nabla\mathcal{T}(\theta)=\Lambda^{sup}\cdot\nabla\mathcal{L}(\theta)+\Lambda^{con}\cdot\nabla\mathcal{C}(\theta)$
(3)

, where $\Lambda^{sup}$, $\Lambda^{con}$ are usually scalar matrices.
$\mathcal{C}$ reflects the human knowledge the algorithm wants to inject and
is typically computed without the true label. To be more precise, for some
hard constraints on output labels, $\mathcal{C}(\theta)$ is computed via
output $f_{\theta}(x)$ given some unlabeled input $x$. Injecting more than one
hard constraint is also possible by expanding
$\Lambda^{con}\cdot\nabla\mathcal{C}(\theta)$ to
$\overset{K}{\underset{i=1}{\sum}}\Lambda^{con}_{i}\cdot\nabla\mathcal{C}_{i}(\theta)$
in equation (3) , where $K$ is a number of constraints.

To unify and distinguish different algorithms that learn with constraints, we
focus on how $\mathcal{C}(\theta)$ is formulated (§2.1), how constraint-
violating examples are explored (§2.2), and how $\Lambda^{sup}$,
$\Lambda^{con}$ (in equation (3)) are determined(§2.3).

###  2.1 Type of constraint loss

Type of constraint loss is related to how the violation of constraints can be
transformed into the form of a differentiable loss function
$\mathcal{C}(\theta)$ in equation (2), which we will refer to as the
constraint loss. How to convert symbolic constraints into a differentiable
loss function can be broadly categorized into two approaches: Probabilistic
Soft Logic (PSL) and REINFORCE.

#### Probabilistic Soft Logic (PSL)

PSL Bröcheler et al. (2010) is associated with expressing logic in terms of
probabilities, and research utilizing PSL measures the degree of constraint
violation in the logic itself, employing it as a loss. Gödel, product,
Łukasiewicz logics can be primarily used to soften logic Minervini and Riedel
(2018); Nandwani et al. (2019); Li et al. (2020), and these examples are
listed in table 1. Generally, PSL is not suitable for representing all types
of hard constraints, since it must be converted to linear constraints before
they can be directly applied Rajaby Faghihi et al. (2023). Section §4.2 is an
example task illustraing the challenges in applying PSL, and the more details
are in Appendix §A.2.

#### REINFORCE

In contrast, studies employing the REINFORCE Williams (1992) evaluate whether
(or to what degree) the model’s output violates constraints. Constraint
injection research during training time using REINFORCE can be further
classified into two ways depending how the reward is formulated. A simple
method is to assign binary reward (e.g.: {1, 0}) when the model satisfies or
violates the constraints Ahmed et al. (2022). This simple method with binary
reward only considers whether the constraint is satisfied or not. On the other
hand, one could make the reward more fine-grained by measuring the degree of
constraint violation and assigning real-valued rewards related to it Mehta et
al. (2018). A significant feature of REINFORCE is that the determination of
constraint loss relies solely on the rule of assigning rewards based on the
presence or absence of constraint violation in sampled examples, regardless of
the specific constraint. This differs from PSL in that it does not require
intricate implementations for generating constraint loss. However, due to the
need for sampling procedures, the computational cost is generally higher than
when using PSL Rajaby Faghihi et al. (2023).

To summarize, PSL and REINFORCE are mainly used approach to generate
$\mathcal{C}(\theta)$ in eq.(2) to reduce expected constraint violation with
following differences. PSL defines constraint violation as a continuous
measure, while REINFORCE relies on the reinforcement learning paradigm to
guide the model towards satisfying constraints. Specifically, the REINFORCE
method is divided into two types based on the method of setting rewards:
binary rewards and real rewards. More specific comparison between types of
constraint losses: PSL and REINFORCE is in Appendix §A.1.

###  2.2 Exploration of constraint-violating examples

Let $f_{\theta}(x)$ represent the output distribution associated with the
model $f$ parameterized by $\theta$ given input $x$. The identification of
constraint-violating examples from $f(x)$ plays a crucial role in determining
constraint loss $\mathcal{C}(\theta)$. Therefore, exploration of constraint-
violating examples can significantly impact the effectiveness and efficiency
of constraint learning. The possible questions we have are as follows: Would
it be better to explore the model’s approximate output space? Would it be best
to examine the model’s best possible effort? Or would it be better to explore
by considering all possible probability distributions? Theses are considered
to determine the magnitude of the constraint loss $\mathcal{C}$. For example,
in REINFORCE with {1, 0} reward, the reward will be 0 if we only visit
constraint-violating examples.

According to the questions posed above, exploration strategies are divided by
three, each explained below: sampling, argmax, and exhaustive.

#### Sampling

Sampling strategy involves drawing samples from the forward propagation
results of the model $f$ to examine different instances that violate
constraints. As demonstrated by Ahmed et al. (2022), this method commonly
employs the REINFORCE algorithm to incorporate constraint violations into the
loss function for the identified examples. The sampling strategy can be
applied independently to all combinations for our other analysis axes,
specifically concerning the type of constraint loss (§2.1) and the integration
mechanism of learning signals from main task and constraint (§2.3).

#### Argmax (Top-1)

Argmax (Top-1) strategy, constraint violation is assesed by choosing the
combination with the highest probability from $f(x)$. Following greedy
decoding process such as beam search or Vitrerbi decoding Mehta et al. (2018),
it evaluates constraint violation for the decoded example. Similar to
sampling, it evaluates constraint violation for the decoded example, but
distinguishes itself by considering the most probable prediction at that
moment without multiple samplings. Like the sampling strategy, the argmax
strategy can also be applied independently to all combinations under our other
axes of anlysis.

#### Exhaustive

Exhaustive strategy considers probabilities of all output class and its
combinations. It is prominently employed in research related to PSL Nandwani
et al. (2019); Li et al. (2020). Since there is no sampling involved, it is
computationally cost-effective rather than sampling strategy. When considering
the type of constraint loss (§2.1), our performance evaluation is exclusively
conducted using PSL for the exhaustive strategy, excluding the REINFORCE in
the constraint loss, as it would be impossible to consider all possible
combinations in REINFORCE. Since the exhaustive strategy can only be applied
for constraint loss type of PSL, exhaustive strategy cannot handle all of
general type of constraints.

###  2.3 Integration mechanism of learning signals from main task and
constraint

This section is related to the integration of main task and the constraint
information. We categorize integration mechanisms of prior studies into static
and monotone ($\lambda\uparrow$). Additionally, we introduce three new
integration mechanisms based on the linear projection: projection-sup,
projection-con, and projection-both, which will be discussed in section §3. We
provide detailed explanations of these mechanisms below.

#### Static

For constraint loss $\mathcal{C}$, a widely used approach incorporating
$\mathcal{C}$ into the existing supervised loss term is to add
$\lambda\cdot\mathcal{C}$ to the previous existing loss, where $\lambda$ is a
fixed positive real number Ahmed et al. (2022); Li et al. (2020); Mehta et al.
(2018); Minervini and Riedel (2018). In this approach, the value of $\lambda$
remains unchanged throughout the training process, serving as a constant
multiplier that determines the relative influence of $\mathcal{C}$ in
comparison to the main task loss $\mathcal{L}$ in eq.(2).

#### Monotone ($\lambda\uparrow$)

On the other hand, the study by Nandwani et al. (2019) deviates from this by
not using a fixed $\lambda$. Instead, it initiates training with $\lambda$
starting from 0 and progressively adjusting its value during the learning
process. This concept emerged from the transformation of the constrained
optimization problem into a max-min problem, employing alternative updates. In
this method, the value of $\lambda$ steadily grows throughout the training,
signifying a progressive emphasis on the constraint loss.

#### Projection

Unlike previous methods, projection methods perform gradient updates
considering the gradients of two losses: $\mathcal{L}$ and $\mathcal{C}$. For
both static and monotone ($\lambda\uparrow$), $\Lambda$’s are all diagonal
matrices in equation (3). However, the projection method results in non-
diagonal matrices depending on the gradients of both loss functions. The
detailed formulation will be introduced in section §3.

It is important to note that the decision on how to integrate two losses
($\mathcal{L}$, $\mathcal{C}$) is entirely separate from the process of
formulating the constraint loss $\mathcal{C}$ (§2.1), and the exploring
strategy of constraint-violation examples (§2.2). Therefore, adjusting
$\Lambda$’s (or, $\lambda$’s) mentioned in this section can be independently
combined with other analysis axes.

##  3 Further exploration on integration of main task and constraint
information

In this section, we propose new methods for integrating the losses of main
task and constraint injection task. Departing from categorized methods used in
previous research, ‘static’ and ‘monotone($\lambda\uparrow$)’, we introduce
three new integration mechanism for the two losses: ‘projection-sup’,
‘projection-con’, and ‘projection-both’.

#### Motivation

Gradient Episodic Memory (GEM) Lopez-Paz and Ranzato (2017) model is designed
for continual learning for positive backward transfer, aiming to store
memories of previous tasks in such a way that the loss does not increase when
learning from new data. It introduces constraints to prevent an increase in
loss for previous tasks stored in memory when learning from new data and
presents a new minimization problem. A-GEM Chaudhry et al. (2018) is a variant
of GEM that is designed for effective memory and computational cost, by
storing the averaged episodic memory across the all tasks. Motivated by these
works, we propose a new method for integrating losses – $\mathcal{L}(\theta)$
and $\mathcal{C}(\theta)$ – for two tasks. In GEM/A-GEM, whenever new data was
deemed to violate positive backward transfer, it applies a projection
operation for the gradients to adjust them. We adapt the concept from
GEM/A-GEM and utilize it in designing the integration mechanism of main task
and constraint information

#### Method

Recall that the derivative of loss function with constraint has form of:

$\displaystyle\nabla\mathcal{T}(\theta)=\Lambda^{sup}\cdot\nabla\mathcal{L}(\theta)+\Lambda^{con}\cdot\nabla\mathcal{C}(\theta)$

Our approach is rooted in the idea that supervised learning and constraint
injection are two distinct tasks, and during their respective updates, we can
prevent negatively effecting each other by executing a projection of gradient
for each other. The followings are explanations of three new algorithms, and
the pseudo-codes are available in Appendix C.

Projection-sup applies the projection method to the gradient of the constraint
loss (namely, adjust $\Lambda^{con}$) to prevent it from negatively affecting
the supervised learning task, while storing the averaged gradient vector of
supervised learning task $g_{sup}$ previously used for training.
Mathematically, project $\nabla\mathcal{C}(\theta)$ via:

$\text{Proj}(\nabla\mathcal{C}(\theta))=\nabla\mathcal{C}(\theta)-\frac{\nabla\mathcal{C}(\theta)\cdot
g_{sup}}{g_{sup}\cdot g_{sup}}g_{sup}$ (4)

, whenever $\nabla\mathcal{C}(\theta)\cdot g_{sup}<0$. Then, the vector
$\text{Proj}(\nabla\mathcal{C}(\theta))$ satisfies
$\text{Proj}(\nabla\mathcal{C}(\theta))\cdot g_{sup}=0$. This ensures that
$\nabla\mathcal{C}$ is transformed orthogonally to $g_{sup}$, preventing ti
from providing information that contradicts supervised learning.

Conversely, projection-con applies the projection method to the gradient of
the supervised loss (namely, adjust $\Lambda^{con}$) to prevent it from
negatively affecting the constraint injection task, while storing the averaged
gradient vector of constraint injection task $g_{sup}$ previously used for
training. Mathematically, project $\nabla\mathcal{L}(\theta)$ via:

$\text{Proj}(\nabla\mathcal{L}(\theta))=\nabla\mathcal{L}(\theta)-\frac{\nabla\mathcal{L}(\theta)\cdot
g_{con}}{g_{con}\cdot g_{con}}g_{con}$ (5)

, whenever $\nabla\mathcal{L}(\theta)\cdot g_{con}<0$. Then, the vector
$\text{Proj}(\nabla\mathcal{L}(\theta))$ satisfies
$\text{Proj}(\nabla\mathcal{L}(\theta))\cdot g_{con}=0$. This ensures that
$\nabla\mathcal{L}$ is transformed orthogonally to $g_{con}$, preventing ti
from providing information that contradicts constraint injection.

Projection-both combines both projection-sup and projection-con, applying
projection to both gradients (namely, adjust both $\Lambda^{sup}$ and
$\Lambda^{con}$) to ensure that neither task negatively impacts the other. It
stores two types of gradients separately by each task used for training
before, and apply two projections (4) and (5) together.

##  4 Tasks

In this section, we introduce the tasks for which we conduct experiments:
Natural Language Inference (NLI), Synthetic Transduction Example (STE), and
Semantic Role Labeling (SRL). Additional details about tasks and
implementations are explained in Appendix §D.

###  4.1 Natural Language Inference (NLI)

NLI is a task that involves understanding the logical relationships between
pairs of text. Given a premise (P) and a hypothesis (H), the task is to
determine whether P entails H, contradicts H, or maintains a neutral
relationship with H. There exists constraints such as if P entails H, then H
must not contradict P. We used the five constraints listed in Minervini and
Riedel (2018), as shown in table 4 . The dataset used is SNLI Bowman et al.
(2015).

###  4.2 Synthetic Transduction Example (STE)

We also present an artificial task utilized in Lee et al. (2019). A sequence
transducer $T:\mathcal{L}_{S}\rightarrow\mathcal{L}_{T}$ converts the source
language $\mathcal{L}_{S}=(az|bz)^{*}$ to the target language
$\mathcal{L}_{T}=(za|bbb)^{*}$, for example, $T(azbzbz)=zabbbbbb$. The
constraint imposed involves the relationship between the number of ‘b’ in the
source and the target. Specifically, the count of ‘b’ in the target must be
exactly three times that in the source.

###  4.3 Semantic Role Labeling (SRL)

SRL is a natural language processing task that predicts the semantic roles of
each word in a sentence with respect to a given verb or predicate. The method
of our work employed for this purpose is BIO tagging.

The Unique Core Roles constraint from Li et al. (2020) is applied as a
constraint, which means that there can be no more than one occurrence of each
core argument. For a predicate $u$, if the model predicts the $i$-th word as
B-X, then other words in the same prediction should not be predicted as B-X.
This can be expressed as follow.

$\displaystyle\forall\;u,i\in s,\texttt{X}$
$\displaystyle\in\mathcal{A}_{core},$ $\displaystyle B_{\texttt{X}}(u,i)$
$\displaystyle\rightarrow\underset{j\in s,j\neq i}{\bigwedge}\neg
B_{\texttt{X}}(u,j).$ (6)

The dataset we used is English Ontonotes v5, with the CoNLL-2012 shared task
format Pradhan et al. (2012).

##  5 Experiments

Our experiment is composed of NLI, STE, and SRL tasks, with accuracy, token
accuracy, and F1 score are used as the main task metrics, respectively. Our
goal is to first observe the performance trends of algorithms according to our
three classification criteria. Then, we will explore combinations that show
particularly strong performance.

#### Experiment environment

We used RTX 3090 GPU, and Adam optimizer for all of trainings. We conducted
training for each case 10 times, and the results are displayed as the mean (in
the larger font above) and standard deviation (in the smaller font below) for
both the main task metric (denoted by Perf) and constraint violation (denoted
by Const.Vio)111For example, $\underset{\pm 00.77}{84.72}$ means that the
average is 84.72, and the standard deviation is 00.77 from 10 experiments.
rate.

#### Metric

Comparing the superiority of experimental results considering two different
metrics simultaneously is very challenging, especially when there is no
occurrence of Pareto-improvement. The $H\beta$-score (Harmonic $\beta$ Score)
we propose is an indicator that allows for a quick and clear evaluation of
experimental outcomes based on two metrics. Assume we have two metrics to
consider, and denote the scores for each metric as $m_{1}$ and $m_{2}$,
respectively. Both metrics are assumed to have values ranging from 0 to 1,
with higher values indicating better performance222Constraint violation rate
is used for table 3. However, when we consider $H\beta$-score, we convert it
to the constraint satisfaction rate, which is $1-$(constraint violation
rate).. The $H\beta$-score is similar in form to the F$\beta$-score and is
defined as follow:

$H_{\beta}(m_{1},m_{2})=\frac{1+\beta^{2}}{\frac{1}{m_{1}}+\frac{\beta^{2}}{m_{2}}}.$

The $H\beta$-score is exactly the same in form as the F$\beta$-score. It is
simply an extension of the F$\beta$-score, which uses precision and recall as
arguments, to be a score for any two arbitrary metrics. If the magnitude of
$\beta$ increases, the evaluation significantly considers the weight of
$m_{2}$. Conversely, as the value of $\beta$ approaches zero, the weight of
$m_{1}$ is significantly considered in the evaluation.

#### Experiment results

Table 3 shows the experiment results for all combinations possible in our
analysis axes which consist of previous methods and our newly proposed
methods. For each task, we present the experimental results based on our three
analysis axes proposed in section §2: type of constraint loss (soft, binary,
real), exploration strategy of constraint-violating examples (top-1, sampling,
exhaustive), and mechanism for integrating the main and the constraint
information (static, monotone, proj-sup, proj-con, proj-both).

As the sheer number of experiments is too large to interpret in table 3, we
try to examine key factors for best main task performance, constraint
violation by dissecting the table 3 from different perspectives.

#### Trends per analysis axes

Figure 4 illustrates the top 5 experimental results with the highest
$H\beta$-scores for each of the five integration mechanisms described in
section §2.3. Among the five integration mechanisms, projection-con and
projection-both consistently demonstrates the best performance across most
$\beta$ values. They excel in a wide range of scenarios, from those
emphasizing main task metrics (lower $\beta$ values) to those prioritizing
constraint injection task performance (higher $\beta$ values). The static and
monotone mechanism seldom performs well , they do not always exhibit excellent
performance across all tasks.

Figure 5 illustrates the top 5 experimental results with the highest
$H\beta$-scores for each of the three types of constraint losses described in
section §2.1. Among the three types of losses, whether soft or real type shows
consistently better performance depends on the task. Our hypothesis is, as
mentioned in appendix A.1, soft and real types of losses can incorporate more
fine-grained information into constraint loss compared to binary types of
loss.

Figure 6 illustrates the top 5 experimental results with the highest
$H\beta$-scores, considering each of the five exploring strategies described
in section §2.2. Among the five strategies, one clear observation is that the
sampling method consistently demonstrates superior performance across all
tasks. Although there are variations, performance tends to improve as the
sample size increases. However, the overall performance of the full strategy
is not favorable, especially in SRL task. In the full strategy, the model
generates errors that significantly different from those expected for
realistic output, resulting in suboptimal performance due to the associated
loss. We hypothesize that the full strategy’s performance of SRL is even worse
than that observed in NLI, due to the significantly larger output space.

To summarize, sampling strategy and projection-con, projection-both mechanisms
consistently demonstrate superior performance across all tasks. However, in
relation to the type of constraint loss, there is no type of loss that
consistently shows superior performance across all tasks; it varies depending
on the task. As shown in Table 3, the number of combinations of learning
algorithms with output constraints based on our analysis criteria is quite
large (65 combinations for the NLI and SRL tasks, and 40 combinations for the
STE task). Therefore, it is practically impossible to experiment with all
learning algorithms. We examined the performance trends of the algorithms
through figures 1, 2, and 3, which provide useful insights for selecting
learning algorithms.

#### Specific combinations of axes outperforming others

In addition to observing overall trends, we dive into a more detailed analysis
of specific algorithm combinations and their performance. We observed in the
previous experimental results (figures 4, 5, 6) that the sampling strategy and
projection-con, projection-both mechanisms generally perform well, with
performance improving as sample size increases. However, the results in
figures 4, 5, and 6 represent averages across multiple algorithm outcomes and
do not depict individual algorithms. In this section, we narrow our focus and
present an analysis for individual algorithms assuming a fixed sampling
strategy with a sample size of 10 (referred to as samp-10 from now on) which
consistently performed the best across different tasks, across different
conditions. Figures 1, 2, and 3 depict the $H\beta$-scores for different
combinations of loss types and integration mechanisms when the sampling
strategy is fixed as samp-10. For visibility, we consider values of $0.3$,
$1$, and $3$.

Notably, our newly proposed projection-based algorithms, projection-con and
projection-both, exhibit the highest-level performance across most situations.
One interesting point is the performance difference between projection-con and
projection-both mechanisms. By examining the average of the top 5 number of
$H\beta$-scores (as previously shown in figure 4), we find that projection-con
outperforms other mechanisms. However, upon observing individual algorithms
per task, we found that for the soft or real types of loss, the projection-
both mechanism shows the best-level performance than other mechanisms for most
combinations. In the case of the SRL task, there are instances where the
monotone mechanism performs well. Particularly, when used in conjunction with
a soft type of loss, the monotone mechanism exhibits higher performance, which
is inconsistent with other experimental results. The reason for this
discrepancy has not been clearly identified yet, but the specific
characteristics of weight updates in constraint loss combined with a soft type
of loss for achieving higher performance remain a subject for future work.

Figure 1: Experiment result from NLI task with samp-10. Three bar plots
represents the $H\beta$-scores with respect to the integration mechanism
(separated by the $x$-axis) and type of constraint losses (separated by the
color). From top to bottom, the corresponding values of $\beta$’s are $0.3$,
$1$, $3$, respectively. Figure 2: Experiment result from STE task with
samp-10. Three bar plots represents the $H\beta$-scores with respect to the
integration mechanism (separated by the $x$-axis) and type of constraint
losses (separated by the color). From top to bottom, the corresponding values
of $\beta$’s are $0.3$, $1$, $3$, respectively. Figure 3: Experiment result
from SRL task with samp-10. Three bar plots represents the $H\beta$-scores
with respect to the integration mechanism (separated by the $x$-axis) and type
of constraint losses (separated by the color). From top to bottom, the
corresponding values of $\beta$’s are $0.3$, $1$, $3$, respectively.

Another noteworthy observation is that under samp-10, the soft type of loss
exhibits the highest performance in most cases. Results from figures 1 and 3
show that, except for the projection-sup instance in SRL, soft type of loss
generally outperforms real type of loss. We previously observed from figure 5
that real type of loss tends to perform best in the SRL task among the three
types of losses. The results in 3, however, demonstrate the opposite,
indicating that soft type of loss performs exceptionally well when combined
with the sampling strategy.

##  6 Additional related work

There are two stages where constraints can be injected: at inference time and
learning time. At inference time, the goal is to remedy nonsensical outputs
that violate human constraints at test time regardless of the training
procedure Lee et al. (2019); Roth and Yih (2005). For example, Lee et al.
(2019) updates the model parameters at test time for each test instance to
satisfy constraints, while Roth and Yih (2005) transforms the constrained
problem into the form of integer linear programming for the inference process
to maximize the log probability score with constraint satisfaction. Both
methods have shown to satisfy constraints well and also improve performances,
however, as these two methods utilize vastly different philosophies, their
formulations are not directly comparable.

On the other hand, another line of research have explored how to practically
use constraint injection in software development Ahmed et al. (2022); Rajaby
Faghihi et al. (2021), which demonstrate the application of constraint
injection techniques in software. These tools can apply constraints across
different domains, highlighting the versatility of constraint learning
methods. Importantly, they are effective not only during the training phase
but also during inference. This research and the platforms mentioned offer
valuable insights into the practical application of constraints.

##  7 Conclusions

We have proposed three axes for classifying and categorizing learning
algorithms related to injecting constraints: type of constraint loss,
exploring strategy of constraint-violating examples, and integration of main
task and constraint information. To the best of our knowledge, this study is
the first to systematically classify existing learning algorithms with
constraints under a unified formulation. We have analyzed the key factors that
affect performance based on our analysis criteria, which helps in
understanding learning algorithms with constraints.

Additionally, we have introduced three projection-based mechanisms as a novel
approach for the integration mechanism of main task and constraint
information. Viewing the main task and constraint injection as two separate
tasks, we started with the motivation to prevent negative effects on each
other during the gradient update process. This introduces a new perspective on
integrating learning signals from main task and constraint, which shows
superior performance compared to existing integration mechanisms.

## References

  * Ahmed et al. (2022) Kareem Ahmed, Tao Li, Thy Ton, Quan Guo, Kai-Wei Chang, Parisa Kordjamshidi, Vivek Srikumar, Guy Van den Broeck, and Sameer Singh. 2022.  Pylon: A pytorch framework for learning with constraints.  In _NeurIPS 2021 Competitions and Demonstrations Track_ , pages 319–324. PMLR.
  * Bowman et al. (2015) Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015.  A large annotated corpus for learning natural language inference.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_ , pages 632–642, Lisbon, Portugal. Association for Computational Linguistics.
  * Bröcheler et al. (2010) Matthias Bröcheler, Lilyana Mihalkova, and Lise Getoor. 2010.  Probabilistic similarity logic.  In _Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence_ , UAI’10, page 73–82, Arlington, Virginia, USA. AUAI Press.
  * Chaudhry et al. (2018) Arslan Chaudhry, Marc’Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. 2018.  Efficient lifelong learning with a-gem.  _arXiv preprint arXiv:1812.00420_.
  * Lee et al. (2019) Jay Yoon Lee, Sanket Vaibhav Mehta, Michael Wick, Jean-Baptiste Tristan, and Jaime Carbonell. 2019.  Gradient-based inference for networks with output constraints.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 33, pages 4147–4154.
  * Li et al. (2020) Tao Li, Parth Anand Jawale, Martha Palmer, and Vivek Srikumar. 2020.  Structured tuning for semantic role labeling.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 8402–8412, Online. Association for Computational Linguistics.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Lopez-Paz and Ranzato (2017) David Lopez-Paz and Marc’Aurelio Ranzato. 2017.  Gradient episodic memory for continual learning.  _Advances in neural information processing systems_ , 30.
  * Mehta et al. (2018) Sanket Vaibhav Mehta, Jay Yoon Lee, and Jaime Carbonell. 2018.  Towards semi-supervised learning for deep semantic role labeling.  _arXiv preprint arXiv:1808.09543_.
  * Minervini and Riedel (2018) Pasquale Minervini and Sebastian Riedel. 2018.  Adversarially regularising neural NLI models to integrate logical background knowledge.  In _Proceedings of the 22nd Conference on Computational Natural Language Learning_ , pages 65–74, Brussels, Belgium. Association for Computational Linguistics.
  * Nandwani et al. (2019) Yatin Nandwani, Abhishek Pathak, and Parag Singla. 2019.  A primal dual formulation for deep learning with constraints.  _Advances in Neural Information Processing Systems_ , 32.
  * Pradhan et al. (2012) Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012.  Conll-2012 shared task: Modeling multilingual unrestricted coreference in ontonotes.  In _Joint conference on EMNLP and CoNLL-shared task_ , pages 1–40.
  * Rajaby Faghihi et al. (2021) Hossein Rajaby Faghihi, Quan Guo, Andrzej Uszok, Aliakbar Nafar, and Parisa Kordjamshidi. 2021.  DomiKnowS: A library for integration of symbolic domain knowledge in deep learning.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 231–241, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Rajaby Faghihi et al. (2023) Hossein Rajaby Faghihi, Aliakbar Nafar, Chen Zheng, Roshanak Mirzaee, Yue Zhang, Andrzej Uszok, Alexander Wan, Tanawan Premsri, Dan Roth, and Parisa Kordjamshidi. 2023.  Gluecons: A generic benchmark for learning under constraints.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 33, pages 9552–9561.
  * Roth and Yih (2005) Dan Roth and Wen-tau Yih. 2005.  Integer linear programming inference for conditional random fields.  In _Proceedings of the 22nd international conference on Machine learning_ , pages 736–743.
  * Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014.  Sequence to sequence learning with neural networks.  _Advances in neural information processing systems_ , 27.
  * Williams (1992) Ronald J Williams. 1992.  Simple statistical gradient-following algorithms for connectionist reinforcement learning.  _Machine learning_ , 8:229–256.
  * Xu et al. (2018) Jingyi Xu, Zilu Zhang, Tal Friedman, Yitao Liang, and Guy Broeck. 2018.  A semantic loss function for deep learning with symbolic knowledge.  In _International conference on machine learning_ , pages 5502–5511. PMLR.

##  Appendix A More specific comparison between types of constraint losses:
PSL and REINFORCE

In this section, we dive deeper into the characteristics and applicability of
the two types of constraint loss mentioned in Section 2.1: Probabilistic Soft
Logic (PSL) and REINFORCE. These types of losses have unique strengths and
weaknesses depending on the different circumstances.

###  A.1 Fine-grained expressiveness of constraints

PSL stands superior in capturing more fine-grained information compared to
REINFORCE. The PSL type of loss function evaluates not only the overall
outcome but also performance of individual components to encourage more
detailed feedback. For instance, consider a multi-label classification
setting, where a particular book’s every possible category needs to be
predicted. An easily understandable example of constraint is associated with
the hierarchical structure between labels: If a model predicts ‘science
fiction’, it must necessarily also make a prediction that includes a hierarchy
higher than that, which is ‘fiction’. Note that, the above hierarchical
constraint can be considered in the form of conditional statement for
propositions, as follows:

$\displaystyle\text{Pred(science fiction)}\implies\text{Pred(fiction)}$

For the sake of simplicity in explanation, we employ the Łukasiewicz logic for
this example. The soft value corresponding to the above logical expression is:

$\displaystyle\text{min}(1,1-P_{s}+P_{f})$ (7)

, where $P_{s}$ and $P_{f}$ represent the probability of being predicted for
the science fiction class, and fiction class, respectively. In constraint
learning using PSL the learning process aims to increase the soft value (7).
In this example, the learning is conducted to increase $P_{f}-P_{s}$. Since
‘fiction’ is a class with higher hierarchy, and more inclusive class than
‘science fiction’, learning to increase $P_{f}-P_{s}$ is highly reasonable.
Likewise, the PSL type of constraint loss can enrich the model’s understanding
and providing more detailed feedback. In REINFORCE, however, if the model’s
prediction violates the constraint, the probability for the prediction is
directly reflected in the loss, regardless of the constraint imposed. This
makes it challenging to provide detailed information about specifically which
part should we penalize in the model’s prediction. To incorporate more fine-
grained information in REINFORCE, there is research that utilizes real
rewards. For example, Mehta et al. defines reward score as $s=1-2g\in[-1,1]$,
where $g\in[0,1]$ stands for normalized error count, so that larger constraint
violations lead to greater constraint loss. Although the loss function cannot
reflect the soft value of logical expression, by assigning rewards differently
based on the degree of constraint violation, it is possible to incorporate
more fine-grained information into the loss function than just assigning
binary rewards.

Logic | Product | Gödel | Łukasiewicz
---|---|---|---
Negation | $1-a$ | $1-a$ | $1-a$
T-conorm | $a+b-ab$ | max$(a,b)$ | min$(1,a+b)$
T-norm | $ab$ | min$(a,b)$ | max$(0,a+b-1)$
Implication | $\begin{cases}1&\text{if }a\leq b\\\ b/a&\text{otherwise}\end{cases}$ | $\begin{cases}1&\text{if }a\leq b\\\ b&\text{otherwise}\end{cases}$ | $\min(1,1-a+b)$

Table 1: Examples of logics. Our experiment used Gödel logic except for the
implication ($\Rightarrow$). For $\Rightarrow$, we used S-implication form,
$\max(1-a,b)$.

###  A.2 Type of constraints that constraint loss can represent

Though PSL stands superior in capturing more fine-grained information compared
to REINFORCE, PSL encounters difficulties when representing a variety of
constraints, while REINFORCE can express arbitrary types of constraints.
Rajaby Faghihi et al. introduced limitations in encoding a specific type of
knowledge in research related to constraint injection during training time
Nandwani et al. (2019); Ahmed et al. (2022). Instead of focusing on individual
characteristics of these studies, we can generalize this limitation using our
view of analysis axes. As in table 2, we can rewrite the constraint types that
each constraint loss type can handle, according to the two types of constraint
loss: PSL and REINFORCE.

NC (Needs Conversion) in table 2 can sometimes be practically challenging due
to significant overhead, making it difficult to leverage effectively. An
example is seen in the STE task discussed in §4, where the constraint is
defined as follows: the count of ‘b’ in the target should be exactly three
times that in the source. Let $s\in(az|bz)^{*}$ be an input sequence data, and
$t$ be a predicted output sequence of model. Also, for a finite set
$X=\\{x_{1},x_{2},\cdots,x_{n}\\}\subseteq\mathbb{N}$, let $s(X)$, $t(X)$
represent the presence of ‘b’ at positions $x_{1},...,x_{n}$ in the sequences
$s$ and $t$, respectively. Note that if the count of ‘b’ in $s$ is 1, then the
count of ‘b’ in $t$ should be 3. While this represents a small portion of the
original constraint, when expressed in the linearized logical expression for
PSL application, it can be represented as follows:

$\displaystyle\overset{|s|}{\underset{i=1}{\bigwedge}}\left[s(\\{i\\})\implies\underset{1\leq
j_{1}<j_{2}<j_{3}\leq|t|}{\bigvee}t(\\{j_{1},j_{2},j_{3}\\})\right]$

However, even this partial inclusion of the overall constraint requires an
excessively high computational cost.

| Seq | Lin | Log | Log+Quan | Prog
---|---|---|---|---|---
PSL | ✓ | ✓ | NC | NC | X
REINFORCE | ✓ | ✓ | ✓ | ✓ | ✓

Table 2: This table classifies constraints that constraint-injection methods
can handle during training time. We reinterpret table 2 of Rajaby Faghihi et
al. (2023) with our axes of analysis: type of constraint loss. The specific
meaning of abbreviations are as follows: Seq=sequential structure, Lin=linear
constraint, Log=logical constraint, Log+Quan=logical constraint with
quantifier, Prog=any constraints encoded as a program, NC=needs conversion.

The comparison between two types of constriant losses illustrates that the
appropriate type of loss may vary depending on task requirements and problem
details, reflecting the inevitable trade-off between the level of detailed
information about constraints and the scope of constraint representation.

##  Appendix B Experiment result

Figure 4: The $H\beta$-score values for different values of $\beta$ for three
tasks: NLI, STE and SRL. For each $\beta$, the top 5 experimental results with
the highest $H\beta$-scores are presented for each of the five integration
mechanisms described in section §2.3. Figure 5: The $H\beta$-score values for
different values of $\beta$ for three tasks: NLI, STE and SRL. For each
$\beta$, the top 5 experimental results with the highest $H\beta$-scores are
presented for each of the three types of constraint losses described in
section §2.1. Figure 6: The $H\beta$-score values for different values of
$\beta$ for three tasks: NLI, STE and SRL. For each $\beta$, the top 5
experimental results with the highest $H\beta$-scores are presented for each
of the five integration mechanisms described in section §2.2.

Table 3 represents the experiment results for all combinations, containing
main task metrics(%, denoted as Perf) and constraint violation rates(%,
denoted as Const.vio). SRL, NLI, and STE tasks used F1 score, accuracy, and
token accuracy for main task metric, respectively. For the types of constraint
losses, soft, binary, and real respectively represents PSL, REINFORCE method
with binary reward, and REINFORCE method with real reward. The term ‘Baseline’
refers to the experiment results without any constraint injection. Ahmed et
al., using the REINFORCE - binary reward method, separates the generation of
constraint loss into two approaches in their
implementation333https://github.com/pylon-lib/pylon: one for decoded samples
that satisfy the constraints and another for those that do not. In Mehta et
al., 2018, they generates constraint loss for decoded samples only when the
constraints are violated. To compare various algorithms under a unified
formulation, experiments involving constraint loss related to REINFORCE were
conducted by generating constraint loss for examples that violated the
constraints.

Note that we can easily extend the learning algorithms with constraints to
semi-supervised learning. For SRL and NLI tasks, we also utilized unlabeled
data during the training process. For SRL, we randomly selected $3\%$ of
training data for unlabeled data. For NLI, we utilized the unlabeled data used
in Ahmed et al. (2022)444https://github.com/pylon-
lib/pylon/tree/master/examples/nli.

Task Top-1 Sampling-1 Sampling-5 Sampling-10 Exhaustive Perf Const.Vio Perf
Const.Vio Perf Const.Vio Perf Const.Vio Perf Const.Vio NLI Baseline Acc:
$\underset{\pm 00.30}{65.14}$, Const.Vio: $\underset{\pm 02.57}{20.81}$ soft
static $\underset{\pm 00.35}{65.18}$ $\underset{\pm 00.88}{04.72}$
$\underset{\pm 00.40}{65.40}$ $\underset{\pm 00.60}{03.60}$ $\underset{\pm
00.26}{65.31}$ $\underset{\pm 00.36}{02.23}$ $\underset{\pm 00.40}{65.22}$
$\underset{\pm 01.88}{02.29}$ $\underset{\pm 00.34}{65.20}$ $\underset{\pm
00.22}{01.95}$ monotone ($\lambda\uparrow$) $\underset{\pm 00.26}{65.21}$
$\underset{\pm 03.03}{21.51}$ $\underset{\pm 00.27}{65.20}$ $\underset{\pm
02.09}{20.24}$ $\underset{\pm 00.38}{65.30}$ $\underset{\pm 01.74}{21.83}$
$\underset{\pm 00.54}{65.33}$ $\underset{\pm 01.67}{20.20}$ $\underset{\pm
00.55}{65.20}$ $\underset{\pm 02.05}{22.72}$ Proj-Sup $\underset{\pm
00.43}{65.28}$ $\underset{\pm 02.99}{20.14}$ $\underset{\pm 00.48}{65.05}$
$\underset{\pm 01.61}{20.73}$ $\underset{\pm 00.40}{65.26}$ $\underset{\pm
00.38}{02.08}$ $\underset{\pm 00.20}{65.38}$ $\underset{\pm 02.55}{13.93}$
$\underset{\pm 00.49}{65.36}$ $\underset{\pm 00.41}{01.95}$ Proj-Con
$\underset{\pm 00.27}{65.46}$ $\underset{\pm 00.42}{03.05}$ $\underset{\pm
00.40}{65.23}$ $\underset{\pm 00.74}{03.54}$ $\underset{\pm 00.26}{65.05}$
$\underset{\pm 00.33}{02.40}$ $\underset{\pm 00.28}{65.48}^{*}$ $\underset{\pm
00.17}{01.73}$ $\underset{\pm 00.29}{65.30}$ $\underset{\pm 00.36}{02.50}$
Proj-Both $\underset{\pm 00.40}{65.20}$ $\underset{\pm 03.13}{17.45}$
$\underset{\pm 00.29}{65.41}$ $\underset{\pm 01.75}{06.16}$ $\underset{\pm
00.21}{65.39}$ $\underset{\pm 05.04}{08.19}$ $\underset{\pm 00.32}{65.23}$
$\underset{\pm 00.84}{01.17}^{*}$ $\underset{\pm 00.43}{65.40}$ $\underset{\pm
03.29}{21.30}$ binary static $\underset{\pm 00.38}{65.20}$ $\underset{\pm
02.65}{02.45}$ $\underset{\pm 00.44}{64.23}$ $\underset{\pm 05.29}{03.24}$
$\underset{\pm 00.38}{63.26}$ $\underset{\pm 11.67}{17.02}$ $\underset{\pm
00.49}{63.26}$ $\underset{\pm 02.40}{24.72}$ - - monotone ($\lambda\uparrow$)
$\underset{\pm 00.31}{65.36}$ $\underset{\pm 03.38}{20.30}$ $\underset{\pm
00.29}{65.10}$ $\underset{\pm 02.66}{22.66}$ $\underset{\pm 00.37}{65.16}$
$\underset{\pm 01.78}{22.00}$ $\underset{\pm 00.36}{65.29}$ $\underset{\pm
02.74}{21.32}$ - - Proj-Sup $\underset{\pm 00.35}{65.21}$ $\underset{\pm
03.30}{14.73}$ $\underset{\pm 00.47}{65.25}$ $\underset{\pm 01.66}{07.06}$
$\underset{\pm 00.23}{65.22}$ $\underset{\pm 00.20}{02.25}$ $\underset{\pm
00.25}{65.18}$ $\underset{\pm 04.97}{07.80}$ - - Proj-Con $\underset{\pm
00.49}{65.11}$ $\underset{\pm 06.08}{07.49}$ $\underset{\pm 00.19}{65.42}$
$\underset{\pm 02.43}{11.80}$ $\underset{\pm 00.35}{65.25}$ $\underset{\pm
00.16}{02.57}$ $\underset{\pm 00.24}{65.33}$ $\underset{\pm 03.01}{20.40}$ - -
Proj-Both $\underset{\pm 00.28}{65.16}$ $\underset{\pm 02.85}{19.76}$
$\underset{\pm 00.34}{65.05}$ $\underset{\pm 00.28}{02.11}$ $\underset{\pm
00.41}{65.23}$ $\underset{\pm 00.37}{02.10}$ $\underset{\pm 00.49}{65.14}$
$\underset{\pm 00.33}{02.18}$ - - real static $\underset{\pm 00.24}{65.26}$
$\underset{\pm 01.29}{20.60}$ $\underset{\pm 00.34}{64.66}$ $\underset{\pm
00.90}{01.92}$ $\underset{\pm 00.44}{63.28}$ $\underset{\pm 01.89}{23.82}$
$\underset{\pm 00.30}{63.26}$ $\underset{\pm 07.16}{22.72}$ - - monotone
($\lambda\uparrow$) $\underset{\pm 00.21}{65.42}$ $\underset{\pm
01.88}{21.45}$ $\underset{\pm 00.49}{65.14}$ $\underset{\pm 02.25}{20.97}$
$\underset{\pm 00.38}{65.26}$ $\underset{\pm 02.26}{21.36}$ $\underset{\pm
00.38}{65.26}$ $\underset{\pm 03.68}{21.23}$ - - Proj-Sup $\underset{\pm
00.26}{65.26}$ $\underset{\pm 01.96}{22.93}$ $\underset{\pm 00.25}{65.21}$
$\underset{\pm 02.23}{22.70}$ $\underset{\pm 00.39}{65.11}$ $\underset{\pm
02.76}{20.68}$ $\underset{\pm 00.48}{65.51}$ $\underset{\pm 02.03}{21.65}$ - -
Proj-Con $\underset{\pm 00.32}{65.27}$ $\underset{\pm 01.67}{20.98}$
$\underset{\pm 00.36}{65.33}$ $\underset{\pm 01.86}{08.75}$ $\underset{\pm
00.24}{65.04}$ $\underset{\pm 00.36}{02.39}$ $\underset{\pm 00.38}{65.49}$
$\underset{\pm 00.20}{01.78}$ - - Proj-Both $\underset{\pm 00.34}{65.09}$
$\underset{\pm 01.27}{23.21}$ $\underset{\pm 00.31}{65.15}$ $\underset{\pm
02.77}{07.04}$ $\underset{\pm 00.39}{65.40}$ $\underset{\pm 03.42}{09.33}$
$\underset{\pm 00.26}{65.29}$ $\underset{\pm 03.62}{20.40}$ - - STE Baseline
Tok-Acc: $\underset{\pm 02.26}{67.26}$, Const.Vio: $\underset{\pm
10.07}{28.89}$ binary static $\underset{\pm 00.43}{69.98}$ $\underset{\pm
12.11}{29.35}$ $\underset{\pm 02.37}{73.59}$ $\underset{\pm 12.72}{18.83}$
$\underset{\pm 02.20}{69.35}$ $\underset{\pm 06.73}{33.32}$ $\underset{\pm
03.22}{69.68}$ $\underset{\pm 17.58}{29.87}$ - - monotone ($\lambda\uparrow$)
$\underset{\pm 06.42}{67.17}$ $\underset{\pm 13.09}{26.93}$ $\underset{\pm
05.72}{71.03}$ $\underset{\pm 11.46}{33.55}$ $\underset{\pm 03.79}{69.43}$
$\underset{\pm 11.15}{24.89}$ $\underset{\pm 04.63}{71.07}$ $\underset{\pm
08.34}{21.74}$ - - Proj-Sup $\underset{\pm 04.03}{43.55}$ $\underset{\pm
03.65}{93.87}$ $\underset{\pm 05.90}{75.01}$ $\underset{\pm 07.28}{10.86}$
$\underset{\pm 03.67}{69.19}$ $\underset{\pm 16.22}{26.21}$ $\underset{\pm
02.96}{71.81}$ $\underset{\pm 17.69}{26.38}$ - - Proj-Con $\underset{\pm
03.27}{49.64}$ $\underset{\pm 02.10}{98.40}$ $\underset{\pm 03.97}{73.58}$
$\underset{\pm 16.16}{22.18}$ $\underset{\pm 06.76}{74.96}$ $\underset{\pm
16.42}{22.71}$ $\underset{\pm 08.92}{77.48}$ $\underset{\pm 14.79}{19.08}$ - -
Proj-Both $\underset{\pm 01.70}{51.19}$ $\underset{\pm 02.08}{98.77}$
$\underset{\pm 03.92}{70.71}$ $\underset{\pm 12.72}{23.43}$ $\underset{\pm
03.34}{68.51}$ $\underset{\pm 10.77}{26.62}$ $\underset{\pm 02.20}{68.94}$
$\underset{\pm 15.75}{26.06}$ - - real static $\underset{\pm 04.25}{67.77}$
$\underset{\pm 12.86}{30.18}$ $\underset{\pm 03.03}{70.16}$ $\underset{\pm
12.04}{22.18}$ $\underset{\pm 04.02}{70.30}$ $\underset{\pm 07.53}{10.86}^{*}$
$\underset{\pm 05.00}{78.13}$ $\underset{\pm 11.15}{15.37}$ - - monotone
($\lambda\uparrow$) $\underset{\pm 04.43}{63.73}$ $\underset{\pm
13.45}{22.92}$ $\underset{\pm 02.93}{60.74}$ $\underset{\pm 34.51}{51.79}$
$\underset{\pm 03.29}{69.91}$ $\underset{\pm 13.41}{19.07}$ $\underset{\pm
03.74}{71.53}$ $\underset{\pm 13.05}{21.99}$ - - Proj-Sup $\underset{\pm
04.91}{46.30}$ $\underset{\pm 04.92}{94.17}$ $\underset{\pm 02.47}{54.32}$
$\underset{\pm 01.72}{98.86}$ $\underset{\pm 04.14}{73.02}$ $\underset{\pm
08.75}{15.04}$ $\underset{\pm 02.49}{72.55}$ $\underset{\pm 10.61}{18.36}$ - -
Proj-Con $\underset{\pm 03.88}{48.20}$ $\underset{\pm 04.58}{95.29}$
$\underset{\pm 01.16}{52.98}$ $\underset{\pm 00.73}{99.74}$ $\underset{\pm
03.04}{72.24}$ $\underset{\pm 07.57}{14.26}$ $\underset{\pm 03.16}{75.86}$
$\underset{\pm 08.36}{18.66}$ - - Proj-Both $\underset{\pm 02.60}{50.60}$
$\underset{\pm 03.16}{98.68}$ $\underset{\pm 05.59}{74.81}$ $\underset{\pm
16.24}{17.40}$ $\underset{\pm 04.31}{72.00}$ $\underset{\pm 14.56}{15.03}$
$\underset{\pm 04.24}{80.92}^{*}$ $\underset{\pm 06.07}{12.91}$ - - SRL
Baseline F1: $\underset{\pm 00.77}{84.72}$, Const.Vio: $\underset{\pm
04.09}{20.43}$ soft static $\underset{\pm 01.49}{85.24}$ $\underset{\pm
02.04}{15.17}$ $\underset{\pm 01.55}{85.02}$ $\underset{\pm 03.02}{14.07}$
$\underset{\pm 01.13}{85.21}$ $\underset{\pm 02.80}{19.53}$ $\underset{\pm
00.74}{85.15}$ $\underset{\pm 36.95}{19.18}$ $\underset{\pm 00.98}{85.31}$
$\underset{\pm 04.61}{21.72}$ monotone ($\lambda\uparrow$) $\underset{\pm
01.07}{84.42}$ $\underset{\pm 04.44}{18.53}$ $\underset{\pm 01.46}{85.78}^{*}$
$\underset{\pm 03.66}{14.40}$ $\underset{\pm 01.23}{85.12}$ $\underset{\pm
03.12}{16.38}$ $\underset{\pm 01.16}{84.49}$ $\underset{\pm 01.42}{05.73}^{*}$
$\underset{\pm 00.81}{85.18}$ $\underset{\pm 02.97}{15.97}$ Proj-Sup
$\underset{\pm 00.98}{85.02}$ $\underset{\pm 04.63}{20.79}$ $\underset{\pm
01.24}{85.19}$ $\underset{\pm 04.85}{20.23}$ $\underset{\pm 01.03}{85.09}$
$\underset{\pm 03.93}{19.59}$ $\underset{\pm 01.26}{85.32}$ $\underset{\pm
04.08}{15.86}$ $\underset{\pm 00.97}{85.18}$ $\underset{\pm 04.03}{18.73}$
Proj-Con $\underset{\pm 00.60}{84.96}$ $\underset{\pm 01.23}{15.72}$
$\underset{\pm 01.53}{85.24}$ $\underset{\pm 02.44}{11.76}$ $\underset{\pm
00.90}{85.07}$ $\underset{\pm 02.85}{12.80}$ $\underset{\pm 01.17}{84.58}$
$\underset{\pm 03.61}{12.11}$ $\underset{\pm 00.57}{84.31}$ $\underset{\pm
04.01}{18.04}$ Proj-Both $\underset{\pm 01.21}{85.21}$ $\underset{\pm
03.13}{21.86}$ $\underset{\pm 01.06}{84.71}$ $\underset{\pm 00.37}{12.11}$
$\underset{\pm 01.68}{85.62}$ $\underset{\pm 03.57}{17.68}$ $\underset{\pm
02.06}{84.93}$ $\underset{\pm 01.83}{10.66}$ $\underset{\pm 01.11}{85.03}$
$\underset{\pm 04.26}{17.64}$ binary static $\underset{\pm 01.51}{85.20}$
$\underset{\pm 00.66}{19.84}$ $\underset{\pm 01.02}{85.52}$ $\underset{\pm
02.34}{19.53}$ $\underset{\pm 00.91}{85.19}$ $\underset{\pm 02.91}{18.90}$
$\underset{\pm 01.28}{84.71}$ $\underset{\pm 04.34}{22.48}$ - - monotone
($\lambda\uparrow$) $\underset{\pm 00.94}{84.51}$ $\underset{\pm
02.93}{14.25}$ $\underset{\pm 00.75}{84.36}$ $\underset{\pm 05.88}{20.98}$
$\underset{\pm 01.44}{85.23}$ $\underset{\pm 03.37}{15.50}$ $\underset{\pm
00.68}{85.19}$ $\underset{\pm 04.40}{16.26}$ - - Proj-Sup $\underset{\pm
01.05}{84.14}$ $\underset{\pm 02.12}{21.20}$ $\underset{\pm 01.20}{84.57}$
$\underset{\pm 02.20}{19.99}$ $\underset{\pm 01.01}{85.70}$ $\underset{\pm
03.92}{22.44}$ $\underset{\pm 00.57}{84.73}$ $\underset{\pm 03.03}{19.05}$ - -
Proj-Con $\underset{\pm 00.71}{84.54}$ $\underset{\pm 03.91}{21.28}$
$\underset{\pm 01.23}{85.56}$ $\underset{\pm 04.38}{19.62}$ $\underset{\pm
01.03}{85.06}$ $\underset{\pm 03.58}{19.79}$ $\underset{\pm 00.86}{84.74}$
$\underset{\pm 06.10}{19.23}$ - - Proj-Both $\underset{\pm 00.77}{85.23}$
$\underset{\pm 04.22}{20.36}$ $\underset{\pm 01.36}{84.89}$ $\underset{\pm
03.33}{19.14}$ $\underset{\pm 01.014}{84.85}$ $\underset{\pm 03.91}{19.46}$
$\underset{\pm 00.63}{84.61}$ $\underset{\pm 03.39}{21.09}$ - - real static
$\underset{\pm 00.68}{85.05}$ $\underset{\pm 03.21}{18.26}$ $\underset{\pm
00.96}{85.12}$ $\underset{\pm 03.15}{09.79}$ $\underset{\pm 01.09}{85.33}$
$\underset{\pm 04.43}{22.37}$ $\underset{\pm 01.23}{84.85}$ $\underset{\pm
04.10}{20.32}$ monotone ($\lambda\uparrow$) $\underset{\pm 00.78}{84.73}$
$\underset{\pm 04.21}{21.99}$ $\underset{\pm 00.96}{85.09}$ $\underset{\pm
03.95}{23.78}$ $\underset{\pm 01.29}{84.94}$ $\underset{\pm 04.50}{19.61}$
$\underset{\pm 01.14}{85.36}$ $\underset{\pm 02.65}{20.00}$ - - Proj-Sup
$\underset{\pm 01.46}{85.19}$ $\underset{\pm 04.13}{19.19}$ $\underset{\pm
00.90}{85.09}$ $\underset{\pm 04.40}{17.16}$ $\underset{\pm 00.94}{84.87}$
$\underset{\pm 02.60}{09.21}$ $\underset{\pm 01.25}{85.29}$ $\underset{\pm
02.20}{09.05}$ - - Proj-Con $\underset{\pm 00.93}{85.06}$ $\underset{\pm
04.32}{18.22}$ $\underset{\pm 01.67}{84.31}$ $\underset{\pm 04.19}{09.47}$
$\underset{\pm 01.32}{85.19}$ $\underset{\pm 05.09}{22.55}$ $\underset{\pm
01.08}{85.11}$ $\underset{\pm 03.98}{17.50}$ - - Proj-Both $\underset{\pm
00.94}{85.05}$ $\underset{\pm 04.55}{18.54}$ $\underset{\pm 00.74}{85.25}$
$\underset{\pm 04.28}{20.36}$ $\underset{\pm 00.41}{84.54}$ $\underset{\pm
02.87}{11.96}$ $\underset{\pm 07.19}{84.33}$ $\underset{\pm 03.35}{10.23}$ - -

Table 3: Experiment results for all combinations. The gray-colored numbers
represent results with main task metrics and constraint violations worse than
the baseline. For each type of constraint loss, results showing the highest
main task metric and lowest constraint violation are highlighted in bold. For
individual task, the highest main task metric and lowest constraint violation
results are marked with an asterisk (*). In SRL and STE tasks, where the
output takes the form of more than one token, the method of selecting the
class with the highest probability for each token was employed for Top-1
strategy.

##  Appendix C Pseudo-code for projection based integration mechanism

Algorithm 1 shows the detail pseudo-code for projection-both mechanism.
Pseudo-codes for projection-sup and projection-con mechanisms are variant of
algorithm 1. For projection-sup, there is no need to store $g_{con}^{ref}$,
nor to calculate the dot product between $g_{sup}$ and $g_{con}^{ref}$.
Likewise, for projection-con, there is no need to store $g_{sup}^{ref}$, nor
to calculate the dot product between $g_{con}$ and $g_{sup}^{ref}$.

Algorithm 1 Pseudo code for projection-both mechanism

Input: labeled data $\mathcal{D}_{L}=\langle x_{i},y_{i}\rangle_{i=1}^{T}$,
unlabeled data $\mathcal{D}_{U}=\langle x_{i}^{u}\rangle_{i=1}^{T}$ (if
available), model parameter $\theta$.

1: Initialize: $g_{sup}^{ref}\leftarrow 0$, $g_{con}^{ref}\leftarrow 0$.

2: while not converge do

3: $\langle x_{L},y_{L}\rangle\leftarrow\text{sample from }\mathcal{D}_{L}$

4: $\langle x_{U}\rangle\leftarrow\text{sample from }\mathcal{D}_{U}$

5: $g_{sup}\leftarrow\nabla\mathcal{L}(x_{L},y_{L};\theta)$

6:
$g_{con}\leftarrow\nabla(\mathcal{C}(x_{L};\theta)+\mathcal{C}(x_{U};\theta))$

7: if $g_{sup}\cdot g_{con}^{ref}<0$ then

8: $g_{sup}\leftarrow\text{ project }g_{sup}\text{ via }g_{con}^{ref}$

9: end if

10: if $g_{con}\cdot g_{sup}^{ref}<0$ then

11: $g_{con}\leftarrow\text{ project }g_{con}\text{ via }g_{sup}^{ref}$

12: end if

13: $g_{sup}^{ref}\leftarrow$ store the averaged vector of $g_{sup}$ across
gradient updates.

14: $g_{con}^{ref}\leftarrow$ store the averaged vector of $g_{con}$ across
gradient updates.

15: Gradient update of $\theta$ for the cumulative gradients: $g_{sup}$ and
$g_{con}$.

16: end while

##  Appendix D Additional details about selected tasks and implementations

###  D.1 SRL

The baseline employs the RoBERTa baseline model Liu et al. (2019), and two
linear layers are added after the last layer of RoBERTa. While the parameters
of the RoBERTa model are fixed, only the parameters of the last two linear
layers are trained.

The model predicts one of 9 tags -$\\{\texttt{O, B0, I0, ..., B3, I3}\\}$ \-
and transforms all other tags into O. During the training process, 3% of the
data is randomly sampled from the training data for use.

For the real type of constraint loss in REINFORCE algorithm, the method
employed to assign rewards is based on the count of duplicates in B. For all
types of B-X that appear more than once, we summed the occurrences of all
number of constraint-violated B-X and divided by the total sequence length,
and this is multiplied by the constraint loss.

###  D.2 NLI

NLI Rules
---
R1 | $\text{T}\implies\text{ent}(X_{1},X_{1})$
R2 | $\text{con}(X_{1},X_{2})\implies\text{con}(X_{2},X_{1})$
R3 | $\text{ent}(X_{1},X_{2})\implies\neg\text{con}(X_{2},X_{1})$
R4 | $\text{neu}(X_{1},X_{2})\implies\neg\text{con}(X_{2},X_{1})$
R5 | $\text{ent}(X_{1},X_{2})\wedge\text{ent}(X_{2},X_{3})\implies\text{ent}(X_{1},X_{1})$

Table 4: NLI Rules in Minervini and Riedel (2018).

The baseline employs the RoBERTa baseline model Liu et al. (2019), and two
linear layers are added after the last layer of RoBERTa. While the parameters
of the RoBERTa model are fixed, only the parameters of the last two linear
layers are trained. During training, 20% of the training data is randomly
sampled for use.

For the real type of constraint loss in REINFORCE algorithm, the method
employed to assign rewards is based on the value in the PSL. In cases where it
violates constraints, the corresponding PSL values are multiplied by the
constraint loss.

###  D.3 STE

The training data includes 3 to 6 instances of az’ and bz’ in the source
language, generating a dataset of 6000 instances. The test data comprises 3 to
8 instances of ‘az’ and ‘bz’ in the source language, transformed into the
target language. We utilize seq2seq Sutskever et al. (2014) LSTM for
prediction.

For the real type of constraint loss in REINFORCE algorithm, the method
employed to assign rewards is a length-normalized quadratic:
$(3x_{b}-y_{b})^{2}/(\text{len}(x)+\text{len}(y))$, where $x$ and $y$
respectively represents the input and output, while $x_{b}$ and $y_{b}$
respectively represents the number of occurrences of ‘b’ in the input and
output.

We don’t utilize a PSL type of constraint loss for STE task. This is because
expressing constraints about the number of ‘b’ occurrences in input and output
is highly intricate for PSL. This constraint serves as an example
demonstrating the difficulty of applying PSL to all types of constraints.


# Energy condition and cosmic censorship conjecture in the perfect fluid
collapse

Javad T. Firouzjaee  Department of Physics, K. N. Toosi University of
Technology,
P. O. Box 15875-4416, Tehran, Iran  School of Physics, Institute for Research
in Fundamental Sciences (IPM),
P. O. Box 19395-5531, Tehran, Iran<EMAIL_ADDRESS>
###### Abstract

The hypothesis of cosmic censorship plays a crucial role in classical general
relativity, namely, to ensure that naked singularities would never occur from
the black hole singularity. In this paper, we will present how energy
conditions prohibit forming the naked singularity in the spherical perfect
fluid collapse, and thus the strong cosmic censorship conjecture hold in this
model. We also show that this result can be extended to the cosmological
constant background.

##  I Introduction

Hawking and Penrose have shown spacetime singularities are unavoidable when
the trapped region form in general relativity and suitable energy conditions
are satisfied HawPen . The key issue is whether these singularities are
covered by the event horizon as in the Schwarzschild case or not, that is
whether they are black holes or naked singularities. Although Penrose’s cosmic
censorship conjecture (CCC) Penrose  states that physically reasonable initial
data cannot end in naked singularities visible to distant observers, various
counterexamples have been proposed. The issue is whether these are
astrophysically plausible counterexamples to the CCC. These counterexamples
usually challenged the strong cosmic censorship conjecture that, asserts that,
generically, timelike singularities never occur, so that even an observer who
falls into a black hole will never “see” the singularity. In counterpoint,
when singularities (or locally naked singularity) of gravitational collapse
are contained in black holes is known as the weak cosmic censorship conjecture
Wald:1997wa .

Indeed CCC counterexamples have been proposed in the case of pressure-free
spherically symmetric (LTB) collapse, which have attracted major interest LTB-
naked ; LTB-naked2 ; LTB-naked3  (see Joshi-book-review  for review). It seems
that naked singularity could occur in cases where there is no pressure, and
although pressure will occur and could alter the situation in more realistic
cases, these examples provide useful toy models as a basis for further
investigation. Most of the attempts are focused on finding timelike
singularities from which the null geodesics emerge (at least locally) other-
ref . In this letter, we will present how energy conditions prohibit forming
the naked singularity.

Our approach to show the CCC in the perfect fluid collapse is based on the
fact that the apparent horizon which is coincident with the singularity at the
center cannot be future timelike. We assume that the weak energy condition,
which states that the energy density is not negative, is satisfied and will
see how this is playing an important role in this context. In the following,
we use units in which $G=c=1$ and our reasoning is based on the radial null
geodesics.

##  II Perfect fluid spherical collapse

A collapsing perfect fluid within a compact spherically symmetric spacetime
region can be described Misner:1964je  by the following metric in the comoving
coordinates $(t,r,\theta,\varphi)$:

$\displaystyle ds^{2}$ $\displaystyle=$
$\displaystyle-e^{2\nu(t,r)}dt^{2}+e^{2\psi(t,r)}dr^{2}+R(t,r)^{2}d\Omega^{2},\,$
$\displaystyle\,u^{\mu}$ $\displaystyle=$ $\displaystyle
e^{-\nu(t,r)}\delta^{\mu}_{0}.$ (1)

Assume the energy momentum tensor has the perfect fluid form

$\displaystyle
T^{t}_{t}=-\rho(t,r),\,\,T^{r}_{r}=T^{\theta}_{\theta}=T^{\varphi}_{\varphi}=p(t,r).$
(2)

The Einstein equations for this energy momentum tensor give,

$\displaystyle\rho(t,r)=\frac{2M^{\prime}(t,r)}{R^{2}(t,r)R^{\prime}(t,r)},$
$\displaystyle~{}~{}p(t,r)=-\frac{2\dot{M(t,r)}}{R^{2}(t,r)\dot{R}(t,r)},$ (3)
$\nu^{\prime}(t,r)=-\frac{p^{\prime}(t,r)}{\rho(t,r)+p(t,r)},$ (4)

where $\dot{}$ and ′ mean the partial derivative relative to $t$ and $r$
respectively. Four velocity of this comoving fluid is
$u^{\mu}=(e^{-\nu},0,0,0)$ and after contraction this vector with the energy
momentum tensor, $\nabla_{\nu}T^{\mu\nu}=0$, we get the energy-momentum
conservation in the form

$\dfrac{d\rho}{d\tau}+\theta(\rho+p)=0,$ (5)

where $\theta=\nabla_{\mu}u^{\mu}$ and
$\dfrac{d}{d\tau}=e^{-\nu}\dfrac{d}{dt}$. As a result, one gets

$\displaystyle
e^{-\nu}\dfrac{d\rho}{dt}+(\dot{\nu}+\dot{\psi}+2\dfrac{\dot{R}}{R})(\rho+p)=0$
$\displaystyle-2\dot{R}^{\prime}+R^{\prime}\frac{\dot{G}}{G}+\dot{R}\frac{H^{\prime}}{H}=0,$
(6)

where

$G=e^{-2\psi}(R^{\prime})^{2}~{}~{},~{}~{}H=e^{-2\nu}(\dot{R})^{2},$ (7)

and $M(t,r)$ quantity is defined as

$\displaystyle G(t,r)-H(t,r)=1-\frac{2M(t,r)}{R(t,r)}.$ (8)

It is obvious from the equation (7) that $G,H>0$ are non-negative quantities.
It is known that the apparent horizon (that is the boundary of the trapped
region) is located at $R=2M$ Hayward:1994bu ; Firouzjaee-penn . The $M$
quantity is the Misner-Sharp Energy

$\displaystyle E_{MS}=M(R)=\frac{1}{2}\int_{0}^{R}\rho R^{2}dR,$ (9)

or

$E_{MS}=\frac{1}{8\pi}\int_{0}^{r}\rho\sqrt{(1+(\frac{dR}{d\tau})^{2}-\frac{2M}{R})}d^{3}V,$
(10)

where $d^{3}V=4\pi e^{\psi}R^{\prime}dr,$ and
$\frac{d}{d\tau}=e^{-\nu}\frac{d}{dt}.$ The last form of the function $M$
indicates that when considered as energy, it includes contribution terms the
kinetic energy and the gravitational potential energy.

Equation (9) shows that non-singular density gives $M(R\rightarrow 0)=0$. This
equation can be held just before the singularity formation (onset of
singularity formation) when $\rho/\rho_{0}\gg 1$ where $\rho_{0}$ can be the
average density of collapsing object.

During the gravitational collapse of a cloud of matter, a singularity will
form when physical quantities like the energy density $\rho$ and the Ricci
scalar $R$ will diverge. There can be two kinds of singularities in this
metric: a big bang (big crunch) singularity (shell-focusing singularity) at
$R(t,r)=0$, and a shell-crossing one at $R^{\prime}(t,r)=0$. Shell-crossing
singularities are gravitationally weak which there are proposals for extending
the spacetime through such singularities and show a breakdown of the
coordinate system being used, rather than a genuine physical singularity.
These singularities are generally not considered as being serious
counterexamples to the cosmic censorship conjecture Shell-crossing-coordinate
. Indeed it has been suggested one need not consider the shell-crossing
singularity because even if they occur in the dust case, they are likely to
not occur in the fluid case because of pressure effects. To show that the
shell-crossing singularity cannot be naked, suppose a shell-crossing
singularity first forms at $R_{0}(t_{sc})\neq 0$. To examine whether a point
is in the trapped surface or not, first, we must calculate the ratio
$\frac{R}{2M}$ to see that it is greater than one or not. If $\frac{R}{2M}<1$
this point will be trapped and no null or timelike geodesic can escape from it
to infinity. Consider a sphere with radius $R_{0}+\epsilon$ which the
$\epsilon$ can has any small positive value before the shell-crossing
singularity form $t<t_{sc}$. The ratio of $\frac{R}{2M}$ for this point gets

$\frac{R}{2M}=\lim_{R\rightarrow R_{0}}\frac{R}{8\pi\int_{0}^{R+\epsilon}\rho
R^{2}dR}.$ (11)

Here, we wrote the Misner-Sharp mass in the $(t,R)$ coordinate. After Taylor
expansion of the above ratio around the $R_{0}$ you’ll get,
$\frac{R}{2M}\sim\frac{1}{8\pi\rho R_{0}^{2}}<1$, which is clearly less than
$1$ as density goes to infinity at $R=R_{0}$. As a result, the trapped surface
will form just before the shell-crossing singularity formation and outside it.
Consequently, no naked singularity forms from these shell-crossing
singularities.

Note that in contrast to the dust collapse which always ends in a singularity
at the center Firouzjaee-2010 , the perfect fluid collapse can be stopped by
pressure at the center and the collapse ends to a compact star, not a black
hole Moradi:2015caa . On the apparent horizon, equation (8) gives $G=H$. The
apparent horizon slope in the $(t,r)$ plane is

$\frac{dt}{dr}|_{AH}=\frac{R^{\prime}-2M^{\prime}}{2\dot{M}-\dot{R}},$ (12)

The tangent vector on the apparent horizon can be presented as

$k^{\mu}=c(1,\frac{dr}{dt},0,0),$ (13)

where $c$ is a constant. To see whether this tangent vector is timelike or
spacelike, we have to calculate:

$k_{\mu}k^{\mu}=c^{2}(-e^{2\nu}+e^{2\psi}(\frac{dr}{dt})^{2}).$ (14)

Using the equation (12), we have

$k_{\mu}k^{\mu}=c^{2}e^{2\nu}(-1+\frac{e^{2\psi}}{e^{2\nu}}(\frac{2\dot{M}-\dot{R}}{R^{\prime}-2M^{\prime}})^{2}).$
(15)

Since on the apparent horizon, we have
$G=H\rightarrow\frac{e^{2\psi}}{e^{2\nu}}=\frac{R^{\prime 2}}{\dot{R}^{2}}$,
and applying the Einstein equation (II) we get
$2\frac{M^{\prime}}{R^{\prime}}=\rho R^{2}>0$ and
$2\frac{\dot{M}}{\dot{R}}=-pR^{2}>0$. Using these equations, the above
equation will be simplified as

$k_{\mu}k^{\mu}=c^{2}e^{2\nu}(\frac{(1+pR^{2})^{2}}{(1-\rho R^{2})^{2}}-1).$
(16)

If the weak energy conditions, $\rho\geq 0$ and $\rho+p>0$, are held in he
model, then we get that $k_{\mu}k^{\mu}>1$. Thus this condition does not allow
the apparent horizon to become future timelike. This argument does not alter
with changing the coordinate because the causal nature of spacetime does not
change by using another coordinate.

To have a more relativistic model, consider a black hole which is formed in
dark energy background with the cosmological constant. If we solve the
Einstein equation with the spherical symmetric metric and the perfect fluid
energy momentum tensor, we get

$\displaystyle\rho(t,r)=\frac{2M^{\prime}(t,r)}{R^{2}(t,r)R^{\prime}(t,r)}-\Lambda,$
$\displaystyle~{}~{}p(t,r)=-\frac{2\dot{M(t,r)}}{R^{2}(t,r)\dot{R}(t,r)}+\Lambda,$
(17)

where $M(t,r)$ quantity is defined in the following equation

$\displaystyle G(t,r)-H(t,r)=1-\frac{2M(t,r)}{R(t,r)}+\frac{\Lambda
R(t,r)^{2}}{3}.$ (18)

The $M(R)$ quantity is the Misner-Sharp Energy

$\displaystyle E_{MS}=\frac{1}{2}\int_{0}^{R}\rho R^{2}dR,$ (19)

or

$E_{MS}=\frac{1}{8\pi}\int_{0}^{r}\rho\sqrt{(1+(\frac{dR}{d\tau})^{2}-\frac{2M}{R}+\frac{\Lambda
R^{2}}{3})}d^{3}V.$ (20)

As discussed in Firouzjaee-penn , the cosmological constant contributes to the
Misner-Sharp Energy, but we can discriminate the matter-energy in a comoving
sphere with radius $r$ and time $t$ with $M(t,r)$. The apparent horizon
location will be changed by cosmological constant as $R=2M-\frac{\Lambda
R^{3}}{3}$. This equation shows that the positive cosmological constant will
reduce the horizon radius for a given matter mass, $M$. Nevertheless, using
the present cosmological constant value for the astrophysical black hole in
this equation leads to negligible correction for the astrophysical apparent
horizon radius.
In the presence of the cosmological constant, the apparent horizon slope in
the $(t,r)$ plane is

$\frac{dt}{dr}|_{AH}=\frac{R^{\prime}-2M^{\prime}+\Lambda
R^{2}R^{\prime}}{2\dot{M}-\dot{R}-\Lambda R^{2}\dot{R}},$ (21)

To see whether this tangent vector is timelike or spacelike in presence of
cosmological constant, we have to calculate the magnitude of the apparent
horizon tangent vector:

$k_{\mu}k^{\mu}=c^{2}(-e^{2\nu}+e^{2\psi}(\frac{dr}{dt})^{2}).$ (22)

Now, using the equation (21)

$k_{\mu}k^{\mu}=c^{2}e^{2\nu}(-1+\frac{e^{2\psi}}{e^{2\nu}}(\frac{2\dot{M}-\dot{R}-\dot{R}-\Lambda
R^{2}\dot{R}}{R^{\prime}-2M^{\prime}-\dot{R}+\Lambda R^{2}R^{\prime}})^{2}).$
(23)

On the apparent horizon, we have
$G=H\rightarrow\frac{e^{2\psi}}{e^{2\nu}}=\frac{R^{\prime 2}}{\dot{R}^{2}}$,
and using the Einstein equation (II) we get
$2\frac{M^{\prime}}{R^{\prime}}=\rho R^{2}+\Lambda R^{2}>0$ and
$2\frac{\dot{M}}{\dot{R}}=-pR^{2}+\Lambda R^{2}>0$. Applying these equations
to the magnitude of the apparent horizon tangent vector

$k_{\mu}k^{\mu}=c^{2}e^{2\nu}(\frac{(1+pR^{2})^{2}}{(1-\rho R^{2})^{2}}-1).$
(24)

It is clear from the weak energy condition, $\rho\geq 0$ and $\rho+p>0$, that
$k_{\mu}k^{\mu}>1$, thus this condition does not allow the apparent horizon to
become future timelike.

As shown in Figure (1), the only point which has a chance to be a naked
singularity is at the center of the black hole. Since the apparent horizon
cannot be future timelike, then no light can escape from this point of the
singularity, and therefore no naked singularity happens in the perfect fluid
collapse which has the central singularity $R=0$.

Figure 1:  Penrose diagram of the fluid collapse in the cosmological de Sitter
background.

In this part, we consider the Eardely and Smarr model LTB-naked  which is
claimed that contains the naked singularity. They consider a flat LTB model
whose mass function is in the form $M(r)=r^{3}$ and the singularity time for
each layer is $t_{0}(r)=\zeta r^{p}$. In Figure (2), the density profile of
collapse at the onset of singularity formation for the parameters, $p=2$ and
$\zeta=10$ is depicted. In Figure (3), a typical null geodesics after $t>0$
(before singularity formation) near the singularity are shown. These three
geodesics cross the apparent horizon and end to the singularity. This diagram
verifies some points:

  * •

The apparent horizon is spacelike.

  * •

Singularity is spacelike.

  * •

This behavior will be held even in the case that the null geodesic is close to
the crossing point i.e $R=M=0$.

  * •

Another reason for holding the CCC is that we see that a typical null geodesic
near the singularity point will fall into the trapped region (and then into a
singularity). Since null geodesic congruences do not cross each other then all
null geodesics originated from the center and after this null geodesics will
also fall into the trapped region and then it holds CCC.

These confirm that no naked singularity occurs in this model.

Figure 2:  Density profile of collapse at onset of singularity formation.
Figure 3: Null geodesic behaviour near singularity formation.

_Conclusion:_

The main issue of the present investigation was to examine whether the end
state of the collapse is a naked singularity or a black hole in the perfect
fluid collapse. Our results show that having the weak energy condition causes
the slope of the apparent horizon can not be future timelike. As a result, no
local naked singularity can occur in the spherical perfect fluid collapse.
This feature will not change if the black holes are located in the
cosmological constant background. This result will be held by dust (LTB model)
collapse which is a special case of the perfect fluid. As an illustrative
example, we took a known model in dust collapse and numerically depict its
horizon and singularity curve, and it was shown how a typical null geodesic
from the central singularity is trapped. The above result provides an
intriguing perspective on black hole physics which tells that not only do
energy conditions play an important role in forming the singularity but also
cover it by forming the apparent horizon. All these results lead to this point
that the strong cosmic censorship conjecture holds in the perfect fluid
spherical collapse.

## References

  * (1) S Hawking and R Penrose Proc Roy Soc A 314 529-548 (1970).
  * (2) R. Penrose, Riv. Nuovo Cimento 1, 252 (1969).
  * (3) R. M. Wald, doi:10.1007/978-94-017-0934-7_5 [arXiv:gr-qc/9710068 [gr-qc]].
  * (4) D. M. Eardley and L. Smarr, Phys. Rev. D 19, 2239 (1979).
  * (5) D. Christodoulou, Commun. Math. Phys. 93, 171 (1984).
  * (6) R. P. A. C. Newman, Class. Quantum Grav. 3, 527 (1986); B. Waugh and K. Lake, Phys. Rev. D 38, 1315 (1988); P. S. Joshi and I. H. Dwivedi, Phys. Rev. D 47, 5357 (1993); P. S. Joshi and I. H. Dwivedi, Class. Quant. Grav. 16, 41 (1999); P. S. Joshi, N. Dadhich and R. Maartens, Phys. Rev. D 65, 101501 (2002).
  * (7) Joshi, Pankaj S. Gravitational collapse and spacetime singularities. Cambridge University Press, 2007; P. S. Joshi and D. Malafarina, Int. J. Mod. Phys. D 20, 2641 (2011).
  * (8) R. Goswami and P. S. Joshi, Phys. Rev. D 69 (2004), 027502 doi:10.1103/PhysRevD.69.027502; P. S. Joshi, R. Goswami and N. Dadhich, [arXiv:gr-qc/0308012 [gr-qc]].
  * (9) Papapetrou A. and Hamoui A., (1967) Ann. Inst. H. Poincare A IV, 343; Hellaby, Charles, and Kayll Lake. ”Shell crossings and the Tolman model.” The Astrophysical Journal 290 (1985): 381-387; Clarke C. J. S. and Kr´olak A., (1985) J. Geom. Phys. 2, 127.
  * (10) J. T. Firouzjaee and R. Mansouri, Gen. Rel. Grav. 42, 2431 (2010).
  * (11) C. W. Misner and D. H. Sharp, Phys. Rev. 136, B571-B576 (1964) doi:10.1103/PhysRev.136.B571
  * (12) S. A. Hayward, Phys. Rev. D 53, 1938-1949 (1996) doi:10.1103/PhysRevD.53.1938
  * (13) J. T. Firouzjaee, Int. J. Mod. Phys. D 21, 1250039 (2012).
  * (14) R. Moradi, J. T. Firouzjaee and R. Mansouri, Class. Quant. Grav. 32, no. 21, 215001 (2015).