11institutetext: Vrije Universiteit Brussel, Brussels, Belgium
22institutetext: Université Libre de Bruxelles, Brussels, Belgium
33institutetext: Delft University of Technology, Delft, The Netherlands
44institutetext: Leiden Institute of Advanced Computer Science (LIACS),
Leiden, The Netherlands
44email<EMAIL_ADDRESS>
# Synthesising Reinforcement Learning Policies through Set-Valued Inductive
Rule Learning

Youri Coppens 1122 0000-0003-1124-0731
Denis Steckelmacher 11 0000-0003-1521-8494
Catholijn M. Jonker 3344 0000-0003-4780-7461
Ann Nowé 11 0000-0001-6346-4564

###### Abstract

Today’s advanced Reinforcement Learning algorithms produce black-box policies,
that are often difficult to interpret and trust for a person. We introduce a
policy distilling algorithm, building on the CN2 rule mining algorithm, that
distills the policy into a rule-based decision system. At the core of our
approach is the fact that an RL process does not just learn a policy, a
mapping from states to actions, but also produces extra meta-information, such
as action values indicating the quality of alternative actions. This meta-
information can indicate whether more than one action is near-optimal for a
certain state. We extend CN2 to make it able to leverage knowledge about
equally-good actions to distill the policy into fewer rules, increasing its
interpretability by a person. Then, to ensure that the rules explain a valid,
non-degenerate policy, we introduce a refinement algorithm that fine-tunes the
rules to obtain good performance when executed in the environment. We
demonstrate the applicability of our algorithm on the Mario AI benchmark, a
complex task that requires modern reinforcement learning algorithms including
neural networks. The explanations we produce capture the learned policy in
only a few rules, that allow a person to understand what the black-box agent
learned. Source code: https://gitlab.ai.vub.ac.be/yocoppen/svcn2.

###### Keywords:

Reinforcement Learning Inductive Rule Learning Explainable AI Policy
Distillation.

##  1 Introduction

Reinforcement Learning (RL) is a machine learning technique that learns from
experience, rather than data that has been collected offline. By interacting
with its environment, an intelligent agent learns to adapt its actions to show
the desired behaviour. This environment is in general stochastic and its
dynamics unknown to the agent.

RL is particularly useful in settings without a model of the environment, or
when the environment is hard to model. RL is also interesting when an accurate
simulation model can be built, but solving that model to obtain a decision or
control strategy is hard. Telecommunication applications such as routing, load
balancing and call admission control are examples where RL has proven its
worth as a machine learning technique. RL has unfortunately one crucial
weakness which may prevent it from being applied in future critical
applications: the control policies learned by the agent, especially using
modern algorithms including neural networks, are black boxes that are hard to
interpret. This shortcoming could prove a major obstacle to the adoption of
RL. For example, the new EU privacy regulation, more specifically Article 22
of the General Data Protection Regulation (GDPR), requires that all AI with an
impact on human lives needs to be accountable, which, for an RL agent, means
that it must be possible to tell the user why the agent took a particular
action in a state. Interpretability of machine learning based models is
therefore crucial, as it forms the basis for other concerns such as
accountability, that subsumes the justification of the decisions taken. Assume
that RL is used to synthesise a prevention strategy for a dangerous infection
disease. The agent should learn what regulations and procedures people should
follow, while optimally allocating a budget, such as vaccines, medication or
closed-school days. While this scenario is plausible, see [14], will a black-
box strategy be accepted? The above observation brings us to the following key
question: Wouldn’t it make more sense to use RL to explore possible strategies
and simulate the potential effects on our society, before we would decide to
implement such a strategy? However, this requires that the strategies found by
RL algorithms are presented in a human-understandable format.

Explainable AI, reviewed in [17], is not new. Even in the early days of Expert
Systems, many approaches have been proposed to allow to automatically generate
explanations and to reflect on the reasoning being used, see e.g. [16].
Recently, explainability has been put more prominently on the agenda in the
Machine Learning (ML) community as well [19]. The massive success of the
black-box ML-techniques has culminated in many applications that display
powerful results, but provide poor insight into how they find their
conclusion. Over the last three years, workshops at major conferences in
Machine Learning (e.g. ICML, NeurIPS), Artificial Intelligence (e.g. IJCAI),
Human-Computer Interaction (e.g. ACM CHI, ACM IUI) and Planning (e.g. ICAPS)
have been held for contributions that increase human interpretability of
intelligent systems. At these workshops, initial research related to data
mining has been proposed. However, ideas to make RL more explainable are
rather limited and often borrowed from existing ML Interpretability literature
[2].

In this paper, we present a method that allows the policy learned by a
reinforcement learning agent, using modern deep learning methods, to be
expressed in a way that is understandable by a person. Our contributions are
as follows:

  1. 1.

We extend the CN2 rule-mining algorithm to make it able to leverage the meta-
information of an RL process on equally-good actions. This allows the
algorithm to map an RL policy to a simpler and compacter set of rules.

  2. 2.

We use our extended CN2 algorithm to produce human-understandable rules from a
black-box RL policy, and evaluate our method on the Mario benchmark.

  3. 3.

We introduce a 2-phased method, that produces a first set of rules providing a
global view on the policy. Then refines the rules when needed. This refinement
is driven by the performance of the rules when used to perform the task,
rather than the accuracy when mining the rules.

##  2 Related Work

Until now, insights in the learnt RL policy were typically limited to showing
a typical trace, i.e. an execution of the policy or through analysing and
visualising agent reward structures [1]. Relevant efforts to mention are [21],
where the authors visualise action versus parameter exploration and [3] that
analyses and visualises the usefulness of heuristic information in a state
space.

Distillation techniques translate the behaviour from large complex models to
simpler surrogate models [10]. While these methods reduce computational
overhead and model complexity, the general focus is mostly set on maintaining
performance rather than increasing the interpretability through the surrogate
model, even in the context of RL policies [20]. In [5], a deep RL policy is
distilled into a surrogate soft decision tree model [7], in an effort to gain
insights in the RL policy. The surrogate model determines a hierarchy of
filters that perform sequential feature extraction from observations. These
filters highlight which features are considered important in the decision
making. Our work is complementary in the sense that the rules we extract look
at the content, i.e. the actual value of the observation features, rather than
the importance of a particular feature.

Relational RL (RRL) could be seen as an interpretable RL approach _by design_
[25] as it allows to express the policy as well as the quality of the policy
using relational expressions. Relational RL requires domain-specific knowledge
from the designer, as a set of concepts have to be defined a priori. If these
concepts are not carefully defined, the expressiveness of the agent is
reduced, which may prevent it from learning any good policy. In [27],
relational inductive biases have been incorporated into a deep learning
process, in an attempt to scale RRL without pre-defined concepts, thus
allowing for relevant concepts to emerge. The approach is shown to reveal the
high-level plan of the RL strategy, thus identifying subgoals and options that
can be generalised. Our work does not consider subgoals and instead focuses on
making the actual policy, the mapping from states to actions, transparent to
the user.

The paper by Madumal et al. [15] introduces an approach to explain a policy
which can either be obtained through model-based RL or planning. It builds
upon causal graphs as a means to calculate counterfactuals. As the causal
graph is assumed to be given, the symbolic concepts are also defined _a
priori_ , and have to be carefully chosen, leading to the same vulnerability
as Relational RL.

In our approach, we don’t want to restrict the policy _a priori_. We allow the
RL process to learn complex policies if needed. Recent advances in (deep)
function approximation for RL have yielded powerful techniques that learn rich
state representations and have advanced state-of-the-art in terms of learning
performance in complex environments, see e.g. [18]. However, these techniques
obscure a proper understanding of the policy that is being learned.

In the approach we propose, we start from existing state-of-the-art RL
techniques and make them more interpretable. Our work performs an _a
posteriori_ simplification and communication of the strategy. This is realised
through a rule-mining approach that exploits the meta-information that is
present in the RL learning algorithm. While decision trees are a more commonly
adopted interpretable surrogate model that can be visualised in a natural
manner, we opted for decision rules, because these have a similar
expressiveness while also being more compact and robust. In addition, the use
of meta-information, such as which actions are equally good in a state, is
more naturally implemented in a rule-mining system, as well as the refinement
process we propose. The RL process not only provides us with the best action
for a given state, but the probability that an RL actor gives to each of the
actions is providing us information on the quality of the sub-optimal actions.
Exploiting this meta-information allows to go beyond a mere translation of the
policy into a set of rules. As, for instance, knowing that two actions have
nearly the same probability in a given state indicates that they are both
equally good, which allows our rule mining algorithm to chose one action or
the other, depending on what leads to the simplest rule set. This exploitation
of the meta-information of an RL process also differentiates our work from
approaches that translate the RL policy in a set of fuzzy rules, such as [11]
and [9].

##  3 Background

Our work covers several domains of research such as Reinforcement Learning and
Inductive Rule Learning.

###  3.1 Policies learnt through Reinforcement Learning

Reinforcement Learning (RL) tackles sequential decision making problems in an
interactive setting. [23] An artificial agent learns by operating in an
unknown environment, from scratch or boosted by some initial domain knowledge,
which actions are optimal over time. Based on the observation of the state of
the environment, the agent executes an action of choice which makes the agent
transition to a next state. In addition to the new state, the agent will also
receive feedback from the environment in the form of a numerical reward. Based
on that feedback, the agent will adapt its behaviour into a desired strategy.

Reinforcement learning problems are formally represented as a Markov Decision
Process (MDP), a 5-tuple $(S,A,T,R,\gamma)$, with $S$ and $A$ the sets of
states and actions available in the environment. Unknown to the agent are $T$
and $R$. $T:S\times A\times S\rightarrow[0,1]$ represents the transition
probability function, assigning probabilities to a possible transition from
state $s\in S$ to another state $s^{\prime}\in S$, by executing action $a\in
A$. $R:S\times A\rightarrow\mathbb{R}$ is the environment’s reward function,
providing the agent with feedback and finally a discount factor
$\gamma\in[0,1]$ regulates the agent’s preference for immediate rewards over
long-term rewards. The agent’s goal is to maximise the expected return
$\mathbb{E}[G_{t}]$, where $G_{t}=\sum_{i=0}^{\infty}\gamma^{i}R_{t+i}$, the
sum of future discounted rewards. Lower values of $\gamma$ set the preference
on immediate reward, whilst higher values result in a more balanced weighing
of current and future reward in $G_{t}$. The agent learns a policy
$\pi:S\times A\rightarrow[0,1]$, mapping state-action pairs to probabilities,
describing the optimal behaviour strategy.

In value-based RL methods, this policy is implicitly present through the use
of a _value function_ in combination with an action-selection strategy. The
value function $Q(s,a)$ is often represented as the expected return when
taking an action $a$ in state $s$: $Q(s,a)=\mathbb{E}[G_{t}|S_{t}=s,A_{t}=a]$,
hence describing which actions are expected to result in more future rewards.

When the Q-values have converged, the highest ranked action is the optimal
one, whilst others are granted to be sub-optimal. However, when the Q-values
are equal or almost equal, this indicates that there is no clear preference
over these actions.

Policy-based algorithms are another family of RL methods. They maintain an
explicit policy, or actor, often described as a parametric expression
$\pi_{\theta}$. The policy is optimised iteratively, using methods such as
computing and following the gradient of its quality [24], or training it to
progressively imitate the greedy policy of a collection of critics [22].
Policy-based methods typically learn a stochastic policy where the difference
in probability can be considered to be a proxy for the difference in quality
of two alternative actions in a state. The BDPI algorithm [22] that we use in
this paper, for instance, learns a policy that associates nearly-equal
probability to actions that are of comparable quality, while actions that are
clearly worse than the other ones have a probability near 0. In other words,
when the probability of two actions in a given state is close to each other,
this expresses there is no clear preference of one action over the other
action. We exploit this property in our framework in order to find a small and
simple set of rules capturing the main elements of the policy.

###  3.2 Inductive Rule Learning

Inductive rule learning allows us to extract knowledge in a symbolic
representation from a set of instances in a supervised manner. The goal of
supervised rule induction is the discovery of rules that reflect relevant
dependencies between classes and attribute values describing the given data
[8]. In our setting, these data instances are state-action pairs drawn from a
learned RL policy. More precisely, the instances are pairs of states with
_set-valued_ actions: for a single instance, a state might be paired with
multiple actions.

The set-valued aspect is important to express that two or more actions are
high-quality alternatives in a certain state. By supporting the possibility to
express this indifference towards which action of that set is selected in the
corresponding state, we can create rules with a larger support and as a
consequence fewer rules. Regular inductive rule learning is not designed to
deal with this set-valued aspect, and therefore we extend this learning
paradigm in Section 4. Before we introduce our set-valued rule mining, we
first summarise the main elements of inductive rule learning and the CN2
algorithm.

Inductive rule learning allows to summarise a set of instances, which might be
discrete valued or continuous valued, along with a discrete target class, into
a set of rules. The rules are of the form ‘IF $Conditions$ THEN $c$’, where
$c$ is the class label. Because of their symbolic nature, the resulting model
is easy to interpret by humans.

We base our work on CN2 [4], a frequently used rule learning algorithm based
on the separate-and-conquer principle, and extend it to support multiple
equally-good labels per input. In CN2, rules are iteratively identified
through the application of beam search, a heuristic tree search algorithm. The
best rule that is identified in this search procedure will be selected as new
rule. The consequent (classifying label) of the rule is the majority class of
the examples covered by the rule.

The commonly used evaluation heuristic in CN2 is _weighted relative accuracy_
(WRA) [13, 26], which aims to find a good balance between a large coverage and
a high accuracy. Equation 1 describes the WRA of a proposed rule $r$:

$\text{WRA}(r)=\frac{\hat{P}(r)+\hat{N}(r)}{P(r)+N(r)}\times\left(\frac{\hat{P}(r)}{\hat{P}(r)+\hat{N}(r)}-\frac{P(r)}{P(r)+N(r)}\right)$
(1)

where $P(r)$ and $N(r)$ represent the number of positive and negative examples
in the current training set and $\hat{P}(r)$ and $\hat{N}(r)$ represent the
number of positive and negative examples covered by the considered rule $r$.
From hereon, we will drop the parameter $r$ as no confusion is to be expected.
Examples are said to be positive when they have the same classification label
as the consequent of the rule, otherwise they are considered to be negative.
$\hat{P}$ and $\hat{N}$ can be thus considered the true and false positive
examples covered by the rule respectively. The first factor in Equation 1,
$\frac{\hat{P}+\hat{N}}{P+N}$, represents the _coverage_ of the proposed rule,
the rate of examples that satisfy the antecedent, and weighs the second factor
of the equation. That factor,
$\left(\frac{\hat{P}}{\hat{P}+\hat{N}}-\frac{P}{P+N}\right)$, represents the
relative classification accuracy of a rule. It is the difference between the
accuracy of the samples covered by the rule and the overall accuracy of the
training examples.

The algorithm stops generating rules if there are no other significant rules
to be found or an insufficient number of samples remain. These thresholds are
expected to be application dependent and are therefore to be determined by the
user.

We now introduce the main contribution of this paper, a rule mining algorithm
based on CN2 that extends two aspects of it relevant for reinforcement
learning applications: allowing states to be mapped to _several_ equally-good
actions, and ensuring that the distilled policy performs comparably to the
original policy.

##  4 Extending CN2 to Set-Valued Labels

Our main contribution consists of extending two important aspects of CN2, to
make it suitable to policy distillation in an explainable reinforcement
learning setting:

  1. 1.

We exploit meta-information generated by the reinforcement learning process to
associate sets of equally-good actions to each state. By offering choice to
the rule miner in which action to map to every state, we potentially (and in
practice do) reduce the number of rules needed to cover the original policy.

  2. 2.

Contrary to classical Supervised Learning settings, the main application
domain of CN2, what matters in our setting is not primarily the classification
accuracy, but the loss in performance of the distilled policy. In other words,
one little misclassification in a particular state can have a big impact on
the performance of the policy, but in other states, this misclassification
might almost be unnoticeable from a policy performance perspective. We propose
a two-step approach that allows to identify those important
misclassifications, leading to short rule sets that, when executed in the
environment, still perform as well as the original policy. This ensures that
the explanations we provide with the mined rules correspond to a good policy,
and not some over-simplified degenerate policy.

The first point, associating sets of equally-good actions to each state,
requires an extension of the CN2 algorithm itself. The second point, and the
application of the whole framework to reinforcement learning policy
distillation, requires an iterative framework that we present in Section 5.

The CN2 algorithm iteratively learns a rule through a beam search procedure.
The objective of this step is to search for a rule that on the one hand covers
a large number of instances, and on the other hand has a high accuracy. This
is steered through the evaluation heuristic, i.e. the weighted relative
accuracy (WRA), which makes a distinction between positive and negative
examples based on the label of the training sample. In order to deal with the
Set-Valued instances, which express an indifference with respect to the actual
label chosen by the rule, we adapted the WRA heuristic.

In the regular single-label setting, the sum of positive and negative examples
covered by the rule and in the overall training set, $\hat{P}+\hat{N}$ and
$P+N$, always corresponds to the number of respective training examples,
$\hat{E}$ and $E$. However, in a multi-label setting this is not necessarily
the case, as an instance can be assigned to any of the labels and is thus
potentially both a positive and negative example at the same time according to
the standard WRA heuristic. The heuristic aims at maximising the correctly
classified instances for a rule by assigning the majority class of the covered
samples as the class label. In order to deal with the indifference to which
instances with multiple class labels are assigned, we propose the following
adaptation to the WRA heuristic, described in Equation 2, where we replace the
sum of positive and negative examples covered by the rule and in the overall
set by the actual number of distinct samples in the set, respectively denoted
by $E$ and $\hat{E}$:

$\text{WRA}_{set}=\frac{\hat{E}}{E}\times\left(\frac{\hat{P}}{\hat{E}}-\frac{P}{E}\right)$
(2)

This will have the effect that the heuristic will only count the multi-
labelled instances as a positive example when the majority class is one of the
options, and as a negative example otherwise.

###  4.1 Impact of Set-Valued Labels on Policy Distillation

(a) In each cell, the most optimal action(s) to perform are shown by arrows.
Notice how some cells in the grid contain two optimal actions leading to the
top rightmost goal state. The cross in the middle indicates the muddy states

(b) Distillation of the muddy gridworld policy with standard CN2: Samples
provided to CN2 are state-action pairs. When multiple optimal actions exist
for a state, an action is selected at random.

Figure 1: Muddy gridworld environment with its optimal policy, and a
distillation of this policy using standard CN2 (no support for equally-good
actions).

  1. 1.

IF X<=18 THEN Class=RIGHT

  2. 2.

IF X=19 THEN Class=UP

(a) An initial distillation of the muddy gridworld policy using our Set-Valued
extension of CN2, taking multiple possible actions into account. The rules are
simple, but do not yet consider the policy details in the muddy states into
account, leading to sub-optimal performance.

  * 1.1

IF X<=18 AND Y=10 AND X>=8 AND X<=9 THEN Class=UP

  * 1.2

IF X<=18 AND X>=10 THEN Class=RIGHT

  * 1.3

IF X<=18 AND X<=8 THEN Class=RIGHT

  * 1.4

IF X<=18 AND Y=11 THEN Class=UP

  * 1.5

IF X<=18 AND Y=9 THEN Class=DOWN

  * 1.6

IF X<=18 THEN Class=RIGHT

  * 2

IF X=19 THEN Class=UP

(b) Refining the rules shown in (a), as detailed in Section 5, identifies the
state near the mud that requires a different action from the surrounding
states, leading to optimal behavior.

Figure 2: Distilled policy with our modified CN2 algorithm, without and with
our refinement step.

Before we discuss our approach on the Mario AI benchmark, we illustrate the
effect of our set-valued rule mining heuristic on a simple 20 by 20 discrete-
state gridworld problem, where a navigating agent has to walk to the top
rightmost cell from any initial position.111Another illustration can be found
on pages 8 and 9 of
https://www.ida.liu.se/~frehe08/tailor2020/TAILOR˙2020˙paper˙48.pdf Actions
are up, down, right and left. Near the center of the grid, a couple of cells
contain mud. The agent receives a reward of -1 per time-step, -10 when
entering a muddy cell.

Figure 1(a) shows the environment, and the optimal policy learned by a
Q-learning agent in this setting. In most cells, two optimal actions exist to
navigate to the goal, namely moving upward or to the right. Meaning there are
quite a lot of optimal policies in this example. In the cells neighboring the
muddy cells however, the agent has learned not to walk towards the muddy
parts, but rather walk around it. In essence, the general behavior of the
agent can be summarized as moving upward or to the right. Hence an acceptable
simplified policy would be to first walk to either the upper or right border
and then walk straight to the goal. Such a simplified policy would be a
desired outcome of the rule mining process, as it provides a first high level
view on the policy. This high level policy can be refined, see Section 5
below. As explained in that section, this refinement is not driven by
increasing the accuracy requirements of the rule-mining algorithm, but rather
by improving the performance of the distilled policy.

Figure 1(b) shows the resulting policy derived from the standard single-
labeled CN2 algorithm, using the regular WRA heuristic. As the algorithm
expects single labeled data, whenever multiple optimal actions exist for a
state, one of these had to be sampled randomly. The resulting rule list covers
several sectors in the grid where the agent either moves upward or to the
right, eventually ending up in the goal. As standard CN2 cannot take
alternative actions into account during the mining process, it cannot distil a
simple policy, meaning a policy expressed with few rules, yet having a good
performance.

Figure 2(a) presents the resulting simplified policy of our Set-Valued
extension of CN2. Our algorithm can nicely summarize the essence of the policy
into two simple rules, making the agent walk to the right border of the grid
first, and afterwards walk upwards to the goal. Note that the rule mining
process did not capture the avoidance of the muddy cells which is an
exceptional region in the environment. However, Figure 2(b) shows that the
refinement step that we propose in Section 5 allows extra rules to be
identified, restoring optimal performance of the agent around the muddy
region.

Our extension of CN2, able to learn from pairs of states and multiple equally-
good labels, is only one component of our explainable reinforcement learning
framework. We now fully detail the iterative algorithm that we built on our
CN2 extension, that allows a policy to be learned, distilled, evaluated and
fine-tuned by the user, to ensure that state-action pairs that are crucial for
the performance of the policy are classified correctly.

##  5 Explaining Policies through Distillation

In this work, we are using inductive rule mining to translate a deep
reinforcement learning policy into a simplified ordered list of rules that
reveal the agent’s behaviour in the environment. The simplification is
performed through a rule-mining approach that exploits the meta-information
that is present in the RL learning algorithm, hereby balancing between
accuracy and readability.

We extended the implementation of CN2 in the open source data mining framework
Orange222https://github.com/biolab/orange3, in order to support multi-label
data and incorporated our proposed $\text{WRA}_{set}$ heuristic to perform
set-valued rule mining. We evaluated our methods on an altered level of the
Mario AI benchmark [12]. The goal of the Mario agent is to collect rewarding
red coins and avoiding penalising green and blue coins. The action space of
the Mario agent is restricted so that it can only press one of the 5 available
controller buttons during each step. The agent can also decide not to press
any controller button, bringing the total number of actions to 6.

The observations provided to the RL agent are built from a description of the
enemies, coins and blocks around the current position of Mario in the
environment. We consider a 10 by 10 receptive field, that is, we look at 10 by
10 unit squares in the environment, centred around Mario’s current position.
Each square can either be empty, or contain a block, enemy or coin. Because
every class of enemy, or every colour of a coin, is uniquely identified, each
square can take 24 different values (0 being empty, then positive values
identifying the kind of object in the square). When producing observations for
the agent, we consider this 10 by 10 set of squares that can each take a value
among 24, and produced a flattened, one-hot encoded, large observation vector
of $10\times 10\times 24=2400$ entries, with each entry either 0 or 1. Figure
3 illustrates the transformation from a game frame to a receptive field.
Because one-hot encoded states can be decoded back to their integer variant (a
10 by 10 grid of integers in our case), and the integers we consider uniquely
identify objects in the Mario game that have human-friendly game, any
observation passed to the agent can also be described in a human-friendly way,
such as “there is a red coin above Mario”.

$\Rightarrow$ $\begin{bmatrix}0&0&0&0&0&0&9&9&0&0\\\ 0&0&0&0&0&0&0&0&0&0\\\
0&0&0&0&0&0&0&0&0&0\\\ 0&0&0&0&0&0&4&4&4&4\\\ 0&0&0&0&0&0&0&0&0&0\\\
0&0&0&0&0&16&0&0&0&0\\\ 0&0&0&0&0&0&9&0&9&9\\\ 0&0&0&23&0&11&11&11&11&11\\\
11&11&11&11&11&11&11&11&11&11\\\ 0&0&0&0&0&0&0&0&0&0\\\ \end{bmatrix}$ Figure
3: Transformation from a MarioAI game frame to a 10 by 10 receptive field grid
centred around Mario, which is returned by the emulator. Each cell of the grid
corresponds to a pixel region about the size of a brick block in the game. The
game objects in each of the grid’s cells are translated into numerical labels
as state representation for the learning agent. Whenever the receptive field
ranges out of the game frame, it is padded with zeros (e.g. the bottom row).

As explained above, we propose a two-phased distillation algorithm, to produce
meaningful rules from a black-box policy learned with Deep Reinforcement
Learning. The first phase produces a list of rules that approximate how the
Deep RL policy maps states to actions. These rules capture the general and
high-level behaviour of the policy, and produce compact and human-friendly
descriptions. However, some of the approximations made by our rule mining
algorithm may lead to rules that, if executed in the environment, lead to a
significant loss in performance. Because small mistakes in a single state may
or may not have an impact on the performance of the agent, increasing the
target accuracy of the rule-mining algorithm does not address the problem.
Instead, we introduce a second phase, that evaluates the rules in the
environment, and corrects small mistakes when they have an impact on the
performance of the agent.

###  5.1 Phase 1: Supervised post-processing

Our first phase distills a Deep Reinforcement Learning policy into rules,
using the rule-mining algorithm that we introduce in Section 4:

  1. 1.

We train a Deep RL agent, that we assume produces a stochastic policy.
Q-Values can be mapped to a stochastic policy with Boltzmann exploration;

  2. 2.

We record multiple trajectories sampled from the policy;

  3. 3.

We translate the trajectories into a collection of symbolic states and set-
valued actions, so that each state can be mapped to several actions if the
Deep RL policy gives similar probability to these actions;

  4. 4.

Our rule-mining algorithm learns rules that cover the state-action pairs
identified above.

First, a deep RL agent is trained to control Mario to play a single level. In
this paper we realise this through the use of Bootstrapped Dual Policy
Iteration (BDPI) [22]. We chose this algorithm because it produces an explicit
actor, from which we can obtain action probabilities, and the off-policy
actor-critic nature of the algorithm leads to an actor that computes the
probability that an action is optimal, instead of a simple number that
empirically led to high returns, as Policy Gradient approaches do.

Once a policy has been learnt, a data set is recorded by executing the
converged RL policy in the environment for 50 episodes. The data set consists
of tuples $(s,\pi(s))$ of state observations and action probabilities for each
step of the performed episodes.333Since BDPI is an actor-critic algorithm, we
use the actor predictions as output, i.e. probabilities over actions. With
value-based algorithms, e.g. DQN [18], one could record the Q-values instead.

Given the data set of states and their corresponding action probabilities, a
translation is applied in order to prepare for mining the rules. The numerical
values of the state observations are transformed back to their symbolic
counterpart, as each value in the observation represents a specific game
object in Mario’s receptive field. Since the state observation represents a
receptive field around Mario, we can assign x,y coordinates to each cell
(feature of the state) with the agent in the center.

For each state observation $s$ in the data set, the predicted action
distribution $\pi(s)$ is transformed into a set, $\mathcal{L}(s)$. This set
denotes which actions are considered suitable to be executed in that
particular state. Equation 3 describes this transformation formally. For each
state $s$ in the recordings, we take the probability of the best action, i.e.
the action with the highest probability,
$\max_{a^{\prime}}(\pi(s,a^{\prime}))$, and set a proportional threshold,
$\tau*\max_{a^{\prime}}\pi(s,a^{\prime})$, where $\tau\in(0,1]$ determines how
tolerant we are with respect to sub-optimal actions. Lower values of $\tau$
will lead to more actions exceeding the threshold, while high values will put
the threshold closer to $\max_{a^{\prime}}(\pi(s,a^{\prime}))$. For our
experiments we picked a value for $\tau=90\%$.

$\forall s\in S:\mathcal{L}(s)=\\{a\in A(s)\
:\pi(s,a)\geq\tau*\max_{a^{\prime}}(\pi(s,a^{\prime}))~{}|~{}\forall
a^{\prime}\in A(s)\\}$ (3)

Once the symbolic data set is created, we can proceed mining rules with our
Set-Valued CN2 algorithm. We explored different settings; providing the rule
mining the full state space of the RL as feature space or reducing the feature
space to the upper right quadrant of Mario’s receptive field, that is the
cells in front of Mario. We also performed rule mining including and excluding
the inequality ($!=$) predicate. The reduced feature set, with the excluded
inequality predicate provides more interpretable rules, and are listed below.
The default parameters of the beam search, expressing the maximal number of
conditions per rule, the minimal number of samples to be covered by a rule and
the beam width, were put respectively to 5, 20 and 10. Learned rules by Set-
Valued CN2 using our $\text{WRA}_{set}$ heuristic, that explain the policy our
BDPI agent has learned in the Mario environment, are listed below. Note that
the rules need to be interpreted sequentially.

  1. 1.

IF (1, 5)=NULL AND (0, 5)=NULL AND (0, 1)=NULL THEN Class=JUMP

  2. 2.

IF (1, 5)=BRICK AND (1, 0)=COIN_RED THEN Class=RIGHT

  3. 3.

IF (1, 5)=BRICK AND (2, 4)=COIN_RED THEN Class=RIGHT

  4. 4.

IF (4, 2)=NULL AND (0, 4)=NULL THEN Class=JUMP

  5. 5.

IF (4, 5)=NULL AND (3, 3)=NULL THEN Class=JUMP

  6. 6.

IF (3, 5)=BRICK THEN Class=JUMP

  7. 7.

IF (2, 2)=BRICK THEN Class=JUMP

  8. 8.

IF (0, 2)=NULL AND (0, 1)=NULL THEN Class=RIGHT

  9. 9.

IF (0, 5)=NULL THEN Class=JUMP

  10. 10.

IF TRUE THEN Class=JUMP

The extraction resulted in a list of 10 rules, capturing the agent’s behaviour
at a high level. In overall, the conditions of the rules focus on empty spaces
(NULL) and the presence of bricks or red coins. The second rule, for instance,
explains that the policy likes to move to the right when there is a (positive
rewarding) red coin in front of Mario. However, a problem with the learned
rules is that they over-emphasise jumping, the action that is most often the
best one, but that is not enough to finish a Mario level (Mario also has to
move right to win). While the rules provide a high-level explanation of the
policy followed by the agent, they do not allow the agent to obtain high
returns when executed. Our second phase, the refinement, addresses this issue
by allowing the rule mining algorithm to produce more refined rules, that lead
to better execution performance.

###  5.2 Phase 2: Refinement

The rules produced by the first phase, described above, are optimised for
classification accuracy. However, accuracy is not the main objective in policy
distillation, as even high accuracy may still lead to poor performance in the
environment. We therefore introduce a second phase, that refines the rules of
the first phase based on roll-outs in the reinforcement learning environment.
The second phase ensures that good policies are produced by our algorithm:

  1. 1.

Use the Deep RL policy to perform an episode in the environment, observe the
predicted actions step-wise;

  2. 2.

In each time-step, predict the action that would be chosen by the obtained
rules from phase 1 and note which rule was applied for the prediction;

  3. 3.

When the predicted action of the Deep RL policy and the rules are different,
store the observed state together with the action of the Deep RL policy in the
training set of the respective rule;

  4. 4.

Once the episode is finished, we extend each training set with samples from
phase 1 that were correctly classified in order to get a balanced data set;

  5. 5.

Launch a new rule mining process for each balanced training set using Set-
Valued CN2 with the respective rule of the first phase as a starting point for
the beam search process.

The procedure described above leads to each rule produced during phase 1 to
have additional datapoints belonging to them, created when the rules and the
black-box policy disagree in some state. By re-running our rule-mining
algorithm on these rules, with extra datapoints, overly-general rules that
lead to poor performance will be split into additional rules that better
capture important actions in important states. If we were to take for example
the fourth rule from the resulting rule list in the first phase, one of the
resulting refined rules would be:

IF (4, 2)=NULL AND (0, 4)=NULL AND (2, 5)!=BRICK AND (0, 3)=NULL AND (1,
2)!=COIN_RED AND (1, 2)!=COIN_BLUE AND (1, 2)!=BORDER_CANNOT_PASS_THROUGH THEN
Class=RIGHT

This additional rule ensures that the agent sometimes moves right, allowing
the level to be finished. Because we keep track of which phase-1 rules are
split into phase-2 rules, we are able to show the user a hierarchical view of
the rules, with simple and general phase-1 rules that capture the overall
policy, and some rules having children that identify refinements required for
top performance.

Performing this refinement procedure in our example of Mario with the rules
obtained from the first phase (listed in Subsection 5.1), caused a refinement
of multiple rules. We executed the refined rule list in the environment for 50
episodes and compared its performance with the performance of the original
BDPI policy. Figure 4 shows that the refined rules are capable to reach a
performance similar to BDPI, which demonstrates that our algorithms are able
to produce intuitive explanations of black-box reinforcement learning
policies, while ensuring that the explanations correspond to high-performance
policies.

Figure 4: Box plot comparison of episodic cumulative reward (y-axis) achieved
by the refined extracted rules of our Set-Valued CN2 on the Mario environment
with the original BDPI agent. 50 episodes were played by both agents (x-axis).
The refined rules we obtain after phase 2 have an equivalent performance to
the original policy since the box plots highly overlap. A bad-performing agent
obtains a score 100 on this environment.

##  6 Conclusion

In this paper we have proposed an approach to translate a policy trained by a
Deep RL algorithm into a set of rules. The set is human-readable, captures the
behaviour of the policy, and can be iteratively refined to obtain higher
returns if executed in the environment. A key element of our approach is that
we associate every state to a _set_ of equally-good actions, obtained by
looking at the action probabilities predicted by the RL policy. This part of
our contributions allows our modified rule mining algorithm to simplify its
rule set. By allowing to select one of the almost equally-good actions for a
state, the number of generated rules is significantly reduced compared to
applying CN2 on the greedy policy. We find the iterative refinements appealing
with respect to explainability. The rules of phase one are not altered, but
only hierarchically expanded. Thus, the rules of the first phase provide a
high-level view and every refinement adds details. Our refinement process is
driven by the performance of the rules, measured by their execution in the
environment. This prevents us from being dependent on magic numbers (i.e. the
setting of the different parameters) of the rule mining algorithm.
Furthermore, our method only assumes that the Deep RL algorithm produces a
probability distribution over the actions, which can also be retrieved from
value-based algorithms by using e.g. a Softmax function over Q-values.
Alternatively, Equation 3 can also be directly applied to Q-values. Finally,
in our experiments, the states are semantically meaningful for users. The
readability of the rules could however be further improved by using rule
simplification or _syntactic sugar_. For example, in the Mario experiment,
position (0, 4) could be translated into _right of Mario_. In settings where
the features are not semantically meaningful, a double representation could be
used as proposed in [6], where states represented in the feature space of the
RL agent are grounded in a semantically meaningful representation for users.
In future work we will also explore other rule-mining algorithms, as the use
of CN2 is not crucial to our approach. What is however important is that the
algorithm can handle Set-Valued labels. Therefore, adaptations of bi-
clustering or Markov Random fields are appealing for exploiting meta-
information of an RL process.

#### Acknowledgements.

This work is supported by the Research Foundation Flanders (FWO) [grant
numbers G062819N and 1129319N], the AI Research Program from the Flemish
Government (Belgium) and the Francqui Foundation. This work is part of the
research program Hybrid Intelligence with project number 024.004.022, which is
(partly) financed by the Dutch Ministry of Education, Culture and Science
(OCW).

## References

  * [1] Agogino, A.K., Tumer, K.: Analyzing and visualizing multiagent rewards in dynamic and stochastic domains. Autonomous Agents and Multi-Agent Systems 17(2), 320–338 (2008). https://doi.org/10.1007/s10458-008-9046-9
  * [2] Alharin, A., Doan, T.N., Sartipi, M.: Reinforcement Learning Interpretation Methods: A Survey. IEEE Access 8, 171058–171077 (2020). https://doi.org/10.1109/ACCESS.2020.3023394
  * [3] Brys, T., Nowé, A., Kudenko, D., Taylor, M.E.: Combining multiple correlated reward and shaping signals by measuring confidence. In: Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence. p. 1687–1693. AAAI Press, Palo Alto, California (2014)
  * [4] Clark, P., Niblett, T.: The CN2 induction algorithm. Machine Learning 3(4), 261–283 (1989). https://doi.org/10.1007/BF00116835
  * [5] Coppens, Y., Efthymiadis, K., Lenaerts, T., Nowé, A.: Distilling Deep Reinforcement Learning Policies in Soft Decision Trees. In: Miller, T., Weber, R., Magazzeni, D. (eds.) Proceedings of the IJCAI 2019 Workshop on Explainable Artificial Intelligence. pp. 1–6. Macau (2019)
  * [6] De Giacomo, G., Iocchi, L., Favorito, M., Patrizi, F.: Restraining Bolts for Reinforcement Learning Agents. In: Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence. vol. 9, pp. 13659–13662. AAAI Press, Palo Alto, California (2020). https://doi.org/10.1609/aaai.v34i09.7114
  * [7] Frosst, N., Hinton, G.: Distilling a Neural Network Into a Soft Decision Tree. In: Besold, T.R., Kutz, O. (eds.) Proceedings of the First International Workshop on Comprehensibility and Explanation in AI and ML 2017. AI*IA Series, vol. 2071. CEUR Workshop Proceedings, Aachen, Germany (2017)
  * [8] Fürnkranz, J., Gamberger, D., Lavrač, N.: Foundations of Rule Learning. Cognitive Technologies, Springer, Berlin, Heidelberg (2012). https://doi.org/10.1007/978-3-540-75197-7
  * [9] Gevaert, A., Peck, J., Saeys, Y.: Distillation of Deep Reinforcement Learning Models using Fuzzy Inference Systems. In: Beuls, K., Bogaerts, B., Bontempi, G., Geurts, P., Harley, N., Lebichot, B., Lenaerts, T., Louppe, G., Van Eecke, P. (eds.) Proceedings of the 31st Benelux Conference on Artificial Intelligence (BNAIC 2019) and the 28th Belgian Dutch Conference on Machine Learning (Benelearn 2019). vol. 2491. CEUR Workshop Proceedings, Aachen, Germany (2019)
  * [10] Hinton, G., Vinyals, O., Dean, J.: Distilling the Knowledge in a Neural Network. arXiv e-prints arXiv:1503.02531 (2015)
  * [11] Huang, J., Angelov, P.P., Yin, C.: Interpretable policies for reinforcement learning by empirical fuzzy sets. Engineering Applications of Artificial Intelligence 91 (2020). https://doi.org/10.1016/j.engappai.2020.103559
  * [12] Karakovskiy, S., Togelius, J.: The Mario AI Benchmark and Competitions. IEEE Transactions on Computational Intelligence and AI in Games 4(1), 55–67 (2012). https://doi.org/10.1109/TCIAIG.2012.2188528
  * [13] Lavrač, N., Flach, P., Zupan, B.: Rule Evaluation Measures: A Unifying View. In: Džeroski, S., Flach, P. (eds.) Inductive Logic Programming. Lecture Notes in Computer Science, vol. 1634, pp. 174–185. Springer, Berlin, Heidelberg (1999). https://doi.org/10.1007/3-540-48751-4_17
  * [14] Libin, P., Moonens, A., Verstraeten, T., Perez Sanjines, F.R., Hens, N., Lemey, P., Nowé, A.: Deep reinforcement learning for large-scale epidemic control. In: Proceedings of the Adaptive and Learning Agents Workshop 2020 (ALA2020) at AAMAS (2020)
  * [15] Madumal, P., Miller, T., Sonenberg, L., Vetere, F.: Explainable reinforcement learning through a causal lens. In: Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence. vol. 3, pp. 2493–2500. AAAI Press, Palo Alto, California (2020). https://doi.org/10.1609/aaai.v34i03.5631
  * [16] Maes, P.: Computational Reflection. In: Morik, K. (ed.) GWAI-87 11th German Workshop on Artifical Intelligence. Informatik-Fachberichte, vol. 152, pp. 251–265. Springer, Berlin, Heidelberg (1987). https://doi.org/10.1007/978-3-642-73005-4_27
  * [17] Miller, T.: Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence 267, 1–38 (2019). https://doi.org/10.1016/j.artint.2018.07.007
  * [18] Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., Hassabis, D.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015). https://doi.org/10.1038/nature14236
  * [19] Molnar, C.: Interpretable Machine Learning. Leanpub, Victoria, Canada (2019)
  * [20] Rusu, A.A., Colmenarejo, S.G., Gulcehre, C., Desjardins, G., Kirkpatrick, J., Pascanu, R., Mnih, V., Kavukcuoglu, K., Hadsell, R.: Policy Distillation. In: International Conference on Learning Representations (2016), arXiv:1511.06295
  * [21] Rückstieß, T., Sehnke, F., Schaul, T., Wierstra, D., Sun, Y., Schmidhuber, J.: Exploring parameter space in reinforcement learning. Paladyn, Journal of Behavioral Robotics 1(1), 14–24 (2010). https://doi.org/10.2478/s13230-010-0002-4
  * [22] Steckelmacher, D., Plisnier, H., Roijers, D.M., Nowé, A.: Sample-efficient model-free reinforcement learning with off-policy critics. In: Brefeld, U., Fromont, E., Hotho, A., Knobbe, A., Maathuis, M., Robardet, C. (eds.) Machine Learning and Knowledge Discovery in Databases. Lecture Notes in Computer Science, vol. 11908, pp. 19–34. Springer, Cham, Switzerland (2020). https://doi.org/10.1007/978-3-030-46133-1_2
  * [23] Sutton, R.S., Barto, A.G.: Reinforcement Learning : An Introduction. MIT Press, Cambridge, Massachusetts, 2nd edn. (2018)
  * [24] Sutton, R.S., McAllester, D., Singh, S., Mansour, Y.: Policy Gradient Methods for Reinforcement Learning with Function Approximation. In: Neural Information Processing Systems (NIPS). pp. 1057–1063 (2000)
  * [25] Tadepalli, P., Givan, R., Driessens, K.: Relational Reinforcement Learning: An Overview. In: Tadepalli, P., Givan, R., Driessens, K. (eds.) Proceedings of the ICML-2004 Workshop on Relational Reinforcement Learning. pp. 1–9. Banff, Canada (2004)
  * [26] Todorovski, L., Flach, P., Lavrač, N.: Predictive Performance of Weighted Relative Accuracy. In: Zighed, D.A., Komorowski, J., Żytkow, J. (eds.) Principles of Data Mining and Knowledge Discovery. Lecture Notes in Computer Science, vol. 1910, pp. 255–264. Springer, Berlin, Heidelberg (2000). https://doi.org/10.1007/3-540-45372-5_25
  * [27] Zambaldi, V., Raposo, D., Santoro, A., Bapst, V., Li, Y., Babuschkin, I., Tuyls, K., Reichert, D., Lillicrap, T., Lockhart, E., Shanahan, M., Langston, V., Pascanu, R., Botvinick, M., Vinyals, O., Battaglia, P.: Deep reinforcement learning with relational inductive biases. In: International Conference on Learning Representations (2019)


# On Restricted Powers of Complete Intersections

Keller VandeBogert

###### Abstract.

A restricted $d$th power of an ideal $I$ is obtained by restricting the
exponent vectors allowed to appear on the “natural” generating set of $I^{d}$,
for some integer $d$. In this paper, we study homological properties of
restricted powers of complete intersections. We construct an explicit minimal
free resolution for any restricted power of a complete intersection which
generalizes the $L$-complex construction of Buchsbaum and Eisenbud. We use
this resolution to compute an explicit basis for the Koszul homology which
allows us to deduce that the quotient defined by any restricted $d$th power of
a complete intersection is a Golod ring for $d\geqslant 2$. Finally, using
techniques of Miller and Rahmati, we show that the minimal free resolution of
the quotient defined by any restricted power of a complete intersection admits
the structure of an associative DG-algebra.

##  1\. Introduction

Let $I=(a_{1},\dots,a_{n})$ be an ideal in some commutative ring $R$ and
$w=(w_{1},\dots,w_{n})\\\ \in\mathbb{Z}_{\geqslant 0}^{n}$ any vector. The
$w$-restricted $d$th power of $I$ is the ideal generated by all elements of
the form $a_{1}^{d_{1}}\cdots a_{n}^{d_{n}}$, where $d_{i}\leqslant w_{i}$ for
each $1\leqslant i\leqslant n$ and $d_{1}+\cdots+d_{n}=d$. From a
combinatorial perspective, if one imagines all exponent vectors with sum $d$
as being encoded by the dilated $(n-1)$-simplex $d\cdot\Delta^{n-1}$, then the
exponent vectors for the $w$-restricted $d$th power are obtained by cutting
$d\cdot\Delta^{n-1}$ by an appropriate collection of hyperplanes. This
perspective is employed in, for instance, [1], where the results of Almousa,
Fløystad, and Lohne are extended to the case of polarizations of restricted
powers of the graded maximal ideal (in a polynomial ring) with a
characterization dependent on the associated $w$-restricted dilated simplex.

Gasharov, Hibi, and Peeva have also considered $w$-stable monomial ideals,
where the idea is similar: simply restrict the minimal generating set of a
stable monomial ideal to all monomials appearing with exponent vector bounded
above by $w$. As it turns out, many of the properties possessed by stable
ideals are inherited by $w$-stable ideals, such as Golodness and a linear
resolution (in the equigenerated case). Moreover, since the Eliahou-Kervaire
resolution is a naturally multigraded minimal free resolution, the minimal
free resolution of a $w$-stable ideal is obtained by restricting to all
multidegrees that are also bounded above by $w$. This recovers and generalizes
the _squarefree_ Eliahou-Kervaire resolution, introduced in [2].

In this paper, we show that the idea of “restricting multidegrees” can be
extended to arbitrary $w$-restricted powers of complete intersections (which
need not be monomial ideals). We introduce a generalization of the well-known
$L$-complexes of Buchsbaum and Eisenbud (see [4]) and prove that these
complexes yield a minimal free resolution of $w$-restricted powers of complete
intersections. We also give an explicit description of the basis elements of
the Koszul homology of any restricted power of a complete intersection by
lifting basis elements of the aforementioned minimal free resolution to the
Koszul homology algebra. This allows us to prove that for $d\geqslant 2$, the
$d$th restricted power of a complete intersection always defines a Golod ring.
Finally, we employ recent results of Miller and Rahmati [10] to show that, for
large enough characteristic, the minimal free resolution of the quotient
defined by the restricted power of any complete intersection admits the
structure of an associative DG-algebra.

${{(2,0,0)}}$${{(2,0,0)}}$${{(1,1,0)}}$${{(1,0,1)}}$${{(1,1,0)}}$${{(1,0,1)}}$${{(0,2,0)}}$${{(0,1,1)}}$${{(0,0,2)}}$${{(0,1,1)}}$
Figure 1. Restriction of the dilated $1$-simplex $2\cdot\Delta^{1}$ to the
vector $(2,1,1)$.

The paper is organized as follows. In Section 2, we introduce notation and
conventions that will be used in the rest of the paper. This includes the
definition of a Golod ring in terms of trivial Massey operations and a
refresher on much of the content of [10] dealing with transferring algebra
structures along special deformation retracts. In Section 3, we define the
pieces that constitute the building blocks of the minimal free resolution of
quotients defined by restricted powers of complete intersections. The
construction generalizes the original construction of Buchsbaum and Eisenbud,
and we give a brief and self-contained proof of acyclicity.

In Section 4, we use the minimal free resolution of Section 3 to obtain
information about the Koszul homology algebra and Golodness. More precisely,
we are able to find an explicit lift of the basis elements in the minimal free
resolution to the Koszul homology algebra. With respect to this basis of the
Koszul homology, we find that the trivial Massey operation with
$\mu(h_{1},\dots,h_{k})=0$ for $k\geqslant 2$ is well-defined, allowing us to
deduce Golodness of all $w$-restricted $d$th powers of complete intersections,
for $d\geqslant 2$. This also gives an explicit minimal free resolution of the
residue field due to a well-known construction of Golod.

In Section 5, we prove that the minimal free resolution constructed in Section
3 admits the structure of an associative DG-algebra. The proof of this fact
relies heavily on recent techniques developed by Miller and Rahmati in [10].
In particular, we construct an explicit algebra structure on an associated
total complex obtained by restricting multidegrees, and show that the scaled
de Rham map satisfies the generalized Leibniz rule with respect to this
product. As a consequence, the perturbation lemma will allow us to transfer
the product structure on this total complex to the generalized $L$-complexes
of Section 3, immediately extending the application to the Buchsbaum-Eisenbud
$L$-complexes given in [10].

##  2\. Restricted Powers, Golod Rings, and Transferring Algebra Structures

The purpose of this section is to provide background and conventions for the
material to be used for the rest of the paper. The main focal points of this
section are the definition of Golod (see Definition 2.5) and the terminology
introduced in the latter half, culminating in Proposition 2.11. For a general
overview of DG-algebra techniques, including exposition on Golod rings, the
reader is encouraged to consult Avramov [3]. The material on deformation
retracts and the perturbation lemma comes from [10]. We begin this section by
defining the $w$-restricted $d$th power of a complete intersection.

######  Definition 2.1.

Let $F$ be a free $R$-module of rank $n$ with basis $f_{1},\dots,f_{n}$. Let
$w=(w_{1},\dots,w_{n})\in\mathbb{N}^{n}$ be any vector and let $\psi:F\to R$
be any $R$-module homomorphism. Then the $w$-_restricted $d$th power_ of
$\operatorname{im}(\psi)$, denoted $(\operatorname{im}\psi)^{d}_{w}$, is
defined to be the ideal generated by all elements of the form

$\displaystyle\psi(f_{1})^{\alpha_{1}}\cdot\psi(f_{2})^{\alpha_{2}}\cdots\psi(f_{n})^{\alpha_{n}},\quad$
$\displaystyle\textrm{where}\ \alpha_{i}\leqslant w_{i}\ \textrm{for all}\
1\leqslant i\leqslant n$ $\displaystyle\quad\textrm{and}\
\alpha_{1}+\cdots+\alpha_{n}=d.$

The idea of restricted powers has been considered before by other authors in
different contexts. Indeed:

  1. (1)

If $w=(d,d,\dots,d)$, then
$(\operatorname{im}\psi)_{w}^{d}=(\operatorname{im}\psi)^{d}$ is just the
$d$th power of $\operatorname{im}\psi$.

  2. (2)

If $w=(1,1,\dots,1)$, then $(\operatorname{im}\psi)_{w}^{d}$ is the
_squarefree_ power of $\operatorname{im}\psi$.

  3. (3)

If $R=k[x_{1},\dots,x_{n}]$ and $\operatorname{im}\psi=(x_{1},\dots,x_{n})$,
then $(\operatorname{im}\psi)^{d}_{w}$ is an example of a $w$-stable ideal as
considered by Gasharov, Hibi, and Peeva in [7].

  4. (4)

Retaining the notation of $(3)$, let $P(d)$ denote the set of all integer
partitions of $d$. Then $(\operatorname{im}\psi)^{d}_{w}=I_{P(d)}$, in the
notation of the $S_{n}$-invariant ideals considered by Raicu in [11].

  5. (5)

In [1], it is shown that any restricted power of the graded maximal ideal has
a cellular resolution, and a complete characterization for polarizations of
all such ideals is given.

######  Notation 2.2.

The notation $(F_{\bullet},d_{\bullet})$ will denote a complex $F_{\bullet}$
with differentials $d_{\bullet}$. When no confusion may occur, $F$ may be
written, where the notation $d^{F}$ is understood to mean the differential of
$F$ (in the appropriate homological degree).

Given a complex $F_{\bullet}$ as above, elements of $F_{n}$ will often be
denoted $f_{n}$, without specifying that $f_{n}\in F_{n}$.

######  Definition 2.3.

A _differential graded algebra_ $(F,d)$ (DG-algebra) over a commutative
Noetherian ring $R$ is a complex of finitely generated free $R$-modules with
differential $d$ and with a unitary, associative multiplication
$F\otimes_{R}F\to F$ satisfying

  1. (a)

$F_{i}F_{j}\subseteq F_{i+j}$,

  2. (b)

$d_{i+j}(x_{i}x_{j})=d_{i}(x_{i})x_{j}+(-1)^{i}x_{i}d_{j}(x_{j})$,

  3. (c)

$x_{i}x_{j}=(-1)^{ij}x_{j}x_{i}$, and

  4. (d)

$x_{i}^{2}=0$ if $i$ is odd,

where $x_{k}\in F_{k}$.

The next definition will be essential for defining Golod rings. If
$z_{\lambda}$ is a cycle in some complex $A$, then the notation
$[z_{\lambda}]$ denotes the homology class of $z_{\lambda}$. In the following
definition, $\overline{a}:=(-1)^{|a|+1}a$, where $|a|$ denotes the homological
degree of $a\in A$.

######  Definition 2.4.

Let $A$ be a DG-algebra with $H_{0}(A)\cong k$. Then $A$ admits a _trivial
Massey operation_ if for some $k$-basis
$\mathcal{B}=\\{h_{\lambda}\\}_{\lambda\in\Lambda}$, there exists a function

$\mu:\coprod_{i=1}^{\infty}\mathcal{B}^{i}\to A$

such that

$\displaystyle\mu(h_{\lambda})=z_{\lambda}\quad\textrm{with}\quad[z_{\lambda}]=h_{\lambda},\
\textrm{and}$ $\displaystyle
d\mu(h_{\lambda_{1}},\dots,h_{\lambda_{p}})=\sum_{j=1}^{p-1}\overline{\mu(h_{\lambda_{1}},\dots,h_{\lambda_{j}})}\mu(h_{\lambda_{j+1}},\dots,h_{\lambda_{p}}).$

Observe that taking $p=2$ in the above definition yields that $H_{\geqslant
1}(A)^{2}=0$, so the induced algebra structure on $H(A)$ is totally trivial
for a DG-algebra admitting a trivial Massey operation.

######  Definition 2.5.

Let $(R,\mathfrak{m})$ be a local ring and let $K^{R}$ denote the Koszul
complex on the generators of $\mathfrak{m}$. If $K^{R}$ admits a trivial
Massey operation $\mu$, then $R$ is called a _Golod ring_.

######  Remark 2.6.

It is worth mentioning that Definition 2.5 is equivalent to saying that the
Poincaré series $P^{R}_{k}(t)$ of $R$ attains a coefficient-wise inequality
originally established by Serre (this is often given as the definition of
Golodness, but Definition 2.5 will be much more convenient for our purposes).
As observed above, the Koszul homology algebra of any Golod ring has trivial
multiplication in positive homological degrees; it is worth noting that for
rings of projective dimension $\geqslant 4$, there exist non-Golod rings with
trivial Koszul homology algebras (see work by Katthän [9]).

Next, we recall some of the terminology and conventions established in [10]
for transferring algebra structures. The main goals for the remainder of this
section are the establishment of Proposition 2.11, which furnishes conditions
for which an algebra structure can be transferred along a perturbed
deformation retract.

######  Definition 2.7.

Let $F_{\bullet}$ and $G_{\bullet}$ be two complexes. A deformation retract is
a quasi-isomorphism of complexes

${{F_{\bullet}}}$${{G_{\bullet}}}$$\scriptstyle{p}$$\scriptstyle{i}$

satisfying:

  1. (1)

$p\circ i=1$ and

  2. (2)

$i\circ p\simeq 1$ via some homotopy $h$ on $F_{\bullet}$.

A deformation retract is _special_ if, furthermore, one has:

  1. (1)

$h\circ i=0$,

  2. (2)

$p\circ h=0$, and

  3. (3)

$h^{2}=0$.

Given a deformation retract

${{F_{\bullet}}}$${{G_{\bullet},}}$$\scriptstyle{p}$$\scriptstyle{i}$

with associated homotopy $h$, a perturbation is a map $\delta$ such that
$d^{F}+\delta$ is a differential on $F_{\bullet}$; that is,
$(d^{F}+\delta)^{2}=0$. The perturbation $\delta$ is _small_ if $1-\delta h$
is invertible.

Note that if $F_{\bullet}$ is a finite complex, then every perturbation is
locally nilpotent and hence small.

######  Setup 2.8.

Let

${{F_{\bullet}}}$${{G_{\bullet},}}$$\scriptstyle{p}$$\scriptstyle{i}$

be a special deformation retract with associated homotopy $h$. Assume that
$\delta$ is a small perturbation on $F_{\bullet}$ and let $A:=(1-\delta
h)^{-1}\delta$. Define the following data:

  1. (1)

$i_{\infty}:=i+hAi$,

  2. (2)

$p_{\infty}:=p+pAh$,

  3. (3)

$d^{F}_{\infty}:=d^{F}+\delta$,

  4. (4)

$d^{G}_{\infty}:=d^{G}+pAi$, and

  5. (5)

$h_{\infty}:=h+hAh$.

######  Lemma 2.9 (Perturbation Lemma).

Adopt notation and hypotheses as in Setup 2.8. Then the data

${{(F_{\bullet},d^{F}_{\infty})}}$${{(G_{\bullet},d^{G}_{\infty})}}$$\scriptstyle{p_{\infty}}$$\scriptstyle{i_{\infty}}$

is a special deformation retract with associated homotopy $h_{\infty}$.

######  Definition 2.10.

Let $F_{\bullet}$ be a complex of $R$-modules equipped with a product, and let
$h:F_{\bullet}\to F_{\bullet}$ be any graded map. Then $h$ satisfies the
_generalized Leibniz rule_ if for every $f$, $f^{\prime}\in F_{\bullet}$,

$h(f\cdot f^{\prime})\subseteq h(f)X+Xh(f^{\prime}).$

The following Proposition is a slight variant on Proposition $3.5$ of [10]; in
the original statement, it was assumed that $F_{\bullet}$ was a DG-algebra
with the unperturbed differential. It turns out that this hypothesis is
unnecessary, and that the existence of an associative product for which the
associated homotopy satisfies the generalized Leibniz rule is sufficient.

######  Proposition 2.11.

Adopt notation and hypotheses as in Setup 2.8, and assume furthermore that:

  1. (1)

$(F_{\bullet},d^{F})$ is an associative algebra (not necessarily satisfying
the Leibniz rule),

  2. (2)

$d^{F}_{\infty}=d^{F}+\delta$ satisfies the Leibniz rule, and

  3. (3)

$h$ satisfies the generalized Leibniz rule.

Then $(G_{\bullet},d^{G}_{\infty})$ is an associative DG-algebra with product:

$g\cdot_{G}g^{\prime}:=p_{\infty}(i_{\infty}(g)\cdot i_{\infty}(g^{\prime})).$

Moreover, the map
$i_{\infty}:(G_{\bullet},d^{G}_{\infty})\to(F_{\bullet},d^{F}_{\infty})$ is a
morphism of DG-algebras.

###### Proof.

The proof is essentially identical to that of [10, Proposition 1.4], where one
only needs to verify that
$(d^{F}_{\infty}h_{\infty}+h_{\infty}d^{F}_{\infty})(i_{\infty}(f)\cdot
i_{\infty}(f^{\prime}))=0$; this is a straightforward computation. ∎

##  3\. The Minimal Free Resolution

In this section, we construct an explicit minimal free resolution for the
quotient defined by the $w$-restricted $d$th power of a complete intersection,
for any vector $w$ and power $d$. The construction is highly reminiscent of
the original construction of Buchsbaum and Eisenbud; the proof of exactness
here is self contained and is distinct from the proof given in [4]. Indeed,
the proof of Theorem 3.10 was inspired by the proof of exactness given in [5,
Theorem 2.12].

Let $F$ be a free $R$-module of rank $n$ with basis elements
$f_{1},\dots,f_{n}$, with $\psi:F\to R$ any map such that
$\psi(f_{1}),\dots,\psi(f_{n})$ forms a regular sequence. Throughout this
section, we will assume either:

  1. (1)

$R$ is a Noetherian local ring, or

  2. (2)

$R$ is a $\mathbb{Z}$-graded ring, in which case the elements
$\psi(f_{1}),\dots,\psi(f_{n})$ are assumed to be homogeneous of positive
degree.

Both of the above conditions guarantee that any subsequence of the regular
sequence $\psi(f_{1}),\dots,\psi(f_{n})$ remains regular. We begin by
establishing the following notation, which will be employed tacitly for the
remainder of the paper.

######  Notation 3.1.

Let $F$ be a free $R$-module of rank $n$ with basis elements
$f_{1},\dots,f_{n}$. Given integers $a$ and $b\geqslant 0$, the following
notation will be used for conciseness:

$\bigwedge^{a}:=\bigwedge^{a}F,\qquad S_{b}:=S_{b}(F),$

denoting the exterior and symmetric algebras, respectively. Let
$\sigma=(\sigma_{1}<\cdots<\sigma_{a})$ be an indexing set and
$\alpha=(\alpha_{1},\dots,\alpha_{n})$ be an exponent vector with
$|\alpha|:=\alpha_{1}+\cdots+\alpha_{n}=b$. Then, the following notation will
be used:

$f_{\sigma}:=f_{\sigma_{1}}\wedge\cdots\wedge
f_{\sigma_{a}}\in\bigwedge^{a},\qquad f^{\alpha}:=f_{1}^{\alpha_{1}}\cdots
f_{n}^{\alpha_{n}}\in S_{b}.$

Given any integer $1\leqslant i\leqslant n$, the notation $\epsilon_{i}$ will
denote the vector with a $1$ in the $i$th spot and $0$’s elsewhere. Throughout
the paper, for any two vectors $\alpha$, $\beta\in\mathbb{Z}^{n}$, write

$\alpha\leqslant\beta\iff\alpha_{i}\leqslant\beta_{i}\ \textrm{for all}\
1\leqslant i\leqslant n.$

######  Definition 3.2.

Adopt notation as in Notation 3.1. Then the _multidegree_ of an element
$f_{\sigma}\otimes f^{\alpha}\in\bigwedge^{a}\otimes S_{b}$, denoted
$\operatorname{mdeg}$, is defined as

$\operatorname{mdeg}(f_{\sigma}\otimes
f^{\alpha}):=(\epsilon_{\sigma_{1}}+\dots+\epsilon_{\sigma_{a}}+\alpha).$

The multigrading of Definition 3.2 should cause no confusion, since it is
induced by the natural choice of multigrading on the symmetric and exterior
algebras $S_{\bullet}$ and $\bigwedge^{\bullet}$, if each $f_{i}$ is given
multidegree $\epsilon_{i}$.

######  Setup 3.3.

Adopt notation as in Notation 3.1 and let $\psi:F\to R$ be such that
$\operatorname{grade}\operatorname{im}(\psi)=n$. Let $w=(w_{1},\dots,w_{n})$
be any vector and set

$(\bigwedge^{a}\otimes S_{b})_{w}:=\\{f_{\sigma}\otimes
f^{\alpha}\mid\operatorname{mdeg}(f_{\sigma}\otimes f^{\alpha})\leqslant
w\\}.$

Let $\kappa_{a,b}^{w}:(\bigwedge^{a}\otimes
S_{b})_{w}\to(\bigwedge^{a-1}\otimes S_{b+1})_{w}$ be the map induced by the
tautological Koszul differential:

$\displaystyle\bigwedge^{a}\otimes S_{b}$
$\displaystyle\xrightarrow{\textrm{comult.}\otimes 1}\bigwedge^{a-1}\otimes
F\otimes S_{b}$
$\displaystyle\xrightarrow{1\otimes\textrm{mult.}}\bigwedge^{a-1}\otimes
S_{b+1}.$

Observe that $\kappa_{a,b}^{w}$ is well defined because it preserves
multidegree. With this notation, define
$L^{a}_{b,w}(F)=L^{a}_{b,w}:=\operatorname{Ker}\kappa_{a,b}^{w}\subseteq(\bigwedge^{a}\otimes
S_{b})_{w}$.

As it turns out, the basis elements for these modules are actually quite
comprehensible. The proof of the following proposition is a consequence of the
much more general statement given in [13, Proposition 2.1.4] combined with the
fact that all tableaux appearing in a given straightening relation have the
same multidegree.

######  Proposition 3.4.

For every $a,b\geqslant 0$, the module $L^{a}_{b,w}$ has basis represented by
the set of semistandard tableaux with multidegree bounded above by $w$.

######  Example 3.5.

Let $R=k[x_{1},\dots,x_{3}]$ and $F=Rf_{1}\oplus Rf_{2}\oplus Rf_{3}$ with
$\psi:F\to R$ induced by sending $f_{i}\mapsto x_{i}$. Suppose that
$w=(3,1,1)$; then

$(x_{1},x_{2},x_{3})^{3}_{w}=(x_{1}^{3},x_{1}^{2}x_{2},x_{1}^{2}x_{3},x_{1}x_{2}x_{3}).$

Likewise, the module $L^{1}_{3,w}(F)$ has basis represented by the tableaux

$\ytableausetup{boxsize=1.7em}\ytableau 1&11\\\ 2\\\
,\quad\ytableausetup{boxsize=1.7em}\ytableau 1&11\\\ 3\\\
,\quad\ytableausetup{boxsize=1.7em}\ytableau 1&12\\\ 3\\\
,\quad\ytableausetup{boxsize=1.7em}\ytableau 1&13\\\ 2\\\ .$

The following observation is a straightforward verification which shows that
the horizontal and vertical differentials of Figure 2 anticommute.

######  Observation 3.6.

Adopt notation and hypotheses as in Setup 3.3. Let
$\textrm{Kos}:\bigwedge^{a}\to\bigwedge^{a-1}$ denote the Koszul differential
induced by $\psi$, for any given $a\geqslant 0$. Then $\textrm{Kos}\otimes
1:\bigwedge^{\bullet}\otimes S_{\bullet}\to\bigwedge^{\bullet}\otimes
S_{\bullet}$ and $\kappa_{\bullet,\bullet}$ anticommute; that is,

$(\textrm{Kos}\otimes
1)\circ\kappa_{a,b}^{w}=-\kappa_{a-1,b}^{w}\circ(\textrm{Kos}\otimes 1).$

######  Notation 3.7.

For conciseness and ease of notation, subscripts indicating homological
degrees will often be omitted. Moreover, the vector $w$ as in Setup 3.3 will
often be omitted in the notation $\kappa_{a,b}^{w}$, and the much simpler
notation $\kappa$ may be used.

Likewise, for any map $\psi:F\to R$, there is a naturally induced map
$S_{d}(\psi):S_{d}\to R$ defined by sending
$f^{\alpha}\mapsto\psi(f)^{\alpha_{1}}\cdots\psi(f_{n})^{\alpha_{n}}$. This
map will also be denoted simply by $\psi$.

Given a double complex $E_{\bullet,\bullet}$ with anticommuting vertical and
horizontal differentials $d^{E,v}$ and $d^{E,h}$, respectively, the total
complex $T(E_{\bullet,\bullet})$ will have differential $d^{E,h}-d^{E,v}$.

The following proposition will be helpful in the proof of Theorem 3.10. Recall
that the assumptions put forth at the beginning of this section imply that any
subsequence of our regular sequence is again a regular sequence.

######  Proposition 3.8.

Adopt notation and hypotheses as in Setup 3.3. Let $E$ denote the double
complex of Figure 2. Then,

  1. (1)

all rows of $E$ except for the bottom-most row are exact, and

  2. (2)

the complexes $(\bigwedge^{\bullet}\otimes S_{b})_{w}$ for $0\leqslant
b\leqslant d-1$ are exact in homological degrees $\geqslant 1$.

###### Proof.

Since the rows of $E$ are obtained from restricting the multidegrees appearing
in the truncated tautological Koszul complex, they must be exact, except for
the bottom-most row. At the bottom, the homology is isomorphic to $R$.

To see that the complexes $(\bigwedge^{\bullet}\otimes S_{b})_{w}$ are
acyclic, observe that if $f_{s}\otimes f^{\alpha}\in(\bigwedge^{1}\otimes
S_{b})_{w}$ for all $s\in S$, where $S$ is some subset of
$[n]=\\{1,\dots,n\\}$, then $f_{\sigma}\otimes
f^{\alpha}\in(\bigwedge^{|\sigma|}\otimes S_{b})_{w}$ for every $\sigma\subset
S$. Thus, $(\bigwedge^{\bullet}\otimes S_{b})_{w}$ decomposes as a direct sum
of Koszul complexes on subsets of the set
$\\{\psi(f_{1}),\dots,\psi(f_{n})\\}$. ∎

######  Definition 3.9.

Adopt notation and hypotheses as in Setup 3.3. Then $L^{w}(\psi,d)$ denotes
the complex

$0\to L_{d,w}^{n-1}(F)\xrightarrow{\textrm{Kos}\otimes
1}\cdots\xrightarrow{\textrm{Kos}\otimes
1}L_{d,w}^{1}(F)\xrightarrow{\textrm{Kos}\otimes
1}(S_{d})_{w}\xrightarrow{\psi}R\to 0.$

We finally arrive at the main result of this section:

######  Theorem 3.10.

Adopt notation and hypotheses as in Setup 3.3. The complex $L^{w}(\psi,d)$ of
Definition 3.9 is the minimal free resolution of
$R/(\operatorname{im}\psi)^{d}_{w}$.

${0}$${0}$${{(\bigwedge^{n}\otimes
S_{d-1})_{w}}}$${{L^{n-1}_{d,w}}}$${\iddots}$${\vdots}$${\vdots}$${\vdots}$${\vdots}$${\iddots}$${{(\bigwedge^{2}\otimes
S_{d-1})_{w}}}$${{L_{d,w}^{1}}}$${\iddots}$${{(\bigwedge^{1}\otimes
S_{d-1})_{w}}}$${{L^{0}_{d,w}}}$${\vdots}$${\vdots}$${\vdots}$${\iddots}$${{(\bigwedge^{2}\otimes
S_{0})_{w}}}$${{(\bigwedge^{1}\otimes S_{1})_{w}}}$${{(\bigwedge^{0}\otimes
S_{2})_{w}}}$${{(\bigwedge^{1}\otimes S_{0})_{w}}}$${{(\bigwedge^{0}\otimes
S_{1})_{w}}}$${{(\bigwedge^{0}\otimes
S_{0})_{w}}}$$\scriptstyle{\kappa}$$\scriptstyle{\kappa}$$\scriptstyle{\textrm{Kos}\otimes
1}$$\scriptstyle{\textrm{Kos}\otimes 1}$$\scriptstyle{\textrm{Kos}\otimes
1}$$\scriptstyle{\kappa}$$\scriptstyle{\kappa}$$\scriptstyle{\textrm{Kos}\otimes
1}$$\scriptstyle{\textrm{Kos}\otimes 1}$$\scriptstyle{\textrm{Kos}\otimes
1}$$\scriptstyle{\textrm{Kos}\otimes
1}$$\scriptstyle{\kappa}$$\scriptstyle{\kappa}$$\scriptstyle{\kappa}$$\scriptstyle{\textrm{Kos}\otimes
1}$$\scriptstyle{\textrm{Kos}\otimes 1}$$\scriptstyle{\textrm{Kos}\otimes
1}$$\scriptstyle{\textrm{Kos}\otimes 1}$$\scriptstyle{\textrm{Kos}\otimes 1}$
Figure 2. The double complex used for the proof of Theorem 3.10; recall the
conventions established in Notation 3.7.

###### Proof.

Throughout the proof, let $T(-)$ denote the totalization of a double complex.
Let $E$ denote the double complex of Figure 2, $E^{\prime}$ the double complex
obtained by deleting the bottom most $\bigwedge^{0}\otimes S_{0}$ term, and
$E^{\prime\prime}$ the rightmost nontrivial column, which is $L^{w}(\psi,d)$
without $R$ in homological degree $0$. Index the total complex $T(E)$ such
that $T(E)_{0}=\Big{(}\bigoplus_{i=0}^{d-1}(\bigwedge^{0}\otimes
S_{i})_{w}\Big{)}\oplus L^{0}_{d,w}$. Then the short exact sequence of total
complexes

$0\to T(E^{\prime})\to T(E)\to T(E/E^{\prime})\to 0$

combined with Proposition 3.8 (1) yields that

$H_{i}(T(E))=H_{i}(T(E^{\prime}))=\begin{cases}R&\textrm{if}\ i=0\\\
0&\textrm{otherwise}.\end{cases}$

Likewise, the short exact sequence

$0\to T(E^{\prime\prime})\to T(E)\to T(E/E^{\prime\prime})\to 0$

combined with Proposition 3.8 (2) yields that $H_{i}(T(E^{\prime\prime}))=0$
for $i\geqslant 1$ and there is an inclusion of homology

$H_{0}(T(E^{\prime\prime}))\hookrightarrow H_{0}(T(E)).$

By definition, this inclusion lifts a cycle $f^{\alpha}=f_{i_{1}}\cdots
f_{i_{d}}\in(S_{d})_{w}=L^{0}_{d,w}$ to a cycle of $T(E)_{0}$; one such lift
is given by

$f^{\alpha}+\sum_{j=1}^{d}\psi(f_{i_{1}}\cdots f_{i_{j}})f_{i_{j+1}}\cdots
f_{i_{d}}.$

Composing with the isomorphism $H_{0}(T(E))\xrightarrow{\sim}R$ induces the
map $f^{\alpha}\mapsto\psi(f^{\alpha})$, whence augmenting $E^{\prime\prime}$
by the map $(S_{d})_{w}\xrightarrow{\psi}R$ remains acyclic. ∎

##  4\. Koszul Homology and Golodness

In this section, we study the Koszul homology of restricted powers of complete
intersections in regular local rings. Since the differentials in the complexes
of Definition 3.9 are induced by Koszul differentials, one can explicitly
compute the Koszul lift to the homology algebra. The main result of this
section is Corollary 4.4, but most of the work is done in the proof of
Proposition 4.3. After computing a basis for the Koszul homology, we are able
to find a straightforward trivial Massey operation allowing for an easy
description of the minimal free resolution of the residue field over the
quotient defined by a restricted power of a complete intersection.

Throughout this section, we will assume either:

  1. (1)

$R$ is a local ring, or

  2. (2)

$R$ is a standard graded polynomial ring over a field $k$, in which case
$\psi(f_{1}),\dots,\psi(f_{n})$ are assumed to be homogeneous of positive
degree.

######  Definition 4.1.

Let $M$ be a finitely generated $R$-module. The _Koszul homology_ of $M$,
denoted $H_{\bullet}(M)$, is defined to be the homology of
$M\otimes_{R}K_{\bullet}$; that is

$H_{\bullet}(M):=H_{\bullet}(M\otimes_{R}K_{\bullet}),$

where $K_{\bullet}$ denote the Koszul complex resolving the residue field,
$k$.

Let $A_{\bullet}$ and $B_{\bullet}$ be complexes with $H_{0}(A)=R$ and
$H_{0}(B)=S$. Recall that there is a functorial isomorphism

$H_{\bullet}(A\otimes S)\cong H_{\bullet}(R\otimes B)$

induced by the natural projections

${A}$${{A\otimes B}}$${B}$

The isomorphism can be explicitly described as follows:

  1. (1)

Choose a cycle $z_{1}$ in $A\otimes S$ representing a basis element of
$H_{\bullet}(A\otimes S)$.

  2. (2)

Lift $z_{1}$ to a cycle $z_{2}$ in $A\otimes B$.

  3. (3)

Project $z_{2}$ onto a cycle $z_{3}$ in $R\otimes B$, then descend to
homology.

######  Definition 4.2.

Adopt notation and hypotheses as in Setup 3.3. Let $K_{\bullet}$ denote the
Koszul complex resolving the residue field $k$. The map
$\phi^{i}_{j}:\bigwedge^{i}F\to K_{j}\otimes\bigwedge^{i-j}F$ is defined to be
the composition

$\displaystyle\bigwedge^{i}F$
$\displaystyle\xrightarrow{\textrm{comult}}\bigwedge^{j}F\otimes\bigwedge^{i-j}F$
$\displaystyle\xrightarrow{\textrm{lift}\otimes
1}K_{j}\otimes\bigwedge^{i-j}F,$

where $\textrm{lift}:\bigwedge^{j}\to K_{j}$ denotes the projection onto
$K_{j}$ of the cycle in $(\bigwedge^{\bullet}F\otimes K_{\bullet})_{j}$ that
represents any $f_{\tau}\in\bigwedge^{j}$.

In the notation of Definition 4.2, let $z_{\tau}$ denote the lift of any
$f_{\tau}\in\bigwedge^{j}$ to the Koszul algebra. Then the map $\phi_{j}^{i}$
of Definition 4.2 can be written explicitly:

$\displaystyle\phi_{j}^{i}(f_{\sigma})=\sum_{\begin{subarray}{c}\tau\subseteq\sigma\\\
|\tau|=j\\\
\end{subarray}}\operatorname{sgn}(\tau\subset\sigma)z_{\tau}\otimes
f_{\sigma\backslash\tau},$

where $\operatorname{sgn}$ is the sign of the permutation that reorders
$(\sigma\backslash\tau)\cup\tau$ into ascending order. The next proposition
tells us that the $\phi_{i}^{j}$ maps can be used to give an explicit lift of
basis elements of the complex of Definition 3.9 to the tensor product complex
$L^{w}(\psi,d)\otimes K_{\bullet}$.

######  Proposition 4.3.

Adopt notation and hypotheses as in Setup 3.3. Let $\bigwedge^{\bullet}F$
denote the Koszul complex induced by the map $\psi:F\to R$ and let
$(\bigwedge^{\bullet}F\otimes S_{d})_{w}$ be the complex obtained by
restricting to terms with multidegrees bounded above by $w$. Given any
$f_{\sigma}\otimes f^{\alpha}\in(\bigwedge^{i}F\otimes S_{d-1})_{w}$, the
element

$f_{\sigma}\otimes f^{\alpha}+\sum_{j=1}^{i-1}\phi_{j}^{i}(f_{\sigma})\otimes
f^{\alpha}+\psi(f^{\alpha})\phi_{i}^{i}(f_{\sigma})$

is a cycle in $\big{(}(\bigwedge^{\bullet}F\otimes S_{d-1})_{w}\otimes
K_{\bullet}\big{)}_{i}$.

###### Proof.

Assume $1\leqslant j\leqslant i-1$ (noting that $\phi_{0}^{i}$ is the
identity). Then:

$\displaystyle(\textrm{Kos}\otimes 1)(\phi_{j}^{i}(f_{\sigma})\otimes
f^{\alpha})$ $\displaystyle=(\textrm{Kos}\otimes
1)\Big{(}\sum_{\begin{subarray}{c}\tau\subset\in\sigma\\\ |\tau|=j\\\
\end{subarray}}\operatorname{sgn}(\tau)z_{\tau}\otimes
f_{\sigma\backslash\tau}\otimes f^{\alpha}\Big{)}$
$\displaystyle=\sum_{\begin{subarray}{c}\tau\subset\sigma\\\ |\tau|=j\\\
\end{subarray}}\sum_{r\in\tau}\operatorname{sgn}(r\in\tau)\operatorname{sgn}(\tau\subset\sigma)\psi(f_{r})z_{\tau\backslash
r}\otimes f_{\sigma\backslash\tau}\otimes f^{\alpha}$
$\displaystyle=\sum_{\begin{subarray}{c}\tau^{\prime}\subset\sigma\\\
|\tau^{\prime}|=j-1\\\
\end{subarray}}\sum_{r\in\sigma\backslash\tau^{\prime}}\operatorname{sgn}(r\in\sigma\backslash\tau^{\prime})\operatorname{sgn}(\tau^{\prime}\subset\sigma)z_{\tau^{\prime}}\otimes\psi(f_{r})f_{(\sigma\backslash\tau^{\prime})\backslash
r}$ $\displaystyle=(1\otimes\textrm{Kos})(\phi_{j-1}^{i}(f_{\sigma})\otimes
f^{\alpha}).$

In order to see that the above signs are equal, for any $r\in\sigma$ and
$\tau^{\prime}\in\sigma$, let $\tau$ be the set $\tau^{\prime}\cup r$ ordered
in ascending order. One can reorder the set
$(\sigma\backslash(\tau^{\prime}\cup r))\cup\tau^{\prime}\cup r$ into
ascending order by either:

  1. (1)

First reorder $\tau^{\prime}\cup r$ into ascending order, then reorder
$(\sigma\backslash(\tau^{\prime}\cup r))\cup\tau$ into ascending order; this
permutation has sign
$\operatorname{sgn}(r\in\tau)\operatorname{sgn}(\tau\subset\sigma)$.

  2. (2)

First reorder $(\sigma\backslash(\tau^{\prime}\cup r))\cup r$ into ascending
order, then reorder $(\sigma\backslash\tau^{\prime})\cup\tau^{\prime}$ into
ascending order; this permutation has sign
$\operatorname{sgn}(r\in\sigma\backslash\tau^{\prime})\operatorname{sgn}(\tau^{\prime}\subset\sigma)$.

Since both of the above cases yield a permutation of the same parity, the
signs are indeed equal.

In the case $j=i$, one has:

$\displaystyle(1\otimes\psi)(\phi_{i-1}^{i}(f_{\sigma})\otimes f^{\alpha})$
$\displaystyle=\sum_{r\in\sigma}\operatorname{sgn}(r\in\sigma)\psi(f^{\alpha}\cdot
f_{r})z_{\sigma\backslash r}$ $\displaystyle=(\textrm{Kos}\otimes
1)(\psi(f^{\alpha})z_{\sigma}).$

This completes the proof. ∎

######  Corollary 4.4.

Adopt notation and hypotheses as in Setup 3.3. The correspondence
$f_{\sigma}\otimes f^{\alpha}\mapsto\psi(f^{\alpha})\phi_{i}^{i}(f_{\sigma})$
induces an isomorphism of homology $L^{w}(\psi,d)\otimes k\to
H_{\bullet}(R/(\operatorname{im}\psi)^{d}_{w})$. Moreover, the ring
$R/(\operatorname{im}\psi)^{d}_{w}$ is Golod whenever $d\geqslant 2$.

###### Proof.

The first part of the statement is clear by construction of the isomorphism
$\operatorname{Tor}_{\bullet}^{R}(-,k)\cong H_{\bullet}(-\otimes K_{\bullet})$
combined with Proposition 4.3. The Golodness follows from noticing that the
product of elements of the form $\psi(f^{\alpha})\phi_{i}^{i}(f_{\sigma})$ in
$R/(\operatorname{im}\psi)^{d}_{w}\otimes K_{\bullet}$ are trivial. This
implies that simply choosing $\mu(h_{1},\dots,h_{n})=0$ for $i>1$ is a well
defined trivial Massey operation on the Koszul homology. ∎

For any $\ell\geqslant 1$, let $V_{\ell}$ denote the free $R$-module with
formal basis elements

$\\{v_{\sigma,\alpha}\mid\ell=|\sigma|+1\\},$

where we are thinking of each $v_{\sigma,\alpha}$ as being a formal stand-in
for the basis element $\psi(f^{\alpha})z_{\sigma}$ in the Koszul homology
algebra. Then, combining Corollary 4.4 with Golod’s construction of the
minimal free resolution of the residue field (see, for instance, [3, Theorem
5.2.2]), we have:

######  Corollary 4.5.

Adopt notation and hypotheses as in Setup 3.3. Let
$(T_{\bullet},\partial_{\bullet})$ denote the complex with

$\displaystyle T_{n}$
$\displaystyle:=\bigoplus_{p+h+i_{1}+\cdots+i_{p}=n}K_{h}\otimes_{R}V_{i_{1}}\otimes_{R}\cdots\otimes_{R}V_{i_{p}},$
$\displaystyle\partial_{n}$ $\displaystyle:T_{n}\to T_{n-1},$
$\displaystyle\partial_{n}(a\otimes
v_{\sigma^{1}\alpha^{1}}\otimes\cdots\otimes v_{\sigma^{p},\alpha^{p}})$
$\displaystyle=d(a)\otimes v_{\sigma^{1}\alpha^{1}}\otimes\cdots\otimes
v_{\sigma^{p},\alpha^{p}}$ $\displaystyle+(-1)^{|a|}$ $\displaystyle
a\psi(f^{\alpha^{1}})f_{\sigma^{1}}\otimes
v_{\alpha^{2},\sigma^{2}}\otimes\cdots\otimes v_{\alpha^{p},\sigma^{p}}.$

Then $T_{\bullet}$ is the minimal free resolution of the residue field $k$
over $R/(\operatorname{im}\psi)^{d}_{w}$.

######  Remark 4.6.

In a slightly different direction, it is easy to see that $w$-restricted
powers of arbitrary monomial ideals are $d$-Golod (hence Golod), in the
terminology of [8].

##  5\. Algebra Structure on the Minimal Free Resolution

In this section, we prove that the generalized $L$-complexes of Definition 3.9
admit the structure of an associative DG-algebra. The methods employed here
come from techniques developed by Miller and Rahmati in [10]. The process of
constructing the algebra structure consists of a few steps. First, we
construct an algebra structure on an associated total complex that surjects
onto the complexes of Definition 3.9. Next, we observe that the scaled de Rham
map (see Lemma 5.4) satisfies the generalized Leibniz rule with respect to
this algebra structure, inducing a special deformation retract. Finally, the
result will follow after combining the previous two sentences with Proposition
2.11.

Let $F$ be a free $R$-module of rank $n$ with basis elements
$f_{1},\dots,f_{n}$, with $\psi:F\to R$ any map such that
$\psi(f_{1}),\dots,\psi(f_{n})$ forms a regular sequence. Throughout this
section, we will assume either:

  1. (1)

$R$ is a Noetherian local ring and $n+d$ is a unit, or

  2. (2)

$R$ is a $\mathbb{Z}$-graded ring and $n+d$ is a unit, in which case the
elements $\psi(f_{1}),\dots,\psi(f_{n})$ are assumed to be homogeneous of
positive degree.

The characteristic assumptions imposed in $(1)$ and $(2)$ are needed because
of the definition of the scaled de Rham differential given in Lemma 5.4. In
the following definition, we show how to build a well-defined algebra
structure after restricting multidegrees.

######  Definition 5.1.

Let $(S_{\bullet})_{w}$ denote the $R$-submodule of $S_{\bullet}$ generated by
all monomials with multidegree bounded by $w$. Observe that
$(S_{\bullet})_{w}$ is not a subalgebra with respect to the ordinary
multiplication on $S_{\bullet}$. However, $(S_{\bullet})_{w}$ may be given an
associative algebra structure as follows (notice: this is not necessarily
graded):

$f^{\alpha}\cdot
f^{\beta}=\psi(f^{\alpha+\beta-\min(\alpha+\beta,w)})f^{\min(\alpha+\beta,w)}.$

Let $X_{d}^{w}$ denote the total complex of the double complex obtained by
deleting the rightmost nontrivial column of the double complex in Figure 2.
Then $X_{d}^{w}$ may be given the structure of an associative algebra with
product defined as follows:

$(f_{\sigma}\otimes f^{\alpha})(f_{\tau}\otimes
f^{\beta})=\begin{cases}0&\textrm{if}\ \alpha_{i}+\beta_{i}\geqslant w_{i},\
\textrm{and}\ i\in\sigma\cup\tau\ \textrm{for some}\ i,\\\ f_{\sigma}\wedge
f_{\tau}\otimes f^{\alpha}\cdot f^{\beta}&\textrm{otherwise},\\\ \end{cases}$

where in the above, it is understood that if $f^{\alpha}\cdot
f^{\beta}\in(S_{\geqslant d})_{w}$, then the product is $0$.

######  Example 5.2.

The product of Definition 5.1 may seem unnecessarily complicated at first
sight, but it is important to notice that the “obvious” choice may not be
well-defined. For instance, let $R=k[x_{1},x_{2}]$ and $F=Rf_{1}\oplus Rf_{2}$
with $\psi:F\to R$ induced by sending $f_{i}\mapsto x_{i}$. If $w=(1,1)$, then
the standard product on $S_{\bullet}$ does not restrict to a product on
$(S_{\bullet})_{w}$ since $f_{1}\cdot f_{1}=f_{1}^{2}\notin(S_{\bullet})_{w}$.
The product of Definition 5.1 pulls the “overflow” out as a coefficient, so
that $f_{1}\cdot f_{1}=x_{1}f_{1}\in(S_{\bullet})_{w}$.

######  Proposition 5.3.

Adopt notation and hypotheses as in Setup 3.3. With product as in Definition
5.1, the total complex $X_{d}^{w}$ is an associative DG-algebra.

###### Proof.

Let $S:=\\{i\mid\alpha_{i}+\beta_{i}\geqslant w_{i}\ \textrm{and}\
i\in\sigma\cup\tau\\}$. If $|S|=0$, then the proof of the Leibniz rule is
essentially identical to that of the Koszul complex.

If $|S|=1$, then let $\ell$ be the unique integer with
$\alpha_{\ell}+\beta_{\ell}\geqslant w_{\ell}$ and $\ell\in\sigma\cup\tau$.
Assume without loss of generality that $\ell\in\sigma$. By definition,
$(f_{\sigma}\otimes f^{\alpha})\cdot(f_{\tau}\otimes f^{\beta})=0$. On the
other hand, one computes:

$\displaystyle d(f_{\sigma}\otimes f^{\alpha})(f_{\tau}\otimes
f^{\beta})+(-1)^{|\sigma|}(f_{\sigma}\otimes f^{\alpha})d(f_{\tau}\otimes
f^{\beta})$ $\displaystyle=$
$\displaystyle\operatorname{sgn}(\ell\in\sigma)\Big{(}-\psi(f_{\ell})f_{\sigma\backslash\ell}\wedge
f_{\tau}\otimes f^{\alpha}\cdot f^{\beta}+f_{\sigma\backslash\ell}\wedge
f_{\tau}\otimes f^{\alpha+\epsilon_{\ell}}\cdot f^{\beta}\Big{)}=0.$

Finally, if $|S|>1$, then all terms appearing in the Leibniz rule still have
trivial multiplication. To conclude the proof, observe that associativity
follows by the associativity of the exterior algebra and the product defined
in Definition 5.1. ∎

######  Lemma 5.4.

Adopt notation and hypotheses as in Setup 3.3. Let $h$ denote the scaled de
Rham map

$h_{a,b}:=\frac{1}{a+b}\sum_{j=1}^{n}f_{j}\otimes\frac{\partial}{\partial
f_{j}}:\bigwedge^{a}\otimes S_{b}\to\bigwedge^{a+1}\otimes S_{b-1}.$

Then,

  1. (1)

$h^{2}=0$,

  2. (2)

$\kappa h+h\kappa=1$, and

  3. (3)

$h$ restricts to a contracting homotopy $(\bigwedge^{a}\otimes
S_{b})_{w}\to(\bigwedge^{a+1}\otimes S_{b-1})_{w}$.

###### Proof.

The proofs of $(1)$ and $(2)$ may be found in [10, Lemma 4.8], and $(3)$
follows because $h$ preserves multidegree. ∎

######  Proposition 5.5.

Adopt notation and hypotheses as in Setup 3.3 and let $h$ denote the scaled de
Rham map of Lemma 5.4. Then $h$ satisfies the generalized Leibniz rule with
respect to the product of Definition 5.1.

###### Proof.

Let $f_{\sigma}\otimes f^{\alpha}\in\bigwedge^{r}\otimes S_{a}$ and
$f_{\tau}\otimes f^{\beta}\in\bigwedge^{s}\otimes S_{b}$. If
$(f_{\sigma}\otimes f^{\alpha})(f_{\tau}\otimes f^{\beta})=0$, then there is
nothing to prove. Assume that the product is not zero and define
$T:=\\{i\mid\alpha_{i}+\beta_{i}>w_{i}\\}$. For each $i\in T$, let
$\alpha_{i}^{\prime}<\alpha_{i}$ and $\beta_{i}^{\prime}<\beta_{i}$ be such
that $\alpha_{i}^{\prime}+\beta_{i}=w_{i}$ and
$\alpha_{i}+\beta_{i}^{\prime}=w_{i}$. Observe that for any $i\in T$,

$\displaystyle(r+a)h(f_{\sigma}\otimes
f^{\alpha})(\psi(f_{i}^{\beta_{i}-\beta_{i}^{\prime}})f_{\tau}\otimes
f^{\beta-(\beta_{i}-\beta_{i}^{\prime})\epsilon_{i}})$
$\displaystyle=\sum_{j\notin T}f_{j}\wedge f_{\sigma}\wedge
f_{\tau}\otimes\frac{\partial(f^{\alpha}\cdot f^{\beta})}{\partial f_{j}}$
$\displaystyle+f_{i}\wedge f_{\sigma}\wedge
f_{\tau}\otimes\frac{\partial(f^{\alpha}\cdot f^{\beta})}{\partial
f_{i}},\quad\textrm{and}$
$\displaystyle(s+b)(\psi(f_{i}^{\alpha_{i}-\alpha_{i}^{\prime}})f_{\sigma}\otimes
f^{\alpha-(\alpha_{i}-\alpha_{i}^{\prime})\epsilon_{i}})h(f_{\tau}\otimes
f^{\beta})$ $\displaystyle=\sum_{j\notin T}f_{\sigma}\wedge f_{j}\wedge
f_{\tau}\otimes\frac{\partial(f^{\alpha}\cdot f^{\beta})}{\partial f_{j}}$
$\displaystyle+f_{\sigma}\wedge f_{i}\wedge
f_{\tau}\otimes\frac{\partial(f^{\alpha}\cdot f^{\beta})}{\partial f_{i}}.$

Using this, one computes:

$\displaystyle(r+a)h(f_{\sigma}\otimes
f^{\alpha})\big{(}(1-|T|)f_{\tau}\otimes f^{\beta}+\sum_{i\in
T}\psi(f_{i}^{\beta_{i}-\beta_{i}^{\prime}})f_{\tau}\otimes
f^{\beta-(\beta_{i}-\beta_{i}^{\prime})\epsilon_{i}}\big{)}$
$\displaystyle+(-1)^{r}(s+b)\big{(}(1-|T|)f_{\sigma}\otimes
f^{\alpha}+\sum_{i\in
T}\psi(f_{i}^{\alpha_{i}-\alpha_{i}^{\prime}})f_{\sigma}\otimes
f^{\alpha-(\alpha_{i}-\alpha_{i}^{\prime})\epsilon_{i}}\big{)}h(f_{\tau}\otimes
f^{\beta})$ $\displaystyle=$ $\displaystyle\sum_{i=1}^{n}f_{i}\wedge
f_{\sigma}\wedge f_{\tau}\otimes\frac{\partial(f^{\alpha}\cdot
f^{\beta})}{\partial f_{i}}$ $\displaystyle=$
$\displaystyle(r+s+a+b)h\big{(}(f_{\sigma}\otimes f^{\alpha})(f_{\tau}\otimes
f^{\beta})\big{)}.$

Dividing the above by $r+s+a+b$, it follows that $h$ satisfies the generalized
Leibniz rule. ∎

######  5.6.

Let us recall the method of transferring algebra structures used in [10] as
applied to our situation. The goal of this method is to find a special
deformation retract

$(X_{d}^{w},\kappa-\textrm{Kos}\otimes
1)\rightleftarrows(L^{w}(\psi,d),\textrm{Kos}\otimes 1)$

that is a perturbation of a special deformation retract, with the homotopy $h$
satisfying the generalized Leibniz rule. Let $\varepsilon$ denote any choice
of isomorphism $\bigwedge^{0}\otimes S_{0}\cong R$. In our situation,

${0}$${0}$${{(\bigwedge^{n}\otimes
S_{d-1})_{w}}}$${{L^{n-1}_{d,w}}}$${\iddots}$${\vdots}$${\vdots}$${\vdots}$${\vdots}$${\cdots}$${{(\bigwedge^{2}\otimes
S_{d-1})_{w}}}$${{L_{d,w}^{1}}}$${\cdots}$${{(\bigwedge^{1}\otimes
S_{d-1})_{w}}}$${{L^{0}_{d,w}}}$${\vdots}$${\vdots}$${\vdots}$${{(\bigwedge^{0}\otimes
S_{d-1})_{w}}}$${R}$${{(\bigwedge^{2}\otimes
S_{0})_{w}}}$${{(\bigwedge^{1}\otimes S_{1})_{w}}}$${{(\bigwedge^{0}\otimes
S_{2})_{w}}}$${{(\bigwedge^{1}\otimes S_{0})_{w}}}$${{(\bigwedge^{0}\otimes
S_{1})_{w}}}$${{(\bigwedge^{0}\otimes
S_{0})_{w}}}$$\scriptstyle{\kappa}$$\scriptstyle{\kappa}$$\scriptstyle{\kappa}$$\scriptstyle{p}$$\scriptstyle{p}$$\scriptstyle{p}$$\scriptstyle{p}$$\scriptstyle{p}$$\scriptstyle{i}$$\scriptstyle{i}$$\scriptstyle{i}$$\scriptstyle{i}$$\scriptstyle{i}$
Figure 3. The unperturbed double complex.

  1. (1)

The special deformation retract to be perturbed comes from the double complex
of Figure 3, with contracting homotopy $h$ coming from Lemma 5.4 and

$\displaystyle i$ $\displaystyle=\begin{cases}h&\textrm{on}\ L^{i}_{a,w}\\\
\varepsilon^{-1}&\textrm{on}\ R,\\\ \end{cases}$ $\displaystyle p$
$\displaystyle=\begin{cases}\kappa&\textrm{on}\ (\bigwedge^{i}\otimes
S_{a-1})_{w},\ i>0,\\\ \varepsilon&\textrm{on}\ R,\\\ 0&\textrm{otherwise}.\\\
\end{cases}$

${0}$${0}$${{(\bigwedge^{n}\otimes
S_{d-1})_{w}}}$${{L^{n-1}_{d,w}}}$${\iddots}$${\vdots}$${\vdots}$${\vdots}$${\vdots}$${{(\bigwedge^{2}\otimes
S_{d-1})_{w}}}$${{L_{d,w}^{1}}}$${\cdots}$${{(\bigwedge^{1}\otimes
S_{d-1})_{w}}}$${{L^{0}_{d,w}}}$${\vdots}$${\vdots}$${\vdots}$${{(\bigwedge^{0}\otimes
S_{d-1})_{w}}}$${R}$${{(\bigwedge^{2}\otimes
S_{0})_{w}}}$${{(\bigwedge^{1}\otimes S_{1})_{w}}}$${{(\bigwedge^{0}\otimes
S_{2})_{w}}}$${{(\bigwedge^{1}\otimes S_{0})_{w}}}$${{(\bigwedge^{0}\otimes
S_{1})_{w}}}$${{(\bigwedge^{0}\otimes
S_{0})_{w}}}$$\scriptstyle{\textrm{Kos}\otimes
1}$$\scriptstyle{\textrm{Kos}\otimes
1}$$\scriptstyle{\kappa}$$\scriptstyle{\kappa}$$\scriptstyle{\kappa}$$\scriptstyle{\textrm{Kos}\otimes
1}$$\scriptstyle{\textrm{Kos}\otimes 1}$$\scriptstyle{\textrm{Kos}\otimes
1}$$\scriptstyle{p_{\infty}}$$\scriptstyle{p_{\infty}}$$\scriptstyle{p_{\infty}}$$\scriptstyle{\textrm{Kos}\otimes
1}$$\scriptstyle{\textrm{Kos}\otimes 1}$$\scriptstyle{\textrm{Kos}\otimes
1}$$\scriptstyle{\textrm{Kos}\otimes 1}$$\scriptstyle{\textrm{Kos}\otimes
1}$$\scriptstyle{\textrm{Kos}\otimes 1}$$\scriptstyle{\textrm{Kos}\otimes
1}$$\scriptstyle{p_{\infty}}$$\scriptstyle{p_{\infty}}$$\scriptstyle{p_{\infty}}$$\scriptstyle{\psi}$$\scriptstyle{\textrm{Kos}\otimes
1}$ Figure 4. The perturbed double complex.

  2. (2)

The perturbed deformation retract comes from the double complex of Figure 4,
where the perturbation is precisely the vertical Koszul differential
$\textrm{Kos}\otimes 1$ and

$\displaystyle i_{\infty}$
$\displaystyle=\begin{cases}(1-h(\textrm{Kos}\otimes 1))^{-1}h&\textrm{on}\
L^{i}_{a,w}\\\ \varepsilon^{-1}&\textrm{on}\ R,\\\ \end{cases}$ $\displaystyle
p_{\infty}$ $\displaystyle=\begin{cases}\kappa&\textrm{on}\
(\bigwedge^{i}\otimes S_{a-1})_{w},\ i>0,\\\ \varepsilon&\textrm{on}\ R,\\\
0&\textrm{otherwise}.\\\ \end{cases}$

The fact that the perturbed differential induced on the rightmost column of
Figure 4 is precisely $\textrm{Kos}\otimes 1$ is an identical computation to
that done in $4.6$ of [10].

######  Theorem 5.7.

Adopt notation and hypotheses as in Setup 3.3. The complex $L^{w}(\psi,d)$ of
Definition 3.9 admits the structure of an associative DG-algebra. If $R$ is a
standard graded polynomial ring and $\operatorname{im}\psi=R_{+}$, then this
product is invariant under the action of the symmetric group on the variables.

###### Proof.

Define the product of the classes of $f_{\sigma}\otimes f^{\alpha}$ and
$f_{\tau}\otimes f^{\beta}\in L^{w}(\psi,d)$ via

$(f_{\sigma}\otimes f^{\alpha})\cdot(f_{\tau}\otimes
f^{\beta}):=p_{\infty}\big{(}i_{\infty}(f_{\sigma}\otimes f^{\alpha})\cdot
i_{\infty}(f_{\tau}\otimes f^{\beta})\big{)}.$

This product will yield an associative DG-algebra structure on $L^{w}(\psi,d)$
by Proposition 2.11 combined with Proposition 5.3, Lemma 5.4, Proposition 5.5,
and the discussion of 5.6. ∎

It is not difficult to see that in the case $R=k[x_{1},\dots,x_{n}]$ and
$\operatorname{im}(\psi)=(x_{1},\dots,x_{n})$ with $w=(1,1,\dots,1)$, the
complex $L^{w}(\psi,d)$ is identical to the complex constructed by Galetto in
[6]. Thus, Theorem 5.7 gives an explicit $S_{n}$-invariant algebra structure
on those complexes.

######  Remark 5.8.

One can alternatively remove the characteristic assumption on $k$ at the
expense of losing $S_{n}$-invariance by restricting the product of Srinivasan
[12] to all standard tableaux with bounded multidegree. It is straightforward
to verify that this restriction does yield a subalgebra.

## References

  * [1] Ayah Almousa and Keller VandeBogert, _Polarizations and hook partitions_ , In Preparation.
  * [2] Annetta Aramova, Jürgen Herzog, and Takayuki Hibi, _Squarefree lexsegment ideals_ , Mathematische Zeitschrift 228 (1998), no. 2, 353–378.
  * [3] Luchezar L Avramov, _Infinite free resolutions_ , Six lectures on commutative algebra, Springer, 1998, pp. 1–118.
  * [4] David A Buchsbaum and David Eisenbud, _Generic free resolutions and a family of generically perfect ideals_ , Advances in Mathematics 18 (1975), no. 3, 245–301.
  * [5] Sabine El Khoury and Andrew R Kustin, _Artinian gorenstein algebras with linear resolutions_ , Journal of Algebra 420 (2014), 402–474.
  * [6] Federico Galetto, _On the ideal generated by all squarefree monomials of a given degree_ , arXiv preprint arXiv:1609.06396 (2016).
  * [7] Vesselin Gasharov, Takayuki Hibi, and Irena Peeva, _Resolutions of a-stable ideals_ , Journal of Algebra 254 (2002), no. 2, 375–394.
  * [8] Jürgen Herzog and Rasoul Ahangari Maleki, _Koszul cycles and golod rings_ , Manuscripta Mathematica 157 (2018), no. 3, 483–495.
  * [9] Lukas Katthän, _A non-golod ring with a trivial product on its koszul homology_ , Journal of Algebra 479 (2017), 244–262.
  * [10] Claudia Miller and Hamidreza Rahmati, _Transferring algebra structures on complexes_ , arXiv preprint arXiv:2007.08040 (2020).
  * [11] Claudiu Raicu, _Regularity of sn-invariant monomial ideals_ , Journal of Combinatorial Theory, Series A 177 (2021), 105307.
  * [12] Hema Srinivasan, _Algebra structures on some canonical resolutions_ , Journal of Algebra 122 (1989), no. 1, 150–187.
  * [13] Jerzy Weyman, _Cohomology of vector bundles and syzygies_ , vol. 149, Cambridge University Press, 2003.