Thevaa Chandereng, Department of Biostatistics & Medical Informatics,
University of Wisconsin-Madison, 201 WARF, 610 Walnut Street, Madison, WI
53726.

# Robust Response-Adaptive Randomization Design

Thevaa Chandereng1,21,2affiliationmark:  and Rick
Chappell1,21,2affiliationmark:  11affiliationmark:  Department of
Biostatistics & Medical Informatics, University of Wisconsin-Madison, Madison,
WI, USA
22affiliationmark:  Department of Statistics, University of Wisconsin-Madison,
Madison, WI, USA<EMAIL_ADDRESS>
###### Abstract

In clinical trials, patients are randomized with equal probability among
treatments to obtain an unbiased estimate of the treatment effect. However,
response-adaptive randomization has been proposed due to ethical reasons,
especially in rare diseases where randomization ratio is tilted to favor the
better performing treatment. The substantial disagreement regarding time-
trends in adaptive randomization is not fully addressed. The type I error is
inflated in the traditional Bayesian adaptive randomization approach when
time-trend is present. In our approach, patients are assigned in blocks and
the randomization ratio is recomputed for blocks rather than traditional
adaptive randomization where it is done per patient. We further investigate
the design with a range of scenario for both frequentist and Bayesian design.
We compare our method with equal randomization and with different number of
blocks including traditional response- adaptive randomization design where
randomization ratio is altered patient by patient basis. The analysis is done
in stratification if there is two or more patient in each block. Having a
large number of blocks or randomizing a few subjects into a block should be
avoided due to the possibility of not acquiring any information from the
block(s). On the other hand, response-adaptive randomization with a small
number of blocks has a good balance between efficiency and treating more
subjects to the better-performing treatment.

###### keywords:

adaptive design, play the winner rule, clinical trials, group-sequential
design, Bayesian design

##  1 Introduction

###  1.1 Response-adaptive randomization

Randomization remains a pivotal methodology for advancement in medical
knowledge. It removes any systematic bias and allows direct inference between
the treatment group and outcome. Traditionally, fixed randomization scheme
(1:1 or 2:1) is used to due to simplicity in design and execution of the
trial. However, response-adaptive randomization (RAR) design utilizes accrual
information to tilt the randomization ratio to the better performing treatment
group. Patients enrolled in these trials are not only treated to obtain the
effectiveness of treatment but also treated to the best way possible
(wei1978randomized, ). Response-adaptive randomization design primarily
targets to solve both the issues mentioned above at once with a good balance.
The biasness in the trial can be greatly affected if one arm is very superior
to the other arm(s) due to a drastic alteration in the randomization ratio.

Response-adaptive randomization has been highly advocated due to ethical
reasons. The primary use of adaptive designs in clinical trials is that it
improves the benefit/risk for the patients enrolled in a trial
(hey2015outcome, ). For instance, in Zidovudine Treatment (AZT) trial
conducted to test the reduction of maternal-infant transmission of HIV type 1,
patients were randomized to 1:1 (connor1994reduction, ). The equal
randomization scheme used placed 239 women in the treatment group (AZT) and
238 patients in the placebo group. Out of the 238 women in the AZT group, 60
of the infants were transmitted with the HIV virus while only 20 infants
contracted the HIV virus for the AZT group (connor1994reduction, ). The
outcome of the trial confirmed that the new treatment works. Given the
seriousness of the outcome of this study, it is reasonable to argue that 50-50
allocation was unethical. As the outcome of the trial becomes available, the
randomization ratio should have tilted in favor of the AZT group. Response-
adaptive randomization design could have reduced the number of infants that
contracted HIV disease from their maternal.

On the other hand, opponents of RAR have argued that adaptive randomization
challenges the whole notion of equipoise begg2015ethical . Hey and Kimmelman
has also argued that most new treatments offer small improvement standard
treatment, thus they offer limited benefit and require a larger sample size
hey2015outcome . Hey and Kimmelman also suggested that equal randomization
helps reduce the trial size and length, thus it benefits future patients
rather than current patients enrolled in the trial hey2015outcome . Korn and
Friedlin measure the difference in non-responders under equal and adaptive
randomization and found that adaptive randomization required a larger trial to
achieve the same power and type I error korn2011outcome . The major drawback
to response-adaptive randomization design is that the trial needs to be short
to be able to obtain the outcome of the trial for future randomization
(karrison2003group, ).

###  1.2 Time-trend issues

However, one of the main critics against response-adaptive randomization is
the time-trend issue. This contributes to the main factor of why RAR is
infrequently used. The type I error rate is usually not controlled at the
nominal level under traditional Bayesian RAR design (thall2015statistical, ).
Besides affecting type I error, studies have shown that there is a large bias
in the estimation of treatment difference under traditional RAR
design(thall2015statistical, ). Figure 1 shows that the increase in response
in the treatment group tilts the randomization ratio in favor of the
treatment. However, this is directly confounded by time.

Figure 1: Effect of time as a blocking factor on randomization ratio

Example of time trend issues in response-adaptive randomization (RAR) design
includes (chappell2007continuous, ; karrison2003group, ):

  * •

The disease itself can change, sometimes radically (e.g., AIDS in the early
1990s).

  * •

Our definition of the disease can change due to new scientific discoveries or
diagnostic methods (e.g., stage migration in nasopharyngeal carcinoma due to
the introduction of CAT scans to Hong Kong 2005).

  * •

Inclusion criteria can change, either formally (in which case we can stratify
analysis on before vs. after the change) or informally due to “recruiting
zeal” or other issues (in which case we can’t).

  * •

Centers can change, such as when VAs enter the trial earlier or later than
academic institutions.

  * •

Patients within centers can change, especially but not only with chronic
diseases, due to the phenomenon of “A queue of desperate patients lining up at
the door”.

  * •

In addition to these examples, an investigator who wants to game the system
could cross his/her fingers that his favored treatment arm is ahead, then
progressively enroll better prognosis patients over time

In long duration trials, time-trends more likely to occur. Patient’s
characteristics might be completely different throughout the trial or even at
the beginning and end of the trial (which is also known as “patient drift”)
(karrison2003group, ). Since most of the RAR design have adaptiveness on the
fly, there is one important assumption made. “The sequence of patients who
arrive for entry into the trial represents samples drawn at random from two
homogenous populations, with no drift in the probabilities of success”
(begg2015ethical, ; karrison2003group, ). However, this assumption is usually
violated. For example, there were more smokers enrolled in the latter part of
the trial than the beginning of the trial in the Lung Cancer
Elimination(BATTLE) (liu2015overview, ).

Given the serious flaw in RAR design, there is not much literature on this
area to address the time-trend issue. Randomization test for adjusting type I
error inflation was proposed by Simon and Simon using different RAR rules for
double-arm trials (simon2011using, ). Jennison and Turnbull explored a group-
sequential method for RAR with continuous outcome utilizing multi-stage
terminology jennison1999group . Karrison et al. introduced a stratified group-
sequential method with a simple example of altering the randomization ratio to
address this issue (karrison2003group, ). Coad used a very similar stratified
analysis to Karrison et al. (coad1992comparative, ). Rosenberger et al.
introduced a covariate-adjusted RAR procedure where time mechanism can be
added as a covariate in the model (rosenberger2001covariate, ). Thall et al.
investigated type I error under a linear time-trend induced in the traditional
response-adaptive randomization design and showed that the type I error is
significantly above the nominal level (thall2015statistical, ). Villar et al.
explored the hypothesis testing procedure and adjusting for covariates for
correcting type I error inflation and the effect on power in RAR design with
time-trend effect added to the model for two-armed and multi-armed trials
(villar2018response, ).

Time-trend can not only greatly affect the biasness in the difference in
treatment effect but it can also wrongly reject a true null hypothesis. We
propose a block (group-sequential) design where the randomization ratio is
altered in a block level instead of a patient by patient basis using both
frequentist and Bayesian approach. In each block, the randomization ratio is
kept constant. The block design is similar to the stratified group design
introduced by Karrison et al. (karrison2003group, ). Then, we further study
the robustness in different block sizes using both frequentist and Bayesian
approach. We also compare these results with traditional RAR design and with
fixed (1:1) randomization.

##  2 Trial Design and Simulation

###  2.1 Block Design for RAR and Why?

In binary outcomes, events (success/failure) are observed within a short
period from the beginning of the treatment. In a block design, patients are
enrolled in a sequential manner. For instance, in two-arm design (treatment A
and B) patients are enrolled in block with sample size of $n_{Ak}$ and
$n_{Bk}$, for $k=1,2,3,...,K$, where $n_{Ak}$ and $n_{Bk}$ represent the
sample sizes in treatment group A and B in block $k$. In this design, the
randomization ratio is constant for patients within each block and the
randomization ratio is altered at the block level. Unlike traditional RAR
which alters the randomization ratio by patient basis, this method speeds up
the process of RAR trials since randomization ratio is modified for a group of
patients in the block. The initial randomization ratio is set to 1:1. Bashir
et al. have implemented two-block design where patients are randomized 1:1 in
a group of 10 and then based on the outcome, they randomized the next ten
patient to the superior treatment (bashir2012randomized, ). However, at the
second block, they randomized all the patient to the lower dosage because the
probability of randomizing patient to the lower dosage was 0.9 compared to 0.1
to the higher dosage. This design should be considered a randomized control
trial for the first block and an observational study for the second block. As
asserted by Karrison et al., the block design will eliminate bias due to drift
through stratification (karrison2003group, ). However, the optimal block size
remains unclear and it is highly dependent on the operating characteristics of
a trial. Blocks with a large number of patients also help reduce the large
probability of imbalance in the wrong direction, where more patients are
assigned to the inferior treatment compared to traditional RAR. Blocks with
fewer patients can reduce the power of the trial because if patients are not
randomized to both treatments in a block, the block becomes uninformative.

###  2.2 Simulation and Design

We investigated the effect of a different number of blocks using simulations
for both frequentist and Bayesian approach. The rules for altering the
randomization ratio in both Bayesian and frequentist design are described in
the subsection below. The target sample size of $N$ = 200 subjects were
considered with number of blocks (strata), $K$ = 1, 2, 4, 5, 10, 20, 100, 200.
When $K$ = 1, this follows the traditional equal allocation design and when
$K$ = 200, this follows the traditional RAR design where stratification is
ignored. Upon completion of enrollment and collection of data of the subjects
in each block, the interim analyses are done to revise the allocation rule.
The interim analysis also allows for early stopping and details of early
stopping for both Bayesian and frequentist design is included below. 10, 000
independent simulations were performed for each design yielding a Monte-Carlo
standard error of 0.25%.

In each simulation, the success rate of treatment A (control group), $p_{A}$
is set to 0.25. The alternatives (the success rate of treatment B, $p_{B}$) is
set to 0.25 (null case), 0.35, 0.45. On top of simulating both Bayesian and
frequentist design for $p_{A}$ and $p_{B}$ specified above, we also simulated
RAR with drift effect. To examine the effects of drift, we increased both
$p_{A}$ and $p_{B}$ linearly from their initial values to a final value of
0.25 larger. The drift was added onto each block rather than patient by
patient basis. For instance, the success rate in strata k for treatment A is
$p_{A}(k)=0.25+0.25((k-1)/K)$, where $K$ is the total number of strata. While
both the $p_{A}$ and $p_{B}$, the treatment effect remains constant throughout
the trial. For both frequentist and Bayesian design, the final analysis is
done using stratification for a number of blocks, K where $1<K<200$. For
traditional RAR and traditional RAR with 1:1 allocation ratio, the standard
analysis is used. Details of the analysis are attached in the Bayesian and
frequentist approach section.

###  2.3 Frequentist Approach

The allocation rule for the frequentist approach is altered using optimal
allocation ratio for 2-armed trial specified by Rosenberger et al
rosenberger2001optimal . The allocation probability for treatment A in stratum
$j$ is defined as

$\pi_{j,A}=\frac{\sqrt{\hat{p_{A}}}}{\sqrt{\hat{p_{A}}}+\sqrt{\hat{p_{B}}}},$

where $\hat{p_{A}}$ is the estimated success rate of treatment A and
$\hat{p_{B}}$ is the estimated success rate of treatment B up to block $j-1$.
However, the allocation probability is only altered upon observed both event
(success and no response) in both the arm. Simulation for early stopping is
also included, the alpha spending approach was incorporated to stop for early
success or failure demets1994interim . For traditional RAR and fixed
allocation, one-sided chi-square test was used to analyze the outcome.
However, for the block design, one-sided Cochran-Mantel-Haenszel test was
utilized to deal with the stratification. Yates’s correction was not applied
for both chi-square and Cochran-Mantel-Haenszel test due to the overly
conservative nature of the correction haviland1990yates . Treatment B is
proclaimed superior to treatment A if the one-sided p-value $<0.05$.

The computation of the mean proportion of treatment difference is computed as
below for stratified design. In two treatment scenario (treatment A & B),
suppose there are $K$ strata, let $p_{Ak}$ and $p_{Bk}$ be the proportion of
success in treatment A and stratum $k$. Let $n_{Ak}$ and $n_{Bk}$ be the
number of patients in treatment A and treatment B and stratum $k$. Let

$\hat{\delta_{k}}=\hat{p_{Bk}}-\hat{p_{Ak}}$

be the observed difference in proportion between treatment A and B in stratum
$k$. The observed proportion of treatment difference ($\delta$) is computed as
below,

$\hat{\delta}=\sum^{K}_{k=1}w_{k}\hat{\delta_{k}}$

where
$w_{k}=\frac{(n_{Ak}^{-1}+n_{Bk}^{-1})^{-1}}{\sum_{k=1}^{K}(n_{Ak}^{-1}+n_{Bk}^{-1})^{-1}}$
is the weight of the stratum k and $\sum_{k=1}^{K}w_{k}=1$. For non-stratified
design, where the block size $K$ = 1, 200, the difference in proportion
between treatment B and A ($\delta$) is obtained as follows,

$\hat{\delta}=\hat{p_{B}}-\hat{p_{A}}$

, where $p_{B}$ is the estimated proportion of success in treatment B and
$p_{A}$ is the estimated proportion of success in treatment A.

###  2.4 Bayesian Approach

In the Bayesian approach, the Bayesian adaptive randomization (BAR(1/2))
method introduced by Thall and Wathen is employed thall2007practical . The
probability of randomizing subjects to treatment A in stratum j is defined as

$\pi_{j,A}=\frac{(p_{A>B}(data))^{1/2}}{(p_{A>B}(data))^{1/2}+(p_{B>A}(data))^{1/2}},$

where $p_{A>B}(data)$ is the posterior probability that treatment A has a
higher success rate than treatment B and $p_{A>B}(data))=1-p_{B>A}(data))$ .
The beta-Binomial conjugate prior is used for the estimation of the posterior
probability. The posterior probability that the treatment A has a higher event
rate than treatment B is

$\begin{split}p_{A>B}(data)\sim&beta(y_{A}+a_{0},N_{A}-y_{A}+b_{0})-\\\
&beta(y_{B}+a_{0},N_{B}-y_{B}+b_{0})>0,\end{split}$

where $y_{A}$ and $y_{B}$ denote the number of events in treatment A and B,
$N_{A}$ and $N_{B}$ denote the number of subjects in treatment A and B,
$a_{0}$ and $b_{0}$ denote the prior rate parameter of the beta distribution.
Similar to Thall and Wathen’s approach, beta(0.5, 0.5) priors were assumed for
both treatment A and B thall2007practical . Since there is no closed-form
solution for the difference in beta distributed random variables, Monte Carlo
simulations were performed to estimate the posterior of the treatment
difference.

Similar to the frequentist design, simulation for the possibility of early
stopping are included. If treatment B is selected to be better than treatment
A, then if $P_{B>A}(data)>.99$, the trial is stopped early for success and if
$P_{B>A}(data)<.01$, the trial is stopped early for failure. For non-
stratified design, where $K$ = 1, 200, if the final posterior probability
$P_{B>A}(data)>.95$, then treatment B is declared superior to treatment A. The
mean estimate of the proportion of treatment difference is estimated using
Monte Carlo simulation. The mean value of $P_{B>A}(data)$ is used to estimate
the proportion of treatment difference.

For stratified design by block, Bayesian logistic regression was implemented
to estimate the posterior probability of treatment difference. The logistic
regression model is defined as below

$Logit(y_{ij})\sim\beta_{0}+\beta_{trt}x_{ij}+\sum_{j=1}^{K}\beta_{j}x_{ij},$

where $y_{ij}$ is the outcome of treatment $i$ and stratum $j$, $x_{ij}$ is
the indicator variable of treatment $i$ and stratum $j$, $\beta_{0}$ is the
intercept term, $\beta_{trt}$ is the treatment effect, $\beta_{j}$ is the
stratum effect of stratum $j$. The uninformative prior applied in the model
are as defined

$\beta_{0},\beta_{trt},\beta_{j}\sim
N(0,\sigma^{2}_{j}),\qquad\sigma_{j}^{2}\sim Inv-\chi^{2}(1,2.5).$

The posterior value of $\beta_{trt}$ is used to estimate the proportion
difference in treatment. The R package arm with quasi family and $\mu(1-\mu)$
link was used to fit Bayesian logistic regression and obtain the posterior
samples of $\beta_{trt}$ arm . The proportion difference in treatment is
estimated using the mean posterior value of $\beta_{trt}$. If difference of
proportion between B and A is estimated using $\beta_{trt}$, then treatment B
is declared superior if $E(\beta_{trt}>0)>0.95$.

##  3 Results

For all cases, the simulation was replicated for 10,000 times. The simulation
is reproduced for both Bayesian and frequentist design with and without time
trends. The number of blocks (K) choosen are as following, K = 1, 2, 4, 5, 10,
20, 100, 200. For all the simulation, the power (treatment B is superior to
treatment A), bias, probability of sample imbalance of more than 20 patients
assigned to treatment A over treatment B, the mean, 2.5% and 97.5% percentile
of difference between $N_{B}-N_{A}$ (sample size imbalance favoring treatment
B over treatment A) are reported. The power reflects the proportion of 10,00
trial that declares treatment B superior to treatment A. The type I error
(false-poitive) is simulated by setting both $p_{A},p_{B}=0.25$. The bias
computes the estimated difference in proportion of treatment estimate, bias =
$\hat{\delta}-\delta$. $\pi_{20}$, the probability of assigning 20 or more
subjects to treatment A over treatment B ($P(N_{A}-N_{B}>20)$) is included in
the simulation. Due to nontrivial possibility of assigning more patients to
the inferior arm by chance, it is vital to analyze $\pi_{2}0$. To highlight
the main advantage of RAR compared to equal randomization, the difference in
sample size between the superior arm and inferior arm is presented. Since the
difference between subjects assigned to treatment B and treatment A
($N_{B}-N_{A}$) is skewed and dispersed as illustrated in Figure 2, the mean,
2.5% and 97.5% of $N_{B}-N_{A}$ are reported.

Figure 2: Distribution of difference in number of patients assigned to
treatment B compared to treatment A ($N_{B}-N_{A}$) for a 200 patient trial
with no early stopping using both Bayesian and frequentist stratified design
with 10 number of blocks. The sample size difference is collected for 100
different simulation with $p_{A}=0.25$ and $p_{B}=0.35$. The black dashed line
included shows $\pi_{20}$.

###  3.1 Simulation with No Time-Trend

Table 1 and Table 2 displays the results of simulations for a number of
blocks, K = 1, 2, 4, 5, 10, 20, 100, 200 using the frequentist and Bayesian
approach with no drift applied and not stopping early for success or failure.
The frequentist approach (Table 1) manages to control the type I error unlike
the Bayesian design in Table 2. Even though stratified RAR design has higher
type I error, the type I error is still controlled under the nominal level
(Table 1). However, type I error is high for 4, 5, 10 and 20 number of blocks
under the Bayesian design (Table 2). Block design with 2, 4, 5 and 10 number
of blocks provides a higher power when $p_{B}=0.35,0.45$ (Table 1). In the
Bayesian design, the fixed randomization ratio provides the best power.
Although the type I error is close to the nominal level under K = 2, 4 and 5,
with a slight increase in sample size it could be lowered to the nominal
level. A small increase in sample size might still be favorable if more
subjects are treated with the best possible cure. Block design with a small
number of subjects in a block should not be considered due to low power in the
design as shown in Table 1 and 2 with 100 blocks. This poor performance is due
to the reality that some of the blocks can be noninformative if subjects in
the block are randomly assigned to the small treatment group. Ethically, this
design (2 subjects per block) places more subjects at risk without
contributing to the advancement in science.

$p_{B}$ | Block | Power | Bias | $\pi_{20}$ | $N_{B}-N_{A}$
---|---|---|---|---|---
0.25 | 1 | 0.03 | 0.00 | 0.06 | 0.00 (-28, 28)
| 2 | 0.05 | 0.00 | 0.03 | 6.97 (-22.00, 36)
| 4 | 0.05 | 0.00 | 0.02 | 11.80 (-18, 44)
| 5 | 0.05 | 0.00 | 0.01 | 13.01 (-18, 46)
| 10 | 0.05 | 0.00 | 0.01 | 14.63 (-16, 48)
| 20 | 0.05 | 0.00 | 0.01 | 15.02 (-16, 48)
| 100 | 0.05 | 0.00 | 0.01 | 15.24 (-16, 48)
| 200 | 0.03 | 0.00 | 0.01 | 15.11 (-16, 48)
0.35 | 1 | 0.33 | 0.00 | 0.07 | 0.15 (-28, 28)
| 2 | 0.45 | 0.00 | 0.02 | 9.94 (-20, 42)
| 4 | 0.45 | 0.00 | 0.01 | 15.83 (-16, 52)
| 5 | 0.46 | 0.00 | 0.01 | 16.83 (-16, 52)
| 10 | 0.44 | 0.00 | 0.01 | 19.28 (-14, 58)
| 20 | 0.42 | 0.00 | 0.01 | 19.97 (-14, 58)
| 100 | 0.20 | 0.00 | 0.01 | 20.26 (-12, 58)
| 200 | 0.35 | 0.00 | 0.01 | 20.61 (-14, 58)
0.45 | 1 | 0.85 | 0.00 | 0.07 | 0.04 (-28, 28)
| 2 | 0.91 | 0.00 | 0.01 | 14.99 (-16, 48)
| 4 | 0.90 | 0.00 | 0.00 | 23.26 (-10, 60)
| 5 | 0.91 | 0.00 | 0.00 | 25.26 (-8, 62)
| 10 | 0.90 | 0.00 | 0.00 | 28.62 (-6, 66)
| 20 | 0.88 | 0.00 | 0.00 | 29.03 (-6, 68)
| 100 | 0.47 | 0.00 | 0.00 | 29.75 (-4, 68)
| 200 | 0.84 | 0.00 | 0.00 | 29.80 (-6, 70)

Table 1: RAR using frequentist approach with no early stopping criteria and no
drift applied. The $p_{A}$ is set to 0.25 for all cases.
$Bias=\hat{\delta}-\delta$. $\pi_{20}$ denote the probability of assigning
more than 20 patients in the inferior treatment group, $P(N_{A}-N_{B}>20)$.
$N_{A}$ and $N_{B}$ denote the number of patient assigned to treatment A and
B. The mean (2.5%, 97.5%) of $N_{B}-N_{A}$ is reported in the last column.
10,000 simulation was done for each case.

The estimation of the actual treatment effect is unbiased under the
frequentist design regardless of the number of blocks used. However, the
estimation of treatment effect is biased for most stratified RAR design and
traditional RAR as shown in Table 2. Unlike the Bayesian approach (Table 2),
the variability in sample size assigned to treatment A and B is smaller in the
frequentist approach (Table 1). The mean difference in the subject’s
assignment of treatment is also smaller in the frequentist design compared to
the Bayesian design. Thus, the frequentist design is a little conservative of
assigning patients to the better-performing treatment, unlike the Bayesian
design. At the largest difference in proportion $p_{B}=0.45$, $p_{A}=0.25$,
there is still room for imbalance in the wrong direction in the frequentist
design compared to the Bayesian design (Table 1 and 2 ). The imbalance in
sample size favoring the inferior treatment is small under all scenario as
shown by $\pi_{20}$ in Table 1. The difference in sample size ($N_{B}-N_{A}$)
is relatively small and close to 0 for the frequentist design. On the other
hand, the imbalance is large for the Bayesian design as illustrated by
$\pi_{20}$ in Table 2. The large difference in sample size (more than half the
total sample size) is seen in the Bayesian design with the most extreme
difference in the outcome of treatment A and B.

Table LABEL:freqearlystopwnodrift and Table LABEL:bayesearlystopnodrift
displays the results of simulations for a number of blocks, K = 1, 2, 4, 5,
10, 20, 100, 200 using the frequentist and Bayesian approach and with early
stopping criteria for success or failure implemented. Parallel to the earlier
results, the drift is higher in Table LABEL:freqearlystopwnodrift and Table
LABEL:bayesearlystopnodrift. Table LABEL:bayesearlystopwdrift emphasizes that
K = 10, 20, 100 and 200 have a inflated type I error of 0.12, 0.16 and 0.18
similar to Table 4. As seen in type I error and power in Table
LABEL:freqearlystopwdrift and LABEL:bayesearlystopwdrift, a large number of
stratum with Bayesian design should not be considered for clinical studies.

$p_{B}$ | Block | Power | Bias | $\pi_{20}$ | $N_{B}-N_{A}$
---|---|---|---|---|---
0.25 | 1 | 0.05 | 0.00 | 0.07 | -0.09 (-28, 28)
| 2 | 0.06 | 0.00 | 0.31 | -0.01 (-78, 76)
| 4 | 0.08 | 0.00 | 0.36 | -1.13 (-106, 104)
| 5 | 0.08 | 0.00 | 0.36 | -0.14 (-112, 110)
| 10 | 0.13 | 0.00 | 0.37 | 0.63 (-124, 124)
| 20 | 0.14 | 0.01 | 0.39 | -1.48 (-132, 128)
| 100 | 0.01 | 0.00 | 0.38 | 0.58 (-132, 132)
| 200 | 0.07 | 0.00 | 0.39 | -1.05 (-132, 132)
0.35 | 1 | 0.46 | 0.00 | 0.07 | -0.06 (-28, 28)
| 2 | 0.46 | 0.01 | 0.06 | 39.73 (-38, 98)
| 4 | 0.46 | 0.01 | 0.07 | 57.26 (-48, 136)
| 5 | 0.46 | 0.01 | 0.07 | 60.77 (-50, 144)
| 10 | 0.50 | 0.02 | 0.07 | 66.64 (-56, 156)
| 20 | 0.58 | 0.01 | 0.07 | 69.57 (-62, 162)
| 100 | 0.08 | -0.08 | 0.07 | 71.26 (-64, 164)
| 200 | 0.44 | 0.01 | 0.07 | 72.48 (-60, 164)
0.45 | 1 | 0.91 | 0.00 | 0.07 | -0.03 (-28, 28)
| 2 | 0.89 | 0.01 | 0.01 | 69.77 (2, 110)
| 4 | 0.87 | 0.03 | 0.01 | 99.58 (12, 150)
| 5 | 0.86 | 0.02 | 0.01 | 103.36 (14, 158)
| 10 | 0.86 | 0.03 | 0.00 | 113.34 (22, 170)
| 20 | 0.88 | 0.02 | 0.01 | 117.73 (26, 176)
| 100 | 0.29 | -0.12 | 0.01 | 119.93 (24, 178)
| 200 | 0.86 | 0.02 | 0.00 | 120.38 (24, 176)

Table 2: RAR using Bayesian approach with no early stopping criteria and with
no drift applied. The $p_{A}$ is set to 0.25 for all cases.
$Bias=\hat{\delta}-\delta$. $\pi_{20}$ denote the probability of assigning
more than 20 patients in the inferior treatment group. $N_{A}$ and $N_{B}$
denote the number of patient assigned to treatment A and B. The mean (2.5%,
97.5%) of $N_{B}-N_{A}$ is reported in the last column. 10,000 simulation was
done for each case.

###  3.2 Simulation with Time-Trend

Table 3 and Table 4 displays the results of simulation for number of blocks, K
= 1, 2, 4, 5, 10, 20, 100, 200 using the frequentist and Bayesian approach
with 0.25 drift applied and not stopping early for success or failure. With
time drift, the false positive rate is still under the nominal level for the
frequentist design with all block sizes as seen in Table 3. However, the type
I error is inflated for traditional RAR, 10 and 20 blocks in the Bayesian
design. Having a 2, 4, 5, 10 and 20 blocks still remain the most powerful
design with the most extreme difference in outcome of $p_{A}$ and $p_{B}$. In
the Bayesian design, block size with 50 subjects is still comparable to the
traditional fixed randomization ratio design, since the type I error is
controlled under 0.05 and it has the highest power of 0.91 under the maximum
difference between $p_{A}$ and $p_{B}$. Large number of blocks still remains
as a poor design as illustrated earlier in Table 2. The estimatation of
treatment difference remains the similar to the simulation with no time-trend
except the bias is a little higher under traditional RAR design as presented
in Table 4. The difference in sample size, the imbalance in the wrong
direction and the variability between the arms ($N_{B}-N_{A}$) remains
comparable to the simulation without time drift applied.

Table LABEL:freqearlystopwdrift and Table LABEL:bayesearlystopwdrift displays
the results of simulation for number of blocks, K = 1, 2, 4, 5, 10, 20, 100,
200 using the frequentist and Bayesian approach and with early stopping
criteria for success or failure implemented. The output in Table
LABEL:freqearlystopwdrift is similar to Table 3 except the treatment different
is slightly biased. However, it is shown that clinical trials with early
stopping are slightly biased compared to trials without early stopping
criteria imposed. Table LABEL:bayesearlystopwdrift emphasizes that K = 10, 20,
100 and 200 have a inflated type I error of 0.14, 0.17 and 0.11 similar to
Table 4. As seen in type I error and power in Table 4 and
LABEL:bayesearlystopwdrift, large number of stratum design should not be
considered for clinical studies.

$p_{B}$ | Block | Power | Bias | $\pi_{20}$ | $N_{B}-N_{A}$
---|---|---|---|---|---
0.25 | 1 | 0.03 | 0.00 | 0.07 | 0.00 (-28, 28)
| 2 | 0.05 | 0.00 | 0.04 | 5.28 (-24, 34.00)
| 4 | 0.05 | 0.00 | 0.02 | 9.56 (-20, 40)
| 5 | 0.05 | 0.00 | 0.02 | 10.81(-20, 42)
| 10 | 0.05 | 0.00 | 0.01 | 12.90 (-16, 44)
| 20 | 0.05 | 0.00 | 0.01 | 13.21 (-16, 46)
| 100 | 0.05 | 0.00 | 0.01 | 13.28 (-16, 44)
| 200 | 0.03 | 0.00 | 0.01 | 13.23 (-16, 46)
0.35 | 1 | 0.30 | 0.00 | 0.07 | 0.06 (-28, 28)
| 2 | 0.40 | 0.00 | 0.03 | 7.09 (-22, 38)
| 4 | 0.41 | 0.00 | 0.02 | 12.16 (-18, 44)
| 5 | 0.41 | 0.00 | 0.02 | 13.55 (-18, 48)
| 10 | 0.40 | 0.00 | 0.01 | 16.36 (-14, 52)
| 20 | 0.38 | 0.00 | 0.01 | 16.92 (-14, 50)
| 100 | 0.18 | 0.00 | 0.01 | 17.30 (-14, 52)
| 200 | 0.29 | 0.00 | 0.01 | 17.61 (-14, 52)
0.45 | 1 | 0.82 | 0.00 | 0.07 | -0.07 (-28, 28)
| 2 | 0.89 | 0.00 | 0.02 | 11.02 (-20, 40)
| 4 | 0.88 | 0.00 | 0.01 | 18.41 (-14, 52)
| 5 | 0.88 | 0.00 | 0.01 | 20.40 (-12, 56)
| 10 | 0.88 | 0.00 | 0.00 | 23.70 (-10, 58)
| 20 | 0.85 | 0.00 | 0.00 | 24.32 (-8, 60)
| 100 | 0.46 | 0.00 | 0.00 | 25.02 (-8, 60)
| 200 | 0.80 | 0.00 | 0.00 | 25.31(-8, 62)

Table 3: RAR using frequentist approach with no early stopping criteria and
with 0.25 drift applied. The $p_{A}$ is set to 0.25 for all cases.
$Bias=\hat{\delta}-\delta$. $\pi_{20}$ denote the probability of assigning
more than 20 patients in the inferior treatment group. $N_{A}$ and $N_{B}$
denote the number of patient assigned to treatment A and B. The mean (2.5%,
97.5%) of $N_{B}-N_{A}$ is reported in the last column. 10,000 simulation was
done for each case.

$p_{B}$ | Block | Power | Bias | $\pi_{20}$ | $N_{B}-N_{A}$
---|---|---|---|---|---
0.25 | 1 | 0.03 | -0.01 | 0.03 | 0.44 (-21, 28)
| 2 | 0.07 | 0.00 | 0.29 | -0.30 (-80, 77)
| 4 | 0.05 | -0.01 | 0.41 | -5.84 (-96, 86)
| 5 | 0.06 | 0.00 | 0.43 | -1.66 (-117, 113)
| 10 | 0.12 | 0.01 | 0.34 | 3.32 (-148, 148)
| 20 | 0.20 | 0.02 | 0.33 | 12.40 (-137, 140)
| 100 | 0.01 | 0.00 | 0.36 | 4.18 (-130, 141)
| 200 | 0.17 | 0.02 | 0.32 | 12.74 (-106, 138)
0.35 | 1 | 0.51 | 0.00 | 0.10 | -1.74 (-32, 26)
| 2 | 0.42 | 0.01 | 0.06 | 38.02 (-35, 97)
| 4 | 0.35 | 0.01 | 0.07 | 52.78 (-87, 140)
| 5 | 0.46 | 0.03 | 0.06 | 68.38 (-48, 148)
| 10 | 0.45 | 0.03 | 0.03 | 72.88 (-34, 149)
| 20 | 0.51 | 0.02 | 0.11 | 73.24 (-50, 162)
| 100 | 0.11 | -0.07 | 0.11 | 74.40 (-80, 162)
| 200 | 0.59 | 0.05 | 0.07 | 75.92 (-60, 162)
0.45 | 1 | 0.91 | 0.00 | 0.08 | -0.10 (-35, 28)
| 2 | 0.93 | 0.02 | 0.00 | 68.28 (20, 111)
| 4 | 0.91 | 0.04 | 0.01 | 105.34 (26, 148)
| 5 | 0.79 | 0.03 | 0.01 | 105.96 (5, 160)
| 10 | 0.82 | 0.04 | 0.00 | 119.52 (27, 173)
| 20 | 0.75 | 0.03 | 0.01 | 119.74 (13, 177)
| 100 | 0.22 | -0.13 | 0.00 | 123.12 (34, 175)
| 200 | 0.89 | 0.06 | 0.00 | 125.48 (54, 179)

Table 4: RAR using Bayesian approach with no early stopping criteria and with
drift applied.The $p_{A}$ is set to 0.25 for all cases.
$Bias=\hat{\delta}-\delta$. $\pi_{20}$ denote the probability of assigning
more than 20 patients in the inferior treatment group. $N_{A}$ and $N_{B}$
denote the number of patient assigned to treatment A and B. The mean (2.5%,
97.5%) of $N_{B}-N_{A}$ is reported in the last column. 10,000 simulation was
done for each case.

##  4 Conclusion

RAR design has a lot of appealing properties mainly assigning more patients to
better-performing treatment. However, trialist needs to be careful with issues
of time-trend and methods to alter the randomization ratio. Time-trends can
significantly impact the type I error rate which can affect the validity of
clinical studies. As a statistician, we cannot emphasize more on the
importance of controlling the false-positive rate in a clinical setting. Thall
et al. have shown that methods that are able to control type-I error have
failed to detect the true treatment difference (thall2015statistical, ).

Besides controlling for false positive rate, trialist needs to make sure the
method of altering the randomization ratio is not too extreme which can
greatly affect the bias and question the notion of randomization in clinical
studies. Thall et al. have emphasized on the difference in simulation between
BAR(1) and BAR(1/2), with BAR(1) having a large imbalance in the wrong
direction and larger false-positive rate. Zelen’s play-the-winner rule was
implemented to an extracorporeal membrane oxygenation (ECMO) trial where the
first patient was assigned to both control and treatment group (zelen1969play,
; mike1993neonatal, ). Due to the failure in the control and success in the
treatment group, all subsequent patient’s were randomized to the treatment
group (mike1993neonatal, ). However, it was later discovered that the first
patient assigned to the control group was much sicker than all the patients
randomized to the treatment groups.

On the other hand, scientists need to be aware of the risk that RAR poses
which includes having assigned more patients to the inferior treatment. As
highlight by Thall et al., “The practical and ethical point is that AR may
behave pathologically in that it carries a nontrivial risk of creating a large
sample size imbalance in favor of the inferior treatment”
(thall2015statistical, ). Large imbalance in the wrong direction can also be
controlled by methods that do not alter the randomization ratio rapidly.

It is shown that a small number of blocks (K = 2, 4 and 5) has a good tradeoff
between efficiency and ethically treating patients to the best known superior
treatment. A large number of blocks should be clearly avoided for both ethical
reason and poor design. Traditional RAR does not only delay the trial but also
affects the clinical conclusion achieved. We have not considered the multiple
treatment design with more than 2-arms. The design would be much more complex
and it should be examined further.

An R package (blockRAR), for the frequentist and Bayesian models, is
implemented in R and released as open source software under the MIT license.
The blockRAR package is available at Comprehensive R Archive Network (CRAN)
and at https://thevaachandereng.github.io/blockRAR/. We used blockRAR version
1.0.0 for all analyses.

Supplementary material are available.

None declared.

None declared.

We are grateful to Tom Cook for his helpful comments and suggestions.

## References

  * (1) Wei L and Durham S.  The randomized play-the-winner rule in medical trials.  _Journal of the American Statistical Association_ 1978; 73(364): 840–843.
  * (2) Hey SP and Kimmelman J.  Are outcome-adaptive allocation trials ethical?  _Clinical trials_ 2015; 12(2): 102–106.
  * (3) Connor EM, Sperling RS, Gelber R et al.  Reduction of maternal-infant transmission of human immunodeficiency virus type 1 with zidovudine treatment.  _New England Journal of Medicine_ 1994; 331(18): 1173–1180.
  * (4) Begg CB.  Ethical concerns about adaptive randomization, 2015.
  * (5) Korn EL and Freidlin B.  Outcome-adaptive randomization: is it useful?  _Journal of Clinical Oncology_ 2011; 29(6): 771.
  * (6) Karrison TG, Huo D and Chappell R.  A group sequential, response-adaptive design for randomized clinical trials.  _Controlled Clinical Trials_ 2003; 24(5): 506–522.
  * (7) Thall P, Fox P and Wathen J.  Statistical controversies in clinical research: scientific and ethical problems with adaptive randomization in comparative clinical trials.  _Annals of Oncology_ 2015; 26(8): 1621–1628.
  * (8) Chappell R and Karrison T.  Continuous bayesian adaptive randomization based on event times with covariates by cheung et al., statistics in medicine 2006; 25: 55–70.  _Statistics in medicine_ 2007; 26(15): 3050–3052.
  * (9) Liu S and Lee JJ.  An overview of the design and conduct of the battle trials.  _Chinese clinical oncology_ 2015; 4(3).
  * (10) Simon R and Simon NR.  Using randomization tests to preserve type i error with response adaptive and covariate adaptive randomization.  _Statistics & probability letters_ 2011; 81(7): 767–772.
  * (11) Jennison C and Turnbull BW.  _Group sequential methods with applications to clinical trials_.  Chapman and Hall/CRC, 1999.
  * (12) Coad D.  A comparative study of some data-dependent allocation rules for bernoulli data.  _Journal of statistical computation and simulation_ 1992; 40(3-4): 219–231.
  * (13) Rosenberger WF, Vidyashankar A and Agarwal DK.  Covariate-adjusted response-adaptive designs for binary response.  _Journal of biopharmaceutical statistics_ 2001; 11(4): 227–236.
  * (14) Villar SS, Bowden J and Wason J.  Response-adaptive designs for binary responses: How to offer patient benefit while being robust to time trends?  _Pharmaceutical statistics_ 2018; 17(2): 182–197.
  * (15) Bashir Q, Munsell MF, Giralt S et al.  Randomized phase ii trial comparing two dose levels of thymoglobulin in patients undergoing unrelated donor hematopoietic cell transplant.  _Leukemia & lymphoma_ 2012; 53(5): 915–919.
  * (16) Rosenberger WF, Stallard N, Ivanova A et al.  Optimal adaptive designs for binary response trials.  _Biometrics_ 2001; 57(3): 909–913.
  * (17) Demets DL and Lan KG.  Interim analysis: the alpha spending function approach.  _Statistics in medicine_ 1994; 13(13-14): 1341–1352.
  * (18) Haviland MG.  Yates’s correction for continuity and the analysis of 2$\times$ 2 contingency tables.  _Statistics in medicine_ 1990; 9(4): 363–367.
  * (19) Thall PF and Wathen JK.  Practical bayesian adaptive randomisation in clinical trials.  _European Journal of Cancer_ 2007; 43(5): 859–866.
  * (20) Gelman A and Su YS.  _arm: Data Analysis Using Regression and Multilevel/Hierarchical Models_ , 2018.  URL https://CRAN.R-project.org/package=arm.  R package version 1.10-1.
  * (21) Zelen M.  Play the winner rule and the controlled clinical trial.  _Journal of the American Statistical Association_ 1969; 64(325): 131–146.
  * (22) Mike V, Krauss AN and Ross GS.  Neonatal extracorporeal membrane oxygenation (ecmo): clinical trials and the ethics of evidence.  _Journal of medical ethics_ 1993; 19(4): 212–218.


# Representations of flat virtual braids by automorphisms of free group

Bogdan Chuzhinov  Novosibirsk State University, 630090 Novosibirsk, Russia
<EMAIL_ADDRESS>and  Andrey Vesnin  Sobolev Institute of Mathematics,
630090 Novosibirsk, Russia; Novosibirsk State University, 630090 Novosibirsk,
Russia<EMAIL_ADDRESS>
###### Abstract.

Representations of braid group $B_{n}$ on $n\geq 2$ strands by automorphisms
of a free group of rank $n$ go back to Artin (1925). In 1991 Kauffman
introduced a theory of virtual braids and virtual knots and links. The virtual
braid group $VB_{n}$ on $n\geq 2$ strands is an extension of the classical
braid group $B_{n}$ by the symmetric group $S_{n}$. In this paper we consider
flat virtual braid groups $FVB_{n}$ on $n\geq 2$ strands and construct a
family of representations of $FVB_{n}$ by automorphisms of free groups of rank
$2n$. It has been established that these representations do not preserve the
forbidden relations between classical and virtual generators. We investigated
some algebraic properties of the constructed representations. In particular,
we established conditions of faithfulness in case $n=2$ and proved that the
kernel contains a free group of rank two for $n\geq 3$.

###### Key words and phrases:

braid, virtual braid, flat virtual braid group, automorphism of free group

###### 2000 Mathematics Subject Classification:

57K12

A.V.’s work was carried out in the framework of the State Task to the Sobolev
Institute of Mathematics (Project FWNF-2022-0004)

##  1\. Introduction

The foundations of the braid groups theory were laid down in the works of E.
Artin in the 1920s. In [1] he defined the braid group $B_{n}$ on $n\geq 2$
strands as a group with generators $\sigma_{1},\ldots,\sigma_{n-1}$ and
defining relations:

(1) $\displaystyle\sigma_{i}\sigma_{j}$ $\displaystyle=\sigma_{j}\sigma_{i},$
$\displaystyle|i-j|\geq 2,$ (2)
$\displaystyle\sigma_{i}\sigma_{i+1}\sigma_{i}$
$\displaystyle=\sigma_{i+1}\sigma_{i}\sigma_{i+1},$ $\displaystyle 1\leq i\leq
n-2.$

A set $\\{\sigma_{1},\ldots,\sigma_{n-1}\\}$ is called the standard
generators, or the Artin generators of the braid group $B_{n}$. The generator
$\sigma_{i}\in B_{n}$ and its inverse $\sigma_{i}^{-1}$ are presented
geometrically in Figure 1.

$\cdots$$\cdots$$1$$i$$i+1$$n$$\sigma_{i}$$\cdots$$\cdots$$1$$i$$i+1$$n$$\sigma^{-1}_{i}$
Figure 1. Generator $\sigma_{i}\in B_{n}$ and its inverse $\sigma_{i}^{-1}$.

There is very nice relation between braid groups and knot theory based on
Alexender’s theorem and Markov’s theorem [2]. Invariants, arising from
representations of braid groups, play an important role in classical knot
theory and its generalizations.

Artin discovered faithful representation $\varphi_{n}\colon
B_{n}\to{\rm{Aut(\mathbb{F}}}_{n})$, where ${\mathbb{F}}_{n}=\langle
x_{1},\ldots,x_{n}\rangle$ – is the free group of rank $n$. Homomorphism
$\varphi_{n}$ maps generator $\sigma_{i}\in B_{n}$ to the following
automorphism $\varphi_{n}(\sigma_{i})$:

$\displaystyle\varphi_{n}(\sigma_{i}):\begin{cases}x_{i}\mapsto x_{i+1},\\\
x_{i+1}\mapsto x_{i+1}^{-1}x_{i}x_{i+1},\\\ x_{j}\mapsto x_{j},\qquad j\neq
i,i+1.\end{cases}$

Note, that for each $i$, $1\leq i\leq n-1$ one has
$\varphi_{n}(\sigma_{i})(x_{1}\cdots x_{n})=x_{1}\cdots x_{n}$. Therefore
$\varphi_{n}(\beta)(x_{1}\cdots x_{n})=(x_{1}\cdots x_{n})$ for every
$\beta\in B_{n}$. Moreover, it is shown by Artin [1, 3] that an automorphism
$g\in\operatorname{Aut}(\mathbb{F}_{n})$ is equal to $\varphi_{n}(\beta)$ for
some $\beta\in B_{n}$ if and only if the following two conditions are
satisfied: (i) $f(x_{i})$ is conjugate of some $x_{j}$ for $i=1,\ldots,n-1$;
and (ii) $f(x_{1}\cdots x_{n})=x_{1}\cdots x_{n}$.

In what follows, if any automorphism acts on a generator identically, we will
not write this action. We write the composition of automorphisms in the order
of their application from left to right, namely,
$\varphi\psi(f)=\psi(\varphi(f))$.

Virtual braids were introduced by Kauffman in his founding paper [4] together
with virtual knots and links, see also [5]. In the same paper he defined the
virtual braid group $VB_{n}$ on $n\geq 2$ strands, generated by the elements
$\sigma_{1},\ldots,\sigma_{n-1}$ similarly to the classical braid group and
$\rho_{1},\ldots\rho_{n-1}$ that satisfy _skew relations_ (1) – (2),
_substitution group relations_ (3) – (5) and _mixed relations_ (6) – (7):

(3) $\displaystyle\rho_{i}^{2}$ $\displaystyle=1,$ $\displaystyle 1\leq i\leq
n-1,$ (4) $\displaystyle\rho_{i}\rho_{j}$ $\displaystyle=\rho_{j}\rho_{i},$
$\displaystyle|i-j|\geq 2,$ (5) $\displaystyle\rho_{i}\rho_{i+1}\rho_{i}$
$\displaystyle=\rho_{i+1}\rho_{i}\rho_{i+1},$ $\displaystyle 1\leq i\leq n-2,$
(6) $\displaystyle\rho_{i}\sigma_{j}$ $\displaystyle=\sigma_{j}\rho_{i},$
$\displaystyle|i-j|\geq 2,$ (7) $\displaystyle\rho_{i}\rho_{i+1}\sigma_{i}$
$\displaystyle=\sigma_{i+1}\rho_{i}\rho_{i+1},$ $\displaystyle 1\leq i\leq
n-2.$

Generator $\rho_{i}\in VB_{n}$ is presented geometrically in Figure 2.

$\cdots$$\cdots$$1$$i$$i+1$$n$$\rho_{i}$ Figure 2. Generator $\rho_{i}\in
VB_{n}$.

Geometric braids corresponding to the mixed relation (7) are presented in
Figure 3.

$i$$i+1$$i+2$$\rho_{i}\rho_{i+1}\sigma_{i}$$=$$i$$i+1$$i+2$$\sigma_{i+1}\rho_{i}\rho_{i+1}$
Figure 3. The mixed relation
$\rho_{i}\rho_{i+1}\sigma_{i}=\sigma_{i+1}\rho_{i}\rho_{i+1}$.

S. Kamada [6] established that the following _Alexander’s theorem for virtual
braids_ holds: If $K$ is a virtual link, then for some $n$ there exists a
virtual braid $\beta\in VB_{n}$ such that $K$ is the closure of $\beta$.

It is known as shown in [7] that relations

(8)
$\displaystyle\rho_{i}\sigma_{i+1}\sigma_{i}=\sigma_{i+1}\sigma_{i}\rho_{i+1},\qquad\qquad
1\leq i\leq n-2,$ (9)
$\displaystyle\rho_{i+1}\sigma_{i}\sigma_{i+1}=\sigma_{i}\sigma_{i+1}\rho_{i},\qquad\qquad
1\leq i\leq n-2.$

do not hold in the $VB_{n}$ group. These relations (8) – (9) are called
_forbidden relations_ , see Figures 4 and 5. The group $WB_{n}$ is obtained
from $VB_{n}$ by adding the relation (8) and is called _welded braid group_
[8]. The same group $WB_{n}$ is obtained by adding the relation (9) to the
group $VB_{n}$. Adding both relations (8) and (9) to $VB_{n}$ leads to
unknotting transformations for virtual knots and links [7, 9, 6]. Other
unknotting operations for links, virtual links and welded links are given, for
example, in [10, 11, 12]. Note that the representations
$VB_{n}\to\rm{Aut}(G_{n})$ were constructed, for example, for groups $G_{n}$
of the following form: $G_{n}=\mathbb{F}_{n}*\mathbb{Z}^{n+1}$ in [13],
$G_{n}=\mathbb{F}_{n}*\mathbb{Z}^{2}$ in [14],
$G_{n}=\mathbb{F}_{n}*\mathbb{Z}^{2n+1}$ and
$G_{n}=\mathbb{F}_{n}*\mathbb{Z}^{n}$ in [15]. For structural properties and
other representations of the virtual braid groups see [16, 17].

$i$$i+1$$i+2$$\rho_{i}\sigma_{i+1}\sigma_{i}$$=$$i$$i+1$$i+2$$\sigma_{i+1}\sigma_{i}\rho_{i+1}$
Figure 4. The forbidden relation
$\rho_{i}\sigma_{i+1}\sigma_{i}=\sigma_{i+1}\sigma_{i}\rho_{i+1}$.
$i$$i+1$$i+2$$\rho_{i+1}\sigma_{i}\sigma_{i+1}$$=$$i$$i+1$$i+2$$\sigma_{i}\sigma_{i+1}\rho_{i}$
Figure 5. The forbidden relation
$\rho_{i+1}\sigma_{i}\sigma_{i+1}=\sigma_{i}\sigma_{i+1}\rho_{i}$.

In [18] the _group of flat virtual braids_ $FVB_{n}$ on $n$ strands was
introduced as a result of adding the relations (10) to the group $VB_{n}$:

(10) $\sigma_{i}^{2}=1,\qquad 1\leq i\leq n-1.$

We summarize the above discussions in the following definition.

######  Definition 1.

For $n\geq 2$ a group with generators $\sigma_{1},\ldots,\sigma_{n-1}$,
$\rho_{1},\ldots,\rho_{n-1}$ and the following defining relations:

$\begin{array}[]{ccl}\sigma_{i}^{2}=1,&\rho_{i}^{2}=1,&1\leq i\leq n-1,\\\
\sigma_{i}\,\sigma_{i+1}\,\sigma_{i}=\sigma_{i+1}\,\sigma_{i}\,\sigma_{i+1},&\rho_{i}\,\rho_{i+1}\,\rho_{i}=\rho_{i+1}\,\rho_{i}\,\rho_{i+1},&1\leq
i\leq n-2,\\\
\sigma_{i}\,\sigma_{j}=\sigma_{j}\,\sigma_{i},&\rho_{i}\,\rho_{j}=\rho_{j}\,\rho_{i},&|i-j|\geq
2\end{array}$

and

$\begin{array}[]{cl}\rho_{i}\,\rho_{i+1}\,\sigma_{i}=\sigma_{i+1}\,\rho_{i}\,\rho_{i+1},&1\leq
i\leq n-2,\\\ \rho_{i}\,\sigma_{j}=\sigma_{j}\,\rho_{i},&|i-j|\geq
2.\end{array}$

is called the _flat virtual braid group $FVB_{n}$ on $n$ strands_.

Generator $\sigma_{i}\in FVB_{n}$ is presented geometrically in Figure 6 and
generator $\rho\in FVB_{n}$ is presented geometrically in Figure 2.

$\cdots$$\cdots$$1$$i$$i+1$$n$$\sigma_{i}$ Figure 6. Generator $\sigma_{i}\in
FVB_{n}$.

In [19] the following problem was formulated: Does it exist a representation
of the $FVB_{n}$ group by automorphisms of some group for which the forbidden
relations would not hold? In [20] such representation $\eta_{n}\colon
FVB_{n}\to{\rm Aut}(\mathbb{F}_{2n})$ was constructed, here
${\mathbb{F}}_{2n}=\langle x_{1},\ldots,x_{n},y_{1},\ldots,y_{n}\rangle$ is a
free group of rank $2n$. The homomorphism $\eta_{n}$ maps generators
$\sigma_{i},\rho_{i}\in FVB_{n}$, where $i=1,\ldots,n-1$, to the following
automorphisms:

(11) $\displaystyle\eta_{n}(\sigma_{i}):\begin{cases}x_{i}\mapsto
x_{i+1}y_{i+1},\\\ x_{i+1}\mapsto
x_{i}y_{i+1}^{-1},\end{cases}\quad\eta_{n}(\rho_{i}):\begin{cases}x_{i}\mapsto
x_{i+1},\\\ x_{i+1}\mapsto x_{i},\\\ y_{i}\mapsto y_{i+1},\\\ y_{i+1}\mapsto
y_{i}.\end{cases}$

It was shown in [20] that the representation
$\eta_{2}:FVB_{2}\to{\rm{Aut(\mathbb{F}}}_{4})$ is faithful.

In this paper we construct a family of representations of the $FVB_{n}$ group
by automorphisms of the free group $\mathbb{F}_{2n}=\langle
x_{1},\ldots,x_{n},y_{1},\ldots,y_{n}\rangle$, which generalize the
representation (11). Namely, we consider a family of homomorphisms
$\Theta_{n}:FVB_{n}\to{\rm{Aut(\mathbb{F}}}_{2n})$, which are given by mapping
generators $\sigma_{i},\rho_{i}\in FVB_{n}$, where $i=1,\ldots,n-1$, to the
following automorphisms:

(12) $\Theta_{n}(\sigma_{i}):\begin{cases}x_{i}\mapsto
x_{i+1}\,a_{i}(y_{i},y_{i+1}),\\\ x_{i+1}\mapsto
x_{i}\,b_{i}(y_{i},y_{i+1}),\end{cases}\quad\Theta_{n}(\rho_{i}):\begin{cases}x_{i}\mapsto
x_{i+1}\,c_{i}(y_{i},y_{i+1}),\\\ x_{i+1}\mapsto
x_{i}\,d_{i}(y_{i},y_{i+1}),\\\ y_{i}\mapsto y_{i+1},\\\ y_{i+1}\mapsto
y_{i},\end{cases}$

where the elements $a_{i}(y_{i},y_{i+1})$, $b_{i}(y_{i},y_{i+1})$,
$c_{i}(y_{i},y_{i+1})$ and $d_{i}(y_{i},y_{i+1})$ are words in a free group of
rank two with generators $\\{y_{i},y_{i+1}\\}$ for each $i=1,\ldots,n-1$.
Thus, the homomorphisms $\Theta_{n}$ depend only on the choice of the words
$a_{i},b_{i},c_{i},d_{i}$, which define the locally nontrivial action of the
automorphism corresponding to the generator of the group $FVB_{n}$, and in
this sense the homomorphisms $\Theta_{n}$ are _local homomorphisms_.

The article has the following structure. In Theorem 1 we establish for which
$a_{i}$, $b_{i}$, $c_{i}$ and $d_{i}$ there exists a local homomorphism
$\Theta_{n}$ of the group $FVB_{n}$ into the automorphism group of the free
group $\mathbb{F}_{2n}$. In Section 3 we obtain results about the structure of
the kernel of the homomorphism $\Theta_{n}$, in particular, in Theorem 3 we
describe the kernel of this homomorphism for $n=2$. In Theorem 4 it will be
established that for $n\geq 3$ the kernel of the homomorphism $\Theta_{n}$
contains a free group of rank 2. We note it was shown earlier in [20] that for
$n\geq 3$ the kernel of the homomorphism $\eta_{n}$, which is a special case
of $\Theta_{n}$, contains an infinite cyclic group.

##  2\. Existence of local representations

Let $\mathbb{F}_{2n}$ be a free group of rank $2n$ with free generators
$x_{1},\ldots,x_{n}$, $y_{1},\ldots,y_{n}$.

######  Theorem 1.

Let $n\geq 2$ and $a_{i}(y_{i},y_{i+1})$, $b_{i}(y_{i},y_{i+1})$,
$c_{i}(y_{i},y_{i+1})$, $d_{i}(y_{i},y_{i+1})$ are words in a free group of
rank two with generators $\\{y_{i},y_{i+1}\\}$, where $1\leq i\leq n-1$.
Define the map $\Theta_{n}:FVB_{n}\to{\rm{Aut(\mathbb{F}}}_{2n})$ by mapping
$\sigma_{i}$ and $\rho_{i}$ to automorphisms:

(13) $\Theta_{n}(\sigma_{i}):\begin{cases}x_{i}\mapsto
x_{i+1}\,a_{i}(y_{i},y_{i+1}),\\\ x_{i+1}\mapsto
x_{i}\,b_{i}(y_{i},y_{i+1}),\end{cases}\quad\Theta_{n}(\rho_{i}):\begin{cases}x_{i}\mapsto
x_{i+1}\,c_{i}(y_{i},y_{i+1}),\\\ x_{i+1}\mapsto
x_{i}\,d_{i}(y_{i},y_{i+1}),\\\ y_{i}\mapsto y_{i+1},\\\ y_{i+1}\mapsto
y_{i}.\end{cases}$

Then the following properties hold.

  * (1)

The map $\Theta_{n}$ is homomorphism iff

$b_{i}(y_{i},y_{i+1})=a_{i}^{-1}(y_{i},y_{i+1}),\qquad
c_{i}(y_{i},y_{i+1})=y_{i+1}^{m_{i}},\qquad
d_{i}(y_{i},y_{i+1})=y_{i}^{-m_{i}},$

where $m_{i}\in\mathbb{Z}$ for $1\leq i\leq n-1$, and

$\displaystyle a_{j}(y_{j},y_{j+1})$
$\displaystyle=y_{j+1}^{m_{j}}\,a_{j-1}(y_{j},y_{j+1})\,y_{j}^{-m_{j-1}},\quad
2\leq j\leq n-1,$

with $n\geq 3$, where $a_{1}=w(y_{1},y_{2})$ for some word
$w(A,B)\in\mathbb{F}_{2}=\langle A,B\rangle$.

  * (2)

The map $\Theta_{n}$ does not preserve the forbidden relations.

###### Proof.

(1) Let us verify that the relations (1) – (7) and the relation (10) are
preserved under the map $\Theta_{n}$. Denote
$s_{i}=\Theta_{n}(\sigma_{i})\in{\rm Aut}(\mathbb{F}_{2n})$ and
$r_{i}=\Theta_{n}(\rho_{i})\in{\rm Aut}(\mathbb{F}_{2n})$.

The relations (1), (4) and (6) are preserved because $s_{i}$ acts non-
trivially only on $x_{i}$ and $x_{i+1}$, while $r_{i}$ acts non-trivially only
on $x_{i}$, $x_{i+1}$, $y_{i}$ and $y_{i+1}$.

Since

$s_{i}^{2}:\begin{cases}x_{i}\mapsto x_{i+1}\,a_{i}(y_{i},y_{i+1})\mapsto
x_{i}\,b_{i}(y_{i},y_{i+1})\,a_{i}(y_{i},y_{i+1}),\\\ x_{i+1}\mapsto
x_{i}\,b_{i}(y_{i},y_{i+1})\mapsto
x_{i+1}\,a_{i}(y_{i},y_{i+1})\,b_{i}(y_{i},y_{i+1}),\end{cases}$

the relation (10) is preserved if and only if
$b_{i}(y_{i},y_{i+1})=a_{i}^{-1}(y_{i},y_{i+1})$ for all $1\leq i\leq n-1$.
Further,

$r_{i}^{2}:\begin{cases}x_{i}\mapsto x_{i+1}\,c_{i}(y_{i},y_{i+1})\mapsto
x_{i}\,d_{i}(y_{i},y_{i+1})\,c_{i}(y_{i+1},y_{i}),\\\ x_{i+1}\mapsto
x_{i}\,d_{i}(y_{i},y_{i+1})\mapsto
x_{i+1}\,c_{i}(y_{i},y_{i+1})\,d_{i}(y_{i+1},y_{i}),\end{cases}$

and relation (3) is preserved iff

(14) $\displaystyle d_{i}(y_{i},y_{i+1})=c_{i}^{-1}(y_{i+1},y_{i}),\qquad
1\leq i\leq n-1.$

Consider the actions of automorphisms $r_{i}r_{i+1}s_{i}$ and
$s_{i+1}r_{i}r_{i+1}$. We have

$r_{i}r_{i+1}s_{i}:\begin{cases}x_{i}\mapsto
x_{i+1}c_{i}(y_{i},y_{i+1})\mapsto
x_{i+2}c_{i+1}(y_{i+1},y_{i+2})c_{i}(y_{i},y_{i+2}),\\\ x_{i+1}\mapsto
x_{i}c_{i}^{-1}(y_{i+1},y_{i})\mapsto x_{i}c_{i}^{-1}(y_{i+2},y_{i})\mapsto
x_{i+1}a_{i}(y_{i},y_{i+1})c_{i}^{-1}(y_{i+2},y_{i}),\\\ x_{i+2}\mapsto
x_{i+2}\mapsto x_{i+1}c_{i+1}^{-1}(y_{i+2},y_{i+1})\mapsto
x_{i}a_{i}^{-1}(y_{i},y_{i+1})c_{i+1}^{-1}(y_{i+2},y_{i+1}),\end{cases}$

and

$s_{i+1}r_{i}r_{i+1}:\begin{cases}x_{i}\mapsto
x_{i+1}c_{i}(y_{i},y_{i+1})\mapsto
x_{i+2}c_{i+1}(y_{i+1},y_{i+2})c_{i}(y_{i},y_{i+2}),\\\
\begin{array}[]{c}x_{i+1}\mapsto x_{i+2}a_{i+1}(y_{i+1},y_{i+2})\mapsto
x_{i+2}a_{i+1}(y_{i},y_{i+2})\\\ \mapsto
x_{i+1}c_{i+1}^{-1}(y_{i+2},y_{i+1})a_{i+1}(y_{i},y_{i+1}),\end{array}\\\
\begin{array}[]{c}x_{i+2}\mapsto x_{i+1}a_{i+1}^{-1}(y_{i+1},y_{i+2})\mapsto
x_{i}c_{i}^{-1}(y_{i+1},y_{i})a_{i+1}^{-1}(y_{i},y_{i+2})\\\ \mapsto
x_{i}c_{i}^{-1}(y_{i+2},y_{i})a_{i+1}^{-1}(y_{i},y_{i+1}).\end{array}\end{cases}$

Thus, to fulfill the relation (7), it is necessary and sufficient that

(15) $\displaystyle
a_{i+1}(y_{i},y_{i+1})c_{i}(y_{i+2},y_{i})=c_{i+1}(y_{i+2},y_{i+1})a_{i}(y_{i},y_{i+1}),$

holds for all $1\leq i\leq n-2$.

A similar consideration of the relations (5) leads to the equalities

(16) $\displaystyle
c_{i+1}(y_{i},y_{i+2})c_{i}(y_{i+1},y_{i+2})=c_{i+1}(y_{i+1},y_{i+2})c_{i}(y_{i},y_{i+2}),$

for all $1\leq i\leq n-2$. This is only possible if
$c_{i}(y_{i},y_{i+1})=y_{i+1}^{m_{i}}$ for some $m_{i}\in\mathbb{Z}$ with
$1\leq i\leq n-1$. Using (15) and (16) we get

$\displaystyle
a_{i+1}(y_{i},y_{i+1})=y_{i+1}^{m_{i+1}}a_{i}(y_{i},y_{i+1})y_{i}^{-m_{i}},\qquad
1\leq i\leq n-2.$

The fulfillment of the relations (2) is checked directly.

(2) Let us now show that the forbidden relations do not hold under the map
$\Theta_{n}$. Indeed, we have:

$\begin{gathered}y_{i}\overset{r_{i}}{\longmapsto}y_{i+1}\overset{s_{i+1}}{\longmapsto}y_{i+1}\overset{s_{i}}{\longmapsto}y_{i+1},\\\
y_{i}\overset{s_{i+1}}{\longmapsto}y_{i}\overset{s_{i}}{\longmapsto}y_{i}\overset{r_{i+1}}{\longmapsto}y_{i},\end{gathered}$

therefore, $r_{i}s_{i+1}s_{i}\neq s_{i+1}s_{i}r_{i+1}$. Similarly

$\begin{gathered}y_{i}\overset{s_{i}}{\longmapsto}y_{i}\overset{s_{i+1}}{\longmapsto}y_{i}\overset{r_{i}}{\longmapsto}y_{i+1},\\\
y_{i}\overset{r_{i+1}}{\longmapsto}y_{i}\overset{s_{i}}{\longmapsto}y_{i}\overset{s_{i+1}}{\longmapsto}y_{i},\end{gathered}$

therefore, $s_{i}s_{i+1}r_{i}\neq r_{i+1}s_{i}s_{i+1}$. ∎

Thus the representation $\Theta_{n}$ given by the formula (13) depends on the
word $a_{1}(A,B)=w(A,B)\in\mathbb{F}_{2}=\langle A,B\rangle$, into which we
substitute $y_{i}$ and $y_{i+1}$ instead of $A$ and $B$ respectively, and a
set of integers $m=(m_{1},\ldots,m_{n-1})$. To emphasize this dependence of
the representation on $w$ and $m$, we denote it $\Theta_{n}^{w,m}$:

(17) $\Theta_{n}^{w,m}(\sigma_{i}):\begin{cases}\displaystyle x_{i}\mapsto
x_{i+1}\,\prod_{k=i}^{2}y_{i+1}^{m_{k}}\,w(y_{i},y_{i+1})\,\prod_{k=1}^{i-1}y_{i}^{-m_{k}},\\\
\displaystyle x_{i+1}\mapsto
x_{i}\,\prod_{k=i-1}^{1}y_{i}^{m_{k}}\,(w(y_{i},y_{i+1}))^{-1}\,\prod_{k=2}^{i}y_{i+1}^{-m_{k}},\end{cases}$

and

(18) $\Theta_{n}^{w,m}(\rho_{i}):\begin{cases}x_{i}\mapsto
x_{i+1}y_{i+1}^{m_{i}},\\\ x_{i+1}\mapsto x_{i}y_{i}^{-m_{i}},\\\ y_{i}\mapsto
y_{i+1},\\\ y_{i+1}\mapsto y_{i},\end{cases}$

where in the products $\displaystyle\prod_{k=i}^{2}$ and
$\displaystyle\prod_{k=i-1}^{1}$ it is assumed that $i\geq 2$ and the indices
are decreasing, and in the products $\displaystyle\prod_{k=1}^{i-1}$ and
$\displaystyle\prod_{k=2}^{i}$ it is assumed $i\geq 2$ and the indices are
increasing.

The word $w$ is called the _defining word_ for the homomorphism
$\Theta_{n}^{w,m}$. In the particular case when $m_{i}=0$ for all
$i=1,\ldots,n-1$, we will write $\Theta_{n}^{w}:FVB_{n}\to{\rm
Aut}(\mathbb{F}_{2n})$ assuming that

(19) $\Theta_{n}^{w}(\sigma_{i}):\begin{cases}x_{i}\mapsto
x_{i+1}w(y_{i},y_{i+1}),\\\ x_{i+1}\mapsto
x_{i}(w(y_{i},y_{i+1}))^{-1},\end{cases}\quad\Theta_{n}^{w}(\rho_{i}):\begin{cases}x_{i}\mapsto
x_{i+1},\\\ x_{i+1}\mapsto x_{i},\\\ y_{i}\mapsto y_{i+1},\\\ y_{i+1}\mapsto
y_{i}.\end{cases}$

Note that the homomorphism (11) constructed in [20] can be represented as
$\eta_{n}=\Theta^{w}_{n}$ for $w(A,B)=B$.

Let $\beta\in FVB_{n}$ and $x\in{\mathbb{F}}_{2n}$. Further, to simplify the
notation, by $\beta(x)$ we mean $\Theta^{w,m}_{n}(\beta)(x)$, where the word
$w$ and the set $m$ are assumed to be clear from the context.

##  3\. The kernel of homomorphism $\Theta_{n}^{w,m}$ and $FVK_{n}$ group

In this section we show that the kernel of the homomorphism $\Theta_{n}^{w,m}$
lies in the intersection of the group of flat virtual pure braids and the
group of flat virtual kure braids group $FVK_{n}$ defined below.

Consider the subgroup ${\rm S}_{n}=\langle\sigma_{1}\ldots\sigma_{n-1}\rangle$
of $FVB_{n}$, which is isomorphic to the permutation group of an $n$-element
set. The map $\pi_{n}:FVB_{n}\to{\rm{S}}_{n}$ defined on the generators
$\sigma_{i},\rho_{i}$ according to the rule:

$\displaystyle\pi_{n}(\sigma_{i})$ $\displaystyle=\sigma_{i},$ $\displaystyle
1\leq i\leq n-1,$ $\displaystyle\pi_{n}(\rho_{i})$ $\displaystyle=\sigma_{i},$
$\displaystyle 1\leq i\leq n-1,$

is obviously a homomorphism.

######  Definition 2.

Denote $FVP_{n}=\operatorname{Ker}(\pi_{n})$ and call it _flat virtual pure
braid group on $n$ strands_.

Similarly, the subgroup
$S^{\prime}_{n}=\langle\rho_{1}\ldots\rho_{n-1}\rangle$ of $FVB_{n}$ is
isomorphic to the permutation group of an $n$-element set, and the map
$\nu_{n}:FVB_{n}\to{\rm{S^{\prime}_{n}}}$ defined on generators $\sigma_{i}$,
$\rho_{i}$ as follows:

$\displaystyle\nu_{n}(\sigma_{i})$ $\displaystyle=1,$ $\displaystyle 1\leq
i\leq n-1,$ $\displaystyle\nu_{n}(\rho_{i})$ $\displaystyle=\rho_{i},$
$\displaystyle 1\leq i\leq n-1,$

is also a homomorphism.

######  Definition 3.

Denote $FVK_{n}=\operatorname{Ker}(\nu_{n})$ and call it _flat virtual kure
braid group on $n$ strands_.

Here we use the term _flat virtual kure braid_ since the term _kure virtual
braid group_ was used in [21] for kernel of the map $\pi_{K}:VB_{n}\to S_{n}$
which is defined analogously to $\nu_{n}:FVB_{n}\to S^{\prime}$. The group
$FVK_{n}=\operatorname{Ker}(\nu_{n})$ also was denoted by $FH_{n}$ in [20]
since is the flat analog of the Rabenda’s group $H_{n}$ from [22, Prop. 17].

######  Lemma 1.

[20, Prop. 4] The group $FVK_{n}$ admits a presentation with generators
$x_{i,j}$, $1\leq i\neq j\leq n$ and defining relations

(20) $x_{i,j}^{2}=1,\qquad x_{i,j}\,x_{k,l}=x_{k,l}\,x_{i,j},\qquad
x_{i,k}\,x_{k,j}\,x_{i,k}=x_{k,j}\,x_{i,k}\,x_{k.j},$

where different letters stand for different indices.

######  Corollary 1.

The group $FVK_{n}$ is a Coxeter group with defining relations

(21)
$x_{i,j}^{2}=1,\qquad(x_{i,j}\,x_{k,l})^{2}=1,\qquad(x_{i,k}\,x_{k,j})^{3}=1.$

The following property is a generalization of the property established in [20,
Prop. 9] for the word $w(A,B)=B$.

######  Lemma 2.

Let $n\geq 2$. For any word $w\in{\mathbb{F}_{2}}$ and any set of integers
$m=(m_{1},\ldots,m_{n-1})$ $\operatorname{Ker}(\Theta_{n}^{w,m})\leq
FVP_{n}\cap FVK_{n}$.

###### Proof.

Let $g\in\operatorname{Ker}(\Theta_{n}^{w,m})$. Then
$y_{i}=g(y_{i})=\nu_{n}(g)(y_{i})$, since all $\sigma_{i}$ act identically on
$y_{i}$. But then $\nu_{n}(g)$ is the identity permutation of the set
$\\{y_{1},\ldots,y_{n}\\}$, which by definition means $g\in FVK_{n}$.

Next, we show that $g\in FVP_{n}$. Denote by $G$ the normal closure of the
subgroup $\langle y_{1},\ldots,y_{n}\rangle$ in ${\mathbb{F}}_{2n}$. It is
clear that $G$ is a $\Theta_{n}^{w,m}(FVB_{n})$–invariant subgroup. Then
$\Theta_{n}^{w,m}$ induces a homomorphism $\Psi_{n}^{w,m}\colon FVB_{n}\to{\rm
Aut}({\mathbb{F}}_{2n}/G)={\rm Aut}({\mathbb{F}}_{n})$, where
${\mathbb{F}}_{n}=\langle x_{1},\ldots,x_{n}\rangle$. From the formulas (13)
we can write out the action of $\Psi_{n}^{w,m}$ on the generators of the group
$FVB_{n}$:

$\displaystyle\Psi_{n}^{w,m}(\sigma_{i})\colon\begin{cases}x_{i}\mapsto
x_{i+1},\\\ x_{i+1}\mapsto
x_{i},\end{cases}\qquad\Psi_{n}^{w,m}(\rho_{i})\colon\begin{cases}x_{i}\mapsto
x_{i+1},\\\ x_{i+1}\mapsto x_{i},\end{cases}\qquad 1\leq i\leq n-1.$

Now it is easy to see that the image of $FVB_{n}$ under the map
$\Psi_{n}^{w,m}$ is a permutation of the set $\\{x_{1},\ldots,x_{n}\\}$. It
remains to note that if $g\in\operatorname{Ker}(\Theta_{n}^{w,m})$, then
$g\in\operatorname{Ker}(\Psi_{n}^{w,m})=FVP_{n}$. ∎

Since $S^{\prime}_{n}\leq FVB_{n}$, then the decomposition of
$FVB_{n}=FVK_{n}\rtimes S^{\prime}_{n}$ follows directly from the definition
of $FVK_{n}$. Considering the restriction of the homomorphism $\pi_{n}$ to
$FVK_{n}$, we obtain the homomorphism $\xi_{n}\colon FVK_{n}\to S_{n}$. Note
that its kernel is $X_{n}=\operatorname{Ker}(\xi_{n})=FVP_{n}\cap FVK_{n}$.
Further, since $S^{\prime}_{n}\leq FVK_{n}$, we obtain the decomposition
$FVK_{n}=X_{n}\rtimes S_{n}$. Thus, we have the following decomposition of the
group of flat virtual braids:

$FVB_{n}=(X_{n}\rtimes S_{n})\rtimes S^{\prime}_{n}.$

As it invented in [22], we denote

$\displaystyle\lambda_{i,i+1}=\rho_{i}\sigma_{i},$ $\displaystyle 1\leq i\leq
n-1,$
$\displaystyle\lambda_{i,j}=\rho_{j-1}\rho_{j-2}\ldots\rho_{i+1}\lambda_{i,i+1}\rho_{i+1}\ldots\rho_{j-2}\rho_{j-1},$
$\displaystyle j-i\geq 2.$

Element $\lambda_{i,j}$ is presented geometrically in Figure 7,

$i$$i+1$$i+2$$j-2$$j-1$$j$$\vdots$$\vdots$$\vdots$$\vdots$$\vdots$$\vdots$$\vdots$$\vdots$$\vdots$$\vdots$$\vdots$$\vdots$$\cdots$
Figure 7. Element $\lambda_{i,j}\in FVB_{n}$.

######  Lemma 3.

[22] The group $FVP_{n}$ is generated by the elements $\lambda_{i,j},~{}1\leq
i<j\leq n$ and the defining relations are:

(22) $\displaystyle\lambda_{i,j}\lambda_{k,l}$
$\displaystyle=\lambda_{k,l}\lambda_{i,j},$ (23)
$\displaystyle\lambda_{k,i}\lambda_{k,j}\lambda_{i,j}$
$\displaystyle=\lambda_{i,j}\lambda_{k,j}\lambda_{k,i},$

where $i,j,k,l$ correspond to different indices.

Let us consider the case $n=3$ in more details. As shown in [22],

(24) $\displaystyle{\rm{FVP}}_{3}=\langle a,b,c\mid[a,c]=1\rangle,$

where $a=\lambda_{2,3}\lambda_{1,3}$, $b=\lambda_{2,3}$ and
$c=\lambda_{2,3}\lambda_{1,2}^{-1}$. These elements presented geometrically in
Figure 8.

$1$$2$$3$$a$$1$$2$$3$$b$$1$$2$$3$$c$ Figure 8. Elements
$a=\lambda_{2,3}\lambda_{1,3},\,b=\lambda_{2,3},\,c=\lambda_{2,3}\lambda^{-1}_{1,2}\in
FVB_{3}$.

######  Theorem 2.

We have the following decomposition

$X_{3}=\mathbb{Z}^{2}*\mathbb{F}_{3}*\Gamma,$

where $\mathbb{F}_{3}$ is a free group of rank $3$ and $\Gamma=\langle
x,y,u,v,p,q~{}|~{}xy=uv,vu=pq,qp=yx\rangle$.

###### Proof.

Consider the restriction of the homomorphism $\nu_{3}\colon FVB_{3}\to
S_{3}^{\prime}$ to $FVP_{3}$. Let us denote it by $\varphi\colon FVP_{3}\to
S_{n}^{\prime}$. Then $X_{3}=\operatorname{Ker}(\varphi)$.

To find the generators and relations of the $X_{3}$ group, we use the
Reidemeisetr-Schreier rewriting process, see for example [23]. Let us write
out the system of Schreier representatives for $\operatorname{Ker}(\varphi)$
using the generators indicated in the presentation (24):
$T=\\{1,a,ab,c,cb,b\\}$. For an element $g$, we denote its representative in
$T$ by $\overline{g}$. Then the kernel $\operatorname{Ker}(\varphi)$ is
generated by the following elements:

$\displaystyle a\cdot a\cdot(\overline{a^{2}})^{-1}=a^{2}c^{-1}=t,$
$\displaystyle a\cdot c\cdot(\overline{ac})^{-1}=ac=m,$ $\displaystyle ab\cdot
a\cdot(\overline{aba})^{-1}=abab^{-1}=v,$ $\displaystyle ab\cdot
b\cdot(\overline{ab^{2}})^{-1}=ab^{2}a^{-1}=w,$ $\displaystyle ab\cdot
c\cdot(\overline{abc})^{-1}=abcb^{-1}c^{-1}=p,$ $\displaystyle c\cdot
a\cdot(\overline{ca})^{-1}=ca=r,$ $\displaystyle c\cdot
c\cdot(\overline{c^{2}})^{-1}=c^{2}a^{-1}=g,$ $\displaystyle cb\cdot
a\cdot(\overline{cba})^{-1}=cbab^{-1}a^{-1}=q,$ $\displaystyle cb\cdot
b\cdot(\overline{cb^{2}})^{-1}=cb^{2}c^{-1}=h,$ $\displaystyle cb\cdot
c\cdot(\overline{cbc})^{-1}=cbcb^{-1}=y,$ $\displaystyle b\cdot
a\cdot(\overline{ba})^{-1}=bab^{-1}c^{-1}=x,$ $\displaystyle b\cdot
b\cdot(\overline{b^{2}})^{-1}=b^{2}=f,$ $\displaystyle b\cdot
c\cdot(\overline{bc})^{-1}=bcb^{-1}a^{-1}=u.$

Further, the relations $taca^{-1}c^{-1}t^{-1}$ for $t\in T$ must be rewritten
in new generators. For example, for $t=ab$ we get:

$\begin{gathered}ab(aca^{-1}c^{-1})b^{-1}a^{-1}\,=\,vbca^{-1}c^{-1}b^{-1}a^{-1}\,=\,vuaba^{-1}c^{-1}b^{-1}a^{-1}\\\
=\,vuq^{-1}cbc^{-1}b^{-1}a^{-1}\,=\,vuq^{-1}p^{-1}.\end{gathered}$

The rest of the relations are found similarly. As a result, we get:

$\displaystyle m=r,\quad$ $\displaystyle m=tg,\quad$ $\displaystyle r=gt,$
$\displaystyle xy=uv,\quad$ $\displaystyle vu=pq,\quad$ $\displaystyle qp=yx.$

It is now clear that the elements $g,t$ generate $\mathbb{Z}^{2}$, the
elements $w,h,f$ generate $\mathbb{F}_{3}$, and the group generated by the
elements $x$ $y$, $u$, $v$, $p$ and $q$ we denote by $\Gamma$. ∎

######  Lemma 4.

[20] Let $G_{n}$ be a subgroup of $FVP_{n}$ generated by the elements:

(25) $\displaystyle t_{i,j}=$ $\displaystyle\lambda_{i,j}^{2},\quad$
$\displaystyle 1\leq i<j\leq n,$ (26) $\displaystyle d_{i,j,k}=$
$\displaystyle\lambda_{j,k}^{-1}\lambda_{i,j}^{-1}\lambda_{j,k}\lambda_{i,k},\quad$
$\displaystyle 1\leq i<j<k\leq n,$ (27) $\displaystyle e_{i,j,k}=$
$\displaystyle\lambda_{j,k}^{-1}\lambda_{i,j}^{-1}\lambda_{i,k}\lambda_{i,j},\quad$
$\displaystyle 1\leq i<j<k\leq n.$

Then the normal closure of $G_{n}$ in $FVP_{n}$ coincides with $X_{n}$.

Let us describe the action of $\Theta_{n}^{w}$ on the generators indicated in
Lemma 4.

######  Lemma 5.

The homomorphism
$\Theta_{n}^{w}:FVB_{n}\to\operatorname{Aut}(\mathbb{F}_{2n})$, defined by the
word $w\in\mathbb{F}_{2}$, maps the generators of the group $G_{n}$ to the
following automorphisms:

(28) $\displaystyle\Theta_{n}^{w}(t_{i,j})$
$\displaystyle:\begin{cases}x_{i}\mapsto x_{i}\,w_{i,j}^{-1}\,w_{j,i}^{-1},\\\
x_{j}\mapsto x_{j}\,w_{i,j}\,w_{j,i},\end{cases}\quad$ $\displaystyle 1\leq
i<j\leq n,$ (29) $\displaystyle\Theta_{n}^{w}(d_{i,j,k})$
$\displaystyle:\begin{cases}x_{j}\mapsto
x_{j}\,w_{j,i}^{-1}\,w_{i,k}^{-1}\,w_{j,k},\\\ x_{k}\mapsto
x_{k}\,w_{i,k}\,w_{j,i}\,w_{j,k}^{-1},\end{cases}\quad$ $\displaystyle 1\leq
i<j<k\leq n,$ (30) $\displaystyle\Theta_{n}^{w}(e_{i,j,k})$
$\displaystyle:\begin{cases}x_{i}\mapsto
x_{i}\,w_{i,j}^{-1}\,w_{j,k}^{-1}\,w_{i,k},\\\ x_{j}\mapsto
x_{j}\,w_{i,j}\,w_{i,k}^{-1}\,w_{j,k},\end{cases}\quad$ $\displaystyle 1\leq
i<j<k\leq n.$

where $w_{i,j}=w(y_{i},y_{j})$ for all $i,j$.

###### Proof.

Let, as before, $s_{i}=\Theta_{n}^{w}(\sigma_{i})$ and
$r_{i}=\Theta_{n}^{w}(\rho_{i})$. First of all, let us establish some
auxiliary formulas. Let $1\leq i<j-1\leq n-1$, then

$\displaystyle
b_{i,j}:=\Theta^{w}_{n}(\rho_{j-1}\ldots\rho_{i+1}):\begin{cases}x_{i+1}\overset{r_{i+1}}{\longmapsto}x_{i+2},\\\
\vdots\\\ x_{j-1}\overset{r_{j-1}}{\longmapsto}x_{j},\\\
x_{j}\overset{r_{j-1}}{\longmapsto}x_{j-1}\overset{r_{j-2}}{\longmapsto}x_{j-2}\ldots\overset{r_{i+1}}{\longmapsto}x_{i+1},\\\
y_{i+1}\overset{r_{i+1}}{\longmapsto}y_{i+2},\\\ \vdots\\\
y_{j-1}\overset{r_{j-1}}{\longmapsto}y_{j},\\\
y_{j}\overset{r_{j-1}}{\longmapsto}x_{j-1}\overset{r_{j-2}}{\longmapsto}x_{j-2}\ldots\overset{r_{i+1}}{\longmapsto}y_{i+1}.\end{cases}$

Further,

$\displaystyle
a_{i,i+1}:=\Theta^{w}_{n}(\lambda_{i,i+1})=\Theta^{w}_{n}(\rho_{i}\sigma_{i}):\begin{cases}x_{i}\overset{r_{i}}{\longmapsto}x_{i+1}\overset{s_{i}}{\longmapsto}x_{i}\,w_{i,i+1}^{-1},\\\
x_{i+1}\overset{r_{i}}{\longmapsto}x_{i}\overset{s_{i}}{\longmapsto}x_{i+1}\,w_{i,i+1},\\\
y_{i}\overset{r_{i}}{\longmapsto}y_{i+1}\overset{s_{i}}{\longmapsto}y_{i+1},\\\
y_{i+1}\overset{r_{i}}{\longmapsto}y_{i}\overset{s_{i}}{\longmapsto}y_{i}.\end{cases}$

Let us show that for $1\leq i<j\leq n$ the formulas

$\displaystyle\Theta^{w}_{n}(\lambda_{i,j}):\begin{cases}x_{i}\mapsto
x_{i}\,w_{i,j}^{-1},\\\ x_{j}\mapsto x_{j}\,w_{i,j},\\\ y_{i}\mapsto y_{j},\\\
y_{j}\mapsto
y_{i},\end{cases}\qquad\Theta^{w}_{n}(\lambda_{i,j}^{-1}):\begin{cases}x_{i}\mapsto
x_{i}\,w_{j,i},\\\ x_{j}\mapsto x_{j}\,w_{j,i}^{-1},\\\ y_{i}\mapsto y_{j},\\\
y_{j}\mapsto y_{i}\end{cases}$

hold. Indeed, we have

$\displaystyle\Theta^{w}_{n}(\lambda_{i,j}):\begin{cases}x_{i}\overset{b_{i,j}}{\longmapsto}x_{i}\overset{a_{i,i+1}}{\longmapsto}x_{i}\,w_{i,i+1}^{-1}\overset{b_{i,j}^{-1}}{\longmapsto}x_{i}\,w_{i,j}^{-1},\\\
x_{i+1}\overset{b_{i,j}}{\longmapsto}x_{i+2}\overset{a_{i,i+1}}{\longmapsto}x_{i+2}\overset{b_{i,j}^{-1}}{\longmapsto}x_{i+1},\\\
\vdots\\\
x_{j-1}\overset{b_{i,j}}{\longmapsto}x_{j}\overset{a_{i,i+1}}{\longmapsto}x_{j}\overset{b_{i,j}^{-1}}{\longmapsto}x_{j-1},\\\
x_{j}\overset{b_{i,j}}{\longmapsto}x_{i+1}\overset{a_{i,i+1}}{\longmapsto}x_{i+1}\,w_{i,i+1}\overset{b_{i,j}^{-1}}{\longmapsto}x_{j}\,w_{i,j},\\\
y_{i}\overset{b_{i,j}}{\longmapsto}y_{i}\overset{a_{i,i+1}}{\longmapsto}y_{i+1}\overset{b_{i,j}^{-1}}{\longmapsto}y_{j},\\\
y_{i+1}\overset{b_{i,j}}{\longmapsto}y_{i+2}\overset{a_{i,i+1}}{\longmapsto}y_{i+2}\overset{b_{i,j}^{-1}}{\longmapsto}y_{i+1},\\\
\vdots\\\
y_{j-1}\overset{b_{i,j}}{\longmapsto}y_{j}\overset{a_{i,i+1}}{\longmapsto}y_{j}\overset{b_{i,j}^{-1}}{\longmapsto}y_{j-1},\\\
y_{j}\overset{b_{i,j}}{\longmapsto}y_{i+1}\overset{a_{i,i+1}}{\longmapsto}y_{i}\overset{b_{i,j}^{-1}}{\longmapsto}y_{i}.\end{cases}$

We are now ready to prove the formulas (28) – (30). For example, let us
establish (28):

$\displaystyle\Theta^{w}_{n}(t_{i,j})=\Theta^{w}_{n}(\lambda_{i,j}^{2}):\begin{cases}x_{i}\overset{\lambda_{i,j}}{\longmapsto}x_{i}\,w_{i,j}^{-1}\overset{\lambda_{i,j}}{\longmapsto}x_{i}\,w_{i,j}^{-1}\,w_{j,i}^{-1},\\\
x_{j}\overset{\lambda_{i,j}}{\longmapsto}x_{i}\,w_{i,j}\overset{\lambda_{i,j}}{\longmapsto}x_{i}\,w_{i,j}\,w_{j,i},\\\
y_{i}\overset{\lambda_{i,j}}{\longmapsto}y_{j}\overset{\lambda_{i,j}}{\longmapsto}y_{i},\\\
y_{j}\overset{\lambda_{i,j}}{\longmapsto}y_{i}\overset{\lambda_{i,j}}{\longmapsto}y_{j}.\end{cases}$

The formulas (29) and (30) are proved in the same way. ∎

The following statement answers the question about the faithfulness of the
representations $\Theta_{n}^{w,m}$ in the case $n=2$.

######  Theorem 3.

The nontrivial representation
$\Theta_{2}^{w,m}:FVB_{2}\to\operatorname{Aut}(\mathbb{F}_{4})$ is not exact
if the defining word $w$ is

$w(A,B)=A^{k_{1}}B^{k_{2}}\ldots A^{k_{m}}B^{-k_{m}}\ldots
A^{-k_{2}}B^{-k_{1}}A^{m_{1}},$

where all $k_{i}$ are nonzero integers except possibly only for $k_{1}$ and
$k_{m}$. In this case,
$\operatorname{Ker}(\Theta_{2}^{w,m})=X_{2}\simeq\mathbb{Z}$. The
representation of $\Theta_{2}^{w,m}$ is exact for other $w$.

###### Proof.

Lemma 2 implies that ${\rm{Ker}}(\Theta_{2}^{w,m})\leq X_{2}$. It is easy to
show that $X_{2}$ is generated by the element $t_{1,2}$.

In the case of $n=2$, the set $m$ consists of a single integer
$m=\\{m_{1}\\}$. For any $k\in\mathbb{Z}$

$\displaystyle\Theta_{n}^{w,m}(t_{1,2}^{k})\colon\begin{cases}x_{1}\mapsto
x_{1}\,(w_{1,2}^{-1}\,y_{2}^{m_{1}}\,w_{2,1}^{-1}\,y_{1}^{m_{1}})^{k},\\\
x_{2}\mapsto
x_{2}\,(w_{1,2}\,y_{1}^{-m_{1}}\,w_{2,1}\,y_{2}^{-m_{1}})^{k}.\end{cases}$

Thus $\Theta_{n}^{w}(t_{1,2}^{k})={\rm id}$ iff either $k=0$ or

$w_{1,2}^{-1}\,y_{2}^{m_{1}}\,w_{2,1}^{-1}\,y_{1}^{m_{1}}=1,$

i.e. $f(y_{1},y_{2})=f^{-1}(y_{2},y_{1})$ for the word
$f(y_{1},y_{2})=w(y_{1},y_{2})y_{1}^{-m_{1}}$.

Let $f(A,B)=A^{k_{1}}B^{k_{2}}\dots B^{k_{s}}A^{k_{s+1}}$. Then

$A^{k_{1}}B^{k_{2}}\dots B^{k_{s}}A^{k_{s+1}}=B^{-k_{s+1}}A^{-k_{s}}\dots
A^{-k_{2}}B^{-k_{1}},$

therefore $f(A,B)=A^{k_{1}}B^{k_{2}}\dots A^{k_{m}}B^{-k_{m}}\ldots
A^{-k_{2}}B^{-k_{1}}$, where all $k_{i}$ — nonzero integers except maybe
$k_{1}$ and $k_{m}$. But then

$w(A,B)=A^{k_{1}}B^{k_{2}}\ldots A^{k_{m}}B^{-k_{m}}\ldots
A^{-k_{2}}B^{-k_{1}}A^{m_{1}}.$

This completes the proof. ∎

##  4\. Nontriviality of the kernel ${\rm{Ker}}(\Theta_{n}^{w,m})$ for $n\geq
3$

Consider the following subgroups of the $FVB_{n}$ group:

(31) $\displaystyle Q_{n}^{i}$ $\displaystyle=\langle
t_{i,i+1},~{}e_{i,i+1,i+2}\rangle,\quad$ $\displaystyle 1\leq i\leq n-2,$ (32)
$\displaystyle M_{n}^{i+1}$ $\displaystyle=\langle
t_{i+1,i+2},~{}d_{i,i+1,i+2}\rangle,\quad$ $\displaystyle 1\leq i\leq n-2,$
(33) $\displaystyle P_{n}^{i+2}$ $\displaystyle=\langle
t_{i,i+2}\rangle,\quad$ $\displaystyle 1\leq i\leq n-2.$

######  Lemma 6.

Let $n\geq 3$ and $w(A,B)\in\mathbb{F}_{2}(A,B)$. Then, for all $i$, $1\leq
i\leq n-2$, we obtain the inclusion

(34)
$\displaystyle\Big{[}Q_{n}^{i},\big{[}M_{n}^{i+1},P_{n}^{i+2}\big{]}\Big{]}\leq{\rm{Ker}}(\Theta_{n}^{w}).$

###### Proof.

Note that $Q_{n}^{i}$ acts non-trivially only on generators $x_{i}$ and
$x_{i+1}$, $M_{n}^{i+1}$ acts non-trivially only on $x_{i+1}$ and $x_{i+2}$,
while $P_{n}^{i+2}$ acts non-trivially only on $x_{i}$ and $x_{i+2}$. Consider
the element

$h=q(mpm^{-1}p^{-1})q^{-1}(mpm^{-1}p^{-1})^{-1},$

where $q\in Q_{n}^{i}$, $m\in M_{n}^{i+1}$ and $p\in P_{n}^{i+2}$. This
element acts non-trivially only on the generators $x_{i}$, $x_{i+1}$ and
$x_{i+2}$. Write out its action:

$h:x_{i}\overset{q}{\longmapsto}qx_{i}\overset{m}{\longmapsto}qx_{i}\overset{p}{\longmapsto}pqx_{i}\overset{m^{-1}}{\longmapsto}pqx_{i}\overset{p^{-1}}{\longmapsto}qx_{i}\overset{q^{-1}}{\longmapsto}x_{i}\overset{p}{\longmapsto}px_{i}\overset{m}{\longmapsto}px_{i}\overset{p^{-1}}{\longmapsto}x_{i}\overset{m^{-1}}{\longmapsto}x_{i};$
$\begin{gathered}h:x_{i+1}\overset{q}{\longmapsto}qx_{i+1}\overset{m}{\longmapsto}mqx_{i+1}\overset{p}{\longmapsto}mqx_{i+1}\overset{m^{-1}}{\longmapsto}qx_{i+1}\overset{p^{-1}}{\longmapsto}qx_{i+1}\overset{q^{-1}}{\longmapsto}x_{i+1}\overset{p}{\longmapsto}x_{i+1}\cr\overset{m}{\longmapsto}mx_{i+1}\overset{p^{-1}}{\longmapsto}mx_{i+1}\overset{m^{-1}}{\longmapsto}x_{i+1};\end{gathered}$
$\begin{gathered}h:x_{i+2}\overset{q}{\longmapsto}x_{i+2}\overset{m}{\longmapsto}mx_{i+2}\overset{p}{\longmapsto}pmx_{i+2}\overset{m^{-1}}{\longmapsto}m^{-1}pmx_{i+2}\overset{p^{-1}}{\longmapsto}p^{-1}m^{-1}pmx_{i+2}\overset{q^{-1}}{\longmapsto}\cr
p^{-1}m^{-1}pmx_{i+2}\overset{pmp^{-1}m^{-1}}{\longmapsto}x_{i+2}\end{gathered}$

Thus, $h\in{\rm{Ker}}(\Theta_{n}^{w})$ and the inclusion (34) is proved. ∎

######  Theorem 4.

Let $n\geq 3$. For any defining word $w(A,B)\in\mathbb{F}_{2}(A,B)$ the kernel
${\rm{Ker}}(\Theta_{n}^{w})$ contains a subgroup isomorphic to a free group of
rank 2.

###### Proof.

Denote a free group of rank 2 by $\mathbb{F}_{2}$. By Lemma 6, it suffices to
show that
$\mathbb{F}_{2}\leq\Big{[}Q_{3}^{1},\big{[}M_{3}^{2},P_{3}^{3}\big{]}\Big{]}\leq
FVP_{3}$.

Consider the elements

$h_{0}=\big{[}t_{1,2},[t_{2,3},t_{1,3}]\big{]}\qquad\text{and}\qquad
h_{1}=\big{[}t_{1,2},[d_{1,2,3},t_{1,3}]\big{]}.$

We write them down in terms of the generators of $FVP_{3}$ group, see (24):

(35)
$h_{0}=\big{[}(c^{-1}b)^{2},[b^{2},(b^{-1}a)^{2}]\big{]}\qquad\text{and}\qquad
h_{1}=\big{[}(c^{-1}b)^{2},[b^{-2}ca,(b^{-1}a)^{2}]\big{]}.$

Let us prove that $h_{0}$ and $h_{1}$ generate $\mathbb{F}_{2}$. To do this,
it suffices to show that there are no relations between them. Let
$\psi:FVP_{3}\to\langle a,b\rangle$ be the homomorphism given by the mapping
$\psi(a)=a$, $\psi(b)=b$ and $\psi(c)=1$. Denote $\bar{h}_{0}=\psi(h_{0})$ and
$\bar{h}_{1}=\psi(h_{1})$. Then

$\displaystyle\bar{h}_{0}$
$\displaystyle=b^{3}ab^{-1}ab^{-2}a^{-1}ba^{-1}b^{-2}ab^{-1}ab^{2}a^{-1}ba^{-1}b^{-1},$
$\displaystyle\bar{h}_{1}$
$\displaystyle=ab^{-1}aba^{-1}ba^{-1}b^{-2}ab^{-1}ab^{-1}a^{-1}ba^{-1}b^{2}.$

The elements $\bar{h}_{0}$ and $\bar{h}_{1}$ lie in the free group $\langle
a,b\rangle$. Hence the group $\langle\bar{h}_{0},\bar{h}_{1}\rangle$ is either
isomorphic to $\mathbb{Z}$ or isomorphic to $\mathbb{F}_{2}$. The first case
means that $\bar{h}_{0}$ and $\bar{h}_{1}$ must be powers of the same element,
i.e. $\bar{h}_{0}=g(a,b)^{k}$ and $\bar{h}_{1}=g(a,b)^{s}$ for some word
$g(a,b)\in\langle a,b\rangle$ and nonzero $k,s\in\mathbb{Z}$. Let
$g(a,b)=f\cdot w\cdot f^{-1}$, where $w(a,b)$ is the cyclic reduced word in
$\langle a,b\rangle$. Then $g^{s}=fw^{s}f^{-1}=\bar{h}_{1}$ and since
$\bar{h}_{1}$ is itself cyclically reduced, we get $f=1$. But then
$g^{k}=fw^{k}f^{-1}=w^{k}=\bar{h}_{0}$ must be cyclically reduced, which is
not the case. Thus $\langle\bar{h}_{0},\bar{h}_{1}\rangle\cong\mathbb{F}_{2}$
and hence $\langle h_{0},h_{1}\rangle\cong\mathbb{F}_{2}$. ∎

######  Corollary 2.

Let $n\geq 3$, then $\operatorname{Ker}(\Theta_{n}^{w,m})$ contains a subgroup
isomorphic to a free group of rank 2 for any integer tuple
$m=(m_{1},\ldots,m_{n-1})$ and arbitrary defining word
$w(A,B)\in\mathbb{F}_{2}(A,B)$.

###### Proof.

There are three points which are sufficient to prove the corollary:

  * •

$t_{1,2}$ acts non-trivially only on generators $x_{1}$ and $x_{2}$,

  * •

$t_{2,3}$ and $d_{1,2,3}$ act non-trivially only on elements $x_{2}$ and
$x_{3}$,

  * •

$t_{1,3}$ acts non-trivially only on $x_{1}$ and $x_{3}$.

Recall we have $t_{i,i+1}=\lambda_{i,i+1}^{2}=(\rho_{i}\sigma_{i})^{2}$ by the
formula (25). Therefore, $t_{1,2}$ and $t_{2,3}$ satisfy property above.

Let’s check that $t_{1,3}=(\rho_{2}\rho_{1}\sigma_{1}\rho_{2})^{2}$ leaves
$x_{2},~{}y_{1},~{}y_{2}$ and $y_{3}$ in place. The element $t_{1,3}$ acts
trivially on the generators $y_{i}$ for $1\leq i\leq 3$, because using
formulas (17, 18) we get $t_{1,3}\cdot
y_{i}=(\rho_{2}\rho_{1}\rho_{2})^{2}\cdot y_{i}=1\cdot y_{i}=y_{i}$. The
trivial action on $x_{2}$ follows from the fact that
$\rho_{2}\rho_{1}\sigma_{1}\rho_{2}$ leaves this element in place. Indeed,
according to (17, 18), we obtain

$\begin{gathered}x_{2}\overset{\rho_{2}}{\longmapsto}x_{3}y_{3}^{m_{2}}\overset{\rho_{1}}{\longmapsto}x_{3}y_{3}^{m_{2}}\overset{\sigma_{1}}{\longmapsto}x_{3}y_{3}^{m_{2}}\overset{\rho_{2}}{\longmapsto}x_{2}y_{2}^{-m_{2}}y_{2}^{m_{2}}=x_{2}.\end{gathered}$

Similarly, we check the action for

$\begin{gathered}x_{1}\overset{\sigma_{2}\rho_{2}}{\longmapsto}x_{1}\overset{\sigma_{1}}{\longmapsto}x_{2}w(y_{1},y_{2})\overset{\rho_{1}}{\longmapsto}x_{1}y_{1}^{-m_{1}}w(y_{2},y_{1})\overset{\rho_{2}}{\longmapsto}x_{1}y_{1}^{-m_{1}}w(y_{3},y_{1})\overset{\sigma_{2}}{\longmapsto}\\\
\overset{\sigma_{2}}{\longmapsto}x_{1}y_{1}^{-m_{1}}w(y_{3},y_{1})\overset{\rho_{2}}{\longmapsto}x_{1}y_{1}^{-m_{1}}w(y_{2},y_{1})\overset{\rho_{1}}{\longmapsto}\\\
\overset{\rho_{1}}{\longmapsto}x_{2}y_{2}^{m_{1}}y_{2}^{-m_{1}}w(y_{1},y_{2})\overset{\sigma_{1}}{\longmapsto}x_{1}w^{-1}(y_{1},y_{2})w(y_{1},y_{2})\overset{\rho_{2}}{\longmapsto}x_{1},\\\
d_{1,2,3}\cdot y_{i}=\rho_{1}\rho_{2}\rho_{1}\rho_{1}\rho_{2}\rho_{1}\cdot
y_{i}=y_{i},\quad 1\leq i\leq 3.\end{gathered}$

This completes the proof. ∎

## References

  * [1] E. Artin, Theorie der Zöpfe, Abh. Math. Semin Univ. Hamburg 4 (1925), 47–72.
  * [2] C. Kassel, V. Turaev, Braid groups, Graduate Texts in Mathematics, Springer, 2008.
  * [3] E. Artin, The theory of braids, Annals of Math. 48 (1947), 101–126.
  * [4] L. Kauffman, Virtual knot theory, European J. Comb. 20(7) (1999), 663–690.
  * [5] L. Kauffman, S. Lambropoulou, Virtual braids, Fundamenta Mathematicae 184 (2004), 159–186.
  * [6] S. Kamada, Braid presentation of virtual knots and welded knots, Osaka J. Math. 44 (2007), 441–458.
  * [7] M. Goussarov, M. Polyak, O. Viro, Finite-type invariants of classical and virtual knots, Topology 39(5) (2000), 1045–1068.
  * [8] R. Fenn, R. Rimányi, C. Rourke, The braid-permutation group, Topology 36 (1997), 123–135.
  * [9] T. Kanenobu, Forbidden moves unknot a virtual knot, J. Knot Theory Ramificatiions 10(1) (2001), 89–96.
  * [10] A. Gill, M. Prabhakar, A. Vesnin, Gordian complexes of knots and virtual knots given be region crossing and arc shift moves, J. Knot Theory Ramifications 20(10) (2020), paper number 2042008.
  * [11] A. Gill, K. Kaur, M. Prabhakar, A. Vesnin, An unknotting invariant for welded knots, Proc. Indian Acad. Sci. (Math. Sci.) 131 (2021), paper number 47.
  * [12] K. Kaur, M. Prabhakar, A. Vesnin, An unknotting index for virtual links, Topology and Its Applications 264 (2019), 352–368.
  * [13] D. Silver, S. Williams, Alexander groups and virtual links, J. Knot Theory Ramifications 10(1) (2001), 151–160.
  * [14] H. Boden, E. Dies, A. Gaudreau, A. Gerlings, E. Harper, A. Nicas, Alexander invariants for virtual knots, J. Knot Theory Ramifications 24(3) (2015), paper number 1550009.
  * [15] V. Bardakov, Yu. Mikhalchishina, M. Neshchadim, Representations of virtual braids by automorphisms and virtual knot groups, J. Knot Theory Ramifications 25(1) (2017), paper number 1750003.
  * [16] P. Bellingeri, L. Paris, Virtual braids and permutations, Annales de L’Institute Fourier 70:3 (2020). 1341–1362.
  * [17] V. Bardakov, I. Emel’yanenkov, M. Ivanov, T. Kozlovskaya, T. Nasybullov, A. Vesnin, Virtual and universal braid groups, their quotients and representations, Journal of Group Theory 25:4 (2022), 679–712.
  * [18] L. Kauffman, A survey of virtual knot theory, in: Knots in Hellas–98 (Delphi), 143–202, Ser. Knots Everything, 24, World Sci. Publ., River Edge, NJ, 2000.
  * [19] R. Fenn, D. Ilyutko, L. Kauffman, V. Manturov, Unsolved problems in virtual knot theory and combinatorial knot theory, In: Banach Center Publications 103 (2014), 9–61.
  * [20] V. Bardakov, B. Chuzhinov, I. Emel’yanenkov, M. Ivanov, E. Markhinina, T. Nasybullov, S. Panov, N. Singh, S. Vasyutkin, A. Vesnin, V. Yakhin, Representations of flat virtual braids which do not preserve the forbidden relations, Preprint https://arxiv.org/abs/2010.03162
  * [21] P. Bellingeri, L. Paris, A. Thiel, Virtual Artin groups, Proceedings of the London Mathematical Society, 126:1 (2023), 192–215.
  * [22] V. Bardakov, P. Bellingeri, C. Damiani, Unrestricted virtual braids, fused links and other quotients of virtual braid groups, J. Knot Theory Ramifications 24(12) (2015), paper number 1550063.
  * [23] W. Magnus, A. Karrass, D. Solitar Combinatorial group theory. New York 1996.