# Auto-Tag: Tagging-Data-By-Example in Data Lakes

Yeye He, Jie Song, Yue Wang, Surajit Chaudhuri
Vishal Anil, Blake Lassiter, Yaron Goland, Gaurav Malhotra  Microsoft
Corporation

###### Abstract.

As data lakes become increasingly popular in large enterprises today, there is
a growing need to tag or classify data assets (e.g., files and databases) in
data lakes with additional metadata (e.g., semantic column-types), as the
inferred metadata can enable a range of downstream applications like data
governance (e.g., GDPR compliance), and dataset search. Given the sheer size
of today’s enterprise data lakes with petabytes of data and millions of data
assets, it is imperative that data assets can be “auto-tagged”, using
lightweight inference algorithms and minimal user input.

In this work, we develop Auto-Tag, a corpus-driven approach that automates
data-tagging of custom data types in enterprise data lakes. Using Auto-Tag,
users only need to provide one example column to demonstrate the desired data-
type to tag. Leveraging an index structure built offline using a lightweight
scan of the data lake, which is analogous to pre-training in machine learning,
Auto-Tag can infer suitable data patterns to best “describe” the underlying
“domain” of the given column at an interactive speed, which can then be used
to tag additional data of the same “type” in data lakes. The Auto-Tag approach
can adapt to custom data-types, and is shown to be both accurate and
efficient. Part of Auto-Tag ships as a “custom-classification” feature in a
cloud-based data governance and catalog solution Azure Purview.

##  1\. Introduction

Large enterprise data lakes are increasingly common today, often with
petabytes of data and millions of data assets (e.g., flat files or databases).
Given their sheer sizes, it has become increasingly important to govern and
catalog data lakes, as evidenced by a growing number of offerings from
startups and established vendors, such as Azure Purview (purview, ), AWS Glue
Catalog (AWSGlue, ), Google Cloud Data Catalog (GoogleCatalog, ), Alation
(Alation, ), Waterline (Waterline, ), Collibra (Collibra, ), etc.

A key challenge in governing data lakes is data tagging (also known as
classification), which is the process of inferring rich metadata (e.g.
semantic column-types) from data. Such inferred metadata are critical for
downstream applications such as data governance and data discovery:

Figure 1. Azure Purview: sample columns in an example file
“Contoso_Dev_IncomeStatement.tsv”, are automatically tagged as “Personal.IP
Address”, “Contoso.CustomerId”, “US.ZIP Code”, etc.

Data governance. Data protection regulations such as GDPR, PCI and CCPA impose
strict requirements on how sensitive personal data can be retained and
accessed. To ensure compliance, it is imperative that enterprises can
automatically identify sensitive data assets in their data lakes, so that
these data assets can be governed in accordance with regulatory requirements.

Data discovery. In order to improve the productivity of enterprise workers, it
is increasingly important for enterprise workers to discover and leverage data
assets relevant to their tasks, using self-service mechanisms such as data-set
search. Given the large number of data assets in modern enterprise data lakes,
and their nondescript, sometimes cryptic, nature, datasets search is clearly
challenging (compared to the web search for exampled) (halevy2016managing, ).
Suitable metadata tags/classifications associated with data assets can
significantly improve search relevance, and enhance the overall usefulness of
enterprise data lakes.

Figure 2. An example spreadsheet from Contoso, with many enterprise-specific
“custom” data-types. Figure 3. Azure Purview UI for custom-classification by-
example: after uploading an example data file, columns with inferred patterns
will be suggested for user to inspect and approve.

Auto-tagging of “standard” data-types. Given the importance of data tagging,
it is no surprise that leading vendors in this space all have features
relating to automated data-tagging.

Figure 1 shows a data-tagging feature in Azure Purview (purview, ). Out of the
box, the system can already recognize 100+ standard data types commonly found
in the public domain (purview-builtin-types, ).111Note that many of these
data-types are sensitive in nature, making them particularly relevant to data
governance and catalog vendors. In this particular example, it detects a few
sample columns to be of type “IP Address”, “US Zip Code”, etc. Note that these
are well-known data-types from the public domain, henceforth refer to as
“standard” data-types.

Because standard data-types are well-known and their corresponding “data-
taggers” can be reliably tested beforehand, auto-tagging features for standard
data-types are readily available and work well “out-of-the-box” in today’s
data-catalog and governance vendors (using a combination of techniques like
predefined regex patterns, bloom-filters, etc.).

Figure 4. Example “custom” data types, crawled from an enterprise data-lake at
Microsoft. Each column has a distinctive data pattern in proprietary formats,
encoding specific meanings. These custom data types are all common, occurring
in at least 5000 columns in our sample crawl with 7M columns.

Auto-tagging of “custom” data-types. While standard data-types capture an
important class of use cases, we observe that there is also a large number of
“custom” enterprise data types, which are unique to different industries and
enterprises.

In the example of Figure 2, column B contains unique IDs that this company
Contoso (a fictional company) assigns to their customers – in this case, these
values have a prefix of “CUST#” followed by six hexadecimal characters (e.g.,
“CUST#0FF125”). Ideally, we want to tag columns of this type as a new custom
data-type “Contoso.CustomerId”, like shown in the second column of Figure 1.
However, Contoso’s way of identifying customers is likely unique, as other
companies may devise their own unique-identifiers for customers – for example,
another company may use customer-identifiers that have a string prefix of “C-”
followed by a unique nine-digit number (e.g., “C-123456789”), while yet
another company may choose to use other types of UUID. These different forms
of customer-ids are custom-made (generated by some programs) and unique to
each enterprise, which are thus not well-known “standard” data-types that a
general-purpose data-catalog solution can possibly anticipate.

There are a large number of such custom data-types in today’s enterprises.
Figure 4 shows a few example custom data-types, harvested from a real
production data lake at Microsoft (chaiken2008scope, ). Each column here has a
distinctive data pattern, which uniquely identifies a custom data-type widely
used inside the company. For instance, the first column is known as Knowledge-
Base entity-id (Satori (gao2018building, )), which is a unique ID assigned to
real-world entities and used by the search engine Bing. Similarly, the second
column encodes the delivery status of Bing ads, etc.

Methods developed for tagging “standard” data types are clearly inapplicable
for these idiosyncratic “custom” data types, because they are unique to
different enterprises, and unlikely to be found from the public domain or
other enterprises.

This motivates us to look into ways that can tag custom enterprise data-types,
with minimal input from users.

Tagging custom-types by-examples. Unlike auto-tagging of standard data types,
which can be expected to work out-of-box, we believe that tagging custom
enterprise data types requires some amount of human input, (e.g. from data-
owners or domain-experts), to (1) determine relevant data of interest to
“tag”, and (2) provide suitable and meaningful tags that can describe the
underlying meaning of the custom data-type (i.e., an algorithm may infer
values of the form “CID-12345” to be a unique data-type, but cannot be sure of
its meaning).

We believe that a human-in-the-loop approach to tagging custom data-types have
two key desiderata:

$\bullet$ Low human-cost. The system should require minimal input from
enterprise users, ideally needing users to provide only one example column to
demonstrate the custom data type of interest (e.g., an example column in
Figure 4). Note that this is different from typical machine-learning tasks –
asking users to repeatedly provide feedback in the form of positive/negative
labels to tag one custom data type can be too costly in this setting.

$\bullet$ Low execution-cost. It is also important that any tag-inference
algorithm needs to be lightweight, in order for the feature to be cost-
effective on large enterprise data lakes. Although a deep analysis (e.g., a
full scan) of the data lake will typically yield better predictive accuracy,
the scale of the data (e.g., petabytes) makes a full scan too expensive. Thus,
auto-tagging algorithms should only perform a lightweight scan (e.g. a row-
wise sample per asset), in order for the feature to be viable in terms of
COGS.

In Section 2, we will discuss why existing techniques (e.g.,
(cortez2015annotating, ; yan2018synthesizing, ; zhao2019auto, ;
hulsebos2019sherlock, ; zhang2019sato, )) may be insufficient in such a
setting, either due to high human-costs, or high execution-cost.

In this work, we develop an initial version of this auto-tagging feature
called Auto-Tag. Unlike standard machine-learning, Auto-Tag has the advantage
of requiring only one labeled example column (low human costs), and unlike
content-based or dictionary-based approaches, does not require a full scan of
data files (low execution costs), because patterns can be reliably generalized
from small samples.

Using a variant of an algorithm we develop in (auto-validate-full, ), we first
perform lightweight (row-wise sampled) scans of data lakes offline, to build a
succinct index structure that is analogous to pre-training in machine-
learning. At online inference time, given an example column of interest users
point us to, Auto-Tag leverages the offline index to produce relevant data
patterns that can accurately describe the underlying domain of the custom type
of interest.

Figure 3 shows a by-example auto-tagging feature in Azure Purview, which uses
this technology to auto-tag custom data-types. From the UI, users can easily
upload a data file with target columns of interest. Leveraging a succinct
index structure built offline, a list of suggested data-tagging patterns can
be produced at an interactive speed, so that users can pick the desired
pattern corresponding to the column of interest, inspect the suggested
pattern, before approving the data-tagging rule. The tagging logic so created
would then be used to tag additional columns in the data lake matching the
given pattern during data scans.

Our experiments using real data from a production enterprise data lake at
Microsoft (chaiken2008scope, ) suggest that Auto-Tag is both accurate and
cost-efficient, for tagging custom data-types. We report experimental results
in Section 4.

##  2\. Related Works

Auto-tagging of data assets is an important topic, given the abundance of data
in data lakes today. We will review related data-tagging techniques below222We
focus on techniques that produce tags based on data-values in columns.
Orthogonal techniques leveraging other types of information also exist, e.g.,
program-flows (sen2014bootstrapping, )., and discuss why they are not
immediately applicable to our problem (high human-cost or execution cost).

We emphasize that different classes of techniques below are often suitable for
orthogonal types of data (e.g., natural-language content vs. machine-generated
data), and thus do not subsume each other.

Data-tagging by value-patterns. It is reported that a substantial fraction of
enterprise data columns have regex-like patterns (auto-validate-full, ), for
which pattern-based approaches are the most suitable.

There are many existing techniques from the data profiling literature, which
infers patterns based on example data-values. These include research
prototypes like Potter’s wheel (raman2001potter, ), X-System
(ilyas2018extracting, ), LearnPads (fisher2005pads, ; fisher2008dirt, ),
FlashProfile (padhi2018flashprofile, ), and commercial implementations like
Microsoft SQL Server SSIS (ssis-profiling, ), Trifacta (trifacta-profiling, ),
Ataccama (ataccama, ).

As we will highlight in Section 3, the goal of data-profiling is distinctively
different from data-tagging – it aims to find patterns to succinctly summarize
given data values only, which tend to produce overly-specific (or under-
generalized) patterns, which yield low recall when used for auto-tagging.
There is significant room for improvement, and is the focus of our corpus-
driven Auto-Tag approach.

Data-tagging by machine learning models. Machine-learning or deep-learning
based approaches, such as Auto-EM (zhao2019auto, ), Sherlock
(hulsebos2019sherlock, ) and Sato (zhang2019sato, ), have been developed to
tag columns with natural-language content (e.g., company-names, people-names,
etc.). Such approaches, however, are often a poor fit for machine-generated
data (e.g., GUID, employee-ID, etc.), and would complement pattern-based
approaches. Such approaches also typically require a non-trivial amount of
labeled data, increasing the cost of adoption for tagging custom enterprise
data-types (high human costs).

Data-tagging by value-overlap. Techniques have also been developed to tag
columns based on value overlap in enterprise tables (cortez2015annotating, )
and web tables (venetis2011recovering, ; taneva2013mining, ;
wang2012understanding, ), where the idea is that if a substantial fraction of
values in a given column match a known dictionary of values (e.g., a known
list of department-names or product-names), then the column can be tagged
accordingly.

When such “dictionaries of values” for enterprise concepts are not known a
priori, techniques are developed to harvest such “dictionaries” for data-
tagging. These techniques are known in the literature as set expansion
(he2011seisa, ; pantel2009web, ; wang2007language, ), concept discovery
(li2017discovering, ; ota2020data, ), and more broadly knowledge-base
construction (hoffart2013yago2, ; bollacker2008freebase, ; dong2014knowledge,
; gao2018building, ). These approaches, however, typically require full-scans
for high recall, thus introducing high execution-costs.

Data-tagging by synthesized programs. Because values of certain data types
(e.g., credit-card numbers, UPC codes) have unique signatures such as check-
sums, which can only be detected via specific program-logic, program-synthesis
based data-tagging have been proposed, which synthesize type-detection
functions using existing code (yan2018synthesizing, ). Such approaches,
however, requires the presence of a enterprise-specific code repository to be
effective.

##  3\. Auto-Tag By-Examples

Given the need of low human-costs and execution-costs discussed above, in this
work we set out to solve the auto-tagging problem, for string-valued custom-
types with syntactic patterns (our prior study (auto-validate-full, ) suggests
that this is an important class accounting for around 40% string-valued
columns in a production data lake).

We will first briefly describe the pattern language used.

###  3.1. Preliminary: Pattern Language

We use a standard pattern language (similar to (raman2001potter, )). Other
languages can also be plugged in Auto-Tag to produce corresponding patterns.

Figure 5 shows a standard generalization hierarchy, where leaf-nodes represent
the English alphabet, and intermediate nodes (e.g., <digit>, <letter>)
represent token that values can generalize into. A pattern is a sequence of
(leaf or intermediate) tokens, and for a given value $v$, this hierarchy
induces a space of all patterns consistent with $v$, denoted by
$\mathbf{P}(v)$. For instance, for a value $v=$ “9:07”, we could generate
$\mathbf{P}(v)=$ {“<digit>:<digit>{2}”, “<digit>+:<digit>{2}”,
“<digit>:<digit>+”, “<num>:<digit>+”, “9:<digit>{2}”, …}, among many other
options.

Given a column $C$ for which patterns need to be generated, we define the
space of candidate patterns, denoted by $\mathbf{P}(C)$, as the set of
patterns consistent with values $v\in C$. We use an in-house implementation to
produce patterns based on the hierarchy in Figure 5 (other hierarchies and
languages can be applied similarly in Auto-Tag).

###  3.2. Find Suitable “Domain” Patterns

Given the pattern language $\mathbf{P}$ described above, and given a data
lake, consisting of a large collection of tables $\mathbf{T}$ (which can be
flat files such as .csv, .tsv, .xls, .json, as well as database files and
database tables, etc.), at a high level our auto-tagging problem can be stated
as follows.

###### Definition 1.

Auto-tag by-examples. Given a data lake of tables $\mathbf{T}$, users
demonstrate a desired action to tag data of type $t$, by providing one example
column $C\in\mathbf{T}$ consisting of a set of values $C=\\{v_{i}\\}$ that are
of type $t$, and a tag $n(t)$ describing this type $t$. Let $\mathbf{P}(C)$ be
the set of all data-patterns consistent with $C$, our goal is to select a
suitable pattern $p\in\mathbf{P}(C)$, such that for any data column
$D\in\mathbf{T}$, if $p\in\mathbf{P}(D)$ or $p$ also matches the column $D$,
then $D$ is also likely of type $t$ (can be tagged as $n(t)$).

###### Example 2.

As a concrete example, users provide an example column $C_{1}$ shown in Figure
8, as well as a tag $n(C_{1})$, say “date”. The system should suggest a
suitable pattern “<letter>{3} <digit>{2} <digit>{4}” that best describe the
underlying data “domain”. Once this is reviewed and approved by users, it can
be used to tag additional columns in $\mathbf{T}$ matching the same pattern
(with tag “date”).

One challenge is that $\mathbf{P}(C)$ is large (there are many ways to
“generalize” a column $C$ into patterns). For a simple column of date-time
strings like in Figure 6, and using a standard generalization hierarchy as in
Figure 5, one could produce over 3 billion possible patterns. For example, the
first part (digit “9” for month) alone can be generalized in 7 different ways
shown in the top-right box of Figure 6, and the cross-product at each position
creates a large space (3.3 billion patterns) for this seemingly simple column.

Given a large space of candidates $\mathbf{P}(C)$, the key is to:

(1) Not “under-generalize”: or use overly restrictive patterns, which lead to
low recall for data-tagging; and

(2) Not “over-generalize”: or use overly generic patterns (e.g. the trivial
“.*”), which lead to low precision.

These are the key reasons why related techniques like pattern-profiling (e.g.,
Potter’s Wheel (raman2001potter, ), PADS (fisher2005pads, ), X-System
(ilyas2018extracting, ), FlashProfile (padhi2017flashprofile, ), etc.) are not
directly applicable to data-tagging, because they have very different
objectives.

Specifically, the goal of pattern-profiling is to succinctly “summarize” a
given set of values in column $C$, so that users can quickly understand what
is in $C$ without needing to scroll/inspect the entire $C$. Such techniques
explicitly consider only values in $C$, without needing to consider values not
present in $C$ (e.g., other valid values that are in the same “domain” as
$C$).

For example, classical pattern profiling methods like Potter’s Wheel
(raman2001potter, ) and FlashProfile (padhi2017flashprofile, ) would correctly
generate a desirable pattern “Mar <digit>{2} 2019” for $C_{1}$ in Figure 8,
which is valuable from a pattern-profiling’s perspective as it succinctly
describes all given values in $C_{1}$. However, this pattern is not suitable
for data-tagging, as it under-generalizes and would miss many similar date-
time columns like “Apr 01 2019”, thus yielding low recall. A more appropriate
data-tagging pattern should instead describe the entire “domain” of possible
values for this data-type, e.g., “<letter>{3} <digit>{2} <digit>{4}”.

A key challenge here is to select suitable patterns from $\mathbf{P}(C)$, when
only one example column $C$ is given. This is intuitively difficult if we only
look at $C$ – for the date examples in Figure 8, we as humans know the
ideally-generalized pattern for this type, but for data from proprietary
domains with ad-hoc formats (e.g., Figure 4), even humans may find it hard and
need to use additional evidence to reason about ideal patterns to describe the
corresponding “domain” (e.g., by inspecting similar-looking columns in the
lake $\mathbf{T}$).

Following this intuition, we propose a corpus-driven approach Auto-Tag that
leverages summary statistics of $\mathbf{T}$ (with similar-looking columns) to
determine the best pattern, which we describe next.

Figure 5. Example generalization hierarchy. Figure 6. Possible ways to
generalize a column of date-time strings, using the hierarchy in Figure 5.

###  3.3. Auto-Tag: Estimate Pattern Quality

Intuitively, a pattern $p(C)\in\mathbf{P}(C)$ is a good domain pattern if it
captures all valid values from the same domain, and a “bad” pattern if it
under-generalizes or over-generalizes.

Avoid under-generalization. We show that it is possible to infer whether
$p(C)$ under-generalizes, using summary statistics from $\mathbf{T}$ (without
human input).

Figure 7. Given a column $C$, leverage tables in a data lake $\mathbf{T}$, to
infer whether candidate patterns $p(C)$ over-generalize or under-generalize.

###### Example 3.

The left of Figure 7 shows a query column $C$ for which domain patterns need
to be generated. A few candidate patterns in $\mathbf{P}(C)$ are listed in the
middle. In this example, we know that $p_{1}(C)$, $p_{2}(C)$, $p_{3}(C)$ are
“bad” because they under-generalize the domain (too “narrow”).

We show that this can be inferred using $\mathbf{T}$ alone. Specifically,
$p_{1}(C)$ likely under-generalizes the domain, because we can find many
columns like $D\in\mathbf{T}$ shown on the right of Figure 7 that are “impure”
– these columns contain values that match $p_{1}(C)$, as well as values that
do not (e.g., “9/11/2019 09:12:03 AM”, where the day part does not match
$p_{1}(C)$). A large number of “impure” columns likely indicate under-
generalizations.

We can show that $p_{2}(C)$ also likely under-generalizes the domain, as it
makes many columns like $D$ “impure” (the “PM” part does not match
$p_{2}(C)$).

The same can be said about $p_{3}(C)$ (values like “10:02:20 AM” are
inconsistent with $p_{3}(C)$ because they have two-digit hours, whereas
$p_{3}(C)$ uses a single <digit>).

Using $p_{5}(C)$ to describe the domain, on the other hand, would not yield
many “impure” columns in $\mathbf{T}$, suggesting that it does not under-
generalize the domain.

Intuitively, we can use the impurity of $p$ on data columns $D\in\mathbf{T}$,
measured as the fraction of values in $D$ not matching $p$, to infer whether
$p$ is an under-generalization:

###### Definition 4.

The impurity of a candidate pattern $p$ on a data column $D\in\mathbf{T}$, is
defined as:

(1) $\text{Imp}_{D}(p)=\frac{|\\{v|v\in D,p\notin\mathbf{P}(v)\\}|}{|\\{v|v\in
D\\}|}$

###### Example 5.

In Figure 7, $\text{Imp}_{D}(p_{1})$ can be calculated as $\frac{2}{12}$,
since the first 2 values (with “9/11/2019”) out of 12 do not match $p_{1}$.
Similarly, $\text{Imp}_{D}(p_{3})$ can be calculated as $\frac{8}{12}$, since
the last 8 values in $D$ (with two-digit hours) do not match $p_{3}$.

Finally, $\text{Imp}_{D}(p_{5})$ is $\frac{0}{12}$, since all values in $D$
match $p_{5}$.

We note that if $p(C)$ is used to tag data in the same domain as $C$, then
Imp${}_{D}(p)$ directly corresponds to expected false-negative-rate (FNR), or
recall-loss for data-tagging tasks.

###### Definition 6.

The expected false-negative-rate (FNR) of using pattern $p(C)$ to tag a data
column $D$ drawn from the same domain as $C$, denoted by $\text{FNR}_{D}(p)$,
is defined as:

(2)
$\text{FNR}_{D}(p)=\frac{\text{FN}_{D}(p)}{\text{TP}_{D}(p)+\text{FN}_{D}(p)}$

Where TP${}_{D}(p)$ and FN${}_{D}(p)$ are the number of false-positive
detection and true-negative detection of $p$ on $D$, respectively.

Note that since $D$ is from the same domain as $C$, ensuring that
TP${}_{D}(p)$ and FN${}_{D}(p)$ = $|D|$, $\text{FNR}_{D}(p)$ can be rewritten
as:

(3) $\text{FNR}_{D}(p)=\frac{|\\{v|v\in D,p\notin\mathbf{P}(v)\\}|}{|\\{v|v\in
D\\}|}=\text{Imp}_{D}(p)$

Thus allowing us to estimate $\text{FNR}_{D}(p)$ using $\text{Imp}_{D}(p)$.

###### Example 7.

Continue with Example 5, it can be verified that the expected FNR of using $p$
as the domain pattern for $D$, directly corresponds to the impurity
$\text{Imp}_{D}(p)$ – e.g., using $p_{1}$ to tag $D$ has
$\text{FNR}_{D}(p_{1})$ = $\text{Imp}_{D}(p_{1})$ = $\frac{2}{12}$; while
using $p_{5}$ to tag $D$ has $\text{FNR}_{D}(p_{5})=\text{Imp}_{D}(p_{5})=0$,
etc.

Based on FNR${}_{D}(p)$ defined for one column $D\in\mathbf{T}$, we can in
turn define the estimated FNR on the entire corpus $\mathbf{T}$, using all
column $D\in\mathbf{T}$ where some value $v\in D$ matches $p$:

###### Definition 8.

Given a corpus $\mathbf{T}$, we estimate the FNR of pattern $p$ on
$\mathbf{T}$, denoted by FNR${}_{\mathbf{T}}(p)$, as:

(4) $\text{FNR}_{\mathbf{T}}(p)=\operatorname*{avg}_{D\in\mathbf{T},v\in
D,p\in\mathbf{P}(v)}{\text{FNR}_{D}(p)}$

###### Example 9.

Continue with the $p_{5}$ in Example 5 and Example 7. Suppose there are 5000
data columns $D\in\mathbf{T}$ where some value $v\in D$ matches $p_{5}$.
Suppose 2000 columns out of the 5000 have $\text{FNR}_{D}(p_{5})=0$, and the
remaining 3000 columns have $\text{FNR}_{D}(p_{5})=50\%$. The overall
$\text{FNR}_{\mathbf{T}}(p_{5})$ can be calculated as
$\frac{3000*50\%}{5000}=30\%$, using Equation (4).

Avoid over-generalization. So far we have focused on avoiding under-
generalization. Similarly we should also avoid over-generalization, such as
$p_{8}$ and $p_{9}$ shown in Figure 8. We achieve this by measuring coverage
of pattern $p$ over $\mathbf{T}$.

###### Definition 10.

The coverage of a candidate pattern $p$ on $\mathbf{T}$, is defined as:

(5) $\text{Cov}(p)=|\\{D|D\in\mathbf{T},p\in\mathbf{P}(D)\\}|$

Intuitively, among all candidate patterns that do not under-generalize (using
impurity-based estimates), we should pick the pattern with the least coverage,
which is the least likely to over-generalize.

###### Example 11.

Recall that in Example 3, we infer that $p_{1}(C)$, $p_{2}(C)$ and $p_{3}(C)$
likely under-generalize (thus can be excluded), while $p_{5}(C)$, $p_{8}(C)$
and $p_{9}(C)$ do not. For the remaining patterns, given a data lake with 10M
columns, we find the coverage of $p_{5}(C)$, $p_{8}(C)$ and $p_{9}(C)$ to be
20K, 500K and 10M, respectively. We can then pick $p_{5}(C)$ as the suitable
pattern for auto-tagging, as it does not under-generalize, and at the same
time is the least likely to over-generalize.

Given this intuition, we formalize pattern-inference as an optimization
problem below.

###  3.4. Problem Formulation: CMDT

We now describe the basic version of Auto-Tag as follows. Given an input query
column $C$ and a background corpus $\mathbf{T}$, we need to produce a domain
pattern $p(C)$, such that $p(C)$ is expected to have a low FNR but also with
few false positives. We formulate this as an optimization problem, called
Coverage-Minimizing version of Data-Tagging (CMDT), defined as:

(6) $\displaystyle\text{(CMDT)}\qquad{}\min_{p\in\mathbf{P}(C)}$
$\displaystyle\text{Cov}_{\mathbf{T}}(p)$ (7) s.t.
$\displaystyle\text{FNR}_{\mathbf{T}}(p)\leq r$ (8)
$\displaystyle\text{Cov}_{\mathbf{T}}(p)\geq m$

Equation (7) states that the expected recall loss of using $p$ as the domain
pattern for $C$, estimated from $\text{FNR}_{\mathbf{T}}(p)$, is lower than a
given threshold $r$. Equation (8) is an optional constraint that requires the
coverage of $p$, $\text{Cov}_{\mathbf{T}}(p)$, defined as the number of
columns in $\mathbf{T}$ that match $p$, to be greater than a given threshold
$m$ (otherwise the custom data-type may be too niche to be interesting).

The domain pattern $p$ we produce for $C$ is then the minimizer of CMDT from
the space of all candidate patterns $\mathbf{P}(C)$ (Equation (6)), which as
discussed minimizes the chance of over-generalization (and false-positives in
auto-tagging) for a given recall constraint.

We should note that the CMDT formulation is closely related to the FMDV
problem in (auto-validate-full, ). The two problems share the same problem
structure but use different objective functions (tailored to data-tagging and
data-validation, respectively). We leverage similar vertical-cut and
horizontal-cut algorithms in (auto-validate-full, ), and also optimization
methods (lightweight scan with offline indexing). Together, these mechanisms
achieve (1) interactive response time and (2) cost effectiveness (by scanning
a small fraction of rows per file). We refer readers to (auto-validate-full, )
for details of the algorithms in the interest of space.

Figure 8. Example showing different patterns produced for data-profiling vs.
data-tagging, because the two have very different objectives.

##  4\. Experiments

We implement our offline indexing algorithm in a Map-Reduce-like language
called Scope (zhou2012scope, ) with UDFs in C#, executed on a production
cluster (chaiken2008scope, ).

###  4.1. Benchmark Evaluation

Data set. We evaluate algorithms using a real corpus $\mathbf{T}$ with 7.2M
data columns, crawled from a production data lake at Microsoft (zhou2012scope,
).

Evaluation methodology. We randomly sample 1000 columns from $\mathbf{T}$ to
produce a benchmark set of columns, denoted by $\mathbf{B}$. We use the first
1000 values of each column to control column size variations.

Given a benchmark $\mathbf{B}$ with 1000 columns,
$\mathbf{B}=\\{C_{i}|i\in[1,1000]\\}$, we manually assign a ground-truth tag-
id for each column. This produces clusters of columns in the same data-type
and should be assigned same tags.

We then evaluate precision and recall of patterns generated on $\mathbf{B}$ as
follows. For each column $C_{i}\in\mathbf{B}$, we use the first $10\%$ of
values in $C_{i}$ (or 100 values) as the “training data”, denoted by
$C_{i}^{\text{train}}$, from which patterns need to be generated. Each
algorithm $A$ can observe $C_{i}^{\text{train}}$ and “learn” pattern
$A(C_{i}^{\text{train}})$. The inferred pattern is denoted as
$A(C_{i}^{\text{train}})$.

To test recall of $A(C_{i}^{\text{train}})$ when $C_{i}$ is used for auto-
tagging, denoted by $R_{A}(C_{i})$, we use the remaining $90\%$ of values from
$C_{i}$, as well as other columns in $\mathbf{B}$ with the same ground-truth
cluster-id. These are data columns drawn from the same data-type as $C_{i}$,
which we expect $A(C_{i}^{\text{train}})$ to match. We take chunks of 100
values from these columns as column-units, and compute recall by testing the
fraction of column-units that can be correctly tagged (under different
matching thresholds).

To test precision, denoted by $P_{A}(C_{i})$, we use columns in $\mathbf{B}$
with a different ground-truth cluster-id, which are from a different data-type
as $C_{i}$. We know that it is a false-positive if $A(C_{i}^{\text{train}})$
were to tag these columns. We compute precision accordingly, by taking chunks
of 100 values from these columns as column-units, and compute the fraction of
column-units that not incorrectly tagged by $A(C_{i}^{\text{train}})$.

The overall precision/recall on benchmark $\mathbf{B}$ is the average across
all cases:
$P_{A}(\mathbf{B})=\operatorname*{avg}_{C_{i}\in\mathbf{B}}P_{A}(C_{i})$, and
$R_{A}(\mathbf{B})=\operatorname*{avg}_{C_{i}\in\mathbf{B}}R_{A}(C_{i})$. Both
of these are between 0 and 1 as usual.

###  4.2. Methods Compared

We compare the following algorithms using benchmark $\mathbf{B}$, by reporting
precision/recall numbers $P_{A}(\mathbf{B})$ and $R_{A}(\mathbf{B})$.

Auto-Tag. This is our proposed approach using CMDT.

Potter’s Wheel (PWheel) (raman2001potter, ). This is an influential pattern-
profiling method that finds the best pattern based on minimal description
length (MDL).

SQL Server Patterns (ssis-profiling, ). SQL Server has a data-profiling
feature in SSIS and Data Tools. We invoke it programmatically to produce regex
patterns for each column.

XSystem (ilyas2018extracting, ). This recent approach develops a flexible
branch and merge strategy to pattern profiling. We use the authors’
implementation on GitHub (xsystem-code, ) to produce patterns.

FlashProfile (padhi2018flashprofile, ). FlashProfile is a recent approach to
pattern profiling, which clusters similar values by a distance score. We use
the authors’ implementation in NuGet (FlashProfileCode, ).

We also compare with Grok (grok, ), which is a popular approach that uses a
collection of curated regex patterns to detect common types in log messages;
schema-matching (do2002comparison, ) based methods (followed by pattern-
profiling); and a simple Value-Union (cortez2015annotating, ) method that is
more suitable for natural-language content. These methods are not as effective
(e.g., producing overly-generic patterns with low precision), and are omitted
from the results (to ensure we can zoom in on the competitive methods in the
figures).

###  4.3. Experimental Results

We evaluate different methods based on tagging quality (precision/recall),
latency, and memory footprint.

Quality. Figure 9 shows precision/recall of all methods using the enterprise
benchmark $\mathbf{B}$ with 1000 randomly sampled test cases. It can be seen
that Auto-Tag is substantially better than other methods in both precision and
recall.

Among all the baselines, we find data profiling techniques like PWheel and
FlashProfile to also be of high-precision. However, these techniques tend to
under-generalize and produce lower recall (because as discussed, data
profiling techniques aim to optimize for a fundamentally different objective
compared to data-tagging).

Figure 9. Precision/Recall on 1000 randomly sampled cosmos data. Results are
scaled to test columns that have patterns.

Latency. Given that it is important to produce regex suggestions at
interactive speed (for users to inspect and verify), we compare the mean and
max latency of different methods on 1000 benchmark test columns. It can be
seen that Auto-Tag is clearly interactive, where the max latency is 0.663
second.

In comparison, methods like FlashProfile and XSystem use expensive clustering,
which on average take 6-7 seconds per input column, where the max latency per
column is close to 6 minutes for both methods. Note that for both FlashProfile
and XSystem we use authors’ original implementations (xsystem-code, ;
FlashProfileCode, ).

Memory footprint. We also evaluate the average and max memory usage for
pattern-learning per input column. Auto-Tag avoids expensive bottom-up
enumeration and is lightweight, which uses an average of 1.9MB memory (2.8MB
max). In comparison, clustering-based pattern-profiling methods like
FlashProfile takes 162MB memory on average, with a max memory usage of 7.9GB.

Method | mean-latency (ms) | max-latency (ms)
---|---|---
Auto-Tag | 12 | 663
FlashProfile | 7076 | 359382
XSystem | 6411 | 346996
Table 1. Mean/max latency on 1000 benchmark cases.

##  5\. Conclusions

Observing the need to data-tagging for custom data types in enterprise data
lakes, we propose a corpus-driven Auto-Tag approach to infer relevant data
patterns. This is shown to be accurate and cost-effective, when evaluated on
real enterprise data from a production data lake.

## References

  * [1] Alation Data Catalog.  https://www.alation.com/.
  * [2] ataccama: Data Profiling.  https://www.ataccama.com/product/data-discovery-and-profiling.
  * [3] AWS Glue Data Catalog.  https://docs.aws.amazon.com/glue/latest/dg/what-is-glue.html.
  * [4] Azure Purview.  https://azure.microsoft.com/en-us/services/purview/.
  * [5] Azure Purview: 100+ standard data-types for auto-tagging.  https://docs.microsoft.com/en-us/azure/purview/supported-classifications.
  * [6] Collibra Data Catalog.  https://www.collibra.com/data-catalog.
  * [7] FlashProfile package.  https://www.nuget.org/packages/Microsoft.ProgramSynthesis.Extraction.Text/.
  * [8] Google Cloud Data Catalog.  https://cloud.google.com/data-catalog.
  * [9] Grok Patterns.  https://github.com/elastic/elasticsearch/blob/master/libs/grok/src/main/resources/patterns/grok-patterns.
  * [10] Informatica Rev.  https://www.informatica.com/products/data-quality/rev.html.
  * [11] Self-service data preparation, worldwide, 2016.  https://www.gartner.com/doc/3204817/forecast-snapshot-selfservice-data-preparation.
  * [12] SQL Server SSIS profiling.  https://docs.microsoft.com/en-us/sql/integration-services/control-flow/data-profiling-task?view=sql-server-ver15.
  * [13] Talend data validation.  https://www.talendbyexample.com/talend-data-validation.html.
  * [14] Trifacta built-in data types.  https://docs.trifacta.com/display/PE/Supported+Data+Types.
  * [15] Trifacta: Data Profiling.  https://www.trifacta.com/data-profiling/.
  * [16] Waterline Data Catalog.  https://www.waterlinedata.com/.
  * [17] XSystem Code.  https://bitbucket.org/andrewiilyas/xsystem-old/src/outlier-detection/.
  * [18] K. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor.  Freebase: a collaboratively created graph database for structuring human knowledge.  In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1247–1250, 2008.
  * [19] R. Chaiken, B. Jenkins, P.-Å. Larson, B. Ramsey, D. Shakib, S. Weaver, and J. Zhou.  Scope: easy and efficient parallel processing of massive data sets.  Proceedings of the VLDB Endowment, 1(2):1265–1276, 2008.
  * [20] E. Cortez, P. A. Bernstein, Y. He, and L. Novik.  Annotating database schemas to help enterprise search.  Proceedings of the VLDB Endowment, 8(12):1936–1939, 2015.
  * [21] H.-H. Do, S. Melnik, and E. Rahm.  Comparison of schema matching evaluations.  In Net. ObjectDays: International Conference on Object-Oriented and Internet-Based Technologies, Concepts, and Applications for a Networked World, pages 221–237. Springer, 2002.
  * [22] X. Dong, E. Gabrilovich, G. Heitz, W. Horn, N. Lao, K. Murphy, T. Strohmann, S. Sun, and W. Zhang.  Knowledge vault: A web-scale approach to probabilistic knowledge fusion.  In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 601–610, 2014.
  * [23] K. Fisher and R. Gruber.  Pads: a domain-specific language for processing ad hoc data.  ACM Sigplan Notices, 40(6):295–304, 2005.
  * [24] K. Fisher, D. Walker, K. Q. Zhu, and P. White.  From dirt to shovels: fully automatic tool generation from ad hoc data.  ACM SIGPLAN Notices, 43(1):421–434, 2008.
  * [25] Y. Gao, J. Liang, B. Han, M. Yakout, and A. Mohamed.  Building a large-scale, accurate and fresh knowledge graph.  In SigKDD, 2018.
  * [26] A. Y. Halevy, F. Korn, N. F. Noy, C. Olston, N. Polyzotis, S. Roy, and S. E. Whang.  Managing google’s data lake: an overview of the goods system.  IEEE Data Eng. Bull., 39(3):5–14, 2016.
  * [27] M. Hazewinkel, editor.  Encyclopedia of Mathematics.  Springer Science+Business Media B.V. / Kluwer Academic Publishers, 2001\.
  * [28] Y. He and D. Xin.  Seisa: set expansion by iterative similarity aggregation.  In Proceedings of the 20th international conference on World wide web, pages 427–436, 2011.
  * [29] Y. He, D. Xin, V. Ganti, S. Rajaraman, and N. Shah.  Crawling deep web entity pages.  In Proceedings of the sixth ACM international conference on Web search and data mining, pages 355–364, 2013.
  * [30] J. Hoffart, F. M. Suchanek, K. Berberich, and G. Weikum.  Yago2: A spatially and temporally enhanced knowledge base from wikipedia.  Artificial Intelligence, 194:28–61, 2013.
  * [31] M. Hulsebos, K. Hu, M. Bakker, E. Zgraggen, A. Satyanarayan, T. Kraska, Ç. Demiralp, and C. Hidalgo.  Sherlock: A deep learning approach to semantic data type detection.  In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1500–1508, 2019.
  * [32] A. Ilyas, J. M. da Trindade, R. C. Fernandez, and S. Madden.  Extracting syntactical patterns from databases.  In 2018 IEEE 34th International Conference on Data Engineering (ICDE), pages 41–52. IEEE, 2018.
  * [33] P. V. Konda.  Magellan: Toward building entity matching management systems.  The University of Wisconsin-Madison, 2018.
  * [34] K. Li, Y. He, and K. Ganjam.  Discovering enterprise concepts using spreadsheet tables.  In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1873–1882, 2017.
  * [35] Y. Li, J. Li, Y. Suhara, A. Doan, and W.-C. Tan.  Deep entity matching with pre-trained language models.  arXiv preprint arXiv:2004.00584, 2020.
  * [36] J. Madhavan, D. Ko, Ł. Kot, V. Ganapathy, A. Rasmussen, and A. Halevy.  Google’s deep web crawl.  Proceedings of the VLDB Endowment, 1(2):1241–1252, 2008.
  * [37] S. Mudgal, H. Li, T. Rekatsinas, A. Doan, Y. Park, G. Krishnan, R. Deep, E. Arcaute, and V. Raghavendra.  Deep learning for entity matching: A design space exploration.  In Proceedings of the 2018 International Conference on Management of Data, pages 19–34, 2018.
  * [38] F. Naumann, C.-T. Ho, X. Tian, L. M. Haas, and N. Megiddo.  Attribute classification using feature analysis.  In Proceedings of ICDE, 2002.
  * [39] M. Ota, H. Müller, J. Freire, and D. Srivastava.  Data-driven domain discovery for structured datasets.  Proceedings of the VLDB Endowment, 13(7):953–967, 2020.
  * [40] S. Padhi, P. Jain, D. Perelman, O. Polozov, S. Gulwani, and T. Millstein.  Flashprofile: a framework for synthesizing data profiles.  Proceedings of the ACM on Programming Languages, 2(OOPSLA):1–28, 2018.
  * [41] S. Padhi, P. Jain, D. Perelman, O. Polozov, S. Gulwani, and T. D. Millstein.  Flashprofile: Interactive synthesis of syntactic profiles.  2017\.
  * [42] P. Pantel, E. Crestan, A. Borkovsky, A.-M. Popescu, and V. Vyas.  Web-scale distributional similarity and entity set expansion.  In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 938–947, 2009.
  * [43] V. Raman and J. M. Hellerstein.  Potter’s wheel: An interactive data cleaning system.  In VLDB, volume 1, 2001.
  * [44] D. Ritze, O. Lehmberg, and C. Bizer.  Matching html tables to dbpedia.  In Proceedings of WIMS, page 10. ACM, 2015.
  * [45] A. Sen.  Metadata management: past, present and future.  Decision Support Systems, (1), 2004.
  * [46] S. Sen, S. Guha, A. Datta, S. K. Rajamani, J. Tsai, and J. M. Wing.  Bootstrapping privacy compliance in big data systems.  In 2014 IEEE Symposium on Security and Privacy, pages 327–342. IEEE, 2014.
  * [47] D. Shepherd, K. Damevski, B. Ropski, and T. Fritz.  Sando: an extensible local code search framework.  In Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering, page 15. ACM, 2012.
  * [48] Y. L. Simmhan, B. Plale, and D. Gannon.  A survey of data provenance in e-science.  SIGMOD Rec., 2005.
  * [49] A. Singhal.  Introducing the knowledge graph: things, not strings.  Official google blog, 2012.
  * [50] J. Song and Y. He.  Auto-validate: Unsupervised data validation using data-domain patterns inferred from data lakes.  In SIGMOD 2021 (to appear).
  * [51] F. M. Suchanek, G. Kasneci, and G. Weikum.  Yago: a core of semantic knowledge.  In Proceedings of WWW, 2007.
  * [52] A. Talaika, J. Biega, A. Amarilli, and F. M. Suchanek.  Ibex: harvesting entities from the web using unique identifiers.  In Proceedings of WebDB, 2015.
  * [53] B. Taneva, T. Cheng, K. Chakrabarti, and Y. He.  Mining acronym expansions and their meanings using query click log.  In Proceedings of the 22nd international conference on World Wide Web, pages 1261–1272, 2013.
  * [54] P. Venetis, A. Y. Halevy, J. Madhavan, M. Pasca, W. Shen, F. Wu, and G. Miao.  Recovering semantics of tables on the web.  2011\.
  * [55] J. Wang, H. Wang, Z. Wang, and K. Q. Zhu.  Understanding tables on the web.  In International Conference on Conceptual Modeling, pages 141–155. Springer, 2012.
  * [56] R. C. Wang and W. W. Cohen.  Language-independent set expansion of named entities using the web.  In Seventh IEEE international conference on data mining (ICDM 2007), pages 342–350. IEEE, 2007.
  * [57] C. Yan and Y. He.  Synthesizing type-detection logic for rich semantic data types using open-source code.  In Proceedings of the 2018 International Conference on Management of Data, pages 35–50, 2018.
  * [58] D. Zhang, Y. Suhara, J. Li, M. Hulsebos, Ç. Demiralp, and W.-C. Tan.  Sato: Contextual semantic type detection in tables.  arXiv preprint arXiv:1911.06311, 2019.
  * [59] C. Zhao and Y. He.  Auto-em: End-to-end fuzzy entity-matching using pre-trained deep models and transfer learning.  In The World Wide Web Conference, pages 2413–2424, 2019.
  * [60] F. Zhao, J. Zhou, C. Nie, H. Huang, and H. Jin.  Smartcrawler: a two-stage crawler for efficiently harvesting deep-web interfaces.  IEEE transactions on services computing, 9(4):608–620, 2015.
  * [61] A. X. Zheng, M. I. Jordan, B. Liblit, M. Naik, and A. Aiken.  Statistical debugging: simultaneous identification of multiple bugs.  In Proceedings of the 23rd international conference on Machine learning, pages 1105–1112. ACM, 2006.
  * [62] Q. Zheng, Z. Wu, X. Cheng, L. Jiang, and J. Liu.  Learning to crawl deep web.  Information Systems, 38(6):801–819, 2013.
  * [63] J. Zhou, N. Bruno, M.-C. Wu, P.-A. Larson, R. Chaiken, and D. Shakib.  Scope: parallel databases meet mapreduce.  The VLDB Journal, 21(5):611–636, 2012.


# Twists of Albanese varieties over function fields with large ranks

Abolfazl Mohajer, Sajad Salami  Johannes Gutenberg Universität Mainz, Institut
für Mathematik, Staudingerweg 9, 55099 Mainz, Germany  Instítuto de Matemática
e EstatísticaUniversidade Estadual do Rio do Janeiro, Brazil  mohajer@uni-
mainz.de<EMAIL_ADDRESS>
###### Abstract.

In this paper we construct abelian varieties of large Mordell-Weil rank over
function fields. We achieve this by using a generalization of the notion of
Prym variety to higher dimensions and a structure theorem for the Mordell-Weil
group of abelian varieties over function fields proven in our previous works.
We consider abelian and dihedral covers of the projective space and apply the
above results to the twists of their Albanese varieties.

###### Key words and phrases:

Prym variety, Rational points, Galois covering

###### 2010 Mathematics Subject Classification:

14G05, 14H30, 14H40

##  1\. Introduction

Let $A$ be an abelian variety over a field $k$ of characteristic $\geq 0$. Let
$A(k)$ be the set of $k$-rational points of $A$. If $k$ is a number field or a
function field satisfying some mild conditions, then it is well-known that
$A(k)$ is a finitely generated abelian group. The rank of this group is called
the Mordell-Weil rank of $A$. Finding abelian varieties with large rank is an
interesting problem in number theory. In this paper we investigate this
problem by considering abelian and dihedral covers of algebraic varieties. Let
$n\in\mathbb{N}$ be coprime to the characteristic of the field $k$ and such
that $k$ contains a primitive $n$-th root of unity $\xi$. Suppose there is an
embedding $G\hookrightarrow\operatorname{Aut}(A)$ with $G$ a finite group of
order $|G|=n$. Further, let $X,Y$ be smooth projective varieties over $k$ with
function fields $L,K$ respectively. Let $f:X\to Y$ be a $G$-Galois covering.
To such a covering we have associated in [8] an abelian varietiy $P(X/Y)$, the
so-called _Prym variety_ , that generalizes the notion of Prym variety in the
case of coverings of curves. The group $G$ is also the Galois group of the
Galois field extension $L|K$. By the results of [2, 4], the twist of $A$ by
the extension $L|K$ is equivalent to the twist by the 1-cocyle $a=(a_{g})\in
Z^{1}(G,\operatorname{Aut}(A))$ given by $a_{g}=g$, where in the notation
$a_{g}$, $g$ is viewed as a group element and on the right side as an
automorphism of $A$ corresponding to $g\in G$ (i.e., we identify $g$ with its
image $g\in G\hookrightarrow\operatorname{Aut}(A)$). By the results of [8]
there is an isomorphism for $A_{a}(K)$ in terms of the Prym variety.
Furthermore, [8] gives a description of the Prym variety of the $m$-times self
product $\prod_{i}f$ of the $G$-cover $f:X\to Y$ with itself, which
generalizes the results of [10] and [4] for cyclic and double covers. Such a
product yields a $G$-Galois field extension (by considering the function
fields of $\prod_{i}X$ and $\prod_{i}Y$) and hence a twist of the abelian
varieties defined over the function field of these varieties. In this paper,
on the one hand we consider abelian covers $X\to{\mathbb{P}}^{\ell}$ of the
$\ell$-dimensional projective space and using the above results show that the
rank of the twisted Albanese variety $\widetilde{\operatorname{Alb}(X)}$
becomes arbitrarily large if one takes $m$ to be large enough. Furthermore, we
consider non-abelian dihedral covers $C\to{\mathbb{P}}^{1}$ of the projective
line and show that the rank of the twist of the Jacobian
$\operatorname{Jac}_{a}(C)$ can be made arbitrarily large. In fact one can
generalize this result also to general dihedral covers of higher-dimensional
projective spaces ${\mathbb{P}}^{\ell}$. Our results also more generally apply
to metacyclic covers of the projective spaces.

##  2\. The Prym variety of Galois coverings and twists

Let $V$ be a smooth projective variety over a field $k$. The _Albanese
variety_ of $V$ is the initial object for the morphisms from $V$ to abelian
varieties. If $V$ is defined over a field of characterictic zero, then it is
the following abelian variety,

(2.0.1)
$\operatorname{Alb}(V)=H^{0}(V,\Omega^{1}_{V})^{*}/H_{1}(V,\mathbb{Z})$

Let $X,Y$ be smooth projective varieties with $f:X\to Y$ a Galois covering
with Galois group $G$. In other words there is a finite group $G$ with $|G|=n$
that has a $G$-linearized action on $X$ such that $Y\coloneqq X/G$ and $f:X\to
Y$ is the quotient map. By the universal property of the albanese variety
there is an induced action of $G$ on $\operatorname{Alb}(X)$. Let
${(\operatorname{Alb}(X)^{G})}^{0}$ be the largest abelian subvariety of
$\operatorname{Alb}(X)$ fixed (pointwise) under this action. Equivalently,
${(\operatorname{Alb}(X)^{G})}^{0}$ is the connected component of the identity
of the subvariety $\operatorname{Alb}(X)^{G}$ of fixed points of
$\operatorname{Alb}(X)$ under the action of $G$.

Let $g\in G$ be an element of the group $G$. The automorphism of
$\operatorname{Alb}(X)$ induced by $g$ is also denoted also by $g$ throughout
the manuscript. Using this notation let
$\operatorname{Nm}G:\operatorname{Alb}(X)\to\operatorname{Alb}(X)$ be the
_norm endomorphism_ of $\operatorname{Alb}(X)$ given by
$\operatorname{Nm}G:=\sum_{g\in G}g$. This definition is motivated by [9],
Prop 3.1 in the case of curves and extends it to higher dimensions.

The following definition of Prym variety for the covering $f$ is given in [8],
Definition 2.1.

######  Definition 2.1.

Let $f:X\to Y$ be $G$-Galois covering of smooth projective varieties over a
field $k$. The Prym variety $P(X/Y)$ of the covering $f$ is defined to be

(2.1.1)
$P(X/Y):=\frac{\operatorname{Alb}(X)}{{(\operatorname{Alb}(X)^{G})}^{0}}$

In particular $P(X/Y)$ is (isogeneous to) the complementary abelian subvariety
in $\operatorname{Alb}(X)$ of (the abelian subvariety)
${(\operatorname{Alb}(X)^{G})}^{0}$ .

For the classical case of Prym varieties of double coverings of curves we
refer to [1]. The following result has been proven in [8], Prop 2.2 and gives
an equivalent description of the Prym variety and shows furthermore that it
coincides with the Prym variety of covers of curves (see [1], [6] and [9]) up
to isogeny.

######  Proposition 2.2.

With the above notation,

$P(X/Y)=\frac{\operatorname{Alb}(X)}{\operatorname{Im}\operatorname{Nm}G}\sim{(\ker\operatorname{Nm}G)}^{0}.$

######  Remark 2.3.

The albanese variety of curves coincides with the Jacobian variety. Hence if
$X$ and $Y$ are curves, the Prym variety $P(X/Y)$ coincides with the Prym
variety for covers of curves, see [6] and [7]. In this case there are two
homomorphisms: the norm homomorphism
$\operatorname{Nm}_{f}:\operatorname{Jac}(X)\to\operatorname{Jac}(Y)$ and the
pull-back homomorphism $f^{*}:\operatorname{Jac}(Y)\to\operatorname{Jac}(X)$
and it holds that: $\operatorname{Nm}G=f^{*}\circ\operatorname{Nm}_{f}$, so
that
$P(X/Y)={(\ker\operatorname{Nm}_{f})}^{0}={(\ker\operatorname{Nm}G)}^{0}$, see
[9], Prop 3.1.

Given a $G$-Galois covering $f:X\to Y$ defined over $k$, one can form the
$m$-times self product $f_{m}:\prod_{i=1}^{m}X\to\prod_{i=1}^{m}Y$. Then the
diagonal embedding $G\hookrightarrow\prod_{i}G:=G\times\cdots\times G$ gives a
subgroup of $\prod_{i}G$ isomorphic to $G$. We denote this subgroup by
$\tilde{G}$. This gives rise to an intermediate Galois covering
$f^{\prime}_{m}:\prod_{i=1}^{m}X\to(\prod_{i=1}^{m}X)/\tilde{G}$. Let us write
$\mathcal{X}=\prod_{i=1}^{m}X$ and $\mathcal{Y}=(\prod_{i}X)/\tilde{G}$. We
are interested in the Prym variety $P(\mathcal{X}/\mathcal{Y})$. We have

######  Proposition 2.4 ([8], Prop 2.5).

With the above notation, there is an isogeny

(2.4.1) $P(\mathcal{X}/\mathcal{Y})\sim_{k}\prod_{i=1}^{m}P(X_{i}/Y_{i})$

We recall from [2] the notion of twist of a smooth projective variety. Suppose
$k^{\prime}/k$ is a $G$-Galois extension of fields. Let ${\mathcal{A}}$ be a
$G$-set and suppose $E$ is another $G$-set which is also a left
${\mathcal{A}}$-set. Let $a=(a_{g})\in
Z^{1}(G,\operatorname{Aut}({\mathcal{A}}))$ be a 1-cocycle of ${\mathcal{A}}$.
For any $g\in G$ and $x\in E$, we denote by ${}^{g}x$ the left action of $g$
on $x$. The $G$-set $E$ with this action of $G$ is denoted by $E_{a}$ and is
called the twist of $E$ by $a$. If $X$ is a projective smooth variety over
$k$, we denote by $\operatorname{Aut}(X)$ the automorphism scheme of $X$ and
let $a=(a_{g})\in Z^{1}(G,\operatorname{Aut}(X))$. There exists a projective
variety $\widetilde{X}$ over $k$ and $k^{\prime}$-isomorphism
$f^{\prime}:X\otimes_{k}k^{\prime}\to\widetilde{X}\otimes_{k}k^{\prime}$ such
that ${}^{g}(f^{\prime})=f^{\prime}\circ a_{g}$ for every $g\in G$. We denote
the variety $\widetilde{X}$ by $X_{a}$ and call it _the twist_ of $X$ by $a$.
Of course the two notion of twist introduced above are compatible in the sense
that the map $f^{\prime}:X(k^{\prime})\to\widetilde{X}(k^{\prime})$ induces an
isomorphism between $X(k^{\prime})_{a}\cong
X_{a}(k^{\prime})(=\widetilde{X}(k^{\prime}))$.

In this paper, we are mainly interested in the case where we have an abelian
variety $A$ and its twist $A_{a}$ by a field extension $L|K$ or equivalently a
twist by the 1-cocycle $a=(a_{g})\in Z^{1}(G,\operatorname{Aut}(A))$ given by
$a_{g}=g$. For such a $1$-cocycle, let
$\operatorname{Alb}(a):=\operatorname{Alb}(a_{g})$ for $g\in G$ which
satisfies also $1$-cocycle conditions, so that $\operatorname{Alb}(a)\in
Z^{1}(G,\operatorname{Aut}(\operatorname{Alb}(X)))$.

The relation between Albanese variety and its twists is given by the following
proposition which is proven in [11], Proposition 3.2.

######  Proposition 2.5.

The twist $\operatorname{Alb}(X)_{\operatorname{Alb}(a)}$ of
$\operatorname{Alb}(X)$ by the $1$-cocycle $\operatorname{Alb}(a)$ is
$K$-isomorphic to $\operatorname{Alb}(X_{a})$. Equivalently
$\widetilde{\operatorname{Alb}(X)}$, the twist of
$\operatorname{Alb}({\mathcal{X}})$, is $K$-isomorphic to
$\operatorname{Alb}(\tilde{X})$.

##  3\. The Main results

Given an integer $n\geq 2$, let
$\pi:{\mathcal{X}}^{\prime}\rightarrow{\mathcal{X}}$ be a $G$-Galois cover of
smooth projective varieties of degree $n$, both as well as $\pi$ defined over
a field $k$. Denote by ${\mathcal{K}}$ and ${\mathcal{L}}$ the function fields
of ${\mathcal{X}}$ and ${\mathcal{X}}^{\prime}$ respectively. Assume that
${\mathcal{A}}$ is an abelian variety such that there is an embedding
$G\hookrightarrow\operatorname{Aut}({\mathcal{A}})$, and let
${\mathcal{A}}[n](k)$ be the group of $k$-rational $n$-division points on
${\mathcal{A}}$. Define $\tilde{{\mathcal{A}}}$ to be the twist of
${\mathcal{A}}$ by the $G$-extension ${\mathcal{L}}|{\mathcal{K}}$, or
equivalently, by the $1$-cocycle
$a=(a_{u})\in{\mathcal{Z}}^{1}(G,\operatorname{Aut}({\mathcal{A}}))$, where
$a_{u}=u$ for $u\in G$ (we denote the image of $u$ under
$G\hookrightarrow\operatorname{Aut}({\mathcal{A}})$ again by $u$) and $G$ is
the Galois group of the extension ${\mathcal{L}}|{\mathcal{K}}$. The following
theorem describes the structure of the Mordell-Weil group of
${\mathcal{K}}$-rational points on the twist $\tilde{{\mathcal{A}}}$. This is
Proven in [8], Theorem 2.4 and generalizes the main results of [5, 10, 12].

######  Theorem 3.1.

Notation being as above, we assume that there exists a $k$-rational point on
${\mathcal{X}}^{\prime}.$ Then, as an isomorphism of abelian groups, we have:

(3.1.1) $\tilde{{\mathcal{A}}}({\mathcal{K}})\cong{\rm
Hom}_{k}(P({{\mathcal{X}}^{\prime}/{\mathcal{X}}}),{\mathcal{A}})\oplus{\mathcal{A}}[n](k).$

Moreover, if $P({{\mathcal{X}}^{\prime}/{\mathcal{X}}})$ is $k$-isogenous to
${\mathcal{A}}^{m}\times{\mathcal{B}}$ for some integer $m>0$ and
${\mathcal{B}}$ is an abelian variety defined over $k$ so that none of its
irreducible components is $k$-isogenous to ${\mathcal{A}}$, then

${\rm rank}(\tilde{{\mathcal{A}}}({\mathcal{K}}))=m\cdot{\rm
rank}(\operatorname{End}_{k}({\mathcal{A}})).$

We apply Theorem 3.1 in the following situations. First, we consider abelian
Galois covers $X\to\mathbb{P}^{\ell}$ with Galois group $G$, where for an
integer $\ell\geq 1$, we denote by ${\mathbb{P}}^{\ell}$ the
$\ell$-dimensional projective space over $\bar{k}$ an algebraically closed
field containing the number field $k$. For an integer $m\geq 1$, we define
$U_{m}$ to be the $m$-times self-fiber product of $X$ over $k$ and we let
$V_{m}$ be the quotient of $U_{m}$ by $G$. Considering a non-singular model
for $X$, say ${\mathcal{X}}$, we obtain the non-singular models
${\mathcal{U}}_{m}$ and ${\mathcal{V}}_{m}$ for $U_{m}$ and $V_{m}$,
respectively. Denoting by ${\mathcal{K}}$ and ${\mathcal{L}}$, the function
field of ${\mathcal{U}}_{m}$ and ${\mathcal{V}}_{m}$, respectively, we obtain
the following theorem which generalizes Theorem 1.2 of [10] and Theorem 1.1 of
[11].

######  Theorem 3.2.

Notation being as above, we assume that there exists a $k$-rational point on
$X$ and hence on ${\mathcal{X}}$. Then, as an isomorphism of abelian groups,
we have:

$\widetilde{\operatorname{Alb}({\mathcal{X}})}({\mathcal{K}})\cong\operatorname{Alb}(\tilde{{\mathcal{X}}})({\mathcal{K}})\cong\big{(}\operatorname{End}_{k}(\operatorname{Alb}({\mathcal{X}}))\big{)}^{m}\oplus\operatorname{Alb}({\mathcal{X}})[n](k))$

and hence,

${\rm
rank}(\widetilde{\operatorname{Alb}({\mathcal{X}})}({\mathcal{K}}))=m\cdot{\rm
rank}(\operatorname{End}_{k}(\operatorname{Alb}({\mathcal{X}}))),$

where $\operatorname{End}_{k}(*)$ is the ring of endomorphisms over $k$ of its
origin $(*)$ and ${\rm rank}(\operatorname{End}_{k}(*))$ denotes its rank as a
${\mathbb{Z}}$-module.

###### Proof.

Without loss of generality, we may identify $\bar{k}$ with $\mathbb{C}$ the
field of complex numbers. By the main result of G. Yun in [13], any finite
abelian Galois cover $X\to\mathbb{P}^{\ell}$ with $\ell\geq 1$ is determined
by a collection of equations of the form
$w^{n_{j}}_{j}=f_{j}(x_{1},\cdots,x_{\ell})$ for $j=1,2,\dots,r$ for some
integer $r\geq 1$ where $n_{1},\cdots,n_{r}$ are integers such that $n_{j}$
divides $n_{j+1}$ for $j=1,\cdots,r-1$. Hence, the fibered product
$U_{m}=X^{(1)}\times_{k}\cdots\times_{k}X^{(m)}$ for any integer $m\geq 1$,
where $X^{(i)}$ is a copy of $X$, can be defined by the affine equations
$w_{i,j}^{n_{j}}=f_{j}(x_{i,1},\cdots,x_{i,\ell})$ for $1\leq i\leq m$ and
$1\leq j\leq r$. Now, consider the fibered product
${\mathcal{U}}_{m}={\mathcal{X}}_{n}^{(1)}\times_{k}\cdots\times_{k}{\mathcal{X}}_{n}^{(m)}$,
where ${\mathcal{X}}_{n}^{(i)}$ is a non-singular projective model of
$X^{(i)}$ for each $1\leq i\leq m$. Note that ${\mathcal{U}}_{m}$ can be
viewed as a non-singular model of $U_{m}$. Denote by $L$ and ${\mathcal{L}}$
the function fields of $U_{m}$ and ${\mathcal{U}}_{m}$, respectively. Let
$V_{m}$ and ${\mathcal{V}}_{m}$ be the quotient of $U_{m}$ and
${\mathcal{U}}_{m}$ by $G$ and $\bar{G}$, the image of $G$ in
$\prod_{i=1}^{m}G$, and denote by $K$ and ${\mathcal{K}}$ the function fields
of $V_{m}$ and ${\mathcal{V}}_{m}$, respectively. Then, both of the extensions
$L|K$ and ${\mathcal{L}}|{\mathcal{K}}$ are finite abelian extension of order
$n$. Indeed, we have $L\subset k(x_{i,e},w_{i,j}\mid 1\leq e\leq\ell,\ 1\leq
i\leq m,\ 1\leq j\leq r)$ where $x_{i,e}$’s are independent transcendental
variables and each $w_{i,j}$ satisfy the above equations. Then, by definition,
$K$ is the subfield of $G$-invariant elements of $L$, i.e.,

$K=L^{G}\subseteq k(x_{i,e},w_{1,j}^{n_{1}-1}\cdot
w_{2,j},\ldots,w_{1,j}^{n_{1}-1}\cdot w_{m,j}).$

Since $(w_{1,j}^{n_{1}-1}\cdot
w_{i+1,j})^{n_{j}}=f_{j}^{n_{j}-d_{j}}(x_{1,1},\cdots,x_{1,\ell})^{n-1}f_{j}(x_{i+1,1},\cdots,x_{i+1,\ell})$
for $1\leq i\leq m-1$, where $d_{j}=n_{j}/n_{1}$ for $1\leq j\leq r$, so by
defining $z_{i,j}:=w_{1,j}^{n_{1}-1}\cdot w_{i+1,j}$ the variety $V_{m}$ can
be expressed by the equations

(3.2.1)
$z_{i,j}^{n_{j}}=f_{j}(x_{1,1},\cdots,x_{1,\ell})^{n_{j}-d_{j}}f_{j}(x_{i+1,1},\cdots,x_{i+1,\ell})\
(i=1,\ldots,m-1,\ j=1,\cdots,r).$

Thus, the extension $L|K$ is an abelian $G$-extension determined by
$w_{1,j}^{n_{1}}=f_{j}(x_{1,1},\cdots,x_{1,\ell})$, i.e.,

$L=K(w_{1,j}\mid j=1,\dots,r)\subseteq k(x_{i,e},z_{i,j})(w_{1,j}\mid
i=1,\ldots,m-1,\ j=1,\dots,r).$

We remark that the abelian field extension ${\mathcal{L}}|{\mathcal{K}}$ can
be determined by considering the homogenization of the equations
$w_{1,j}^{n_{1}}=f_{j}(x_{1,1},\cdots,x_{1,\ell})$. Furthermore, the variety
${\mathcal{V}}_{m}$ can be expressed by the homogenization of equations
(3.2.1). Let $\tilde{X}$ and $\tilde{{\mathcal{X}}}$ be the twists of $X$ and
${\mathcal{X}}$ by the extensions $L|K$ and ${\mathcal{L}}|{\mathcal{K}}$,
respectively. Then, as in Example 2.6 of [8], one can check that the twist
$\tilde{X}$ is given by the following set of affine equations

(3.2.2)
$f_{j}(x_{1,1},\cdots,x_{1,\ell})^{d_{j}}z_{1,j}^{n_{j}}=f_{j}(x_{i+1,1},\cdots,x_{i+1,\ell}),(i=1,\ldots,m-1,\
j=1,\dots,r)$

and $\tilde{{\mathcal{X}}}$ can be determined by its homogenization. Moreover,
it is easy to check that $\tilde{X}$ contains the $m$ $K$-rational points:

(3.2.3) $P_{1}:=\left(x_{1,1},\dots,x_{1,\ell},1\right),\text{and}\
P_{i+1}:=(x_{i+1,1},\dots,x_{i+1,\ell},w_{i+1,j}/w_{1,j}),\ (1\leq i\leq
m-1).$

Let us denote by $\tilde{P}_{i}$ the point corresponding to $P_{i}$’s on
$\tilde{{\mathcal{X}}}$. Now, by applying Proposition 2.2 to the $G$-cover
$\pi:{\mathcal{X}}\rightarrow{\mathbb{P}}^{\ell}$, we obtain

$P({{\mathcal{X}}^{(i)}/{\mathbb{P}}^{\ell}})=\frac{\operatorname{Alb}({\mathcal{X}}^{(i)})}{\operatorname{Im}\operatorname{Nm}G}.$

Since $\operatorname{Alb}({\mathbb{P}}^{\ell})=0$, we have
$P({{\mathcal{X}}^{(i)}/{\mathbb{P}}^{\ell}})=\operatorname{Alb}({\mathcal{X}}^{(i)})=\operatorname{Alb}({\mathcal{X}})$
for $i=1,\ldots,m$ , and hence using Proposition 2.4, we get the following
$k$-isogeny of abelian varieties

(3.2.4)
$P({{\mathcal{U}}_{m}/{\mathcal{V}}_{m}})\sim_{k}\prod_{i=1}^{m}P({{\mathcal{X}}^{(i)}/{\mathbb{P}}^{\ell}})=\operatorname{Alb}({\mathcal{X}})^{m}.$

Now, let us consider the $1$-cocycle
$a=(a_{g})\in{\mathcal{Z}}^{1}(\bar{G},\operatorname{Aut}(\operatorname{Alb}({\mathcal{X}})))$
defined by $a_{g}=g$. Using Proposition 2.5, we conclude that
$\widetilde{\operatorname{Alb}({\mathcal{X}})}\sim_{\mathcal{K}}\operatorname{Alb}(\tilde{{\mathcal{X}}})$.
Thus, by Theorem 3.1 for ${\mathcal{X}}^{\prime}={\mathcal{U}}_{m}$,
${\mathcal{X}}={\mathcal{V}}_{m}$, and
${\mathcal{A}}=\operatorname{Alb}({\mathcal{X}})$, we get:

$\displaystyle\widetilde{\operatorname{Alb}({\mathcal{X}})}({\mathcal{K}})$
$\displaystyle\cong{\rm
Hom}_{k}(P({{\mathcal{U}}_{m}/{\mathcal{V}}_{m}}),\operatorname{Alb}({\mathcal{X}}))\oplus\operatorname{Alb}({\mathcal{X}})[n](k)$
$\displaystyle\cong{\rm
Hom}_{k}(\operatorname{Alb}({\mathcal{X}})^{m},\operatorname{Alb}({\mathcal{X}}))\oplus\operatorname{Alb}({\mathcal{X}})[n](k)$
$\displaystyle\cong(\operatorname{End}_{k}(\operatorname{Alb}({\mathcal{X}})))^{m}\oplus\operatorname{Alb}({\mathcal{X}})[n](k).$

We denote by $\tilde{Q}_{i}$ the image of $\tilde{P}_{i}$’s by the Albanese
map
$\tilde{\alpha}:\tilde{{\mathcal{X}}}\rightarrow\operatorname{Alb}(\tilde{{\mathcal{X}}})$
for $i=1,\ldots,m$. Then, by tracing back the above isomorphisms, one can see
that the points $\tilde{Q}_{i}$’s form a subset of independent generators for
the Mordell-Weil group
$\operatorname{Alb}(\tilde{{\mathcal{X}}})({\mathcal{K}})$. Hence, as
${\mathbb{Z}}$-modules, we have

${\rm rank}(\widetilde{\operatorname{Alb}({\mathcal{X}})}({\mathcal{K}}))={\rm
rank}(\operatorname{Alb}(\tilde{{\mathcal{X}}}))({\mathcal{K}})=m\cdot{\rm
rank}(\operatorname{End}_{k}(\operatorname{Alb}({\mathcal{X}}))).$

Therefore, we have completed the proof of Theorem 3.2.

∎

Next, we apply Theorem 3.1 to the case of dihedral covers of
${\mathbb{P}}^{1}$. The dihedral group of order $2n$ is the group with
presentation

$D_{n}=\langle\sigma,\tau\mid\sigma^{n}=\tau^{2}=1,\tau\sigma\tau=\sigma^{-1}\rangle.$

This group sits in the following exact sequence

(3.2.5) $0\to{\mathbb{Z}}_{n}\to D_{n}\to{\mathbb{Z}}_{2}\to 0.$

A $D_{n}$-Galois cover $f:X\to Y$ corresponds to the function field extension
$k(X)=k(Y)(u,z)$, where $u^{2}=f\in k(Y)$ and $z^{n}=g\in k(Y)(u)$, see [3],
Section 5. The map $f:X\to Y$ factors as $X\xrightarrow{p}W\xrightarrow{q}Y$,
where $W=X/\langle\sigma\rangle$, $p$ is a ${\mathbb{Z}}_{n}$-cover and $q$ is
a ${\mathbb{Z}}_{2}$-cover.

If $Y={\mathbb{P}}^{1}$, we know that $k({\mathbb{P}}^{1})=k(x)$. Consider a
$D_{n}$-cover $C\to\mathbb{P}^{1}$ and let $\mathcal{C}_{m}=\prod_{i=1}^{m}C$,
i.e., the product of $m$ copies of the same dihedral cover as before. We
denote the image of $D_{n}$ under the diagonal embedding
$D_{n}\hookrightarrow\prod_{i=1}^{m}D_{n}$ again by $D_{n}$. Set
$\mathcal{Y}_{m}=\mathcal{C}_{m}/D_{n}$. By Proposition 2.4, we have that

(3.2.6) $P(\mathcal{C}_{m}/\mathcal{Y}_{m})=\prod_{i}P(C/\mathbb{P}^{1})$

By Remark 2.3, $P(C/\mathbb{P}^{1})={(\ker\operatorname{Nm}_{f})}^{0}$.
However as $\operatorname{Jac}(\mathbb{P}^{1})=0$, it follows that
$P(C/\mathbb{P}^{1})=\operatorname{Jac}(C)$. Now Proposition 2.4 gives that

(3.2.7)
$P(\mathcal{C}_{m}/\mathcal{Y}_{m})=\prod_{i}P(C/\mathbb{P}^{1})=(\operatorname{Jac}(C))^{m}.$

Let $L$ be the function field of ${\mathcal{C}}_{m}$. Then
$L=k(x_{1},\dots,x_{m},u_{1},\dots,u_{m},z_{1},\dots,z_{m})$. Let
$L^{\prime}=k(x_{1},\dots,x_{m},u_{1},\dots,u_{m})$. Set
$L^{\prime}_{1}={L^{\prime}}^{{\mathbb{Z}}_{2}}=k(x_{1},\dots,x_{m},u_{1}u_{2},\dots,u_{1}u_{m})$.
Setting $U_{i}=u_{1}u_{i+1}$, it holds that $U_{i}^{2}=f(x_{1})f(x_{i+1})$ for
$1,\dots,m-1$. So $L^{\prime}=L^{\prime}_{1}(u_{1})$. It holds that
$L=L^{\prime}(z_{1},\dots,z_{m})$. In this case,
$L=L^{\prime}(s_{1},\dots,s_{m},z_{1},\dots,z_{m})$, where $s_{i}$ are the
coordinates of $W_{i}$.
$L^{\prime}_{2}={L}^{{\mathbb{Z}}_{n}}=k(t_{1},\dots,t_{m},z_{1}^{n-1}z_{2},\dots,z_{1}^{n-1}z_{m})$
and so $L=L^{\prime}_{2}(z_{1})$. Setting $Z_{i}=z_{1}^{n-1}z_{i+1}$, it holds
that $Z_{i}^{n}=g(s_{1})^{n-1}g(s_{i+1})$ for $1,\dots,m-1$. Suppose $C_{a}$
is the twist of the dihedral cover $C$ by the extension $L\mid K$. Then
$C_{a}$ can be defined by the equation

(3.2.8) $\left\\{\begin{array}[]{ll}f(x_{1})U^{2}=f(x)\\\
g(s_{1})Z^{n}=g(s),\end{array}\right.$

A computation shows that $C_{a}$ has the following rational points.

$P_{1}=(x_{1},s_{1},1,1),P_{i}=(x_{i+1},s_{i+1},U_{i}/f(x_{1}),Z_{i}/g(s_{1})),i=1,\dots,m-1.$

We define the 1-cocycle $Z^{1}(G,\operatorname{Aut}(C))$ by $a_{g}=g$. Let
$\operatorname{Jac}(C)_{a}$ be the twist corresponding to this 1-cocycle. By
applying 3.1.1 and 3.2.7, it follows that

$\displaystyle\operatorname{Jac}(C)_{a}(K)\cong{\rm
Hom}_{k}(P(\mathcal{C}_{m}/\mathcal{Y}_{m}),\operatorname{Jac}(C))\oplus\operatorname{Jac}(C)[2n](k)$
$\displaystyle\cong{\rm
Hom}_{k}((\operatorname{Jac}(C))^{m},\operatorname{Jac}(C))\oplus\operatorname{Jac}(C)[2n](k)$
$\displaystyle\cong\operatorname{End}_{k}(\operatorname{Jac}(C))^{m}\oplus\operatorname{Jac}(C)[2n](k).$

So that ${\rm rank}(\operatorname{Jac}(C)_{a}(K))=m\cdotp{\rm
rank}(\operatorname{End}_{k}(\operatorname{Jac}(C)))$. Let $Q_{i}$ be the
image of $P_{i}$ under the embedding
$C_{a}\hookrightarrow\operatorname{Jac}(C_{a})$. Then, the $Q_{i}$ are among
the generators of $\operatorname{Jac}(C)_{a}(K)$.

######  Remark 3.3.

By similar arguments as in the proof of Theorem 3.2 and the above example, one
can easily show that the same results hold for dihedral covers of the
projective space ${\mathbb{P}}^{\ell}$ for for all $l\geq 1$. In fact the
above results can be generalized to metacyclic Galois covers of
${\mathbb{P}}^{\ell}$.

## References

  * [1] A. Beauville,_Variétiés de Prym et Jacobiennes intermediares._ Ann.scient. Éc. Norm. Sup.,10, (1977), 309-391.
  * [2] A. Borel., J.P. Serre, _Théorèmes de finitude en cohomolo-gie galoisienne._ Comment. Math. Helv.,39, (1964), 111-164.
  * [3] F. Catenese, F. Perroni, _Dihedral Galois covers of algebraic varieties and the simple cases._ Journal of Geometry and Physics 118 (2017), 67–93.
  * [4] F. Hazama, _On the Mordell-Weil group of certain abelian varieties defined over function fields._ J. Number Theory,37, (1991), 168-172.
  * [5] F. Hazama, _Rational points on certain abelian varieties over function fields._ J. Number Theory,50, (1995), 278-285.
  * [6] H. Lange, A. Ortega, _Prym varieties of cyclic coverings._ Geom. Dedicata.,150, (2011), 391-403.
  * [7] A. Mohajer, _On the Prym map of Galois coverings._ arXiv:2004.09678 (2020). To appear in Rocky Mountain Journal of Mathematics.
  * [8] A. Mohajer, _Rational points on abelian varieties over function fields and Prym varieties._ Arch. Math. 116, pp. 293-300 (2021). https://doi.org/10.1007/s00013-020-01550-4.
  * [9] S. Recillas, R. Rodríguez, _Prym varieties and fourfold covers._ arXiv:math/0303155, 2003.
  * [10] S. Salami, _The rational points on certain abelian varieties over function fields._ J. of Number Theory,195, February (2019), 330-337.
  * [11] S. Salami, _Twists of the Albanese varieties of cyclic multipleplanes with large ranks over higher dimensionfunction fields._ Journal de Théorie des Nombres de Bordeaux 32(2020), 861–876.
  * [12] W. B. Wang: _On the twist of abelian varieties defined by the Galois extension of prime degree_ , Journal of Algebra, 163 (3), (1994), 813 - 818.
  * [13] G. Yun: _A note on finite abelian covers_ , SCIENCE CHINA Mathematics, Vol. 54 No. 7, (2011): 1333–1342