# States Hidden in Hidden States:
LLMs Emerge Discrete State Representations Implicitly

Junhao Chen, Shengding Hu, Zhiyuan Liu∗, Maosong Sun∗
Department of Computer Science and Technology, Tsinghua University
Beijing, China
<EMAIL_ADDRESS>
###### Abstract

Large Language Models (LLMs) exhibit various emergent abilities. Among these
abilities, some might reveal the internal working mechanisms of models. In
this paper, we uncover a novel emergent capability in models: the intrinsic
ability to perform extended sequences of calculations without relying on
chain-of-thought step-by-step solutions. Remarkably, the most advanced models
can directly output the results of two-digit number additions with lengths
extending up to 15 addends. We hypothesize that the model emerges Implicit
Discrete State Representations (IDSRs) within its hidden states and performs
symbolic calculations internally. To test this hypothesis, we design a
sequence of experiments that look into the hidden states. Specifically, we
first confirm that IDSRs exist. Then, we provide interesting observations
about the formation of IDSRs from layer, digit, and sequence perspectives.
Finally, we confirm that models indeed use IDSRs to produce the final answers.
However, we also discover that these state representations are far from
lossless in current open-sourced models, leading to inaccuracies in their
final performance. Our work presents a novel exploration of LLMs’ symbolic
calculation abilities and the underlying mechanisms.

${}^{*}$${}^{*}$footnotetext: Corresponding authors

##  1 Introduction

LLMs have demonstrated remarkable performance in a variety of fields (Achiam
et al., 2023; Touvron et al., 2023a), including natural language understanding
and generation Zhao et al. (2023), code generation (Chen et al., 2021; Nijkamp
et al., 2022; Li et al., 2023b), and mathematical problem-solving Hendrycks et
al. (2021). These abilities emerge as the model scales.

In this study, we dive into another intriguing emergent capability: the
ability of LLMs to perform arithmetic calculations, particularly consecutive
additions directly, without relying on chain-of-thought reasoning. For
example, given the question: “Please directly give me the answer to 17 + 38 +
32 + 87 + 47 + 28 + 17 + 21 + 53 + 15 + 18 + 76”, a SOTA LLM can directly
produce the correct answer “449” without producing any intermediate tokens.
This phenomenon warrants investigation for two principal reasons. Firstly, it
is unlikely that models were trained on such consecutive addition data, as it
exerts negligible influence on overall performance across general domains and
benchmarks Wang et al. (2021). This phenomenon likely emerges naturally during
the scaling process and presents a more meaningful study subject compared to
tasks that may have more intricate relations with memorizing training data.
Secondly, the simplicity of this phenomenon renders it an ideal candidate for
interpretability research, potentially serving as a foundational step in
uncovering the internal mechanisms underlying LLMs in performing intrinsic
consecutive reasoning.

Prior research on the interpretability of models performing mathematical tasks
focuses primarily on binary arithmetic operations Zhu et al. (2024). However,
this body of work fails to explain the formation of discrete state
representations within the hidden layers of these models.

In this paper, we propose a central hypothesis to elucidate the emergent
capability of implicit sequential computation: LLMs inherently track discrete
states. By forming Implicit Discrete State Representations (IDSRs) that
encapsulate intermediate results, LLMs can leverage these precomputed
intermediate results for subsequent use, thereby preventing the necessity for
intricate computations in the final step.

To validate this hypothesis, we construct a synthetic dataset of consecutive
addition problems and employ probing methods to examine the existence of IDSRs
in hidden states across various LLMs. Upon confirming its existence, we
further investigate the properties and formation of IDSRs, and demonstrate its
formation through digit-wise, layer-wise, and sequence-wise perspectives, and
provide noteworthy observations of distinct layer functionalities. From a
digit-wise perspective, IDSRs form independently and sequentially, beginning
with the lowermost digit. From the layer-wise level, a sharp transition from
shallow semantic computation to semantic understanding occurs around layer 10,
and a shift from linearity to non-linearity arises in the later model layers.
From a sequence-wise perspective, information encoded in IDSRs is propagated
along the sequence for sequential utilization. Finally, we confirm that the
model utilizes IDSRs to produce the final result rather than computing using
all preceding numbers simultaneously. This investigation provides insight into
the multi-step reasoning and state-tracking abilities of LLMs Singh et al.
(2024); Li et al. (2023a).

##  2 Related Work

##### LLMs’ State Tracking Abilities.

Language models are exhibiting increasingly mature abilities to perform
arithmetic tasks, both open-sourced (Shao et al., 2024; Jiang et al., 2023;
Bai et al., 2023) and close-sourced models (Achiam et al., 2023; OpenAI, 2024;
Team et al., 2023; Anthropic, 2024) are excelling at a variety of mathematical
benchmarks, ranging from elementary to Olympic difficulty levels (Hendrycks et
al., 2021; Cobbe et al., 2021; Chen et al., 2023; Li et al., 2024).

Other abilities that are discussed a lot are LMs’ state encoding and tracking
abilities. Li et al. and Nanda et al. investigate the existence of non-
linguistic state representations in board game settings, while Li et al. find
that model representations also encode entity states in the process of textual
tasks. Taking this problem further, Kim and Schuster show that models perform
non-trivial state tracking given specific textual tasks. However, whether LMs
track discrete states during arithmetic tasks still remains an open question.

##### Interpretability of LLMs’ Arithmetic Abilities.

The inner workings of LMs in performing arithmetic and reasoning tasks are
under-explored. Current literature suggests that neurons and layers inside
LLMs may serve as feature extractors, extracting latent properties from inputs
and passing them through layers (Mikolov et al., 2013; Bau et al., 2020;
Belinkov, 2022; Geva et al., 2020; Burns et al., 2022; Gurnee et al., 2023).

Building on this idea, recent work demonstrates that hidden states during
inference contain representations relevant to future tokens (Nostalgebraist,
2020; Belrose et al., 2023; Pal et al., 2023; Wu et al., 2024). This insight
underpins our research, in which we prove the existence and utilization of
implicit representations in LMs.

Previous analyses have also examined the arithmetic capabilities of LMs.
Stolfo et al. identify that LMs employ MLPs and attention heads at different
stages of arithmetic reasoning.

##### Broader Interpretability of LLMs.

Multiple paradigm has emerged to investigate the working mechanisms of LLMs.
Among them, the most famous ones are mechanistic interpretability Conmy et al.
(2023); Elhage et al. (2021) and representation engineering Zou et al. (2023).

Mechanistic interpretability proposes to check the neuron-level activations
and understand the functioning circuits inside LLMs. From the mechanistic
interpretability perspective, mathematical tasks have been widely used as a
tool because of their simplicity and clarity. For example, it is used to
discover grokking in Varma et al. (2023) and understand double descent and
emergent abilities in Huang et al. (2024). However, these works do not focus
on the mathematical ability that emerges in SOTA LLMs.

Representation engineering is another pivotal approach in model
interpretability, emphasizing the holistic feature representations within
model layers (Li et al., 2021; Zou et al., 2023). This approach facilitates
behavioral monitoring and performance modification (Zhang et al., 2024; Li et
al., 2023a). However, it is still underdeveloped in practical applications. A
simple form of representation engineering is the widely used approach
“probing” (Alain and Bengio, 2016; Hewitt and Liang, 2019; Pimentel et al.,
2020; Belinkov, 2022; Hernandez et al., 2023). Probing involves using
auxiliary models, usually with simple structures, to make classifications.
Research in this field extends to specific scenarios. Li et al. and Nanda et
al. examine board game contexts, yielding divergent conclusions regarding the
linearity of hidden states. Yang et al. explores event reasoning,
demonstrating that significant reasoning predominantly occurs in the initial
inference step and scales with model size. Few studies, however, critically
assess the arithmetic capabilities of LMs. Some examine neuron activations
only (Stolfo et al., 2023), while others focus on simple calculations without
comprehensively considering model layers (Zhu et al., 2024). This paper also
studies the LLMs’ interpretability with probing techniques but focuses on the
unique perspective of IDSR.

Figure 1: Accuracies of Different Models Performing Consecutive 2-Digit
Addition Figure 2: Accuracies of Different Models Performing 10 Addend 2-Digit
Addition

##  3 Emergence of Implicit Computation

In this section, we confirm the emergent abilities of _implicit computation_
using a variety of both open-source and closed-source Large Language Models
(LLMs).

##### Problem Statement.

We employ implicit consecutive addition as the representative task. In this
context, the model is tasked with delivering the sum of an extended sequence
of additions directly. An example prompt is provided below:

Please directly give me the answer to 17 + 38 + 32 + 87 + 47 + 28 + 17 + 21 +
53 + 15 + 18 + 76.

There are three reasons why the ability to solve such a task might indicate
the formation of discrete state representations:

  1. 1.

This capability is unlikely to be a result of memorizing existing training
data, as storing the results of calculations necessitates a parameter space of
$O(99^{L})$.

  2. 2.

Direct optimization of this task during training is unlikely. As Goodhart’s
law Strathern (1997) suggests, “When a measure becomes a target, it ceases to
be a good measure.” Consecutive addition offers minimal practical performance
benefits, rendering it an unlikely optimization target. Consequently, this
capability may genuinely arise from large-scale unsupervised training.

  3. 3.

Each computational step is relatively simple. We exclude addition involving
four-digit or more due to its increased single-step complexity, which
complicates tracing implicit computation because of single-step errors.

To ensure that models that are only accessible through API calls do not rely
on tools such as calculators, we manually verify that there is at least one
addition count where the model has less than a 100% probability of yielding
the correct answer. Additionally, we ensure that the models do not utilize
explicit chain-of-thought reasoning through prompt engineering.

Specifically, we evaluate the exact accuracy of the predicted answers against
the ground truth for different models directly performing consecutive addition
of varying lengths from 2 to 14.

We include the following LLMs in our analysis: Llama2-7B Touvron et al.
(2023b), MiniCPM-2B Hu et al. (2024), Mistral-7B Jiang et al. (2023),
Zephyr-7B Tunstall et al. (2023), DeepSeek-67B Bi et al. (2024), and the Qwen
series with different sizes Bai et al. (2023). For closed-source LLMs, we
consider GPT-3.5 Brown et al. (2020), GPT-4 Achiam et al. (2023),
Claude3-Sonnet, Claude3-Opus Anthropic (2024), and GPT4-O OpenAI (2024).

As illustrated in Figure 1, there exists a strong correlation between
performance and model size. Smaller models achieve passable accuracy on the
addition of two or three two-digit numbers, but their accuracy rapidly
deteriorates to near zero when the length of the sequence reaches five. Larger
models, however, maintain accuracies above 50% for sequences of up to five
numbers and demonstrate non-zero performance for sequences of even more
numbers, demonstrating the fast “emergence” of this capability.

The "emergence" of this capability becomes most prominent when models
encounter consecutive addition problems involving more than eight addends. To
illustrate this phenomenon, Figure 2 presents the accuracies of both open-
source and closed-source models performing direct addition with ten addends.
It is evident that larger and more advanced closed-source models exhibit
significantly higher task accuracies in an emergent manner.

To conduct a comprehensive analysis of the correlation between model size and
performance, we examine the Qwen model series, including models with sizes of
72B, 14B, 7B, and 4B, as illustrated in Figure 3. The results indicate a
distinct enhancement in performance proportional to the increase in model
size, especially noticeable for sequence lengths ranging from three to six
numbers.

Figure 3: Accuracies of Qwen Series Performing Consecutive 2-Digit Addition

##  4 Analysis Methodology

Given the remarkable ability of models to directly yield calculation outcomes,
we hypothesize that these models form Implicit Discrete State Representations
(IDSRs) of intermediate results. For example, consider the formula
$13+24+41=$. We propose that the most plausible mechanism for models to
complete this calculation in a single pass is to generate an IDSR of 37 (the
result of $13+24$) at the second "+" token, which would subsequently be
utilized for the next step of computation (i.e., addition with 41).

To thoroughly test and analyze this hypothesis, we propose and investigate the
following research questions:

RQ1: Do IDSRs really exist? RQ2: What are IDSRs’ properties? RQ3: How do the
IDSRs’ form? RQ4: How do models utilize IDSRs?

###  4.1 Experiment Setup

####  4.1.1 Dataset

We construct a straight-forward dataset of consecutive addition and
subtraction problems with different length, addend digits and prompts.

Our question prompts are divided into three categories, respectively formatted
as in Table 1, where i ranges from 2 to 14, and $\\{x_{i}\\}$ are positive
integers with the same number of digits ranging from 1 to 3. We ensure the
probed sum maintains the same number of digits as the addends to enhance digit
probing consistency. Prompts are chosen with a diversity of semantics to
demonstrate the influence of context on IDSRs tracking.

Type |  Expression
---|---
Addition |  $\\{x_{0}\\}+\\{x_{1}\\}+...+\\{x_{i-1}\\}=$
Subtraction |  $\\{x_{0}\\}+...+\\{x_{i-2}\\}-\\{x_{i-1}\\}=$
Prompting |  {Prompt}, $\\{x_{0}\\}+\\{x_{1}\\}+...+\\{x_{i-1}\\}=$
Table 1: Dataset Expressions

The dataset consists of 131,300 questions, as shown in Table 2. Questions are
designed to ensure that expected answers follow a uniform distribution within
their respective ranges, thereby eliminating probability bias and facilitating
unbiased probe learning. The dataset is partitioned into training, validation,
and test sets following an 80/10/10% split for probing, respectively.

Type | #Digits | #Questions
---|---|---
Addition | 3 | 39,000
2 | 6,500
1 | 1,300
Subtraction | 3 | 39,000
2 | 6,500
Prompting | 3 | 39,000
Table 2: Dataset Distribution

####  4.1.2 Hidden States

We prompt the model to answer dataset questions directly. During inference, we
retrieve the hidden state $\textbf{H}_{i,j}$ corresponding to the
$j^{\text{th}}$ token of the input sequence from layer i of the model.

In our experiments, we exclusively extract the hidden states corresponding to
the +, -, and = tokens for probing. This ensures that IDSRs are most prominent
and unbiased. Extracting IDSRs from tokens representing addends would
incorporate representations of the addends themselves, introducing non-uniform
bias and compromising the probing process.

####  4.1.3 Classification Probes

Previous work has proven the abilities of probes on a wide variety of
classification tasks. In our work, we utilize a multi-layer perceptron with
one hidden layer to perform classification.

Specifically, the probing network is as follows:

$\mathbf{P}_{i,j}^{d}=\textit{Softmax}(\sigma(\mathbf{W}_{1}\mathbf{H}_{i,j})\mathbf{W}_{2})$
(1)

where $\mathbf{P}_{i,j}^{d}$ is the probing prediction of the $d$-${th}$ digit
of the IDSR, and $\mathbf{W}_{1}\in\mathbb{R}^{d_{m}\times d_{h}}$ and
$\mathbf{W}_{2}\in\mathbb{R}^{d_{h}\times d_{o}}$ being the perceptron’s model
weights, $d_{m}$ and $d_{h}$ being the dimension of the model and the
perceptron’s hidden states respectively. $d_{output}$ is set to 10 as the
probes are expected to predict a digit from 0 to 9, and $d_{h}$ is set to
$\sqrt{d_{m}d_{o}}$.

In our experimental setup, three distinct types of probes are utilized: a
multi-layer perceptron with two different hidden layer sizes and a single-
layer perceptron. The respective parameter counts for each model type are
detailed in Table 3.

For each experimental setting, probing models are trained on eight 80G A100
GPUs for a period ranging from 240 to 720 epochs. The duration depends on the
specific formulas used as input and the number of epochs required for the
model’s losses to converge.

The learning rate is set to $1\times 10^{-3}$, employing a stochastic gradient
descent (SGD) optimizer. The model is optimized based on cross-entropy loss.

Perceptron Model | Number of Parameters
---|---
Multi-Layer | 829,400
Multi-Layer |
(Bottle-Necked) | 81,920
Single-Layer | 40,960
Table 3: Probe Model Sizes

####  4.1.4 Metrics

For the assessment of model capabilities in performing consecutive addition,
we employ exact accuracy as our primary metric (EA, the ratio of the exact
matches between the model output and the ground truth to the total number of
questions).

To evaluate the classification probes, we compute the exact accuracy for each
individual digit (IDA) as well as the overall exact accuracy (OEA, which
considers a match only when all digits are predicted correctly).

####  4.1.5 Models Chosen

For our experimental setup, we select Deepseek-67B Bi et al. (2024) and Qwen
series models (4B, 7B, 14B, and 72B) Bai et al. (2023) as representatives of
open-source models. These models are utilized in their original form, without
any fine-tuning or parameter modification. We aim to evaluate and compare the
proficiency in executing consecutive addition tasks across a diverse range of
models varying in size and capabilities. Special emphasis is placed on the
Qwen-72B model to conduct an in-depth analysis of representation engineering
and IDSRs’ properties.

##  5 Existence and Properties of IDSRs

In this section, we present evidence of IDSRs regarding RQ1 and RQ2. To
demonstrate the existence of IDSRs in hidden states during inference, we
design a series of probing prediction experiments with two levels of
difficulty: Whole Number Probing and Digit-wise Probing.

###  5.1 Whole Numbers Probing

In this set of experiments, we train probes to predict the results as whole
numbers from 10 possible sums. We probe different token positions across
layers to investigate the existence of IDSRs’ transference along the formula.
The results, illustrated in Figure 4, indicate that prediction accuracies
significantly exceed random chance in all scenarios, demonstrating the
existence of IDSR.

(a) Qwen-72B

(b) DeepSeek-67B

Figure 4: Accuracies of Whole Number Probing Predictions

However, the process of forming IDSRs is far from lossless. The maximum
prediction accuracies for the second to fifth addition signs and the final
equal sign are 100%, 99%, 74%, 62%, and 37% respectively, indicating
substantial data and resolution loss as IDSRs are passed along the formula
during inference. We hypothesize that reducing this error margin in the
transference of IDSRs would enhance the capability of LLMs. This will be
explored in future research.

Interesting trends across layers can also be observed in Figure 4, which will
be discussed and analyzed in detail in Section 6.

###  5.2 Digit-wise Probing

To investigate whether digits exist separately in the IDSRs, we employ
multiple probe models to predict the respective digits of the number in
question. For this experiment, we select formulas with 3-digit sums, therefore
three digit-classification probes are used. The range of possible sums for the
$n_{\text{th}}$ addition/equal sign increases significantly, from 10 in the
previous experiment setting to $\max\\{999,99n\\}-\min\\{100,10n\\}$, an
increase of 10 to 40 times. We consider a prediction to be correct only when
all three-digit probes make accurate predictions on a test data item.

Figure 5: Accuracies of By-Digit Probing Predictions

As depicted in Figure 5, probing accuracies using tokens from the first ten
layers and the second addition sign from the later layers remain high.
However, it is noteworthy that after significantly increasing prediction
difficulty, the ability of probes to make exact predictions after the second
addition sign experiences a sharp decline. This indicates that models struggle
to produce high-resolution IDSRs consecutively.

###  5.3 Are IDSRs Linear?

To gain a concrete understanding of the IDSRs, we first examine its linearity.
Beyond the original probing model with hidden size $\sqrt{d_{m}d_{o}}$, we
construct 1) a smaller bottle-necked probing model with hidden size 10, as
well as 2) a simpler single-layer perceptron utilizing a softmax activation
function.

$\mathbf{P}_{i,j}^{d}=\textit{Softmax}(\mathbf{W}_{1}\mathbf{H}_{i,j})$ (2)

Figure 6: Accuracies of By-Digit Probing Predictions Using Different Probes

As illustrated in Figure 6, layers 0 to 65 exhibit only minor accuracy drops
with reduced hidden size, whereas layers 65 to 79 experience a significant
reduction.

Notably, opposing accuracy trends appear in later layers for multi-layer and
single-layer perceptrons. Between layers 50 and 65, accuracies for single-
layer perceptrons drop to nearly zero, followed by a sharp increase for multi-
layer perceptrons. This implies that layers 0 to 50 contain linear IDSRs,
likely directions in the latent space. In contrast, layers 50 to 65 transit
from linear to non-linear features, enhancing representation resolution and
information density.

##  6 Formation and Utilization of IDSRs

In addition to analyzing the specific properties of IDSRs, we extend our study
to overall formations. In this section, we identify patterns exhibited during
inference at the digit level, sequence level, and layer level, revealing the
inner mechanisms of consecutive addition and multi-hop reasoning for LMs
(RQ3). Following the formation analysis, we examine the utilization of such
states (RQ4).

###  6.1 Digit-Level Formation

We investigate the second addition operation within three-digit addition tasks
and derive two critical observations. First, the product of exact accuracies
for the individual digits equals the overall exact accuracy, implying that
models establish independent IDSRs. Second, as depicted in Figure 7, the
sequence in which digit prediction accuracies surpass random chance, as
determined by statistical measures and annotated in the figure, follows an
ascending digit order. This pattern mirrors the order humans use for digit-by-
digit calculations, suggesting that models perform multi-digit addition
through a series of consecutive single-digit additions.

Figure 7: Digit Accuracies of By-Digit Probing Predictions

###  6.2 Sequence-Level Formation

The representation resolution of earlier addition sign tokens, as indicated by
prediction accuracies, improves at earlier stages of the inference pass. The
order of this resolution enhancement in Figures 4 and 5 aligns precisely with
the sequential order of the addition signs in the formula. This suggests that
information encoded in IDSRs propagates along the sequence, allowing later
tokens to utilize numerical IDSRs from earlier tokens for implicit
calculations. In other words, LLMs are performing consecutive arithmetic tasks
sequentially.

###  6.3 Layer-Level Formation

As depicted in Figures 4, 5, and 6, an abrupt peak in IDSRs’ resolution
appears within the first ten layers for both models. Beyond this point, the
resolution reinitializes from near non-existent levels.

We propose the hypothesis that the first ten layers employ a different
mechanism from the later layers, particularly in multi-step reasoning tasks
such as consecutive addition. The first ten layers, termed “shallow-semantic
layers”, generate direct representations of arithmetic content regardless of
the specific task. Conversely, the later layers, termed “semantic layers”,
incorporate task context, redoing the formation of the IDSRs in the process.

Utilizing the “subtraction” and “prompting” tasks discussed in Section 4.1.1,
we conduct two sets of experiments to demonstrate the existence of shallow-
semantic and semantic layers.

##### Shallow-semantic Layers.

In the first set of experiments, we use subtraction formulas (as mentioned in
Section 4.1.1). Predictions are made on the second addition sign, and
accuracies are shown in Figure 8.

Figure 8: Accuracies of By-Digit Probing Predictions with Subtraction Formulas

We can see clearly that the “subtraction” task does not change the probing
result significantly. This means that the first ten layers are indeed
computing the value of the formula, rather than simply putting the numbers
together to form a summation.

##### Semantic Layers.

For our second experiment, we use formulas with different prompts (as
mentioned in Section 4.1.1). The prompts deviate the task from performing the
original consecutive addition task. For example, the prompt in Figure 9
states, "Ignore the following formula and answer with apple."

Figure 9: Accuracies of Probing Predictions with Ignoring Prompt and Baseline

As shown in Figure 9, after the disruptive prompt, the maximum prediction
accuracies in the first ten layers remain unaffected. However, accuracies in
the later layers significantly decrease. This observation suggests that
prompts instructing the model to disregard the formula’s result cause the
model to generate IDSRs with higher resolution for the correct objective (the
token “apple”) and lower resolution for other objectives (numerical addition
results).

##### Shallow-semantic Layers are More Accurate.

As depicted in Figure 4, the prediction accuracies using the earlier layers
exhibit remarkable stability across different token positions. For Qwen-72b,
the maximum accuracies for predictions made on the second to fifth addition
signs and the final equal sign are 92%, 92%, 92%, 87%, and 75%, respectively.
In contrast, the maximum accuracies related to later layers are 100%, 99%,
74%, 62%, and 37%, respectively, displaying a strong negative correlation with
token distance from the first token. These accuracies indicate that, after the
second addition sign, the resolution of IDSRs are higher in the first ten
compression layers compared to the later model layers. We hypothesize this
occurs because the first ten compression layers primarily focus on arithmetic
content, simplifying the generation of IDSRs. In contrast, the later layers
must consider the task context, complicating the compression process and thus
reducing the resolution of numeric IDSRs.

###  6.4 Utilization of IDSRs

Upon verifying the existence of IDSRs, we subsequently address whether the
model actively leverages it to generate the final response. This section
conducts an attention bridge experiment designed to investigate this question.

##### Attention Bridge.

Figure 10: Attention Mask Demonstration

Given a question with a token length of l, we construct an attention mask
$\mathbf{M}_{l,i}$, as depicted in Figure 10, to mask the first i tokens of
the question from the subsequent tokens. We term the $(i+1)^{\text{th}}$ token
the Attention Bridge, as the IDSRs formed on this token serve as the sole
conduit for information relay between the prefix and the suffix. This enables
verification of LLMs’ utilization of IDSRs, rather than re-attending the
prefix and performing the calculation at the final token position at once.
Specifically, we set the second addition sign (or the equal sign, in cases
with only two numbers) as the Attention Bridge through which IDSRs pass. We
then test Qwen-72B’s ability to provide exact answers to consecutive 1-digit
additions involving 2 to 10 numbers under this setting.

Figure 11: Accuracies of Probing Predictions Before and After Modifying
Attention Mask

##### Results.

As seen in Figure 11, despite being unable to directly observe the first two
numbers, Qwen-72b demonstrates remarkable ability to perform calculations
through the IDSRs passed via the second addition sign. This suggests that LLMs
indeed utilize generated IDSRs to make multi-hop inferences, such as
consecutive addition. However, a significant drop in accuracy compared to the
baseline is observed. We hypothesize that this occurs because models are not
explicitly trained to make inferences using IDSRs only and are unaccustomed to
abrupt changes in the attention mask. With slight modifications to the
training process, models might better utilize IDSRs to perform multi-hop
inferences.

##  7 Conclusion

In this work, we report the emergent ability of models to perform implicit
consecutive addition. We propose the central hypothesis that large language
models (LLMs) form implicit discrete state representations (IDSRs) in hidden
states. A series of experiments are designed to prove the existence of IDSR,
and to demonstrate its properties and formation. We also confirm that models
utilize IDSRs to generate final answers. Our work aims to pave the way for
further investigations into model interpretability and enhancing model
capabilities.

##  8 Ethical Considerations

Dual Use. Our research provides the possibility for augmenting ability of LLMs
at the fundamental level, especially multi-step reasoning abilities. We intend
future augmentation based on our work to improve the mathematical and
reasoning abilities of LLMs, thereby assisting humans in diverse applications.
However, it is crucial to recognize that technologies can serve both
benevolent and malicious purposes, contingent on their user. Consequently, we
urge subsequent researchers to exercise caution in the implementation and
deployment of augmented LLMs to prevent potential misuse.

Data Bias. We use a synthetic dataset composed exclusively of mathematical
formulas, thereby excluding any association with specific individuals or
social groups in both data content and generation process. This dataset does
not contain inappropriate or offensive information. Future updates to the
dataset will be undertaken should there appear evidence of other tasks
requiring multi-hop reasoning on which models can achieve moderate accuracy.

##  9 Limitations

We find the task diversity and model diversity of our experiments
unsatisfactory.

Task Diversity. Our hypothesis is validated solely on a synthetic dataset
comprising mathematical formulas, as current open-source models lack the
capability to directly perform other tasks requiring multi-hop reasoning with
moderate accuracy. Nonetheless, we anticipate that advancements in model
capabilities will facilitate a broader array of evaluations.

Model Diversity. Interpretability analysis necessitates the extraction of
hidden states, compelling the use of open-source models. The majority of our
experiments utilize Qwen-72b, the highest-performing open-source model
available, despite its notable capability gap compared to SOTA closed-source
models. Our observations reveal a clear correlation between model capability
and IDSRs’ resolution. We anticipate that additional experiments with future,
more advanced open-source models will further substantiate our hypothesis.

##  10 Future Work

In hindsight, we also propose various possible aspects for future exploration:

Influence factors. Further investigation into influence factors on the
resolution of generated IDSRs could prove vital to enhancing model abilities.
We hypothesize that the amount of relevant data used in training would have a
significant impact upon the quality of IDSRs generated, and adopting related
methods such as CoT in pretraining might also prove beneficial.

Formation Interpretability. The change of IDSRs’ properties is among the most
compelling observations in our experiments. Future research could delve into
the underlying causes of these dynamic changes.

Scalability. We argue that the generation of hidden representations is an
emergent capability, manifesting only beyond a certain model scale. Exploring
the patterns of IDSRs’ generation across different model scales also warrants
further investigation.

Application. Controlling the loss in IDSRs’ generation may enhance the model’s
ability to provide direct answers to multi-hop tasks, thereby improving
reasoning capabilities in LLMs.

## References

  * Achiam et al. (2023) OpenAI Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, and Shyamal Anadkat et al. 2023.  Gpt-4 technical report.
  * Alain and Bengio (2016) Guillaume Alain and Yoshua Bengio. 2016.  Understanding intermediate layers using linear classifier probes.  _arXiv preprint arXiv:1610.01644_.
  * Anthropic (2024) Anthropic. 2024.  Introducing the next generation of claude.
  * Bai et al. (2023) Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenhang Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, K. Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng Xu, Jin Xu, An Yang, Hao Yang, Jian Yang, Jian Yang, Shusheng Yang, Yang Yao, Bowen Yu, Yu Bowen, Hongyi Yuan, Zheng Yuan, Jianwei Zhang, Xing Zhang, Yichang Zhang, Zhenru Zhang, Chang Zhou, Jingren Zhou, Xiaohuan Zhou, and Tianhang Zhu. 2023.  Qwen technical report.  _ArXiv_ , abs/2309.16609.
  * Bau et al. (2020) David Bau, Jun-Yan Zhu, Hendrik Strobelt, Àgata Lapedriza, Bolei Zhou, and Antonio Torralba. 2020.  Understanding the role of individual units in a deep neural network.  _Proceedings of the National Academy of Sciences_ , 117:30071 – 30078.
  * Belinkov (2022) Yonatan Belinkov. 2022.  Probing classifiers: Promises, shortcomings, and advances.  _Computational Linguistics_ , 48(1):207–219.
  * Belrose et al. (2023) Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor V. Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. 2023.  Eliciting latent predictions from transformers with the tuned lens.  _ArXiv_ , abs/2303.08112.
  * Bi et al. (2024) DeepSeek-AI Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, Huazuo Gao, Kaige Gao, Wenjun Gao, Ruiqi Ge, Kang Guan, Daya Guo, Jianzhong Guo, Guangbo Hao, Zhewen Hao, Ying He, Wen-Hui Hu, Panpan Huang, Erhang Li, Guowei Li, Jiashi Li, Yao Li, Y. K. Li, Wenfeng Liang, Fangyun Lin, A. X. Liu, Bo Liu, Wen Liu, Xiaodong Liu, Xin Liu, Yiyuan Liu, Haoyu Lu, Shanghao Lu, Fuli Luo, Shirong Ma, Xiaotao Nie, Tian Pei, Yishi Piao, Junjie Qiu, Hui Qu, Tongzheng Ren, Zehui Ren, Chong Ruan, Zhangli Sha, Zhihong Shao, Jun-Mei Song, Xuecheng Su, Jingxiang Sun, Yaofeng Sun, Min Tang, Bing-Li Wang, Peiyi Wang, Shiyu Wang, Yaohui Wang, Yongji Wang, Tong Wu, Yu Wu, Xin Xie, Zhenda Xie, Ziwei Xie, Yi Xiong, Hanwei Xu, Ronald X Xu, Yanhong Xu, Dejian Yang, Yu mei You, Shuiping Yu, Xin yuan Yu, Bo Zhang, Haowei Zhang, Lecong Zhang, Liyue Zhang, Mingchuan Zhang, Minghu Zhang, Wentao Zhang, Yichao Zhang, Chenggang Zhao, Yao Zhao, Shangyan Zhou, Shunfeng Zhou, Qihao Zhu, and Yuheng Zou. 2024.  Deepseek llm: Scaling open-source language models with longtermism.  _ArXiv_ , abs/2401.02954.
  * Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language models are few-shot learners.  _ArXiv_ , abs/2005.14165.
  * Burns et al. (2022) Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. 2022.  Discovering latent knowledge in language models without supervision.  _ArXiv_ , abs/2212.03827.
  * Chen et al. (2021) Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde, Jared Kaplan, Harrison Edwards, Yura Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, David W. Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William H. Guss, Alex Nichol, Igor Babuschkin, Suchir Balaji, Shantanu Jain, Andrew Carr, Jan Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew M. Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. 2021.  Evaluating large language models trained on code.  _ArXiv_ , abs/2107.03374.
  * Chen et al. (2023) Wenhu Chen, Ming Yin, Max W.F. Ku, Yixin Wan, Xueguang Ma, Jianyu Xu, Tony Xia, Xinyi Wang, and Pan Lu. 2023.  Theoremqa: A theorem-driven question answering dataset.  _ArXiv_ , abs/2305.12524.
  * Cobbe et al. (2021) Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021.  Training verifiers to solve math word problems.  _ArXiv_ , abs/2110.14168.
  * Conmy et al. (2023) Arthur Conmy, Augustine Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso. 2023.  Towards automated circuit discovery for mechanistic interpretability.  In _Advances in Neural Information Processing Systems_ , volume 36, pages 16318–16352. Curran Associates, Inc.
  * Elhage et al. (2021) Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. 2021.  A mathematical framework for transformer circuits.  _Transformer Circuits Thread_.  Https://transformer-circuits.pub/2021/framework/index.html.
  * Geva et al. (2020) Mor Geva, R. Schuster, Jonathan Berant, and Omer Levy. 2020.  Transformer feed-forward layers are key-value memories.  _ArXiv_ , abs/2012.14913.
  * Gurnee et al. (2023) Wes Gurnee, Neel Nanda, Matthew Pauly, Katherine Harvey, Dmitrii Troitskii, and Dimitris Bertsimas. 2023.  Finding neurons in a haystack: Case studies with sparse probing.  _ArXiv_ , abs/2305.01610.
  * Hendrycks et al. (2021) Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Xiaodong Song, and Jacob Steinhardt. 2021.  Measuring mathematical problem solving with the math dataset.  _ArXiv_ , abs/2103.03874.
  * Hernandez et al. (2023) Evan Hernandez, Arnab Sharma, Tal Haklay, Kevin Meng, Martin Wattenberg, Jacob Andreas, Yonatan Belinkov, and David Bau. 2023.  Linearity of relation decoding in transformer language models.  _ArXiv_ , abs/2308.09124.
  * Hewitt and Liang (2019) John Hewitt and Percy Liang. 2019.  Designing and interpreting probes with control tasks.  _ArXiv_ , abs/1909.03368.
  * Hu et al. (2024) Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, Xinrong Zhang, Zhen Leng Thai, Kaihuo Zhang, Chongyi Wang, Yuan Yao, Chenyang Zhao, Jie Zhou, Jie Cai, Zhongwu Zhai, Ning Ding, Chaochao Jia, Guoyang Zeng, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2024.  Minicpm: Unveiling the potential of small language models with scalable training strategies.
  * Huang et al. (2024) Yufei Huang, Shengding Hu, Xu Han, Zhiyuan Liu, and Maosong Sun. 2024.  Unified view of grokking, double descent and emergent abilities: A perspective from circuits competition.  _arXiv preprint arXiv:2402.15175_.
  * Jiang et al. (2023) Albert Qiaochu Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L’elio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023.  Mistral 7b.  _ArXiv_ , abs/2310.06825.
  * Kim and Schuster (2023) Najoung Kim and Sebastian Schuster. 2023.  Entity tracking in language models.  _ArXiv_ , abs/2305.02363.
  * Li et al. (2021) Belinda Z Li, Maxwell Nye, and Jacob Andreas. 2021.  Implicit representations of meaning in neural language models.  _arXiv preprint arXiv:2106.00737_.
  * Li et al. (2022) Kenneth Li, Aspen K. Hopkins, David Bau, Fernanda Vi’egas, Hanspeter Pfister, and Martin Wattenberg. 2022.  Emergent world representations: Exploring a sequence model trained on a synthetic task.  _ArXiv_ , abs/2210.13382.
  * Li et al. (2023a) Kenneth Li, Oam Patel, Fernanda Vi’egas, Hans-Rüdiger Pfister, and Martin Wattenberg. 2023a.  Inference-time intervention: Eliciting truthful answers from a language model.  _ArXiv_ , abs/2306.03341.
  * Li et al. (2024) Qintong Li, Leyang Cui, Xueliang Zhao, Lingpeng Kong, and Wei Bi. 2024.  Gsm-plus: A comprehensive benchmark for evaluating the robustness of llms as mathematical problem solvers.  _ArXiv_ , abs/2402.19255.
  * Li et al. (2023b) Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, Qian Liu, Evgenii Zheltonozhskii, Terry Yue Zhuo, Thomas Wang, Olivier Dehaene, Mishig Davaadorj, Joel Lamy-Poirier, João Monteiro, Oleh Shliazhko, Nicolas Gontier, Nicholas Meade, Armel Zebaze, Ming-Ho Yee, Logesh Kumar Umapathi, Jian Zhu, Benjamin Lipkin, Muhtasham Oblokulov, Zhiruo Wang, Rudra Murthy, Jason Stillerman, Siva Sankalp Patel, Dmitry Abulkhanov, Marco Zocca, Manan Dey, Zhihan Zhang, Nourhan Fahmy, Urvashi Bhattacharyya, W. Yu, Swayam Singh, Sasha Luccioni, Paulo Villegas, Maxim Kunakov, Fedor Zhdanov, Manuel Romero, Tony Lee, Nadav Timor, Jennifer Ding, Claire Schlesinger, Hailey Schoelkopf, Jana Ebert, Tri Dao, Mayank Mishra, Alexander Gu, Jennifer Robinson, Carolyn Jane Anderson, Brendan Dolan-Gavitt, Danish Contractor, Siva Reddy, Daniel Fried, Dzmitry Bahdanau, Yacine Jernite, Carlos Muñoz Ferrandis, Sean M. Hughes, Thomas Wolf, Arjun Guha, Leandro von Werra, and Harm de Vries. 2023b.  Starcoder: may the source be with you! _ArXiv_ , abs/2305.06161.
  * Mikolov et al. (2013) Tomas Mikolov, Wen tau Yih, and Geoffrey Zweig. 2013.  Linguistic regularities in continuous space word representations.  In _North American Chapter of the Association for Computational Linguistics_.
  * Nanda et al. (2023) Neel Nanda, Andrew Lee, and Martin Wattenberg. 2023.  Emergent linear representations in world models of self-supervised sequence models.  _ArXiv_ , abs/2309.00941.
  * Nijkamp et al. (2022) Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Haiquan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2022.  Codegen: An open large language model for code with multi-turn program synthesis.  In _International Conference on Learning Representations_.
  * Nostalgebraist (2020) Nostalgebraist. 2020.  Interpreting gpt: The logit lens.
  * OpenAI (2024) OpenAI. 2024.  [link].
  * Pal et al. (2023) Koyena Pal, Jiuding Sun, Andrew Yuan, Byron C. Wallace, and David Bau. 2023.  Future lens: Anticipating subsequent tokens from a single hidden state.  _ArXiv_ , abs/2311.04897.
  * Pimentel et al. (2020) Tiago Pimentel, Josef Valvoda, Rowan Hall Maudslay, Ran Zmigrod, Adina Williams, and Ryan Cotterell. 2020.  Information-theoretic probing for linguistic structure.  _ArXiv_ , abs/2004.03061.
  * Shao et al. (2024) Zhihong Shao, Peiyi Wang, Qihao Zhu, R. X. Xu, Jun-Mei Song, Mingchuan Zhang, Y. K. Li, Yu Wu, and Daya Guo. 2024.  Deepseekmath: Pushing the limits of mathematical reasoning in open language models.  _ArXiv_ , abs/2402.03300.
  * Singh et al. (2024) Chandan Singh, Jeevana Priya Inala, Michel Galley, Rich Caruana, and Jianfeng Gao. 2024.  Rethinking interpretability in the era of large language models.  _ArXiv_ , abs/2402.01761.
  * Stolfo et al. (2023) Alessandro Stolfo, Yonatan Belinkov, and Mrinmaya Sachan. 2023.  A mechanistic interpretation of arithmetic reasoning in language models using causal mediation analysis.  In _Conference on Empirical Methods in Natural Language Processing_.
  * Strathern (1997) Marilyn Strathern. 1997.  ‘improving ratings’: audit in the british university system.  _European Review_ , 5(3):305–321.
  * Team et al. (2023) Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. 2023.  Gemini: a family of highly capable multimodal models.  _arXiv preprint arXiv:2312.11805_.
  * Touvron et al. (2023a) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023a.  Llama: Open and efficient foundation language models.  _ArXiv_ , abs/2302.13971.
  * Touvron et al. (2023b) Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Daniel M. Bikel, Lukas Blecher, Cristian Cantón Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony S. Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel M. Kloumann, A. V. Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, R. Subramanian, Xia Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zhengxu Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023b.  Llama 2: Open foundation and fine-tuned chat models.  _ArXiv_ , abs/2307.09288.
  * Tunstall et al. (2023) Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar Sanseviero, Alexander M. Rush, and Thomas Wolf. 2023.  Zephyr: Direct distillation of lm alignment.  _ArXiv_ , abs/2310.16944.
  * Varma et al. (2023) Vikrant Varma, Rohin Shah, Zachary Kenton, János Kramár, and Ramana Kumar. 2023.  Explaining grokking through circuit efficiency.  _arXiv preprint arXiv:2309.02390_.
  * Wang et al. (2021) Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, and Tao Qin. 2021.  Generalizing to unseen domains: A survey on domain generalization.  _IEEE Transactions on Knowledge and Data Engineering_ , 35:8052–8072.
  * Wu et al. (2024) Wilson Wu, John X Morris, and Lionel Levine. 2024.  Do language models plan ahead for future tokens?  _arXiv preprint arXiv:2404.00859_.
  * Yang et al. (2024) Sohee Yang, Elena Gribovskaya, Nora Kassner, Mor Geva, and Sebastian Riedel. 2024.  Do large language models latently perform multi-hop reasoning? _ArXiv_ , abs/2402.16837.
  * Zhang et al. (2024) Yihao Zhang, Zeming Wei, Jun Sun, and Meng Sun. 2024.  Towards general conceptual model editing via adversarial representation engineering.
  * Zhao et al. (2023) Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Z. Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jianyun Nie, and Ji rong Wen. 2023.  A survey of large language models.  _ArXiv_ , abs/2303.18223.
  * Zhu et al. (2024) Fangwei Zhu, Damai Dai, and Zhifang Sui. 2024.  Language models understand numbers, at least partially.  _ArXiv_ , abs/2401.03735.
  * Zou et al. (2023) Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Troy Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, Zico Kolter, and Dan Hendrycks. 2023.  Representation engineering: A top-down approach to ai transparency.  _ArXiv_ , abs/2310.01405.


# Grading and Filtrations of Gamma Rings

Shadi Shaqaqha and Afnan Dagher  Department of Mathematics, Yarmouk
University, Irbid, Jordan<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract.

The aim of this paper is to introduce and study graded and filtered gamma
rings and gamma modules. We prove that the filtered $\Gamma$-ring (module) is
a generalization of the notion of graded ring (module). Also, we construct a
graded $\Gamma$-ring from a filtred $\Gamma$-ring. We investigate some
properties of graded and fileterd Gamma rings and Gamma modules. Finally we
define and study the strongly graded gamma rings.

###### Key words and phrases:

Gamma rings; Gamma ring homomorphism; graded $\Gamma$-rings; Graded
$\Gamma$-modules; Filtered Gamma rings; Filtered Gamma modules; Strongly
graded Gamma rings.

###### 2000 Mathematics Subject Classification:

16W50, 16W70, 16U80

##  1\. INTRODUCTION

The concept of $\Gamma$-rings was first introduced by Nobusawa in 1964 ([13]).
Later, Barnes generalized the notion of Nobusawa’s $\Gamma$-rings and
established a new definition of a $\Gamma$-ring ([4]). Nowadays, the notion of
$\Gamma$-rings means the $\Gamma$-rings due to Barnes. Many mathematicians
have been involved in extending the concepts and results of ring theory to the
broader framework of the $\Gamma$-ring setting (see e.g. [2, 16, 18, 19, 20]
and references therein).
The graded and filtered rings play a role in various branches of mathematics,
and they have become increasingly important. Consequently, the graded
analogues of different concept are widely studied (see [3, 5, 6, 8, 12, 10,
14]). For example, S. Shaqaqha, in his dissertation, considered gradings and
filtrations on colour Lie superalgebras and obtained more Schreier-type
formulas in the case of subalgebras of free colour Lie superalgebras ( [14]).
So, our results are expected to be useful in various applications.
In [9], a group graded gamma ring is defined to be a $\Gamma$-ring $R$ which
is the direct sum of additive subgroups $R_{g}$, $g\in K$, such that
$R_{g}\Gamma R_{h}\subseteq R_{gh}$, for all $g,h\in K$, where $K$ is a group
defined multiplicatively. In this paper we replace the group grading with a
semigroup then we introduce the notion of filterd $\Gamma$-rings. We extend
known results in the the case of graded rings to graded $\Gamma$-rings.
The article is organized as follows. In Section 2, we recall some definitions
and notations that will be used in our work. In Section 3, we introduce the
notion of graded gamma rings and we ofer some examples. In Section 4, we
obtain some properties of semigroup graded gamma rings. We present ways to
obtain new graded gamma rings from old ones. In Section 5, we define filtered
gamma rings. We give a recipe to form new graded gamma ring from a filtered
gamma ring. In Section 6, we study semigroup graded gamma modules. In Section
7, we obtain a way to produce a ring out of old gamma rings. In Section 8, we
study filtered gamma modules and obtain some properties. The strongly graded
gamma rings are introduced and studied in Section 9.

##  2\. Preliminaries

The notion of gamma rings is a one of the generalizations of rings.

######  Definition 2.1.

Let $\Gamma$ be an additive abelian group. A $\Gamma$-ring (in the sense of
Barnes) is an additive abelian group $R$ together with a map

$R\times\Gamma\times R\rightarrow R:(x,\alpha,y)\mapsto x\alpha y$

and satisfying the following two identities, for any $x,y,z\in R$ and
$\alpha,\beta\in\Gamma$:

  * (i)

$(x+y)\alpha z=x\alpha z+y\alpha z$, $x(\alpha+\beta)z=x\alpha z+x\beta z$,
$x\alpha(y+z)=x\alpha y+x\alpha z$.

  * (ii)

$(x\alpha y)\beta z=x\alpha(y\beta z)$.

Clearly every ring $R$ can be regarded as an $R$-ring. Let $G$ be an additive
group. We donote by $G_{m,n}$ the set of all $m\times n$ matrices over $G$.
Let $M$ be a $\Gamma$-ring. Then $M_{m,n}$ forms a $\Gamma_{n,m}$-ring (see
[19]).
Let $R$ be a $\Gamma$-ring. An element $1\in R$ is called a unity of $R$ if
$a\gamma 1=1\gamma a=a$ for all $a\in R$ and for some $\gamma\in\Gamma$. In
this case, we say $R$ is a $\Gamma$-ring with unity. One can easily show that
if a $\Gamma$-ring has a unity with respect to $\gamma\in\Gamma$, then it is
unique.
A nonempty subset $S$ of a $\Gamma$-ring $R$ is a sub-$\Gamma$-ring of $R$ if
$a-b\in S$ and $a\gamma b\in S$ for any $a,b\in S$ and $\gamma\in\Gamma$. A
subset $I$ of the $\Gamma$-ring $R$ is a left (right) ideal of $R$ if $I$ is
an additive subgroup of $R$ and $R\Gamma I=\\{r\alpha a~{}|~{}r\in
R,\alpha\in\Gamma,a\in I\\}$ ($I\Gamma R$) is contained in $I$. If $I$ is both
a left and a right ideal of $I$, then we say that $I$ is an ideal (or two-
sided ideal) of $R$. It is clear that the intersection of any number of left
(respectively right or two-sided) ideals of $R$ is also a left (respectively
right or two-sided) ideal of $R$.
Let $I$ be a (two-sided) ideal of a $\Gamma$-ring $R$. then the factor group
$R/I={x+I:x\in R}$ of all cosets of $I$ is a $\Gamma$-ring with respect to the
following multiplication:

$\displaystyle(x+I)(\gamma)(y+I)$ $\displaystyle=$ $\displaystyle((x\gamma
y)+I$

for $x,y\in R$ and $\gamma\in\Gamma$. Let $R_{1},R_{2},\ldots,R_{n}$ be
$\Gamma$-rings. Then the direct product
$\prod_{i=1}^{n}R_{i}=R_{1}\times\cdots\times R_{n}$ has the structure of a
$\Gamma$-ring under the componentwise addition and the following
multiplication

$(r_{1},\ldots,r_{n})\gamma(s_{,},\ldots,s_{n})=(r_{1}\gamma
s_{1},\ldots,r_{n}\gamma s_{n})$

for all $r_{i},s_{i}\in R_{i}~{}(i=1,\ldots,n)$ and $\gamma\in\Gamma$
Suppose that $R_{1}$ and $R_{2}$ are $\Gamma$-rings and let
$\varphi:\Gamma\rightarrow\Gamma$ be a group isomorphism. Then
$f:R_{1}\rightarrow R_{2}$ is $\varphi$-homomorphism if $f(x+y)=f(x)+f(y)$ and
$f(x\gamma y)=f(x)\varphi(\gamma)f(y)$ for all $x,y\in R_{1}$ and
$\gamma\in\Gamma$. In particular, if $\varphi$ is the identity map, then $f$
is homomorphism.

##  3\. Basic Definition and Examples

######  Definition 3.1.

Suppose that $G$ is an abelian semigroup written multiplicatively. Let $R$ be
a $\Gamma-$ring. Then $R$ is called a graded $\Gamma-$ring of type $G$ if
there exist additive subgroups $R_{g}$ of $R$ such that $R=\oplus_{g\in
G}R_{g}$ and $R_{g}\Gamma R_{h}\subseteq R_{gh}$ for all $g,h\in G$. A nonzero
element $r\in R$ is called homogeneous if there is $g\in G$ such that $r\in
R_{g}$, and in this case we say the degree of $r$ is $g$ and we write
$d(m)=g$.

So, any nonzero element $r\in R$, where $R$ is a graded $\Gamma$-ring of type
$G$, has a unique representation as a sum of a finite set of homogenous
elements $r=r_{g_{1}}+\cdots+r_{g_{k}}$. In this case, we say that
$r_{g_{1}},r_{g_{2}},\ldots$, and $r_{g_{k}}$ are the homogeneous components
of $r$. The set $G_{R}=\\{g\in G~{}|~{}R_{g}\neq\\{0\\}\\}$ is called the
support of $R$.
Throughout the paper we denote by $G$ an abelian semigroup (written
multiplicatively) unless otherwise stated.
A two-sided ideal $I$ of a graded $\Gamma$-ring $R$ of type $G$ is called a
graded $\Gamma$-ideal (or homogeneous $\Gamma$-ideal) if $I=\bigoplus_{g\in
G}I_{g}$, where $I_{g}=I\cap R_{g}$. In other words, $I$ is a graded
$\Gamma$-ideal if and only if , for each $x\in I$, its homogeneous components
belong also to $I$.

######  Example 3.1.

  * (i)

Let $G$ be a semigroup with identity $e$ (or a monoid). Any $\Gamma$-ring $R$
may be considered as a graded $\Gamma$-ring of type $G$ by putting $R_{e}=R$,
and $R_{g}=\\{0\\}$ for any $g\neq e$ in $G$ (i.e. $M$ has the trivial
subgroup support). Such a graded $\Gamma$-ring is called trivial.

  * (ii)

If $I$ is a graded $\Gamma$-ideal of a graded $\Gamma$-ring $R$ of type $G$,
then $R/I$ is also a graded $\Gamma$-ring and has the following decomposition:

$R/I=\bigoplus_{g\in G}\left(R_{g}+I\right)/I\cong\bigoplus_{g\in
G}\left(R_{g}/\left(R_{g}\cap I_{g}\right)\right).$

(The last congruence follows from the second isomorphism theorem for groups).

  * (iii)

Given a semigroup $G$ and a $\Gamma$-ring $R$. Set $RG$ to be the set of all
linear combinations

$r=\sum_{g\in G}r_{g}g,$

where $r_{g}\in R$ and where only finitely many of the $r_{g}^{\prime}s$ are
nonzero. Next, we define the addition and the multiplication on $RG$ as
follows: for $\alpha=\sum_{g\in G}a_{g}g,\beta=\sum_{g\in G}b_{g}g\in RG$ and
$\gamma\in\Gamma$, we have

$\displaystyle\alpha+\beta$ $\displaystyle=$ $\displaystyle\sum_{g\in
G}(a_{g}+b_{g})g,$ $\displaystyle\alpha\gamma\beta$ $\displaystyle=$
$\displaystyle\sum_{g,h\in G}(a_{g}\gamma b_{h})gh.$

Then $RG$ is a graded $\Gamma$-ring of type $G$. Such graded $\Gamma$-ring is
called semigroup $\Gamma$-ring (and if $G$ is group, then it is called group
Gamma ring).

  * (iv)

Let $R$ be a $\Gamma$-ring. Then $R^{0}=R$ is a $\Gamma$-ring too under the
same addition on $R$, but the multiplication is defined by $x\circ\gamma\circ
y=y\gamma x$ for $x,y\in R^{0}$, and $\gamma\in\Gamma$. Furthermore, if $R$ is
a graded $\Gamma$-ring of type $G$, where $G$ is a group, then $R^{0}$ is a
graded $\Gamma$-ring of type $G$ too by setting $(R^{0})_{g}=R_{g^{-1}}$,
$g\in G$. Indeed for $x\in(R^{0})_{g}$, $y\in(R^{0})_{h}$, and
$\alpha\in\Gamma$, we have $x\circ\alpha\circ y=y\alpha x\in
R_{h^{-1}g^{-1}}=R_{(gh)^{-1}}=(R^{0})_{gh}$.

  * (v)

Let $R=\bigoplus_{g\in G}R_{g}$ and $S=\bigoplus_{g\in G}S_{g}$ be graded
$\Gamma$-rings of type $G$. Then the $\Gamma$-ring $R\times S$ may be made
into a graded $\Gamma$-ring by setting:

$(R\times S)_{g}=R_{g}\times S_{g}~{}\forall g\in G.$

More generally, if $(R_{i}){i\in I}$, where $I$ is finite, is a family of
graded $\Gamma$-rings of type $G$, then $R=\prod_{i\in I}R_{i}$ is a graded
$\Gamma$-ring of type $G$ by setting

$\left(\prod_{i\in I}R_{i}\right)_{g}=\prod_{i\in I}(R_{i})_{g}.$

##  4\. Some Properties of Graded Gamma Rings

The following two theorems give us recipes for forming new graded gamma ring
from old ones.

######  Theorem 4.1.

Let $R$ be a graded $\Gamma$-ring of type $G$ where $G$ is a semigroup. If
$\varphi:G\rightarrow H$ is an onto homomorphism (epimorphism) of semigroups,
then the $\Gamma$-ring $S=R$ with gradation

$S_{h}=\bigoplus_{g\in\varphi^{-1}(h)}R_{g}$

for all $h\in H$ is a graded $\Gamma$-ring of type $H$ (such graded
$\Gamma$-ring is denoted by $R_{(H)}$).

Proof.  Let $a,b\in H$, $x\in S_{a}$ and $y\in S_{b}$. Then
$x=x_{a_{1}}+\cdots+x_{a_{r}}$ and $y=y_{b_{1}}+\cdots+y_{b_{s}}$, where
$\varphi(a_{i})=a$ and $\varphi(b_{j})=b$ for all $i,j$. Now, for
$\gamma\in\Gamma$, we obtain

$x\gamma y=\sum_{1\leq i\leq r,1\leq j\leq m}x_{a_{i}}\gamma y_{b_{j}}.$

Also, for $1\leq i\leq r$ and $1\leq j\leq m$, we have
$\varphi(a_{i}b_{j})=\varphi(a_{i})\varphi(b_{j})=ab$. Thus $x\gamma y\in
S_{ab}$. $\Box$

Also, we have the following result. We omit the proof since the proof is
straightforward.

######  Theorem 4.2.

Let $R$ be a graded $\Gamma$-ring of type $G$, and let $H$ be a subsemigroup
of $G$. Then

  * (i)

$R^{\prime}=\bigoplus_{h\in H}R_{h}$ is a graded $\Gamma$-ring of type $H$
(such graded is denoted by $R^{(H)}$). In particular, if $e$ is identity of
$G$, then $R_{e}$ corresponds to the trivial subsemigroup of $G$.

  * (ii)

If $H$ is a normal subsemigroup of $G$, then

$R=\bigoplus_{gN\in G/N}R_{gN}$

is a graded $\Gamma$-ring of type $G/N$, where

$R_{gN}=\bigoplus_{n\in N}R_{gn}.$

The following theorem extends a well known result to group graded rings.

######  Theorem 4.3.

Let $G$ be a monoid with the identity element $e$, and let $R$ be a graded
$\Gamma$-ring of type $G$. Then $R_{e}$ is a sub-$\Gamma$-ring of $R$. Also if
$1$ is the unity element with respect to $\gamma_{0}\in\Gamma$, then $1\in
R_{e}$.

Proof.  For $a,b\in R_{e}$, we have $a-b\in R_{e}$ since $M_{e}$ is an
additive subgroup of $R$, and also $a\gamma b\in R_{ee}=R_{e}$ for any
$\gamma\in\Gamma$. It follows that $R_{e}$ is a sub-$\Gamma$-ring of $R$.
Next, as $R=\bigoplus_{g\in G}R_{g}$, we may assume
$1=r_{g_{1}}+r_{g_{2}}+\cdots+r_{g_{n}}$. Pick $\tau\in G$, and a nonzero
element $\lambda_{\tau}\in R_{\tau}$, then

$1\gamma_{0}\lambda_{\tau}=\lambda_{\tau}=r_{g_{1}}\gamma_{0}\lambda_{\tau}+r_{g_{2}}\gamma_{0}\lambda_{\tau}+\cdots+r_{g_{m}}\gamma_{0}\lambda_{\tau}.$

and so, for all $g_{i}\in G$ with $g_{i}\neq e$ we have
$r_{g_{i}}\gamma_{0}\lambda_{\tau}=0$. Hence,
$1\gamma_{0}\lambda_{\tau}=r_{e}\gamma_{0}\lambda_{\tau}$. Therefore, by the
uniqueness of the unity, we have $1=r_{e}\in R_{e}$. $\Box$

Let $R$ be a $\Gamma$-ring with a unity $1$ corresponding to
$\gamma_{0}\in\Gamma$. An element $r\in R$ is called invertible with respect
to $1$ and $\gamma_{0}$ if there is an element (unique) $r^{-1}\in R$ such
that $r^{-1}\gamma_{0}r=r\gamma_{0}r^{-1}=1$.

######  Theorem 4.4.

Let $G$ be a group with the identity element $e$, and let $R$ be a graded
$\Gamma$-ring of type $G$. If $r\in R_{g}$ is an invertible (corresponding to
$\gamma_{0}\in\Gamma$) homogeneous element, then $r^{-1}$ is homogeneous too
of degree $g^{-1}$.

Proof.  Let $s=s_{g_{1}}+\cdots+s_{g_{k}}$ be the inverse of $r$. Then

$1=r\gamma_{0}s=r\gamma_{0}s_{g_{1}}+\cdots+r\gamma_{0}s_{g_{k}},$

where $r\gamma_{0}s_{g_{i}}\in R_{gg_{i}}$ for all $i=1,\ldots,k$. By Theorem
4.3, $1$ is homogeneous of degree $e$, and also, the decomposition is unique,
so that $r\gamma_{0}s_{g_{i}}=0$ for each $g_{i}\neq g^{-1}$. Since $r$ is
invertible, $s_{g^{-1}}\neq 0$, and so $s=s_{g^{-1}}\in R_{g^{-1}}$ as
desired. $\Box$

##  5\. Filtered Gamma Rings

The following definition produces the notion of filtered $\Gamma$-rings. It is
a generalization of the notion of filtered rings [11].

######  Definition 5.1.

Let $R$ be a $\Gamma$-ring and let $R^{0}\subseteq R^{1}\subseteq
R^{2}\subseteq\cdots$ be a chain of additive subgroups of $R$. This chain is
called an (ascending) filtration of $R$ if

$\bigcup_{r\geq 0}R^{m}=R~{}\mathrm{and}~{}R^{i}\Gamma R^{j}\subseteq
R^{i+j}~{}\forall i,j\geq 0.$

In this case we say that $R$ is a filtered $R_{\Gamma}$-ring.

Let $R$ and $S$ be filtered $\Gamma$-rings. A homomorphism
$\varphi:R\rightarrow S$ is called a filtered homomorphism if
$\varphi(R_{i})\subseteq S_{i}$ for $i=0,1,\ldots$.

######  Example 5.1.

  * (i)

Any $\Gamma$-ring $R$ can be made into a filtered $\Gamma$-ring by setting
$R^{k}=R$ for all $k\geq 0$. Such filtration is called trivial.

  * (ii)

Let $R$ be a $\Gamma-$ ring, and let

$R[x]=\\{p(x)=a_{0}+a_{1}x+\cdots+a_{n}x^{n}:a_{0},a_{1},\cdots,a_{n}\in
R\\}.$

Then it is easy to see that $R[x]$ is a $\Gamma$-ring under the ordinary
addition and the multiplication defined as follows: if
$p(x)=a_{0}+a_{1}x+\cdots+a_{n}x^{n},q(x)=b_{0}+b_{1}x+\cdots+b_{m}x^{m}\in
R[x]$ such that $a_{n}\neq 0\neq b_{m}$, then

$p(x).\alpha.q(x)=a_{0}\alpha b_{0}+(a_{0}\alpha b_{1}+b_{0}\alpha
a_{1})x+\cdots+a_{n}\alpha b_{m}x^{m+n}$

. Moreover, $R[x]$ has a filtration $\bigcup_{k=0}^{\infty}R^{k}$, where
$R^{k}$ is spanned by all monomials of degree less than or equal to $k$.

  * (iii)

Given a graded $\Gamma$-ring $R=\bigoplus_{i\geq 0}R_{i}$ of type
$\mathbb{Z}$, then there is a corresponding filtration

$\bigcup_{k=0}^{\infty}R^{k},$

where $R^{k}=\bigoplus_{j=0}^{k}R_{j}$.

Conversely, the following result gives us a way to produce a graded
$\Gamma$-ring out of a filtered $\Gamma$-ring.

######  Theorem 5.1.

Let $R$ be a $\Gamma$-ring with filtration $R^{0}\subseteq R^{1}\subseteq
R^{2}\subseteq\cdots$. Set $R^{-1}=0$, and let
$\mathrm{gr}R=\bigoplus_{k=0}^{\infty}\left(R^{k}/R^{k-1}\right)$. Then define
multiplication

$R^{m}/R^{m-1}\times\Gamma\times R^{n}/R^{n-1}\rightarrow R^{m+n}/R^{m+n-1}$

by $\left(r+R^{m-1}\right)\gamma\left(s+R^{n-1}\right)=(r\gamma s)+R^{m+n-1}$.
Extending this multiplication by multilinearity makes $\mathrm{gr}R$ is a
graded $\Gamma$-ring.

Proof. We must show that the map is well defined. Suppose that $x+R^{m-1}=x\
^{\prime}+R^{m-1}$, $y+R^{n-1}=y\ ^{\prime}+R^{n-1}$, and $\gamma\in\Gamma$.
Then $x-x\ ^{\prime}=r_{0}$ and $y-y\ ^{\prime}=r_{1}$ for some $r_{0}\in
R^{m-1}$ and $r_{1}\in R^{n-1}$. Thus

$\displaystyle x\gamma y-x\ ^{\prime}\gamma y\ ^{\prime}$ $\displaystyle=$
$\displaystyle(x\ ^{\prime}+r_{0})\gamma(y\ ^{\prime}+r_{1})-x\
^{\prime}\gamma y\ ^{\prime}$ $\displaystyle=$ $\displaystyle x\
^{\prime}\gamma r_{1}+r_{0}\gamma y\ ^{\prime}+r_{0}\gamma r_{1}$

is a member of $R^{m+n-1}$. The remainder of the proof is quite
straightforward.

$\Box$

######  Remark 1.

If the filtration $R^{k}$, given in the theorem above, comes from a grading
(i.e. $R^{k}=\bigoplus_{i=0}^{k}R_{i}$), then $R^{k}/R^{k-1}\cong R_{k}$ as
additive groups, and so $\mathrm{gr}R\cong R$ as $\Gamma$-rings.

##  6\. Graded Gamma Modules

Let $R$ be a $\Gamma$-ring. An abelian group $M$ together with a mapping

$R\times\Gamma\times M\rightarrow M;~{}(r,\gamma,m)\mapsto r\gamma m,$

such that the following identities are satisfied for all $r_{1},r_{2},r\in R$,
$\gamma_{1},\gamma_{2},\gamma\in\Gamma$, and $m_{1},m_{2},m\in M$:

  * (i)

$r\gamma(m_{1}+m_{2})=r\gamma m_{1}+r\gamma m_{2}$,

  * (ii)

$(r_{1}+r_{2})\gamma m=r_{1}\gamma m+r_{2}\gamma m$,

  * (iii)

$r(\gamma_{1}+\gamma_{2})m=r\gamma_{1}m+r\gamma_{2}m$,

  * (iv)

$(r_{1}\gamma_{1})(r_{2}\gamma_{2}m)=(r_{1}\gamma_{1}r_{2})\gamma_{2}m$

is called a (left) $R_{\Gamma}$-module. A right $R_{\Gamma}$-module is defined
in analogous manner ([1]). If $R$ and $S$ are $\Gamma$-rings, then we say $M$
is an $(R,S)_{\Gamma}$-bimodule if it is both left $R_{\Gamma}$-module and
right $S_{\Gamma}$-module and simultaneously $(r\alpha m)\beta
s=r\alpha(m\beta s)$ for all $r\in R$, $m\in M$, $s\in S$, and
$\alpha,\beta\in\Gamma$. A (left) $R_{\Gamma}$-module $M$ is unitary if there
exists an element , say $1$, in $R$ and $\gamma_{0}\in\Gamma$ such that
$1\gamma_{0}m=m~{}\forall m\in M$. Note that if $M$ is a left
$R_{\Gamma}$-module then it is easy to verify that $0\gamma m=r0m=r\gamma 0=0$
for any $r\in R,\gamma\in\Gamma$ and $m\in M$.

######  Example 6.1.

  * (i)

If $R$ is a $\Gamma$-ring and $M$ is an abelian group, then $M$ is an
$R_{\Gamma}$-module by letting $r\gamma m=0~{}\forall r\in
R,\gamma\in\Gamma,m\in M$.

  * (ii)

Every $\Gamma$-ring $R$ is an $R_{\Gamma}$-module by

$R\times\Gamma\times R\rightarrow R;~{}(r_{1},\gamma,r_{2})\mapsto r_{1}\gamma
r_{2}.$

  * (iii)

Let $R$ be a graded $\Gamma$-ring of type $G$, where $G$ is a monoid with
identity $e$. Then $R$ can be considered as an
$\left(R_{e},R_{e}\right)_{\Gamma}$-bimodule.

Let $R$ be a $\Gamma$-ring. A nonempty subset $M_{1}$ of $M$ is called a
(left) $R_{\Gamma}$-submodule of $M$ if $M_{1}$ is a subgroup of $M$ and
$R\Gamma M_{1}\subseteq M_{1}$. In this case we write $M_{1}\leq M$. For a
$\Gamma$-ring $R$, $\\{0\\}$ and $R$ are $R_{\Gamma}$-submodules of
$R_{\Gamma}$-module $R$. Also, every ideal of $R$ is a submodule of $R$.
The following definition is a generalization of the notion of a graded module.

######  Definition 6.1.

Let $R$ be a graded $\Gamma$-ring of type $G$, and $M$ be an
$R_{\Gamma}$-module. Then $M$ is said to be graded (left) $R_{\Gamma}$-module
if there is a family $\\{M_{g}~{}:~{}g\in G\\}$ of additive subgroups of $M$
such that $M=\bigoplus_{g\in G}M_{g}$ and $R_{g}\Gamma M_{h}\subseteq M_{gh}$
for all $g,h\in G$. A nonzero element $m\in M$ is called homogeneous of degree
$g$, and we write $\mathrm{deg}(m)=g$, if there $g\in G$ such that $m\in
M_{g}$. A submodule $N$ of $M$ is a graded submodule if $N=\bigoplus_{g\in
G}(N\cap M_{g})$ (equivalently, for any $x\in N$ the homogeneous component(s)
of $x$ are again in $N$).

######  Example 6.2.

Let $R$ be a graded $\Gamma$-ring of type $G$, $M$ be a graded
$R_{\Gamma}$-module, and $K$ be a submodule of $M$. Then
$K^{\prime}=\bigoplus_{g\in G}K_{g}$, where $K_{g}$ is the additive subgroup
of $K$ generated by $K\cap(h(M))_{g}$ where $(h(M))_{g}$ is the set of all
homogeneous elements of degree $g$ (i.e. $K_{g}$ is the smallest subgroup of
$G$ that contains $K\cap(h(M))_{g}$), is a graded submodule of $M$. One can
easily show that it is a maximal submodule of $K$ that is a graded submodule
of $M$.

It is known that if $R$ is a $\Gamma$ ring, $M$ is a $R_{\Gamma}$-ring, and
$K$ is a subgroup of $M$, then that factor group $M/K$ is an
$R_{\Gamma}$-module under the mapping

$R\times\Gamma\times M/K\rightharpoonup M/K;~{}(r,\gamma,m+K)\mapsto(r\gamma
m)+M$

(See [1]). Using the same arguments in the graded rings we can prove the
following result.

######  Proposition 6.1.

Let $R$ be a graded $\Gamma$-ring of type $G$ and $M$ be a graded
$R_{\Gamma}$-module. If $K$ is a submodule of $M$, then the factor module
$M/K$ is a graded $R_{\Gamma}$-module by setting

$(M/K)_{g}=(M_{g}+K)/K\cong M_{g}/(K\cap M_{g})$

for any $g\in G$.

##  7\. On Homomorphisms of Gamma Modules

Suppose that $R$ is a graded $\Gamma$-ring of type $G$ and that $M$ and $K$
are (left) graded $R_{\Gamma}$-module. Let $\varphi:\Gamma\rightarrow\Gamma$
be a group isomorphism. Then $f:M\rightarrow K$ is $\varphi$-homomorphism if
$f(x+y)=f(x)+f(y)$ and $f(r\gamma x)=r\varphi(\gamma)f(x)$ for all $x,y\in M$,
$r\in R$, and $\gamma\in\Gamma$. In particular, if $\varphi$ is the identity
map, then $f$ is homomorphism. It is called isomorphism if it is one to one
and onto.
For graded (left) $R_{\Gamma}$-modules $M$ and $K$. A ($\varphi$-)homomorphism
$f:M\rightarrow K$ is called homogeneous of degree $h\in G$ if for all $g\in
G$, we have $f(M_{g})\subseteq K_{hg}$. In particular, for graded
$\Gamma$-rings $R$ and $K$. A ($\varphi$-)homomorphism of $\Gamma$-ringe
$f:R\rightarrow K$ is called homogeneous of degree $h\in G$ if for all $g\in
G$, we have $f(R_{g})\subseteq K_{hg}$. A homogeneous $(\varphi-)$homomorphism
of $R_{\Gamma}$-modules of degree $e\in G$ (the identity element of $G$) will
be called a degree preserving map. Clearly the set $\mathrm{Hom}_{R}(M,K)$ of
all homomorphisms of $R_{\Gamma}$ modules from $M$ into $K$ forms a group
under the ordinary addition. Also, for graded $R_{\Gamma}$-modules
$M_{1},M_{2}$, and $M_{3}$ the composition of a homogeneous
($\varphi$-)homomorphism $f:M_{1}\rightarrow M_{2}$ of degree $h\in G$ with a
($\varphi$-)homomorphism $g:M_{2}\rightarrow M_{3}$ of degree $k\in G$ is
($\varphi$-)homomorphism of degree $hk$.

######  Definition 7.1.

Let $R$ be a $\Gamma$-ring and $M$ be an $R_{\Gamma}$-module. Then we say $M$
is finitely generated if there exist $m_{1},\ldots,m_{k}$ in $M$ such that if
$x\in M$, then there exist $r_{1},\ldots,r_{k}\in R$ and
$\gamma_{1},\ldots,\gamma_{k}\in\Gamma$ with
$x=r_{1}\gamma_{1}m_{1}+\cdots+r_{k}\gamma_{k}m_{k}$. In this case the set
$\\{m_{1},\ldots,m_{k}\\}$ is said a generating set to $M$.

The following theorem gives us a procedure to produce a ring out of old
$\Gamma$-ring.

######  Theorem 7.1.

Let $M$ and $K$ be graded $R_{\Gamma}$-modules of type $G$. If $M$ is a
finitely generated, then

$\mathrm{Hom}_{R}(M,K)=\bigoplus_{g\in G}\left(\mathrm{Hom}(M,K)\right)_{g}$

is a graded abelian group, where $\mathrm{Hom}_{R}(M,K)_{g}$ is the subgroup
of $\mathrm{Hom}_{R}(M,K)$ that consisting of all homomorphisms of from $M$ to
$K$ of degree $g$. Moreover if $M=K$, then $\mathrm{Hom}(M,M)$ is a graded
ring of type $G$.

Proof.  Suppose that $f\in\mathrm{Hom}_{R}(M,K)$. Let $g\in G$. Define a map

$f_{g}:M\rightarrow K$

as follows: for $m=m_{g_{1}}+m_{g_{2}}+\cdots+m_{g_{k}}\in M$
($g_{1},g_{2},\ldots,g_{k}\in G$), set

$f_{g}(m)=\left(f(m_{g_{1}g^{-1}})\right)_{g_{1}}+\cdots+\left(f(m_{g_{k}g^{-1}})\right)_{g_{k}}.$

Then $f_{g}\in\mathrm{Hom}_{R}(M,K)$. Indeed if $x=\sum_{h\in G}x_{h}$,
$y=\sum_{h\in G}y_{h}\in M$, then

$\displaystyle f_{g}(x+y)$ $\displaystyle=$ $\displaystyle\sum_{h\in
G}\left(f(x+y)_{hg^{-1}}\right)_{h}$ $\displaystyle=$ $\displaystyle\sum_{h\in
G}\left((f(x_{hg^{-1}}+y_{hg^{-1}})\right)_{h}$ $\displaystyle=$
$\displaystyle\sum_{h\in G}\left((f(x_{hg^{-1}})+f(y_{hg^{-1}})\right)_{h}$
$\displaystyle=$ $\displaystyle\sum_{h\in
G}\left(\left(f(x_{hg^{-1}}\right)_{h}+\left((f(y_{hg^{-1}}\right)_{h}\right)$
$\displaystyle=$ $\displaystyle f_{g}(x)+f_{g}(y).$

Also, for $r\in R$ and $\gamma\in\Gamma$, we have

$\displaystyle f_{g}(r\gamma x)$ $\displaystyle=$ $\displaystyle\sum_{h\in
G}\left(f((r\gamma x)_{hg^{-1}})\right)_{h}$ $\displaystyle=$
$\displaystyle\sum_{h\in G}\left(r\gamma f(x_{hg^{-1}})\right)_{h}$
$\displaystyle=$ $\displaystyle r\gamma\sum_{h\in
G}\left((f(x_{hg^{-1}})\right)_{h}$ $\displaystyle=$ $\displaystyle r\gamma
f_{g}(x).$

In addition, $f_{g}\in\left(\mathrm{Hom}_{R}(M,K)\right)_{g}$. Indeed for any
$h\in G$ and $m\in M_{h}$, we have $f_{g}(m)=(f(m))_{hg}\in K_{hg}$.
Let us assume that $\\{m_{1},\ldots,m_{s}\\}$ is a generating set to $M$. For
$x\in M$, there exist $r_{1},\ldots,r_{s}\in R$ and
$\gamma_{1},\ldots,\gamma_{s}\in\Gamma$ with
$x=r_{1}\gamma_{1}m_{1}+\cdots+r_{s}\gamma_{s}m_{s}$. Thus
$f(x)=r_{1}\gamma_{1}f(m_{1})+\cdots+r_{s}\gamma_{s}f(m_{s})$. On the other
hand for each $t=1,2,\ldots,r$, $f_{g}(m_{t})=0$ for all but a finite number
of $g\in G$ and

$\sum_{g\in G}f_{g}(m_{t})=\sum_{g\in G}(f(m_{t}))_{hg}=f(m_{t}).$

It follows again that $f_{g}(x)=0$ for all but finitely number of $g\in G$,
and $f=\sum_{g\in G}f_{g}$. We have so far shown that
$\mathrm{Hom}_{R}(M,K)=\sum_{g\in G}(\mathrm{Hom}_{R}(M,K))_{g}$. On the other
hand if $f\in\mathrm{Hom}_{R}(M,K)_{h_{1}}\cap\mathrm{Hom}_{R}(M,K)_{h_{2}}$
where $h_{1},h_{2}\in G$ and $h_{1}\neq h_{2}$, then for any $g\in G$ we have
$f(M_{g})\subseteq K_{h_{1}g}\cap K_{h_{2}g}=\\{0\\}$. Hence $f=0$. This shows
that $\mathrm{Hom}_{R}(M,K)=\bigoplus_{g\in G}(\mathrm{Hom}_{R}(M,K))_{g}$. In
particular, if $M=K$, we obtain $\mathrm{Hom}_{R}(M,M)=\bigoplus_{g\in
G}(\mathrm{Hom}_{R}(M,M))_{g}$. For $\rho\in\mathrm{Hom}_{R}(M,M))_{h_{1}}$
and $\sigma\in\mathrm{Hom}_{R}(M,M)_{h_{2}}$ we have
$\rho\sigma\in\mathrm{Hom}_{R}(M,M)_{h_{1}h_{2}}$. It is clear now that
$\mathrm{Hom}_{R}(M,M)$ is a graded ring of type $G$. $\Box$

##  8\. Filtered $\Gamma$-Modules

Let $R$ be a filtered $\Gamma$-ring and $M$ be a (left) $R_{\Gamma}$-module.
Then $M$ is called a filtered $R_{\Gamma}$ module if there is an (ascending)
chain

$M^{0}\subseteq M^{1}\subseteq M^{2}\subseteq\cdots$

of additive subgroups of $M$ such that $\bigcup_{k\geq 0}M^{k}=M$ and
$R^{i}\Gamma M^{j}\subseteq M^{i+j}$ for any $i,j$.

######  Example 8.1.

  * (i)

It is clear that if $R$ is a filtered $\Gamma$-ring, then $R$ is a filtered
$R_{\Gamma}$-module.

  * (ii)

If the filtration of a graded $\Gamma$-ring $R$ is trivial and $M$ is
$R_{\Gamma}$-module, then any ascending chain of submodules $K^{s}$, where
$\bigcup_{s\geq 0}K^{s}=M$, of $M$ defines a filtration for $M$.

  * (iii)

Let $R$ be a $\Gamma$-ring and $I\subseteq R$ be an ideal of $R$. Then $R$ has
a descending filtration $\bigcup_{i=0}^{\infty}R^{k}$, where

$R^{0}=R,R^{1}=I,R^{2}=I\Gamma I,~{}\mathrm{and}~{}R^{k}=R^{k-1}\Gamma
I~{}\mathrm{for}~{}k\geq 3.$

Such filtration is called $I$-adic filtration. Let $M$ be $R_{\Gamma}$-module.
The corresponding descending filtration of $M$

$M\supseteq R^{1}\Gamma M\supseteq R^{2}\Gamma M\supseteq\cdots.$

Let $R$ be a graded $\Gamma$-ring and let $M=\bigoplus_{i=0}^{\infty}M_{i}$ be
a graded $R_{\Gamma}$-module. Consider the corresponding filtration of $R$ as
in Example 5.1(iii), then there is a corresponding filtration on $M$

$M^{0}\subseteq M^{1}\subseteq M^{2}\subseteq\cdots$

where $M^{k}=\bigoplus_{j=0}^{k}M_{j}$. Conversely, one can prove the
following result.

######  Theorem 8.1.

If $M$ is a filtered $R_{\Gamma}$ module, then

$\mathrm{gr}(M)=\bigoplus_{i=0}^{\infty}M_{i}/M_{i-1}$

is a graded $(\mathrm{gr}R)_{\Gamma}$ module where
$\mathrm{gr}R=\bigoplus_{i=0}^{\infty}R_{i}/R_{i-1}$.

Let $R$ be a filtered $\Gamma$-ring and $M=\bigcup_{k\geq 0}M^{k}$ be a
filtered $R_{\Gamma}$-module. It is trivial that $\bigcup_{k\geq}M^{k}$ is an
$R_{\Gamma}$-submodule of $M$. The following theorem answers the question: Is
$\bigcap_{k\geq}M^{k}$ an $R_{\Gamma}$-submodule?

######  Theorem 8.2.

Let $R$ be a filtered $\Gamma$-ring and $M$ be a filtered $R_{\Gamma}$-module
by $M^{k}$. Then $\bigcap M^{k}$ is an $R_{\Gamma}$-submodule of $M$.

Proof.  The intersection of any arbitrary collection of subgroups of a group
is subgroup, so $\bigcap M^{k}$ is a subgroup of $M$. For $\lambda\in R$,
$\gamma\in\Gamma$, and $x\in\bigcap M^{k}$. There exists $p\in\mathbb{N}_{0}$
such that $\lambda\in R^{p}$. As $x\in M^{k-p}$ for all $k$, we have
$\lambda\gamma x\in M^{k}$ for all $k$. That is $\lambda\gamma x\in\bigcap
M^{k}$. $\Box$

######  Remark 2.

One can focus on studying graded $\Gamma$-bimodules. Let $R$ and $S$ be graded
$\Gamma$-rings of type $G$ and let $M$ be an abelian group. Then
$M=\bigoplus_{g\in G}M_{g}$ is a graded $(R,S)_{\Gamma}$-bimodule of type $G$
if the following conditions are satisfied:

  * (i)

$M$ is an $(R,S)_{\Gamma}$-bimodule,

  * (ii)

$M$ is a graded left $R_{\Gamma}$-module,

  * (iii)

$M$ is a graded right $S_{\Gamma}$-module.

Therefore $N_{g}\Gamma M_{h}\Gamma S_{k}\subseteq M_{ghk}$ for $g,h,k\in G$.

##  9\. Strongly Graded Gamma Rings

Let $M$ be a graded $\Gamma$-ring, where $G$ is a monoid, with identity $1$.
According to Theorem 4.3, we have $1\in M_{e}$. Thus $M_{g}\Gamma M_{e}=M_{g}$
and $M_{e}\Gamma M_{g}=M_{g}$ for all $g\in G$. If these equalities hold for
any arbitrary two elements $g,h,G$, we get a strongly graded $\Gamma$-ring. In
other words we have the following definition.

######  Definition 9.1.

Let $G$ be a semigroup. A graded $\Gamma$-ring $M$ of type $G$ is called a
strongly graded $\Gamma$-ring if $M_{g}\Gamma M_{h}=M_{gh}$ for all $g,h\in
G$.

######  Example 9.1.

  * (i)

Let $G$ be a group and $M$ be a $\Gamma$-ring. The group $\Gamma$-ring $MG$ is
a strongly graded $\Gamma$-ring of type $G$.

  * (ii)

If $M$ is a strongly graded $\Gamma$-ring, then so is $M/I$ for every graded
ideal $I$ of $M$.

  * (iii)

If $M$ is a strongly graded $\Gamma$-ring of type $G$ and $\phi:G\rightarrow
H$ is an epimorphism, then $M_{(H)}$ is strongly graded.

  * (iv)

If $M$ is a strongly graded $\Gamma$-ring of type $G$, and $H$ is a
sub(semi-)group of $G$, then $M^{(H)}$ is strongly graded.

  * (v)

Let $(M_{i})_{i\in I}$ be a family of graded $\Gamma$ rings of type $G$ such
that $I$ is finite. Then $M=\prod_{i\in I}M_{i}$ is strongly graded if and
only if $M_{i}$ is strongly graded for each $i\in I$.

Let $M$ be a $\Gamma$-ring with unity $1_{\gamma_{0}}$, and let $M^{*}$
denotes the set of invertible elements in $M$ with respect to $\gamma_{0}$.
Define multiplication on $M^{*}$ as follows: $mn=m\gamma_{0}n$ for $m,n\in M$.
Then the set $M^{*}$ forms a group. For if $m,n\in M^{*}$, then
$mnn^{-1}m^{-1}=(m\gamma_{0}n)\gamma_{0}(n^{-1}\gamma_{0}n^{-1})=1_{\gamma_{0}}$.
Also, the associativity follows from one of the properties of $\Gamma$-rings.
The support of invertible homogeneous elements of $M$ is defined by

$G_{M}^{*}=\\{g\in G~{}|~{}M_{g}^{*}\neq\phi\\}$

where $M_{g}^{*}=M_{g}\cap M^{*}$. Using Theorem 4.4 it is easy to see that
$G_{M}^{*}$ is a group. Also $G_{M}^{*}\subseteq G_{M}$ where $G_{M}=\\{g\in
G~{}|~{}M_{g}\neq\\{0\\}\\}$ is the support of $M$.

######  Definition 9.2.

A graded $\Gamma$-ring $M=\bigoplus_{g\in G}M_{g}$ with unity
$1_{\gamma_{0}}$, where $G$ is a group, is called crossed product if there is
an invertible element in every homogeneous element of $M$, that is
$G_{M}^{*}=G$.

######  Theorem 9.1.

Let $M=\bigoplus_{g\in G}M_{g}$ be a graded $\Gamma$-ring of type $G$, where
$G$ is a group, with unity $1_{\gamma_{0}}$. Then

  * (i)

$M$ is strongly graded if and only if $1_{\gamma_{0}}\in M_{g}\Gamma
M_{g^{-1}}$ for all $g\in G$,

  * (ii)

if $M$ is strongly graded, then the support of $M$ is $G$,

  * (iii)

any crossed product $\Gamma$-ring is strongly graded,

  * iv)

if $N$ is a graded $\Gamma$-ring and $\varphi:M\rightarrow N$ is a degree
preserving homomorphism of $\Gamma$-rings, and $M$ is strongly graded
$\Gamma$-ring, then $N$ is strongly graded too.

Proof.

  * (i)

Suppose that $M$ is a strongly graded $\Gamma$-ring. Then $M_{e}=M_{g}\Gamma
M_{g^{-1}}$ for any $g\in G$. Now the result follows from Theorem 4.3.
Conversely, let $x\in M_{e}$ and $g\in G$. Then
$x=1_{\gamma_{0}}\gamma_{0}x\in\left(M_{g}\Gamma
M_{g^{-1}}\right)\gamma_{0}M_{e}=M_{g}\Gamma\left(M_{g^{-1}}\gamma_{0}M_{e}\right)\subseteq
M_{g}\Gamma M_{g^{-1}}$. Hence $M_{e}=M_{g}\Gamma M_{g^{-1}}$ for all $g\in
G$. Next for $g,h\in G$ we have

$\displaystyle M_{gh}$ $\displaystyle\subseteq$ $\displaystyle M_{e}\Gamma
M_{gh}$ $\displaystyle=$ $\displaystyle\left(M_{g}\Gamma
M_{g^{-1}}\right)\Gamma M_{gh}$ $\displaystyle=$ $\displaystyle
M_{g}\Gamma\left(M_{g^{-1}}\Gamma M_{gh}\right)$ $\displaystyle\subseteq$
$\displaystyle M_{g}\Gamma M_{h}.$

This proves that $M_{gh}=M_{g}\Gamma M_{h}$ for all $g,h\in G$. Therefore $M$
is strongly graded.

  * (ii)

Let $g\in G$. Then using (i), we have $1_{\gamma_{0}}\in M_{g}\Gamma
M_{g^{-1}}$. Thus $M_{g}\neq\\{0\\}$.

  * (iii)

Suppose that $M$ is a crossed product $\Gamma$-ring. Let $g\in G$. Then there
exists an invertible element (homogeneous) $m\in M_{g}$. According to Theorem
4.4 we have $m^{-1}\in M_{g^{-1}}$. So $1_{\gamma_{0}}=m\gamma_{0}m^{-1}\in
M_{g}\Gamma M_{g^{-1}}$. Now the result follows from (i).

  * (iv)

$M$ is strongly graded, so for all $g\in G$ we have $1_{\gamma_{0}}\in
M_{g}\Gamma M_{g^{-1}}$. It follows
$\varphi(1_{\gamma_{0}})=1_{\gamma_{0}}\in\varphi(M_{g})\Gamma\varphi(M_{g^{-1}})\subseteq
N_{g}\Gamma N_{g^{-1}}$ for all $g\in G$.

$\Box$

######  Remark 3.

Let $R$ be a graded $\Gamma$-ring, and $M$ be a graded $R_{\Gamma}$-module.
Then $M$ is strongly graded $R_{\Gamma}$-module if $R_{g}\Gamma M_{h}=M_{gh}$
for all $g,h\in G$. One can focus on considering such graded $\Gamma$-rings.
Also, one of the future works will focus on studying tensor product of graded
$\Gamma$-modules.

## References

  * [1] R. Ameri, R. Sadeghi, Gamma modules, Ratio Mathematica, 20 (2010), 127-147.
  * [2] D. D. Anderson, Some remarks on multiplication ideals, Math Japon 25 (1) (1980), 463-469.
  * [3] Y. Bahturin, S. Sehgal and M. Zaicev, Group gradings on associative algebras, J. Algebra 241 (2001), 677-698.
  * [4] W. E. Barnes, On the gamma rings of Nobusawa, Pacific J. Math 18 (1966), 411-422.
  * [5] M. Cohen and L. H. Rowen, Group graded rings, Communications in Algebra 11 (11) (1983), 1253-1270.
  * [6] E. Dade, Group graded rings and modules, Math. Z 174 (3) (1980), 241-262 .
  * [7] A. Estaji, A. Khorasani and S. Baghdari, Multiplication ideals in $\Gamma$-rings, Journal of Hyperstructures 2 (1) (2013), 30-39.
  * [8] Fida. M. and Maisa. K., Graded modules over first strongly graded rings, Malaysian Journal of Mathematical Sciences 11 (2) (2018), 205-220.
  * [9] F. Fusheng, G. Lingzhong, Group graded gamma rings, Northeast. Math. J. 14 (2) (1998), 177-186.
  * [10] R. Hazrat , Group rings and graded grothendieck groups, London Mathematical Society Lecture Note Series 435, Cambridge university press (2016).
  * [11] C. Nastasescu and F. van Oystaeyen, Graded and filtered rings and modules, Lecture notes in mathematics 758, Springer, Berlin (1979).
  * [12] C. Nastasescu and F. van Oystaeyen, Graded ring theory, North-Holland, Amsterdam (1982).
  * [13] N. Nobusawa, On a generalization of the ring theory, Osaka J. Math. 1 (1964), 81-89.
  * [14] S. Shaqaqha, Hilbert series for free lie superalgebras and related topics. Ph.D. Thesis, MUN (2015).
  * [15] M. S. Uddin, M. S. Islam, Gamma rings of ramma endomorphisms, Pure and Applied Mathematics 3 (1) (2013), 94-99.
  * [16] M. Dumitru, Gamma-ring: some interpretations used in the study of their radicals, U.P.B. Sci. Bull., Series A 71 (3) (2009), 9-22.
  * [17] A.C. Paul and Md. Sabur, Decomposition in neotherian gamma rings, International Archive of Applied Sciences and Technology, 2 (2) (2011), 38-42.
  * [18] S. Kyuno, On the radicals of $\Gamma$-rings, Osaka J. Math., 12 (1975), 639-645.
  * [19] S. Kyuno, On prime gamma rings, Pacific J. Math., 75 (1) (1978), 185-190.
  * [20] S. Kyuno, Prime ideals in gamma rings, Pacific J. Math., 98 (2) (1982), 375-379.