#
GPT-NeoX-Japanese

##
개요

일본어를 위한 자동회귀 언어 모델인 GPT-NeoX-Japanese를 소개합니다. 이 모델은 https://github.com/EleutherAI/gpt-neox에서 학습되었습니다. 일본어는 많은 어휘와 히라가나, 가타카나, 한자의 조합으로 이루어진 독특한 언어입니다. 이러한 일본어의 독특한 구조를 해결하기 위해 특수 서브워드 토크나이저를 사용했습니다. 이 유용한 토크나이저를 오픈소스로 제공해 준 tanreinama에게 매우 감사드립니다.

이 모델은 Google의 PaLM 연구 권장 사항을 따르며, 트랜스포머 블록에서 편향 파라미터를 제거하여 모델 성능을 향상시켰습니다. 자세한 내용은 이 기사를 참조하세요.

모델 개발은 ABEJA, Inc.의 신야 오타니, 타카요시 마카베, 안주 아로라, 쿄 하토리에 의해 주도되었습니다. 이 모델 개발 활동에 대한 자세한 내용은 여기를 참조하세요.

###
사용 예시

`generate()` 메서드를 사용하면 GPT NeoX Japanese 모델을 통해 텍스트를 생성할 수 있습니다.

    >>> from transformers import GPTNeoXJapaneseForCausalLM, GPTNeoXJapaneseTokenizer

    >>> model = GPTNeoXJapaneseForCausalLM.from_pretrained("abeja/gpt-neox-japanese-2.7b")
    >>> tokenizer = GPTNeoXJapaneseTokenizer.from_pretrained("abeja/gpt-neox-japanese-2.7b")

    >>> prompt = "人とAIが協調するためには、"

    >>> input_ids = tokenizer(prompt, return_tensors="pt").input_ids

    >>> gen_tokens = model.generate(
    ...     input_ids,
    ...     do_sample=True,
    ...     temperature=0.9,
    ...     max_length=100,
    ... )
    >>> gen_text = tokenizer.batch_decode(gen_tokens, skip_special_tokens=True)[0]

    >>> print(gen_text)
    人とAIが協調するためには、AIと人が共存し、AIを正しく理解する必要があります。

##
자료

- 일상 언어 모델링 작업 가이드

##
GPTNeoXJapanese 설정 (GPTNeoXJapaneseConfig)

###
class transformers.GPTNeoXJapaneseConfig

< source >

( vocab_size = 32000 hidden_size = 2560 num_hidden_layers = 32 num_attention_heads = 32 intermediate_multiple_size = 4 hidden_act = 'gelu' rotary_pct = 1.0 rotary_emb_base = 10000 max_position_embeddings = 2048 initializer_range = 0.02 layer_norm_eps = 1e-05 use_cache = True bos_token_id = 31996 eos_token_id = 31999 rope_scaling = None attention_dropout = 0.1 hidden_dropout = 0.0 **kwargs )

Parameters

- vocab_size (`int`, optional, defaults to 32000) — Vocabulary size of the GPTNeoXJapanese model. Defines the number of different tokens that can be represented by the `inputs_ids` passed when calling `GPTNeoXJapanese`.
- hidden_size (`int`, optional, defaults to 2560) — Dimension of the encoder layers and the pooler layer.
- num_hidden_layers (`int`, optional, defaults to 32) — Number of hidden layers in the Transformer encoder.
- num_attention_heads (`int`, optional, defaults to 32) — Number of attention heads for each attention layer in the Transformer encoder.
- intermediate_multiple_size (`int`, optional, defaults to 4) — Dimension of the “intermediate” layer in the Transformer encoder is calculated by hidden_size * intermediate_multiple_size.
- hidden_act (`str` or `function`, optional, defaults to `"gelu"`) — The non-linear activation function (function or string) in the encoder and pooler.
- rotary_pct (`float`, optional, defaults to 1.00) — percentage of hidden dimensions to allocate to rotary embeddings
- rotary_emb_base (`int`, optional, defaults to 10000) — base for computing rotary embeddings frequency
- max_position_embeddings (`int`, optional, defaults to 2048) — The maximum sequence length that this model might ever be used with.
- initializer_range (`float`, optional, defaults to 0.02) — The standard deviation of the truncated_normal_initializer for initializing all weight matrices.
- layer_norm_eps (`float`, optional, defaults to 1e-5) — The epsilon used by the layer normalization layers.
- use_cache (`bool`, optional, defaults to `True`) — Whether or not the model should return the last key/values attentions (not used by all models). Only relevant if `config.is_decoder=True`.
- rope_scaling (`Dict`, optional) — Dictionary containing the scaling configuration for the RoPE embeddings. NOTE: if you apply new rope type and you expect the model to work on longer `max_position_embeddings`, we recommend you to update this value accordingly. Expected contents: `rope_type` (`str`): The sub-variant of RoPE to use. Can be one of [‘default’, ‘linear’, ‘dynamic’, ‘yarn’, ‘longrope’, ‘llama3’], with ‘default’ being the original RoPE implementation. `factor` (`float`, optional): Used with all rope types except ‘default’. The scaling factor to apply to the RoPE embeddings. In most scaling types, a `factor` of x will enable the model to handle sequences of length x original maximum pre-trained length. `original_max_position_embeddings` (`int`, optional): Used with ‘dynamic’, ‘longrope’ and ‘llama3’. The original max position embeddings used during pretraining. `attention_factor` (`float`, optional): Used with ‘yarn’ and ‘longrope’. The scaling factor to be applied on the attention computation. If unspecified, it defaults to value recommended by the implementation, using the `factor` field to infer the suggested value. `beta_fast` (`float`, optional): Only used with ‘yarn’. Parameter to set the boundary for extrapolation (only) in the linear ramp function. If unspecified, it defaults to 32. `beta_slow` (`float`, optional): Only used with ‘yarn’. Parameter to set the boundary for interpolation (only) in the linear ramp function. If unspecified, it defaults to 1. `short_factor` (`List[float]`, optional): Only used with ‘longrope’. The scaling factor to be applied to short contexts (< `original_max_position_embeddings`). Must be a list of numbers with the same length as the hidden size divided by the number of attention heads divided by 2 `long_factor` (`List[float]`, optional): Only used with ‘longrope’. The scaling factor to be applied to long contexts (< `original_max_position_embeddings`). Must be a list of numbers with the same length as the hidden size divided by the number of attention heads divided by 2 `low_freq_factor` (`float`, optional): Only used with ‘llama3’. Scaling factor applied to low frequency components of the RoPE `high_freq_factor` (`float`, optional*): Only used with ‘llama3’. Scaling factor applied to high frequency components of the RoPE
- attention_dropout (`float`, optional, defaults to 0.1) — The dropout ratio for the attention.
- hidden_dropout (`float`, optional, defaults to 0.0) — The dropout ratio for the hidden layer.
- Example —

This is the configuration class to store the configuration of a `GPTNeoXModelJapanese`. It is used to instantiate a GPTNeoX model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the GPTNeoXJapanese abeja/gpt-neox-japanese-2.7b architecture.

Configuration objects inherit from PretrainedConfig and can be used to control the model outputs. Read the documentation from PretrainedConfig for more information. Default configs is set as 2.7B model

    >>> from transformers import GPTNeoXJapaneseConfig, GPTNeoXJapaneseModel

    >>> # Initializing a GPTNeoXJapanese gpt-neox-japanese-2.7b style configuration
    >>> configuration = GPTNeoXJapaneseConfig()

    >>> # Initializing a model (with random weights) from the gpt-neox-japanese-2.7b style configuration
    >>> model = GPTNeoXJapaneseModel(configuration)

    >>> # Accessing the model configuration
    >>> configuration = model.config

##
GPTNeoXJapanese토큰화 (GPTNeoXJapaneseTokenizer)

###
class transformers.GPTNeoXJapaneseTokenizer

< source >

( vocab_file emoji_file unk_token = '<|endoftext|>' pad_token = '<|endoftext|>' bos_token = '<|startoftext|>' eos_token = '<|endoftext|>' do_clean_text = False **kwargs )

Parameters

- vocab_file (`str`) — File containing the vocabulary.
- emoji_file (`str`) — File containing the emoji.
- unk_token (`str`, optional, defaults to `"<|endoftext|>"`) — The unknown token. A token that is not in the vocabulary cannot be converted to an ID and is set to be this token instead.
- pad_token (`str`, optional, defaults to `"<|endoftext|>"`) — The token used for padding
- bos_token (`str`, optional, defaults to `"<|startoftext|>"`) — The beginning of sequence token.
- eos_token (`str`, optional, defaults to `"<|endoftext|>"`) — The end of sequence token.
- do_clean_text (`bool`, optional, defaults to `False`) — Whether or not to clean text for URL, EMAIL, TEL, Japanese DATE and Japanese PRICE.

This tokenizer inherits from `PreTrainedTokenizer` and is based on Japanese special Sub-Word-Encoding that is used in this repository (https://github.com/tanreinama/Japanese-BPEEncoder_V2). Check the repository for details. Japanese has a relatively large vocabulary and there is no separation between words. Furthermore, the language is a combination of hiragana, katakana, and kanji, and variants such as “1” and “①” are often used. In order to cope with these, this tokenizer has the following features

- Subword-by-subword segmentation, which is intermediate between byte strings and morphological analysis.
- BPEs are created for each Kanji, Hiragana, and Katakana character, and there are no BPEs that cross character types, such as Kanji + Hiragana or Hiragana + Katakana.
- All-byte encoding that does not require <unk>.
- Independent of UTF codes such as 2-byte and 3-byte characters
- Conversion of heterographs to the same token_id
- Emoji and Emoticon are grouped into 12 types as special tags.

Example:

    >>> from transformers import GPTNeoXJapaneseTokenizer

    >>> tokenizer = GPTNeoXJapaneseTokenizer.from_pretrained("abeja/gpt-neox-japanese-2.7b")
    >>> # You can confirm both 慶応 and 慶應 are encoded to 17749
    >>> tokenizer("吾輩は猫である🐯。実は慶応(慶應)大学出身")["input_ids"]
    [30014, 26883, 26638, 27228, 25, 26650, 31732, 31679, 27809, 26638, 17749, 31592, 17749, 31593, 321, 1281]

    >>> # Both 慶応 and 慶應 are decoded to 慶応
    >>> tokenizer.decode(tokenizer("吾輩は猫である🐯。実は慶応(慶應)大学出身")["input_ids"])
    '吾輩は猫である🐯。実は慶応(慶応)大学出身'

####
convert_tokens_to_string

< source >

( tokens )

Converts a sequence of tokens (string) in a single string.

##
GPTNeoXJapaneseModel

###
class transformers.GPTNeoXJapaneseModel

< source >

( config )

Parameters

- config (~GPTNeoXJapaneseConfig) — Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the from_pretrained() method to load the model weights.

The bare GPTNeoXJapanese Model transformer outputting raw hidden-states without any specific head on top. This model is a PyTorch torch.nn.Module sub-class. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior.

####
forward

< source >

( input_ids: typing.Optional[torch.LongTensor] = None attention_mask: typing.Optional[torch.FloatTensor] = None position_ids: typing.Optional[torch.LongTensor] = None head_mask: typing.Optional[torch.FloatTensor] = None inputs_embeds: typing.Optional[torch.FloatTensor] = None past_key_values: typing.Union[transformers.cache_utils.Cache, typing.Tuple[typing.Tuple[torch.FloatTensor]], NoneType] = None use_cache: typing.Optional[bool] = None output_attentions: typing.Optional[bool] = None output_hidden_states: typing.Optional[bool] = None return_dict: typing.Optional[bool] = None cache_position: typing.Optional[torch.LongTensor] = None ) →
transformers.modeling_outputs.BaseModelOutputWithPast or `tuple(torch.FloatTensor)`

Parameters

- input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`) — Indices of input sequence tokens in the vocabulary.

  Indices can be obtained using AutoTokenizer.

- attention_mask (`torch.FloatTensor` of shape `(batch_size, sequence_length)`, optional) — Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`:
  - 1 for tokens that are not masked,
  - 0 for tokens that are masked.
- token_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, optional) — Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0, 1]`:
  - 0 corresponds to a sentence A token,
  - 1 corresponds to a sentence B token.
- position_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, optional) — Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.max_position_embeddings - 1]`.
- head_mask (`torch.FloatTensor` of shape `(num_heads,)` or `(num_layers, num_heads)`, optional) — Mask to nullify selected heads of the self-attention modules. Mask values selected in `[0, 1]`:
  - 1 indicates the head is not masked,
  - 0 indicates the head is masked.
- inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, optional) — Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert input_ids indices into associated vectors than the model’s internal embedding lookup matrix.
- past_key_values (`Cache` or `tuple(tuple(torch.FloatTensor))`, optional) — Pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used to speed up sequential decoding. This typically consists in the `past_key_values` returned by the model at a previous stage of decoding, when `use_cache=True` or `config.use_cache=True`.

  Two formats are allowed:

  - a Cache instance;
  - Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`). This is also known as the legacy cache format.

  The model will output the same cache format that is fed as input. If no `past_key_values` are passed, the legacy cache format will be returned.

  If `past_key_values` are used, the user can optionally input only the last `input_ids` (those that don’t have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `input_ids` of shape `(batch_size, sequence_length)`.

- use_cache (`bool`, optional) — If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`).
- output_attentions (`bool`, optional) — Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail.
- output_hidden_states (`bool`, optional) — Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail.
- return_dict (`bool`, optional) — Whether or not to return a ModelOutput instead of a plain tuple.
- cache_position (`torch.LongTensor` of shape `(sequence_length)`, optional) — Indices depicting the position of the input sequence tokens in the sequence. Contrarily to `position_ids`, this tensor is not affected by padding. It is used to update the cache in the correct position and to infer the complete sequence length.

Returns

transformers.modeling_outputs.BaseModelOutputWithPast or `tuple(torch.FloatTensor)`

A transformers.modeling_outputs.BaseModelOutputWithPast or a tuple of `torch.FloatTensor` (if `return_dict=False` is passed or when `config.return_dict=False`) comprising various elements depending on the configuration (GPTNeoXJapaneseConfig) and inputs.

- last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`) — Sequence of hidden-states at the output of the last layer of the model.

  If `past_key_values` is used only the last hidden-state of the sequences of shape `(batch_size, 1, hidden_size)` is output.

- past_key_values (`tuple(tuple(torch.FloatTensor))`, optional, returned when `use_cache=True` is passed or when `config.use_cache=True`) — Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) and optionally if `config.is_encoder_decoder=True` 2 additional tensors of shape `(batch_size, num_heads, encoder_sequence_length, embed_size_per_head)`.

  Contains pre-computed hidden-states (key and values in the self-attention blocks and optionally if `config.is_encoder_decoder=True` in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding.

- hidden_states (`tuple(torch.FloatTensor)`, optional, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`) — Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.

  Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.

- attentions (`tuple(torch.FloatTensor)`, optional, returned when `output_attentions=True` is passed or when `config.output_attentions=True`) — Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`.

  Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads.

The GPTNeoXJapaneseModel forward method, overrides the `__call__` special method.

Although the recipe for forward pass needs to be defined within this function, one should call the `Module` instance afterwards instead of this since the former takes care of running the pre and post processing steps while the latter silently ignores them.

Example:

    >>> from transformers import AutoTokenizer, GPTNeoXJapaneseModel
    >>> import torch

    >>> tokenizer = AutoTokenizer.from_pretrained("abeja/gpt-neox-japanese-2.7b")
    >>> model = GPTNeoXJapaneseModel.from_pretrained("abeja/gpt-neox-japanese-2.7b")

    >>> inputs = tokenizer("日本語のGPT-neoxがHugging Faceで使えます😀", return_tensors="pt")
    >>> outputs = model(**inputs)

    >>> last_hidden_states = outputs.last_hidden_state

##
일상 LLM 을 위한 GPTNeoXJapanese(GPTNeoXJapaneseForCausalLM)

###
class transformers.GPTNeoXJapaneseForCausalLM

< source >

( config )

Parameters

- config (~GPTNeoXJapaneseConfig) — Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the from_pretrained() method to load the model weights.

GPTNeoXJapanese Model with a `language modeling` head on top for Classifier Model fine-tuning. This model is a PyTorch torch.nn.Module sub-class. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior.

####
forward

< source >

( input_ids: typing.Optional[torch.LongTensor] = None attention_mask: typing.Optional[torch.FloatTensor] = None position_ids: typing.Optional[torch.LongTensor] = None inputs_embeds: typing.Optional[torch.FloatTensor] = None head_mask: typing.Optional[torch.FloatTensor] = None past_key_values: typing.Union[transformers.cache_utils.Cache, typing.Tuple[typing.Tuple[torch.FloatTensor]], NoneType] = None labels: typing.Optional[torch.LongTensor] = None use_cache: typing.Optional[bool] = None output_attentions: typing.Optional[bool] = None output_hidden_states: typing.Optional[bool] = None return_dict: typing.Optional[bool] = None cache_position: typing.Optional[torch.LongTensor] = None ) →
transformers.modeling_outputs.CausalLMOutputWithPast or `tuple(torch.FloatTensor)`

Parameters

- input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`) — Indices of input sequence tokens in the vocabulary.

  Indices can be obtained using AutoTokenizer.

- attention_mask (`torch.FloatTensor` of shape `(batch_size, sequence_length)`, optional) — Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`:
  - 1 for tokens that are not masked,
  - 0 for tokens that are masked.
- token_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, optional) — Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0, 1]`:
  - 0 corresponds to a sentence A token,
  - 1 corresponds to a sentence B token.
- position_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, optional) — Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.max_position_embeddings - 1]`.
- head_mask (`torch.FloatTensor` of shape `(num_heads,)` or `(num_layers, num_heads)`, optional) — Mask to nullify selected heads of the self-attention modules. Mask values selected in `[0, 1]`:
  - 1 indicates the head is not masked,
  - 0 indicates the head is masked.
- inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, optional) — Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert input_ids indices into associated vectors than the model’s internal embedding lookup matrix.
- past_key_values (`Cache` or `tuple(tuple(torch.FloatTensor))`, optional) — Pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used to speed up sequential decoding. This typically consists in the `past_key_values` returned by the model at a previous stage of decoding, when `use_cache=True` or `config.use_cache=True`.

  Two formats are allowed:

  - a Cache instance;
  - Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`). This is also known as the legacy cache format.

  The model will output the same cache format that is fed as input. If no `past_key_values` are passed, the legacy cache format will be returned.

  If `past_key_values` are used, the user can optionally input only the last `input_ids` (those that don’t have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `input_ids` of shape `(batch_size, sequence_length)`.

- use_cache (`bool`, optional) — If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`).
- output_attentions (`bool`, optional) — Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail.
- output_hidden_states (`bool`, optional) — Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail.
- return_dict (`bool`, optional) — Whether or not to return a ModelOutput instead of a plain tuple.
- cache_position (`torch.LongTensor` of shape `(sequence_length)`, optional) — Indices depicting the position of the input sequence tokens in the sequence. Contrarily to `position_ids`, this tensor is not affected by padding. It is used to update the cache in the correct position and to infer the complete sequence length.
- labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, optional) — Labels for computing the left-to-right language modeling loss (next word prediction). Indices should be in `[-100, 0, ..., config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels n `[0, ..., config.vocab_size]`.

Returns

transformers.modeling_outputs.CausalLMOutputWithPast or `tuple(torch.FloatTensor)`

A transformers.modeling_outputs.CausalLMOutputWithPast or a tuple of `torch.FloatTensor` (if `return_dict=False` is passed or when `config.return_dict=False`) comprising various elements depending on the configuration (GPTNeoXJapaneseConfig) and inputs.

- loss (`torch.FloatTensor` of shape `(1,)`, optional, returned when `labels` is provided) — Language modeling loss (for next-token prediction).

- logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`) — Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).

- past_key_values (`tuple(tuple(torch.FloatTensor))`, optional, returned when `use_cache=True` is passed or when `config.use_cache=True`) — Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`)

  Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding.

- hidden_states (`tuple(torch.FloatTensor)`, optional, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`) — Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.

  Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.

- attentions (`tuple(torch.FloatTensor)`, optional, returned when `output_attentions=True` is passed or when `config.output_attentions=True`) — Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`.

  Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads.

The GPTNeoXJapaneseForCausalLM forward method, overrides the `__call__` special method.

Although the recipe for forward pass needs to be defined within this function, one should call the `Module` instance afterwards instead of this since the former takes care of running the pre and post processing steps while the latter silently ignores them.

Example:

    >>> from transformers import AutoTokenizer, GPTNeoXJapaneseForCausalLM, GPTNeoXJapaneseConfig
    >>> import torch

    >>> tokenizer = AutoTokenizer.from_pretrained("abeja/gpt-neox-japanese-2.7b")
    >>> config = GPTNeoXJapaneseConfig.from_pretrained("abeja/gpt-neox-japanese-2.7b")
    >>> config.is_decoder = True
    >>> model = GPTNeoXJapaneseForCausalLM.from_pretrained("abeja/gpt-neox-japanese-2.7b", config=config)

    >>> inputs = tokenizer("日本語のGPT-neoxがHugging Faceで使えます😀", return_tensors="pt")
    >>> outputs = model(**inputs)

    >>> prediction_logits = outputs.logits

< > Update on GitHub