com.microsoft - BeamSearch#

BeamSearch - 1#

Version

  • name: BeamSearch (GitHub)

  • domain: com.microsoft

  • since_version: 1

  • function:

  • support_level: SupportType.COMMON

  • shape inference: True

This version of the operator has been available since version 1 of domain com.microsoft.

Summary

Attributes

  • decoder - GRAPH (required) : Decoder subgraph to execute in a loop.

  • decoder_start_token_id - INT : The id of the token that indicates decoding starts.

  • early_stopping - INT : early stop or not

  • encoder - GRAPH : The subgraph for initialization of encoder and decoder. It will be called once before decoder subgraph.

  • eos_token_id - INT (required) : The id of the end-of-sequence token

  • init_decoder - GRAPH : The subgraph for the first decoding run. It will be called once before decoder subgraph. This is relevant only for the GPT2 model. If this attribute is missing, the decoder subgraph will be used for all decoding runs

  • model_type - INT : model type: 0 for GPT-2; 1 for encoder decoder like T5

  • no_repeat_ngram_size - INT : no repeat ngrams size

  • pad_token_id - INT (required) : The id of the padding token

  • vocab_size - INT : Size of the vocabulary. If not provided, it will be inferred from the decoder subgraph’s output shape

Inputs

Between 5 and 10 inputs.

  • input_ids (heterogeneous) - I:

  • max_length (heterogeneous) - I:

  • min_length (optional, heterogeneous) - I:

  • num_beams (heterogeneous) - I:

  • num_return_sequences (heterogeneous) - I:

  • length_penalty (optional, heterogeneous) - T:

  • repetition_penalty (optional, heterogeneous) - T:

  • vocab_mask (optional, heterogeneous) - M:

  • prefix_vocab_mask (optional, heterogeneous) - M:

  • attention_mask (optional, heterogeneous) - I:

Outputs

Between 1 and 3 outputs.

  • sequences (heterogeneous) - I:

  • sequences_scores (optional, heterogeneous) - T:

  • scores (optional, heterogeneous) - T:

Type Constraints

  • T in ( tensor(float) ): Constrain input and output types to float tensors.

  • I in ( tensor(int32) ): Constrain to integer types

  • M in ( tensor(int32) ): Constrain mask to integer types

Examples