Reference documentation and code samples for the Vertex AI V1 API class Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec.
Configuration for Speculative Decoding.
Inherits
- Object
Extended By
- Google::Protobuf::MessageExts::ClassMethods
Includes
- Google::Protobuf::MessageExts
Methods
#draft_model_speculation
def draft_model_speculation() -> ::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::DraftModelSpeculation- (::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::DraftModelSpeculation) — draft model speculation.
Note: The following fields are mutually exclusive:
draft_model_speculation,ngram_speculation. If a field in that set is populated, all other fields in the set will automatically be cleared.
#draft_model_speculation=
def draft_model_speculation=(value) -> ::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::DraftModelSpeculation- value (::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::DraftModelSpeculation) — draft model speculation.
Note: The following fields are mutually exclusive:
draft_model_speculation,ngram_speculation. If a field in that set is populated, all other fields in the set will automatically be cleared.
- (::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::DraftModelSpeculation) — draft model speculation.
Note: The following fields are mutually exclusive:
draft_model_speculation,ngram_speculation. If a field in that set is populated, all other fields in the set will automatically be cleared.
#ngram_speculation
def ngram_speculation() -> ::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::NgramSpeculation- (::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::NgramSpeculation) — N-Gram speculation.
Note: The following fields are mutually exclusive:
ngram_speculation,draft_model_speculation. If a field in that set is populated, all other fields in the set will automatically be cleared.
#ngram_speculation=
def ngram_speculation=(value) -> ::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::NgramSpeculation- value (::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::NgramSpeculation) — N-Gram speculation.
Note: The following fields are mutually exclusive:
ngram_speculation,draft_model_speculation. If a field in that set is populated, all other fields in the set will automatically be cleared.
- (::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::NgramSpeculation) — N-Gram speculation.
Note: The following fields are mutually exclusive:
ngram_speculation,draft_model_speculation. If a field in that set is populated, all other fields in the set will automatically be cleared.
#speculative_token_count
def speculative_token_count() -> ::Integer- (::Integer) — The number of speculative tokens to generate at each step.
#speculative_token_count=
def speculative_token_count=(value) -> ::Integer- value (::Integer) — The number of speculative tokens to generate at each step.
- (::Integer) — The number of speculative tokens to generate at each step.