Skip to content

Commit

Permalink
Adds option to return stop tokens to PredictRequest.
Browse files Browse the repository at this point in the history
PiperOrigin-RevId: 694270327
  • Loading branch information
tensorflower-gardener authored and tensorflow-copybara committed Nov 7, 2024
1 parent 1352a52 commit 2ece3e4
Showing 1 changed file with 4 additions and 0 deletions.
4 changes: 4 additions & 0 deletions tensorflow_serving/apis/predict.proto
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,10 @@ message PredictRequest {
// Only supported in disaggregated serving. When set, additional arrays from
// prefill will be returned if available.
optional bool return_additional_arrays_from_prefill = 3;

// Returns these stop tokens in response if the model stops at them. The
// model may stop at other tokens, but will not return them in the response.
repeated int64 return_stoptokens = 4;
}

optional RequestOptions request_options = 7;
Expand Down

0 comments on commit 2ece3e4

Please sign in to comment.