Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/cpp baby llama rework #2903

Merged
merged 8 commits into from
Jan 26, 2024
Prev Previous commit
Next Next commit
Add last successful location to error message in handle function
  • Loading branch information
mreso committed Jan 25, 2024
commit 4e0aa8675d24de1b7e85b7f9c949793d035d927c
6 changes: 5 additions & 1 deletion cpp/src/backends/handler/base_handler.cc
Original file line number Diff line number Diff line change
@@ -10,13 +10,17 @@ void BaseHandler::Handle(
std::map<uint8_t, std::string> map_idx_to_req_id;
std::pair<std::string&, std::map<uint8_t, std::string>&> idx_to_req_id(
req_ids, map_idx_to_req_id);
std::string just_passed = "";
try {
auto start_time = std::chrono::system_clock::now();
auto inputs =
Preprocess(device, idx_to_req_id, request_batch, response_batch);
just_passed = "Preprocessing";
auto outputs =
Inference(model, inputs, device, idx_to_req_id, response_batch);
just_passed = "Inference";
Postprocess(outputs, idx_to_req_id, response_batch);
just_passed = "Postprocessing";
auto stop_time = std::chrono::system_clock::now();
std::chrono::duration<double, std::milli> duration = stop_time - start_time;
try {
@@ -44,7 +48,7 @@ void BaseHandler::Handle(
TS_LOGF(ERROR, "Failed to record PredictionTime metric. {}", e.what());
}
} catch (...) {
TS_LOG(ERROR, "Failed to handle this batch");
TS_LOG(ERROR, "Failed to handle this batch after: {}", just_passed);
}
}