Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unify API error types #450

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions examples/crd_api.rs
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,8 @@ async fn main() -> anyhow::Result<()> {
info!("Created {} ({:?})", Meta::name(&o), o.status.unwrap());
debug!("Created CRD: {:?}", o.spec);
}
Err(kube::Error::Api(ae)) => assert_eq!(ae.code, 409), // if you skipped delete, for instance
Err(e) => return Err(e.into()), // any other case is probably bad
Err(kube::Error::Api(ae)) => assert_eq!(ae.code, Some(409)), // if you skipped delete, for instance
Err(e) => return Err(e.into()), // any other case is probably bad
}
// Wait for the api to catch up
sleep(Duration::from_secs(1)).await;
Expand Down
10 changes: 5 additions & 5 deletions examples/crd_derive_schema.rs
Original file line number Diff line number Diff line change
Expand Up @@ -203,12 +203,12 @@ async fn main() -> Result<()> {
assert!(res.is_err());
match res.err() {
Some(kube::Error::Api(err)) => {
assert_eq!(err.code, 422);
assert_eq!(err.reason, "Invalid");
assert_eq!(err.status, "Failure");
assert_eq!(err.code, Some(422));
assert_eq!(err.reason.as_deref(), Some("Invalid"));
assert_eq!(err.status.as_deref(), Some("Failure"));
assert_eq!(
err.message,
"Foo.clux.dev \"qux\" is invalid: spec.non_nullable: Required value"
err.message.as_deref(),
Some("Foo.clux.dev \"qux\" is invalid: spec.non_nullable: Required value")
);
}
_ => assert!(false),
Expand Down
2 changes: 1 addition & 1 deletion examples/job_api.rs
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ async fn main() -> anyhow::Result<()> {
}
}
WatchEvent::Deleted(s) => info!("Deleted {}", Meta::name(&s)),
WatchEvent::Error(s) => error!("{}", s),
WatchEvent::Error(s) => error!("{:?}", s),
_ => {}
}
}
Expand Down
6 changes: 3 additions & 3 deletions examples/pod_api.rs
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ async fn main() -> anyhow::Result<()> {
// wait for it..
std::thread::sleep(std::time::Duration::from_millis(5_000));
}
Err(kube::Error::Api(ae)) => assert_eq!(ae.code, 409), // if you skipped delete, for instance
Err(e) => return Err(e.into()), // any other case is probably bad
Err(kube::Error::Api(ae)) => assert_eq!(ae.code, Some(409)), // if you skipped delete, for instance
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a bit uglier, and a breaking change, but a small one.

Copy link
Member

@kazk kazk Mar 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, API error handling already felt awkward when writing controllers. Matches like Err(kube::Error::Api(kube::error::ErrorResponse { code: 404, .. })) are common.

I was going to propose switching to snafu and add new Error specific for client/API (let's say kube::client::Error for now). Providing specific variants like Error::NotFound and Error::Conflict makes sense because these are very common. We can include our wrapper for meta::v1::Status in variants for those who need them.

Also, I'm not sure if there's any reason to keep the deprecated runtime, but I think it makes sense to move kube-runtime under kube soon and this (snafu) is a necessary step.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, there's definitely bigger improvements to be made in client error handling, separating it out more like how it's done for ConfigError, and special purpose creating specific errors would help a lot like you say.

A small question though. How are you finding working with snafu? What benefits do you find with it over thiserror?

I ask because I was kind of thinking to propose unifying under thiserror instead because every time i make another app, I have started reaching for it instead because of ergonomics:

  • snafu's error enum ends up being 5 lines per impl (when using a source + backtrace) rather than thiserror's 2
  • snafu's error variants all have to be imported into scope and causes extra cycles when developing

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A small question though. How are you finding working with snafu? What benefits do you find with it over thiserror?

IMO, the big benefit is that the context selectors push you towards having many smaller and more specific error types that act as a sort of semantic backtrace ("file not found while finding Y while reconciling X"), rather than the more or less unmanageable kube::Error ("something has gone wrong in library X, have fun figuring out why").

  • snafu's error enum ends up being 5 lines per impl (when using a source + backtrace) rather than thiserror's 2

snafu is agnostic to struct vs tuple variants, you can do:

#[derive(Debug, Snafu)
pub enum Error {
    #[snafu(display("IO error"))]
    Io(#[snafu(source)] std::io::Error, #[snafu(backtrace)] Backtrace)
}

The current state in kube-derive is more about my own bias towards struct-style enums.

Backtraces can also be inherited from other snafu errors, which would save you an item.

  • snafu's error variants all have to be imported into scope and causes extra cycles when developing

Not really, if you follow snafu's recommended error-enum-per-module style. The enum is the external interface of the module, the context selectors are an implementation detail.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move to #453? It's easier for others to join, and we can leave this PR alone.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Providing specific variants like Error::NotFound and Error::Conflict makes sense because these are very common. We can include our wrapper for meta::v1::Status in variants for those who need them.

I agree that that would be more comfortable, but it's a big compatibility footgun, sadly.

Adding a Conflict arm would mean that a match on Err(kube::Error::Api(kube::error::ErrorResponse { code: 409, .. })) would still compile just fine, but silently never match anything.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we keep kube::client::Status, we can add something like:

impl Status {
    pub fn classify(&self) -> Reason;
}

#[non_exhaustive]
pub enum Reason {
    NotExists,
    Confict,
    PermissionDenied,
    Unknown
}

Otherwise, it can be implemented as a free function or an extension trait for k8s_openapi::Status

Err(e) => return Err(e.into()), // any other case is probably bad
}

// Watch it phase for a few seconds
Expand All @@ -59,7 +59,7 @@ async fn main() -> anyhow::Result<()> {
info!("Modified: {} with phase: {}", Meta::name(&o), phase);
}
WatchEvent::Deleted(o) => info!("Deleted {}", Meta::name(&o)),
WatchEvent::Error(e) => error!("Error {}", e),
WatchEvent::Error(e) => error!("Error {:?}", e),
_ => {}
}
}
Expand Down
12 changes: 5 additions & 7 deletions kube-runtime/src/watcher.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

use derivative::Derivative;
use futures::{stream::BoxStream, Stream, StreamExt};
use k8s_openapi::apimachinery::pkg::apis::meta::v1::Status;
use kube::{
api::{ListParams, Meta, WatchEvent},
Api,
Expand All @@ -23,11 +24,8 @@ pub enum Error {
source: kube::Error,
backtrace: Backtrace,
},
#[snafu(display("error returned by apiserver during watch: {}", source))]
WatchError {
source: kube::error::ErrorResponse,
backtrace: Backtrace,
},
#[snafu(display("error returned by apiserver during watch: {:?}", status))]
WatchError { status: Status, backtrace: Backtrace },
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this now the debug print?

#[snafu(display("watch stream failed: {}", source))]
WatchFailed {
source: kube::Error,
Expand Down Expand Up @@ -153,15 +151,15 @@ async fn step_trampolined<K: Meta + Clone + DeserializeOwned + Send + 'static>(
}),
Some(Ok(WatchEvent::Error(err))) => {
// HTTP GONE, means we have desynced and need to start over and re-list :(
let new_state = if err.code == 410 {
let new_state = if err.code == Some(410) {
State::Empty
} else {
State::Watching {
resource_version,
stream,
}
};
(Some(Err(err).context(WatchError)), new_state)
(Some(WatchError { status: err }.fail()), new_state)
}
Some(Err(err)) => (Some(Err(err).context(WatchFailed)), State::Watching {
resource_version,
Expand Down
8 changes: 3 additions & 5 deletions kube/src/api/object.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
use crate::{
api::metadata::{ListMeta, Meta, ObjectMeta, TypeMeta},
error::ErrorResponse,
};
use crate::api::metadata::{ListMeta, Meta, ObjectMeta, TypeMeta};
use k8s_openapi::apimachinery::pkg::apis::meta::v1::Status;
use serde::{Deserialize, Serialize};
use std::fmt::Debug;

Expand All @@ -27,7 +25,7 @@ where
/// NB: This became Beta first in Kubernetes 1.16.
Bookmark(Bookmark),
/// There was some kind of error
Error(ErrorResponse),
Error(Status),
}

impl<K> Debug for WatchEvent<K>
Expand Down
2 changes: 1 addition & 1 deletion kube/src/api/subresource.rs
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
use bytes::Bytes;
use futures::Stream;
use k8s_openapi::apimachinery::pkg::apis::meta::v1::Status;
use serde::de::DeserializeOwned;

use crate::{
api::{Api, DeleteParams, Patch, PatchParams, PostParams, Resource},
client::Status,
Error, Result,
};

Expand Down
5 changes: 3 additions & 2 deletions kube/src/api/typed.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
use either::Either;
use futures::Stream;
use k8s_openapi::apimachinery::pkg::apis::meta::v1::Status;
use serde::{de::DeserializeOwned, Serialize};
use std::iter;

use crate::{
api::{DeleteParams, ListParams, Meta, ObjectList, Patch, PatchParams, PostParams, Resource, WatchEvent},
client::{Client, Status},
client::Client,
Result,
};

Expand Down Expand Up @@ -309,7 +310,7 @@ where
/// WatchEvent::Modified(s) => println!("Modified: {}", Meta::name(&s)),
/// WatchEvent::Deleted(s) => println!("Deleted {}", Meta::name(&s)),
/// WatchEvent::Bookmark(s) => {},
/// WatchEvent::Error(s) => println!("{}", s),
/// WatchEvent::Error(s) => println!("{:?}", s),
/// }
/// }
/// Ok(())
Expand Down
89 changes: 11 additions & 78 deletions kube/src/client/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@
use crate::{
api::{Meta, WatchEvent},
config::Config,
error::ErrorResponse,
service::Service,
Error, Result,
};
Expand All @@ -22,8 +21,8 @@ use either::{Either, Left, Right};
use futures::{self, Stream, StreamExt, TryStream, TryStreamExt};
use http::{self, Request, Response, StatusCode};
use hyper::Body;
use k8s_openapi::apimachinery::pkg::apis::meta::v1 as k8s_meta_v1;
use serde::{de::DeserializeOwned, Deserialize};
use k8s_openapi::apimachinery::pkg::apis::meta::v1::{self as k8s_meta_v1, Status};
use serde::de::DeserializeOwned;
use serde_json::{self, Value};
use tokio_util::{
codec::{FramedRead, LinesCodec, LinesCodecError},
Expand Down Expand Up @@ -239,7 +238,7 @@ impl Client {
}

// Got general error response
if let Ok(e_resp) = serde_json::from_str::<ErrorResponse>(&line) {
if let Ok(e_resp) = serde_json::from_str::<Status>(&line) {
return Some(Err(Error::Api(e_resp)));
}
// Parsing error
Expand Down Expand Up @@ -328,17 +327,19 @@ fn handle_api_errors(text: &str, s: StatusCode) -> Result<()> {
if s.is_client_error() || s.is_server_error() {
// Print better debug when things do fail
// trace!("Parsing error: {}", text);
if let Ok(errdata) = serde_json::from_str::<ErrorResponse>(text) {
if let Ok(errdata) = serde_json::from_str::<Status>(text) {
debug!("Unsuccessful: {:?}", errdata);
Err(Error::Api(errdata))
} else {
warn!("Unsuccessful data error parse: {}", text);
// Propagate errors properly via reqwest
let ae = ErrorResponse {
status: s.to_string(),
code: s.as_u16(),
message: format!("{:?}", text),
reason: "Failed to parse error data".into(),
let ae = Status {
metadata: Default::default(),
status: Some(s.to_string()),
code: Some(s.as_u16().into()),
message: Some(format!("{:?}", text)),
reason: Some("Failed to parse error data".into()),
details: None,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This now causes a lot more option types that always exist (like most k8s-openapi structs), and that was the main reason for the separation of the structs.

};
debug!("Unsuccessful: {:?} (reconstruct)", ae);
Err(Error::Api(ae))
Expand All @@ -357,74 +358,6 @@ impl TryFrom<Config> for Client {
}
}

// TODO: replace with Status in k8s openapi?

/// A Kubernetes status object
#[allow(missing_docs)]
#[derive(Deserialize, Debug)]
pub struct Status {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At least we are getting rid of these duplicate structs.

// TODO: typemeta
// TODO: metadata that can be completely empty (listmeta...)
#[serde(default, skip_serializing_if = "String::is_empty")]
pub status: String,
#[serde(default, skip_serializing_if = "String::is_empty")]
pub message: String,
#[serde(default, skip_serializing_if = "String::is_empty")]
pub reason: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub details: Option<StatusDetails>,
#[serde(default, skip_serializing_if = "num::Zero::is_zero")]
pub code: u16,
}

/// Status details object on the [`Status`] object
#[derive(Deserialize, Debug)]
#[serde(rename_all = "camelCase")]
#[allow(missing_docs)]
pub struct StatusDetails {
#[serde(default, skip_serializing_if = "String::is_empty")]
pub name: String,
#[serde(default, skip_serializing_if = "String::is_empty")]
pub group: String,
#[serde(default, skip_serializing_if = "String::is_empty")]
pub kind: String,
#[serde(default, skip_serializing_if = "String::is_empty")]
pub uid: String,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub causes: Vec<StatusCause>,
#[serde(default, skip_serializing_if = "num::Zero::is_zero")]
pub retry_after_seconds: u32,
}

/// Status cause object on the [`StatusDetails`] object
#[derive(Deserialize, Debug)]
#[allow(missing_docs)]
pub struct StatusCause {
#[serde(default, skip_serializing_if = "String::is_empty")]
pub reason: String,
#[serde(default, skip_serializing_if = "String::is_empty")]
pub message: String,
#[serde(default, skip_serializing_if = "String::is_empty")]
pub field: String,
}

#[cfg(test)]
mod test {
use super::Status;

// ensure our status schema is sensible
#[test]
fn delete_deserialize_test() {
let statusresp = r#"{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"some-app","group":"clux.dev","kind":"foos","uid":"1234-some-uid"}}"#;
let s: Status = serde_json::from_str::<Status>(statusresp).unwrap();
assert_eq!(s.details.unwrap().name, "some-app");

let statusnoname = r#"{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"group":"clux.dev","kind":"foos","uid":"1234-some-uid"}}"#;
let s2: Status = serde_json::from_str::<Status>(statusnoname).unwrap();
assert_eq!(s2.details.unwrap().name, ""); // optional probably better..
}
}

#[cfg(feature = "ws")]
// Verify upgrade response according to RFC6455.
// Based on `tungstenite` and added subprotocol verification.
Expand Down
22 changes: 3 additions & 19 deletions kube/src/error.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
//! Error handling in [`kube`][crate]

use http::header::InvalidHeaderValue;
use serde::{Deserialize, Serialize};
use k8s_openapi::apimachinery::pkg::apis::meta::v1::Status;
use std::path::PathBuf;
use thiserror::Error;

Expand All @@ -14,8 +14,8 @@ pub enum Error {
/// It's also used in `WatchEvent` from watch calls.
///
/// It's quite common to get a `410 Gone` when the `resourceVersion` is too old.
#[error("ApiError: {0} ({0:?})")]
Api(#[source] ErrorResponse),
#[error("ApiError: {0:?}")]
Api(Status),

/// ConnectionError for when TcpStream fails to connect.
#[error("ConnectionError: {0}")]
Expand Down Expand Up @@ -261,19 +261,3 @@ impl From<OAuthError> for Error {
ConfigError::OAuth(e).into()
}
}

/// An error response from the API.
#[derive(Error, Deserialize, Serialize, Debug, Clone, Eq, PartialEq)]
#[error("{message}: {reason}")]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason for expanding the nice shorthand to the full debug print?

pub struct ErrorResponse {
/// The status
pub status: String,
/// A message about the error
#[serde(default)]
pub message: String,
/// The reason for the error
#[serde(default)]
pub reason: String,
/// The error code
pub code: u16,
}
2 changes: 1 addition & 1 deletion kube/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@
//! println!("Modified: {} with phase: {}", Meta::name(&o), phase);
//! }
//! WatchEvent::Deleted(o) => println!("Deleted {}", Meta::name(&o)),
//! WatchEvent::Error(e) => println!("Error {}", e),
//! WatchEvent::Error(e) => println!("Error {:?}", e),
//! _ => {}
//! }
//! }
Expand Down
2 changes: 1 addition & 1 deletion kube/src/runtime/informer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ where
}
Ok(WatchEvent::Error(e)) => {
// 410 Gone => we need to restart from latest next call
if e.code == 410 {
if e.code == Some(410) {
warn!("Stream desynced: {:?}", e);
*needs_resync.lock().await = true;
}
Expand Down
2 changes: 1 addition & 1 deletion tests/dapp.rs
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ async fn main() -> anyhow::Result<()> {
}
}
WatchEvent::Deleted(s) => info!("Deleted {}", Meta::name(&s)),
WatchEvent::Error(s) => error!("{}", s),
WatchEvent::Error(s) => error!("{:?}", s),
_ => {}
}
}
Expand Down