Client-side debug build crash aggregator and uploader

[JakeWharton]

In every app I've ever written, I've had the debug build on my phone as well as the release build. I tend to use the debug build day-to-day and perhaps some others do to be on the bleeding edge. I usually also have an out-of-band update mechanism to grab the latest debug build from CI.

This is fun as a developer to use the bleeding edge of your product. The downside, however, is crashes caused by the inherent unstability of these builds. It's expected!

Most developers disable crash reporting in debug builds. This prevents cluttering the website with problems that are programming mistakes which never will see the light of day. This significantly reduces noise.

The problem is that a lot of times legitimate crashes occur on these builds and are lost. If I'm actively developing chances are I'll say "I'll get back to that" or "I'll report a bug for that" and never do. If i'm away from a computer, the stacktrace and information is long gone by the time that I am.

In most of the apps I've built, we distinguish CI-built debug builds vs. local-built debug builds. This lets us turn on crash reporting for people who are just driving the latest build vs. those who are actively working on it. This works great, but it only solves half the problem and legitimate crashes during development are oft forgot.

So I'd like to propose a new module be added to this project which provides a debug-only activity and infrastructure that captures these crashes and aggregates them locally. When the crash occurs, a notification is shown with a summary. It has notification actions to "report" or "ignore". Clicking on it opens the debug-only activity which provides all of the information that would have otherwise been uploaded to Bugsnag with the familar tab interface. Here we can also choose to report or ignore this notification or others we've encountered in the past.

If you've every used or seen LeakCanary, I'm envisioning it works something like that. When enabled, as an alternative form of the notifier, it captures and aggregates locally and provides options to upload or ignore.

This certainly isn't something that's specific to Android and could be applied across nearly all of your products. But I'm an Android developer, and so maybe we can be the platform which beta tests such functionality.

[fractalwrench]

Thanks for the detailed proposal @JakeWharton. I think this would make a lot of sense, and has been added to our roadmap. A few things we would need to think about are:

Error Grouping

Currently individual reports are grouped on the backend. If we grouped errors locally, this could lead to discrepancies in how they are grouped on the Dashboard. Showing each individual event separately would be the simplest solution if this functionality were to support multiple platforms.

Error Actions

Upload and ignore are useful actions - I would also propose Discard, which deletes an error from local storage entirely, and Export, which shares a crash report via an Intent.

[ColtonIdle]

@JakeWharton I might be trivializing your ask/needs here (I think the option to choose to send or not makes sense as a feature), but want to play devils advocate here.

"This prevents cluttering the website with problems that are programming mistakes which never will see the light of day"

Wouldn't something like adding a new debug "stage" to your bugsnag configuration get you past that problem? We currently have production and internal stages so that I can see all of our real production crashes (and vice versa) without being cluttered with our internal build crashes.

If that's not enough separation, our iOS team actually just created a new BugSnag project called iOS internal (you could name yours Android Debug), which would give you a new api key that you can use on debug only builds and really have things sandboxed.

I'm sure you thought of those two solutions already, but kind of curious why they weren't good enough or not solving the problem you're after. Interested to hear back.

[JakeWharton]

I do/did separate production and internal already. You certainly can use a third stage for local builds by a developer but there's a few things at play:

1. the app needs to wake back up, report the crash, the crash has to be ingested, the email enqueued and sent, and gmail has to notify me. on the best case scenario side this takes about 20 seconds but certainly can scale upwards of minutes.
2. nearly 100% of the features of server-side bugsnag are useless. i don't need grouping, i don't need breadcrumbs, i don't need app data, i don't need device data, i don't need user data, etc.
3. crashes that occur on my machine and crashes that occur on other developer machines are all put into the same bucket and they all notify me. i don't care when my coworker crashes his development build, but i care about mine. i don't want to go digging through the web UI when I crash I want to be able to quickly click and see the info
4. if the crash is a real crash, i don't have the ability to promote it from the development stage to the internal stage with the rest of the pre-release crashes.

It is for that reason that I want something which can preempt crash reporting with a local UI for local, debug builds.