CKAN install statistics and bug reporting tool

[dewiniaid]

I started mentioning this in #1679 but it probably deserves its own issue with further elaboration.

It would be useful if CKAN provided some opt-in statistics collection, particularly in conjunction with the idea of staging/unstable repositories that has been tossed around. Additionally, this data collection can be used to help facilitate better bug reports and ideas of mod compatibility for KSP versions.

There's a few layers of this: General statistics, installation reports, and bug reports.

Note some of this (mainly the statistics collection and install reporting) requires creation of a web service somewhere -- but the clientside implementation still deserves merit for the bug reporting capabilities alone.

General Statistics
General statistics being collected can include:

• CKAN version
• KSP version
• System architecture
• Possibly some limited hardware information
• List of all installed mods and mod versions that CKAN is aware of
• Possibly: List of the contents of Gamedata/ that are not from any installed mod (based on CKAN manifests). It should be possible to opt out of this separately from everything else, and some folders should probably be wildcard-ignored (like .git)

Installation Reports
Users should have a way to do per-install per-KSP-version option to look at their mod list and indicate "Works", "Mostly Works" or "Broken" for each mod. Users selecting anything other than "Works" should receive information on how to check if the issue is already reported and a means to report the issues there (i.e. a link to the mod's website).

The general data of these statistics can then be aggregated in a number of ways:

• New metadata version, 100% of users report Broken --> probably bad metadata.
• New KSP version --> reports can be used to get a general idea of what mods work/don't work with the new version.
• ~10% of users report a mod as Broken --> Mod author can look at common traits (are they all on Linux? Do they all also have a certain other mod installed?) and use that to narrow down the cause.

The prompting for these statistics needs to happen in such a way where users aren't biased towards only doing it for broken mods (i.e. the "review website" problem). Possibly there can be a banner of some sort on the top of the modlist prompting users for install reports and perhaps select mods that are particularly needing of an install report (e.g. the first installers of a new version, or mods that have a low number of install reports for the current KSP+Mod version combination). Ideally it should be timed in such a way where the user has probably launched KSP once since installing the mod. (Maybe by comparing timestamps on KSP's log files)

Bug reports
CKAN can help mod authors by having a standardized way of compiling information for bug reports for a given mod. This can collect

• All of the general statistics above
• The specific metadata for the mod in question (or a URL linking to it?)
• A fast way to create a .zip of KSP.log, output_log.txt, and any log files specifically named in the module's manifest. (Users should be informed of all log files being included). A bonus would be a built-in way to upload it somewhere and get a link directly from CKAN; failing that a list of ideal sites for filesharing could be linked.

With the exception of the log .zip, this should be output in a way suitable for copy/paste into the forums (i.e. with appropriate forum code if our forums support that), a Github Issue (i.e. with Markdown formatting), or just plain text.

The UI for producing bug reports should, as a bonus, link to a guide on writing good bug reports (i.e. http://www.chiark.greenend.org.uk/~sgtatham/bugs.html written by the author of PuTTY)

[TeddyDD]

I did not heard about this guide before. It's pretty good!
Edit:
Now I think that bug report helper should not be exclusively part of CKAN. We could write standalone tool and add it as plugin to CKAN. This way people who don't use CKAN could benefit from this tool too.

[dewiniaid]

I agree on the merits of it being a standalone tool, but CKAN makes it possible to get installed mod versions since it owns it. A non-CKAN tool would have trouble even knowing what mods are installed, which is a huge piece of the data for the reporting tool.

[HebaruSan]

Replaced by #3242.