Testing in a Distributed World

Accompanying repository for the "Testing in a distributed world" talk given at RICON 2014. Feel free to open any issues for questions and/or to say hi :)

Talk Outline

See the image credits and link to slides. The video of the talk is available!

The talk is broken down in 4 main sessions:

• Testing Distributed Systems
• Challenges of testing
• Distributed System Testing in Academia
• Formal Methods (Human Assisted Proofs, Model Checking, Lightweight Formal Methods)
• Scholarly testing (3 highlighted papers)
• Distributed System Testing in Industry
• Highlighted papers
• Conclusions

References

Spin

• Spin website
• Spin Wikipedia
• http://www.cs.toronto.edu/~chechik/courses01/csc2108/lectures/spin.2up.pdf
• SPIN Model checking

Alloy

• http://people.csail.mit.edu/aleks/website/arby/
• Allow Download
• Software abstractions book
• Embedding Alloy in Ruby

TLA+

• Use of Formal Methods at Amazon Web Services
• Raft and TLA+
• Raft TLA+ spec
• TLA tools

Coq

• http://en.wikipedia.org/wiki/Coq
• http://coq.inria.fr/what-is-coq
• https://video.ias.edu/univalent/appel
• Introduction to Coq

MoDIST - MOdel checker for DISTributed systems

• MoDIST Paper
• MS Research page
• Microsoft's MoDist Tool Designed to Hunt for Bugs
• Modist talk

More Papers

• Paxos Made Live - An Engineering Perspective
• QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs
• Simple testing can prevent most critical failures
• A Formal Approach to Distributed System Tests Design
• Specification based testing
• Pip: Detecting the Unexpected in Distributed Systems
• Model-Checking Verification for Reliable Web Service
• FATE and DESTINI: A Framework for Cloud Recovery Testing
• WiDS Checker: Combating Bugs in Distributed Systems
• http://www.ida.liu.se/conferences/p2p/p2p2004/papers/hughes.pdf

Industry

• Chaos engineer
• "Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson
• https://foundationdb.com/key-value-store/white-papers/testing
• http://blog.foundationdb.com/correctness-testing-comparing-foundationdb-netflix-chaos-monkey
• http://blog.foundationdb.com/correctness-testing-ii-creating-ridiculously-paranoid-failures-at-foundationdb
• http://techblog.netflix.com/2014/10/a-state-of-xen-chaos-monkey-cassandra.html?m=1
• Netflix Canary Testing
• http://blog.jessitron.com/2014/08/property-based-testing-for-better-code.html

Tools

• Aspirator: A simple static checker
• Aspirator github
• Monitor Oriented Programming

Fault Injection

• Fastly Avalanche
• Simian Army
• Chaos monkey and Cassandra

Video Resources

• Leslie Lamport's Thinking for Programmers
• https://www.youtube.com/watch?v=lKXe3HUG2l4
• https://www.youtube.com/watch?v=Q663XyxmGUk