Skip to content

Latest commit

 

History

History
322 lines (225 loc) · 19.6 KB

README.adoc

File metadata and controls

322 lines (225 loc) · 19.6 KB

Travesty

Build Status Latest 2.12 version

What is it?

Travesty is a utility library for Akka Streams. It has two uses:

  • generating structural diagrams of your Akka Streams, both as graphics and text (the latter useful for logging). Example:

example graph
  • generating Tinkerpop 3 / Gremlin graph data structures - usable for e.g. writing tests that check stage sequencing in dynamically constructed Streams.

What’s new?

0.9.1

  • Fixed issue with graph not generating when some of the stages have no names (thanks to @FXHibon for the fix).

0.9

  • Added support for partial graphs (see below for example),

  • Added support for Akka 2.5.10-11.

0.8

  • Added "manual support" for typed diagrams/graphs (see Typed diagrams),

  • made Graphviz engine selection configurable for better speed/compatibility (#3),

  • fixed #4,

  • extended Akka version support up to 2.5.9 .

How to use it

Include in your project

Add to your build.sbt

//repo of dependency used for text rendering
resolvers += "indvd00m-github-repo" at "https://raw.githubusercontent.com/indvd00m/maven-repo/master/repository"

"net.mikolak" %% "travesty" % "0.9_AKKAVERSION"
//for example "net.mikolak" %% "travesty" % "0.9_2.5.11"

Where AKKAVERSION is the version of Akka (Streams) you’re using in your project. Currently supported are versions 2.5.[4-11], and Scala 2.12.

You can review all available versions on Maven Central.

Diagram generation

import net.mikolak.travesty
import net.mikolak.travesty.OutputFormat

val graph = ??? //your graph here

//render as image to file, PNG is supported as well
travesty.toFile(graph, OutputFormat.SVG)("/tmp/stream.svg")

//render as text "image"
log.info(travesty.toString(graph)) //take care NOT to wrap the text

Example text output

Here’s how a typical travesty.toString(graph) may look like:

                 *********                                                                                                                                  *******
              ****       ****                                                                                                                             **       **
             *               *
                  *INLET*                                                                                                                               *   seqSink   *
             *               *
              ****       ****                                                                                                                  ●●●        **       **
                 *********         ●●●                                                                                                        ●●  ●●        *******
                                      ●●●●●                                                                                                 ●●   ●●
                                           ●●●●●●  ●                                                                                   ●●●●●    ●
                                                 ●●●●●           *********                                  *********             ●●●●●
                                                   ●●●●       ****       ****                             ***       ***        ●●●
                                                             *               *               ●●          *             *
                                                                 ZipWith2            ●●●●●●●●● ●●       *   broadcast   *
                                                             *               *                ●●         *             *
                                                              ****       ****                ●            ***       ***
                                                   ●●●●          *********                                  *********          ●●
                                            ●●●●●●●●●●                                                                           ●●●●       ●
                                        ●●●●       ●                                                                                 ●●●●●   ●●
               *************          ●●                                                                                                  ●●●●●●         *************
            *******************                                                                                                            ●●          *******   *******
           **                 **                                                                                                                      *                 *
               singleSource                                                                                                                          *     *OUTLET*      *
           **                 **                                                                                                                      *                 *
            *******************                                                                                                                        *******   *******
               *************                                                                                                                             *************






                                                                          Flow[String,String,akka.NotUsed]

And here’s the same example in a vertical orientation, i.e. travesty.toString(graph, direction = BottomToTop):

               *************                               *********
            *******************                         ****       ****
           **                 **                       *               *
               singleSource                                 *INLET*
           **                 **                       *               *
            *******************                         ****       ****
               *************                               *********


                          ●                              ●
                           ●                            ●
                           ●                            ●
                            ●                          ●
                             ●                        ●
                              ●                      ●
                              ●                      ●
                               ●                    ●
                                ●                  ●
                                 ● ●            ● ●
                                  ●●           ● ●
                                 ● ●           ● ●●
                                  ●●●         ●●●
                                    ●         ●

                                     *********
                                  ****       ****
                                 *               *
                                     ZipWith2
                                 *               *
                                  ****       ****
                                     *********


                                         ●
                                         ●
                                         ●
                                         ●
                                         ●
                                         ●
                                         ●
                                         ●
                                         ●
                                        ●● ●
                                        ● ●
                                         ●●
                                         ●


                                     *********
                                   ***       ***
                                  *             *
                                 *   broadcast   *
                                  *             *
                                   ***       ***
                                     *********


                                     ●        ●
                                    ●          ●
                                   ●           ●
                                  ●             ●
                                 ●               ●
                                ●                 ●
                               ●                   ●
                              ●                     ●
                              ●                      ●
                            ●●                        ●●
                            ● ●                      ● ●
                           ●●●                        ●●●
                           ●                            ●


                *************                             *******
              *******   *******                         **       **
             *    *OUTLET*     *                          seqSink
            *                   *                     *             *
             *                 *
              *******   *******                         **       **
                *************                             *******





                        Flow[String,String,akka.NotUsed]

Partial graphs

Travesty now supports Akka Stream graphs with any shape.

For example, this:

Flow[String].map(_ + "a").to(Sink.ignore)

will render as:

example partial graph

The labels for open inlets and outlets are configurable via the partial-names section of the config:

travesty.partial-names {
        inlet  = "*INLET*"
        outlet = "*OUTLET*"
}

Typed diagrams

Currently, it works like this:

import net.mikolak.travesty
import net.mikolak.travesty.OutputFormat
import registry._ //adds special .↓ and .register methods to stages

val graph = Source.single("1")..via(Flow[String].map(_.toInt).).to(Sink.seq)

//render as image to file, PNG is supported as well
travesty.toFile(graph, OutputFormat.SVG)("/tmp/stream.svg")
example typed graph

register, aliased to , is a special pass-through extension method that allows Travesty to recognize the types going through your stream. Append .register/.↓ to every stage you need type labels for.

Automatic support is coming, but unfortunately is a non-trivial problem to solve. For more details, see issue #1.

Graph testing

import net.mikolak.travesty
import gremlin.scala._ //traversal operations

val graph = ??? //your graph here

val tested = travesty.toAbstractGraph(graph)

//checks whether the only path through the stream has length two
tested.E().simplePath().toList() must have size 2

For more examples, see e.g. TravestyToGraphSpec.

For general examples of what you can do with Gremlin in Scala, see the appropriately named gremlin-scala project.

Advanced usage

For further tweaking the rendering, you can use LowLevelApi:

val vizGraph = LowLevelApi.toVizGraph(travesty.toAbstractGraph(graph))

//use the instance to change splines, node shapes, etc. etc.

//and finally, use the Java API to render
vizGraph.render(Format.PNG)

How does it work?

Generally, creating a graph of an Akka Stream is hard. This is because it’s difficult to "get to" the internals of a Stream and infer its structure. There definitely is no easy solution.

Travesty "cheats" by using the internal Traversal API. The Traversal is a stack-like structure containing instructions on how to construct a running Stream.

This stack is parsed and converted into a Gremlin graph, convenient for annotating, pre-processing (e.g. additional decoration of Sources and Sinks), and testing.

The Gremlin graph is converted into a Graphviz graph, using graphviz-java.

Finally, the Graphviz graph is rendered into the required output format.

Caveats

No materialization annotations (for now)

Completely doable, but not present in the current version. Track #2 to be notified when this gets added.

Represents the graph "blueprint", not running stream

The graph/diagram generated from the Traversal object does not correspond 1:1 to what will be present in the running Stream. There are at least two reasons for this:

  • the default materializer uses fusing to join stages that can be processed synchronously;

  • there can be other optimizations used by the materializer, such as ignoring stages, adding new stages, etc. Currently, the most prominent are the "virtual" Sink stages that can appear in some scenarios.

Slooooow

graphviz-java provides several implementations of Graphviz to use. However, the one selected as default by travesty, for maximum portability, is also the slowest one. While generating the graph is always fast, rendering the diagram may take up to ~10 seconds.

If you would like to try switching to a faster engine, see reference.conf for more info.

Tight coupling with Akka Stream’s internals

As mentioned before, travesty uses the internal API for graph/diagram generation. This is why the version number follows Akka’s versioning scheme.