Refactoring streams and storage

[jeqo]

This PR is attempting to simplify Kafka Streams design and removing the need for an additional Lucene store.

Changes:

• Topics has been reduced to 3:
  • spans (partitioned by traceId),
  • traces (partitioned by traceId), and
  • dependency-links (partitioned by pair parent:child)
• Kafka Streams topologies reduced to 2, 1 for aggregation, 1 for storage:
  • trace-aggregation: key topology to collect spans, aggregate, and emit traces and dependency links. Based on Session window.
  • trace-store: Stores for traces (Key value store), service names (Key value store), and dependency links (Time window based store).
• Lucene full-index replaced by a time-based index (based on key-value store) to first get ids, and then apply filtering. [TODO: test performance]. This approach keeps the same limitation for stores: Only standalone instance supported.
• Kafka Streams unit-testing added.
• Autocomplete tags support added.

[codefromthecrypt]

possible to add autocomplete tag support?

[jeqo]

possible to add autocomplete tag support?

added :)

[codefromthecrypt]

this might need a little work when you update to zipkin 2.16 because we changed the integration tests to jupiter

[jeqo]

@adriancole I've created openzipkin/zipkin#2762 and openzipkin/zipkin-reporter-java#158 to follow up some of the ideas we've discussed. Is there anything else missing from your pov?

[codefromthecrypt]

most comments of importance are around changing single-span message to list span message even when same traceID

[codefromthecrypt]

Te lo mereces un par de chelas!

[jeqo]

Solo un par? 😂🍻

…

On Fri, 23 Aug 2019 at 11:42, Adrian Cole ***@***.***> wrote: ***@***.**** approved this pull request. Te lo mereces un par de chelas! — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#25>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABPE6XMO4U6GTF3QGS4N4S3QF6WJRANCNFSM4IKTYNTQ> .

[codefromthecrypt]

un par significa ocho, ya? On Fri, Aug 23, 2019, 5:46 PM Jorge Quilcate Otoya <[email protected]> wrote:

…

Solo un par? 😂🍻 On Fri, 23 Aug 2019 at 11:42, Adrian Cole ***@***.***> wrote: > ***@***.**** approved this pull request. > > Te lo mereces un par de chelas! > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > < #25 >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/ABPE6XMO4U6GTF3QGS4N4S3QF6WJRANCNFSM4IKTYNTQ > > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#25>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAPVVZMPEX3IGATZASKMLLQF6WXHANCNFSM4IKTYNTQ> .

[jeqo]

Ahora si nos entendemos 😂😂😂

…

On Fri, 23 Aug 2019 at 12:01, Adrian Cole ***@***.***> wrote: un par significa ocho, ya? On Fri, Aug 23, 2019, 5:46 PM Jorge Quilcate Otoya < ***@***.***> wrote: > Solo un par? 😂🍻 > > On Fri, 23 Aug 2019 at 11:42, Adrian Cole ***@***.***> > wrote: > > > ***@***.**** approved this pull request. > > > > Te lo mereces un par de chelas! > > > > — > > You are receiving this because you authored the thread. > > Reply to this email directly, view it on GitHub > > < > #25 > >, > > or mute the thread > > < > https://github.com/notifications/unsubscribe-auth/ABPE6XMO4U6GTF3QGS4N4S3QF6WJRANCNFSM4IKTYNTQ > > > > . > > > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > < #25 >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/AAAPVVZMPEX3IGATZASKMLLQF6WXHANCNFSM4IKTYNTQ > > . > — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#25>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABPE6XP5KU4OWCZ3ZQTTBTLQF6YRHANCNFSM4IKTYNTQ> .