Skip to content

Latest commit

 

History

History

tadis-analyzer

README.md

tadis-analyzer

tadis-analyzer can discover the architecture of a microservice system by collecting information about the system from different sources. These sources include infrastructure services, e.g. Kubernetes and RabbitMQ, which provide real-time information, and Git repositories, which add information from static source code analysis.

TADIS is designed as a DEV tool that a gives a team an overview of the services and their dependencies when planning further development of the system. It allows to customize the analysis in order to visualize system-specific architecture aspects. TADIS is not intended for system performance analysis.

The analyzer is provided as a stand-alone microservice as well as a NPM package which can be integrated into a self-made microservice. Either way, it provides a generic REST controller with the following endpoints:

  • /collect/system uses a CollectorService to create a graph of a microservice system by using different transformers. A small example is given in here for Kubernetes and RabbitMQ. A larger example is given in the customized-analyzer-example. The available transformers are described in the next section.
  • /collect/source triggers a source code import for all microservices. The CollectorService provides a list of available microservices. For each microservice, an import of its Git repository is triggered by using the GitStorage service.
  • /source/store/repository/:repositoryName allows to trigger the import of a specific microservice repository.
  • /source/status gives a info object of the imported repositories.

Configuration

Environment Variables:

  • PORT: defines the PORT of the REST endpoints
  • SOURCE_FOLDER: defines the folder where source code is stored on import
  • GIT_BASE_URLS: is a comma-separated list of base URLs that are inspected for microservice repository locations.

Usage

  • install: yarn install
  • run: yarn start
  • build Docker image: yarn docker-build
  • run Docker image: yarn docker-run

Getting Started

Please see customized-analyzer-example.

Transformers

Transformers are used by a collector to assemble a system by collecting information from different sources. Each transformer gets an input system and transforms the system into an output system.

A system uses concepts of microservice systems which are defined by this model. The system model is a typed extension of a generic model.

The following transformers are available out-of-the-box:

  • Kubernetes Transformers
    • MicroservicesFromKubernetesCreator
    • EnvDefinitionFromPodDecorator
    • LabelsFromDeploymentDecorator
  • Java Transformers
    • FeignClientAnnotationAnalyzer
    • JavaAnnotationAnalyzer
  • RabbitMQ Transformers
    • RabbitMqBindingsFromApiAnalyzer
    • ExchangesFromEnvPayloadCreator
    • OutgoingExchangesFromSourceCreator
    • MicroserviceWithOutgoingExchangeMerger
  • Common Transformers
    • SourceLocationDecorator
    • StaticNodeFilter
    • SubSystemFromPayloadTransformer
  • Source Pattern Analyzer

The transformers and their configuration options are described in next sections.

Kubernetes Transformers

Environment Variables:

  • KUBERNETES_NAMESPACE: All Kubernetes transformers use this fixed namespace.

Assumptions in each transformer:

  • pod.metadata.name starts with a microservice name which is followed by minus character, e.g. microservicename-8648fc44bb-5qnm8
  • this assumption is violated if you use minus in your microservice names and if you have multiple microservices beginning with the same name

MicroservicesFromKubernetesCreator

  • gets all services and pods from the Kubernetes namespace
  • creates a microservice for each service that also has a pod
  • assumptions:
    • service.metadata.name is a microservice name

EnvDefinitionFromPodDecorator

  • gets all pods from the Kubernetes namespace
  • adds attribute env of each container in pod.spec.containers to the payload of the microservice

LabelsFromDeploymentDecorator

  • gets all deployments from the Kubernetes namespace
  • adds attribute labels from deployment.metadata.labels to the payload of the microservice

Java Transformers

FeignClientAnnotationAnalyzer

  • searches in the source code of each microservice for *.java files for the pattern /@FeignClient\s*\(\s*(value\s*=)?\s*"([\w-]+)"/
  • for each value creates a SyncDataFlow between the current microservice and the target microservice referred to in value

JavaAnnotationAnalyzer (non-MSA)

  • an attempt to generalize analyzers for Java annotations
  • the transform method is passed an annotation to search for and actions defined as mapping of annotation elements to graph extensions
  • see source code for details
transform(system: System, annotation: string, elementMappings: ElementMapping[]): Promise<System> { ... }

RabbitMQ Transformers

RabbitMqBindingsFromApiAnalyzer

  • uses the RabbitMQ Management Plugin
  • gets all queue to exchange bindings for each queue via /api/queues/<vhost>/<queueName>/bindings/ (only supports vhost /)
  • if queue name equals <microservice-name>.<text> for any existing microservice
    • then an AsyncEventFlow is created between the exchange and microservice
    • else a MessageQueue is created and used instead of the microservice

Environment Variables:

  • RABBIT_USER and RABBIT_PASSWORD for use in basic auth
  • RABBIT_URL: base URL to RabbitMQ Management API

ExchangesFromEnvPayloadCreator

  • searches in each node payload for an env attribute
  • the env attribute must be of type EnvEntry[]
type EnvEntry = {
  name: string,
  value: string
}
  • for each env.name that contains the string EXCHANGE
    • if the env.name also contains the string OUTGOING then an AsyncEventFlow is added between the current microservice and the exchange defined in env.value
    • if instead the env.name also contains the string INCOMINGthen an AsyncEventFlow is added from the exchange towards the current microservice

OutgoingExchangesFromSourceCreator

  • searches in the source code of each microservice for .java files that also have send in their filename
  • searches for variable assignments matching the pattern /exchange[\w]*[\s]*=\s*"(\w+)"/ig where the expression identifies the exchange that is used for sending messages.
  • for each sending exchange found, creates an AsyncEventFlow between the current microservice and the exchange

MicroserviceWithOutgoingExchangeMerger

  • searches for microservice nodes having just one outgoing exchange whose name equals the name of the microservice node
  • merges each such exchange into the microservice node, including payload and metadata, and adds the payload reduced=true to the node

Common Transformers (not microservice system specific)

SourceLocationDecorator

Adds payload sourceLocation to each microservice where source code was fetched via Git.

StaticNodeFilter

Filters nodes from a flat system.

Environment Variables:

  • EXCLUDED_NODE_NAMES
    • comma-separated list of names
    • regular expressions may be used
    • example: dummy.*, foo

SubSystemFromPayloadTransformer

Use this transformer when you have a flat system where some nodes have a payload defining their association to a sub-system and you want the system to be transformed to a system of sub-systems. This transformer creates new sub-system nodes and moves each node to its sub-system node.

How sub-system information is to be extracted from a nodes payload must be specified when calling the transformer.

see SubSystemTransformer.ts

Source Pattern Analyzer

This transformer allows to provide patterns for static source code analysis. An example is provided in the customized-analyzer-example project.

License

Apache License, Version 2.0

Copyright 2017-2020 Andreas Blunk, MaibornWolff GmbH