Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: add configuration management and authentication features #57

Closed
wants to merge 3 commits into from

Commits on Aug 3, 2019

  1. add configuration management and authentication features

    Resolves: epoch8#54, epoch8#55, epoch8#56
    
    More info:
    
    This commit adds the support for the configuration of the exporter via
    `prometheus_exporter` section of Airflow configuration.
    
    The keys with `expose_` prefix enable or disable the exposure of certain class
    of metrics. For example, Airflow variables-related metrics are not exposed by
    default. However, with `expose_variables` set to `True`, the exporter will start
    exposing the metrics. Similarly, by default, the exporter exposes scheduler-related
    metrics. However, with `expose_scheduler` set to `False`, the exporter will
    not export them.
    
    Additionally, when `expose_config` is being enabled, the exporter will expose
    a subset of `core` and `prometheus_exporter` configuration settings.
    
    ```
    [prometheus_exporter]
    auth_enabled = True
    auth_token = 4e1dba1c-2b66-4275-b8ae-292ee9665fa1
    expose_variables = True
    expose_config = False
    expose_scheduler = False
    ```
    
    It is possible to disable the exporter:
    
    ```
    [prometheus_exporter]
    disabled = True
    ```
    
    When authentication is enabled, the metrics are accessible via:
    
    ```bash
    curl -v -H 'Authorization: Bearer 4e1dba1c-2b66-4275-b8ae-292ee9665fa1' https://localhost:8443/admin/metrics/
    ```
    
    Also, when the authentication is enabled, Prometheus scrape job
    might look like this:
    
    ```yaml
      - job_name: 'airflow_exporters'
        metrics_path: /admin/metrics/
        scheme: https
        tls_config:
          insecure_skip_verify: true
        bearer_token: '4e1dba1c-2b66-4275-b8ae-292ee9665fa1'
        scrape_interval: 5m
        static_configs:
        - targets:
          - '127.0.0.1:8443'
          labels:
            environment: 'dev'
        relabel_configs:
        - source_labels: [__address__]
          regex: "^(.*):(.*)$"
          target_label: instance
          replacement: ${1}
        - source_labels: [instance]
          regex: "^(127.0.0.1)$"
          target_label: instance
          replacement: "airflow"
    ```
    greenpau committed Aug 3, 2019
    Configuration menu
    Copy the full SHA
    d42cd5e View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2019

  1. move code to init

    greenpau committed Aug 15, 2019
    Configuration menu
    Copy the full SHA
    39cbac1 View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2019

  1. Configuration menu
    Copy the full SHA
    5d100a6 View commit details
    Browse the repository at this point in the history