Merge pull request #137 from snyk/feat/healthcheck-with-internal-api

feat: broker client systemcheck endpoint

README.md

@@ -187,7 +187,7 @@ To use the the broker client with a Jira deployment, run `docker pull snyk/broke
 - `BROKER_TOKEN` - the snyk broker token, obtained from your Jira integration settings view.
 - `JIRA_USERNAME` - the Jira username.
 - `JIRA_PASSWORD` - the Jira password.
-- `JIRA_BASE_URL` - the URL of your Jira deployment, such as `https://your.jira.domain.com`.
+- `JIRA_HOSTNAME` - the hostname of your Jira deployment, such as `your.jira.domain.com`.
 - `BROKER_CLIENT_URL` - the full URL of the broker client as it will be accessible by your Jira for webhooks, such as `http://my.broker.client:7341`
 - `PORT` - the local port at which the broker client accepts connections. Default is 7341.
 
@@ -201,7 +201,7 @@ docker run --restart=always \
            -e BROKER_TOKEN=secret-broker-token \
            -e JIRA_USERNAME=username \
            -e JIRA_PASSWORD=password \
-           -e JIRA_BASE_URL=https://your.jira.domain.com \
+           -e JIRA_HOSTNAME=your.jira.domain.com \
            -e BROKER_CLIENT_URL=http://my.broker.client:8000 \
            -e PORT=8000 \
        snyk/broker:jira
@@ -217,10 +217,43 @@ FROM snyk/broker:jira
 ENV BROKER_TOKEN        secret-broker-token
 ENV JIRA_USERNAME       username
 ENV JIRA_PASSWORD       password
-ENV JIRA_BASE_URL       https://your.jira.domain.com
+ENV JIRA_HOSTNAME       your.jira.domain.com
 ENV PORT                8000
 ```
 
+### Monitoring
+
+#### Healthcheck
+
+The broker exposes an endpoint at `/healthcheck`, which can be used to monitor the health of the running application. This endpoint responds with status code `200 OK` when the internal request is successful, and returns `{ ok: true }` in the response body.
+
+In the case of the broker client, this endpoint also reports on the status of the broker websocket connection.  If the websocket connection is not open, this endpoint responds with status code `500 Internal Server Error` and `{ ok: false }` in the response body.
+
+To change the location of the healthcheck endpoint, you can specify an alternative path via an environment variable:
+
+```
+ENV BROKER_HEALTHCHECK_PATH /path/to/healthcheck
+```
+
+#### Systemcheck
+
+The broker client exposes an endpoint at `/systemcheck`, which can be used to validate the brokered service (SCM or the like) connectivity and credentials. This endpoint causes the broker client to make a request to a preconfigured URL, and report on the success of the request. The supported configuration is:
+
+* `BROKER_CLIENT_VALIDATION_URL` - the URL to which the request will be made.
+* `BROKER_CLIENT_VALIDATION_AUTHORIZATION_HEADER` - [optional] the `Authorization` header value of the request. Mutually exclusive with `BROKER_CLIENT_VALIDATION_BASIC_AUTH`.
+* `BROKER_CLIENT_VALIDATION_BASIC_AUTH` - [optional] the basic auth credentials (`username:password`) to be base64 encoded and placed in the `Authorization` header value of the request. Mutually exclusive with `BROKER_CLIENT_VALIDATION_AUTHORIZATION_HEADER`.
+* `BROKER_CLIENT_VALIDATION_METHOD` - [optional] the HTTP method of the request (default is `GET`).
+* `BROKER_CLIENT_VALIDATION_TIMEOUT_MS` - [optional] the request timeout in milliseconds (default is 5000 ms).
+
+This endpoint responds with status code `200 OK` when the internal request is successful, and returns `{ ok: true }` in the response body. If the internal request fails, this endpoint responds with status code `500 Internal Server Error` and `{ ok: false }` in the response body.
+
+To change the location of the systemcheck endpoint, you can specify an alternative path via an environment variable:
+
+```
+ENV BROKER_SYSTEMCHECK_PATH /path/to/systemcheck
+```
+
+
 ### Advanced Configuration
 
 #### HTTPS
@@ -244,18 +277,6 @@ docker run --restart=always \
 
 Note that `BROKER_CLIENT_URL` now has the HTTPS scheme.
 
-
-#### Monitoring
-
-The broker exposes an endpoint at `/healthcheck`, which can be used to monitor the health of the running application. This endpoint returns `200 OK` status code when the application is healthy, and will return a JSON object containing `ok: true`.
-
-To change the location of this endpoint, you can specify an alternative path via an environment variable:
-
-```
-ENV BROKER_HEALTHCHECK_PATH /path/to/healthcheck
-```
-
-
 #### SCM with an internal certificate
 
 By default, the broker client establishes HTTPS connections to the SCM. If your SCM is serving an internal certificate (signed by your own CA), you can provide the CA certificate to the broker client.

client-templates/bitbucket-server/.env.sample

@@ -20,6 +20,12 @@ BITBUCKET_API=$BITBUCKET/rest/api/1.0
 # the url of your broker client (including scheme and port)
 # BROKER_CLIENT_URL=
 
+# Bitbucket server validation url, checked by broker client systemcheck endpoint
+BROKER_CLIENT_VALIDATION_URL=https://$BITBUCKET/rest/api/1.0/projects
+
+# Bitbucket server basic auth creds
+BROKER_CLIENT_VALIDATION_BASIC_AUTH="$BITBUCKET_USERNAME:$BITBUCKET_PASSWORD"
+
 # The URL of the Snyk broker server
 BROKER_SERVER_URL=https://broker.snyk.io
 

client-templates/github-com/.env.sample

@@ -4,25 +4,27 @@ BROKER_TOKEN=
 # your personal access token to your github.com account
 GITHUB_TOKEN=
 
-# the host GitHub, excluding scheme. For github.com
-# this should be "github.com"
-GITHUB=
+# the host for GitHub, excluding scheme
+GITHUB=github.com
 
-# the host for GitHub's raw content, excluding scheme. For github.com
-# this should be "raw.githubusercontent.com"
-GITHUB_RAW=
+# the host for GitHub's raw content, excluding scheme
+GITHUB_RAW=raw.githubusercontent.com
 
-# the url that the github API should be accessed at. For github.com this should be
-# changed to "api.github.com"
-GITHUB_API=$GITHUB/api/v3
+# the GitHub REST API url, excluding scheme
+GITHUB_API=api.github.com
 
-# the url that the github graphql API should be accessed at.
-# For github.com this should be changed to "api.github.com"
-GITHUB_GRAPHQL=$GITHUB/api
+# the GitHub GraphQL API url, excluding scheme
+GITHUB_GRAPHQL=api.github.com
 
 # the url of your broker client (including scheme and port)
 # BROKER_CLIENT_URL=
 
+# GitHub validation url, checked by broker client systemcheck endpoint
+BROKER_CLIENT_VALIDATION_URL=https://$GITHUB_API/user
+
+# GitHub validation request Authorization header
+BROKER_CLIENT_VALIDATION_AUTHORIZATION_HEADER="token $GITHUB_TOKEN"
+
 # The URL of the Snyk broker server
 BROKER_SERVER_URL=https://broker.snyk.io
 

client-templates/github-enterprise/.env.sample

@@ -4,21 +4,24 @@ BROKER_TOKEN=
 # your personal access token to your github enterprise account
 GITHUB_TOKEN=
 
-# the host where your GitHub Enterprise is running, excluding scheme. For github.com
-# this should be "github.com"
+# the host for your GitHub Enterprise deployment, excluding scheme
 GITHUB=
 
-# the url that the github API should be accessed at. For github.com this should be
-# changed to "api.github.com"
+# the GitHub Enterprise REST API url, excluding scheme
 GITHUB_API=$GITHUB/api/v3
 
-# the url that the github graphql API should be accessed at.
-# For github.com this should be changed to "api.github.com"
+# the GitHub Enterprise GraphQL API url, excluding scheme
 GITHUB_GRAPHQL=$GITHUB/api
 
 # the url of your broker client (including scheme and port)
 # BROKER_CLIENT_URL=
 
+# GitHub Enterprise validation url, checked by broker client systemcheck endpoint
+BROKER_CLIENT_VALIDATION_URL=https://$GITHUB_API/user
+
+# GitHub Enterprise validation request Authorization header
+BROKER_CLIENT_VALIDATION_AUTHORIZATION_HEADER="token $GITHUB_TOKEN"
+
 # The URL of the Snyk broker server
 BROKER_SERVER_URL=https://broker.snyk.io
 

client-templates/gitlab/.env.sample

@@ -11,6 +11,9 @@ GITLAB=
 # the url of your broker client (including scheme and port)
 # BROKER_CLIENT_URL=
 
+# GitLab validation url, checked by broker client systemcheck endpoint
+BROKER_CLIENT_VALIDATION_URL=https://$GITLAB/api/v3/user?private_token=$GITLAB_TOKEN
+
 # The URL of the Snyk broker server
 BROKER_SERVER_URL=https://broker.snyk.io
 

client-templates/jira/.env.sample

@@ -4,16 +4,24 @@ BROKER_TOKEN=
 # your personal username to your Jira Server account
 JIRA_USERNAME=
 
-# your personal password to your Jira Server account
+# your personal password or API token to your Jira Server account
 JIRA_PASSWORD=
 
+# your Jira Server hostname, i.e. jira.yourdomain.com
+JIRA_HOSTNAME=
+
 # Your Jira Server URL, including scheme and hostname
-# i.e. https://jira.yourdomain.com
-JIRA_BASE_URL=
+JIRA_BASE_URL=https://$JIRA_HOSTNAME
 
 # the url of your broker client (including scheme and port)
 # BROKER_CLIENT_URL=
 
+# Jira validation url, checked by broker client systemcheck endpoint
+BROKER_CLIENT_VALIDATION_URL=https://$JIRA_HOSTNAME/rest/api/2/myself
+
+# Jira basic auth creds
+BROKER_CLIENT_VALIDATION_BASIC_AUTH="$JIRA_USERNAME:$JIRA_PASSWORD"
+
 # The URL of the Snyk broker server
 BROKER_SERVER_URL=https://broker.snyk.io
 

dockerfiles/bitbucket-server/Dockerfile

@@ -59,10 +59,13 @@ ENV ACCEPT accept.json
 # The path for the broker's internal healthcheck URL. Must start with a '/'.
 ENV BROKER_HEALTHCHECK_PATH /healthcheck
 
+# Bitbucket server validation url, checked by broker client healthcheck endpoint
+ENV BROKER_CLIENT_VALIDATION_URL https://$BITBUCKET/rest/api/1.0/projects
+
+# Bitbucket server basic auth creds
+ENV BROKER_CLIENT_VALIDATION_BASIC_AUTH "$BITBUCKET_USERNAME:$BITBUCKET_PASSWORD"
 
-EXPOSE $PORT
 
-HEALTHCHECK --interval=10s --timeout=1s \
-  CMD wget -q --spider http://localhost:${PORT}${BROKER_HEALTHCHECK_PATH}
+EXPOSE $PORT
 
 CMD ["broker", "--verbose"]

dockerfiles/github-com/Dockerfile

@@ -62,10 +62,13 @@ ENV ACCEPT accept.json
 # The path for the broker's internal healthcheck URL. Must start with a '/'.
 ENV BROKER_HEALTHCHECK_PATH /healthcheck
 
+# GitHub validation url, checked by broker client healthcheck endpoint
+ENV BROKER_CLIENT_VALIDATION_URL https://$GITHUB_API/user
+
+# GitHub validation request Authorization header
+ENV BROKER_CLIENT_VALIDATION_AUTHORIZATION_HEADER "token $GITHUB_TOKEN"
 
-EXPOSE $PORT
 
-HEALTHCHECK --interval=10s --timeout=1s \
-  CMD wget -q --spider http://localhost:${PORT}${BROKER_HEALTHCHECK_PATH}
+EXPOSE $PORT
 
 CMD ["broker", "--verbose"]

dockerfiles/github-enterprise/Dockerfile

@@ -59,10 +59,13 @@ ENV ACCEPT accept.json
 # The path for the broker's internal healthcheck URL. Must start with a '/'.
 ENV BROKER_HEALTHCHECK_PATH /healthcheck
 
+# GitHub Enterprise validation url, checked by broker client healthcheck endpoint
+ENV BROKER_CLIENT_VALIDATION_URL https://$GITHUB_API/user
+
+# GitHub Enterprise validation request Authorization header
+ENV BROKER_CLIENT_VALIDATION_AUTHORIZATION_HEADER "token $GITHUB_TOKEN"
 
-EXPOSE $PORT
 
-HEALTHCHECK --interval=10s --timeout=1s \
-  CMD wget -q --spider http://localhost:${PORT}${BROKER_HEALTHCHECK_PATH}
+EXPOSE $PORT
 
 CMD ["broker", "--verbose"]

dockerfiles/gitlab/Dockerfile

@@ -53,10 +53,11 @@ ENV ACCEPT accept.json
 # The path for the broker's internal healthcheck URL. Must start with a '/'.
 ENV BROKER_HEALTHCHECK_PATH /healthcheck
 
+# GitLab validation url, checked by broker client healthcheck endpoint
+ENV BROKER_CLIENT_VALIDATION_URL https://$GITLAB/api/v3/user?private_token=$GITLAB_TOKEN
+
 
-EXPOSE $PORT
 
-HEALTHCHECK --interval=10s --timeout=1s \
-  CMD wget -q --spider http://localhost:${PORT}${BROKER_HEALTHCHECK_PATH}
+EXPOSE $PORT
 
 CMD ["broker", "--verbose"]

dockerfiles/jira/Dockerfile

@@ -27,8 +27,11 @@ ENV BROKER_TOKEN <broker-token>
 ENV JIRA_USERNAME <username>
 ENV JIRA_PASSWORD <password>
 
+# Your Jira Server host
+ENV JIRA_HOSTNAME your.jira.server.hostname
+
 # Your Jira Server URL, including scheme and hostname
-ENV JIRA_BASE_URL https://your.jira.server.hostname
+ENV JIRA_BASE_URL https://$JIRA_HOSTNAME
 
 # The port used by the broker client to accept internal connections
 # Default value is 7341
@@ -54,10 +57,13 @@ ENV ACCEPT accept.json
 # The path for the broker's internal healthcheck URL. Must start with a '/'.
 ENV BROKER_HEALTHCHECK_PATH /healthcheck
 
+# Jira validation url, checked by broker client healthcheck endpoint
+ENV BROKER_CLIENT_VALIDATION_URL https://$JIRA_HOSTNAME/rest/api/2/myself
 
-EXPOSE $PORT
+# Jira basic auth creds
+ENV BROKER_CLIENT_VALIDATION_BASIC_AUTH "$JIRA_USERNAME:$JIRA_PASSWORD"
 
-HEALTHCHECK --interval=10s --timeout=1s \
-  CMD wget -q --spider http://localhost:${PORT}${BROKER_HEALTHCHECK_PATH}
+
+EXPOSE $PORT
 
 CMD ["broker", "--verbose"]

lib/client/index.js

@@ -1,4 +1,5 @@
 const primus = require('primus');
+const request = require('request');
 const socket = require('./socket');
 const relay = require('../relay');
 const logger = require('../log');
@@ -25,7 +26,7 @@ module.exports = ({ port = null, config = {}, filters = {} }) => {
 
   // IMPORTANT: defined before relay (`app.all('/*', ...`)
   app.get(config.brokerHealthcheckPath || '/healthcheck', (req, res) => {
-    // io.readyState sets the success of the healthcheck
+    // healthcheck state depends on websocket connection status
     // value of primus.Spark.OPEN means the websocket connection is open
     const isConnOpen = (io.readyState === primus.Spark.OPEN);
     const status = isConnOpen ? 200 : 500;
@@ -39,6 +40,70 @@ module.exports = ({ port = null, config = {}, filters = {} }) => {
     return res.status(status).json(data);
   });
 
+  app.get(config.brokerSystemcheckPath || '/systemcheck', (req, res) => {
+    // Systemcheck is the broker client's ability to assert the network 
+    // reachability and some correctness of credentials for the service
+    // being proxied by the broker client.
+
+    const brokerClientValidationMethod =
+      config.brokerClientValidationMethod || 'GET';
+    const brokerClientValidationTimeoutMs =
+      config.brokerClientValidationTimeoutMs || 5000;
+
+    const data = {
+      brokerClientValidationUrl: logger.sanitise(config.brokerClientValidationUrl),
+      brokerClientValidationMethod,
+      brokerClientValidationTimeoutMs,
+    };
+
+    const validationRequestHeaders = {
+      'user-agent': 'Snyk Broker client ' + version,
+    };
+
+    // set auth header according to config
+    if (config.brokerClientValidationAuthorizationHeader) {
+      validationRequestHeaders.authorization = config.brokerClientValidationAuthorizationHeader;
+    } else if (config.brokerClientValidationBasicAuth) {
+      validationRequestHeaders.authorization =
+        `Basic ${new Buffer(config.brokerClientValidationBasicAuth).toString('base64')}`;
+    }
+
+    // make the internal validation request
+    request({
+      url: config.brokerClientValidationUrl,
+      headers: validationRequestHeaders,
+      method: brokerClientValidationMethod,
+      timeout: brokerClientValidationTimeoutMs,
+      json: true,
+    }, (error, response) => {
+      // test logic requires to surface internal data
+      // which is best not exposed in production
+      if (process.env.TAP) {
+        data.testError = error;
+        data.testResponse = response;
+      }
+
+      if (error) {
+        data.ok = false;
+        data.error = error.message;
+        return res.status(500).json(data);
+      }
+
+      data.brokerClientValidationUrlStatusCode = response && response.statusCode;
+      // check for 2xx status code
+      const goodStatusCode = /^2/.test(response && response.statusCode);
+      if (!goodStatusCode) {
+        data.ok = false;
+        data.error = 'Status code is not 2xx';
+        return res.status(500).json(data);
+      }
+
+      data.ok = true;
+      return res.status(200).json(data);
+    });
+  });
+
+  // relay all other URL paths
   app.all('/*', (req, res, next) => {
     res.locals.io = io;
     next();