poc: send data to segment

[JameelB]

Description

POC on how to send data to Segment. This tracks two events: Kafka Instance Create and Kafka Instance Delete.

Note that this isn't the final changes but an initial rough implementation to show what needs to be done in order to add the client and send data to Segment. The event names and properties/values tied to an event may still change depending on requirements. Client configuration may also change.

This doesn't include:

• unit/integration test changes, some are still broken as mocks for the Segment client hasn't been created.
• monitoring
• secret file configuration for reading the API Key

Events seen in Segment:

Complete JSON Data

{
  "context": {
    "app": {
      "name": "K*** F*** M***",
      "namespace": "development",
      "version": "93ffb4679d2a777486603907e380b3dfd43bb780"
    },
    "library": {
      "name": "analytics-go",
      "version": "3.0.0"
    }
  },
  "event": "Kafka Instance Create",
  "integrations": {},
  "messageId": "cb108064-a4a3-4286-9f6e-37b444460344",
  "originalTimestamp": "2022-12-21T12:30:44.269733Z",
  "properties": {
    "result": "success"
  },
  "receivedAt": "2022-12-21T12:30:45.229Z",
  "sentAt": "2022-12-21T12:30:44.269Z",
  "timestamp": "2022-12-21T12:30:45.228Z",
  "type": "track",
  "userId": "kfm-test-user",
  "writeKey": "<redacted>"
}

Complete JSON Data

{
  "context": {
    "app": {
      "name": "K*** F*** M***",
      "namespace": "development",
      "version": "93ffb4679d2a777486603907e380b3dfd43bb780"
    },
    "library": {
      "name": "analytics-go",
      "version": "3.0.0"
    }
  },
  "event": "Kafka Instance Delete",
  "integrations": {},
  "messageId": "82780273-55a1-4a9c-829f-705b321bc4bb",
  "originalTimestamp": "2022-12-21T12:30:59.54096Z",
  "properties": {
    "result": "failed"
  },
  "receivedAt": "2022-12-21T12:30:59.654Z",
  "sentAt": "2022-12-21T12:30:59.541Z",
  "timestamp": "2022-12-21T12:30:59.653Z",
  "type": "track",
  "userId": "kfm-test-user",
  "writeKey": "<redacted>"
}

Verification Steps

N/A

Checklist (Definition of Done)

• All acceptance criteria specified in JIRA have been completed
• Unit and integration tests added that prove the fix is effective or the feature works (tested against emulated and non-emulated OCM environment)
• Documentation added for the feature
• CI and all relevant tests are passing
• Code Review completed
• Verified independently by reviewer
• All PR comments are resolved either by addressing them or creating follow up tasks
• Required metrics/dashboards/alerts have been added (or PR created).
• Required Standard Operating Procedure (SOP) is added.
• JIRA has been created for changes required on the client side

[machi1990]

@JameelB @stehessel there are rooms to collaborate & share experience between Kafka & ACS services as both set of teams are looking at this.

The ACS team have opened up a draft PR regarding this as well stackrox/acs-fleet-manager#733

[stehessel]

@machi1990 Sure, thanks for letting us know. FYI, we have recently started to add telemetry via Segment to our ACS core product, so we already have some experience. cc @0x656b694d

[parametalol]

Hi, here's our framework for telemetry gathering and reporting: https://github.com/stackrox/stackrox/tree/master/pkg/telemetry/phonehome.
It includes a thin wrapper over segment SDK, support for intercepting gRPC and HTTP calls and for periodic gathering.

[machi1990]

Thanks @stehessel @0x656b694d

Hi, here's our framework for telemetry gathering and reporting: https://github.com/stackrox/stackrox/tree/master/pkg/telemetry/phonehome. It includes a thin wrapper over segment SDK, support for intercepting gRPC and HTTP calls and for periodic gathering.

Awesome. /cc @JameelB