Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(metricsforwarder): Directly write to Log Cache via Syslog protocol #2913

Merged
merged 131 commits into from
May 29, 2024

Conversation

bonzofenix
Copy link
Contributor

@bonzofenix bonzofenix commented Apr 30, 2024

As part of an ongoing effort to get app autoscaler components running on CF, an alternative way to send custom metrics to log-cache from metricforwarder needs to be found.

Current implementation relays on direct connection within the metrics forwarder VM to a locally run metro-agent process. If the goals is to get Metricfowarder running as a CF app, CF container would need access to a metron-agent on the host machine, in this case a Diego cell, which would not be set up by as the same way it is being run now.

Alternatively we can relay on the log-cache syslog-server, while opening traffic from a cf container to the log-cache vm provision by the CF deployment.

This PR intends to default metricforwarder to write to the syslog-server directly instead of using a locally running agent.

@bonzofenix bonzofenix added the allow-acceptance-tests This label needs to be added to enable the acceptance tests to run. label Apr 30, 2024
@bonzofenix bonzofenix marked this pull request as draft April 30, 2024 15:10
ca_file: metron_client/ca.crt
db:
policy_db:
url: postgres://postgres:CHANGE_ME@some-service:5432/autoscaler

Check failure

Code scanning / SonarCloud

PostgreSQL database passwords should not be disclosed

<!--SONAR_ISSUE_KEY:AY8vjuDtqnDMGaaRqvuB-->Make sure this PostgreSQL database password gets changed and removed from the code. <p>See more on <a href="https://sonarcloud.io/project/issues?id=cloudfoundry_app-autoscaler-release&issues=AY8vjuDtqnDMGaaRqvuB&open=AY8vjuDtqnDMGaaRqvuB&pullRequest=2913">SonarCloud</a></p>
connection_max_lifetime: 60s

storedprocedure_db:
url: postgres://postgres:CHANGE_ME@some-service:5432/autoscaler

Check failure

Code scanning / SonarCloud

PostgreSQL database passwords should not be disclosed

<!--SONAR_ISSUE_KEY:AY8vjuDtqnDMGaaRqvuC-->Make sure this PostgreSQL database password gets changed and removed from the code. <p>See more on <a href="https://sonarcloud.io/project/issues?id=cloudfoundry_app-autoscaler-release&issues=AY8vjuDtqnDMGaaRqvuC&open=AY8vjuDtqnDMGaaRqvuC&pullRequest=2913">SonarCloud</a></p>
@bonzofenix bonzofenix changed the title Add initial files to push metricsforwarder as a cf app via binary POC of metricsforwarder running on CF Apr 30, 2024
@bonzofenix bonzofenix force-pushed the metricforwarder-on-cf branch 2 times, most recently from b2abedd to 6718757 Compare May 14, 2024 08:55
@bonzofenix bonzofenix self-assigned this May 14, 2024
Copy link

linux-foundation-easycla bot commented May 14, 2024

@bonzofenix bonzofenix force-pushed the metricforwarder-on-cf branch from 5da618c to f6ef81b Compare May 15, 2024 11:20
@bonzofenix bonzofenix force-pushed the metricforwarder-on-cf branch from 6ec0348 to b6a52a1 Compare May 21, 2024 14:52
@bonzofenix bonzofenix changed the title POC of metricsforwarder running on CF Enable Syslog for metricsforwarder May 21, 2024
@bonzofenix bonzofenix marked this pull request as ready for review May 21, 2024 15:04
@bonzofenix bonzofenix force-pushed the metricforwarder-on-cf branch from a3d39de to 676613d Compare May 28, 2024 10:53
bonzofenix and others added 3 commits May 28, 2024 12:54
 Long deployments names generate longer than posible urls and can
 break certificate generation process when running a bosh deploy.
@bonzofenix bonzofenix requested a review from geigerj0 May 28, 2024 13:27
bonzofenix and others added 4 commits May 29, 2024 13:06
Prometheus metrics and health enpoint will be address globally
once the we know whats the best way to address this issue when
running autoscaler components as cf apps.
@bonzofenix bonzofenix requested a review from geigerj0 May 29, 2024 13:19
Copy link
Contributor

@geigerj0 geigerj0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice and clean PR!

thanks also for making the world a better place by improving the pipeline creation, fixing the test-certs folders etc. 👍

@bonzofenix bonzofenix enabled auto-merge (rebase) May 29, 2024 16:39
Copy link

Quality Gate Passed Quality Gate passed

Issues
4 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

auto-merge was automatically disabled May 29, 2024 18:25

Rebase failed

@bonzofenix bonzofenix merged commit 8bce277 into main May 29, 2024
33 checks passed
@bonzofenix bonzofenix deleted the metricforwarder-on-cf branch May 29, 2024 18:27
@silvestre silvestre added enhancement and removed dependencies Pull requests that update a dependency file labels Jun 4, 2024
@silvestre silvestre changed the title Enable Syslog for metricsforwarder feat(metricsforwarder): Directly write to Log Cache via Syslog protocol Jun 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
allow-acceptance-tests This label needs to be added to enable the acceptance tests to run. enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants