Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Incremental config updates #187

Merged
merged 34 commits into from
Jul 12, 2024
Merged

Incremental config updates #187

merged 34 commits into from
Jul 12, 2024

Conversation

sukhwinder33445
Copy link
Contributor

@sukhwinder33445 sukhwinder33445 self-assigned this May 2, 2024
@cla-bot cla-bot bot added the cla/signed CLA is signed by all contributors of a PR label May 2, 2024
@sukhwinder33445 sukhwinder33445 marked this pull request as draft May 2, 2024 13:19
@sukhwinder33445 sukhwinder33445 force-pushed the incremental-config-updates branch 3 times, most recently from 5541f2e to caa996f Compare May 3, 2024 07:05
@oxzi oxzi mentioned this pull request May 3, 2024
Merged
@sukhwinder33445 sukhwinder33445 force-pushed the incremental-config-updates branch from caa996f to 8a6a7c9 Compare May 3, 2024 09:22
@nilmerg nilmerg added this to the Beta milestone May 3, 2024
@sukhwinder33445 sukhwinder33445 force-pushed the incremental-config-updates branch 3 times, most recently from d733192 to 8813a6a Compare May 6, 2024 14:16
@sukhwinder33445
Copy link
Contributor Author

sukhwinder33445 commented May 14, 2024 •
edited
Loading

Note to myself:
No changes have been made to the following forms, as they will change completely based on the existing PRs:

  • Event Rule forms
  • Schedule Forms (changes made, but not yet final and not working properly)

@sukhwinder33445 sukhwinder33445 force-pushed the incremental-config-updates branch 2 times, most recently from 04e7569 to 70355f3 Compare May 14, 2024 08:26
@sukhwinder33445 sukhwinder33445 force-pushed the incremental-config-updates branch 4 times, most recently from 3fcc783 to a0d6bcb Compare June 20, 2024 10:29
oxzi added a commit to Icinga/icinga-notifications that referenced this pull request Jun 20, 2024
@oxzi
Incremental Configuration Updates
b746738 
Previously, the entire configuration stored in the database was
synchronized every second. With the growth of configurations in live
environments on the horizon, this would simply not scale well. This
brings us to incremental updates.

By introducing two new columns - "changed_at" as a Unix millisecond
timestamp and "deleted" as a boolean - for all tables referenced in the
ConfigSet structure, SQL queries can be modified to retrieve only those
rows with a more recent timestamp. The "deleted" column became necessary
to detect disappearances, since the synchronization now only takes newer
items into account. Some additional fields needed to be added to the
ConfigSet to track relationships.

Even though the codebase served well at the time, there was some code
that did almost the same thing as other code, just in different ways.
So a huge refactoring was done. This resulted in an internal generic
function that handles all synchronization with custom callbacks.

The web counterpart is being developed in
<Icinga/icinga-notifications-web#187>.

Closes #5.
oxzi added a commit to Icinga/icinga-notifications that referenced this pull request Jun 20, 2024
@oxzi
Incremental Configuration Updates
90f793f 
Previously, the entire configuration stored in the database was
synchronized every second. With the growth of configurations in live
environments on the horizon, this would simply not scale well. This
brings us to incremental updates.

By introducing two new columns - "changed_at" as a Unix millisecond
timestamp and "deleted" as a boolean - for all tables referenced in the
ConfigSet structure, SQL queries can be modified to retrieve only those
rows with a more recent timestamp. The "deleted" column became necessary
to detect disappearances, since the synchronization now only takes newer
items into account. Some additional fields needed to be added to the
ConfigSet to track relationships.

Even though the codebase served well at the time, there was some code
that did almost the same thing as other code, just in different ways.
So a huge refactoring was done. This resulted in an internal generic
function that handles all synchronization with custom callbacks.

The web counterpart is being developed in
<Icinga/icinga-notifications-web#187>.

Closes #5.
@sukhwinder33445 sukhwinder33445 force-pushed the incremental-config-updates branch 7 times, most recently from da9e9a8 to 215a532 Compare June 24, 2024 13:45
oxzi added a commit to Icinga/icinga-notifications that referenced this pull request Jun 24, 2024
@oxzi
Incremental Configuration Updates
2798275 
Previously, the entire configuration stored in the database was
synchronized every second. With the growth of configurations in live
environments on the horizon, this would simply not scale well. This
brings us to incremental updates.

By introducing two new columns - "changed_at" as a Unix millisecond
timestamp and "deleted" as a boolean - for all tables referenced in the
ConfigSet structure, SQL queries can be modified to retrieve only those
rows with a more recent timestamp. The "deleted" column became necessary
to detect disappearances, since the synchronization now only takes newer
items into account. Some additional fields needed to be added to the
ConfigSet to track relationships.

Even though the codebase served well at the time, there was some code
that did almost the same thing as other code, just in different ways.
So a huge refactoring was done. This resulted in an internal generic
function that handles all synchronization with custom callbacks.

The web counterpart is being developed in
<Icinga/icinga-notifications-web#187>.

Closes #5.
@sukhwinder33445 sukhwinder33445 force-pushed the incremental-config-updates branch from ed5d34d to e73f309 Compare July 10, 2024 13:26
oxzi added a commit to Icinga/icinga-notifications that referenced this pull request Jul 10, 2024
@oxzi
Incremental Configuration Updates
252db66 
Previously, the entire configuration stored in the database was
synchronized every second. With the growth of configurations in live
environments on the horizon, this would simply not scale well. This
brings us to incremental updates.

By introducing two new columns - "changed_at" as a Unix millisecond
timestamp and "deleted" as a boolean - for all tables referenced in the
ConfigSet structure, SQL queries can be modified to retrieve only those
rows with a more recent timestamp. The "deleted" column became necessary
to detect disappearances, since the synchronization now only takes newer
items into account. Some additional fields needed to be added to the
ConfigSet to track relationships.

Even though the codebase served well at the time, there was some code
that did almost the same thing as other code, just in different ways.
So a huge refactoring was done. This resulted in an internal generic
function that handles all synchronization with custom callbacks.

The web counterpart is being developed in
<Icinga/icinga-notifications-web#187>.

Closes #5.
@sukhwinder33445 sukhwinder33445 force-pushed the incremental-config-updates branch from e73f309 to 49ca621 Compare July 11, 2024 08:41
@sukhwinder33445 sukhwinder33445 force-pushed the incremental-config-updates branch from 49ca621 to 2bf142a Compare July 11, 2024 09:02
nilmerg
nilmerg requested changes Jul 11, 2024
View reviewed changes
application/forms/ChannelForm.php Outdated Show resolved Hide resolved
@sukhwinder33445 sukhwinder33445 force-pushed the incremental-config-updates branch from 3dfd349 to 4217e45 Compare July 11, 2024 11:44
@sukhwinder33445 sukhwinder33445 force-pushed the incremental-config-updates branch from 4217e45 to ebfc217 Compare July 11, 2024 11:48
@sukhwinder33445
Forms: Add changed_at column manualy on db::insert
013a57d 
Mariadb ≥10.1 does not support the default (UNIX_TIMESTAMP() * 1000)
More info: Icinga/icinga-notifications#191 (comment)
oxzi added a commit to Icinga/icinga-notifications that referenced this pull request Jul 11, 2024
@oxzi
Incremental Configuration Updates
09ad259 
Previously, the entire configuration stored in the database was
synchronized every second. With the growth of configurations in live
environments on the horizon, this would simply not scale well. This
brings us to incremental updates.

By introducing two new columns - "changed_at" as a Unix millisecond
timestamp and "deleted" as a boolean - for all tables referenced in the
ConfigSet structure, SQL queries can be modified to retrieve only those
rows with a more recent timestamp. The "deleted" column became necessary
to detect disappearances, since the synchronization now only takes newer
items into account. Some additional fields needed to be added to the
ConfigSet to track relationships.

Even though the codebase served well at the time, there was some code
that did almost the same thing as other code, just in different ways.
So a huge refactoring was done. This resulted in an internal generic
function that handles all synchronization with custom callbacks.

The web counterpart is being developed in
<Icinga/icinga-notifications-web#187>.

Closes #5.
oxzi added a commit to Icinga/icinga-notifications that referenced this pull request Jul 11, 2024
@oxzi
Incremental Configuration Updates
fb36770 
Previously, the entire configuration stored in the database was
synchronized every second. With the growth of configurations in live
environments on the horizon, this would simply not scale well. This
brings us to incremental updates.

By introducing two new columns - "changed_at" as a Unix millisecond
timestamp and "deleted" as a boolean - for all tables referenced in the
ConfigSet structure, SQL queries can be modified to retrieve only those
rows with a more recent timestamp. The "deleted" column became necessary
to detect disappearances, since the synchronization now only takes newer
items into account. Some additional fields needed to be added to the
ConfigSet to track relationships.

Even though the codebase served well at the time, there was some code
that did almost the same thing as other code, just in different ways.
So a huge refactoring was done. This resulted in an internal generic
function that handles all synchronization with custom callbacks.

The web counterpart is being developed in
<Icinga/icinga-notifications-web#187>.

Closes #5.
oxzi added a commit to Icinga/icinga-notifications that referenced this pull request Jul 11, 2024
@oxzi
Incremental Configuration Updates
beef754 
Previously, the entire configuration stored in the database was
synchronized every second. With the growth of configurations in live
environments on the horizon, this would simply not scale well. This
brings us to incremental updates.

By introducing two new columns - "changed_at" as a Unix millisecond
timestamp and "deleted" as a boolean - for all tables referenced in the
ConfigSet structure, SQL queries can be modified to retrieve only those
rows with a more recent timestamp. The "deleted" column became necessary
to detect disappearances, since the synchronization now only takes newer
items into account. Some additional fields needed to be added to the
ConfigSet to track relationships.

Even though the codebase served well at the time, there was some code
that did almost the same thing as other code, just in different ways.
So a huge refactoring was done. This resulted in an internal generic
function that handles all synchronization with custom callbacks.

The web counterpart is being developed in
<Icinga/icinga-notifications-web#187>.

Closes #5.
oxzi added a commit to Icinga/icinga-notifications that referenced this pull request Jul 11, 2024
@oxzi
Incremental Configuration Updates
2d2048e 
Previously, the entire configuration stored in the database was
synchronized every second. With the growth of configurations in live
environments on the horizon, this would simply not scale well. This
brings us to incremental updates.

By introducing two new columns - "changed_at" as a Unix millisecond
timestamp and "deleted" as a boolean - for all tables referenced in the
ConfigSet structure, SQL queries can be modified to retrieve only those
rows with a more recent timestamp. The "deleted" column became necessary
to detect disappearances, since the synchronization now only takes newer
items into account. Some additional fields needed to be added to the
ConfigSet to track relationships.

Even though the codebase served well at the time, there was some code
that did almost the same thing as other code, just in different ways.
So a huge refactoring was done. This resulted in an internal generic
function that handles all synchronization with custom callbacks.

The web counterpart is being developed in
<Icinga/icinga-notifications-web#187>.

Closes #5.
julianbrost pushed a commit to Icinga/icinga-notifications that referenced this pull request Jul 12, 2024
@oxzi @julianbrost
Incremental Configuration Updates
860ea0c 
Previously, the entire configuration stored in the database was
synchronized every second. With the growth of configurations in live
environments on the horizon, this would simply not scale well. This
brings us to incremental updates.

By introducing two new columns - "changed_at" as a Unix millisecond
timestamp and "deleted" as a boolean - for all tables referenced in the
ConfigSet structure, SQL queries can be modified to retrieve only those
rows with a more recent timestamp. The "deleted" column became necessary
to detect disappearances, since the synchronization now only takes newer
items into account. Some additional fields needed to be added to the
ConfigSet to track relationships.

Even though the codebase served well at the time, there was some code
that did almost the same thing as other code, just in different ways.
So a huge refactoring was done. This resulted in an internal generic
function that handles all synchronization with custom callbacks.

The web counterpart is being developed in
<Icinga/icinga-notifications-web#187>.

Closes #5.
@nilmerg nilmerg merged commit c677b3c into main Jul 12, 2024
22 checks passed
@nilmerg nilmerg deleted the incremental-config-updates branch July 12, 2024 07:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@oxzi oxzi oxzi left review comments

@nilmerg nilmerg nilmerg approved these changes

Assignees

@sukhwinder33445 sukhwinder33445

Labels
cla/signed CLA is signed by all contributors of a PR
Projects
None yet
Milestone
Beta
Development

Successfully merging this pull request may close these issues.
3 participants
@sukhwinder33445 @oxzi @nilmerg