Add more projects to the automation

[rgildein]

I need to add juju_channels to the configuration and use it in
charm_check template. Add TEST_CHARM env variable to snap_check
template.

Added the check, promote and release for:

• charm-duplicty
• charm-promethues-blackbox-exporter
• charm-prometheus-juju-exporter
• charm-storage-connector
• prometheus-backupall-juju-exporter
• prometheus-juju-exporter

related: #111

got new information

[chanchiwai-ray]

@rgildein I think you will need to drop charm-prometheus-juju-exporter for now, since it might need a new template to support corner case canonical/charm-prometheus-juju-exporter#68

[samuelallan72]

I think you will need to drop charm-prometheus-juju-exporter for now, since it might need a new template to support corner case

Note: ideally we would use the same template for all related projects, except for cases where there is a strong case for a major difference. Otherwise we will end up with many templates and lose the advantage of enforcing consistency.

I agree that for now, with the limitations of our current structure and use of terraform templates, it makes sense to split templates. But longer term, when we switch to a better templating engine, we can definitely ensure we support these cases in a single template - especially in the two recent corner cases involving more complex matrix definition ( canonical/charm-prometheus-juju-exporter#68 and another from @rgildein ) - this should be quite possible to achieve.

[rgildein]

Why are we putting all these inactive projects to automation repos?

To be able to drop bootstack-actions.

I'm strongly for dropping terraform for managing the files and PRs - it's not what terraform is designed for and it's really awkward to use and maintain.

Why do you think it's not designed for these steps? I think it's not designed to be used without states, and we are using it exactly like that. The benefit of using states is that terraform can know that PR was created with exact commits and not push anything + git configuration will be applying only new changes and not always changing everything.

@rgildein could you link to your PoC from the sprint too?

rgildein#1

@rgildein I think you will need to drop charm-prometheus-juju-exporter for now, since it might need a new template to support corner case canonical/charm-prometheus-juju-exporter#68

We can merge it with it, and simply closed PR create by this automation and then @chanchiwai-ray can open new PR here with proper template. I do not want to remove it from the list, because we simply forgot to add.

Note: ideally we would use the same template for all related projects, except for cases where there is a strong case for a major difference. Otherwise we will end up with many templates and lose the advantage of enforcing consistency.

What is an issue to have multiple templates? I think we should avoid to creating some hacks (as we did for bootstack-action) to avoid having multiple templates. These hacks make any maintaining more complicated. The benefit of these automation is not to have singe template, but have single place where we can do simple changes, like 3.4/stable -> 3.5/stable for Juju.

With Jinja, we can do some logic inside templates, however I believe it will simply hide stuff and make it more complex to maintain.

[chanchiwai-ray]

We can merge it with it, and simply closed PR create by this automation and then @chanchiwai-ray can open new PR here with proper template. I do not want to remove it from the list, because we simply forgot to add.

That works too, let me know how we are going to handle these corner case (multiple templates or something Samuel suggested)

[samuelallan72]

What is an issue to have multiple templates? I think we should avoid to creating some hacks (as we did for bootstack-action) to avoid having multiple templates. These hacks make any maintaining more complicated. The benefit of these automation is not to have singe template, but have single place where we can do simple changes, like 3.4/stable -> 3.5/stable for Juju.

With Jinja, we can do some logic inside templates, however I believe it will simply hide stuff and make it more complex to maintain.

Agreed, I don't want to enforce a single template at the expense of losing simplicity. However, I believe we should strive to have as small a number of templates as possible, to help keep our workflows as consistent as possible. Consistency is the main benefit here (consistency includes having a single place to do simple changes like changing the juju version).

let me know how we are going to handle these corner case

My take on this is that for every change that deviates from the template is to consider how much of an edge case it is. If it's something truly radical and not likely to be used by other projects, then go for it and make a new template. However, if it's not really that much of a corner case, then we should consider refactoring the templates to better accomodate these changes. For exmaple, here we're taking about adding a new combination of matrix variables, which I've seen in at least one other PR recently too - so it's not really a strong corner case - here we already use template variables for matrix, so we can refactor how we use variables in matrix to accomodate how we want to structure it. That's the benefit of being able to refactor.

[rgildein]

Agreed, I don't want to enforce a single template at the expense of losing simplicity. However, I believe we should strive to have as small a number of templates as possible, to help keep our workflows as consistent as possible. Consistency is the main benefit here (consistency includes having a single place to do simple changes like changing the juju version).

There will be always exceptions, and with thous I believe using single value is not good. This is single repo where you can change it, so I do not think you need also single value. At least that's me.

My take on this is that for every change that deviates from the template is to consider how much of an edge case it is. If it's something truly radical and not likely to be used by other projects, then go for it and make a new template. However, if it's not really that much of a corner case, then we should consider refactoring the templates to better accomodate these changes. For exmaple, here we're taking about adding a new combination of matrix variables, which I've seen in at least one other PR recently too - so it's not really a strong corner case - here we already use template variables for matrix, so we can refactor how we use variables in matrix to accomodate how we want to structure it. That's the benefit of being able to refactor.

I'm ok to do any refactor, but I simply want to avoid doing hacks in original repo in the workflow or here. We can add exclude as parameter for our matrixes here instead of creating new template (but that will require to add empty value to all repos except one). I agree that if change is not radical, why not.

[Pjack]

Most of the child PR are pass , we could merge and investigate further issues if any.

[samuelallan72]

Agreed, let's merge and go from there. :)

[samuelallan72]

Actually no wait, we need to drop commit 9638ed6

[samuelallan72]

I did some updates here, so it should be ready for review again:

• dropped the cherry picked commit from Update apply logic #112 (it's been reverted, and needs more discussion and work before it can be reintroduced)
• rebased on main
• updated the test commands for two charms that have had their structure updated recently.