Stack monitoring best practices

[niqo01]

Feature scope

Documentation

Describe your suggested feature

Hello,

I was wondering if you could shed some light on monitoring stack best practices for standard infrastructure setup.
For example, I am setting up a DbStack (with nested stacks), ApiStack, VpnStack, HostedZoneStack and a CDK Pipeline stack.

My first thought was to create a Monitoring stack and use introspection like below:

    val monitoringStack = MonitoringStack(this, "MonitoringStack", StackProps.builder().build())
    val monitoring = MonitoringFacade(
      monitoringStack,
      "MonitoringFacade",
      MonitoringFacadeProps.builder()
        .build(),
    )
    val defaultAspectProps =  MonitoringAspectProps.builder()
      .billing(MonitoringAspectType.builder().enabled(false).build())
      .ec2(MonitoringAspectType.builder().enabled(false).build())
      .elasticCache(MonitoringAspectType.builder().enabled(false).build())
      .build()
    monitoring.monitorScope(delegatedZoneStack, defaultAspectProps)
    monitoring.monitorScope(hangoutDbStack, defaultAspectProps,)
    monitoring.monitorScope(hangoutsApiStack, defaultAspectProps,)
    monitoring.monitorBilling()

Which works great.
But is that configuration ideal? Now all of my stacks have exports set for the monitoring stacks automatically and moving pieces around in any of my stacks are likely to result in issues like Cannot delete export Preprod-DbStack:ExportsOutputRefDbMigrateDbFnEFD6AD2752F9BDB0 as it is in use by Preprod-MonitoringStack.

Thanks for this great tool!

[echeung-amzn]

This is realistically going to be highly opinionated on what works for particular individuals/teams.

My first thought was to create a Monitoring stack and use introspection like below

monitorScope is generally convenient if you don't have specific needs and just want to have some dashboards for everything to look through.

Now all of my stacks have exports set for the monitoring stacks automatically and moving pieces around in any of my stacks are likely to result in issues

I generally see the export problem being referred to as "deadly embrace" and some posts like this one explain how to deal with those.

I've personally had decent experiences putting my monitoring resources within nested stacks close to the actual resources, e.g. an ApiStack containing some resources like an API Gateway, Lambda, etc. would contain a NestedApiMonitoringStack that has the facade monitoring those particular resources. That does help to avoid the deadly embrace problem a bit, so long as there's no actual cross-stack references.

I've certainly seen plenty of people just put things in separate stacks too though, and it could work fine if the infrastructure is relatively stable and doesn't change much.