Update azure_rm_appgateway to support setting WAF policy

[l3ender]

SUMMARY

This PR updates the app gateway module so that a web application firewall (WAF) policy can be configured.

Due to previous updates in dependencies (#1697), the WAF settings for an app gateway can only be configured via a separate WAF policy resource. Therefore other options are now deprecated as they are unused. If it is attempted to use the deprecated policies, the following error occurs:

Code: ApplicationGatewayWafConfigurationCannotBeAddedToWAFV2Gateway
Message: WebApplicationFirewallConfiguration cannot be added to WAF_V2 Gateway from API version 2024-01-01. Please use a WAF Policy instead.

ISSUE TYPE

• Bugfix Pull Request
• Feature Pull Request

COMPONENT NAME

azure_rm_appgateway

ADDITIONAL INFORMATION

This change was tested by running the integration test for the module. As there is no module to create a WAF policy, that resource needs to be created separately before referencing it from this module!

Click to expand output of integration test

-> cd ~/repos/azure && rm -f azure-azcollection-*.tar.gz && ansible-galaxy collection build . --force && ansible-galaxy collection install   azure-azcollection-*.tar.gz --force && cd ~/repos/azure/ansible_collections/azure/azcollection/ && ansible-test integration azure_rm_appgateway   --allow-destructive --allow-disabled -v
[DEPRECATION WARNING]: [defaults]collections_paths option, does not fit var naming standard, use the singular form collections_path instead. This   feature will be removed from ansible-core in version 2.19.
Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
Created collection for azure.azcollection at /Users/ross/repos/azure/azure-azcollection-2.7.0.tar.gz
[DEPRECATION WARNING]: [defaults]collections_paths option, does not fit var naming standard, use the singular form collections_path instead. This   feature will be removed from ansible-core in version 2.19.
Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
Starting galaxy collection install process
Process install dependency map
Starting collection install process
Installing 'azure.azcollection:2.7.0' to '/Users/ross/repos/azure/ansible_collections/azure/azcollection'
azure.azcollection:2.7.0 was installed successfully
Configured locale: en_US.UTF-8
Falling back to tests in "tests/integration/targets/" because "roles/test/" was not found.
Detected architecture aarch64 for Python interpreter: /Users/ross/repos/azure/venv/bin/python3.12
Using existing azure cloud config: tests/integration/cloud-config-azure.ini
Creating container database.
Run command: /Users/ross/repos/azure/venv/bin/python3.12 /Users/ross/repos/azure/venv/lib/python3.12/site-packages/ansible_test/_util/target/tools/  yamlcheck.py
Configuring target inventory.
Running azure_rm_appgateway integration test role
Initializing "/tmp/ansible-test-kqn8ovda-injector" as the temporary injector directory.
Injecting "/tmp/python-07m8dfvk-ansible/python" as a execv wrapper for the "/Users/ross/repos/azure/venv/bin/python3.12" interpreter.
Stream command: ansible-playbook azure_rm_appgateway-m6ajjl6v.yml -i inventory -v
Using /Users/ross/repos/azure/ansible_collections/azure/azcollection/tests/output/.tmp/integration/azure_rm_appgateway-19gz9d3n-ÅÑŚÌβŁÈ/tests/  integration/integration.cfg as config file
running playbook inside collection azure.azcollection

PLAY [testhost] ****************************************************************

TASK [Gathering Facts] *********************************************************
ok: [testhost]

TASK [azure_rm_appgateway : Gather Resource Group info] ************************
ok: [testhost] => {"changed": false, "resourcegroups": [{"id": "/subscriptions/15d7b290-260a-4ea7-a138-37b3f781f644/resourceGroups/  automated-testing", "location": "southcentralus", "name": "automated-testing", "properties": {"provisioning_state": "Succeeded"}, "tags": {}, "type":   "Microsoft.Resources/resourceGroups"}]}

TASK [azure_rm_appgateway : Prepare random number] *****************************
ok: [testhost] => {"ansible_facts": {"cert1_file": "cert1.txt", "cert1_password": "your-password", "cert2_file": "cert2.txt", "cert2_password":   "your-password", "cert3b64_file": "cert3b64.txt", "location": "southcentralus", "rpfx": "ef45326369"}, "changed": false}

TASK [azure_rm_appgateway : Create a virtual network] **************************
changed: [testhost] => {"changed": true, "check_mode": false, "state": {"address_prefixes": ["10.1.0.0/16", "172.100.0.0/16"], "dns_servers": ["127.0.  0.1", "127.0.0.2"], "etag": "W/\"ef8269e7-7d36-49aa-927e-57a4b55b1d67\"", "flow_timeout_in_minutes": null, "id": "/subscriptions/  15d7b290-260a-4ea7-a138-37b3f781f644/resourceGroups/automated-testing/providers/Microsoft.Network/virtualNetworks/vnetef45326369", "location":   "southcentralus", "name": "vnetef45326369", "provisioning_state": "Succeeded", "tags": null, "type": "Microsoft.Network/virtualNetworks"}}

TASK [azure_rm_appgateway : Create a subnet] ***********************************
changed: [testhost] => {"changed": true, "state": {"address_prefix": "10.1.0.0/24", "address_prefixes": null, "id": "/subscriptions/  15d7b290-260a-4ea7-a138-37b3f781f644/resourceGroups/automated-testing/providers/Microsoft.Network/virtualNetworks/vnetef45326369/subnets/  subnetef45326369", "name": "subnetef45326369", "nat_gateway": null, "network_security_group": {}, "private_endpoint_network_policies": "Enabled",   "private_link_service_network_policies": "Enabled", "provisioning_state": "Succeeded", "route_table": {}}}

TASK [azure_rm_appgateway : Configure public IP for waf policy gateway] ********
changed: [testhost] => {"changed": true, "state": {"dns_settings": {}, "etag": "W/\"831c7046-91e9-43e2-98ab-4ebe5f002f23\"",   "idle_timeout_in_minutes": 4, "ip_address": "13.65.47.176", "location": "southcentralus", "name": "appgateway-waf-policy-ef45326369-pip",   "provisioning_state": "Succeeded", "public_ip_address_version": "ipv4", "public_ip_allocation_method": "static", "sku": "Standard", "tags": null,   "type": "Microsoft.Network/publicIPAddresses", "zones": null}}

TASK [azure_rm_appgateway : Try to create waf policy instance of Application Gateway] ***
changed: [testhost] => {"changed": true, "compare": [], "id": "/subscriptions/15d7b290-260a-4ea7-a138-37b3f781f644/resourceGroups/automated-testing/  providers/Microsoft.Network/applicationGateways/appgateway-waf-policy-ef45326369", "location": "southcentralus", "name":   "appgateway-waf-policy-ef45326369", "operational_state": "Running", "provisioning_state": "Succeeded", "resource_group": "automated-testing"}

TASK [azure_rm_appgateway : Assert the resource instance is well created] ******
ok: [testhost] => {
    "changed": false,
    "msg": "All assertions passed"
}

TASK [azure_rm_appgateway : Try to create waf policy instance of Application Gateway - no update] ***
ok: [testhost] => {"changed": false, "compare": ["new item was empty, using old [/probes/*][ match ]", "new item was empty, using old [/probes/*][   match ]", "changed [/ssl_certificates/*/data] MIIKsQIBAzCCCm0GCSqGSIb3DQEHAaCCCl4EggpaMIIKVjCCBg8GCSqGSIb3DQEHAaCCBgAEggX8MIIF  +DCCBfQGCyqGSIb3DQEMCgECoIIE/jCCBPowHAYKKoZIhvcNAQwBAzAOBAj37r+wRsc6/gICB9AEggTY1V5HNscO+2bZb7JSMme1ljERe7DCiRE2cJsPKlwW2/  NEiSqAUX7gXKK0ISr6Dto71rFdza0uPwbcQoPN1QwkJHSAkxWg4OcAx2kf7077tlHhf5rzxTn5V3wXM0Q3h6NsDjSqSHjYVISIiXItUzlDaBpMY/NUFNCTyJR5I91MWsljrJ/  bQaAIo57HJR9nzjY5DaBA9P3bAhmX5LJRGsJWoCEaGeeVQ3Yn6yD06ordiJnf6dNxqQGN+o2x54gqfmw  +RnoC2f8VAsTIfb3fwJPKdg2JiJIa6Ms2Sc8VR7VGmZt34qZwTPBrzeqJjtIMT41bBae46lmma8ypYwErqzOYSrHqXPXzaxlloYy81HYWAsJTWyBxTsVBcLom5m9ru79  +SKG35xY1wSkzZmWMNFfVRFCJy/X+h2ErrGYjogCHYaIUmiosvUccwRUXGU083ul9iTcz/Dl79VBz63OFX/  CnZMDTQ8ugbqpvW78pAnBU0r8MUubHciD1sJG2zmMlxCAzan6BLm9OMyhTNIbzYOjQQw99MQQys/ZeyNLqTFHTeGRfU2ewqgHjbH2PYCQfjipXSmdmsSsGxlLA9AOtwAk3QKJ77P03HRGOeXmy/  I4iIHuIQuaQcjfprNR2fI36dftDo7U4gvRQHkiti+zNVqpi3/hIc2k7O8bCcMeSvfIlUvWIPUrUceZmpVPpLdcFcQbN9  +1nZwiFYydOhrPnlp40rSO3RM08EmQUfRYt8fwRFcoWBX3b411vOqZVGeMfMtThMYI53R4Cmh5tUp93FslHNmIfnuewhHfIm+vtCicLcW6TaC2l4EqmNf0flK5m5nANotCfqj87MPsB83qPwol/  91BTKaxuH2hKrZDgU1ibPE8NhzBinp2ANi0BHK3Sl0CsC2MPyZpFY+4MWvk/  SI9ex4VsKYKmhubOFkhDLLBZH0UEmUdNTH4Gd76GsDnfI9arR2ctM9ecTPeu74hKiHlNZhc4U3TX20FBeqF5tZYnfCLRhvdiNM9AlwEKqqQEe0W7PrALcNVdjhJl0X9  +0Br28E3RKZQRITWa10Vjmh0WcYrzEQ3/qEZYbqVpHMp  +kdrHxB65v0zlGxjdwyKzafLzqYXmaHOyVlFnkayNaAkVVxOCzNrxB9HfhjvhjWafeMvA0p7O9CxTD2xPEhUaHQ5j7L8F0alfMYcg73SdGHAcY6AV8  +eh0jqs3IF68cquXOl5Bm8uYKRjtgl9nY6hYc0lRDdtFHZo8ayNDr0cltNU7XZTaCKVNSDTRn92rTNJY0E3PD5HSKcRi58WJrIgEDGasyleRkRlGTY7512Qut0rg7m1Eyp6MK  +sNmSSA7cR70pH7I1dwy4VrJMODdMH11y1QJF2EQWQdN00Js54tjVgTIO3btb5N7jhNYpRedv0a4UZ8TdDI4ZMCMf3SdP3xbQ06M1pFrS8WQzwp3KTk8vmnseJL84n0hC8KqWmGmTWHTa9dwmopeM6  Xh/Jm1pkrgrloxqfSlscGEJE0plAnk1mLx29FxswfZ6a7pNKg7CydK4SiDkqM+pWukPbgKODqYPUvS0nk3RGGXvZSIzTbvm77tF+MqXOb6Rn+IflIk4yZsjIBQA0I/bQ78YDzXUVsrtAi9waRoCZs  +L48NUy4zpKn25FMqkrziVn  +TGB4jANBgkrBgEEAYI3EQIxADATBgkqhkiG9w0BCRUxBgQEAQAAADBdBgkqhkiG9w0BCRQxUB5OAHQAZQAtAGMANwBmADEAYwBhADYAMQAtADQAOAA1ADQALQA0ADgAZQBmAC0AYQAwADgANQAtAG  QANABhADIAZgA1AGYAOAAyADcAZAAxMF0GCSsGAQQBgjcRATFQHk4ATQBpAGMAcgBvAHMAbwBmAHQAIABTAHQAcgBvAG4AZwAgAEMAcgB5AHAAdABvAGcAcgBhAHAAaABpAGMAIABQAHIAbwB2AGkA  ZABlAHIwggQ/BgkqhkiG9w0BBwagggQwMIIELAIBADCCBCUGCSqGSIb3DQEHATAcBgoqhkiG9w0BDAEDMA4ECFcAfrkm3ibUAgIH0ICCA/hlBog8GY9GCpucTwAxBGa0cOGQ29EK0xfrmY/  Dv59IeJhRr47Mvl1XNk5PIInb64RsOVr00jrJAbfgLLKMBowcQLT7k6jGbGBdOzC57x9DNP0VuHsIIym0Z+SpJgEWBbtdBTGzgNw/  YoXYxT4Rtka9ScSyCFjmmCzXz7bGLqC7yrGb7BzigQ9y4u4bg0pf75pERzN8rJM29Ob2IydkgARfpmbNKjdMCtD6dI7tafG2lQfUX6sgQY  +Sy5HTz3ansN8X1yv2WQTu8Drxf2ce55v4WrFbPTTND94ubgDt7jvbCe1DuNP1DAYmQ5pbW0GGqF1x2csK5WWD7J8FD08VaQFM8y8pGIUeUkN4rYU3eTdTAQe+ec2hOr9QZn1Sb/  p5u4KqIMn4MSCQ8EU0gXa2JETdUjXPr/5JFZTidJYagRyMIkYnwg9uusikctulaBsHMBKMYQ0Z19CEbcd2phdoxWTrtp7kwwjnu64zPgE6ALe9yJOT8AFEB6H1c16Z  +aPGj9hbhkh6tcdGWUvzDYq08wjKjP3nA78StIisUmeZPfAXJUquPzRZr0pmcwYWfyP54TdC2BvPlLW/QXVV44IGxUdLuI6mz4p+O2  +xKu9QMFwdcpij2ZK4uMrBLDo7ZoTQ4rBRnn471AMUKgeP0D5tbl8PygUU1RqHv34ok3fwx0WglzdMQJyt53PiPWW4lipwUtUfd0eD8CXoMccf8XJmugVUBCD1wQsyCW6RrR9RX8HXVBrm5O2HKfJc  QYznl3qHqXb6ofvbOQ3S+v0ALN+sma8Tn6JceVTAOH+UuMdcu0FIDYnpmrvMecnJ2kbs1Y35mj4rSJyP5PGLg+ygb9VlBPwCCem/jHL+YivN38+0oWqfn2slyI4FNKX  +5U8M6xpiEaq6McKwKZC1d51A4dUdMAkO2d1Z6rVjqhKeqE6HWD9A0cyPBFZpNQskUfNDW8qILLEfEjhBi+s1LkHzKDykCN/  ReFfRiQS84DekoC59cymM8Hs1geMWCMFWfut4HTd7ItYaiJz5qpYVY4U/8myhyWnrktjLjQ6OkdM9bBDIpRHj95MYEC26NlWQZwjk  +yynPTZf0w5p2Ok6Dq2shJFykuJ1VkelgvsPe8qMa55Wp11dpREIFzXouPXNP/vFpziZcl1OKTd7Dwa  +ruQFRfsoZGzupsBOYxrmYqr6kOm5mzMW0HAlsWwl5mY2aSQMvXBE6k32xTkPIlIp763Ee4m6cmx4+SDcO5D+a9t05QY4JmssL  +x3T9qsbXSXDPEsg0cfVvuQYy9AYkIFOes4G45IagRAvhQQj9bEh8kTvp8CFDDtIrbWjX50zreb51VcAcEkIOLyROtIdLem0zA7MB8wBwYFKw4DAhoEFC0i4I5iwNYQug0vTVS0JC/Qm+/  NBBTsUM0D9QxIZYUi+qlDy14sOcEaUwICB9A= != None - default", "changed [/ssl_certificates/*/password] ******** != None - default"], "id": "/subscriptions/  15d7b290-260a-4ea7-a138-37b3f781f644/resourceGroups/automated-testing/providers/Microsoft.Network/applicationGateways/  appgateway-waf-policy-ef45326369", "location": "southcentralus", "modifiers": {}, "name": "appgateway-waf-policy-ef45326369", "operational_state":   "Running", "provisioning_state": "Succeeded", "resource_group": "automated-testing"}

TASK [azure_rm_appgateway : Assert the resource instance is not updated] *******
ok: [testhost] => {
    "changed": false,
    "msg": "All assertions passed"
}

PLAY RECAP *********************************************************************
testhost                   : ok=10   changed=4    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

[l3ender]

Hi @Fred-sun or @xuzhang3 , can you please review this PR when able? Thank you!

[Fred-sun]

@l3ender In addition, I refer to the azure-mgmt-network file, where the parameter 'web_application_firewall_configuration', 'firewall_policy' and 'force_firewall_policy_association' are equal. Is it better to treat them side by side? Thank you!

[l3ender]

I refer to the azure-mgmt-network file, where the parameter 'web_application_firewall_configuration', 'firewall_policy' and 'force_firewall_policy_association' are equal. Is it better to treat them side by side?

I saw the same but thought to keep the existing Ansible structure intact (everything under web_application_firewall_configuration) because a) it is the existing approach of the module so it minimizes design changes; and b) it seems like the cleaner approach--both these items are related so the module can encapsulate them together, regardless of how the upstream python library handles it.

Thanks!

[Fred-sun]

@l3ender Your comment has a comment on several parameters will be abandoned, but there is no message indicating that the reference document should be abandoned, could you please inform your reference document? Thank you!

[l3ender]

@Fred-sun, sorry for delay, I missed your message!

The azure-mgmt-network library does not indicate the change/deprecation; I believe it is enforced on the API and not in the client library so there is nothing in library to indicate!

I found the WAF configuration on an app gateway was deprecated when I tried running existing integration tests; when I did that, I received the error mentioned in the PR description.

I then noticed that the Azure Portal similarly no longer allows creating app gateways with WAF configuration; a separate policy must be used instead.

The best I can find that highlights this change is on this page describing WAF policies for app gateways, which notes:

If you have a Custom Rules only WAF Policy, then you may want to move to the new WAF Policy. Going forward, the firewall policy supports WAF policy settings, managed rulesets, exclusions, and disabled rule-groups. Essentially, all the WAF configurations that were previously done inside the Application Gateway are now done through the WAF Policy.

And similarly, the doc for configuring WAF on an app gateway mentions:

To enable a Web Application Firewall on Application Gateway, you must create a WAF policy.

So, that is the best/most detail I can find but I hope it answers your question! Thank you!