Git workflow for rules creation
- The main development is done in the feature branches.
- One feature branch should contain one new rule.
- Or you can use one feature branch, if you update a bunch of files at once, based on the same common characteristic
- Features are merged into the main branch (after all tests in Github CI/CD pipeline have successfully passed).
When you have a (Issue) ticket assigned for you to create a new rule, you should follow the next steps:
-
Fork and clone the repository of the cloud stream and navigate to its folder.
git clone https://github.com/epam/ecc-azure-rulepack.git cd ecc-azure-rulepack -
Checkout to a main branch, and from this branch create a new one for a new rule.
The new branch should have the following name format: feature/<rule_name>.
For instance, a new branch should be created like this:git switch main git switch -c "feature/ecc-azure-010-policy_name" -
Now you can start working on a rule.
a. Create a file <policy_name>.yml for future rule in policies/ folder. How to name a file you can find here - Rule file name convention
b. Create in terraform/<policy_name>/ green, red, and iam folders.
c. Create <policy_number>-policy.json file in terraform/<policy_name>/iam/ with minimum required permissions for a user to run Custodian policy.policy_name=<policy_name> policy_number=<policy_index_number> touch policies/$policy_name.yml mkdir -p terraform/$policy_name/{green,red,iam} touch terraform/$policy_name/iam/$policy_number-policy.json
Click here to expand for example...
touch policies/ecc-azure-002-cis_iam_owner_roles.yml mkdir -p terraform/ecc-azure-002-cis_iam_owner_roles/{green,red,iam} touch terraform/ecc-azure-002-cis_iam_owner_roles/iam/002-policy.json -
When you've finished with a rule, execute
git statusto check that all necessary files are ready, and you accidentally didn't edit unnecessary files. -
Now, you can stage, commit and push your changes.
Commit message structure is type: message
For example, new: added policy ecc-azure-010-policy_name
git add . git commit -m "new: added policy ecc-azure-010-policy_name" git push -u origin feature/ecc-azure-010-policy_name
Standard types:
Type Group name Comment example Description new New Policies added policy ecc-azure-010-policy_name for new policies with terraform files and tests upd Updates updated policies 017, 124
updated a number of policies (see the list in the commit message)upd due to new policy vision (for example, expanding rule filter conditions) p-fix Policy Fixes fixed policies 043, 342
fixed a number of policies (see the list in the commit message)fix of the incorrect implementation tf-fix Terraform Fixes fixed terraform for policies 043, 342
fixed a number of terraform files for policies (see the list in the commit message)t-fix Test Fixes fixed tests for policies 043, 342
fixed a number of tests for policies (see the list in the commit message)del Deletions deleted policies 012, 129
deleted terraform for policy 235
deleted tests for policy 012doc Documentation Changes added "How to use" info updated "Smth" info *commit with type [new] must include only one policy
*if it's more than 20 updates/fixes in one commit, use the commit message with '(see the list in the commit message)' and then write policy numbers after the blank row:
upd: updated a number of policies (see the list in the commit message)
012, 623, 124, 227...Commits with messages, that have no type, will be added to the group Other Changes. To exclude a commit from the changelog file use type [skip].
ℹ️ Note: To avoid merge conflicts, it's highly recommended before pushing changes to a remote branch, switch to a develop branch, pull changes, switch back to your working branch and merge develop into your branch.
git switch main git pull git switch feature/<rule_name> git merge main
-
Open Git repository in a Web-browser and create a pull request into main branch.
- For title enter your branch name: feature/<rule_name>.
- Select an Assignee. It must be someone from your cloud stream or if it isn't possible, assign a pull request (PR) to someone with a sufficient level of competence in your cloud.
- In "Pull options", check both box "Delete source branch when the pull request is accepted." and "Squash commits when the pull request is accepted.".
- Navigate to the "Changes" tab, to check your changes again that everything is correct.
- Click "Create pull request".
ℹ️ Note: If you pushed a branch to a remote repository, but after sometime you do not need it anymore, delete it from there. Do not keep unnecessary branches in a remote repository, keep it only if you need it in the future.
⚠️ warning: Under no circumstances should you commit and push secret keys, credentials, or any kind of secret information to a remote repository.
The layout represents a base folder and file structure for repositories with static rules. This structure may vary slightly from cloud to cloud, but in most cases, you need to adhere to it.
- iam - contains JSON files (for ARM and Azure CLI) to create a custom role with minimal permissions (read-only) in Azure Subscription that is able to run all Cloud Custodian rules in the repository. For more information: https://docs.microsoft.com/en-us/azure/role-based-access-control/overview.
- policies - contains rules in .yml format. Check our naming convention - Rules name convention.
- terraform - contains terraform scripts for each rule in the repository. Check how to write terraform scripts - Terraform - Guide.
- terraform//iam - contains JSON file with permissions for creating a custom role with minimal permissions that is able to run a specific Cloud Custodian rule in the repository.
- tests - (only AWS and GCP) contains the python test file red_policy_test.py and folders with records of green and red infrastructure in json files. In some rare cases, it may contain the green_policy_test.py test file.
- CHANGELOG.md and version - contains information about recent updates in the repository. It is updated once a week at the end of the sprint.
