Case Study: NIST 800 53 Mappings

The primary goal of ATT&CK Sync is to reduce the effort required to stay in sync with new ATT&CK releases. In order to prove the concept, we use the Center’s NIST 800-53 Mappings project as a case study. This project is an ideal case study because it contains a large number of mappings that depend on ATT&CK (over 9k line items). Furthermore, we have previously updated these mappings from v8.2 to v9.0 and from v9.0 to v10.1, which gives us empirical evidence about the cost and level of effort to conduct these upgrades. For this case study, we are migrating the mappings from v10.1 to v12.1.

The concept of operation for this case study begins with the ATT&CK Sync changelog, which is obtained in a machine-readable format (JSON). We created a Python script to read that changelog and use that data to mark up the control mappings spreadsheet, adding annotations for each mapping that is potentially affected by ATT&CK changes (such as modified language in the description, new detections, or new mitigations). This step immediately reduces the level effort by 50% because over half of the mapped ATT&CK techniques have no changes and do not require review. The modified technique descriptions are displayed with red and green text to indicate word-by-word edits, which helps mapping analysts quickly review the changes to assess the impact on the validity of that mapping. Approximately another 20% were easily reviewed as the analysts were able to easily identify that changes were insignificant to the mappings (e.g., typo corrections, new but similar detections). This left about 30% where the associated ATT&CK techniques had changes requiring more in-depth analysis and research, such as significant changes to technique descriptions, new techniques, and technique deprecations. The ability to easily and efficiently identify control mappings that need to be re-assessed, in this case streamlining down from over 9000 to around 3000 mappings, assists analysts with focusing their efforts on those re-assessments and the highlighted underlying changes.

Download the related resources:

Sample JSON Changelog

Sample NIST 800-53 Mappings Excel

Running Python code:

$ poetry shell

$ python -m attack_sync.generate_mapping_excel \
    samples/attack-changelog-v10.1-v12.1.json \
    data/nist800-53-r4-mappings.xlsx \
    data/nist800-53-r4-mappings-output.xlsx