A common, but misguided threat modeling approach for systems incorporating ML models is to augment a typical system threat model with analysis of ML model-level attacks such as prompt injections. This effectively disconnects the ML model, the underlying system, and safety/privacy requirements in the threat modeling process and ultimately sets it up to fail by missing critical risks and controls to mitigate them. This talk presents Trail of Bits's approach to ML threat modeling and how it enables robust and practical control recommendations throughout the ML life cycle that account for the full complexity of the ML tech stack.
Presented at:
Authored by:
- Adelin Travers