From the course: Security Risks in AI and Machine Learning: Categorizing Attacks and Failure Modes (2022)

Unlock the full course today

Join today to access over 24,900 courses taught by industry experts.

System manipulation

System manipulation

From the course: Security Risks in AI and Machine Learning: Categorizing Attacks and Failure Modes (2022)

Start my 1-month free trial Buy for my team

System manipulation

“

- If a model is looking for unusual patterns over time, one way to manipulate the system is for an attacker to train it that's something that is abnormal is actually normal behavior. Then when the attacker goes to download large amounts of data, rather flagging or alerting this as abnormal behavior, the system has been trained by the attacker to think that this is normal. That is why it is so important to monitor systems for drift and other system manipulation and then to retool or retrain them as needed to ensure ongoing and reliable operation. While the vision of ML and AI for many encompasses a super smart system that gets smarter and better over time the reality isn't quite so futuristic. Many machine learning systems are trained in highly controlled settings and don't actually do a lot of learning in production. For some systems that are learning continuously, system or model manipulation is a concern. Over time, machine…

Contents