As machine learning becomes part of industrial automation, its security depends on more than protecting servers and networks. Models can also be affected by deliberate changes to their training data, carefully crafted inputs at inference time, or attempts to extract information through model queries. Adversarial machine learning (AML) provides a way to describe these threats and assess their possible consequences.
A useful taxonomy can help engineering, operations, and cybersecurity teams discuss the same risks in consistent terms. The 2019 NISTIR 8269 draft introduced a framework organized around attacks, defenses, and consequences. That draft was withdrawn on March 8, 2023, and superseded by NIST AI 100-2e2023, an initial public draft. It should therefore be treated as historical material, not as current NIST guidance. Its risk-based concepts nevertheless offer a practical way to understand the security challenges involved.
Adversarial machine learning spans the model lifecycle
Machine learning systems rely on data, models, and processes for training, validation, and inference. This creates security considerations across more than one operational stage. An attacker might target training data or the learning process, or try to manipulate inputs and obtain information from a deployed model.
The target may be a physical input sensor, data as represented after preprocessing, the model itself, or a physical action produced by a system. In an industrial setting, that distinction can help teams map a model’s boundaries: where information enters, where it is transformed, how predictions are made, and whether those predictions influence an operational decision or action.
AML analysis also distinguishes intentional attacks from unintentional issues such as design flaws or biased data. Both can affect model performance, but they are different problem categories and should not be conflated in a security assessment.
Two attack windows: training and inference
Training-time attacks
Training-time attacks seek to access or influence the data or model used to build a system. In a data access attack, an adversary obtains some training information that may help them study the model or develop a substitute. In a poisoning attack, the adversary changes the learning process by affecting training data or, in some cases, the model logic itself.
Poisoning can take different forms. Data injection adds adversarial samples to a dataset; data manipulation changes existing inputs or labels; and logic corruption tampers with the learning algorithm. Indirect poisoning occurs before data reaches the target system’s preprocessing stage. These distinctions matter when reviewing data provenance, access controls, preprocessing, and the integrity of model-development workflows.
Inference-time attacks
Inference-time attacks generally leave the training data and target model untouched. Instead, the adversary interacts with the deployed model. Evasion attacks use altered inputs to cause an incorrect prediction. Depending on the attacker’s knowledge and access, the input changes may be generated using information about the model, a substitute model, or feedback from queries.
Oracle attacks use a model’s input-output interface to learn about the model or its training data. Repeated queries may support model extraction, data inference, or membership inference—attempts to determine whether particular information was included in the training set. The taxonomy describes attackers as black box, gray box, or white box according to how much they know about the target.
Assess consequences, not just attack techniques
Identifying an attack type is only one part of understanding risk. The same technique can have different implications depending on the application, the model’s role, and the defenses in place. The taxonomy separates consequences into three broad security concerns:
- Integrity: Predictions or actions are altered, including reduced confidence or misclassification. An error may be general or may direct an input toward a particular incorrect class.
- Availability: The model’s output becomes inaccessible or unusable, for example because service access or inference quality is impaired.
- Confidentiality and privacy: Information about the model or its training data is revealed. Privacy violations are a specific concern when personal information is exposed.
For industrial teams, the operational significance depends on how a model is used. A classification error, degraded inference speed, and disclosure of sensitive training information are different outcomes and call for different risk responses. Assessment should connect the model’s function to the consequences of incorrect, unavailable, or revealing outputs rather than treating all model failures as equivalent.
Defenses involve tradeoffs and layered review
Defenses can target training-time or inference-time threats. Training-data protections include encryption, sanitization to identify and remove problematic samples, and robust statistical methods intended to limit the influence of poisoned data. Inference-related approaches include adversarial training, ensembles, feature transformations, and other methods intended to improve model robustness.
Privacy-focused techniques address a different part of the problem. Differential privacy uses randomization to limit what model outputs reveal about individual training records; the source notes a potential tradeoff with prediction accuracy. Homomorphic encryption can allow computation on encrypted data, but introduces computational overhead and limits operations to those supported by the encryption approach.
No single defense should be assumed to address every threat. Some robustness methods may be challenged by attackers who use model queries or other information to build a substitute model. Defensive choices should therefore be assessed against a defined threat model: the attacker’s goals, knowledge, and capabilities. Teams should also account for possible effects on model accuracy and operational performance.
Applying the framework to industrial AI risk
A practical review can use the taxonomy as a set of questions rather than a checklist of guaranteed controls:
- Which parts of the ML pipeline are exposed, from data collection and preprocessing to model queries and outputs?
- Could an attacker access training data, influence it, manipulate inputs, or query the deployed model?
- What does the attacker know about the model, and what information can the system reveal in response to queries?
- Which outcomes matter most: incorrect decisions, loss of service, or exposure of model or training information?
- What tradeoffs could proposed defenses introduce for accuracy, computation, or system operation?
These questions help connect AI security to system architecture and operational risk without assuming that every ML application has the same exposure or impact. They can also support clearer communication among model developers, integrators, plant cybersecurity teams, and decision-makers.
Conclusion
Adversarial machine learning broadens the security conversation from infrastructure to the data, models, and interfaces that shape machine-learning behavior. Separating attacks by target and lifecycle stage, then evaluating integrity, availability, confidentiality, and privacy consequences, gives industrial teams a structured way to reason about risk. Because the 2019 NISTIR 8269 draft has been withdrawn and superseded, organizations should consult the superseding publication for current NIST material while using the historical taxonomy only as context.
Access Official Technical PDF