Generative AI and machine learning models are not deployed as standalone systems. They are integrated into traditional applications, including web interfaces, APIs, email services, and internal tools. This means that ML application security depends on the same fundamentals as any other web-facing system, from input validation and authentication through to access control and secure data handling. This article covers the traditional security risks that carry over to the application layer of ML-based systems and the tactics adversaries use to exploit them.<\/p>\n\n\n\n
An ML-based system typically has several components. The model itself, the training data pipeline, the serving infrastructure, and the application that wraps it all together. The application component is the part that faces users and external services. It handles HTTP requests, renders interfaces, manages sessions, and passes input to the model for inference.<\/p>\n\n\n\n
Because this layer is architecturally identical to any traditional web application, it inherits the full catalogue of traditional web vulnerabilities. The OWASP Top 10:2025<\/a> lists broken access control at number one, with injection at number five. These risks do not disappear because a model sits behind the endpoint. The addition of an ML component increases the attack surface by introducing new data flows and integration points that may not receive the same security scrutiny as the core application logic.<\/p>\n\n\n\n OWASP maintains a separate Top 10 for LLM Applications<\/a> that addresses model-specific risks such as prompt injection and training data poisoning. The application-layer risks covered here are distinct from those model-level concerns, though both categories can exist in the same system and compound each other.<\/p>\n\n\n\n Unauthorised access occurs when an attacker reaches sensitive areas of the system without proper credentials. In ML-based applications, this can mean accessing administrative interfaces for model management, viewing training data through the application’s UI, or reaching inference endpoints that should be restricted.<\/p>\n\n\n\n The consequences follow the same pattern as any other application, including privilege escalation, data exfiltration, and potential full system compromise. The difference in an ML context is that “sensitive data” may include training datasets, model weights, hyperparameter configurations, or inference logs. These assets have value to an attacker beyond what traditional application data offers, because they can be used to reverse-engineer the model, poison future training runs, or extract proprietary intellectual property.<\/p>\n\n\n\n Injection attacks exploit poor input handling to manipulate back-end databases or system processes. SQL injection and command injection are the two most common variants, and both apply directly to ML-integrated applications.<\/p>\n\n\n\n Consider a web application that takes user input, passes it to a model for classification, and logs the results to a database. If the logging mechanism constructs SQL queries using unsanitised user input, the application is vulnerable to SQL injection regardless of what the model does with that input. A successful SQL injection can allow an attacker to retrieve user data, bypass authentication mechanisms, or modify database records.<\/p>\n\n\n\n Command injection follows the same principle. If the application calls system processes based on user-supplied input, for example triggering a model retraining script with user-specified parameters, an attacker can inject shell commands into those parameters and execute arbitrary code on the server.<\/p>\n\n\n\n The fix is the same as it has always been. Parameterised queries for database interactions, strict input validation and sanitisation, and never passing unsanitised user input to a shell. The presence of an ML model in the architecture does not change the remediation. It changes the blast radius, because a compromised application server that also hosts model training infrastructure gives an attacker access to both the application data and the model pipeline.<\/p>\n\n\n\n Weak authentication mechanisms remain one of the most exploited entry points. In ML-based applications, authentication failures are particularly damaging because they can expose model management interfaces, training pipelines, and data labelling tools that were built as internal utilities and never hardened for external access.<\/p>\n\n\n\n Common weaknesses include<\/p>\n\n\n\nUnauthorised access<\/h2>\n\n\n\n
Injection attacks<\/h2>\n\n\n\n
Insecure authentication<\/h2>\n\n\n\n
\n