AI systems learn from data. Their performance, reliability, and security all depend on it, and when that data is corrupted, manipulated, or leaked, the model built on it inherits the problem. The model cannot tell the difference between legitimate patterns and patterns an attacker planted. This article maps the standard AI data pipeline stage by stage, drawn from the AI Data Attacks module on HackTheBox Academy, and identifies where each stage creates opportunities for compromise. It sits within the broader AI Red Teaming Series, and connects directly to LLM04:2025 Data and Model Poisoning in the OWASP LLM Top 10 and the Data component of Google’s Secure AI Framework.

How the AI data pipeline works

Most AI implementations follow a common sequence. Raw data is collected from various sources, processed and transformed into a usable format, then fed into models for training or prediction. The specifics vary by organisation and application, but the stages are consistent enough to reason about generically.

Each stage relies on different technologies, carries different assumptions about data integrity, and presents its own attack surface.

Data collection

Data collection pulls raw information from wherever it originates. User interactions arrive from web applications as JSON logs streamed through Apache Kafka. Structured transaction records come out of PostgreSQL. IoT devices push sensor readings over MQTT, web scrapers extract content from public sites using tools like Scrapy, and third parties deliver batch files in formats ranging from CSV and Parquet to JPEG images and WAV audio.

The variety is part of the challenge. Every source is a separate trust boundary, and every format carries different assumptions about integrity. Quality problems introduced at this stage, whether malicious or accidental, ride the pipeline forward into processing, training, and eventually production. The pipeline does not inherently validate what it collects.

Storage

Where the data lands depends on its shape. Structured records go into relational databases like PostgreSQL. Semi-structured logs fit NoSQL stores like MongoDB. Large, mixed datasets end up in data lakes built on distributed file systems or cloud object storage such as S3, HDFS, or Azure Blob. Time-series data gets specialised tools like InfluxDB.

Training data is one category of stored asset. The other is the models themselves. Trained models are serialised into files (.pkl, ONNX, .pt, .pth) and sit alongside the data in the same storage infrastructure. This matters because a .pkl file can contain arbitrary Python code, and a .pt file can be swapped without the pipeline raising an alert. If an attacker gets write access to the storage layer, they do not need to poison training data at all. They can replace the model directly.

Data processing and transformation

Raw data is almost never usable as-is. The processing and transformation stage turns it into something a model can learn from, and this is where much of the actual engineering effort lives.

Missing values are handled with tools like Pandas and scikit-learn imputers. Numeric features are scaled to comparable ranges using StandardScaler. Feature engineering then builds new inputs from existing data, whether that means extracting date components from timestamps, generating text embeddings with spaCy or NLTK, or augmenting image datasets with OpenCV and Pillow. At scale, distributed frameworks like Apache Spark and Dask handle the compute, and orchestrators like Apache Airflow or Kubeflow keep the workflow running in order.

All of this logic is code. Cleaning scripts, transformation jobs, feature extractors, all running automatically and shaping exactly what the model sees. If an attacker compromises the processing logic rather than the raw data, the corruption enters the pipeline at a point where nobody is looking for it. The raw data upstream still looks clean.

Modelling

Processed data feeds into the analysis and modelling stage. Data scientists explore the dataset in environments like Jupyter Notebooks or managed platforms like AWS SageMaker and Azure ML, then train models with frameworks like PyTorch, TensorFlow, or scikit-learn. The cycle is iterative, starting with algorithm selection (a random forest, a CNN, a transformer), then tuning hyperparameters with tools like Optuna, validating performance, and repeating until the model meets its benchmarks.

From a security perspective, the modelling stage is a downstream consumer. It faithfully learns whatever the data teaches it. If the upstream data was poisoned, the model absorbs the poison without flagging or resisting it. The actual attack surface lives in collection, storage, and processing. Modelling is simply where the consequences become real.

Deployment

Deployment marks the transition from trained artefact to live system. The most common pattern wraps the model in a REST API using FastAPI or Flask, containerises it with Docker, and orchestrates it through Kubernetes. Models can also run as serverless functions on platforms like AWS Lambda, or be compiled directly for edge devices using TensorFlow Lite.

The security concern at deployment is the model file itself. A model loaded into production is implicitly trusted to produce correct outputs. If that file has been tampered with, whether swapped for a trojaned version or loaded through an insecure deserialisation path, the compromise reaches production silently. Wrong predictions are one outcome. Arbitrary code execution is the more severe one.

Monitoring and retraining

Deployed models do not stay static. Monitoring tracks both standard operational health (latency and throughput via tools like Prometheus and Grafana) and ML-specific metrics like data drift and prediction quality (using platforms like WhyLabs or Arize AI).

When performance degrades, the model gets retrained. Feedback from predictions and user interactions is logged, combined with newly collected data, and run through the pipeline again. Orchestrators like Airflow manage these retraining cycles automatically, and the updated model replaces the deployed version.

This is where the security problem compounds. Retraining is designed to incorporate new data. That is the whole point. But the pipeline cannot reliably distinguish between a genuine shift in user behaviour and data an attacker deliberately injected through feedback mechanisms. Malicious data introduced through these loops gets absorbed into the next model version. This is online poisoning, and it works precisely because the system is structurally designed to trust its own inputs. The attack exploits the feature, not a bug.

Two pipeline examples

The stages above become easier to reason about with concrete systems. Two examples illustrate how this architecture looks in practice and where the attack surface sits in each case.

E-commerce recommendation engine

An e-commerce platform building product recommendations collects user activity streamed through Kafka alongside review text. Raw data lands in a data lake on AWS S3. Apache Spark processes it, reconstructing user sessions and running sentiment analysis on reviews, then outputs processed files. Inside AWS SageMaker, a recommendation model trains on that data. The model is serialised as a pickle file, stored back on S3, and deployed through a containerised API on Kubernetes. Monitoring tracks click-through rates, while user feedback feeds into periodic retraining cycles to keep the model current.

Every stage in that flow is a potential entry point for an attacker, from the Kafka stream and S3 bucket through to the Spark jobs, the serialised model file, and the feedback-driven retraining loop.

Healthcare diagnostic tool

A healthcare provider building a predictive diagnostic tool faces a tighter version of the same architecture. Anonymised patient images and clinical notes come from internal hospital systems. Storage is encrypted and access-controlled. Python scripts standardise images and extract text features before a PyTorch training run on specialised hardware. The validated model deploys via an internal API to a clinical support system. Retraining happens less frequently and under stricter governance.

The fundamental problem persists. Incorporating new data still requires trusting that the incoming data is legitimate. The stakes are higher because the model’s output influences patient care, but the pipeline’s structural vulnerabilities are identical. Tighter controls reduce the likelihood of compromise at each stage without eliminating the underlying architectural risk.

Summary

The AI data pipeline follows a consistent architecture across domains: collect, store, process, model, deploy, monitor, retrain. Each stage introduces its own attack surface, from collection’s lack of inherent validation through to the retraining loop’s structural trust in its own inputs. The pipeline does not distinguish between legitimate data and data an attacker planted. Security has to be layered across every stage because no single point catches everything, and the most dangerous attacks, such as online poisoning through feedback loops, exploit the system’s own design rather than breaking it.