{"id":24135,"date":"2024-10-11T14:22:58","date_gmt":"2024-10-11T13:22:58","guid":{"rendered":"https:\/\/www.riskinsight-wavestone.com\/?p=24135"},"modified":"2024-10-11T14:22:59","modified_gmt":"2024-10-11T13:22:59","slug":"data-poisoning-a-threat-to-llms-integrity-and-security","status":"publish","type":"post","link":"https:\/\/www.riskinsight-wavestone.com\/en\/2024\/10\/data-poisoning-a-threat-to-llms-integrity-and-security\/","title":{"rendered":"Data Poisoning: a threat to LLM’s Integrity and Security"},"content":{"rendered":"\n

Large Language Models (LLMs) such as GPT-4 have revolutionized Natural Language Processing (NLP) by achieving unprecedented levels of performance. Their performance relies on a <\/span>high dependency of various data<\/span><\/b>: model training data, over-training data and\/or Retrieval-Augmented Generation (RAG) enrichment data. However, this dependence on data not only constitutes a pillar for improving the performance of any AI system, but also a <\/span>vector for attacks <\/span><\/b>enabling these models to be compromised. <\/span>\u00a0<\/span><\/p>\n

Poisoning attacks disrupt the behavior of an AI system by introducing corrupted data into the learning process. These attacks are one of the best-known families of attacks that can compromise a model. And this is far from a new topic. In 2017, researchers demonstrated that this method could corrupt autonomous cars to cause them to mistake a “stop” sign for a speed limit sign.<\/span>\u00a0<\/span><\/p>\n

This article focuses specifically on poisoning attacks on AI systems, with particular attention to their impact on LLM models.<\/span>\u00a0<\/span><\/p>\n

\u00a0<\/span><\/p>\n

Data Poisoning: What Does it all Mean?<\/span>\u00a0<\/span><\/h2>\n

Data poisoning is an attack aimed at corrupting AI model data. <\/span>This data is intended to mislead the system <\/span><\/b>into making incorrect predictions. <\/span>\u00a0<\/span><\/p>\n

The impacts are varied: degraded performance (biased response, offensive comments, etc.), introduction of vulnerabilities (backdoors that change the model’s behaviour), hijacking of the model. For example, a compromised model used in a customer service department could promise compensation or offend customers, while an anti-virus classification model could let through threats that resemble the injected fish.\u00a0<\/span>\u00a0<\/span><\/p>\n

Once a training dataset is corrupted and the model trained, <\/span>it is difficult, if not almost impossible, to correct the problem<\/span><\/b>. It is therefore important to ensure the integrity of the data and to incorporate anti-fish controls from the outset of the system design.<\/span>\u00a0<\/span><\/p>\n

How do you Poison a Model?<\/span>\u00a0<\/span><\/h2>\n

There are several possible techniques for poisoning data:<\/span>\u00a0<\/span><\/p>\n

Technique 1: Inverting labels<\/span><\/b>\u00a0<\/span><\/h3>\n

During Training\u00a0<\/em><\/p>\n

Label inversion involves assigning incorrect labels to the training data. Consider a model that classifies items according to their sentiment (positive, neutral or negative). During training, the model associates specific text features with sentiment labels. By inverting the data labels, the model learns from false examples, thereby degrading its performance. Here is an example of data with inverted labels:<\/span>\u00a0<\/span><\/p>\n