CLEARING-IT

Cost-effective machine unlearning with privacy guarantees

Personal data are commonly employed for training conventional ML models. Regarding FMs, they owe their success to being trained on large-scale diverse data, potentially sensitive or copyrighted (usually text scraped from the web for LLMs and any type of data for multimodal FMs). For personal data, individuals may invoke the right to be forgotten (RTBF) to request removal of their data used to train an ML model. The RTBF is recognized by several data protection regulations, such as the EU General Data Protection Regulation and the EU AI Act. Similarly, copyright owners may request removal of the information they own from the data used to train an ML model.

Goals

The overarching aim of CLEARING-IT is to achieve efficient unlearning with privacy guarantees, both in conventional Machine Learning models and in Large Language Models / Foundation Models

Privacy guarantees in conventional machine learning

We will develop efficient unlearning methods with privacy guarantees for conventional ML models. The framework will be general enough to accommodate any privacy model. The utility and privacy achieved must be comparable to exact unlearning, with the computational cost being substantially lower.

  • We will train the ML model on both protected and original data, and in a way that is amenable to enforce unlearning requests
  • We will design the processing of forgetting requests in a way that is scalable as the number of requests increases

Privacy guarantees in FMs

We will seek efficient unlearning with privacy guarantees for foundation models.

  • We will seek to enforce a privacy model on the training data, with the important difference that such training data now consist of large amounts of unstructured web data.
  • We will seek to precisely locate the undesired knowledge in internal components of the FM.
  • We will design efficient target interventions on the identified components to unlearn the undesired knowledge, and we will validate unlearning empirically.
  • We will seek to make the unlearning interventions compliant with a privacy model, in order to obtain formal unlearning guarantees.

Efficient tools to enforce privacy models on unstructured and multimedia data

We will design efficient and utility-preserving methods to enforce privacy models on unstructured and multimedia data, which constitute the vast majority of training data used by FMs

  • We will focus on unstructured text data.
  • We will focus on images.

Improving the effective privacy of approximate unlearning via explainability

Approximate unlearning methods are the most numerous in the literature. However, the unlearning they achieve may be insufficient because they offer no guarantee. We plan to resort to explainable AI to improve the effective privacy achieved by approximate unlearning.

  • We will leverage AI explainability techniques to guide the improvement of approximate unlearning in an iterative manner
  • We will extend this iterative, explainability-based approach to improve the effectiveness of approximate unlearning in FMs.

Team

Main researchers

Dr. Josep Domingo-Ferrer

Universitat Rovira i Virgili josep.domingo (at) urv.cat 0000-0001-7213-4962

Dr. David Sánchez

Universitat Rovira i Virgili david.sanchez (at) urv.cat 0000-0001-7275-7887

Research team

  • Dr. Alberto Blanco Justicia
  • Dr. Jesús A. Manjón Paniagua

Work team

  • Dra. Montserrat Batet, URV
  • Dr. Najeeb Jebreel, URV
  • Dr. Benet Manzanares, URV
  • Dr. Carles Domingo-Escrich, Microsoft Research New England, USA
  • Dr. Pierre Lison, Norwegian Computing Center, Norway
  • Dr. Nuno Moniz, IBM Tech Ethics Lab, University of Notre Dame, USA
  • Dr. Anna Monreale, University of Pisa, Italy
  • Dr. Melek Önen, Eurécom, France
  • Sara Lanuza
  • Mona Khalil
  • Faisal Ahmed
  • Tamim Al Mahmud