I think I can define myself as an enthusiastic and open-minded person. I am interested in a wide variety of things, from nature to art, of course the human being and especially science and technological progress. Clearly, I am invested in the future. I study and prepare myself for any situation because I believe in knowledge more than in improvisation. I also have a high regard for human beings, and I try to ensure that my loved ones evolve in a favourable environment.
Find me on
,
and
.
Publications
-
What Makes Multimodal In-Context Learning Work? - CVPR 2024
Folco Bertini Baldassini, Mustafa Shukor, Matthieu Cord, Laure Soulier, Benjamin Piwowarski
Large Language Models have demonstrated remarkable performance across various tasks, exhibiting the capacity to swiftly acquire new skills, such as through In-Context Learning (ICL) with minimal demonstration examples. In this work, we present a comprehensive framework for investigating Multimodal ICL (M-ICL) in the context of Large Multimodal Models. We consider the best open-source multimodal models (e.g., IDEFICS, OpenFlamingo) and a wide range of multimodal tasks. Our study unveils several noteworthy findings: (1) M-ICL primarily relies on text-driven mechanisms, showing little to no influence from the image modality. (2) When used with advanced-ICL strategy (like RICES), M-ICL is not better than a simple strategy based on majority voting over context examples. Moreover, we identify several biases and limitations of M-ICL that warrant consideration prior to deployment.
-
Cross-Attention Watermarking of Large Language Models - ICASSP 2024
Folco Bertini Baldassini, Huy H. Nguyen, Ching-Chung Chang, Isao Echizen
A new approach to linguistic watermarking of language models is presented in which information is imperceptibly inserted into the output text while preserving its readability and original meaning. A cross-attention mechanism is used to embed watermarks in the text during inference. Two methods using cross-attention are presented that minimize the effect of watermarking on the performance of a pretrained model. Exploration of different training strategies for optimizing the watermarking and of the challenges and implications of applying this approach in real-world scenarios clarified the tradeoff between watermark robustness and text quality. Watermark selection substantially affects the generated output for high entropy sentences. This proactive watermarking approach has potential application in future model development.
Mastodon