1FAIR, Meta, 2HuggingFace, 3AutoGPT, 4GenAI, MetaWe introduce GAIA, a benchmark for General AI Assistants that, if solved, would represent amilestone in AI research. GAIA proposes real-world questions that require a set of fundamentalabilities such as reasoning, multi-modality handling, web browsing, and generally tool-use proficiency.GAIA questions are conceptually simple for humans yet challenging for most advanced AIs: weshow that human respondents obtain 92% vs. 15% for GPT-4 equipped with plugins. This notableperformance disparity contrasts with the recent trend of LLMs outperforming humans on tasksrequiring professional skills in e.g. law or chemistry. GAIA’s philosophy departs from the currenttrend in AI benchmarks suggesting to target tasks that are ever more difficult for humans. We positthat the advent of Artificial General Intelligence (AGI) hinges on a system’s capability to exhibitsimilar robustness as the average human does on such questions. Using GAIA’s methodology, wedevise 466 questions and their answer. We release our questions while retaining answers to 300 ofthem to power a leader-board hereby accessible.Date: November 23, 2023Correspondence: {gmialon,tscialom}@meta.com, clementine@huggingface.coCode: https://huggingface.co/gaia-benchmark