UAE's Falcon 40B Dominates Leaderboard: Ranks #1 Globally in Latest Hugging Face Independent Verification

Falcon 40B, the UAE's first large-scale open-source, 40-billion-parameter AI model launched by Abu Dhabi's Technology Innovation Institute (TII) last week, soared to the top spot on Hugging Face's latest Open Large Language Model (LLM) Leaderboard. Hugging Face, an American company seeking to democratize artificial intelligence through open-source and open science, is considered the world's definitive independent verifier of AI models.

This press release features multimedia. View the full release here: https://www.businesswire.com/news/home/20230529005045/en/

Falcon 40B ranks 1st globally in Hugging Face Open LLM Leaderboard. (Graphic: AETOSWire)

Falcon 40B managed to beat back established models such as LLaMA from Meta (including its 65B model), StableLM from Stability AI, and RedPajama from Together to achieve the coveted ranking. The index utilizes four key benchmarks from the Eleuther AI Language Model Evaluation Harness, a consolidated framework that assesses generative language models on: the AI2 Reasoning Challenge (25-shot), a set of grade-school science questions; HellaSwag (10-shot), a test of common sense inference, which is easy for humans but challenging for SOTA models; MMLU (5-shot), a test to measure a text model's multitask accuracy; and TruthfulQA (0-shot), a test to measure whether a language model is truthful in generating answers to questions.