.png){kind=icon}
返回
Benchmarking large and small MLLMs
.png){kind=icon}
OA
AI
DOI：10.1007/s00138-025-01762-0.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Large multimodal language models (MLLMs) such as GPT-4V and GPT-4o have achieved remarkable advancements in understanding and generating multimodal content, showcasing superior quality and capabilities across diverse tasks. However, their deployment faces significant challenges, including slow inference, high computational cost, and impracticality for on-device applications. In contrast, the emergence of small MLLMs, exemplified by the LLava-series models and Phi-3-Vision, offers promising alternatives with faster inference, reduced deployment costs, and the ability to handle domain-specific scenarios. Despite their growing presence, the capability boundaries between large and small MLLMs remain underexplored. In this work, we conduct a systematic and comprehensive evaluation to benchmark both small and large MLLMs, spanning general capabilities such as object recognition, temporal reasoning, and multimodal comprehension, as well as real-world applications in domains like industry and automotive. Our evaluation reveals that small MLLMs can achieve comparable performance to large models in specific scenarios but lag significantly in complex tasks requiring deeper reasoning or nuanced understanding. Furthermore, we identify common failure cases in both small and large MLLMs, highlighting domains where even state-of-the-art models struggle. We hope our findings will guide the research community in pushing the quality boundaries of MLLMs, advancing their usability and effectiveness across diverse applications.
Keyword:
Multi-modal large language model
Evaluation
General capabilities
Application
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
M
IF:
2.3
论文数:
95
被引数:
2.7K
机构
引用论文
.png){kind=icon}
Towards General Purpose Vision Systems: An End-to-End Task-Agnostic Vision-Language Architecture
PREAI
分享收藏
分享收藏DenseCap: Fully Convolutional Localization Networks for Dense CaptioningDenseCap: 用于密集字幕的完全卷积定位网络OAAI
OAAI分享收藏
查看更多内容