Highlights:
- Introduces SmolVLM2 variants for blind and low-vision accessibility research.
- Develops two novel evaluation frameworks – Multi-Context BLV and Navigational Assistance.
- Assesses 500M and 2.2B parameter models on AVCaps (outdoor) and Charades (indoor) datasets.
- Optimizes models for smartphones using FP32 and INT8 precision variants.
TLDR:
A research team led by Shruti Singh Baghel and collaborators has proposed lightweight Vision-Language Models and new evaluation frameworks to improve real-world accessibility for blind and low-vision users, enabling efficient AI deployment on mobile devices.
In a significant step toward inclusive artificial intelligence, researchers Shruti Singh Baghel, Yash Pratap Singh Rathore, Sushovan Jena, Anurag Pradhan, Amit Shukla, Arnav Bhavsar, and Pawan Goyal have unveiled a novel study titled *“Towards Blind and Low-Vision Accessibility of Lightweight VLMs and Custom LLM-Evals.”* This research, published on arXiv (arXiv:2511.10615v1), addresses one of the biggest challenges in modern AI—making Vision-Language Models (VLMs) practical for users with visual impairments. While large VLMs demonstrate remarkable ability to interpret and describe complex visual scenes, their massive computational requirements often limit deployment, especially on mobile and low-power devices used by blind and low-vision (BLV) individuals.
The study evaluates two compact SmolVLM2 model variants containing 500 million and 2.2 billion parameters respectively across two comprehensive datasets: AVCaps, representing outdoor scenarios, and Charades, depicting indoor environments. The researchers’ primary objective was to determine how smaller models perform in generating detailed, accessible, and context-aware descriptions tailored for BLV use cases while maintaining efficiency suitable for portable devices. Notably, the study introduces two bespoke evaluation systems — the *Multi-Context BLV Framework*, which measures understanding across spatial orientation, social interaction, action events, and ambient context; and the *Navigational Assistance Framework*, designed to assess mobility-relevant information essential for orientation and movement.
From a technical perspective, the work delves into performance comparisons between FP32 and INT8 precision variants to measure how quantization impacts descriptive quality and processing speed on smartphones. This real-world evaluation offers new benchmarks for running VLMs in resource-limited environments. The team also experimented with four distinct prompt strategies, providing deeper insights into optimizing description generation under varying accessibility goals. The result is a comprehensive foundation for developing scalable, mobile-first AI tools that make visual content more interpretable for users with vision impairments. Their findings may influence future progress in assistive AI, empowering devices to deliver richer, safer, and more context-aware interactions for millions of visually impaired individuals worldwide.
By integrating cutting-edge computer vision and natural language understanding while addressing the practical deployment gap, the research marks an important contribution to the domain of accessible computing. It bridges the divide between academic innovation and real-world usability, paving the way for the next generation of inclusive AI applications designed to function seamlessly on everyday consumer hardware.
Source:
Source:
Original research paper on arXiv: https://doi.org/10.48550/arXiv.2511.10615