Microsoft Teams up With Non-Profit Organizations To Develop Inclusive AI Models

Microsoft has partnered with non-profit organizations and researchers to develop AI products and services that are more inclusive of people with disabilities.

Smartphones have transformed the lives of the visually impaired and those with low vision. Thanks to emerging technologies, such as AI, facial recognition, cloud, IoT, visually impaired people can now perform multiple tasks independently. Over the years, big tech companies and booming startups have developed promising apps, wearable devices, and services to cater to the visually impaired population. 

After the launch of Apple’s VoiceOver and Google’s TalkBack, visually impaired people were able to interact with their smartphones easily. Gradually, a host of popular apps, including Microsoft’s Seeing AI, Google’s Lookout TapTapSee, Be My Eyes, and KNFB reader, enhanced the quality of those with disabilities. 

While a slate of apps and AI-powered services have been developed, the technology industry is still struggling to develop a robust, inclusive, and personalized AI system that helps visually impaired or disabled people live independently. Addressing the disparity, Microsoft recently announced three partnerships with non-profit organizations and researchers. 

1. Object Recognition for Blind Image Training (ORBIT) Launched

Microsoft and City, University of London have collaborated to launch object recognition for blind image training (ORBIT) research project to build new AI systems for personalized object recognition. The Microsoft-funded project will source a richer set of data directly from visually impaired users and construct a large dataset to help researchers build inclusive AI algorithms in apps, such as Seeing AI or TapTapSee and novel wearable systems. 

The dataset will also be publicly available in two phases: 

• Phase 1 will include around100 users and thousands of videos.
• Phase 2 will consist of 1,000 users and contain more than 10,000 videos. 

On successful implementation, the project will help the visually impaired and people with low vision identify personal items easily.

Simone Stumpf, senior lecturer at the center for human-computer interaction design at City, University of London, who leads ORBIT, saidOpens a new window , “Without data, there is no machine learning, and there’s really been no dataset of a size that anyone could use to introduce a step-change in this relatively new area of AI.”

2. Team Gleason and Microsoft Partnership on Project Insight

Microsoft and Team Gleason Foundation, founded by former NFL player Steve Gleason, launched Project Insight to build a computer vision dataset for people living with amyotrophic lateral sclerosis (ALS). Team GleasonOpens a new window and Microsoft Health Next Enable will create an open dataset, including the facial imagery of people living with ALS, to train and build inclusive AI models. 

For this project, a medical history questionnaire will be sent to the participants. Furthermore, the participants will be prompted through an app to submit images of themselves using their computers. An estimated 5 TB of anonymized data will be collected and shared with researchers in data science collectives, such as GitHub and Kaggle.

Blair Casey, chief impact officer, Team Gleason, saidOpens a new window , “ALS progression can be as diverse as the individuals themselves, so accessing computers and communication devices should not be one size fits all. We will capture as much information as possible from 100 people living with ALS, so we can develop tools for all to effectively use.”

3. Expansion of VizWiz

The third project is between Microsoft and the University of Texas, Austin. Both organizations will improvise VizWizOpens a new window , a talking mobile app that offers nearly real-time answers to visually impaired people’s questions. App creator Danna Gurari and her team have worked with Microsoft to develop a new public dataset to train, validate, and test image captioning algorithmsOpens a new window . 

This week, on October 14, 2020, Microsoft announced the development of a new AI system for image-captioning that describes images with human accuracy. This has been a significant milestone for the Redmond giant in building AI systems that are inclusive of people with disabilities. The company implemented a visual vocabulary pre-training approach to large AI models and fine-tuned it for better image captioning. 

Xuedong Huang, chief technology officer, Microsoft Azure AI Cognitive Services, saidOpens a new window , “Image captioning is one of the core computer vision capabilities that can enable a broad range of services. The visual vocabulary pre-training essentially is the education needed to train the system, we are trying to educate this motor memory.”

The new AI system has been incorporated in the Seeing AI app — it will be rolled out in other Microsoft products, such as Microsoft Word and Outlook for Windows and Mac, and PowerPoint for Windows, Mac, and web by the end of 2020.

Smart technology has greatly enhanced the lives of visually impaired people. It has empowered them to interact and navigate with the world in unique ways. With some promising partnerships, Microsoft is further driving innovation and improving accessibility for people with disabilities. However, creating inclusive AI models is an industry-wide problem that won’t be solved by one project or organization alone. Now is the time for companies to collaborate and develop empowering products to simplify the lives of people living with disabilities.

Do you think artificial intelligence can promote inclusive prosperity for all? Comment below or let us know on LinkedInOpens a new window , TwitterOpens a new window , or FacebookOpens a new window . We’d love to hear from you!