Microsoft has released a major update for its AI-based application for the visually impaired. Intended for the blind or those who have a low vision, Seeing AI has a list of features that is obviously intended to provide a wide range of features. This latest update apparently adds the ability to explore to the touch the photographs already taken. Artificial intelligence is used to analyze a scene and map specific objects in the photo. Swipe the image so that the phone detects nearby objects and identifies them.
If you have ever tried using a screen reader, you probably know how bad the tech cards are. Although they work pretty well for some applications, screen readers do not fit very easily in the modern web. A certain number of authors sure accessibility have talked about it pretty good. The bottom line is that assistive technology can absolutely use a boost. I looked at Seeing AI, hoping it would indicate a better future. After spending some time on it, I would say that it has a real promise and some problems that prevent recommending a reliable source of information right away. Since the purpose of the application is to identify things, this can be worrying.
- Short text
- Product (barcode search)
- Nobody (can be taught to recognize individuals)
- Light (uses audio cues to tell you when the camera is pointing to the brightest light source it can see)
The photo identification and analysis feature was the best we wanted to test today, because that's what Microsoft added to this update, but we also tested other functions. The application works well enough to identify documents and text, especially if you are using a white background. We could not correctly identify any product barcode; this aspect of the application always failed with a message "Sorry, something went wrong". When we tried to use the "Scene" function, the system often misidentified the furniture, for example by taking a dresser for a desk.
As for the photography section, well … it might be easier to show you than to describe it. Allowing AI to analyze various images to search for content has yielded some interesting results. The slideshow below contains our results. I threw various images on the app to analyze what she could identify and what she could not.
The overall performance was just not very good. The analysis of the text was quite strong and the application read the content well, but everything else was a child's game. See AI does not distinguish very easily dogs and cats. He can not tell if there is a plaque on the stove. He can not identify different foods.
With regard to tactile photographic analysis, this feature is also lacking.
Seeing that artificial intelligence was correct, it was a cat, although he confused Alisdair with Persian (it was mainly Maine Coon). He even insisted on "reading" the V in his fur as a true V. (He also pulled this trick on the rug patterns, insisting that a rug in the dining room was labeled AAAAAAAAAAA repeating). The application describes it as a "sleeping Persian cat", which is not so bad. But this level of description is rare. Most of the time, you get "Person" and a big blue box surrounding a single individual. The app will tell you how many objects it recognizes, but it does not really tell you where the objects are. are. You must draw your finger all over the screen hoping to touch the object that the camera has seen. Only one object at a time is highlighted in blue (as above) and it is always the last object you touched.
It does not make any difference to the blind, who will have to explore the image in any case, but it is surprising that Microsoft did not realize that visually impaired users would also be interested in seeing all the boxes where objects have have been detected in a given image. immediately, rather than looking for them individually by blindly searching for the fingers. Several photos also came back with "No information detected", without any data on the reason for this problem or how we could solve the problem.
Overall, we give the app an "A" for the efforts. It is really trying to provide exciting new features in an area where they are sorely needed. The current execution is however still sorely lacking. It is clear that these models require more training before anyone can really rely on them to navigate or identify an object. The objects were not identified correctly, they were not placed in the context of the photo and the many problems that would have the knowledge of artificial intelligence with the depth would make us nervous if we were really trying to use the app to navigate the physical world. Tell a blind man that there is a plate Above a pan on the stove when no such plate exists, it can be downright dangerous. The ability to browse photos to the touch, though interesting, was ultimately not accurate enough to provide a compelling case of use.
We support what Microsoft is trying to do here. We understand the difficulty and hope that the company will continue. But seeing the AI can be useful for specific things, as it was a proper identification of the currency (I could not test this aspect), he did not impress us with his ability to interpret the awakened world
AI modular wheelchairs can monitor obstacles, incorporate head tracking
Microsoft's new Adaptive Controller is for people with mobility issues
Chinese researchers use Kinect to translate sign language into text