While the World Wide Web is colorful and extremely engaging, not everyone can experience that to the fullest.
Among those people who cannot experience the web as it is, include the vision impaired and blind people. Because these people, despite having the lacks, deserve to have an equal share of the web, Microsoft Edge is delivering an update that should improve accessibility for those users.
Traditionally, the blind and people with low vision could use screen readers, which can automatically read text and image content as speech or braille output. Screen readers are essential to those people, as well as to those who have a learning disability.
However, when dealing with pictures displayed on websites, screen readers can be rendered useless when dealing with unlabeled images.
While web owners and webmasters are urged to add 'Alt text' to images on their websites, Microsoft found that more than half of the images on the web are missing these image descriptions.
Realizing this, Microsoft updates its Edge with an image-recognition technology, so the browser can read out what's inside an unlabeled image, automatically.
"When a screen reader finds an image without a label, that image can be automatically processed by Machine Learning (ML) algorithms to describe the image in words and capture any text it contains," said Travis Leithead, a Program Manager on Microsoft’s Edge platform team, in the video.
"The algorithms are not perfect, and the quality of the descriptions will vary, but for users of screen readers, having some description for an image is often better than no context at all."
But before the AI can work, it needs users' permission.
After the user has granted permission, Microsoft Edge will send unlabeled images to Azure Cognitive Services’ Computer Vision API for processing. The AI supports common web image formats, like JPEG, PNG, GIF, WEBP, etc., as explained by Microsoft in a blog post.
In all, the AI can analyze images and create descriptive summaries in 5 languages, and recognize text inside of images in over 120 languages.
On the good side, the AI will not read out images that are meant to be part of a website's design, and images that are smaller than 50 x 50 pixels.
This should help avoid confusion to the user, and negate unnecessary information.
The downside is that, it won't read out excessively large images, images that are categorized as pornographic in nature, gory, or sexually suggestive images.
Images in these categories will not be sent to the auto-image caption service, nor provided to the screen reader.
"Additionally, the feature can be controlled through an enterprise policy setting. Under the policy name AccessibilityImageLabelsEnabled there is an option to disable the feature," said Microsoft.
This is an important feature, especially for blind and people with low vision, tech giant Microsoft is adding a new feature to its Edge browser that will automatically add “Alt Text” to unlabelled images.
Alternative Text or simply Alt text, is known as image label.
It can be added by web developers to describe images, charts and graphics.
The thing is, screen readers will read out "unlabeled graphic" whenever they encounter images without the label.
With image recognition AI, Microsoft wants to improve its Edge browser by increasing its accessibility feature.
It's worth noting though, that the AI will prioritize any image label a site is providing (if any is available), over generating its own Alt text.
This is because Microsoft knows algorithms are far from perfect, and also because of the fact that only the site author that fully understands the context and intent of the image and its creative meaning.
What's more, Microsoft doesn't want its AI to be viewed as a reason for authors to avoid properly labeling images.