Researchers Develop System To Make Memes Accessible For The Visually Impaired

Meme reading

The internet has social media networks which are a world on their own.

While social media networks were once tools to keep users in touch with others, they have evolved to become much more than that. Scrolling down feeds, there will be memes after memes. Within those platforms, memes thrives.

While emojis changed the way people communicate in the online culture, memes put communication to a whole different level.

Memes are images that are copied and then overlaid with slight variations of text. They are often humorous and convey a shared experience, Their popularity is caused by the fun and joy they bring. For those who have the vision to enjoy them, yes, they are indeed hilarious.

But for the visually impaired, which accounted to more than 1 billion people around the world, memes don't mean anything to them.

Researchers from Carnegie Mellon University are changing that, by developing a system to make memes more accessible for everyone.

"If you're blind, you miss that part of the conversation," said Cole Gleason, a Ph.D. student in CMU's Human-Computer Interaction Institute (HCII.)

"Memes may not seem like the most important problem, but a vital part of accessibility is not choosing for people what deserves their attention," said Jeff Bigham, an associate professor in the HCII. "Many people use memes, and so they should be made accessible."

In the study “Making Memes Accessible,” six researchers trained a system to classify and translate memes with up to 92% accuracy.

Visually impaired and blind people typically use screen readers, built-in accessibility features, and even outsource a pair of eyes to help them navigate the web. But these tools are limited. With the rise of memes, screen readers struggle to detail the full picture of memes.

According to the study, social platforms like Facebook, Instagram, and Twitter allow users to add alternative text to their images, but most users who upload memes aren’t aware of this feature, or they simply don’t use it.

As a matter of fact, only 0.1% of images on the web are accessible by screen readers and accessible to the visually impaired people.

Meme reading

To change this, the Carnegie Mellon system scans through memes found on the web, and cross-check them with a growing database of meme templates to output three formats: meme text only, an alt-text + meme text pair, and an audio macro meme.

With the system, the researchers provide alternative forms of meme descriptions, that include audio, alt-text, and text templates.

While this system should indeed help those visually impaired people, the system has some drawbacks.

The most prominent, is its ability to cross-check. With the increasing speed people are creating memes, the system cannot keep up with the growing library of online memes. While technology like this has the power to make the internet more inclusive, the system isn't reliable on its own.

This is why social media platforms should encourage their users to add alt-text when posting memes, just to make them accessible to everyone, no matter what their conditions are.

The meme culture has evolved into spreading joy online and relating to something in pop culture. To make the whole web more accessible, platforms should also be looking into ways and the technologies to prevent visually impaired people from missing out and feeling isolated.

Internet grows as a society, and this particular reason means that everyone should be more inclusive.