Huge thanks to everyone who left insightful replies in this thread where I posted about the initial results of my study, which I will cite throughout this article.
It is interesting to see how different fediverse apps approach image description. Ice Cubes, for example, has an AI-powered image description generator. Ivory uses the built-in iOS Live Text feature to detect text in images (as of version 1.4).
Toot!, on the other hand, doesn’t use any fancy technologies, but it makes it harder to opt-out of only posting media with alt text. And the official Mastodon apps have an option to remind you if you forget to add it.
I was curious about the effects of which client people use to post in the fediverse on the presence of image descriptions.
Results
I analyzed 29,856 posts from mastodon.social, and here are the highlights. (Full dataset, which contains no personal information or the contents of the posts, is available here.)
The full dataset is available on kaggle.com, here are some highlights.
| Client app | Percentage of posts where all media has alt text | Note |
| Web | 17.4% | This is the main Mastodon web app. |
| Mastodon for Android | 14.3% | The official Android app. |
| Mastodon for iOS | 21.6% | The official iOS app. |
| Tusky | 33.4% | A free and open-source Mastodon client for Android. |
| Ice Cubes App | 53.7% | A free and open-source Mastodon client for iOS. As mentioned earlier, it lets you use AI to generate image descriptions. |
| Ivory for iOS | 40.4% | iOS version of the Ivory app. |
| Ivory for Mac | 40% | Mac version of the Ivory app. |
| Mona for iPhone | 30.1% | An iOS Mastodon app “packed with customization features”. |
| Mona for Mac | 64.3% | Mac version of the Mona app. |
| Metatext | 34.1% | An open-source Mastodon app for iOS. |
| Fedilab | 44.9% | A multi-account client for Mastodon, Pleroma, Friendica and Pixelfed. |
| Toot! | 13.8% | A Mastodon app for Android and iOS. |
| Phanpy | 63.6% | An alternative web client for Mastodon. |
| Buffer | 15.8% | A platform for sharing content across social media sites, including Mastodon. |
| Zoho Social | 0.4% | A platform for sharing content across social media sites, including Mastodon. |
| Cheap Bots, Toot Sweet! | 1% | A platform for making Mastodon bots. |
An earlier version of the analysis that looks at the broader fediverse beyond Mastodon is available here. Note that the client app information is missing for a large portion of the posts as per the earlier explanation.
Methodology
I used Mastodon’s API to read the public timeline on the mastodon.social instance. I only analyzed local posts from this instance, as the information about the client app used is not available for remote posts. This does have the benefit of removing the influence of the fact that the use of alt text differs between communities.
The URL of the API request has the following format:
https://mastodon.social/api/v1/timelines/public?limit=40&local=true&only_media=true- the
local=trueparameter filters out posts coming from instances other than mastodon.social - passing
only_media=truewill let you filter out posts that don’t have images or videos attached - and
limit=40increases the default limit of 20 posts fetched per request
The results need to be paginated through, and the API requests should be spaced out appropriately.
The full script used to collect and analyze the data will be released at a later date.
Conclusions
While some apps implement simple text detection from uploaded images, this technology is not useful for images that contain more than just text. Apps that provide AI-based image detection, either through functionality provided by the platform they run on, or custom integrations, do have a higher rate of image descriptions.
The alternative web client for Mastodon, Phanpy, stands out over the other clients, and looking at the post compose dialog, one possible explanation is the prominent field for adding image description.
It is worth noting that the presence of an alt text does not imply its quality, so a separate study would have to be done on that.
Interesting to compare the use of alt text based on which apps people use.
https://stefanbohacek.com/blog/impact-of-fediverse-clients-on-the-use-of-alt-text/