It\u2019s also now available to app developers through the Computer Vision API in Azure Cognitive Services, and will start rolling out in Microsoft Word, Outlook, and PowerPoint later this year.<\/span><\/p>\n The model can generate \u201calt text\u201d image descriptions for web pages and documents, an important feature for people with limited vision that\u2019s all-too-often unavailable.<\/p>\n \u201cIdeally, everyone would include alt text for all images in documents, on the web, in social media \u2013 as this enables people who are blind to access the content and participate in the conversation,\u201d said Saqib Shaikh, a software engineering manager at Microsoft\u2019s AI platform group. \u201cBut, alas, people don\u2019t. So, there are several apps that use image captioning as [a] way to fill in alt text when it\u2019s missing.\u201d<\/p>\n [Read: Microsoft unveils efforts to make AI more accessible to people with disabilities<\/a>]<\/em><\/p>\n<\/div>\n<\/div>\n<\/section>\n The algorithm now tops the leaderboard of an image-captioning benchmark called nocaps<\/a>. Microsoft achieved this by pre-training a large AI model on a dataset of images paired with word tags \u2014 rather than full captions, which are less efficient to create. Each of the tags was mapped to a specific object in an image.<\/span><\/p>\n The pre-trained model was then fine-tuned on a dataset of captioned images, which enabled it to compose sentences. It then used its \u201cvisual vocabulary\u201d to create captions for images containing novel objects.<\/p>\n Microsoft said the model is twice as good as the one it\u2019s used in products since 2015. The image below shows how these improvements work in practice:<\/p>\n However, the benchmark performance achievement doesn\u2019t mean the model will be better than humans at image captioning in the real world. Harsh Agrawal, one of the creators of the benchmark, told The Verge<\/a> that its evaluation metrics \u201conly roughly correlate with human preferences\u201d and that it \u201conly covers a small percentage of all the possible visual concepts.\u201d<\/p>\n Published October 15, 2020 \u2014 11:21 UTC <\/p>\n![](\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2020\/10\/Screenshot-2020-10-15-at-11.39.02.png\")