Belfast Telegraph

UK Website Of The Year

CaptionBot: Microsoft AI describes what's in pictures - how did it handle famous faces from Northern Ireland?

Published 14/04/2016

CaptionBot uses a number of complex programs to analyse and caption images Microsoft
CaptionBot uses a number of complex programs to analyse and caption images Microsoft

Microsoft's CaptionBot is an artificial intelligence (AI) system capable of analysing and describing pictures automatically.

The program, now available to try on the company's website, is from the Microsoft Cognitive Services team, and it works surprisingly well - most of the time. We tested it with a few faces that are well-known to people in Northern Ireland and the results were mixed.

CaptionBot uses a number of complex programs to analyse and caption images Microsoft
CaptionBot uses a number of complex programs to analyse and caption images Microsoft
CaptionBot uses a number of complex programs to analyse and caption images Microsoft
CaptionBot uses a number of complex programs to analyse and caption images Microsoft

Users can upload any image to CaptionBot and have it return a description in seconds. It manages this by using image and face analysing programs, as well as a language processor which returns descriptions in understandable English.

The program isn't entirely spot-on all the time, but it's generally close enough to be impressive.

CaptionBot is similar to previous Microsoft AI programs, like 'How Old Do I Look?' and 'What Dog?', both of which used similar technology.

Image captioning software is nothing new, but the way Microsoft has made the program public gives everyone a chance to see how it works.

CaptionBot should get more accurate over time - users can rate how well it did at captioning pictures on a one-to-five scale, and it will slowly 'learn' to identify different elements as it gets fed more content.

A word of warning - Microsoft says it will hold on to all images uploaded to CaptionBot, in order to improve its capabilities in future. However, it insists it won't record any personal information about users.

CaptionBot uses a number of complex programs to analyse and caption images Microsoft
CaptionBot uses a number of complex programs to analyse and caption images Microsoft
CaptionBot uses a number of complex programs to analyse and caption images Microsoft
CaptionBot uses a number of complex programs to analyse and caption images Microsoft
CaptionBot uses a number of complex programs to analyse and caption images Microsoft
CaptionBot uses a number of complex programs to analyse and caption images Microsoft
CaptionBot uses a number of complex programs to analyse and caption images Microsoft
CaptionBot uses a number of complex programs to analyse and caption images Microsoft
CaptionBot uses a number of complex programs to analyse and caption images Microsoft
CaptionBot uses a number of complex programs to analyse and caption images Microsoft
CaptionBot uses a number of complex programs to analyse and caption images Microsoft
CaptionBot uses a number of complex programs to analyse and caption images Microsoft
CaptionBot uses a number of complex programs to analyse and caption images Microsoft
CaptionBot uses a number of complex programs to analyse and caption images Microsoft
CaptionBot uses a number of complex programs to analyse and caption images Microsoft
CaptionBot uses a number of complex programs to analyse and caption images Microsoft

Online Editors

Your Comments

COMMENT RULES: Comments that are judged to be defamatory, abusive or in bad taste are not acceptable and contributors who consistently fall below certain criteria will be permanently blacklisted. The moderator will not enter into debate with individual contributors and the moderator’s decision is final. It is Belfast Telegraph policy to close comments on court cases, tribunals and active legal investigations. We may also close comments on articles which are being targeted for abuse. Problems with commenting? customercare@belfasttelegraph.co.uk

Read More