Understanding the 1,000 words of a picture

Online Social Networks (OSNs) are populated with billions of images

With more than 350 million images being uploaded daily on Facebook itself, extracting meaningful content from images has become more important that ever. As a preliminary step towards understanding and interpreting image content from a human perspective, we propose Helix, a system that tries to extract some meaningful, human understandable features from an image. Helix summarizes an image with the following: sentiment of the image, sentiment of faces present in the image (if any), and a text label for the image. These features can be used by researchers for image coding, categorization and summarization tasks on a large scale, which are currently carried out manually through human annotation.


We offer an API as well HelixAPI.

Our API provides the following endpoints:

More API details

Format of JSON returned:
{'text': {'text':'string, text found in image', 'sentiment':{'Positive':'', 'Negative':''}}, 'tag':{'tag': 'text, tag produced by inceptionv3, 'confidence':'confidence value associated with tag', 'tag_id':'id of the tag corresponding to the tag (not required for normal API use'}, 'sentiment':{'Tensorflow_SentiBank:{'Positive': 'fraction of positive sentiment in image overall', 'Negative': 'fraction of negative sentiment in image overall'}, 'Faces':{'Face_0':{'Positive':'', 'Negative':''},....}, 'Average':{'Positive':'', 'Negative':''}}}

Error codes:

Try our extensions on Chrome and Firefox

Available on the Chrome Web Store and Mozilla's Web Store. Read more about this here. Try uploading an image and see it for yourself!

Upload image (JPEG):