Image Captioning can be useful for physically disabled people like semi-blind or blind people if voice output is added to the generated captions. It can also be used in virtual assistants such as Sirior Cortana to help searching images of a particular type. For e.g.:- ”Show me pictures of myselfwearing a blue shirt.” Thus, we can see that there is plenty of motivation and usefulness involved in the image captioning task.
In this project, we implemented three different techniques used for Image Captioning:
- CNN-RNN (Google’s Implementation as ourbaseline)
- CNN-BRNN (Deep Visual-Semantic alignments for Gen-erating Image Description - Andrej Karpathy)
- Attention-based mode (Show Tell and Attend)
The various evaluation metrics used are:
- BLEU (Bilingual Evaluation Understudy)score
- METEOR
- Cider
Datasets Used were:
- Flickr8k (8000 im-ages comprising of 1GB)
- Flickr30K (31K imagescomprising of 6GB)
- MSCOCO (123K im-ages comprising of 18GB)