This repository contains the source code(in a google drive if you require code please ask for permission to view files) for the text to image synthesis using GAN's all the structres and codes are referenced from the original papers please look at references section for further information. All the scripts from the source code were in python 2 but all the scripts are modified to use the following packages. The jupyter notebook has the code to run validation or generate images using the text and all the codes are modified to run semantic object accuracy for all the gan's. Please change the relevent paths in coco_eval.yml in all the GAN's folders and change the corresponding shell scripts too for using the gan's to generate images or for calculating semantic object accuracy.
backports.functools-lru-cache==1.5 cloudpickle==0.6.1 cycler==0.10.0 dask==0.20.0 decorator==4.3.0 easydict==1.9 funcsigs==1.0.2 kiwisolver==1.0.1 matplotlib==2.2.3 mock==2.0.0 networkx==2.2 nltk==3.3 numpy==1.15.4 pbr==5.1.0 Pillow==5.3.0 pkg-resources==0.0.0 protobuf==3.6.1 pyparsing==2.3.0 python-dateutil==2.7.5 pytz==2018.7 PyWavelets==1.0.1 PyYAML==3.13 scikit-image==0.14.1 scipy==1.1.0 six==1.11.0 subprocess32==3.5.3 tensorboard==1.0.0a4 toolz==0.9.0 torch==0.4.1 torchfile==0.1.0 torchvision==0.2.1 Werkzeug==0.14.1
One example on all the generated images for
Input (five texts describing the required image)
baseball players practicing their batting skills in a filled arena.
One baseball player is swinging a club and the other is swinging a bat.
A batter practices his swing as his teammate swings at home plate.
A baseball player swinging a bat on top of a field.
Baseball players swing bats during a baseball game.
Image generated by Stack- GAN:\
Image generated by Attn GAN:
Where two images are generated from different noise
Image generated by OP- GAN:
Get the pretrained models and put them in relevant folders
For damsm pretrained on coco download and add them to DAMSM in text2img
https://drive.google.com/file/d/1zIrXCE9F6yfbEJIbNP5-YrEe2pZcPSGJ/view
For attn gan pretrained model on coco and add them to models in text2img
https://drive.google.com/file/d/1i9Xkg9nU74RAvkcqKE-rJYhjvzKAMnCi/view
For stack gan pretrained model on coco and add them to models in text2img
https://www2.informatik.uni-hamburg.de/wtm/software/multiple-objects-gan/model-ms-coco-stackgan.zip
For OP gan pretrained model on coco and add them to models in text2img opgan folders
https://www2.informatik.uni-hamburg.de/wtm/software/semantic-object-accuracy/op-gan.pth
For calculating SOA get coco val 2014 split from http://images.cocodataset.org/zips/val2014.zip
https://github.com/taoxugit/AttnGAN