Image-Captions-and-Generating-Images

Searching and Generating Images by Using Sentences

In this projects,I am trying to training two model which are pseudo siamese network and GAN. Besides, I am trying to use the pretrained Speech2Text model which released by facebook's team.

Targets :

1. Searching twenty relatively images which are described by five specific sentences.
2. Generating one similarly image by using the previous five specific sentences.
3. Convert speech files(.wav) to text.

Inputs :

1. Five sentences in english which describe :
   1.1 What's kinds of fabrics
   1.2 What's the color series
   1.3 Visualization patterns of fabric
   1.4 More details of pattern descriptions
   1.5 Handfeel of fabric

Model_Used :

1. (pseudo siamese network)
   1.1 pretrained_Bert
   1.1.1 Ref : https://huggingface.co/bert-base-uncased
   1.2 CNN model
   
2. GAN-GP
   
3. facebook/hubert-large-ls960-ft model
   3.1 Ref : https://huggingface.co/facebook/hubert-large-ls960-ft
   

Result