Siamese Neural Network

This tutorial to learn how to develop Siamese Neural network with Tensorflow framework. This is converted from [1]

Siamese networks were originally introduced for comparing signatures

Introduction

Assume that we want to build face recognition system for a small organization with only 10 employees. if we use traditional classification apporach, we might come up with a system

Problems

Tro train such a system, we need alot of different images of each of the 10 person in the organization.
What if a new person joins or leaves the organization? you need to take the pain of collecting data again an dre-train the entire model again. This practically not possible specially for large organizations where recruitment and attrition in happening almost every week.

Advantages of One-short learning

👍 Require only one or afew training example for each class.

Solution

one short classification which helps to solve both problems above.

Instead of directly classifying an input(test) image to one of th e10 people in the organization, this network instead takes an extra reference image of the person as input and will produce a similarity score denoting the chances tha tthe two input images belong to the same person. Typically the similarity score is squished between 0 and 1 using a sigmoid function; Wherein 0 denotes no similarity and 1 denotes full similarity. Any number between 0 and 1 is interpreted accordingly.

Notice: Thjsi network is not learnign to classify and image directly to any of the output classes. Rather, it is learning a similarity functiono, which takes two images as input and expresses how similar they are.

How does this solve the two problems above:

In a short while we will see that to train this network, we do not need too many instances of a class ans only few are enought to build a good model.
Biggest advantages is that let's say in case of face recognition, we have a new employee who has joined the organization. Now inorder for the network to detect his face, we only require a single of afew images of his face which will be stored in the database. Using this as the reference image, the network will calculate the similarity for any new instance presented to it. Thus we say that network predicts the score in one short.

Application

Face recognition [2]
Drug discovery [3]
Offline signature verification system [4]

Dataset

Omniglot dataset: 1623 hand drawn characters from 50 different alphabets. Every character there are just 20 examples. each image have size of 105x105. All of the data file is stored in the link . We download two file image_background.zip and images_eveluation.zip.

code_example_download_omniglot_dataset

code_example_load_data_to_tensor

output of load_data_to_tensor function are X, y, c

X are images
y index of character
c are label

Mapping the problem to binary classification task

We map this problem into classification problem, (we need {X, Y} := {input, target} data type)

input := Pair of images

target := {1 if both contain the same character, 0 if both images contain different class}

Thus we need to create pairs of images along with the target variable, as shown above, to be fed as input to the Siamese Network. Note that even though characters from Sanskrit alphabet are shown above, but in practice we will generate pairs randomly from all the alphabets in the training data.