Hybrid Algorithm to Explore Properties of GPT in Quantum Transformer Models

[hykavitha]

Description

• This project is intended to explore couple papers in literature of Quantum Transformer models [self attention model: https://arxiv.org/abs/2205.05625 , Quantum vision transformers : https://arxiv.org/abs/2209.08167, GPT models: https://arxiv.org/pdf/2303.12712.pdf] . There are a couple limitations to produce a quantum GPT model, one being that transformers generally require a lot of data to be more accurate then convolutional or recurrent architectures, and another being the size of the model, inputs, and outputs, Input is the key property of transformers - each block updates the existing input vector in the sequence, transforming it slightly, and this updated vector is passed to the next block.
• In this project we want to explore and build a quantum circuit as a transformer block - if we have some inputs encoded in some way as input, and some other set of inputs corresponding to the last measurement outputs of the model - using the quantum circuit for part of the transformer blocks - i.e., as a sub component of a classical transformer neural net.

This project should take approximately 3 months to reasonably explore the literature of the Quantum transformer model and generate Qiskit Code. To Build Quantum Machine Learning - technical skills, and work on meaningful projects we want to integrate into Qiskit or maintain;
Deliverables

• Interactive tutorial Jupyter Notebook
• Journal / conference paper

Mentors details
Mentor 1
Name: Kavitha Yogaraj, Brian Quanz
GitHub ID: @hykavitha, @bquanz
What they do:

• Kavitha: IBM Quantum Computational Scientist, IBM Qiskit Advocate-2020, IBM Quantum
• Brian : IBM Senior Research Scientist, IBM Quantum

Number of mentees required : 3

Type of mentees
Mentee 1

Required:

• Deep Learning - RNN, CNN Expert, deep generative models - Classical
• Knowledge in Quantum Transformer model
• Experience in qiskit
• Proficient in Linear Alzebra
• Gradient descent
• Convert Pennylane code to Qiskit.
• Paper implementation knowledge

Mentee 2

Required:

• Proficient in Qiskit & QNN Hybrid algorithms, Quantum Machine Learning
• Deep Learning - RNN, CNN Expert, deep generative models - Classical
• Knowledge in Quantum Transformer model
• Experience in Qiskit open source contributions
• Proficient in Linear Alzebra
• Gradient descent
• Convert Pennylane code to Qiskit.
• Paper implementation knowledge

Mentee 3

Required:

• Proficient in Linear Alzebra
• Deep Learning - RNN, CNN Expert, deep generative models - Classical
• Knowledge in Quantum Transformer model
• Experience in Qiskit open source contributions
• Proficient in Linear Alzebra
• Gradient descent
• Convert Pennylane code to Qiskit.
• Paper implementation knowledge

[Tarun2021]

I am interested to work in this project

[divshacker]

I am also interested in this project

[poig]

interested

[ichen17]

Interesting.

[harshdeeps036]

Interested.

[AnuVadali]

I am interested to work on this project.

This is very intriguing considering the fact that just some years ago, there were studies on how to simulate quantum circuits using transformer models (https://arxiv.org/abs/1912.11052). And now we are studying how we can make transformer models using quantum circuits. Very interesting :D

Genuine question : Won't we have an issue with the number of qubits?

[hykavitha]

I am interested to work on this project.

This is very intriguing considering the fact that just some years ago, there were studies on how to simulate quantum circuits using transformer models (https://arxiv.org/abs/1912.11052). And now we are studying how we can make transformer models using quantum circuits. Very interesting :D

Genuine question : Won't we have an issue with the number of qubits?

Thats a study we want to do here.

[hykavitha]

@GemmaDawson : Please assign these interested people as the mentees for this project.

1. @harshdeeps036 Harshdeep Singh
2. @ichen17 I-Chi-Chen
3. @AnuVadali Anu Vadil

Thanks

[GemmaDawson]

@bquanz - please add a comment so that I may assign this issue to you! TY 😊

[bquanz]

@GemmaDawson added

[AnuVadali]

Uploading checkpoint 1 slide deck
QAMP_31_checkpoint_1.pptx

[harshdeeps036]

Checkpoint 2:
Checkpoint 2.pdf

[ichen17]

QAMP_31_final.pdf
Final Presentation Materials