Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Master #6

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Master #6

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1e324ac
Sentiment Analysis using NLP
pavankumarhm May 2, 2023
4c6734c
Sentiment Analysis using NLP
pavankumarhm May 2, 2023
161d352
Sentiment Analysis using NLP
pavankumarhm May 2, 2023
fc27b7b
Sentiment Analysis using NLP
pavankumarhm May 2, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 42 additions & 36 deletions Sentiment_Analysis_using_NLP.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@
{
"cell_type": "code",
"execution_count": 94,
"id": "1de51a4a",
"id": "d88d92b9",
"metadata": {},
"outputs": [
{
Expand All @@ -119,7 +119,7 @@
},
{
"cell_type": "markdown",
"id": "1aeb27aa",
"id": "f561e9a0",
"metadata": {},
"source": [
"## First, training a word2vec model (with the arguments that we want) on your training sentence. Store it into the `word2vec` variable. "
Expand All @@ -141,7 +141,7 @@
},
{
"cell_type": "markdown",
"id": "cd936167",
"id": "b633a171",
"metadata": {},
"source": [
"## Embedding the training and test sentences."
Expand Down Expand Up @@ -188,7 +188,7 @@
},
{
"cell_type": "markdown",
"id": "a5e7be40",
"id": "290583a6",
"metadata": {},
"source": [
"### It's a good practice to check check the following for `X_train_pad` and `X_test_pad`:\n",
Expand Down Expand Up @@ -216,7 +216,7 @@
},
{
"cell_type": "markdown",
"id": "8806e733",
"id": "4bf6b14b",
"metadata": {},
"source": [
"## Baseline Model"
Expand Down Expand Up @@ -255,7 +255,7 @@
},
{
"cell_type": "markdown",
"id": "220caaf9",
"id": "ca363914",
"metadata": {},
"source": [
"## The Model"
Expand Down Expand Up @@ -371,7 +371,7 @@
{
"cell_type": "code",
"execution_count": 102,
"id": "711f5974",
"id": "4c762a37",
"metadata": {},
"outputs": [
{
Expand Down Expand Up @@ -411,31 +411,31 @@
},
{
"cell_type": "markdown",
"id": "eae91b7f",
"id": "b93555d3",
"metadata": {},
"source": [
"## Trained Word2Vec - Transfer Learning\n"
]
},
{
"cell_type": "markdown",
"id": "7bfb7d0f",
"id": "427ae5ca",
"metadata": {},
"source": [
"### The accuracy of the above the baseline model, might be quite low. By improving the quality of the embedding we can Improve accuracy of the model."
]
},
{
"cell_type": "markdown",
"id": "f3bec45a",
"id": "8ebc1eb8",
"metadata": {},
"source": [
"### Let's improve the quality of our embedding, instead of just loading a larger corpus, let's benefit from the embedding that others have learned. Because, the quality of an embedding, i.e. the proximity of the words, can be derived from different tasks. This is exactly what transfer learning is."
]
},
{
"cell_type": "markdown",
"id": "17063f95",
"id": "d5ab2de5",
"metadata": {},
"source": [
"### Listing all the different models available in the word2vec using gensim api."
Expand Down Expand Up @@ -626,7 +626,7 @@
{
"cell_type": "code",
"execution_count": 110,
"id": "762034c1",
"id": "9c344fc0",
"metadata": {},
"outputs": [
{
Expand Down Expand Up @@ -664,15 +664,15 @@
},
{
"cell_type": "markdown",
"id": "cc341e2b",
"id": "ee3893f0",
"metadata": {},
"source": [
"### There is a significant improvement in the accuracy after Transfer learning."
]
},
{
"cell_type": "markdown",
"id": "db7feb2d",
"id": "ec279c0d",
"metadata": {},
"source": [
"## Comparing Accuracy of Baseline model, The model and Improved model."
Expand All @@ -681,7 +681,7 @@
{
"cell_type": "code",
"execution_count": 113,
"id": "18ec7fe1",
"id": "f51eb1ad",
"metadata": {},
"outputs": [
{
Expand All @@ -705,7 +705,7 @@
},
{
"cell_type": "markdown",
"id": "0ce88f63",
"id": "1315fb78",
"metadata": {},
"source": [
"## Predicting the model for new review."
Expand All @@ -714,7 +714,7 @@
{
"cell_type": "code",
"execution_count": 114,
"id": "ddf33a02",
"id": "6586edc0",
"metadata": {},
"outputs": [],
"source": [
Expand Down Expand Up @@ -760,21 +760,21 @@
},
{
"cell_type": "code",
"execution_count": 116,
"id": "92bdd2db",
"execution_count": 141,
"id": "fc098a62",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1/1 [==============================] - 0s 17ms/step\n",
"Negative review\n"
"1/1 [==============================] - 0s 70ms/step\n",
"Positive review\n"
]
}
],
"source": [
"review = \"The movie is bad.\"\n",
"review = \"The movie is good.\"\n",
"sentiment = predict_sentiment(review)\n",
"if sentiment > 0.5:\n",
" print(\"Positive review\")\n",
Expand All @@ -784,7 +784,7 @@
},
{
"cell_type": "code",
"execution_count": 117,
"execution_count": 138,
"id": "1949d6e3",
"metadata": {},
"outputs": [],
Expand All @@ -794,16 +794,16 @@
},
{
"cell_type": "code",
"execution_count": 118,
"execution_count": 139,
"id": "9de36bef",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[master 3f85d13] Sentiment Analysis using NLP\r\n",
" 1 file changed, 135 insertions(+), 261 deletions(-)\r\n"
"[master 161d352] Sentiment Analysis using NLP\r\n",
" 1 file changed, 24 insertions(+), 24 deletions(-)\r\n"
]
}
],
Expand All @@ -813,25 +813,31 @@
},
{
"cell_type": "code",
"execution_count": 119,
"id": "008724a6",
"execution_count": 140,
"id": "db054a16",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Enumerating objects: 5, done.\n",
"Counting objects: 100% (5/5), done.\n",
"Enumerating objects: 47, done.\n",
"Counting objects: 100% (47/47), done.\n",
"Delta compression using up to 8 threads\n",
"Compressing objects: 100% (3/3), done.\n",
"Writing objects: 100% (3/3), 150.80 KiB | 1.77 MiB/s, done.\n",
"Total 3 (delta 1), reused 0 (delta 0), pack-reused 0\n",
"remote: Resolving deltas: 100% (1/1), completed with 1 local object.\u001b[K\n",
"Compressing objects: 100% (47/47), done.\n",
"Writing objects: 100% (47/47), 294.53 KiB | 5.77 MiB/s, done.\n",
"Total 47 (delta 24), reused 4 (delta 0), pack-reused 0\n",
"remote: Resolving deltas: 100% (24/24), done.\u001b[K\n",
"remote: This repository moved. Please use the new location:\u001b[K\n",
"remote: git@github.com:pavankumarhm/Sentiment-Analysis-for-MYM-Intern-Assesment.git\u001b[K\n",
"remote: \n",
"remote: Create a pull request for 'master' on GitHub by visiting:\u001b[K\n",
"remote: https://github.com/pavankumarhm/Sentiment-Analysis-for-MYM-Intern-Assesment/pull/new/master\u001b[K\n",
"remote: \n",
"remote: Heads up! The branch 'master' that you pushed to was renamed to 'main'.\u001b[K\n",
"remote: \n",
"To github.com:pavankumarhm/Sentiment-Analysis-Assesment.git\n",
" 4a22313..3f85d13 master -> master\n",
" * [new branch] master -> master\n",
"branch 'master' set up to track 'origin/master'.\n"
]
}
Expand All @@ -843,7 +849,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "699f6f10",
"id": "5c4856a8",
"metadata": {},
"outputs": [],
"source": []
Expand Down