Skip to content

Commit

Permalink
Updated relation extraction slides again
Browse files Browse the repository at this point in the history
  • Loading branch information
@isabelleaugenstein
isabelleaugenstein committed Oct 21, 2018
1 parent 2512c00 commit a6ba8a6
Showing 1 changed file with 53 additions and 51 deletions.
104 changes: 53 additions & 51 deletions chapters/relation_extraction_slides.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -273,7 +273,7 @@
"source": [
"* **Temporal** Information Extraction:\n",
" * Recognise and/or normalise temporal expressions, e.g. \"tomorrow morning at 8\" -> \"2016-11-26 08:00:00\"\n",
" * sequence labelling task"
" * sequence labelling or generation task"
]
},
{
Expand Down Expand Up @@ -450,9 +450,9 @@
},
"source": [
"## Relation Extraction as Structured Prediction\n",
"* **Goal**: train model \\\\(s_{\\params}(\\x,y)\\\\) that assigns high *scores* to the label $\\mathcal{y}$ that fits the arguments and supporting text $\\mathcal{x}$, low scores otherwise \n",
"* **Training**: Learn parameters \\\\(\\params\\\\) from training set of $\\mathcal{x,y}$ pairs\n",
"* **Prediction** of labels for input instances $\\mathcal{x}$: solve maximisation problem $\\argmax_y s_{\\params}(\\x,y)$."
"* **Goal**: train model \\\\(s_{\\params}(\\x,y)\\\\) that assigns high *scores* to the label $\\mathcal{y}$ that fits the arguments and supporting text $\\mathcal{\\x}$, low scores otherwise \n",
"* **Training**: Learn parameters \\\\(\\params\\\\) from training set of $\\mathcal{\\x,y}$ pairs\n",
"* **Prediction** of labels for input instances $\\mathcal{\\x}$: solve maximisation problem $\\argmax_y s_{\\params}(\\x,y)$."
]
},
{
Expand Down Expand Up @@ -521,7 +521,7 @@
"## Pattern-Based Extraction\n",
"* The simplest relation extraction method\n",
"* Set of textual patterns for each relation\n",
"* Assigns labels to entity pairs whose sentences match that pattern\n",
"* Assign labels to entity pairs whose sentences match that pattern\n",
" * Labels: relation types or \"NONE\"\n",
"* Data: entity pairs $\\mathcal{E}$, patterns $A$, labels $Y$"
]
Expand Down Expand Up @@ -612,10 +612,10 @@
}
},
"source": [
"* **Scoring model**: determine which instance belongs to which relation\n",
" * A pattern scoring model \\\\(s_{\\params}(\\x,y)\\\\) only has one parameter\n",
" * Assignes scores to each relation label \\\\(y\\\\) proportional to the matches with the set of textual patterns\n",
" * The final label assigned to each instance is then the one with the highest score"
"**Scoring model**: determine which instance belongs to which relation\n",
" * A pattern scoring model \\\\(s_{\\params}(\\x,y)\\\\) only has one parameter\n",
" * Assignes scores to each relation label \\\\(y\\\\) proportional to the matches with the set of textual patterns\n",
" * The final label assigned to each instance is then the one with the highest score"
]
},
{
Expand All @@ -638,7 +638,7 @@
}
},
"source": [
"Closer look at pattern matching:\n",
"### Closer look at pattern matching\n",
"* Patterns in the training data: sentences where entity pairs are blanked with 'XXXXX'\n",
"* Suggested improvement:\n",
" * We could use those patterns to find more sentences\n",
Expand Down Expand Up @@ -700,7 +700,7 @@
}
},
"source": [
"* There are many different alternatives to this method shortening patterns\n",
"* There are many different alternatives to this method for shortening patterns\n",
"* **Thought exercise**: \n",
" * what is a possible problem with this way of shortening patterns and what are better ways of generalising patterns?\n",
" \n",
Expand All @@ -715,8 +715,8 @@
}
},
"source": [
"Revised pattern extraction approach:\n",
" * Define sentences shortening / **pattern generalisation method**\n",
"### Revised pattern extraction approach\n",
" * Define sentence shortening / **pattern generalisation method**\n",
" * Apply patterns to testing instances to classify them into 'method used for task' and 'NONE'\n",
"\n",
"Example: return instances which contain a 'method used for task' pattern"
Expand Down Expand Up @@ -758,7 +758,7 @@
" # convert training and testing sentences to short paths to obtain patterns\n",
" training_patterns = set([sentenceToShortPath(train_sent) for train_sent in training_sentences])\n",
" testing_patterns = [sentenceToShortPath(test_sent) for test_sent in testing_sentences]\n",
" # look for training patterns in testing patterns\n",
" # look for match of training and testing patterns\n",
" testing_extractions = []\n",
" for i, testing_pattern in enumerate(testing_patterns):\n",
" if testing_pattern in training_patterns: # look for exact matches of patterns\n",
Expand Down Expand Up @@ -798,53 +798,53 @@
"* Overall idea: extract patterns and entity pairs **iteratively**\n",
"* One of the first algorithms: [DIPRE (Sergey Brin, 1999)](http://ilpubs.stanford.edu:8090/421/1/1999-65.pdf)\n",
"* Two helper methods: \n",
" * *use entity pairs* to find/generate (more) patterns and entity pairs\n",
" * *apply patterns* to find more patterns and entity pairs\n"
" * *use entity pairs* to find/generate (more) patterns\n",
" * *apply patterns* to find entity pairs\n"
]
},
{
"cell_type": "code",
"execution_count": 256,
"execution_count": 300,
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"# use entity pairs to generate patterns and entity pairs\n",
"def searchForPatternsAndEntpairsByEntpairs(training_entpairs, testing_patterns, testing_entpairs, testing_sentences):\n",
"# use entity pairs to find more patterns\n",
"def searchForEntpairsByPatterns(training_patterns, testing_patterns, testing_entpairs, testing_sentences):\n",
" testing_extractions = []\n",
" appearing_testing_patterns = []\n",
" appearing_testing_entpairs = []\n",
" for i, testing_entpair in enumerate(testing_entpairs):\n",
" if testing_entpair in training_entpairs: # if there is an exact match of an entity pair\n",
" for i, testing_pattern in enumerate(testing_patterns):\n",
" if testing_pattern in training_patterns: # if there is an exact match of a pattern\n",
" testing_extractions.append(testing_sentences[i])\n",
" appearing_testing_entpairs.append(testing_entpair) # add that entity pair\n",
" appearing_testing_patterns.append(testing_patterns[i])\n",
" appearing_testing_patterns.append(testing_pattern) # add that pattern\n",
" appearing_testing_entpairs.append(testing_entpairs[i])\n",
" return testing_extractions, appearing_testing_patterns, appearing_testing_entpairs"
]
},
{
"cell_type": "code",
"execution_count": 255,
"execution_count": 301,
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"# use entity pairs to find more patterns and entity pairs\n",
"def searchForPatternsAndEntpairsByPatterns(training_patterns, testing_patterns, testing_entpairs, testing_sentences):\n",
"# use patterns to find entity pairs\n",
"def searchForPatternsByEntpairs(training_entpairs, testing_patterns, testing_entpairs, testing_sentences):\n",
" testing_extractions = []\n",
" appearing_testing_patterns = []\n",
" appearing_testing_entpairs = []\n",
" for i, testing_pattern in enumerate(testing_patterns):\n",
" if testing_pattern in training_patterns: # if there is an exact match of a pattern\n",
" for i, testing_entpair in enumerate(testing_entpairs):\n",
" if testing_entpair in training_entpairs: # if there is an exact match of an entity pair\n",
" testing_extractions.append(testing_sentences[i])\n",
" appearing_testing_patterns.append(testing_pattern) # add that pattern\n",
" appearing_testing_entpairs.append(testing_entpairs[i])\n",
" appearing_testing_entpairs.append(testing_entpair) # add that entity pair\n",
" appearing_testing_patterns.append(testing_patterns[i])\n",
" return testing_extractions, appearing_testing_patterns, appearing_testing_entpairs"
]
},
Expand All @@ -861,7 +861,7 @@
},
{
"cell_type": "code",
"execution_count": 286,
"execution_count": 302,
"metadata": {
"slideshow": {
"slide_type": "subslide"
Expand All @@ -884,7 +884,7 @@
},
{
"cell_type": "code",
"execution_count": 287,
"execution_count": 303,
"metadata": {
"slideshow": {
"slide_type": "subslide"
Expand All @@ -904,8 +904,8 @@
" print(\"Number patterns at iteration\", str(i), \":\", str(len(train_patterns)))\n",
" print(\"Number entpairs at iteration\", str(i), \":\", str(len(train_entpairs)))\n",
" # get more patterns and entity pairs\n",
" test_extracts_e, ext_test_patterns_e, ext_test_entpairs_e = searchForPatternsAndEntpairsByEntpairs(train_entpairs, test_patterns, test_entpairs, test_sents)\n",
" test_extracts_p, ext_test_patterns_p, ext_test_entpairs_p = searchForPatternsAndEntpairsByPatterns(train_patterns, test_patterns, test_entpairs, test_sents)\n",
" test_extracts_e, ext_test_patterns_e, ext_test_entpairs_e = searchForPatternsByEntpairs(train_entpairs, test_patterns, test_entpairs, test_sents)\n",
" test_extracts_p, ext_test_patterns_p, ext_test_entpairs_p = searchForEntpairsByPatterns(train_patterns, test_patterns, test_entpairs, test_sents)\n",
" # add them to the existing entity pairs for the next iteration\n",
" train_patterns.update(ext_test_patterns_p)\n",
" train_patterns.update(ext_test_patterns_e)\n",
Expand All @@ -919,7 +919,7 @@
},
{
"cell_type": "code",
"execution_count": 284,
"execution_count": 304,
"metadata": {
"slideshow": {
"slide_type": "subslide"
Expand All @@ -930,31 +930,31 @@
"name": "stdout",
"output_type": "stream",
"text": [
"Number extractions at iteration 1 : 11735\n",
"Number extractions at iteration 1 : 13136\n",
"Number patterns at iteration 1 : 19\n",
"Number entpairs at iteration 1 : 20\n",
"Number extractions at iteration 2 : 11813\n",
"Number extractions at iteration 2 : 13214\n",
"Number patterns at iteration 2 : 19\n",
"Number entpairs at iteration 2 : 98\n",
"Number extractions at iteration 3 : 11974\n",
"Number extractions at iteration 3 : 13375\n",
"Number patterns at iteration 3 : 24\n",
"Number entpairs at iteration 3 : 259\n",
"Number extractions at iteration 4 : 12140\n",
"Number extractions at iteration 4 : 13541\n",
"Number patterns at iteration 4 : 24\n",
"Number entpairs at iteration 4 : 425\n",
"Number extractions at iteration 5 : 12306\n",
"Number extractions at iteration 5 : 13707\n",
"Number patterns at iteration 5 : 24\n",
"Number entpairs at iteration 5 : 591\n",
"Number extractions at iteration 6 : 12472\n",
"Number extractions at iteration 6 : 13873\n",
"Number patterns at iteration 6 : 24\n",
"Number entpairs at iteration 6 : 757\n",
"Number extractions at iteration 7 : 12638\n",
"Number extractions at iteration 7 : 14039\n",
"Number patterns at iteration 7 : 24\n",
"Number entpairs at iteration 7 : 923\n",
"Number extractions at iteration 8 : 12804\n",
"Number extractions at iteration 8 : 14205\n",
"Number patterns at iteration 8 : 24\n",
"Number entpairs at iteration 8 : 1089\n",
"Number extractions at iteration 9 : 12970\n",
"Number extractions at iteration 9 : 14371\n",
"Number patterns at iteration 9 : 24\n",
"Number entpairs at iteration 9 : 1255\n"
]
Expand Down Expand Up @@ -1064,7 +1064,7 @@
}
},
"source": [
"* Such as 'noisy' pattern is e.g. 'in' matches many contexts that are not 'method used for task' \n",
"* Such a 'noisy' pattern is e.g. 'in': it matches many contexts that are not 'method used for task' \n",
"* **Thought exercise**: \n",
" * how would a confidence weighting for patterns work here?\n",
" \n",
Expand Down Expand Up @@ -11108,9 +11108,9 @@
"source": [
"## Universal Schema\n",
"* Goal: overcome limitation of having to pre-define relations, within the supervised learning paradigm\n",
"* This is possible by viewing paths between entity pairs **as relations themselves**\n",
"* Simplified paths between entity pairs and relation labels are no longer considered separately\n",
" * instead paths between entity pairs and relations are **modelled in the same space**\n"
"* This is possible by viewing relation paths **as relations themselves**\n",
"* Simplified paths between entity pairs and relations defined in knowledge base are **no longer considered separately**\n",
" * instead they are **modelled in the same space**\n"
]
},
{
Expand Down Expand Up @@ -16319,7 +16319,9 @@
"* The scores shown here are for the relation 'method used for task'. However, we could also use our model to score the compatibility of entity pairs with other relations, e.g. 'demonstrates XXXXX for XXXXXX'. How could this be done here?\n",
"* How could we get around the problem of unseen words, as described above?\n",
"* What other possible problems can you see with the above formulation of universal schema relation extraction?\n",
"* What possible problems can you see with using latent word representations?"
"* What possible problems can you see with using latent word representations?\n",
"\n",
"Enter answer: https://b.socrative.com/login/student/, code: AUGENSTEIN"
]
},
{
Expand All @@ -16336,7 +16338,7 @@
"* Pattern-based extraction\n",
"* Bootstrapping\n",
"* Supervised\n",
"* Distantly supervised\n",
"* Distantly supervised extraction\n",
"* Universal schema\n",
"\n",
"Features often a mix of \n",
Expand All @@ -16356,7 +16358,7 @@
"\n",
"* Jurafky, Dan and Martin, James H. (2016). Speech and Language Processing, Chapter 21 (Information Extraction): https://web.stanford.edu/~jurafsky/slp3/21.pdf\n",
"\n",
"* Riedel, Sebastian and Yao, Limin and McCallum, Andrew and Marlin, Benjamin M. (2013). Extraction with Matrix Factorization and Universal Schemas. Proceedings of NAACL. http://www.aclweb.org/anthology/N13-1008"
"* Riedel, Sebastian and Yao, Limin and McCallum, Andrew and Marlin, Benjamin M. (2013). Relation extraction with Matrix Factorization and Universal Schemas. Proceedings of NAACL. http://www.aclweb.org/anthology/N13-1008"
]
}
],
Expand Down

0 comments on commit a6ba8a6

Please sign in to comment.