EXPLORATION/Node_06/[E-06] Naver_movie_sentiment_analysis.ipynb

{"cells":[{"cell_type":"markdown","metadata":{"id":"fd77O6MkaVd0"},"source":["# 6. 영화리뷰 텍스트 감성분석하기"]},{"cell_type":"markdown","metadata":{},"source":["---"]},{"cell_type":"markdown","metadata":{"id":"XUpYh1Ryp6PZ"},"source":["## 목차\n","\n","### 1) 데이터 준비와 확인\n","활용할 데이터셋은 네이버 영화의 댓글을 모아 구성된 [Naver sentiment movie corpus](https://github.com/e9t/nsmc)입니다.\n","\n","### 2) 데이터로더 구성\n","- 데이터의 중복 제거\n","- NaN 결측치 제거\n","- 한국어 토크나이저로 토큰화\n","- 불용어(Stopwords) 제거\n","- 사전word_to_index 구성\n","- 텍스트 스트링을 사전 인덱스 스트링으로 변환\n","- X_train, y_train, X_test, y_test, word_to_index 리턴\n","\n","### 3) 모델 구성을 위한 데이터 분석 및 가공\n","- 데이터셋 내 문장 길이 분포\n","- 적절한 최대 문장 길이 지정\n","- keras.preprocessing.sequence.pad_sequences 을 활용한 패딩 추가\n","\n","### 4) 모델 구성 및 validation set 구성\n","모델은 3가지 이상 다양하게 구성하여 실험해 보세요.\n","\n","### 5) 모델 훈련 개시\n","\n","### 6) Loss, Accuracy 그래프 시각화\n","\n","### 7) 학습된 Embedding 레이어 분석\n","\n","### 8) 한국어 Word2Vec 임베딩 활용하여 성능 개선\n","한국어 Word2Vec은 다음 경로에서 구할 수 있습니다.\n","\n","- [Pre-trained word vectors of 30+ languages](https://github.com/Kyubyong/wordvectors)\n","\n","위 링크에서 적절한 ko.bin을 찾아 이용하세요. 그리고 gensim 버전을 3.x.x로 낮춰야 오류가 나지 않습니다."]},{"cell_type":"markdown","metadata":{},"source":["---"]},{"cell_type":"markdown","metadata":{"id":"QfAzbPQLaPXZ"},"source":["## 6-11. 프로젝트 : 네이버 영화리뷰 감성분석 도전하기\n","- 활용할 데이터셋은 네이버 영화의 댓글을 모아 구성된 [Naver sentiment movie corpus](https://github.com/e9t/nsmc)"]},{"cell_type":"markdown","metadata":{"id":"1fUKMMKNMeaw"},"source":["#### **참고: [Colab 에 konlpy 와 mecab 설치](https://sanghyu.tistory.com/170)**\n","```bash\n","# 1. bash 셀로 명령어를 입력하여 설치\n","%%bash\n","apt-get update\n","apt-get install g++ openjdk-8-jdk python-dev python3-dev\n","pip3 install JPype1\n","pip3 install konlpy\n","```\n","```bash\n","# 2. jdk 환경 변수 설정\n","%env JAVA_HOME \"/usr/lib/jvm/java-8-openjdk-amd64\"\n","```\n","```bash\n","# 3. mecab 설치\n","%%bash\n","bash <(curl -s https://raw.githubusercontent.com/konlpy/konlpy/master/scripts/mecab.sh)\n","pip3 install /tmp/mecab-python-0.996\n","```"]},{"cell_type":"code","execution_count":4,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":2661,"status":"ok","timestamp":1642993680110,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"_MFP7fzI5SYs","outputId":"8b386c86-e78c-4665-a16b-245c1bb3f28e"},"outputs":[{"name":"stderr","output_type":"stream","text":["debconf: unable to initialize frontend: Dialog\n","debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 5.)\n","debconf: falling back to frontend: Readline\n","debconf: unable to initialize frontend: Readline\n","debconf: (This frontend requires a controlling tty.)\n","debconf: falling back to frontend: Teletype\n","dpkg-preconfigure: unable to re-open stdin: \n","  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n","                                 Dload  Upload   Total   Spent    Left  Speed\n","\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\n","\r 21 1381k   21  303k    0     0   302k      0  0:00:04  0:00:01  0:00:03  302k\r100 1381k  100 1381k    0     0  1228k      0  0:00:01  0:00:01 --:--:-- 8766k\n","./configure: line 7378: /usr/bin/file: No such file or directory\n","In file included from viterbi.cpp:14:0:\n","param.h:30:13: warning: 'Target {anonymous}::lexical_cast(Source) [with Target = std::__cxx11::basic_string<char>; Source = std::__cxx11::basic_string<char>]' defined but not used [-Wunused-function]\n"," std::string lexical_cast<std::string, std::string>(std::string arg) {\n","             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n","learner_tagger.cpp:25:7: warning: 'char* MeCab::{anonymous}::mystrdup(const string&)' defined but not used [-Wunused-function]\n"," char *mystrdup(const std::string &str) {\n","       ^~~~~~~~\n","ar: `u' modifier ignored since `D' is the default (see `U')\n","./pos-id.def is not found. minimum setting is used\n","./pos-id.def is not found. minimum setting is used\n","./pos-id.def is not found. minimum setting is used\n","./pos-id.def is not found. minimum setting is used\n","./pos-id.def is not found. minimum setting is used\n","./pos-id.def is not found. minimum setting is used\n","./pos-id.def is not found. minimum setting is used\n","./pos-id.def is not found. minimum setting is used\n","./pos-id.def is not found. minimum setting is used\n","./pos-id.def is not found. minimum setting is used\n","./pos-id.def is not found. minimum setting is used\n","./pos-id.def is not found. minimum setting is used\n","./pos-id.def is not found. minimum setting is used\n","./pos-id.def is not found. minimum setting is used\n","seed/pos-id.def is not found. minimum setting is used\n","seed/pos-id.def is not found. minimum setting is used\n","model-ipadic.c1.0.f1.dic/pos-id.def is not found. minimum setting is used\n","model-ipadic.c1.0.f1.dic/pos-id.def is not found. minimum setting is used\n","  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n","                                 Dload  Upload   Total   Spent    Left  Speed\n","\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\n","\r 20 47.4M   20  9.9M    0     0  7631k      0  0:00:06  0:00:01  0:00:05 7631k\r100 47.4M  100 47.4M    0     0  22.7M      0  0:00:02  0:00:02 --:--:-- 50.0M\n","configure.ac:2: warning: AM_INIT_AUTOMAKE: two- and three-arguments forms are deprecated.  For more info, see:\n","configure.ac:2: http://www.gnu.org/software/automake/manual/automake.html#Modernize-AM_005fINIT_005fAUTOMAKE-invocation\n","/tmp/mecab-ko-dic-2.1.1-20180720/missing: Unknown `--is-lightweight' option\n","Try `/tmp/mecab-ko-dic-2.1.1-20180720/missing --help' for more information\n","configure: WARNING: 'missing' script is too old or missing\n","/sbin/ldconfig.real: /usr/local/lib/python3.7/dist-packages/ideep4py/lib/libmkldnn.so.0 is not a symbolic link\n","\n","Cloning into 'mecab-python-0.996'...\n","  DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.\n","   pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.\n","  WARNING: Built wheel for mecab-python is invalid: Metadata 1.2 mandates PEP 440 version, but '0.996-ko-0.9.2' is not\n","  DEPRECATION: mecab-python was installed using the legacy 'setup.py install' method, because a wheel could not be built for it. A possible replacement is to fix the wheel build issue reported above. You can find discussion regarding this at https://github.com/pypa/pip/issues/8368.\n","  DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.\n","   pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.\n","  WARNING: Built wheel for mecab-python is invalid: Metadata 1.2 mandates PEP 440 version, but '0.996-ko-0.9.2' is not\n","  DEPRECATION: mecab-python was installed using the legacy 'setup.py install' method, because a wheel could not be built for it. A possible replacement is to fix the wheel build issue reported above. You can find discussion regarding this at https://github.com/pypa/pip/issues/8368.\n"]}],"source":["# 라이브러리 불러오기\n","import pandas as pd\n","\n","from konlpy.tag import Mecab\n","import numpy as np\n","from collections import Counter\n","\n","import tensorflow as tf\n","import os\n","\n","import matplotlib.pyplot as plt"]},{"cell_type":"markdown","metadata":{"id":"c2DAVYEATSNW"},"source":["### 1) 데이터 준비와 확인\n","긍정 1, 부정 0 으로 라벨링"]},{"cell_type":"code","execution_count":5,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":206},"executionInfo":{"elapsed":1869,"status":"ok","timestamp":1642993682263,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"TAEFXbx3E-R8","outputId":"0bd0cf5d-981c-4471-87c6-456985c50d26"},"outputs":[{"data":{"text/html":["\n","  <div id=\"df-8f194ccd-5016-4f21-846f-8e655875037e\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>id</th>\n","      <th>document</th>\n","      <th>label</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>9976970</td>\n","      <td>아 더빙.. 진짜 짜증나네요 목소리</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>3819312</td>\n","      <td>흠...포스터보고 초딩영화줄....오버연기조차 가볍지 않구나</td>\n","      <td>1</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>10265843</td>\n","      <td>너무재밓었다그래서보는것을추천한다</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>9045019</td>\n","      <td>교도소 이야기구먼 ..솔직히 재미는 없다..평점 조정</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>6483659</td>\n","      <td>사이몬페그의 익살스런 연기가 돋보였던 영화!스파이더맨에서 늙어보이기만 했던 커스틴 ...</td>\n","      <td>1</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8f194ccd-5016-4f21-846f-8e655875037e')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-8f194ccd-5016-4f21-846f-8e655875037e button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-8f194ccd-5016-4f21-846f-8e655875037e');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "],"text/plain":["         id                                           document  label\n","0   9976970                                아 더빙.. 진짜 짜증나네요 목소리      0\n","1   3819312                  흠...포스터보고 초딩영화줄....오버연기조차 가볍지 않구나      1\n","2  10265843                                  너무재밓었다그래서보는것을추천한다      0\n","3   9045019                      교도소 이야기구먼 ..솔직히 재미는 없다..평점 조정      0\n","4   6483659  사이몬페그의 익살스런 연기가 돋보였던 영화!스파이더맨에서 늙어보이기만 했던 커스틴 ...      1"]},"execution_count":5,"metadata":{},"output_type":"execute_result"}],"source":["# 데이터를 읽어봅시다. \n","train_data = pd.read_table('drive/MyDrive/Colab Notebooks/sentiment_classification/data/ratings_train.txt')\n","test_data = pd.read_table('drive/MyDrive/Colab Notebooks/sentiment_classification/data/ratings_test.txt')\n","\n","train_data.head()"]},{"cell_type":"markdown","metadata":{"id":"TQlFlLRrTWSn"},"source":["### 2) 데이터로더 구성\n","- 데이터의 중복 제거\n","- NaN 결측치 제거\n","- 한국어 토크나이저로 토큰화\n","- 불용어(Stopwords) 제거\n","- 사전word_to_index 구성\n","- 텍스트 스트링을 사전 인덱스 스트링으로 변환\n","- X_train, y_train, X_test, y_test, word_to_index 리턴"]},{"cell_type":"code","execution_count":6,"metadata":{"executionInfo":{"elapsed":10,"status":"ok","timestamp":1642993682264,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"uq6lO9waktMZ"},"outputs":[],"source":["tokenizer = Mecab()\n","stopwords = ['의','가','이','은','들','는','좀','잘','걍','과','도','를','으로','자','에','와','한','하다']\n","\n","def load_data(train_data, test_data, num_words=10000):\n","  train_data.drop_duplicates(subset=['document'], inplace=True)  # 데이터 중복 제거\n","  train_data = train_data.dropna(how = 'any')  # NaN 결측치 제거\n","  test_data.drop_duplicates(subset=['document'], inplace=True)  # 데이터 중복 제거\n","  test_data = test_data.dropna(how = 'any')  # NaN 결측치 제거\n","  \n","  X_train = []  # 학습용 데이터\n","  for sentence in train_data['document']:\n","      temp_X = tokenizer.morphs(sentence) # 토큰화\n","      temp_X = [word for word in temp_X if not word in stopwords] # 불용어 제거\n","      X_train.append(temp_X)  # 단어 저장\n","\n","  X_test = []  # 테스트용 데이터\n","  for sentence in test_data['document']:\n","      temp_X = tokenizer.morphs(sentence) # 토큰화\n","      temp_X = [word for word in temp_X if not word in stopwords] # 불용어 제거\n","      X_test.append(temp_X)  # 단어 저장\n","  \n","  words = np.concatenate(X_train).tolist()  # array 합치고 list로 변환\n","  counter = Counter(words)  # list의 요소 개수 세기 => 딕셔너리 {단어: 개수} 형태로 반환\n","  counter = counter.most_common(10000-4)  # 최빈값 (10000-4)개 => 가장 마지막 4개를 제외한 나머지\n","  vocab = ['<PAD>', '<BOS>', '<UNK>', '<UNUSED>'] + [key for key, _ in counter]  # 단어 저장 list # <PAD>, <BOS>, <UNK>, <UNUSED>\n","  word_to_index = {word:index for index, word in enumerate(vocab)}  # 딕셔너리 {단어: 인덱스} 형태\n","  \n","  # 중첩 함수\n","  def wordlist_to_indexlist(wordlist):\n","      return [word_to_index[word] if word in word_to_index else word_to_index['<UNK>'] for word in wordlist]\n","      \n","  # list(map(함수, 리스트)): 리스트의 모든 요소를 지정된 함수로 처리한 결과를 리스트로 만듦\n","  X_train = list(map(wordlist_to_indexlist, X_train))\n","  X_test = list(map(wordlist_to_indexlist, X_test))\n","      \n","  return X_train, np.array(list(train_data['label'])), X_test, np.array(list(test_data['label'])), word_to_index"]},{"cell_type":"code","execution_count":7,"metadata":{"executionInfo":{"elapsed":24020,"status":"ok","timestamp":1642993706274,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"3R7m31OcRRpv"},"outputs":[],"source":["X_train, y_train, X_test, y_test, word_to_index = load_data(train_data, test_data) "]},{"cell_type":"code","execution_count":8,"metadata":{"executionInfo":{"elapsed":23,"status":"ok","timestamp":1642993706275,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"hpZxiT6Pa1mN"},"outputs":[],"source":["index_to_word = {index:word for word, index in word_to_index.items()}"]},{"cell_type":"code","execution_count":9,"metadata":{"executionInfo":{"elapsed":22,"status":"ok","timestamp":1642993706275,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"ODw0C7ama3Zh"},"outputs":[],"source":["# 문장 1개를 활용할 딕셔너리와 함께 주면, 단어 인덱스 리스트 벡터로 변환해 주는 함수입니다. \n","# 단, 모든 문장은 <BOS>로 시작하는 것으로 합니다. \n","def get_encoded_sentence(sentence, word_to_index):\n","    return [word_to_index['<BOS>']]+[word_to_index[word] if word in word_to_index else word_to_index['<UNK>'] for word in sentence.split()]\n","\n","# 여러 개의 문장 리스트를 한꺼번에 단어 인덱스 리스트 벡터로 encode해 주는 함수입니다. \n","def get_encoded_sentences(sentences, word_to_index):\n","    return [get_encoded_sentence(sentence, word_to_index) for sentence in sentences]\n","\n","# 숫자 벡터로 encode된 문장을 원래대로 decode하는 함수입니다. \n","def get_decoded_sentence(encoded_sentence, index_to_word):\n","    return ' '.join(index_to_word[index] if index in index_to_word else '<UNK>' for index in encoded_sentence[1:])  #[1:]를 통해 <BOS>를 제외\n","\n","# 여러 개의 숫자 벡터로 encode된 문장을 한꺼번에 원래대로 decode하는 함수입니다. \n","def get_decoded_sentences(encoded_sentences, index_to_word):\n","    return [get_decoded_sentence(encoded_sentence, index_to_word) for encoded_sentence in encoded_sentences]"]},{"cell_type":"markdown","metadata":{"id":"DV9lIpmpaHxS"},"source":["### 3) 모델 구성을 위한 데이터 분석 및 가공\n","- 데이터셋 내 문장 길이 분포"]},{"cell_type":"code","execution_count":10,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":23,"status":"ok","timestamp":1642993706276,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"zWGQ7y2oRoRt","outputId":"ee4b1e56-d73b-41cf-833a-00df33b85fd4"},"outputs":[{"name":"stdout","output_type":"stream","text":["[32, 74, 919, 4, 4, 39, 228, 20, 33, 748]\n","라벨:  0\n","1번째 리뷰 문장 길이:  10\n","2번째 리뷰 문장 길이:  17\n"]}],"source":["print(X_train[0])  # 1번째 리뷰데이터\n","print('라벨: ', y_train[0])  # 1번째 리뷰데이터의 라벨\n","\n","print('1번째 리뷰 문장 길이: ', len(X_train[0]))\n","print('2번째 리뷰 문장 길이: ', len(X_train[1]))"]},{"cell_type":"markdown","metadata":{"id":"9WS4mevNoB0p"},"source":["- 적절한 최대 문장 길이 지정"]},{"cell_type":"code","execution_count":11,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":17,"status":"ok","timestamp":1642993706276,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"m0jqJzCEtuwS","outputId":"849dba47-46c2-43b2-887a-2463191351a3"},"outputs":[{"name":"stdout","output_type":"stream","text":["문장길이 평균 :  15.96938143432699\n","문장길이 최대 :  116\n","문장길이 표준편차 :  12.843571939469296\n","pad_sequences maxlen :  41\n","전체 문장의 0.9342988343341575%가 maxlen 설정값 이내에 포함됩니다. \n"]}],"source":["total_data_text = list(X_train) + list(X_test)\n","# 텍스트데이터 문장길이의 리스트를 생성한 후\n","num_tokens = [len(tokens) for tokens in total_data_text]\n","num_tokens = np.array(num_tokens)\n","# 문장길이의 평균값, 최대값, 표준편차를 계산해 본다. \n","print('문장길이 평균 : ', np.mean(num_tokens))\n","print('문장길이 최대 : ', np.max(num_tokens))\n","print('문장길이 표준편차 : ', np.std(num_tokens))\n","\n","# 예를들어, 최대 길이를 (평균 + 2*표준편차)로 한다면,  \n","max_tokens = np.mean(num_tokens) + 2 * np.std(num_tokens)\n","maxlen = int(max_tokens)\n","print('pad_sequences maxlen : ', maxlen)\n","print('전체 문장의 {}%가 maxlen 설정값 이내에 포함됩니다. '.format(np.sum(num_tokens < max_tokens) / len(num_tokens)))"]},{"cell_type":"markdown","metadata":{"id":"7U8VSOZRRfsA"},"source":["- keras.preprocessing.sequence.pad_sequences 을 활용한 패딩 추가"]},{"cell_type":"code","execution_count":12,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":895,"status":"ok","timestamp":1642993707158,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"geRjxUwqdfHH","outputId":"d8511234-092c-404a-f787-8d8d140ea917"},"outputs":[{"name":"stdout","output_type":"stream","text":["(146182, 41)\n","(49157, 41)\n"]}],"source":["# 짧은 문장 앞(pre)에 패딩 추가. post보다 pre가 효율이 더 좋음!\n","X_train = tf.keras.preprocessing.sequence.pad_sequences(X_train,\n","                                                        value=word_to_index[\"<PAD>\"],\n","                                                        padding='pre', # 혹은 'post'\n","                                                        maxlen=maxlen)\n","\n","X_test = tf.keras.preprocessing.sequence.pad_sequences(X_test,\n","                                                       value=word_to_index[\"<PAD>\"],\n","                                                       padding='pre', # 혹은 'post'\n","                                                       maxlen=maxlen)\n","\n","print(X_train.shape)  # (146182, 41)\n","print(X_test.shape)  # (49157, 41)"]},{"cell_type":"markdown","metadata":{"id":"NA8b7GGcbEIh"},"source":["### 4) 모델 구성 및 validation set 구성\n","모델은 3가지 이상 다양하게 구성하여 실험해 보세요."]},{"cell_type":"code","execution_count":13,"metadata":{"executionInfo":{"elapsed":3,"status":"ok","timestamp":1642993707430,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"V-zuqlY5NaLM"},"outputs":[],"source":["vocab_size = len(word_to_index)  # 어휘 사전의 크기: 10000개 단어\n","word_vector_dim = 200   # 단어 하나를 표현하는 임베딩 벡터의 차원 수입니다. (변경 가능한 하이퍼 파라미터)"]},{"cell_type":"markdown","metadata":{"id":"jeMNFYeqNsIN"},"source":["- Model 1: LSTM"]},{"cell_type":"code","execution_count":14,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":260,"status":"ok","timestamp":1642993707687,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"plcH83GKNp62","outputId":"73ccd8f5-c027-462c-dcf9-a133ba2a66f1"},"outputs":[{"name":"stdout","output_type":"stream","text":["Model: \"sequential\"\n","_________________________________________________________________\n"," Layer (type)                Output Shape              Param #   \n","=================================================================\n"," embedding (Embedding)       (None, None, 200)         2000000   \n","                                                                 \n"," lstm (LSTM)                 (None, 8)                 6688      \n","                                                                 \n"," dense (Dense)               (None, 8)                 72        \n","                                                                 \n"," dense_1 (Dense)             (None, 1)                 9         \n","                                                                 \n","=================================================================\n","Total params: 2,006,769\n","Trainable params: 2,006,769\n","Non-trainable params: 0\n","_________________________________________________________________\n"]}],"source":["# 모델 1: RNN \n","\n","model_lstm = tf.keras.Sequential()\n","model_lstm.add(tf.keras.layers.Embedding(vocab_size, word_vector_dim, input_shape=(None,)))\n","model_lstm.add(tf.keras.layers.LSTM(8))   # 가장 널리 쓰이는 RNN인 LSTM 레이어를 사용하였습니다. 이때 LSTM state 벡터의 차원수는 8로 하였습니다. (변경 가능)\n","model_lstm.add(tf.keras.layers.Dense(8, activation='relu'))\n","model_lstm.add(tf.keras.layers.Dense(1, activation='sigmoid'))  # 최종 출력은 긍정/부정을 나타내는 1dim 입니다.\n","\n","model_lstm.summary()"]},{"cell_type":"markdown","metadata":{"id":"UG8QvDOlNwuV"},"source":["- Model 2: 1-D CNN"]},{"cell_type":"code","execution_count":15,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":414,"status":"ok","timestamp":1642993708097,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"4V2m1P7wN6zs","outputId":"81f775cb-8975-47f5-ad7a-c29d718bc74b"},"outputs":[{"name":"stdout","output_type":"stream","text":["Model: \"sequential_1\"\n","_________________________________________________________________\n"," Layer (type)                Output Shape              Param #   \n","=================================================================\n"," embedding_1 (Embedding)     (None, None, 200)         2000000   \n","                                                                 \n"," conv1d (Conv1D)             (None, None, 16)          22416     \n","                                                                 \n"," max_pooling1d (MaxPooling1D  (None, None, 16)         0         \n"," )                                                               \n","                                                                 \n"," conv1d_1 (Conv1D)           (None, None, 16)          1808      \n","                                                                 \n"," global_max_pooling1d (Globa  (None, 16)               0         \n"," lMaxPooling1D)                                                  \n","                                                                 \n"," dense_2 (Dense)             (None, 8)                 136       \n","                                                                 \n"," dense_3 (Dense)             (None, 1)                 9         \n","                                                                 \n","=================================================================\n","Total params: 2,024,369\n","Trainable params: 2,024,369\n","Non-trainable params: 0\n","_________________________________________________________________\n"]}],"source":["# 모델 2: 1-D CNN\n","\n","model_1d_cnn = tf.keras.Sequential()\n","model_1d_cnn.add(tf.keras.layers.Embedding(vocab_size, word_vector_dim, input_shape=(None,)))\n","model_1d_cnn.add(tf.keras.layers.Conv1D(16, 7, activation='relu'))\n","model_1d_cnn.add(tf.keras.layers.MaxPooling1D(5))\n","model_1d_cnn.add(tf.keras.layers.Conv1D(16, 7, activation='relu'))\n","model_1d_cnn.add(tf.keras.layers.GlobalMaxPooling1D())\n","model_1d_cnn.add(tf.keras.layers.Dense(8, activation='relu'))\n","model_1d_cnn.add(tf.keras.layers.Dense(1, activation='sigmoid'))  # 최종 출력은 긍정/부정을 나타내는 1dim 입니다.\n","\n","model_1d_cnn.summary()"]},{"cell_type":"markdown","metadata":{"id":"qpUjMGsxN7gg"},"source":["- Model 3: GlobalMaxPooling1D() 레이어 1개만 사용"]},{"cell_type":"code","execution_count":16,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":13,"status":"ok","timestamp":1642993708097,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"S0OnjpbdN_IL","outputId":"ec67db36-5777-4d00-e100-d82f0bceab8a"},"outputs":[{"name":"stdout","output_type":"stream","text":["Model: \"sequential_2\"\n","_________________________________________________________________\n"," Layer (type)                Output Shape              Param #   \n","=================================================================\n"," embedding_2 (Embedding)     (None, None, 200)         2000000   \n","                                                                 \n"," global_max_pooling1d_1 (Glo  (None, 200)              0         \n"," balMaxPooling1D)                                                \n","                                                                 \n"," dense_4 (Dense)             (None, 8)                 1608      \n","                                                                 \n"," dense_5 (Dense)             (None, 1)                 9         \n","                                                                 \n","=================================================================\n","Total params: 2,001,617\n","Trainable params: 2,001,617\n","Non-trainable params: 0\n","_________________________________________________________________\n"]}],"source":["# 모델 3: GlobalMaxPooling1D() 레이어 1개만 사용\n","\n","model_gmp1d = tf.keras.Sequential()\n","model_gmp1d.add(tf.keras.layers.Embedding(vocab_size, word_vector_dim, input_shape=(None,)))\n","model_gmp1d.add(tf.keras.layers.GlobalMaxPooling1D())\n","model_gmp1d.add(tf.keras.layers.Dense(8, activation='relu'))\n","model_gmp1d.add(tf.keras.layers.Dense(1, activation='sigmoid'))  # 최종 출력은 긍정/부정을 나타내는 1dim 입니다.\n","\n","model_gmp1d.summary()"]},{"cell_type":"markdown","metadata":{"id":"u7axHfrn4FZR"},"source":["### 5) 모델 훈련 개시"]},{"cell_type":"code","execution_count":17,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":11,"status":"ok","timestamp":1642993708098,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"8m5JjSc7OXiZ","outputId":"7bf56644-3ff4-44d7-95b0-24da9614cae2"},"outputs":[{"name":"stdout","output_type":"stream","text":["(29236, 41)\n","(29236,)\n","(116946, 41)\n","(116946,)\n"]}],"source":["# Train : Validation : Test 을 일반적으로 6 : 2 : 2로 이용\n","\n","# 일반적으로 전체 데이터 중 80%를 학습으로, 20%를 검증으로 사용하는 것이 좋다고 한다.\n","# validation set (총 데이터 개수 * 0.2)건 분리\n","x_val_len = int(len(X_train)*0.2)\n","y_val_len = int(len(y_train)*0.2)\n","\n","x_val = X_train[:x_val_len]   \n","y_val = y_train[:y_val_len]\n","\n","print(x_val.shape)  # (29236, 41)\n","print(y_val.shape)  # (29236,)\n","\n","# validation set을 제외한 나머지 (총 데이터 개수 * 0.8)건\n","partial_x_train = X_train[x_val_len:]  \n","partial_y_train = y_train[y_val_len:]\n","\n","print(partial_x_train.shape)  # (116946, 41)\n","print(partial_y_train.shape)  # (116946,)"]},{"cell_type":"markdown","metadata":{"id":"jXkjh2g2flVm"},"source":["#### **참고: [Early Stopping 의 개념과 Keras 를 통한 구현](https://3months.tistory.com/424)**\n","너무 많은 Epoch 은 overfitting 을 일으킨다. 하지만 너무 적은 Epoch 은 underfitting 을 일으킨다. \n","\n","=> Epoch 을 정할 때 Early stopping 을 사용한다! Early stopping 은 무조건 Epoch 을 많이 돌린 후, 특정 시점에서 멈추는 것이다."]},{"cell_type":"code","execution_count":18,"metadata":{"executionInfo":{"elapsed":8,"status":"ok","timestamp":1642993708098,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"MvnMU-iFgKS7"},"outputs":[],"source":["from keras.callbacks import EarlyStopping\n","\n","# monitor='val_loss': validation set 의 loss 를 monitoring 한다.\n","# mode='min': loss 의 경우, performance measure가 최소화 시키는 방향으로 training 이 진행되므로 min 을 지정한다. => keras에서 알아서 적절한 epoch에서 training을 멈춘다.\n","# verbose=1: 언제 keras 에서 training 을 멈추었는지를 화면에 출력할 수 있다.\n","# patience=5: patience 는 성능이 증가하지 않는 epoch 을 몇 번이나 허용할 것인가를 주관적 기준으로 정의한다. 성능이 증가하지 않는다고 바로 멈추는 것은 효과적이지 않을 수 있기 때문이다. 사용한 데이터와 모델의 설계에 따라 최적의 값이 바뀔 수 있다. \n","early_stopping = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=5)"]},{"cell_type":"markdown","metadata":{"id":"6-9AhpS0OBwM"},"source":["- Model 1: LSTM"]},{"cell_type":"code","execution_count":19,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":125211,"status":"ok","timestamp":1642993833301,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"jXxkErBdOEah","outputId":"36c6260a-c75b-4615-e00f-d5ddc5fac419"},"outputs":[{"name":"stdout","output_type":"stream","text":["Epoch 1/5\n","229/229 [==============================] - 27s 109ms/step - loss: 0.4387 - accuracy: 0.8103 - val_loss: 0.3482 - val_accuracy: 0.8498\n","Epoch 2/5\n","229/229 [==============================] - 24s 107ms/step - loss: 0.3185 - accuracy: 0.8662 - val_loss: 0.3327 - val_accuracy: 0.8540\n","Epoch 3/5\n","229/229 [==============================] - 24s 106ms/step - loss: 0.2850 - accuracy: 0.8809 - val_loss: 0.3420 - val_accuracy: 0.8541\n","Epoch 4/5\n","229/229 [==============================] - 24s 107ms/step - loss: 0.2577 - accuracy: 0.8935 - val_loss: 0.3428 - val_accuracy: 0.8544\n","Epoch 5/5\n","229/229 [==============================] - 24s 107ms/step - loss: 0.2323 - accuracy: 0.9047 - val_loss: 0.3602 - val_accuracy: 0.8531\n"]}],"source":["model_lstm.compile(optimizer='adam',\n","              loss='binary_crossentropy',\n","              metrics=['accuracy'])\n","              \n","epochs=5  # 몇 epoch를 훈련하면 좋을지 결과를 보면서 바꾸어 봅시다. \n","\n","history_lstm = model_lstm.fit(partial_x_train,\n","                    partial_y_train,\n","                    epochs=epochs,\n","                    batch_size=512,\n","                    validation_data=(x_val, y_val),\n","                    verbose=1,\n","                    callbacks=[early_stopping])"]},{"cell_type":"code","execution_count":20,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":5952,"status":"ok","timestamp":1642993839243,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"bzigcF4MORTD","outputId":"6a220857-c7a8-4b56-90fe-19c380a79d19"},"outputs":[{"name":"stdout","output_type":"stream","text":["1537/1537 - 6s - loss: 0.3667 - accuracy: 0.8479 - 6s/epoch - 4ms/step\n","[0.36666378378868103, 0.8479158878326416]\n"]}],"source":["# 모델 평가\n","results_lstm = model_lstm.evaluate(X_test,  y_test, verbose=2)  # (loss, accuracy)\n","\n","print(results_lstm)  # (0.3667, 0.8479)"]},{"cell_type":"markdown","metadata":{"id":"JisZPUCfcdsu"},"source":["- Model 2: 1-D CNN"]},{"cell_type":"code","execution_count":21,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":202638,"status":"ok","timestamp":1642994041869,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"PKiORjNhcf31","outputId":"d3ae4e73-fa25-4082-8c0b-87a0139bcb53"},"outputs":[{"name":"stdout","output_type":"stream","text":["Epoch 1/5\n","229/229 [==============================] - 37s 160ms/step - loss: 0.4387 - accuracy: 0.7907 - val_loss: 0.3404 - val_accuracy: 0.8519\n","Epoch 2/5\n","229/229 [==============================] - 36s 158ms/step - loss: 0.3004 - accuracy: 0.8741 - val_loss: 0.3326 - val_accuracy: 0.8543\n","Epoch 3/5\n","229/229 [==============================] - 36s 158ms/step - loss: 0.2395 - accuracy: 0.9053 - val_loss: 0.3391 - val_accuracy: 0.8584\n","Epoch 4/5\n","229/229 [==============================] - 36s 158ms/step - loss: 0.1718 - accuracy: 0.9372 - val_loss: 0.3820 - val_accuracy: 0.8519\n","Epoch 5/5\n","229/229 [==============================] - 36s 158ms/step - loss: 0.1117 - accuracy: 0.9624 - val_loss: 0.4451 - val_accuracy: 0.8475\n"]}],"source":["model_1d_cnn.compile(optimizer='adam',\n","              loss='binary_crossentropy',\n","              metrics=['accuracy'])\n","              \n","epochs=5  # 몇 epoch를 훈련하면 좋을지 결과를 보면서 바꾸어 봅시다. \n","\n","history_1d_cnn = model_1d_cnn.fit(partial_x_train,\n","                    partial_y_train,\n","                    epochs=epochs,\n","                    batch_size=512,\n","                    validation_data=(x_val, y_val),\n","                    verbose=1,\n","                    callbacks=[early_stopping])"]},{"cell_type":"code","execution_count":22,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":4091,"status":"ok","timestamp":1642994045956,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"C2zY9i1Bcgm-","outputId":"98b16ecf-8f52-4c46-f3fb-045deccea924"},"outputs":[{"name":"stdout","output_type":"stream","text":["1537/1537 - 4s - loss: 0.4554 - accuracy: 0.8424 - 4s/epoch - 3ms/step\n","[0.45540377497673035, 0.8424029350280762]\n"]}],"source":["# 모델 평가\n","results_1d_cnn = model_1d_cnn.evaluate(X_test,  y_test, verbose=2)  # (loss, accuracy)\n","\n","print(results_1d_cnn)  # (0.4554, 0.8424)"]},{"cell_type":"markdown","metadata":{"id":"huwxYGAsczbF"},"source":["- Model 3: GlobalMaxPooling1D() 1개만 사용"]},{"cell_type":"code","execution_count":23,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":51727,"status":"ok","timestamp":1642994097644,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"yYDm1ivpc6cf","outputId":"497122b3-e16f-4881-8823-83eca09b2116"},"outputs":[{"name":"stdout","output_type":"stream","text":["Epoch 1/5\n","229/229 [==============================] - 11s 45ms/step - loss: 0.4748 - accuracy: 0.8013 - val_loss: 0.3482 - val_accuracy: 0.8472\n","Epoch 2/5\n","229/229 [==============================] - 10s 45ms/step - loss: 0.3153 - accuracy: 0.8666 - val_loss: 0.3333 - val_accuracy: 0.8552\n","Epoch 3/5\n","229/229 [==============================] - 10s 45ms/step - loss: 0.2682 - accuracy: 0.8900 - val_loss: 0.3379 - val_accuracy: 0.8557\n","Epoch 4/5\n","229/229 [==============================] - 10s 45ms/step - loss: 0.2286 - accuracy: 0.9101 - val_loss: 0.3503 - val_accuracy: 0.8549\n","Epoch 5/5\n","229/229 [==============================] - 10s 45ms/step - loss: 0.1888 - accuracy: 0.9299 - val_loss: 0.3710 - val_accuracy: 0.8533\n"]}],"source":["model_gmp1d.compile(optimizer='adam',\n","              loss='binary_crossentropy',\n","              metrics=['accuracy'])\n","              \n","epochs=5  # 몇 epoch를 훈련하면 좋을지 결과를 보면서 바꾸어 봅시다. \n","\n","history_gmp1d = model_gmp1d.fit(partial_x_train,\n","                    partial_y_train,\n","                    epochs=epochs,\n","                    batch_size=512,\n","                    validation_data=(x_val, y_val),\n","                    verbose=1,\n","                    callbacks=[early_stopping])"]},{"cell_type":"code","execution_count":24,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":2221,"status":"ok","timestamp":1642994099852,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"9Xixu_sCc6ZX","outputId":"3a711e95-4428-4429-f754-338b7782bad1"},"outputs":[{"name":"stdout","output_type":"stream","text":["1537/1537 - 2s - loss: 0.3834 - accuracy: 0.8461 - 2s/epoch - 1ms/step\n","[0.38342639803886414, 0.8461053371429443]\n"]}],"source":["# 모델 평가\n","results_gmp1d = model_gmp1d.evaluate(X_test,  y_test, verbose=2)  # (loss, accuracy)\n","\n","print(results_gmp1d)  # (0.3834, 0.8461)"]},{"cell_type":"markdown","metadata":{"id":"YGSw46_ULh46"},"source":["- Model 비교 (loss, accuracy)\n","\n","|Num|Model|Loss|Accuracy|\n","|:---:|:---:|:---:|:---:|\n","|1|LSTM|0.3667|0.8479|\n","|2|1-D CNN|0.4554|0.8424|\n","|3|GlobalMaxPooling1D() 레이어 1개|0.3834|0.8461|\n","\n","세 모델 다 성능이 비슷하다. 그 중에서 상대적으로 LSTM 성능이 제일 좋다.\n","- accuracy만 생각하면 값이 가장 높은 LSTM 성능이 제일 좋다.\n","- loss만 생각하면 값이 가장 낮은 LSTM 성능이 제일 좋다."]},{"cell_type":"markdown","metadata":{"id":"DkeeV0CK4Mzq"},"source":["### 6) Loss, Accuracy 그래프 시각화"]},{"cell_type":"markdown","metadata":{"id":"KpL4b2fvObHw"},"source":["- Model 1: LSTM"]},{"cell_type":"code","execution_count":25,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":14,"status":"ok","timestamp":1642994099852,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"ndJnZEyaOdDs","outputId":"f10e4bd9-4907-4060-cffb-685c573b23c4"},"outputs":[{"name":"stdout","output_type":"stream","text":["dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])\n"]}],"source":["history_dict = history_lstm.history\n","print(history_dict.keys()) # epoch에 따른 그래프를 그려볼 수 있는 항목들"]},{"cell_type":"code","execution_count":26,"metadata":{"executionInfo":{"elapsed":8,"status":"ok","timestamp":1642994099852,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"ohKPhTUjOe-s"},"outputs":[],"source":["acc = history_dict['accuracy']\n","val_acc = history_dict['val_accuracy']\n","loss = history_dict['loss']\n","val_loss = history_dict['val_loss']\n","\n","epochs = range(1, len(acc) + 1)"]},{"cell_type":"code","execution_count":27,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":313},"executionInfo":{"elapsed":856,"status":"ok","timestamp":1642994100701,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"SByfW36fOgIe","outputId":"52ed6ef5-11ed-4566-8c64-f729f48bb62b"},"outputs":[{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAbgAAAEoCAYAAAAqrOTwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXxU1fn48c9DCAQIO4KsgsqirAkBBATBpUWlUBRU9KumtCq444JoVRBr7cKvWhfc92JxaaVUoFrcwKXKIiogCmiQICiELawm4fn9ce4kk8lMMgkzmSXP+/W6r9y599w7z9zAPDnnnnuOqCrGGGNMsqkV6wCMMcaYaLAEZ4wxJilZgjPGGJOULMEZY4xJSpbgjDHGJCVLcMYYY5KSJTiTFERkoYhcGumysSQiOSJyehTOqyJyvLf+qIjcEU7ZKrzPRSLyZlXjLOe8w0QkN9LnNcmndqwDMDWXiOz1e1kfOAQUea+vUNXZ4Z5LVc+MRtlkp6oTI3EeEekIfAukqmqhd+7ZQNi/Q2MizRKciRlVTfeti0gO8BtVXRRYTkRq+740jTEmXNZEaeKOrwlKRG4Rka3AMyLSVEReF5FtIrLTW2/nd8y7IvIbbz1bRN4XkZle2W9F5Mwqlu0kIotFJF9EFonIwyLytxBxhxPj3SLygXe+N0Wkhd/+i0Vko4jkichvy7k+A0Rkq4ik+G0bIyKfe+v9ReQjEdklIltE5CERqRPiXM+KyO/8Xt/sHfO9iEwIKHu2iHwqIntEZJOITPfbvdj7uUtE9orIQN+19Tt+kIgsFZHd3s9B4V6b8ojICd7xu0RktYiM8tt3lois8c65WURu8ra38H4/u0Rkh4gsERH7Pkwy9gs18epooBlwDHA57t/qM97rDsAB4KFyjh8AfAW0AP4EPCUiUoWyLwKfAM2B6cDF5bxnODFeCPwKaAnUAXxfuCcCj3jnb+O9XzuCUNWPgX3AqQHnfdFbLwIme59nIHAacGU5cePFMMKL5wygMxB4/28fcAnQBDgbmCQiv/T2DfV+NlHVdFX9KODczYD5wAPeZ/sLMF9Emgd8hjLXpoKYU4F/A296x10DzBaRrl6Rp3DN3Q2BHsDb3vYbgVzgKKAVcBtg4xYmGUtwJl4dBqap6iFVPaCqear6D1Xdr6r5wD3AKeUcv1FVn1DVIuA5oDXuiyzssiLSAegH3KmqP6nq+8C8UG8YZozPqOrXqnoAeBno420fC7yuqotV9RBwh3cNQvk7MB5ARBoCZ3nbUNXlqvo/VS1U1RzgsSBxBHOeF98qVd2HS+j+n+9dVf1CVQ+r6ufe+4VzXnAJcZ2qvuDF9XdgLfALvzKhrk15TgLSgT94v6O3gdfxrg1QAJwoIo1UdaeqrvDb3ho4RlULVHWJ2sC8SccSnIlX21T1oO+FiNQXkce8Jrw9uCaxJv7NdAG2+lZUdb+3ml7Jsm2AHX7bADaFCjjMGLf6re/3i6mN/7m9BJMX6r1wtbVzRKQucA6wQlU3enF08Zrftnpx/B5Xm6tIqRiAjQGfb4CIvOM1we4GJoZ5Xt+5NwZs2wi09Xsd6tpUGLOq+v8x4H/ec3HJf6OIvCciA73tfwbWA2+KyDciMjW8j2ESiSU4E68C/5q+EegKDFDVRpQ0iYVqdoyELUAzEanvt619OeWPJMYt/uf23rN5qMKqugb3RX4mpZsnwTV1rgU6e3HcVpUYcM2s/l7E1WDbq2pj4FG/81ZU+/ke13TrrwOwOYy4Kjpv+4D7Z8XnVdWlqjoa13w5F1czRFXzVfVGVT0WGAXcICKnHWEsJs5YgjOJoiHuntYu737OtGi/oVcjWgZMF5E63l//vyjnkCOJ8VVgpIic7HUImUHF/z9fBK7DJdJXAuLYA+wVkW7ApDBjeBnIFpETvQQbGH9DXI32oIj0xyVWn224JtVjQ5x7AdBFRC4Ukdoicj5wIq458Uh8jKvtTRGRVBEZhvsdzfF+ZxeJSGNVLcBdk8MAIjJSRI737rXuxt23LK9J2CQgS3AmUdwP1AO2A/8D/lNN73sRrqNGHvA74CXc83rBVDlGVV0NXIVLWluAnbhOEOXx3QN7W1W3+22/CZd88oEnvJjDiWGh9xnexjXfvR1Q5EpghojkA3fi1Ya8Y/fj7jl+4PVMPCng3HnASFwtNw+YAowMiLvSVPUnXEI7E3fdZwGXqOpar8jFQI7XVDsR9/sE14lmEbAX+AiYparvHEksJv6I3Vc1Jnwi8hKwVlWjXoM0xhwZq8EZUw4R6Scix4lILa8b/WjcvRxjTJyzkUyMKd/RwD9xHT5ygUmq+mlsQzLGhMOaKI0xxiQla6I0xhiTlCzBGWOMSUpJcw+uRYsW2rFjx1iHYYwxphotX758u6oeFWxf0iS4jh07smzZsliHYYwxphqJSOAQcMWsidLPoVCP7xpjjEk4luA8+/fDCSfA9dfD9iMaW8EYY0w8sATnOXQITj8dHnwQjjsO7r3XJT1jjDGJKWmeg8vKytJI3INbswZuvRXmzYO2bWHGDLj0UkgJNSmLSSgFBQXk5uZy8ODBigubmEpLS6Ndu3akpqbGOhQTx0RkuapmBd1nCS64JUvg5pvh44+he3f44x/hrLMg5JzQJiF8++23NGzYkObNmxN6gm8Ta6pKXl4e+fn5dOrUKdbhmDhWXoKzJsoQhgyBjz6CV15xzZcjR8Kpp8LSpbGOzByJgwcPWnJLACJC8+bNraZtjogluHKIwNixrtnyoYdg9Wro3x/OPx82bIh1dKaqLLklBvs9mSNlCS4Mqalw1VWwfj3ccQe8/rrrcXnttbBtW6yjM4kkLy+PPn360KdPH44++mjatm1b/Pqnn34q99hly5Zx7bXXVvgegwYNikis7777LiNHjozIuYyJBUtwldCoket0sn49/OpXMGuW63H5+99bj0sTnubNm7Ny5UpWrlzJxIkTmTx5cvHrOnXqUFhYGPLYrKwsHnjggQrf48MPP4xkyMYkLEtwVdC6NTz2GHzxhbsv99vfQufO8NRTUM73kzFBZWdnM3HiRAYMGMCUKVP45JNPGDhwIBkZGQwaNIivvvoKKF2jmj59OhMmTGDYsGEce+yxpRJfenp6cflhw4YxduxYunXrxkUXXYSvU9mCBQvo1q0bffv25dprr62wprZjxw5++ctf0qtXL0466SQ+//xzAN57773iGmhGRgb5+fls2bKFoUOH0qdPH3r06MGSJUsifs2MCUfSDNUVCyecAHPnwvvvux6Xv/kN/OUvrsfl2Wdbj8t4d/31sHJlZM/Zpw/cf3/lj8vNzeXDDz8kJSWFPXv2sGTJEmrXrs2iRYu47bbb+Mc//lHmmLVr1/LOO++Qn59P165dmTRpUpku9Z9++imrV6+mTZs2DB48mA8++ICsrCyuuOIKFi9eTKdOnRg/fnyF8U2bNo2MjAzmzp3L22+/zSWXXMLKlSuZOXMmDz/8MIMHD2bv3r2kpaXx+OOP8/Of/5zf/va3FBUVsd+aN0yMWA0uAk4+GT78EF59FQoK4Be/gGHD4JNPYh2ZSRTjxo0jxXvYcvfu3YwbN44ePXowefJkVq9eHfSYs88+m7p169KiRQtatmzJDz/8UKZM//79adeuHbVq1aJPnz7k5OSwdu1ajj322OLu9+EkuPfff5+LL74YgFNPPZW8vDz27NnD4MGDueGGG3jggQfYtWsXtWvXpl+/fjzzzDNMnz6dL774goYNG1b1shhzRKwGFyEicO65MGoUPPkkTJ8OAwbAuHHuHt3xx8c6QhOoKjWtaGnQoEHx+h133MHw4cN57bXXyMnJYdiwYUGPqVu3bvF6SkpK0Pt34ZQ5ElOnTuXss89mwYIFDB48mDfeeIOhQ4eyePFi5s+fT3Z2NjfccAOXXHJJRN/XmHBYDS7CUlNh0iTXEWXaNJg/3zVlXnMN/PhjrKMziWD37t20bdsWgGeffTbi5+/atSvffPMNOTk5ALz00ksVHjNkyBBmz54NuHt7LVq0oFGjRmzYsIGePXtyyy230K9fP9auXcvGjRtp1aoVl112Gb/5zW9YsWJFxD+DMeGISYITkREi8pWIrBeRqeWUO1dEVESCPqUezxo2dLW49evh17+GRx5xtbjf/Q727Yt1dCaeTZkyhVtvvZWMjIyI17gA6tWrx6xZsxgxYgR9+/alYcOGNG7cuNxjpk+fzvLly+nVqxdTp07lueeeA+D++++nR48e9OrVi9TUVM4880zeffddevfuTUZGBi+99BLXXXddxD+DMeGo9qG6RCQF+Bo4A8gFlgLjVXVNQLmGwHygDnC1qpY7Dlekh+qKtLVr3RiXc+e6Xph33eUeNahtjcTV6ssvv+SEE06IdRgxt3fvXtLT01FVrrrqKjp37szkyZNjHVYZ9vsyFYm3obr6A+tV9RtV/QmYA4wOUu5u4I9AUozV060bvPaa63HZsSNcfjn06uUGdU6S4UBNAnniiSfo06cP3bt3Z/fu3VxxxRWxDsmYiItFgmsLbPJ7nettKyYimUB7VZ1f3olE5HIRWSYiy7YlyJAigwfDBx/AP/8JRUUwejSccoob1NmY6uJ7wHzNmjXMnj2b+vXrxzokYyIu7jqZiEgt4C/AjRWVVdXHVTVLVbOOOuqo6AcXISIwZgysWuXuzX39NZx0kutxuW5drKMzxpjkEIsEtxlo7/e6nbfNpyHQA3hXRHKAk4B5idjRpCKpqTBxouuIMn06LFwIJ54IV19tPS6NMeZIxSLBLQU6i0gnEakDXADM8+1U1d2q2kJVO6pqR+B/wKiKOpkksvR090jB+vVuNJRHH3VjXN59t/W4NMaYqqr2BKeqhcDVwBvAl8DLqrpaRGaIyKjqjieeHH20a7JcvRrOOAPuvNM9WvD44zbGpTHGVFZM7sGp6gJV7aKqx6nqPd62O1V1XpCyw5K59hZM166uE8oHH7ia3BVXQM+e8K9/WY/LRDd8+HDeeOONUtvuv/9+Jk2aFPKYYcOG4XsE5qyzzmLXrl1lykyfPp2ZM2eW+95z585lzZqSp3HuvPNOFi1aVJnwg7JpdUy8irtOJqbEoEGwZIl7vEAVfvlLGDoU/ve/WEdmqmr8+PHMmTOn1LY5c+aENR4kuFkAmjRpUqX3DkxwM2bM4PTTT6/SuYxJBJbg4pyIS2yrVrl7c+vWwcCBbqbxr7+OdXSmssaOHcv8+fOLJzfNycnh+++/Z8iQIUyaNImsrCy6d+/OtGnTgh7fsWNHtm/fDsA999xDly5dOPnkk4un1AH3jFu/fv3o3bs35557Lvv37+fDDz9k3rx53HzzzfTp04cNGzaQnZ3Nq6++CsBbb71FRkYGPXv2ZMKECRw6dKj4/aZNm0ZmZiY9e/Zk7dq15X4+m1bHxBMbRyNB1K7tmiovushNyfOnP7lRUa64wt2ra9Uq1hEmoBjMl9OsWTP69+/PwoULGT16NHPmzOG8885DRLjnnnto1qwZRUVFnHbaaXz++ef06tUr6HmWL1/OnDlzWLlyJYWFhWRmZtK3b18AzjnnHC677DIAbr/9dp566imuueYaRo0axciRIxk7dmypcx08eJDs7GzeeustunTpwiWXXMIjjzzC9ddfD0CLFi1YsWIFs2bNYubMmTz55JMhP59Nq2PiidXgEkx6uktoGza45PbYY64jyowZsHdvrKMz4fBvpvRvnnz55ZfJzMwkIyOD1atXl2pODLRkyRLGjBlD/fr1adSoEaNGlfTPWrVqFUOGDKFnz57Mnj075HQ7Pl999RWdOnWiS5cuAFx66aUsXry4eP8555wDQN++fYsHaA7FptUx8cRqcAmqVSt4+GG47jq47Tb3mMGsWW6My1//2sa4DEuM5ssZPXo0kydPZsWKFezfv5++ffvy7bffMnPmTJYuXUrTpk3Jzs7m4MGqjVKXnZ3N3Llz6d27N88++yzvvvvuEcXrm3LnSKbbsWl1TCxYDS7BdeniJlr98EPo3Nk9ON6jh2u+tB6X8Sk9PZ3hw4czYcKE4trbnj17aNCgAY0bN+aHH35g4cKF5Z5j6NChzJ07lwMHDpCfn8+///3v4n35+fm0bt2agoKC4iluABo2bEh+fn6Zc3Xt2pWcnBzWr18PwAsvvMApp5xSpc9m0+qYeGIJLkkMHAiLF7vE5hsKzDfTuIk/48eP57PPPitOcL7pZbp168aFF17I4MGDyz0+MzOT888/n969e3PmmWfSr1+/4n133303AwYMYPDgwXTr1q14+wUXXMCf//xnMjIy2LBhQ/H2tLQ0nnnmGcaNG0fPnj2pVasWEydOrNLnsml1TDyp9ulyoiXep8upToWF8PTTrtly61aX7O691z1fV9PZ9CuJxX5fpiLxNl2OibLatd10POvXu84n//0vdO/uZhrfujXW0RljTPWwrghJrEEDuOMO19tyxgzX4/KFF+Cmm9ySnh7rCI0xSeXQIcjLg+3bS5Zt20q/9t922WWuqSlKLMHVAC1bwkMPlfS4vOsu99D4tGlucOfU1FhHaIyJO4cPw44dZZNTeUkrSCemYk2bQosWbunQATIzXY+4KDriBCciDYADqnpYRLoA3YCFqlpwxNGZiOrcGV55xQ31NWUKXHml6yn/hz+40VJEYh1h9VBVpKZ82ASWLP0D4oKqe1A2nFqVb9mxwyW5YOrXL0lWRx3lvlx8r33b/F83axaTv6QjUYNbDAwRkabAm7jpcM4HLorAuU0UnHQSvPce/PvfMHUqnHOOG/fyT39yM44ns7S0NPLy8mjevLkluTimquTl5ZGWlhbrUOLToUPh1ar8t3vDw5VRu3bpZNSjR9kE5Z+0mjd3CS4BHHEvShFZoaqZInINUE9V/yQiK1W1T2RCDE9EelGuWOGq0c2aQaNGNaJKU1gIzzzjmiu3bHE1uXvvBb/e5UmloKCA3NzcKj9EbapPWloa7dq1IzXZ29CLimDnzvBqVb6loqbA8hJU4NK4cUJ/15XXizISNTgRkYG4GtuvvW0pEThv9Tp0CLyx/ABISSlJdpVZmjRxxyaI2rXdfd4LL4T77nO1uB493L256dPdHHXJJDU1lU6dOsU6DJOsVF3yCbdW5WsKDFXRaNCgdILq2rXipkAbxqhYJGpwpwA3Ah+o6h9F5FjgelW9NhIBhuuIa3AFBbBwofvHVtGye3f552rSpPKJsWlTqFOn6vFHyI8/upnEH30U6taFG290PS5tmECTFFQrv+zeHX4ni+3b3XdJMKmp4dWq/JsC69Wr3uuTgMqrwUX0QW8RqQWkq+qeiJ00TNX6oHdhIezaFV4y9F927gx90xZcv/3yEmCoffXqRbyJYf161+PylVdcL8xp01xNL9lbi6qdKhw4APv2lSz794d+HWpfYWHoL2jf+1TnEo/vGUkilWsKPOoo91diAjcFxquoJjgReRGYCBThOpg0Av6qqn8+ohNXUkKMZHL4MOzZU/nEuGNH6L8KwVW1KltjbNYsrP9wH3/selwuXuw6St17r+uUUmP+n6q65utwE1BlX1dliph69VzTVYMG7mZ//fruLw+R4AuE3heNpbrfL9rv2ahR2aTVtKk1BcaJaCe4laraR0QuAjKBqcByVQ0+kVWUJESCqypV92XoXxMMNzGW9wWakhJWItSmzXh/TTN+/2gzPl7fjBMGNOaPM1M4+eTquwQhqbrkH80EVF6tO5i6dUsnIN96JF7Xqwe1bAAiY3yi3ckkVURSgV8CD6lqgYjYAyyRJOKaL9PT3QOSlXHwYPgJccsWWL3are8paWUWYAjgG9/+8MfCriFN2NKgGU2ObUa9NmHeYywqinwC2rfPnbcyUlODJ5HGjaFNm8olnWD7EqiTkTHJLBIJ7jEgB/gMWCwixwDVfg/OhJCWBq1bu6UyCgpC3mcs+mEH37yzg2+W7aDRFzvounUH7dM3UHu3V7s80vsdKSmhE0rLlkdWC/I15xljkl5UZhMQkdqqWrWZEasoqZso49S2ba7H5SOPuA6gN94IN994mIaHdwevIdauHV5C8t1PMsaYCkT7HlxjYBow1Nv0HjBDVSvoSx9ZluBiZ/16+O1v4eWX3b34adPcbAZWUTLGRFu0p8t5GsgHzvOWPcAzETivSRDHHw8vveR6XJ54Ilx9tfv56qs2q7gxJnYikeCOU9VpqvqNt9wFHBuB85oE078/vPMOvP6660g4blzJTOPGGFPdIpHgDohIcYdxERkMHIjAeU0CEoGzz4bPPoOnnoJNm+CUU2DUKHjxRVi+vPxh9IwxJlIicQ+uN/A80NjbtBO4VFU/P8LYKsXuwcWn/fvhr391U/L4PXlAmzZuWL0uXdxP39Kxo/WyN8aEr1qG6hKRRgCqukdErlfV+yNy4jBZgotvhw7BunXw1Vfw9dfup2/ZubOkXJ067p6ef9LzJcLmzWMXvzEmPlXbWJR+b/idqlbyieQjYwkuMam68Wn9E55v2bDBDbHo07x52cTXtSscd1xcjFNtjImBaI9kEvQ9o3Rek2RE3KMFRx1FmaG/Cgvh22/LJr4FC9wcdj4pKdCpU9nmzq5d3XQ/9kidMTVTtBJcudVCERkB/BU3b9yTqvqHgP0TgatwAzjvBS5X1TVRitXEqdq13QDPnTvDyJGl9+3aVbqp07f+9ttudDKfRo3KJr4uXdySIJMSG2OqqMpNlCKST/BEJriZvYMmTxFJAb4GzgBycTMQjPdPYCLSyDfljoiMAq5U1RHlxWNNlAbcuMibNgVv8ty0qXTZ9u2DN3m2b2/jGRuTKKLSRKmqVZ0Csz+wXlW/8YKbA4wGihNcwHxyDaigRmiMT61acMwxbvnZz0rv27+/pKOL//L886UfXUhLc7XGYMmvcWOMMQkiFhMatQX8/5bOBQYEFhKRq4AbgDrAqdUTmklm9etD795u8acKW7eW7eG5ciW89lrpyQpatSrd1Olb79TJhiYzJt7E7Yx9qvow8LCIXAjcDlwaWEZELgcuB+hQ2WlkjPGIlEy4MGxY6X0//eR6cwbW+l57zfX+9Kld2/XmDFbra9HCOroYEwuxSHCbgfZ+r9t520KZAzwSbIeqPg48Du4eXKQCNManTh044QS3BNqxI/i9vv/8xyVGn6ZNg/fwPP541xxqjImOWCS4pUBnEemES2wXABf6FxCRzqq6znt5NrAOY+JMs2ZurM2BA0tvLyqCnJyyD7QvWuTu9/mIuJFbAnt4du0Kbdtarc+YI1XtCU5VC0XkauAN3GMCT6vqahGZASxT1XnA1SJyOlCAN/RXdcdpTFWlpLjmyuOOgzPPLL0vP79s4vv6azcg9f79JeUaNAhe6+vSxU3sboypWFRGMokFe0zAJDJV2Lw5eJPnxo2lpx3yjeMZOJpLhw7W5GlqnliMZGKMqQQRaNfOLaedVnrfgQPBO7rMmeMeePd39NGu2fOYY9xP3+J7dMIebjc1iSU4Y+JcvXrQo4db/KnCtm0u2eXklCwbN8KyZfDPf0JBQeljWrYMnvx8P6350yQTS3DGJCgRl7BatoQhQ8ruLyqCLVtcwvNPfjk57hm/f/2rdG9PcANaB0t+vvVGjaL7mYyJJEtwxiSplJSSZs/Bg8vuP3wYfvihbPLLyYFVq2D+/NLjeoJ75CFU8uvYEZo0iepHMqZSLMEZU0PVqlXygHvgow7gmkB//LFs8svJcc2ib7xRuucnuKHMQiW/jh1dgrTHH0x1sQRnjAlKxA1N1qoVDCgzmF7JXH6ByW/jRvjmGzezw969pY9JTy+/CdRGfTGRZAnOGFMl/nP5ZQXppK3qZmsPTH6+9cWLYc+e0sfUr19+E2jLlpYATfgswRljokLEjfbSrBlkZgYvs2tX8OS3cSN89JFLkP7q1St55CFYImzVyqY6MiUswRljYqZJE+jTxy3B7NkTvAk0J8c9CpGXV7p83brugfdQtcDWrV3nG1MzWIIzxsStRo2gZ0+3BLN3b/DHIHJy3GMQP/5YunxqakkCDPY8YNu2lgCTiSU4Y0zCSk+H7t3dEsy+ffDdd8F7gi5Y4OYB9Fe7tpvR3T/5dejgtvl+1qsXzU9kIskSnDEmaTVoEHq6I3DDoH33XfBa4Jtvwvfflz2mefOySc9/vU0blyhN7NmvwRhTY9WrVzJgdTCHDkFuLmza5Jbvviv5+c038N57sHt36WNq1XJJzj/pBSZCexyieliCM8aYEOrWLZn6KJQ9e0onP//1Zctg7lyXKP2lpZUkvFCJsGHD6H62msASnDHGHIFGjcq/D+gbFNu/9uefCBctck2hgTOXNWkSuhm0QwfXIaZOneh/vkRmCc4YY6LIf1DsYA/Eg5v14fvvyzaD+hLhRx/Bjh1lz9uqVfn3A2v6c4GW4IwxJsZSU0seYA9l376ShBeYAFetgoULy44NmprqBtsu735g48bJez/QEpwxxiSABg2gWze3BKPqannBOsRs2gRLlrgOM0VFpY9LTw/dDNq+vUuQifpohCU4Y4xJAiLuEYbmzUOPDFNU5J79C9Yh5rvv4NNPyz4cD2680fLuB8brCDGW4IwxpoZISXGdU9q2DT5FErg5AH2PRgQmwnXr4K23ID+/7HnbtCn/fmDz5tXfFGoJzhhjTLG0NDj+eLeEsnt36A4xn3wC//xn2dni69Urm/SGD4dhw6L3WSzBGWOMqZTGjd3So0fw/YcPu6bOUPcD33gDtmxxSdASnDHGmIRRqxYcfbRb+vULXuann8rW8iLNEpwxxphqV6dO9B9Ur8GPABpjjElmooHjwyQoEdkGbIzAqVoA2yNwnuqQSLFCYsWbSLFCYsWbSLFCYsWbSLFCZOI9RlWPCrYjaRJcpIjIMlUNMaBOfEmkWCGx4k2kWCGx4k2kWCGx4k2kWCH68VoTpTHGmKRkCc4YY0xSsgRX1uOxDqASEilWSKx4EylWSKx4EylWSKx4EylWiHK8dg/OGGNMUrIanDHGmKRkCc4YY0xSqpEJTkSeFpEfRWRViP0iIg+IyHoR+VxEMqs7Rr9YKop1mIjsFpGV3nJndccYEE97EXlHRNaIyGoRuS5Imbi4vmHGGjfXV0TSROQTEfnMi/euIGXqishL3rX9WEQ6Vn+kYceaLSLb/O27mDcAACAASURBVK7tb2IRq188KSLyqYi8HmRfXFzXgJjKizferm2OiHzhxbIsyP7ofCeoao1bgKFAJrAqxP6zgIWAACcBH8dxrMOA12N9Tf3iaQ1keusNga+BE+Px+oYZa9xcX+96pXvrqcDHwEkBZa4EHvXWLwBeiuNYs4GHYn1d/eK5AXgx2O87Xq5rJeKNt2ubA7QoZ39UvhNqZA1OVRcDO8opMhp4Xp3/AU1EpHX1RFdaGLHGFVXdoqorvPV84EugbUCxuLi+YcYaN7zrtdd7meotgb3ERgPPeeuvAqeJVPcsXGHHGjdEpB1wNvBkiCJxcV19wog30UTlO6FGJrgwtAU2+b3OJY6/+ICBXlPQQhHpHutgfLxmnAzcX+/+4u76lhMrxNH19ZqlVgI/Av9V1ZDXVlULgd1A8+qN0gkjVoBzvSapV0WkfTWH6O9+YApwOMT+uLmunorihfi5tuD+uHlTRJaLyOVB9kflO8ESXOJbgRuLrTfwIDA3xvEAICLpwD+A61V1T6zjKU8FscbV9VXVIlXtA7QD+otIiBm5Yi+MWP8NdFTVXsB/KakhVSsRGQn8qKrLY/H+lRVmvHFxbf2crKqZwJnAVSIytDre1BJccJsB/7942nnb4o6q7vE1BanqAiBVRFrEMiYRScUljNmq+s8gReLm+lYUazxeXy+WXcA7wIiAXcXXVkRqA42BvOqNrrRQsapqnqoe8l4+CfSt7tg8g4FRIpIDzAFOFZG/BZSJp+taYbxxdG198Wz2fv4IvAb0DygSle8ES3DBzQMu8Xr2nATsVtUtsQ4qGBE52ncvQET6436nMftC82J5CvhSVf8SolhcXN9wYo2n6ysiR4lIE2+9HnAGsDag2DzgUm99LPC2enfxq1M4sQbcYxmFuwda7VT1VlVtp6odcR1I3lbV/wsoFhfXFcKLN16urRdLAxFp6FsHfgYE9gqPyndCjZzwVET+jusd10JEcoFpuJvgqOqjwAJcr571wH7gV7GJNKxYxwKTRKQQOABcEKv/eJ7BwMXAF979F4DbgA4Qd9c3nFjj6fq2Bp4TkRRcon1ZVV8XkRnAMlWdh0vYL4jIelznpAviONZrRWQUUOjFmh2jWIOK0+saUhxf21bAa97fibWBF1X1PyIyEaL7nWBDdRljjElK1kRpjDEmKVmCM8YYk5QswRljjElKluCMMcYkJUtwxhhjkpIlOGNiTESK/EZ9XykiUyN47o4SYiYKY5JdjXwOzpg4c8Ab0soYE0FWgzMmTnlzaP3Jm0frExE53tveUUTe9gbSfUtEOnjbW4nIa97A0J+JyCDvVCki8oS4edne9EYWQUSuFTcX3uciMidGH9OYqLEEZ0zs1Qtoojzfb99uVe0JPIQbQR7coM/PeQPpzgYe8LY/ALznDQydCaz2tncGHlbV7sAu4Fxv+1QgwzvPxGh9OGNixUYyMSbGRGSvqqYH2Z4DnKqq33iDQm9V1eYish1oraoF3vYtqtpCRLYB7fwG2fVNA/RfVe3svb4FSFXV34nIf4C9uBkS5vrN32ZMUrAanDHxTUOsV8Yhv/UiSu69nw08jKvtLfVGyTcmaViCMya+ne/38yNv/UNKBvu9CFjirb8FTILiyUYbhzqpiNQC2qvqO8AtuOlfytQijUlk9hebMbFXz282A4D/qKrvUYGmIvI5rhY23tt2DfCMiNwMbKNk5PXrgMdF5Ne4mtokINSUIynA37wkKMAD3rxtxiQNuwdnTJzy7sFlqer2WMdiTCKyJkpjjDFJyWpwxhhjkpLV4IwxxiQlS3DGGGOSkiU4Y4wxSckSnDHGmKRkCc4YY0xSsgRnjDEmKVmCM8YYk5QswRljjElKluBM3BCRhSJyaaTLxpI3aenpUTiv+k2A+qiI3BFO2Sq8z0Ui8mZV4zQmlmwkE3NERMR/DrH6uEGBi7zXV6jq7OqPKn5440n+RlUXRfi8CnRW1fWRKuvNHfctbr64wkjEaUws2WwC5oj4T9RZ3pe5iNS2L00TL+zfY81gTZQmKkRkmIjkisgtIrIVN71LUxF5XUS2ichOb72d3zHvishvvPVsEXlfRGZ6Zb8VkTOrWLaTiCwWkXwRWSQiD4vI30LEHU6Md4vIB9753hSRFn77LxaRjSKSJyK/Lef6DBCRrSKS4rdtjDc1DiLSX0Q+EpFdIrJFRB4SkTohzvWsiPzO7/XN3jHfi8iEgLJni8inIrJHRDaJyHS/3Yu9n7tEZK+IDPRdW7/jB4nIUhHZ7f0cFO61qeR1biYiz3ifYaeIzPXbN1pEVnqfYYOIjPC2l2oOFpHpvt+ziHT0mmp/LSLfAW9721/xfg+7vX8j3f2Oryci/8/7fe72/o3VE5H5InJNwOf5XETGBPusJnYswZloOhpoBhwDXI779/aM97oDcAB4qJzjBwBfAS2APwFPiYhUoeyLwCdAc2A6cHE57xlOjBfi5mBrCdQBbgIQkROBR7zzt/Herx1BqOrHwD7g1IDzvuitFwGTvc8zEDgNuLKcuPFiGOHFcwbQGQi8/7cPuARogpvRe5KI/NLbN9T72URV01X1I/8DRaQZMB94wPtsfwHmi0jzgM9Q5toEUdF1fgHX5N3dO9d9Xgz9geeBm73PMBTICXU9gjgFOAH4ufd6Ie46tQRWAP5N6jOBvsAg3L/jKcBh4Dng/3yFRKQ30BZ3bUw8UVVbbInIgvuiOd1bHwb8BKSVU74PsNPv9bu4Jk6AbGC93776gAJHV6Ys7suzEKjvt/9vwN/C/EzBYrzd7/WVuAlKAe4E5vjta+Bdg9NDnPt3wNPeekNc8jkmRNnrgdf8XitwvLf+LPA7b/1p4A9+5br4lw1y3vuB+7z1jl7Z2n77s4H3vfWLgU8Cjv8IyK7o2lTmOgOtcYmkaZByj/niLe/fn/d6uu/37PfZji0nhiZemca4BHwA6B2kXBqwE3dfE1winFXd/99sqXixGpyJpm2qetD3QkTqi8hjXpPPHlyTWBP/ZroAW30rqrrfW02vZNk2wA6/bQCbQgUcZoxb/db3+8XUxv/cqroPyAv1Xrja2jkiUhc4B1ihqhu9OLp4zXZbvTh+j6vNVaRUDMDGgM83QETe8ZoGdwMTwzyv79wbA7ZtxNVefEJdm1IquM7tcb+znUEObQ9sCDPeYIqvjYikiMgfvGbOPZTUBFt4S1qw9/L+Tb8E/J+I1MLNtP7CEcRkosQSnImmwC66NwJdgQGq2oiSJrFQzY6RsAVoJiL1/ba1L6f8kcS4xf/c3ns2D1VYVdfgEsSZlG6eBNfUuRZXS2gE3FaVGHA1WH8vAvOA9qraGHjU77wVdan+Htek6K8DsDmMuAKVd5034X5nTYIctwk4LsQ59+Fq7z5HBynj/xkvBEbjmnEb42p5vhi2AwfLea/ngItwTcf7NaA518QHS3CmOjXENfvs8u7nTIv2G3o1omXAdBGpIyIDgV9EKcZXgZEicrLXIWQGFf8fexG4DvcF/0pAHHuAvSLSDZgUZgwvA9kicqKXYAPjb4irHR307mdd6LdvG65p8NgQ514AdBGRC0WktoicD5wIvB5mbIFxBL3OqroFd29sltcZJVVEfAnwKeBXInKaiNQSkbbe9QFYCVzglc8CxoYRwyFcLbs+rpbsi+Ewrrn3LyLSxqvtDfRq23gJ7TDw/7DaW9yyBGeq0/1APdxfx/8D/lNN73sRrqNGHu6+10u4L7Zgqhyjqq4GrsIlrS24+zS5FRz2d1zHh7dVdbvf9ptwyScfeMKLOZwYFnqf4W1gvffT35XADBHJx90zfNnv2P3APcAH4npvnhRw7jxgJK72lYfrdDEyIO5wVXSdLwYKcLXYH3H3IFHVT3CdWO4DdgPvUVKrvANX49oJ3EXpGnEwz+Nq0JuBNV4c/m4CvgCWAjuAP1L6O/N5oCfunq6JQ/agt6lxROQlYK2qRr0GaZKXiFwCXK6qJ8c6FhOc1eBM0hORfiJynNekNQJ332VuRccZE4rX/Hsl8HisYzGhWYIzNcHRuC7se3HPcE1S1U9jGpFJWCLyc9z9yh+ouBnUxJA1URpjjElKVoMzxhiTlJJmsOUWLVpox44dYx2GMcaYarR8+fLtqnpUsH1Jk+A6duzIsmXLYh2GMcaYaiQigaPrFLMmSmOMMUkpaWpwxhhj4pcq5OXBpk2Qm+t+9ugBQ4dWfGxVWYIzxhhzRIIlr9zcsusHD5Y+7pprEjjBeQ/V/hVIAZ5U1T8E7D8GN97bUbihcP5PVXO9fZcCt3tFf6eqz0UzVmOMMWWFSl7+iStY8qpdG9q2hfbtISsLxoyBdu3c0r69+9mqVXRjj1qC86a9eBg38WIusFRE5nkjqPvMBJ5X1edE5FTgXuBiv8FXs3Cjfy/3jg02fYYxxpgqiHbyqhXjXh7RrMH1x01C+Q2AiMzBDZHkn+BOBG7w1t+hZPiknwP/VdUd3rH/BUbgBqY1xhhTgWRPXuGIZoJrS+mJF3OBAQFlPsNN9PhXYAzQUESahzi2bcCxiMjlwOUAHToETntljDHJqaLk5ft5KGDOjGRKXuGIdSeTm4CHRCQbN6PvZqAo3INV9XG8wU6zsrJszDFjTMKLRPLq1w/OOad04kq25BWOaCa4zZSeWbgdATP/qur3uBocIpIOnKuqu0RkMzAs4Nh3oxirMcZEnSps3162h6Elr+iIZoJbCnQWkU64xHYBpWcPRkRa4GYXPgzciutRCfAG8HsRaeq9/pm33xhj4pIlr/gTtQSnqoUicjUuWaUAT6vqahGZASxT1Xm4Wtq9IqK4JsqrvGN3iMjduCQJMMPX4cQYY6qbJa/ElDTT5WRlZamNRWmMqYrCQti8GXJy3LJxY8n6d99VnLz8k5Ylr+olIstVNSvYvlh3MjHGmKgLTGCBy6ZNUOTXvU0E2rSBY46xmlciswRnjEl4VU1gHTvC4MHup//Svj3UrVvNH8JEnCU4Y0zcswRmqsISnDEm5goL3X2uUAksN9cSmKk8S3DGmKirSgJr29YlqyFDgiewOnWq+UOYhGMJzhhzxCyBmXhkCc4YU6GCgtIJzL8bvSUwE68swRljyiSwYDWww4dLylsCM4nAEpwxNcCRJLChQy2BmcRkCc6YJFCVBNaunUtWp5ziHmi2BGaSTYUJTkR+Acz3BkQ2xsTITz/Bt9/CunUly9dfw/r17jmw8hJYYA2sXTtLYCb5hVODOx+4X0T+gRsweW2UYzKmxioqch04/BOYbz0np3RHjiZNoHNn9xzYccdZAjMmUIUJTlX/T0QaAeOBZ72R/58B/q6q+dEO0Jhkc/iwG5UjMIGtWwcbNrjmRp8GDaBLF+jbFy64wCW0Ll3cz+bNXU3NGBNcWPfgVHWPiLwK1AOuB8YAN4vIA6r6YDQDNCYRqcIPP5RNYF9/7ZLYgQMlZdPS4Pjj4YQTYNSokgTWuTMcfbQlMWOqKpx7cKOAXwHHA88D/VX1RxGpD6wBLMGZGkkV8vLKJjDf+t69JWVTU+HYY13S+tnPShJY586uOdFGpTcm8sKpwZ0L3Keqi/03qup+Efl1dMIyJn7s3h08ga1bBzt3lpSrVcvd/+rSBU4+uSSBdekCHTq4+cOMMdUnnP9y04EtvhciUg9opao5qvpWtAIzpjrt2+d6IwZrUty2rXTZDh1c4jr//NL3xDp1so4dxsSTcBLcK8Agv9dF3rZ+UYnImCg5eBC++aZ0EvOtf/996bKtW7ukFXhP7LjjoF692MRvjKmccBJcbVX9yfdCVX8SEfs71cSlgoLgz4qtWwfffefum/m0aOGS1hlnlG5OPP54SE+P3WcwxkRGOAlum4iMUtV5ACIyGtge3bCMCa2oyCWrYPfFvv229LNijRu7pDV4MGRnl25SbNIkZh/BGFMNwklwE4HZIvIQIMAm4JKoRmVqvMOHXbNhqGfFfvqppGyDBi5hZWTAeeeVblJs0cK62RtTU4XzoPcG4CQRSfde763gEGPCogo//hi8Y8f69aWfFatb1zUddu0Kv/hF6SZFe1bMGBNMWB2XReRsoDuQJt43iarOiGJcJsmsWQPLl5etkeX7jYVTu7brxNG5M5x+eukkZs+KGWMqK5wHvR8F6gPDgSeBscAnUY7LJIkNG+DWW+GVV9xr37NinTvDoEGlmxOPOcaeFTPGRE44XyeDVLWXiHyuqneJyP8DFkY7MJPYduyA3/0OHnrIjeJx550wfrwbzcOeFTPGVIdwEtxB7+d+EWkD5AGtoxeSSWSHDsHDD8Pdd8OePTBhAtx1F7RpE+vIjDE1TTgJ7t8i0gT4M7ACUOCJqEZlEo6qa4acOtV11R8xAv70J+jZM9aRGWNqqnITnIjUAt5S1V3AP0TkdSBNVXdXS3QmIbz/Ptx0E3z8MfTqBW+84QYUNsaYWCq3X5o3i/fDfq8PWXIzPuvWwbnnwpAhbkbpp5+GFSssuRlj4kM4Ha/fEpFzRexJI+Ns3w7XXQcnnuhqa3ff7br+/+pXkJIS6+iMMcYJ5x7cFcANQKGIHMSNZqKq2iiqkZm4c/AgPPgg3HOPe37tsstg+nT3oLUxxsSbcEYyaVgdgZj4dfgwzJkDt90GGzfC2We7DiQnnhjryIwxJrRwHvQeGmx74ASoJjm9957rQLJsGfTpA089BaedFuuoTNLyTfegWrIEvq5qmaocB250gpSUkqV27dLrtWrZWHFxKpwmypv91tOA/sBy4NSoRGTiwldfwS23wL/+pRzf9iBzHtrHuLP2UevAPli2380Q6r/s31/xa/9h/v3nralpr6N17mh8wVdngklkviQYmPzKS4yVLRftc1R3jNXwh0E4TZS/8H8tIu2B+8M5uYiMAP4KpABPquofAvZ3AJ4DmnhlpqrqAhHpCHwJfOUV/Z+qTgznPWscVTe0fmUSTTnJqXD3PnZt2U+TPfv4G/toIPuRzQpXVyKmtDQ3xL//Ur9+2SFMAv9x16TX0Tq3SOklcFs4Zap6XKzPHa33P3wYCgvdH2i+xf91uPsqU+7gwSM7h+91vP/hcPPN7n5HlFRl5L9c4ISKColICu4RgzO8Y5aKyDxVXeNX7HbgZVV9REROBBYAHb19G1S1TxXiiz8FBZVONJV67V8zCkedOqUTT4MGFNVrwMadTVn1bVv2FDXgmBMb0HdoA6RF/eDJKtTr+vVtVGRj4oVqdJJwpM4xaFBUP3449+AexI1eAu6xgj64EU0q0h9Yr6rfeOeZA4wG/BOcAr7emI2B78MLOwpU4YcfIlILKvO6oKBysaSkhE4qLVsGTzIVJR7/134jGh8+DLNnw29/655lGzUK/vhH6NYtwtfXGFP9RNz/9xo6ink4n3qZ33oh8HdV/SCM49riJkf1yQUGBJSZDrwpItcADYDT/fZ1EpFPgT3A7aq6JPANRORy4HKADh06hBFSOQ4dgtZhDrEpEjqJNG1a9cTjW6ppNOK333YdSD79FPr2heefh2HDquWtjTEm6sJJcK8CB1W1CFzTo4jUV9X9EXj/8cCzqvr/RGQg8IKI9AC2AB1UNU9E+gJzRaS7qu7xP1hVHwceB8jKyjqyxua6dd0oweUlHt+2tLSE7jW1Zg1MmQLz50OHDq4Gd8EF1rJoaq6CggJyc3M5ePBgxYVNTKSlpdGuXTtSU1PDPiacBPcWrmblm8m7HvAmUFHj6Wagvd/rdt42f78GRgCo6kcikga0UNUfgUPe9uUisgHoQunaZGSJwJVXRu308eCHH2DaNHjiCUhPd02R117r8rUxNVlubi4NGzakY8eO2KBN8UdVycvLIzc3l06dOoV9XDh/s6epqi+54a3XD+O4pUBnEekkInWAC4B5AWW+A04DEJETcI8hbBORo7xOKojIsUBn4Jsw3tMEsX+/m5vt+OPdc2xXXeUmIp0yxZKbMQAHDx6kefPmltzilIjQvHnzSteww6nB7RORTFVd4b1RX+BARQepaqGIXA28gXsE4GlVXS0iM4BlqjoPuBF4QkQm4zqcZKuqeg+XzxCRAuAwMFFVd1TqkxmKiuCFF+D222HzZhgzBv7wBzeLtjGmNEtu8a0qv59wEtz1wCsi8j1uHMqjgfPDObmqLsB1/fffdqff+hpgcJDj/gH8I5z3MMEtWuQ6kHz2GfTv74baOvnkWEdljAkmLy+P07whgrZu3UpKSgpHHXUUAJ988gl1yul4tmzZMp5//nkeeOCBct9j0KBBfPjhh5ELOgGE86D3UhHpBnT1Nn2lqpXs926qy6pVrulx4ULo2NEltvPOS+g+McYkvebNm7Ny5UoApk+fTnp6OjfddFPx/sLCQmqH6OqflZVFVlZWhe9R05IbhHEPTkSuAhqo6ipVXQWki0hy98ZIQFu2uNH9e/eGjz6CmTNh7Vo4/3xLbsYkouzsbCZOnMiAAQOYMmUKn3zyCQMHDiQjI4NBgwbx1VduoKd3332XkSNHAi45TpgwgWHDhnHssceWqtWlp6cXlx82bBhjx46lW7duXHTRRag34smCBQvo1q0bffv25dprry0+r7+cnByGDBlCZmYmmZmZpRLnH//4R3r27Env3r2ZOnUqAOvXr+f000+nd+/eZGZmsmHDhuhcsCDCaaK8TFX9Jz3dKSKXAbOiF5YJ1759Lpn9+c9uxK5rr3X33Jo3j3VkxiSm668HrzIVMX36wP1hDXBYWm5uLh9++CEpKSns2bOHJUuWULt2bRYtWsRtt93GP/5R9k7O2rVreeedd8jPz6dr165MmjSpTNf6Tz/9lNWrV9OmTRsGDx7MBx98QFZWFldccQWLFy+mU6dOjB8/PmhMLVu25L///S9paWmsW7eO8ePHs2zZMhYuXMi//vUvPv74Y+rXr8+OHa7bxEUXXcTUqVMZM2YMBw8e5PDhw5W/EFUUToJLERFRL8V7vRur50lkE1JRETz7LNxxh6u9jRsH994Lxx0X68iMMZEybtw4UrxZhHfv3s2ll17KunXrEBEKQoyQdPbZZ1O3bl3q1q1Ly5Yt+eGHH2jXrl2pMv379y/e1qdPH3JyckhPT+fYY48t7oY/fvx4Hn/88TLnLygo4Oqrr2blypWkpKTw9ddfA7Bo0SJ+9atfUb++62TfrFkz8vPz2bx5M2PGjAHcs2zVKZwE9x/gJRF5zHt9BbAweiGZirzxhutAsmoVDBwIr74a9SHdjKkxqlLTipYGDRoUr99xxx0MHz6c1157jZycHIaFGHaobt26xespKSkUFhZWqUwo9913H61ateKzzz7j8OHD1Z60KiOc5+BuAd4GJnrLF7iHvU01++wz+NnPYMQI92zbK6/ABx9YcjOmJti9ezdt27YF4Nlnn434+bt27co333xDTk4OAC+99FLIOFq3bk2tWrV44YUXKPIGez/jjDN45pln2L/fDXK1Y8cOGjZsSLt27Zg7dy4Ahw4dKt5fHSpMcKp6GPgYyMENoHwqbiobU002b4YJEyAjw008et99britsWOtA4kxNcWUKVO49dZbycjIqFSNK1z16tVj1qxZjBgxgr59+9KwYUMaN25cptyVV17Jc889R+/evVm7dm1xLXPEiBGMGjWKrKws+vTpw8yZMwF44YUXeOCBB+jVqxeDBg1i69atEY89FNEQ8wWJSBfcWJHjge3AS8BNqnpMtUVXCVlZWbpsWfRG8oqF/HzXeWTmTHfP7Zpr3Kj/TZvGOjJjksuXX37JCSdUOAtY0tu7dy/p6emoKldddRWdO3dm8uTJsQ6rWLDfk4gsV9Wgz0mUV4Nbi6utjVTVk1X1QaCSE4+ZqigshMceg86d4e67YfRo1+V/5kxLbsaY6HniiSfo06cP3bt3Z/fu3VxxxRWxDumIlNfJ5Bzc+JHviMh/gDm4kUxMlKjCggVuktsvv3Qjj/zrXzAgcJIhY4yJgsmTJ8dVje1IhazBqepcVb0A6Aa8gxuyq6WIPCIiP6uuAGuKTz+F00+HkSPd/Kj//CcsXmzJzRhjqiqcTib7VPVFVf0FbsqbT3E9K00EbNoEl17qJhz97DN44AFYvdoNjGwdSIwxpuoqNY+5qu7ETTBa9uk/Uyl79riR/e+7zzVNTpkCU6dCkyaxjswYY5JDpRKcOXIFBW7C0enTYds2uOgiuOceOCYu+6YaY0ziCudBbxMBqjBvHvTs6SYcPfFEWLoU/vY3S27G1HTDhw/njTfeKLXt/vvvZ9KkSSGPGTZsGL5Ho8466yx27dpVpsz06dOLn0cLZe7cuaxZs6b49Z133smiRYsqE37csgRXDZYtg+HDXXd/cD0j33kHwpjhwhhTA4wfP545c+aU2jZnzpyQAx4HWrBgAU2qeH8jMMHNmDGD008/vUrnijeW4KJo40bXBNmvnxt5ZNYs+OILGDXKOpAYY0qMHTuW+fPn89NPPwFuSprvv/+eIUOGMGnSJLKysujevTvTpk0LenzHjh3Zvn07APfccw9dunTh5JNPLp5SB9wzbv369aN3796ce+657N+/nw8//JB58+Zx880306dPHzZs2EB2djavvvoqAG+99RYZGRn07NmTCRMmcOjQoeL3mzZtGpmZmfTs2ZO1a9eWiSkeptWxe3BRsGuXG9n/r391iey22+CWW6BRo1hHZoypUAzmy2nWrBn9+/dn4cKFjB49mjlz5nDeeechItxzzz00a9aMoqIiTjvtND7//HN69eoV9DzLly9nzpw5rFy5ksLCQjIzM+nbty8A55xzDpdddhkAt99+O0899RTXXHMNo0aNYuTIkYwdO7bUuQ4ePEh2djZvvfUWXbp04ZJLLuGRRx7h+uuvB6BFixasWLGCWbNmMXPmTJ588slSx8fDtDpWg4ugn36CBx+E4493Q2ydfz58/bXrRGLJzRhTHv9mSv/myZdffpnMzEwyMjJYvXp1qebEQEuWLGHMmDHUr1+fRo0aMWrUqOJ9q1atYsiQIfTs2ZPZs2ez1JrgdQAACXdJREFUevXqcuP56quv6NSpE126dAHg0ksvZfHixcX7zznnHAD69u1bPECzv4KCAi677DJ69uzJuHHjiuMOd1od3/4jYTW4CFCFuXNdLW3dOjj1VJfgMjNjHZkxptJiNF/O6NGjmTx5MitWrGD//v307duXb7/9lpkzZ7J06VKaNm1KdnY2Bw8erNL5s7OzmTt3Lr179+bZZ5/l3XffPaJ4fVPuhJpuJx6m1bEa3BH6+GMYMgTOOQdSU2H+fFi0yJKbMaZy0tPTGT58OBMmTCiuve3Zs4cGDRrQuHFjfvjhBxYuLH8qzqFDhzJ37lwOHDhAfn4+//73v4v35efn07p1awoKCpg9e3bx9oYNG5Kfn1/mXF27diUnJ4f169cDblaAU045JezPEw/T6liCq6Jvv4ULLoCTTnK1tscecyORnHWWdSAxxlTN+PHj+eyzz4oTXO/evcnIyKBbt25ceOGFDB48uNzjMzMzOf/88+nduzdnnnkm/fr1K9539913M2DAAAYPHky3bt2Kt19wwQX8+c9/JiMjo1THjrS0NJ555hnGjRtHz549qVWrFhMnTgz7s8TDtDohp8tJNNU1Xc7One6e2oMPQkqKm1n75puhYcOov7UxJkpsupzEUNnpcuweXJgOHXLd/O++2/WSzM52694Eu8YYY+KMNVFWQBVefdWNPHLDDW5Q5E8/haeftuRmjDHxzBJcOT76CAYPhnHjoH59WLgQ3nwTeveOdWTGGGMqYgkuiA0bXFIbNMh1JnnySffc54gR1oHEmGSVLP0RklVVfj+W4Pzk5cHkyXDCCW5m7enTXQ/JX//adSgxxiSntLQ08vLyLMnFKVUlLy+v0s/SWScTz/ffQ/fubp62CRNgxgxo3TrWURljqkO7du3Izc1l27ZtsQ7FhJCWlka7du0qdYwlOE+bNq72NmaMm9LGGFNzpKam0qlTp1iHYSLMEpyfO++MdQTGGGMixe7BGWOMSUqW4IwxxiSlpBmqS0S2ARsjcKoWwPYInKc6JFKskFjxJlKskFjxJlKskFjxJlKsEJl4j1HVo4LtSJoEFykisizUuGbxJpFihcSKN5FihcSKN5FihcSKN5FihejHa02UxhhjkpIlOGOMMUnJElxZj8c6gEpIpFghseJNpFghseJNpFghseJNpFghyvHaPThjjDFJyWpwxhhjklKNTHAi8rSI/Cgiq0LsFxF5QETWi8jnIpJZ3TH6xVJRrMNEZLeIrPSWmI7HIiLtReQdEVkjIqtF5LogZeLi+oYZa9xcXxFJE5FPROQzL967gpSpKyIvedf2YxHpWP2Rhh1rtohs87u2v4lFrH7xpIjIpyLyepB9cXFdA2IqL954u7Y5IvKFF8uyIPuj852gqjVuAYYCmcCqEPvPAhYCApwEfBzHsQ4DXo/1NfWLpzWQ6a03BL4GTozH6xtmrHFzfb3rle6tpwIfAycFlLkSeNRbvwB4KY5jzQYeivV19YvnBuDFYL/veLmulYg33q5tDtCinP1R+U6okTU4VV0M7CinyGjgeXX+BzQRkZjMLRBGrHFFVbeo6gpvPR/4Egic+zwurm+YscYN73rt9V6mekvgTfTRwHPe+qvAaSLVP4thmLHGDRFpB5wNPBmiSFxc1//f3v2FSFWGcRz//lr3Ysmw0DBplb3IK/unhVRGF0JRFN4oaPSfIJTIuimtmyC66iJiS4j+EJKWRKVsYYuLKxEUVIYaURcSQoqxKqhIImlPF+cdd5pm3BGcOe+e/X1u5sw7h5lnnmXPc97znnnfmjbinWw6ckyYkgWuDdcCf9Q9P0jGBz7g9nQp6CtJC8oOpiZdxllIcfZeL7v8XiBWyCi/6bLUHmAMGImIlrmNiLPACWBmd6MstBErwPJ0SepTSXO7HGK9N4AXgH9avJ5NXpOJ4oV8cgvFyc0OSbslPdXk9Y4cE1zgJr+fKKaquQl4E9hWcjwASJoOfAY8FxEny47nQiaINav8RsS5iLgZ6AcWS7q+zHgupI1YvwAGIuJGYITxHlJXSXoAGIuI3WV8/sVqM94sclvnzohYBNwHPC3prm58qAtcc4eA+jOe/tSWnYg4WbsUFBHbgV5Js8qMSVIvRcHYHBGfN9klm/xOFGuO+U2xHAd2Afc2vHQ+t5KmATOAY92N7r9axRoRxyLiTHr6HnBLt2NLlgDLJB0AtgBLJW1q2CenvE4Yb0a5rcVzKD2OAVuBxQ27dOSY4ALX3BDwaLqz5zbgREQcLjuoZiRdUxsLkLSY4m9a2gEtxfI+8GtEvN5ityzy206sOeVX0tWSrkzbfcDdwG8Nuw0Bj6XtFcBopFH8bmon1oYxlmUUY6BdFxEvRkR/RAxQ3EAyGhEPN+yWRV6hvXhzyW2K5XJJV9S2gXuAxrvCO3JMmJILnkr6mOLuuFmSDgIvUwyCExFvA9sp7urZD/wFPFFOpG3FugJYI+kscBpYVdY/XrIEeAT4OY2/ALwEzIPs8ttOrDnldw6wUVIPRaH9JCK+lPQK8GNEDFEU7A8l7ae4OWlVxrGulbQMOJtifbykWJvKNK8tZZzb2cDWdJ44DfgoIoYlrYbOHhM8k4mZmVWSL1GamVklucCZmVklucCZmVklucCZmVklucCZmVklucCZlUzSubpZ3/dIWn8J33tALVaiMKu6Kfk7OLPMnE5TWpnZJeQenFmm0hpar6V1tL6XdF1qH5A0mibS3SlpXmqfLWlrmhh6r6Q70lv1SHpXxbpsO9LMIkhaq2ItvH2StpT0Nc06xgXOrHx9DZcoV9a9diIibgDeophBHopJnzemiXQ3A4OpfRD4Ok0MvQj4JbXPBzZExALgOLA8ta8HFqb3Wd2pL2dWFs9kYlYySaciYnqT9gPA0oj4PU0K/WdEzJR0FJgTEX+n9sMRMUvSEaC/bpLd2jJAIxExPz1fB/RGxKuShoFTFCskbKtbv82sEtyDM8tbtNi+GGfqts8xPvZ+P7CBorf3Q5ol36wyXODM8ray7vG7tP0t45P9PgR8k7Z3Amvg/GKjM1q9qaTLgLkRsQtYR7H8y/96kWaTmc/YzMrXV7eaAcBwRNR+KnCVpH0UvbAHU9szwAeSngeOMD7z+rPAO5KepOiprQFaLTnSA2xKRVDAYFq3zawyPAZnlqk0BndrRBwtOxazyciXKM3MrJLcgzMzs0pyD87MzCrJBc7MzCrJBc7MzCrJBc7MzCrJBc7MzCrJBc7MzCrpXwk3Ujl49x8dAAAAAElFTkSuQmCC","text/plain":["<Figure size 432x288 with 2 Axes>"]},"metadata":{"needs_background":"light"},"output_type":"display_data"}],"source":["# Subplot들의 간격 조정하기 참고: https://tempdev.tistory.com/35\n","\n","# plt.subplots\n","fig, axs = plt.subplots(2 ,1, constrained_layout=True)\n","\n","# Training and Validation Loss 그래프\n","axs[0].plot(epochs, loss, 'b', label='Training loss')\n","axs[0].plot(epochs, val_loss, 'r', label='Validation loss')\n","axs[0].set_title('Training and validation loss')\n","axs[0].set_xlabel('Epochs')\n","axs[0].set_ylabel('Loss')\n","axs[0].legend()\n","\n","# Training and Validation Accuracy 그래프\n","axs[1].plot(epochs, acc, 'b', label='Training acc')\n","axs[1].plot(epochs, val_acc, 'r', label='Validation acc')\n","axs[1].set_title('Training and validation accuracy')\n","axs[1].set_xlabel('Epochs')\n","axs[1].set_ylabel('Accuracy')\n","axs[1].legend()\n","\n","\n","plt.show()"]},{"cell_type":"markdown","metadata":{"id":"dfF3p0rfdtdI"},"source":["- Model 2: 1-D CNN"]},{"cell_type":"code","execution_count":28,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":19,"status":"ok","timestamp":1642994100701,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"cAHdZigidxdB","outputId":"5439c557-fe54-4a4b-a8d6-7eccd90be82f"},"outputs":[{"name":"stdout","output_type":"stream","text":["dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])\n"]}],"source":["history_dict = history_1d_cnn.history\n","print(history_dict.keys()) # epoch에 따른 그래프를 그려볼 수 있는 항목들"]},{"cell_type":"code","execution_count":29,"metadata":{"executionInfo":{"elapsed":13,"status":"ok","timestamp":1642994100702,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"BbaEnnw1dxaM"},"outputs":[],"source":["acc = history_dict['accuracy']\n","val_acc = history_dict['val_accuracy']\n","loss = history_dict['loss']\n","val_loss = history_dict['val_loss']\n","\n","epochs = range(1, len(acc) + 1)"]},{"cell_type":"code","execution_count":30,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":313},"executionInfo":{"elapsed":600,"status":"ok","timestamp":1642994101289,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"sVtVBzoUd4E3","outputId":"b7430fdb-8686-4e39-ab60-78d5f0d8e0d1"},"outputs":[{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAbgAAAEoCAYAAAAqrOTwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydeXxU5fX/34eQkEDYA7IECPu+byp1weUnioJsCqIScUVtXaqtbV0o1da2fFu1LhVRZNHGnVqEalERq20FAZVVtoAgKIQtMQSScH5/PHeYJQtDyGQmyXm/Xvc1d3nuvWfuJPOZc57nOUdUFcMwDMOoatSItgGGYRiGEQlM4AzDMIwqiQmcYRiGUSUxgTMMwzCqJCZwhmEYRpXEBM4wDMOokpjAGVUCEVkkIpPKu200EZFMEbkgAtdVEengrf9VRB4Ip20Z7jNRRN4rq52lXPdcEdlR3tc1qh41o22AUX0RkZyAzdrAEaDQ275ZVV8K91qqenEk2lZ1VPWW8riOiKQBW4F4VS3wrv0SEPZnaBjljQmcETVUNdm3LiKZwA2quji0nYjU9H1pGoZhhIuFKI2YwxeCEpGfi8huYJaINBSRBSKyR0T2e+upAecsEZEbvPV0Efm3iEz32m4VkYvL2LatiCwVkWwRWSwiT4nIvBLsDsfG34jIJ9713hORlIDj14jINhHJEpFflfJ8BovIbhGJC9g3SkS+9NYHich/ROSAiOwSkSdFJKGEa70oIg8HbN/rnfOtiEwOaTtcRFaKyCER+UZEpgYcXuq9HhCRHBE5w/dsA84/U0SWichB7/XMcJ9NaYhIV+/8AyKyRkRGBBy7RETWetfcKSL3ePtTvM/ngIjsE5GPRcS+D6sY9oEasUozoBHQBrgJ97c6y9tuDRwGnizl/MHABiAF+APwvIhIGdq+DHwGNAamAteUcs9wbLwKuA5oCiQAvi/cbsAz3vVbePdLpRhU9X/AD8B5Idd92VsvBO7y3s8ZwPnAraXYjWfDMM+eC4GOQGj/3w/AtUADYDgwRUQu946d7b02UNVkVf1PyLUbAe8AT3jv7U/AOyLSOOQ9FHk2J7A5HvgH8J533o+Bl0Sks9fkeVy4uy7QA/jA2/9TYAfQBDgN+CVgeQurGCZwRqxyDHhIVY+o6mFVzVLVN1Q1V1WzgUeAc0o5f5uqPqeqhcBsoDnuiyzstiLSGhgIPKiqR1X138DbJd0wTBtnqerXqnoYeBXo4+0fCyxQ1aWqegR4wHsGJfE3YAKAiNQFLvH2oaqfq+p/VbVAVTOBZ4uxoziu8Oxbrao/4AQ98P0tUdWvVPWYqn7p3S+c64ITxI2qOtez62/AeuCygDYlPZvSOB1IBh71PqMPgAV4zwbIB7qJSD1V3a+qKwL2NwfaqGq+qn6slpi3ymECZ8Qqe1Q1z7chIrVF5FkvhHcIFxJrEBimC2G3b0VVc73V5JNs2wLYF7AP4JuSDA7Txt0B67kBNrUIvLYnMFkl3QvnrY0WkVrAaGCFqm7z7Ojkhd92e3b8FufNnYggG4BtIe9vsIh86IVgDwK3hHld37W3hezbBrQM2C7p2ZzQZlUN/DEQeN0xOPHfJiIficgZ3v4/ApuA90Rki4jcF97bMCoTJnBGrBL6a/qnQGdgsKrWwx8SKynsWB7sAhqJSO2Afa1KaX8qNu4KvLZ3z8YlNVbVtbgv8osJDk+CC3WuBzp6dvyyLDbgwqyBvIzzYFupan3grwHXPZH38y0udBtIa2BnGHad6LqtQvrPjl9XVZep6khc+HI+zjNEVbNV9aeq2g4YAdwtIuefoi1GjGECZ1QW6uL6tA54/TkPRfqGnke0HJgqIgner//LSjnlVGx8HbhURH7kDQiZxon/P18G7sAJ6WshdhwCckSkCzAlTBteBdJFpJsnsKH218V5tHkiMggnrD724EKq7Uq49kKgk4hcJSI1ReRKoBsunHgq/A/n7f1MROJF5FzcZ5ThfWYTRaS+qubjnskxABG5VEQ6eH2tB3H9lqWFhI1KiAmcUVl4DEgC9gL/Bf5ZQfediBuokQU8DLyCm69XHGW2UVXXALfhRGsXsB83CKI0fH1gH6jq3oD99+DEJxt4zrM5HBsWee/hA1z47oOQJrcC00QkG3gQzxvyzs3F9Tl+4o1MPD3k2lnApTgvNwv4GXBpiN0njaoexQnaxbjn/jRwraqu95pcA2R6odpbcJ8nuEE0i4Ec4D/A06r64anYYsQeYv2qhhE+IvIKsF5VI+5BGoZxapgHZxilICIDRaS9iNTwhtGPxPXlGIYR41gmE8MonWbAm7gBHzuAKaq6MromGYYRDhaiNAzDMKokFqI0DMMwqiQmcIZhGEaVpMr0waWkpGhaWlq0zTAMwzAqkM8//3yvqjYp7liVEbi0tDSWL18ebTMMwzCMCkREQlPAHcdClIZhGEaVpMp4cIZhGEaMcuwYbNsGX34JX33lli+/hIkT4f77I3ZbE7gA5s+Hnj2hfftoW2IYhlFJ2bfPL2I+IVu9GnJy/G3atXNfth06RNQUEziPo0fhuuvgwAEYOhSuvx5Gj4akpGhbZhiGEYMcOQLr1wcL2Vdfwc6AAhGNGjkhS0+HXr3cevfuULduhZhoAueRkOA+m9mz4fnn4eqroUED50HfcAP0Caf0omEYRlVDFbZvLypkGzZAQYFrk5AAXbs678AnZD17QosWIJGsaFU6VSaTyYABA7S8RlEeOwZLlsDMmfDmm+6HSr9+TugmTHDCZxiGUeU4eLCokH31FRw65G/Tpo0Tr0Ah69QJ4uOjYrKIfK6qA4o9ZgJXOvv2wUsvOa/uiy8gMRHGjXMhzLPPjuqPE8MwjLKRn+88sFAh277d36Z+/aJC1qOH2x9DmMCVA6qwYoXz6l5+2f2g6dgRJk+GSZOgefOI3dowDKNsqLo+sVAhW7fOiRxAzZrQpUuwkPXqBampleIXvAlcOZObC6+/7ry6pUshLg6GD3de3SWXuL8XwzCMCiU7241WDB2Kf+CAv01qalEh69zZ9aFVUmJO4Ly6Wo8DccBMVX20hHZjgNeBgapaqnpVpMAF8vXX8MIL8OKL8N13zpObNMl5dh07Vrg5hmFUdQoKYOPGokKWmelvU7euX8R8QtajBzRsGDWzI0VMCZyIxAFfAxfi6mstAyao6tqQdnWBd4AE4PZYFTgf+fmwcKHz6t55xw1UOeccNzBlzBibbmAYxkmiCrt3FxWydevcyDdw4aPOnYOFrGdPNxCkEoQXy4PSBC4awbRBwCZV3QIgIhm4KslrQ9r9Bvg9cG/Fmlc24uNh5Ei3fPutf7rBNdfA7bfDVVc5sevXL9qWGoYRc/zwgwsvho5gzMryt2nRwonXBRf4haxrV6hVK3p2xzjRELiWwDcB2zuAwYENRKQf0EpV3xGRSiFwgbRoAb/4Bfz8566P7vnnYdYseOYZN5/uhhuc4FXBaIFhGKVRWAibNhUVsi1bnMcGUKeOCyeOGhXcX9a4cXRtr4TE3HAIEakB/AlID6PtTcBNAK1bt46sYWWgRg0491y3/OUvbvTlzJnOo7vnHhe6vP56F8qsYWmvDaNq8d13RYVszRrIy3PHa9RwHfV9+7qOe5+QtW1rXwjlRDT64M4ApqrqRd72LwBU9Xfedn1gM+BLXNYM2AeMKK0fLtp9cCfDihXOq3vpJTevsn17NyglPd15f4ZhVCJyc2Ht2qJD8b//3t/mtNOC+8h69XLhReucP2VibZBJTdwgk/OBnbhBJlep6poS2i8B7on1QSZl4fBhlyll5kyXOaVGDTfN4Prr3bSDKCUGMAyjOI4dc6HEUCHbuNEfXkxKcrkWA0OLPXtC06bRtb0KE1ODTFS1QERuB97FTRN4QVXXiMg0YLmqvl3RNkWLpCSX63LiRBeW9003WLDA/eCbNMmJXadO0bbUMKoJqrB3ryvt4lt83tnq1c5bAzdCsUMHJ14TJvi9snbt3MhGIyawid4xRkEBLFrkQpgLFrg+6bPOcgNTxo6F2rWjbaFhVGIKCtww50AB8y3bt7vXw4eDz0lJKTo5uls3NxjEiDoxFaKMFFVF4ALZtQvmzHFit3Ej1KvnfizecAP0719tprkYRvgcPuwXquKWnTvdr8ZAmjRx88YCl9at/esNG9o/WwxjAlfJUYWPP3ZC99pr7n+4Vy8ndBMnupJLhlHlUYX9+4t6XIHLnj3B58TFQcuWRQXMJ2KtW1tYpJJjAleFOHgQ/vY3NzDl88/dHM/Ro11f3dChNrrYqMQcO+bCFsWFDX1LYFVocB3Zgd5WqIC1bGnJYas4JnBVlFWrnFc3b57Lp9q2rX+6QWpqtK0zjBCOHIFvvim5/+ubb/wZ7n00alSygLVp4/rHLHxYrTGBq+IcPgxvveXE7oMPnBc3bJjz6i67zKYbGBXEwYOl93/t3h3cXsRN/Cyu38u3XbdudN6LUWkwgatGbNniphvMmuUGizVtCtde68SuS5doW2dUWlRdZo6S+r62bXMCF0hCQrBohQpYamqlLtNixAYmcNWQggJ4913n1f3jH257yBA3MGXcOBvhbISQnw87dpTc97V9uz+DvY969UoffXjaadYpbEQcE7hqznffuekGM2e6+nV167rpBtdfDwMHWhdGteCHH0qe97Vtm3P3jx0LPue000oevNGmDTRoEJ33YhgBmMCFy4IFztVJSHAdVwkJwevF7fOtx8fH/K9VVfjkE+fVvfqqS8rQo4fz6q6+2pKVV1pUXVmV0iYvB5ZdATeyMDW15MEbrVpBYmJ03o9hnAQmcOHSrJlzd8pKXFzJAlgW0Szr8TDOOZSXwCtvJfDcC3EsWy4kJLjqHNdfD+efH/NaXTaOHXOhuPx890Mm8DWc9VM9Hsm2od5X7doli1ebNq70vKWUMqoAJnDhsn69G5KYnw9Hj7rFtx7uvkicU1BQPg+pBI7FJ3CUBA4XxHNEEzgWF09ivQTqNk4gPqkchVo1uoJQ0X/rNWq4912zpt/LP9F6WY7XrFk0G0ejRhZ7NqoFMZVsOaaJ1WGGPmEoT9EMWK+Rn0/i0aPUzD1K1sZ8Nq09yp5d+STsP0qrpkdpm5rPabWPUqPgqItrhnOfo0fDE5S4uLJ94depUz7iEKm2NWtWUTfYMCoPJnCVARG/RxRBagIdvGXrVjfV4J5ZsGOFm0/rm27QrVuYFywsDBY+kWAhqFnTvAzDMCKGhSiNUikshPfecwNT/v53F/E74wwndFdeCcnJ0bbQMIzqTGkhSouhGKUSFwcXXwyvv+4SsU+f7vLd3nCDG6dwww3w3/9WfPeWYRjGiTCBM8KmaVP46U9d/cdPPoErrnCJn884w003+POfXa1IwzCMWOCUBU5E6ohIDW+9k4iMEJFSsx+KyDAR2SAim0TkvmKO3yIiX4nIKhH5t4iE2+tjVAAicOaZLmy5ezc895ybPH733S614LhxLotKaNktwzCMiuSU++BE5HPgLKAh8AmwDDiqqhNLaB8HfA1cCOzw2k9Q1bUBbeqp6iFvfQRwq6oOK80O64OLPqtXO9GbO9fNK27VylU3uO46N3LdMAyjvIl0H5yoai4wGnhaVccB3UtpPwjYpKpbVPUokAGMDGzgEzePOoD18FQCfGHKnTvhlVega1eYNs2V8bnoInj8cVi82Hl91mdnGEakKY9pAiIiZwATgeu9faWlSGgJfBOwvQMYXMxFbwPuBhKA80q48U3ATQCtW7c+acONyFCrluufu+IKlyVq1ix48UU3GtNH48ZOELt3d6++datObhhGeVEeIcpzgJ8Cn6jq70WkHXCnqv6khPZjgWGqeoO3fQ0wWFVvL6H9VcBFqjqpNDssRBn7fP+9C2OuXg1r1vjXDwX4682b+wXPJ3rdullZMMMwiieimUxU9SPgI+9GNYC9JYmbx06gVcB2qrevJDKAZ07VTiP6NG0K553nFh+qrkpLoOCtXg1//avLmuYjLS3Y2+vRwyWesXzAhmGUxCkLnIi8DNwCFOIGjNQTkcdV9Y8lnLIM6CgibXHCNh64KuSaHVV1o7c5HNiIUSURcYNRWrVyVch9FBZCZmZRj++991xiFHCZsDp0CPb2evSAjh2tirlhGOXTB9dNVQ+JyERgEXAf8DlQrMCpaoGI3A68i+ure0FV14jINGC5qr4N3C4iFwD5wH6g1PCkUfWIi4P27d0yMmAIUn4+bNxY1OObP9+fUD8+3nl3oR5f27aWHtIwqhPl0Qe3BugDvAw8qaoficgXqtq7PAwMF+uDq97k5bliEKEeX2amv01SkuvPC/X4UlMtJaZhVFYiXU3gWSAT+AJYKiJtgEOlnmEY5UxiIvTp45ZAsrNd5pVAj++992D2bH+bevWKjujs0cP1GRqGUXmJSLJlEampqpEtYhaCeXDGybBvX7DorVkDX33l9vtISSnq7XXvDg0bRs9uwzCCiagHJyL1gYeAs71dHwHTgIOnem3DiBSNGsFZZ7nFh6or6B7avzd7tvMEfbRsWdTb69bNlagzDCN2KI8Q5QvAauAKb/saYBYus4lhVBpEoFkzt5x/vn+/KnzzTdH+vaefdn1/Ptq2LerxdeniJr4bhlHxlMcgk1Wq2udE+yKNhSiNiqawELZsKerxbdjg6uaBGw3asWNRj69DB1fv1TCMUyPSg0wOi8iPVPXf3s2GAIdPcI5hVHp84tWxI1x+uX//0aNuKkOgx/fll/Dmm/4cnAkJzrsL9fjS0mwqg2GUF+XhwfUG5gD1vV37gUmq+uUp2nZSFOfB5efns2PHDvIC40hGzJKYmEhqairxVXSWdm6um8oQ6vFt3+5vU7u2E7tQj69FC5vKYBjFUZoHV26jKEWkHrhKACJyp6o+Vi4XDpPiBG7r1q3UrVuXxo0bI/btENOoKllZWWRnZ9O2bdtom1OhHDrkpjKE9vHt3u1vU79+0RydPXpAkybRs9swYoEKEbiQG25X1QpN71+cwK1bt44uXbqYuFUSVJX169fTtWvXaJsSE+zd68Qu1OPbv9/fpmlTJ3a9e7s5gH37ujJFVdQJNowiRLoPrth7Rui6J42JW+XBPqtgUlLgnHPc4kPVeXaBgrd6NTz7rD85dUKC8+769vWLXu/ekJwcnfdhGNEiUgJn5SwNIwKIuJJCzZvDhRf69xcWwtdfw6pVsHKlW+bPdxXWfed16ODELlD4TjstOu/DMCqCMguciGRTvJAJkFRmi6oQWVlZnO9NqNq9ezdxcXE08TpNPvvsMxISEko8d/ny5cyZM4cnnnii1HuceeaZfPrpp6ds65IlS5g+fToLFiw45WsZFU9cnAtNdu0KEya4faquuvrKlX7h++wzePVV/3nNmhUVvXbtbCSnUTUos8CpqpWgPAGNGzdm1apVAEydOpXk5GTuueee48cLCgqoWcJkqAEDBjBgQLFh5SDKQ9yMqomISySdmgqXXebff+CAE7xAb++995wXCK64bO/ewcLXvbsLfRpGZcKmmlYw6enpJCYmsnLlSoYMGcL48eO54447yMvLIykpiVmzZtG5c+cgj2rq1Kls376dLVu2sH37du68805+8hNXUzY5OZmcnByWLFnC1KlTSUlJYfXq1fTv35958+YhIixcuJC7776bOnXqMGTIELZs2VKqp7Zv3z4mT57Mli1bqF27NjNmzKBXr1589NFH3HHHHYDrL1u6dCk5OTlceeWVHDp0iIKCAp555hnOCsx/ZcQcDRrAuee6xUdenhvMEujtvfAC/PCDOx4f79KRBYpenz4uUbVhxCrVRuDuvNP945YnffrAY2WYDLFjxw4+/fRT4uLiOHToEB9//DE1a9Zk8eLF/PKXv+SNN94ocs769ev58MMPyc7OpnPnzkyZMqXIfLGVK1eyZs0aWrRowZAhQ/jkk08YMGAAN998M0uXLqVt27ZM8MWvSuGhhx6ib9++zJ8/nw8++IBrr72WVatWMX36dJ566imGDBlCTk4OiYmJzJgxg4suuohf/epXFBYWkpube/IPxIg6iYnQv79bfBw7Bps2BYvewoXw4ov+Nu3aFQ1xNm9uc/aM2CAqAiciw4DHcQVPZ6rqoyHH7wZuAAqAPcBkVd1W4YZGiHHjxhEXFwfAwYMHmTRpEhs3bkREyPeVqw5h+PDh1KpVi1q1atG0aVO+++47UlNTg9oMGjTo+L4+ffqQmZlJcnIy7dq1Oz63bMKECcyYMaNU+/79738fF9nzzjuPrKwsDh06xJAhQ7j77ruZOHEio0ePJjU1lYEDBzJ58mTy8/O5/PLL6RNar8aotNSoAZ06ueXKK/37d+0KFr2VKyHwN1nTpn6x8wlfx47Wr2dUPBUucCISBzwFXAjsAJaJyNuqujag2UpggKrmisgU4A/AlUWvFj5l8bQiRZ2AtPMPPPAAQ4cO5a233iIzM5NzA+NGAdQKyNgbFxdHQUHRakThtDkV7rvvPoYPH87ChQsZMmQI7777LmeffTZLly7lnXfeIT09nbvvvptrr722XO9rxBa+UZyXXOLfd+gQfPFFsPD96U+uAju4Sgu9egWLXo8eznM0jEgRDQ9uELBJVbcAiEgGMBI4LnCq+mFA+/8CV1eohRXIwYMHadmyJQAvBsZ+yonOnTuzZcsWMjMzSUtL45VXXjnhOWeddRYvvfQSDzzwAEuWLCElJYV69eqxefNmevbsSc+ePVm2bBnr168nKSmJ1NRUbrzxRo4cOcKKFStM4Koh9eoVLT909KjL0BIoenPnuioM4EZ+duvm9/Z8/XpWb88oL6IhcC2BbwK2dwCDS2l/PbAoohZFkZ/97GdMmjSJhx9+mOHDh5f79ZOSknj66acZNmwYderUYeDAgSc8Z+rUqUyePJlevXpRu3ZtZnvlrx977DE+/PBDatSoQffu3bn44ovJyMjgj3/8I/Hx8SQnJzNnzpxyfw9G5SQhoWiV9WPHYOvWYNFbvNgJn4+0tKIhztRU69czTp6IpOoq9YYiY4FhqnqDt30NMFhVby+m7dXA7cA5qnqkmOM3ATcBtG7duv+2bcHddOvWrbO0T0BOTg7JycmoKrfddhsdO3bkrrvuirZZxWKfWfXku+/8gud73bjRX32hcePggSx9+kDnzs4LNKo30UjVVRo7gVYB26neviBE5ALgV5QgbgCqOgOYAS4XZfmbWjV47rnnmD17NkePHqVv377cfPPN0TbJMII47TS46CK3+MjJcWWGAkXviSdc6BMgKcn16wWKXs+eriKDYUB0PLiawNfA+ThhWwZcpaprAtr0BV7HeXobw7luScmWzRuoXNhnZpRGfr4rORQoeqtWucnr4EZqdulSNMTZuHF07TYiR0x5cKpaICK3A+/ipgm8oKprRGQasFxV3wb+CCQDr3kJeLer6oiKttUwjNgiPt55aT17gm8skyps2+afsrBqFSxdCi+/7D+vVauiotemjfXrVXWiMg9OVRcCC0P2PRiwfkGFG2UYRqVExA1MSUuDUaP8+/fuLdqv9847bqALuNGavkEwPuHr0gVKyJ5nVELsozQMo0qSkgIXXOAWH7m58NVXwaL3zDMuVRlArVrOOwwUvR49XH5Oo/JhAmcYRrWhdm0YPNgtPgoKXKmhwBDnm2/CzJn+Nq1b+yup+5auXd0EdiN2seQ5EWTo0KG8++67Qfsee+wxpkyZUuI55557Lr7BMpdccgkHfL3nAUydOpXp06eXeu/58+ezdq0/OcyDDz7I4sWLT8b8YlmyZAmXXnrpKV/HMGKFmjXdhPOJE2H6dDcvb+9e16/397/DI4/AkCGwYwc8/jikp8PAgc6ra98eRoyAX/wC5s1z4ujzBo3oYx5cBJkwYQIZGRlcFDD2OSMjgz/84Q9hnb9w4cITNyqB+fPnc+mll9KtWzcApk2bVuZrGUZ1Q8R5ba1bOwHzUVAAmze7ygurV7vXNWtg0SJ3DNxIzvbtnZcX6PV17mwlhyoa8+AiyNixY3nnnXc46k3cyczM5Ntvv+Wss85iypQpDBgwgO7du/PQQw8Ve35aWhp79+4F4JFHHqFTp0786Ec/YsOGDcfbPPfccwwcOJDevXszZswYcnNz+fTTT3n77be599576dOnD5s3byY9PZ3XX38dgPfff5++ffvSs2dPJk+ezJEjR47f76GHHqJfv3707NmT9evXl/r+9u3bx+WXX06vXr04/fTT+fLLLwH46KOP6NOnD3369KFv375kZ2eza9cuzj77bPr06UOPHj34+OOPT+3hGkYUqFnTCdXo0fDgg/DKK07ofvjBvb7yCtx/v5uft24d/O53rgBtr14uPNqtG4wbB1OnwmuvuVRmJeRXN8qB6uPBRaFeTqNGjRg0aBCLFi1i5MiRZGRkcMUVVyAiPPLIIzRq1IjCwkLOP/98vvzyS3r16lXsdT7//HMyMjJYtWoVBQUF9OvXj/5eXZPRo0dz4403AnD//ffz/PPP8+Mf/5gRI0Zw6aWXMnbs2KBr5eXlkZ6ezvvvv0+nTp249tpreeaZZ7jzzjsBSElJYcWKFTz99NNMnz6dmYEdESFYWR3DcCQk+D21QI4cgQ0b/J7e6tXua+iNN/xZWuLjnWj6zvd5fe3bW6aWU6X6CFyU8IUpfQL3/PPPA/Dqq68yY8YMCgoK2LVrF2vXri1R4D7++GNGjRpFbS9Fw4iAmMnq1au5//77OXDgADk5OUHh0OLYsGEDbdu2pVOnTgBMmjSJp5566rjAjR49GoD+/fvz5ptvlnotK6tjGKVTq5bz3kL/tQ8fdhPWA0Odn33mPMDAc7t0KRrqbNvWSg+FS/URuCjVyxk5ciR33XUXK1asIDc3l/79+7N161amT5/OsmXLaNiwIenp6eSVsWc6PT2d+fPn07t3b1588UWWLFlySvb6Su6cSrkdK6tjGKWTlOSfhhBITo4Lbfo8vjVr4N//Dp60npTkQp2BIzp79HD9hTZxPZjqI3BRIjk5maFDhzJ58uTj1bQPHTpEnTp1qF+/Pt999x2LFi0qsQ4cwNlnn016ejq/+MUvKCgo4B//+MfxfJLZ2dk0b96c/Px8XnrppeOld+rWrUt2dnaRa3Xu3JnMzEw2bdpEhw4dmDt3Luecc0Zoge0AACAASURBVE6Z3puV1TGM8iU52Y3QDC36ceiQ668LDHUuXgyBxTuSk4OFz+f1tWxZfYXPBK4CmDBhAqNGjSIjIwOA3r1707dvX7p06UKrVq0YMmRIqef369ePK6+8kt69e9O0adOgkje/+c1vGDx4ME2aNGHw4MHHRW38+PHceOONPPHEE8cHlwAkJiYya9Ysxo0bR0FBAQMHDuSWW24p0/uysjqGUTHUqwenn+6WQPbv9wufL9S5cCHMmuVvU7++E77QeXzNmlV94avwZMuRwpItVw3sMzOMU2fv3uAwp08As7L8bRo2LCp6PXpAkybRs7ssxFSyZcMwDCOypKTAOee4xYcqfP99UeHLyPBXYwAncKGi1707NGpU8e/jVDGBMwzDqAaIuLp7p50G553n368Ku3YFe3pr1rj+vcBu/GbNio7o7N7dhUBjlSovcKqKVPVAcxWhqoTLDaMyIQItWrjlwgv9+1VderLQrC0zZ7qJ7T5atiwqet26xUaC6iotcImJiWRlZdG4cWMTuRhHVcnKyiIxMTHaphiGgRO+Vq3cMmyYf/+xYy5PZ2io8+mng/NwtmlTNMzZtWvFVlyv0oNM8vPz2bFjR5nnmBkVS2JiIqmpqcTHx0fbFMMwTpLCQti6tWioc/168LIVIuImqvtE76KLgvsJy0LMDTIRkWHA47iK3jNV9dGQ42cDjwG9gPGq+nrRq5yY+Ph42rZte6rmGoZhGCcgLg46dHDLyJH+/aUlqFY9dYErjQoXOBGJA54CLgR2AMtE5G1VXRvQbDuQDtxT0fYZhmEY5YcvQbUvSbWPo0ddrs6I3juyly+WQcAmVd0CICIZwEjguMCpaqZ37FgU7DMMwzAiTEJC5MsHRSNlZ0vgm4DtHd6+k0ZEbhKR5SKyfM+ePeVinGEYhlE1qNSjKFV1BjADQET2iMi2crhsCrC3HK5TEVQmW6Fy2VuZbIXKZW9lshUql72VyVYoH3vblHQgGgK3E2gVsJ3q7TslVLVcEsyIyPKSRuTEGpXJVqhc9lYmW6Fy2VuZbIXKZW9lshUib280QpTLgI4i0lZEEoDxwNtRsMMwDMOowlS4wKlqAXA78C6wDnhVVdeIyDQRGQEgIgNFZAcwDnhWRNZUtJ2GYRhG5SYqfXCquhBYGLLvwYD1ZbjQZTSYEaX7loXKZCtULnsrk61QueytTLZC5bK3MtkKEba3ymQyMQzDMIxAotEHZxiGYRgRxwTOMAzDqJJUS4ETkRdE5HsRWV3CcRGRJ0Rkk4h8KSL9KtrGAFtOZOu5InJQRFZ5y4PFtasoRKSViHwoImtFZI2I3FFMm5h4vmHaGjPPV0QSReQzEfnCs/fXxbSpJSKveM/2fyKSVvGWhm1rujd/1fdsb4iGrQH2xInIShFZUMyxmHiuITaVZm+sPdtMEfnKs2V5Mccj852gqtVuAc4G+gGrSzh+CbAIEOB04H8xbOu5wIJoP9MAe5oD/bz1usDXQLdYfL5h2hozz9d7XsneejzwP+D0kDa3An/11scDr8SwrenAk9F+rgH23A28XNznHSvP9STsjbVnmwmklHI8It8J1dKDU9WlwL5SmowE5qjjv0ADEWleMdYFE4atMYWq7lLVFd56Nm4qSGgqtph4vmHaGjN4zyvH24z3ltBRYiOB2d7668D5EoViiGHaGjOISCowHJhZQpOYeK4+wrC3shGR74RqKXBhUG75MiuIM7xQ0CIR6R5tY3x4YZy+uF/vgcTc8y3FVoih5+uFpVYB3wP/UtUSn626OacHgcYVa6UjDFsBxnghqddFpFUxxyuKx4CfASUleI+Z5+pxInshdp4tuB8374nI5yJyUzHHI/KdYAJX+VkBtFHV3sBfgPlRtgcAEUkG3gDuVNVD0banNE5ga0w9X1UtVNU+uHmig0SkRzTtKY0wbP0HkKaqvYB/4feQKhQRuRT4XlU/j8b9T5Yw7Y2JZxvAj1S1H3AxcJu4mp8RxwSueCKSLzMSqOohXyhI3QT6eBFJiaZNIhKPE4yXVPXNYprEzPM9ka2x+Hw9Ww4AHwLDQg4df7YiUhOoD2RVrHXBlGSrqmapqq8i2Eygf0Xb5jEEGCEimUAGcJ6IzAtpE0vP9YT2xtCz9dmz03v9HngLVzYtkIh8J5jAFc/bwLXeyJ7TgYOquivaRhWHiDTz9QWIyCDcZxq1LzTPlueBdar6pxKaxcTzDcfWWHq+ItJERBp460m4osHrQ5q9DUzy1scCH6jXi1+RhGNrSB/LCFwfaIWjqr9Q1VRVTcMNIPlAVa8OaRYTzxXCszdWnq1nSx0RqetbB/4fEDoqPCLfCZW6XE5ZEZG/4UbHpYjLefkQrhMcVf0rLo3YJcAmIBe4LjqWhmXrWGCKiBQAh4Hx0frH8xgCXAN85fW/APwSaA0x93zDsTWWnm9zYLaIxOGE9lVVXSAi04Dlqvo2TrDnisgm3OCk8TFs60/E5Z8t8GxNj5KtxRKjz7VEYvjZnga85f1OrAm8rKr/FJFbILLfCZaqyzAMw6iSWIjSMAzDqJKYwBmGYRhVEhM4wzAMo0piAmcYhmFUSUzgDMMwjCqJCZxhRBkRKQzI+r5KRO4rx2unSQmVKAyjqlMt58EZRoxx2EtpZRhGOWIenGHEKF4NrT94dbQ+E5EO3v40EfnAS6T7voi09vafJiJveYmhvxCRM71LxYnIc+Lqsr3nZRZBRH4irhbelyKSEaW3aRgRwwTOMKJPUkiI8sqAYwdVtSfwJC6DPLikz7O9RLovAU94+58APvISQ/cD1nj7OwJPqWp34AAwxtt/H9DXu84tkXpzhhEtLJOJYUQZEclR1eRi9mcC56nqFi8p9G5VbSwie4Hmqprv7d+lqikisgdIDUiy6ysD9C9V7eht/xyIV9WHReSfQA6uQsL8gPpthlElMA/OMGIbLWH9ZDgSsF6Iv+99OPAUzttb5mXJN4wqgwmcYcQ2Vwa8/sdb/xR/st+JwMfe+vvAFDhebLR+SRcVkRpAK1X9EPg5rvxLES/SMCoz9ovNMKJPUkA1A4B/qqpvqkBDEfkS54VN8Pb9GJglIvcCe/BnXr8DmCEi1+M8tSlASSVH4oB5nggK8IRXt80wqgzWB2cYMYrXBzdAVfdG2xbDqIxYiNIwDMOokpgHZxiGYVRJzIMzDMMwqiQmcIZhGEaVxATOMAzDqJKYwBmGYRhVEhM4wzAMo0piAmcYhmFUSUzgDMMwjCqJCZxhGIZRJTGBM2IGEVkkIpPKu2008YqWXhCB62pAAdS/isgD4bQtw30mish7ZbXTMKKJZTIxTgkRCawhVhuXFLjQ275ZVV+qeKtiBy+f5A2quricr6tAR1XdVF5tvdpxW3H14grKw07DiCZWTcA4JQILdZb2ZS4iNe1L04gV7O+xemAhSiMiiMi5IrJDRH4uIrtx5V0aisgCEdkjIvu99dSAc5aIyA3eerqI/FtEpnttt4rIxWVs21ZElopItogsFpGnRGReCXaHY+NvROQT73rviUhKwPFrRGSbiGSJyK9KeT6DRWS3iMQF7BvllcZBRAaJyH9E5ICI7BKRJ0UkoYRrvSgiDwds3+ud862ITA5pO1xEVorIIRH5RkSmBhxe6r0eEJEcETnD92wDzj9TRJaJyEHv9cxwn81JPudGIjLLew/7RWR+wLGRIrLKew+bRWSYtz8oHCwiU32fs4ikeaHa60VkO/CBt/8173M46P2NdA84P0lE/s/7PA96f2NJIvKOiPw45P18KSKjinuvRvQwgTMiSTOgEdAGuAn39zbL224NHAaeLOX8wcAGIAX4A/C8iEgZ2r4MfAY0BqYC15Ryz3BsvApXg60pkADcAyAi3YBnvOu38O6XSjGo6v+AH4DzQq77srdeCNzlvZ8zgPOBW0uxG8+GYZ49FwIdgdD+vx+Aa4EGuIreU0Tkcu/Y2d5rA1VNVtX/BJ4oIo2Ad4AnvPf2J+AdEWkc8h6KPJtiONFznosLeXf3rvVnz4ZBwBzgXu89nA1klvQ8iuEcoCtwkbe9CPecmgIrgMCQ+nSgP3Am7u/4Z8AxYDZwta+RiPQGWuKejRFLqKottpTLgvuiucBbPxc4CiSW0r4PsD9gewkuxAmQDmwKOFYbUKDZybTFfXkWALUDjs8D5oX5noqz8f6A7VtxBUoBHgQyAo7V8Z7BBSVc+2HgBW+9Lk582pTQ9k7grYBtBTp46y8CD3vrLwCPBrTrFNi2mOs+BvzZW0/z2tYMOJ4O/Ntbvwb4LOT8/wDpJ3o2J/OcgeY4IWlYTLtnffaW9vfnbU/1fc4B761dKTY08NrUxwnwYaB3Me0Sgf24fk1wQvh0Rf+/2XLixTw4I5LsUdU834aI1BaRZ72QzyFcSKxBYJguhN2+FVXN9VaTT7JtC2BfwD6Ab0oyOEwbdwes5wbY1CLw2qr6A5BV0r1w3tpoEakFjAZWqOo2z45OXthut2fHb3He3IkIsgHYFvL+BovIh15o8CBwS5jX9V17W8i+bTjvxUdJzyaIEzznVrjPbH8xp7YCNodpb3EcfzYiEicij3phzkP4PcEUb0ks7l7e3/QrwNUiUgNXaX3uKdhkRAgTOCOShA7R/SnQGRisqvXwh8RKCjuWB7uARiJSO2Bfq1Lan4qNuwKv7d2zcUmNVXUtTiAuJjg8CS7UuR7nJdQDflkWG3AebCAvA28DrVS1PvDXgOueaEj1t7iQYiCtgZ1h2BVKac/5G9xn1qCY874B2pdwzR9w3ruPZsW0CXyPVwEjcWHc+jgvz2fDXiCvlHvNBibiQse5GhLONWIDEzijIqmLC/sc8PpzHor0DT2PaDkwVUQSROQM4LII2fg6cKmI/MgbEDKNE/+PvQzcgfuCfy3EjkNAjoh0AaaEacOrQLqIdPMENtT+ujjvKM/rz7oq4NgeXGiwXQnXXgh0EpGrRKSmiFwJdAMWhGlbqB3FPmdV3YXrG3vaG4wSLyI+AXweuE5EzheRGiLS0ns+AKuA8V77AcDYMGw4gvOya+O8ZJ8Nx3Dh3j+JSAvP2zvD87bxBO0Y8H+Y9xazmMAZFcljQBLu1/F/gX9W0H0n4gZqZOH6vV7BfbEVR5ltVNU1wG040dqF66fZcYLT/oYb+PCBqu4N2H8PTnyygec8m8OxYZH3Hj4ANnmvgdwKTBORbFyf4asB5+YCjwCfiBu9eXrItbOAS3HeVxZu0MWlIXaHy4me8zVAPs6L/R7XB4mqfoYbxPJn4CDwEX6v8gGcx7Uf+DXBHnFxzMF50DuBtZ4dgdwDfAUsA/YBvyf4O3MO0BPXp2vEIDbR26h2iMgrwHpVjbgHaVRdRORa4CZV/VG0bTGKxzw4o8ojIgNFpL0X0hqG63eZf6LzDKMkvPDvrcCMaNtilIwJnFEdaIYbwp6Dm8M1RVVXRtUio9IiIhfh+iu/48RhUCOKWIjSMAzDqJKYB2cYhmFUSapMsuWUlBRNS0uLthmGYRhGBfL555/vVdUmxR2rMgKXlpbG8uXLo22GYRiGUYGISGh2neNYiNIwDMOoklQZD84wDMOIbQoKYPt22LQJNm+Gbt3gnHMidz8TOMMwDKPcyM2FLVucgG3e7BezzZth2zYncj5uu80Erszk5+ezY8cO8vLyTtzYiAqJiYmkpqYSHx8fbVMMwwiTffuChStQzHbtCm5bvz506AD9+8MVV0D79m67fXto0SKydlZpgduxYwd169YlLS2NkutkGtFCVcnKymLHjh20bds22uYYhuFx7Bh8+21RD8y3HDgQ3L55cydYF13kXgOXRo0gWl+/ERU4Ly3S40AcMFNVHw053gaXsbsJLpnp1aq6wztWiEt0CrBdVUec7P3z8vJM3GIYEaFx48bs2bMn2qYYRrXj6FHIzCzqgW3eDFu3QmDgKy4O0tKcYA0aFOyFtWsHtWuXdJfoEjGB8woXPgVciMuovkxE3vZqYPmYDsxR1dkich7wO1wWcYDDqtqnHOw41UsYEcQ+H8OIHNnZRb0v37J9u/PUfNSu7QSrc2e45BK/B9ahA7RuDTUrYbwvkiYPAjap6hYAEcnAJbkNFLhuwN3e+odYAlzDMIywUYU9e4of0LF5M3z/fXD7xo2daJ1xBlxzTXAosVmz6IUSI0UkBa4lAeXhcV7c4JA2XwCjcWHMUUBdEWns1Z1KFJHlQAHwqKoWET8RuQm4CaB169DCxdEnKyuL888/H4Ddu3cTFxdHkyZuwv1nn31GQkJCiecuX76cOXPm8MQTT5R6jzPPPJNPP/20/Iw2DCOmKCyEb74pfkDH5s2Qk+NvKwKpqU6wLrss2Atr394N+KhORNvpvAd4UkTSgaW4woOF3rE2qrpTRNoBH4jIV6q6OfBkVZ2BV65iwIABMZc1unHjxqxatQqAqVOnkpyczD333HP8eEFBATVL8PsHDBjAgAEDTngPEzfDqPzk5bl+r+K8sK1bIT/f3zY+Htq2dYJ19tnBXljbtpCYGL33EWtEUuB2Aq0CtlO9fcdR1W9xHhwikgyMUdUD3rGd3usWEVkC9AWCBK4ykp6eTmJiIitXrmTIkCGMHz+eO+64g7y8PJKSkpg1axadO3dmyZIlTJ8+nQULFjB16lS2b9/Oli1b2L59O3feeSc/+clPAEhOTiYnJ4clS5YwdepUUlJSWL16Nf3792fevHmICAsXLuTuu++mTp06DBkyhC1btrBgwYIguzIzM7nmmmv44YcfAHjyySc588wzAfj973/PvHnzqFGjBhdffDGPPvoomzZt4pZbbmHPnj3ExcXx2muv0b59+4p9mIZRiThwoGQvbOdOF270UbeuE6yePeHyy/0eWPv2zkOLi4ve+6hMRFLglgEdRaQtTtjGA1cFNhCRFGCfqh4DfoEbUYmINARyVfWI12YI8IdTMebOO8FzpsqNPn3gscdO/rwdO3bw6aefEhcXx6FDh/j444+pWbMmixcv5pe//CVvvPFGkXPWr1/Phx9+SHZ2Np07d2bKlClF5o6tXLmSNWvW0KJFC4YMGcInn3zCgAEDuPnmm1m6dClt27ZlwoQJxdrUtGlT/vWvf5GYmMjGjRuZMGECy5cvZ9GiRfz973/nf//7H7Vr12bfvn0ATJw4kfvuu49Ro0aRl5fHscDeasOohqjC7t0lD63Pygpu37SpE66hQ4O9sA4dICWl6vWHRYOICZyqFojI7cC7uGkCL6jqGhGZBixX1beBc4HfiYjiQpS3ead3BZ4VkWO4fJmPhoy+rNSMGzeOOO8n2MGDB5k0aRIbN25ERMgPjEUEMHz4cGrVqkWtWrVo2rQp3333HampqUFtBg0adHxfnz59yMzMJDk5mXbt2h2fZzZhwgRmzChahDg/P5/bb7+dVatWERcXx9dffw3A4sWLue6666jtjQNu1KgR2dnZ7Ny5k1GjRgFusrZhVAcKClw2juIGdWzZ4rJ4+KhRw40+bN8exowJ9sLatXNemhFZItoHp6oLgYUh+x4MWH8deL2Y8z4FepanLWXxtCJFnTp1jq8/8MADDB06lLfeeovMzEzOPffcYs+pVavW8fW4uDgKAvPdnESbkvjzn//MaaedxhdffMGxY8dMtIxqiyp89x2sXu2Wr7/2i9m2bW7Qh4/ERCdW7dvDBRcED+ho0wZKGUdmVADRHmRS7Tl48CAtW7YE4MUXXyz363fu3JktW7aQmZlJWloar7zySol2pKamUqNGDWbPnk2h91984YUXMm3aNCZOnHg8RNmoUSNSU1OZP38+l19+OUeOHKGwsPC4l2cYlYVDh/xC9tVX/tfAcGKDBk6wBg6E8eODw4ktWjhPzYhNTOCizM9+9jMmTZrEww8/zPDhw8v9+klJSTz99NMMGzaMOnXqMHDgwGLb3XrrrYwZM4Y5c+YcbwswbNgwVq1axYABA0hISOCSSy7ht7/9LXPnzuXmm2/mwQcfJD4+ntdee4127dqVu/2GUR4cOQLr1xcVsu3b/W3q1IEePWDUKPfas6d7bdo0enYbp4aoxtzo+jIxYMAADS14um7dOrp27Roli2KHnJwckpOTUVVuu+02OnbsyF133RVts45jn5NRXhQWumH1oUL29df+0GJ8PHTpEixiPXq4kKJ5Y5UPEflcVYudU2UeXDXgueeeY/bs2Rw9epS+ffty8803R9skwzglfCMWA0Vs9WpYswYOH/a3a9fOidfo0X5B69jR+saqCyZw1YC77rorpjw2wzgZDh4svp/Mm7ECwGmnOQG7+Wa/kHXrBsnJ0bPbiD4mcIZhxAR5ecX3k30TkPCvbl0nYGPGBIcXvQx4hhGECZxhGBVKYaGbMxYaXty4MbifrGtXOOssv5D17OnmldkEaCNcTOAMw4gIqq66c6iQrV3r7ycT8feTjR0b3E9mRd6NU+WEAicilwHveOm0DMMwinDgQPH9ZPv3+9s0a+YE7JZbgvvJAvIeGEa5Es6g2CuBjSLyBxHpEmmDqhJDhw7l3XffDdr32GOPMWXKlBLPOffcc/FNd7jkkks4EFobHleZYPr06aXee/78+axd689u9uCDD7J48eKTMd8wipCXBytXwpw58LOfwcUXQ6tW0LChCydOmQLz5rmUVuPGwV/+Ah9+6GqW7doF//oX/OlPMHmymzht4mZEkhN6cKp6tYjUAyYAL3p5I2cBf1PV7EgbWJmZMGECGRkZXHTRRcf3ZWRk8Ic/hJc3euHChSduVALz58/n0ksvpVu3bgBMmzatzNcyqh+FhS49VXH9ZL682gkJrp/snHOC+8latbJ+MiM2CGtao6oewuWMzACa44qTrhCRH0fQtkrP2LFjeeeddzh69CjgStJ8++23nHXWWUyZMoUBAwbQvXt3HnrooWLPT0tLY+/evQA88sgjdOrUiR/96Eds2LDheJvnnnuOgQMH0rt3b8aMGUNubi6ffvopb7/9Nvfeey99+vRh8+bNpKen8/rrLu3n+++/T9++fenZsyeTJ0/myJEjx+/30EMP0a9fP3r27Mn69euL2JSZmclZZ51Fv3796NevX1A9ut///vf07NmT3r17c9999wGwadMmLrjgAnr37k2/fv3YvLnSVzyqUqi6Ui3//CdMnw6TJkH//m54fefOrl/s17+GL75wYvarX8Err7h+tJwcV6Fj3jz4+c9h+HAbBGLEFuH0wY0ArgM6AHOAQar6vYjUBtYCf4msieVEFOrlNGrUiEGDBrFo0SJGjhxJRkYGV1xxBSLCI488QqNGjSgsLOT888/nyy+/pFevXsVe5/PPPycjI4NVq1ZRUFBAv3796N+/PwCjR4/mxhtvBOD+++/n+eef58c//jEjRozg0ksvZezYsUHXysvLIz09nffff59OnTpx7bXX8swzz3DnnXcCkJKSwooVK3j66aeZPn06M2fODDrfyupUXvbvL9pHtnq16z/z0by588JuvdXvlXXtaqFEo3ISzijKMcCfVXVp4E5VzRWR6yNjVtXBF6b0Cdzzzz8PwKuvvsqMGTMoKChg165drF27tkSB+/jjjxk1atTxZMYjRow4fmz16tXcf//9HDhwgJycnKBwaHFs2LCBtm3b0qlTJwAmTZrEU089dVzgRo8eDUD//v158803i5xvZXVin8OHYd26okK2M6DccL16TsCuvDJ4PlnjxtGz2zDKm3AEbiqwy7chIknAaaqaqarvl3aiiAwDHsfVg5upqo+GHG+DK3LaBNgHXK2qO7xjk4D7vaYPq+rssN5RSUSpXs7IkSO56667WLFiBbm5ufTv35+tW7cyffp0li1bRsOGDUlPTycvL69M109PT2f+/Pn07t2bF198kSVLlpySvb6SOyWV27GyOrFDQYHrE1uzJljINm3y95PVquU8sKFDg/vJUlMtlGhUfcLpg3sNCIwrFXr7SkVE4oCngIuBbsAEEekW0mw6MEdVewHTgN955zYCHgIGA4OAh7wq35WO5ORkhg4dyuTJk49X0z506BB16tShfv36fPfddyxatKjUa5x99tnMnz+fw4cPk52dzT/+8Y/jx7Kzs2nevDn5+fm89NJLx/fXrVuX7OyiY4A6d+5MZmYmmzZtAmDu3Lmcc845Yb+fgwcP0rx5c2rUqMHcuXODyurMmjWLXK/i4759+6hbt+7xsjoAR44cOX7cCJ/CQidaf/87PPIIXHUV9OrlwobdurnRitOmOYHr0QPuvx9ee815cTk5btTj3Llu1OMll9ggEKP6EI4HV1NVj/o2VPWoiISTqnQQsElVtwCISAYwEtdv56MbcLe3/iEw31u/CPiXqu7zzv0XMAz4Wxj3jTkmTJjAqFGjyMjIAKB379707duXLl260KpVK4YMGVLq+f369ePKK6+kd+/eNG3aNKjkzW9+8xsGDx5MkyZNGDx48HFRGz9+PDfeeCNPPPHE8cEl4MKEs2bNYty4cRQUFDBw4EBuueWWsN+LldWJHKqufIsvabDvdd264ATCbdpA9+5uiH737m7p2hWsHJ9hBHPCcjmeuPxFVd/2tkcCP1HV809w3lhgmKre4G1fAwxW1dsD2rwM/E9VHxeR0cAbQApuUEuiqj7stXsAOKyqJU7+snI5lZfq9jn5MnyECtmaNc7j8tGihfPIfCLWo4fz2OrWjZ7thhFrnGq5nFuAl0TkSUCAb4Bry8m2e4AnRSQdWArsxIVAw0JEbgJuAmjdunU5mWQY5ceePUWFLHTkYpMmTrzS04MFrWGlDMobRuwQzkTvzcDpIpLsbeec4BQfO4FWAdup3r7Aa38LjAbwrj9GVQ+IyE7g3JBzlxRj2wxgBjgPLky7DKPc2b+/qIitWeMEzkeDBk7ArrwyWMisYrRhRIawki2LyHCgO5AoXu+0qp4oNcYyoKOItMUJ23jgqpDrpgD7vDyXv8CNqAR4F/htwMCS/+cdP2lUFbEe9ZilslWUz852k5xDhezbb/1tkpOdMDWPnQAAEzhJREFUcI0Y4Q8tdu/u5pjZn6JhVBzhTPT+K1AbGArMBMYCn53oPFUtEJHbcWIVB7ygqmtEZBqw3OvTOxf4nZf+aylwm3fuPhH5DU4kAab5BpycDImJiWRlZdG4cWMTuRhEVcnKyorJqQa5uW5wR6hXtn27v01SkusTu+ACv4j16GGjFA0jVghnkMmXqtor4DUZWKSqZ1WMieFR3CCT/Px8duzYUeY5ZkbkSUxMJDU1lfgo1UY5cgQ2bCgqZFu2uMEg4HIudukS7I316AFpaRAXFxWzDcPwONVBJj51yBWRFkAWLh9lzBMfH0/btm2jbYYRA+Tnu7lkoUIWWGQzLg46dYJ+/eCaa/xi1qED1LTKiYZR6Qjn3/YfItIA+COwAlDguYhaZRhlpLAQtm4t2ke2fr0TOXDhw/btnYCNGeMXsk6dXOYPwzCqBqUKnIjUAN5X1QPAGyKyADc/7WCFWGcYJVDSpOi1a13NMh9paU68LrnEP2qxSxebFG0Y1YFSBU5Vj4nIU0Bfb/sIcKQiDDMMcEL27bdFQ4u+ci0+WrZ04nXrrf4+sq5dbVK0YVRnwglRvi8iY4A3tbKN6TYqFd9/X/xcssBJ0U2bOvG67rrguWQNGkTPbsMwYpNwBO5mXL7IAhHJw2UzUVWtF1HLjCrLvn3+1FQlTYpu2NAJ2PjxwULWpEn07DYMo3IRTiYTC/IYp8SuXfDss/Dpp07Mdu3yH6tb1wnXyJHBw/CbNbO5ZIZhnBrhTPQ+u7j9oQVQDSOUtWvh//4P5s1ztcv69oX/9/+ChcwmRRuGESnCCVHeG7CeiCuD8zlwXkQsMio1qrB0Kfzxj/DOOy7bx403wl13uaH5hmEYFUU4IcrLArdFpBUQnfLYRsxSUABvvAHTp8Py5a6vbNo0N6qxceNoW2cYRnWkLPkZdgDVp3iXUSo5OfDCC/DnP0Nmppss/eyzLhNIUlK0rTMMozoTTh/cX3DZSwBqAH1wGU2Maszu3fCXv8Azz7hSMUOGwGOPwWWXQY0a0bbOMAwjPA8uMINxAfA3Vf0kQvYYMc66dW7gyNy5LvXV6NHw05/CGWdE2zLDMIxgwhG414E8VS0EEJE4EamtqrmRNc2IFVTh44/dwJEFC1zo8YYb3MCRDh2ibV01Q9V1eB496pb8fPdas6b7YJKSID7ehqYaBmFmMgEuAHyJkZKA94AzI2WUERsUFMBbbzlhW7YMUlLg1792A0dSUqJtXTlw7FhRofC9lrR+ouORvpYvY3Rp1KjhF7vyWhITSz9ucWkjBglH4BJV9XjWP1XNEZGwUtWKyDDgcVzB05mq+mjI8dbAbKCB1+Y+VV0oImnAOmCD1/S/qnpLOPc0Tp0ffoBZs+BPf3KZ+Tt2hL/+Fa69Rkmqdcyl7M8tcK+FhU4Ji3sN3Vec5xFNITl2LHIPsUYNV0guIcF5VKHrofuSksJvW9zxggI4fDi85dCh4vcfPVr295uQUP6iWtpiXqoRBuEI3A8i0k9VVwCISH/g8IlOEpE44CngQtzIy2Ui8raqrg1odj/wqqo+IyLdgIVAmndss6r2Cf+tlAN5eSV/OYfzBV7JjxUeLSD3UCFHcgu4SgtJjysksVYBcdsLkdsK4JbCCv04jhP6RV/aF35iokuPUpognIx4nKzQ+F4rYyXUwkL3P5CXF75YhrNkZ7tEo8UdK+uPjJK81BN5mmX1Xn2fq4lqpSIcgbsTeE1Evv3/7d1/kFXlfcfx94ddVAZStAEDFbZoixpNa4M7aBLG2Ni01iibqFOhrUJjQ7VIYjRWdBxNnU5mYlLHUDARrYJGBZMIQaIEG5n6C4ElGiOJ4kpRMFYWTLQqYtj99o/nrHtZ98clu/fec+9+XjM7e+4559797AN7vvec89znIY1DOQY4p4jnTQZaImILgKQlQBNQWOAC6BjTciTwqyJzl8aECfDqqxWNsI8hQ9KBsr6+9+/FbBs2rMdtr79Vx6Zn69m8rY532+tpOLyO4ybV8fvj+/HzuvveW6HoqXjU1/ugUi51dTB8ePoqh4h0Rt216A1EgW1t7X79nn5OhlL4f/OAA9IEgoWP+1pfjufU1flvJlPMB703SDoaOCpb9VxEFHEjgMOAbQWPtwMndNnnq8BqSXOA4aR7fR0Ol/Qk8AZwVUQ8UsTP7J+rrkp/BP0tJgOxrcT/SSPg0UfT/bX77ktvUmfOgksuSZckzUpO6jwojxxZnp/Z3t57Ae1u2549nZe13333/Y+7W79nTzpz7Wv/vXsH/nfsaNf9LaTlLMpDh5blvm0xn4ObDdwZEc9kjw+RND0ibhyAnz8dWBQR/y7pY8Adkj4CvAI0RMSu7JLocknHRsQbXbLNAmYBNDQ09D/NRRf1/zVyrq2ts+PI+vVplJFrroHZsz1Svw0CQ4ak2W7zMuNte/u+94WLLaADtf7NN4vbvxQzpdXXp67Y11038K/d8SOK2OcLEbGg40FE/FrSF4C+CtzLwPiCx+OydYXOB07NXnetpIOAURGxg2xi1YjYKOkF4Ej2/UweEbEQWAjQ2Njouep68dZbsGhR6jiyZUvq3n/jjTBjRn7+1s0GnSFD0hnNgQdWOknPItI741IU3I+XtjN+MQWuTpI6JjvNOo8cUMTzNgATJR1OKmzTgL/tss9LwCnAIkkfJg3m3CppNPBaRLRJOgKYCGwp6jeyfbz6KixYkL5eew1OPDGdvTU1VWc/CDMrMymdbdXXV9274WIK3CpgqaSbssf/BDzQ15MiYq+ki4Afkz4CcGtEbJJ0LdAcESuAS4GbJX2Z1OFkZkRENkXPtZJ+C7QDF0TEa/v92w1izz2XztYWL05vlJqa4CtfSUNqmZkNBoo+rq1KGkK6z3VKtuppYExEzC5xtv3S2NgYzc3Nfe9YwyLgscfSGdqKFemqx8yZ6TL3UUf1+XQzs6ojaWNENHa3rZhelO2S1gF/BPwNMAr4wcBGtP5oa4Ply9NUNU88kTqOXH116jhy6KGVTmdmVhk9FjhJR5J6OU4HdgJLASLiz8sTzfry9tudHUdeeCFNKLpgQTprq7JL5WZmA663M7hngUeA0yOiBSC7V2YVtmNHZ8eRXbvghBPg61+Hz37WHUfMzDr0VuDOJPV8XCNpFbCENJKJVcjmzZ0dR955B6ZOhcsuSx1HPHCBmdm+eixwEbGc9AHr4aQhti4GDpX0bWBZRKwuU8ZB77HH0v21H/4wDQQwY0YaccQdR8zMetbnWCkR8VZE3BURZ5A+rP0kcHnJkw1ybW1w773pc5BTpsDDD6eRxF58EW66ycXNzKwvxXwO7j0R8WvSyCELSxPH3n47XYK8/npoaYEjjoD581PHkXKNgWtmVgv2q8BZ6bS2dnYc2bkTJk+G730PPvc5dxwxM/tduMBV2PPPp7O1RYtSx5EzzkgdR6ZMcccRM7P+cIGrkMcfTx1Hli9PM0ecdx5ceikcfXSlk5mZ1QYXuDJqa0tDaH3zm6nAHXIIXHllmqVnzJhKpzMzqy0ucGWwe3dnx5Hnn08Th8+bB5//vDuOmJmVigtcCe3cmTqNzJ+flhsbYelSOPPMNPOEmZmVjg+zJdDS0tlxZPduOP30NFXNSSe544iZWbm4wA2gtWvT/bVly1LHkXPPTSOOHHNMpZOZmQ0+fY5k0h+STpX0nKQWSXO72d4gaY2kJyU9Lem0gm1XZM97TtJflTJnf7S3p56QU6akUUfWrIErroCtW+GWW1zczMwqpWRncJLqgAXAp4HtwAZJKyLiFwW7XQXcExHflnQMcD8wIVueBhwL/AHwX5KOjIi2UuXdX7t3w+23p0uRmzenjiPf+lbqODJiRKXTmZlZKS9RTgZaImILgKQlpEGbCwtcAL+XLY8EfpUtNwFLImIP8D+SWrLXW1vCvEXZuRNuvDF1HGltheOPhyVL4Kyz3HHEzCxPSnlIPgzYVvB4O3BCl32+CqyWNAcYDvxFwXOf6PLcw7r+AEmzgFkADQ0NAxK6Jy+8kM7Wbrstnb2ddloaceSTn3THETOzPCrpPbgiTAcWRcQ44DTgDklFZ4qIhRHRGBGNo0ePLknAdevg7LNh4sR0T236dHjmGfjRj+Dkk13czMzyqpRncC8D4wsej8vWFTofOBUgItZKOggYVeRzS6a9HVauhG98Ax59FA4+GObOhTlzYOzYcqUwM7P+KOUZ3AZgoqTDJR1A6jSyoss+LwGnAEj6MHAQ0JrtN03SgZIOByYC60uYFUiDHd98c+r52NQE27bBDTfASy/B177m4mZmVk1KdgYXEXslXQT8GKgDbo2ITZKuBZojYgVwKXCzpC+TOpzMjIgANkm6h9QhZS8wu9Q9KN95J12G3L4dJk2Cu+9OlybdccTMrDop1ZPq19jYGM3Nzf16jRtugOOO8701M7NqIWljRDR2t83nJwUuvrjSCczMbKBUuhelmZlZSbjAmZlZTaqZe3CSWoEXB+ClRgE7B+B1yqGaskJ15a2mrFBdeaspK1RX3mrKCgOT9w8jotsPQtdMgRsokpp7umGZN9WUFaorbzVlherKW01ZobryVlNWKH1eX6I0M7Oa5AJnZmY1yQXu/RZWOsB+qKasUF15qykrVFfeasoK1ZW3mrJCifP6HpyZmdUkn8GZmVlNGpQFTtKtknZIeqaH7ZI0T1KLpKclTSp3xoIsfWU9WdLrkp7Kvq4ud8YuecZLWiPpF5I2SfpSN/vkon2LzJqb9pV0kKT1kn6W5f3XbvY5UNLSrG3XSZpQ/qRFZ50pqbWgbf+xElkL8tRJelLSym625aJdu2TqLW/e2narpJ9nWd43pmLJjgkRMei+gJOAScAzPWw/DXgAEHAisC7HWU8GVla6TQvyjAUmZcsfADYDx+SxfYvMmpv2zdprRLY8FFgHnNhln38GvpMtTwOW5jjrTGB+pdu1IM8lwF3d/XvnpV33I2/e2nYrMKqX7SU5JgzKM7iIeBh4rZddmoDbI3kCOFhSRSbLKSJrrkTEKxHx02z5/4Bf8v7Z2HPRvkVmzY2svd7MHg7NvrreRG8CFmfL3wdOkco/dHiRWXND0jjgM8AtPeySi3btUETealOSY8KgLHBFOAzYVvB4Ozk+8AEfyy4FPSDp2EqH6ZBdxvko6d17ody1by9ZIUftm12WegrYATwYET22bUTsBV4HPljelEkRWQHOyi5JfV/S+G62l8sNwL8A7T1sz027ZvrKC/lpW0hvblZL2ihpVjfbS3JMcIGrfj8lDVVzHPAfwPIK5wFA0gjgB8DFEfFGpfP0po+suWrfiGiLiD8jzXI/WdJHKpmnN0VkvQ+YEBF/CjxI5xlSWUk6HdgRERsr8fP3V5F5c9G2BaZExCTgr4HZkk4qxw91gevey0DhO55x2brciYg3Oi4FRcT9wFBJoyqZSdJQUsG4MyLu7WaX3LRvX1nz2L5Zlt8Aa4BTu2x6r20l1QMjgV3lTbevnrJGxK6I2JM9vAU4vtzZMp8ApkraCiwBPiXpu132yVO79pk3R23bkefl7PsOYBkwucsuJTkmuMB1bwVwXtaz50Tg9Yh4pdKhuiNpTMe9AEmTSf+mFTugZVn+E/hlRFzfw265aN9isuapfSWNlnRwtjwM+DTwbJfdVgAzsuWzgYciu4tfTsVk7XKPZSrpHmjZRcQVETEuIiaQOpA8FBF/32W3XLQrFJc3L22bZRku6QMdy8BfAl17hZfkmDAoJzyVdDepd9woSduBa0g3wYmI7wD3k3r1tABvA/9QmaRFZT0buFDSXmA3MK1Sf3iZTwDnAj/P7r8AXAk0QO7at5iseWrfscBiSXWkQntPRKyUdC3QHBErSAX7DkktpM5J03Kc9YuSpgJ7s6wzK5S1Wzlt1x7luG0/BCzL3ifWA3dFxCpJF0BpjwkeycTMzGqSL1GamVlNcoEzM7Oa5AJnZmY1yQXOzMxqkgucmZnVJBc4swqT1FYw6vtTkuYO4GtPUA8zUZjVukH5OTiznNmdDWllZgPIZ3BmOZXNoXVdNo/Wekl/nK2fIOmhbCDdn0hqyNZ/SNKybGDon0n6ePZSdZJuVpqXbXU2sgiSvqg0F97TkpZU6Nc0KxkXOLPKG9blEuU5Bdtej4g/AeaTRpCHNOjz4mwg3TuBedn6ecB/ZwNDTwI2ZesnAgsi4ljgN8BZ2fq5wEez17mgVL+cWaV4JBOzCpP0ZkSM6Gb9VuBTEbElGxT6fyPig5J2AmMj4rfZ+lciYpSkVmBcwSC7HdMAPRgRE7PHlwNDI+LfJK0C3iTNkLC8YP42s5rgMzizfIselvfHnoLlNjrvvX8GWEA629uQjZJvVjNc4Mzy7ZyC72uz5cfpHOz374BHsuWfABfCe5ONjuzpRSUNAcZHxBrgctL0L+87izSrZn7HZlZ5wwpmMwBYFREdHxU4RNLTpLOw6dm6OcBtki4DWukcef1LwEJJ55PO1C4EeppypA74blYEBczL5m0zqxm+B2eWU9k9uMaI2FnpLGbVyJcozcysJvkMzszMapLP4MzMrCa5wJmZWU1ygTMzs5rkAmdmZjXJBc7MzGqSC5yZmdWk/weipuEpHCdbvwAAAABJRU5ErkJggg==","text/plain":["<Figure size 432x288 with 2 Axes>"]},"metadata":{"needs_background":"light"},"output_type":"display_data"}],"source":["# Subplot들의 간격 조정하기 참고: https://tempdev.tistory.com/35\n","\n","# plt.subplots\n","fig, axs = plt.subplots(2 ,1, constrained_layout=True)\n","\n","# Training and Validation Loss 그래프\n","axs[0].plot(epochs, loss, 'b', label='Training loss')\n","axs[0].plot(epochs, val_loss, 'r', label='Validation loss')\n","axs[0].set_title('Training and validation loss')\n","axs[0].set_xlabel('Epochs')\n","axs[0].set_ylabel('Loss')\n","axs[0].legend()\n","\n","# Training and Validation Accuracy 그래프\n","axs[1].plot(epochs, acc, 'b', label='Training acc')\n","axs[1].plot(epochs, val_acc, 'r', label='Validation acc')\n","axs[1].set_title('Training and validation accuracy')\n","axs[1].set_xlabel('Epochs')\n","axs[1].set_ylabel('Accuracy')\n","axs[1].legend()\n","\n","\n","plt.show()"]},{"cell_type":"markdown","metadata":{"id":"qQtnbT3jdyDK"},"source":["- Model 3: GloablMaxPooling1D()  레이어 1개만 사용"]},{"cell_type":"code","execution_count":31,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":17,"status":"ok","timestamp":1642994101291,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"EcWN4kwwd1H_","outputId":"3bdb1ede-5152-4a0c-d593-42fe46a69b40"},"outputs":[{"name":"stdout","output_type":"stream","text":["dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])\n"]}],"source":["history_dict = history_gmp1d.history\n","print(history_dict.keys()) # epoch에 따른 그래프를 그려볼 수 있는 항목들"]},{"cell_type":"code","execution_count":32,"metadata":{"executionInfo":{"elapsed":13,"status":"ok","timestamp":1642994101292,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"1dZESSg5d1Ff"},"outputs":[],"source":["acc = history_dict['accuracy']\n","val_acc = history_dict['val_accuracy']\n","loss = history_dict['loss']\n","val_loss = history_dict['val_loss']\n","\n","epochs = range(1, len(acc) + 1)"]},{"cell_type":"code","execution_count":33,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":313},"executionInfo":{"elapsed":590,"status":"ok","timestamp":1642994101869,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"Rsl8rKSkd-_y","outputId":"b908ad13-bb29-4654-d37f-fb94ca80a303"},"outputs":[{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAbgAAAEoCAYAAAAqrOTwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXhU5fXA8e8hLAHCHpAlYIICiiKrYEWoCFYUBMEFkCopVsF9qVpqXahiq5aq9SdSccUFgytVhNa6IO6yqoCgEIKCoBgkJEAggfP7472TTCYzyWSZzCQ5n+eZJ3fuNmfuJHPyvvddRFUxxhhjapo60Q7AGGOMiQRLcMYYY2okS3DGGGNqJEtwxhhjaiRLcMYYY2okS3DGGGNqJEtwpkYQkcUiMqmy940mEckQkWEROK+KyNHe8r9E5LZw9i3H60wUkbfKG2cJ5z1VRLZW9nlNzVM32gGY2ktEcvyeNgIOAIe851NU9flwz6WqZ0Zi35pOVadWxnlEJBnYDNRT1Xzv3M8DYX+GxlQ2S3AmalQ1wbcsIhnA71X17cD9RKSu70vTGGPCZVWUJub4qqBE5I8isgN4SkRaiMhCEdkpIr94y0l+xywRkd97y6ki8qGIzPT23SwiZ5Zz3xQRWSoi2SLytojMEpHnQsQdTox3ichH3vneEpFEv+0XicgWEckUkT+XcH0GiMgOEYnzWzdGRL70lvuLyCcisltEtovIwyJSP8S5nhaRGX7Pb/KO+UFEJgfsO0JEVonIHhH5XkSm+21e6v3cLSI5IvIr37X1O/5kEVkmIlnez5PDvTYlEZFjveN3i8haERnlt+0sEVnnnXObiNzorU/0Pp/dIrJLRD4QEfs+rGHsAzWxqi3QEjgSuAz3u/qU97wTsB94uITjBwAbgETgPuAJEZFy7DsP+BxoBUwHLirhNcOJ8ULgd0AboD7g+8LtDsz2zt/ee70kglDVz4C9wGkB553nLR8Crvfez6+AocAVJcSNF8NwL57TgS5A4P2/vcDFQHNgBHC5iJzjbRvs/Wyuqgmq+knAuVsCbwIPee/tfuBNEWkV8B6KXZtSYq4HvAG85R13NfC8iHTzdnkCV93dBDgeeNdb/wdgK9AaOAK4BbBxC2sYS3AmVh0G7lDVA6q6X1UzVfUVVd2nqtnA3cCvSzh+i6o+pqqHgLlAO9wXWdj7ikgn4ETgdlU9qKofAq+HesEwY3xKVb9R1f3Ai0Avb/15wEJVXaqqB4DbvGsQygvABAARaQKc5a1DVVeo6qeqmq+qGcCjQeII5gIvvjWquheX0P3f3xJV/UpVD6vql97rhXNecAnxW1V91ovrBWA9cLbfPqGuTUlOAhKAe7zP6F1gId61AfKA7iLSVFV/UdWVfuvbAUeqap6qfqA2MG+NYwnOxKqdqprreyIijUTkUa8Kbw+uSqy5fzVdgB2+BVXd5y0mlHHf9sAuv3UA34cKOMwYd/gt7/OLqb3/ub0EkxnqtXCltbEi0gAYC6xU1S1eHF296rcdXhx/xZXmSlMkBmBLwPsbICLveVWwWcDUMM/rO/eWgHVbgA5+z0Ndm1JjVlX/fwb8z3suLvlvEZH3ReRX3vq/AxuBt0QkXUSmhfc2THViCc7EqsD/pv8AdAMGqGpTCqvEQlU7VobtQEsRaeS3rmMJ+1ckxu3+5/Zes1WonVV1He6L/EyKVk+Cq+pcD3Tx4rilPDHgqln9zcOVYDuqajPgX37nLa308wOu6tZfJ2BbGHGVdt6OAffPCs6rqstUdTSu+nIBrmSIqmar6h9UtTMwCrhBRIZWMBYTYyzBmeqiCe6e1m7vfs4dkX5Br0S0HJguIvW9//7PLuGQisT4MjBSRE7xGoTcSel/n/OAa3GJ9KWAOPYAOSJyDHB5mDG8CKSKSHcvwQbG3wRXos0Vkf64xOqzE1el2jnEuRcBXUXkQhGpKyLjgO646sSK+AxX2rtZROqJyKm4zyjN+8wmikgzVc3DXZPDACIyUkSO9u61ZuHuW5ZUJWyqIUtwprp4EGgI/Ax8Cvynil53Iq6hRiYwA5iP668XTLljVNW1wJW4pLUd+AXXCKIkvntg76rqz37rb8Qln2zgMS/mcGJY7L2Hd3HVd+8G7HIFcKeIZAO345WGvGP34e45fuS1TDwp4NyZwEhcKTcTuBkYGRB3manqQVxCOxN33R8BLlbV9d4uFwEZXlXtVNznCa4RzdtADvAJ8IiqvleRWEzsEbuvakz4RGQ+sF5VI16CNMZUjJXgjCmBiJwoIkeJSB2vGf1o3L0cY0yMs5FMjClZW+BVXIOPrcDlqroquiEZY8JhVZTGGGNqJKuiNMYYUyNZgjPGGFMj1Zh7cImJiZqcnBztMIwxxlShFStW/KyqrYNtqzEJLjk5meXLl0c7DGOMMVVIRAKHgCtgVZR+DoTqvmuMMabasQTn2b8fjj0Wrr4aduwofX9jjDGxzRKcJzcXTj8dZs+Gzp1h2jTYtSvaURljjCkvS3CeFi3g0Udh/XoYOxbuuw9SUmDGDMjOjnZ0xhhjysoSXICjj4bnnoMvv4TTToPbbnMlugcecKU8Y4wx1YMluBCOPx5eew0++wx694YbbnDJb84cyMuLdnTGGGNKYwmuFP37w1tvwXvvwZFHwpQprjHK88/DoUPRjs4YY0woluDCdOqp8OGHsHAhJCTAb38LvXrBggVgw3kaY0zssQRXBiIwYgSsXAnz58PBgzBmDAwYAP/7nyU6Y4yJJZbgyqFOHbjgAli7Fp58En78EX7zG9co5eOPox2dMcYYsARXIXXrwu9+B998Aw89BF9/DQMHwsiRsHp1tKMzxpjazRJcJWjQwI2AsmkT/O1vrhTXuzeMGwcbNkQ7OmOMqZ0swVWixo3dCCjp6XDrrfDmm9C9O0yeDFtCDgdqjDEmEizBRUDz5nDXXS7RXXstzJsHXbrANdfYOJfGGFNVLMFFUJs2cP/9sHGju1f3yCNw1FHwpz/ZOJfGGBNpluCqQFJS4TiX55wD995r41waY0ykWYKrQkcf7UZA+eKLwnEujzrKxrk0xphIsAQXBT16FI5z2bOnG+eySxcb59IYYyqTJbgo6t/fjYDy7rvQsWPhOJfz5sHhw9GOzhhjqjdLcDFgyBD46KPCcS4nTnQlu3//24b/MsaY8opKghOR4SKyQUQ2isi0EvY7V0RURPpVZXzR4D/OZVqaG+fynHPgpJPg7bct0RljTFlVeYITkThgFnAm0B2YICLdg+zXBLgW+KxqI4yuOnXcCChr18ITT7h+c6efbuNcGmNMWUWjBNcf2Kiq6ap6EEgDRgfZ7y7gXqBWti+sW9eNgOIb53LdOhvn0hhjyiIaCa4D8L3f863eugIi0gfoqKpvlnQiEblMRJaLyPKdO3dWfqQxwDfOZXq6G+fyo4/cOJfjx9s4l8YYU5KYa2QiInWA+4E/lLavqs5R1X6q2q9169aRDy6KfONcbt7sxrlcuNCNc3nJJTbOpTHGBBONBLcN6Oj3PMlb59MEOB5YIiIZwEnA67WhoUk4/Me5vOYa13G8a1cb59IYYwJFI8EtA7qISIqI1AfGA6/7NqpqlqomqmqyqiYDnwKjVHV5FGKNWW3auBFQvv0WJk2ycS6NMSZQlSc4Vc0HrgL+C3wNvKiqa0XkThEZVdXxVHcdO7oRUPzHuezc2ca5NMYY0RrSwapfv366fLkV8r76yo1x+e9/Q+vWcMstMHUqxMdHOzJjjKl8IrJCVYPewoq5RiamYnr0gAUL4NNP3Wgo11/vxrl87DEb59IYU7tYgquhBgwoOs7lZZfZOJfGmCjavx+2bYM1a+CDD1w105dfRvQlrYqyFlCFN9903Qu++MKV8u66C0aNckOEGWNMWA4ehF9+cY9du8q2fOBA8fPdcAP84x8VCqmkKsq6FTqzqRZE3AgoZ50FL73k7tGdc46bzeDuu2HoUEt0xtQa+fmwe3dh4ilLstq3r+RzN20KLVq4R8uWrrOub9l/vW+5Y8eSz1dBVoKrhfLz4ZlnYPp0+P57N5vB3XfDr34V7ciMMWE5fBj27ClfaWrPnpLP3ahR6IRUUrJq3tyNMVjFSirBVSjBiUhjYL+qHhaRrsAxwGJVrfLmDJbgyu7AAdfFYMYM+OknV8qbMcM1TjHGRJgq7N1bNAmFm6x27y75Znr9+oXJp6zJqn79qrsGlSCSCW4FMAhoAXyE68R9UFUnlvuk5WQJrvz27nUDOt93n/u7GTcO7rzTjZBijCnF/v3lq+775RdXnRJKXFzJCaqkxNWwYa257xDJBLdSVfuIyNVAQ1W9T0RWq2qvcp+0nCzBVdzu3TBzJjz4IOTmuhFSbr8djjwy2pEZU4lUXZ+ZvXvdPSXfz2DLe/dCVlbJySq3hAlPRFzVXVkSlG85IaHWJKmKiGSCWwVcATwAXOKNSPKVqvYo90nLqcIJ7tAhOOYY90uVkABNmpT/Z/361foX86ef3MwFjzzink+Z4jqMt20b3bhMLXHoUOjEE+660rYfOlS2mJo0Kd99qWbN3CSPJmIimeB+jRv1/yNVvVdEOgPXqeo15T5pOVU4weXmugnYcnLcGFfBfobbgaxu3YolyMCfjRtH5Y/k++9dd4Inn3TT9lx7Ldx0k/u7NbWUqquSK29iCeeYgwfLHlfDhq5xRKNG7u8lcDncdcG2N20alcYTJjwRS3ABL1IHSFDVUproREbEqyh9f9glJcCy/iypaiNQ48aVlzATElzGCtO337oWly+84P7Wb7rJJbuEhLJfRhNBgVVvFS3lhDqmrOrVq3iSKWldw4ZWSqrFIlmCmwdMBQ7hGpg0Bf6pqn8v90nLqVreg/N9GVVWwixLKbNevdAJMERS3LKrCXNfTeDtz5pQr3kCF13RhPG/TyC+dRP3ZRMrXzKqrgoqP7/oI9i6mrLdV6oqa9WbSOQSj2+5Xr3IfM7GENkEt1pVe4nIRKAPMA1YoaonlPuk5VQtE1xli3Yps7T7lyJVkwDK+iUfKXFxrmrL9zPUoyLbfdt8VXRlTUwNGlTr+8XGRHIkk3oiUg84B3hYVfNEpGb0HK+ORAq/vNq0qZxzllDKXPtZDm+mZfPT5hyObJHNsJNyOKZDNrLXb78ff4RNmwrn7gn3izw+vvKSQGUlkbJsj4uzxGFMlFU0wT0KZABfAEtF5EggKvfgTITUq+eaOTdvXmzTcaOh+91unMs//xmuWezGuZwxA84+277fjTHRVaGbJqr6kKp2UNWz1NkCDKmk2Ew14BvnctUqSEtzNZqjR8NJJ8E770Q7OmNMbVahBCcizUTkfhFZ7j3+ATSupNhMNVKnjhsBZd06ePxx2L4dhg2D006DTz6JdnTGmNqoos3engSygQu8xx7gqYoGZaqvunXhkktc14J//hPWroWTT3ZVls88A5995kZMMcaYSKuUVpSlrasK1ooyNvnGufz7392oRj5HHOEGjunWrfBnt26QnOzaZxhjTDgi2Ypyv4icoqofei80ENgfRkDDgX8CccDjqnpPwPapwJW4/nU5wGWquq6CsZooaNwY/vQnuPFGSE+HDRtg/frCny+/7Ib182nQAI4+Onjya9Yseu/DGFP9VLQE1xN4BvB99fwCTFLVkPOQi0gc8A1wOrAV10F8gn8CE5GmvhFRRGQUcIWqDi8plmAluLy8PLZu3UpuWfpymSp36BAcOBDPmjVJrF1bryABbtpUtEtb27ZFE55v+cgjrdRnTG0VsRKcqn4B9BSRpt7zPSJyHRAywQH9gY2qmu4FlwaMBgoSXMBwX42BcmXhrVu30qRJE5KTkxFrsx6zVJXMzExattzKpEkpBesPHgxe6nvxxaLVnQ0aQJcuwUt9TZtG4Q0ZY2JCpYwgGpCQbgAeLGH3DsD3fs+3AgMCdxKRK71z1QdOK09cubm5ltyqARGhVatW7Ny5s8j6+vVdsjrmGNf1wEcVfv7ZJTz/5PfFF/Daa0VLfe3aFU16vp+dOlmpz5iaLhJDZFdKNlHVWcAsEbkQuBWYVOyFRC4DLgPo1KlT8GAsuVULZfmcRKB1a/c45ZSi2w4edFWbgaW++fOLlvri44uW+vyTX5MmlfSmjDFRFYkEV1p14jago9/zJG9dKGnA7KAvpDoHmAPuHlwZYjQ1VP36cOyx7uFPFXbuLF7qW7UKXnml6BjV7duHLvXFynjSxpjSlSvBiUg2wROZAA1LOXwZ0EVEUnCJbTxwYcD5u6jqt97TEcC3VEOZmZkMHToUgB07dhAXF0fr1q0B+Pzzz6lfv37IY5cvX84zzzzDQw89VOJrnHzyyXz88ccVjnXJkiXMnDmThQsXVvhcsUjEDc/Zpg0MGlR024EDhaU+/+T3wgtF++zFx0PXrsWTX9euVuozJhaVK8Gparn/nFU1X0SuAv6L6ybwpDcT+J3AclV9HbhKRIYBeXgtM8v7etHUqlUrVq9eDcD06dNJSEjgxhtvLNien59P3RATKfbr149+/YI2DCqiMpJbbdegAXTv7h7+fKU+/6rODRtg5cripb4OHYKX+jp2tFKfMdESlWlqVXURsChg3e1+y9dWeVBVJDU1lfj4eFatWsXAgQMZP3481157Lbm5uTRs2JCnnnqKbt26FSlRTZ8+ne+++4709HS+++47rrvuOq65xk2anpCQQE5ODkuWLGH69OkkJiayZs0a+vbty3PPPYeIsGjRIm644QYaN27MwIEDSU9PL7GktmvXLiZPnkx6ejqNGjVizpw5nHDCCbz//vtce637aESEpUuXkpOTw7hx49izZw/5+fnMnj2bQYFFpGrKv9Q3eHDRbb5SX2Dye/55yMoq3K9hw9ClPpsw1pjIqjXzsF93HXiFqUrTqxc8WFJ70RC2bt3Kxx9/TFxcHHv27OGDDz6gbt26vP3229xyyy288sorxY5Zv3497733HtnZ2XTr1o3LL7+cegETSa5atYq1a9fSvn17Bg4cyEcffUS/fv2YMmUKS5cuJSUlhQkTJpQa3x133EHv3r1ZsGAB7777LhdffDGrV69m5syZzJo1i4EDB5KTk0N8fDxz5szhjDPO4M9//jOHDh1iX3lmfK6GSir1/fRTYcLzJb8VK1yndv9SX1JS8W4Nxxzj1lupz5iKqzUJLpacf/75xHlt1LOyspg0aRLffvstIkJeXl7QY0aMGEGDBg1o0KABbdq04ccffyQpKanIPv379y9Y16tXLzIyMkhISKBz586kpLj+ZRMmTGDOnDklxvfhhx8WJNnTTjuNzMxM9uzZw8CBA7nhhhuYOHEiY8eOJSkpiRNPPJHJkyeTl5fHOeecQ69eVT5KW0wRccOQHXEE/PrXRbcdOAAbNxYv9T37LOzx62jTsGHxlp2+Ul9jG8rcmLDVmgRXnpJWpDT2+5a67bbbGDJkCK+99hoZGRmceuqpQY9p0KBBwXJcXBz5+fnl2qcipk2bxogRI1i0aBEDBw7kv//9L4MHD2bp0qW8+eabpKamcsMNN3DxxRdX6uvWFA0awHHHuYc/VTcvbGDXhmXLXKd2/8GGOnYMnvw6dLBSnzGBak2Ci1VZWVl06NABgKeffrrSz9+tWzfS09PJyMggOTmZ+fPnl3rMoEGDeP7557nttttYsmQJiYmJNG3alE2bNtGjRw969OjBsmXLWL9+PQ0bNiQpKYlLL72UAwcOsHLlSktwZSTihiFr27Z4qS83t2ipz5f8Akt9DRq4Ics6d4aUlKKPzp2hRYuqfU/GxAJLcFF28803M2nSJGbMmMGIESMq/fwNGzbkkUceYfjw4TRu3JgTTzyx1GOmT5/O5MmTOeGEE2jUqBFz584F4MEHH+S9996jTp06HHfccZx55pmkpaXx97//nXr16pGQkMAzzzxT6e+hNouPh+OPdw9/qrBjR2HCS0+HzZvd47PPinZqBzdQdWDS8y0nJ7tqUWNqmgoNthxLgg22/PXXX3NsYI/fWignJ4eEhARUlSuvvJIuXbpw/fXXRzusYuzzqjxZWYUJb/Pmoglw82ZXMvTXtm3o0l9Skg1rZmJXJKfLMdXAY489xty5czl48CC9e/dmypQp0Q7JRFizZq6Vb7A2P757foFJb/Nm+PBD18Hdv7Vn3bpuFJdgpb+UFDdkmo2IZ2KRleBMzLDPKzbk5cH33wcv+W3e7LpB+GvcOHjJz7ds/f1MJFkJzhgTtnr1XILq3Bm8keaKyMmBjIzgVaDvvee2+0tMDF7yS0lxJcMSRqwzpkIswRljyiQhIXjDF3DVn5mZwUt/K1a4Ic78e6/UqePu8YUq/bVta90fTPlZgjPGVBoRV2JLTIRgDXYPHYJt24I3fnnrLfjhh6L7N2jgWnmGagDTvHmVvC1TTVmCM8ZUmbg4Vy3ZqVPxPn/gWncGVn/6EuEnnxSd3QFcgiup+0N8fFW8KxOrLMFF0JAhQ5g2bRpnnHFGwboHH3yQDRs2MHt20CnuOPXUU5k5cyb9+vXjrLPOYt68eTQP+Dc12MwEgRYsWEDXrl3p7g2WePvttzN48GCGDRtWofdU06fVMdEVH184i3swu3cHL/2tWweLFhXv/tCuXejSX4cO1v2hprMEF0ETJkwgLS2tSIJLS0vjvvvuC+v4RYsWlb5TCAsWLGDkyJEFCe7OO+8s97mMiRXNm0Pv3u4R6PBh1/k9WOOXpUth3ryi3R/q1Su5+0NionV/qO4swUXQeeedx6233srBgwepX78+GRkZ/PDDDwwaNIjLL7+cZcuWsX//fs477zz+8pe/FDs+OTmZ5cuXk5iYyN13383cuXNp06YNHTt2pG/fvoDr4zZnzhwOHjzI0UcfzbPPPsvq1at5/fXXef/995kxYwavvPIKd911FyNHjuS8887jnXfe4cYbbyQ/P58TTzyR2bNn06BBA5KTk5k0aRJvvPEGeXl5vPTSSxwT6l9pbFodE1vq1HGzsbdvDwMHFt9+8KDr/hCs68Nrr8HPPxfdPyEhdPVnSooNfF0d1J4EF4X5clq2bEn//v1ZvHgxo0ePJi0tjQsuuAAR4e6776Zly5YcOnSIoUOH8uWXX3LCCScEPc+KFStIS0tj9erV5Ofn06dPn4IEN3bsWC699FIAbr31Vp544gmuvvpqRo0aVZDQ/OXm5pKamso777xD165dufjii5k9ezbXXXcdAImJiaxcuZJHHnmEmTNn8vjjj4d8fzatjqlO6teHo45yj2Cys4ve//MlwvR0eOcd2Lu36P5t2oROfh07uhKiia7ak+CixFdN6UtwTzzxBAAvvvgic+bMIT8/n+3bt7Nu3bqQCe6DDz5gzJgxNGrUCIBRo0YVbFuzZg233noru3fvJicnp0h1aDAbNmwgJSWFrl27AjBp0iRmzZpVkODGjh0LQN++fXn11VdLPJdNq2NqkiZNoEcP9wjkm9092P2/zz+Hl15yLUR94uJckguW/Dp3dsnRqj8jr/YkuCjNlzN69Giuv/56Vq5cyb59++jbty+bN29m5syZLFu2jBYtWpCamkpu4N3xMKWmprJgwQJ69uzJ008/zZIlSyoUr2/KnYpMt2PT6piaxn929wEDim/Pz4etW4P3/1u40A2N5q9Ro+DdH3zPmzSpkrdV49WeBBclCQkJDBkyhMmTJxfMpr1nzx4aN25Ms2bN+PHHH1m8eHHIeeAABg8eTGpqKn/605/Iz8/njTfeKBhPMjs7m3bt2pGXl8fzzz9fMPVOkyZNyM7OLnaubt26kZGRwcaNGwvu2f06WHvtMNi0OsY4deu6hJWcDEOGFN++b5+r/gxMfunp8P77rnrUX6tWxUt/vmUb/SV8luCqwIQJExgzZgxpaWkA9OzZk969e3PMMcfQsWNHBga7I+6nT58+jBs3jp49e9KmTZsiU97cddddDBgwgNatWzNgwICCpDZ+/HguvfRSHnroIV5++eWC/ePj43nqqac4//zzCxqZTJ06tVzvy6bVMSY8jRpB9+7uEUgVdu0KnvxWrnQNYPLyCvevU8d1cQiW/Gz0l6KiMtiyiAwH/gnEAY+r6j0B228Afg/kAzuByaq6paRz2mDL1Z99XsYUFzj6S2A1aODoL/HxriQZqgFMTRv9JaYGWxaROGAWcDqwFVgmIq+r6jq/3VYB/VR1n4hcDtwHjKvqWI0xJtpKG/1l/37YsiV48vv4Yzc3oL8WLUInv+RkNzxaTRGNKsr+wEZVTQcQkTRgNFCQ4FT1Pb/9PwV+W6URGmNMNdGwYcmjv/zyS/Dkt2YNvPGG6x/oI+L6EYZq/dm+ffWq/oxGgusAfO/3fCsQpF1SgUuAxcE2iMhlwGUAnTp1CnqwqiLWHjfm1ZR5CY2JNS1auEefPsW3HT4M27cHb/357ruuatT/T7N+fTjyyNCtP1u0iK3uDzHdyEREfgv0A4I281PVOcAccPfgArfHx8eTmZlJq1atLMnFMFUlMzOTeBsZ15gq5Wuw0qEDnHJK8e0HDsB33wVvALNsmWsc469p09DJLznZlTarUjQS3Dago9/zJG9dESIyDPgz8GtVPVCeF0pKSmLr1q3s3LmzXIGaqhMfH09SUlK0wzDG+GnQALp0cY9gsrKCz/ywfj0sXhx88Gv/5Dd0KJTQQ6rCopHglgFdRCQFl9jGAxf67yAivYFHgeGq+lN5X6hevXqkpKRUJFZjjDEhNGvmRiwMNjCRaujBrz/8EF54wbUQrVEJTlXzReQq4L+4bgJPqupaEbkTWK6qrwN/BxKAl7yqxe9UdVTIkxpjjIkpIq7E1q4dnHxy8e15ea4KNJKicg9OVRcBiwLW3e63XLFJy4wxxsS0evUiPyB1NWrwaYwxxoQvKiOZRIKI7ARKHO0kTInAz6XuFRuqU6xQveKtTrFC9Yq3OsUK1Sve6hQrVE68R6pq62AbakyCqywisjzUsC+xpjrFCtUr3uoUK1SveKtTrFC94q1OsULk47UqSmOMMTWSJThjjDE1kiW44uZEO4AyqE6xQvWKtzrFCtUr3uoUK1SveKtTrBDheO0enDHGmBrJSnDGGGNqJEtwxhhjaqRameBE5EkR+UlE1oTYLiLykIhsFJEvRSTIRBNVI4xYTxWRLBFZ7T1uD7ZfVRGRjiLynoisE9C//c4AACAASURBVJG1InJtkH1i4vqGGWvMXF8RiReRz0XkCy/evwTZp4GIzPeu7Wciklz1kYYda6qI7PS7tr+PRqx+8cSJyCoRWRhkW0xc14CYSoo31q5thoh85cWyPMj2yHwnqGqtewCDgT7AmhDbz8LNQSfAScBnMRzrqcDCaF9Tv3jaAX285SbAN0D3WLy+YcYaM9fXu14J3nI94DPgpIB9rgD+5S2PB+bHcKypwMPRvq5+8dwAzAv2ecfKdS1DvLF2bTOAxBK2R+Q7oVaW4FR1KbCrhF1GA8+o8ynQXETaVU10RYURa0xR1e2qutJbzga+xk1y6y8mrm+YscYM73rleE/reY/AVmKjgbne8svAUInCZIhhxhozRCQJGAE8HmKXmLiuPmHEW91E5DuhVia4MASbdTxmv/iAX3lVQYtF5LhoB+PjVeP0xv337i/mrm8JsUIMXV+vWmo18BPwP1UNeW1VNR/IAlpVbZROGLECnOtVSb0sIh2DbK8qDwI3A4dDbI+Z6+opLV6InWsL7p+bt0RkhYhcFmR7RL4TLMFVfytxY7H1BP4PWBDleAAQkQTgFeA6Vd0T7XhKUkqsMXV9VfWQqvbCTRTcX0SOj2Y8JQkj1jeAZFU9AfgfhSWkKiUiI4GfVHVFNF6/rMKMNyaurZ9TVLUPcCZwpYgMrooXtQQXXFizjscCVd3jqwpSNw1RPRFJjGZMIlIPlzCeV9VXg+wSM9e3tFhj8fp6sewG3gOGB2wquLYiUhdoBmRWbXRFhYpVVTNV1Tcj2ONA36qOzTMQGCUiGUAacJqIPBewTyxd11LjjaFr64tnm/fzJ+A1oH/ALhH5TrAEF9zrwMVey56TgCxV3R7toIIRkba+ewEi0h/3mUbtC82L5Qnga1W9P8RuMXF9w4k1lq6viLQWkebeckPgdGB9wG6vA5O85fOAd9W7i1+Vwok14B7LKNw90Cqnqn9S1SRVTcY1IHlXVX8bsFtMXFcIL95YubZeLI1FpIlvGfgNENgqPCLfCVGZ8DTaROQFXOu4RBHZCtyBuwmOqv4LNxnrWcBGYB/wu+hEGlas5wGXi0g+sB8YH60/PM9A4CLgK+/+C8AtQCeIuesbTqyxdH3bAXNFJA6XaF9U1YUiciewXFVfxyXsZ0VkI65x0vgYjvUaERkF5HuxpkYp1qBi9LqGFMPX9gjgNe//xLrAPFX9j4hMhch+J9hQXcYYY2okq6I0xhhTI1mCM8YYUyNZgjPGGFMjWYIzxhhTI1mCM8YYUyNZgjMmykTkkN+o76tFZFolnjtZQsxEYUxNVyv7wRkTY/Z7Q1oZYyqRleCMiVHeHFr3efNofS4iR3vrk0XkXW8g3XdEpJO3/ggRec0bGPoLETnZO1WciDwmbl62t7yRRRCRa8TNhfeliKRF6W0aEzGW4IyJvoYBVZTj/LZlqWoP4GHcCPLgBn2e6w2k+zzwkLf+IeB9b2DoPsBab30XYJaqHgfsBs711k8DenvnmRqpN2dMtNhIJsZEmYjkqGpCkPUZwGmqmu4NCr1DVVuJyM9AO1XN89ZvV9VEEdkJJPkNsuubBuh/qtrFe/5HoJ6qzhCR/wA5uBkSFvjN32ZMjWAlOGNim4ZYLosDfsuHKLz3PgKYhSvtLfNGyTemxrAEZ0xsG+f38xNv+WMKB/udCHzgLb8DXA4Fk402C3VSEakDdFTV94A/4qZ/KVaKNKY6s//YjIm+hn6zGQD8R1V9XQVaiMiXuFLYBG/d1cBTInITsJPCkdevBeaIyCW4ktrlQKgpR+KA57wkKMBD3rxtxtQYdg/OmBjl3YPrp6o/RzsWY6ojq6I0xhhTI1kJzhhjTI1kJThjjDE1kiU4Y4wxNZIlOGOMMTWSJThjjDE1kiU4Y4wxNZIlOGOMMTWSJThjjDE1kiU4Y4wxNZIlOBMzRGSxiEyq7H2jyZu0dFgEzqt+E6D+S0RuC2ffcrzORBF5q7xxGhNNNpKJqRAR8Z9DrBFuUOBD3vMpqvp81UcVO7zxJH+vqm9X8nkV6KKqGytrX2/uuM24+eLyKyNOY6LJZhMwFeI/UWdJX+YiUte+NE2ssN/H2sGqKE1EiMipIrJVRP4oIjtw07u0EJGFIrJTRH7xlpP8jlkiIr/3llNF5EMRmentu1lEziznvikislREskXkbRGZJSLPhYg7nBjvEpGPvPO9JSKJftsvEpEtIpIpIn8u4foMEJEdIhLnt26MNzUOItJfRD4Rkd0isl1EHhaR+iHO9bSIzPB7fpN3zA8iMjlg3xEiskpE9ojI9yIy3W/zUu/nbhHJEZFf+a6t3/Eni8gyEcnyfp4c7rUp43VuKSJPee/hFxFZ4LdttIis9t7DJhEZ7q0vUh0sItN9n7OIJHtVtZeIyHfAu976l7zPIcv7HTnO7/iGIvIP7/PM8n7HGorImyJydcD7+VJExgR7ryZ6LMGZSGoLtASOBC7D/b495T3vBOwHHi7h+AHABiARuA94QkSkHPvOAz4HWgHTgYtKeM1wYrwQNwdbG6A+cCOAiHQHZnvnb++9XhJBqOpnwF7gtIDzzvOWDwHXe+/nV8BQ4IoS4saLYbgXz+lAFyDw/t9e4GKgOW5G78tF5Bxv22DvZ3NVTVDVT/wPFJGWwJvAQ957ux94U0RaBbyHYtcmiNKu87O4Ku/jvHM94MXQH3gGuMl7D4OBjFDXI4hfA8cCZ3jPF+OuUxtgJeBfpT4T6AucjPs9vhk4DMwFfuvbSUR6Ah1w18bEElW1hz0q5YH7ohnmLZ8KHATiS9i/F/CL3/MluCpOgFRgo9+2RoACbcuyL+7LMx9o5Lf9OeC5MN9TsBhv9Xt+BW6CUoDbgTS/bY29azAsxLlnAE96y01wyefIEPteB7zm91yBo73lp4EZ3vKTwD1++3X13zfIeR8EHvCWk7196/ptTwU+9JYvAj4POP4TILW0a1OW6wy0wyWSFkH2e9QXb0m/f97z6b7P2e+9dS4hhubePs1wCXg/0DPIfvHAL7j7muAS4SNV/fdmj9IfVoIzkbRTVXN9T0SkkYg86lX57MFViTX3r6YLsMO3oKr7vMWEMu7bHtjltw7g+1ABhxnjDr/lfX4xtfc/t6ruBTJDvRautDZWRBoAY4GVqrrFi6OrV223w4vjr7jSXGmKxABsCXh/A0TkPa9qMAuYGuZ5fefeErBuC6704hPq2hRRynXuiPvMfglyaEdgU5jxBlNwbUQkTkTu8ao591BYEkz0HvHBXsv7nZ4P/FZE6uBmWn+2AjGZCLEEZyIpsInuH4BuwABVbUphlVioasfKsB1oKSKN/NZ1LGH/isS43f/c3mu2CrWzqq7DJYgzKVo9Ca6qcz2ulNAUuKU8MeBKsP7mAa8DHVW1GfAvv/OW1qT6B1yVor9OwLYw4gpU0nX+HveZNQ9y3PfAUSHOuRdXevdpG2Qf//d4ITAaV43bDFfK88XwM5BbwmvNBSbiqo73aUB1rokNluBMVWqCq/bZ7d3PuSPSL+iViJYD00Wkvoj8Cjg7QjG+DIwUkVO8BiF3Uvrf2DzgWtwX/EsBcewBckTkGODyMGN4EUgVke5egg2MvwmudJTr3c+60G/bTlzVYOcQ514EdBWRC0WkroiMA7oDC8OMLTCOoNdZVbfj7o094jVGqScivgT4BPA7ERkqInVEpIN3fQBWA+O9/fsB54URwwFcKbsRrpTsi+Ewrrr3fhFp75X2fuWVtvES2mHgH1jpLWZZgjNV6UGgIe6/40+B/1TR607ENdTIxN33mo/7Ygum3DGq6lrgSlzS2o67T7O1lMNewDV8eFdVf/ZbfyMu+WQDj3kxhxPDYu89vAts9H76uwK4U0SycfcMX/Q7dh9wN/CRuNabJwWcOxMYiSt9ZeIaXYwMiDtcpV3ni4A8XCn2J9w9SFT1c1wjlgeALOB9CkuVt+FKXL8Af6FoiTiYZ3Al6G3AOi8OfzcCXwHLgF3AvRT9znwG6IG7p2tikHX0NrWOiMwH1qtqxEuQpuYSkYuBy1T1lGjHYoKzEpyp8UTkRBE5yqvSGo6777KgtOOMCcWr/r0CmBPtWExoluBMbdAW14Q9B9eH63JVXRXViEy1JSJn4O5X/kjp1aAmiqyK0hhjTI1kJThjjDE1Uo0ZbDkxMVGTk5OjHYYxxpgqtGLFip9VtXWwbTUmwSUnJ7N8+fJoh2GMMaYKiUjg6DoFrIrSGGNMjVRjSnDGGGNi2+HD8P338O237nHccTB4cOnHlZclOGOMMZXm8GH44YfCJOb/2LQJDviNIXT11Zbgyi0vL4+tW7eSm5tb+s4mKuLj40lKSqJevXrRDsUYEyZV2LEjeBLbuBH27y/ct0EDOOoo6NIFzjrL/fQ92rePbJw1OsFt3bqVJk2akJycTOh5Mk20qCqZmZls3bqVlJSUaIdjjPGjCjt3hk5iOTmF+9arB507u6Q1bFjRJJaUBHGhJsSKsBqd4HJzcy25xTARoVWrVuzcuTPaoRhTa2VmBk9i334Le/YU7hcXBykpLmkNHlw0iXXqBHVjMJvEYEiVy5JbbLPPx5jI++WXoqUv/yT2i9+0snXqwJFHuqR10klFk1hysiupVSc1PsEZY0xtsGdP6JJYpt+88iKuxHX00TBuXNEk1rkz1K8fvfdQ2SzBRVBmZiZDhw4FYMeOHcTFxdG6tetw//nnn1O/hN+k5cuX88wzz/DQQw+V+Bonn3wyH3/8ceUFbYyJWTk5xUtgvsdPPxXdNynJJa1zzy2exOLjoxN/VbMEF0GtWrVi9erVAEyfPp2EhARuvPHGgu35+fnUDVFx3a9fP/r161fqa1hyM6Zm2b8/dBLbvr3ovu3auaR19tlFk9hRR0GjRtGJP5ZYgqtiqampxMfHs2rVKgYOHMj48eO59tpryc3NpWHDhjz11FN069aNJUuWMHPmTBYuXMj06dP57rvvSE9P57vvvuO6667jmmuuASAhIYGcnByWLFnC9OnTSUxMZM2aNfTt25fnnnsOEWHRokXccMMNNG7cmIEDB5Kens7ChQuLxJWRkcFFF13E3r17AXj44Yc5+eSTAbj33nt57rnnqFOnDmeeeSb33HMPGzduZOrUqezcuZO4uDheeukljjrqqKq9mMZUU7m5kJ4ePIltDZgDvk0bl7TOOKNoEjv6aEhIiE781UWtSXDXXQdeYarS9OoFDz5Y9uO2bt3Kxx9/TFxcHHv27OGDDz6gbt26vP3229xyyy288sorxY5Zv3497733HtnZ2XTr1o3LL7+8WN+xVatWsXbtWtq3b8/AgQP56KOP6NevH1OmTGHp0qWkpKQwYcKEoDG1adOG//3vf8THx/Ptt98yYcIEli9fzuLFi/n3v//NZ599RqNGjdi1axcAEydOZNq0aYwZM4bc3FwOHz5c9gthTA128CBs3hw8iX33nWuG79OqlUtaQ4YUT2LNmkXvPVR3tSbBxZLzzz+fOK9jSFZWFpMmTeLbb79FRMjLywt6zIgRI2jQoAENGjSgTZs2/PjjjyQlJRXZp3///gXrevXqRUZGBgkJCXTu3Lmgn9mECROYM6f4JMR5eXlcddVVrF69mri4OL755hsA3n77bX73u9/RyKvvaNmyJdnZ2Wzbto0xY8YArrO2MbVRXh5kZASvUszIcKN6+DRv7pLWwIGQmlo0kbVoEaU3UMPVmgRXnpJWpDRu3Lhg+bbbbmPIkCG89tprZGRkcOqppwY9pkGDBgXLcXFx5Ofnl2ufUB544AGOOOIIvvjiCw4fPmxJyxjPoUOwZUvwklhGBvj/mTVp4hLWiSfChRcWTWKtWrkWjKbq1JoEF6uysrLo0KEDAE8//XSln79bt26kp6eTkZFBcnIy8+fPDxlHUlISderUYe7cuRw6dAiA008/nTvvvJOJEycWVFG2bNmSpKQkFixYwDnnnMOBAwc4dOhQQSnPmOrm8GHYts0lrW++KT5+on/FSuPGLmH17g0XXFC0OrFNG0tiscQSXJTdfPPNTJo0iRkzZjBixIhKP3/Dhg155JFHGD58OI0bN+bEE08Mut8VV1zBueeeyzPPPFOwL8Dw4cNZvXo1/fr1o379+px11ln89a9/5dlnn2XKlCncfvvt1KtXj5deeonOnTtXevzGVBZV+PHHwsTln8gCx0+Mj3cJ69hjYfTooiWxtm0tiVUXov53Oquxfv36aeCEp19//TXHHntslCKKHTk5OSQkJKCqXHnllXTp0oXrr78+2mEVsM/JVCb/oacCS2PZ2YX7+cZP7Nq1MHn5ljt0cKN6mNgnIitUNWifKivB1QKPPfYYc+fO5eDBg/Tu3ZspU6ZEOyRjKiRw1A7/ROY19AVckkpOLmzc4Z/IYnX8RFN5Ivrxishw4J9AHPC4qt4TsP1I4EmgNbAL+K2qbvW2TQJu9XadoapzIxlrTXb99dfHVInNmHDs21e8daIvkf34Y9F9O3Z0icv/nljXrm5w4Jo09JQpm4glOBGJA2YBpwNbgWUi8rqqrvPbbSbwjKrOFZHTgL8BF4lIS+AOoB+gwArv2F8wxtQYBw+6Ds/+JTDfcmCH57ZtXeIaOdJG7TDhiWQJrj+wUVXTAUQkDRgN+Ce47sAN3vJ7wAJv+Qzgf6q6yzv2f8Bw4IUIxmuMiYD8fNfMPvB+2DffuPX+fcX8Ozz73xvr0sU1wTemLCKZ4DoA3/s93woMCNjnC2AsrhpzDNBERFqFOLZD4AuIyGXAZQCdOnWqtMCNMWVz+LArcQXeD/vmGzeah38z+6ZNXcIaMAB++9uiiaxly+i9B1PzlJrgRORs4E1VjcRYTDcCD4tIKrAU2AYcCvdgVZ0DzAHXijIC8RljPKqwY0fw1okbN7rxFX0aNnQJq0cPGDu2aBKzvmKmqoRTghsHPCgirwBPqur6MM+9Dejo9zzJW1dAVX/AleAQkQTgXFXdLSLbgFMDjl0S5uvGjCFDhjBt2jTOOOOMgnUPPvggGzZsYPbs2UGPOfXUU5k5cyb9+vXjrLPOYt68eTRv3rzIPsFmJgi0YMECunbtSvfu3QG4/fbbGTx4MMOGDauEd2ZqKtWSm9nn5BTuW7++u//lGwjYP4m1b2/N7E30lZrgVPW3ItIUmAA8LSIKPAW8oKrZJRy6DOgiIim4xDYeuNB/BxFJBHZ5pcM/4VpUAvwX+KuI+EZo+423vVqZMGECaWlpRRJcWloa9913X1jHL1q0qNyvvWDBAkaOHFmQ4O68885yn8vUPFlZoZOY/wzPcXGumX3XrjBoUNG+Yp06ue3GxKqw/sdS1T3Ay0Aa0A53v2yliFxdwjH5wFW4ZPU18KKqrhWRO0VklLfbqcAGEfkGOAK42zt2F3AXLkkuA+70NTipTs477zzefPNNDh48CLgpaX744QcGDRrE5ZdfTr9+/TjuuOO44447gh6fnJzMzz//DMDdd99N165dOeWUU9iwYUPBPo899hgnnngiPXv25Nxzz2Xfvn18/PHHvP7669x000306tWLTZs2kZqayssvvwzAO++8Q+/evenRoweTJ0/mwIEDBa93xx130KdPH3r06MH69cUL6xkZGQwaNIg+ffrQp0+fIvPR3XvvvfTo0YOePXsybdo0ADZu3MiwYcPo2bMnffr0YdOmTZVwZU049u6FL76Al1+Gv/0Nfvc7OOUUOOIIN/Cvb7zEv/wFPvjA3RsbPx7uvx8WLoQNGwqb6i9aBP/8J1x1FfzmN675vSU3E+vCuQc3CvgdcDTwDNBfVX8SkUa4FpH/F+pYVV0ELApYd7vf8su4xBns2CcpLNFVXBTmy2nZsiX9+/dn8eLFjB49mrS0NC644AJEhLvvvpuWLVty6NAhhg4dypdffskJJ5wQ9DwrVqwgLS2N1atXk5+fT58+fejbty8AY8eO5dJLLwXg1ltv5YknnuDqq69m1KhRjBw5kvPOO6/IuXJzc0lNTeWdd96ha9euXHzxxcyePZvrrrsOgMTERFauXMkjjzzCzJkzefzxx4scb9PqxJb8fJeANmwoXiLbtq3ovr7JMUeNKtpX7Kijas8Mz6Z2Cece3LnAA6q61H+lqu4TkUsiE1bN4aum9CW4J554AoAXX3yROXPmkJ+fz/bt21m3bl3IBPfBBx8wZsyYgsGMR40aVbBtzZo13HrrrezevZucnJwi1aHBbNiwgZSUFLp27QrApEmTmDVrVkGCGzt2LAB9+/bl1VdfLXa8TasTHYcPu9aIa9fCmjXusXYtrF/v+pL5JCa6xDVsWPF5xayZvaltwklw04GCidJFpCFwhKpmqOo7kQqs0kVpvpzRo0dz/fXXs3LlSvbt20ffvn3ZvHkzM2fOZNmyZbRo0YLU1FRy/ZuglUFqaioLFiygZ8+ePP300yxZsqRC8fqm3Ak13Y5NqxNZqq65vS+B+ZLZ11+76kKfI4+E44+H4cPhuOPgmGNsXjFjAoVzD+4lwL9e6ZC3zoQhISGBIUOGMHny5ILZtPfs2UPjxo1p1qwZP/74I4sXLy7xHIMHD2bBggXs37+f7Oxs3njjjYJt2dnZtGvXjry8PJ5//vmC9U2aNCE7u3gboG7dupGRkcHGjRsBePbZZ/n1r38d9vvJysqiXbt21KlTh2effbbItDpPPfUU+7xv4V27dtGkSZOCaXUADhw4ULC9tvONbP/OO+7e1mWXwcknu3tjnTrBWWfBTTfBW2+5vmGXXQaPPw6ffurGYczIcPfJ7r0XLr4Y+ve35GZMoHBKcHVVtaASRFUPioiN7lYGEyZMYMyYMaSlpQHQs2dPevfuzTHHHEPHjh0ZOHBgicf36dOHcePG0bNnT9q0aVNkypu77rqLAQMG0Lp1awYMGFCQ1MaPH8+ll17KQw89VNC4BFw14VNPPcX5559Pfn4+J554IlOnTg37vdi0OmW3a1dhacy/VJaZWbhPq1auRHbRRa5Edvzx7qd1fDam/EqdLscbJuv/VPV17/lo4BpVHVoF8YXNpsupvmrK55Sd7RJYYDLbvr1wn6ZNCxOYL4kdf7x1fjamvCo6Xc5U4HkReRgQ3BBaF1difMZUK/v3u3tigQ0+tmwp3KdhQ5e8fvObosksKckSmTFVJZyO3puAk7yRRlDVnFIOMaZGOHjQNbkPrFpMTy8cILh+fdfAY+BAd5/Ml8hSUmwkD2OiLazBlkVkBHAcEC/ev5+qWi2GxlBVxP5ljlmxMKP8oUOwaVPR0tiaNS65+RqSxsW5Voq9esHEiYWlsqOPtkkzjYlV4XT0/hfQCBgCPA6cB3we4bgqRXx8PJmZmbRq1cqSXAxSVTIzM6usq8Hhw64aMfAe2ddfgzeYCyLQubMrhZ1zTuE9sm7dwOtBYYypJsL53/NkVT1BRL5U1b+IyD+Aktu1x4ikpCS2bt3Kzp07ox2KCSE+Pp6kpKRKPacq/PBD0STma/yxd2/hfh07ugQ2bFhh1eKxx4LXMNQYU82Fk+B8PZD3iUh7IBM3HmXMq1evHikpKdEOw0TQzp3F75GtXQu7dxfuc8QRLoFdcklh1WL37tCsWfTiNsZEXjgJ7g0RaQ78HVgJKPBYRKMyJsDu3cH7kvkXzlu0cMlrwoSifckSE6MXtzEmekpMcCJSB3hHVXcDr4jIQiBeVbOqJDpT6+zdC+vWFW/w4T9wcEKCS1yjRhXtU9a2rTXBN8YUKjHBqephEZkF9PaeHwAOVEVgpmbLzXUDBQeWyjZvLtwnPt7dEzvttKKdojt1skRmjCldOFWU74jIucCrGgttuk21c+gQ/Oc/8Pnnhcns228L+5LVrev6kvXvD5MnFyazzp1tzjFjTPmFk+CmADcA+SKSixvNRFW1aUQjM9XewYPw/PNwzz2uT1mdOq7f2PHHwwUXFFYtdukC9epFO1pjTE0TzkgmNouUKZN9+9zI9zNnwvffu87R8+e7e2Y2u44xpqqE09F7cLD1gROgGrN7N8ya5abe+/lnGDQI5syBM86we2bGmKoXThXlTX7L8UB/YAVwWkQiMtXOjz+6pDZrlhtR/6yz4E9/glNOiXZkxpjaLJwqyrP9n4tIRyA602ObmLJlC/z97/DEE26oqwsugGnTXJWkMcZEW3mGid0KVP/Ju0y5ff21azgyb56repw0CW6+2TUWMcaYWBHOPbj/w41eAlAH6IUb0cTUMsuWwd/+BgsWuPnOrroK/vAHN8dZSIcOwZ497pGX59apFn8EW1+WfStrfXU8t0+dOu4/DpHC5apcF+3XL8s6uylcK4RTgvOfJjsfeEFVPwrn5CIyHPgnEAc8rqr3BGzvBMwFmnv7TFPVRSKSDHwNbPB2/VRVp4bzmqYSHTyIZu3h0/9m8ezDWXz9WRbtG2Xx8tlZ/GZAFgmH98A/syAr4LFnT+Fyjk0faGJUuMmxTh3XITPwUbdu8PXleVTWuWItpij/IyGl9d0WkcZArqoe8p7HAQ1UdV8px8UB3wCn46o1lwETVHWd3z5zgFWqOltEugOLVDXZS3ALVfX4cN9Iv379dPny5aXvWBuouqFCSko8pWzTrCwkN7f012rYEJo2dSMXB3v4tjVt6mYH9f3C+/8nHWpdpNfXpNf0leYOHy6+XJXrov36kXg/hw+7mojAR35+8PXleZTlXL4REqoDkZKT5RVXwO23V/AlZIWq9gu2LayRTIBhgO9f8YbAW8DJpRzXH9ioquleEGnAaGCd3z4K+DqMNwN+CCOemk3VlXrKkZSKrPdVB5YkIaFoMmrVisPJKaT/3IwPvmrGptxm1G3ZjEEj3aN+YtPiyat+/chfE2NMIV/ijaWkW95zde8e0UsVToKLV9WCeiZVzRGRRmEc1wH43u/5VmBAwD7TgbdE5GqgMS6R+qSIyCpgD3Crqn4QxmtGl//9pvIkJd/P0v5Dq1OnaMmoWTNo394N3Bis9BSsVNW0aZFxsHJzYe5cuO8+SE93Q2VNewDGj7cZqI2Y2wAADVZJREFUq42JKb5SkY1jV6pwvrr2ikgfVV0JICJ9gf2V9PoTgKdV9R8i8ivgWRE5HtgOdFLVTO/1FojIcaq6x/9gEbkMuAygU6dOFYtEFTIzK1StF9b9prp1iyeczp1LT0j+zxMSKq1uOzsbHn0U7r8ftm9340Hefz+cfbbLo8YYU12Fk+CuA14SkR8AAdoC48I4bhvQ0e95krfO3yXAcABV/URE4oFEVf0Jb9YCVV0hIpuArhRt8IKqzgHmgLsHF0ZMoeXmQuvWJe8TH1888bRrV3pC8n/Ex0f9xiu4XP5//wcPPQS//AJDh8Jzz8GQITERnjHGVFg4Hb2XicgxQDdv1QZVDeMGD8uALiKSgkts44ELA/b5DhgKPC0ix+JGStkpIq2BXap6SEQ6A12A9LDeUXnFx7tv+8Dk5P+8Btxv2rbNldAefdTNvXbOOW7Ukf79ox2ZMcZUrnD6wV0JPK+qa7znLURkgqo+UtJxqpovIlcB/8V1AXhSVdeKyJ3AclV9HfgD8JiIXI9rcJKqquqNf3mniOQBh4GpqrqrIm+0VCJw9dURfYlo2rjR3V+bO9fdKrzwQvjjH929NmOMqYnC6SawWlV7Baxbpaq9IxpZGVk3geC+/NKNOjJ/vpuS5pJL4MYbISUl2pEZY0zFVbSbQJyIiG+yU69/W/Wvq6vhPv7YjTqycCE0aeKS2vXXQ9u20Y7MGGOqRjgJ7j/AfBF51Hs+BVgcuZBMeanCW2+5xPb++9CqFdx1F1x5JbRoEe3ojDGmaoWT4P6Ia4rvGyrrS1xLShMjDh+G116Dv/4VVq6EDh3c9DW//z00bhzt6IwxJjpK7emkqoeBz4AM3Ogkp+HGiTRRlpcHTz/tBgM47zzXp+2JJ1xH7WuvteRmjKndQpbgRKQrriP2BOBnYD6Aqg6pmtBMKPv2uUQ2cyZ8952bf23+fDj3XBvcwBhjfEqqolwPfACMVNWNAF5zfhMlWVnwyCPwwAOwcycMGgT/+hcMH26ds40xJlBJCW4srnP2eyLyHyANN5KJqWI//eTuqc2a5UYHO/NM1zl70KBoR2aMMbErZIJT1QW4MSAb42YBuA5oIyKzgddU9a0qirHW2rLFVUM+/jgcOADnnw/TpkHvmOqBaIwxsSmcRiZ7VXWeqp6NG09yFa5lpYmQ9eshNRWOPtoNqTVxols3f74lN2OMCVeZJkJR1V9wgxvPiUw4tduKFa4P26uvuqExr7wS/vAH6Nix9GONMcYUZTN9RZkqLF3q+rC99RY0bw5//jNcc03pkxsYY4wJzRJclKjCm2+6xPbJJ3DEEXDvvTB1qpvAwBhjTMVYgqti+fnw0kuuKvKrryA52TX9T02Fhg2jHZ0xxtQcluCqyIEDbqqa++6DTZvc6CPPPgvjxrlR/o0xxlSuUltRmorJyXETjHbuDFOmQMuWbtzIr776//buNUauso7j+PdnKYVQgqQluLbFNrEmXLxQG0ArhBTRAkJfQEIbUasVAsotBCigQSX6xheGlBIaQORaW4LSFAKkXNVECrQILVAxBasCNV1oKIINgfL3xfMsPQyz26nsmXPO7O+TbPbMnDOzv/03Pf89l3keOO00Nzczs7L4CK4kW7bAVVelScK3bIGZM+Hmm9N3jzpiZlY+N7hh9sor6Yht8WJ46y2YPTuNOnL44VUnMzMbWdzghskLL6TrazfeCNu3w9y5sGABHHJI1cnMzEYmN7iPaN26dEfksmXpetr8+XDRRTBlStXJzMxGNje4/9OqVekzbHfdBWPHwoUXwvnnQ19f1cnMzAzc4HZJBDzwQGpsjzwC48bBFVfA2WfDvvtWnc7MzIrc4Drw3nuwfHlqbGvWwIQJaU6200/3rNlmZnXlBjeEd96BJUvSEFrr16fR/a+/Pn1+bcyYqtOZmdlQSv2gt6RZkp6XtEHSJW3WHyDpYUl/kbRW0vGFdZfm1z0v6etl5my1bRssWpQa2rx5sPvusHRpmrJm/nw3NzOzJijtCE7SKOBq4FjgJeAJSSsi4rnCZj8Gbo+IayQdBNwDTM7Lc4CDgU8CD0j6TERsLysvwNatcM016fTj5s0wY0Z6fNxx/nC2mVnTlHmK8jBgQ0S8CCBpKWlm8GKDC2Bg7Px9gFfy8mxgaUS8Dfxd0ob8fo+WFXbTJjjwwNTkZs2Cyy6DI48s66eZmVnZymxwE4B/FR6/BLSO5/FTYKWkc4C9gK8WXruq5bUTyomZ9PXBBRfAiSd61mwzs15Q9WDLc4EbI2IicDxwi6SOM0k6Q9JqSav7+/s/cpjLL3dzMzPrFWU2uJeBSYXHE/NzRfOB2wEi4lFgD2B8h68lIq6NiOkRMX0/T39tZmYFZTa4J4CpkqZI2p1008iKlm3+CRwDIOlAUoPrz9vNkTRG0hRgKvB4iVnNzKzHKCLKe/N02/+VwCjghoj4haQrgNURsSLfLXkdMJZ0w8nFEbEyv/ZHwPeAd4HzI+LenfysfuAfwxB7PPDqMLxPNzQpKzQrb5OyQrPyNikrNCtvk7LC8OT9VES0PYVXaoNrIkmrI2J61Tk60aSs0Ky8TcoKzcrbpKzQrLxNygrl5636JhMzM7NSuMGZmVlPcoP7sGurDrALmpQVmpW3SVmhWXmblBWalbdJWaHkvL4GZ2ZmPclHcGZm1pNGZIOTdIOkzZKeGWS9JC3MsxmslTSt2xkLWXaW9WhJWyU9lb8u73bGljyT8gwRz0l6VtJ5bbapRX07zFqb+kraQ9Ljkp7OeX/WZpsxkpbl2j4maXL3k3acdZ6k/kJtv19F1kKeUXlmk7vbrKtFXVsyDZW3brXdKGldzrK6zfpy9gkRMeK+gKOAacAzg6w/HrgXEHAE8FiNsx4N3F11TQt5+oBpeXlv4G/AQXWsb4dZa1PfXK+xeXk08BhwRMs2PwAW5+U5wLIaZ50HLKq6roU8FwBL2v1716Wuu5C3brXdCIwfYn0p+4QReQQXEX8EtgyxyWzg5khWAR+X1NeddB/UQdZaiYhNEfFkXv4PsJ4PD5Rdi/p2mLU2cr3ezA9H56/Wi+izgZvy8h3AMVL3J3vqMGttSJoInABcP8gmtajrgA7yNk0p+4QR2eA60G4mhNru+IAv5VNB90o6uOowA/JpnENJf70X1a6+Q2SFGtU3n5Z6CtgM3B8Rg9Y2It4FtgLjupsy6SArwMn5lNQdkia1Wd8tVwIXA+8Nsr42dc12lhfqU1tIf9yslLRG0hlt1peyT3CDa74nSUPVfB64ClhecR4AJI0FfkcaZu2NqvMMZSdZa1XfiNgeEV8gDUB+mKRDqswzlA6y3gVMjojPAfez4wipqyR9A9gcEWuq+Pm7qsO8tahtwVciYhpwHPBDSUd144e6wbXX0WwGdRARbwycCoqIe4DRksZXmUnSaFLDuC0ift9mk9rUd2dZ61jfnOV14GFgVsuq92sraTfSRMKvdTfdBw2WNSJeizSpMaRTbV/sdrZsBnCSpI3AUmCmpFtbtqlTXXeat0a1Hcjzcv6+GbiTNIF1USn7BDe49lYA38539hwBbI2ITVWHakfSJwauBUg6jPRvWtkOLWf5NbA+In41yGa1qG8nWetUX0n7Sfp4Xt4TOBb4a8tmK4Dv5OVTgIciX8Xvpk6ytlxjOYl0DbTrIuLSiJgYEZNJN5A8FBGntWxWi7pCZ3nrUtucZS9Jew8sA18DWu8KL2WfUOaM3rUl6beku+PGS3oJ+AnpIjgRsRi4h3RXzwbgv8B3q0naUdZTgLMkvQtsA+ZU9R8vmwF8C1iXr78AXAYcALWrbydZ61TfPuAmSaNIjfb2iLhbhRk6SA37FkkbSDcnzalx1nMlnUSaMWQL6c6/2qhpXQdV49ruD9yZ/07cDVgSEfdJOhPK3Sd4JBMzM+tJPkVpZmY9yQ3OzMx6khucmZn1JDc4MzPrSW5wZmbWk9zgzComaXth1PenJF0yjO89WYPMRGHW60bk5+DMamZbHtLKzIaRj+DMairPofXLPI/W45I+nZ+fLOmhPJDug5IOyM/vL+nOPDD005K+nN9qlKTrlOZlW5lHFkHSuUpz4a2VtLSiX9OsNG5wZtXbs+UU5amFdVsj4rPAItII8pAGfb4pD6R7G7AwP78Q+EMeGHoa8Gx+fipwdUQcDLwOnJyfvwQ4NL/PmWX9cmZV8UgmZhWT9GZEjG3z/EZgZkS8mAeF/ndEjJP0KtAXEe/k5zdFxHhJ/cDEwiC7A9MA3R8RU/PjBcDoiPi5pPuAN0kzJCwvzN9m1hN8BGdWbzHI8q54u7C8nR3X3k8AriYd7T2RR8k36xlucGb1dmrh+6N5+c/sGOz3m8Cf8vKDwFnw/mSj+wz2ppI+BkyKiIeBBaTpXz50FGnWZP6Lzax6exZmMwC4LyIGPiqwr6S1pKOwufm5c4DfSLoI6GfHyOvnAddKmk86UjsLGGzKkVHArbkJCliY520z6xm+BmdWU/ka3PSIeLXqLGZN5FOUZmbWk3wEZ2ZmPclHcGZm1pPc4MzMrCe5wZmZWU9ygzMzs57kBmdmZj3JDc7MzHrS/wDEtP0L8LuTPQAAAABJRU5ErkJggg==","text/plain":["<Figure size 432x288 with 2 Axes>"]},"metadata":{"needs_background":"light"},"output_type":"display_data"}],"source":["# Subplot들의 간격 조정하기 참고: https://tempdev.tistory.com/35\n","\n","# plt.subplots\n","fig, axs = plt.subplots(2 ,1, constrained_layout=True)\n","\n","# Training and Validation Loss 그래프\n","axs[0].plot(epochs, loss, 'b', label='Training loss')\n","axs[0].plot(epochs, val_loss, 'r', label='Validation loss')\n","axs[0].set_title('Training and validation loss')\n","axs[0].set_xlabel('Epochs')\n","axs[0].set_ylabel('Loss')\n","axs[0].legend()\n","\n","# Training and Validation Accuracy 그래프\n","axs[1].plot(epochs, acc, 'b', label='Training acc')\n","axs[1].plot(epochs, val_acc, 'r', label='Validation acc')\n","axs[1].set_title('Training and validation accuracy')\n","axs[1].set_xlabel('Epochs')\n","axs[1].set_ylabel('Accuracy')\n","axs[1].legend()\n","\n","\n","plt.show()"]},{"cell_type":"markdown","metadata":{"id":"MK_8VTQD70Qk"},"source":["### 7) 학습된 Embedding 레이어 분석"]},{"cell_type":"code","execution_count":34,"metadata":{"executionInfo":{"elapsed":423,"status":"ok","timestamp":1642994102278,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"HFz9xj6qTWuV"},"outputs":[],"source":["from gensim.models.keyedvectors import Word2VecKeyedVectors"]},{"cell_type":"markdown","metadata":{"id":"d0Xx9NhreWE6"},"source":["- Model 1: LSTM"]},{"cell_type":"code","execution_count":35,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":18,"status":"ok","timestamp":1642994102278,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"TYFFfov3Oj44","outputId":"b1356c24-ad6b-4c20-e9dc-0182586d7535"},"outputs":[{"name":"stdout","output_type":"stream","text":["(10000, 200)\n"]}],"source":["embedding_layer = model_lstm.layers[0]\n","weights = embedding_layer.get_weights()[0]\n","print(weights.shape)    # shape: (vocab_size, embedding_dim)"]},{"cell_type":"code","execution_count":36,"metadata":{"executionInfo":{"elapsed":2195,"status":"ok","timestamp":1642994104457,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"_ZUTJ5pWOl8s"},"outputs":[],"source":["# 학습한 Embedding 파라미터를 파일에 써서 저장합니다. \n","word2vec_file_path = 'drive/MyDrive/Colab Notebooks/sentiment_classification/data/word2vec_LSTM.txt'\n","f = open(word2vec_file_path, 'w')\n","f.write('{} {}\\n'.format(vocab_size-4, word_vector_dim))  # 몇개의 벡터를 얼마 사이즈로 기재할지 타이틀을 씁니다.\n","\n","# 단어 개수(에서 특수문자 4개는 제외하고)만큼의 워드 벡터를 파일에 기록합니다. \n","vectors = model_lstm.get_weights()[0]\n","for i in range(4,vocab_size):\n","    f.write('{} {}\\n'.format(index_to_word[i], ' '.join(map(str, list(vectors[i, :])))))\n","f.close()"]},{"cell_type":"code","execution_count":37,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":2488,"status":"ok","timestamp":1642994106943,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"aHV4xhujOnw4","outputId":"589d8790-a4e3-4ba7-eddb-18adcce7ea16"},"outputs":[{"data":{"text/plain":["array([ 1.15599511e-02, -3.59604508e-02,  1.51203508e-02,  1.89386144e-01,\n","        5.75726740e-02,  2.73495000e-02,  1.68933660e-01, -7.67221227e-02,\n","        1.17870811e-02,  1.64730400e-02, -1.94924660e-02, -3.31952944e-02,\n","       -9.97470506e-03,  6.26773909e-02,  8.14430043e-02,  3.93945538e-02,\n","       -1.29168347e-01,  1.49938852e-01,  7.31094107e-02, -9.43839084e-03,\n","       -6.83573782e-02,  9.68413148e-03, -2.68183276e-02,  7.90559351e-02,\n","        2.21854839e-02,  3.15193459e-02, -7.72445872e-02,  2.69467564e-04,\n","        7.53839016e-02,  9.49576199e-02,  6.06733561e-02,  9.53467563e-02,\n","       -5.11434898e-02,  4.29606698e-02, -1.21458821e-01,  4.56217863e-02,\n","        2.94061881e-02, -1.76502727e-02, -6.95532784e-02, -3.17792557e-02,\n","        1.05763242e-01, -1.70692783e-02,  1.66010652e-02, -6.05575070e-02,\n","        2.72291973e-02,  6.61186427e-02, -3.28605250e-03,  1.97056793e-02,\n","        2.85470393e-02, -9.69168246e-02, -5.85180819e-02, -2.78378930e-02,\n","        6.57828450e-02,  4.56302688e-02,  3.06355655e-02,  2.68973976e-01,\n","       -2.69970074e-02,  2.50300281e-02, -1.51505142e-01, -7.38346800e-02,\n","       -3.75884883e-02,  2.71481760e-02, -1.78256318e-01, -7.61985555e-02,\n","        1.35375299e-02,  4.00303379e-02,  4.33051400e-02, -1.00087471e-01,\n","        8.92524049e-02, -1.25224471e-01, -6.74773566e-03, -6.05343617e-02,\n","        6.24410398e-02, -1.49847809e-02,  4.14633527e-02, -3.26999999e-03,\n","       -2.51978129e-01,  1.12940483e-01,  1.07343636e-01, -1.41071364e-01,\n","        5.89978509e-02,  7.33124167e-02, -7.91905224e-02, -6.22675605e-02,\n","       -8.41747373e-02, -4.16889526e-02,  1.97491683e-02,  3.05254180e-02,\n","       -3.65726165e-02,  1.11188680e-01,  2.55921930e-02,  5.35429046e-02,\n","       -3.48790921e-02,  3.38803940e-02,  8.25596526e-02, -4.35320102e-02,\n","        2.45707128e-02, -2.38350332e-02, -6.80163354e-02, -1.33174071e-02,\n","        3.26766074e-02,  1.01064436e-01,  5.57657592e-02,  1.26566499e-01,\n","        4.25787549e-03, -8.72997218e-04,  3.13539281e-02,  9.71220154e-03,\n","       -8.09606351e-03, -1.23815350e-01,  7.81948566e-02, -7.21547753e-02,\n","       -8.71811137e-02, -5.64607382e-02, -2.71488670e-02, -9.70166177e-03,\n","       -8.90007243e-02,  1.46563590e-01, -6.87253997e-02,  4.64332551e-02,\n","       -4.89165932e-02,  2.32720580e-02,  9.50930566e-02, -3.02216113e-02,\n","        6.13412494e-03, -7.08185583e-02, -2.68123597e-02, -4.55307364e-02,\n","        2.95634549e-02,  3.91585827e-02,  4.00378415e-03,  5.29363044e-02,\n","        1.56936064e-01, -1.84428059e-02,  1.64692216e-02, -2.93729361e-02,\n","        3.86451967e-02,  1.07642844e-01, -6.58588856e-03, -7.88305402e-02,\n","       -2.65829042e-02,  6.56149536e-02, -8.22590478e-03,  5.99956559e-03,\n","       -2.17965040e-02, -4.33195010e-02,  5.27508557e-02, -1.86270997e-02,\n","        3.51839662e-02, -2.98410952e-02,  9.26311407e-03,  2.55369861e-03,\n","        4.72785942e-02, -7.00350292e-03, -4.13951874e-02,  5.78216556e-03,\n","        1.67844385e-01,  5.78147471e-02,  3.61288190e-02,  1.09137870e-01,\n","       -1.39698103e-01, -8.51167142e-02, -3.44190523e-02,  4.66798097e-02,\n","       -6.45019859e-02,  2.04306263e-02, -2.48714127e-02, -5.88788791e-03,\n","        7.51897991e-02, -8.96684732e-03,  9.51584503e-02,  1.60026439e-02,\n","        1.80507023e-02, -1.25387773e-01,  2.14377902e-02, -9.73970518e-02,\n","       -6.12526871e-02,  4.63036522e-02, -1.04129329e-01,  7.14459121e-02,\n","        1.34823825e-02,  8.41542780e-02, -8.83828253e-02,  6.60495386e-02,\n","       -8.53917152e-02, -2.74094511e-02, -2.63365060e-02, -1.97014771e-04,\n","        7.02475086e-02,  9.32830051e-02,  6.99753612e-02,  1.63306184e-02,\n","       -5.17390706e-02,  4.76108603e-02, -8.62614252e-03,  8.81360751e-03,\n","        1.59141570e-02, -4.62636873e-02,  9.20892693e-03,  2.84449477e-02],\n","      dtype=float32)"]},"execution_count":37,"metadata":{},"output_type":"execute_result"}],"source":["# gensim에서 제공하는 패키지를 이용해, 위에 남긴 임베딩 파라미터를 읽어서 word vector로 활용\n","word_vectors = Word2VecKeyedVectors.load_word2vec_format(word2vec_file_path, binary=False)\n","vector = word_vectors['재미']\n","vector"]},{"cell_type":"code","execution_count":38,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":38,"status":"ok","timestamp":1642994106944,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"bIrcnspNOp7Y","outputId":"96d2d307-90a8-400b-a1db-739e8dac2fd9"},"outputs":[{"data":{"text/plain":["[('완전', 0.757716178894043),\n"," ('넘', 0.7401788830757141),\n"," ('더불', 0.7306267023086548),\n"," ('너무', 0.7235556840896606),\n"," ('이리', 0.7154935002326965),\n"," ('띠', 0.7124409675598145),\n"," ('긴장감', 0.7105696201324463),\n"," ('기대', 0.7052869200706482),\n"," ('닿', 0.6945632100105286),\n"," ('거의', 0.6874486804008484)]"]},"execution_count":38,"metadata":{},"output_type":"execute_result"}],"source":["# 워드 벡터가 의미 벡터 공간상에 유의미하게 학습되었는지 확인하기\n","# => 단어를 하나 주고 그와 가장 유사한 단어와 그 유사도를 확인하는 방법\n","word_vectors.similar_by_word(\"재미\")  # 유사 단어와 유사도 확인"]},{"cell_type":"code","execution_count":39,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":35,"status":"ok","timestamp":1642994106944,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"Y8m_A2noRD8k","outputId":"c0807808-7bbd-48af-ae57-5fdeb9f88c5e"},"outputs":[{"data":{"text/plain":["[('명품', 0.8917285203933716),\n"," ('굿', 0.88321852684021),\n"," ('미소', 0.8820233941078186),\n"," ('흥미진진', 0.8697075843811035),\n"," ('아담스', 0.8673045039176941),\n"," ('완벽', 0.8664607405662537),\n"," ('신들린', 0.8623718619346619),\n"," ('따스', 0.8615207672119141),\n"," ('유머러스', 0.8603526949882507),\n"," ('탱고', 0.8556028008460999)]"]},"execution_count":39,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"최고\")"]},{"cell_type":"code","execution_count":40,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":30,"status":"ok","timestamp":1642994106944,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"AlOvG3kIRPgT","outputId":"46d87dfe-6e5a-42e7-e2b7-388cc05a44ea"},"outputs":[{"data":{"text/plain":["[('쓰레기', 0.8883030414581299),\n"," ('졸작', 0.8764959573745728),\n"," ('재미없', 0.8747398853302002),\n"," ('억지', 0.8687449097633362),\n"," ('어설픈', 0.8685830235481262),\n"," ('제로', 0.8680387139320374),\n"," ('따분', 0.8662077784538269),\n"," ('구림', 0.8648359775543213),\n"," ('꽝', 0.8611047267913818),\n"," ('뻔해서', 0.8603533506393433)]"]},"execution_count":40,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"별로\")"]},{"cell_type":"code","execution_count":41,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":28,"status":"ok","timestamp":1642994106945,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"juoqQCYmRoss","outputId":"3fdebfe6-5d17-4f69-bb76-45c08cd87b34"},"outputs":[{"data":{"text/plain":["[('끌리', 0.8639836311340332),\n"," ('못지않', 0.8490253686904907),\n"," ('깨끗', 0.8455284833908081),\n"," ('명장면', 0.8426334857940674),\n"," ('최상', 0.8368562459945679),\n"," ('여사', 0.8356959819793701),\n"," ('고조', 0.8347816467285156),\n"," ('진하', 0.8337646722793579),\n"," ('쌀', 0.8337490558624268),\n"," ('후편', 0.8334569931030273)]"]},"execution_count":41,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"엉엉\")"]},{"cell_type":"code","execution_count":42,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":25,"status":"ok","timestamp":1642994106945,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"jFpcvJ7PlnCS","outputId":"20237730-f6dc-4361-a32b-ccfc5f53834a"},"outputs":[{"data":{"text/plain":["[('유쾌', 0.7988952398300171),\n"," ('긴장감', 0.7773101329803467),\n"," ('캐', 0.7601171731948853),\n"," ('창의', 0.7469446659088135),\n"," ('미학', 0.7443838119506836),\n"," ('암튼', 0.7424423694610596),\n"," ('최소', 0.7421683073043823),\n"," ('소화', 0.7309916615486145),\n"," ('왤케', 0.7304494380950928),\n"," ('삶', 0.7279245257377625)]"]},"execution_count":42,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"매력\")"]},{"cell_type":"code","execution_count":43,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":23,"status":"ok","timestamp":1642994106946,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"uKWytHgblpY8","outputId":"3a27fc75-97dc-414e-d53a-88c7b28ead8c"},"outputs":[{"data":{"text/plain":["[('긴장감', 0.6452733278274536),\n"," ('모두', 0.6123091578483582),\n"," ('인정', 0.6122713685035706),\n"," ('로렌스', 0.6050076484680176),\n"," ('독특', 0.6021087169647217),\n"," ('또한', 0.5990393757820129),\n"," ('항상', 0.5943559408187866),\n"," ('어쨋든', 0.5876557230949402),\n"," ('진정', 0.5873904228210449),\n"," ('투쟁', 0.5847823023796082)]"]},"execution_count":43,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"반전\")"]},{"cell_type":"markdown","metadata":{"id":"Jy0qlfuN3-R3"},"source":["- Model 2: 1-D CNN"]},{"cell_type":"code","execution_count":44,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":19,"status":"ok","timestamp":1642994106946,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"Qc4qM3B54biB","outputId":"c59e27bf-de8d-4c16-d9fe-50a0f7c8949c"},"outputs":[{"name":"stdout","output_type":"stream","text":["(10000, 200)\n"]}],"source":["embedding_layer = model_1d_cnn.layers[0]\n","weights = embedding_layer.get_weights()[0]\n","print(weights.shape)    # shape: (vocab_size, embedding_dim)"]},{"cell_type":"code","execution_count":45,"metadata":{"executionInfo":{"elapsed":2466,"status":"ok","timestamp":1642994109396,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"BmEQY9O-5AJ_"},"outputs":[],"source":["# 학습한 Embedding 파라미터를 파일에 써서 저장합니다. \n","word2vec_file_path = 'drive/MyDrive/Colab Notebooks/sentiment_classification/data/word2vec_1-D_CNN.txt'\n","f = open(word2vec_file_path, 'w')\n","f.write('{} {}\\n'.format(vocab_size-4, word_vector_dim))  # 몇개의 벡터를 얼마 사이즈로 기재할지 타이틀을 씁니다.\n","\n","# 단어 개수(에서 특수문자 4개는 제외하고)만큼의 워드 벡터를 파일에 기록합니다. \n","vectors = model_1d_cnn.get_weights()[0]\n","for i in range(4,vocab_size):\n","    f.write('{} {}\\n'.format(index_to_word[i], ' '.join(map(str, list(vectors[i, :])))))\n","f.close()"]},{"cell_type":"code","execution_count":46,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":2141,"status":"ok","timestamp":1642994111528,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"eLBHfMTT5AG6","outputId":"dbfc5c24-e274-4195-ab2b-0e0c5e153baa"},"outputs":[{"data":{"text/plain":["array([ 0.04134703, -0.02434341, -0.02596116,  0.0071864 , -0.02185209,\n","       -0.0587077 , -0.01844174,  0.03430754,  0.00327818, -0.03799032,\n","        0.05521839,  0.03988498, -0.04963054,  0.04630915,  0.03676181,\n","       -0.01683879, -0.00232387, -0.11235008, -0.06199582, -0.13553055,\n","        0.02810752, -0.04475434,  0.00866413,  0.06013425, -0.09264623,\n","       -0.06783593, -0.11980495,  0.0358476 , -0.06785541, -0.01609808,\n","        0.00483533,  0.08515575,  0.08373234,  0.09358314, -0.02778801,\n","        0.02954536, -0.05550599, -0.01628357, -0.05057786, -0.13453467,\n","       -0.05892323,  0.07378894, -0.06506231, -0.0629918 ,  0.02726937,\n","       -0.0136245 ,  0.01256663,  0.0590753 ,  0.03902242,  0.00727947,\n","        0.06979833, -0.06752416, -0.02422572,  0.01719517, -0.00983879,\n","       -0.02463958, -0.0661689 , -0.01416873,  0.01329864, -0.04046849,\n","        0.01704728, -0.02586146, -0.09884497, -0.03335802,  0.09599873,\n","       -0.05473552, -0.05671005,  0.011891  ,  0.04988721, -0.00938463,\n","        0.01405562, -0.04227621,  0.0294939 ,  0.01826822,  0.12634856,\n","       -0.0386922 , -0.05611722,  0.00100576,  0.01449321,  0.02682138,\n","       -0.00811747, -0.02088471, -0.06865143, -0.04534618, -0.01238543,\n","        0.01429939, -0.00196961, -0.11857808,  0.0113981 , -0.02010052,\n","       -0.00838749,  0.02256009,  0.02290786,  0.0195952 ,  0.06981657,\n","        0.01198311,  0.00754081, -0.11771896, -0.04552408,  0.00025504,\n","        0.08390615, -0.04604164,  0.05884627,  0.00798626, -0.02830367,\n","        0.03808177, -0.00477564, -0.06306519,  0.00066281,  0.06990898,\n","       -0.02447404, -0.02221356, -0.03619851, -0.06788822, -0.05170999,\n","       -0.0309211 ,  0.03452538,  0.01000396,  0.07025704, -0.02917412,\n","        0.11723495, -0.02998932, -0.00899033, -0.06166574, -0.05974387,\n","        0.04707124, -0.00138708, -0.03693607, -0.03428502,  0.01350579,\n","       -0.00781378, -0.01857121, -0.00289697,  0.03164862, -0.02647259,\n","        0.01369326, -0.10788389,  0.00845351, -0.03599028, -0.05586057,\n","        0.01711028, -0.02852936,  0.1147607 , -0.05587469,  0.02967203,\n","       -0.04597401,  0.01108977, -0.03666905, -0.0159916 , -0.03589112,\n","       -0.00862835, -0.05167798, -0.06590357,  0.00820157,  0.03137938,\n","        0.02865746, -0.04119446,  0.05334031,  0.10465113, -0.09758931,\n","       -0.02269778,  0.03959584, -0.01034478,  0.00045507,  0.00066695,\n","       -0.01406533,  0.0577856 ,  0.13152961,  0.02264036, -0.007156  ,\n","        0.11950058,  0.01631357,  0.09240403,  0.06608698,  0.05304947,\n","        0.01839153,  0.04889895,  0.00960542,  0.03065591, -0.08107387,\n","       -0.01348072,  0.03950563,  0.02274153, -0.00142874,  0.11916889,\n","        0.00479792,  0.06315044, -0.04404753, -0.02347233,  0.07042438,\n","       -0.0587756 ,  0.03867217,  0.0020741 , -0.03650598,  0.10705162,\n","       -0.02557315,  0.06009254,  0.05049419,  0.08856811, -0.04389209],\n","      dtype=float32)"]},"execution_count":46,"metadata":{},"output_type":"execute_result"}],"source":["# gensim에서 제공하는 패키지를 이용해, 위에 남긴 임베딩 파라미터를 읽어서 word vector로 활용\n","word_vectors = Word2VecKeyedVectors.load_word2vec_format(word2vec_file_path, binary=False)\n","vector = word_vectors['재미']\n","vector"]},{"cell_type":"code","execution_count":47,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":20,"status":"ok","timestamp":1642994111528,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"o0SB4Q0_5G0q","outputId":"155a467d-991e-4c98-da08-02f1ebbb348b"},"outputs":[{"data":{"text/plain":["[('정사', 0.5040515661239624),\n"," ('존재감', 0.49690523743629456),\n"," ('오빠', 0.4709305763244629),\n"," ('흠흠', 0.4703681766986847),\n"," ('넘어서', 0.46792352199554443),\n"," ('겐', 0.4583006799221039),\n"," ('정권', 0.4521726965904236),\n"," ('열라', 0.44678810238838196),\n"," ('나탈리', 0.4447680711746216),\n"," ('달라', 0.4421963393688202)]"]},"execution_count":47,"metadata":{},"output_type":"execute_result"}],"source":["# 워드 벡터가 의미 벡터 공간상에 유의미하게 학습되었는지 확인하기\n","# => 단어를 하나 주고 그와 가장 유사한 단어와 그 유사도를 확인하는 방법\n","word_vectors.similar_by_word(\"재미\")  # 유사 단어와 유사도 확인"]},{"cell_type":"code","execution_count":48,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":16,"status":"ok","timestamp":1642994111529,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"OEhR5rk3RxOB","outputId":"daf1486e-e20e-4995-a3f8-f3e128bd9924"},"outputs":[{"data":{"text/plain":["[('이제서야', 0.7765339612960815),\n"," ('담백', 0.7701098918914795),\n"," ('펑펑', 0.7490707635879517),\n"," ('멋져요', 0.7483293414115906),\n"," ('여운', 0.7477136850357056),\n"," ('편도', 0.7412523627281189),\n"," ('호랑이', 0.7389379739761353),\n"," ('동포', 0.7374215126037598),\n"," ('황홀', 0.7369143962860107),\n"," ('본방', 0.73538738489151)]"]},"execution_count":48,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"최고\")"]},{"cell_type":"code","execution_count":49,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":13,"status":"ok","timestamp":1642994111529,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"L3fnsxhyRzCD","outputId":"1ff753d9-d409-4113-b971-7a52382e3058"},"outputs":[{"data":{"text/plain":["[('포장', 0.7981038093566895),\n"," ('착오', 0.7945014238357544),\n"," ('과대평가', 0.791346549987793),\n"," ('쓰레기', 0.7884174585342407),\n"," ('유치찬란', 0.7863763570785522),\n"," ('낭패', 0.7775011658668518),\n"," ('아까', 0.7750105857849121),\n"," ('반개', 0.7684745788574219),\n"," ('콱', 0.7667334675788879),\n"," ('늘어진다', 0.7660669088363647)]"]},"execution_count":49,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"별로\")"]},{"cell_type":"code","execution_count":50,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":11,"status":"ok","timestamp":1642994111529,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"tTKrrPYsRzzI","outputId":"217b0ae4-dac1-4dbd-8257-cc3c999e37a7"},"outputs":[{"data":{"text/plain":["[('훈훈', 0.7759348750114441),\n"," ('주몽', 0.7446386814117432),\n"," ('되돌아보', 0.7440555095672607),\n"," ('헌신', 0.7367801666259766),\n"," ('담백', 0.7242134809494019),\n"," ('곱씹', 0.7204722166061401),\n"," ('드라마틱', 0.7182658910751343),\n"," ('명불허전', 0.7164402008056641),\n"," ('뗄', 0.7143599390983582),\n"," ('레고', 0.7123239040374756)]"]},"execution_count":50,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"엉엉\")"]},{"cell_type":"code","execution_count":51,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":19,"status":"ok","timestamp":1642994111945,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"cZIKfLS779Gy","outputId":"e730153c-00e2-46a0-9d4d-b4b4a3409dde"},"outputs":[{"data":{"text/plain":["[('감춰진', 0.6063684821128845),\n"," ('입감', 0.6037587523460388),\n"," ('무거운', 0.5950450897216797),\n"," ('웨스턴', 0.594286322593689),\n"," ('헌신', 0.5871021747589111),\n"," ('굳', 0.5846222043037415),\n"," ('감동', 0.5839760303497314),\n"," ('장난아', 0.5780368447303772),\n"," ('소화', 0.5743797421455383),\n"," ('그해', 0.5711315870285034)]"]},"execution_count":51,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"매력\")"]},{"cell_type":"code","execution_count":52,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":14,"status":"ok","timestamp":1642994111945,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"v5fFJtyt7-tD","outputId":"ca58b6c1-c41d-4e7e-c22c-728264a8c363"},"outputs":[{"data":{"text/plain":["[('두고두고', 0.49870771169662476),\n"," ('쌍', 0.492639422416687),\n"," ('야말로', 0.4858424663543701),\n"," ('으신', 0.4822346270084381),\n"," ('엘', 0.4804536700248718),\n"," ('지날수록', 0.4743923246860504),\n"," ('이토록', 0.468650221824646),\n"," ('아름다운', 0.46455609798431396),\n"," ('거울', 0.46072012186050415),\n"," ('긴장감', 0.4595382511615753)]"]},"execution_count":52,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"반전\")"]},{"cell_type":"markdown","metadata":{"id":"qKRw14_z4KuK"},"source":["- Model 3: # GlobalMaxPooling1D() 레이어 1개만 사용"]},{"cell_type":"code","execution_count":53,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":11,"status":"ok","timestamp":1642994111946,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"eSh9-CdQ4b39","outputId":"d742e73e-47f5-4b76-df19-8e82e81bb73d"},"outputs":[{"name":"stdout","output_type":"stream","text":["(10000, 200)\n"]}],"source":["embedding_layer = model_gmp1d.layers[0]\n","weights = embedding_layer.get_weights()[0]\n","print(weights.shape)    # shape: (vocab_size, embedding_dim)"]},{"cell_type":"code","execution_count":54,"metadata":{"executionInfo":{"elapsed":2479,"status":"ok","timestamp":1642994114417,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"u4ng2z4T5Nay"},"outputs":[],"source":["# 학습한 Embedding 파라미터를 파일에 써서 저장합니다. \n","word2vec_file_path = 'drive/MyDrive/Colab Notebooks/sentiment_classification/data/word2vec_GlobalMaxPooling1D.txt'\n","f = open(word2vec_file_path, 'w')\n","f.write('{} {}\\n'.format(vocab_size-4, word_vector_dim))  # 몇개의 벡터를 얼마 사이즈로 기재할지 타이틀을 씁니다.\n","\n","# 단어 개수(에서 특수문자 4개는 제외하고)만큼의 워드 벡터를 파일에 기록합니다. \n","vectors = model_gmp1d.get_weights()[0]\n","for i in range(4,vocab_size):\n","    f.write('{} {}\\n'.format(index_to_word[i], ' '.join(map(str, list(vectors[i, :])))))\n","f.close()"]},{"cell_type":"code","execution_count":55,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":2001,"status":"ok","timestamp":1642994116416,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"Dr4rkEhV5OWe","outputId":"ec703122-c36c-425c-a83c-b58598f449f9"},"outputs":[{"data":{"text/plain":["array([-0.0523678 ,  0.12495606, -0.04349257, -0.0554188 , -0.00609744,\n","       -0.04880491, -0.02695485, -0.02475937, -0.05744374,  0.03399743,\n","       -0.02575667, -0.10672331, -0.06338732, -0.09608673, -0.05874534,\n","       -0.0363675 , -0.00102614, -0.01469714, -0.08175734, -0.04322144,\n","       -0.06141543, -0.0726357 ,  0.03567281, -0.01352176,  0.1466846 ,\n","       -0.05210067, -0.06121134,  0.0293897 , -0.03626779, -0.04324828,\n","       -0.00815098, -0.03624084,  0.10856152,  0.04644124, -0.04558814,\n","       -0.05752408, -0.05428651, -0.02277846, -0.07116133, -0.05423525,\n","       -0.03111629, -0.05247708,  0.09282546,  0.04470009, -0.05723304,\n","        0.0411249 , -0.00986466, -0.04593084, -0.01793112, -0.03854621,\n","       -0.0487396 ,  0.06610681,  0.0356398 , -0.04463397,  0.33385885,\n","        0.04116601, -0.0520281 , -0.0697555 , -0.05887554, -0.06267077,\n","        0.02370942,  0.12722993, -0.01772998, -0.06101006, -0.04731524,\n","       -0.05947956, -0.05084667, -0.00774187, -0.07058977, -0.00769562,\n","        0.01825573, -0.09766031,  0.01171733, -0.07471231, -0.05558429,\n","       -0.03477369, -0.00190679, -0.04281137, -0.03499711, -0.07286585,\n","       -0.03133897, -0.06821264, -0.09127499, -0.03742083, -0.06685144,\n","        0.14190227, -0.02755663, -0.04676033, -0.07320062, -0.09185988,\n","        0.02324599, -0.06177331, -0.05587384, -0.05707911, -0.04865352,\n","        0.08949786, -0.09406406,  0.00627435, -0.0112028 , -0.05959734,\n","       -0.02511694, -0.06418872, -0.08620974,  0.01947826, -0.002181  ,\n","       -0.05465831, -0.06348017, -0.06629202, -0.03913575, -0.06067978,\n","       -0.00789379, -0.0787468 , -0.00413492, -0.02230423, -0.04449713,\n","       -0.04916576, -0.0520482 , -0.05078012,  0.02250485, -0.03999405,\n","       -0.02040707, -0.07175688, -0.02017715, -0.07254167, -0.01954835,\n","       -0.02755936, -0.0251391 ,  0.10370518, -0.09959006, -0.04860874,\n","       -0.12483389,  0.01221966, -0.05500047, -0.06237043, -0.06913634,\n","       -0.06507053, -0.09737162, -0.04596897, -0.04965904, -0.08210429,\n","       -0.04661398, -0.06812911, -0.029217  , -0.07475292, -0.03869118,\n","       -0.04373318,  0.07061995, -0.01912174, -0.05564994,  0.01857706,\n","       -0.07334994,  0.07565088, -0.01448111, -0.02997441, -0.07816254,\n","       -0.0323096 , -0.00152351,  0.04447246, -0.0429823 , -0.01198557,\n","       -0.08086596, -0.04126002, -0.04716232, -0.05089356,  0.03712721,\n","       -0.05272984,  0.10380194, -0.07909078, -0.02551376, -0.04082807,\n","       -0.08585772,  0.09804302, -0.02901005,  0.04786892, -0.09934554,\n","        0.01026891, -0.03492602, -0.03872825,  0.04593427,  0.00895297,\n","        0.14896345, -0.06726147,  0.01401379, -0.00453565, -0.03771841,\n","       -0.0644561 ,  0.12717955, -0.0450495 , -0.04418118, -0.01923404,\n","       -0.06222741, -0.08327743, -0.03984153, -0.02309057,  0.0746792 ,\n","       -0.09654612, -0.04241182, -0.02030847, -0.01920683, -0.02076588],\n","      dtype=float32)"]},"execution_count":55,"metadata":{},"output_type":"execute_result"}],"source":["# gensim에서 제공하는 패키지를 이용해, 위에 남긴 임베딩 파라미터를 읽어서 word vector로 활용\n","word_vectors = Word2VecKeyedVectors.load_word2vec_format(word2vec_file_path, binary=False)\n","vector = word_vectors['재미']\n","vector"]},{"cell_type":"code","execution_count":56,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":560,"status":"ok","timestamp":1642994116972,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"aQeF-ju65OTL","outputId":"13ca42e0-84ab-4f7f-ddcd-47219645c4e3"},"outputs":[{"data":{"text/plain":["[('스토리', 0.4149767756462097),\n"," ('긴장감', 0.4109301269054413),\n"," ('완', 0.4071023464202881),\n"," ('감동', 0.4030129611492157),\n"," ('예뻐서', 0.4027698040008545),\n"," ('하늘', 0.3982482850551605),\n"," ('어울리', 0.39427632093429565),\n"," ('진짜', 0.39338165521621704),\n"," ('는데', 0.39296314120292664),\n"," ('깊이', 0.3925923705101013)]"]},"execution_count":56,"metadata":{},"output_type":"execute_result"}],"source":["# 워드 벡터가 의미 벡터 공간상에 유의미하게 학습되었는지 확인하기\n","# => 단어를 하나 주고 그와 가장 유사한 단어와 그 유사도를 확인하는 방법\n","word_vectors.similar_by_word(\"재미\")  # 유사 단어와 유사도 확인"]},{"cell_type":"code","execution_count":57,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":23,"status":"ok","timestamp":1642994116973,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"-RhnMS6DR6fo","outputId":"e824ee92-43e4-4285-8358-462a6d42ada3"},"outputs":[{"data":{"text/plain":["[('누군가', 0.36325740814208984),\n"," ('당대', 0.3536795377731323),\n"," ('수작', 0.33141815662384033),\n"," ('아날로그', 0.314446359872818),\n"," ('바치', 0.31333059072494507),\n"," ('곡', 0.31100383400917053),\n"," ('의외로', 0.31060802936553955),\n"," ('공권력', 0.30855754017829895),\n"," ('감수성', 0.30527111887931824),\n"," ('잼따', 0.2995571196079254)]"]},"execution_count":57,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"최고\")"]},{"cell_type":"code","execution_count":58,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":20,"status":"ok","timestamp":1642994116973,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"l_MANApER8cD","outputId":"8958e77d-3d92-4437-b7ed-d2c8740993cc"},"outputs":[{"data":{"text/plain":["[('어설픈', 0.5763288736343384),\n"," ('진부', 0.5739359855651855),\n"," ('-_', 0.5682641863822937),\n"," ('표절', 0.5643913149833679),\n"," ('짱나', 0.5627702474594116),\n"," ('그지', 0.5605618953704834),\n"," ('어휴', 0.5605323314666748),\n"," ('뻔한', 0.5575464963912964),\n"," ('별루', 0.5521450042724609),\n"," ('난해', 0.5379496812820435)]"]},"execution_count":58,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"별로\")"]},{"cell_type":"code","execution_count":59,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":17,"status":"ok","timestamp":1642994116973,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"Af182rH_R90P","outputId":"f42a99c6-ff79-4a02-9adf-86d7fdd383bd"},"outputs":[{"data":{"text/plain":["[('감회', 0.45710986852645874),\n"," ('정화', 0.45638078451156616),\n"," ('는다는', 0.4455082416534424),\n"," ('이뻐', 0.4291670620441437),\n"," ('연락', 0.4234762489795685),\n"," ('최고봉', 0.4205551743507385),\n"," ('발견', 0.4170006513595581),\n"," ('다가왔', 0.41380515694618225),\n"," ('감탄', 0.40907543897628784),\n"," ('힐링', 0.4064022898674011)]"]},"execution_count":59,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"엉엉\")"]},{"cell_type":"code","execution_count":60,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":16,"status":"ok","timestamp":1642994116974,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"YGq60aXT8E6L","outputId":"6730dc18-04d2-45e0-8718-b70a9cda5dbb"},"outputs":[{"data":{"text/plain":["[('나루토', 0.46137771010398865),\n"," ('게', 0.4294614791870117),\n"," ('롭', 0.42611876130104065),\n"," ('바랍니다', 0.41053926944732666),\n"," ('이쁜', 0.409149706363678),\n"," ('음', 0.40361204743385315),\n"," ('뮤', 0.3967868983745575),\n"," ('로맨스', 0.39550620317459106),\n"," ('시', 0.3941178321838379),\n"," ('미셸', 0.39407801628112793)]"]},"execution_count":60,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"매력\")"]},{"cell_type":"code","execution_count":61,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":14,"status":"ok","timestamp":1642994116974,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"SZGH41kf8GYK","outputId":"4f16c839-ebd0-4daf-9854-5641de82befb"},"outputs":[{"data":{"text/plain":["[('에서', 0.5754741430282593),\n"," ('는데', 0.5681638121604919),\n"," ('하', 0.547177791595459),\n"," ('봤', 0.5454105138778687),\n"," (',', 0.5425621271133423),\n"," ('롭', 0.5361525416374207),\n"," ('숨겨진', 0.5272085666656494),\n"," ('기억', 0.5176573991775513),\n"," ('너무', 0.5171259641647339),\n"," ('영화', 0.516504168510437)]"]},"execution_count":61,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"반전\")"]},{"cell_type":"markdown","metadata":{"id":"BFATMGxsYxEZ"},"source":["- word_vectors.wimiar_by_word()로 유사 단어 찾기 결론\n","  - 감정과 관련한 단어를 인자로 넣으면 세 모델 모두 그 감정과 연관있는 유사 단어를 뽑아냈다. 특히 '별로' 라는 단어의 유사 단어들을 잘 찾아낸 것을 볼 수 있다. 부정적인 감정 단어가 데이터셋에 많아서 그런 것인지는 모르겠지만 긍정보다 더 잘 찾는다고 느껴졌다.\n","  - 감정과 관련 없는 일반적인 단어를 인자로 넣으면 비슷한 단어를 찾지 못하고 상관없는 단어를 뱉어냈다. 감정 분석이다 보니 당연한 결과인 듯 하다."]},{"cell_type":"markdown","metadata":{"id":"S08rCxwn8IB0"},"source":["### 8) 한국어 Word2Vec 임베딩 활용하여 성능 개선\n","\n","#### **gensim 3.x.x 버전으로 다운그레이드**\n","4.0.x 버전부터는 vocab 코드가 gensim 패키지 안에서 사라졌기 때문에 ko.bin 을 로드하려면 gensim 3.x.x 버전이 필요하다.\n","```bash\n","pip install --upgrade gensim==3.8.3\n","```"]},{"cell_type":"markdown","metadata":{"id":"LzOIEXglKldJ"},"source":["#### **Word2Vec 모델 로드 코드 수정**\n","```python\n","word2vec_path = 'drive/MyDrive/Colab Notebooks/sentiment_classification/data/ko.bin'\n","```\n","기존의 lms 코드로는 에러가 범벅이었다. UnicodeDecodeError 다음에는 ValueError 가 나를 반겼다.\n","```python\n","# 기존의 lms 코드\n","word2vec = KeyedVectors.load_word2vec_format(word2vec_path, binary=True, limit=1000000)\n","```\n","그래서 구글링한 코드로 해결했다.\n","```python\n","# 구글링한 코드\n","word2vec = Word2Vec.load(word2vec_path)\n","```\n","  - 두 코드의 차이점\n","    - KeyedVectors.load_word2vec_format(): 학습된 word vector 로드\n","    - Word2Vec.load(): 학습한 model 로드"]},{"cell_type":"markdown","metadata":{"id":"zpXaW1xoOroH"},"source":["#### - Mdel: LSTM\n","  - 카피한 임베딩(ko.bin) 활용\n","  - 앞서 세 가지 모델로 테스트해본 결과 LSTM 성능이 제일 좋았기 때문에 LSTM으로 진행\n","  - LSTM state vector의 차원 수를 256 으로 변경해서 진행"]},{"cell_type":"code","execution_count":62,"metadata":{"executionInfo":{"elapsed":11,"status":"ok","timestamp":1642994116974,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"jsj9EHbJ52gq"},"outputs":[],"source":["from gensim.models import Word2Vec\n","from gensim.models import KeyedVectors"]},{"cell_type":"code","execution_count":63,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":2151,"status":"ok","timestamp":1642994119114,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"D5mmiWRxTjeC","outputId":"23e6a695-f425-4f21-b631-6e06607d1343"},"outputs":[{"name":"stdout","output_type":"stream","text":["(30185, 200)\n"]},{"data":{"text/plain":["[('취미', 0.5857348442077637),\n"," ('유머', 0.5140613913536072),\n"," ('매력', 0.5105490684509277),\n"," ('흥미', 0.4988338351249695),\n"," ('공짜', 0.4960595667362213),\n"," ('일자리', 0.49294644594192505),\n"," ('즐거움', 0.48700767755508423),\n"," ('비애', 0.4836210310459137),\n"," ('관객', 0.48286449909210205),\n"," ('향수', 0.4823310971260071)]"]},"execution_count":63,"metadata":{},"output_type":"execute_result"}],"source":["word2vec_path = 'drive/MyDrive/Colab Notebooks/sentiment_classification/data/ko.bin'\n","\n","word2vec = Word2Vec.load(word2vec_path)  # 학습한 모델 로드\n","\n","# 완성된 임베딩 매트릭스의 크기 확인\n","print(word2vec.wv.vectors.shape)\n","\n","vector = word2vec.wv.most_similar(\"재미\")  # 메모리를 다소 많이 소비하는 작업이니 유의해 주세요.\n","vector     # 무려 300dim의 워드 벡터입니다."]},{"cell_type":"code","execution_count":64,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":433,"status":"ok","timestamp":1642994119542,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"0jRxiFm8UqJ8","outputId":"fbd73d38-4121-48f4-ac1b-0238fbaa65cd"},"outputs":[{"name":"stderr","output_type":"stream","text":["/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:7: DeprecationWarning: Call to deprecated `__contains__` (Method will be removed in 4.0.0, use self.wv.__contains__() instead).\n","  import sys\n","/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:8: DeprecationWarning: Call to deprecated `__getitem__` (Method will be removed in 4.0.0, use self.wv.__getitem__() instead).\n","  \n"]}],"source":["vocab_size = len(word_to_index)    # 어휘 사전의 크기입니다(10,000개의 단어)\n","word_vector_dim = word2vec.wv.vectors.shape[1]  # 워드 벡터의 차원수 200\n","embedding_matrix = np.random.rand(vocab_size, word_vector_dim)\n","\n","# embedding_matrix에 Word2Vec 워드 벡터를 단어 하나씩마다 차례차례 카피한다.\n","for i in range(4,vocab_size):\n","    if index_to_word[i] in word2vec:\n","        embedding_matrix[i] = word2vec[index_to_word[i]]"]},{"cell_type":"code","execution_count":65,"metadata":{"executionInfo":{"elapsed":10,"status":"ok","timestamp":1642994119543,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"5_3nTV7wU9am"},"outputs":[],"source":["from tensorflow.keras.initializers import Constant"]},{"cell_type":"markdown","metadata":{"id":"IAFGM_BCmfwt"},"source":["- LSTM"]},{"cell_type":"code","execution_count":66,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":9,"status":"ok","timestamp":1642994119543,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"FfGezILAWhEk","outputId":"57770242-891c-4d9e-ff1f-82163cbf7285"},"outputs":[{"name":"stdout","output_type":"stream","text":["Model: \"sequential_3\"\n","_________________________________________________________________\n"," Layer (type)                Output Shape              Param #   \n","=================================================================\n"," embedding_3 (Embedding)     (None, 41, 200)           2000000   \n","                                                                 \n"," lstm_1 (LSTM)               (None, 256)               467968    \n","                                                                 \n"," dense_6 (Dense)             (None, 8)                 2056      \n","                                                                 \n"," dense_7 (Dense)             (None, 1)                 9         \n","                                                                 \n","=================================================================\n","Total params: 2,470,033\n","Trainable params: 2,470,033\n","Non-trainable params: 0\n","_________________________________________________________________\n"]}],"source":["# 모델 구성\n","model = tf.keras.Sequential()\n","model.add(tf.keras.layers.Embedding(vocab_size, \n","                                 word_vector_dim, \n","                                 embeddings_initializer=Constant(embedding_matrix),  # 카피한 임베딩을 여기서 활용\n","                                 input_length=maxlen, \n","                                 trainable=True))   # trainable을 True로 주면 Fine-tuning\n","model.add(tf.keras.layers.LSTM(256))   # 가장 널리 쓰이는 RNN인 LSTM 레이어를 사용하였습니다. LSTM state 벡터의 차원수는 변경 가능\n","model.add(tf.keras.layers.Dense(8, activation='relu'))\n","model.add(tf.keras.layers.Dense(1, activation='sigmoid')) \n","\n","model.summary()"]},{"cell_type":"code","execution_count":67,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":1523951,"status":"ok","timestamp":1642995643488,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"jz5PU3rGWjX6","outputId":"d49f1a64-11b6-482c-d546-2e35ba735537"},"outputs":[{"name":"stdout","output_type":"stream","text":["Epoch 1/5\n","229/229 [==============================] - 296s 1s/step - loss: 0.4604 - accuracy: 0.7775 - val_loss: 0.3572 - val_accuracy: 0.8413\n","Epoch 2/5\n","229/229 [==============================] - 294s 1s/step - loss: 0.3241 - accuracy: 0.8604 - val_loss: 0.3187 - val_accuracy: 0.8618\n","Epoch 3/5\n","229/229 [==============================] - 294s 1s/step - loss: 0.2782 - accuracy: 0.8836 - val_loss: 0.3124 - val_accuracy: 0.8652\n","Epoch 4/5\n","229/229 [==============================] - 293s 1s/step - loss: 0.2441 - accuracy: 0.9004 - val_loss: 0.3123 - val_accuracy: 0.8666\n","Epoch 5/5\n","229/229 [==============================] - 296s 1s/step - loss: 0.2138 - accuracy: 0.9137 - val_loss: 0.3221 - val_accuracy: 0.8686\n"]}],"source":["# 학습의 진행\n","model.compile(optimizer='adam',\n","              loss='binary_crossentropy',\n","              metrics=['accuracy'])\n","              \n","epochs=5  # 몇 epoch를 훈련하면 좋을지 결과를 보면서 바꾸어 봅시다. \n","\n","history = model.fit(partial_x_train,\n","                    partial_y_train,\n","                    epochs=epochs,\n","                    batch_size=512,\n","                    validation_data=(x_val, y_val),\n","                    verbose=1,\n","                    callbacks=[early_stopping])"]},{"cell_type":"code","execution_count":68,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":73333,"status":"ok","timestamp":1642995716801,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"HSEm4CLUWk9D","outputId":"3de31f55-90d1-42d0-b9ae-ca73a707fa4f"},"outputs":[{"name":"stdout","output_type":"stream","text":["1537/1537 - 73s - loss: 0.3288 - accuracy: 0.8648 - 73s/epoch - 48ms/step\n","[0.32877132296562195, 0.8648208975791931]\n"]}],"source":["# 테스트셋을 통한 모델 평가\n","results = model.evaluate(X_test,  y_test, verbose=2)\n","\n","print(results)  # (0.3288, 0.8648)"]},{"cell_type":"code","execution_count":69,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":12,"status":"ok","timestamp":1642995716802,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"bSzqwIXFavRE","outputId":"02a5a1dd-6e2e-4669-881f-625e6c508744"},"outputs":[{"name":"stdout","output_type":"stream","text":["dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])\n"]}],"source":["history_dict = history.history\n","print(history_dict.keys()) # epoch에 따른 그래프를 그려볼 수 있는 항목들"]},{"cell_type":"code","execution_count":70,"metadata":{"executionInfo":{"elapsed":7,"status":"ok","timestamp":1642995716802,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"W8Zg8NIxay70"},"outputs":[],"source":["acc = history_dict['accuracy']\n","val_acc = history_dict['val_accuracy']\n","loss = history_dict['loss']\n","val_loss = history_dict['val_loss']\n","\n","epochs = range(1, len(acc) + 1)"]},{"cell_type":"code","execution_count":71,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":313},"executionInfo":{"elapsed":630,"status":"ok","timestamp":1642995717425,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"Nw-mdBkUa10k","outputId":"60a9c46c-2620-4e5a-ee17-16bf0f7f8e74"},"outputs":[{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAbgAAAEoCAYAAAAqrOTwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXxU5fX48c8hEEJI2EGWgIDsCIQQQEAQXFpUClWxilZNtSq4Vdy1KhTrt7bSaqlL61K1LkWrlR8qVAWloGhlEVEUZAuCAkIQCEtISM7vj+dOMpnMJJNlMjPhvF+vec2du82ZG5gzz3KfR1QVY4wxpq6pF+0AjDHGmEiwBGeMMaZOsgRnjDGmTrIEZ4wxpk6yBGeMMaZOsgRnjDGmTrIEZ+oEEZkvIpfV9L7RJCLZInJ6BM6rItLNW/6riNwTzr5VeJ+LReSdqsZZznlHi8i2mj6vqXvqRzsAc+wSkQN+L5OBI0Ch9/pqVX0x3HOp6pmR2LeuU9XJNXEeEekMbAYaqOpR79wvAmH/DY2paZbgTNSoaopvWUSygV+q6oLA/USkvu9L0xhjwmVVlCbm+KqgROR2EdkBPCMizUXkTRHZJSI/eMtpfscsEpFfestZIvKBiMz09t0sImdWcd8uIrJYRHJFZIGIPCoiL4SIO5wY7xORD73zvSMirfy2XyIiW0QkR0R+Xc71GSoiO0QkwW/dOSKy2lseIiIficheEdkuIo+ISGKIcz0rIr/1e32rd8x3InJ5wL5ni8inIrJfRLaKyHS/zYu9570ickBEhvmurd/xw0VkmYjs856Hh3ttyiMivb3j94rIGhEZ77ftLBH50jvntyJyi7e+lff32Ssie0RkiYjY92EdY39QE6vaAi2A44GrcP9Wn/FedwIOA4+Uc/xQYB3QCvgD8LSISBX2fQn4BGgJTAcuKec9w4nxIuAXQBsgEfB94fYBHvfO3957vzSCUNX/AQeBUwPO+5K3XAhM9T7PMOA04Jpy4saLYawXzxlAdyCw/e8gcCnQDDgbmCIiP/W2jfKem6lqiqp+FHDuFsBbwCzvs/0JeEtEWgZ8hjLXpoKYGwBvAO94x10PvCgiPb1dnsZVd6cCJwLveetvBrYBrYHjgLsAG7ewjrEEZ2JVETBNVY+o6mFVzVHV11T1kKrmAvcDp5Rz/BZVfVJVC4HngHa4L7Kw9xWRTsBg4F5VzVfVD4C5od4wzBifUdWvVfUw8AqQ7q2fCLypqotV9Qhwj3cNQvknMAlARFKBs7x1qOoKVf1YVY+qajbwtyBxBPMzL74vVPUgLqH7f75Fqvq5qhap6mrv/cI5L7iEuF5Vn/fi+iewFviJ3z6hrk15TgJSgAe8v9F7wJt41wYoAPqISBNV/UFVV/qtbwccr6oFqrpEbWDeOscSnIlVu1Q1z/dCRJJF5G9eFd5+XJVYM/9qugA7fAuqeshbTKnkvu2BPX7rALaGCjjMGHf4LR/yi6m9/7m9BJMT6r1wpbVzRaQhcC6wUlW3eHH08Krfdnhx/B+uNFeRUjEAWwI+31ARed+rgt0HTA7zvL5zbwlYtwXo4Pc61LWpMGZV9f8x4H/e83DJf4uI/FdEhnnrHwQ2AO+IyCYRuSO8j2HiiSU4E6sCf03fDPQEhqpqE0qqxEJVO9aE7UALEUn2W9exnP2rE+N2/3N779ky1M6q+iXui/xMSldPgqvqXAt09+K4qyox4KpZ/b2EK8F2VNWmwF/9zltR6ec7XNWtv07At2HEVdF5Owa0nxWfV1WXqeoEXPXlHFzJEFXNVdWbVbUrMB64SUROq2YsJsZYgjPxIhXXprXXa8+ZFuk39EpEy4HpIpLo/fr/STmHVCfGV4FxInKy1yFkBhX//3wJ+BUukf4rII79wAER6QVMCTOGV4AsEenjJdjA+FNxJdo8ERmCS6w+u3BVql1DnHse0ENELhKR+iJyAdAHV51YHf/DlfZuE5EGIjIa9zea7f3NLhaRpqpagLsmRQAiMk5Eunltrftw7ZblVQmbOGQJzsSLh4FGwG7gY+A/tfS+F+M6auQAvwVext2vF0yVY1TVNcC1uKS1HfgB1wmiPL42sPdUdbff+ltwyScXeNKLOZwY5nuf4T1c9d17AbtcA8wQkVzgXrzSkHfsIVyb44dez8STAs6dA4zDlXJzgNuAcQFxV5qq5uMS2pm46/4YcKmqrvV2uQTI9qpqJ+P+nuA60SwADgAfAY+p6vvVicXEHrF2VWPCJyIvA2tVNeIlSGNM9VgJzphyiMhgETlBROp53egn4NpyjDExzkYyMaZ8bYF/4zp8bAOmqOqn0Q3JGBMOq6I0xhhTJ1kVpTHGmDrJEpwxxpg6qc60wbVq1Uo7d+4c7TCMMcbUohUrVuxW1dbBttWZBNe5c2eWL18e7TCMMcbUIhEJHAKumFVR+jkS6vZdY4wxcccSnOfwYejdG667DrZvj3Y0xhhjqssSnCcvD370I/jb36BrV7j1VthdrUGEjDHGRFOduQ8uMzNTa6INbuNGmDEDXngBkpNh6lS46SZo1qwGgjTGxJSCggK2bdtGXl5exTubqEpKSiItLY0GDRqUWi8iK1Q1M9gxluBC+OormDYN/vUvaN7cleiuvx5SwpmhyhgTFzZv3kxqaiotW7Yk9ITvJtpUlZycHHJzc+nSpUupbeUlOKuiDKF3b3jlFfj0UxgxAu66C044AR5+2FVnGmPiX15eniW3OCAitGzZstIlbUtwFUhPhzfegI8+gv79XZVlt27w179Cfn60ozPGVJclt/hQlb+TJbgwnXQSvPsuvP8+HH88TJkCvXrBc89BYWG0ozPGxKOcnBzS09NJT0+nbdu2dOjQofh1fgW/oJcvX84NN9xQ4XsMHz68RmJdtGgR48aNq5Fz1RZLcJU0ejR88AHMm+fa5rKy4MQTXXVmkc0HbIyphJYtW7Jq1SpWrVrF5MmTmTp1avHrxMREjh49GvLYzMxMZs2aVeF7LF26tCZDjiuW4KpABM48E5Yvh9deg3r14IILICPDVWfWkX47xpgoyMrKYvLkyQwdOpTbbruNTz75hGHDhjFw4ECGDx/OunXrgNIlqunTp3P55ZczevRounbtWirxpXg94xYtWsTo0aOZOHEivXr14uKLL8bXyXDevHn06tWLQYMGccMNN1RYUtuzZw8//elP6d+/PyeddBKrV68G4L///W9xCXTgwIHk5uayfft2Ro0aRXp6OieeeCJLliyp8WsWSp0ZqisaRODcc2HCBJg92/W6HD8ehg6F3/4WTjvN7WOMiX033girVtXsOdPTXce0ytq2bRtLly4lISGB/fv3s2TJEurXr8+CBQu46667eO2118ocs3btWt5//31yc3Pp2bMnU6ZMKdOl/tNPP2XNmjW0b9+eESNG8OGHH5KZmcnVV1/N4sWL6dKlC5MmTaowvmnTpjFw4EDmzJnDe++9x6WXXsqqVauYOXMmjz76KCNGjODAgQMkJSXxxBNP8OMf/5hf//rXFBYWcujQocpfkCqyElwNSEiAiy92txY8+SR89x2ccQaMGQMffhjt6Iwx8eb8888nISEBgH379nH++edz4oknMnXqVNasWRP0mLPPPpuGDRvSqlUr2rRpw86dO8vsM2TIENLS0qhXrx7p6elkZ2ezdu1aunbtWtz9PpwE98EHH3DJJZcAcOqpp5KTk8P+/fsZMWIEN910E7NmzWLv3r3Ur1+fwYMH88wzzzB9+nQ+//xzUlNTq3pZKs1KcDWoQQP45S/hkktcorv/fjj5ZBg71pXoBg2KdoTGmFCqUtKKlMaNGxcv33PPPYwZM4bXX3+d7OxsRo8eHfSYhg0bFi8nJCQEbb8LZ5/quOOOOzj77LOZN28eI0aM4O2332bUqFEsXryYt956i6ysLG666SYuvfTSGn3fUKwEFwENG7oxLTduhD/8AT75BDIzXXXmF19EOzpjTDzZt28fHTp0AODZZ5+t8fP37NmTTZs2kZ2dDcDLL79c4TEjR47kxRdfBFzbXqtWrWjSpAkbN26kX79+3H777QwePJi1a9eyZcsWjjvuOK688kp++ctfsnLlyhr/DKFYgoug5GQ3AsrmzfCb38DChe5euosvhvXrox2dMSYe3Hbbbdx5550MHDiwxktcAI0aNeKxxx5j7NixDBo0iNTUVJo2bVruMdOnT2fFihX079+fO+64g+eeew6Ahx9+mBNPPJH+/fvToEEDzjzzTBYtWsSAAQMYOHAgL7/8Mr/61a9q/DOEYkN11aI9e+DBB2HWLDc1T1YW3HOPu6/OGFP7vvrqK3r37h3tMKLuwIEDpKSkoKpce+21dO/enalTp0Y7rDKC/b1sqK4Y0aIF/O53sGmTG9fyhRege3eboscYE11PPvkk6enp9O3bl3379nH11VdHO6QaYSW4KNq61XVEefppqF/fJbrbb4dWraIdmTHHBivBxZe4KMGJyFgRWSciG0TkjnL2O09EVESCBh/vOnZ0Y1quXQs/+xn86U/QpQvcey/s3Rvt6IwxJr7VeoITkQTgUeBMoA8wSUT6BNkvFfgV8L/ajbD2nXCCG9Pyiy/cCCn33ecmXf3d7+DAgWhHZ4wx8SkaJbghwAZV3aSq+cBsYEKQ/e4Dfg8cM5PT2BQ9xhhTc6KR4DoAW/1eb/PWFRORDKCjqr5Vm4HFCpuixxhjqi/melGKSD3gT8DNYex7lYgsF5Hlu3btinxwtcym6DGmbhszZgxvv/12qXUPP/wwU6ZMCXnM6NGj8XWoO+uss9gbpMF++vTpzJw5s9z3njNnDl9++WXx63vvvZcFCxZUJvygYmlanWgkuG+Bjn6v07x1PqnAicAiEckGTgLmButooqpPqGqmqma2bt06giFHl03RY0zdNGnSJGbPnl1q3ezZs8MaDxLcLADNmjWr0nsHJrgZM2Zw+umnV+lcsSoaCW4Z0F1EuohIInAhMNe3UVX3qWorVe2sqp2Bj4Hxqhpf9wDUMP8pev79bzfAs03RY0x8mzhxIm+99Vbx5KbZ2dl89913jBw5kilTppCZmUnfvn2ZNm1a0OM7d+7M7t27Abj//vvp0aMHJ598cvGUOuDucRs8eDADBgzgvPPO49ChQyxdupS5c+dy6623kp6ezsaNG8nKyuLVV18FYOHChQwcOJB+/fpx+eWXc+TIkeL3mzZtGhkZGfTr14+1a9eW+/miPa1OrQ+2rKpHReQ64G0gAfi7qq4RkRnAclWdW/4Zjm0icM45blqel1+2KXqMqTFRmC+nRYsWDBkyhPnz5zNhwgRmz57Nz372M0SE+++/nxYtWlBYWMhpp53G6tWr6d+/f9DzrFixgtmzZ7Nq1SqOHj1KRkYGg7zR3c8991yuvPJKAO6++26efvpprr/+esaPH8+4ceOYOHFiqXPl5eWRlZXFwoUL6dGjB5deeimPP/44N954IwCtWrVi5cqVPPbYY8ycOZOnnnoq5OeL9rQ6UWmDU9V5qtpDVU9Q1fu9dfcGS26qOvpYL70Fk5AAF10EX34JTz3lRkKxKXqMiT/+1ZT+1ZOvvPIKGRkZDBw4kDVr1pSqTgy0ZMkSzjnnHJKTk2nSpAnjx48v3vbFF18wcuRI+vXrx4svvhhyuh2fdevW0aVLF3r06AHAZZddxuLFi4u3n3vuuQAMGjSoeIDmUKI9rY5NlxPnGjSAK66An//cpugxplqiNF/OhAkTmDp1KitXruTQoUMMGjSIzZs3M3PmTJYtW0bz5s3Jysoir4r3CmVlZTFnzhwGDBjAs88+y6JFi6oVr2/KnepMt1Nb0+rEXC9KUzWBU/QsW2ZT9BgTD1JSUhgzZgyXX355celt//79NG7cmKZNm7Jz507mz59f7jlGjRrFnDlzOHz4MLm5ubzxxhvF23Jzc2nXrh0FBQXFU9wApKamkpubW+ZcPXv2JDs7mw0bNgDw/PPPc8opp1Tps0V7Wh1LcHWMb4qeTZtgxgyboseYeDBp0iQ+++yz4gTnm16mV69eXHTRRYwYMaLc4zMyMrjgggsYMGAAZ555JoMHDy7edt999zF06FBGjBhBr169itdfeOGFPPjggwwcOJCNGzcWr09KSuKZZ57h/PPPp1+/ftSrV4/JkydX6XNFe1odG2y5jrMpeowJzQZbji9xMdiyqT02RY8x5lhlCe4Ycdxx8NBDrpry8svhb39zAzrfeit4t9EYY0ydYgnuGGNT9BhjjhXVTnAi0tgbPxIR6SEi40WkQfVDM5FkU/QY49SVfgh1XVX+TjVRglsMJIlIB+Ad4BLg2Ro4b+0qKoJ+/eDss92IBo8+6kY6zs6u0yMb2xQ95liWlJRETk6OJbkYp6rk5OSQlJRUqeOq3YtSRFaqaoaIXA80UtU/iMgqVU2v1okrqdq9KHNz3R3T69fD11+D/zAxDRu6b/3u3aFHD/fsW27Xrk6NjfXxx66X5YIF0KED3H23a7NLTIx2ZMbUvIKCArZt21blm6hN7UlKSiItLY0GDUpXEJbXi7ImEtynwDXAQ8AV3riSn6tqv2qduJJq9DYBVdfF8OuvXcLzJb31692d1N7AowA0blyS8AITYKtWcZv8Fi2CX/8ali51bXTTprnRUhISoh2ZMcaUiHSCOwU3d9uHqvp7EekK3KiqN1TrxJVUa/fBFRbC1q2lk55vefPm0tWZzZoFT3zdu7ttMU4V/vMfV4pbudLNRfeb38DEiVDPuicZY2JARBNcwBvVA1JUdX+NnTRMMXGjd0GBa7MLVvL75pvSc9q0bl068fmeu3VzpcIYogpz5riqyzVrYMAA1yll3Li4LaAaY+qISJfgXgImA4W4ud6aAH9W1QerdeJKiokEV568PFe9GazkF3jHdYcOwUt+J5zg2gOjpLCwZIqeDRtsih5jTPRFOsGtUtV0EbkYyADuAFaoavCJiyIk5hNceQ4ccBnDl/j8E6D/Xdj16kGnTmU7unTvDp07Q/3amRyioAD+8Q831uU338App7hZDCoYLs8YY2pcpBPcGiAdeAl4RFX/KyKfqeqAap24kuI6wZXnhx/KVnf6lvf71QTXr+9uZAtMfD16QFpaRBrNjhwpmaJnxw6boscYU/sineBuAG4HPgPOBjoBL6jqyGqduJLqbIILRRV27Qre3rd+PRw+XLJvUpKr3gxW8mvbttr1i4cOudsGf/97yMlxM47PmAEnnljNz2iMMRWotU4mfm9YX1WrNhNeFR1zCa48RUXw3XdlE9/XX7t2wIKCkn1TUoLf5tCjB7RsWam33b8f/vxnmDnT3VY4aRJMn+5OZ4wxkRDpElxTYBowylv1X2CGqu6r1okryRJcmAoLXcNZsJJf4KgtzZsHv7m9e3do0iTkW+zZ45Lcn/9sU/QYYyIr0gnuNeAL4Dlv1SXAAFU9t1onriRLcDUgP9/dyxesvW/r1tL7tmlT9hYH320OyckA7NwJDzwAjz/uCpXDhrnR0E480T337RsXtwMaY6pD1dUaHT7s2jP8n1u3dh3kqqFWelFWtC7SLMFF2OHDrnozWMlvx47S+6allUp8u5t35+nF3flgTXM+X5fI7txE8kmkgAZ07CjFCc/33KuXazY0xkSIL+kEJpxgSaiyz8HWhRrP9+abXXVPNZSX4GqiX/lhETlZVT/w3mwEcLiCY0y8adTIZaBgPUf273e3OQSW/P71L9izh1a4Xki3BzltwbeJ5H+bSN58l/TySWQriZDYkITkRBJTEklKTSS5WSKNmjVEGia6gTH9Hw0bll1X2fXl7WvDtpjaECrpVCexlPdcVFS1OBs1co/k5NLPjRq5Kplg20I9R7iBviYS3GTgH15bHMAPwGU1cF4TL5o0gYwM9wi0Z49Ldhs2uPv9jhxxVaHeo4H3SDp8hP278sn7Pp9DOfkc3JPP4b1HKNjmS3t5JMl+UpPySU08QnL9fBqK21bvaD7iO28kZn5ISKh+koxkAk5MjFwSjtQd/PFyXlX376omE0t556hO0vFPNP6JpFmz8BNORfskJcXVqA7VTnCq+hkwQESaeK/3i8iNwOrqntvUAS1auCFPhg4td7cEoLn38HfwIHz1FXz+uZu7zvfsP/hL8+bQb7BXxdmnkH69CujbPZ9myfllEir5IdaFWl/ZfQ8dCm9fU/eVlyyaN69eovF/jrOkU5sidZvAN6raqZztY4E/477XnlLVBwK2TwauxQ3/dQC4SlW/LO89rQ3u2JKTUzrh+Z79731PS6NM+17v3jHQvueriqqpJHvkSOlxTmsyzkiIp/OqulJyZRJOcrI7xpJOrYjGfXBbVbVjiG0JwNfAGcA23PiVk/wTmIg08Q3YLCLjgWtUdWx572kJzqi6zp6Bie+rr0oKTfXquWr/wMR3wgk2FZAx8SjSnUyCKS9rDgE2qOomABGZDUwAihNcwGwEjSs4nzGA+8HcqZN7nHVWyfqjR10zoH/i++wz+Pe/S370JyVBnz5lE1/79vZD3Jh4VeUEJyK5BE88AjQq59AOgP9NVduAMg00InItcBOQCJxa1TiNqV/fVU327g3nn1+y/tAh+PLL0onv3XfdQNI+zZuXdB71Jb4TT3TrjTGxrcoJTlVTazKQIOd/FHhURC4C7iZIz0wRuQq4CqBTp5BNfsYElZwMmZnu4c/Xvuef+F58sXT7XocOpUt6J57oEmij8n7aGWNqVUTa4Mp9Q5FhwHRV/bH3+k4AVf1diP3rAT+oatNg232sDc5Ekips21a2U8uXX5Zu3+vWrWzi69bN2veMiZRotMGVZxnQXUS6AN8CFwIX+e8gIt1Vdb338mxgPcZEkQh07Ogege17GzaUTnyrV5dt3+vdu2zi69DB2veMiaRaT3CqelRErgPext0m8HdVXSMiM4DlqjoXuE5ETgcKsBvHTQyrX98NLdarV9n2vcD79xYsKN2+16xZ2U4t1r5nTM2p9SrKSLEqShMPcnJgzZqyVZ37/Obe6NAh+P171r5nTFmxVkVpzDGrZUsYNco9fHzte4H37y1a5O7hhpL2vWD379W3/8XGBGX/NYyJMv/2vTPPLFnva98LTHyvv17Svtewobt/r3t36NKl9OP4490wlcYcq6yK0pg442vf8098GzfCli2lJ2sXccOVBSa+rl3dc7t2NlGCiX9WRWlMHZKcDIMGuYe/wkL47jvYtMnNW+v/WLAAvv229P4NG7pSXrDk16WL6+xivTxNPLMEZ0wdkZBQUtV5yillt+fluVJeYPLbvBk++QR++KH0/k2ahE5+nTsXT9xuTMyyBGfMMSIpCXr2dI9g9u0LnvzWrYO333ZTlvk77rjgya9LF5dkrfOLiTb7J2iMAaBpU0hPd49AqrBzZ9nkt2kTfPQRvPJK6blmfaXJwMTnexx3nFV/msizBGeMqZAItG3rHsOGld1+9Kibqsg/8fmW33zTJUd/jRoFT3y+R9NyB+YzJjyW4Iwx1Va/fklyCubQIcjOLpv8Nm+GJUtKD2QNbiL4UL0/jz/edZAxpiKW4IwxEZec7O7X69On7DZV18ElWOlv9WqYO7dkQGtwpcn27UN3gGnf3ga3No4lOGNMVIm4EluLFmVvfQAoKoLt24Pf/rBoEbzwQsmN7wANGpS+/SGwHbBlS2v/O1ZYgjPGxLR69dz4nB06wMiRZbcfOQLffBO8B+i//w27d5fePyUldPLr0gUaN66dz2UizxKcMSauNWzohirr3j349tzc4Mlv40Z3A/yhQ6X3b926dOI7/njXI7RTJ/dsHWDihyU4Y0ydlpoK/fu7RyBV2LUr9M3vr77qeoj6a9KkdMLzPfuW09KsE0yssARnjDlmiUCbNu4xdGjZ7YWFrv1v61ZXDRr4vHy5S5CBjjsueBL0Pbdta+OA1gZLcMYYE0JCgiuRpaUFv/8P3Agv27aVJD7/JPjVV24UmIMHSx/ToIFrUwws/fk/N2tmnWGqyxKcMcZUQ6NG5bcBqsLevaFLgR9+6BJkYFVoSkr5pcCOHd3wayY0S3DGGBNBIm5mhubNg7cDgqsK3bmzbAL0La9aVXY0GHAdYspLgu3aHdv3BFqCM8aYKEtIcDeot28PJ50UfJ+8PDflUbBS4Pr1sHCh6zEaeF7/qtBgz3V5WiRLcMYYEweSkuCEE9wjlH37gifArVvh449dr1D/SXHBjTJTUVVovE6NZAnOGGPqiKZNoV8/9wimqMhVdQZ2hvE9f/457NhR9riWLctPgO3bx+b0SDEYkjHGmEioV8+1y7VrB0OGBN/nyBFXFRosAW7aBP/9ryspBp63ffvyq0KjMUSaJThjjDHFGjZ0I7l07Rp6n/37g3eG2brV3Rv4+uulB8gG19s08JaIMWNg9OjIfRZLcMYYYyqlSRPo29c9gikqcjfAh7o14u233Q30+fmW4IwxxsSRevXcaC7HHQeZmcH3yc8vW8qraZbgjDHG1LrERPeIJBsNzRhjTJ0k6j9TYBwTkV3Alho4VStgd4V7xYZ4ihXiK954ihXiK954ihXiK954ihVqJt7jVbV1sA11JsHVFBFZrqohao1jSzzFCvEVbzzFCvEVbzzFCvEVbzzFCpGP16oojTHG1EmW4IwxxtRJluDKeiLaAVRCPMUK8RVvPMUK8RVvPMUK8RVvPMUKEY7X2uCMMcbUSVaCM8YYUydZgjPGGFMnHZMJTkT+LiLfi8gXIbaLiMwSkQ0islpEMmo7Rr9YKop1tIjsE5FV3uPe2o4xIJ6OIvK+iHwpImtE5FdB9omJ6xtmrDFzfUUkSUQ+EZHPvHh/E2SfhiLysndt/ycinWs/0rBjzRKRXX7X9pfRiNUvngQR+VRE3gyyLSaua0BM5cUba9c2W0Q+92JZHmR7ZL4TVPWYewCjgAzgixDbzwLmAwKcBPwvhmMdDbwZ7WvqF087IMNbTgW+BvrE4vUNM9aYub7e9UrxlhsA/wNOCtjnGuCv3vKFwMsxHGsW8Ei0r6tfPDcBLwX7e8fKda1EvLF2bbOBVuVsj8h3wjFZglPVxcCecnaZAPxDnY+BZiLSrnaiKy2MWGOKqm5X1ZXeci7wFdAhYLeYuL5hxhozvOt1wHvZwHsE9hKbADznLb8KnCZS27NwhR1rzBCRNE/96YgAACAASURBVOBs4KkQu8TEdfUJI954E5HvhGMywYWhA7DV7/U2YviLDxjmVQXNF5EQE1jUPq8aZyDu17u/mLu+5cQKMXR9vWqpVcD3wLuqGvLaqupRYB/QsnajdMKIFeA8r0rqVRHpWMsh+nsYuA0oCrE9Zq6rp6J4IXauLbgfN++IyAoRuSrI9oh8J1iCi38rcWOxDQD+AsyJcjwAiEgK8Bpwo6ruj3Y85akg1pi6vqpaqKrpQBowREROjGY85Qkj1jeAzqraH3iXkhJSrRKRccD3qroiGu9fWWHGGxPX1s/JqpoBnAlcKyKjauNNLcEF9y3g/4snzVsXc1R1v68qSFXnAQ1EpFU0YxKRBriE8aKq/jvILjFzfSuKNRavrxfLXuB9YGzApuJrKyL1gaZATu1GV1qoWFU1R1WPeC+fAgbVdmyeEcB4EckGZgOnisgLAfvE0nWtMN4Yura+eL71nr8HXgeGBOwSke8ES3DBzQUu9Xr2nATsU9Xt0Q4qGBFp62sLEJEhuL9p1L7QvFieBr5S1T+F2C0mrm84scbS9RWR1iLSzFtuBJwBrA3YbS5wmbc8EXhPvVb82hROrAFtLONxbaC1TlXvVNU0Ve2M60Dynqr+PGC3mLiuEF68sXJtvVgai0iqbxn4ERDYKzwi3wnH5ISnIvJPXO+4ViKyDZiGawRHVf8KzMP16tkAHAJ+EZ1Iw4p1IjBFRI4Ch4ELo/UfzzMCuAT43Gt/AbgL6AQxd33DiTWWrm874DkRScAl2ldU9U0RmQEsV9W5uIT9vIhswHVOujCGY71BRMYDR71Ys6IUa1Axel1DiuFrexzwuvc7sT7wkqr+R0QmQ2S/E2yoLmOMMXWSVVEaY4ypkyzBGWOMqZMswRljjKmTLMEZY4ypkyzBGWOMqZMswRkTZSJS6Dfq+yoRuaMGz91ZQsxEYUxdd0zeB2dMjDnsDWlljKlBVoIzJkZ5c2j9wZtH6xMR6eat7ywi73kD6S4UkU7e+uNE5HVvYOjPRGS4d6oEEXlS3Lxs73gjiyAiN4ibC2+1iMyO0sc0JmIswRkTfY0Cqigv8Nu2T1X7AY/gRpAHN+jzc95Aui8Cs7z1s4D/egNDZwBrvPXdgUdVtS+wFzjPW38HMNA7z+RIfThjosVGMjEmykTkgKqmBFmfDZyqqpu8QaF3qGpLEdkNtFPVAm/9dlVtJSK7gDS/QXZ90wC9q6rdvde3Aw1U9bci8h/gAG6GhDl+87cZUydYCc6Y2KYhlivjiN9yISVt72cDj+JKe8u8UfKNqTMswRkT2y7we/7IW15KyWC/FwNLvOWFwBQonmy0aaiTikg9oKOqvg/cjpv+pUwp0ph4Zr/YjIm+Rn6zGQD8R1V9two0F5HVuFLYJG/d9cAzInIrsIuSkdd/BTwhIlfgSmpTgFBTjiQAL3hJUIBZ3rxtxtQZ1gZnTIzy2uAyVXV3tGMxJh5ZFaUxxpg6yUpwxhhj6iQrwRljjKmTLMEZY4ypkyzBGWOMqZMswRljjKmTLMEZY4ypkyzBGWOMqZMswRljjKmTLMEZY4ypkyzBmZghIvNF5LKa3jeavElLT4/AedVvAtS/isg94exbhfe5WETeqWqcxkSTjWRiqkVE/OcQS8YNClzovb5aVV+s/ahihzee5C9VdUENn1eB7qq6oab29eaO24ybL+5oTcRpTDTZbAKmWvwn6izvy1xE6tuXpokV9u/x2GBVlCYiRGS0iGwTkdtFZAduepfmIvKmiOwSkR+85TS/YxaJyC+95SwR+UBEZnr7bhaRM6u4bxcRWSwiuSKyQEQeFZEXQsQdToz3iciH3vneEZFWftsvEZEtIpIjIr8u5/oMFZEdIpLgt+4cb2ocRGSIiHwkIntFZLuIPCIiiSHO9ayI/Nbv9a3eMd+JyOUB+54tIp+KyH4R2Soi0/02L/ae94rIAREZ5ru2fscPF5FlIrLPex4e7rWp5HVuISLPeJ/hBxGZ47dtgois8j7DRhEZ660vVR0sItN9f2cR6exV1V4hIt8A73nr/+X9HfZ5/0b6+h3fSET+6P0993n/xhqJyFsicn3A51ktIucE+6wmeizBmUhqC7QAjgeuwv17e8Z73Qk4DDxSzvFDgXVAK+APwNMiIlXY9yXgE6AlMB24pJz3DCfGi3BzsLUBEoFbAESkD/C4d/723vulEYSq/g84CJwacN6XvOVCYKr3eYYBpwHXlBM3XgxjvXjOALoDge1/B4FLgWa4Gb2niMhPvW2jvOdmqpqiqh/5HygiLYC3gFneZ/sT8JaItAz4DGWuTRAVXefncVXefb1zPeTFMAT4B3Cr9xlGAdmhrkcQpwC9gR97r+fjrlMbYCXgX6U+ExgEDMf9O74NKAKeA37u20lEBgAdcNfGxBJVtYc9auSB+6I53VseDeQDSeXsnw784Pd6Ea6KEyAL2OC3LRlQoG1l9sV9eR4Fkv22vwC8EOZnChbj3X6vr8FNUApwLzDbb1tj7xqcHuLcvwX+7i2n4pLP8SH2vRF43e+1At285WeB33rLfwce8Nuvh/++Qc77MPCQt9zZ27e+3/Ys4ANv+RLgk4DjPwKyKro2lbnOQDtcImkeZL+/+eIt79+f93q67+/s99m6lhNDM2+fprgEfBgYEGS/JOAHXLsmuET4WG3/f7NHxQ8rwZlI2qWqeb4XIpIsIn/zqnz246rEmvlX0wXY4VtQ1UPeYkol920P7PFbB7A1VMBhxrjDb/mQX0zt/c+tqgeBnFDvhSutnSsiDYFzgZWqusWLo4dXbbfDi+P/cKW5ipSKAdgS8PmGisj7XtXgPmBymOf1nXtLwLotuNKLT6hrU0oF17kj7m/2Q5BDOwIbw4w3mOJrIyIJIvKAV825n5KSYCvvkRTsvbx/0y8DPxeReriZ1p+vRkwmQizBmUgK7KJ7M9ATGKqqTSipEgtV7VgTtgMtRCTZb13HcvavTozb/c/tvWfLUDur6pe4BHEmpasnwVV1rsWVEpoAd1UlBlwJ1t9LwFygo6o2Bf7qd96KulR/h6tS9NcJ+DaMuAKVd5234v5mzYIctxU4IcQ5D+JK7z5tg+zj/xkvAibgqnGb4kp5vhh2A3nlvNdzwMW4quNDGlCda2KDJThTm1Jx1T57vfacaZF+Q69EtByYLiKJIjIM+EmEYnwVGCciJ3sdQmZQ8f+xl4Bf4b7g/xUQx37ggIj0AqaEGcMrQJaI9PESbGD8qbjSUZ7XnnWR37ZduKrBriHOPQ/oISIXiUh9EbkA6AO8GWZsgXEEvc6quh3XNvaY1xmlgYj4EuDTwC9E5DQRqSciHbzrA7AKuNDbPxOYGEYMR3Cl7GRcKdkXQxGuuvdPItLeK+0N80rbeAmtCPgjVnqLWZbgTG16GGiE+3X8MfCfWnrfi3EdNXJw7V4v477YgqlyjKq6BrgWl7S249pptlVw2D9xHR/eU9XdfutvwSWfXOBJL+ZwYpjvfYb3gA3es79rgBkikotrM3zF79hDwP3Ah+J6b54UcO4cYByu9JWD63QxLiDucFV0nS8BCnCl2O9xbZCo6ie4TiwPAfuA/1JSqrwHV+L6AfgNpUvEwfwDV4L+FvjSi8PfLcDnwDJgD/B7Sn9n/gPoh2vTNTHIbvQ2xxwReRlYq6oRL0GauktELgWuUtWTox2LCc5KcKbOE5HBInKCV6U1FtfuMqei44wJxav+vQZ4ItqxmNAswZljQVtcF/YDuHu4pqjqp1GNyMQtEfkxrr1yJxVXg5oosipKY4wxdZKV4IwxxtRJdWaw5VatWmnnzp2jHYYxxphatGLFit2q2jrYtjqT4Dp37szy5cujHYYxxphaJCKBo+sUsypKY4wxdVKdKcEZY4yJTapw4ADs2AE7d5Y89+sHo0ZVfHxVWYIzxhhTJQcPliQr/8QV7Pnw4bLH33CDJThjjDG15PDhipOV7/ngwbLHi0CrVnDccdC2LZxwgnv2vfZ/bhXuPBZVZAnOGGPquCNHwk9aubnBz9GyZUlyGjIkdNJq3Rrqx0hmiZEwjDHGVEZ+Pnz/fXhJa9++4Odo3rwkOWVkhE5abdpAgwa1+/lqgiU4Y4yJEQUFsGtXeEnrh2DTwQJNm5Ykp/794Uc/Kp2sfMtt2kDDhrX7+WpbRBOcN7Dtn4EE4ClVfSBg+/G4OZda46aj+LmqbvO2XQbc7e36W1V9LpKxGmNMJBw9Crt3V5y0du50+wWTmlqSmPr0gTFjQpe2kpJq9/PFsoglOG/q+UeBM3BzYi0TkbneLMY+M4F/qOpzInIq8DvgEr8JEDNxM/Cu8I4N8ZvFGGNqT2Eh5OSUn6x8y7t2uW7ygZKTXVJq2xZ69ICRI0MnreTkssebikWyBDcE2KCqmwBEZDZumhL/BNcHuMlbfp+SKUx+DLyrqnu8Y98FxuImhzTGmIgoKnJJaetW9/jmG5eoAhPY99+7fQMlJZUkpa5dYdiw0EkrJaX2P9+xJpIJrgOw1e/1NmBowD6fAefiqjHPAVJFpGWIYztELlRjTF2n6jpb+BKX/7Nveds21w7mLzGxJCl16gSDB4dOWqmprpu8iQ3R7mRyC/CIiGQBi3FTxxeGe7CIXAVcBdCpU6dIxGeMiRN5eS5BBSYv/yQW2AU+IQHS0qBjR1fa6tjRPTp1Kllu0cKSVryKZIL7Fujo9zrNW1dMVb/DleAQkRTgPFXdKyLfAqMDjl0U+Aaq+gTejLqZmZk2sZ0xdVRhIWzfXrq0Ffi8a1fZ49q0cUmqZ084/fSSxOV7btvWJTlTN0UywS0DuotIF1xiuxC4yH8HEWkF7FHVIuBOXI9KgLeB/xOR5t7rH3nbjTF1jCrs2RO8ytD3/N13rjeiv9TUkkSVkVE2eaWlWY/CY13EEpyqHhWR63DJKgH4u6quEZEZwHJVnYsrpf1ORBRXRXmtd+weEbkPlyQBZvg6nBhj4suhQ6FLXb5kduhQ6WMaNCipIjzllLJVh506ufu9jCmPaLD+q3EoMzNTbT44Y2pXQYErXZVXdbgn4KepiKsaDExY/kmsTRuoZ5N5mTCIyApVzQy2LdqdTIwxMUrVtWuV1+tw+/ay3eWbNStJVCedVDaJdejgeiYaE2mW4Iw5RuXmhm7z2rrV9UjMyyt9TMOGJYkqWKeNjh1d25gxscASnDF1UH6+S1DlddwIHIC3Xj1o394lqUGD4Kc/LVuN2KqVdZk38cMSnDFxSNUlqq++grVrITu7dBLbubPs8FAtW7pE1aWLm2QysN2rffvYmebEmJpg/5yNiWEFBbBxo0tk/o+1a0tPNpmcXJKozjoreNWhjWdojjWW4IyJAQcPuqQVmMg2bCh9/1daGvTuDVdcAb16ueXevV2vQ6s6NKa0ChOciPwEeMu7GdsYUw27dpUuhfmWv/mmZJ+EBOjWzSWwn/60JIn16mUdOIypjHBKcBcAD4vIa7ibtddGOCZj4lpRUUn7WOAjJ6dkv0aNXNI6+eSSJNa7t0tu1o3emOqrMMGp6s9FpAkwCXjWG3XkGeCfqppb/tHG1F35+a4KMTCJrVtXemSOli1d4jr33JKSWO/ern3MbmY2JnLCaoNT1f0i8irQCLgRN7XNrSIyS1X/EskAjYm23NyS6kT/asUNG9wgwD6dOrnkNWpU6RJZ69bRi92YY1k4bXDjgV8A3YB/AENU9XsRScZNXmoJzsQ936gdwaoVt20r2a9+fVeF2KcPnHdeSRLr2dMmsDQm1oRTgjsPeEhVF/uvVNVDInJFZMIyJjKKimDLluDd7v3HTGzc2JXGRo8uXa3YrZsbCNgYE/vCSXDTge2+FyLSCDhOVbNVdWGkAjOmOo4cgfXry1YrrlsHhw+X7Ne6tUtcEyeWrlZMS7P2MVOHFRW5+08KC0ueK7tcE/tmZLjpIiIknAT3L2C43+tCb93giERkTCXs3x/8/rFNm0q3jx1/vEtcY8aUTmQtW0YvdlMJqu5RWOi+nP2fw11X2f39l2vyy76mk0RV9o0VN98c9QRXX1XzfS9UNV9ErBOzqTWqbuipYO1j331Xsl+DBtC9O/TvDxdcUFK12LOnq3KMOapuqJL8/JJH4OtwH0eP1swXeTSSR7jb4pGIu7Gxfn337L8cbF15yw0auBlcK3NcpPatqfeI8Iy04SS4XSIy3pugFBGZAOyOaFTmmFRY6MZUDKxW/Oor2Lu3ZL+UFJe8TjutdGmsa1doUF/dl73/l/+efNhRhaRR3aQTznlrS7167uH7cvEtBz6Huy7YNv8vr6qeoyb3r6n3rOoXuO94EzXhJLjJwIsi8gggwFbg0ohGZeoGVfj+e9effuNGOHAA8vMpOJhPzo58crbn88OOfPbtyic3J5+De/NJKMwnkXw6k0+/xHyaN86naXI+qc3zaZyYT6OEfBpQgPyQD//Nh3eDJI5IEHFzxTRo4O7CLu+RlARNmlS8X+AjnHOHOs73xRrsS1vExvEyx6RwbvTeCJwkIine6wMRj8rEj6IiN+vlhg3BHwfK/nNpALQF2iDkk0iBJFJUPxGSEqmXlEj9Rok0SHHPJV/kSZAYZtKoaqIo75GQUOuX1hhTPWHd6C0iZwN9gSTxfgmq6owIxmViSVGRuxksVBLz75ZYv76rK+zWjaKRo1j6fTcemd+NT/efQJuezenaK5ETeifSo28DevVNoEdPIdVGuTfGREA4N3r/FUgGxgBPAROBTyIcl6ltR4+6ARR9SWv9+pLlTZtcv3ufxEQ44QR3U9gZZ7hn36NjR6hfn4ULYepU+PxzGDkSXnrITaJpjDG1JZwS3HBV7S8iq1X1NyLyR2B+pAMzEVBQ4HpxBCuFbd5cutNDo0YlQ9qPG1c6iXXoELLKbv16uOUWmDsXOneGf/3LjfhhTUDGmNoWToLL854PiUh7IAdoF87JRWQs8GcgAXhKVR8I2N4JeA5o5u1zh6rOE5HOwFfAOm/Xj1V1cjjvecw7csQlq2BJLDu79D0wKSkuYfXv70YC9iWw7t2hXbtKZaUffoD77oNHHnF9LB54AH71q4j3AjbGmJDCSXBviEgz4EFgJaDAkxUdJCIJwKPAGcA2YJmIzFXVL/12uxt4RVUfF5E+wDygs7dto6qmh/1JjiWHD7tqQ//k5atS/OYb13vRp2lTl7QGD4ZJk0qXxGpglsyjR+GJJ+Dee91QV1dc4RJd27bV/IzGGFNN5SY4EakHLFTVvcBrIvImkKSq+8I49xBgg6pu8s41G5iAG6DZR4Em3nJT4DuMc/Cg61rv3xbme/iP/gtuOI5u3dzEYv4JrFs3ty1C9YNvvw033QRffunGbHzoIUi3nyTGmBhRboJT1SIReRQY6L0+Ahwp7xg/HXD3zPlsA4YG7DMdeEdErgcaA6f7besiIp8C+4G7VXVJ4BuIyFXAVQCdOnUKM6wYsn9/6J6J27eX3rdNG5ewTj21bBJr3rxWw1671o2wM2+e62vy+uswYYK1sxljYks4VZQLReQ84N+q/nVfNWIS8Kyq/lFEhgHPi8iJuMGdO6lqjogMAuaISF9V3e9/sKo+ATwBkJmZWdOx1YwffgiewNavd/Oz+GvXziWsM88sncBOOMHdOBxlOTnwm9/AY4+5oa8efBCuv97d/2yMMbEmnAR3NXATcFRE8nCjmaiqVvSN+y3Q0e91mrfO3xXAWNwJPxKRJKCVqn6PV1JU1RUishHoASwPI97apeq++QO71vse/nOwgOtG360b/PSnZZNYTA6Y6DpXPv44TJ8O+/bBVVe5RNemTbQjM8aY0MIZySS1iudeBnQXkS64xHYhcFHAPt8ApwHPikhvIAk39mVrYI+qFopIV6A7sKmKcVSfb7TfUNWJ+/yaJOvVc1M7d+sGP/tZ6STWtavrfh8nVF015M03u2lmTj8d/vQn6Ncv2pEZY0zFwrnRe1Sw9YEToAbZflRErgPext0C8HdVXSMiM4Dl3uDNNwNPishUXIeTLFVV7z1niEgBUARMVtU9Id6qZqi6oenDGXIqIcHd5NWtGwwbVjqJde5cJ+rs1qxxHUjeeQd69IA33oCzz7Z2NmNM/JCKmtVE5A2/l0m43pErVPXUSAZWWZmZmbp8eTVqMHNzS7dzNWhQPORUmcfxx9fZaZ1374Zp0+Bvf4PUVLd8zTVu8BJjjIk1IrJCVTODbQunivInASfrCDxcQ7HFjtRUeOqpkurFTp2OqQF28/PdTdozZrjC6pQprs3NJgQ1xsSrsAZbDrAN6F3TgcSEK66IdgS1TtVVP958s6uJHTsW/vhH6NMn2pEZY0z1hNMG9xdc+xhAPSAdN6KJiXOrV7t2toUL3ZCT8+a5OxSMMaYuCKcE59+wdRT4p6p+GKF4TC34/nu45x5XI9usGfzlL3D11XW2WdEYc4wKJ8G9CuSpaiG4MSZFJFlVD0U2NFPTjhyBWbPcWJGHD7ubtO+9F1q0iHZkxkRXQUEB27ZtIy8vr+KdTVQkJSWRlpZGg0r8Eg9rJBPcEFq+fvKNgHeA4ZWO0ESFqhtO69Zb3RjN48bBzJnQs2e0IzMmNmzbto3U1FQ6d+6M2L0wMUdVycnJYdu2bXTp0iXs4+qFsU+SqhbfBOYt2xzMceLTT2HMGDcnW6NGboDkN96w5GaMv7y8PFq2bGnJLUaJCC1btqx0CTucBHdQRDL83mgQcLiS8ZlatmOH6xQ6aJC7afvxx2HVKvjRj6IdmTGxyZJbbKvK3yecBHcj8C8RWSIiHwAvA9dV+p1MrcjLg9/9zs1Z+vzzrpfk+vUweTLUr8pNIcaYiMvJySE9PZ309HTatm1Lhw4dil/n5+eXe+zy5cu54YYbKnyP4cOPvValcG70XiYivQBfpdY6VS2IbFimslTh1VddO9uWLW4s5z/8wSU6Y0xsa9myJatWrQJg+vTppKSkcMsttxRvP3r0KPVD/ELNzMwkMzPoQB6lLF26tGaCjSMVluBE5Fqgsap+oapfACkick3kQzPhWr4cRo1yYzs3berua3v9dUtuxsSzrKwsJk+ezNChQ7ntttv45JNPGDZsGAMHDmT48OGsW7cOgEWLFjFu3DjAJcfLL7+c0aNH07VrV2bNmlV8vpSUlOL9R48ezcSJE+nVqxcXX3wxviEb582bR69evRg0aBA33HBD8Xn9ZWdnM3LkSDIyMsjIyCiVOH//+9/Tr18/BgwYwB133AHAhg0bOP300xkwYAAZGRls3LgxMhcsiHAqra5U1Ud9L1T1BxG5EngscmGZcHz7Ldx1F/zjH27qmiefhF/84pgaYcyYGnfjja69uialp8PDVRjgcNu2bSxdupSEhAT279/PkiVLqF+/PgsWLOCuu+7itddeK3PM2rVref/998nNzaVnz55MmTKlTNf6Tz/9lDVr1tC+fXtGjBjBhx9+SGZmJldffTWLFy+mS5cuTJo0KWhMbdq04d133yUpKYn169czadIkli9fzvz58/l//+//8b///Y/k5GT2eFOFXXzxxdxxxx2cc8455OXlUVRUVPkLUUXhJLgEERHfZKcikgDY0LtRdOiQG07rgQfg6FG4/XaX6GJgTlRjTA06//zzSfB+se7bt4/LLruM9evXIyIUFARvKTr77LNp2LAhDRs2pE2bNuzcuZO0tLRS+wwZMqR4XXp6OtnZ2aSkpNC1a9fibviTJk3iiSeeKHP+goICrrvuOlatWkVCQgJff/01AAsWLOAXv/gFycmuk32LFi3Izc3l22+/5ZxzzgHcvWy1KZwE9x/gZRH5m/f6amB+5EIyoajC7NkuoW3d6rr+/+EPbtIDY0zNqEpJK1Ia+02CfM899zBmzBhef/11srOzGT16dNBjGvpN15WQkMDRo0ertE8oDz30EMcddxyfffYZRUVFtZ60KiOcXpS3A+8Bk73H57ibvU0t+vhjGD4cLroIWrWCRYtcpxJLbsYcG/bt20eHDh0AePbZZ2v8/D179mTTpk1kZ2cD8PLLL4eMo127dtSrV4/nn3+ewsJCAM444wyeeeYZDh1yg1zt2bOH1NRU0tLSmDNnDgBHjhwp3l4bKkxwqloE/A/Ixs0FdyrwVWTDMj5bt8LPf+7mVc3Ohr//HZYtg1NOiXZkxpjadNttt3HnnXcycODASpW4wtWoUSMee+wxxo4dy6BBg0hNTaVp06Zl9rvmmmt47rnnGDBgAGvXri0uZY4dO5bx48eTmZlJeno6M2fOBOD5559n1qxZ9O/fn+HDh7Njx44ajz2UkBOeikgPYJL32I27/+0WVT2+1qKrhGpPeBpjDh501Y8PPghFRXDLLa5qMjU12pEZU/d89dVX9O5dN2cBq4wDBw6QkpKCqnLttdfSvXt3pk6dGu2wigX7O5U34Wl5Jbi1uNLaOFU9WVX/AhTWWKQmqKIid4N2jx5u8tHx42HtWvjtby25GWMi68knnyQ9PZ2+ffuyb98+rr766miHVC3ldTI5F7gQeF9E/gPMBmwsmwhautR1UV62DAYPhldegREjoh2VMeZYMXXq1JgqsVVXyBKcqs5R1QuBXsD7uCG72ojI4yJiIxrWoC1b4MILXTL79lt3X9vHH1tyM8aY6gink8lBVX1JVX8CpAGf4npWmmo6cADuvtuN7D93rpub7euv4ZJLoF44/VuNMcaEVKmvUVX9QVWfUNXTwtlfRMaKyDoR2SAidwTZ3klE3heRT0VktYic5bftTu+4dSLy48rEGeuKiuCZZ9xQWvffDxMnwrp18JvfgN9tL8YYY6ohYuPLeyOePAqcAWwDlonIXFX90m+3u4FXVPVxEekDzAM6e8sXAn2B9sACEenhm1U8ni1eDFOnwsqVMHSoGzPypJOiHZUxxtQ9kawIGwJsUNVNqpqP66QyIWAfBXwDTDUFvvOWJwCzVfWIqm4GNnjni1ubNrmS2imnwK5d8OKL8NFHltyMMTBmzBjefvvtUusefvhhpkyZEvKY0aNHpcI7nQAADGhJREFU47s16qyzzmLv3r1l9pk+fXrx/WihzJkzhy+/LCl33HvvvSxYsKAy4cesSCa4DsBWv9fbvHX+pgM/F5FtuNLb9ZU4FhG5SkSWi8jyXbt21VTcNWr/frjjDujdG+bPd13/1651I5LY/IrGGHDjPs6ePbvUutmzZ4cc8DjQvHnzaNasWZXeOzDBzZgxg9NPP71K54o10e7KMAl4VlXTgLOA50Uk7Ji89sBMVc1s3bp1xIKsisJCN7p/9+7w+9+7XpJffw333APeWKTGGAPAxIkTeeutt4onN83Ozua7775j5MiRTJkyhczMTPr27cu0adOCHt+5c2d2794NwP3330+PHj04+eSTi6fUAXeP2+DBgxkwYADnnXcehw4dYunSpcydO5dbb72V9PR0Nm7cSFZWFq+++ioACxcuZODAgfTr14/LL7+cI0eOFL/ftGnTyMjIoF+/fqxdu7ZMTLEwrU4k53j+Fujo9zrNW+fvCmAsgKp+JCJJQKswj41Z77/v7mdbvdp19X/zTXdfmzEmDkRhvpwWLVowZMgQ5s+fz4QJE5g9ezY/+9nPEBHuv/9+WrRoQWFhIaeddhqrV6+mf//+Qc+zYsUKZs+ezapVqzh69CgZGRkMGjQIgHPPPZcrr7wSgLvvvpunn36a66+/nvHjxzNu3DgmTpxY6lx5eXlkZWWxcOFCevTowaWXXsrjjz/OjTfeCECrVq1YuXIljz32GDNnzuSpp54qdXwsTKsTyRLcMqC7iHQRkURcp5G5Aft8A5wGICK9gSRgl7ffhSLSUES6AN2BTyIYa43YsAHOOQdOPRX27YOXX4YlSyy5GWMq5l9N6V89+corr5CRkcHAgQNZs2ZNqerEQEuWLOGcc84hOTmZJk2aMH78+OJtX3zxBSNHjqRfv368+OKLrFmzptx41q1bR5cuXejRowcAl112GYsXLy7efu655wIwaNCg4gGa/RUUFHDllVfSr18/zj///OK4w51WJ7kGqroiVoJT1aMich3wNpAA/F1V14jIDGC5qs4FbgaeFJGpuA4nWd68c2tE5BXgS+AocG0s96Dcu9cNpTVrFjRsCP/3f+5HYCObc8GY+BOl+XImTJjA1KlTWblyJYcOHWLQoEFs3ryZmTNnsmzZMpo3b05WVhZ5eXlVOn9WVhZz5sxhwIABPPvssyxatKha8fqm3Ak13U4sTKsT0TY4VZ2nqj1U9QRVvd9bd6+X3FDVL1V1hKoOUNV0VX3H79j7veN6qmpMzj939Cj89a+une1Pf3I3aH/9Ndx5pyU3Y0zlpKSkMGbMGC6//PLi0tv+/ftp3LgxTZs2ZefOncyfX/5X4ahRo5gzZw6HDx8mNzeXN954o3hbbm4u7dq1o6CggBdffLF4fWpqKrm5uWXO1bNnT7Kzs9mwYQPgZgU4pRLTmMTCtDrR7mQSt95911WrT5kCffrA8uXw9NPQrl20IzPGxKtJkybx2WefFSe4AQMGMHDgQHr16sVFF13EiArG78vIyOCCCy5gwIABnHnmmQz2ax+57777GDp0KCNGjPj/7d1/rFd1Hcfx5wu867Jw2qQRejXYcmMhENQYZXPOVotsuKHM2/pFi7VLDW1tpfWHWOuv0NZINzWLUWrqLJSYMlEYtVUYEqJANtZooTTgphjLOaB3f5zPhS/X7/fyve1+v+dzzn09tjvOPefw5XXfd5z39/z4fj7MnDnz9Pr+/n5Wr17NvHnzznqwo7e3l7Vr17J06VJmz57NhAkTGBgYaPtnyWFanZbT5VRNt6bLefnlYuqajRthxgy4447ivpsf+TerLk+XUw1jOV2ONXjttWIEkiuugG3bikf/9+2DJUvc3MzMctTJjwnUwokTcO+9sGpV8TDJ8uXFh7WnTi07mZmZjcRncCPYtAnmzoWVK4v7bTt3Fs3Ozc3MLH9ucE3s3QuLFhVfJ07AE0/AM88Uzc7M6qkuzyPU1f/z+3GDazA4WJytzZlTDIR8552wZw8sXuz7bGZ11tvby+DgoJtcpiKCwcHBUX+WzvfgkldfhVmzisGRBwbg9tshs+EtzaxD+vr6OHjwILkO2m7Fm5C+vr5R/R03uOTii4unJJcsKZ6UNLPxo6enhxkzZpQdw8aYG1yD224rO4GZmY0V34MzM7NacoMzM7Naqs1QXZKOAH8fg5eaAhwdg9fphiplhWrlrVJWqFbeKmWFauWtUlYYm7zvjYimjwTWpsGNFUk7Wo1rlpsqZYVq5a1SVqhW3iplhWrlrVJW6HxeX6I0M7NacoMzM7NacoN7u/vKDjAKVcoK1cpbpaxQrbxVygrVylulrNDhvL4HZ2ZmteQzODMzq6Vx2eAk/UzSYUkvtdguSWsk7Ze0W9L8bmdsyHKurFdLOiZpV/oqdTwWSZdK2ippr6Q9km5usk8W9W0zazb1ldQr6TlJL6S8322yzzskPZJqu13S9O4nbTvrMklHGmq7vIysDXkmSvqzpI1NtmVR12GZRsqbW20PSHoxZdnRZHtnjgkRMe6+gKuA+cBLLbZ/CngKELAQ2J5x1quBjWXXtCHPNGB+Wj4f+Cvw/hzr22bWbOqb6jU5LfcA24GFw/b5KnBPWu4HHsk46zLgrrLr2pDnG8BDzX7fudR1FHlzq+0BYMoI2ztyTBiXZ3AR8VvgXyPsch3w8yj8EbhQ0rTupDtbG1mzEhGHImJnWv43sA+4ZNhuWdS3zazZSPU6nr7tSV/Db6JfB6xLy48BH5O6P9lTm1mzIakPuBa4v8UuWdR1SBt5q6Yjx4Rx2eDacAnwj4bvD5LxgQ/4cLoU9JSkWWWHGZIu48yjePfeKLv6jpAVMqpvuiy1CzgMbI6IlrWNiJPAMeCi7qYstJEV4Pp0SeoxSZd2OWKjHwHfAv7bYns2dU3OlRfyqS0Ub26elvS8pK802d6RY4IbXPXtpBiqZi7wY+DxkvMAIGky8Cvg6xHxRtl5RnKOrFnVNyJORcQHgD5ggaRsJ3dqI+tvgOkRMQfYzJkzpK6S9GngcEQ8X8a/P1pt5s2itg0+GhHzgUXA1yRd1Y1/1A2uuVeAxnc8fWlddiLijaFLQRHxJNAjaUqZmST1UDSMByPi1012yaa+58qaY31TlteBrcAnh206XVtJ5wEXAIPdTXe2VlkjYjAi3krf3g98sNvZkiuBxZIOAA8D10h6YNg+OdX1nHkzqu1QnlfSn4eB9cCCYbt05JjgBtfcBuAL6cmehcCxiDhUdqhmJL1n6F6ApAUUv9PSDmgpy0+BfRHxwxa7ZVHfdrLmVF9J75Z0YVqeBHwc+Muw3TYAX0zLNwBbIt3F76Z2sg67x7KY4h5o10XEtyOiLyKmUzxAsiUiPjdstyzqCu3lzaW2Kcs7JZ0/tAx8Ahj+VHhHjgnjcsJTSb+keDpuiqSDwCqKm+BExD3AkxRP9ewH/gN8qZykbWW9AVgh6STwJtBf1n+85Erg88CL6f4LwHeAyyC7+raTNaf6TgPWSZpI0WgfjYiNkr4H7IiIDRQN+xeS9lM8nNSfcdabJC0GTqasy0rK2lSmdW0p49pOBdan94nnAQ9FxCZJA9DZY4JHMjEzs1ryJUozM6slNzgzM6slNzgzM6slNzgzM6slNzgzM6slNzizkkk61TDq+y5Jt47ha09Xi5kozOpuXH4Oziwzb6YhrcxsDPkMzixTaQ6tH6R5tJ6T9L60frqkLWkg3WclXZbWT5W0Pg0M/YKkj6SXmijpJyrmZXs6jSyCpJtUzIW3W9LDJf2YZh3jBmdWvknDLlHe2LDtWETMBu6iGEEeikGf16WBdB8E1qT1a4BtaWDo+cCetP5y4O6ImAW8Dlyf1t8KzEuvM9CpH86sLB7JxKxkko5HxOQm6w8A10TE39Kg0P+MiIskHQWmRcSJtP5QREyRdAToaxhkd2gaoM0RcXn6/hagJyK+L2kTcJxihoTHG+ZvM6sFn8GZ5S1aLI/GWw3Lpzhz7/1a4G6Ks70/pVHyzWrDDc4sbzc2/PmHtPx7zgz2+1ngd2n5WWAFnJ5s9IJWLyppAnBpRGwFbqGY/uVtZ5FmVeZ3bGblm9QwmwHApogY+qjAuyTtpjgL+0xatxJYK+mbwBHOjLx+M3CfpC9TnKmtAFpNOTIReCA1QQFr0rxtZrXhe3BmmUr34D4UEUfLzmJWRb5EaWZmteQzODMzqyWfwZmZWS25wZmZWS25wZmZWS25wZmZWS25wZmZWS25wZmZWS39Dz4mTJY74/2zAAAAAElFTkSuQmCC","text/plain":["<Figure size 432x288 with 2 Axes>"]},"metadata":{"needs_background":"light"},"output_type":"display_data"}],"source":["# Subplot들의 간격 조정하기 참고: https://tempdev.tistory.com/35\n","\n","# plt.subplots\n","fig, axs = plt.subplots(2 ,1, constrained_layout=True)\n","\n","# Training and Validation Loss 그래프\n","axs[0].plot(epochs, loss, 'b', label='Training loss')\n","axs[0].plot(epochs, val_loss, 'r', label='Validation loss')\n","axs[0].set_title('Training and validation loss')\n","axs[0].set_xlabel('Epochs')\n","axs[0].set_ylabel('Loss')\n","axs[0].legend()\n","\n","# Training and Validation Accuracy 그래프\n","axs[1].plot(epochs, acc, 'b', label='Training acc')\n","axs[1].plot(epochs, val_acc, 'r', label='Validation acc')\n","axs[1].set_title('Training and validation accuracy')\n","axs[1].set_xlabel('Epochs')\n","axs[1].set_ylabel('Accuracy')\n","axs[1].legend()\n","\n","\n","plt.show()"]},{"cell_type":"code","execution_count":72,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":303,"status":"ok","timestamp":1643001418992,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"ueZzDGqwg-uN","outputId":"07888789-0cdc-4109-8151-728c3b34fba9"},"outputs":[{"data":{"text/plain":["[('스토리', 0.4149767756462097),\n"," ('긴장감', 0.4109301269054413),\n"," ('완', 0.4071023464202881),\n"," ('감동', 0.4030129611492157),\n"," ('예뻐서', 0.4027698040008545),\n"," ('하늘', 0.3982482850551605),\n"," ('어울리', 0.39427632093429565),\n"," ('진짜', 0.39338165521621704),\n"," ('는데', 0.39296314120292664),\n"," ('깊이', 0.3925923705101013)]"]},"execution_count":72,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"재미\")"]},{"cell_type":"code","execution_count":73,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":314,"status":"ok","timestamp":1643001448948,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"6l0B2JPQhHOX","outputId":"c708ae0e-0008-4295-906a-b2881e58d135"},"outputs":[{"data":{"text/plain":["[('누군가', 0.36325740814208984),\n"," ('당대', 0.3536795377731323),\n"," ('수작', 0.33141815662384033),\n"," ('아날로그', 0.314446359872818),\n"," ('바치', 0.31333059072494507),\n"," ('곡', 0.31100383400917053),\n"," ('의외로', 0.31060802936553955),\n"," ('공권력', 0.30855754017829895),\n"," ('감수성', 0.30527111887931824),\n"," ('잼따', 0.2995571196079254)]"]},"execution_count":73,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"최고\")"]},{"cell_type":"code","execution_count":74,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":299,"status":"ok","timestamp":1643001459223,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"x7cYeEtShJY0","outputId":"cff36180-367e-4b86-cf30-89e712c49a40"},"outputs":[{"data":{"text/plain":["[('어설픈', 0.5763288736343384),\n"," ('진부', 0.5739359855651855),\n"," ('-_', 0.5682641863822937),\n"," ('표절', 0.5643913149833679),\n"," ('짱나', 0.5627702474594116),\n"," ('그지', 0.5605618953704834),\n"," ('어휴', 0.5605323314666748),\n"," ('뻔한', 0.5575464963912964),\n"," ('별루', 0.5521450042724609),\n"," ('난해', 0.5379496812820435)]"]},"execution_count":74,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"별로\")"]},{"cell_type":"code","execution_count":75,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":320,"status":"ok","timestamp":1643001469587,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"ADceTYsBhLwX","outputId":"ddbf1018-33d6-46b5-f354-71a22e72b60d"},"outputs":[{"data":{"text/plain":["[('감회', 0.45710986852645874),\n"," ('정화', 0.45638078451156616),\n"," ('는다는', 0.4455082416534424),\n"," ('이뻐', 0.4291670620441437),\n"," ('연락', 0.4234762489795685),\n"," ('최고봉', 0.4205551743507385),\n"," ('발견', 0.4170006513595581),\n"," ('다가왔', 0.41380515694618225),\n"," ('감탄', 0.40907543897628784),\n"," ('힐링', 0.4064022898674011)]"]},"execution_count":75,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"엉엉\")"]},{"cell_type":"code","execution_count":76,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":293,"status":"ok","timestamp":1643001478419,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"xNN5VErwhOFz","outputId":"975a4d25-9f83-4c95-8d64-0ded8df75ad3"},"outputs":[{"data":{"text/plain":["[('나루토', 0.46137771010398865),\n"," ('게', 0.4294614791870117),\n"," ('롭', 0.42611876130104065),\n"," ('바랍니다', 0.41053926944732666),\n"," ('이쁜', 0.409149706363678),\n"," ('음', 0.40361204743385315),\n"," ('뮤', 0.3967868983745575),\n"," ('로맨스', 0.39550620317459106),\n"," ('시', 0.3941178321838379),\n"," ('미셸', 0.39407801628112793)]"]},"execution_count":76,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"매력\")"]},{"cell_type":"code","execution_count":77,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":305,"status":"ok","timestamp":1643001487687,"user":{"displayName":"박혜령","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GisbEQvBKpkmHTFCo4Ec0oObN6XH6kJNzcctpw0-Q=s64","userId":"14403397615004068576"},"user_tz":-540},"id":"djXQ8KAdhQeB","outputId":"4309fcab-abc0-496c-c862-2da040c53384"},"outputs":[{"data":{"text/plain":["[('에서', 0.5754741430282593),\n"," ('는데', 0.5681638121604919),\n"," ('하', 0.547177791595459),\n"," ('봤', 0.5454105138778687),\n"," (',', 0.5425621271133423),\n"," ('롭', 0.5361525416374207),\n"," ('숨겨진', 0.5272085666656494),\n"," ('기억', 0.5176573991775513),\n"," ('너무', 0.5171259641647339),\n"," ('영화', 0.516504168510437)]"]},"execution_count":77,"metadata":{},"output_type":"execute_result"}],"source":["word_vectors.similar_by_word(\"반전\")"]},{"cell_type":"markdown","metadata":{"id":"Be8N7br3d6UI"},"source":["- 비교\n","\n","|Num|Model|Loss|Accuracy|\n","|:---:|:---:|:---:|:---:|\n","|1|LSTM|0.3667|0.8479|\n","|2|ko.bin 임베딩을 활용한 LSTM|0.3288|0.8648|\n","\n","- 결론\n","  - 앞서 진행한 LSTM 모델과 카피한 임베딩을 활용한 LSTM 모델의 loss 와 accuracy 를 비교하면 후자가 더 성능이 좋다.\n","  - 유사 단어 찾기는 두 모델 모두 비슷하게 잘 찾아냈다."]},{"cell_type":"markdown","metadata":{},"source":["---"]},{"cell_type":"markdown","metadata":{"id":"RTNOJlqU5w_k"},"source":["# 회고\n","\n","### - 이번 프로젝트에서 **어려웠던 점**.\n","- 라이브러리 때문에 시간을 버려서 굉장히 화났다. 클라우드 환경이 불난 집에 기름을 들이부었다. 매우 화난다.\n","- ko.bin 파일을 불러오는 기존의 lms 코드가 오류나서 해결하기까지 시간이 걸렸다.\n","\n","### - 프로젝트를 진행하면서 **알아낸 점** 혹은 **아직 모호한 점**.\n","1. 윈도우는 라이브러리 지원이 안 되는 경우(e.g. KoNLPy의 Mecab)가 종종 있다.\n","2. 라이브러리, 운영체제, 파이썬 버전 모든 것이 호환돼야 설치가 잘 진행된다.\n","3. 설치와 실행(import)은 별개다. 설치가 무사히 끝났어도 실행이 안 될 수가 있다.\n","3. ko.bin 파일을 불러올 때 Word2Vec.load() 코드로 바꿔 써야 제대로 동작했다.\n","4. padding 추가는 post 보다 pre 가 더 효율적이다. (RNN 모델의 경우, 입력데이터가 순차 처리되어, 가장 마지막 입력이 최종 state 값에 가장 영향을 많이 미치게 된다.)\n","5. 적절한 epoch을 찾을 때 early stopping 방식을 사용한다.\n","6. LSTM 파라미터 계산 공식은 '$size of kernel*((dim+1)*unit+unit^2)$'이다.\n","\n","```python\n","# LSTM 파라미터 계산 결과 416\n","# => 계산식: 4*((4+1)*8+8^2) = 416\n","\n","vocab_size = 10  # sample 개수\n","word_vector_dim = 4  # input_dim\n","\n","model.add(layers.Embedding(10,4))  # layers.Embedding(sample, dim) => dim == input_dim\n","model.add(layers.LSTM(8))  # layers.LSTM(unit) => unit == output_dim\n","```\n","\n","### - 루브릭 평가 지표를 맞추기 위해 **시도한 것들**.\n","\n",">#### **루브릭**\n",">|번호|평가문항|상세기준|\n",">|:---:|---|---|\n",">|1|다양한 방법으로 Text Classification 태스크를 성공적으로 구현하였다.|3가지 이상의 모델이 성공적으로 시도됨|\n",">|2|gensim을 활용하여 자체학습된 혹은 사전학습된 임베딩 레이어를 분석하였다.|gensim의 유사단어 찾기를 활용하여 자체학습한 임베딩과 사전학습 임베딩을 적절히 분석함|\n",">|3|한국어 Word2Vec을 활용하여 가시적인 성능향상을 달성했다.|네이버 영화리뷰 데이터 감성분석 정확도를 85% 이상 달성함|\n","\n","Model 3가지 비교는 기존 lms 코드를 그대로 썼다. 사용한 모델은 LSTM, 1-D CNN, GlobalMaxPooling1D() 레이어 1개이다. 구글링해보니 적절한 epoch 값은 early_stopping 인 시점이라고 해서 적용해 보았다. model.fit() 속도가 너무 느리고 학습할수록 validation_loss 값이 올라가서(overfitting 발생) 기존 epochs=20 에서 5로 값을 낮췄다. epoch이 20이었을 때보다 훨씬 안정적으로 변했다.\n","\n","gensim으로 유사 단어 찾기를 해봤다. 일반 단어보다 감정 관련 단어여야 잘 찾아냈다. 그 중에서 부정적인 감정 단어(e.g. '별로')를 잘 추출해냈다. 불용어 처리를 더 세게 걸어야 했나 싶은 생각도 들지만 시간이 없는 관계로 일단 제출하기로 결정했다.\n","\n","기존의 LSTM 모델(accuracy 84.79%)보다 ko.bin 임베딩을 활용한 LSTM 성능(accuracy 86.48%)이 더 좋았음을 확인했다.\n","\n","### - 만약에 루브릭 평가 관련 지표를 **달성 하지 못했을 때, 이유에 관한 추정**.\n","없음\n","\n","### - **자기 다짐**\n","- 클라우드 환경은 한번씩 둘러봐야 한다. 보고 있어도 연결 끊기는 건 함정 ^^\n","- 그러므로 클라우드 환경에서 작업한 주피터 노트북 파일을 자주 다운받아서 로컬에 저장해놔야 한다.\n","\n","---\n","\n","- 대나무숲\n","\n","예전부터 lms 할 때마다 로컬 컴퓨터에서 패키지 설치가 안 되어서 난항을 겪었는데 이번 노드에서 제대로 포텐 터졌다. 빨간색 글자로 도배된 Error와 끝이 보이지 않는 Examining의 늪에 빠졌다. 어처구니가 없다 정말. 요즘 손가락이 너무 아프다. 이게 마음가짐으로 해결되는 것도 아니고 서럽네. 이정도면 감정 분석의 대상이 사실 내가 아니었을까? 라는 생각이 든다.\n","\n","- 화났음!\n","\n","항상 로컬에서만 작업해왔기에 이번에도 로컬 컴퓨터에서 하려고 라이브러리 설치부터 시도했다. 그러나 KoNLPy 공식 홈페이지에서 안내하기로 윈도우에서는 Mecab 을 지원하지 않는다고 되어 있었다. 현재 내 파이썬 버전이 lms 클라우드 환경과 동일한 3.9.7 버전인데 3.8.x 버전으로 다운그레이드해야 윈도우에서 사용할 수 있다고 되어 있다. 주말을 투자했으나 아나콘다 패키지가 꼬였는지 Examining 만 이틀 넘게 돌아가고 있어서 실패했다. 심지어 지금도 돌아가고 있다. 그렇게 로컬 컴퓨터에서 하겠다는 계획은 박살났다. GPU 사용하려고 CUDA까지 다 설정해놔서 그런 것인지 원인이 무엇인지 감을 잡을 수가 없다. 제출할 시간은 다가오고 마음이 급해서 lms에서 하려고 해봤으나 lms 오류가 나서 리셋이 됐고 다 날아가버렸다. 수업 시간 때에도 한번씩 OoOoOOOoooh!! 글자가 나타나면서 404 error 페이지가 날 반길 때가 자주 있었는데 그 화면을 이렇게 급할 때 또 보니 정말 부수고 싶었다. Colab 으로 왔으나 여기서도 이놈의 KoNLPy와 MeCab 설정하느라 머리가 아팠다. KoNLPy가 되면 MeCab이 안 되고 MeCab이 되면 KoNLPy가 안 되는 어이 없는 현상이 나타나서 해결하느라 결국 시간이 또 많이 걸렸다. Colab 런타임 GPU 설정을 했어도 model.fit() 코드 실행 속도가 느려서 매우 화났다. 한번씩 Colab 연결이 끊어져서 실행했던 내역이 싹 증발해서 처음부터 다시 실행하느라 분노폭발했다."]}],"metadata":{"colab":{"authorship_tag":"ABX9TyMmXMLjLQj/Z6TXlm0FC0KL","collapsed_sections":[],"mount_file_id":"15jpqbcAKEQlJGO2vpDrshwIDK10NS70s","name":"[E-06] Naver movie sentiment analysis .ipynb","provenance":[]},"kernelspec":{"display_name":"Python 3","name":"python3"},"language_info":{"name":"python"}},"nbformat":4,"nbformat_minor":0}