diff --git a/docs/_downloads/032d653a4f5a9c1ec32b9fc7c989ffe1/seq2seq_translation_tutorial.ipynb b/docs/_downloads/032d653a4f5a9c1ec32b9fc7c989ffe1/seq2seq_translation_tutorial.ipynb index a5bf39b3d..3cd3765f9 100644 --- a/docs/_downloads/032d653a4f5a9c1ec32b9fc7c989ffe1/seq2seq_translation_tutorial.ipynb +++ b/docs/_downloads/032d653a4f5a9c1ec32b9fc7c989ffe1/seq2seq_translation_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n\uae30\ucd08\ubd80\ud130 \uc2dc\uc791\ud558\ub294 NLP: Sequence to Sequence \ub124\ud2b8\uc6cc\ud06c\uc640 Attention\uc744 \uc774\uc6a9\ud55c \ubc88\uc5ed\n********************************************************************************\n**Author**: `Sean Robertson `_\n **\ubc88\uc5ed**: `\ud669\uc131\uc218 `_\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc740 \"\uae30\ucd08\ubd80\ud130 \uc2dc\uc791\ud558\ub294 NLP\"\uc758 \uc138\ubc88\uc9f8\uc774\uc790 \ub9c8\uc9c0\ub9c9 \ud3b8\uc73c\ub85c, NLP \ubaa8\ub378\ub9c1 \uc791\uc5c5\uc744\n\uc704\ud55c \ub370\uc774\ud130 \uc804\ucc98\ub9ac\uc5d0 \uc0ac\uc6a9\ud560 \uc790\uccb4 \ud074\ub798\uc2a4\uc640 \ud568\uc218\ub4e4\uc744 \uc791\uc131\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc744 \ub9c8\uce5c \ub4a4\uc5d0\ub294 `torchtext` \uac00 \uc5b4\ub5bb\uac8c \uc9c0\uae08\uae4c\uc9c0\uc758 \ud29c\ud1a0\ub9ac\uc5bc\ub4e4\uc5d0\uc11c\uc758\n\uc804\ucc98\ub9ac \uacfc\uc815\uc744 \ub2e4\ub8e8\ub294\uc9c0\ub97c \uc774\ud6c4 \ud29c\ud1a0\ub9ac\uc5bc\ub4e4\uc5d0\uc11c \ubc30\uc6b8 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc774 \ud504\ub85c\uc81d\ud2b8\uc5d0\uc11c\ub294 \uc2e0\uacbd\ub9dd\uc774 \ubd88\uc5b4\ub97c \uc601\uc5b4\ub85c \ubc88\uc5ed\ud558\ub3c4\ub85d \uac00\ub974\uce60 \uc608\uc815\uc785\ub2c8\ub2e4.\n\n::\n\n [KEY: > input, = target, < output]\n\n > il est en train de peindre un tableau .\n = he is painting a picture .\n < he is painting a picture .\n\n > pourquoi ne pas essayer ce vin delicieux ?\n = why not try that delicious wine ?\n < why not try that delicious wine ?\n\n > elle n est pas poete mais romanciere .\n = she is not a poet but a novelist .\n < she not not a poet but a novelist .\n\n > vous etes trop maigre .\n = you re too skinny .\n < you re all alone .\n\n... \uc131\uacf5\uc728\uc740 \ubcc0\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\ud558\ub098\uc758 \uc2dc\ud000\uc2a4\ub97c \ub2e4\ub978 \uc2dc\ud000\uc2a4\ub85c \ubc14\uafb8\ub294 \ub450\uac1c\uc758 RNN\uc774 \ud568\uaed8 \ub3d9\uc791\ud558\ub294\n`sequence to sequence network `__ \uc758 \uac04\ub2e8\ud558\uc9c0\ub9cc \uac15\ub825\ud55c \uc544\uc774\ub514\uc5b4\uac00\n\uc774\uac83(\ubc88\uc5ed)\uc744 \uac00\ub2a5\ud558\uac8c \ud569\ub2c8\ub2e4. \uc778\ucf54\ub354 \ub124\ud2b8\uc6cc\ud06c\ub294 \uc785\ub825 \uc2dc\ud000\uc2a4\ub97c \ubca1\ud130\ub85c \uc555\ucd95\ud558\uace0,\n\ub514\ucf54\ub354 \ub124\ud2b8\uc6cc\ud06c\ub294 \ud574\ub2f9 \ubca1\ud130\ub97c \uc0c8\ub85c\uc6b4 \uc2dc\ud000\uc2a4\ub85c \ud3bc\uce69\ub2c8\ub2e4.\n\n.. figure:: /_static/img/seq-seq-images/seq2seq.png\n :alt:\n\n\uc774 \ubaa8\ub378\uc744 \uac1c\uc120\ud558\uae30 \uc704\ud574 `Attention Mechanism `__ \uc744\n\uc0ac\uc6a9\ud558\uba74 \ub514\ucf54\ub354\uac00 \uc785\ub825 \uc2dc\ud000\uc2a4\uc758 \ud2b9\uc815 \ubc94\uc704\uc5d0 \uc9d1\uc911\ud560 \uc218 \uc788\ub3c4\ub85d \ud569\ub2c8\ub2e4.\n\n**\ucd94\ucc9c \uc790\ub8cc:**\n\n\ucd5c\uc18c\ud55c Pytorch\ub97c \uc124\uce58\ud588\uace0, Python\uc744 \uc54c\uace0, Tensor\ub97c \uc774\ud574\ud55c\ub2e4\uace0 \uac00\uc815\ud569\ub2c8\ub2e4.:\n\n- http://pytorch.org/ \uc124\uce58 \uc548\ub0b4\ub97c \uc704\ud55c \uc790\ub8cc\n- :doc:`/beginner/deep_learning_60min_blitz` \uc77c\ubc18\uc801\uc778 PyTorch \uc2dc\uc791\uc744 \uc704\ud55c \uc790\ub8cc\n- :doc:`/beginner/pytorch_with_examples` \ub113\uace0 \uae4a\uc740 \ud1b5\ucc30\uc744 \uc704\ud55c \uc790\ub8cc\n- :doc:`/beginner/former_torchies_tutorial` \uc774\uc804 Lua Torch \uc0ac\uc6a9\uc790\ub97c \uc704\ud55c \uc790\ub8cc\n\n\nSequence to Sequence \ub124\ud2b8\uc6cc\ud06c\uc640 \ub3d9\uc791 \ubc29\ubc95\uc5d0 \uad00\ud574\uc11c \uc544\ub294 \uac83\uc740 \uc720\uc6a9\ud569\ub2c8\ub2e4:\n\n- `Learning Phrase Representations using RNN Encoder-Decoder for\n Statistical Machine Translation `__\n- `Sequence to Sequence Learning with Neural\n Networks `__\n- `Neural Machine Translation by Jointly Learning to Align and\n Translate `__\n- `A Neural Conversational Model `__\n\n\uc774\uc804 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0 \uc788\ub294\n:doc:`/intermediate/char_rnn_classification_tutorial`\n\uc640 :doc:`/intermediate/char_rnn_generation_tutorial` \ub294\n\uac01\uac01 \uc778\ucf54\ub354, \ub514\ucf54\ub354 \ubaa8\ub378\uacfc \ube44\uc2b7\ud55c \ucee8\uc13c\uc744 \uac00\uc9c0\uae30 \ub54c\ubb38\uc5d0 \ub3c4\uc6c0\uc774 \ub429\ub2c8\ub2e4.\n\n\ucd94\uac00\ub85c \uc774 \ud1a0\ud53d\ub4e4\uc744 \ub2e4\ub8e8\ub294 \ub17c\ubb38\uc744 \uc77d\uc5b4 \ubcf4\uc2ed\uc2dc\uc624:\n\n- `Learning Phrase Representations using RNN Encoder-Decoder for\n Statistical Machine Translation `__\n- `Sequence to Sequence Learning with Neural\n Networks `__\n- `Neural Machine Translation by Jointly Learning to Align and\n Translate `__\n- `A Neural Conversational Model `__\n\n\n**\uc694\uad6c \uc0ac\ud56d**\n\n" + "\n\uae30\ucd08\ubd80\ud130 \uc2dc\uc791\ud558\ub294 NLP: Sequence to Sequence \ub124\ud2b8\uc6cc\ud06c\uc640 Attention\uc744 \uc774\uc6a9\ud55c \ubc88\uc5ed\n********************************************************************************\n**Author**: `Sean Robertson `_\n **\ubc88\uc5ed**: `\ud669\uc131\uc218 `_\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc740 \"\uae30\ucd08\ubd80\ud130 \uc2dc\uc791\ud558\ub294 NLP\"\uc758 \uc138\ubc88\uc9f8\uc774\uc790 \ub9c8\uc9c0\ub9c9 \ud3b8\uc73c\ub85c, NLP \ubaa8\ub378\ub9c1 \uc791\uc5c5\uc744\n\uc704\ud55c \ub370\uc774\ud130 \uc804\ucc98\ub9ac\uc5d0 \uc0ac\uc6a9\ud560 \uc790\uccb4 \ud074\ub798\uc2a4\uc640 \ud568\uc218\ub4e4\uc744 \uc791\uc131\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc744 \ub9c8\uce5c \ub4a4\uc5d0\ub294 `torchtext` \uac00 \uc5b4\ub5bb\uac8c \uc9c0\uae08\uae4c\uc9c0\uc758 \ud29c\ud1a0\ub9ac\uc5bc\ub4e4\uc5d0\uc11c\uc758\n\uc804\ucc98\ub9ac \uacfc\uc815\uc744 \ub2e4\ub8e8\ub294\uc9c0\ub97c \uc774\ud6c4 \ud29c\ud1a0\ub9ac\uc5bc\ub4e4\uc5d0\uc11c \ubc30\uc6b8 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc774 \ud504\ub85c\uc81d\ud2b8\uc5d0\uc11c\ub294 \uc2e0\uacbd\ub9dd\uc774 \ubd88\uc5b4\ub97c \uc601\uc5b4\ub85c \ubc88\uc5ed\ud558\ub3c4\ub85d \uac00\ub974\uce60 \uc608\uc815\uc785\ub2c8\ub2e4.\n\n::\n\n [KEY: > input, = target, < output]\n\n > il est en train de peindre un tableau .\n = he is painting a picture .\n < he is painting a picture .\n\n > pourquoi ne pas essayer ce vin delicieux ?\n = why not try that delicious wine ?\n < why not try that delicious wine ?\n\n > elle n est pas poete mais romanciere .\n = she is not a poet but a novelist .\n < she not not a poet but a novelist .\n\n > vous etes trop maigre .\n = you re too skinny .\n < you re all alone .\n\n... \uc131\uacf5\uc728\uc740 \ubcc0\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\ud558\ub098\uc758 \uc2dc\ud000\uc2a4\ub97c \ub2e4\ub978 \uc2dc\ud000\uc2a4\ub85c \ubc14\uafb8\ub294 \ub450\uac1c\uc758 RNN\uc774 \ud568\uaed8 \ub3d9\uc791\ud558\ub294\n`sequence to sequence network `__ \uc758 \uac04\ub2e8\ud558\uc9c0\ub9cc \uac15\ub825\ud55c \uc544\uc774\ub514\uc5b4\uac00\n\uc774\uac83(\ubc88\uc5ed)\uc744 \uac00\ub2a5\ud558\uac8c \ud569\ub2c8\ub2e4. \uc778\ucf54\ub354 \ub124\ud2b8\uc6cc\ud06c\ub294 \uc785\ub825 \uc2dc\ud000\uc2a4\ub97c \ubca1\ud130\ub85c \uc555\ucd95\ud558\uace0,\n\ub514\ucf54\ub354 \ub124\ud2b8\uc6cc\ud06c\ub294 \ud574\ub2f9 \ubca1\ud130\ub97c \uc0c8\ub85c\uc6b4 \uc2dc\ud000\uc2a4\ub85c \ud3bc\uce69\ub2c8\ub2e4.\n\n.. figure:: /_static/img/seq-seq-images/seq2seq.png\n :alt:\n\n\uc774 \ubaa8\ub378\uc744 \uac1c\uc120\ud558\uae30 \uc704\ud574 `Attention Mechanism `__ \uc744\n\uc0ac\uc6a9\ud558\uba74 \ub514\ucf54\ub354\uac00 \uc785\ub825 \uc2dc\ud000\uc2a4\uc758 \ud2b9\uc815 \ubc94\uc704\uc5d0 \uc9d1\uc911\ud560 \uc218 \uc788\ub3c4\ub85d \ud569\ub2c8\ub2e4.\n\n**\ucd94\ucc9c \uc790\ub8cc:**\n\n\ucd5c\uc18c\ud55c Pytorch\ub97c \uc124\uce58\ud588\uace0, Python\uc744 \uc54c\uace0, Tensor\ub97c \uc774\ud574\ud55c\ub2e4\uace0 \uac00\uc815\ud569\ub2c8\ub2e4.:\n\n- http://pytorch.org/ \uc124\uce58 \uc548\ub0b4\ub97c \uc704\ud55c \uc790\ub8cc\n- :doc:`/beginner/deep_learning_60min_blitz` \uc77c\ubc18\uc801\uc778 PyTorch \uc2dc\uc791\uc744 \uc704\ud55c \uc790\ub8cc\n- :doc:`/beginner/pytorch_with_examples` \ub113\uace0 \uae4a\uc740 \ud1b5\ucc30\uc744 \uc704\ud55c \uc790\ub8cc\n- :doc:`/beginner/former_torchies_tutorial` \uc774\uc804 Lua Torch \uc0ac\uc6a9\uc790\ub97c \uc704\ud55c \uc790\ub8cc\n\n\nSequence to Sequence \ub124\ud2b8\uc6cc\ud06c\uc640 \ub3d9\uc791 \ubc29\ubc95\uc5d0 \uad00\ud574\uc11c \uc544\ub294 \uac83\uc740 \uc720\uc6a9\ud569\ub2c8\ub2e4:\n\n- `Learning Phrase Representations using RNN Encoder-Decoder for\n Statistical Machine Translation `__\n- `Sequence to Sequence Learning with Neural\n Networks `__\n- `Neural Machine Translation by Jointly Learning to Align and\n Translate `__\n- `A Neural Conversational Model `__\n\n\uc774\uc804 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0 \uc788\ub294\n:doc:`/intermediate/char_rnn_classification_tutorial`\n\uc640 :doc:`/intermediate/char_rnn_generation_tutorial` \ub294\n\uac01\uac01 \uc778\ucf54\ub354, \ub514\ucf54\ub354 \ubaa8\ub378\uacfc \ube44\uc2b7\ud55c \ucee8\uc13c\uc744 \uac00\uc9c0\uae30 \ub54c\ubb38\uc5d0 \ub3c4\uc6c0\uc774 \ub429\ub2c8\ub2e4.\n\n\ucd94\uac00\ub85c \uc774 \ud1a0\ud53d\ub4e4\uc744 \ub2e4\ub8e8\ub294 \ub17c\ubb38\uc744 \uc77d\uc5b4 \ubcf4\uc2ed\uc2dc\uc624:\n\n- `Learning Phrase Representations using RNN Encoder-Decoder for\n Statistical Machine Translation `__\n- `Sequence to Sequence Learning with Neural\n Networks `__\n- `Neural Machine Translation by Jointly Learning to Align and\n Translate `__\n- `A Neural Conversational Model `__\n\n\n**\uc694\uad6c \uc0ac\ud56d**\n" ] }, { @@ -423,7 +423,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/03a48646520c277662581e858e680809/model_parallel_tutorial.ipynb b/docs/_downloads/03a48646520c277662581e858e680809/model_parallel_tutorial.ipynb index cb7448ebd..85ae0e229 100644 --- a/docs/_downloads/03a48646520c277662581e858e680809/model_parallel_tutorial.ipynb +++ b/docs/_downloads/03a48646520c277662581e858e680809/model_parallel_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n\ub2e8\uc77c \uba38\uc2e0\uc744 \uc774\uc6a9\ud55c \ubaa8\ub378 \ubcd1\ub82c\ud654 \uc2e4\uc2b5 \uc608\uc81c\n===================================================\n**\uc800\uc790** : `Shen Li `_\n**\ubc88\uc5ed** : `\uc548\uc0c1\uc900 `_\n\n\ubaa8\ub378 \ubcd1\ub82c \ucc98\ub9ac\ub294 \ubd84\uc0b0 \ud559\uc2b5 \uae30\uc220\uc5d0 \ubc94\uc6a9\uc801\uc73c\ub85c \uc0ac\uc6a9\ub418\uace0 \uc788\uc2b5\ub2c8\ub2e4. \n\uc774\uc804 \ud29c\ud1a0\ub9ac\uc5bc\ub4e4\uc5d0\uc11c\ub294 'DataParallel' ``_\n\uc5ec\ub7ec GPU\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc744 \ud559\uc2b5 \uc2dc\ud0ac \ub54c \uc5b4\ub5bb\uac8c DataParallel\uc744 \uc0ac\uc6a9\ud558\ub294\uc9c0\uc5d0 \ub300\ud574\uc11c \uc0b4\ud3b4\ubcf4\uc558\uc2b5\ub2c8\ub2e4.\n\uc774 \ubc29\ubc95\uc740 \uac01 GPU\uc5d0 \uc785\ub825 \ub370\uc774\ud130\ub97c \ubd80\ubd84\uc801\uc73c\ub85c \ud560\ub2f9\ud558\uace0 \ub3d9\uc77c\ud55c \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc744 \ubcf5\uc81c\ud558\uc5ec \uc774\uc6a9\ud558\ub294 \ubc29\uc2dd\uc774\uc5c8\uc2b5\ub2c8\ub2e4. \n\uc774 \ubc29\ubc95\uc740 \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc744 \uc0c1\ub2f9\ud788 \ube60\ub974\uac8c \ud559\uc2b5\uc2dc\ud0ac \uc218 \uc788\ub294 \uc7a5\uc810\uc774 \uc788\uc9c0\ub9cc, \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc774 \ub108\ubb34 \ucee4\uc11c \ub2e8\uc77c GPU\uc5d0 \ud560\ub2f9\uc774 \ub418\uc9c0 \uc54a\ub294 \uacbd\uc6b0\uc5d0\ub294 \ub3d9\uc791\ud558\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4.\n\n\uc774\ubc88 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 ``\ub370\uc774\ud130 \ubcd1\ub82c \ucc98\ub9ac`` \uac00 \uc544\ub2cc **\ubaa8\ub378 \ubcd1\ub82c \ucc98\ub9ac** \ubb38\uc81c\ub97c \ud574\uacb0\ud558\ub294 \ubc29\ubc95\uc744 \uc18c\uac1c\ud569\ub2c8\ub2e4.\n\uac01 GPU\uc5d0 \ubaa8\ub378 \uc804\uccb4\ub97c \ubcf5\uc81c\ud558\ub294 \uac83\uc774 \uc544\ub2cc, \ud558\ub098\uc758 \ubaa8\ub378\uc744 \uc5ec\ub7ec GPU\uc5d0 \ubd84\ud560\ud558\uc5ec \ud560\ub2f9\ud558\ub294 \ubc29\ubc95\uc785\ub2c8\ub2e4.\n\uad6c\uccb4\uc801\uc73c\ub85c, 10\uac1c\uc758 \uce35\uc73c\ub85c \uad6c\uc131\ub41c ``m`` \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc5d0 \ub300\ud574\uc11c ``\ub370\uc774\ud130 \ubcd1\ub82c \ucc98\ub9ac`` \ubc29\ubc95\uc740 10\uac1c\uc758 \uce35\uc744 \uc804\ubd80 \ubcf5\uc81c\ud558\uc5ec \uac01 GPU\uc5d0 \ud560\ub2f9\ud558\uc5ec \ucc98\ub9ac\ud558\uc9c0\ub9cc,\n\uc774\uc640 \ubc18\ub300\ub85c 2\uac1c\uc758 GPU\uc5d0 \ubaa8\ub378\uc744 \ubcd1\ub82c \ucc98\ub9ac\ud55c\ub2e4\uba74, \uac01 GPU\uc5d0 5\uac1c\uc758 \uce35\uc529 \uac01\uac01 \ud560\ub2f9\ud558\uc5ec \ud638\uc2a4\ud305\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\ubaa8\ub378 \ubcd1\ub82c \ucc98\ub9ac\uc758 \uc804\ubc18\uc801\uc778 \uc544\uc774\ub514\uc5b4\ub294 \ubaa8\ub378\uc758 \uc11c\ube0c \ub124\ud2b8\uc6cc\ud06c\ub4e4\uc744 \uac01\uac01 \ub2e4\ub978 GPU\uc5d0 \ud560\ub2f9\ud558\uace0, \n\uac01 \uc7a5\ube44 \ubcc4\ub85c \uc21c\uc804\ud30c\ub97c \uc9c4\ud589\ud558\uc5ec \uacc4\uc0b0\ub418\ub294 \ucd9c\ub825\uac12\ub4e4\uc744 \uac01 \uc7a5\ube44 \uac04 \uacf5\uc720\ud558\uc5ec \uc774\uc6a9\ud558\ub294 \uac83\uc785\ub2c8\ub2e4. \n\uc774\uc6a9\ud558\uace0\uc790 \ud558\ub294 \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc744 \ubd80\ubd84\uc801\uc73c\ub85c \uac01 GPU\uc5d0 \ud560\ub2f9\ud558\ub294 \uac83\uc774\uae30 \ub54c\ubb38\uc5d0, \uc5ec\ub7ec GPU\ub97c \uc774\uc6a9\ud558\uc5ec \ub354 \ud070 \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc744 \ud560\ub2f9\ud558\uace0 \ud559\uc2b5\uc2dc\ud0ac \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uc774\ubc88 \ud29c\ud1a0\ub9ac\uc5bc\uc740 \uac70\ub300\ud55c \ubaa8\ub378\uc744 \uc81c\ud55c\ub41c \uc218\uc758 GPU\uc5d0 \ubd84\ud560\ud558\uc5ec \ud560\ub2f9\ud558\uc9c0 \uc54a\uace0, \uadf8 \ub300\uc2e0, \ubaa8\ub378 \ubcd1\ub82c \ucc98\ub9ac\uc758 \uc544\uc774\ub514\uc5b4\ub97c \uc774\ud574\ud558\ub294 \ubaa9\uc801\uc73c\ub85c \uc791\uc131\ub418\uc5c8\uc2b5\ub2c8\ub2e4. \n\ubaa8\ub378 \ubcd1\ub82c \ucc98\ub9ac\uc758 \uc544\uc774\ub514\uc5b4\ub97c \ud65c\uc6a9\ud558\uc5ec \uc2e4\uc81c \uc5b4\ud50c\ub9ac\ucf00\uc774\uc158\uc5d0 \uc801\uc6a9\ud558\ub294 \uac83\uc740 \uc5ec\ub7ec\ubd84\uc758 \ubaab\uc785\ub2c8\ub2e4.\n\n

Note

\uc2e0\uacbd\ub9dd \ubaa8\ub378\uc744 \uc5ec\ub7ec \uc11c\ubc84\ub97c \uc774\uc6a9\ud558\uc5ec \ud559\uc2b5\uc2dc\ud0a4\ub294 \ubcd1\ub82c \ud559\uc2b5 \ubc29\ubc95\uc5d0 \ub300\ud574\uc11c\ub294 \ub2e4\uc74c \ud29c\ud1a0\ub9ac\uc5bc\uc744 \ucc38\uace0\ud558\uc138\uc694.\n `\ubd84\uc0b0 \ud504\ub808\uc784\uc6cc\ud06c RPC \uc2dc\uc791\ud574\ubcf4\uae30 `__

\n\nBasic Usage\n-----------\n\n" + "\n\ub2e8\uc77c \uba38\uc2e0\uc744 \uc774\uc6a9\ud55c \ubaa8\ub378 \ubcd1\ub82c\ud654 \uc2e4\uc2b5 \uc608\uc81c\n===================================================\n**\uc800\uc790** : `Shen Li `_\n**\ubc88\uc5ed** : `\uc548\uc0c1\uc900 `_\n\n\ubaa8\ub378 \ubcd1\ub82c \ucc98\ub9ac\ub294 \ubd84\uc0b0 \ud559\uc2b5 \uae30\uc220\uc5d0 \ubc94\uc6a9\uc801\uc73c\ub85c \uc0ac\uc6a9\ub418\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n\uc774\uc804 \ud29c\ud1a0\ub9ac\uc5bc\ub4e4\uc5d0\uc11c\ub294 'DataParallel' ``_\n\uc5ec\ub7ec GPU\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc744 \ud559\uc2b5 \uc2dc\ud0ac \ub54c \uc5b4\ub5bb\uac8c DataParallel\uc744 \uc0ac\uc6a9\ud558\ub294\uc9c0\uc5d0 \ub300\ud574\uc11c \uc0b4\ud3b4\ubcf4\uc558\uc2b5\ub2c8\ub2e4.\n\uc774 \ubc29\ubc95\uc740 \uac01 GPU\uc5d0 \uc785\ub825 \ub370\uc774\ud130\ub97c \ubd80\ubd84\uc801\uc73c\ub85c \ud560\ub2f9\ud558\uace0 \ub3d9\uc77c\ud55c \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc744 \ubcf5\uc81c\ud558\uc5ec \uc774\uc6a9\ud558\ub294 \ubc29\uc2dd\uc774\uc5c8\uc2b5\ub2c8\ub2e4.\n\uc774 \ubc29\ubc95\uc740 \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc744 \uc0c1\ub2f9\ud788 \ube60\ub974\uac8c \ud559\uc2b5\uc2dc\ud0ac \uc218 \uc788\ub294 \uc7a5\uc810\uc774 \uc788\uc9c0\ub9cc, \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc774 \ub108\ubb34 \ucee4\uc11c \ub2e8\uc77c GPU\uc5d0 \ud560\ub2f9\uc774 \ub418\uc9c0 \uc54a\ub294 \uacbd\uc6b0\uc5d0\ub294 \ub3d9\uc791\ud558\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4.\n\n\uc774\ubc88 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 ``\ub370\uc774\ud130 \ubcd1\ub82c \ucc98\ub9ac`` \uac00 \uc544\ub2cc **\ubaa8\ub378 \ubcd1\ub82c \ucc98\ub9ac** \ubb38\uc81c\ub97c \ud574\uacb0\ud558\ub294 \ubc29\ubc95\uc744 \uc18c\uac1c\ud569\ub2c8\ub2e4.\n\uac01 GPU\uc5d0 \ubaa8\ub378 \uc804\uccb4\ub97c \ubcf5\uc81c\ud558\ub294 \uac83\uc774 \uc544\ub2cc, \ud558\ub098\uc758 \ubaa8\ub378\uc744 \uc5ec\ub7ec GPU\uc5d0 \ubd84\ud560\ud558\uc5ec \ud560\ub2f9\ud558\ub294 \ubc29\ubc95\uc785\ub2c8\ub2e4.\n\uad6c\uccb4\uc801\uc73c\ub85c, 10\uac1c\uc758 \uce35\uc73c\ub85c \uad6c\uc131\ub41c ``m`` \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc5d0 \ub300\ud574\uc11c ``\ub370\uc774\ud130 \ubcd1\ub82c \ucc98\ub9ac`` \ubc29\ubc95\uc740 10\uac1c\uc758 \uce35\uc744 \uc804\ubd80 \ubcf5\uc81c\ud558\uc5ec \uac01 GPU\uc5d0 \ud560\ub2f9\ud558\uc5ec \ucc98\ub9ac\ud558\uc9c0\ub9cc,\n\uc774\uc640 \ubc18\ub300\ub85c 2\uac1c\uc758 GPU\uc5d0 \ubaa8\ub378\uc744 \ubcd1\ub82c \ucc98\ub9ac\ud55c\ub2e4\uba74, \uac01 GPU\uc5d0 5\uac1c\uc758 \uce35\uc529 \uac01\uac01 \ud560\ub2f9\ud558\uc5ec \ud638\uc2a4\ud305\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\ubaa8\ub378 \ubcd1\ub82c \ucc98\ub9ac\uc758 \uc804\ubc18\uc801\uc778 \uc544\uc774\ub514\uc5b4\ub294 \ubaa8\ub378\uc758 \uc11c\ube0c \ub124\ud2b8\uc6cc\ud06c\ub4e4\uc744 \uac01\uac01 \ub2e4\ub978 GPU\uc5d0 \ud560\ub2f9\ud558\uace0,\n\uac01 \uc7a5\ube44 \ubcc4\ub85c \uc21c\uc804\ud30c\ub97c \uc9c4\ud589\ud558\uc5ec \uacc4\uc0b0\ub418\ub294 \ucd9c\ub825\uac12\ub4e4\uc744 \uac01 \uc7a5\ube44 \uac04 \uacf5\uc720\ud558\uc5ec \uc774\uc6a9\ud558\ub294 \uac83\uc785\ub2c8\ub2e4.\n\uc774\uc6a9\ud558\uace0\uc790 \ud558\ub294 \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc744 \ubd80\ubd84\uc801\uc73c\ub85c \uac01 GPU\uc5d0 \ud560\ub2f9\ud558\ub294 \uac83\uc774\uae30 \ub54c\ubb38\uc5d0, \uc5ec\ub7ec GPU\ub97c \uc774\uc6a9\ud558\uc5ec \ub354 \ud070 \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc744 \ud560\ub2f9\ud558\uace0 \ud559\uc2b5\uc2dc\ud0ac \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uc774\ubc88 \ud29c\ud1a0\ub9ac\uc5bc\uc740 \uac70\ub300\ud55c \ubaa8\ub378\uc744 \uc81c\ud55c\ub41c \uc218\uc758 GPU\uc5d0 \ubd84\ud560\ud558\uc5ec \ud560\ub2f9\ud558\uc9c0 \uc54a\uace0, \uadf8 \ub300\uc2e0, \ubaa8\ub378 \ubcd1\ub82c \ucc98\ub9ac\uc758 \uc544\uc774\ub514\uc5b4\ub97c \uc774\ud574\ud558\ub294 \ubaa9\uc801\uc73c\ub85c \uc791\uc131\ub418\uc5c8\uc2b5\ub2c8\ub2e4.\n\ubaa8\ub378 \ubcd1\ub82c \ucc98\ub9ac\uc758 \uc544\uc774\ub514\uc5b4\ub97c \ud65c\uc6a9\ud558\uc5ec \uc2e4\uc81c \uc5b4\ud50c\ub9ac\ucf00\uc774\uc158\uc5d0 \uc801\uc6a9\ud558\ub294 \uac83\uc740 \uc5ec\ub7ec\ubd84\uc758 \ubaab\uc785\ub2c8\ub2e4.\n\n

Note

\uc2e0\uacbd\ub9dd \ubaa8\ub378\uc744 \uc5ec\ub7ec \uc11c\ubc84\ub97c \uc774\uc6a9\ud558\uc5ec \ud559\uc2b5\uc2dc\ud0a4\ub294 \ubcd1\ub82c \ud559\uc2b5 \ubc29\ubc95\uc5d0 \ub300\ud574\uc11c\ub294 \ub2e4\uc74c \ud29c\ud1a0\ub9ac\uc5bc\uc744 \ucc38\uace0\ud558\uc138\uc694.\n `\ubd84\uc0b0 \ud504\ub808\uc784\uc6cc\ud06c RPC \uc2dc\uc791\ud574\ubcf4\uae30 `__

\n\nBasic Usage\n-----------\n" ] }, { @@ -33,7 +33,7 @@ }, "outputs": [], "source": [ - "import torch\nimport torch.nn as nn\nimport torch.optim as optim\n\n\nclass ToyModel(nn.Module):\n def __init__(self):\n super(ToyModel, self).__init__()\n self.net1 = torch.nn.Linear(10, 10).to('cuda:0') # \uccab \ubc88\uc9f8 \uce35\uc744 \uccab \ubc88\uc9f8 GPU\uc5d0 \ud560\ub2f9\n self.relu = torch.nn.ReLU()\n self.net2 = torch.nn.Linear(10, 5).to('cuda:1') # \ub450 \ubc88\uc9f8 \uce35\uc744 \ub450 \ubc88\uc9f8 GPU\uc5d0 \ud560\ub2f9\n\n def forward(self, x):\n x = self.relu(self.net1(x.to('cuda:0'))) \n return self.net2(x.to('cuda:1')) # \uccab \ubc88\uc9f8 \uce35\uc758 \uc0b0\ucd9c\ubb3c\uc744 \ub450 \ubc88\uc9f8 GPU\uc5d0 \ud560\ub2f9\ud558\uc5ec \uc9c4\ud589" + "import torch\nimport torch.nn as nn\nimport torch.optim as optim\n\n\nclass ToyModel(nn.Module):\n def __init__(self):\n super(ToyModel, self).__init__()\n self.net1 = torch.nn.Linear(10, 10).to('cuda:0') # \uccab \ubc88\uc9f8 \uce35\uc744 \uccab \ubc88\uc9f8 GPU\uc5d0 \ud560\ub2f9\n self.relu = torch.nn.ReLU()\n self.net2 = torch.nn.Linear(10, 5).to('cuda:1') # \ub450 \ubc88\uc9f8 \uce35\uc744 \ub450 \ubc88\uc9f8 GPU\uc5d0 \ud560\ub2f9\n\n def forward(self, x):\n x = self.relu(self.net1(x.to('cuda:0')))\n return self.net2(x.to('cuda:1')) # \uccab \ubc88\uc9f8 \uce35\uc758 \uc0b0\ucd9c\ubb3c\uc744 \ub450 \ubc88\uc9f8 GPU\uc5d0 \ud560\ub2f9\ud558\uc5ec \uc9c4\ud589" ] }, { @@ -58,7 +58,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uae30\uc874\uc5d0 \uc874\uc7ac\ud558\ub294 \ubaa8\ub4c8\uc5d0 \ubaa8\ub378 \ubcd1\ub82c \ucc98\ub9ac \uc801\uc6a9\ud574\ubcf4\uae30\n----------------------------------------\n\n\uae30\uc874\uc5d0 \ub2e8\uc77c GPU\uc5d0 \uc874\uc7ac\ud558\ub294 \ubaa8\ub4c8\uc744 \uc5ec\ub7ec GPU\uc5d0 \ud560\ub2f9\ud558\ub294 \uac83\uc740 \ub2e8\uc9c0 \uba87 \uc904\uc758 \ucf54\ub4dc\ub97c \uc218\uc815\ud558\ub294 \uac83\uc73c\ub85c\ub3c4 \uc27d\uac8c \uac00\ub2a5\ud569\ub2c8\ub2e4. \n\uc544\ub798\uc5d0 \uc788\ub294 \ucf54\ub4dc\ub4e4\uc740 ResNet50 \ubaa8\ub378\uc744 \ubd84\ud560\ud558\ub294 \ubc29\ubc95\uc785\ub2c8\ub2e4. \uc774 \uc544\uc774\ub514\uc5b4\ub294, \uae30\uc874\uc5d0 \uc874\uc7ac\ud558\ub294 ResNet \ubaa8\ub4c8\uc744 \uc0c1\uc18d\ubc1b\uc544 \uc124\uacc4\ud560 \ub54c, 2\uac1c\uc758 GPU\uc5d0 \uce35\uc744 \ub098\ub204\uc5b4 \uc124\uacc4\ud558\ub294 \ubc29\uc2dd\uc73c\ub85c \uc9c4\ud589\ub429\ub2c8\ub2e4. \n\uadf8 \ud6c4, 2\uac1c GPU\uc5d0\uc11c \uacc4\uc0b0\ub418\ub294 \uc911\uac04 \uc0b0\ucd9c\ubb3c \ud150\uc11c\uac12\uc744 \uc801\uc808\ud788 \ubc30\uce58\ud558\uae30 \uc704\ud5e4 \uc21c\uc804\ud30c \uba54\uc18c\ub4dc\ub97c \uc218\uc815\ud569\ub2c8\ub2e4.\n\n" + "\uae30\uc874\uc5d0 \uc874\uc7ac\ud558\ub294 \ubaa8\ub4c8\uc5d0 \ubaa8\ub378 \ubcd1\ub82c \ucc98\ub9ac \uc801\uc6a9\ud574\ubcf4\uae30\n---------------------------------------------------\n\n\uae30\uc874\uc5d0 \ub2e8\uc77c GPU\uc5d0 \uc874\uc7ac\ud558\ub294 \ubaa8\ub4c8\uc744 \uc5ec\ub7ec GPU\uc5d0 \ud560\ub2f9\ud558\ub294 \uac83\uc740 \ub2e8\uc9c0 \uba87 \uc904\uc758 \ucf54\ub4dc\ub97c \uc218\uc815\ud558\ub294 \uac83\uc73c\ub85c\ub3c4 \uc27d\uac8c \uac00\ub2a5\ud569\ub2c8\ub2e4.\n\uc544\ub798\uc5d0 \uc788\ub294 \ucf54\ub4dc\ub4e4\uc740 ResNet50 \ubaa8\ub378\uc744 \ubd84\ud560\ud558\ub294 \ubc29\ubc95\uc785\ub2c8\ub2e4. \uc774 \uc544\uc774\ub514\uc5b4\ub294, \uae30\uc874\uc5d0 \uc874\uc7ac\ud558\ub294 ResNet \ubaa8\ub4c8\uc744 \uc0c1\uc18d\ubc1b\uc544 \uc124\uacc4\ud560 \ub54c, 2\uac1c\uc758 GPU\uc5d0 \uce35\uc744 \ub098\ub204\uc5b4 \uc124\uacc4\ud558\ub294 \ubc29\uc2dd\uc73c\ub85c \uc9c4\ud589\ub429\ub2c8\ub2e4.\n\uadf8 \ud6c4, 2\uac1c GPU\uc5d0\uc11c \uacc4\uc0b0\ub418\ub294 \uc911\uac04 \uc0b0\ucd9c\ubb3c \ud150\uc11c\uac12\uc744 \uc801\uc808\ud788 \ubc30\uce58\ud558\uae30 \uc704\ud5e4 \uc21c\uc804\ud30c \uba54\uc18c\ub4dc\ub97c \uc218\uc815\ud569\ub2c8\ub2e4.\n\n" ] }, { @@ -76,7 +76,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uc704\uc758 \uc608\uc81c\uc5d0\uc11c\ub294 \ub2e8\uc77c GPU\uc5d0 \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc744 \ud560\ub2f9\ud558\uc5ec \ud559\uc2b5\uc2dc\ud0a4\uae30\uc5d0\ub294 \ubaa8\ub378 \ud06c\uae30\uac00 \ub108\ubb34 \ud074 \ub54c \ubc1c\uc0dd\ud558\ub294 \ubb38\uc81c\ub97c \ud574\uacb0\ud558\ub294 \ubc29\ubc95\uc785\ub2c8\ub2e4. \n\ud558\uc9c0\ub9cc, \uc5ec\ub7ec\ubd84\uc740 \ub2e8\uc77c GPU\ub97c \uc774\uc6a9\ud560 \ub54c\ubcf4\ub2e4 \ud559\uc2b5 \uacfc\uc815\uc774 \uc624\ub798\uac78\ub9ac\uba70, \uc774\ub294 \uc5ec\ub7ec\ubd84\ub4e4\uc774 \uc774\ubbf8 \uc54c\uace0 \uc788\ub294 \ub0b4\uc6a9\uc774\uc5c8\uc744 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \n\uadf8 \uc774\uc720\ub294, \ub450 \uac1c\uc758 GPU\uac00 \ub3d9\uc2dc\uc5d0 \uacc4\uc0b0\ud558\ub294 \uac83\uc774 \uc544\ub2c8\ub77c 1\uac1c\uc758 GPU\ub294 \uacc4\uc0b0\ud558\uc9c0 \uc54a\uace0 \ub300\uae30\ud558\uace0 \uc788\uae30 \ub54c\ubb38\uc785\ub2c8\ub2e4.\n\ub610\ud55c, \ub450 \ubc88\uc9f8 \uce35 (layer2)\uc774 \ud560\ub2f9\ub41c \uccab \ubc88\uc9f8 GPU\uc5d0\uc11c \uacc4\uc0b0\ub41c \uacb0\uacfc\ub97c \uc138 \ubc88\uc9f8 \uce35 (layer3)\uc774 \ud560\ub2f9\ub41c \ub450 \ubc88\uc9f8 GPU\ub85c \ud150\uc11c\uac12\uc744 \ubcf5\uc0ac\ud558\uae30 \ub54c\ubb38\uc5d0 \uacc4\uc0b0 \uacfc\uc815\uc774 \ub354 \uae38\uc5b4\uc9c0\uac8c \ub429\ub2c8\ub2e4.\n\n\ucf54\ub4dc \uc2e4\ud589 \uc2dc\uac04\uc744 \uc815\ub7c9\uc801\uc73c\ub85c \uc0b4\ud3b4\ubcf4\uae30 \uc704\ud574 \uc2e4\ud5d8\uc744 \ud558\ub098 \ud574\ubd05\uc2dc\ub2e4. \uc785\ub825 \ud150\uc11c\uac12\uacfc \ub808\uc774\ube14\uac12\uc744 \ub79c\ub364\uc73c\ub85c \uc124\uc815\ud55c \ud6c4, \n\uc774\ubbf8 \uc874\uc7ac\ud558\ub294 torchvision.models.reset50() \uacfc, \ubaa8\ub378 \ubcd1\ub82c \ucc98\ub9ac\ub97c \uc9c4\ud589\ud55c ``ModelParallelResNet50`` \uc744 \ud1b5\ud574 \ud559\uc2b5\uc744 \uc9c4\ud589\ud569\ub2c8\ub2e4.\n\ud559\uc2b5 \uc9c4\ud589\uc744 \uc644\ub8cc\ud55c \ud6c4, \ub450 \ubaa8\ub378\ub4e4\uc740 \ub79c\ub364\uc73c\ub85c \uc0dd\uc131\ub41c \ub370\uc774\ud130\ub85c \ud559\uc2b5\uc744 \uc9c4\ud589\ud588\uae30 \ub54c\ubb38\uc5d0 \uc2e4\uc6a9\uc801\uc778 \uc608\uce21\uc744 \ud558\uc9c4 \ubabb\ud558\uc9c0\ub9cc, \ud559\uc2b5 \uc9c4\ud589 \uc2dc\uac04\uc744 \uc2e4\uc6a9\uc801\uc73c\ub85c \ube44\uad50\ud558\uc5ec \ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n" + "\uc704\uc758 \uc608\uc81c\uc5d0\uc11c\ub294 \ub2e8\uc77c GPU\uc5d0 \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc744 \ud560\ub2f9\ud558\uc5ec \ud559\uc2b5\uc2dc\ud0a4\uae30\uc5d0\ub294 \ubaa8\ub378 \ud06c\uae30\uac00 \ub108\ubb34 \ud074 \ub54c \ubc1c\uc0dd\ud558\ub294 \ubb38\uc81c\ub97c \ud574\uacb0\ud558\ub294 \ubc29\ubc95\uc785\ub2c8\ub2e4.\n\ud558\uc9c0\ub9cc, \uc5ec\ub7ec\ubd84\uc740 \ub2e8\uc77c GPU\ub97c \uc774\uc6a9\ud560 \ub54c\ubcf4\ub2e4 \ud559\uc2b5 \uacfc\uc815\uc774 \uc624\ub798\uac78\ub9ac\uba70, \uc774\ub294 \uc5ec\ub7ec\ubd84\ub4e4\uc774 \uc774\ubbf8 \uc54c\uace0 \uc788\ub294 \ub0b4\uc6a9\uc774\uc5c8\uc744 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uadf8 \uc774\uc720\ub294, \ub450 \uac1c\uc758 GPU\uac00 \ub3d9\uc2dc\uc5d0 \uacc4\uc0b0\ud558\ub294 \uac83\uc774 \uc544\ub2c8\ub77c 1\uac1c\uc758 GPU\ub294 \uacc4\uc0b0\ud558\uc9c0 \uc54a\uace0 \ub300\uae30\ud558\uace0 \uc788\uae30 \ub54c\ubb38\uc785\ub2c8\ub2e4.\n\ub610\ud55c, \ub450 \ubc88\uc9f8 \uce35 (layer2)\uc774 \ud560\ub2f9\ub41c \uccab \ubc88\uc9f8 GPU\uc5d0\uc11c \uacc4\uc0b0\ub41c \uacb0\uacfc\ub97c \uc138 \ubc88\uc9f8 \uce35 (layer3)\uc774 \ud560\ub2f9\ub41c \ub450 \ubc88\uc9f8 GPU\ub85c \ud150\uc11c\uac12\uc744 \ubcf5\uc0ac\ud558\uae30 \ub54c\ubb38\uc5d0 \uacc4\uc0b0 \uacfc\uc815\uc774 \ub354 \uae38\uc5b4\uc9c0\uac8c \ub429\ub2c8\ub2e4.\n\n\ucf54\ub4dc \uc2e4\ud589 \uc2dc\uac04\uc744 \uc815\ub7c9\uc801\uc73c\ub85c \uc0b4\ud3b4\ubcf4\uae30 \uc704\ud574 \uc2e4\ud5d8\uc744 \ud558\ub098 \ud574\ubd05\uc2dc\ub2e4. \uc785\ub825 \ud150\uc11c\uac12\uacfc \ub808\uc774\ube14\uac12\uc744 \ub79c\ub364\uc73c\ub85c \uc124\uc815\ud55c \ud6c4,\n\uc774\ubbf8 \uc874\uc7ac\ud558\ub294 torchvision.models.reset50() \uacfc, \ubaa8\ub378 \ubcd1\ub82c \ucc98\ub9ac\ub97c \uc9c4\ud589\ud55c ``ModelParallelResNet50`` \uc744 \ud1b5\ud574 \ud559\uc2b5\uc744 \uc9c4\ud589\ud569\ub2c8\ub2e4.\n\ud559\uc2b5 \uc9c4\ud589\uc744 \uc644\ub8cc\ud55c \ud6c4, \ub450 \ubaa8\ub378\ub4e4\uc740 \ub79c\ub364\uc73c\ub85c \uc0dd\uc131\ub41c \ub370\uc774\ud130\ub85c \ud559\uc2b5\uc744 \uc9c4\ud589\ud588\uae30 \ub54c\ubb38\uc5d0 \uc2e4\uc6a9\uc801\uc778 \uc608\uce21\uc744 \ud558\uc9c4 \ubabb\ud558\uc9c0\ub9cc, \ud559\uc2b5 \uc9c4\ud589 \uc2dc\uac04\uc744 \uc2e4\uc6a9\uc801\uc73c\ub85c \ube44\uad50\ud558\uc5ec \ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n" ] }, { @@ -94,7 +94,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uc704\uc5d0\uc11c \uc815\uc758\ud55c ``train(model)`` \uba54\uc18c\ub4dc\ub294 nn.MSELoss (Mean Squared Error ; \ud3c9\uade0 \uc81c\uacf1 \uc624\ucc28) \ub85c \uc190\uc2e4 \ud568\uc218\ub97c \uc815\uc758\ud558\uc5ec \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc744 \ud559\uc2b5\ud558\ub294 \uac83\uc744 \uc758\ubbf8\ud569\ub2c8\ub2e4. \n\uadf8\ub9ac\uace0, ``optim.SGD`` \uba54\uc18c\ub4dc\ub294 \ucd5c\uc801\ud654 \ubc29\uc2dd\uc744 \uc758\ubbf8\ud569\ub2c8\ub2e4. \uc704 \ubc29\uc2dd\uc740 128 * 128 \ud06c\uae30\uc758 \uc774\ubbf8\uc9c0\uac00 120\uac1c\ub85c \uad6c\uc131\ub41c \ubc30\uce58 \ub370\uc774\ud130\uac00 3\uac1c \uc874\uc7ac\ud558\ub294 \uc0c1\ud669\uc744 \ubaa8\ubc29\ud558\uae30 \uc704\ud574 \ub79c\ub364\uc73c\ub85c \uc0dd\uc131\ud558\uc600\uc2b5\ub2c8\ub2e4. \n\uadf8\ub9ac\uace0\ub098\uc11c, \uc6b0\ub9ac\ub294 ``timeit`` \uc744 \uc774\uc6a9\ud558\uc5ec ``train(model)`` \uba54\uc18c\ub4dc\ub97c 10\ud68c \uc2e4\ud589\ud558\uc5ec \ud559\uc2b5\uc744 \uc9c4\ud589\ud558\uace0, \ud559\uc2b5 \uc2e4\ud589 \uc2dc\uac04\uc5d0 \ub300\ud574\uc11c \ud45c\uc900 \ud3b8\ucc28\uac12\uc744 \ubc18\uc601\ud558\ub294 \uc774\ubbf8\uc9c0\ub97c \uc0dd\uc131\ud558\uc5ec \uc800\uc7a5\ud569\ub2c8\ub2e4.\n\n" + "\uc704\uc5d0\uc11c \uc815\uc758\ud55c ``train(model)`` \uba54\uc18c\ub4dc\ub294 nn.MSELoss (Mean Squared Error ; \ud3c9\uade0 \uc81c\uacf1 \uc624\ucc28) \ub85c \uc190\uc2e4 \ud568\uc218\ub97c \uc815\uc758\ud558\uc5ec \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc744 \ud559\uc2b5\ud558\ub294 \uac83\uc744 \uc758\ubbf8\ud569\ub2c8\ub2e4.\n\uadf8\ub9ac\uace0, ``optim.SGD`` \uba54\uc18c\ub4dc\ub294 \ucd5c\uc801\ud654 \ubc29\uc2dd\uc744 \uc758\ubbf8\ud569\ub2c8\ub2e4. \uc704 \ubc29\uc2dd\uc740 128 * 128 \ud06c\uae30\uc758 \uc774\ubbf8\uc9c0\uac00 120\uac1c\ub85c \uad6c\uc131\ub41c \ubc30\uce58 \ub370\uc774\ud130\uac00 3\uac1c \uc874\uc7ac\ud558\ub294 \uc0c1\ud669\uc744 \ubaa8\ubc29\ud558\uae30 \uc704\ud574 \ub79c\ub364\uc73c\ub85c \uc0dd\uc131\ud558\uc600\uc2b5\ub2c8\ub2e4.\n\uadf8\ub9ac\uace0\ub098\uc11c, \uc6b0\ub9ac\ub294 ``timeit`` \uc744 \uc774\uc6a9\ud558\uc5ec ``train(model)`` \uba54\uc18c\ub4dc\ub97c 10\ud68c \uc2e4\ud589\ud558\uc5ec \ud559\uc2b5\uc744 \uc9c4\ud589\ud558\uace0, \ud559\uc2b5 \uc2e4\ud589 \uc2dc\uac04\uc5d0 \ub300\ud574\uc11c \ud45c\uc900 \ud3b8\ucc28\uac12\uc744 \ubc18\uc601\ud558\ub294 \uc774\ubbf8\uc9c0\ub97c \uc0dd\uc131\ud558\uc5ec \uc800\uc7a5\ud569\ub2c8\ub2e4.\n\n" ] }, { @@ -123,14 +123,14 @@ }, "outputs": [], "source": [ - "# \uc2e4\ud5d8 \uacb0\uacfc, \ubaa8\ub378 \ubcd1\ub82c \ucca0\ub9ac\ud558\uc5ec \ud559\uc2b5\ud558\ub294 \uc2dc\uac04\uc774 \ub2e8\uc77c GPU\ub85c \ud559\uc2b5\ud558\ub294 \uc2dc\uac04\ubcf4\ub2e4 \uc57d 7% ``4.02/3.75-1=7%``\uc815\ub3c4\n# \uc624\ub798 \uac78\ub9ac\ub294 \uac83\uc744 \ud655\uc778\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uadf8\ub7ec\ubbc0\ub85c, \uc21c\uc804\ud30c\uc640 \uc5ed\uc804\ud30c\ub97c \uc9c4\ud589\ud558\uba74\uc11c GPU \uac04 \ud150\uc11c\uac12\ub4e4\uc774\n# \ubcf5\uc81c\ub418\uc5b4 \uc774\uc6a9\ud558\ub294 \uc2dc\uac04\uc774 \uc57d 7%\uc815\ub3c4 \uc18c\uc694\ub418\ub294 \uac83\uc73c\ub85c \uacb0\ub860\uc9c0\uc744 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \ud559\uc2b5\ud558\ub294 \uacfc\uc815 \uc18d\uc5d0\uc11c\n# 2\uac1c\uc758 GPU \uc911 1\uac1c\uc758 GPU\uac00 \uacc4\uc0b0\ud558\uc9c0 \uc54a\uace0 \ub300\uae30\ud558\uace0 \uc788\uae30 \ub54c\ubb38\uc5d0, \uc774\ub97c \ud574\uacb0\ud558\uc5ec \n# \ud559\uc2b5 \uc2dc\uac04\uc744 \ube60\ub974\uac8c \uac1c\uc120\uc2dc\ud0ac \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uadf8 \uc911 \ud55c \uac00\uc9c0 \ubc29\ubc95\uc740, \ud559\uc2b5 \ub2e8\uc704\uc778 \ubbf8\ub2c8 \ubc30\uce58 1\uac1c\uc758 \ub370\uc774\ud130\ub97c\n# 2\uac1c\ub85c \ubd84\ud560\ud558\ub294 \ud30c\uc774\ud504\ub77c\uc778\uc744 \uc0dd\uc131\ud558\uc5ec, \ubd84\ud560\ub41c \uccab \ubc88\uc9f8 \ub370\uc774\ud130\uac00 \uccab \ubc88\uc9f8 \uce35\uc744 \ud1b5\uacfc\ud558\uc5ec \ub450 \ubc88\uc9f8 \uce35\uc73c\ub85c\n# \ubcf5\uc81c\ub418\uace0, \ub450 \ubc88\uc9f8 \uce35\uc744 \ud1b5\uacfc\ud560 \ub54c, \ub450\ubc88\uc7ac\ub85c \ubd84\ud560\ub41c \ub370\uc774\ud130\uac00 \uccab \ubc88\uca30 \uce35\uc744 \ud1b5\ud574 \uacc4\uc0b0\ub418\ub294 \ubc29\uc2dd\uc73c\ub85c \uc124\uc815\ud558\ub294 \uac83\uc785\ub2c8\ub2e4.\n# \uc774\ub7ec\ud55c \ubc29\ubc95\uc744 \ud1b5\ud574\uc11c 2\uac1c\uc758 GPU\uac00 2\uac1c\ub85c \ubd84\ud560\ub41c \ub370\uc774\ud130\ub97c \ub3d9\uc2dc\uc5d0 \ucc98\ub9ac\ud560 \uc218 \uc788\uc73c\uba70 \ud559\uc2b5 \uc2dc\uac04\uc744 \ub2e8\ucd95\uc2dc\ud0ac \uc218 \uc788\uc2b5\ub2c8\ub2e4." + "# \uc2e4\ud5d8 \uacb0\uacfc, \ubaa8\ub378 \ubcd1\ub82c \ucca0\ub9ac\ud558\uc5ec \ud559\uc2b5\ud558\ub294 \uc2dc\uac04\uc774 \ub2e8\uc77c GPU\ub85c \ud559\uc2b5\ud558\ub294 \uc2dc\uac04\ubcf4\ub2e4 \uc57d 7% ``4.02/3.75-1=7%``\uc815\ub3c4\n# \uc624\ub798 \uac78\ub9ac\ub294 \uac83\uc744 \ud655\uc778\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uadf8\ub7ec\ubbc0\ub85c, \uc21c\uc804\ud30c\uc640 \uc5ed\uc804\ud30c\ub97c \uc9c4\ud589\ud558\uba74\uc11c GPU \uac04 \ud150\uc11c\uac12\ub4e4\uc774\n# \ubcf5\uc81c\ub418\uc5b4 \uc774\uc6a9\ud558\ub294 \uc2dc\uac04\uc774 \uc57d 7%\uc815\ub3c4 \uc18c\uc694\ub418\ub294 \uac83\uc73c\ub85c \uacb0\ub860\uc9c0\uc744 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \ud559\uc2b5\ud558\ub294 \uacfc\uc815 \uc18d\uc5d0\uc11c\n# 2\uac1c\uc758 GPU \uc911 1\uac1c\uc758 GPU\uac00 \uacc4\uc0b0\ud558\uc9c0 \uc54a\uace0 \ub300\uae30\ud558\uace0 \uc788\uae30 \ub54c\ubb38\uc5d0, \uc774\ub97c \ud574\uacb0\ud558\uc5ec\n# \ud559\uc2b5 \uc2dc\uac04\uc744 \ube60\ub974\uac8c \uac1c\uc120\uc2dc\ud0ac \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uadf8 \uc911 \ud55c \uac00\uc9c0 \ubc29\ubc95\uc740, \ud559\uc2b5 \ub2e8\uc704\uc778 \ubbf8\ub2c8 \ubc30\uce58 1\uac1c\uc758 \ub370\uc774\ud130\ub97c\n# 2\uac1c\ub85c \ubd84\ud560\ud558\ub294 \ud30c\uc774\ud504\ub77c\uc778\uc744 \uc0dd\uc131\ud558\uc5ec, \ubd84\ud560\ub41c \uccab \ubc88\uc9f8 \ub370\uc774\ud130\uac00 \uccab \ubc88\uc9f8 \uce35\uc744 \ud1b5\uacfc\ud558\uc5ec \ub450 \ubc88\uc9f8 \uce35\uc73c\ub85c\n# \ubcf5\uc81c\ub418\uace0, \ub450 \ubc88\uc9f8 \uce35\uc744 \ud1b5\uacfc\ud560 \ub54c, \ub450\ubc88\uc7ac\ub85c \ubd84\ud560\ub41c \ub370\uc774\ud130\uac00 \uccab \ubc88\uca30 \uce35\uc744 \ud1b5\ud574 \uacc4\uc0b0\ub418\ub294 \ubc29\uc2dd\uc73c\ub85c \uc124\uc815\ud558\ub294 \uac83\uc785\ub2c8\ub2e4.\n# \uc774\ub7ec\ud55c \ubc29\ubc95\uc744 \ud1b5\ud574\uc11c 2\uac1c\uc758 GPU\uac00 2\uac1c\ub85c \ubd84\ud560\ub41c \ub370\uc774\ud130\ub97c \ub3d9\uc2dc\uc5d0 \ucc98\ub9ac\ud560 \uc218 \uc788\uc73c\uba70 \ud559\uc2b5 \uc2dc\uac04\uc744 \ub2e8\ucd95\uc2dc\ud0ac \uc218 \uc788\uc2b5\ub2c8\ub2e4." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "\uc785\ub825 \ud150\uc11c\uac12\uc744 \ubd84\ud560\ud558\ub294 \ud30c\uc774\ud504\ub77c\uc778\uc744 \uc124\uacc4\ud558\uc5ec \ud559\uc2b5 \uc2dc\uac04\uc744 \ub2e8\ucd95\ud558\ub294 \ubc29\ubc95\uc5d0 \ub300\ud55c \uc608\uc81c\n-----------------------------\n\n\uc544\ub798\uc5d0 \uc788\ub294 \uc2e4\ud5d8\uc740, 120\uac1c\uc758 \uc774\ubbf8\uc9c0\ub85c \uad6c\uc131\ub41c 1\uac1c\uc758 \ubbf8\ub2c8 \ubc30\uce58 \ub370\uc774\ud130\ub97c 20\uac1c\uc529 \ub098\ub204\uc5b4 \uc9c4\ud589\ud558\ub294 \n\uacfc\uc815\uc785\ub2c8\ub2e4. \uc544\ub798\uc758 \uacfc\uc815\uc744 \uc2e4\ud589\ud560 \ub54c, PyTorch\uac00 CUDA \uc5f0\uc0b0\uc744 \ube44\ub3d9\uae30\uc801\uc73c\ub85c \uc774\uc6a9\ud558\uae30 \ub54c\ubb38\uc5d0, \n\ud504\ub85c\uc138\uc2a4\ub97c \uc2e4\ud589\ud558\ub294 \uc2a4\ub808\ub4dc\ub97c \uc5ec\ub7ec\uac1c \uc0dd\uc131\ud560 \ud544\uc694\uac00 \uc5c6\uc2b5\ub2c8\ub2e4.\n\n" + "\uc785\ub825 \ud150\uc11c\uac12\uc744 \ubd84\ud560\ud558\ub294 \ud30c\uc774\ud504\ub77c\uc778\uc744 \uc124\uacc4\ud558\uc5ec \ud559\uc2b5 \uc2dc\uac04\uc744 \ub2e8\ucd95\ud558\ub294 \ubc29\ubc95\uc5d0 \ub300\ud55c \uc608\uc81c\n---------------------------------------------------------------------------------------\n\n\uc544\ub798\uc5d0 \uc788\ub294 \uc2e4\ud5d8\uc740, 120\uac1c\uc758 \uc774\ubbf8\uc9c0\ub85c \uad6c\uc131\ub41c 1\uac1c\uc758 \ubbf8\ub2c8 \ubc30\uce58 \ub370\uc774\ud130\ub97c 20\uac1c\uc529 \ub098\ub204\uc5b4 \uc9c4\ud589\ud558\ub294\n\uacfc\uc815\uc785\ub2c8\ub2e4. \uc544\ub798\uc758 \uacfc\uc815\uc744 \uc2e4\ud589\ud560 \ub54c, PyTorch\uac00 CUDA \uc5f0\uc0b0\uc744 \ube44\ub3d9\uae30\uc801\uc73c\ub85c \uc774\uc6a9\ud558\uae30 \ub54c\ubb38\uc5d0,\n\ud504\ub85c\uc138\uc2a4\ub97c \uc2e4\ud589\ud558\ub294 \uc2a4\ub808\ub4dc\ub97c \uc5ec\ub7ec\uac1c \uc0dd\uc131\ud560 \ud544\uc694\uac00 \uc5c6\uc2b5\ub2c8\ub2e4.\n\n" ] }, { @@ -141,21 +141,21 @@ }, "outputs": [], "source": [ - "class PipelineParallelResNet50(ModelParallelResNet50):\n def __init__(self, split_size=20, *args, **kwargs):\n super(PipelineParallelResNet50, self).__init__(*args, **kwargs)\n self.split_size = split_size\n\n def forward(self, x):\n splits = iter(x.split(self.split_size, dim=0))\n s_next = next(splits)\n s_prev = self.seq1(s_next).to('cuda:1')\n ret = []\n\n for s_next in splits:\n # A. s_prev\ub294 \ub450 \ubc88\uc9f8 GPU\uc5d0\uc11c \uc2e4\ud589\ub429\ub2c8\ub2e4.\n s_prev = self.seq2(s_prev)\n ret.append(self.fc(s_prev.view(s_prev.size(0), -1)))\n\n # B. s_next\ub294 A.\uc640 \ub3d9\uc2dc\uc5d0 \uc9c4\ud589\ub418\uba74\uc11c \uccab \ubc88\uc9f8 GPU\uc5d0\uc11c \uc2e4\ud589\ub429\ub2c8\ub2e4. \n s_prev = self.seq1(s_next).to('cuda:1')\n\n s_prev = self.seq2(s_prev)\n ret.append(self.fc(s_prev.view(s_prev.size(0), -1)))\n\n return torch.cat(ret)\n\n\nsetup = \"model = PipelineParallelResNet50()\"\npp_run_times = timeit.repeat(\n stmt, setup, number=1, repeat=num_repeat, globals=globals())\npp_mean, pp_std = np.mean(pp_run_times), np.std(pp_run_times)\n\nplot([mp_mean, rn_mean, pp_mean],\n [mp_std, rn_std, pp_std],\n ['Model Parallel', 'Single GPU', 'Pipelining Model Parallel'],\n 'mp_vs_rn_vs_pp.png')" + "class PipelineParallelResNet50(ModelParallelResNet50):\n def __init__(self, split_size=20, *args, **kwargs):\n super(PipelineParallelResNet50, self).__init__(*args, **kwargs)\n self.split_size = split_size\n\n def forward(self, x):\n splits = iter(x.split(self.split_size, dim=0))\n s_next = next(splits)\n s_prev = self.seq1(s_next).to('cuda:1')\n ret = []\n\n for s_next in splits:\n # A. s_prev\ub294 \ub450 \ubc88\uc9f8 GPU\uc5d0\uc11c \uc2e4\ud589\ub429\ub2c8\ub2e4.\n s_prev = self.seq2(s_prev)\n ret.append(self.fc(s_prev.view(s_prev.size(0), -1)))\n\n # B. s_next\ub294 A.\uc640 \ub3d9\uc2dc\uc5d0 \uc9c4\ud589\ub418\uba74\uc11c \uccab \ubc88\uc9f8 GPU\uc5d0\uc11c \uc2e4\ud589\ub429\ub2c8\ub2e4.\n s_prev = self.seq1(s_next).to('cuda:1')\n\n s_prev = self.seq2(s_prev)\n ret.append(self.fc(s_prev.view(s_prev.size(0), -1)))\n\n return torch.cat(ret)\n\n\nsetup = \"model = PipelineParallelResNet50()\"\npp_run_times = timeit.repeat(\n stmt, setup, number=1, repeat=num_repeat, globals=globals())\npp_mean, pp_std = np.mean(pp_run_times), np.std(pp_run_times)\n\nplot([mp_mean, rn_mean, pp_mean],\n [mp_std, rn_std, pp_std],\n ['Model Parallel', 'Single GPU', 'Pipelining Model Parallel'],\n 'mp_vs_rn_vs_pp.png')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "GPU \uac04 \ud150\uc11c\uac12\uc774 \ubcf5\uc0ac\ub418\ub294 \uac83\uc740 \ud604\uc7ac \uacc4\uc0b0\ub418\uace0 \uc788\ub294 \uc18c\uc2a4\uac12\uacfc, \uc18c\uc2a4\uac12\uc758 \ubaa9\uc801\uc9c0 GPU \uac04 \uc5f0\uc0b0\ub418\uace0 \uc788\ub294\n\uc2a4\ud2b8\ub9bc\uacfc \ub3d9\uae30\ud654\ub418\ub294 \uac83\uc744 \uc8fc\uc758\ud558\uc138\uc694. \ub9cc\uc57d \uc5ec\ub7ec \uc2a4\ud2b8\ub9bc\uc744 \uc0dd\uc131\ud558\uc5ec \uc9c4\ud589\ud558\uace0 \uc788\ub2e4\uba74, GPU \uac04 \ud150\uc11c\uac12\uc774\n\uc815\uc0c1\uc801\uc73c\ub85c \ubcf5\uc0ac\ub418\uc5b4 \uacc4\uc0b0\ub418\uace0 \uc788\ub294\uc9c0 \uaf2d \ud655\uc778\ud574\uc57c \ud569\ub2c8\ub2e4. \ub9cc\uc57d \ubcf5\uc0ac\ub418\ub294 \uacfc\uc815 \uc911\uc5d0 \uc18c\uc2a4\uac12\uc744 \uc774\uc6a9\ud558\uac70\ub098,\nGPU\uc758 \ud150\uc11c\uac12\uc744 \uc77d\uac70\ub098 \uc4f0\ub294 \uac83\uc740 \uc62c\ubc14\ub974\uac8c \uacc4\uc0b0\ub418\uc9c0 \uc54a\uc744 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc704\uc758 \uc608\uc81c\uc5d0\uc11c\ub294 \uc18c\uc2a4\uac12 \ubc0f GPU \n\ud150\uc11c\uac12\uc744 \uae30\ubcf8 \uc2a4\ud2b8\ub9bc\ub9cc \uc774\uc6a9\ud558\uc5ec \uc9c4\ud589\ud558\ubbc0\ub85c \ucd94\uac00\uc801\uc778 \ub3d9\uae30\ud654 \uacfc\uc815\uc744 \uc9c4\ud589\ud560 \ud544\uc694\ub294 \uc5c6\uc2b5\ub2c8\ub2e4.\n\n" + "GPU \uac04 \ud150\uc11c\uac12\uc774 \ubcf5\uc0ac\ub418\ub294 \uac83\uc740 \ud604\uc7ac \uacc4\uc0b0\ub418\uace0 \uc788\ub294 \uc18c\uc2a4\uac12\uacfc, \uc18c\uc2a4\uac12\uc758 \ubaa9\uc801\uc9c0 GPU \uac04 \uc5f0\uc0b0\ub418\uace0 \uc788\ub294\n\uc2a4\ud2b8\ub9bc\uacfc \ub3d9\uae30\ud654\ub418\ub294 \uac83\uc744 \uc8fc\uc758\ud558\uc138\uc694. \ub9cc\uc57d \uc5ec\ub7ec \uc2a4\ud2b8\ub9bc\uc744 \uc0dd\uc131\ud558\uc5ec \uc9c4\ud589\ud558\uace0 \uc788\ub2e4\uba74, GPU \uac04 \ud150\uc11c\uac12\uc774\n\uc815\uc0c1\uc801\uc73c\ub85c \ubcf5\uc0ac\ub418\uc5b4 \uacc4\uc0b0\ub418\uace0 \uc788\ub294\uc9c0 \uaf2d \ud655\uc778\ud574\uc57c \ud569\ub2c8\ub2e4. \ub9cc\uc57d \ubcf5\uc0ac\ub418\ub294 \uacfc\uc815 \uc911\uc5d0 \uc18c\uc2a4\uac12\uc744 \uc774\uc6a9\ud558\uac70\ub098,\nGPU\uc758 \ud150\uc11c\uac12\uc744 \uc77d\uac70\ub098 \uc4f0\ub294 \uac83\uc740 \uc62c\ubc14\ub974\uac8c \uacc4\uc0b0\ub418\uc9c0 \uc54a\uc744 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc704\uc758 \uc608\uc81c\uc5d0\uc11c\ub294 \uc18c\uc2a4\uac12 \ubc0f GPU\n\ud150\uc11c\uac12\uc744 \uae30\ubcf8 \uc2a4\ud2b8\ub9bc\ub9cc \uc774\uc6a9\ud558\uc5ec \uc9c4\ud589\ud558\ubbc0\ub85c \ucd94\uac00\uc801\uc778 \ub3d9\uae30\ud654 \uacfc\uc815\uc744 \uc9c4\ud589\ud560 \ud544\uc694\ub294 \uc5c6\uc2b5\ub2c8\ub2e4.\n\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - ".. figure:: /_static/img/model-parallel-images/mp_vs_rn_vs_pp.png\n :alt:\n\n\ud30c\uc774\ud504\ub77c\uc778\uc744 \uc774\uc6a9\ud558\uc5ec \ubbf8\ub2c8 \ubc30\uce58 \ub0b4 \ub370\uc774\ud130\ub97c \ubd84\ud560\ud558\uc5ec \uc801\uc6a9\ud558\uc600\uc744 \ub54c, ResNet50 \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc758\n\ud559\uc2b5 \uc2dc\uac04\uc774 \uc57d 49% ``3.75/2.51-1=49%`` \uc815\ub3c4 \ub2e8\ucd95\ub41c \uac83\uc744 \uc774\ubc88 \uc2e4\ud5d8\uc744 \ud1b5\ud574 \ud655\uc778\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \ud558\uc9c0\ub9cc, \uc774\uc0c1\uc801\uc73c\ub85c\n\ud559\uc2b5 \uc2dc\uac04\uc774 2\ubc30 \ub2e8\ucd95\ub418\ub294 \uac83\uc5d0 \ube44\ud574 \ub2e4\uc18c \uc801\uac8c \ud559\uc2b5 \uc2dc\uac04\uc774 \ub2e8\ucd95\ub418\uc5c8\uc2b5\ub2c8\ub2e4. \ud30c\uc774\ud504\ub77c\uc778\uc744 \uc774\uc6a9\ud560 \ub54c,\n``split_sizes`` \ub9e4\uac1c\ubcc0\uc218\ub97c \ub3c4\uc785\ud558\uc600\uae30 \ub54c\ubb38\uc5d0, \ud30c\uc774\ud504\ub77c\uc778\uc744 \uc774\uc6a9\ud558\ub294 \uac83\uc774 \ud559\uc2b5 \uc2dc\uac04 \ub2e8\ucd95\uc5d0 \uc5bc\ub9c8\ub098\n\uc601\ud5a5\uc744 \ubbf8\ucce4\ub294\uc9c0 \ubd88\ubd84\uba85\ud569\ub2c8\ub2e4. \uc9c1\uad00\uc801\uc73c\ub85c \uc0dd\uac01\ud558\uc600\uc744 \ub54c, ``split_sizes`` \ub9e4\uac1c\ubcc0\uc218 \uac12\uc744 \uc791\uac8c \uc124\uc815\ud55c\ub2e4\uba74,\n\uc544\uc8fc \uc18c\uaddc\ubaa8\uc758 CUDA \uc5f0\uc0b0\uc774 \ub9ce\uc774 \uc9c4\ud589\ub418\uace0, ``split_sizes`` \ub9e4\uac1c\ubcc0\uc218 \uac12\uc744 \ud06c\uac8c \uc124\uc815\ud55c\ub2e4\uba74, \uccab \ubc88\uc9f8\uc640 \n\ub9c8\uc9c0\ub9c9 \ubd84\ub9ac\ub420 \ub54c \ube44\uad50\uc801 \uae34 \uc2dc\uac04 \ub3d9\uc548 CUDA \uc5f0\uc0b0\uc774 \uc774\ub8e8\uc5b4\uc9c0\uac8c \ub429\ub2c8\ub2e4. \ub458 \ub2e4 \ucd5c\uc801\uc758 \uc124\uc815\uc774 \uc544\ub2d9\ub2c8\ub2e4.\n\ub530\ub77c\uc11c, ``split_sizes`` \ub9e4\uac1c\ubcc0\uc218 \uac12\uc744 \ucd5c\uc801\uc73c\ub85c \uc124\uc815\ud558\uc600\uc744 \ub54c, \ud559\uc2b5 \uc2dc\uac04 \uacfc\uc815\uc774 \ub2e8\ucd95\ub420 \uc218 \uc788\uc744 \uac83\uc774\ub77c\n\uae30\ub300\ub429\ub2c8\ub2e4. ``split_sizes`` \ub9e4\uac1c\ubcc0\uc218 \uac12\uc744 \uc870\uc815\ud558\uc5ec \uc2e4\ud5d8\ud558\uba74\uc11c \ucd5c\uc801\uc758 \uac12\uc744 \ucc3e\uc544\ubd05\uc2dc\ub2e4.\n\n" + ".. figure:: /_static/img/model-parallel-images/mp_vs_rn_vs_pp.png\n :alt:\n\n\ud30c\uc774\ud504\ub77c\uc778\uc744 \uc774\uc6a9\ud558\uc5ec \ubbf8\ub2c8 \ubc30\uce58 \ub0b4 \ub370\uc774\ud130\ub97c \ubd84\ud560\ud558\uc5ec \uc801\uc6a9\ud558\uc600\uc744 \ub54c, ResNet50 \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc758\n\ud559\uc2b5 \uc2dc\uac04\uc774 \uc57d 49% ``3.75/2.51-1=49%`` \uc815\ub3c4 \ub2e8\ucd95\ub41c \uac83\uc744 \uc774\ubc88 \uc2e4\ud5d8\uc744 \ud1b5\ud574 \ud655\uc778\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \ud558\uc9c0\ub9cc, \uc774\uc0c1\uc801\uc73c\ub85c\n\ud559\uc2b5 \uc2dc\uac04\uc774 2\ubc30 \ub2e8\ucd95\ub418\ub294 \uac83\uc5d0 \ube44\ud574 \ub2e4\uc18c \uc801\uac8c \ud559\uc2b5 \uc2dc\uac04\uc774 \ub2e8\ucd95\ub418\uc5c8\uc2b5\ub2c8\ub2e4. \ud30c\uc774\ud504\ub77c\uc778\uc744 \uc774\uc6a9\ud560 \ub54c,\n``split_sizes`` \ub9e4\uac1c\ubcc0\uc218\ub97c \ub3c4\uc785\ud558\uc600\uae30 \ub54c\ubb38\uc5d0, \ud30c\uc774\ud504\ub77c\uc778\uc744 \uc774\uc6a9\ud558\ub294 \uac83\uc774 \ud559\uc2b5 \uc2dc\uac04 \ub2e8\ucd95\uc5d0 \uc5bc\ub9c8\ub098\n\uc601\ud5a5\uc744 \ubbf8\ucce4\ub294\uc9c0 \ubd88\ubd84\uba85\ud569\ub2c8\ub2e4. \uc9c1\uad00\uc801\uc73c\ub85c \uc0dd\uac01\ud558\uc600\uc744 \ub54c, ``split_sizes`` \ub9e4\uac1c\ubcc0\uc218 \uac12\uc744 \uc791\uac8c \uc124\uc815\ud55c\ub2e4\uba74,\n\uc544\uc8fc \uc18c\uaddc\ubaa8\uc758 CUDA \uc5f0\uc0b0\uc774 \ub9ce\uc774 \uc9c4\ud589\ub418\uace0, ``split_sizes`` \ub9e4\uac1c\ubcc0\uc218 \uac12\uc744 \ud06c\uac8c \uc124\uc815\ud55c\ub2e4\uba74, \uccab \ubc88\uc9f8\uc640\n\ub9c8\uc9c0\ub9c9 \ubd84\ub9ac\ub420 \ub54c \ube44\uad50\uc801 \uae34 \uc2dc\uac04 \ub3d9\uc548 CUDA \uc5f0\uc0b0\uc774 \uc774\ub8e8\uc5b4\uc9c0\uac8c \ub429\ub2c8\ub2e4. \ub458 \ub2e4 \ucd5c\uc801\uc758 \uc124\uc815\uc774 \uc544\ub2d9\ub2c8\ub2e4.\n\ub530\ub77c\uc11c, ``split_sizes`` \ub9e4\uac1c\ubcc0\uc218 \uac12\uc744 \ucd5c\uc801\uc73c\ub85c \uc124\uc815\ud558\uc600\uc744 \ub54c, \ud559\uc2b5 \uc2dc\uac04 \uacfc\uc815\uc774 \ub2e8\ucd95\ub420 \uc218 \uc788\uc744 \uac83\uc774\ub77c\n\uae30\ub300\ub429\ub2c8\ub2e4. ``split_sizes`` \ub9e4\uac1c\ubcc0\uc218 \uac12\uc744 \uc870\uc815\ud558\uc5ec \uc2e4\ud5d8\ud558\uba74\uc11c \ucd5c\uc801\uc758 \uac12\uc744 \ucc3e\uc544\ubd05\uc2dc\ub2e4.\n\n" ] }, { @@ -173,7 +173,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - ".. figure:: /_static/img/model-parallel-images/split_size_tradeoff.png\n :alt:\n\n\uc2e4\ud5d8 \uacb0\uacfc, ``split_size`` \ub9e4\uac1c\ubcc0\uc218\uac12\uc744 12\ub85c \uc124\uc815\ud558\uc600\uc744 \ub54c, \ud559\uc2b5 \uc2dc\uac04\uc774 54% \uc218\uc900\uc73c\ub85c \n\uac00\uc7a5 \ub9ce\uc774 \ub2e8\ucd95\ub418\uc5c8\uc2b5\ub2c8\ub2e4. \uc544\uc9c1 \ud559\uc2b5 \uc2dc\uac04\uc744 \ub354 \ub2e8\ucd95\uc2dc\ud0ac \uc218 \uc788\ub294 \ubc29\ubc95\uc740 \ub2e4\uc591\ud558\uac8c \uc874\uc7ac\ud569\ub2c8\ub2e4.\n\uc608\ub97c \ub4e4\uc5b4, \uccab \ubc88\uc9f8 GPU\uc5d0\uc11c \ubaa8\ub4e0 \uc5f0\uc0b0\uacfc\uc815\uc774 \uae30\ubcf8\uc73c\ub85c \uc124\uc815\ub418\uc5b4 \uc9c4\ud589\ub429\ub2c8\ub2e4. \uc774\ub294 \ubbf8\ub2c8\ubc30\uce58 \ubd84\ud560 \uacfc\uc815 \uc911,\n\ud604\uc7ac \uc9c4\ud589\ub418\ub294 \uacfc\uc815\uc758 \ub2e4\uc74c \ub2e8\uacc4\ub294 \ud604\uc7ac \uc9c4\ud589\ub418\ub294 \uacfc\uc815\uacfc \ub3d9\uc2dc\uc5d0 \ubcf5\uc81c\uac00 \uc774\ub8e8\uc5b4\uc9c8 \uc218 \uc5c6\ub294 \uac83\uc744 \uc758\ubbf8\ud569\ub2c8\ub2e4.\n\uadf8\ub7ec\ub098, \uc774\uc804\uacfc \ub2e4\uc74c \ub2e8\uacc4\uc758 \ubd84\ud560\uacfc\uc815\uc774 \ub2e4\ub978 \ud150\uc11c\uac12\uc744 \uc774\uc6a9\ud558\uae30 \ub54c\ubb38\uc5d0, \ub2e4\ub978 \uacc4\uc0b0\uacfc \uc911\ubcf5\ub418\uc5b4 \uc9c4\ud589\ub418\uc5b4\ub3c4\n\ubb38\uc81c\uac00 \uc5c6\uc2b5\ub2c8\ub2e4. \uc774\uc5d0 \ub300\ud574\uc11c, 2\uac1c GPU\uc5d0 \uc5ec\ub7ec\uac1c\uc758 \uc2a4\ud2b8\ub9bc\uc744 \uc0ac\uc6a9\ud558\ub294 \uac83\uc774 \ud544\uc694\ud558\uba70, \uc11c\ub85c \ub2e4\ub978 \uc11c\ube0c \ub124\ud2b8\uc6cc\ud06c \n\uad6c\uc870\uac00 \uc11c\ub85c \ub2e4\ub978 \uc2a4\ud2b8\ub9bc\uc744 \uad00\ub9ac\ud558\ub294 \uc804\ub7b5\uc774 \uc694\uad6c\ub429\ub2c8\ub2e4. \ubaa8\ub378 \ubcd1\ub82c \ucc98\ub9ac\uc5d0 \ub300\ud574\uc11c \uc5ec\ub7ec \uc2a4\ud2b8\ub9bc\uc744 \uc0ac\uc6a9\ud558\ub294 \ubc29\ubc95\uc774 \n\uc77c\ubc18\uc801\uc744\ub85c \uc874\uc7ac\ud558\uc9c0 \uc54a\uae30 \ub54c\ubb38\uc5d0 \uc774\ubc88 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 \uc124\uba85\ud558\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4.\n\n" + ".. figure:: /_static/img/model-parallel-images/split_size_tradeoff.png\n :alt:\n\n\uc2e4\ud5d8 \uacb0\uacfc, ``split_size`` \ub9e4\uac1c\ubcc0\uc218\uac12\uc744 12\ub85c \uc124\uc815\ud558\uc600\uc744 \ub54c, \ud559\uc2b5 \uc2dc\uac04\uc774 54% \uc218\uc900\uc73c\ub85c\n\uac00\uc7a5 \ub9ce\uc774 \ub2e8\ucd95\ub418\uc5c8\uc2b5\ub2c8\ub2e4. \uc544\uc9c1 \ud559\uc2b5 \uc2dc\uac04\uc744 \ub354 \ub2e8\ucd95\uc2dc\ud0ac \uc218 \uc788\ub294 \ubc29\ubc95\uc740 \ub2e4\uc591\ud558\uac8c \uc874\uc7ac\ud569\ub2c8\ub2e4.\n\uc608\ub97c \ub4e4\uc5b4, \uccab \ubc88\uc9f8 GPU\uc5d0\uc11c \ubaa8\ub4e0 \uc5f0\uc0b0\uacfc\uc815\uc774 \uae30\ubcf8\uc73c\ub85c \uc124\uc815\ub418\uc5b4 \uc9c4\ud589\ub429\ub2c8\ub2e4. \uc774\ub294 \ubbf8\ub2c8\ubc30\uce58 \ubd84\ud560 \uacfc\uc815 \uc911,\n\ud604\uc7ac \uc9c4\ud589\ub418\ub294 \uacfc\uc815\uc758 \ub2e4\uc74c \ub2e8\uacc4\ub294 \ud604\uc7ac \uc9c4\ud589\ub418\ub294 \uacfc\uc815\uacfc \ub3d9\uc2dc\uc5d0 \ubcf5\uc81c\uac00 \uc774\ub8e8\uc5b4\uc9c8 \uc218 \uc5c6\ub294 \uac83\uc744 \uc758\ubbf8\ud569\ub2c8\ub2e4.\n\uadf8\ub7ec\ub098, \uc774\uc804\uacfc \ub2e4\uc74c \ub2e8\uacc4\uc758 \ubd84\ud560\uacfc\uc815\uc774 \ub2e4\ub978 \ud150\uc11c\uac12\uc744 \uc774\uc6a9\ud558\uae30 \ub54c\ubb38\uc5d0, \ub2e4\ub978 \uacc4\uc0b0\uacfc \uc911\ubcf5\ub418\uc5b4 \uc9c4\ud589\ub418\uc5b4\ub3c4\n\ubb38\uc81c\uac00 \uc5c6\uc2b5\ub2c8\ub2e4. \uc774\uc5d0 \ub300\ud574\uc11c, 2\uac1c GPU\uc5d0 \uc5ec\ub7ec\uac1c\uc758 \uc2a4\ud2b8\ub9bc\uc744 \uc0ac\uc6a9\ud558\ub294 \uac83\uc774 \ud544\uc694\ud558\uba70, \uc11c\ub85c \ub2e4\ub978 \uc11c\ube0c \ub124\ud2b8\uc6cc\ud06c\n\uad6c\uc870\uac00 \uc11c\ub85c \ub2e4\ub978 \uc2a4\ud2b8\ub9bc\uc744 \uad00\ub9ac\ud558\ub294 \uc804\ub7b5\uc774 \uc694\uad6c\ub429\ub2c8\ub2e4. \ubaa8\ub378 \ubcd1\ub82c \ucc98\ub9ac\uc5d0 \ub300\ud574\uc11c \uc5ec\ub7ec \uc2a4\ud2b8\ub9bc\uc744 \uc0ac\uc6a9\ud558\ub294 \ubc29\ubc95\uc774\n\uc77c\ubc18\uc801\uc744\ub85c \uc874\uc7ac\ud558\uc9c0 \uc54a\uae30 \ub54c\ubb38\uc5d0 \uc774\ubc88 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 \uc124\uba85\ud558\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4.\n\n" ] }, { @@ -184,7 +184,7 @@ }, "outputs": [], "source": [ - "\"\"\"\n.. note::\n \uc774\ubc88 \uac8c\uc2dc\ubb3c\uc5d0\uc11c\ub294 \ub2e4\uc591\ud55c \uc131\ub2a5 \uce21\uc815\uac12\uc744 \ud655\uc778\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc5ec\ub7ec\ubd84\uc740 \uc704\uc758 \uc608\uc81c\ub97c \uc2e4\ud589\ud560 \ub54c \ub9c8\ub2e4 \ub9e4\ubc88\n \ub2e4\ub978 \uacb0\uacfc\ub97c \ud655\uc778\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uadf8 \uc774\uc720\ub294, \uc774\uc6a9\ud558\ub294 \uc18c\ud504\ud2b8\uc6e8\uc5b4 \ubc0f \ud558\ub4dc\uc6e8\uc5b4\uc5d0 \ub530\ub77c \uacb0\uacfc\uac00 \n \ub2e4\ub974\uac8c \ub098\ud0c0\ub098\uae30 \ub54c\ubb38\uc785\ub2c8\ub2e4. \uc5ec\ub7ec\ubd84\uc774 \uc774\uc6a9\ud558\uace0 \uc788\ub294 \ud658\uacbd \ub0b4\uc5d0\uc11c \uac00\uc7a5 \uc88b\uc740 \uc131\ub2a5\uc744 \uc5bb\uae30 \uc704\ud574\uc11c\ub294, \uace1\uc120\uc744 \uadf8\ub824\uc11c\n \ucd5c\uc801\uc758 ``split_size`` \uac12\uc744 \ub3c4\ucd9c\ud55c \ud6c4, \ud574\ub2f9 \uac12\uc744 \uc774\uc6a9\ud558\uc5ec \ubbf8\ub2c8 \ubc30\uce58 \ub0b4 \ub370\uc774\ud130\ub97c \ubd84\ub9ac\ud558\ub294 \ud30c\uc774\ud504\ub77c\uc778\uc744 \n \uc0dd\uc131\ud558\ub294 \uac83\uc785\ub2c8\ub2e4.\n\"\"\"" + "\"\"\"\n.. note::\n \uc774\ubc88 \uac8c\uc2dc\ubb3c\uc5d0\uc11c\ub294 \ub2e4\uc591\ud55c \uc131\ub2a5 \uce21\uc815\uac12\uc744 \ud655\uc778\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc5ec\ub7ec\ubd84\uc740 \uc704\uc758 \uc608\uc81c\ub97c \uc2e4\ud589\ud560 \ub54c \ub9c8\ub2e4 \ub9e4\ubc88\n \ub2e4\ub978 \uacb0\uacfc\ub97c \ud655\uc778\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uadf8 \uc774\uc720\ub294, \uc774\uc6a9\ud558\ub294 \uc18c\ud504\ud2b8\uc6e8\uc5b4 \ubc0f \ud558\ub4dc\uc6e8\uc5b4\uc5d0 \ub530\ub77c \uacb0\uacfc\uac00\n \ub2e4\ub974\uac8c \ub098\ud0c0\ub098\uae30 \ub54c\ubb38\uc785\ub2c8\ub2e4. \uc5ec\ub7ec\ubd84\uc774 \uc774\uc6a9\ud558\uace0 \uc788\ub294 \ud658\uacbd \ub0b4\uc5d0\uc11c \uac00\uc7a5 \uc88b\uc740 \uc131\ub2a5\uc744 \uc5bb\uae30 \uc704\ud574\uc11c\ub294, \uace1\uc120\uc744 \uadf8\ub824\uc11c\n \ucd5c\uc801\uc758 ``split_size`` \uac12\uc744 \ub3c4\ucd9c\ud55c \ud6c4, \ud574\ub2f9 \uac12\uc744 \uc774\uc6a9\ud558\uc5ec \ubbf8\ub2c8 \ubc30\uce58 \ub0b4 \ub370\uc774\ud130\ub97c \ubd84\ub9ac\ud558\ub294 \ud30c\uc774\ud504\ub77c\uc778\uc744\n \uc0dd\uc131\ud558\ub294 \uac83\uc785\ub2c8\ub2e4.\n\"\"\"" ] } ], @@ -204,7 +204,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/0abb91e66579f9acfdfbf93bc4b69955/named_tensor_tutorial.ipynb b/docs/_downloads/0abb91e66579f9acfdfbf93bc4b69955/named_tensor_tutorial.ipynb index 28298040e..097c6be46 100644 --- a/docs/_downloads/0abb91e66579f9acfdfbf93bc4b69955/named_tensor_tutorial.ipynb +++ b/docs/_downloads/0abb91e66579f9acfdfbf93bc4b69955/named_tensor_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n(prototype) Introduction to Named Tensors in PyTorch\n*******************************************************\n**Author**: `Richard Zou `_\n\nNamed Tensors aim to make tensors easier to use by allowing users to associate\nexplicit names with tensor dimensions. In most cases, operations that take\ndimension parameters will accept dimension names, avoiding the need to track\ndimensions by position. In addition, named tensors use names to automatically\ncheck that APIs are being used correctly at runtime, providing extra safety.\nNames can also be used to rearrange dimensions, for example, to support\n\"broadcasting by name\" rather than \"broadcasting by position\".\n\nThis tutorial is intended as a guide to the functionality that will\nbe included with the 1.3 launch. By the end of it, you will be able to:\n\n- Create Tensors with named dimensions, as well as remove or rename those\n dimensions\n- Understand the basics of how operations propagate dimension names\n- See how naming dimensions enables clearer code in two key areas:\n - Broadcasting operations\n - Flattening and unflattening dimensions\n\nFinally, we'll put this into practice by writing a multi-head attention module\nusing named tensors.\n\nNamed tensors in PyTorch are inspired by and done in collaboration with\n`Sasha Rush `_.\nSasha proposed the original idea and proof of concept in his\n`January 2019 blog post `_.\n\nBasics: named dimensions\n========================\n\nPyTorch now allows Tensors to have named dimensions; factory functions\ntake a new `names` argument that associates a name with each dimension.\nThis works with most factory functions, such as\n\n- `tensor`\n- `empty`\n- `ones`\n- `zeros`\n- `randn`\n- `rand`\n\nHere we construct a tensor with names:\n\n" + "\n(prototype) Introduction to Named Tensors in PyTorch\n*******************************************************\n**Author**: `Richard Zou `_\n\nNamed Tensors aim to make tensors easier to use by allowing users to associate\nexplicit names with tensor dimensions. In most cases, operations that take\ndimension parameters will accept dimension names, avoiding the need to track\ndimensions by position. In addition, named tensors use names to automatically\ncheck that APIs are being used correctly at runtime, providing extra safety.\nNames can also be used to rearrange dimensions, for example, to support\n\"broadcasting by name\" rather than \"broadcasting by position\".\n\nThis tutorial is intended as a guide to the functionality that will\nbe included with the 1.3 launch. By the end of it, you will be able to:\n\n- Create Tensors with named dimensions, as well as remove or rename those\n dimensions\n- Understand the basics of how operations propagate dimension names\n- See how naming dimensions enables clearer code in two key areas:\n - Broadcasting operations\n - Flattening and unflattening dimensions\n\nFinally, we'll put this into practice by writing a multi-head attention module\nusing named tensors.\n\nNamed tensors in PyTorch are inspired by and done in collaboration with\n`Sasha Rush `_.\nSasha proposed the original idea and proof of concept in his\n`January 2019 blog post `_.\n\nBasics: named dimensions\n========================\n\nPyTorch now allows Tensors to have named dimensions; factory functions\ntake a new `names` argument that associates a name with each dimension.\nThis works with most factory functions, such as\n\n- `tensor`\n- `empty`\n- `ones`\n- `zeros`\n- `randn`\n- `rand`\n\nHere we construct a tensor with names:\n" ] }, { @@ -503,7 +503,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/13b143c2380f4768d9432d808ad50799/char_rnn_classification_tutorial.ipynb b/docs/_downloads/13b143c2380f4768d9432d808ad50799/char_rnn_classification_tutorial.ipynb index a5a12b000..82eccb9c0 100644 --- a/docs/_downloads/13b143c2380f4768d9432d808ad50799/char_rnn_classification_tutorial.ipynb +++ b/docs/_downloads/13b143c2380f4768d9432d808ad50799/char_rnn_classification_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n\uae30\ucd08\ubd80\ud130 \uc2dc\uc791\ud558\ub294 NLP: \ubb38\uc790-\ub2e8\uc704 RNN\uc73c\ub85c \uc774\ub984 \ubd84\ub958\ud558\uae30\n********************************************************************************\n**Author**: `Sean Robertson `_\n **\ubc88\uc5ed**: `\ud669\uc131\uc218 `_\n\n\n\ub2e8\uc5b4\ub97c \ubd84\ub958\ud558\uae30 \uc704\ud574 \uae30\ucd08\uc801\uc778 \ubb38\uc790-\ub2e8\uc704 RNN\uc744 \uad6c\ucd95\ud558\uace0 \ud559\uc2b5 \ud560 \uc608\uc815\uc785\ub2c8\ub2e4.\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 (\uc774\ud6c4 2\uac1c \ud29c\ud1a0\ub9ac\uc5bc\uacfc \ud568\uaed8) NLP \ubaa8\ub378\ub9c1\uc744 \uc704\ud55c \ub370\uc774\ud130 \uc804\ucc98\ub9ac\ub97c\n`torchtext` \uc758 \ud3b8\ub9ac\ud55c \ub9ce\uc740 \uae30\ub2a5\ub4e4\uc744 \uc0ac\uc6a9\ud558\uc9c0 \uc54a\uace0 \uc5b4\ub5bb\uac8c \ud558\ub294\uc9c0 \"\uae30\ucd08\ubd80\ud130(from scratch)\"\n\ubcf4\uc5ec\uc8fc\uae30 \ub584\ubb38\uc5d0 NLP \ubaa8\ub378\ub9c1\uc744 \uc704\ud55c \uc804\ucc98\ub9ac\uac00 \uc800\uc218\uc900\uc5d0\uc11c \uc5b4\ub5bb\uac8c \uc9c4\ud589\ub418\ub294\uc9c0\ub97c \uc54c \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\ubb38\uc790-\ub2e8\uc704 RNN\uc740 \ub2e8\uc5b4\ub97c \ubb38\uc790\uc758 \uc5f0\uc18d\uc73c\ub85c \uc77d\uc5b4 \ub4e4\uc5ec\uc11c \uac01 \ub2e8\uacc4\uc758 \uc608\uce21\uacfc\n\"\uc740\ub2c9 \uc0c1\ud0dc(Hidden State)\" \ucd9c\ub825\ud558\uace0, \ub2e4\uc74c \ub2e8\uacc4\uc5d0 \uc774\uc804 \uc740\ub2c9 \uc0c1\ud0dc\ub97c \uc804\ub2ec\ud569\ub2c8\ub2e4.\n\ub2e8\uc5b4\uac00 \uc18d\ud55c \ud074\ub798\uc2a4\ub85c \ucd9c\ub825\uc774 \ub418\ub3c4\ub85d \ucd5c\uc885 \uc608\uce21\uc73c\ub85c \uc120\ud0dd\ud569\ub2c8\ub2e4.\n\n\uad6c\uccb4\uc801\uc73c\ub85c, 18\uac1c \uc5b8\uc5b4\ub85c \ub41c \uc218\ucc9c \uac1c\uc758 \uc131(\u59d3)\uc744 \ud6c8\ub828\uc2dc\ud0a4\uace0,\n\ucca0\uc790\uc5d0 \ub530\ub77c \uc774\ub984\uc774 \uc5b4\ub5a4 \uc5b8\uc5b4\uc778\uc9c0 \uc608\uce21\ud569\ub2c8\ub2e4:\n\n::\n\n $ python predict.py Hinton\n (-0.47) Scottish\n (-1.52) English\n (-3.57) Irish\n\n $ python predict.py Schmidhuber\n (-0.19) German\n (-2.48) Czech\n (-2.68) Dutch\n\n\n**\ucd94\ucc9c \uc790\ub8cc:**\n\nPytorch\ub97c \uc124\uce58\ud588\uace0, Python\uc744 \uc54c\uace0, Tensor\ub97c \uc774\ud574\ud55c\ub2e4\uace0 \uac00\uc815\ud569\ub2c8\ub2e4:\n\n- https://pytorch.org/ \uc124\uce58 \uc548\ub0b4\n- :doc:`/beginner/deep_learning_60min_blitz` PyTorch \uc2dc\uc791\ud558\uae30\n- :doc:`/beginner/pytorch_with_examples` \ub113\uace0 \uae4a\uc740 \ud1b5\ucc30\uc744 \uc704\ud55c \uc790\ub8cc\n- :doc:`/beginner/former_torchies_tutorial` \uc774\uc804 Lua Torch \uc0ac\uc6a9\uc790\ub97c \uc704\ud55c \uc790\ub8cc\n\nRNN\uacfc \uc791\ub3d9 \ubc29\uc2dd\uc744 \uc544\ub294 \uac83 \ub610\ud55c \uc720\uc6a9\ud569\ub2c8\ub2e4:\n\n- `The Unreasonable Effectiveness of Recurrent Neural\n Networks `__\n \uc2e4\uc0dd\ud65c \uc608\uc81c\ub97c \ubcf4\uc5ec \uc90d\ub2c8\ub2e4.\n- `Understanding LSTM\n Networks `__\n LSTM\uc5d0 \uad00\ud55c \uac83\uc774\uc9c0\ub9cc RNN\uc5d0 \uad00\ud574\uc11c\ub3c4 \uc720\uc775\ud569\ub2c8\ub2e4.\n\n\ub370\uc774\ud130 \uc900\ube44\n==================\n\n.. NOTE::\n `\uc5ec\uae30 `__ \uc5d0\uc11c \ub370\uc774\ud130\ub97c \ub2e4\uc6b4 \ubc1b\uace0,\n \ud604\uc7ac \ub514\ub809\ud1a0\ub9ac\uc5d0 \uc555\ucd95\uc744 \ud478\uc2ed\uc2dc\uc624.\n\n``data/names`` \ub514\ub809\ud1a0\ub9ac\uc5d0\ub294 \"[Language].txt\" \ub77c\ub294 18 \uac1c\uc758 \ud14d\uc2a4\ud2b8 \ud30c\uc77c\uc774 \uc788\uc2b5\ub2c8\ub2e4.\n\uac01 \ud30c\uc77c\uc5d0\ub294 \ud55c \uc904\uc5d0 \ud558\ub098\uc758 \uc774\ub984\uc774 \ud3ec\ud568\ub418\uc5b4 \uc788\uc73c\uba70 \ub300\ubd80\ubd84 \ub85c\ub9c8\uc790\ub85c \ub418\uc5b4 \uc788\uc2b5\ub2c8\ub2e4\n(\uadf8\ub7ec\ub098, \uc720\ub2c8\ucf54\ub4dc\uc5d0\uc11c ASCII\ub85c \ubcc0\ud658\ud574\uc57c \ud568).\n\n\uac01 \uc5b8\uc5b4 \ubcc4\ub85c \uc774\ub984 \ubaa9\ub85d \uc0ac\uc804 ``{language: [names ...]}`` \uc744 \ub9cc\ub4ed\ub2c8\ub2e4.\n\uc77c\ubc18 \ubcc0\uc218 \"category\" \uc640 \"line\" (\uc6b0\ub9ac\uc758 \uacbd\uc6b0 \uc5b8\uc5b4\uc640 \uc774\ub984)\uc740 \uc774\ud6c4\uc758 \ud655\uc7a5\uc131\uc744 \uc704\ud574 \uc0ac\uc6a9\ub429\ub2c8\ub2e4.\n\n.. NOTE::\n\uc5ed\uc790 \uc8fc: \"line\" \uc5d0 \uc785\ub825\uc744 \"category\"\uc5d0 \ud074\ub798\uc2a4\ub97c \uc801\uc6a9\ud558\uc5ec \ub2e4\ub978 \ubb38\uc81c\uc5d0\ub3c4 \ud65c\uc6a9 \ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uc5ec\uae30\uc11c\ub294 \"line\"\uc5d0 \uc774\ub984(ex. Robert )\ub97c \uc785\ub825\uc73c\ub85c \"category\"\uc5d0 \ud074\ub798\uc2a4(ex. english)\ub85c \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n\n" + "\n\uae30\ucd08\ubd80\ud130 \uc2dc\uc791\ud558\ub294 NLP: \ubb38\uc790-\ub2e8\uc704 RNN\uc73c\ub85c \uc774\ub984 \ubd84\ub958\ud558\uae30\n********************************************************************************\n**Author**: `Sean Robertson `_\n **\ubc88\uc5ed**: `\ud669\uc131\uc218 `_\n\n\n\ub2e8\uc5b4\ub97c \ubd84\ub958\ud558\uae30 \uc704\ud574 \uae30\ucd08\uc801\uc778 \ubb38\uc790-\ub2e8\uc704 RNN\uc744 \uad6c\ucd95\ud558\uace0 \ud559\uc2b5 \ud560 \uc608\uc815\uc785\ub2c8\ub2e4.\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 (\uc774\ud6c4 2\uac1c \ud29c\ud1a0\ub9ac\uc5bc\uacfc \ud568\uaed8) NLP \ubaa8\ub378\ub9c1\uc744 \uc704\ud55c \ub370\uc774\ud130 \uc804\ucc98\ub9ac\ub97c\n`torchtext` \uc758 \ud3b8\ub9ac\ud55c \ub9ce\uc740 \uae30\ub2a5\ub4e4\uc744 \uc0ac\uc6a9\ud558\uc9c0 \uc54a\uace0 \uc5b4\ub5bb\uac8c \ud558\ub294\uc9c0 \"\uae30\ucd08\ubd80\ud130(from scratch)\"\n\ubcf4\uc5ec\uc8fc\uae30 \ub584\ubb38\uc5d0 NLP \ubaa8\ub378\ub9c1\uc744 \uc704\ud55c \uc804\ucc98\ub9ac\uac00 \uc800\uc218\uc900\uc5d0\uc11c \uc5b4\ub5bb\uac8c \uc9c4\ud589\ub418\ub294\uc9c0\ub97c \uc54c \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\ubb38\uc790-\ub2e8\uc704 RNN\uc740 \ub2e8\uc5b4\ub97c \ubb38\uc790\uc758 \uc5f0\uc18d\uc73c\ub85c \uc77d\uc5b4 \ub4e4\uc5ec\uc11c \uac01 \ub2e8\uacc4\uc758 \uc608\uce21\uacfc\n\"\uc740\ub2c9 \uc0c1\ud0dc(Hidden State)\" \ucd9c\ub825\ud558\uace0, \ub2e4\uc74c \ub2e8\uacc4\uc5d0 \uc774\uc804 \uc740\ub2c9 \uc0c1\ud0dc\ub97c \uc804\ub2ec\ud569\ub2c8\ub2e4.\n\ub2e8\uc5b4\uac00 \uc18d\ud55c \ud074\ub798\uc2a4\ub85c \ucd9c\ub825\uc774 \ub418\ub3c4\ub85d \ucd5c\uc885 \uc608\uce21\uc73c\ub85c \uc120\ud0dd\ud569\ub2c8\ub2e4.\n\n\uad6c\uccb4\uc801\uc73c\ub85c, 18\uac1c \uc5b8\uc5b4\ub85c \ub41c \uc218\ucc9c \uac1c\uc758 \uc131(\u59d3)\uc744 \ud6c8\ub828\uc2dc\ud0a4\uace0,\n\ucca0\uc790\uc5d0 \ub530\ub77c \uc774\ub984\uc774 \uc5b4\ub5a4 \uc5b8\uc5b4\uc778\uc9c0 \uc608\uce21\ud569\ub2c8\ub2e4:\n\n::\n\n $ python predict.py Hinton\n (-0.47) Scottish\n (-1.52) English\n (-3.57) Irish\n\n $ python predict.py Schmidhuber\n (-0.19) German\n (-2.48) Czech\n (-2.68) Dutch\n\n\n**\ucd94\ucc9c \uc790\ub8cc:**\n\nPytorch\ub97c \uc124\uce58\ud588\uace0, Python\uc744 \uc54c\uace0, Tensor\ub97c \uc774\ud574\ud55c\ub2e4\uace0 \uac00\uc815\ud569\ub2c8\ub2e4:\n\n- https://pytorch.org/ \uc124\uce58 \uc548\ub0b4\n- :doc:`/beginner/deep_learning_60min_blitz` PyTorch \uc2dc\uc791\ud558\uae30\n- :doc:`/beginner/pytorch_with_examples` \ub113\uace0 \uae4a\uc740 \ud1b5\ucc30\uc744 \uc704\ud55c \uc790\ub8cc\n- :doc:`/beginner/former_torchies_tutorial` \uc774\uc804 Lua Torch \uc0ac\uc6a9\uc790\ub97c \uc704\ud55c \uc790\ub8cc\n\nRNN\uacfc \uc791\ub3d9 \ubc29\uc2dd\uc744 \uc544\ub294 \uac83 \ub610\ud55c \uc720\uc6a9\ud569\ub2c8\ub2e4:\n\n- `The Unreasonable Effectiveness of Recurrent Neural\n Networks `__\n \uc2e4\uc0dd\ud65c \uc608\uc81c\ub97c \ubcf4\uc5ec \uc90d\ub2c8\ub2e4.\n- `Understanding LSTM\n Networks `__\n LSTM\uc5d0 \uad00\ud55c \uac83\uc774\uc9c0\ub9cc RNN\uc5d0 \uad00\ud574\uc11c\ub3c4 \uc720\uc775\ud569\ub2c8\ub2e4.\n\n\ub370\uc774\ud130 \uc900\ube44\n==================\n\n

Note

`\uc5ec\uae30 `__ \uc5d0\uc11c \ub370\uc774\ud130\ub97c \ub2e4\uc6b4 \ubc1b\uace0,\n \ud604\uc7ac \ub514\ub809\ud1a0\ub9ac\uc5d0 \uc555\ucd95\uc744 \ud478\uc2ed\uc2dc\uc624.

\n\n``data/names`` \ub514\ub809\ud1a0\ub9ac\uc5d0\ub294 \"[Language].txt\" \ub77c\ub294 18 \uac1c\uc758 \ud14d\uc2a4\ud2b8 \ud30c\uc77c\uc774 \uc788\uc2b5\ub2c8\ub2e4.\n\uac01 \ud30c\uc77c\uc5d0\ub294 \ud55c \uc904\uc5d0 \ud558\ub098\uc758 \uc774\ub984\uc774 \ud3ec\ud568\ub418\uc5b4 \uc788\uc73c\uba70 \ub300\ubd80\ubd84 \ub85c\ub9c8\uc790\ub85c \ub418\uc5b4 \uc788\uc2b5\ub2c8\ub2e4\n(\uadf8\ub7ec\ub098, \uc720\ub2c8\ucf54\ub4dc\uc5d0\uc11c ASCII\ub85c \ubcc0\ud658\ud574\uc57c \ud568).\n\n\uac01 \uc5b8\uc5b4 \ubcc4\ub85c \uc774\ub984 \ubaa9\ub85d \uc0ac\uc804 ``{language: [names ...]}`` \uc744 \ub9cc\ub4ed\ub2c8\ub2e4.\n\uc77c\ubc18 \ubcc0\uc218 \"category\" \uc640 \"line\" (\uc6b0\ub9ac\uc758 \uacbd\uc6b0 \uc5b8\uc5b4\uc640 \uc774\ub984)\uc740 \uc774\ud6c4\uc758 \ud655\uc7a5\uc131\uc744 \uc704\ud574 \uc0ac\uc6a9\ub429\ub2c8\ub2e4.\n\n

Note

\uc5ed\uc790 \uc8fc: \"line\" \uc5d0 \uc785\ub825\uc744 \"category\"\uc5d0 \ud074\ub798\uc2a4\ub97c \uc801\uc6a9\ud558\uc5ec \ub2e4\ub978 \ubb38\uc81c\uc5d0\ub3c4 \ud65c\uc6a9 \ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n \uc5ec\uae30\uc11c\ub294 \"line\"\uc5d0 \uc774\ub984(ex. Robert )\ub97c \uc785\ub825\uc73c\ub85c \"category\"\uc5d0 \ud074\ub798\uc2a4(ex. english)\ub85c \uc0ac\uc6a9\ud569\ub2c8\ub2e4.

\n" ] }, { @@ -308,7 +308,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/16b28c991d1ecd15e9c3baccf501a92e/torchscript_freezing.ipynb b/docs/_downloads/16b28c991d1ecd15e9c3baccf501a92e/torchscript_freezing.ipynb index b25fec4e4..6201beee3 100644 --- a/docs/_downloads/16b28c991d1ecd15e9c3baccf501a92e/torchscript_freezing.ipynb +++ b/docs/_downloads/16b28c991d1ecd15e9c3baccf501a92e/torchscript_freezing.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nModel Freezing in TorchScript\n=============================\n\nIn this tutorial, we introduce the syntax for *model freezing* in TorchScript.\nFreezing is the process of inlining Pytorch module parameters and attributes\nvalues into the TorchScript internal representation. Parameter and attribute\nvalues are treated as final values and they cannot be modified in the resulting\nFrozen module.\n\nBasic Syntax\n------------\nModel freezing can be invoked using API below:\n\n ``torch.jit.freeze(mod : ScriptModule, names : str[]) -> SciptModule``\n\nNote the input module can either be the result of scripting or tracing.\nSee https://pytorch.org/tutorials/beginner/Intro_to_TorchScript_tutorial.html\n\nNext, we demonstrate how freezing works using an example:\n\n" + "\nModel Freezing in TorchScript\n=============================\n\nIn this tutorial, we introduce the syntax for *model freezing* in TorchScript.\nFreezing is the process of inlining Pytorch module parameters and attributes\nvalues into the TorchScript internal representation. Parameter and attribute\nvalues are treated as final values and they cannot be modified in the resulting\nFrozen module.\n\nBasic Syntax\n------------\nModel freezing can be invoked using API below:\n\n ``torch.jit.freeze(mod : ScriptModule, names : str[]) -> SciptModule``\n\nNote the input module can either be the result of scripting or tracing.\nSee https://pytorch.org/tutorials/beginner/Intro_to_TorchScript_tutorial.html\n\nNext, we demonstrate how freezing works using an example:\n" ] }, { @@ -67,7 +67,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/1b58d206e701317cf46c92dcf2a8978d/parallelism_tutorial.ipynb b/docs/_downloads/1b58d206e701317cf46c92dcf2a8978d/parallelism_tutorial.ipynb index 1b453dcd2..ef0d775f4 100644 --- a/docs/_downloads/1b58d206e701317cf46c92dcf2a8978d/parallelism_tutorial.ipynb +++ b/docs/_downloads/1b58d206e701317cf46c92dcf2a8978d/parallelism_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n\uba40\ud2f0-GPU \uc608\uc81c\n==================\n\n\ub370\uc774\ud130 \ubcd1\ub82c \ucc98\ub9ac(Data Parallelism)\ub294 \ubbf8\ub2c8-\ubc30\uce58\ub97c \uc5ec\ub7ec \uac1c\uc758 \ub354 \uc791\uc740 \ubbf8\ub2c8-\ubc30\uce58\ub85c\n\uc790\ub974\uace0 \uac01\uac01\uc758 \uc791\uc740 \ubbf8\ub2c8\ubc30\uce58\ub97c \ubcd1\ub82c\uc801\uc73c\ub85c \uc5f0\uc0b0\ud558\ub294 \uac83\uc785\ub2c8\ub2e4.\n\n\ub370\uc774\ud130 \ubcd1\ub82c \ucc98\ub9ac\ub294 ``torch.nn.DataParallel`` \uc744 \uc0ac\uc6a9\ud558\uc5ec \uad6c\ud604\ud569\ub2c8\ub2e4.\n``DataParallel`` \ub85c \uac10\uc300 \uc218 \uc788\ub294 \ubaa8\ub4c8\uc740 \ubc30\uce58 \ucc28\uc6d0(batch dimension)\uc5d0\uc11c\n\uc5ec\ub7ec GPU\ub85c \ubcd1\ub82c \ucc98\ub9ac\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\nDataParallel\n-------------\n\n" + "\n\uba40\ud2f0-GPU \uc608\uc81c\n==================\n\n\ub370\uc774\ud130 \ubcd1\ub82c \ucc98\ub9ac(Data Parallelism)\ub294 \ubbf8\ub2c8-\ubc30\uce58\ub97c \uc5ec\ub7ec \uac1c\uc758 \ub354 \uc791\uc740 \ubbf8\ub2c8-\ubc30\uce58\ub85c\n\uc790\ub974\uace0 \uac01\uac01\uc758 \uc791\uc740 \ubbf8\ub2c8\ubc30\uce58\ub97c \ubcd1\ub82c\uc801\uc73c\ub85c \uc5f0\uc0b0\ud558\ub294 \uac83\uc785\ub2c8\ub2e4.\n\n\ub370\uc774\ud130 \ubcd1\ub82c \ucc98\ub9ac\ub294 ``torch.nn.DataParallel`` \uc744 \uc0ac\uc6a9\ud558\uc5ec \uad6c\ud604\ud569\ub2c8\ub2e4.\n``DataParallel`` \ub85c \uac10\uc300 \uc218 \uc788\ub294 \ubaa8\ub4c8\uc740 \ubc30\uce58 \ucc28\uc6d0(batch dimension)\uc5d0\uc11c\n\uc5ec\ub7ec GPU\ub85c \ubcd1\ub82c \ucc98\ub9ac\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\nDataParallel\n-------------\n" ] }, { @@ -107,7 +107,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/256861ec2cab5f0dc50c523f520dfefd/dynamic_quantization_tutorial.py b/docs/_downloads/256861ec2cab5f0dc50c523f520dfefd/dynamic_quantization_tutorial.py index cabdd90d2..94555b115 100644 --- a/docs/_downloads/256861ec2cab5f0dc50c523f520dfefd/dynamic_quantization_tutorial.py +++ b/docs/_downloads/256861ec2cab5f0dc50c523f520dfefd/dynamic_quantization_tutorial.py @@ -1,26 +1,27 @@ +# -*- coding: utf-8 -*- """ -(beta) Dynamic Quantization on an LSTM Word Language Model +(베타) LSTM 기반 단어 단위 언어 모델의 동적 양자화 ================================================================== **Author**: `James Reed `_ **Edited by**: `Seth Weidman `_ -Introduction +**번역**: `박경림 `_ `Myungha Kwon `_ + +시작하기 ------------ -Quantization involves converting the weights and activations of your model from float -to int, which can result in smaller model size and faster inference with only a small -hit to accuracy. +양자화는 모델의 크기를 줄이고 추론 속도를 높이면서도 정확도는 별로 낮아지지 않도록, +모델의 가중치와 활성 함수를 실수형에서 정수형으로 변환합니다. -In this tutorial, we'll apply the easiest form of quantization - -`dynamic quantization `_ - -to an LSTM-based next word-prediction model, closely following the -`word language model `_ -from the PyTorch examples. +이 튜토리얼에서는 PyTorch의 `단어 단위 언어 모델 `_ +예제를 따라하면서, LSTM 기반의 단어 예측 모델에 가장 간단한 양자화 기법인 +`동적 양자화 `_ +를 적용해 보겠습니다. """ -# imports +# 불러오기 import os from io import open import time @@ -30,15 +31,14 @@ import torch.nn.functional as F ###################################################################### -# 1. Define the model +# 1. 모델 정의하기 # ------------------- # -# Here we define the LSTM model architecture, following the -# `model `_ -# from the word language model example. +# 단어 단위 언어 모델 예제에서 사용된 `모델 `_ 을 +# 따라 LSTM 모델 아키텍처를 정의합니다. class LSTMModel(nn.Module): - """Container module with an encoder, a recurrent module, and a decoder.""" + """인코더, 반복 모듈 및 디코더가 있는 컨테이너 모듈.""" def __init__(self, ntoken, ninp, nhid, nlayers, dropout=0.5): super(LSTMModel, self).__init__() @@ -71,14 +71,11 @@ def init_hidden(self, bsz): weight.new_zeros(self.nlayers, bsz, self.nhid)) ###################################################################### -# 2. Load in the text data +# 2. 텍스트 데이터 불러오기 # ------------------------ # -# Next, we load the -# `Wikitext-2 dataset `_ into a `Corpus`, -# again following the -# `preprocessing `_ -# from the word language model example. +# 다음으로, 단어 단위 언어 모델 예제의 `전처리 `_ +# 과정을 따라 `Wikitext-2 데이터셋 `_ 을 `Corpus` 인스턴스에 불러옵니다. class Dictionary(object): def __init__(self): @@ -103,16 +100,17 @@ def __init__(self, path): self.test = self.tokenize(os.path.join(path, 'test.txt')) def tokenize(self, path): - """Tokenizes a text file.""" assert os.path.exists(path) - # Add words to the dictionary + """텍스트 파일 토큰화""" + assert os.path.exists(path) + # 사전에 단어 추가 with open(path, 'r', encoding="utf8") as f: for line in f: words = line.split() + [''] for word in words: self.dictionary.add_word(word) - # Tokenize file content + # 파일 내용 토큰화 with open(path, 'r', encoding="utf8") as f: idss = [] for line in f: @@ -130,14 +128,13 @@ def tokenize(self, path): corpus = Corpus(model_data_filepath + 'wikitext-2') ###################################################################### -# 3. Load the pre-trained model +# 3. 사전 학습된 모델 불러오기 # ----------------------------- # -# This is a tutorial on dynamic quantization, a quantization technique -# that is applied after a model has been trained. Therefore, we'll simply load some -# pre-trained weights into this model architecture; these weights were obtained -# by training for five epochs using the default settings in the word language model -# example. +# 이 튜토리얼은 모델이 학습된 후 적용되는 양자화 기술인 동적 양자화에 대한 튜토리얼입니다. +# 따라서 우리는 미리 학습된 가중치를 모델 아키텍처에 로드할 것 입니다. 이 가중치는 word +# language 모델 예제의 기본 설정을 사용하여 5개의 epoch 동안 학습하여 얻은 것입니다. +# ntokens = len(corpus.dictionary) @@ -159,9 +156,8 @@ def tokenize(self, path): print(model) ###################################################################### -# Now let's generate some text to ensure that the pre-trained model is working -# properly - similarly to before, we follow -# `here `_ +# 이제 사전 학습된 모델이 잘 동작하는지 확인해보기 위해 텍스트를 생성해 보겠습니다. +# 지금까지 튜토리얼을 진행했던 방식처럼 `이 예제 `_ 를 따라 하겠습니다. input_ = torch.randint(ntokens, (1, 1), dtype=torch.long) hidden = model.init_hidden(1) @@ -169,7 +165,7 @@ def tokenize(self, path): num_words = 1000 with open(model_data_filepath + 'out.txt', 'w') as outf: - with torch.no_grad(): # no tracking history + with torch.no_grad(): # 기록을 추적하지 않습니다. for i in range(num_words): output, hidden = model(input_, hidden) word_weights = output.squeeze().div(temperature).exp().cpu() @@ -188,28 +184,27 @@ def tokenize(self, path): print(all_output) ###################################################################### -# It's no GPT-2, but it looks like the model has started to learn the structure of -# language! +# 이 모델이 GPT-2는 아니지만, 언어의 구조를 배우기 시작한 것처럼 보입니다! +# # -# We're almost ready to demonstrate dynamic quantization. We just need to define a few more -# helper functions: +# 동적 양자화를 시연할 준비가 거의 끝났습니다. 몇 가지 helper 함수를 정의하기만 하면 됩니다: bptt = 25 criterion = nn.CrossEntropyLoss() eval_batch_size = 1 -# create test data set +# 테스트 데이터셋 만들기 def batchify(data, bsz): - # Work out how cleanly we can divide the dataset into bsz parts. + # 데이터셋을 bsz 부분으로 얼마나 깔끔하게 나눌 수 있는지 계산합니다. nbatch = data.size(0) // bsz - # Trim off any extra elements that wouldn't cleanly fit (remainders). + # 깔끔하게 맞지 않는 추가적인 부분(나머지들)을 잘라냅니다. data = data.narrow(0, 0, nbatch * bsz) - # Evenly divide the data across the bsz batches. + # 데이터에 대하여 bsz 배치들로 동등하게 나눕니다. return data.view(bsz, -1).t().contiguous() test_data = batchify(corpus.test, eval_batch_size) -# Evaluation functions +# 평가 함수들 def get_batch(source, i): seq_len = min(bptt, len(source) - 1 - i) data = source[i:i+seq_len] @@ -217,7 +212,7 @@ def get_batch(source, i): return data, target def repackage_hidden(h): - """Wraps hidden states in new Tensors, to detach them from their history.""" + """은닉 상태를 변화도 기록에서 제거된 새로운 tensor로 만듭니다.""" if isinstance(h, torch.Tensor): return h.detach() @@ -225,7 +220,7 @@ def repackage_hidden(h): return tuple(repackage_hidden(v) for v in h) def evaluate(model_, data_source): - # Turn on evaluation mode which disables dropout. + # Dropout을 중지시키는 평가 모드로 실행합니다. model_.eval() total_loss = 0. hidden = model_.init_hidden(eval_batch_size) @@ -239,15 +234,14 @@ def evaluate(model_, data_source): return total_loss / (len(data_source) - 1) ###################################################################### -# 4. Test dynamic quantization +# 4. 동적 양자화 테스트하기 # ---------------------------- # -# Finally, we can call ``torch.quantization.quantize_dynamic`` on the model! -# Specifically, +# 마지막으로 모델에서 ``torch.quantization.quantize_dynamic`` 을 호출 할 수 있습니다! +# 구체적으로, # -# - We specify that we want the ``nn.LSTM`` and ``nn.Linear`` modules in our -# model to be quantized -# - We specify that we want weights to be converted to ``int8`` values +# - 모델의 ``nn.LSTM`` 과 ``nn.Linear`` 모듈을 양자화 하도록 명시합니다. +# - 가중치들이 ``int8`` 값으로 변환되도록 명시합니다. import torch.quantization @@ -257,8 +251,8 @@ def evaluate(model_, data_source): print(quantized_model) ###################################################################### -# The model looks the same; how has this benefited us? First, we see a -# significant reduction in model size: +# 모델은 동일하게 보입니다. 이것이 어떻게 이득을 주는 것일까요? 첫째, 모델 크기가 +# 상당히 줄어 듭니다: def print_size_of_model(model): torch.save(model.state_dict(), "temp.p") @@ -269,10 +263,10 @@ def print_size_of_model(model): print_size_of_model(quantized_model) ###################################################################### -# Second, we see faster inference time, with no difference in evaluation loss: -# -# Note: we number of threads to one for single threaded comparison, since quantized -# models run single threaded. +# 두 번째로, 평가 손실값은 같으나 추론(inference) 속도가 빨라졌습니다. + +# 메모: 양자화 된 모델은 단일 스레드로 실행되기 때문에 단일 스레드 비교를 위해 +# 스레드 수를 1로 설정했습니다. torch.set_num_threads(1) @@ -286,14 +280,14 @@ def time_model_evaluation(model, test_data): time_model_evaluation(quantized_model, test_data) ###################################################################### -# Running this locally on a MacBook Pro, without quantization, inference takes about 200 seconds, -# and with quantization it takes just about 100 seconds. +# MacBook Pro에서 로컬로 실행하는 경우, 양자화 없이는 추론(inference)에 약 200초가 걸리고 +# 양자화를 사용하면 약 100초가 걸립니다. # -# Conclusion +# 마치며 # ---------- # -# Dynamic quantization can be an easy way to reduce model size while only -# having a limited effect on accuracy. +# 동적 양자화는 정확도에 제한적인 영향을 미치면서 모델 크기를 줄이는 +# 쉬운 방법이 될 수 있습니다. # -# Thanks for reading! As always, we welcome any feedback, so please create an issue -# `here `_ if you have any. +# 읽어주셔서 감사합니다. 언제나처럼 어떠한 피드백도 환영이니, 의견이 있다면 +# `여기 `_ 에 이슈를 남겨 주세요. diff --git a/docs/_downloads/27f4b1d04aea1872e99a8fc10c08ba13/two_layer_net_tensor.ipynb b/docs/_downloads/27f4b1d04aea1872e99a8fc10c08ba13/two_layer_net_tensor.ipynb index 6cf3e28b5..797865072 100644 --- a/docs/_downloads/27f4b1d04aea1872e99a8fc10c08ba13/two_layer_net_tensor.ipynb +++ b/docs/_downloads/27f4b1d04aea1872e99a8fc10c08ba13/two_layer_net_tensor.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nPyTorch: Tensors\n----------------\n\n\ud558\ub098\uc758 \uc740\ub2c9\uce35(hidden layer)\uacfc \ud3b8\ud5a5(bias)\uc774 \uc5c6\ub294 \uc644\uc804\ud788 \uc5f0\uacb0\ub41c ReLU \uc2e0\uacbd\ub9dd\uc744,\n\uc720\ud074\ub9ac\ub4dc \uac70\ub9ac(Euclidean distance) \uc81c\uacf1\uc744 \ucd5c\uc18c\ud654\ud558\ub294 \uc2dd\uc73c\ub85c x\ub85c\ubd80\ud130 y\ub97c \uc608\uce21\ud558\ub3c4\ub85d\n\ud559\uc2b5\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc5ec\uae30\uc11c\ub294 PyTorch tensor\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc21c\uc804\ud30c, \uc190\uc2e4(loss), \uadf8\ub9ac\uace0 \uc5ed\uc804\ud30c \uc5f0\uc0b0\uae4c\uc9c0\n\uc9c1\uc811 \uad6c\ud604\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\nPyTorch Tensor\ub294 \uae30\ubcf8\uc801\uc73c\ub85c NumPy \ubc30\uc5f4\uacfc \ub3d9\uc77c\ud569\ub2c8\ub2e4; \ub525\ub7ec\ub2dd\uc774\ub098 \uc5f0\uc0b0 \uadf8\ub798\ud504\n(computational graph), \ubcc0\ud654\ub3c4(gradient)\ub294 \uc54c\uc9c0 \ubabb\ud558\uba70 \uc784\uc758\uc758 \uc22b\uc790 \uacc4\uc0b0\uc5d0 \uc0ac\uc6a9\ud558\ub294\n\uc77c\ubc18\uc801\uc778 N\ucc28\uc6d0 \ubc30\uc5f4\uc785\ub2c8\ub2e4.\n\nNumPy \ubc30\uc5f4\uacfc PyTorch Tensor\uc758 \uac00\uc7a5 \ud070 \ucc28\uc774\uc810\uc740 PyTorch Tensor\ub294 CPU\ub098 GPU \uc5b4\ub514\uc11c\ub4e0\n\uc2e4\ud589\uc774 \uac00\ub2a5\ud558\ub2e4\ub294 \uc810\uc785\ub2c8\ub2e4. GPU\uc5d0\uc11c \uc5f0\uc0b0\uc744 \ud558\uae30 \uc704\ud574\uc11c\ub294, Tensor\ub97c CUDA \uc790\ub8cc\ud615\uc73c\ub85c\n\ubcc0\ud658(cast)\ud574\uc8fc\uae30\ub9cc \ud558\uba74 \ub429\ub2c8\ub2e4.\n\n" + "\nPyTorch: Tensors\n----------------\n\n\ud558\ub098\uc758 \uc740\ub2c9\uce35(hidden layer)\uacfc \ud3b8\ud5a5(bias)\uc774 \uc5c6\ub294 \uc644\uc804\ud788 \uc5f0\uacb0\ub41c ReLU \uc2e0\uacbd\ub9dd\uc744,\n\uc720\ud074\ub9ac\ub4dc \uac70\ub9ac(Euclidean distance) \uc81c\uacf1\uc744 \ucd5c\uc18c\ud654\ud558\ub294 \uc2dd\uc73c\ub85c x\ub85c\ubd80\ud130 y\ub97c \uc608\uce21\ud558\ub3c4\ub85d\n\ud559\uc2b5\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc5ec\uae30\uc11c\ub294 PyTorch tensor\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc21c\uc804\ud30c, \uc190\uc2e4(loss), \uadf8\ub9ac\uace0 \uc5ed\uc804\ud30c \uc5f0\uc0b0\uae4c\uc9c0\n\uc9c1\uc811 \uad6c\ud604\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\nPyTorch Tensor\ub294 \uae30\ubcf8\uc801\uc73c\ub85c NumPy \ubc30\uc5f4\uacfc \ub3d9\uc77c\ud569\ub2c8\ub2e4; \ub525\ub7ec\ub2dd\uc774\ub098 \uc5f0\uc0b0 \uadf8\ub798\ud504\n(computational graph), \ubcc0\ud654\ub3c4(gradient)\ub294 \uc54c\uc9c0 \ubabb\ud558\uba70 \uc784\uc758\uc758 \uc22b\uc790 \uacc4\uc0b0\uc5d0 \uc0ac\uc6a9\ud558\ub294\n\uc77c\ubc18\uc801\uc778 N\ucc28\uc6d0 \ubc30\uc5f4\uc785\ub2c8\ub2e4.\n\nNumPy \ubc30\uc5f4\uacfc PyTorch Tensor\uc758 \uac00\uc7a5 \ud070 \ucc28\uc774\uc810\uc740 PyTorch Tensor\ub294 CPU\ub098 GPU \uc5b4\ub514\uc11c\ub4e0\n\uc2e4\ud589\uc774 \uac00\ub2a5\ud558\ub2e4\ub294 \uc810\uc785\ub2c8\ub2e4. GPU\uc5d0\uc11c \uc5f0\uc0b0\uc744 \ud558\uae30 \uc704\ud574\uc11c\ub294, Tensor\ub97c CUDA \uc790\ub8cc\ud615\uc73c\ub85c\n\ubcc0\ud658(cast)\ud574\uc8fc\uae30\ub9cc \ud558\uba74 \ub429\ub2c8\ub2e4.\n" ] }, { @@ -46,7 +46,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/2a6717c6d63c217a6ca736f62c4dc768/tf_two_layer_net.ipynb b/docs/_downloads/2a6717c6d63c217a6ca736f62c4dc768/tf_two_layer_net.ipynb index f6f932fe0..9e6073762 100644 --- a/docs/_downloads/2a6717c6d63c217a6ca736f62c4dc768/tf_two_layer_net.ipynb +++ b/docs/_downloads/2a6717c6d63c217a6ca736f62c4dc768/tf_two_layer_net.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nTensorFlow: \uc815\uc801 \uadf8\ub798\ud504(Static Graph)\n--------------------------------------\n\n\ud558\ub098\uc758 \uc740\ub2c9\uce35(hidden layer)\uacfc \ud3b8\ud5a5(bias)\uc774 \uc5c6\ub294 \uc644\uc804\ud788 \uc5f0\uacb0\ub41c ReLU \uc2e0\uacbd\ub9dd\uc744,\n\uc720\ud074\ub9ac\ub4dc \uac70\ub9ac(Euclidean distance) \uc81c\uacf1\uc744 \ucd5c\uc18c\ud654\ud558\ub294 \uc2dd\uc73c\ub85c x\ub85c\ubd80\ud130 y\ub97c \uc608\uce21\ud558\ub3c4\ub85d\n\ud559\uc2b5\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uae30\ubcf8\uc801\uc778 TensorFlow \uc5f0\uc0b0\uc744 \uc0ac\uc6a9\ud558\uc5ec \uc5f0\uc0b0 \uadf8\ub798\ud504\ub97c \uad6c\uc131\ud55c \ub2e4\uc74c, \uadf8\ub798\ud504\ub97c \uc5ec\ub7ec \ucc28\ub840\n\uc2e4\ud589\ud558\uc5ec \uc2e4\uc81c\ub85c \uc2e0\uacbd\ub9dd\uc744 \ud559\uc2b5\uc2dc\ucf1c\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\nTensorflow\uc640 PyTorch\uc758 \uc8fc\uc694\ud55c \ucc28\uc774\uc810 \uc911 \ud558\ub098\ub294, PyTorch\ub294 \ub3d9\uc801 \uc5f0\uc0b0 \uadf8\ub798\ud504\ub97c\n\uc0ac\uc6a9\ud558\ub294\ub370 \ubc18\ud574 Tensorflow\ub294 \uc815\uc801 \uc5f0\uc0b0 \uadf8\ub798\ud504\ub97c \uc0ac\uc6a9\ud55c\ub2e4\ub294 \uac83\uc785\ub2c8\ub2e4.\n\n\uba3c\uc800 Tensorflow\uc5d0\uc11c \uc5f0\uc0b0 \uadf8\ub798\ud504\ub97c \uad6c\uc131\ud55c \ub2e4\uc74c, \ud574\ub2f9 \uadf8\ub798\ud504\ub97c \uc5ec\ub7ec \ucc28\ub840\n\uc2e4\ud589\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n" + "\nTensorFlow: \uc815\uc801 \uadf8\ub798\ud504(Static Graph)\n--------------------------------------\n\n\ud558\ub098\uc758 \uc740\ub2c9\uce35(hidden layer)\uacfc \ud3b8\ud5a5(bias)\uc774 \uc5c6\ub294 \uc644\uc804\ud788 \uc5f0\uacb0\ub41c ReLU \uc2e0\uacbd\ub9dd\uc744,\n\uc720\ud074\ub9ac\ub4dc \uac70\ub9ac(Euclidean distance) \uc81c\uacf1\uc744 \ucd5c\uc18c\ud654\ud558\ub294 \uc2dd\uc73c\ub85c x\ub85c\ubd80\ud130 y\ub97c \uc608\uce21\ud558\ub3c4\ub85d\n\ud559\uc2b5\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uae30\ubcf8\uc801\uc778 TensorFlow \uc5f0\uc0b0\uc744 \uc0ac\uc6a9\ud558\uc5ec \uc5f0\uc0b0 \uadf8\ub798\ud504\ub97c \uad6c\uc131\ud55c \ub2e4\uc74c, \uadf8\ub798\ud504\ub97c \uc5ec\ub7ec \ucc28\ub840\n\uc2e4\ud589\ud558\uc5ec \uc2e4\uc81c\ub85c \uc2e0\uacbd\ub9dd\uc744 \ud559\uc2b5\uc2dc\ucf1c\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\nTensorflow\uc640 PyTorch\uc758 \uc8fc\uc694\ud55c \ucc28\uc774\uc810 \uc911 \ud558\ub098\ub294, PyTorch\ub294 \ub3d9\uc801 \uc5f0\uc0b0 \uadf8\ub798\ud504\ub97c\n\uc0ac\uc6a9\ud558\ub294\ub370 \ubc18\ud574 Tensorflow\ub294 \uc815\uc801 \uc5f0\uc0b0 \uadf8\ub798\ud504\ub97c \uc0ac\uc6a9\ud55c\ub2e4\ub294 \uac83\uc785\ub2c8\ub2e4.\n\n\uba3c\uc800 Tensorflow\uc5d0\uc11c \uc5f0\uc0b0 \uadf8\ub798\ud504\ub97c \uad6c\uc131\ud55c \ub2e4\uc74c, \ud574\ub2f9 \uadf8\ub798\ud504\ub97c \uc5ec\ub7ec \ucc28\ub840\n\uc2e4\ud589\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n" ] }, { @@ -46,7 +46,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/2a7469f7250c4f405b1c4f885994ea69/advanced_tutorial.py b/docs/_downloads/2a7469f7250c4f405b1c4f885994ea69/advanced_tutorial.py index 0d03593f9..b4cda9e34 100644 --- a/docs/_downloads/2a7469f7250c4f405b1c4f885994ea69/advanced_tutorial.py +++ b/docs/_downloads/2a7469f7250c4f405b1c4f885994ea69/advanced_tutorial.py @@ -1,120 +1,112 @@ # -*- coding: utf-8 -*- r""" -Advanced: Making Dynamic Decisions and the Bi-LSTM CRF +심화 과정 : Bi-LSTM CRF와 동적 결정 ====================================================== -Dynamic versus Static Deep Learning Toolkits +동적, 정적 딥 러닝 툴킷(toolkits) 비교 -------------------------------------------- -Pytorch is a *dynamic* neural network kit. Another example of a dynamic -kit is `Dynet `__ (I mention this because -working with Pytorch and Dynet is similar. If you see an example in -Dynet, it will probably help you implement it in Pytorch). The opposite -is the *static* tool kit, which includes Theano, Keras, TensorFlow, etc. -The core difference is the following: - -* In a static toolkit, you define - a computation graph once, compile it, and then stream instances to it. -* In a dynamic toolkit, you define a computation graph *for each - instance*. It is never compiled and is executed on-the-fly - -Without a lot of experience, it is difficult to appreciate the -difference. One example is to suppose we want to build a deep -constituent parser. Suppose our model involves roughly the following -steps: - -* We build the tree bottom up -* Tag the root nodes (the words of the sentence) -* From there, use a neural network and the embeddings - of the words to find combinations that form constituents. Whenever you - form a new constituent, use some sort of technique to get an embedding - of the constituent. In this case, our network architecture will depend - completely on the input sentence. In the sentence "The green cat - scratched the wall", at some point in the model, we will want to combine - the span :math:`(i,j,r) = (1, 3, \text{NP})` (that is, an NP constituent - spans word 1 to word 3, in this case "The green cat"). - -However, another sentence might be "Somewhere, the big fat cat scratched -the wall". In this sentence, we will want to form the constituent -:math:`(2, 4, NP)` at some point. The constituents we will want to form -will depend on the instance. If we just compile the computation graph -once, as in a static toolkit, it will be exceptionally difficult or -impossible to program this logic. In a dynamic toolkit though, there -isn't just 1 pre-defined computation graph. There can be a new -computation graph for each instance, so this problem goes away. - -Dynamic toolkits also have the advantage of being easier to debug and -the code more closely resembling the host language (by that I mean that -Pytorch and Dynet look more like actual Python code than Keras or -Theano). - -Bi-LSTM Conditional Random Field Discussion +Pytorch는 *동적* 신경망 툴킷입니다. 다른 동적 신경망 툴킷으로는 +`Dynet `_ 이 있습니다.(이 툴킷을 +예로 든 이유는 사용하는 법이 Pytorch와 비슷하기 때문입니다. Dynet의 예제를 보면 +Pytorch로 구현할 때도 도움이 될 것입니다.) 반대로 *정적* 툴킷들로 +Theano, Keras, TensorFlow 등이 있습니다. 주요 차이점은 다음과 같습니다: + +* 정적 툴킷을 쓸 때는 계산 그래프를 한 번만 정의하고, 컴파일 한 후, + 데이터를 계산 그래프에 넘깁니다. +* 동적 툴킷에서는 *각 데이터* 의 계산 그래프를 정의하며 컴파일하지 + 않고 즉각 실행됩니다. + +경험이 많지 않다면 두 방식의 차이를 알기 어렵습니다. 딥 러닝 기반의 +구구조 분석기(constituent parser)를 예로 들어보겠습니다. 모델은 대략 +다음과 같은 과정을 수행합니다: + +* 트리를 상향식(bottom-up)으로 만들어 나갑니다. +* 최상위 노드를 태깅합니다. (문장의 각 단어) +* 거기서부터 신경망과 단어들의 임베딩을 이용해 구구조를 이루는 조합을 + 찾아냅니다. 새로운 구구조를 생성할 때마다 구구조의 임베딩을 얻기 위한 + 어떤 기술이 필요합니다. 지금은 신경망이 오직 입력 문장만 참고할 + 것입니다. "The green cat scratched the wall"이란 문장에서, 모델의 어느 시점에 + :math:`(i,j,r) = (1, 3, \text{NP})` 범위 (단어 1에서부터 단어 3까지가 + NP 구구조라는 뜻이며, 이 문장에서는 "The green cat") 를 결합하길 원할 + 것입니다. + +그런데, 또다른 문장 "Somewhere, the big fat cat scratched the wall" 에서는 +어느 시점에 :math:`(2, 4, NP)` 구구조를 만들기를 원할 것입니다. 우리가 +만들기 원하는 구구조들은 문장에 따라 다릅니다. 만약 정적 툴킷에서처럼 +계산 그래프를 한 번만 컴파일한다면, 이 과정을 프로그래밍하기 매우 어렵거나 +불가능할 것입니다. 하지만 동적 툴킷에서는 하나의 계산 그래프만 있지 +않습니다. 각 문장들마다 새로운 계산 그래프가 있을 수 있기 때문에 이런 +문제가 없습니다. + +동적 틀킷은 디버깅 하기 더 쉽고, 코드가 기반 언어와 더 비슷합니다 +(Pytorch와 Dynet이 Keras 또는 Theano 보다 Python 코드와 더 비슷합니다). + +Bi-LSTM Conditional Rnadom Field 설명 ------------------------------------------- -For this section, we will see a full, complicated example of a Bi-LSTM -Conditional Random Field for named-entity recognition. The LSTM tagger -above is typically sufficient for part-of-speech tagging, but a sequence -model like the CRF is really essential for strong performance on NER. -Familiarity with CRF's is assumed. Although this name sounds scary, all -the model is is a CRF but where an LSTM provides the features. This is -an advanced model though, far more complicated than any earlier model in -this tutorial. If you want to skip it, that is fine. To see if you're -ready, see if you can: - -- Write the recurrence for the viterbi variable at step i for tag k. -- Modify the above recurrence to compute the forward variables instead. -- Modify again the above recurrence to compute the forward variables in - log-space (hint: log-sum-exp) - -If you can do those three things, you should be able to understand the -code below. Recall that the CRF computes a conditional probability. Let -:math:`y` be a tag sequence and :math:`x` an input sequence of words. -Then we compute +이 영역에서는 개체명 인식을 수행하는 완성된 Bi-LSTM Conditional Random +Field 예시를 살펴보겠습니다. 위에 나온 LSTM 태거(tagger)는 일반적으로 +품사 태깅을 하기에 충분합니다. 하지만 CRF 같은 연속된 데이터를 다루는 +모델은 좋은 개체명 인식 모델(NER)에 꼭 필요합니다. 여러분이 CRF를 잘 알고 +있다고 가정하겠습니다. 이름이 무섭게 들릴 수도 있지만, LSTM이 특징을 +제공하는 점을 제외하면 이 모델은 CRF 입니다. 하지만 더 발전된 모델이며, +이 튜토리얼의 앞부분에 나왔던 모델보다 훨씬 복잡합니다. 넘어가고 싶다면 +넘어가도 괜찮습니다. 이해할 수 있다고 생각한다면, 아래를 읽어보세요: + +- 태그 k에 대한 i번째 단계의 비터비(viterbi) 변수를 위해 순환 흐름을 만든다. +- 순방향 변수를 계산하기 위해 위의 순한 흐름을 조정한다. +- 순방향 변수를 로그 공간에서 계산하기 위해 다시 한 번 조정한다. + (힌트 : 로그-합-지수승) + +위의 세가지를 할 수 있다면, 아래의 코드를 이해할 수 있을 것입니다. +CRF는 조건부 확률을 계산한다는 점을 기억하세요. :math:`y` 를 연속된 +태그라 하고, :math:`x` 를 연속된 입력 단어라 하겠습니다. 그러면 아래의 +식을 계산할 수 있습니다. .. math:: P(y|x) = \frac{\exp{(\text{Score}(x, y)})}{\sum_{y'} \exp{(\text{Score}(x, y')})} -Where the score is determined by defining some log potentials -:math:`\log \psi_i(x,y)` such that +점수(score) 함수는 아래와 같이 정의된 로그 포텐셜(potential) :math:`\log \psi_i(x,y)` +함수에 의해 결정됩니다. .. math:: \text{Score}(x,y) = \sum_i \log \psi_i(x,y) -To make the partition function tractable, the potentials must look only -at local features. +분배 함수(partition function)를 단순화하기 위해서, 포텐셜이 주변의 +특징들만 반영한다고 하겠습니다. -In the Bi-LSTM CRF, we define two kinds of potentials: emission and -transition. The emission potential for the word at index :math:`i` comes -from the hidden state of the Bi-LSTM at timestep :math:`i`. The -transition scores are stored in a :math:`|T|x|T|` matrix -:math:`\textbf{P}`, where :math:`T` is the tag set. In my -implementation, :math:`\textbf{P}_{j,k}` is the score of transitioning -to tag :math:`j` from tag :math:`k`. So: +Bi-LSTM CRF 안에 배출(emission), 전이(transition) 두 종류의 포텐셜을 +정의합니다. :math:`i` 번째 단어에 대한 배출 포텐셜은 Bi-LSTM의 +:math:`i` 번째 시점의 은닉 상태가 결정합니다. 전이 점수는 :math:`|T|x|T|` +형태인 행렬 :math:`\textbf{P}` 에 저장되어 있습니다. :math:`T` 는 +태그의 집합입니다. 이 구현에서, :math:`\textbf{P}_{j,k}` 는 tag :math:`j` 에서 +tag :math:`k` 로의 전이 점수를 의미합니다. 따라서: .. math:: \text{Score}(x,y) = \sum_i \log \psi_\text{EMIT}(y_i \rightarrow x_i) + \log \psi_\text{TRANS}(y_{i-1} \rightarrow y_i) .. math:: = \sum_i h_i[y_i] + \textbf{P}_{y_i, y_{i-1}} -where in this second expression, we think of the tags as being assigned -unique non-negative indices. +두 번째 식에서 고유하고 음수가 아닌 인덱스에 의해 태그가 부여됐다고 +간주합니다. -If the above discussion was too brief, you can check out -`this `__ write up from -Michael Collins on CRFs. +위의 설명이 너무 간단하다고 생각한다면 CRF에 대한 Michael Collins의 +글을 `여기 `__ 에서 +읽어보세요. -Implementation Notes +구현 문서 -------------------- -The example below implements the forward algorithm in log space to -compute the partition function, and the viterbi algorithm to decode. -Backpropagation will compute the gradients automatically for us. We -don't have to do anything by hand. +아래의 예시는 로그 공간에서 분배 함수를 계산하기 위한 순방향 알고리즘과 +복호화하기 위한 비터비 알고리즘을 구현한 것입니다. 역전파 +단계에서 변화도는 자동으로 계산될 것입니다. 우리가 직접 할 일은 +없습니다. -The implementation is not optimized. If you understand what is going on, -you'll probably quickly see that iterating over the next tag in the -forward algorithm could probably be done in one big operation. I wanted -to code to be more readable. If you want to make the relevant change, -you could probably use this tagger for real tasks. +이 구현은 최적의 상태가 아닙니다. 과정을 이해했다면, 순방향 알고리즘 +상에서 다음 태그를 순차적으로 처리하는 과정을 하나의 큰 연산으로 줄일 +수 있다는 것을 아마 빠르게 알 수 있을 것입니다. 이 코드는 가능한 읽기 +쉽게 작성했습니다. 적절하게 수정하면, 이 태거를 실제 문제들에 사용할 +수도 있을 것입니다. """ -# Author: Robert Guthrie +# 작성자: Robert Guthrie import torch import torch.autograd as autograd @@ -124,11 +116,11 @@ torch.manual_seed(1) ##################################################################### -# Helper functions to make the code more readable. +# 코드 가독성을 높여주는 보조 함수들 def argmax(vec): - # return the argmax as a python int + # argmax를 파이썬 정수형으로 반환합니다. _, idx = torch.max(vec, 1) return idx.item() @@ -138,7 +130,7 @@ def prepare_sequence(seq, to_ix): return torch.tensor(idxs, dtype=torch.long) -# Compute log sum exp in a numerically stable way for the forward algorithm +# 순방향 알고리즘을 위해 수치적으로 안정적인 방법으로 로그 합 지수승을 계산합니다. def log_sum_exp(vec): max_score = vec[0, argmax(vec)] max_score_broadcast = max_score.view(1, -1).expand(1, vec.size()[1]) @@ -146,7 +138,7 @@ def log_sum_exp(vec): torch.log(torch.sum(torch.exp(vec - max_score_broadcast))) ##################################################################### -# Create model +# 모델 생성 class BiLSTM_CRF(nn.Module): @@ -163,16 +155,15 @@ def __init__(self, vocab_size, tag_to_ix, embedding_dim, hidden_dim): self.lstm = nn.LSTM(embedding_dim, hidden_dim // 2, num_layers=1, bidirectional=True) - # Maps the output of the LSTM into tag space. + # LSTM의 출력을 태그 공간으로 대응시킵니다. self.hidden2tag = nn.Linear(hidden_dim, self.tagset_size) - # Matrix of transition parameters. Entry i,j is the score of - # transitioning *to* i *from* j. + # 전이 매개변수 행렬. i, j 성분은 i에서 j로 변할 때의 점수입니다. self.transitions = nn.Parameter( torch.randn(self.tagset_size, self.tagset_size)) - # These two statements enforce the constraint that we never transfer - # to the start tag and we never transfer from the stop tag + # 이 두 코드는 시작 태그로 전이하지 않고, 정지 태그에서부터 + # 전이하지 않도록 강제합니다. self.transitions.data[tag_to_ix[START_TAG], :] = -10000 self.transitions.data[:, tag_to_ix[STOP_TAG]] = -10000 @@ -183,30 +174,28 @@ def init_hidden(self): torch.randn(2, 1, self.hidden_dim // 2)) def _forward_alg(self, feats): - # Do the forward algorithm to compute the partition function + # 분배 함수를 계산하기 위해 순방향 알고리즘을 수행합니다. init_alphas = torch.full((1, self.tagset_size), -10000.) - # START_TAG has all of the score. + # START_TAG는 모든 점수를 갖고 있습니다. init_alphas[0][self.tag_to_ix[START_TAG]] = 0. - # Wrap in a variable so that we will get automatic backprop + # 자동으로 역전파 되도록 변수로 감쌉니다. forward_var = init_alphas - # Iterate through the sentence + # 문장의 각 성분을 반복 처리합니다. for feat in feats: - alphas_t = [] # The forward tensors at this timestep + alphas_t = [] # 현재 시점의 순방향 텐서 for next_tag in range(self.tagset_size): - # broadcast the emission score: it is the same regardless of - # the previous tag + # 이전의 태그와 상관없이 배출 점수를 전파합니다. emit_score = feat[next_tag].view( 1, -1).expand(1, self.tagset_size) - # the ith entry of trans_score is the score of transitioning to - # next_tag from i + # trans_score의 i번째 성분은 i로부터 next_tag로 전이할 점수입니다. trans_score = self.transitions[next_tag].view(1, -1) - # The ith entry of next_tag_var is the value for the - # edge (i -> next_tag) before we do log-sum-exp + # next_tag_var의 i번째 성분은 로그-합-지수승을 계산하기 전 + # i에서 next_tag로 가는 간선의 값입니다. next_tag_var = forward_var + trans_score + emit_score - # The forward variable for this tag is log-sum-exp of all the - # scores. + # 이 태그의 순방향 변수는 모든 점수들의 로그-합-지수승 계산 + # 결과입니다. alphas_t.append(log_sum_exp(next_tag_var).view(1)) forward_var = torch.cat(alphas_t).view(1, -1) terminal_var = forward_var + self.transitions[self.tag_to_ix[STOP_TAG]] @@ -222,7 +211,7 @@ def _get_lstm_features(self, sentence): return lstm_feats def _score_sentence(self, feats, tags): - # Gives the score of a provided tag sequence + # 주어진 태그 순열에 점수를 매깁니다. score = torch.zeros(1) tags = torch.cat([torch.tensor([self.tag_to_ix[START_TAG]], dtype=torch.long), tags]) for i, feat in enumerate(feats): @@ -234,44 +223,43 @@ def _score_sentence(self, feats, tags): def _viterbi_decode(self, feats): backpointers = [] - # Initialize the viterbi variables in log space + # 비터비 변수를 로그 공간 상에 초기화합니다. init_vvars = torch.full((1, self.tagset_size), -10000.) init_vvars[0][self.tag_to_ix[START_TAG]] = 0 - # forward_var at step i holds the viterbi variables for step i-1 + # i 단계의 forward_var는 i-1 단계의 비터비 변수를 갖고 있습니다. forward_var = init_vvars for feat in feats: - bptrs_t = [] # holds the backpointers for this step - viterbivars_t = [] # holds the viterbi variables for this step + bptrs_t = [] # 현재 단계의 backpointer를 갖고 있습니다. + viterbivars_t = [] # 현재 단계의 비터비 변수를 갖고 있습니다. for next_tag in range(self.tagset_size): - # next_tag_var[i] holds the viterbi variable for tag i at the - # previous step, plus the score of transitioning - # from tag i to next_tag. - # We don't include the emission scores here because the max - # does not depend on them (we add them in below) + # next_tag_var[i]는 이전 단계의 태그 i에 대한 비터비 변수와, + # 태그 i에서 next_tag로 전이할 점수를 더한 값을 갖고 있습니다. + # 배출 점수는 argmax와 상관 없기 때문에(아래 코드에서 추가할 것입니다) + # 여기에 포함되지 않습니다. next_tag_var = forward_var + self.transitions[next_tag] best_tag_id = argmax(next_tag_var) bptrs_t.append(best_tag_id) viterbivars_t.append(next_tag_var[0][best_tag_id].view(1)) - # Now add in the emission scores, and assign forward_var to the set - # of viterbi variables we just computed + # 이제 배출 점수를 더합니다. 그리고 방금 계산한 비터비 변수의 + # 집합을 forward_var에 할당합니다. forward_var = (torch.cat(viterbivars_t) + feat).view(1, -1) backpointers.append(bptrs_t) - # Transition to STOP_TAG + # STAP_TAG로의 전이 terminal_var = forward_var + self.transitions[self.tag_to_ix[STOP_TAG]] best_tag_id = argmax(terminal_var) path_score = terminal_var[0][best_tag_id] - # Follow the back pointers to decode the best path. + # 최적의 경로를 구하기 위해 back pointer를 따라갑니다. best_path = [best_tag_id] for bptrs_t in reversed(backpointers): best_tag_id = bptrs_t[best_tag_id] best_path.append(best_tag_id) - # Pop off the start tag (we dont want to return that to the caller) + # 시작 태그를 빼냅니다 (시작 태그는 반환된 필요가 없습니다) start = best_path.pop() - assert start == self.tag_to_ix[START_TAG] # Sanity check + assert start == self.tag_to_ix[START_TAG] # 완결성 검사 (Sanity check) best_path.reverse() return path_score, best_path @@ -281,16 +269,16 @@ def neg_log_likelihood(self, sentence, tags): gold_score = self._score_sentence(feats, tags) return forward_score - gold_score - def forward(self, sentence): # dont confuse this with _forward_alg above. - # Get the emission scores from the BiLSTM + def forward(self, sentence): # 이 함수와 위의 _forward_alg를 헷갈리지 마세요. + # Bi-LSTM으로부터 배출 점수를 얻습니다. lstm_feats = self._get_lstm_features(sentence) - # Find the best path, given the features. + # 주어진 특징(배출 점수)들로 최적의 경로를 찾아냅니다. score, tag_seq = self._viterbi_decode(lstm_feats) return score, tag_seq ##################################################################### -# Run training +# 훈련 실행 START_TAG = "" @@ -298,7 +286,7 @@ def forward(self, sentence): # dont confuse this with _forward_alg above. EMBEDDING_DIM = 5 HIDDEN_DIM = 4 -# Make up some training data +# 훈련용 데이터를 만듭니다. training_data = [( "the wall street journal reported today that apple corporation made money".split(), "B I I I O O O B I O O".split() @@ -318,57 +306,55 @@ def forward(self, sentence): # dont confuse this with _forward_alg above. model = BiLSTM_CRF(len(word_to_ix), tag_to_ix, EMBEDDING_DIM, HIDDEN_DIM) optimizer = optim.SGD(model.parameters(), lr=0.01, weight_decay=1e-4) -# Check predictions before training +# 훈련 전 예측 결과를 확인합니다. with torch.no_grad(): precheck_sent = prepare_sequence(training_data[0][0], word_to_ix) precheck_tags = torch.tensor([tag_to_ix[t] for t in training_data[0][1]], dtype=torch.long) print(model(precheck_sent)) # Make sure prepare_sequence from earlier in the LSTM section is loaded +# 위의 보조 함수 영역에 있는 prepare_sequence 함수가 불러와 졌는지 확인합니다. for epoch in range( - 300): # again, normally you would NOT do 300 epochs, it is toy data + 300): # 다시 말하지만, 아마 300 에폭을 실행하진 않을 것입니다. 이것은 연습용 데이터입니다. for sentence, tags in training_data: - # Step 1. Remember that Pytorch accumulates gradients. - # We need to clear them out before each instance + # 1단계. Pytorch가 변화도를 누적한다는 것을 기억하세요. + # 그것들을 제거합니다. model.zero_grad() - # Step 2. Get our inputs ready for the network, that is, - # turn them into Tensors of word indices. + # 2단계. 입력 데이터를 신경망에 사용될 수 있도록 단어 + # 인덱스들의 텐서로 변환합니다. sentence_in = prepare_sequence(sentence, word_to_ix) targets = torch.tensor([tag_to_ix[t] for t in tags], dtype=torch.long) - # Step 3. Run our forward pass. + # 3단계. 순방향 계산을 수행합니다. loss = model.neg_log_likelihood(sentence_in, targets) - # Step 4. Compute the loss, gradients, and update the parameters by - # calling optimizer.step() + # 4단계. 손실값, 변화도를 계산하고 optimizer.step()을 호출하여 + # 매개변수들을 갱신합니다. loss.backward() optimizer.step() -# Check predictions after training +# 훈련이 끝난 후 예측 결과를 확인합니다. with torch.no_grad(): precheck_sent = prepare_sequence(training_data[0][0], word_to_ix) print(model(precheck_sent)) -# We got it! +# 다 했습니다! ###################################################################### -# Exercise: A new loss function for discriminative tagging -# -------------------------------------------------------- +# 연습 : 판별적(discriminative) 태깅을 위한 새로운 손실 함수 +# ------------------------------------------------------------ # -# It wasn't really necessary for us to create a computation graph when -# doing decoding, since we do not backpropagate from the viterbi path -# score. Since we have it anyway, try training the tagger where the loss -# function is the difference between the Viterbi path score and the score -# of the gold-standard path. It should be clear that this function is -# non-negative and 0 when the predicted tag sequence is the correct tag -# sequence. This is essentially *structured perceptron*. +# 사실 복호화 할 때는 비터비 경로 점수로 역전파를 하지 않기 때문에 계산 +# 그래프를 만들 필요가 없었습니다. 그러나 이미 만들었으니, 비터비 경로 +# 점수와 실제 정답 경로 점수의 차이를 손실 함수로 사용해서 태거를 +# 학습시켜 보세요. 손실 함수의 값은 음수가 아니어야 하며, 예측된 태그 +# 순열이 정답이라면 손실 함수의 값은 0이어야 합니다. 이것은 본질적으로 +# *구조화된 퍼셉트론* 입니다. # -# This modification should be short, since Viterbi and score\_sentence are -# already implemented. This is an example of the shape of the computation -# graph *depending on the training instance*. Although I haven't tried -# implementing this in a static toolkit, I imagine that it is possible but -# much less straightforward. -# -# Pick up some real data and do a comparison! +# 이미 비터비와 score_sentence 함수가 구현되어 있기 때문에 간단히 수정할 +# 수 있습니다. 이 모델은 *학습 데이터에 따라 변하는* 계산 그래프의 한 +# 예시입니다. 이 모델을 정적 툴킷에서 구현해 보지는 않았는데, 구현이 +# 가능하지만 덜 직관적일 수 있습니다. # +# 실제 데이터를 사용해보고 비교해보세요! diff --git a/docs/_downloads/303a511ec52e0933ffe70a0320cc68dc/tensor_tutorial_old.ipynb b/docs/_downloads/303a511ec52e0933ffe70a0320cc68dc/tensor_tutorial_old.ipynb index 5233d3d5f..bcc852cfd 100644 --- a/docs/_downloads/303a511ec52e0933ffe70a0320cc68dc/tensor_tutorial_old.ipynb +++ b/docs/_downloads/303a511ec52e0933ffe70a0320cc68dc/tensor_tutorial_old.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nTensor\n=======\n\nPyTorch\uc5d0\uc11c\uc758 Tensor\ub294 Torch\uc5d0\uc11c\uc640 \uac70\uc758 \ub3d9\uc77c\ud558\uac8c \ub3d9\uc791\ud569\ub2c8\ub2e4.\n\n\ucd08\uae30\ud654\ub418\uc9c0 \uc54a\uc740 (5 x 7) \ud06c\uae30\uc758 Tensor\ub97c \uc0dd\uc131\ud569\ub2c8\ub2e4:\n\n\n" + "\nTensor\n=======\n\nPyTorch\uc5d0\uc11c\uc758 Tensor\ub294 Torch\uc5d0\uc11c\uc640 \uac70\uc758 \ub3d9\uc77c\ud558\uac8c \ub3d9\uc791\ud569\ub2c8\ub2e4.\n\n\ucd08\uae30\ud654\ub418\uc9c0 \uc54a\uc740 (5 x 7) \ud06c\uae30\uc758 Tensor\ub97c \uc0dd\uc131\ud569\ub2c8\ub2e4:\n" ] }, { @@ -234,7 +234,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/35afa48d3c96a0db7f505da116ff8978/learning_hybrid_frontend_through_example_tutorial.ipynb b/docs/_downloads/35afa48d3c96a0db7f505da116ff8978/learning_hybrid_frontend_through_example_tutorial.ipynb index 08d625a2c..f3985b4f4 100644 --- a/docs/_downloads/35afa48d3c96a0db7f505da116ff8978/learning_hybrid_frontend_through_example_tutorial.ipynb +++ b/docs/_downloads/35afa48d3c96a0db7f505da116ff8978/learning_hybrid_frontend_through_example_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nLearning Hybrid Frontend Syntax Through Example\n===============================================\n**Author:** `Nathan Inkawhich `_\n\nThis document is meant to highlight the syntax of the Hybrid Frontend\nthrough a non-code intensive example. The Hybrid Frontend is one of the\nnew shiny features of Pytorch 1.0 and provides an avenue for developers\nto transition their models from **eager** to **graph** mode. PyTorch\nusers are very familiar with eager mode as it provides the ease-of-use\nand flexibility that we all enjoy as researchers. Caffe2 users are more\naquainted with graph mode which has the benefits of speed, optimization\nopportunities, and functionality in C++ runtime environments. The hybrid\nfrontend bridges the gap between the the two modes by allowing\nresearchers to develop and refine their models in eager mode (i.e.\nPyTorch), then gradually transition the proven model to graph mode for\nproduction, when speed and resouce consumption become critical.\n\nHybrid Frontend Information\n---------------------------\n\nThe process for transitioning a model to graph mode is as follows.\nFirst, the developer constructs, trains, and tests the model in eager\nmode. Then they incrementally **trace** and **script** each\nfunction/module of the model with the Just-In-Time (JIT) compiler, at\neach step verifying that the output is correct. Finally, when each of\nthe components of the top-level model have been traced and scripted, the\nmodel itself is traced. At which point the model has been transitioned\nto graph mode, and has a complete python-free representation. With this\nrepresentation, the model runtime can take advantage of high-performance\nCaffe2 operators and graph based optimizations.\n\nBefore we continue, it is important to understand the idea of tracing\nand scripting, and why they are separate. The goal of **trace** and\n**script** is the same, and that is to create a graph representation of\nthe operations taking place in a given function. The discrepency comes\nfrom the flexibility of eager mode that allows for **data-dependent\ncontrol flows** within the model architecture. When a function does NOT\nhave a data-dependent control flow, it may be *traced* with\n``torch.jit.trace``. However, when the function *has* a data-dependent\ncontrol flow it must be *scripted* with ``torch.jit.script``. We will\nleave the details of the interworkings of the hybrid frontend for\nanother document, but the code example below will show the syntax of how\nto trace and script different pure python functions and torch Modules.\nHopefully, you will find that using the hybrid frontend is non-intrusive\nas it mostly involves adding decorators to the existing function and\nclass definitions.\n\nMotivating Example\n------------------\n\nIn this example we will implement a strange math function that may be\nlogically broken up into four parts that do, and do not contain\ndata-dependent control flows. The purpose here is to show a non-code\nintensive example where the use of the JIT is highlighted. This example\nis a stand-in representation of a useful model, whose implementation has\nbeen divided into various pure python functions and modules.\n\nThe function we seek to implement, $Y(x)$, is defined for\n$x \\epsilon \\mathbb{N}$ as\n\n\\begin{align}z(x) = \\Biggl \\lfloor \\frac{\\sqrt{\\prod_{i=1}^{|2 x|}i}}{5} \\Biggr \\rfloor\\end{align}\n\n\\begin{align}Y(x) = \\begin{cases}\n \\frac{z(x)}{2} & \\text{if } z(x)\\%2 == 0, \\\\\n z(x) & \\text{otherwise}\n \\end{cases}\\end{align}\n\n\\begin{align}\\begin{array}{| r | r |} \\hline\n x &1 &2 &3 &4 &5 &6 &7 \\\\ \\hline\n Y(x) &0 &0 &-5 &20 &190 &-4377 &-59051 \\\\ \\hline\n \\end{array}\\end{align}\n\nAs mentioned, the computation is split into four parts. Part one is the\nsimple tensor calculation of $|2x|$, which can be traced. Part two\nis the iterative product calculation that represents a data dependent\ncontrol flow to be scripted (the number of loop iteration depends on the\ninput at runtime). Part three is a trace-able\n$\\lfloor \\sqrt{a/5} \\rfloor$ calculation. Finally, part 4 handles\nthe output cases depending on the value of $z(x)$ and must be\nscripted due to the data dependency. Now, let's see how this looks in\ncode.\n\nPart 1 - Tracing a pure python function\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nWe can implement part one as a pure python function as below. Notice, to\ntrace this function we call ``torch.jit.trace`` and pass in the function\nto be traced. Since the trace requires a dummy input of the expected\nruntime type and shape, we also include the ``torch.rand`` to generate a\nsingle valued torch tensor.\n\n\n" + "\nLearning Hybrid Frontend Syntax Through Example\n===============================================\n**Author:** `Nathan Inkawhich `_\n\nThis document is meant to highlight the syntax of the Hybrid Frontend\nthrough a non-code intensive example. The Hybrid Frontend is one of the\nnew shiny features of Pytorch 1.0 and provides an avenue for developers\nto transition their models from **eager** to **graph** mode. PyTorch\nusers are very familiar with eager mode as it provides the ease-of-use\nand flexibility that we all enjoy as researchers. Caffe2 users are more\naquainted with graph mode which has the benefits of speed, optimization\nopportunities, and functionality in C++ runtime environments. The hybrid\nfrontend bridges the gap between the the two modes by allowing\nresearchers to develop and refine their models in eager mode (i.e.\nPyTorch), then gradually transition the proven model to graph mode for\nproduction, when speed and resouce consumption become critical.\n\nHybrid Frontend Information\n---------------------------\n\nThe process for transitioning a model to graph mode is as follows.\nFirst, the developer constructs, trains, and tests the model in eager\nmode. Then they incrementally **trace** and **script** each\nfunction/module of the model with the Just-In-Time (JIT) compiler, at\neach step verifying that the output is correct. Finally, when each of\nthe components of the top-level model have been traced and scripted, the\nmodel itself is traced. At which point the model has been transitioned\nto graph mode, and has a complete python-free representation. With this\nrepresentation, the model runtime can take advantage of high-performance\nCaffe2 operators and graph based optimizations.\n\nBefore we continue, it is important to understand the idea of tracing\nand scripting, and why they are separate. The goal of **trace** and\n**script** is the same, and that is to create a graph representation of\nthe operations taking place in a given function. The discrepency comes\nfrom the flexibility of eager mode that allows for **data-dependent\ncontrol flows** within the model architecture. When a function does NOT\nhave a data-dependent control flow, it may be *traced* with\n``torch.jit.trace``. However, when the function *has* a data-dependent\ncontrol flow it must be *scripted* with ``torch.jit.script``. We will\nleave the details of the interworkings of the hybrid frontend for\nanother document, but the code example below will show the syntax of how\nto trace and script different pure python functions and torch Modules.\nHopefully, you will find that using the hybrid frontend is non-intrusive\nas it mostly involves adding decorators to the existing function and\nclass definitions.\n\nMotivating Example\n------------------\n\nIn this example we will implement a strange math function that may be\nlogically broken up into four parts that do, and do not contain\ndata-dependent control flows. The purpose here is to show a non-code\nintensive example where the use of the JIT is highlighted. This example\nis a stand-in representation of a useful model, whose implementation has\nbeen divided into various pure python functions and modules.\n\nThe function we seek to implement, $Y(x)$, is defined for\n$x \\epsilon \\mathbb{N}$ as\n\n\\begin{align}z(x) = \\Biggl \\lfloor \\frac{\\sqrt{\\prod_{i=1}^{|2 x|}i}}{5} \\Biggr \\rfloor\\end{align}\n\n\\begin{align}Y(x) = \\begin{cases}\n \\frac{z(x)}{2} & \\text{if } z(x)\\%2 == 0, \\\\\n z(x) & \\text{otherwise}\n \\end{cases}\\end{align}\n\n\\begin{align}\\begin{array}{| r | r |} \\hline\n x &1 &2 &3 &4 &5 &6 &7 \\\\ \\hline\n Y(x) &0 &0 &-5 &20 &190 &-4377 &-59051 \\\\ \\hline\n \\end{array}\\end{align}\n\nAs mentioned, the computation is split into four parts. Part one is the\nsimple tensor calculation of $|2x|$, which can be traced. Part two\nis the iterative product calculation that represents a data dependent\ncontrol flow to be scripted (the number of loop iteration depends on the\ninput at runtime). Part three is a trace-able\n$\\lfloor \\sqrt{a/5} \\rfloor$ calculation. Finally, part 4 handles\nthe output cases depending on the value of $z(x)$ and must be\nscripted due to the data dependency. Now, let's see how this looks in\ncode.\n\nPart 1 - Tracing a pure python function\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nWe can implement part one as a pure python function as below. Notice, to\ntrace this function we call ``torch.jit.trace`` and pass in the function\nto be traced. Since the trace requires a dummy input of the expected\nruntime type and shape, we also include the ``torch.rand`` to generate a\nsingle valued torch tensor.\n" ] }, { @@ -161,7 +161,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/363afc3b7c522e4e56981679c22f1ad6/word_embeddings_tutorial.ipynb b/docs/_downloads/363afc3b7c522e4e56981679c22f1ad6/word_embeddings_tutorial.ipynb index 035b6c7c2..6db7f7875 100644 --- a/docs/_downloads/363afc3b7c522e4e56981679c22f1ad6/word_embeddings_tutorial.ipynb +++ b/docs/_downloads/363afc3b7c522e4e56981679c22f1ad6/word_embeddings_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n\ub2e8\uc5b4 \uc784\ubca0\ub529: \uc5b4\ud718\uc758 \uc758\ubbf8\ub97c \uc778\ucf54\ub529\ud558\uae30\n===========================================\n**\ubc88\uc5ed**: `\uc784\uc131\uc5f0 `_\n\n\ub2e8\uc5b4 \uc784\ubca0\ub529(word embedding)\uc774\ub780 \ub9d0\ubb49\uce58(\ud639\uc740 \ucf54\ud37c\uc2a4, corpus) \ub0b4 \uac01 \ub2e8\uc5b4\uc5d0 \uc77c\ub300\uc77c\ub85c \ub300\uc751\ud558\ub294 \ubc00\uc9d1\ub41c \uc2e4\uc218 \ubca1\ud130(dense vector)\uc758 \uc9d1\ud569, \ud639\uc740 \uc774 \ubca1\ud130\ub97c\n\uad6c\ud558\ub294 \ud589\uc704\ub97c \uac00\ub9ac\ud0b5\ub2c8\ub2e4. \uc8fc\ub85c \ub2e8\uc5b4\ub97c \ud53c\ucc98(feature)\ub85c \uc0ac\uc6a9\ud558\ub294 \uc790\uc5f0\uc5b4 \ucc98\ub9ac \ubd84\uc57c\uc5d0\uc11c\ub294 \ub2e8\uc5b4\ub97c \ucef4\ud4e8\ud130 \uce5c\ud654\uc801\uc778\n\ud615\ud0dc\ub85c \ubc14\uafb8\uc5b4 \uc8fc\ub294 \uc791\uc5c5\uc774 \ud544\uc218\uc801\uc785\ub2c8\ub2e4. \ucef4\ud4e8\ud130\uac00 \ub2e8\uc5b4\ub97c \ubc14\ub85c \uc774\ud574\ud558\uae30\ub294 \uc0c1\ub2f9\ud788 \uc5b4\ub835\uae30 \ub54c\ubb38\uc774\uc8e0.\n\uadf8\ub807\ub2e4\uba74, \ub2e8\uc5b4\ub97c \uc5b4\ub5bb\uac8c \ud45c\ud604\ud558\ub294 \uac83\uc774 \uc88b\uc744\uae4c\uc694? \ubb3c\ub860 \uac01 \ubb38\uc790\uc5d0 \ud574\ub2f9\ud558\ub294 ASCII\ucf54\ub4dc\ub97c \uc0ac\uc6a9\ud560 \uc218 \uc788\uaca0\uc9c0\ub9cc,\nASCII\ucf54\ub4dc\ub294 \uc774 \ub2e8\uc5b4\uac00 *\ubb34\uc5c7* \uc778\uc9c0\ub97c \uc54c\ub824\uc904 \ubfd0, \ub2e8\uc5b4\uac00 \uc5b4\ub5a4 *\uc758\ubbf8* \ub97c \uac00\uc9c0\ub294\uc9c0\ub294 \uc54c\ub824\uc8fc\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4.\n(\ub8f0\ubca0\uc774\uc2a4\ub85c \uc5b4\ubbf8 \ub4f1 \ubb38\ubc95\uc801 \ud2b9\uc9d5\uc744 \ud65c\uc6a9\ud558\uac70\ub098 \uc601\uc5b4\uc758 \uacbd\uc6b0 \ub300\ubb38\uc790\ub97c \uc0ac\uc6a9\ud560 \uc218 \uc788\uaca0\uc9c0\ub9cc \ucda9\ubd84\ud558\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4.)\n\ub2e8\uc5b4\ub97c \uc5b4\ub5bb\uac8c \ud45c\ud604\ud560\uc9c0 \ubfd0 \uc544\ub2c8\ub77c, \uc774 \ud45c\ud604\ubc95\uc744 \uc5b4\ub5a0\ud55c \ubc29\uc2dd\uc73c\ub85c \uc5f0\uc0b0\ud574\uc57c \ud560 \uc9c0 \ub610\ud55c \ud070 \ubb38\uc81c\uc785\ub2c8\ub2e4.\n\ubcf4\ud1b5 \uc774\ub7ec\ud55c \ubc00\ub3c4 \ub192\uc740 \ubca1\ud130\ub97c \uc5bb\uae30 \uc704\ud574 \uc0ac\uc6a9\ud558\ub294 \ub274\ub7f4\ub137 \ubaa8\ub378\uc740 $|V|$ (\ub9d0\ubb49\uce58\uc758 \ub2e8\uc5b4 \uac1c\uc218)\uc758\n\ud070 \uc785\ub825 \ucc28\uc6d0\uacfc \uba87 \uc548\ub418\ub294 (\ud14d\uc2a4\ub97c \ubd84\ub958\ud558\ub294 \ubb38\uc81c\ub77c\uace0 \ud560 \uacbd\uc6b0) \uc791\uc740 \ucd9c\ub825 \ucc28\uc6d0\uc744 \uac00\uc9d1\ub2c8\ub2e4.\n\uc989, \ub2e8\uc5b4\ub4e4 \uac04\uc758 \uc5f0\uc0b0\uc774 \ud544\uc218\uc785\ub2c8\ub2e4. \uc5b4\ub5bb\uac8c \uc774 \ud070 \ucc28\uc6d0\uc758 \uacf5\uac04\uc744 \uc791\uc740 \uacf5\uac04\uc73c\ub85c \ubcc0\ud615\uc2dc\ud0ac \uc218 \uc788\uc744\uae4c\uc694?\n\n\uba3c\uc800, \uc0c1\uae30\ud55c ASCII\ucf54\ub4dc \ub300\uc2e0 \uc6d0\ud56b \uc778\ucf54\ub529(one-hot encoding)\uc744 \uc0ac\uc6a9\ud574\ubcf4\ub294 \uac83\uc740 \uc5b4\ub5a8\uae4c\uc694? \uc6d0\ud56b \uc778\ucf54\ub529\uc774\ub780\n\ud558\ub098\uc758 \ub2e8\uc5b4 $w$ \ub97c \uc544\ub798\uc758 \ubca1\ud130\ub85c \ud45c\ud604\ud558\ub294 \uac83\uc744 \ub9d0\ud569\ub2c8\ub2e4.\n\n\\begin{align}\\overbrace{\\left[ 0, 0, \\dots, 1, \\dots, 0, 0 \\right]}^\\text{|V| elements}\\end{align}\n\n\uc5ec\uae30\uc11c 1\uc740 \ud574\ub2f9 \ubca1\ud130\uac00 \ud45c\ud604\ud558\uace0\uc790 \ud558\ub294 \ub2e8\uc5b4\uc5d0 \ud574\ub2f9\ud558\ub294 \uc704\uce58 1\uacf3\uc5d0 \uc790\ub9ac\ud569\ub2c8\ub2e4. (\ub098\uba38\uc9c0\ub294 \uc804\ubd80\n0\uc785\ub2c8\ub2e4.) \ub2e4\ub978 \ub2e8\uc5b4\ub97c \ub098\ud0c0\ub0b4\ub294 \ubca1\ud130\uc5d0\uc120 1\uc774 \ub2e4\ub978 \uacf3\uc5d0 \uc704\uce58\ud574 \uc788\uaca0\uc8e0.\n\n\uc6d0\ud56b \uc778\ucf54\ub529\uc740 \ub9cc\ub4e4\uae30\uac00 \uc27d\ub2e4\ub294 \uc7a5\uc810\uc774 \uc788\uc9c0\ub9cc, \ub2e8\uc21c\ud55c \ub9cc\ud07c \ub2e8\uc810\ub3c4 \uc788\uc2b5\ub2c8\ub2e4. \uc77c\ub2e8 \ub2e8\uc5b4 \ubca1\ud130 \ud55c \uac1c\ub294\n\ubaa8\ub4e0 \ub2e8\uc5b4\ub97c \ud45c\ud604\ud560 \uc218 \uc788\uc744 \ub9cc\ud55c \ud06c\uae30\uac00 \ub418\uc5b4\uc57c \ud569\ub2c8\ub2e4. \uc6b0\ub9ac\uac00 \uc5bc\ub9c8\ub098 \ub9ce\uc740 \uc885\ub958\uc758 \ub2e8\uc5b4\ub97c\n\uc0ac\uc6a9\ud558\ub294\uc9c0\ub97c \uc0dd\uac01 \ud55c\ub2e4\uba74 \uc5b4\ub9c8\uc5b4\ub9c8\ud558\uac8c \ud070 \ubca1\ud130\ub77c\ub294 \uac83\uc744 \uc54c \uc218 \uc788\uc8e0. \uc774 \ubfd0\ub9cc\uc774 \uc544\ub2d9\ub2c8\ub2e4.\n\uc6d0\ud56b \ubca1\ud130\ub294 \ubaa8\ub4e0 \ub2e8\uc5b4\ub97c \ub3c5\ub9bd\uc801\uc778 \uac1c\uccb4\ub85c \uac00\uc815\ud558\ub294 \uac83\uc744 \ubcfc \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc989, \uacf5\uac04\uc0c1\uc5d0\uc11c\n\uc644\uc804\ud788 \ub2e4\ub978 \ucd95\uc5d0 \uc704\uce58\ud574 \uc788\uc5b4\uc11c \ub2e8\uc5b4\uac04\uc758 \uad00\uacc4\ub97c \ub098\ud0c0\ub0bc \uc218\uac00 \uc5c6\uc2b5\ub2c8\ub2e4. \ud558\uc9c0\ub9cc \uc6b0\ub9ac\ub294 \ub2e8\uc5b4\n\uc0ac\uc774\uc758 *\uc720\uc0ac\ub3c4* \ub97c \uc5b4\ub5bb\uac8c\ub4e0 \uacc4\uc0b0\ud558\uace0 \uc2f6\uc740\uac70\uc8e0. \uc65c \uc720\uc0ac\ub3c4\uac00 \uc911\uc694\ud558\ub0d0\uad6c\uc694? \ub2e4\uc74c \uc608\uc81c\ub97c \ubd05\uc2dc\ub2e4.\n\n\uc6b0\ub9ac\uc758 \ubaa9\ud45c\uac00 \uc5b8\uc5b4 \ubaa8\ub378\uc744 \ub9cc\ub4dc\ub294 \uac83\uc774\ub77c\uace0 \uac00\uc815\ud558\uace0 \ub2e4\uc74c\uc758 \ubb38\uc7a5\uc774 \ud559\uc2b5 \ub370\uc774\ud130\ub85c\uc368 \uc8fc\uc5b4\uc84c\ub2e4\uace0 \ud574\ubd05\uc2dc\ub2e4.\n\n* \uc218\ud559\uc790\uac00 \uac00\uac8c\ub85c \ub6f0\uc5b4\uac14\ub2e4.\n* \ubb3c\ub9ac\ud559\uc790\uac00 \uac00\uac8c\ub85c \ub6f0\uc5b4\uac14\ub2e4.\n* \uc218\ud559\uc790\uac00 \ub9ac\ub9cc \uac00\uc124\uc744 \uc99d\uba85\ud588\ub2e4.\n\n\ub610\ud55c \ud559\uc2b5 \ub370\uc774\ud130\uc5d0\ub294 \uc5c6\ub294 \uc544\ub798 \ubb38\uc7a5\uc774 \uc788\ub2e4\uace0 \uc0dd\uac01\ud574\ubd05\uc2dc\ub2e4.\n\n* \ubb3c\ub9ac\ud559\uc790\uac00 \ub9ac\ub9cc \uac00\uc124\uc744 \uc99d\uba85\ud588\ub2e4.\n\nASCII \ucf54\ub4dc\ub098 \uc6d0\ud56b \uc778\ucf54\ub529 \uae30\ubc18 \uc5b8\uc5b4 \ubaa8\ub378\uc740 \uc704 \ubb38\uc7a5\uc744 \uc5b4\ub290\uc815\ub3c4 \ub2e4\ub8f0 \uc218 \uc788\uaca0\uc9c0\ub9cc, \uac1c\uc120\uc758 \uc5ec\uc9c0\uac00 \uc788\uc9c0 \uc54a\uc744\uae4c\uc694?\n\uba3c\uc800 \uc544\ub798\uc758 \ub450 \uc0ac\uc2e4\uc744 \uc0dd\uac01\ud574\ubd05\uc2dc\ub2e4.\n\n* '\uc218\ud559\uc790'\uc640 '\ubb3c\ub9ac\ud559\uc790'\uac00 \ubb38\uc7a5 \ub0b4\uc5d0\uc11c \uac19\uc740 \uc5ed\ud560\uc744 \ub9e1\uace0 \uc788\uc2b5\ub2c8\ub2e4. \uc774 \ub450 \ub2e8\uc5b4\ub294 \uc5b4\ub5bb\uac8c\ub4e0 \uc758\ubbf8\uc801\uc778 \uc5f0\uad00\uc131\uc774 \uc788\uc744 \uac81\ub2c8\ub2e4.\n* \uc0c8\ub85c\uc6b4 \ubb38\uc7a5\uc5d0\uc11c '\ubb3c\ub9ac\ud559\uc790'\uac00 \ub9e1\uc740 \uc5ed\ud560\uc744 '\uc218\ud559\uc790'\uac00 \ub9e1\ub294 \uac83\uc744 \ud559\uc2b5 \ub370\uc774\ud130\uc5d0\uc11c \ubcf8 \uc801\uc774 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc6b0\ub9ac \ubaa8\ub378\uc774 \uc704\uc758 \uc0ac\uc2e4\uc744 \ud1b5\ud574 '\ubb3c\ub9ac\ud559\uc790'\uac00 \uc0c8 \ubb38\uc7a5\uc5d0 \uc798 \ub4e4\uc5b4 \ub9de\ub294\ub2e4\ub294 \uac83\uc744 \ucd94\ub860\ud560 \uc218\n\uc788\ub2e4\uba74 \ucc38 \uc88b\uc744 \uac83\uc785\ub2c8\ub2e4. \uc774\uac83\uc774 \uc704\uc5d0\uc11c \uc5b8\uae09\ud55c \uc720\uc0ac\ub3c4\uc758 \uc758\ubbf8\uc785\ub2c8\ub2e4. \ucca0\uc790\uc801 \uc720\uc0ac\ub3c4 \ubfd0\n\uc544\ub2c8\ub77c *\uc758\ubbf8\uc801 \uc720\uc0ac\ub3c4* \uc778 \uac83\uc785\ub2c8\ub2e4. \uc774\uac83\uc774\uc57c\ub9d0\ub85c \uc5b8\uc5b4 \ub370\uc774\ud130\uc5d0 \ub0b4\uc7ac\ud558\ub294 \ud76c\ubc15\uc131(sparsity)\uc5d0\n\ub300\ud55c \ucc98\ubc29\uc774 \ub420 \uac83\uc785\ub2c8\ub2e4. \uc6b0\ub9ac\uac00 \ubcf8 \uac83\uacfc \uc544\uc9c1 \ubcf4\uc9c0 \uc54a\uc740 \uac83 \uc0ac\uc774\ub97c \uc774\uc5b4\uc8fc\ub294 \uac83\uc774\uc8e0.\n\uc55e\uc73c\ub85c\ub294 \ub2e4\uc74c\uc758 \uc5b8\uc5b4\ud559\uc801 \uae30\ubcf8 \uba85\uc81c\ub97c \uac00\uc815\ud558\ub3c4\ub85d \ud569\uc2dc\ub2e4. \ubc14\ub85c \ube44\uc2b7\ud55c \ub9e5\ub77d\uc5d0\uc11c \ub4f1\uc7a5\ud558\ub294\n\ub2e8\uc5b4\ub4e4\uc740 \uc11c\ub85c \uc758\ubbf8\uc801 \uc5f0\uad00\uc131\uc744 \uac00\uc9c4\ub2e4\ub294 \uac83\uc785\ub2c8\ub2e4. \uc5b8\uc5b4\ud559\uc801\uc73c\ub85c\ub294 `\ubd84\uc0b0 \uc758\ubbf8 \uac00\uc124(distributional\nhypothesis) `__ \uc774\ub77c\uace0\ub3c4 \ud569\ub2c8\ub2e4.\n\n\n\ubc00\uc9d1\ub41c \ub2e8\uc5b4 \uc784\ubca0\ub529 \uad6c\ud558\uae30\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n\uc5b4\ub5bb\uac8c \ub2e8\uc5b4\uc758 \uc758\ubbf8\uc801 \uc720\uc0ac\ub3c4\ub97c \uc778\ucf54\ub529 \ud560 \uc218 \uc788\uc744\uae4c\uc694? \ub2e4\uc2dc \ub9d0\ud574, \uc5b4\ub5bb\uac8c \ud574\uc57c \ub2e8\uc5b4\uc758 \uc720\uc0ac\ub3c4\ub97c\n\ub2e8\uc5b4 \ubca1\ud130\uc5d0 \ubc18\uc601\ud560 \uc218 \uc788\uc744\uae4c\uc694? \ub2e8\uc5b4 \ub370\uc774\ud130\uc5d0 \uc758\ubbf8\uc801 \uc18d\uc131(attribute)\uc744 \ubd80\uc5ec\ud558\ub294 \uac74 \uc5b4\ub5a4\uac00\uc694?\n\uc608\ub97c \ub4e4\uc5b4 '\uc218\ud559\uc790'\uc640 '\ubb3c\ub9ac\ud559\uc790'\uac00 \ubaa8\ub450 \ub6f8 \uc218 \uc788\ub2e4\uba74, \ud574\ub2f9 \ub2e8\uc5b4\uc758 '\ub6f8 \uc218 \uc788\uc74c' \uc18d\uc131\uc5d0 \ub192\uc740 \uc810\uc218\ub97c \uc8fc\ub294 \uac81\ub2c8\ub2e4.\n\uacc4\uc18d \ud574\ubd05\uc2dc\ub2e4. \ub2e4\ub978 \ub2e8\uc5b4\ub4e4\uc5d0 \ub300\ud574\uc11c\ub294 \uc5b4\ub5a0\ud55c \uc18d\uc131\uc744 \ub9cc\ub4e4 \uc218 \uc788\uc744\uc9c0 \uc0dd\uac01\ud574\ubd05\uc2dc\ub2e4.\n\n\ub9cc\uc57d \uac01 \uc18d\uc131\uc744 \ud558\ub098\uc758 \ucc28\uc6d0\uc774\ub77c\uace0 \ubcf8\ub2e4\uba74 \ud558\ub098\uc758 \ub2e8\uc5b4\uc5d0 \uc544\ub798\uc640 \uac19\uc740 \ubca1\ud130\ub97c \ubc30\uc815\ud560 \uc218 \uc788\uc744\uac81\ub2c8\ub2e4.\n\n\\begin{align}q_\\text{\uc218\ud559\uc790} = \\left[ \\overbrace{2.3}^\\text{\ub6f8 \uc218 \uc788\uc74c},\n \\overbrace{9.4}^\\text{\ucee4\ud53c\ub97c \uc88b\uc544\ud568}, \\overbrace{-5.5}^\\text{\ubb3c\ub9ac \uc804\uacf5\uc784}, \\dots \\right]\\end{align}\n\n\\begin{align}q_\\text{\ubb3c\ub9ac\ud559\uc790} = \\left[ \\overbrace{2.5}^\\text{\ub6f8 \uc218 \uc788\uc74c},\n \\overbrace{9.1}^\\text{\ucee4\ud53c\ub97c \uc88b\uc544\ud568}, \\overbrace{6.4}^\\text{\ubb3c\ub9ac \uc804\uacf5\uc784}, \\dots \\right]\\end{align}\n\n\uadf8\ub7ec\uba74 \uc544\ub798\uc640 \uac19\uc774 \ub450 \ub2e8\uc5b4 \uc0ac\uc774\uc758 \uc720\uc0ac\ub3c4\ub97c \uad6c\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. ('\uc720\uc0ac\ub3c4'\ub77c\ub294 \ud568\uc218\ub97c \uc815\uc758\ud558\ub294 \uac81\ub2c8\ub2e4)\n\n\\begin{align}\\text{\uc720\uc0ac\ub3c4}(\\text{\ubb3c\ub9ac\ud559\uc790}, \\text{\uc218\ud559\uc790}) = q_\\text{\ubb3c\ub9ac\ud559\uc790} \\cdot q_\\text{\uc218\ud559\uc790}\\end{align}\n\n\ubb3c\ub860 \ubcf4\ud1b5\uc740 \uc774\ub807\uac8c \ubca1\ud130\uc758 \uae38\uc774\ub85c \ub098\ub220\uc8fc\uc9c0\ub9cc\uc694.\n\n\\begin{align}\\text{\uc720\uc0ac\ub3c4}(\\text{\ubb3c\ub9ac\ud559\uc790}, \\text{\uc218\ud559\uc790}) = \\frac{q_\\text{\ubb3c\ub9ac\ud559\uc790} \\cdot q_\\text{\uc218\ud559\uc790}}\n {\\| q_\\text{\ubb3c\ub9ac\ud559\uc790} \\| \\| q_\\text{\uc218\ud559\uc790} \\|} = \\cos (\\phi)\\end{align}\n\n$\\phi$ \ub294 \ub450 \ubca1\ud130 \uc0ac\uc774\uc758 \uac01\uc785\ub2c8\ub2e4. \uc774\ub7f0 \uc2dd\uc774\uba74 \uc815\ub9d0 \ube44\uc2b7\ud55c \ub2e8\uc5b4\ub294 \uc720\uc0ac\ub3c4 1\uc744 \uac16\uace0,\n\uc815\ub9d0 \ub2e4\ub978 \ub2e8\uc5b4\ub294 \uc720\uc0ac\ub3c4 -1\uc744 \uac16\uaca0\uc8e0. \ube44\uc2b7\ud55c \uc758\ubbf8\ub97c \uac00\uc9c8\uc218\ub85d \uac19\uc740 \ubc29\ud5a5\uc744 \uac00\ub9ac\ud0a4\uace0 \uc788\uc744 \ud14c\ub2c8\uae4c\uc694.\n\n\uc774 \uae00 \ucd08\ubc18\uc5d0 \ub098\uc628 \ud76c\ubc15\ud55c \uc6d0\ud56b \ubca1\ud130\uac00 \uc0ac\uc2e4\uc740 \uc6b0\ub9ac\uac00 \ubc29\uae08 \uc815\uc758\ud55c \uc758\ubbf8 \ubca1\ud130\uc758\n\ud2b9\uc774 \ucf00\uc774\uc2a4\ub77c\ub294 \uac83\uc744 \uae08\ubc29 \uc54c \uc218 \uc788\uc2b5\ub2c8\ub2e4. \ub2e8\uc5b4 \ubca1\ud130\uc758 \uac01 \uc6d0\uc18c\ub294 \uadf8 \ub2e8\uc5b4\uc758 \uc758\ubbf8\uc801 \uc18d\uc131\uc744\n\ud45c\ud604\ud558\uace0, \ubaa8\ub4e0 \ub2e8\uc5b4 \uc30d\uc758 \uc720\uc0ac\ub3c4\ub294 0\uc774\uae30 \ub54c\ubb38\uc774\uc8e0. \uc704\uc5d0\uc11c \uc815\uc758\ud55c \uc758\ubbf8 \ubca1\ud130\ub294 *\ubc00\uc9d1* \ub418\uc5b4 \uc788\uc2b5\ub2c8\ub2e4.\n\uc989, \uc6d0\ud56b \ubca1\ud130\uc5d0 \ube44\ud574 0 \uc6d0\uc18c\uc758 \uc218\uac00 \uc801\ub2e4\uace0 \ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\ud558\uc9c0\ub9cc \uc774 \ubca1\ud130\ub4e4\uc740 \uad6c\ud558\uae30\uac00 \uc9c4\uc9dc \uc5b4\ub835\uc2b5\ub2c8\ub2e4. \ub2e8\uc5b4\uac04\uc758 \uc720\uc0ac\ub3c4\ub97c \uacb0\uc815 \uc9c0\uc744 \ub9cc\ud55c\n\uc758\ubbf8\uc801 \uc18d\uc131\uc740 \uc5b4\ub5bb\uac8c \uacb0\uc815\ud560 \uac83\uc774\uba70, \uc18d\uc131\uc744 \uacb0\uc815\ud588\ub2e4\uace0 \ud558\ub354\ub77c\ub3c4 \uac01 \uc18d\uc131\uc5d0\n\ud574\ub2f9\ud558\ub294 \uac12\uc740 \ub3c4\ub300\uccb4 \uc5b4\ub5a0\ud55c \uae30\uc900\uc73c\ub85c \uc815\ud574\uc57c \ud560\uae4c\uc694? \uc18d\uc131\uacfc \uac12\uc744 \ub370\uc774\ud130\uc5d0 \uae30\ubc18\ud574\n\ub9cc\ub4e4\uace0 \uc790\ub3d9\uc73c\ub85c \ub2e8\uc5b4 \ubca1\ud130\ub97c \ub9cc\ub4e4 \uc218\ub294 \uc5c6\uc744\uae4c\uc694? \uc788\uc2b5\ub2c8\ub2e4. \ub525\ub7ec\ub2dd\uc744 \uc0ac\uc6a9\ud558\uba74 \ub9d0\uc774\uc8e0.\n\ub525\ub7ec\ub2dd\uc740 \uc778\uacf5\uc2e0\uacbd\ub9dd\uc744 \uc774\uc6a9\ud558\uc5ec \uc0ac\ub78c\uc758 \uac1c\uc785 \uc5c6\uc774 \uc18d\uc131\uc758 \ud45c\ud604 \ubc29\ubc95\uc744 \uc790\ub3d9\uc73c\ub85c \ud559\uc2b5\ud569\ub2c8\ub2e4.\n\uc774\ub97c \uc774\uc6a9\ud574 \ub2e8\uc5b4 \ubca1\ud130\ub97c \ubaa8\ub378 \ubaa8\uc218\ub85c \uc124\uc815\ud558\uace0 \ubaa8\ub378 \ud559\uc2b5\uc2dc\uc5d0 \ub2e8\uc5b4 \ubca1\ud130\ub3c4 \ud568\uaed8 \uc5c5\ub370\uc774\ud2b8 \ud558\uba74\n\ub420 \uac83\uc785\ub2c8\ub2e4. \uc774\ub807\uac8c \uc6b0\ub9ac \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc740 \uc801\uc5b4\ub3c4 \uc774\ub860\uc0c1\uc73c\ub85c\ub294 \ucda9\ubd84\ud788 \ud559\uc2b5\ud560 \uc218 \uc788\ub294\n*\uc7a0\uc7ac \uc758\ubbf8 \uc18d\uc131* \uc744 \ucc3e\uc744 \uac83\uc785\ub2c8\ub2e4. \uc5ec\uae30\uc11c \ub9d0\ud558\ub294 \uc7a0\uc7ac \uc758\ubbf8 \uc18d\uc131\uc73c\ub85c \uc774\ub8e8\uc5b4\uc9c4 \ubca1\ud130\ub294 \uc0ac\ub78c\uc774\n\ud574\uc11d\ud558\uae30 \uc0c1\ub2f9\ud788 \uc5b4\ub835\ub2e4\ub294 \uc810\uc744 \uae30\uc5b5\ud574 \ub450\uc138\uc694. \uc704\uc5d0\uc11c \uc218\ud559\uc790\uc640 \ubb3c\ub9ac\ud559\uc790\uc5d0\uac8c \ucee4\ud53c\ub97c \uc88b\uc544\ud55c\ub2e4\ub294\n\ub4f1 \uc0ac\ub78c\uc774 \uc784\uc758\uc801\uc73c\ub85c \ub2e8\uc5b4\uc5d0 \ubd80\uc5ec\ud55c \uc18d\uc131\uacfc\ub294 \ub2ec\ub9ac, \uc778\uacf5\uc2e0\uacbd\ub9dd\uc774 \uc790\ub3d9\uc73c\ub85c \ub2e8\uc5b4\uc758 \uc18d\uc131\uc744 \ucc3e\ub294\ub2e4\uba74\n\uadf8 \uc18d\uc131\uacfc \uac12\uc774 \uc758\ubbf8\ud558\ub294 \ubc14\ub97c \uc54c\uae30\uac00 \uc5b4\ub824\uc6b8 \uac83\uc785\ub2c8\ub2e4. \uc608\ub97c \ub4e4\uc5b4\uc11c \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc774 \ucc3e\uc740 '\uc218\ud559\uc790'\uc640\n'\ubb3c\ub9ac\ud559\uc790'\uc758 \ud45c\ud604 \ubca1\ud130 \ub458 \ub2e4 \ub450\ubc88\uc9f8 \uc6d0\uc18c\uac00 \ud06c\ub2e4\uace0 \uac00\uc815\ud574 \ubd05\uc2dc\ub2e4. \ub458\uc774 \ube44\uc2b7\ud558\ub2e4\ub294 \uac74 \uc54c\uaca0\uc9c0\ub9cc,\n\ub3c4\ub300\uccb4 \ub450\ubc88\uc9f8 \uc6d0\uc18c\uac00 \ubb34\uc5c7\uc744 \uc758\ubbf8\ud558\ub294\uc9c0\ub294 \uc54c\uae30\uac00 \ub9e4\uc6b0 \ud798\ub4e0 \uac83\uc785\ub2c8\ub2e4. \ud45c\ud604 \ubca1\ud130 \uacf5\uac04\uc0c1\uc5d0\uc11c\n\ube44\uc2b7\ud558\ub2e4\ub294 \uc815\ubcf4 \uc678\uc5d0\ub294 \uc544\ub9c8 \ub9ce\uc740 \uc815\ubcf4\ub97c \uc8fc\uae34 \uc5b4\ub824\uc6b8 \uac83\uc785\ub2c8\ub2e4.\n\n\uc694\uc57d\ud558\uc790\uba74, **\ub2e8\uc5b4 \uc784\ubca0\ub529\uc740 \ub2e8\uc5b4\uc758 *\uc758\ubbf8* \ub97c \ud45c\ud604\ud558\ub294 \ubc29\ubc95\uc774\uba70, \ucc28\ud6c4\uc5d0 \uc784\ubca0\ub529\uc744 \uc0ac\uc6a9\ud574\uc11c\n\ud480\uace0\uc790 \ud558\ub294 \ubb38\uc81c\uc5d0 \uc720\uc6a9\ud560 \uc758\ubbf8 \uc815\ubcf4\ub97c \ud6a8\uc728\uc801\uc73c\ub85c \uc778\ucf54\ub529\ud55c \uac83\uc785\ub2c8\ub2e4.** \ud488\uc0ac \ud0dc\uadf8, \ud30c\uc2a4 \ud2b8\ub9ac(parse tree) \ub4f1\n\ub2e8\uc5b4\uc758 \uc758\ubbf8 \uc678\uc5d0 \ub2e4\ub978 \uac83\ub3c4 \uc778\ucf54\ub529 \ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4! \ud53c\ucc98 \uc784\ubca0\ub529\uc758 \uac1c\ub150\uc744 \uc7a1\ub294 \uac83\uc774 \uc911\uc694\ud569\ub2c8\ub2e4.\n\n\n\ud30c\uc774\ud1a0\uce58\uc5d0\uc11c \ub2e8\uc5b4 \uc784\ubca0\ub529 \ud558\uae30\n~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n\uc2e4\uc81c\ub85c \ucf54\ub4dc\uc640 \uc608\uc2dc\ub97c \ubcf4\uae30 \uc804\uc5d0, \ud30c\uc774\ud1a0\uce58\ub97c \ube44\ub86f\ud574 \ub525\ub7ec\ub2dd \uad00\ub828 \ud504\ub85c\uadf8\ub798\ubc0d\uc744 \ud560 \ub54c\n\ub2e8\uc5b4 \uc784\ubca0\ub529\uc744 \uc5b4\ub5bb\uac8c \uc0ac\uc6a9\ud558\ub294\uc9c0\uc5d0 \ub300\ud574 \uc870\uae08 \uc54c\uc544\ubd05\uc2dc\ub2e4. \ub9e8 \uc704\uc5d0\uc11c \uc6d0\ud56b \ubca1\ud130\ub97c\n\uc815\uc758\ud588\ub358 \uac83 \ucc98\ub7fc, \ub2e8\uc5b4 \uc784\ubca0\ub529\uc744 \uc0ac\uc6a9\ud560 \ub54c\uc5d0\ub3c4 \uac01 \ub2e8\uc5b4\uc5d0 \uc778\ub371\uc2a4\ub97c \ubd80\uc5ec\ud574\uc57c \ud569\ub2c8\ub2e4.\n\uc774 \uc778\ub371\uc2a4\ub97c \ucc38\uc870 \ud14c\uc774\ube14(look-up table)\uc5d0\uc11c \uc0ac\uc6a9\ud560 \uac83\uc785\ub2c8\ub2e4. \uc989, $|V| \\times D$ \ud06c\uae30\uc758 \ud589\ub82c\uc5d0\n\ub2e8\uc5b4 \uc784\ubca0\ub529\uc744 \uc800\uc7a5\ud558\ub294\ub370, $D$ \ucc28\uc6d0\uc758 \uc784\ubca0\ub529 \ubca1\ud130\uac00 \ud589\ub82c\uc758 $i$ \ubc88\uc9f8 \ud589\uc5d0\n\uc800\uc7a5\ub418\uc5b4\uc788\uc5b4 $i$ \ub97c \uc778\ub371\uc2a4\ub85c \ud65c\uc6a9\ud574 \uc784\ubca0\ub529 \ubca1\ud130\ub97c \ucc38\uc870\ud558\ub294 \uac83\uc785\ub2c8\ub2e4.\n\uc544\ub798\uc758 \ubaa8\ub4e0 \ucf54\ub4dc\uc5d0\uc11c\ub294 \ub2e8\uc5b4\uc640 \uc778\ub371\uc2a4\ub97c \ub9e4\ud551\ud574\uc8fc\ub294 \ub515\uc154\ub108\ub9ac\ub97c word\\_to\\_ix\ub77c \uce6d\ud569\ub2c8\ub2e4.\n\n\ud30c\uc774\ud1a0\uce58\ub294 \uc784\ubca0\ub529\uc744 \uc190\uc27d\uac8c \uc0ac\uc6a9\ud560 \uc218 \uc788\uac8c torch.nn.Embedding\uc5d0 \uc704\uc5d0\uc11c \uc124\uba85\ud55c \ucc38\uc870 \ud14c\uc774\ube14\n\uae30\ub2a5\uc744 \uc9c0\uc6d0\ud569\ub2c8\ub2e4. \uc774 \ubaa8\ub4c8\uc740 \ub2e8\uc5b4\uc758 \uac1c\uc218\uc640 \uc784\ubca0\ub529\uc758 \ucc28\uc6d0, \ucd1d 2\uac1c\uc758 \ubcc0\uc218\ub97c \uc785\ub825 \ubcc0\uc218\ub85c \ubc1b\uc2b5\ub2c8\ub2e4.\n\ntorch.nn.Embedding \ud14c\uc774\ube14\uc758 \uc784\ubca0\ub529\uc744 \ucc38\uc870\ud558\uae30 \uc704\ud574\uc120 torch.LongTensor \ud0c0\uc785\uc758 \uc778\ub371\uc2a4 \ubcc0\uc218\ub97c\n\uaf2d \uc0ac\uc6a9\ud574\uc57c \ud569\ub2c8\ub2e4. (\uc778\ub371\uc2a4\ub294 \uc2e4\uc218\uac00 \uc544\ub2cc \uc815\uc218\uc774\uae30 \ub54c\ubb38\uc785\ub2c8\ub2e4.)\n\n\n" + "\n\ub2e8\uc5b4 \uc784\ubca0\ub529: \uc5b4\ud718\uc758 \uc758\ubbf8\ub97c \uc778\ucf54\ub529\ud558\uae30\n===========================================\n**\ubc88\uc5ed**: `\uc784\uc131\uc5f0 `_\n\n\ub2e8\uc5b4 \uc784\ubca0\ub529(word embedding)\uc774\ub780 \ub9d0\ubb49\uce58(\ud639\uc740 \ucf54\ud37c\uc2a4, corpus) \ub0b4 \uac01 \ub2e8\uc5b4\uc5d0 \uc77c\ub300\uc77c\ub85c \ub300\uc751\ud558\ub294 \ubc00\uc9d1\ub41c \uc2e4\uc218 \ubca1\ud130(dense vector)\uc758 \uc9d1\ud569, \ud639\uc740 \uc774 \ubca1\ud130\ub97c\n\uad6c\ud558\ub294 \ud589\uc704\ub97c \uac00\ub9ac\ud0b5\ub2c8\ub2e4. \uc8fc\ub85c \ub2e8\uc5b4\ub97c \ud53c\ucc98(feature)\ub85c \uc0ac\uc6a9\ud558\ub294 \uc790\uc5f0\uc5b4 \ucc98\ub9ac \ubd84\uc57c\uc5d0\uc11c\ub294 \ub2e8\uc5b4\ub97c \ucef4\ud4e8\ud130 \uce5c\ud654\uc801\uc778\n\ud615\ud0dc\ub85c \ubc14\uafb8\uc5b4 \uc8fc\ub294 \uc791\uc5c5\uc774 \ud544\uc218\uc801\uc785\ub2c8\ub2e4. \ucef4\ud4e8\ud130\uac00 \ub2e8\uc5b4\ub97c \ubc14\ub85c \uc774\ud574\ud558\uae30\ub294 \uc0c1\ub2f9\ud788 \uc5b4\ub835\uae30 \ub54c\ubb38\uc774\uc8e0.\n\uadf8\ub807\ub2e4\uba74, \ub2e8\uc5b4\ub97c \uc5b4\ub5bb\uac8c \ud45c\ud604\ud558\ub294 \uac83\uc774 \uc88b\uc744\uae4c\uc694? \ubb3c\ub860 \uac01 \ubb38\uc790\uc5d0 \ud574\ub2f9\ud558\ub294 ASCII\ucf54\ub4dc\ub97c \uc0ac\uc6a9\ud560 \uc218 \uc788\uaca0\uc9c0\ub9cc,\nASCII\ucf54\ub4dc\ub294 \uc774 \ub2e8\uc5b4\uac00 *\ubb34\uc5c7* \uc778\uc9c0\ub97c \uc54c\ub824\uc904 \ubfd0, \ub2e8\uc5b4\uac00 \uc5b4\ub5a4 *\uc758\ubbf8* \ub97c \uac00\uc9c0\ub294\uc9c0\ub294 \uc54c\ub824\uc8fc\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4.\n(\ub8f0\ubca0\uc774\uc2a4\ub85c \uc5b4\ubbf8 \ub4f1 \ubb38\ubc95\uc801 \ud2b9\uc9d5\uc744 \ud65c\uc6a9\ud558\uac70\ub098 \uc601\uc5b4\uc758 \uacbd\uc6b0 \ub300\ubb38\uc790\ub97c \uc0ac\uc6a9\ud560 \uc218 \uc788\uaca0\uc9c0\ub9cc \ucda9\ubd84\ud558\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4.)\n\ub2e8\uc5b4\ub97c \uc5b4\ub5bb\uac8c \ud45c\ud604\ud560\uc9c0 \ubfd0 \uc544\ub2c8\ub77c, \uc774 \ud45c\ud604\ubc95\uc744 \uc5b4\ub5a0\ud55c \ubc29\uc2dd\uc73c\ub85c \uc5f0\uc0b0\ud574\uc57c \ud560 \uc9c0 \ub610\ud55c \ud070 \ubb38\uc81c\uc785\ub2c8\ub2e4.\n\ubcf4\ud1b5 \uc774\ub7ec\ud55c \ubc00\ub3c4 \ub192\uc740 \ubca1\ud130\ub97c \uc5bb\uae30 \uc704\ud574 \uc0ac\uc6a9\ud558\ub294 \ub274\ub7f4\ub137 \ubaa8\ub378\uc740 $|V|$ (\ub9d0\ubb49\uce58\uc758 \ub2e8\uc5b4 \uac1c\uc218)\uc758\n\ud070 \uc785\ub825 \ucc28\uc6d0\uacfc \uba87 \uc548\ub418\ub294 (\ud14d\uc2a4\ub97c \ubd84\ub958\ud558\ub294 \ubb38\uc81c\ub77c\uace0 \ud560 \uacbd\uc6b0) \uc791\uc740 \ucd9c\ub825 \ucc28\uc6d0\uc744 \uac00\uc9d1\ub2c8\ub2e4.\n\uc989, \ub2e8\uc5b4\ub4e4 \uac04\uc758 \uc5f0\uc0b0\uc774 \ud544\uc218\uc785\ub2c8\ub2e4. \uc5b4\ub5bb\uac8c \uc774 \ud070 \ucc28\uc6d0\uc758 \uacf5\uac04\uc744 \uc791\uc740 \uacf5\uac04\uc73c\ub85c \ubcc0\ud615\uc2dc\ud0ac \uc218 \uc788\uc744\uae4c\uc694?\n\n\uba3c\uc800, \uc0c1\uae30\ud55c ASCII\ucf54\ub4dc \ub300\uc2e0 \uc6d0\ud56b \uc778\ucf54\ub529(one-hot encoding)\uc744 \uc0ac\uc6a9\ud574\ubcf4\ub294 \uac83\uc740 \uc5b4\ub5a8\uae4c\uc694? \uc6d0\ud56b \uc778\ucf54\ub529\uc774\ub780\n\ud558\ub098\uc758 \ub2e8\uc5b4 $w$ \ub97c \uc544\ub798\uc758 \ubca1\ud130\ub85c \ud45c\ud604\ud558\ub294 \uac83\uc744 \ub9d0\ud569\ub2c8\ub2e4.\n\n\\begin{align}\\overbrace{\\left[ 0, 0, \\dots, 1, \\dots, 0, 0 \\right]}^\\text{|V| elements}\\end{align}\n\n\uc5ec\uae30\uc11c 1\uc740 \ud574\ub2f9 \ubca1\ud130\uac00 \ud45c\ud604\ud558\uace0\uc790 \ud558\ub294 \ub2e8\uc5b4\uc5d0 \ud574\ub2f9\ud558\ub294 \uc704\uce58 1\uacf3\uc5d0 \uc790\ub9ac\ud569\ub2c8\ub2e4. (\ub098\uba38\uc9c0\ub294 \uc804\ubd80\n0\uc785\ub2c8\ub2e4.) \ub2e4\ub978 \ub2e8\uc5b4\ub97c \ub098\ud0c0\ub0b4\ub294 \ubca1\ud130\uc5d0\uc120 1\uc774 \ub2e4\ub978 \uacf3\uc5d0 \uc704\uce58\ud574 \uc788\uaca0\uc8e0.\n\n\uc6d0\ud56b \uc778\ucf54\ub529\uc740 \ub9cc\ub4e4\uae30\uac00 \uc27d\ub2e4\ub294 \uc7a5\uc810\uc774 \uc788\uc9c0\ub9cc, \ub2e8\uc21c\ud55c \ub9cc\ud07c \ub2e8\uc810\ub3c4 \uc788\uc2b5\ub2c8\ub2e4. \uc77c\ub2e8 \ub2e8\uc5b4 \ubca1\ud130 \ud55c \uac1c\ub294\n\ubaa8\ub4e0 \ub2e8\uc5b4\ub97c \ud45c\ud604\ud560 \uc218 \uc788\uc744 \ub9cc\ud55c \ud06c\uae30\uac00 \ub418\uc5b4\uc57c \ud569\ub2c8\ub2e4. \uc6b0\ub9ac\uac00 \uc5bc\ub9c8\ub098 \ub9ce\uc740 \uc885\ub958\uc758 \ub2e8\uc5b4\ub97c\n\uc0ac\uc6a9\ud558\ub294\uc9c0\ub97c \uc0dd\uac01 \ud55c\ub2e4\uba74 \uc5b4\ub9c8\uc5b4\ub9c8\ud558\uac8c \ud070 \ubca1\ud130\ub77c\ub294 \uac83\uc744 \uc54c \uc218 \uc788\uc8e0. \uc774 \ubfd0\ub9cc\uc774 \uc544\ub2d9\ub2c8\ub2e4.\n\uc6d0\ud56b \ubca1\ud130\ub294 \ubaa8\ub4e0 \ub2e8\uc5b4\ub97c \ub3c5\ub9bd\uc801\uc778 \uac1c\uccb4\ub85c \uac00\uc815\ud558\ub294 \uac83\uc744 \ubcfc \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc989, \uacf5\uac04\uc0c1\uc5d0\uc11c\n\uc644\uc804\ud788 \ub2e4\ub978 \ucd95\uc5d0 \uc704\uce58\ud574 \uc788\uc5b4\uc11c \ub2e8\uc5b4\uac04\uc758 \uad00\uacc4\ub97c \ub098\ud0c0\ub0bc \uc218\uac00 \uc5c6\uc2b5\ub2c8\ub2e4. \ud558\uc9c0\ub9cc \uc6b0\ub9ac\ub294 \ub2e8\uc5b4\n\uc0ac\uc774\uc758 *\uc720\uc0ac\ub3c4* \ub97c \uc5b4\ub5bb\uac8c\ub4e0 \uacc4\uc0b0\ud558\uace0 \uc2f6\uc740\uac70\uc8e0. \uc65c \uc720\uc0ac\ub3c4\uac00 \uc911\uc694\ud558\ub0d0\uad6c\uc694? \ub2e4\uc74c \uc608\uc81c\ub97c \ubd05\uc2dc\ub2e4.\n\n\uc6b0\ub9ac\uc758 \ubaa9\ud45c\uac00 \uc5b8\uc5b4 \ubaa8\ub378\uc744 \ub9cc\ub4dc\ub294 \uac83\uc774\ub77c\uace0 \uac00\uc815\ud558\uace0 \ub2e4\uc74c\uc758 \ubb38\uc7a5\uc774 \ud559\uc2b5 \ub370\uc774\ud130\ub85c\uc368 \uc8fc\uc5b4\uc84c\ub2e4\uace0 \ud574\ubd05\uc2dc\ub2e4.\n\n* \uc218\ud559\uc790\uac00 \uac00\uac8c\ub85c \ub6f0\uc5b4\uac14\ub2e4.\n* \ubb3c\ub9ac\ud559\uc790\uac00 \uac00\uac8c\ub85c \ub6f0\uc5b4\uac14\ub2e4.\n* \uc218\ud559\uc790\uac00 \ub9ac\ub9cc \uac00\uc124\uc744 \uc99d\uba85\ud588\ub2e4.\n\n\ub610\ud55c \ud559\uc2b5 \ub370\uc774\ud130\uc5d0\ub294 \uc5c6\ub294 \uc544\ub798 \ubb38\uc7a5\uc774 \uc788\ub2e4\uace0 \uc0dd\uac01\ud574\ubd05\uc2dc\ub2e4.\n\n* \ubb3c\ub9ac\ud559\uc790\uac00 \ub9ac\ub9cc \uac00\uc124\uc744 \uc99d\uba85\ud588\ub2e4.\n\nASCII \ucf54\ub4dc\ub098 \uc6d0\ud56b \uc778\ucf54\ub529 \uae30\ubc18 \uc5b8\uc5b4 \ubaa8\ub378\uc740 \uc704 \ubb38\uc7a5\uc744 \uc5b4\ub290\uc815\ub3c4 \ub2e4\ub8f0 \uc218 \uc788\uaca0\uc9c0\ub9cc, \uac1c\uc120\uc758 \uc5ec\uc9c0\uac00 \uc788\uc9c0 \uc54a\uc744\uae4c\uc694?\n\uba3c\uc800 \uc544\ub798\uc758 \ub450 \uc0ac\uc2e4\uc744 \uc0dd\uac01\ud574\ubd05\uc2dc\ub2e4.\n\n* '\uc218\ud559\uc790'\uc640 '\ubb3c\ub9ac\ud559\uc790'\uac00 \ubb38\uc7a5 \ub0b4\uc5d0\uc11c \uac19\uc740 \uc5ed\ud560\uc744 \ub9e1\uace0 \uc788\uc2b5\ub2c8\ub2e4. \uc774 \ub450 \ub2e8\uc5b4\ub294 \uc5b4\ub5bb\uac8c\ub4e0 \uc758\ubbf8\uc801\uc778 \uc5f0\uad00\uc131\uc774 \uc788\uc744 \uac81\ub2c8\ub2e4.\n* \uc0c8\ub85c\uc6b4 \ubb38\uc7a5\uc5d0\uc11c '\ubb3c\ub9ac\ud559\uc790'\uac00 \ub9e1\uc740 \uc5ed\ud560\uc744 '\uc218\ud559\uc790'\uac00 \ub9e1\ub294 \uac83\uc744 \ud559\uc2b5 \ub370\uc774\ud130\uc5d0\uc11c \ubcf8 \uc801\uc774 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc6b0\ub9ac \ubaa8\ub378\uc774 \uc704\uc758 \uc0ac\uc2e4\uc744 \ud1b5\ud574 '\ubb3c\ub9ac\ud559\uc790'\uac00 \uc0c8 \ubb38\uc7a5\uc5d0 \uc798 \ub4e4\uc5b4 \ub9de\ub294\ub2e4\ub294 \uac83\uc744 \ucd94\ub860\ud560 \uc218\n\uc788\ub2e4\uba74 \ucc38 \uc88b\uc744 \uac83\uc785\ub2c8\ub2e4. \uc774\uac83\uc774 \uc704\uc5d0\uc11c \uc5b8\uae09\ud55c \uc720\uc0ac\ub3c4\uc758 \uc758\ubbf8\uc785\ub2c8\ub2e4. \ucca0\uc790\uc801 \uc720\uc0ac\ub3c4 \ubfd0\n\uc544\ub2c8\ub77c *\uc758\ubbf8\uc801 \uc720\uc0ac\ub3c4* \uc778 \uac83\uc785\ub2c8\ub2e4. \uc774\uac83\uc774\uc57c\ub9d0\ub85c \uc5b8\uc5b4 \ub370\uc774\ud130\uc5d0 \ub0b4\uc7ac\ud558\ub294 \ud76c\ubc15\uc131(sparsity)\uc5d0\n\ub300\ud55c \ucc98\ubc29\uc774 \ub420 \uac83\uc785\ub2c8\ub2e4. \uc6b0\ub9ac\uac00 \ubcf8 \uac83\uacfc \uc544\uc9c1 \ubcf4\uc9c0 \uc54a\uc740 \uac83 \uc0ac\uc774\ub97c \uc774\uc5b4\uc8fc\ub294 \uac83\uc774\uc8e0.\n\uc55e\uc73c\ub85c\ub294 \ub2e4\uc74c\uc758 \uc5b8\uc5b4\ud559\uc801 \uae30\ubcf8 \uba85\uc81c\ub97c \uac00\uc815\ud558\ub3c4\ub85d \ud569\uc2dc\ub2e4. \ubc14\ub85c \ube44\uc2b7\ud55c \ub9e5\ub77d\uc5d0\uc11c \ub4f1\uc7a5\ud558\ub294\n\ub2e8\uc5b4\ub4e4\uc740 \uc11c\ub85c \uc758\ubbf8\uc801 \uc5f0\uad00\uc131\uc744 \uac00\uc9c4\ub2e4\ub294 \uac83\uc785\ub2c8\ub2e4. \uc5b8\uc5b4\ud559\uc801\uc73c\ub85c\ub294 `\ubd84\uc0b0 \uc758\ubbf8 \uac00\uc124(distributional\nhypothesis) `__ \uc774\ub77c\uace0\ub3c4 \ud569\ub2c8\ub2e4.\n\n\n\ubc00\uc9d1\ub41c \ub2e8\uc5b4 \uc784\ubca0\ub529 \uad6c\ud558\uae30\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n\uc5b4\ub5bb\uac8c \ub2e8\uc5b4\uc758 \uc758\ubbf8\uc801 \uc720\uc0ac\ub3c4\ub97c \uc778\ucf54\ub529 \ud560 \uc218 \uc788\uc744\uae4c\uc694? \ub2e4\uc2dc \ub9d0\ud574, \uc5b4\ub5bb\uac8c \ud574\uc57c \ub2e8\uc5b4\uc758 \uc720\uc0ac\ub3c4\ub97c\n\ub2e8\uc5b4 \ubca1\ud130\uc5d0 \ubc18\uc601\ud560 \uc218 \uc788\uc744\uae4c\uc694? \ub2e8\uc5b4 \ub370\uc774\ud130\uc5d0 \uc758\ubbf8\uc801 \uc18d\uc131(attribute)\uc744 \ubd80\uc5ec\ud558\ub294 \uac74 \uc5b4\ub5a4\uac00\uc694?\n\uc608\ub97c \ub4e4\uc5b4 '\uc218\ud559\uc790'\uc640 '\ubb3c\ub9ac\ud559\uc790'\uac00 \ubaa8\ub450 \ub6f8 \uc218 \uc788\ub2e4\uba74, \ud574\ub2f9 \ub2e8\uc5b4\uc758 '\ub6f8 \uc218 \uc788\uc74c' \uc18d\uc131\uc5d0 \ub192\uc740 \uc810\uc218\ub97c \uc8fc\ub294 \uac81\ub2c8\ub2e4.\n\uacc4\uc18d \ud574\ubd05\uc2dc\ub2e4. \ub2e4\ub978 \ub2e8\uc5b4\ub4e4\uc5d0 \ub300\ud574\uc11c\ub294 \uc5b4\ub5a0\ud55c \uc18d\uc131\uc744 \ub9cc\ub4e4 \uc218 \uc788\uc744\uc9c0 \uc0dd\uac01\ud574\ubd05\uc2dc\ub2e4.\n\n\ub9cc\uc57d \uac01 \uc18d\uc131\uc744 \ud558\ub098\uc758 \ucc28\uc6d0\uc774\ub77c\uace0 \ubcf8\ub2e4\uba74 \ud558\ub098\uc758 \ub2e8\uc5b4\uc5d0 \uc544\ub798\uc640 \uac19\uc740 \ubca1\ud130\ub97c \ubc30\uc815\ud560 \uc218 \uc788\uc744\uac81\ub2c8\ub2e4.\n\n\\begin{align}q_\\text{\uc218\ud559\uc790} = \\left[ \\overbrace{2.3}^\\text{\ub6f8 \uc218 \uc788\uc74c},\n \\overbrace{9.4}^\\text{\ucee4\ud53c\ub97c \uc88b\uc544\ud568}, \\overbrace{-5.5}^\\text{\ubb3c\ub9ac \uc804\uacf5\uc784}, \\dots \\right]\\end{align}\n\n\\begin{align}q_\\text{\ubb3c\ub9ac\ud559\uc790} = \\left[ \\overbrace{2.5}^\\text{\ub6f8 \uc218 \uc788\uc74c},\n \\overbrace{9.1}^\\text{\ucee4\ud53c\ub97c \uc88b\uc544\ud568}, \\overbrace{6.4}^\\text{\ubb3c\ub9ac \uc804\uacf5\uc784}, \\dots \\right]\\end{align}\n\n\uadf8\ub7ec\uba74 \uc544\ub798\uc640 \uac19\uc774 \ub450 \ub2e8\uc5b4 \uc0ac\uc774\uc758 \uc720\uc0ac\ub3c4\ub97c \uad6c\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. ('\uc720\uc0ac\ub3c4'\ub77c\ub294 \ud568\uc218\ub97c \uc815\uc758\ud558\ub294 \uac81\ub2c8\ub2e4)\n\n\\begin{align}\\text{\uc720\uc0ac\ub3c4}(\\text{\ubb3c\ub9ac\ud559\uc790}, \\text{\uc218\ud559\uc790}) = q_\\text{\ubb3c\ub9ac\ud559\uc790} \\cdot q_\\text{\uc218\ud559\uc790}\\end{align}\n\n\ubb3c\ub860 \ubcf4\ud1b5\uc740 \uc774\ub807\uac8c \ubca1\ud130\uc758 \uae38\uc774\ub85c \ub098\ub220\uc8fc\uc9c0\ub9cc\uc694.\n\n\\begin{align}\\text{\uc720\uc0ac\ub3c4}(\\text{\ubb3c\ub9ac\ud559\uc790}, \\text{\uc218\ud559\uc790}) = \\frac{q_\\text{\ubb3c\ub9ac\ud559\uc790} \\cdot q_\\text{\uc218\ud559\uc790}}\n {\\| q_\\text{\ubb3c\ub9ac\ud559\uc790} \\| \\| q_\\text{\uc218\ud559\uc790} \\|} = \\cos (\\phi)\\end{align}\n\n$\\phi$ \ub294 \ub450 \ubca1\ud130 \uc0ac\uc774\uc758 \uac01\uc785\ub2c8\ub2e4. \uc774\ub7f0 \uc2dd\uc774\uba74 \uc815\ub9d0 \ube44\uc2b7\ud55c \ub2e8\uc5b4\ub294 \uc720\uc0ac\ub3c4 1\uc744 \uac16\uace0,\n\uc815\ub9d0 \ub2e4\ub978 \ub2e8\uc5b4\ub294 \uc720\uc0ac\ub3c4 -1\uc744 \uac16\uaca0\uc8e0. \ube44\uc2b7\ud55c \uc758\ubbf8\ub97c \uac00\uc9c8\uc218\ub85d \uac19\uc740 \ubc29\ud5a5\uc744 \uac00\ub9ac\ud0a4\uace0 \uc788\uc744 \ud14c\ub2c8\uae4c\uc694.\n\n\uc774 \uae00 \ucd08\ubc18\uc5d0 \ub098\uc628 \ud76c\ubc15\ud55c \uc6d0\ud56b \ubca1\ud130\uac00 \uc0ac\uc2e4\uc740 \uc6b0\ub9ac\uac00 \ubc29\uae08 \uc815\uc758\ud55c \uc758\ubbf8 \ubca1\ud130\uc758\n\ud2b9\uc774 \ucf00\uc774\uc2a4\ub77c\ub294 \uac83\uc744 \uae08\ubc29 \uc54c \uc218 \uc788\uc2b5\ub2c8\ub2e4. \ub2e8\uc5b4 \ubca1\ud130\uc758 \uac01 \uc6d0\uc18c\ub294 \uadf8 \ub2e8\uc5b4\uc758 \uc758\ubbf8\uc801 \uc18d\uc131\uc744\n\ud45c\ud604\ud558\uace0, \ubaa8\ub4e0 \ub2e8\uc5b4 \uc30d\uc758 \uc720\uc0ac\ub3c4\ub294 0\uc774\uae30 \ub54c\ubb38\uc774\uc8e0. \uc704\uc5d0\uc11c \uc815\uc758\ud55c \uc758\ubbf8 \ubca1\ud130\ub294 *\ubc00\uc9d1* \ub418\uc5b4 \uc788\uc2b5\ub2c8\ub2e4.\n\uc989, \uc6d0\ud56b \ubca1\ud130\uc5d0 \ube44\ud574 0 \uc6d0\uc18c\uc758 \uc218\uac00 \uc801\ub2e4\uace0 \ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\ud558\uc9c0\ub9cc \uc774 \ubca1\ud130\ub4e4\uc740 \uad6c\ud558\uae30\uac00 \uc9c4\uc9dc \uc5b4\ub835\uc2b5\ub2c8\ub2e4. \ub2e8\uc5b4\uac04\uc758 \uc720\uc0ac\ub3c4\ub97c \uacb0\uc815 \uc9c0\uc744 \ub9cc\ud55c\n\uc758\ubbf8\uc801 \uc18d\uc131\uc740 \uc5b4\ub5bb\uac8c \uacb0\uc815\ud560 \uac83\uc774\uba70, \uc18d\uc131\uc744 \uacb0\uc815\ud588\ub2e4\uace0 \ud558\ub354\ub77c\ub3c4 \uac01 \uc18d\uc131\uc5d0\n\ud574\ub2f9\ud558\ub294 \uac12\uc740 \ub3c4\ub300\uccb4 \uc5b4\ub5a0\ud55c \uae30\uc900\uc73c\ub85c \uc815\ud574\uc57c \ud560\uae4c\uc694? \uc18d\uc131\uacfc \uac12\uc744 \ub370\uc774\ud130\uc5d0 \uae30\ubc18\ud574\n\ub9cc\ub4e4\uace0 \uc790\ub3d9\uc73c\ub85c \ub2e8\uc5b4 \ubca1\ud130\ub97c \ub9cc\ub4e4 \uc218\ub294 \uc5c6\uc744\uae4c\uc694? \uc788\uc2b5\ub2c8\ub2e4. \ub525\ub7ec\ub2dd\uc744 \uc0ac\uc6a9\ud558\uba74 \ub9d0\uc774\uc8e0.\n\ub525\ub7ec\ub2dd\uc740 \uc778\uacf5\uc2e0\uacbd\ub9dd\uc744 \uc774\uc6a9\ud558\uc5ec \uc0ac\ub78c\uc758 \uac1c\uc785 \uc5c6\uc774 \uc18d\uc131\uc758 \ud45c\ud604 \ubc29\ubc95\uc744 \uc790\ub3d9\uc73c\ub85c \ud559\uc2b5\ud569\ub2c8\ub2e4.\n\uc774\ub97c \uc774\uc6a9\ud574 \ub2e8\uc5b4 \ubca1\ud130\ub97c \ubaa8\ub378 \ubaa8\uc218\ub85c \uc124\uc815\ud558\uace0 \ubaa8\ub378 \ud559\uc2b5\uc2dc\uc5d0 \ub2e8\uc5b4 \ubca1\ud130\ub3c4 \ud568\uaed8 \uc5c5\ub370\uc774\ud2b8 \ud558\uba74\n\ub420 \uac83\uc785\ub2c8\ub2e4. \uc774\ub807\uac8c \uc6b0\ub9ac \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc740 \uc801\uc5b4\ub3c4 \uc774\ub860\uc0c1\uc73c\ub85c\ub294 \ucda9\ubd84\ud788 \ud559\uc2b5\ud560 \uc218 \uc788\ub294\n*\uc7a0\uc7ac \uc758\ubbf8 \uc18d\uc131* \uc744 \ucc3e\uc744 \uac83\uc785\ub2c8\ub2e4. \uc5ec\uae30\uc11c \ub9d0\ud558\ub294 \uc7a0\uc7ac \uc758\ubbf8 \uc18d\uc131\uc73c\ub85c \uc774\ub8e8\uc5b4\uc9c4 \ubca1\ud130\ub294 \uc0ac\ub78c\uc774\n\ud574\uc11d\ud558\uae30 \uc0c1\ub2f9\ud788 \uc5b4\ub835\ub2e4\ub294 \uc810\uc744 \uae30\uc5b5\ud574 \ub450\uc138\uc694. \uc704\uc5d0\uc11c \uc218\ud559\uc790\uc640 \ubb3c\ub9ac\ud559\uc790\uc5d0\uac8c \ucee4\ud53c\ub97c \uc88b\uc544\ud55c\ub2e4\ub294\n\ub4f1 \uc0ac\ub78c\uc774 \uc784\uc758\uc801\uc73c\ub85c \ub2e8\uc5b4\uc5d0 \ubd80\uc5ec\ud55c \uc18d\uc131\uacfc\ub294 \ub2ec\ub9ac, \uc778\uacf5\uc2e0\uacbd\ub9dd\uc774 \uc790\ub3d9\uc73c\ub85c \ub2e8\uc5b4\uc758 \uc18d\uc131\uc744 \ucc3e\ub294\ub2e4\uba74\n\uadf8 \uc18d\uc131\uacfc \uac12\uc774 \uc758\ubbf8\ud558\ub294 \ubc14\ub97c \uc54c\uae30\uac00 \uc5b4\ub824\uc6b8 \uac83\uc785\ub2c8\ub2e4. \uc608\ub97c \ub4e4\uc5b4\uc11c \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc774 \ucc3e\uc740 '\uc218\ud559\uc790'\uc640\n'\ubb3c\ub9ac\ud559\uc790'\uc758 \ud45c\ud604 \ubca1\ud130 \ub458 \ub2e4 \ub450\ubc88\uc9f8 \uc6d0\uc18c\uac00 \ud06c\ub2e4\uace0 \uac00\uc815\ud574 \ubd05\uc2dc\ub2e4. \ub458\uc774 \ube44\uc2b7\ud558\ub2e4\ub294 \uac74 \uc54c\uaca0\uc9c0\ub9cc,\n\ub3c4\ub300\uccb4 \ub450\ubc88\uc9f8 \uc6d0\uc18c\uac00 \ubb34\uc5c7\uc744 \uc758\ubbf8\ud558\ub294\uc9c0\ub294 \uc54c\uae30\uac00 \ub9e4\uc6b0 \ud798\ub4e0 \uac83\uc785\ub2c8\ub2e4. \ud45c\ud604 \ubca1\ud130 \uacf5\uac04\uc0c1\uc5d0\uc11c\n\ube44\uc2b7\ud558\ub2e4\ub294 \uc815\ubcf4 \uc678\uc5d0\ub294 \uc544\ub9c8 \ub9ce\uc740 \uc815\ubcf4\ub97c \uc8fc\uae34 \uc5b4\ub824\uc6b8 \uac83\uc785\ub2c8\ub2e4.\n\n\uc694\uc57d\ud558\uc790\uba74, **\ub2e8\uc5b4 \uc784\ubca0\ub529\uc740 \ub2e8\uc5b4\uc758 *\uc758\ubbf8* \ub97c \ud45c\ud604\ud558\ub294 \ubc29\ubc95\uc774\uba70, \ucc28\ud6c4\uc5d0 \uc784\ubca0\ub529\uc744 \uc0ac\uc6a9\ud574\uc11c\n\ud480\uace0\uc790 \ud558\ub294 \ubb38\uc81c\uc5d0 \uc720\uc6a9\ud560 \uc758\ubbf8 \uc815\ubcf4\ub97c \ud6a8\uc728\uc801\uc73c\ub85c \uc778\ucf54\ub529\ud55c \uac83\uc785\ub2c8\ub2e4.** \ud488\uc0ac \ud0dc\uadf8, \ud30c\uc2a4 \ud2b8\ub9ac(parse tree) \ub4f1\n\ub2e8\uc5b4\uc758 \uc758\ubbf8 \uc678\uc5d0 \ub2e4\ub978 \uac83\ub3c4 \uc778\ucf54\ub529 \ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4! \ud53c\ucc98 \uc784\ubca0\ub529\uc758 \uac1c\ub150\uc744 \uc7a1\ub294 \uac83\uc774 \uc911\uc694\ud569\ub2c8\ub2e4.\n\n\n\ud30c\uc774\ud1a0\uce58\uc5d0\uc11c \ub2e8\uc5b4 \uc784\ubca0\ub529 \ud558\uae30\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n\uc2e4\uc81c\ub85c \ucf54\ub4dc\uc640 \uc608\uc2dc\ub97c \ubcf4\uae30 \uc804\uc5d0, \ud30c\uc774\ud1a0\uce58\ub97c \ube44\ub86f\ud574 \ub525\ub7ec\ub2dd \uad00\ub828 \ud504\ub85c\uadf8\ub798\ubc0d\uc744 \ud560 \ub54c\n\ub2e8\uc5b4 \uc784\ubca0\ub529\uc744 \uc5b4\ub5bb\uac8c \uc0ac\uc6a9\ud558\ub294\uc9c0\uc5d0 \ub300\ud574 \uc870\uae08 \uc54c\uc544\ubd05\uc2dc\ub2e4. \ub9e8 \uc704\uc5d0\uc11c \uc6d0\ud56b \ubca1\ud130\ub97c\n\uc815\uc758\ud588\ub358 \uac83 \ucc98\ub7fc, \ub2e8\uc5b4 \uc784\ubca0\ub529\uc744 \uc0ac\uc6a9\ud560 \ub54c\uc5d0\ub3c4 \uac01 \ub2e8\uc5b4\uc5d0 \uc778\ub371\uc2a4\ub97c \ubd80\uc5ec\ud574\uc57c \ud569\ub2c8\ub2e4.\n\uc774 \uc778\ub371\uc2a4\ub97c \ucc38\uc870 \ud14c\uc774\ube14(look-up table)\uc5d0\uc11c \uc0ac\uc6a9\ud560 \uac83\uc785\ub2c8\ub2e4. \uc989, $|V| \\times D$ \ud06c\uae30\uc758 \ud589\ub82c\uc5d0\n\ub2e8\uc5b4 \uc784\ubca0\ub529\uc744 \uc800\uc7a5\ud558\ub294\ub370, $D$ \ucc28\uc6d0\uc758 \uc784\ubca0\ub529 \ubca1\ud130\uac00 \ud589\ub82c\uc758 $i$ \ubc88\uc9f8 \ud589\uc5d0\n\uc800\uc7a5\ub418\uc5b4\uc788\uc5b4 $i$ \ub97c \uc778\ub371\uc2a4\ub85c \ud65c\uc6a9\ud574 \uc784\ubca0\ub529 \ubca1\ud130\ub97c \ucc38\uc870\ud558\ub294 \uac83\uc785\ub2c8\ub2e4.\n\uc544\ub798\uc758 \ubaa8\ub4e0 \ucf54\ub4dc\uc5d0\uc11c\ub294 \ub2e8\uc5b4\uc640 \uc778\ub371\uc2a4\ub97c \ub9e4\ud551\ud574\uc8fc\ub294 \ub515\uc154\ub108\ub9ac\ub97c word\\_to\\_ix\ub77c \uce6d\ud569\ub2c8\ub2e4.\n\n\ud30c\uc774\ud1a0\uce58\ub294 \uc784\ubca0\ub529\uc744 \uc190\uc27d\uac8c \uc0ac\uc6a9\ud560 \uc218 \uc788\uac8c torch.nn.Embedding\uc5d0 \uc704\uc5d0\uc11c \uc124\uba85\ud55c \ucc38\uc870 \ud14c\uc774\ube14\n\uae30\ub2a5\uc744 \uc9c0\uc6d0\ud569\ub2c8\ub2e4. \uc774 \ubaa8\ub4c8\uc740 \ub2e8\uc5b4\uc758 \uac1c\uc218\uc640 \uc784\ubca0\ub529\uc758 \ucc28\uc6d0, \ucd1d 2\uac1c\uc758 \ubcc0\uc218\ub97c \uc785\ub825 \ubcc0\uc218\ub85c \ubc1b\uc2b5\ub2c8\ub2e4.\n\ntorch.nn.Embedding \ud14c\uc774\ube14\uc758 \uc784\ubca0\ub529\uc744 \ucc38\uc870\ud558\uae30 \uc704\ud574\uc120 torch.LongTensor \ud0c0\uc785\uc758 \uc778\ub371\uc2a4 \ubcc0\uc218\ub97c\n\uaf2d \uc0ac\uc6a9\ud574\uc57c \ud569\ub2c8\ub2e4. (\uc778\ub371\uc2a4\ub294 \uc2e4\uc218\uac00 \uc544\ub2cc \uc815\uc218\uc774\uae30 \ub54c\ubb38\uc785\ub2c8\ub2e4.)\n" ] }, { @@ -93,7 +93,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/377bf4a7b1761e5f081e057385870d8e/fgsm_tutorial.py b/docs/_downloads/377bf4a7b1761e5f081e057385870d8e/fgsm_tutorial.py index 374272351..ea0b4c651 100644 --- a/docs/_downloads/377bf4a7b1761e5f081e057385870d8e/fgsm_tutorial.py +++ b/docs/_downloads/377bf4a7b1761e5f081e057385870d8e/fgsm_tutorial.py @@ -20,37 +20,37 @@ ###################################################################### # 위협 모델 # ------------ -# +# # 상황에 따라 다양한 범주의 적대적 공격이 있는데 각각 목표가 다르고 공격자가 알고 있는 정보 -# 대한 가정도 다릅니다. 그러나 보통 가장 중요한 목표는 입력 데이터에 최소한의 작은 변화를 -# 추가하여 이것이 의도적으로 잘못 분류되게 하는 것입니다. 공격자가 가지고 있는 정보에 대한 -# 가정에는 여러 종류가 있는데, 보통 **화이트박스** 와 **블랙박스** 두 가지가 있습니다. +# 대한 가정도 다릅니다. 그러나 보통 가장 중요한 목표는 입력 데이터에 최소한의 작은 변화를 +# 추가하여 이것이 의도적으로 잘못 분류되게 하는 것입니다. 공격자가 가지고 있는 정보에 대한 +# 가정에는 여러 종류가 있는데, 보통 **화이트박스** 와 **블랙박스** 두 가지가 있습니다. # *화이트박스* 공격은 공격자가 모델에 대해 아키텍처, 입력, 출력, 가중치를 포함한 모든 것을 -# 알고 있고 접근할 수 있다고 가정합니다. *블랙박스* 공격은 공격자가 모델의 입력과 출력에 -# 대해서만 접근 가능하고 모델의 가중치와 아키텍처에 관한 내용은 모른다고 가정합니다. -# 공격자의 목표는 오분류 및 **소스/타겟 오분류** 를 포함하는 여러 유형이 있습니다. +# 알고 있고 접근할 수 있다고 가정합니다. *블랙박스* 공격은 공격자가 모델의 입력과 출력에 +# 대해서만 접근 가능하고 모델의 가중치와 아키텍처에 관한 내용은 모른다고 가정합니다. +# 공격자의 목표는 오분류 및 **소스/타겟 오분류** 를 포함하는 여러 유형이 있습니다. # *오분류* 의 목표는 공격자가 출력으로 나온 분류 결과가 잘못 되도록 하나 새로운 분류 결과가 -# 어떤 것이 나오는지 신경 쓰지 않는 것을 의미합니다. *소스/타겟 오분류* 는 공격자가 +# 어떤 것이 나오는지 신경 쓰지 않는 것을 의미합니다. *소스/타겟 오분류* 는 공격자가 # 원래 특정 소스 클래스의 이미지를 다른 특정 대상 클래스로 분류하도록 변경하려고 함을 의미합니다. # -# -# 이 경우 FGSM 공격은 *오분류* 를 목표로 하는 화이트 박스 공격입니다. +# +# 이 경우 FGSM 공격은 *오분류* 를 목표로 하는 화이트 박스 공격입니다. # 이런 배경 정보를 갖고 공격에 대해 자세히 알아 보겠습니다. -# +# # 빠른 변화도 부호 공격 # ------------------------- -# -# 공격 방법에 있어 초기 방식이면서 가장 유명한 방식은 *빠른 변화도 부호 공격 (FGSM)* 이라고 하며 -# `적대적 예제에 대한 설명과 활용 `__ 에서 +# +# 공격 방법에 있어 초기 방식이면서 가장 유명한 방식은 *빠른 변화도 부호 공격 (FGSM)* 이라고 하며 +# `적대적 예제에 대한 설명과 활용 `__ 에서 # 이안 갓펠로우가 기고하였습니다. -# 이 공격법은 놀랍도록 강력하지만 직관적입니다. 학습 방식, *변화도(gradients)* 를 활용하여 신경망을 공격하도록 +# 이 공격법은 놀랍도록 강력하지만 직관적입니다. 학습 방식, *변화도(gradients)* 를 활용하여 신경망을 공격하도록 # 설계 되었습니다. 아이디어는 간단합니다. 역전파 변화도를 기반으로 가중치를 조정하여 손실을 최소화하기보다는 -# 공격이 동일한 역전파 변화도를 기반으로 *손실을 최대화하하는 방향으로 입력 데이터를 조정* 합니다. +# 공격이 동일한 역전파 변화도를 기반으로 *손실을 최대화하하는 방향으로 입력 데이터를 조정* 합니다. # 다시 말해 공격은 입력 데이터에서 계산된 손실 변화도를 사용하고 입력 데이터를 조정하여 손실이 최대가 되게 합니다. -# -# 코드로 넘어가기 전에 유명한 `FGSM `__ 판다 예제를 +# +# 코드로 넘어가기 전에 유명한 `FGSM `__ 판다 예제를 # 보고 몇 가지 표기법을 정리하겠습니다. # # .. 그림:: /_static/img/fgsm_panda_image.png @@ -60,13 +60,13 @@ # :math:`y` 는 :math:`\mathbf{x}` 를 위한 정답 라벨이며, :math:`\mathbf{\theta}` 는 모델의 # 파라미터를, :math:`J(\mathbf{\theta}, \mathbf{x}, y)` 는 네트워크의 학습을 위해서 사용되는 손실을 나타냅니다. # 공격은 :math:`\nabla_{x} J(\mathbf{\theta}, \mathbf{x}, y)` 계산을 위해 입력 데이터에 변화도를 역전파합니다. -# 그러고 나서, 변화도는 손실 값이 최대화되는 방향으로 (예를 들면, :math:`sign(\nabla_{x} J(\mathbf{\theta}, \mathbf{x}, y))` ) +# 그러고 나서, 변화도는 손실 값이 최대화되는 방향으로 (예를 들면, :math:`sign(\nabla_{x} J(\mathbf{\theta}, \mathbf{x}, y))` ) # 작은 스텝(step) 만큼 (그림에서는 :math:`\epsilon` 혹은 :math:`0.007`) 입력 데이터에 적용됩니다. # 결과로 나오는 작은 변화된 이미지( :math:`x'` )는 타겟 네트워크에 의해 "긴팔원숭이"로 *오분류* 되나 여전히 육안으로는 # 분명히 "판다" 입니다. -# +# # 이제 본 튜토리얼의 동기가 명확해지길 바라며, 구현으로 넘어가 보겠습니다. -# +# from __future__ import print_function import torch @@ -81,27 +81,27 @@ ###################################################################### # 구현 # -------------- -# +# # 이 섹션에서는 튜토리얼의 입력 매개 변수에 대해 설명하고 공격중인 모델을 # 정의한 다음 공격을 코딩하고 일부 테스트를 실행합니다. -# +# # 입력 # ~~~~~~ -# +# # 이 학습서에는 입력이 3 개이며 다음과 같이 정의됩니다: -# +# # - **epsilons** - 실행에 사용할 엡실론의 리스트입니다. 엡실론 0의 값은 원래 테스트 셋의 모델 성능을 # 나타내므로 목록에 유지하는 것이 중요합니다. 또한 직관적으로 엡실론이 클수록 작은 변화가 더 눈에 띄지만 -# 모델 정확도를 저하 시키는 측면에서 더 효과가 있습니다. 여기서 데이터의 범위는 0-1 이기 때문에 +# 모델 정확도를 저하 시키는 측면에서 더 효과가 있습니다. 여기서 데이터의 범위는 0-1 이기 때문에 # 엡실론의 값은 1을 초과할 수 없습니다. -# -# - **pretrained_model** - `pytorch/examples/mnist `__ -# 를 통해 미리 학습된 MNIST 모델의 경로. +# +# - **pretrained_model** - `pytorch/examples/mnist `__ +# 를 통해 미리 학습된 MNIST 모델의 경로. # 튜토리얼을 간편하게 하려면 `여기 `__ 에서 미리 학습된 모델을 다운로드하세요. -# -# - **use_cuda** - CUDA 를 사용할지 말지 정하는 이진 플래그. +# +# - **use_cuda** - CUDA 를 사용할지 말지 정하는 이진 플래그. # 본 튜토리얼에서는 CPU 시간이 오래 걸리지 않으므로 CUDA를 지원하는 GPU 의 여부는 중요하지 않습니다. -# +# epsilons = [0, .05, .1, .15, .2, .25, .3] pretrained_model = "data/lenet_mnist_model.pth" @@ -111,12 +111,12 @@ ###################################################################### # 공격을 받는 모델 # ~~~~~~~~~~~~~~~~~~ -# +# # 앞서 말한대로, 공격을 받는 모델은 `pytorch/examples/mnist `__ # 와 동일한 MNIST 모델입니다. 본인의 MNIST 모델을 학습 및 저장하는 방식으로 하거나 제공된 모델을 다운로드 해 사용하는 식으로 진행할 수 있습니다. -# 여기서 *Net* 정의 및 테스트 데이터 로더는 MNIST 예제에서 복사 하였습니다. +# 여기서 *Net* 정의 및 테스트 데이터 로더는 MNIST 예제에서 복사 하였습니다. # 이 섹션의 목적은 모델과 데이터 로더를 정의한 다음, 모델을 초기화하고 미리 학습된 가중치를 읽어오는 것입니다. -# +# # LeNet 모델 정의 class Net(nn.Module): @@ -141,7 +141,7 @@ def forward(self, x): test_loader = torch.utils.data.DataLoader( datasets.MNIST('../data', train=False, download=True, transform=transforms.Compose([ transforms.ToTensor(), - ])), + ])), batch_size=1, shuffle=True) # 어떤 디바이스를 사용할지 정의 @@ -161,17 +161,17 @@ def forward(self, x): ###################################################################### # FGSM 공격 # ~~~~~~~~~~~ -# -# 이제 원래 입력을 교란시켜 적대적인 예를 만드는 함수를 정의 할 수 있습니다. -# ``fgsm_attack`` 함수는 입력 파라미터로 3가지를 가집니다. 첫번째는 원본 *이미지* ( :math:`x` ), -# 두번째는 *엡실론* 으로 픽셀 단위의 작은 변화를 주는 값입니다 ( :math:`\epsilon` ). -# 마지막은 *data_grad* 로 입력 영상 ( :math:`\nabla_{x} J(\mathbf{\theta}, \mathbf{x}, y)` ) 에 대한 변화도 손실 값입니다. +# +# 이제 원래 입력을 교란시켜 적대적인 예를 만드는 함수를 정의 할 수 있습니다. +# ``fgsm_attack`` 함수는 입력 파라미터로 3가지를 가집니다. 첫번째는 원본 *이미지* ( :math:`x` ), +# 두번째는 *엡실론* 으로 픽셀 단위의 작은 변화를 주는 값입니다 ( :math:`\epsilon` ). +# 마지막은 *data_grad* 로 입력 영상 ( :math:`\nabla_{x} J(\mathbf{\theta}, \mathbf{x}, y)` ) 에 대한 변화도 손실 값입니다. # 아래 식에 따른 작은 변화가 적용된 이미지를 생성합니다. -# +# # .. math:: perturbed\_image = image + epsilon*sign(data\_grad) = x + \epsilon * sign(\nabla_{x} J(\mathbf{\theta}, \mathbf{x}, y)) -# +# # 마지막으로 데이터의 원래 범위를 유지하기 위해, 작은 변화가 적용된 이미지가 :math:`[0,1]` 범위로 잘립니다. -# +# # FGSM 공격 코드 @@ -189,14 +189,14 @@ def fgsm_attack(image, epsilon, data_grad): ###################################################################### # 테스팅 함수 # ~~~~~~~~~~~~~~~~ -# -# 마지막으로 본 튜토리얼의 핵심 결과는 ``테스트`` 함수에서 오게 됩니다. +# +# 마지막으로 본 튜토리얼의 핵심 결과는 ``테스트`` 함수에서 오게 됩니다. # 이 테스트 기능을 호출 할 때마다 MNIST 테스트 셋에서 전체 테스트 단계를 수행하고 최종 정확도를 보고합니다. -# 그러나 이 함수에는 *엡실론* 입력도 필요합니다. 이는 ``테스트``` 함수가 :math:`\epsilon` 크기에 따라 공격자의 공격을 받는 모델의 -# 정확도을 보고하기 때문입니다. 더 구체적으로 보면 테스트 셋의 각각의 샘플에서 테스트 함수는 입력 데이터에 대한 손실 변화도( :math:`data\_grad` )를 계산하고, -# ``FGSM 공격``( :math:`perturbed\_data` )을 받은 작은 변화가 적용된 이미지를 만들고 나서 작은 변화가 적용된 이미지가 적대적인지 확인을 합니다. +# 그러나 이 함수에는 *엡실론* 입력도 필요합니다. 이는 ``테스트``` 함수가 :math:`\epsilon` 크기에 따라 공격자의 공격을 받는 모델의 +# 정확도을 보고하기 때문입니다. 더 구체적으로 보면 테스트 셋의 각각의 샘플에서 테스트 함수는 입력 데이터에 대한 손실 변화도( :math:`data\_grad` )를 계산하고, +# ``FGSM 공격`` (:math:`perturbed\_data`) 을 받은 작은 변화가 적용된 이미지를 만들고 나서 작은 변화가 적용된 이미지가 적대적인지 확인을 합니다. # 추가로 모델의 정확도를 테스트하기 위해서 테스트 함수는 나중에 시각화하여 볼 수 있도록 성공적으로 얻은 적대적 이미지를 저장하고 반환합니다. -# +# def test( model, device, test_loader, epsilon ): @@ -264,12 +264,12 @@ def test( model, device, test_loader, epsilon ): ###################################################################### # 공격 실행 # ~~~~~~~~~~ -# -# 구현의 마지막 부분은 공격을 실행하는 것입니다. 여기서 전체 테스트 스텝을 각 *엡실론* 값에 실행합니다. -# 각 엡실론마다 최종 정확도와 성공적인 일부 적대 사례를 저장하여 다음 섹션에 표시합니다. -# 엡실론 값이 증가함에 따라 출력된 정확도가 어떻게 감소하는지 보십시오. +# +# 구현의 마지막 부분은 공격을 실행하는 것입니다. 여기서 전체 테스트 스텝을 각 *엡실론* 값에 실행합니다. +# 각 엡실론마다 최종 정확도와 성공적인 일부 적대 사례를 저장하여 다음 섹션에 표시합니다. +# 엡실론 값이 증가함에 따라 출력된 정확도가 어떻게 감소하는지 보십시오. # 또한, :math:`\epsilon=0` 인 경우에는 공격이 없는 원본 테스트 정확도임을 보입니다. -# +# accuracies = [] examples = [] @@ -284,11 +284,11 @@ def test( model, device, test_loader, epsilon ): ###################################################################### # 결과 # ------- -# +# # 정확도 vs 엡실론 # ~~~~~~~~~~~~~~~~~~~ -# -# 첫 번째 결과는 정확도 vs 엡실론 을 도식화 한 것 입니다. +# +# 첫 번째 결과는 정확도 vs 엡실론 을 도식화 한 것 입니다. # 앞에서 언급했듯이, 엡실론이 증가함에 따라 우리는 테스트 정확도가 감소할 것으로 예상합니다. # 이는 학습을 더 진행해 갈수록 엡실론이 클수록 손실을 극대화 할 방향으로 진행되기 때문입니다. # 엡실론 값이 선형적으로 분포하더라도 곡선의 추세는 선형의 형태가 아닙니다. @@ -310,8 +310,8 @@ def test( model, device, test_loader, epsilon ): ###################################################################### # 샘플 적대적 예제들 # ~~~~~~~~~~~~~~~~~~~~~~~~~~~ -# -# 공짜 점심은 없다는 것을 기억하시나요? 이 경우에는 엡실론이 증가할수록 테스트 정확도는 떨어집니다. +# +# 공짜 점심은 없다는 것을 기억하시나요? 이 경우에는 엡실론이 증가할수록 테스트 정확도는 떨어집니다. # **그러나** 작은 변화는 더 쉽게 인식할 수 있게 됩니다. # 실제로 정확도 저하와 공격자가 고려해야 하는 이해도 사이에는 상충 관계(tradeoff)가 있습니다. # 여기서 우리는 각 엡실론 값에서 성공적인 대적 사례를 보이는 몇 가지 예를 보겠습니다. @@ -341,19 +341,19 @@ def test( model, device, test_loader, epsilon ): ###################################################################### # 다음 단계는? # ----------------- -# +# # 이번 튜토리얼에서 적대적 머신 러닝에 대한 통찰을 얻을 수 있었기를 바랍니다. # 튜토리얼의 내용으로부터 앞으로 더 많은 것들을 알아나갈 수 있습니다. -# 튜토리얼의 적대적 공격 예제는 본 분야의 초급 단계이며 +# 튜토리얼의 적대적 공격 예제는 본 분야의 초급 단계이며 # 적대적 상황으로부터 ML 모델을 공격하고 방어하는 방법에 대한 많은 후속 아이디어가 있습니다. # 사실 NIPS 2017 에서 적대적 공격과 방어에 대한 경쟁(competition)이 있었고 여기서 사용된 # 다양한 방법들은 다음 논문에 정리 되어 있습니다: `적대적 공격과 방어 경쟁 `__. # 방어에 대한 연구는 자연스럽게 교란 및 해킹 목적으로 제작된 입력에 대해 머신 러닝 모델을 # 보다 *견고하게(robust)* 만드는 아이디어로 이어집니다. -# -# 또 다른 방향은 다른 도메인에서의 적의 공격과 방어입니다. 적대적 연구는 이미지 도메인에 제한되어 있지 않습니다. +# +# 또 다른 방향은 다른 도메인에서의 적의 공격과 방어입니다. 적대적 연구는 이미지 도메인에 제한되어 있지 않습니다. # `여기 `__ 에서 음성-텍스트 변환 모델에서의 공격도 확인할 수 있습니다. # 그러나 적대적 머신 러닝 분야에 대해서 많은 것을 알기 위한 최고의 방법은 많이 시도해보는 것입니다. # NIPS 2017 경쟁에서 소개된 다양한 공격 방법을 직접 구현해 보고, FGSM 과 어떤 점이 다른지 연구해 보세요. # 그리고 나서 직접 만든 공격으로부터 모델을 방어해 보세요. -# +# diff --git a/docs/_downloads/37c8905519d3fd3f437b783a48d06eac/char_rnn_classification_tutorial.py b/docs/_downloads/37c8905519d3fd3f437b783a48d06eac/char_rnn_classification_tutorial.py index 4d766380e..da8d22535 100644 --- a/docs/_downloads/37c8905519d3fd3f437b783a48d06eac/char_rnn_classification_tutorial.py +++ b/docs/_downloads/37c8905519d3fd3f437b783a48d06eac/char_rnn_classification_tutorial.py @@ -51,7 +51,7 @@ 데이터 준비 ================== -.. NOTE:: +.. note:: `여기 `__ 에서 데이터를 다운 받고, 현재 디렉토리에 압축을 푸십시오. @@ -62,9 +62,10 @@ 각 언어 별로 이름 목록 사전 ``{language: [names ...]}`` 을 만듭니다. 일반 변수 "category" 와 "line" (우리의 경우 언어와 이름)은 이후의 확장성을 위해 사용됩니다. -.. NOTE:: -역자 주: "line" 에 입력을 "category"에 클래스를 적용하여 다른 문제에도 활용 할 수 있습니다. -여기서는 "line"에 이름(ex. Robert )를 입력으로 "category"에 클래스(ex. english)로 사용합니다. +.. note:: + 역자 주: "line" 에 입력을 "category"에 클래스를 적용하여 다른 문제에도 활용 할 수 있습니다. + 여기서는 "line"에 이름(ex. Robert )를 입력으로 "category"에 클래스(ex. english)로 사용합니다. + """ from __future__ import unicode_literals, print_function, division from io import open diff --git a/docs/_downloads/3dbbd6931d76adb0dc37d4e88b328852/tensor_tutorial.ipynb b/docs/_downloads/3dbbd6931d76adb0dc37d4e88b328852/tensor_tutorial.ipynb index 6f6a85cd5..83981770a 100644 --- a/docs/_downloads/3dbbd6931d76adb0dc37d4e88b328852/tensor_tutorial.ipynb +++ b/docs/_downloads/3dbbd6931d76adb0dc37d4e88b328852/tensor_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nPyTorch\uac00 \ubb34\uc5c7\uc778\uac00\uc694?\n=======================\n\nPython \uae30\ubc18\uc758 \uacfc\ud559 \uc5f0\uc0b0 \ud328\ud0a4\uc9c0\ub85c \ub2e4\uc74c\uacfc \uac19\uc740 \ub450 \uc9d1\ub2e8\uc744 \ub300\uc0c1\uc73c\ub85c \ud569\ub2c8\ub2e4:\n\n- NumPy\ub97c \ub300\uccb4\ud558\uba74\uc11c GPU\ub97c \uc774\uc6a9\ud55c \uc5f0\uc0b0\uc774 \ud544\uc694\ud55c \uacbd\uc6b0\n- \ucd5c\ub300\ud55c\uc758 \uc720\uc5f0\uc131\uacfc \uc18d\ub3c4\ub97c \uc81c\uacf5\ud558\ub294 \ub525\ub7ec\ub2dd \uc5f0\uad6c \ud50c\ub7ab\ud3fc\uc774 \ud544\uc694\ud55c \uacbd\uc6b0\n\n\uc2dc\uc791\ud558\uae30\n-----------\n\nTensors\n^^^^^^^\n\nTensor\ub294 NumPy\uc758 ndarray\uc640 \uc720\uc0ac\ud558\uba70, GPU\ub97c \uc0ac\uc6a9\ud55c \uc5f0\uc0b0 \uac00\uc18d\ub3c4 \uac00\ub2a5\ud569\ub2c8\ub2e4.\n\n" + "\nPyTorch\uac00 \ubb34\uc5c7\uc778\uac00\uc694?\n=======================\n\nPython \uae30\ubc18\uc758 \uacfc\ud559 \uc5f0\uc0b0 \ud328\ud0a4\uc9c0\ub85c \ub2e4\uc74c\uacfc \uac19\uc740 \ub450 \uc9d1\ub2e8\uc744 \ub300\uc0c1\uc73c\ub85c \ud569\ub2c8\ub2e4:\n\n- NumPy\ub97c \ub300\uccb4\ud558\uba74\uc11c GPU\ub97c \uc774\uc6a9\ud55c \uc5f0\uc0b0\uc774 \ud544\uc694\ud55c \uacbd\uc6b0\n- \ucd5c\ub300\ud55c\uc758 \uc720\uc5f0\uc131\uacfc \uc18d\ub3c4\ub97c \uc81c\uacf5\ud558\ub294 \ub525\ub7ec\ub2dd \uc5f0\uad6c \ud50c\ub7ab\ud3fc\uc774 \ud544\uc694\ud55c \uacbd\uc6b0\n\n\uc2dc\uc791\ud558\uae30\n-----------\n\nTensors\n^^^^^^^\n\nTensor\ub294 NumPy\uc758 ndarray\uc640 \uc720\uc0ac\ud558\uba70, GPU\ub97c \uc0ac\uc6a9\ud55c \uc5f0\uc0b0 \uac00\uc18d\ub3c4 \uac00\ub2a5\ud569\ub2c8\ub2e4.\n" ] }, { @@ -370,7 +370,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/40e628499d80c649ce6bf96fef98fcd2/two_layer_net_numpy.ipynb b/docs/_downloads/40e628499d80c649ce6bf96fef98fcd2/two_layer_net_numpy.ipynb index 75eb1e3f0..f6b0706bd 100644 --- a/docs/_downloads/40e628499d80c649ce6bf96fef98fcd2/two_layer_net_numpy.ipynb +++ b/docs/_downloads/40e628499d80c649ce6bf96fef98fcd2/two_layer_net_numpy.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n\uc900\ube44 \uc6b4\ub3d9: NumPy\n-----------------\n\n\ud558\ub098\uc758 \uc740\ub2c9\uce35(hidden layer)\uacfc \ud3b8\ud5a5(bias)\uc774 \uc5c6\ub294 \uc644\uc804\ud788 \uc5f0\uacb0\ub41c ReLU \uc2e0\uacbd\ub9dd\uc744,\n\uc720\ud074\ub9ac\ub4dc \uc624\ucc28(Euclidean error)\ub97c \ucd5c\uc18c\ud654\ud558\ub294 \uc2dd\uc73c\ub85c x\ub85c\ubd80\ud130 y\ub97c \uc608\uce21\ud558\ub3c4\ub85d\n\ud559\uc2b5\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\nNumPy\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc218\ub3d9\uc73c\ub85c \uc21c\uc804\ud30c, \uc190\uc2e4(loss), \uadf8\ub9ac\uace0 \uc5ed\uc804\ud30c \uc5f0\uc0b0\uc744 \ud558\ub294 \uac83\uc744\n\uad6c\ud604\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\nNumPy \ubc30\uc5f4\uc740 \uc77c\ubc18\uc801\uc740 N\ucc28\uc6d0 \ubc30\uc5f4\uc785\ub2c8\ub2e4; \ub525\ub7ec\ub2dd\uc774\ub098 \ubcc0\ud654\ub3c4(gradient), \uc5f0\uc0b0\n\uadf8\ub798\ud504(computational graph)\ub294 \uc54c\uc9c0 \ubabb\ud558\uba70, \uc77c\ubc18\uc801\uc778 \uc218\uce58 \uc5f0\uc0b0\uc744 \uc218\ud589\ud569\ub2c8\ub2e4.\n\n" + "\n\uc900\ube44 \uc6b4\ub3d9: NumPy\n-----------------\n\n\ud558\ub098\uc758 \uc740\ub2c9\uce35(hidden layer)\uacfc \ud3b8\ud5a5(bias)\uc774 \uc5c6\ub294 \uc644\uc804\ud788 \uc5f0\uacb0\ub41c ReLU \uc2e0\uacbd\ub9dd\uc744,\n\uc720\ud074\ub9ac\ub4dc \uc624\ucc28(Euclidean error)\ub97c \ucd5c\uc18c\ud654\ud558\ub294 \uc2dd\uc73c\ub85c x\ub85c\ubd80\ud130 y\ub97c \uc608\uce21\ud558\ub3c4\ub85d\n\ud559\uc2b5\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\nNumPy\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc218\ub3d9\uc73c\ub85c \uc21c\uc804\ud30c, \uc190\uc2e4(loss), \uadf8\ub9ac\uace0 \uc5ed\uc804\ud30c \uc5f0\uc0b0\uc744 \ud558\ub294 \uac83\uc744\n\uad6c\ud604\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\nNumPy \ubc30\uc5f4\uc740 \uc77c\ubc18\uc801\uc740 N\ucc28\uc6d0 \ubc30\uc5f4\uc785\ub2c8\ub2e4; \ub525\ub7ec\ub2dd\uc774\ub098 \ubcc0\ud654\ub3c4(gradient), \uc5f0\uc0b0\n\uadf8\ub798\ud504(computational graph)\ub294 \uc54c\uc9c0 \ubabb\ud558\uba70, \uc77c\ubc18\uc801\uc778 \uc218\uce58 \uc5f0\uc0b0\uc744 \uc218\ud589\ud569\ub2c8\ub2e4.\n" ] }, { @@ -46,7 +46,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/443402f98df8c37ed150fc9266eb8dee/super_resolution_with_onnxruntime.py b/docs/_downloads/443402f98df8c37ed150fc9266eb8dee/super_resolution_with_onnxruntime.py index 029987847..635bca82b 100644 --- a/docs/_downloads/443402f98df8c37ed150fc9266eb8dee/super_resolution_with_onnxruntime.py +++ b/docs/_downloads/443402f98df8c37ed150fc9266eb8dee/super_resolution_with_onnxruntime.py @@ -107,10 +107,10 @@ def _initialize_weights(self): # 이 함수는 모델을 실행하여 어떤 연산자들이 출력값을 계산하는데 사용되었는지를 기록합니다. # ``export`` 함수가 모델을 실행하기 때문에, 우리가 직접 텐서를 입력값으로 넘겨주어야 합니다. # 이 텐서의 값은 알맞은 자료형과 모양이라면 랜덤하게 결정되어도 무방합니다. -# 특정 차원을 동적인 차원으로 지정하지 않는 이상, ONNX로 변환된 그래프의 경우 입력값의 사이즈는 모든 차원에 대해 고정됩니다. +# 특정 차원을 동적인 차원으로 지정하지 않는 이상, ONNX로 변환된 그래프의 경우 입력값의 사이즈는 모든 차원에 대해 고정됩니다. # 예시에서는 모델이 항상 배치 사이즈 1을 사용하도록 변환하였지만, 첫번째 차원을 ``torch.onnx.export()`` 의 # ``dynamic_axes`` 인자에 동적인 차원으로 지정해주면 됩니다. -# +# # PyTorch의 변환 인터페이스에 대해 더 자세히 알고 싶다면 # `torch.onnx 문서 `__ 를 참고해주세요. # @@ -156,7 +156,7 @@ def _initialize_weights(self): # 이 부분은 보통 별도의 프로세스 또는 별도의 머신에서 실행되지만, 이 튜토리얼에서는 모델이 # ONNX 런타임과 PyTorch에서 동일한 결과를 출력하는지를 확인하기 위해 동일한 프로세스에서 계속 실행하도록 # 하겠습니다. -# +# # 모델을 ONNX 런타임에서 실행하기 위해서는 미리 설정된 인자들(본 예제에서는 기본값을 사용합니다)로 # 모델을 위한 추론 세션을 생성해야 합니다. # 세션이 생성되면, 모델의 run() API를 사용하여 모델을 실행합니다. @@ -189,7 +189,7 @@ def to_numpy(tensor): ###################################################################### # ONNX 런타임에서 이미지를 입력값으로 모델을 실행하기 -# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ # @@ -275,7 +275,7 @@ def to_numpy(tensor): # 더 자세한 내용은 `링크 `__ 를 # 참조해주십시오. # -# ONNX 런타임의 성능에 관한 것은 `여기 `__ 에서 +# ONNX 런타임의 성능에 관한 것은 `여기 `__ 에서 # 확인하실 수 있습니다. # # ONNX 런타임에 관한 더 자세한 내용은 `이 곳 `__ 을 참조해 주세요. diff --git a/docs/_downloads/44a84f8c1764dbf61662d306ff9ed43a/chatbot_tutorial.ipynb b/docs/_downloads/44a84f8c1764dbf61662d306ff9ed43a/chatbot_tutorial.ipynb index d9bd42a2f..94d46168e 100644 --- a/docs/_downloads/44a84f8c1764dbf61662d306ff9ed43a/chatbot_tutorial.ipynb +++ b/docs/_downloads/44a84f8c1764dbf61662d306ff9ed43a/chatbot_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n\ucc57\ubd07 \ud29c\ud1a0\ub9ac\uc5bc\n================\n**Author:** `Matthew Inkawhich `_\n **\ubc88\uc5ed**: `\uae40\uc9c4\ud604 `_\n\n" + "\n\ucc57\ubd07 \ud29c\ud1a0\ub9ac\uc5bc\n================\n**Author:** `Matthew Inkawhich `_\n **\ubc88\uc5ed**: `\uae40\uc9c4\ud604 `_\n" ] }, { @@ -353,7 +353,7 @@ }, "outputs": [], "source": [ - "# \ud559\uc2b5 \ubc0f \ucd5c\uc801\ud654 \uc124\uc815\nclip = 50.0\nteacher_forcing_ratio = 1.0\nlearning_rate = 0.0001\ndecoder_learning_ratio = 5.0\nn_iteration = 4000\nprint_every = 1\nsave_every = 500\n\n# Dropout \ub808\uc774\uc5b4\ub97c \ud559\uc2b5 \ubaa8\ub4dc\ub85c \ub461\ub2c8\ub2e4\nencoder.train()\ndecoder.train()\n\n# Optimizer\ub97c \ucd08\uae30\ud654\ud569\ub2c8\ub2e4\nprint('Building optimizers ...')\nencoder_optimizer = optim.Adam(encoder.parameters(), lr=learning_rate)\ndecoder_optimizer = optim.Adam(decoder.parameters(), lr=learning_rate * decoder_learning_ratio)\nif loadFilename:\n encoder_optimizer.load_state_dict(encoder_optimizer_sd)\n decoder_optimizer.load_state_dict(decoder_optimizer_sd)\n\n# cuda\uac00 \uc788\ub2e4\uba74 cuda\ub97c \uc124\uc815\ud569\ub2c8\ub2e4\nfor state in encoder_optimizer.state.values():\n for k, v in state.items():\n if isinstance(v, torch.Tensor):\n state[k] = v.cuda()\n\nfor state in decoder_optimizer.state.values():\n for k, v in state.items():\n if isinstance(v, torch.Tensor):\n state[k] = v.cuda()\n \n# \ud559\uc2b5 \ub2e8\uacc4\ub97c \uc218\ud589\ud569\ub2c8\ub2e4\nprint(\"Starting Training!\")\ntrainIters(model_name, voc, pairs, encoder, decoder, encoder_optimizer, decoder_optimizer,\n embedding, encoder_n_layers, decoder_n_layers, save_dir, n_iteration, batch_size,\n print_every, save_every, clip, corpus_name, loadFilename)" + "# \ud559\uc2b5 \ubc0f \ucd5c\uc801\ud654 \uc124\uc815\nclip = 50.0\nteacher_forcing_ratio = 1.0\nlearning_rate = 0.0001\ndecoder_learning_ratio = 5.0\nn_iteration = 4000\nprint_every = 1\nsave_every = 500\n\n# Dropout \ub808\uc774\uc5b4\ub97c \ud559\uc2b5 \ubaa8\ub4dc\ub85c \ub461\ub2c8\ub2e4\nencoder.train()\ndecoder.train()\n\n# Optimizer\ub97c \ucd08\uae30\ud654\ud569\ub2c8\ub2e4\nprint('Building optimizers ...')\nencoder_optimizer = optim.Adam(encoder.parameters(), lr=learning_rate)\ndecoder_optimizer = optim.Adam(decoder.parameters(), lr=learning_rate * decoder_learning_ratio)\nif loadFilename:\n encoder_optimizer.load_state_dict(encoder_optimizer_sd)\n decoder_optimizer.load_state_dict(decoder_optimizer_sd)\n\n# cuda\uac00 \uc788\ub2e4\uba74 cuda\ub97c \uc124\uc815\ud569\ub2c8\ub2e4\nfor state in encoder_optimizer.state.values():\n for k, v in state.items():\n if isinstance(v, torch.Tensor):\n state[k] = v.cuda()\n\nfor state in decoder_optimizer.state.values():\n for k, v in state.items():\n if isinstance(v, torch.Tensor):\n state[k] = v.cuda()\n\n# \ud559\uc2b5 \ub2e8\uacc4\ub97c \uc218\ud589\ud569\ub2c8\ub2e4\nprint(\"Starting Training!\")\ntrainIters(model_name, voc, pairs, encoder, decoder, encoder_optimizer, decoder_optimizer,\n embedding, encoder_n_layers, decoder_n_layers, save_dir, n_iteration, batch_size,\n print_every, save_every, clip, corpus_name, loadFilename)" ] }, { @@ -398,7 +398,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/4e865243430a47a00d551ca0579a6f6c/cifar10_tutorial.ipynb b/docs/_downloads/4e865243430a47a00d551ca0579a6f6c/cifar10_tutorial.ipynb index 4c98387b7..16e7d46f5 100644 --- a/docs/_downloads/4e865243430a47a00d551ca0579a6f6c/cifar10_tutorial.ipynb +++ b/docs/_downloads/4e865243430a47a00d551ca0579a6f6c/cifar10_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n\ubd84\ub958\uae30(Classifier) \ud559\uc2b5\ud558\uae30\n============================\n\n\uc9c0\uae08\uae4c\uc9c0 \uc5b4\ub5bb\uac8c \uc2e0\uacbd\ub9dd\uc744 \uc815\uc758\ud558\uace0, \uc190\uc2e4\uc744 \uacc4\uc0b0\ud558\uba70 \ub610 \uac00\uc911\uce58\ub97c \uac31\uc2e0\ud558\ub294\uc9c0\uc5d0\n\ub300\ud574\uc11c \ubc30\uc6e0\uc2b5\ub2c8\ub2e4.\n\n\uc774\uc81c \uc544\ub9c8\ub3c4 \uc774\ub7f0 \uc0dd\uac01\uc744 \ud558\uace0 \uacc4\uc2e4\ud150\ub370\uc694,\n\n\ub370\uc774\ud130\ub294 \uc5b4\ub5bb\uac8c \ud558\ub098\uc694?\n------------------------\n\n\uc77c\ubc18\uc801\uc73c\ub85c \uc774\ubbf8\uc9c0\ub098 \ud14d\uc2a4\ud2b8, \uc624\ub514\uc624\ub098 \ube44\ub514\uc624 \ub370\uc774\ud130\ub97c \ub2e4\ub8f0 \ub54c\ub294 \ud45c\uc900 Python \ud328\ud0a4\uc9c0\ub97c\n\uc774\uc6a9\ud558\uc5ec NumPy \ubc30\uc5f4\ub85c \ubd88\ub7ec\uc624\uba74 \ub429\ub2c8\ub2e4. \uadf8 \ud6c4 \uadf8 \ubc30\uc5f4\uc744 ``torch.*Tensor`` \ub85c \ubcc0\ud658\ud569\ub2c8\ub2e4.\n\n- \uc774\ubbf8\uc9c0\ub294 Pillow\ub098 OpenCV \uac19\uc740 \ud328\ud0a4\uc9c0\uac00 \uc720\uc6a9\ud569\ub2c8\ub2e4.\n- \uc624\ub514\uc624\ub97c \ucc98\ub9ac\ud560 \ub54c\ub294 SciPy\uc640 LibROSA\uac00 \uc720\uc6a9\ud558\uace0\uc694.\n- \ud14d\uc2a4\ud2b8\uc758 \uacbd\uc6b0\uc5d0\ub294 \uadf8\ub0e5 Python\uc774\ub098 Cython\uc744 \uc0ac\uc6a9\ud574\ub3c4 \ub418\uace0, NLTK\ub098 SpaCy\ub3c4\n \uc720\uc6a9\ud569\ub2c8\ub2e4.\n\n\ud2b9\ubcc4\ud788 \uc601\uc0c1 \ubd84\uc57c\ub97c \uc704\ud55c ``torchvision`` \uc774\ub77c\ub294 \ud328\ud0a4\uc9c0\uac00 \ub9cc\ub4e4\uc5b4\uc838 \uc788\ub294\ub370,\n\uc5ec\uae30\uc5d0\ub294 Imagenet\uc774\ub098 CIFAR10, MNIST \ub4f1\uacfc \uac19\uc774 \uc77c\ubc18\uc801\uc73c\ub85c \uc0ac\uc6a9\ud558\ub294 \ub370\uc774\ud130\uc14b\uc744 \uc704\ud55c\n\ub370\uc774\ud130 \ub85c\ub354(data loader), \uc989 ``torchvision.datasets`` \uacfc \uc774\ubbf8\uc9c0\uc6a9 \ub370\uc774\ud130 \ubcc0\ud658\uae30\n(data transformer), \uc989 ``torch.utils.data.DataLoader`` \uac00 \ud3ec\ud568\ub418\uc5b4 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc774\ub7ec\ud55c \uae30\ub2a5\uc740 \uc5c4\uccad\ub098\uac8c \ud3b8\ub9ac\ud558\uba70, \ub9e4\ubc88 \uc720\uc0ac\ud55c \ucf54\ub4dc(boilerplate code)\ub97c \ubc18\ubcf5\ud574\uc11c\n\uc791\uc131\ud558\ub294 \uac83\uc744 \ud53c\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 CIFAR10 \ub370\uc774\ud130\uc14b\uc744 \uc0ac\uc6a9\ud569\ub2c8\ub2e4. \uc5ec\uae30\uc5d0\ub294 \ub2e4\uc74c\uacfc \uac19\uc740 \ubd84\ub958\ub4e4\uc774\n\uc788\uc2b5\ub2c8\ub2e4: '\ube44\ud589\uae30(airplane)', '\uc790\ub3d9\ucc28(automobile)', '\uc0c8(bird)', '\uace0\uc591\uc774(cat)',\n'\uc0ac\uc2b4(deer)', '\uac1c(dog)', '\uac1c\uad6c\ub9ac(frog)', '\ub9d0(horse)', '\ubc30(ship)', '\ud2b8\ub7ed(truck)'.\n\uadf8\ub9ac\uace0 CIFAR10\uc5d0 \ud3ec\ud568\ub41c \uc774\ubbf8\uc9c0\uc758 \ud06c\uae30\ub294 3x32x32\ub85c, \uc774\ub294 32x32 \ud53d\uc140 \ud06c\uae30\uc758 \uc774\ubbf8\uc9c0\uac00\n3\uac1c \ucc44\ub110(channel)\uc758 \uc0c9\uc0c1\ub85c \uc774\ub904\uc838 \uc788\ub2e4\ub294 \uac83\uc744 \ub73b\ud569\ub2c8\ub2e4.\n\n.. figure:: /_static/img/cifar10.png\n :alt: cifar10\n\n cifar10\n\n\n\uc774\ubbf8\uc9c0 \ubd84\ub958\uae30 \ud559\uc2b5\ud558\uae30\n----------------------------\n\n\ub2e4\uc74c\uacfc \uac19\uc740 \ub2e8\uacc4\ub85c \uc9c4\ud589\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4:\n\n1. ``torchvision`` \uc744 \uc0ac\uc6a9\ud558\uc5ec CIFAR10\uc758 \ud559\uc2b5\uc6a9 / \uc2dc\ud5d8\uc6a9 \ub370\uc774\ud130\uc14b\uc744\n \ubd88\ub7ec\uc624\uace0, \uc815\uaddc\ud654(nomarlizing)\ud569\ub2c8\ub2e4.\n2. \ud569\uc131\uacf1 \uc2e0\uacbd\ub9dd(Convolution Neural Network)\uc744 \uc815\uc758\ud569\ub2c8\ub2e4.\n3. \uc190\uc2e4 \ud568\uc218\ub97c \uc815\uc758\ud569\ub2c8\ub2e4.\n4. \ud559\uc2b5\uc6a9 \ub370\uc774\ud130\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc2e0\uacbd\ub9dd\uc744 \ud559\uc2b5\ud569\ub2c8\ub2e4.\n5. \uc2dc\ud5d8\uc6a9 \ub370\uc774\ud130\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc2e0\uacbd\ub9dd\uc744 \uac80\uc0ac\ud569\ub2c8\ub2e4.\n\n1. CIFAR10\ub97c \ubd88\ub7ec\uc624\uace0 \uc815\uaddc\ud654\ud558\uae30\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n``torchvision`` \uc744 \uc0ac\uc6a9\ud558\uc5ec \ub9e4\uc6b0 \uc27d\uac8c CIFAR10\uc744 \ubd88\ub7ec\uc62c \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n" + "\n\ubd84\ub958\uae30(Classifier) \ud559\uc2b5\ud558\uae30\n============================\n\n\uc9c0\uae08\uae4c\uc9c0 \uc5b4\ub5bb\uac8c \uc2e0\uacbd\ub9dd\uc744 \uc815\uc758\ud558\uace0, \uc190\uc2e4\uc744 \uacc4\uc0b0\ud558\uba70 \ub610 \uac00\uc911\uce58\ub97c \uac31\uc2e0\ud558\ub294\uc9c0\uc5d0\n\ub300\ud574\uc11c \ubc30\uc6e0\uc2b5\ub2c8\ub2e4.\n\n\uc774\uc81c \uc544\ub9c8\ub3c4 \uc774\ub7f0 \uc0dd\uac01\uc744 \ud558\uace0 \uacc4\uc2e4\ud150\ub370\uc694,\n\n\ub370\uc774\ud130\ub294 \uc5b4\ub5bb\uac8c \ud558\ub098\uc694?\n------------------------\n\n\uc77c\ubc18\uc801\uc73c\ub85c \uc774\ubbf8\uc9c0\ub098 \ud14d\uc2a4\ud2b8, \uc624\ub514\uc624\ub098 \ube44\ub514\uc624 \ub370\uc774\ud130\ub97c \ub2e4\ub8f0 \ub54c\ub294 \ud45c\uc900 Python \ud328\ud0a4\uc9c0\ub97c\n\uc774\uc6a9\ud558\uc5ec NumPy \ubc30\uc5f4\ub85c \ubd88\ub7ec\uc624\uba74 \ub429\ub2c8\ub2e4. \uadf8 \ud6c4 \uadf8 \ubc30\uc5f4\uc744 ``torch.*Tensor`` \ub85c \ubcc0\ud658\ud569\ub2c8\ub2e4.\n\n- \uc774\ubbf8\uc9c0\ub294 Pillow\ub098 OpenCV \uac19\uc740 \ud328\ud0a4\uc9c0\uac00 \uc720\uc6a9\ud569\ub2c8\ub2e4.\n- \uc624\ub514\uc624\ub97c \ucc98\ub9ac\ud560 \ub54c\ub294 SciPy\uc640 LibROSA\uac00 \uc720\uc6a9\ud558\uace0\uc694.\n- \ud14d\uc2a4\ud2b8\uc758 \uacbd\uc6b0\uc5d0\ub294 \uadf8\ub0e5 Python\uc774\ub098 Cython\uc744 \uc0ac\uc6a9\ud574\ub3c4 \ub418\uace0, NLTK\ub098 SpaCy\ub3c4\n \uc720\uc6a9\ud569\ub2c8\ub2e4.\n\n\ud2b9\ubcc4\ud788 \uc601\uc0c1 \ubd84\uc57c\ub97c \uc704\ud55c ``torchvision`` \uc774\ub77c\ub294 \ud328\ud0a4\uc9c0\uac00 \ub9cc\ub4e4\uc5b4\uc838 \uc788\ub294\ub370,\n\uc5ec\uae30\uc5d0\ub294 Imagenet\uc774\ub098 CIFAR10, MNIST \ub4f1\uacfc \uac19\uc774 \uc77c\ubc18\uc801\uc73c\ub85c \uc0ac\uc6a9\ud558\ub294 \ub370\uc774\ud130\uc14b\uc744 \uc704\ud55c\n\ub370\uc774\ud130 \ub85c\ub354(data loader), \uc989 ``torchvision.datasets`` \uacfc \uc774\ubbf8\uc9c0\uc6a9 \ub370\uc774\ud130 \ubcc0\ud658\uae30\n(data transformer), \uc989 ``torch.utils.data.DataLoader`` \uac00 \ud3ec\ud568\ub418\uc5b4 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc774\ub7ec\ud55c \uae30\ub2a5\uc740 \uc5c4\uccad\ub098\uac8c \ud3b8\ub9ac\ud558\uba70, \ub9e4\ubc88 \uc720\uc0ac\ud55c \ucf54\ub4dc(boilerplate code)\ub97c \ubc18\ubcf5\ud574\uc11c\n\uc791\uc131\ud558\ub294 \uac83\uc744 \ud53c\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 CIFAR10 \ub370\uc774\ud130\uc14b\uc744 \uc0ac\uc6a9\ud569\ub2c8\ub2e4. \uc5ec\uae30\uc5d0\ub294 \ub2e4\uc74c\uacfc \uac19\uc740 \ubd84\ub958\ub4e4\uc774\n\uc788\uc2b5\ub2c8\ub2e4: '\ube44\ud589\uae30(airplane)', '\uc790\ub3d9\ucc28(automobile)', '\uc0c8(bird)', '\uace0\uc591\uc774(cat)',\n'\uc0ac\uc2b4(deer)', '\uac1c(dog)', '\uac1c\uad6c\ub9ac(frog)', '\ub9d0(horse)', '\ubc30(ship)', '\ud2b8\ub7ed(truck)'.\n\uadf8\ub9ac\uace0 CIFAR10\uc5d0 \ud3ec\ud568\ub41c \uc774\ubbf8\uc9c0\uc758 \ud06c\uae30\ub294 3x32x32\ub85c, \uc774\ub294 32x32 \ud53d\uc140 \ud06c\uae30\uc758 \uc774\ubbf8\uc9c0\uac00\n3\uac1c \ucc44\ub110(channel)\uc758 \uc0c9\uc0c1\ub85c \uc774\ub904\uc838 \uc788\ub2e4\ub294 \uac83\uc744 \ub73b\ud569\ub2c8\ub2e4.\n\n.. figure:: /_static/img/cifar10.png\n :alt: cifar10\n\n cifar10\n\n\n\uc774\ubbf8\uc9c0 \ubd84\ub958\uae30 \ud559\uc2b5\ud558\uae30\n----------------------------\n\n\ub2e4\uc74c\uacfc \uac19\uc740 \ub2e8\uacc4\ub85c \uc9c4\ud589\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4:\n\n1. ``torchvision`` \uc744 \uc0ac\uc6a9\ud558\uc5ec CIFAR10\uc758 \ud559\uc2b5\uc6a9 / \uc2dc\ud5d8\uc6a9 \ub370\uc774\ud130\uc14b\uc744\n \ubd88\ub7ec\uc624\uace0, \uc815\uaddc\ud654(nomarlizing)\ud569\ub2c8\ub2e4.\n2. \ud569\uc131\uacf1 \uc2e0\uacbd\ub9dd(Convolution Neural Network)\uc744 \uc815\uc758\ud569\ub2c8\ub2e4.\n3. \uc190\uc2e4 \ud568\uc218\ub97c \uc815\uc758\ud569\ub2c8\ub2e4.\n4. \ud559\uc2b5\uc6a9 \ub370\uc774\ud130\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc2e0\uacbd\ub9dd\uc744 \ud559\uc2b5\ud569\ub2c8\ub2e4.\n5. \uc2dc\ud5d8\uc6a9 \ub370\uc774\ud130\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc2e0\uacbd\ub9dd\uc744 \uac80\uc0ac\ud569\ub2c8\ub2e4.\n\n1. CIFAR10\ub97c \ubd88\ub7ec\uc624\uace0 \uc815\uaddc\ud654\ud558\uae30\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n``torchvision`` \uc744 \uc0ac\uc6a9\ud558\uc5ec \ub9e4\uc6b0 \uc27d\uac8c CIFAR10\uc744 \ubd88\ub7ec\uc62c \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n" ] }, { @@ -298,7 +298,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/52d4aaa33601a2b3990ace6aa45546ce/numpy_extensions_tutorial.ipynb b/docs/_downloads/52d4aaa33601a2b3990ace6aa45546ce/numpy_extensions_tutorial.ipynb index 3a87a6f52..155f96c8f 100644 --- a/docs/_downloads/52d4aaa33601a2b3990ace6aa45546ce/numpy_extensions_tutorial.ipynb +++ b/docs/_downloads/52d4aaa33601a2b3990ace6aa45546ce/numpy_extensions_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nCreating Extensions Using numpy and scipy\n=========================================\n**Author**: `Adam Paszke `_\n\n**Updated by**: `Adam Dziedzic `_\n\nIn this tutorial, we shall go through two tasks:\n\n1. Create a neural network layer with no parameters.\n\n - This calls into **numpy** as part of its implementation\n\n2. Create a neural network layer that has learnable weights\n\n - This calls into **SciPy** as part of its implementation\n\n" + "\nnumpy \uc640 scipy \ub97c \uc774\uc6a9\ud55c \ud655\uc7a5(Extensions) \ub9cc\ub4e4\uae30\n=====================================================\n**Author**: `Adam Paszke `_\n\n**Updated by**: `Adam Dziedzic `_\n\n**\ubc88\uc5ed**: `Ajin Jeong `_\n\n\uc774\ubc88 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 \ub450 \uac00\uc9c0 \uc791\uc5c5\uc744 \uc218\ud589\ud560 \uac83\uc785\ub2c8\ub2e4:\n\n1. \ub9e4\uac1c \ubcc0\uc218\uac00 \uc5c6\ub294 \uc2e0\uacbd\ub9dd \uacc4\uce35(layer) \ub9cc\ub4e4\uae30\n - \uc774\ub294 \uad6c\ud604\uc758 \uc77c\ubd80\ub85c **numpy** \ub97c \ud638\ucd9c\ud569\ub2c8\ub2e4.\n\n2. \ud559\uc2b5 \uac00\ub2a5\ud55c \uac00\uc911\uce58\uac00 \uc788\ub294 \uc2e0\uacbd\ub9dd \uacc4\uce35(layer) \ub9cc\ub4e4\uae30\n - \uc774\ub294 \uad6c\ud604\uc758 \uc77c\ubd80\ub85c **Scipy** \ub97c \ud638\ucd9c\ud569\ub2c8\ub2e4.\n" ] }, { @@ -33,7 +33,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Parameter-less example\n----------------------\n\nThis layer doesn\u2019t particularly do anything useful or mathematically\ncorrect.\n\nIt is aptly named BadFFTFunction\n\n**Layer Implementation**\n\n" + "\ub9e4\uac1c \ubcc0\uc218\uac00 \uc5c6\ub294(Parameter-less) \uc608\uc2dc\n----------------------------------------\n\n\uc774 \uacc4\uce35(layer)\uc740 \ud2b9\ubcc4\ud788 \uc720\uc6a9\ud558\uac70\ub098 \uc218\ud559\uc801\uc73c\ub85c \uc62c\ubc14\ub978 \uc791\uc5c5\uc744 \uc218\ud589\ud558\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4.\n\n\uc774\ub984\uc740 \ub300\ucda9 BadFFTFunction\uc73c\ub85c \uc9c0\uc5c8\uc2b5\ub2c8\ub2e4.\n\n**\uacc4\uce35(layer) \uad6c\ud604**\n\n" ] }, { @@ -44,14 +44,14 @@ }, "outputs": [], "source": [ - "from numpy.fft import rfft2, irfft2\n\n\nclass BadFFTFunction(Function):\n @staticmethod\n def forward(ctx, input):\n numpy_input = input.detach().numpy()\n result = abs(rfft2(numpy_input))\n return input.new(result)\n\n @staticmethod\n def backward(ctx, grad_output):\n numpy_go = grad_output.numpy()\n result = irfft2(numpy_go)\n return grad_output.new(result)\n\n# since this layer does not have any parameters, we can\n# simply declare this as a function, rather than as an nn.Module class\n\n\ndef incorrect_fft(input):\n return BadFFTFunction.apply(input)" + "from numpy.fft import rfft2, irfft2\n\n\nclass BadFFTFunction(Function):\n @staticmethod\n def forward(ctx, input):\n numpy_input = input.detach().numpy()\n result = abs(rfft2(numpy_input))\n return input.new(result)\n\n @staticmethod\n def backward(ctx, grad_output):\n numpy_go = grad_output.numpy()\n result = irfft2(numpy_go)\n return grad_output.new(result)\n\n# \uc774 \uacc4\uce35\uc5d0\ub294 \ub9e4\uac1c \ubcc0\uc218\uac00 \uc5c6\uc73c\ubbc0\ub85c nn.Module \ud074\ub798\uc2a4\uac00 \uc544\ub2cc \ud568\uc218\ub85c \uac04\ub2e8\ud788 \uc120\uc5b8\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\ndef incorrect_fft(input):\n return BadFFTFunction.apply(input)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "**Example usage of the created layer:**\n\n" + "**\uc0dd\uc131\ub41c \uacc4\uce35(layer)\uc758 \uc0ac\uc6a9 \uc608\uc2dc:**\n\n" ] }, { @@ -69,7 +69,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Parametrized example\n--------------------\n\nIn deep learning literature, this layer is confusingly referred\nto as convolution while the actual operation is cross-correlation\n(the only difference is that filter is flipped for convolution,\nwhich is not the case for cross-correlation).\n\nImplementation of a layer with learnable weights, where cross-correlation\nhas a filter (kernel) that represents weights.\n\nThe backward pass computes the gradient wrt the input and the gradient wrt the filter.\n\n" + "\ub9e4\uac1c \ubcc0\uc218\uac00 \uc788\ub294(Parameterized) \uc608\uc2dc\n----------------------------------------\n\n\ub525\ub7ec\ub2dd \ubb38\ud5cc\uc5d0\uc11c \uc774 \uacc4\uce35(layer)\uc758 \uc2e4\uc81c \uc5f0\uc0b0\uc740 \uc0c1\ud638 \uc0c1\uad00(cross-correlation)\uc774\uc9c0\ub9cc\n\ud569\uc131\uacf1(convolution)\uc774\ub77c\uace0 \ud5f7\uac08\ub9ac\uac8c \ubd80\ub974\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n(\ud569\uc131\uacf1\uc740 \ud544\ud130\ub97c \ub4a4\uc9d1\uc5b4\uc11c \uc5f0\uc0b0\uc744 \ud558\ub294 \ubc18\uba74, \uc0c1\ud638 \uc0c1\uad00\uc740 \uadf8\ub807\uc9c0 \uc54a\uc740 \ucc28\uc774\uac00 \uc788\uc2b5\ub2c8\ub2e4)\n\n\ud559\uc2b5 \uac00\ub2a5\ud55c \uac00\uc911\uce58\ub97c \uac00\ub294 \ud544\ud130(\ucee4\ub110)\ub97c \uac16\ub294 \uc0c1\ud638 \uc0c1\uad00 \uacc4\uce35\uc744 \uad6c\ud604\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc5ed\uc804\ud30c \ub2e8\uacc4(backward pass)\uc5d0\uc11c\ub294 \uc785\ub825\uc5d0 \ub300\ud55c \uae30\uc6b8\uae30(gradient)\uc640 \ud544\ud130\uc5d0 \ub300\ud55c \uae30\uc6b8\uae30\ub97c \uacc4\uc0b0\ud569\ub2c8\ub2e4.\n\n" ] }, { @@ -80,14 +80,14 @@ }, "outputs": [], "source": [ - "from numpy import flip\nimport numpy as np\nfrom scipy.signal import convolve2d, correlate2d\nfrom torch.nn.modules.module import Module\nfrom torch.nn.parameter import Parameter\n\n\nclass ScipyConv2dFunction(Function):\n @staticmethod\n def forward(ctx, input, filter, bias):\n # detach so we can cast to NumPy\n input, filter, bias = input.detach(), filter.detach(), bias.detach()\n result = correlate2d(input.numpy(), filter.numpy(), mode='valid')\n result += bias.numpy()\n ctx.save_for_backward(input, filter, bias)\n return torch.as_tensor(result, dtype=input.dtype)\n\n @staticmethod\n def backward(ctx, grad_output):\n grad_output = grad_output.detach()\n input, filter, bias = ctx.saved_tensors\n grad_output = grad_output.numpy()\n grad_bias = np.sum(grad_output, keepdims=True)\n grad_input = convolve2d(grad_output, filter.numpy(), mode='full')\n # the previous line can be expressed equivalently as:\n # grad_input = correlate2d(grad_output, flip(flip(filter.numpy(), axis=0), axis=1), mode='full')\n grad_filter = correlate2d(input.numpy(), grad_output, mode='valid')\n return torch.from_numpy(grad_input), torch.from_numpy(grad_filter).to(torch.float), torch.from_numpy(grad_bias).to(torch.float)\n\n\nclass ScipyConv2d(Module):\n def __init__(self, filter_width, filter_height):\n super(ScipyConv2d, self).__init__()\n self.filter = Parameter(torch.randn(filter_width, filter_height))\n self.bias = Parameter(torch.randn(1, 1))\n\n def forward(self, input):\n return ScipyConv2dFunction.apply(input, self.filter, self.bias)" + "from numpy import flip\nimport numpy as np\nfrom scipy.signal import convolve2d, correlate2d\nfrom torch.nn.modules.module import Module\nfrom torch.nn.parameter import Parameter\n\n\nclass ScipyConv2dFunction(Function):\n @staticmethod\n def forward(ctx, input, filter, bias):\n # \ubd84\ub9ac(detach)\ud558\uc5ec NumPy\ub85c \ubcc0\ud658(cast)\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n input, filter, bias = input.detach(), filter.detach(), bias.detach()\n result = correlate2d(input.numpy(), filter.numpy(), mode='valid')\n result += bias.numpy()\n ctx.save_for_backward(input, filter, bias)\n return torch.as_tensor(result, dtype=input.dtype)\n\n @staticmethod\n def backward(ctx, grad_output):\n grad_output = grad_output.detach()\n input, filter, bias = ctx.saved_tensors\n grad_output = grad_output.numpy()\n grad_bias = np.sum(grad_output, keepdims=True)\n grad_input = convolve2d(grad_output, filter.numpy(), mode='full')\n # \uc717\uc904\uc740 \ub2e4\uc74c\uacfc \uac19\uc774 \ud45c\ud604\ud560 \uc218\ub3c4 \uc788\uc2b5\ub2c8\ub2e4:\n # grad_input = correlate2d(grad_output, flip(flip(filter.numpy(), axis=0), axis=1), mode='full')\n grad_filter = correlate2d(input.numpy(), grad_output, mode='valid')\n return torch.from_numpy(grad_input), torch.from_numpy(grad_filter).to(torch.float), torch.from_numpy(grad_bias).to(torch.float)\n\n\nclass ScipyConv2d(Module):\n def __init__(self, filter_width, filter_height):\n super(ScipyConv2d, self).__init__()\n self.filter = Parameter(torch.randn(filter_width, filter_height))\n self.bias = Parameter(torch.randn(1, 1))\n\n def forward(self, input):\n return ScipyConv2dFunction.apply(input, self.filter, self.bias)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "**Example usage:**\n\n" + "**\uc0ac\uc6a9 \uc608\uc2dc:**\n\n" ] }, { @@ -105,7 +105,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "**Check the gradients:**\n\n" + "**\uae30\uc6b8\uae30(gradient) \ud655\uc778:**\n\n" ] }, { @@ -136,7 +136,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/56c122e1c18e5e07666673e900acaed5/fgsm_tutorial.ipynb b/docs/_downloads/56c122e1c18e5e07666673e900acaed5/fgsm_tutorial.ipynb index 138b61a85..df7cebe47 100644 --- a/docs/_downloads/56c122e1c18e5e07666673e900acaed5/fgsm_tutorial.ipynb +++ b/docs/_downloads/56c122e1c18e5e07666673e900acaed5/fgsm_tutorial.ipynb @@ -15,14 +15,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n\uc801\ub300\uc801 \uc608\uc81c \uc0dd\uc131(Adversarial Example Generation)\n====================================================\n\n**\uc800\uc790:** `Nathan Inkawhich `__\n**\ubc88\uc5ed:** `BONGMO KIM `__\n\n\uc774 \uae00\uc744 \uc77d\uace0 \uc788\ub2e4\uba74, \uc5ec\ub7ec\ubd84\uc740 \uc774\ubbf8 \uba38\uc2e0\ub7ec\ub2dd \ubaa8\ub378\uc774 \uc5bc\ub9c8\ub098 \ud6a8\uacfc\uc801\uc778\uc9c0 \uadf8 \uc9c4\uac00\ub97c \uc54c\uace0 \uc788\uc744 \uac83\uc785\ub2c8\ub2e4.\n\uba38\uc2e0 \ub7ec\ub2dd \uc5f0\uad6c\ub294 ML(Machine Learning) \ubaa8\ub378\uc744 \ub354\uc6b1 \ube60\ub974\uace0 \uc815\ud655\ud558\uba70 \ud6a8\uc728\uc801\uc774\uac8c \ud558\ub294 \ubc29\ud5a5\uc73c\ub85c \uc9c4\ud589 \ub418\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n\uadf8\ub7ec\ub098 \ubaa8\ub378\uc744 \uc18d\uc774\ub824\ud558\ub294 \uc801\uc5d0 \ub300\ud55c \ubcf4\uc548\uacfc \uacac\uace0\ud568\uc740 \ubaa8\ub378\uc744 \uc124\uacc4\ud558\uace0 \ud6c8\ub828\ud560 \ub54c \uc885\uc885 \uac04\uacfc\ub418\ub294 \ubd80\ubd84\uc785\ub2c8\ub2e4.\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc740 ML \ubaa8\ub378\ub4e4\uc758 \ubcf4\uc548 \ucde8\uc57d\uc810\uc5d0 \ub300\ud55c \uc778\uc2dd\uc744 \ub192\uc774\uace0, \uc694\uc998 \ud654\ub450\uac00 \ub418\uace0\uc788\ub294 \uc801\ub300\uc801 \uba38\uc2e0 \ub7ec\ub2dd\uc5d0 \ub300\ud55c \ud1b5\ucc30\ub825\uc744 \uc81c\uacf5\ud560 \uac83\uc785\ub2c8\ub2e4.\n\uc774\ubbf8\uc9c0\uc5d0 \ub208\uce58\ucc4c \uc218 \uc5c6\ub294 \uc791\uc740 \ubcc0\ud654(perturbation)\ub97c \ucd94\uac00\ud558\uba74 \ubaa8\ub378 \uc131\ub2a5\uc774 \ud06c\uac8c \ub2ec\ub77c\uc9c8 \uc218 \uc788\ub2e4\ub294 \uc0ac\uc2e4\uc5d0 \ub180\ub784 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uc774\ubc88 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 \uc774\ubbf8\uc9c0 \ubd84\ub958\uae30\uc758 \uc608\uc81c\ub97c \ud1b5\ud574 \uc704 \ub0b4\uc6a9\uc5d0 \ub300\ud574 \uc0b4\ud3b4\ubcfc \uac83\uc785\ub2c8\ub2e4.\n\ud2b9\ud788 \uc6b0\ub9ac\ub294 \uac00\uc7a5 \ub9ce\uc774 \uc0ac\uc6a9\ub418\ub294 \uacf5\uaca9 \ubc29\ubc95 \uc911 \ud558\ub098\uc778 FGSM (Fast Gradient Sign Attack)\uc744 \uc774\uc6a9\ud574 MNIST \ubd84\ub958\uae30\ub97c \uc18d\uc5ec \ubcfc \uac83\uc785\ub2c8\ub2e4.\n\n" + "\n\uc801\ub300\uc801 \uc608\uc81c \uc0dd\uc131(Adversarial Example Generation)\n====================================================\n\n**\uc800\uc790:** `Nathan Inkawhich `__\n**\ubc88\uc5ed:** `BONGMO KIM `__\n\n\uc774 \uae00\uc744 \uc77d\uace0 \uc788\ub2e4\uba74, \uc5ec\ub7ec\ubd84\uc740 \uc774\ubbf8 \uba38\uc2e0\ub7ec\ub2dd \ubaa8\ub378\uc774 \uc5bc\ub9c8\ub098 \ud6a8\uacfc\uc801\uc778\uc9c0 \uadf8 \uc9c4\uac00\ub97c \uc54c\uace0 \uc788\uc744 \uac83\uc785\ub2c8\ub2e4.\n\uba38\uc2e0 \ub7ec\ub2dd \uc5f0\uad6c\ub294 ML(Machine Learning) \ubaa8\ub378\uc744 \ub354\uc6b1 \ube60\ub974\uace0 \uc815\ud655\ud558\uba70 \ud6a8\uc728\uc801\uc774\uac8c \ud558\ub294 \ubc29\ud5a5\uc73c\ub85c \uc9c4\ud589 \ub418\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n\uadf8\ub7ec\ub098 \ubaa8\ub378\uc744 \uc18d\uc774\ub824\ud558\ub294 \uc801\uc5d0 \ub300\ud55c \ubcf4\uc548\uacfc \uacac\uace0\ud568\uc740 \ubaa8\ub378\uc744 \uc124\uacc4\ud558\uace0 \ud6c8\ub828\ud560 \ub54c \uc885\uc885 \uac04\uacfc\ub418\ub294 \ubd80\ubd84\uc785\ub2c8\ub2e4.\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc740 ML \ubaa8\ub378\ub4e4\uc758 \ubcf4\uc548 \ucde8\uc57d\uc810\uc5d0 \ub300\ud55c \uc778\uc2dd\uc744 \ub192\uc774\uace0, \uc694\uc998 \ud654\ub450\uac00 \ub418\uace0\uc788\ub294 \uc801\ub300\uc801 \uba38\uc2e0 \ub7ec\ub2dd\uc5d0 \ub300\ud55c \ud1b5\ucc30\ub825\uc744 \uc81c\uacf5\ud560 \uac83\uc785\ub2c8\ub2e4.\n\uc774\ubbf8\uc9c0\uc5d0 \ub208\uce58\ucc4c \uc218 \uc5c6\ub294 \uc791\uc740 \ubcc0\ud654(perturbation)\ub97c \ucd94\uac00\ud558\uba74 \ubaa8\ub378 \uc131\ub2a5\uc774 \ud06c\uac8c \ub2ec\ub77c\uc9c8 \uc218 \uc788\ub2e4\ub294 \uc0ac\uc2e4\uc5d0 \ub180\ub784 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uc774\ubc88 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 \uc774\ubbf8\uc9c0 \ubd84\ub958\uae30\uc758 \uc608\uc81c\ub97c \ud1b5\ud574 \uc704 \ub0b4\uc6a9\uc5d0 \ub300\ud574 \uc0b4\ud3b4\ubcfc \uac83\uc785\ub2c8\ub2e4.\n\ud2b9\ud788 \uc6b0\ub9ac\ub294 \uac00\uc7a5 \ub9ce\uc774 \uc0ac\uc6a9\ub418\ub294 \uacf5\uaca9 \ubc29\ubc95 \uc911 \ud558\ub098\uc778 FGSM (Fast Gradient Sign Attack)\uc744 \uc774\uc6a9\ud574 MNIST \ubd84\ub958\uae30\ub97c \uc18d\uc5ec \ubcfc \uac83\uc785\ub2c8\ub2e4.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "\uc704\ud611 \ubaa8\ub378\n------------\n\n\uc0c1\ud669\uc5d0 \ub530\ub77c \ub2e4\uc591\ud55c \ubc94\uc8fc\uc758 \uc801\ub300\uc801 \uacf5\uaca9\uc774 \uc788\ub294\ub370 \uac01\uac01 \ubaa9\ud45c\uac00 \ub2e4\ub974\uace0 \uacf5\uaca9\uc790\uac00 \uc54c\uace0 \uc788\ub294 \uc815\ubcf4\n\ub300\ud55c \uac00\uc815\ub3c4 \ub2e4\ub985\ub2c8\ub2e4. \uadf8\ub7ec\ub098 \ubcf4\ud1b5 \uac00\uc7a5 \uc911\uc694\ud55c \ubaa9\ud45c\ub294 \uc785\ub825 \ub370\uc774\ud130\uc5d0 \ucd5c\uc18c\ud55c\uc758 \uc791\uc740 \ubcc0\ud654\ub97c \n\ucd94\uac00\ud558\uc5ec \uc774\uac83\uc774 \uc758\ub3c4\uc801\uc73c\ub85c \uc798\ubabb \ubd84\ub958\ub418\uac8c \ud558\ub294 \uac83\uc785\ub2c8\ub2e4. \uacf5\uaca9\uc790\uac00 \uac00\uc9c0\uace0 \uc788\ub294 \uc815\ubcf4\uc5d0 \ub300\ud55c \n\uac00\uc815\uc5d0\ub294 \uc5ec\ub7ec \uc885\ub958\uac00 \uc788\ub294\ub370, \ubcf4\ud1b5 **\ud654\uc774\ud2b8\ubc15\uc2a4** \uc640 **\ube14\ub799\ubc15\uc2a4** \ub450 \uac00\uc9c0\uac00 \uc788\uc2b5\ub2c8\ub2e4. \n*\ud654\uc774\ud2b8\ubc15\uc2a4* \uacf5\uaca9\uc740 \uacf5\uaca9\uc790\uac00 \ubaa8\ub378\uc5d0 \ub300\ud574 \uc544\ud0a4\ud14d\ucc98, \uc785\ub825, \ucd9c\ub825, \uac00\uc911\uce58\ub97c \ud3ec\ud568\ud55c \ubaa8\ub4e0 \uac83\uc744\n\uc54c\uace0 \uc788\uace0 \uc811\uadfc\ud560 \uc218 \uc788\ub2e4\uace0 \uac00\uc815\ud569\ub2c8\ub2e4. *\ube14\ub799\ubc15\uc2a4* \uacf5\uaca9\uc740 \uacf5\uaca9\uc790\uac00 \ubaa8\ub378\uc758 \uc785\ub825\uacfc \ucd9c\ub825\uc5d0 \n\ub300\ud574\uc11c\ub9cc \uc811\uadfc \uac00\ub2a5\ud558\uace0 \ubaa8\ub378\uc758 \uac00\uc911\uce58\uc640 \uc544\ud0a4\ud14d\ucc98\uc5d0 \uad00\ud55c \ub0b4\uc6a9\uc740 \ubaa8\ub978\ub2e4\uace0 \uac00\uc815\ud569\ub2c8\ub2e4. \n\uacf5\uaca9\uc790\uc758 \ubaa9\ud45c\ub294 \uc624\ubd84\ub958 \ubc0f **\uc18c\uc2a4/\ud0c0\uac9f \uc624\ubd84\ub958** \ub97c \ud3ec\ud568\ud558\ub294 \uc5ec\ub7ec \uc720\ud615\uc774 \uc788\uc2b5\ub2c8\ub2e4. \n*\uc624\ubd84\ub958* \uc758 \ubaa9\ud45c\ub294 \uacf5\uaca9\uc790\uac00 \ucd9c\ub825\uc73c\ub85c \ub098\uc628 \ubd84\ub958 \uacb0\uacfc\uac00 \uc798\ubabb \ub418\ub3c4\ub85d \ud558\ub098 \uc0c8\ub85c\uc6b4 \ubd84\ub958 \uacb0\uacfc\uac00\n\uc5b4\ub5a4 \uac83\uc774 \ub098\uc624\ub294\uc9c0 \uc2e0\uacbd \uc4f0\uc9c0 \uc54a\ub294 \uac83\uc744 \uc758\ubbf8\ud569\ub2c8\ub2e4. *\uc18c\uc2a4/\ud0c0\uac9f \uc624\ubd84\ub958* \ub294 \uacf5\uaca9\uc790\uac00 \n\uc6d0\ub798 \ud2b9\uc815 \uc18c\uc2a4 \ud074\ub798\uc2a4\uc758 \uc774\ubbf8\uc9c0\ub97c \ub2e4\ub978 \ud2b9\uc815 \ub300\uc0c1 \ud074\ub798\uc2a4\ub85c \ubd84\ub958\ud558\ub3c4\ub85d \ubcc0\uacbd\ud558\ub824\uace0 \ud568\uc744 \uc758\ubbf8\ud569\ub2c8\ub2e4.\n\n\n\uc774 \uacbd\uc6b0 FGSM \uacf5\uaca9\uc740 *\uc624\ubd84\ub958* \ub97c \ubaa9\ud45c\ub85c \ud558\ub294 \ud654\uc774\ud2b8 \ubc15\uc2a4 \uacf5\uaca9\uc785\ub2c8\ub2e4. \n\uc774\ub7f0 \ubc30\uacbd \uc815\ubcf4\ub97c \uac16\uace0 \uacf5\uaca9\uc5d0 \ub300\ud574 \uc790\uc138\ud788 \uc54c\uc544 \ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\ube60\ub978 \ubcc0\ud654\ub3c4 \ubd80\ud638 \uacf5\uaca9\n-------------------------\n\n\uacf5\uaca9 \ubc29\ubc95\uc5d0 \uc788\uc5b4 \ucd08\uae30 \ubc29\uc2dd\uc774\uba74\uc11c \uac00\uc7a5 \uc720\uba85\ud55c \ubc29\uc2dd\uc740 *\ube60\ub978 \ubcc0\ud654\ub3c4 \ubd80\ud638 \uacf5\uaca9 (FGSM)* \uc774\ub77c\uace0 \ud558\uba70 \n`\uc801\ub300\uc801 \uc608\uc81c\uc5d0 \ub300\ud55c \uc124\uba85\uacfc \ud65c\uc6a9 `__ \uc5d0\uc11c \n\uc774\uc548 \uac13\ud3a0\ub85c\uc6b0\uac00 \uae30\uace0\ud558\uc600\uc2b5\ub2c8\ub2e4.\n\uc774 \uacf5\uaca9\ubc95\uc740 \ub180\ub78d\ub3c4\ub85d \uac15\ub825\ud558\uc9c0\ub9cc \uc9c1\uad00\uc801\uc785\ub2c8\ub2e4. \ud559\uc2b5 \ubc29\uc2dd, *\ubcc0\ud654\ub3c4(gradients)* \ub97c \ud65c\uc6a9\ud558\uc5ec \uc2e0\uacbd\ub9dd\uc744 \uacf5\uaca9\ud558\ub3c4\ub85d \n\uc124\uacc4 \ub418\uc5c8\uc2b5\ub2c8\ub2e4. \uc544\uc774\ub514\uc5b4\ub294 \uac04\ub2e8\ud569\ub2c8\ub2e4. \uc5ed\uc804\ud30c \ubcc0\ud654\ub3c4\ub97c \uae30\ubc18\uc73c\ub85c \uac00\uc911\uce58\ub97c \uc870\uc815\ud558\uc5ec \uc190\uc2e4\uc744 \ucd5c\uc18c\ud654\ud558\uae30\ubcf4\ub2e4\ub294\n\uacf5\uaca9\uc774 \ub3d9\uc77c\ud55c \uc5ed\uc804\ud30c \ubcc0\ud654\ub3c4\ub97c \uae30\ubc18\uc73c\ub85c *\uc190\uc2e4\uc744 \ucd5c\ub300\ud654\ud558\ud558\ub294 \ubc29\ud5a5\uc73c\ub85c \uc785\ub825 \ub370\uc774\ud130\ub97c \uc870\uc815* \ud569\ub2c8\ub2e4. \n\ub2e4\uc2dc \ub9d0\ud574 \uacf5\uaca9\uc740 \uc785\ub825 \ub370\uc774\ud130\uc5d0\uc11c \uacc4\uc0b0\ub41c \uc190\uc2e4 \ubcc0\ud654\ub3c4\ub97c \uc0ac\uc6a9\ud558\uace0 \uc785\ub825 \ub370\uc774\ud130\ub97c \uc870\uc815\ud558\uc5ec \uc190\uc2e4\uc774 \ucd5c\ub300\uac00 \ub418\uac8c \ud569\ub2c8\ub2e4.\n\n" + "\uc704\ud611 \ubaa8\ub378\n------------\n\n\uc0c1\ud669\uc5d0 \ub530\ub77c \ub2e4\uc591\ud55c \ubc94\uc8fc\uc758 \uc801\ub300\uc801 \uacf5\uaca9\uc774 \uc788\ub294\ub370 \uac01\uac01 \ubaa9\ud45c\uac00 \ub2e4\ub974\uace0 \uacf5\uaca9\uc790\uac00 \uc54c\uace0 \uc788\ub294 \uc815\ubcf4\n\ub300\ud55c \uac00\uc815\ub3c4 \ub2e4\ub985\ub2c8\ub2e4. \uadf8\ub7ec\ub098 \ubcf4\ud1b5 \uac00\uc7a5 \uc911\uc694\ud55c \ubaa9\ud45c\ub294 \uc785\ub825 \ub370\uc774\ud130\uc5d0 \ucd5c\uc18c\ud55c\uc758 \uc791\uc740 \ubcc0\ud654\ub97c\n\ucd94\uac00\ud558\uc5ec \uc774\uac83\uc774 \uc758\ub3c4\uc801\uc73c\ub85c \uc798\ubabb \ubd84\ub958\ub418\uac8c \ud558\ub294 \uac83\uc785\ub2c8\ub2e4. \uacf5\uaca9\uc790\uac00 \uac00\uc9c0\uace0 \uc788\ub294 \uc815\ubcf4\uc5d0 \ub300\ud55c\n\uac00\uc815\uc5d0\ub294 \uc5ec\ub7ec \uc885\ub958\uac00 \uc788\ub294\ub370, \ubcf4\ud1b5 **\ud654\uc774\ud2b8\ubc15\uc2a4** \uc640 **\ube14\ub799\ubc15\uc2a4** \ub450 \uac00\uc9c0\uac00 \uc788\uc2b5\ub2c8\ub2e4.\n*\ud654\uc774\ud2b8\ubc15\uc2a4* \uacf5\uaca9\uc740 \uacf5\uaca9\uc790\uac00 \ubaa8\ub378\uc5d0 \ub300\ud574 \uc544\ud0a4\ud14d\ucc98, \uc785\ub825, \ucd9c\ub825, \uac00\uc911\uce58\ub97c \ud3ec\ud568\ud55c \ubaa8\ub4e0 \uac83\uc744\n\uc54c\uace0 \uc788\uace0 \uc811\uadfc\ud560 \uc218 \uc788\ub2e4\uace0 \uac00\uc815\ud569\ub2c8\ub2e4. *\ube14\ub799\ubc15\uc2a4* \uacf5\uaca9\uc740 \uacf5\uaca9\uc790\uac00 \ubaa8\ub378\uc758 \uc785\ub825\uacfc \ucd9c\ub825\uc5d0\n\ub300\ud574\uc11c\ub9cc \uc811\uadfc \uac00\ub2a5\ud558\uace0 \ubaa8\ub378\uc758 \uac00\uc911\uce58\uc640 \uc544\ud0a4\ud14d\ucc98\uc5d0 \uad00\ud55c \ub0b4\uc6a9\uc740 \ubaa8\ub978\ub2e4\uace0 \uac00\uc815\ud569\ub2c8\ub2e4.\n\uacf5\uaca9\uc790\uc758 \ubaa9\ud45c\ub294 \uc624\ubd84\ub958 \ubc0f **\uc18c\uc2a4/\ud0c0\uac9f \uc624\ubd84\ub958** \ub97c \ud3ec\ud568\ud558\ub294 \uc5ec\ub7ec \uc720\ud615\uc774 \uc788\uc2b5\ub2c8\ub2e4.\n*\uc624\ubd84\ub958* \uc758 \ubaa9\ud45c\ub294 \uacf5\uaca9\uc790\uac00 \ucd9c\ub825\uc73c\ub85c \ub098\uc628 \ubd84\ub958 \uacb0\uacfc\uac00 \uc798\ubabb \ub418\ub3c4\ub85d \ud558\ub098 \uc0c8\ub85c\uc6b4 \ubd84\ub958 \uacb0\uacfc\uac00\n\uc5b4\ub5a4 \uac83\uc774 \ub098\uc624\ub294\uc9c0 \uc2e0\uacbd \uc4f0\uc9c0 \uc54a\ub294 \uac83\uc744 \uc758\ubbf8\ud569\ub2c8\ub2e4. *\uc18c\uc2a4/\ud0c0\uac9f \uc624\ubd84\ub958* \ub294 \uacf5\uaca9\uc790\uac00\n\uc6d0\ub798 \ud2b9\uc815 \uc18c\uc2a4 \ud074\ub798\uc2a4\uc758 \uc774\ubbf8\uc9c0\ub97c \ub2e4\ub978 \ud2b9\uc815 \ub300\uc0c1 \ud074\ub798\uc2a4\ub85c \ubd84\ub958\ud558\ub3c4\ub85d \ubcc0\uacbd\ud558\ub824\uace0 \ud568\uc744 \uc758\ubbf8\ud569\ub2c8\ub2e4.\n\n\n\uc774 \uacbd\uc6b0 FGSM \uacf5\uaca9\uc740 *\uc624\ubd84\ub958* \ub97c \ubaa9\ud45c\ub85c \ud558\ub294 \ud654\uc774\ud2b8 \ubc15\uc2a4 \uacf5\uaca9\uc785\ub2c8\ub2e4.\n\uc774\ub7f0 \ubc30\uacbd \uc815\ubcf4\ub97c \uac16\uace0 \uacf5\uaca9\uc5d0 \ub300\ud574 \uc790\uc138\ud788 \uc54c\uc544 \ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\ube60\ub978 \ubcc0\ud654\ub3c4 \ubd80\ud638 \uacf5\uaca9\n-------------------------\n\n\uacf5\uaca9 \ubc29\ubc95\uc5d0 \uc788\uc5b4 \ucd08\uae30 \ubc29\uc2dd\uc774\uba74\uc11c \uac00\uc7a5 \uc720\uba85\ud55c \ubc29\uc2dd\uc740 *\ube60\ub978 \ubcc0\ud654\ub3c4 \ubd80\ud638 \uacf5\uaca9 (FGSM)* \uc774\ub77c\uace0 \ud558\uba70\n`\uc801\ub300\uc801 \uc608\uc81c\uc5d0 \ub300\ud55c \uc124\uba85\uacfc \ud65c\uc6a9 `__ \uc5d0\uc11c\n\uc774\uc548 \uac13\ud3a0\ub85c\uc6b0\uac00 \uae30\uace0\ud558\uc600\uc2b5\ub2c8\ub2e4.\n\uc774 \uacf5\uaca9\ubc95\uc740 \ub180\ub78d\ub3c4\ub85d \uac15\ub825\ud558\uc9c0\ub9cc \uc9c1\uad00\uc801\uc785\ub2c8\ub2e4. \ud559\uc2b5 \ubc29\uc2dd, *\ubcc0\ud654\ub3c4(gradients)* \ub97c \ud65c\uc6a9\ud558\uc5ec \uc2e0\uacbd\ub9dd\uc744 \uacf5\uaca9\ud558\ub3c4\ub85d\n\uc124\uacc4 \ub418\uc5c8\uc2b5\ub2c8\ub2e4. \uc544\uc774\ub514\uc5b4\ub294 \uac04\ub2e8\ud569\ub2c8\ub2e4. \uc5ed\uc804\ud30c \ubcc0\ud654\ub3c4\ub97c \uae30\ubc18\uc73c\ub85c \uac00\uc911\uce58\ub97c \uc870\uc815\ud558\uc5ec \uc190\uc2e4\uc744 \ucd5c\uc18c\ud654\ud558\uae30\ubcf4\ub2e4\ub294\n\uacf5\uaca9\uc774 \ub3d9\uc77c\ud55c \uc5ed\uc804\ud30c \ubcc0\ud654\ub3c4\ub97c \uae30\ubc18\uc73c\ub85c *\uc190\uc2e4\uc744 \ucd5c\ub300\ud654\ud558\ud558\ub294 \ubc29\ud5a5\uc73c\ub85c \uc785\ub825 \ub370\uc774\ud130\ub97c \uc870\uc815* \ud569\ub2c8\ub2e4.\n\ub2e4\uc2dc \ub9d0\ud574 \uacf5\uaca9\uc740 \uc785\ub825 \ub370\uc774\ud130\uc5d0\uc11c \uacc4\uc0b0\ub41c \uc190\uc2e4 \ubcc0\ud654\ub3c4\ub97c \uc0ac\uc6a9\ud558\uace0 \uc785\ub825 \ub370\uc774\ud130\ub97c \uc870\uc815\ud558\uc5ec \uc190\uc2e4\uc774 \ucd5c\ub300\uac00 \ub418\uac8c \ud569\ub2c8\ub2e4.\n\n" ] }, { @@ -33,14 +33,14 @@ }, "outputs": [], "source": [ - "# \n# \ucf54\ub4dc\ub85c \ub118\uc5b4\uac00\uae30 \uc804\uc5d0 \uc720\uba85\ud55c `FGSM `__ \ud310\ub2e4 \uc608\uc81c\ub97c \n# \ubcf4\uace0 \uba87 \uac00\uc9c0 \ud45c\uae30\ubc95\uc744 \uc815\ub9ac\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n#\n# .. \uadf8\ub9bc:: /_static/img/fgsm_panda_image.png\n# :alt: fgsm_panda_image\n#\n# \uadf8\ub9bc\uc73c\ub85c\ubd80\ud130, :math:`\\mathbf{x}` \ub294 \uc6d0\ubcf8 \uc785\ub825 \uc774\ubbf8\uc9c0\uac00 \"\ud310\ub2e4\" \ub85c \uc62c\ubc14\ub974\uac8c \ubd84\ub958\ub41c \uac83\uc744 \uc758\ubbf8\ud558\uace0,\n# :math:`y` \ub294 :math:`\\mathbf{x}` \ub97c \uc704\ud55c \uc815\ub2f5 \ub77c\ubca8\uc774\uba70, :math:`\\mathbf{\\theta}` \ub294 \ubaa8\ub378\uc758\n# \ud30c\ub77c\ubbf8\ud130\ub97c, :math:`J(\\mathbf{\\theta}, \\mathbf{x}, y)` \ub294 \ub124\ud2b8\uc6cc\ud06c\uc758 \ud559\uc2b5\uc744 \uc704\ud574\uc11c \uc0ac\uc6a9\ub418\ub294 \uc190\uc2e4\uc744 \ub098\ud0c0\ub0c5\ub2c8\ub2e4.\n# \uacf5\uaca9\uc740 :math:`\\nabla_{x} J(\\mathbf{\\theta}, \\mathbf{x}, y)` \uacc4\uc0b0\uc744 \uc704\ud574 \uc785\ub825 \ub370\uc774\ud130\uc5d0 \ubcc0\ud654\ub3c4\ub97c \uc5ed\uc804\ud30c\ud569\ub2c8\ub2e4.\n# \uadf8\ub7ec\uace0 \ub098\uc11c, \ubcc0\ud654\ub3c4\ub294 \uc190\uc2e4 \uac12\uc774 \ucd5c\ub300\ud654\ub418\ub294 \ubc29\ud5a5\uc73c\ub85c (\uc608\ub97c \ub4e4\uba74, :math:`sign(\\nabla_{x} J(\\mathbf{\\theta}, \\mathbf{x}, y))` ) \n# \uc791\uc740 \uc2a4\ud15d(step) \ub9cc\ud07c (\uadf8\ub9bc\uc5d0\uc11c\ub294 :math:`\\epsilon` \ud639\uc740 :math:`0.007`) \uc785\ub825 \ub370\uc774\ud130\uc5d0 \uc801\uc6a9\ub429\ub2c8\ub2e4.\n# \uacb0\uacfc\ub85c \ub098\uc624\ub294 \uc791\uc740 \ubcc0\ud654\ub41c \uc774\ubbf8\uc9c0( :math:`x'` )\ub294 \ud0c0\uac9f \ub124\ud2b8\uc6cc\ud06c\uc5d0 \uc758\ud574 \"\uae34\ud314\uc6d0\uc22d\uc774\"\ub85c *\uc624\ubd84\ub958* \ub418\ub098 \uc5ec\uc804\ud788 \uc721\uc548\uc73c\ub85c\ub294\n# \ubd84\uba85\ud788 \"\ud310\ub2e4\" \uc785\ub2c8\ub2e4.\n# \n# \uc774\uc81c \ubcf8 \ud29c\ud1a0\ub9ac\uc5bc\uc758 \ub3d9\uae30\uac00 \uba85\ud655\ud574\uc9c0\uae38 \ubc14\ub77c\uba70, \uad6c\ud604\uc73c\ub85c \ub118\uc5b4\uac00 \ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n# \n\nfrom __future__ import print_function\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.optim as optim\nfrom torchvision import datasets, transforms\nimport numpy as np\nimport matplotlib.pyplot as plt" + "#\n# \ucf54\ub4dc\ub85c \ub118\uc5b4\uac00\uae30 \uc804\uc5d0 \uc720\uba85\ud55c `FGSM `__ \ud310\ub2e4 \uc608\uc81c\ub97c\n# \ubcf4\uace0 \uba87 \uac00\uc9c0 \ud45c\uae30\ubc95\uc744 \uc815\ub9ac\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n#\n# .. \uadf8\ub9bc:: /_static/img/fgsm_panda_image.png\n# :alt: fgsm_panda_image\n#\n# \uadf8\ub9bc\uc73c\ub85c\ubd80\ud130, :math:`\\mathbf{x}` \ub294 \uc6d0\ubcf8 \uc785\ub825 \uc774\ubbf8\uc9c0\uac00 \"\ud310\ub2e4\" \ub85c \uc62c\ubc14\ub974\uac8c \ubd84\ub958\ub41c \uac83\uc744 \uc758\ubbf8\ud558\uace0,\n# :math:`y` \ub294 :math:`\\mathbf{x}` \ub97c \uc704\ud55c \uc815\ub2f5 \ub77c\ubca8\uc774\uba70, :math:`\\mathbf{\\theta}` \ub294 \ubaa8\ub378\uc758\n# \ud30c\ub77c\ubbf8\ud130\ub97c, :math:`J(\\mathbf{\\theta}, \\mathbf{x}, y)` \ub294 \ub124\ud2b8\uc6cc\ud06c\uc758 \ud559\uc2b5\uc744 \uc704\ud574\uc11c \uc0ac\uc6a9\ub418\ub294 \uc190\uc2e4\uc744 \ub098\ud0c0\ub0c5\ub2c8\ub2e4.\n# \uacf5\uaca9\uc740 :math:`\\nabla_{x} J(\\mathbf{\\theta}, \\mathbf{x}, y)` \uacc4\uc0b0\uc744 \uc704\ud574 \uc785\ub825 \ub370\uc774\ud130\uc5d0 \ubcc0\ud654\ub3c4\ub97c \uc5ed\uc804\ud30c\ud569\ub2c8\ub2e4.\n# \uadf8\ub7ec\uace0 \ub098\uc11c, \ubcc0\ud654\ub3c4\ub294 \uc190\uc2e4 \uac12\uc774 \ucd5c\ub300\ud654\ub418\ub294 \ubc29\ud5a5\uc73c\ub85c (\uc608\ub97c \ub4e4\uba74, :math:`sign(\\nabla_{x} J(\\mathbf{\\theta}, \\mathbf{x}, y))` )\n# \uc791\uc740 \uc2a4\ud15d(step) \ub9cc\ud07c (\uadf8\ub9bc\uc5d0\uc11c\ub294 :math:`\\epsilon` \ud639\uc740 :math:`0.007`) \uc785\ub825 \ub370\uc774\ud130\uc5d0 \uc801\uc6a9\ub429\ub2c8\ub2e4.\n# \uacb0\uacfc\ub85c \ub098\uc624\ub294 \uc791\uc740 \ubcc0\ud654\ub41c \uc774\ubbf8\uc9c0( :math:`x'` )\ub294 \ud0c0\uac9f \ub124\ud2b8\uc6cc\ud06c\uc5d0 \uc758\ud574 \"\uae34\ud314\uc6d0\uc22d\uc774\"\ub85c *\uc624\ubd84\ub958* \ub418\ub098 \uc5ec\uc804\ud788 \uc721\uc548\uc73c\ub85c\ub294\n# \ubd84\uba85\ud788 \"\ud310\ub2e4\" \uc785\ub2c8\ub2e4.\n#\n# \uc774\uc81c \ubcf8 \ud29c\ud1a0\ub9ac\uc5bc\uc758 \ub3d9\uae30\uac00 \uba85\ud655\ud574\uc9c0\uae38 \ubc14\ub77c\uba70, \uad6c\ud604\uc73c\ub85c \ub118\uc5b4\uac00 \ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n#\n\nfrom __future__ import print_function\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.optim as optim\nfrom torchvision import datasets, transforms\nimport numpy as np\nimport matplotlib.pyplot as plt" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "\uad6c\ud604\n--------------\n\n\uc774 \uc139\uc158\uc5d0\uc11c\ub294 \ud29c\ud1a0\ub9ac\uc5bc\uc758 \uc785\ub825 \ub9e4\uac1c \ubcc0\uc218\uc5d0 \ub300\ud574 \uc124\uba85\ud558\uace0 \uacf5\uaca9\uc911\uc778 \ubaa8\ub378\uc744\n\uc815\uc758\ud55c \ub2e4\uc74c \uacf5\uaca9\uc744 \ucf54\ub529\ud558\uace0 \uc77c\ubd80 \ud14c\uc2a4\ud2b8\ub97c \uc2e4\ud589\ud569\ub2c8\ub2e4.\n\n\uc785\ub825\n~~~~~~\n\n\uc774 \ud559\uc2b5\uc11c\uc5d0\ub294 \uc785\ub825\uc774 3 \uac1c\uc774\uba70 \ub2e4\uc74c\uacfc \uac19\uc774 \uc815\uc758\ub429\ub2c8\ub2e4:\n\n- **epsilons** - \uc2e4\ud589\uc5d0 \uc0ac\uc6a9\ud560 \uc5e1\uc2e4\ub860\uc758 \ub9ac\uc2a4\ud2b8\uc785\ub2c8\ub2e4. \uc5e1\uc2e4\ub860 0\uc758 \uac12\uc740 \uc6d0\ub798 \ud14c\uc2a4\ud2b8 \uc14b\uc758 \ubaa8\ub378 \uc131\ub2a5\uc744\n \ub098\ud0c0\ub0b4\ubbc0\ub85c \ubaa9\ub85d\uc5d0 \uc720\uc9c0\ud558\ub294 \uac83\uc774 \uc911\uc694\ud569\ub2c8\ub2e4. \ub610\ud55c \uc9c1\uad00\uc801\uc73c\ub85c \uc5e1\uc2e4\ub860\uc774 \ud074\uc218\ub85d \uc791\uc740 \ubcc0\ud654\uac00 \ub354 \ub208\uc5d0 \ub744\uc9c0\ub9cc\n \ubaa8\ub378 \uc815\ud655\ub3c4\ub97c \uc800\ud558 \uc2dc\ud0a4\ub294 \uce21\uba74\uc5d0\uc11c \ub354 \ud6a8\uacfc\uac00 \uc788\uc2b5\ub2c8\ub2e4. \uc5ec\uae30\uc11c \ub370\uc774\ud130\uc758 \ubc94\uc704\ub294 0-1 \uc774\uae30 \ub54c\ubb38\uc5d0 \n \uc5e1\uc2e4\ub860\uc758 \uac12\uc740 1\uc744 \ucd08\uacfc\ud560 \uc218 \uc5c6\uc2b5\ub2c8\ub2e4.\n\n- **pretrained_model** - `pytorch/examples/mnist `__ \n \ub97c \ud1b5\ud574 \ubbf8\ub9ac \ud559\uc2b5\ub41c MNIST \ubaa8\ub378\uc758 \uacbd\ub85c. \n \ud29c\ud1a0\ub9ac\uc5bc\uc744 \uac04\ud3b8\ud558\uac8c \ud558\ub824\uba74 `\uc5ec\uae30 `__ \uc5d0\uc11c \ubbf8\ub9ac \ud559\uc2b5\ub41c \ubaa8\ub378\uc744 \ub2e4\uc6b4\ub85c\ub4dc\ud558\uc138\uc694.\n\n- **use_cuda** - CUDA \ub97c \uc0ac\uc6a9\ud560\uc9c0 \ub9d0\uc9c0 \uc815\ud558\ub294 \uc774\uc9c4 \ud50c\ub798\uadf8. \n \ubcf8 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 CPU \uc2dc\uac04\uc774 \uc624\ub798 \uac78\ub9ac\uc9c0 \uc54a\uc73c\ubbc0\ub85c CUDA\ub97c \uc9c0\uc6d0\ud558\ub294 GPU \uc758 \uc5ec\ubd80\ub294 \uc911\uc694\ud558\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4.\n\n\n" + "\uad6c\ud604\n--------------\n\n\uc774 \uc139\uc158\uc5d0\uc11c\ub294 \ud29c\ud1a0\ub9ac\uc5bc\uc758 \uc785\ub825 \ub9e4\uac1c \ubcc0\uc218\uc5d0 \ub300\ud574 \uc124\uba85\ud558\uace0 \uacf5\uaca9\uc911\uc778 \ubaa8\ub378\uc744\n\uc815\uc758\ud55c \ub2e4\uc74c \uacf5\uaca9\uc744 \ucf54\ub529\ud558\uace0 \uc77c\ubd80 \ud14c\uc2a4\ud2b8\ub97c \uc2e4\ud589\ud569\ub2c8\ub2e4.\n\n\uc785\ub825\n~~~~~~\n\n\uc774 \ud559\uc2b5\uc11c\uc5d0\ub294 \uc785\ub825\uc774 3 \uac1c\uc774\uba70 \ub2e4\uc74c\uacfc \uac19\uc774 \uc815\uc758\ub429\ub2c8\ub2e4:\n\n- **epsilons** - \uc2e4\ud589\uc5d0 \uc0ac\uc6a9\ud560 \uc5e1\uc2e4\ub860\uc758 \ub9ac\uc2a4\ud2b8\uc785\ub2c8\ub2e4. \uc5e1\uc2e4\ub860 0\uc758 \uac12\uc740 \uc6d0\ub798 \ud14c\uc2a4\ud2b8 \uc14b\uc758 \ubaa8\ub378 \uc131\ub2a5\uc744\n \ub098\ud0c0\ub0b4\ubbc0\ub85c \ubaa9\ub85d\uc5d0 \uc720\uc9c0\ud558\ub294 \uac83\uc774 \uc911\uc694\ud569\ub2c8\ub2e4. \ub610\ud55c \uc9c1\uad00\uc801\uc73c\ub85c \uc5e1\uc2e4\ub860\uc774 \ud074\uc218\ub85d \uc791\uc740 \ubcc0\ud654\uac00 \ub354 \ub208\uc5d0 \ub744\uc9c0\ub9cc\n \ubaa8\ub378 \uc815\ud655\ub3c4\ub97c \uc800\ud558 \uc2dc\ud0a4\ub294 \uce21\uba74\uc5d0\uc11c \ub354 \ud6a8\uacfc\uac00 \uc788\uc2b5\ub2c8\ub2e4. \uc5ec\uae30\uc11c \ub370\uc774\ud130\uc758 \ubc94\uc704\ub294 0-1 \uc774\uae30 \ub54c\ubb38\uc5d0\n \uc5e1\uc2e4\ub860\uc758 \uac12\uc740 1\uc744 \ucd08\uacfc\ud560 \uc218 \uc5c6\uc2b5\ub2c8\ub2e4.\n\n- **pretrained_model** - `pytorch/examples/mnist `__\n \ub97c \ud1b5\ud574 \ubbf8\ub9ac \ud559\uc2b5\ub41c MNIST \ubaa8\ub378\uc758 \uacbd\ub85c.\n \ud29c\ud1a0\ub9ac\uc5bc\uc744 \uac04\ud3b8\ud558\uac8c \ud558\ub824\uba74 `\uc5ec\uae30 `__ \uc5d0\uc11c \ubbf8\ub9ac \ud559\uc2b5\ub41c \ubaa8\ub378\uc744 \ub2e4\uc6b4\ub85c\ub4dc\ud558\uc138\uc694.\n\n- **use_cuda** - CUDA \ub97c \uc0ac\uc6a9\ud560\uc9c0 \ub9d0\uc9c0 \uc815\ud558\ub294 \uc774\uc9c4 \ud50c\ub798\uadf8.\n \ubcf8 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 CPU \uc2dc\uac04\uc774 \uc624\ub798 \uac78\ub9ac\uc9c0 \uc54a\uc73c\ubbc0\ub85c CUDA\ub97c \uc9c0\uc6d0\ud558\ub294 GPU \uc758 \uc5ec\ubd80\ub294 \uc911\uc694\ud558\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4.\n\n\n" ] }, { @@ -58,7 +58,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uacf5\uaca9\uc744 \ubc1b\ub294 \ubaa8\ub378\n~~~~~~~~~~~~~~~~~~\n\n\uc55e\uc11c \ub9d0\ud55c\ub300\ub85c, \uacf5\uaca9\uc744 \ubc1b\ub294 \ubaa8\ub378\uc740 `pytorch/examples/mnist `__\n\uc640 \ub3d9\uc77c\ud55c MNIST \ubaa8\ub378\uc785\ub2c8\ub2e4. \ubcf8\uc778\uc758 MNIST \ubaa8\ub378\uc744 \ud559\uc2b5 \ubc0f \uc800\uc7a5\ud558\ub294 \ubc29\uc2dd\uc73c\ub85c \ud558\uac70\ub098 \uc81c\uacf5\ub41c \ubaa8\ub378\uc744 \ub2e4\uc6b4\ub85c\ub4dc \ud574 \uc0ac\uc6a9\ud558\ub294 \uc2dd\uc73c\ub85c \uc9c4\ud589\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uc5ec\uae30\uc11c *Net* \uc815\uc758 \ubc0f \ud14c\uc2a4\ud2b8 \ub370\uc774\ud130 \ub85c\ub354\ub294 MNIST \uc608\uc81c\uc5d0\uc11c \ubcf5\uc0ac \ud558\uc600\uc2b5\ub2c8\ub2e4. \n\uc774 \uc139\uc158\uc758 \ubaa9\uc801\uc740 \ubaa8\ub378\uacfc \ub370\uc774\ud130 \ub85c\ub354\ub97c \uc815\uc758\ud55c \ub2e4\uc74c, \ubaa8\ub378\uc744 \ucd08\uae30\ud654\ud558\uace0 \ubbf8\ub9ac \ud559\uc2b5\ub41c \uac00\uc911\uce58\ub97c \uc77d\uc5b4\uc624\ub294 \uac83\uc785\ub2c8\ub2e4.\n\n\n" + "\uacf5\uaca9\uc744 \ubc1b\ub294 \ubaa8\ub378\n~~~~~~~~~~~~~~~~~~\n\n\uc55e\uc11c \ub9d0\ud55c\ub300\ub85c, \uacf5\uaca9\uc744 \ubc1b\ub294 \ubaa8\ub378\uc740 `pytorch/examples/mnist `__\n\uc640 \ub3d9\uc77c\ud55c MNIST \ubaa8\ub378\uc785\ub2c8\ub2e4. \ubcf8\uc778\uc758 MNIST \ubaa8\ub378\uc744 \ud559\uc2b5 \ubc0f \uc800\uc7a5\ud558\ub294 \ubc29\uc2dd\uc73c\ub85c \ud558\uac70\ub098 \uc81c\uacf5\ub41c \ubaa8\ub378\uc744 \ub2e4\uc6b4\ub85c\ub4dc \ud574 \uc0ac\uc6a9\ud558\ub294 \uc2dd\uc73c\ub85c \uc9c4\ud589\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uc5ec\uae30\uc11c *Net* \uc815\uc758 \ubc0f \ud14c\uc2a4\ud2b8 \ub370\uc774\ud130 \ub85c\ub354\ub294 MNIST \uc608\uc81c\uc5d0\uc11c \ubcf5\uc0ac \ud558\uc600\uc2b5\ub2c8\ub2e4.\n\uc774 \uc139\uc158\uc758 \ubaa9\uc801\uc740 \ubaa8\ub378\uacfc \ub370\uc774\ud130 \ub85c\ub354\ub97c \uc815\uc758\ud55c \ub2e4\uc74c, \ubaa8\ub378\uc744 \ucd08\uae30\ud654\ud558\uace0 \ubbf8\ub9ac \ud559\uc2b5\ub41c \uac00\uc911\uce58\ub97c \uc77d\uc5b4\uc624\ub294 \uac83\uc785\ub2c8\ub2e4.\n\n\n" ] }, { @@ -69,14 +69,14 @@ }, "outputs": [], "source": [ - "# LeNet \ubaa8\ub378 \uc815\uc758\nclass Net(nn.Module):\n def __init__(self):\n super(Net, self).__init__()\n self.conv1 = nn.Conv2d(1, 10, kernel_size=5)\n self.conv2 = nn.Conv2d(10, 20, kernel_size=5)\n self.conv2_drop = nn.Dropout2d()\n self.fc1 = nn.Linear(320, 50)\n self.fc2 = nn.Linear(50, 10)\n\n def forward(self, x):\n x = F.relu(F.max_pool2d(self.conv1(x), 2))\n x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))\n x = x.view(-1, 320)\n x = F.relu(self.fc1(x))\n x = F.dropout(x, training=self.training)\n x = self.fc2(x)\n return F.log_softmax(x, dim=1)\n\n# MNIST \ud14c\uc2a4\ud2b8 \ub370\uc774\ud130\uc14b\uacfc \ub370\uc774\ud130\ub85c\ub354 \uc120\uc5b8\ntest_loader = torch.utils.data.DataLoader(\n datasets.MNIST('../data', train=False, download=True, transform=transforms.Compose([\n transforms.ToTensor(),\n ])), \n batch_size=1, shuffle=True)\n\n# \uc5b4\ub5a4 \ub514\ubc14\uc774\uc2a4\ub97c \uc0ac\uc6a9\ud560\uc9c0 \uc815\uc758\nprint(\"CUDA Available: \",torch.cuda.is_available())\ndevice = torch.device(\"cuda\" if (use_cuda and torch.cuda.is_available()) else \"cpu\")\n\n# \ubaa8\ub378 \ucd08\uae30\ud654\ud558\uae30\nmodel = Net().to(device)\n\n# \ubbf8\ub9ac \ud559\uc2b5\ub41c \ubaa8\ub378 \uc77d\uc5b4\uc624\uae30\nmodel.load_state_dict(torch.load(pretrained_model, map_location='cpu'))\n\n# \ubaa8\ub378\uc744 \ud3c9\uac00 \ubaa8\ub4dc\ub85c \uc124\uc815\ud558\uae30. \ub4dc\ub86d\uc544\uc6c3 \ub808\uc774\uc5b4\ub4e4\uc744 \uc704\ud574 \uc0ac\uc6a9\ub428\nmodel.eval()" + "# LeNet \ubaa8\ub378 \uc815\uc758\nclass Net(nn.Module):\n def __init__(self):\n super(Net, self).__init__()\n self.conv1 = nn.Conv2d(1, 10, kernel_size=5)\n self.conv2 = nn.Conv2d(10, 20, kernel_size=5)\n self.conv2_drop = nn.Dropout2d()\n self.fc1 = nn.Linear(320, 50)\n self.fc2 = nn.Linear(50, 10)\n\n def forward(self, x):\n x = F.relu(F.max_pool2d(self.conv1(x), 2))\n x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))\n x = x.view(-1, 320)\n x = F.relu(self.fc1(x))\n x = F.dropout(x, training=self.training)\n x = self.fc2(x)\n return F.log_softmax(x, dim=1)\n\n# MNIST \ud14c\uc2a4\ud2b8 \ub370\uc774\ud130\uc14b\uacfc \ub370\uc774\ud130\ub85c\ub354 \uc120\uc5b8\ntest_loader = torch.utils.data.DataLoader(\n datasets.MNIST('../data', train=False, download=True, transform=transforms.Compose([\n transforms.ToTensor(),\n ])),\n batch_size=1, shuffle=True)\n\n# \uc5b4\ub5a4 \ub514\ubc14\uc774\uc2a4\ub97c \uc0ac\uc6a9\ud560\uc9c0 \uc815\uc758\nprint(\"CUDA Available: \",torch.cuda.is_available())\ndevice = torch.device(\"cuda\" if (use_cuda and torch.cuda.is_available()) else \"cpu\")\n\n# \ubaa8\ub378 \ucd08\uae30\ud654\ud558\uae30\nmodel = Net().to(device)\n\n# \ubbf8\ub9ac \ud559\uc2b5\ub41c \ubaa8\ub378 \uc77d\uc5b4\uc624\uae30\nmodel.load_state_dict(torch.load(pretrained_model, map_location='cpu'))\n\n# \ubaa8\ub378\uc744 \ud3c9\uac00 \ubaa8\ub4dc\ub85c \uc124\uc815\ud558\uae30. \ub4dc\ub86d\uc544\uc6c3 \ub808\uc774\uc5b4\ub4e4\uc744 \uc704\ud574 \uc0ac\uc6a9\ub428\nmodel.eval()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "FGSM \uacf5\uaca9\n~~~~~~~~~~~\n\n\uc774\uc81c \uc6d0\ub798 \uc785\ub825\uc744 \uad50\ub780\uc2dc\ucf1c \uc801\ub300\uc801\uc778 \uc608\ub97c \ub9cc\ub4dc\ub294 \ud568\uc218\ub97c \uc815\uc758 \ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \n``fgsm_attack`` \ud568\uc218\ub294 \uc785\ub825 \ud30c\ub77c\ubbf8\ud130\ub85c 3\uac00\uc9c0\ub97c \uac00\uc9d1\ub2c8\ub2e4. \uccab\ubc88\uc9f8\ub294 \uc6d0\ubcf8 *\uc774\ubbf8\uc9c0* ( $x$ ), \n\ub450\ubc88\uc9f8\ub294 *\uc5e1\uc2e4\ub860* \uc73c\ub85c \ud53d\uc140 \ub2e8\uc704\uc758 \uc791\uc740 \ubcc0\ud654\ub97c \uc8fc\ub294 \uac12\uc785\ub2c8\ub2e4 ( $\\epsilon$ ). \n\ub9c8\uc9c0\ub9c9\uc740 *data_grad* \ub85c \uc785\ub825 \uc601\uc0c1 ( $\\nabla_{x} J(\\mathbf{\\theta}, \\mathbf{x}, y)$ ) \uc5d0 \ub300\ud55c \ubcc0\ud654\ub3c4 \uc190\uc2e4 \uac12\uc785\ub2c8\ub2e4. \n\uc544\ub798 \uc2dd\uc5d0 \ub530\ub978 \uc791\uc740 \ubcc0\ud654\uac00 \uc801\uc6a9\ub41c \uc774\ubbf8\uc9c0\ub97c \uc0dd\uc131\ud569\ub2c8\ub2e4.\n\n\\begin{align}perturbed\\_image = image + epsilon*sign(data\\_grad) = x + \\epsilon * sign(\\nabla_{x} J(\\mathbf{\\theta}, \\mathbf{x}, y))\\end{align}\n\n\ub9c8\uc9c0\ub9c9\uc73c\ub85c \ub370\uc774\ud130\uc758 \uc6d0\ub798 \ubc94\uc704\ub97c \uc720\uc9c0\ud558\uae30 \uc704\ud574, \uc791\uc740 \ubcc0\ud654\uac00 \uc801\uc6a9\ub41c \uc774\ubbf8\uc9c0\uac00 $[0,1]$ \ubc94\uc704\ub85c \uc798\ub9bd\ub2c8\ub2e4.\n\n\n" + "FGSM \uacf5\uaca9\n~~~~~~~~~~~\n\n\uc774\uc81c \uc6d0\ub798 \uc785\ub825\uc744 \uad50\ub780\uc2dc\ucf1c \uc801\ub300\uc801\uc778 \uc608\ub97c \ub9cc\ub4dc\ub294 \ud568\uc218\ub97c \uc815\uc758 \ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n``fgsm_attack`` \ud568\uc218\ub294 \uc785\ub825 \ud30c\ub77c\ubbf8\ud130\ub85c 3\uac00\uc9c0\ub97c \uac00\uc9d1\ub2c8\ub2e4. \uccab\ubc88\uc9f8\ub294 \uc6d0\ubcf8 *\uc774\ubbf8\uc9c0* ( $x$ ),\n\ub450\ubc88\uc9f8\ub294 *\uc5e1\uc2e4\ub860* \uc73c\ub85c \ud53d\uc140 \ub2e8\uc704\uc758 \uc791\uc740 \ubcc0\ud654\ub97c \uc8fc\ub294 \uac12\uc785\ub2c8\ub2e4 ( $\\epsilon$ ).\n\ub9c8\uc9c0\ub9c9\uc740 *data_grad* \ub85c \uc785\ub825 \uc601\uc0c1 ( $\\nabla_{x} J(\\mathbf{\\theta}, \\mathbf{x}, y)$ ) \uc5d0 \ub300\ud55c \ubcc0\ud654\ub3c4 \uc190\uc2e4 \uac12\uc785\ub2c8\ub2e4.\n\uc544\ub798 \uc2dd\uc5d0 \ub530\ub978 \uc791\uc740 \ubcc0\ud654\uac00 \uc801\uc6a9\ub41c \uc774\ubbf8\uc9c0\ub97c \uc0dd\uc131\ud569\ub2c8\ub2e4.\n\n\\begin{align}perturbed\\_image = image + epsilon*sign(data\\_grad) = x + \\epsilon * sign(\\nabla_{x} J(\\mathbf{\\theta}, \\mathbf{x}, y))\\end{align}\n\n\ub9c8\uc9c0\ub9c9\uc73c\ub85c \ub370\uc774\ud130\uc758 \uc6d0\ub798 \ubc94\uc704\ub97c \uc720\uc9c0\ud558\uae30 \uc704\ud574, \uc791\uc740 \ubcc0\ud654\uac00 \uc801\uc6a9\ub41c \uc774\ubbf8\uc9c0\uac00 $[0,1]$ \ubc94\uc704\ub85c \uc798\ub9bd\ub2c8\ub2e4.\n\n\n" ] }, { @@ -94,7 +94,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\ud14c\uc2a4\ud305 \ud568\uc218\n~~~~~~~~~~~~~~~~\n\n\ub9c8\uc9c0\ub9c9\uc73c\ub85c \ubcf8 \ud29c\ud1a0\ub9ac\uc5bc\uc758 \ud575\uc2ec \uacb0\uacfc\ub294 ``\ud14c\uc2a4\ud2b8`` \ud568\uc218\uc5d0\uc11c \uc624\uac8c \ub429\ub2c8\ub2e4. \n\uc774 \ud14c\uc2a4\ud2b8 \uae30\ub2a5\uc744 \ud638\ucd9c \ud560 \ub54c\ub9c8\ub2e4 MNIST \ud14c\uc2a4\ud2b8 \uc14b\uc5d0\uc11c \uc804\uccb4 \ud14c\uc2a4\ud2b8 \ub2e8\uacc4\ub97c \uc218\ud589\ud558\uace0 \ucd5c\uc885 \uc815\ud655\ub3c4\ub97c \ubcf4\uace0\ud569\ub2c8\ub2e4.\n\uadf8\ub7ec\ub098 \uc774 \ud568\uc218\uc5d0\ub294 *\uc5e1\uc2e4\ub860* \uc785\ub825\ub3c4 \ud544\uc694\ud569\ub2c8\ub2e4. \uc774\ub294 ``\ud14c\uc2a4\ud2b8``` \ud568\uc218\uac00 $\\epsilon$ \ud06c\uae30\uc5d0 \ub530\ub77c \uacf5\uaca9\uc790\uc758 \uacf5\uaca9\uc744 \ubc1b\ub294 \ubaa8\ub378\uc758 \n\uc815\ud655\ub3c4\uc744 \ubcf4\uace0\ud558\uae30 \ub54c\ubb38\uc785\ub2c8\ub2e4. \ub354 \uad6c\uccb4\uc801\uc73c\ub85c \ubcf4\uba74 \ud14c\uc2a4\ud2b8 \uc14b\uc758 \uac01\uac01\uc758 \uc0d8\ud50c\uc5d0\uc11c \ud14c\uc2a4\ud2b8 \ud568\uc218\ub294 \uc785\ub825 \ub370\uc774\ud130\uc5d0 \ub300\ud55c \uc190\uc2e4 \ubcc0\ud654\ub3c4( $data\\_grad$ )\ub97c \uacc4\uc0b0\ud558\uace0, \n ``FGSM \uacf5\uaca9``( $perturbed\\_data$ )\uc744 \ubc1b\uc740 \uc791\uc740 \ubcc0\ud654\uac00 \uc801\uc6a9\ub41c \uc774\ubbf8\uc9c0\ub97c \ub9cc\ub4e4\uace0 \ub098\uc11c \uc791\uc740 \ubcc0\ud654\uac00 \uc801\uc6a9\ub41c \uc774\ubbf8\uc9c0\uac00 \uc801\ub300\uc801\uc778\uc9c0 \ud655\uc778\uc744 \ud569\ub2c8\ub2e4. \n\ucd94\uac00\ub85c \ubaa8\ub378\uc758 \uc815\ud655\ub3c4\ub97c \ud14c\uc2a4\ud2b8\ud558\uae30 \uc704\ud574\uc11c \ud14c\uc2a4\ud2b8 \ud568\uc218\ub294 \ub098\uc911\uc5d0 \uc2dc\uac01\ud654\ud558\uc5ec \ubcfc \uc218 \uc788\ub3c4\ub85d \uc131\uacf5\uc801\uc73c\ub85c \uc5bb\uc740 \uc801\ub300\uc801 \uc774\ubbf8\uc9c0\ub97c \uc800\uc7a5\ud558\uace0 \ubc18\ud658\ud569\ub2c8\ub2e4.\n\n\n" + "\ud14c\uc2a4\ud305 \ud568\uc218\n~~~~~~~~~~~~~~~~\n\n\ub9c8\uc9c0\ub9c9\uc73c\ub85c \ubcf8 \ud29c\ud1a0\ub9ac\uc5bc\uc758 \ud575\uc2ec \uacb0\uacfc\ub294 ``\ud14c\uc2a4\ud2b8`` \ud568\uc218\uc5d0\uc11c \uc624\uac8c \ub429\ub2c8\ub2e4.\n\uc774 \ud14c\uc2a4\ud2b8 \uae30\ub2a5\uc744 \ud638\ucd9c \ud560 \ub54c\ub9c8\ub2e4 MNIST \ud14c\uc2a4\ud2b8 \uc14b\uc5d0\uc11c \uc804\uccb4 \ud14c\uc2a4\ud2b8 \ub2e8\uacc4\ub97c \uc218\ud589\ud558\uace0 \ucd5c\uc885 \uc815\ud655\ub3c4\ub97c \ubcf4\uace0\ud569\ub2c8\ub2e4.\n\uadf8\ub7ec\ub098 \uc774 \ud568\uc218\uc5d0\ub294 *\uc5e1\uc2e4\ub860* \uc785\ub825\ub3c4 \ud544\uc694\ud569\ub2c8\ub2e4. \uc774\ub294 ``\ud14c\uc2a4\ud2b8``` \ud568\uc218\uac00 $\\epsilon$ \ud06c\uae30\uc5d0 \ub530\ub77c \uacf5\uaca9\uc790\uc758 \uacf5\uaca9\uc744 \ubc1b\ub294 \ubaa8\ub378\uc758\n\uc815\ud655\ub3c4\uc744 \ubcf4\uace0\ud558\uae30 \ub54c\ubb38\uc785\ub2c8\ub2e4. \ub354 \uad6c\uccb4\uc801\uc73c\ub85c \ubcf4\uba74 \ud14c\uc2a4\ud2b8 \uc14b\uc758 \uac01\uac01\uc758 \uc0d8\ud50c\uc5d0\uc11c \ud14c\uc2a4\ud2b8 \ud568\uc218\ub294 \uc785\ub825 \ub370\uc774\ud130\uc5d0 \ub300\ud55c \uc190\uc2e4 \ubcc0\ud654\ub3c4( $data\\_grad$ )\ub97c \uacc4\uc0b0\ud558\uace0,\n``FGSM \uacf5\uaca9`` ($perturbed\\_data$) \uc744 \ubc1b\uc740 \uc791\uc740 \ubcc0\ud654\uac00 \uc801\uc6a9\ub41c \uc774\ubbf8\uc9c0\ub97c \ub9cc\ub4e4\uace0 \ub098\uc11c \uc791\uc740 \ubcc0\ud654\uac00 \uc801\uc6a9\ub41c \uc774\ubbf8\uc9c0\uac00 \uc801\ub300\uc801\uc778\uc9c0 \ud655\uc778\uc744 \ud569\ub2c8\ub2e4.\n\ucd94\uac00\ub85c \ubaa8\ub378\uc758 \uc815\ud655\ub3c4\ub97c \ud14c\uc2a4\ud2b8\ud558\uae30 \uc704\ud574\uc11c \ud14c\uc2a4\ud2b8 \ud568\uc218\ub294 \ub098\uc911\uc5d0 \uc2dc\uac01\ud654\ud558\uc5ec \ubcfc \uc218 \uc788\ub3c4\ub85d \uc131\uacf5\uc801\uc73c\ub85c \uc5bb\uc740 \uc801\ub300\uc801 \uc774\ubbf8\uc9c0\ub97c \uc800\uc7a5\ud558\uace0 \ubc18\ud658\ud569\ub2c8\ub2e4.\n\n\n" ] }, { @@ -112,7 +112,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uacf5\uaca9 \uc2e4\ud589\n~~~~~~~~~~\n\n\uad6c\ud604\uc758 \ub9c8\uc9c0\ub9c9 \ubd80\ubd84\uc740 \uacf5\uaca9\uc744 \uc2e4\ud589\ud558\ub294 \uac83\uc785\ub2c8\ub2e4. \uc5ec\uae30\uc11c \uc804\uccb4 \ud14c\uc2a4\ud2b8 \uc2a4\ud15d\uc744 \uac01 *\uc5e1\uc2e4\ub860* \uac12\uc5d0 \uc2e4\ud589\ud569\ub2c8\ub2e4. \n\uac01 \uc5e1\uc2e4\ub860\ub9c8\ub2e4 \ucd5c\uc885 \uc815\ud655\ub3c4\uc640 \uc131\uacf5\uc801\uc778 \uc77c\ubd80 \uc801\ub300 \uc0ac\ub840\ub97c \uc800\uc7a5\ud558\uc5ec \ub2e4\uc74c \uc139\uc158\uc5d0 \ud45c\uc2dc\ud569\ub2c8\ub2e4. \n\uc5e1\uc2e4\ub860 \uac12\uc774 \uc99d\uac00\ud568\uc5d0 \ub530\ub77c \ucd9c\ub825\ub41c \uc815\ud655\ub3c4\uac00 \uc5b4\ub5bb\uac8c \uac10\uc18c\ud558\ub294\uc9c0 \ubcf4\uc2ed\uc2dc\uc624. \n\ub610\ud55c, $\\epsilon=0$ \uc778 \uacbd\uc6b0\uc5d0\ub294 \uacf5\uaca9\uc774 \uc5c6\ub294 \uc6d0\ubcf8 \ud14c\uc2a4\ud2b8 \uc815\ud655\ub3c4\uc784\uc744 \ubcf4\uc785\ub2c8\ub2e4.\n\n\n" + "\uacf5\uaca9 \uc2e4\ud589\n~~~~~~~~~~\n\n\uad6c\ud604\uc758 \ub9c8\uc9c0\ub9c9 \ubd80\ubd84\uc740 \uacf5\uaca9\uc744 \uc2e4\ud589\ud558\ub294 \uac83\uc785\ub2c8\ub2e4. \uc5ec\uae30\uc11c \uc804\uccb4 \ud14c\uc2a4\ud2b8 \uc2a4\ud15d\uc744 \uac01 *\uc5e1\uc2e4\ub860* \uac12\uc5d0 \uc2e4\ud589\ud569\ub2c8\ub2e4.\n\uac01 \uc5e1\uc2e4\ub860\ub9c8\ub2e4 \ucd5c\uc885 \uc815\ud655\ub3c4\uc640 \uc131\uacf5\uc801\uc778 \uc77c\ubd80 \uc801\ub300 \uc0ac\ub840\ub97c \uc800\uc7a5\ud558\uc5ec \ub2e4\uc74c \uc139\uc158\uc5d0 \ud45c\uc2dc\ud569\ub2c8\ub2e4.\n\uc5e1\uc2e4\ub860 \uac12\uc774 \uc99d\uac00\ud568\uc5d0 \ub530\ub77c \ucd9c\ub825\ub41c \uc815\ud655\ub3c4\uac00 \uc5b4\ub5bb\uac8c \uac10\uc18c\ud558\ub294\uc9c0 \ubcf4\uc2ed\uc2dc\uc624.\n\ub610\ud55c, $\\epsilon=0$ \uc778 \uacbd\uc6b0\uc5d0\ub294 \uacf5\uaca9\uc774 \uc5c6\ub294 \uc6d0\ubcf8 \ud14c\uc2a4\ud2b8 \uc815\ud655\ub3c4\uc784\uc744 \ubcf4\uc785\ub2c8\ub2e4.\n\n\n" ] }, { @@ -130,7 +130,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uacb0\uacfc\n-------\n\n\uc815\ud655\ub3c4 vs \uc5e1\uc2e4\ub860\n~~~~~~~~~~~~~~~~~~~\n\n\uccab \ubc88\uc9f8 \uacb0\uacfc\ub294 \uc815\ud655\ub3c4 vs \uc5e1\uc2e4\ub860 \uc744 \ub3c4\uc2dd\ud654 \ud55c \uac83 \uc785\ub2c8\ub2e4. \n\uc55e\uc5d0\uc11c \uc5b8\uae09\ud588\ub4ef\uc774, \uc5e1\uc2e4\ub860\uc774 \uc99d\uac00\ud568\uc5d0 \ub530\ub77c \uc6b0\ub9ac\ub294 \ud14c\uc2a4\ud2b8 \uc815\ud655\ub3c4\uac00 \uac10\uc18c\ud560 \uac83\uc73c\ub85c \uc608\uc0c1\ud569\ub2c8\ub2e4.\n\uc774\ub294 \ud559\uc2b5\uc744 \ub354 \uc9c4\ud589\ud574 \uac08\uc218\ub85d \uc5e1\uc2e4\ub860\uc774 \ud074\uc218\ub85d \uc190\uc2e4\uc744 \uadf9\ub300\ud654 \ud560 \ubc29\ud5a5\uc73c\ub85c \uc9c4\ud589\ub418\uae30 \ub54c\ubb38\uc785\ub2c8\ub2e4.\n\uc5e1\uc2e4\ub860 \uac12\uc774 \uc120\ud615\uc801\uc73c\ub85c \ubd84\ud3ec\ud558\ub354\ub77c\ub3c4 \uace1\uc120\uc758 \ucd94\uc138\ub294 \uc120\ud615\uc758 \ud615\ud0dc\uac00 \uc544\ub2d9\ub2c8\ub2e4.\n\uc608\ub97c \ub4e4\uba74, math:`\\epsilon=0.05` \uc5d0\uc11c\uc758 \uc815\ud655\ub3c4\uac00 $\\epsilon=0$ \ubcf4\ub2e4 \uc57d 4% \ub0ae\uc9c0\ub9cc\n$\\epsilon=0.2$ \uc5d0\uc11c\uc758 \uc815\ud655\ub3c4\ub294 $\\epsilon=0.15$ \ubcf4\ub2e4 \uc57d 25% \uc815\ub3c4 \ub0ae\uc2b5\ub2c8\ub2e4.\n\ub610\ud55c, $\\epsilon=0.25$ \uc640 $\\epsilon=0.3$ \uc0ac\uc774\uc758 \ubaa8\ub378 \uc815\ud655\ub3c4\ub294 \ub79c\ub364\uc73c\ub85c\n10\uac1c\uc911 1\uac1c\ub97c \uc120\ud0dd\ud588\uc744 \ub54c\uc758 \uc815\ud655\ub3c4\uc640 \uc720\uc0ac\ud55c \uc218\uc900\uc785\ub2c8\ub2e4.\n\n" + "\uacb0\uacfc\n-------\n\n\uc815\ud655\ub3c4 vs \uc5e1\uc2e4\ub860\n~~~~~~~~~~~~~~~~~~~\n\n\uccab \ubc88\uc9f8 \uacb0\uacfc\ub294 \uc815\ud655\ub3c4 vs \uc5e1\uc2e4\ub860 \uc744 \ub3c4\uc2dd\ud654 \ud55c \uac83 \uc785\ub2c8\ub2e4.\n\uc55e\uc5d0\uc11c \uc5b8\uae09\ud588\ub4ef\uc774, \uc5e1\uc2e4\ub860\uc774 \uc99d\uac00\ud568\uc5d0 \ub530\ub77c \uc6b0\ub9ac\ub294 \ud14c\uc2a4\ud2b8 \uc815\ud655\ub3c4\uac00 \uac10\uc18c\ud560 \uac83\uc73c\ub85c \uc608\uc0c1\ud569\ub2c8\ub2e4.\n\uc774\ub294 \ud559\uc2b5\uc744 \ub354 \uc9c4\ud589\ud574 \uac08\uc218\ub85d \uc5e1\uc2e4\ub860\uc774 \ud074\uc218\ub85d \uc190\uc2e4\uc744 \uadf9\ub300\ud654 \ud560 \ubc29\ud5a5\uc73c\ub85c \uc9c4\ud589\ub418\uae30 \ub54c\ubb38\uc785\ub2c8\ub2e4.\n\uc5e1\uc2e4\ub860 \uac12\uc774 \uc120\ud615\uc801\uc73c\ub85c \ubd84\ud3ec\ud558\ub354\ub77c\ub3c4 \uace1\uc120\uc758 \ucd94\uc138\ub294 \uc120\ud615\uc758 \ud615\ud0dc\uac00 \uc544\ub2d9\ub2c8\ub2e4.\n\uc608\ub97c \ub4e4\uba74, math:`\\epsilon=0.05` \uc5d0\uc11c\uc758 \uc815\ud655\ub3c4\uac00 $\\epsilon=0$ \ubcf4\ub2e4 \uc57d 4% \ub0ae\uc9c0\ub9cc\n$\\epsilon=0.2$ \uc5d0\uc11c\uc758 \uc815\ud655\ub3c4\ub294 $\\epsilon=0.15$ \ubcf4\ub2e4 \uc57d 25% \uc815\ub3c4 \ub0ae\uc2b5\ub2c8\ub2e4.\n\ub610\ud55c, $\\epsilon=0.25$ \uc640 $\\epsilon=0.3$ \uc0ac\uc774\uc758 \ubaa8\ub378 \uc815\ud655\ub3c4\ub294 \ub79c\ub364\uc73c\ub85c\n10\uac1c\uc911 1\uac1c\ub97c \uc120\ud0dd\ud588\uc744 \ub54c\uc758 \uc815\ud655\ub3c4\uc640 \uc720\uc0ac\ud55c \uc218\uc900\uc785\ub2c8\ub2e4.\n\n" ] }, { @@ -148,7 +148,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uc0d8\ud50c \uc801\ub300\uc801 \uc608\uc81c\ub4e4\n~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n\uacf5\uc9dc \uc810\uc2ec\uc740 \uc5c6\ub2e4\ub294 \uac83\uc744 \uae30\uc5b5\ud558\uc2dc\ub098\uc694? \uc774 \uacbd\uc6b0\uc5d0\ub294 \uc5e1\uc2e4\ub860\uc774 \uc99d\uac00\ud560\uc218\ub85d \ud14c\uc2a4\ud2b8 \uc815\ud655\ub3c4\ub294 \ub5a8\uc5b4\uc9d1\ub2c8\ub2e4. \n**\uadf8\ub7ec\ub098** \uc791\uc740 \ubcc0\ud654\ub294 \ub354 \uc27d\uac8c \uc778\uc2dd\ud560 \uc218 \uc788\uac8c \ub429\ub2c8\ub2e4.\n\uc2e4\uc81c\ub85c \uc815\ud655\ub3c4 \uc800\ud558\uc640 \uacf5\uaca9\uc790\uac00 \uace0\ub824\ud574\uc57c \ud558\ub294 \uc774\ud574\ub3c4 \uc0ac\uc774\uc5d0\ub294 \uc0c1\ucda9 \uad00\uacc4(tradeoff)\uac00 \uc788\uc2b5\ub2c8\ub2e4.\n\uc5ec\uae30\uc11c \uc6b0\ub9ac\ub294 \uac01 \uc5e1\uc2e4\ub860 \uac12\uc5d0\uc11c \uc131\uacf5\uc801\uc778 \ub300\uc801 \uc0ac\ub840\ub97c \ubcf4\uc774\ub294 \uba87 \uac00\uc9c0 \uc608\ub97c \ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\uc544\ub798 \uc774\ubbf8\uc9c0\uc758 \uccab\ubc88\uc9f8\ub85c \uc5f4\uc740 $\\epsilon=0$ \uc778 \uc608\uc81c\ub4e4\ub85c \uc791\uc740 \ubcc0\ud654\uac00 \uc5c6\ub294 \uc6d0\ubcf8\uc758 \"\uae68\ub057\ud55c\" \uc774\ubbf8\uc9c0\ub4e4\uc744 \ub098\ud0c0\ub0c5\ub2c8\ub2e4.\n\uac01 \uc774\ubbf8\uc9c0\uc758 \uc704\uc758 \uae00\uc790\ub294 \"\uc6d0\ub798 \ubd84\ub958 \uacb0\uacfc -> \uc801\ub300\uc801 \ubd84\ub958 \uacb0\uacfc\"\ub97c \ub098\ud0c0\ub0c5\ub2c8\ub2e4.\n$\\epsilon=0.15$ \uc5d0\uc11c \uc791\uc740 \ubcc0\ud654\uac00 \ub208\uc5d0 \ub744\uae30 \uc2dc\uc791\ud558\uace0 $\\epsilon=0.3$ \uc5d0\uc11c\ub294 \ud655\uc2e4\ud574 \ubcf4\uc785\ub2c8\ub2e4.\n\uadf8\ub7ec\ub098 \ubaa8\ub4e0 \uacbd\uc6b0\uc5d0 \ub300\ud574\uc11c \ub178\uc774\uc988\uac00 \ucd94\uac00\ub418\uc5c8\ub354\ub77c\ub3c4 \uc0ac\ub78c\uc740 \uc62c\ubc14\ub974\uac8c \ubd84\ub958\ub97c \uc218\ud589\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n" + "\uc0d8\ud50c \uc801\ub300\uc801 \uc608\uc81c\ub4e4\n~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n\uacf5\uc9dc \uc810\uc2ec\uc740 \uc5c6\ub2e4\ub294 \uac83\uc744 \uae30\uc5b5\ud558\uc2dc\ub098\uc694? \uc774 \uacbd\uc6b0\uc5d0\ub294 \uc5e1\uc2e4\ub860\uc774 \uc99d\uac00\ud560\uc218\ub85d \ud14c\uc2a4\ud2b8 \uc815\ud655\ub3c4\ub294 \ub5a8\uc5b4\uc9d1\ub2c8\ub2e4.\n**\uadf8\ub7ec\ub098** \uc791\uc740 \ubcc0\ud654\ub294 \ub354 \uc27d\uac8c \uc778\uc2dd\ud560 \uc218 \uc788\uac8c \ub429\ub2c8\ub2e4.\n\uc2e4\uc81c\ub85c \uc815\ud655\ub3c4 \uc800\ud558\uc640 \uacf5\uaca9\uc790\uac00 \uace0\ub824\ud574\uc57c \ud558\ub294 \uc774\ud574\ub3c4 \uc0ac\uc774\uc5d0\ub294 \uc0c1\ucda9 \uad00\uacc4(tradeoff)\uac00 \uc788\uc2b5\ub2c8\ub2e4.\n\uc5ec\uae30\uc11c \uc6b0\ub9ac\ub294 \uac01 \uc5e1\uc2e4\ub860 \uac12\uc5d0\uc11c \uc131\uacf5\uc801\uc778 \ub300\uc801 \uc0ac\ub840\ub97c \ubcf4\uc774\ub294 \uba87 \uac00\uc9c0 \uc608\ub97c \ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\uc544\ub798 \uc774\ubbf8\uc9c0\uc758 \uccab\ubc88\uc9f8\ub85c \uc5f4\uc740 $\\epsilon=0$ \uc778 \uc608\uc81c\ub4e4\ub85c \uc791\uc740 \ubcc0\ud654\uac00 \uc5c6\ub294 \uc6d0\ubcf8\uc758 \"\uae68\ub057\ud55c\" \uc774\ubbf8\uc9c0\ub4e4\uc744 \ub098\ud0c0\ub0c5\ub2c8\ub2e4.\n\uac01 \uc774\ubbf8\uc9c0\uc758 \uc704\uc758 \uae00\uc790\ub294 \"\uc6d0\ub798 \ubd84\ub958 \uacb0\uacfc -> \uc801\ub300\uc801 \ubd84\ub958 \uacb0\uacfc\"\ub97c \ub098\ud0c0\ub0c5\ub2c8\ub2e4.\n$\\epsilon=0.15$ \uc5d0\uc11c \uc791\uc740 \ubcc0\ud654\uac00 \ub208\uc5d0 \ub744\uae30 \uc2dc\uc791\ud558\uace0 $\\epsilon=0.3$ \uc5d0\uc11c\ub294 \ud655\uc2e4\ud574 \ubcf4\uc785\ub2c8\ub2e4.\n\uadf8\ub7ec\ub098 \ubaa8\ub4e0 \uacbd\uc6b0\uc5d0 \ub300\ud574\uc11c \ub178\uc774\uc988\uac00 \ucd94\uac00\ub418\uc5c8\ub354\ub77c\ub3c4 \uc0ac\ub78c\uc740 \uc62c\ubc14\ub974\uac8c \ubd84\ub958\ub97c \uc218\ud589\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n" ] }, { @@ -166,7 +166,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\ub2e4\uc74c \ub2e8\uacc4\ub294?\n-----------------\n\n\uc774\ubc88 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c \uc801\ub300\uc801 \uba38\uc2e0 \ub7ec\ub2dd\uc5d0 \ub300\ud55c \ud1b5\ucc30\uc744 \uc5bb\uc744 \uc218 \uc788\uc5c8\uae30\ub97c \ubc14\ub78d\ub2c8\ub2e4.\n\ud29c\ud1a0\ub9ac\uc5bc\uc758 \ub0b4\uc6a9\uc73c\ub85c\ubd80\ud130 \uc55e\uc73c\ub85c \ub354 \ub9ce\uc740 \uac83\ub4e4\uc744 \uc54c\uc544\ub098\uac08 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\ud29c\ud1a0\ub9ac\uc5bc\uc758 \uc801\ub300\uc801 \uacf5\uaca9 \uc608\uc81c\ub294 \ubcf8 \ubd84\uc57c\uc758 \ucd08\uae09 \ub2e8\uacc4\uc774\uba70 \n\uc801\ub300\uc801 \uc0c1\ud669\uc73c\ub85c\ubd80\ud130 ML \ubaa8\ub378\uc744 \uacf5\uaca9\ud558\uace0 \ubc29\uc5b4\ud558\ub294 \ubc29\ubc95\uc5d0 \ub300\ud55c \ub9ce\uc740 \ud6c4\uc18d \uc544\uc774\ub514\uc5b4\uac00 \uc788\uc2b5\ub2c8\ub2e4.\n\uc0ac\uc2e4 NIPS 2017 \uc5d0\uc11c \uc801\ub300\uc801 \uacf5\uaca9\uacfc \ubc29\uc5b4\uc5d0 \ub300\ud55c \uacbd\uc7c1(competition)\uc774 \uc788\uc5c8\uace0 \uc5ec\uae30\uc11c \uc0ac\uc6a9\ub41c\n\ub2e4\uc591\ud55c \ubc29\ubc95\ub4e4\uc740 \ub2e4\uc74c \ub17c\ubb38\uc5d0 \uc815\ub9ac \ub418\uc5b4 \uc788\uc2b5\ub2c8\ub2e4: `\uc801\ub300\uc801 \uacf5\uaca9\uacfc \ubc29\uc5b4 \uacbd\uc7c1 `__.\n\ubc29\uc5b4\uc5d0 \ub300\ud55c \uc5f0\uad6c\ub294 \uc790\uc5f0\uc2a4\ub7fd\uac8c \uad50\ub780 \ubc0f \ud574\ud0b9 \ubaa9\uc801\uc73c\ub85c \uc81c\uc791\ub41c \uc785\ub825\uc5d0 \ub300\ud574 \uba38\uc2e0 \ub7ec\ub2dd \ubaa8\ub378\uc744\n\ubcf4\ub2e4 *\uacac\uace0\ud558\uac8c(robust)* \ub9cc\ub4dc\ub294 \uc544\uc774\ub514\uc5b4\ub85c \uc774\uc5b4\uc9d1\ub2c8\ub2e4.\n\n\ub610 \ub2e4\ub978 \ubc29\ud5a5\uc740 \ub2e4\ub978 \ub3c4\uba54\uc778\uc5d0\uc11c\uc758 \uc801\uc758 \uacf5\uaca9\uacfc \ubc29\uc5b4\uc785\ub2c8\ub2e4. \uc801\ub300\uc801 \uc5f0\uad6c\ub294 \uc774\ubbf8\uc9c0 \ub3c4\uba54\uc778\uc5d0 \uc81c\ud55c\ub418\uc5b4 \uc788\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4. \n`\uc5ec\uae30 `__ \uc5d0\uc11c \uc74c\uc131-\ud14d\uc2a4\ud2b8 \ubcc0\ud658 \ubaa8\ub378\uc5d0\uc11c\uc758 \uacf5\uaca9\ub3c4 \ud655\uc778\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uadf8\ub7ec\ub098 \uc801\ub300\uc801 \uba38\uc2e0 \ub7ec\ub2dd \ubd84\uc57c\uc5d0 \ub300\ud574\uc11c \ub9ce\uc740 \uac83\uc744 \uc54c\uae30 \uc704\ud55c \ucd5c\uace0\uc758 \ubc29\ubc95\uc740 \ub9ce\uc774 \uc2dc\ub3c4\ud574\ubcf4\ub294 \uac83\uc785\ub2c8\ub2e4.\nNIPS 2017 \uacbd\uc7c1\uc5d0\uc11c \uc18c\uac1c\ub41c \ub2e4\uc591\ud55c \uacf5\uaca9 \ubc29\ubc95\uc744 \uc9c1\uc811 \uad6c\ud604\ud574 \ubcf4\uace0, FGSM \uacfc \uc5b4\ub5a4 \uc810\uc774 \ub2e4\ub978\uc9c0 \uc5f0\uad6c\ud574 \ubcf4\uc138\uc694.\n\uadf8\ub9ac\uace0 \ub098\uc11c \uc9c1\uc811 \ub9cc\ub4e0 \uacf5\uaca9\uc73c\ub85c\ubd80\ud130 \ubaa8\ub378\uc744 \ubc29\uc5b4\ud574 \ubcf4\uc138\uc694.\n\n\n" + "\ub2e4\uc74c \ub2e8\uacc4\ub294?\n-----------------\n\n\uc774\ubc88 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c \uc801\ub300\uc801 \uba38\uc2e0 \ub7ec\ub2dd\uc5d0 \ub300\ud55c \ud1b5\ucc30\uc744 \uc5bb\uc744 \uc218 \uc788\uc5c8\uae30\ub97c \ubc14\ub78d\ub2c8\ub2e4.\n\ud29c\ud1a0\ub9ac\uc5bc\uc758 \ub0b4\uc6a9\uc73c\ub85c\ubd80\ud130 \uc55e\uc73c\ub85c \ub354 \ub9ce\uc740 \uac83\ub4e4\uc744 \uc54c\uc544\ub098\uac08 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\ud29c\ud1a0\ub9ac\uc5bc\uc758 \uc801\ub300\uc801 \uacf5\uaca9 \uc608\uc81c\ub294 \ubcf8 \ubd84\uc57c\uc758 \ucd08\uae09 \ub2e8\uacc4\uc774\uba70\n\uc801\ub300\uc801 \uc0c1\ud669\uc73c\ub85c\ubd80\ud130 ML \ubaa8\ub378\uc744 \uacf5\uaca9\ud558\uace0 \ubc29\uc5b4\ud558\ub294 \ubc29\ubc95\uc5d0 \ub300\ud55c \ub9ce\uc740 \ud6c4\uc18d \uc544\uc774\ub514\uc5b4\uac00 \uc788\uc2b5\ub2c8\ub2e4.\n\uc0ac\uc2e4 NIPS 2017 \uc5d0\uc11c \uc801\ub300\uc801 \uacf5\uaca9\uacfc \ubc29\uc5b4\uc5d0 \ub300\ud55c \uacbd\uc7c1(competition)\uc774 \uc788\uc5c8\uace0 \uc5ec\uae30\uc11c \uc0ac\uc6a9\ub41c\n\ub2e4\uc591\ud55c \ubc29\ubc95\ub4e4\uc740 \ub2e4\uc74c \ub17c\ubb38\uc5d0 \uc815\ub9ac \ub418\uc5b4 \uc788\uc2b5\ub2c8\ub2e4: `\uc801\ub300\uc801 \uacf5\uaca9\uacfc \ubc29\uc5b4 \uacbd\uc7c1 `__.\n\ubc29\uc5b4\uc5d0 \ub300\ud55c \uc5f0\uad6c\ub294 \uc790\uc5f0\uc2a4\ub7fd\uac8c \uad50\ub780 \ubc0f \ud574\ud0b9 \ubaa9\uc801\uc73c\ub85c \uc81c\uc791\ub41c \uc785\ub825\uc5d0 \ub300\ud574 \uba38\uc2e0 \ub7ec\ub2dd \ubaa8\ub378\uc744\n\ubcf4\ub2e4 *\uacac\uace0\ud558\uac8c(robust)* \ub9cc\ub4dc\ub294 \uc544\uc774\ub514\uc5b4\ub85c \uc774\uc5b4\uc9d1\ub2c8\ub2e4.\n\n\ub610 \ub2e4\ub978 \ubc29\ud5a5\uc740 \ub2e4\ub978 \ub3c4\uba54\uc778\uc5d0\uc11c\uc758 \uc801\uc758 \uacf5\uaca9\uacfc \ubc29\uc5b4\uc785\ub2c8\ub2e4. \uc801\ub300\uc801 \uc5f0\uad6c\ub294 \uc774\ubbf8\uc9c0 \ub3c4\uba54\uc778\uc5d0 \uc81c\ud55c\ub418\uc5b4 \uc788\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4.\n`\uc5ec\uae30 `__ \uc5d0\uc11c \uc74c\uc131-\ud14d\uc2a4\ud2b8 \ubcc0\ud658 \ubaa8\ub378\uc5d0\uc11c\uc758 \uacf5\uaca9\ub3c4 \ud655\uc778\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uadf8\ub7ec\ub098 \uc801\ub300\uc801 \uba38\uc2e0 \ub7ec\ub2dd \ubd84\uc57c\uc5d0 \ub300\ud574\uc11c \ub9ce\uc740 \uac83\uc744 \uc54c\uae30 \uc704\ud55c \ucd5c\uace0\uc758 \ubc29\ubc95\uc740 \ub9ce\uc774 \uc2dc\ub3c4\ud574\ubcf4\ub294 \uac83\uc785\ub2c8\ub2e4.\nNIPS 2017 \uacbd\uc7c1\uc5d0\uc11c \uc18c\uac1c\ub41c \ub2e4\uc591\ud55c \uacf5\uaca9 \ubc29\ubc95\uc744 \uc9c1\uc811 \uad6c\ud604\ud574 \ubcf4\uace0, FGSM \uacfc \uc5b4\ub5a4 \uc810\uc774 \ub2e4\ub978\uc9c0 \uc5f0\uad6c\ud574 \ubcf4\uc138\uc694.\n\uadf8\ub9ac\uace0 \ub098\uc11c \uc9c1\uc811 \ub9cc\ub4e0 \uacf5\uaca9\uc73c\ub85c\ubd80\ud130 \ubaa8\ub378\uc744 \ubc29\uc5b4\ud574 \ubcf4\uc138\uc694.\n\n\n" ] } ], @@ -186,7 +186,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/58bdf45884d385ba7031225104b471d3/word_embeddings_tutorial.py b/docs/_downloads/58bdf45884d385ba7031225104b471d3/word_embeddings_tutorial.py index ca7d68ca6..7cc00ea6b 100644 --- a/docs/_downloads/58bdf45884d385ba7031225104b471d3/word_embeddings_tutorial.py +++ b/docs/_downloads/58bdf45884d385ba7031225104b471d3/word_embeddings_tutorial.py @@ -115,7 +115,7 @@ 파이토치에서 단어 임베딩 하기 -~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 실제로 코드와 예시를 보기 전에, 파이토치를 비롯해 딥러닝 관련 프로그래밍을 할 때 단어 임베딩을 어떻게 사용하는지에 대해 조금 알아봅시다. 맨 위에서 원핫 벡터를 diff --git a/docs/_downloads/591028d309d0401740cd71eb6b14bf93/memory_format_tutorial.py b/docs/_downloads/591028d309d0401740cd71eb6b14bf93/memory_format_tutorial.py index 244e23ac2..7bec8ef8e 100644 --- a/docs/_downloads/591028d309d0401740cd71eb6b14bf93/memory_format_tutorial.py +++ b/docs/_downloads/591028d309d0401740cd71eb6b14bf93/memory_format_tutorial.py @@ -1,112 +1,115 @@ # -*- coding: utf-8 -*- """ -(beta) Channels Last Memory Format in PyTorch +(실험용) PyTorch를 사용한 Channels Last Memory Format ******************************************************* + **Author**: `Vitaly Fedyunin `_ -What is Channels Last ---------------------- +**번역**: `Choi Yoonjeong `_ -Channels Last memory format is an alternative way of ordering NCHW tensors in memory preserving dimensions ordering. Channels Last tensors ordered in such a way that channels become the densest dimension (aka storing images pixel-per-pixel). +Channels Last가 무엇인가요 +---------------------------- +Channels Last 메모리 형식(memory format)은 차원 순서를 유지하면서 메모리 상의 NCHW 텐서(tensor)를 정렬하는 또 다른 방식입니다. +Channels Last 텐서는 채널(Channel)이 가장 밀도가 높은(densest) 차원으로 정렬(예. 이미지를 픽셀x픽셀로 저장)됩니다. -For example, classic (contiguous) storage of NCHW tensor (in our case it is two 2x2 images with 3 color channels) look like this: +예를 들어, (2개의 2 x 2 이미지에 3개의 채널이 존재하는 경우) 전형적인(연속적인) NCHW 텐서의 저장 방식은 다음과 같습니다: .. figure:: /_static/img/classic_memory_format.png :alt: classic_memory_format -Channels Last memory format orders data differently: +Channels Last 메모리 형식은 데이터를 다르게 정렬합니다: .. figure:: /_static/img/channels_last_memory_format.png :alt: channels_last_memory_format -Pytorch supports memory formats (and provides back compatibility with existing models including eager, JIT, and TorchScript) by utilizing existing strides structure. -For example, 10x3x16x16 batch in Channels Last format will have strides equal to (768, 1, 48, 3). +PyTorch는 기존의 스트라이드(strides) 구조를 사용함으로써 메모리 형식을 지원(하며, eager, JIT 및 TorchScript를 포함한 +기존의 모델들과 하위 호환성을 제공)합니다. 예를 들어, Channels Last 형식에서 10x3x16x16 배치(batch)는 (768, 1, 48, 3)와 +같은 폭(strides)을 가지고 있게 됩니다. + """ ###################################################################### -# Channels Last memory format is implemented for 4D NCWH Tensors only. +# Channels Last 메모리 형식은 오직 4D NCWH Tensors에서만 실행할 수 있습니다. # import torch N, C, H, W = 10, 3, 32, 32 ###################################################################### -# Memory Format API -# ----------------------- +# 메모리 형식(Memory Format) API +# --------------------------------- # -# Here is how to convert tensors between contiguous and channels -# last memory formats. +# 연속 메모리 형식과 channels last 메모리 형식 간에 텐서를 변환하는 방법은 다음과 같습니다. ###################################################################### -# Classic PyTorch contiguous tensor +# 전형적인 PyTorch의 연속적인 텐서(tensor) x = torch.empty(N, C, H, W) -print(x.stride()) # Ouputs: (3072, 1024, 32, 1) +print(x.stride()) # 결과: (3072, 1024, 32, 1) ###################################################################### -# Conversion operator +# 변환 연산자 x = x.contiguous(memory_format=torch.channels_last) -print(x.shape) # Outputs: (10, 3, 32, 32) as dimensions order preserved -print(x.stride()) # Outputs: (3072, 1, 96, 3) +print(x.shape) # 결과: (10, 3, 32, 32) 차원 순서는 보존함 +print(x.stride()) # 결과: (3072, 1, 96, 3) ###################################################################### -# Back to contiguous +# 연속적인 형식으로 되돌리기 x = x.contiguous(memory_format=torch.contiguous_format) -print(x.stride()) # Outputs: (3072, 1024, 32, 1) +print(x.stride()) # 결과: (3072, 1024, 32, 1) ###################################################################### -# Alternative option +# 다른 방식 x = x.to(memory_format=torch.channels_last) -print(x.stride()) # Ouputs: (3072, 1, 96, 3) +print(x.stride()) # 결과: (3072, 1, 96, 3) ###################################################################### -# Format checks -print(x.is_contiguous(memory_format=torch.channels_last)) # Ouputs: True +# 형식(format) 확인 +print(x.is_contiguous(memory_format=torch.channels_last)) # 결과: True ###################################################################### -# Create as Channels Last +# Channels Last 방식으로 생성하기 x = torch.empty(N, C, H, W, memory_format=torch.channels_last) -print(x.stride()) # Ouputs: (3072, 1, 96, 3) +print(x.stride()) # 결과: (3072, 1, 96, 3) ###################################################################### -# ``clone`` preserves memory format +# ``clone`` 은 메모리 형식을 보존합니다. y = x.clone() -print(y.stride()) # Ouputs: (3072, 1, 96, 3) +print(y.stride()) # 결과: (3072, 1, 96, 3) ###################################################################### -# ``to``, ``cuda``, ``float`` ... preserves memory format +# ``to``, ``cuda``, ``float`` ... 등도 메모리 형식을 보존합니다. if torch.cuda.is_available(): y = x.cuda() - print(y.stride()) # Ouputs: (3072, 1, 96, 3) + print(y.stride()) # 결과: (3072, 1, 96, 3) ###################################################################### -# ``empty_like``, ``*_like`` operators preserves memory format +# ``empty_like``, ``*_like`` 연산자도 메모리 형식을 보존합니다. y = torch.empty_like(x) -print(y.stride()) # Ouputs: (3072, 1, 96, 3) +print(y.stride()) # 결과: (3072, 1, 96, 3) ###################################################################### -# Pointwise operators preserves memory format +# Pointwise 연산자도 메모리 형식을 보존합니다. z = x + y -print(z.stride()) # Ouputs: (3072, 1, 96, 3) +print(z.stride()) # 결과: (3072, 1, 96, 3) ###################################################################### -# Conv, Batchnorm modules support Channels Last -# (only works for CudNN >= 7.6) +# Conv, Batchnorm 모듈은 Channels Last를 지원합니다. (단, CudNN >=7.6 에서만 동작) if torch.backends.cudnn.version() >= 7603: input = torch.randint(1, 10, (2, 8, 4, 4), dtype=torch.float32, device="cuda", requires_grad=True) model = torch.nn.Conv2d(8, 4, 3).cuda().float() input = input.contiguous(memory_format=torch.channels_last) - model = model.to(memory_format=torch.channels_last) # Module parameters need to be Channels Last + model = model.to(memory_format=torch.channels_last) # 모듈 인자들은 Channels Last로 변환이 필요합니다 out = model(input) - print(out.is_contiguous(memory_format=torch.channels_last)) # Ouputs: True + print(out.is_contiguous(memory_format=torch.channels_last)) # 결과: True ###################################################################### -# Performance Gains +# 성능 향상 # ------------------------------------------------------------------------------------------- -# The most significant performance gains are observed on Nvidia's hardware with -# Tensor Cores support. We were able to archive over 22% perf gains while running ' -# AMP (Automated Mixed Precision) training scripts supplied by Nvidia https://github.com/NVIDIA/apex. +# Tensor Cores를 지원하는 Nvidia의 하드웨어에서 가장 의미심장한 성능 향상을 보였습니다. +# Nvidia가 제공하는 AMP(Automated Mixed Precision) 학습 스크립트로 22% 이상의 성능 향상을 확인할 수 있었습니다. +# https://github.com/NVIDIA/apex # # ``python main_amp.py -a resnet50 --b 200 --workers 16 --opt-level O2 ./data`` @@ -143,7 +146,7 @@ # Epoch: [0][80/125] Time 0.260 (0.335) Speed 770.324 (597.659) Loss 2.2505953312 (1.0879) Prec@1 50.500 (52.938) Prec@5 100.000 (100.000) ###################################################################### -# Passing ``--channels-last true`` allows running a model in Channels Last format with observed 22% perf gain. +# ``--channels-last true`` 인자를 전달하여 Channels Last 형식으로 모델을 실행하면 22%의 성능 향상을 보입니다. # # ``python main_amp.py -a resnet50 --b 200 --workers 16 --opt-level O2 --channels-last true ./data`` @@ -184,32 +187,31 @@ # Epoch: [0][80/125] Time 0.198 (0.269) Speed 1011.827 (743.883) Loss 2.8196096420 (2.4011) Prec@1 47.500 (50.938) Prec@5 100.000 (100.000) ###################################################################### -# The following list of models has the full support of Channels Last and showing 8%-35% perf gains on Volta devices: +# 아래 목록의 모델들은 Channels Last 형식을 전적으로 지원(full support)하며 Volta 장비에서 8%-35%의 성능 향상을 보입니다: # ``alexnet``, ``mnasnet0_5``, ``mnasnet0_75``, ``mnasnet1_0``, ``mnasnet1_3``, ``mobilenet_v2``, ``resnet101``, ``resnet152``, ``resnet18``, ``resnet34``, ``resnet50``, ``resnext50_32x4d``, ``shufflenet_v2_x0_5``, ``shufflenet_v2_x1_0``, ``shufflenet_v2_x1_5``, ``shufflenet_v2_x2_0``, ``squeezenet1_0``, ``squeezenet1_1``, ``vgg11``, ``vgg11_bn``, ``vgg13``, ``vgg13_bn``, ``vgg16``, ``vgg16_bn``, ``vgg19``, ``vgg19_bn``, ``wide_resnet101_2``, ``wide_resnet50_2`` # ###################################################################### -# Converting existing models +# 기존 모델들 변환하기 # -------------------------- # -# Channels Last support not limited by existing models, as any model can be converted to Channels Last and propagate format through the graph as soon as input formatted correctly. +# Channels Last 지원은 기존 모델이 무엇이냐에 따라 제한되지 않으며, 어떠한 모델도 Channels Last로 변환할 수 있으며 +# 입력(input)의 형식만 맞춰주면 (신경망) 그래프를 통해 바로 전파(propagate)할 수 있습니다. # -# Need to be done once, after model initialization (or load) -model = model.to(memory_format=torch.channels_last) # Replace with your model +# 모델을 초기화한(또는 불러온) 이후, 한 번 실행이 필요합니다. +model = model.to(memory_format=torch.channels_last) # 원하는 모델로 교체하기 -# Need to be done for every input -input = input.to(memory_format=torch.channels_last) # Replace with your input +# 모든 입력에 대해서 실행이 필요합니다. +input = input.to(memory_format=torch.channels_last) # 원하는 입력으로 교체하기 output = model(input) ####################################################################### -# However, not all operators fully converted to support Channels Last (usually returning -# contiguous output instead). That means you need to verify the list of used operators -# against supported operators list https://github.com/pytorch/pytorch/wiki/Operators-with-Channels-Last-support, -# or introduce memory format checks into eager execution mode and run your model. +# 그러나, 모든 연산자들이 Channels Last를 지원하도록 완전히 바뀐 것은 아닙니다(일반적으로는 연속적인 출력을 대신 반환합니다). +# 즉, Channel Last 지원 연산자 목록 https://github.com/pytorch/pytorch/wiki/Operators-with-Channels-Last-support 에서 사용한 연산자들이 존재하는지 확인하거나, +# eager 실행 모드에서 메모리 형식 검사를 도입하고 모델을 실행해야 합니다. # -# After running the code below, operators will raise an exception if the output of the -# operator doesn't match the memory format of the input. +# 아래 코드에서, 연산자들의 출력이 입력의 메모리 형식과 일치하지 않으면 예외(exception)를 발생시킵니다. # # def contains_cl(args): @@ -281,19 +283,19 @@ def attribute(m): ###################################################################### -# If you found an operator that doesn't support Channels Last tensors -# and you want to contribute, feel free to use following developers -# guide https://github.com/pytorch/pytorch/wiki/Writing-memory-format-aware-operators. +# 만약 Channels Last 텐서를 지원하지 않는 연산자를 발견하였고, 기여하기를 원한다면 +# 다음 개발 문서를 참고해주세요. +# https://github.com/pytorch/pytorch/wiki/Writing-memory-format-aware-operators # ###################################################################### -# Work to do +# 해야할 일 # ---------- -# There are still many things to do, such as: +# 다음과 같이 여전히 해야 할 일이 많이 남아있습니다: +# +# - N1HW와 NC11 Tensors의 모호성 해결하기; +# - 분산 학습을 지원하는지 확인하기; +# - 연산자 범위(operators coverage) 개선(improve)하기 # -# - Resolving ambiguity of N1HW and NC11 Tensors; -# - Testing of Distributed Training support; -# - Improving operators coverage. +# 개선할 부분에 대한 피드백 또는 제안이 있다면 `이슈를 만들어 `_ 알려주세요. # -# If you have feedback and/or suggestions for improvement, please let us -# know by creating `an issue `_. diff --git a/docs/_downloads/597dbaac5c207608e108534fea081ff9/what_is_state_dict.ipynb b/docs/_downloads/597dbaac5c207608e108534fea081ff9/what_is_state_dict.ipynb index 1b96fd96b..785478c7c 100644 --- a/docs/_downloads/597dbaac5c207608e108534fea081ff9/what_is_state_dict.ipynb +++ b/docs/_downloads/597dbaac5c207608e108534fea081ff9/what_is_state_dict.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nPyTorch\uc5d0\uc11c state_dict\ub780 \ubb34\uc5c7\uc778\uac00\uc694?\n======================================\nPyTorch\uc5d0\uc11c ``torch.nn.Module`` \ubaa8\ub378\uc758 \ud559\uc2b5 \uac00\ub2a5\ud55c\n\ub9e4\uac1c\ubcc0\uc218(\uc608. \uac00\uc911\uce58\uc640 \ud3b8\ud5a5)\ub4e4\uc740 \ubaa8\ub378\uc758 \ub9e4\uac1c\ubcc0\uc218\uc5d0 \ud3ec\ud568\ub418\uc5b4 \uc788\uc2b5\ub2c8\ub2e4.\n(model.parameters()\ub85c \uc811\uadfc\ud569\ub2c8\ub2e4)\n``state_dict`` \ub294 \uac04\ub2e8\ud788 \ub9d0\ud574 \uac01 \uacc4\uce35\uc744 \ub9e4\uac1c\ubcc0\uc218 \ud150\uc11c\ub85c \ub9e4\ud551\ub418\ub294\nPython \uc0ac\uc804(dict) \uac1d\uccb4\uc785\ub2c8\ub2e4.\n\n\uac1c\uc694\n------------\n``state_dict`` \ub294 PyTorch\uc5d0\uc11c \ubaa8\ub378\uc744 \uc800\uc7a5\ud558\uac70\ub098 \ubd88\ub7ec\uc624\ub294 \ub370 \uad00\uc2ec\uc774\n\uc788\ub2e4\uba74 \ud544\uc218\uc801\uc778 \ud56d\ubaa9\uc785\ub2c8\ub2e4.\n``state_dict`` \uac1d\uccb4\ub294 Python \uc0ac\uc804\uc774\uae30 \ub54c\ubb38\uc5d0 \uc27d\uac8c \uc800\uc7a5, \uc5c5\ub370\uc774\ud2b8,\n\ubcc0\uacbd \ubc0f \ubcf5\uc6d0\ud560 \uc218 \uc788\uc73c\uba70, \uc774\ub294 PyTorch \ubaa8\ub378\uacfc \uc635\ud2f0\ub9c8\uc774\uc800\uc5d0 \uc5c4\uccad\ub09c\n\ubaa8\ub4c8\uc131(modularity)\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.\n\uc774 \ub54c, \ud559\uc2b5 \uac00\ub2a5\ud55c \ub9e4\uac1c\ubcc0\uc218\ub97c \uac16\ub294 \uacc4\uce35(\ud569\uc131\uacf1 \uacc4\uce35, \uc120\ud615 \uacc4\uce35 \ub4f1)\n\ubc0f \ub4f1\ub85d\ub41c \ubc84\ud37c\ub4e4(batchnorm\uc758 running_mean)\ub9cc \ubaa8\ub378\uc758 ``state_dict``\n \ud56d\ubaa9\uc744 \uac00\uc9c4\ub2e4\ub294 \uc810\uc5d0 \uc720\uc758\ud558\uc2dc\uae30 \ubc14\ub78d\ub2c8\ub2e4. \uc635\ud2f0\ub9c8\uc774\uc800 \uac1d\uccb4\n( ``torch.optim`` ) \ub610\ud55c \uc635\ud2f0\ub9c8\uc774\uc800\uc758 \uc0c1\ud0dc \ubfd0\ub9cc \uc544\ub2c8\ub77c \uc0ac\uc6a9\ub41c\n\ud558\uc774\ud37c \ub9e4\uac1c\ubcc0\uc218 (Hyperparameter) \uc815\ubcf4\uac00 \ud3ec\ud568\ub41c ``state_dict`` \uc744\n\uac16\uc2b5\ub2c8\ub2e4.\n\ub808\uc2dc\ud53c\uc5d0\uc11c ``state_dict`` \uc774 \uac04\ub2e8\ud55c \ubaa8\ub378\uc5d0\uc11c \uc5b4\ub5bb\uac8c \uc0ac\uc6a9\ub418\ub294\uc9c0\n\uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc124\uc815\n----------\n\uc2dc\uc791\ud558\uae30 \uc804\uc5d0 ``torch`` \uac00 \uc5c6\ub2e4\uba74 \uc124\uce58\ud574\uc57c \ud569\ub2c8\ub2e4.\n\n::\n\n pip install torchaudio\n\n\n" + "\nPyTorch\uc5d0\uc11c state_dict\ub780 \ubb34\uc5c7\uc778\uac00\uc694?\n======================================\nPyTorch\uc5d0\uc11c ``torch.nn.Module`` \ubaa8\ub378\uc758 \ud559\uc2b5 \uac00\ub2a5\ud55c\n\ub9e4\uac1c\ubcc0\uc218(\uc608. \uac00\uc911\uce58\uc640 \ud3b8\ud5a5)\ub4e4\uc740 \ubaa8\ub378\uc758 \ub9e4\uac1c\ubcc0\uc218\uc5d0 \ud3ec\ud568\ub418\uc5b4 \uc788\uc2b5\ub2c8\ub2e4.\n(model.parameters()\ub85c \uc811\uadfc\ud569\ub2c8\ub2e4)\n``state_dict`` \ub294 \uac04\ub2e8\ud788 \ub9d0\ud574 \uac01 \uacc4\uce35\uc744 \ub9e4\uac1c\ubcc0\uc218 \ud150\uc11c\ub85c \ub9e4\ud551\ub418\ub294\nPython \uc0ac\uc804(dict) \uac1d\uccb4\uc785\ub2c8\ub2e4.\n\n\uac1c\uc694\n------------\n``state_dict`` \ub294 PyTorch\uc5d0\uc11c \ubaa8\ub378\uc744 \uc800\uc7a5\ud558\uac70\ub098 \ubd88\ub7ec\uc624\ub294 \ub370 \uad00\uc2ec\uc774\n\uc788\ub2e4\uba74 \ud544\uc218\uc801\uc778 \ud56d\ubaa9\uc785\ub2c8\ub2e4.\n``state_dict`` \uac1d\uccb4\ub294 Python \uc0ac\uc804\uc774\uae30 \ub54c\ubb38\uc5d0 \uc27d\uac8c \uc800\uc7a5, \uc5c5\ub370\uc774\ud2b8,\n\ubcc0\uacbd \ubc0f \ubcf5\uc6d0\ud560 \uc218 \uc788\uc73c\uba70, \uc774\ub294 PyTorch \ubaa8\ub378\uacfc \uc635\ud2f0\ub9c8\uc774\uc800\uc5d0 \uc5c4\uccad\ub09c\n\ubaa8\ub4c8\uc131(modularity)\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.\n\uc774 \ub54c, \ud559\uc2b5 \uac00\ub2a5\ud55c \ub9e4\uac1c\ubcc0\uc218\ub97c \uac16\ub294 \uacc4\uce35(\ud569\uc131\uacf1 \uacc4\uce35, \uc120\ud615 \uacc4\uce35 \ub4f1)\n\ubc0f \ub4f1\ub85d\ub41c \ubc84\ud37c\ub4e4(batchnorm\uc758 running_mean)\ub9cc \ubaa8\ub378\uc758 ``state_dict``\n \ud56d\ubaa9\uc744 \uac00\uc9c4\ub2e4\ub294 \uc810\uc5d0 \uc720\uc758\ud558\uc2dc\uae30 \ubc14\ub78d\ub2c8\ub2e4. \uc635\ud2f0\ub9c8\uc774\uc800 \uac1d\uccb4\n( ``torch.optim`` ) \ub610\ud55c \uc635\ud2f0\ub9c8\uc774\uc800\uc758 \uc0c1\ud0dc \ubfd0\ub9cc \uc544\ub2c8\ub77c \uc0ac\uc6a9\ub41c\n\ud558\uc774\ud37c \ub9e4\uac1c\ubcc0\uc218 (Hyperparameter) \uc815\ubcf4\uac00 \ud3ec\ud568\ub41c ``state_dict`` \uc744\n\uac16\uc2b5\ub2c8\ub2e4.\n\ub808\uc2dc\ud53c\uc5d0\uc11c ``state_dict`` \uc774 \uac04\ub2e8\ud55c \ubaa8\ub378\uc5d0\uc11c \uc5b4\ub5bb\uac8c \uc0ac\uc6a9\ub418\ub294\uc9c0\n\uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc124\uc815\n----------\n\uc2dc\uc791\ud558\uae30 \uc804\uc5d0 ``torch`` \uac00 \uc5c6\ub2e4\uba74 \uc124\uce58\ud574\uc57c \ud569\ub2c8\ub2e4.\n\n::\n\n pip install torchaudio\n" ] }, { @@ -114,7 +114,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/5edaebfc06ec3968b8c1da100da2253d/sequence_models_tutorial.ipynb b/docs/_downloads/5edaebfc06ec3968b8c1da100da2253d/sequence_models_tutorial.ipynb index 5bd07626a..2928c05cc 100644 --- a/docs/_downloads/5edaebfc06ec3968b8c1da100da2253d/sequence_models_tutorial.ipynb +++ b/docs/_downloads/5edaebfc06ec3968b8c1da100da2253d/sequence_models_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nSequence Models and Long-Short Term Memory Networks\n===================================================\n\nAt this point, we have seen various feed-forward networks. That is,\nthere is no state maintained by the network at all. This might not be\nthe behavior we want. Sequence models are central to NLP: they are\nmodels where there is some sort of dependence through time between your\ninputs. The classical example of a sequence model is the Hidden Markov\nModel for part-of-speech tagging. Another example is the conditional\nrandom field.\n\nA recurrent neural network is a network that maintains some kind of\nstate. For example, its output could be used as part of the next input,\nso that information can propogate along as the network passes over the\nsequence. In the case of an LSTM, for each element in the sequence,\nthere is a corresponding *hidden state* $h_t$, which in principle\ncan contain information from arbitrary points earlier in the sequence.\nWe can use the hidden state to predict words in a language model,\npart-of-speech tags, and a myriad of other things.\n\n\nLSTM's in Pytorch\n~~~~~~~~~~~~~~~~~\n\nBefore getting to the example, note a few things. Pytorch's LSTM expects\nall of its inputs to be 3D tensors. The semantics of the axes of these\ntensors is important. The first axis is the sequence itself, the second\nindexes instances in the mini-batch, and the third indexes elements of\nthe input. We haven't discussed mini-batching, so lets just ignore that\nand assume we will always have just 1 dimension on the second axis. If\nwe want to run the sequence model over the sentence \"The cow jumped\",\nour input should look like\n\n\\begin{align}\\begin{bmatrix}\n \\overbrace{q_\\text{The}}^\\text{row vector} \\\\\n q_\\text{cow} \\\\\n q_\\text{jumped}\n \\end{bmatrix}\\end{align}\n\nExcept remember there is an additional 2nd dimension with size 1.\n\nIn addition, you could go through the sequence one at a time, in which\ncase the 1st axis will have size 1 also.\n\nLet's see a quick example.\n\n" + "\nSequence Models and Long-Short Term Memory Networks\n===================================================\n\nAt this point, we have seen various feed-forward networks. That is,\nthere is no state maintained by the network at all. This might not be\nthe behavior we want. Sequence models are central to NLP: they are\nmodels where there is some sort of dependence through time between your\ninputs. The classical example of a sequence model is the Hidden Markov\nModel for part-of-speech tagging. Another example is the conditional\nrandom field.\n\nA recurrent neural network is a network that maintains some kind of\nstate. For example, its output could be used as part of the next input,\nso that information can propogate along as the network passes over the\nsequence. In the case of an LSTM, for each element in the sequence,\nthere is a corresponding *hidden state* $h_t$, which in principle\ncan contain information from arbitrary points earlier in the sequence.\nWe can use the hidden state to predict words in a language model,\npart-of-speech tags, and a myriad of other things.\n\n\nLSTM's in Pytorch\n~~~~~~~~~~~~~~~~~\n\nBefore getting to the example, note a few things. Pytorch's LSTM expects\nall of its inputs to be 3D tensors. The semantics of the axes of these\ntensors is important. The first axis is the sequence itself, the second\nindexes instances in the mini-batch, and the third indexes elements of\nthe input. We haven't discussed mini-batching, so lets just ignore that\nand assume we will always have just 1 dimension on the second axis. If\nwe want to run the sequence model over the sentence \"The cow jumped\",\nour input should look like\n\n\\begin{align}\\begin{bmatrix}\n \\overbrace{q_\\text{The}}^\\text{row vector} \\\\\n q_\\text{cow} \\\\\n q_\\text{jumped}\n \\end{bmatrix}\\end{align}\n\nExcept remember there is an additional 2nd dimension with size 1.\n\nIn addition, you could go through the sequence one at a time, in which\ncase the 1st axis will have size 1 also.\n\nLet's see a quick example.\n" ] }, { @@ -118,7 +118,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/5f81194dd43910d586578638f83205a3/dcgan_faces_tutorial.ipynb b/docs/_downloads/5f81194dd43910d586578638f83205a3/dcgan_faces_tutorial.ipynb index a45b8b6b6..540cfe3e9 100644 --- a/docs/_downloads/5f81194dd43910d586578638f83205a3/dcgan_faces_tutorial.ipynb +++ b/docs/_downloads/5f81194dd43910d586578638f83205a3/dcgan_faces_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nDCGAN Tutorial\n==============\n\n**Author**: `Nathan Inkawhich `__\n\n\n" + "\nDCGAN Tutorial\n==============\n\n**Author**: `Nathan Inkawhich `__\n" ] }, { @@ -283,7 +283,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/6042bacf7948939030769777afe22e55/data_loading_tutorial.py b/docs/_downloads/6042bacf7948939030769777afe22e55/data_loading_tutorial.py index a607d3104..8799d24ce 100644 --- a/docs/_downloads/6042bacf7948939030769777afe22e55/data_loading_tutorial.py +++ b/docs/_downloads/6042bacf7948939030769777afe22e55/data_loading_tutorial.py @@ -187,11 +187,12 @@ def __getitem__(self, idx): # 그러므로 우리는 신경망에 주기 전에 처리할 과정을 작성해야 합니다. # # 3가지의 transforms 을 만들어 봅시다: -# - ``Rescale``: 이미지의 크기를 조절합니다. -# - ``RandomCrop``: 이미지를 무작위로 자릅니다. -# 이것을 data augmentation이라 합니다. -# - ``ToTensor``: numpy 이미지에서 torch 이미지로 변경합니다. -# (축변환이 필요합니다) +# +# - ``Rescale``: 이미지의 크기를 조절합니다. +# - ``RandomCrop``: 이미지를 무작위로 자릅니다. +# 이것을 data augmentation이라 합니다. +# - ``ToTensor``: numpy 이미지에서 torch 이미지로 변경합니다. +# (축변환이 필요합니다) # # 간단한 함수대신에 호출 할 수 있는 클래스로 작성 합니다. # 이렇게 한다면, 클래스가 호출 될 때마다 전이(Transform)의 매개변수가 전달 되지 않아도 됩니다. diff --git a/docs/_downloads/61a76849444a0a65d843361c26d1de16/Intro_to_TorchScript_tutorial.ipynb b/docs/_downloads/61a76849444a0a65d843361c26d1de16/Intro_to_TorchScript_tutorial.ipynb index 5a7628307..d8a6cc207 100644 --- a/docs/_downloads/61a76849444a0a65d843361c26d1de16/Intro_to_TorchScript_tutorial.ipynb +++ b/docs/_downloads/61a76849444a0a65d843361c26d1de16/Intro_to_TorchScript_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nTorchScript \uc18c\uac1c\n===========================\n\n**Author**: James Reed (jamesreed@fb.com), Michael Suo (suo@fb.com), rev2\n\n**\ubc88\uc5ed**: `\uac15\uc900\ud601 `_\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc740 C++\uc640 \uac19\uc740 \uace0\uc131\ub2a5 \ud658\uacbd\uc5d0\uc11c \uc2e4\ud589\ub420 \uc218 \uc788\ub294\nPyTorch \ubaa8\ub378(``nn.Module`` \uc758 \ud558\uc704\ud074\ub798\uc2a4)\uc758 \uc911\uac04 \ud45c\ud604\uc778\nTorchScript\uc5d0 \ub300\ud55c \uc18c\uac1c\uc785\ub2c8\ub2e4.\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 \ub2e4\uc74c\uc744 \ub2e4\ub8f0 \uac83\uc785\ub2c8\ub2e4:\n\n1. \ub2e4\uc74c\uc744 \ud3ec\ud568\ud55c PyTorch\uc758 \ubaa8\ub378 \uc81c\uc791\uc758 \uae30\ubcf8:\n\n- \ubaa8\ub4c8(Modules)\n- ``forward`` \ud568\uc218 \uc815\uc758\ud558\uae30\n- \ubaa8\ub4c8\uc744 \uacc4\uce35 \uad6c\uc870\ub85c \uad6c\uc131\ud558\uae30\n\n2. PyTorch \ubaa8\ub4c8\uc744 \uace0\uc131\ub2a5 \ubc30\ud3ec \ub7f0\ud0c0\uc784\uc778 TorchScript\ub85c \ubcc0\ud658\ud558\ub294 \ud2b9\uc815 \ubc29\ubc95\n\n- \uae30\uc874 \ubaa8\ub4c8 \ucd94\uc801\ud558\uae30\n- \uc2a4\ud06c\ub9bd\ud2b8\ub97c \uc0ac\uc6a9\ud558\uc5ec \ubaa8\ub4c8\uc744 \uc9c1\uc811 \ucef4\ud30c\uc77c\ud558\uae30\n- \ub450 \uac00\uc9c0 \uc811\uadfc \ubc29\ubc95\uc744 \uad6c\uc131\ud558\ub294 \ubc29\ubc95\n- TorchScript \ubaa8\ub4c8 \uc800\uc7a5 \ubc0f \ubd88\ub7ec\uc624\uae30\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc744 \uc644\ub8cc\ud55c \ud6c4\uc5d0\ub294\n`\ub2e4\uc74c \ud559\uc2b5\uc11c `_\n\ub97c \ud1b5\ud574 C++\uc5d0\uc11c TorchScript \ubaa8\ub378\uc744 \uc2e4\uc81c\ub85c \ud638\ucd9c\ud558\ub294 \uc608\uc81c\ub97c \uc548\ub0b4\ud569\ub2c8\ub2e4.\n\n\n" + "\nTorchScript \uc18c\uac1c\n===========================\n\n**Author**: James Reed (jamesreed@fb.com), Michael Suo (suo@fb.com), rev2\n\n**\ubc88\uc5ed**: `\uac15\uc900\ud601 `_\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc740 C++\uc640 \uac19\uc740 \uace0\uc131\ub2a5 \ud658\uacbd\uc5d0\uc11c \uc2e4\ud589\ub420 \uc218 \uc788\ub294\nPyTorch \ubaa8\ub378(``nn.Module`` \uc758 \ud558\uc704\ud074\ub798\uc2a4)\uc758 \uc911\uac04 \ud45c\ud604\uc778\nTorchScript\uc5d0 \ub300\ud55c \uc18c\uac1c\uc785\ub2c8\ub2e4.\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 \ub2e4\uc74c\uc744 \ub2e4\ub8f0 \uac83\uc785\ub2c8\ub2e4:\n\n1. \ub2e4\uc74c\uc744 \ud3ec\ud568\ud55c PyTorch\uc758 \ubaa8\ub378 \uc81c\uc791\uc758 \uae30\ubcf8:\n\n- \ubaa8\ub4c8(Modules)\n- ``forward`` \ud568\uc218 \uc815\uc758\ud558\uae30\n- \ubaa8\ub4c8\uc744 \uacc4\uce35 \uad6c\uc870\ub85c \uad6c\uc131\ud558\uae30\n\n2. PyTorch \ubaa8\ub4c8\uc744 \uace0\uc131\ub2a5 \ubc30\ud3ec \ub7f0\ud0c0\uc784\uc778 TorchScript\ub85c \ubcc0\ud658\ud558\ub294 \ud2b9\uc815 \ubc29\ubc95\n\n- \uae30\uc874 \ubaa8\ub4c8 \ucd94\uc801\ud558\uae30\n- \uc2a4\ud06c\ub9bd\ud2b8\ub97c \uc0ac\uc6a9\ud558\uc5ec \ubaa8\ub4c8\uc744 \uc9c1\uc811 \ucef4\ud30c\uc77c\ud558\uae30\n- \ub450 \uac00\uc9c0 \uc811\uadfc \ubc29\ubc95\uc744 \uad6c\uc131\ud558\ub294 \ubc29\ubc95\n- TorchScript \ubaa8\ub4c8 \uc800\uc7a5 \ubc0f \ubd88\ub7ec\uc624\uae30\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc744 \uc644\ub8cc\ud55c \ud6c4\uc5d0\ub294\n`\ub2e4\uc74c \ud559\uc2b5\uc11c `_\n\ub97c \ud1b5\ud574 C++\uc5d0\uc11c TorchScript \ubaa8\ub378\uc744 \uc2e4\uc81c\ub85c \ud638\ucd9c\ud558\ub294 \uc608\uc81c\ub97c \uc548\ub0b4\ud569\ub2c8\ub2e4.\n" ] }, { @@ -294,7 +294,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/63192d95d276e1eaaa08642db0e4ef7f/profiler.py b/docs/_downloads/63192d95d276e1eaaa08642db0e4ef7f/profiler.py index b32333021..14ae58b50 100644 --- a/docs/_downloads/63192d95d276e1eaaa08642db0e4ef7f/profiler.py +++ b/docs/_downloads/63192d95d276e1eaaa08642db0e4ef7f/profiler.py @@ -1,20 +1,18 @@ """ -PyTorch Profiler +PyTorch 프로파일러(Profiler) ==================================== -This recipe explains how to use PyTorch profiler and measure the time and -memory consumption of the model's operators. +이 레시피에서는 어떻게 PyTorch 프로파일러를 사용하는지, 그리고 모델의 연산자들이 소비하는 메모리와 시간을 측정하는 방법을 살펴보겠습니다. -Introduction +개요 ------------ -PyTorch includes a simple profiler API that is useful when user needs -to determine the most expensive operators in the model. +PyTorch는 사용자가 모델 내의 연산 비용이 큰(expensive) 연산자들이 무엇인지 알고싶을 때 유용하게 사용할 수 있는 간단한 프로파일러 API를 포함하고 있습니다. -In this recipe, we will use a simple Resnet model to demonstrate how to -use profiler to analyze model performance. +이 레시피에서는 모델의 성능(performance)을 분석하려고 할 때 어떻게 프로파일러를 사용해야 하는지를 보여주기 위해 간단한 ResNet 모델을 사용하겠습니다. + +설정(Setup) +------------- +``torch`` 와 ``torchvision`` 을 설치하기 위해서 아래의 커맨드를 입력합니다: -Setup ------ -To install ``torch`` and ``torchvision`` use the following command: :: @@ -25,20 +23,20 @@ ###################################################################### -# Steps -# ----- +# 단계(Steps) +# ------------- # -# 1. Import all necessary libraries -# 2. Instantiate a simple Resnet model -# 3. Use profiler to analyze execution time -# 4. Use profiler to analyze memory consumption -# 5. Using tracing functionality +# 1. 필요한 라이브러리들 불러오기 +# 2. 간단한 ResNet 모델 인스턴스화 하기 +# 3. 프로파일러를 사용하여 실행시간 분석하기 +# 4. 프로파일러를 사용하여 메모리 소비 분석하기 +# 5. 추적기능 사용하기 # -# 1. Import all necessary libraries +# 1. 필요한 라이브러리들 불러오기 # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ # -# In this recipe we will use ``torch``, ``torchvision.models`` -# and ``profiler`` modules: +# 이 레시피에서는 ``torch`` 와 ``torchvision.models``, +# 그리고 ``profiler`` 모듈을 사용합니다: # import torch @@ -47,53 +45,53 @@ ###################################################################### -# 2. Instantiate a simple Resnet model +# 2. 간단한 ResNet 모델 인스턴스화 하기 # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ # -# Let's create an instance of a Resnet model and prepare an input -# for it: +# ResNet 모델 인스턴스를 만들고 입력값을 +# 준비합니다 : # model = models.resnet18() inputs = torch.randn(5, 3, 224, 224) ###################################################################### -# 3. Use profiler to analyze execution time +# 3. 프로파일러를 사용하여 실행시간 분석하기 # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ # -# PyTorch profiler is enabled through the context manager and accepts -# a number of parameters, some of the most useful are: +# PyTorch 프로파일러는 컨텍스트 메니저(context manager)를 통해 활성화되고, +# 여러 매개변수를 받을 수 있습니다. 유용한 몇 가지 매개변수는 다음과 같습니다: # -# - ``record_shapes`` - whether to record shapes of the operator inputs; -# - ``profile_memory`` - whether to report amount of memory consumed by -# model's Tensors; -# - ``use_cuda`` - whether to measure execution time of CUDA kernels. +# - ``record_shapes`` - 연사자 입력(input)의 shape을 기록할지 여부; +# - ``profile_memory`` - 모델의 텐서(Tensor)들이 소비하는 메모리 양을 보고(report)할지 여부; +# - ``use_cuda`` - CUDA 커널의 실행시간을 측정할지 여부; # -# Let's see how we can use profiler to analyze the execution time: +# 프로파일러를 사용하여 어떻게 실행시간을 분석하는지 보겠습니다: with profiler.profile(record_shapes=True) as prof: with profiler.record_function("model_inference"): model(inputs) ###################################################################### -# Note that we can use ``record_function`` context manager to label -# arbitrary code ranges with user provided names -# (``model_inference`` is used as a label in the example above). -# Profiler allows one to check which operators were called during the -# execution of a code range wrapped with a profiler context manager. -# If multiple profiler ranges are active at the same time (e.g. in -# parallel PyTorch threads), each profiling context manager tracks only -# the operators of its corresponding range. -# Profiler also automatically profiles the async tasks launched -# with ``torch.jit._fork`` and (in case of a backward pass) -# the backward pass operators launched with ``backward()`` call. -# -# Let's print out the stats for the execution above: +# ``record_function`` 컨텍스트 관리자를 사용하여 임의의 코드 범위에 +# 사용자가 지정한 이름으로 레이블(label)을 표시할 수 있습니다. +# (위 예제에서는 ``model_inference`` 를 레이블로 사용했습니다.) +# 프로파일러를 사용하면 프로파일러 컨텍스트 관리자로 감싸진(wrap) 코드 범위를 +# 실행하는 동안 어떤 연산자들이 호출되었는지 확인할 수 있습니다. +# +# 만약 여러 프로파일러의 범위가 동시에 활성화된 경우(예. PyTorch 쓰레드가 병렬로 +# 실행 중인 경우), 각 프로파일링 컨텍스트 관리자는 각각의 범위 내의 연산자들만 +# 추적(track)합니다. +# 프로파일러는 또한 ``torch.jit._fork`` 로 실행된 비동기 작업과 +# (역전파 단계의 경우) ``backward()`` 의 호출로 실행된 역전파 연산자들도 +# 자동으로 프로파일링합니다. +# +# 위 코드를 실행한 통계를 출력해보겠습니다: print(prof.key_averages().table(sort_by="cpu_time_total", row_limit=10)) ###################################################################### -# The output will look like (omitting some columns): +# (몇몇 열을 제외하고) 출력값이 이렇게 보일 것입니다: # ------------------------- -------------- ---------- ------------ --------- # Name Self CPU total CPU total CPU time avg # Calls @@ -111,16 +109,18 @@ # ------------------------- -------------- ---------- ------------ --------- ###################################################################### -# Here we see that, as expected, most of the time is spent in convolution (and specifically in ``mkldnn_convolution`` -# for PyTorch compiled with MKL-DNN support). -# Note the difference between self cpu time and cpu time - operators can call other operators, self cpu time exludes time -# spent in children operator calls, while total cpu time includes it. +# 예상했던 대로, 대부분의 시간이 합성곱(convolution) 연산(특히 MKL-DNN을 지원하도록 +# 컴파일된 PyTorch의 경우에는 ``mkldnn_convolution`` )에서 소요되는 것을 확인할 수 있습니다. +# (결과 열들 중) Self CPU time과 CPU time의 차이에 유의해야 합니다 - +# 연산자는 다른 연산자들을 호출할 수 있으며, Self CPU time에는 하위(child) 연산자 호출에서 발생한 +# 시간을 제외해서, Totacl CPU time에는 포함해서 표시합니다. # -# To get a finer granularity of results and include operator input shapes, pass ``group_by_input_shape=True``: +# 보다 세부적인 결과 정보 및 연산자의 입력 shape을 함께 보려면 ``group_by_input_shape=True`` 를 +# 인자로 전달하면 됩니다: print(prof.key_averages(group_by_input_shape=True).table(sort_by="cpu_time_total", row_limit=10)) -# (omitting some columns) +# (몇몇 열은 제외하였습니다) # ------------------------- ----------- -------- ------------------------------------- # Name CPU total # Calls Input Shapes # ------------------------- ----------- -------- ------------------------------------- @@ -138,21 +138,21 @@ ###################################################################### -# 4. Use profiler to analyze memory consumption +# 4. 프로파일러를 사용하여 메모리 소비 분석하기 # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ # -# PyTorch profiler can also show the amount of memory (used by the model's tensors) -# that was allocated (or released) during the execution of the model's operators. -# In the output below, 'self' memory corresponds to the memory allocated (released) -# by the operator, excluding the children calls to the other operators. -# To enable memory profiling functionality pass ``profile_memory=True``. +# PyTorch 프로파일러는 모델의 연산자들을 실행하며 (모델의 텐서들이 사용하며) 할당(또는 해제)한 +# 메모리의 양도 표시할 수 있습니다. +# 아래 출력 결과에서 'Self' memory는 해당 연산자에 의해 호출된 하위(child) 연산자들을 제외한, +# 연산자 자체에 할당(해제)된 메모리에 해당합니다. +# 메모리 프로파일링 기능을 활성화하려면 ``profile_memory=True`` 를 인자로 전달하면 됩니다. with profiler.profile(profile_memory=True, record_shapes=True) as prof: model(inputs) print(prof.key_averages().table(sort_by="self_cpu_memory_usage", row_limit=10)) -# (omitting some columns) +# (몇몇 열은 제외하였습니다) # --------------------------- --------------- --------------- --------------- # Name CPU Mem Self CPU Mem Number of Calls # --------------------------- --------------- --------------- --------------- @@ -165,7 +165,7 @@ print(prof.key_averages().table(sort_by="cpu_memory_usage", row_limit=10)) -# (omitting some columns) +# (몇몇 열은 제외하였습니다) # --------------------------- --------------- --------------- --------------- # Name CPU Mem Self CPU Mem Number of Calls # --------------------------- --------------- --------------- --------------- @@ -187,10 +187,10 @@ # --------------------------- --------------- --------------- --------------- ###################################################################### -# 5. Using tracing functionality +# 5. 추적기능 사용하기 # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ # -# Profiling results can be outputted as a .json trace file: +# 프로파일링 결과는 .json 형태의 추적 파일(trace file)로 출력할 수 있습니다: with profiler.profile() as prof: with profiler.record_function("model_inference"): @@ -199,17 +199,17 @@ prof.export_chrome_trace("trace.json") ###################################################################### -# User can examine the sequence of profiled operators after loading the trace file -# in Chrome (``chrome://tracing``): +# 사용자는 Chrome 브라우저( ``chrome://tracing`` )에서 추적 파일을 불러와 +# 프로파일된 일련의 연산자들을 검토해볼 수 있습니다: # # .. image:: ../../_static/img/trace_img.png # :scale: 25 % ###################################################################### -# Learn More -# ---------- +# 더 알아보기 +# ------------- # -# Take a look at the following tutorial to learn how to visualize your model with TensorBoard: +# 다음의 튜토리얼을 통해 Tensorboard를 사용하여 모델 시각화하는 방법을 살펴보세요: # -# - `Visualizing models, data, and training with TensorBoard `_ tutorial +# - :doc:`/intermediate/tensorboard_tutorial` # diff --git a/docs/_downloads/642248c95070825e7ac912504a919140/Captum_Recipe.ipynb b/docs/_downloads/642248c95070825e7ac912504a919140/Captum_Recipe.ipynb index 850659bce..b1e3f783e 100644 --- a/docs/_downloads/642248c95070825e7ac912504a919140/Captum_Recipe.ipynb +++ b/docs/_downloads/642248c95070825e7ac912504a919140/Captum_Recipe.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nCaptum\uc744 \uc0ac\uc6a9\ud558\uc5ec \ubaa8\ub378 \ud574\uc11d\ud558\uae30\n===================================\n\n**\ubc88\uc5ed**: `\uc815\uc7ac\ubbfc `_\n\n" + "\nCaptum\uc744 \uc0ac\uc6a9\ud558\uc5ec \ubaa8\ub378 \ud574\uc11d\ud558\uae30\n===================================\n\n**\ubc88\uc5ed**: `\uc815\uc7ac\ubbfc `_\n" ] }, { @@ -152,7 +152,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/67710f3d40ca3863323ce02dddbf8618/save_load_across_devices.ipynb b/docs/_downloads/67710f3d40ca3863323ce02dddbf8618/save_load_across_devices.ipynb index b76d25ebc..380a07590 100644 --- a/docs/_downloads/67710f3d40ca3863323ce02dddbf8618/save_load_across_devices.ipynb +++ b/docs/_downloads/67710f3d40ca3863323ce02dddbf8618/save_load_across_devices.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nPyTorch\uc5d0\uc11c \ub2e4\uc591\ud55c \uc7a5\uce58 \uac04 \ubaa8\ub378\uc744 \uc800\uc7a5\ud558\uace0 \ubd88\ub7ec\uc624\uae30\n===================================================\n\n\ub2e4\uc591\ud55c \uc7a5\uce58(device)\uc5d0\uc11c \ub2f9\uc2e0\uc758 \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc744 \uc800\uc7a5\ud558\uac70\ub098 \ubd88\ub7ec\uc624\uace0 \uc2f6\uc740 \n\uacbd\uc6b0\uac00 \uc0dd\uae38 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uac1c\uc694\n------------\n\nPyTorch\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc7a5\uce58 \uac04\uc758 \ubaa8\ub378\uc744 \uc800\uc7a5\ud558\uac70\ub098 \ubd88\ub7ec\uc624\ub294 \uac83\uc740 \ube44\uad50\uc801 \uac04\ub2e8\ud569\ub2c8\ub2e4.\n\uc774\ubc88 \ub808\uc2dc\ud53c\uc5d0\uc11c\ub294, CPU\uc640 GPU\uc5d0\uc11c \ubaa8\ub378\uc744 \uc800\uc7a5\ud558\uace0 \ubd88\ub7ec\uc624\ub294 \ubc29\ubc95\uc744 \uc2e4\ud5d8\ud560 \uac83\uc785\ub2c8\ub2e4.\n\n\uc124\uc815\n-----\n\n\uc774\ubc88 \ub808\uc2dc\ud53c\uc5d0\uc11c \ubaa8\ub4e0 \ucf54\ub4dc \ube14\ub85d\uc774 \uc81c\ub300\ub85c \uc2e4\ud589\ub418\uac8c \ud558\ub824\uba74, \n\uc6b0\uc120 \ub7f0\ud0c0\uc784(runtime) \uc124\uc815\uc744 \"GPU\"\ub098 \ub354 \ub192\uac8c \uc9c0\uc815\ud574\uc8fc\uc5b4\uc57c \ud569\ub2c8\ub2e4. \n\uc774\ud6c4, \uc544\ub798\uc640 \uac19\uc774 ``torch``\ub97c \uc124\uce58\ud574\uc57c PyTorch\ub97c \uc0ac\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n::\n\n pip install torch\n\n\n" + "\nPyTorch\uc5d0\uc11c \ub2e4\uc591\ud55c \uc7a5\uce58 \uac04 \ubaa8\ub378\uc744 \uc800\uc7a5\ud558\uace0 \ubd88\ub7ec\uc624\uae30\n===================================================\n\n\ub2e4\uc591\ud55c \uc7a5\uce58(device)\uc5d0\uc11c \ub2f9\uc2e0\uc758 \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc744 \uc800\uc7a5\ud558\uac70\ub098 \ubd88\ub7ec\uc624\uace0 \uc2f6\uc740 \n\uacbd\uc6b0\uac00 \uc0dd\uae38 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uac1c\uc694\n------------\n\nPyTorch\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc7a5\uce58 \uac04\uc758 \ubaa8\ub378\uc744 \uc800\uc7a5\ud558\uac70\ub098 \ubd88\ub7ec\uc624\ub294 \uac83\uc740 \ube44\uad50\uc801 \uac04\ub2e8\ud569\ub2c8\ub2e4.\n\uc774\ubc88 \ub808\uc2dc\ud53c\uc5d0\uc11c\ub294, CPU\uc640 GPU\uc5d0\uc11c \ubaa8\ub378\uc744 \uc800\uc7a5\ud558\uace0 \ubd88\ub7ec\uc624\ub294 \ubc29\ubc95\uc744 \uc2e4\ud5d8\ud560 \uac83\uc785\ub2c8\ub2e4.\n\n\uc124\uc815\n-----\n\n\uc774\ubc88 \ub808\uc2dc\ud53c\uc5d0\uc11c \ubaa8\ub4e0 \ucf54\ub4dc \ube14\ub85d\uc774 \uc81c\ub300\ub85c \uc2e4\ud589\ub418\uac8c \ud558\ub824\uba74, \n\uc6b0\uc120 \ub7f0\ud0c0\uc784(runtime) \uc124\uc815\uc744 \"GPU\"\ub098 \ub354 \ub192\uac8c \uc9c0\uc815\ud574\uc8fc\uc5b4\uc57c \ud569\ub2c8\ub2e4. \n\uc774\ud6c4, \uc544\ub798\uc640 \uac19\uc774 ``torch``\ub97c \uc124\uce58\ud574\uc57c PyTorch\ub97c \uc0ac\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n::\n\n pip install torch\n" ] }, { @@ -150,7 +150,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/695ce7a76ce07582023a85d63a717ce2/saving_and_loading_models_for_inference.ipynb b/docs/_downloads/695ce7a76ce07582023a85d63a717ce2/saving_and_loading_models_for_inference.ipynb index 5432d037e..f08d0146b 100644 --- a/docs/_downloads/695ce7a76ce07582023a85d63a717ce2/saving_and_loading_models_for_inference.ipynb +++ b/docs/_downloads/695ce7a76ce07582023a85d63a717ce2/saving_and_loading_models_for_inference.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nPyTorch\uc5d0\uc11c \ucd94\ub860(inference)\uc744 \uc704\ud574 \ubaa8\ub378 \uc800\uc7a5\ud558\uae30 & \ubd88\ub7ec\uc624\uae30\n================================================================\n\nPyTorch\uc5d0\uc11c\ub294 \ucd94\ub860(inference)\uc744 \uc704\ud574 \ubaa8\ub378\uc744 \uc800\uc7a5\ud558\uace0 \ubd88\ub7ec\uc624\ub294\ub370 2\uac00\uc9c0 \uc811\uadfc\ubc95\uc774\n\uc788\uc2b5\ub2c8\ub2e4. \uccab\ubc88\uc9f8\ub294 ``state_dict`` \ub97c \uc800\uc7a5\ud558\uace0 \ubd88\ub7ec\uc624\ub294 \uac83\uc774\uace0, \ub450\ubc88\uc9f8\ub294 \uc804\uccb4\n\ubaa8\ub378\uc744 \uc800\uc7a5\ud558\ub294 \uac83\uc785\ub2c8\ub2e4.\n\n\uac1c\uc694\n------------\ntorch.save() \ud568\uc218\ub97c \uc0ac\uc6a9\ud558\uc5ec \ubaa8\ub378\uc758 ``state_dict`` \ub97c \uc800\uc7a5\ud558\uba74 \uc774\ud6c4\uc5d0 \ubaa8\ub378\uc744\n\ubd88\ub7ec\uc62c \ub54c \uc720\uc5f0\ud568\uc744 \ud06c\uac8c \uc0b4\ub9b4 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \ud559\uc2b5\ub41c \ubaa8\ub378\uc758 \ub9e4\uac1c\ubcc0\uc218(parameter)\ub9cc\uc744\n\uc800\uc7a5\ud558\uba74\ub418\ubbc0\ub85c \ubaa8\ub378 \uc800\uc7a5 \uc2dc\uc5d0 \uad8c\uc7a5\ud558\ub294 \ubc29\ubc95\uc785\ub2c8\ub2e4. \ubaa8\ub378 \uc804\uccb4\ub97c \uc800\uc7a5\ud558\uace0 \ubd88\ub7ec\uc62c\n\ub54c\uc5d0\ub294 Python\uc758 `pickle `__ \ubaa8\ub4c8\uc744\n\uc0ac\uc6a9\ud558\uc5ec \uc804\uccb4 \ubaa8\ub4c8\uc744 \uc800\uc7a5\ud569\ub2c8\ub2e4. \uc774 \ubc29\uc2dd\uc740 \uc9c1\uad00\uc801\uc778 \ubb38\ubc95\uc744 \uc0ac\uc6a9\ud558\uba70 \ucf54\ub4dc\uc758 \uc591\ub3c4\n\uc801\uc2b5\ub2c8\ub2e4. \uc774 \ubc29\uc2dd\uc758 \ub2e8\uc810\uc740 \uc9c1\ub82c\ud654(serialized)\ub41c \ub370\uc774\ud130\uac00 \ubaa8\ub378\uc744 \uc800\uc7a5\ud560 \ub54c \uc0ac\uc6a9\ud55c\n\ud2b9\uc815 \ud074\ub798\uc2a4 \ubc0f \ub514\ub809\ud1a0\ub9ac \uad6c\uc870\uc5d0 \uc885\uc18d(bind)\ub41c\ub2e4\ub294 \uac83\uc785\ub2c8\ub2e4. \uadf8 \uc774\uc720\ub294 pickle\uc774\n\ubaa8\ub378 \ud074\ub798\uc2a4 \uc790\uccb4\ub97c \uc800\uc7a5\ud558\uc9c0 \uc54a\uae30 \ub54c\ubb38\uc785\ub2c8\ub2e4. \uc624\ud788\ub824 \ubd88\ub7ec\uc62c \ub54c \uc0ac\uc6a9\ub418\ub294 \ud074\ub798\uc2a4\uac00\n\ud3ec\ud568\ub41c \ud30c\uc77c\uc758 \uacbd\ub85c\ub97c \uc800\uc7a5\ud569\ub2c8\ub2e4. \uc774 \ub54c\ubb38\uc5d0 \uc791\uc131\ud55c \ucf54\ub4dc\uac00 \ub2e4\ub978 \ud504\ub85c\uc81d\ud2b8\uc5d0\uc11c\n\uc0ac\uc6a9\ub418\uac70\ub098 \ub9ac\ud329\ud1a0\ub9c1\uc744 \uac70\uce58\ub294 \ub4f1\uc758 \uacfc\uc815\uc5d0\uc11c \ub3d9\uc791\ud558\uc9c0 \uc54a\uc744 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc774 \ub808\uc2dc\ud53c\uc5d0\uc11c\ub294\n\ucd94\ub860\uc744 \uc704\ud574 \ubaa8\ub378\uc744 \uc800\uc7a5\ud558\uace0 \ubd88\ub7ec\uc624\ub294 \ub450 \uac00\uc9c0 \ubc29\ubc95 \ubaa8\ub450\ub97c \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc124\uc815\n----------\n\uc2dc\uc791\ud558\uae30 \uc804\uc5d0 ``torch`` \uac00 \uc5c6\ub2e4\uba74 \uc124\uce58\ud574\uc57c \ud569\ub2c8\ub2e4.\n\n\n::\n\n pip install torch\n\n\n\n" + "\nPyTorch\uc5d0\uc11c \ucd94\ub860(inference)\uc744 \uc704\ud574 \ubaa8\ub378 \uc800\uc7a5\ud558\uae30 & \ubd88\ub7ec\uc624\uae30\n================================================================\n\nPyTorch\uc5d0\uc11c\ub294 \ucd94\ub860(inference)\uc744 \uc704\ud574 \ubaa8\ub378\uc744 \uc800\uc7a5\ud558\uace0 \ubd88\ub7ec\uc624\ub294\ub370 2\uac00\uc9c0 \uc811\uadfc\ubc95\uc774\n\uc788\uc2b5\ub2c8\ub2e4. \uccab\ubc88\uc9f8\ub294 ``state_dict`` \ub97c \uc800\uc7a5\ud558\uace0 \ubd88\ub7ec\uc624\ub294 \uac83\uc774\uace0, \ub450\ubc88\uc9f8\ub294 \uc804\uccb4\n\ubaa8\ub378\uc744 \uc800\uc7a5\ud558\ub294 \uac83\uc785\ub2c8\ub2e4.\n\n\uac1c\uc694\n------------\ntorch.save() \ud568\uc218\ub97c \uc0ac\uc6a9\ud558\uc5ec \ubaa8\ub378\uc758 ``state_dict`` \ub97c \uc800\uc7a5\ud558\uba74 \uc774\ud6c4\uc5d0 \ubaa8\ub378\uc744\n\ubd88\ub7ec\uc62c \ub54c \uc720\uc5f0\ud568\uc744 \ud06c\uac8c \uc0b4\ub9b4 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \ud559\uc2b5\ub41c \ubaa8\ub378\uc758 \ub9e4\uac1c\ubcc0\uc218(parameter)\ub9cc\uc744\n\uc800\uc7a5\ud558\uba74\ub418\ubbc0\ub85c \ubaa8\ub378 \uc800\uc7a5 \uc2dc\uc5d0 \uad8c\uc7a5\ud558\ub294 \ubc29\ubc95\uc785\ub2c8\ub2e4. \ubaa8\ub378 \uc804\uccb4\ub97c \uc800\uc7a5\ud558\uace0 \ubd88\ub7ec\uc62c\n\ub54c\uc5d0\ub294 Python\uc758 `pickle `__ \ubaa8\ub4c8\uc744\n\uc0ac\uc6a9\ud558\uc5ec \uc804\uccb4 \ubaa8\ub4c8\uc744 \uc800\uc7a5\ud569\ub2c8\ub2e4. \uc774 \ubc29\uc2dd\uc740 \uc9c1\uad00\uc801\uc778 \ubb38\ubc95\uc744 \uc0ac\uc6a9\ud558\uba70 \ucf54\ub4dc\uc758 \uc591\ub3c4\n\uc801\uc2b5\ub2c8\ub2e4. \uc774 \ubc29\uc2dd\uc758 \ub2e8\uc810\uc740 \uc9c1\ub82c\ud654(serialized)\ub41c \ub370\uc774\ud130\uac00 \ubaa8\ub378\uc744 \uc800\uc7a5\ud560 \ub54c \uc0ac\uc6a9\ud55c\n\ud2b9\uc815 \ud074\ub798\uc2a4 \ubc0f \ub514\ub809\ud1a0\ub9ac \uad6c\uc870\uc5d0 \uc885\uc18d(bind)\ub41c\ub2e4\ub294 \uac83\uc785\ub2c8\ub2e4. \uadf8 \uc774\uc720\ub294 pickle\uc774\n\ubaa8\ub378 \ud074\ub798\uc2a4 \uc790\uccb4\ub97c \uc800\uc7a5\ud558\uc9c0 \uc54a\uae30 \ub54c\ubb38\uc785\ub2c8\ub2e4. \uc624\ud788\ub824 \ubd88\ub7ec\uc62c \ub54c \uc0ac\uc6a9\ub418\ub294 \ud074\ub798\uc2a4\uac00\n\ud3ec\ud568\ub41c \ud30c\uc77c\uc758 \uacbd\ub85c\ub97c \uc800\uc7a5\ud569\ub2c8\ub2e4. \uc774 \ub54c\ubb38\uc5d0 \uc791\uc131\ud55c \ucf54\ub4dc\uac00 \ub2e4\ub978 \ud504\ub85c\uc81d\ud2b8\uc5d0\uc11c\n\uc0ac\uc6a9\ub418\uac70\ub098 \ub9ac\ud329\ud1a0\ub9c1\uc744 \uac70\uce58\ub294 \ub4f1\uc758 \uacfc\uc815\uc5d0\uc11c \ub3d9\uc791\ud558\uc9c0 \uc54a\uc744 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc774 \ub808\uc2dc\ud53c\uc5d0\uc11c\ub294\n\ucd94\ub860\uc744 \uc704\ud574 \ubaa8\ub378\uc744 \uc800\uc7a5\ud558\uace0 \ubd88\ub7ec\uc624\ub294 \ub450 \uac00\uc9c0 \ubc29\ubc95 \ubaa8\ub450\ub97c \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc124\uc815\n----------\n\uc2dc\uc791\ud558\uae30 \uc804\uc5d0 ``torch`` \uac00 \uc5c6\ub2e4\uba74 \uc124\uce58\ud574\uc57c \ud569\ub2c8\ub2e4.\n\n\n::\n\n pip install torch\n" ] }, { @@ -132,7 +132,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/6b6889455ef3d6c74e64c3fc1c12815b/dynamic_net.ipynb b/docs/_downloads/6b6889455ef3d6c74e64c3fc1c12815b/dynamic_net.ipynb index 6c9e1d4d1..531061c2f 100644 --- a/docs/_downloads/6b6889455ef3d6c74e64c3fc1c12815b/dynamic_net.ipynb +++ b/docs/_downloads/6b6889455ef3d6c74e64c3fc1c12815b/dynamic_net.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nPyTorch: \uc81c\uc5b4 \ud750\ub984(Control Flow) + \uac00\uc911\uce58 \uacf5\uc720(Weight Sharing)\n---------------------------------------------------------------\n\nPyTorch \ub3d9\uc801 \uadf8\ub798\ud504\uc758 \uac15\ub825\ud568\uc744 \ubcf4\uc5ec\uc8fc\uae30 \uc704\ud574, \ub9e4\uc6b0 \uc774\uc0c1\ud55c \ubaa8\ub378\uc744 \uad6c\ud604\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4:\n\uac01 \uc21c\uc804\ud30c \ub2e8\uacc4\uc5d0\uc11c \ub9ce\uc740 \uc740\ub2c9 \uacc4\uce35\uc744 \uac16\ub294 \uc644\uc804\ud788 \uc5f0\uacb0(fully-connected)\ub41c ReLU\n\uc2e0\uacbd\ub9dd\uc774 \ubb34\uc791\uc704\ub85c 0 ~ 3 \uc0ac\uc774\uc758 \uc22b\uc790\ub97c \uc120\ud0dd\ud558\uace0, \uac00\uc7a5 \uc548\ucabd(innermost)\uc758 \uc740\ub2c9\uce35\ub4e4\uc744\n\uacc4\uc0b0\ud558\uae30 \uc704\ud574 \ub3d9\uc77c\ud55c \uac00\uc911\uce58\ub97c \uc5ec\ub7ec \ubc88 \uc7ac\uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n\n" + "\nPyTorch: \uc81c\uc5b4 \ud750\ub984(Control Flow) + \uac00\uc911\uce58 \uacf5\uc720(Weight Sharing)\n---------------------------------------------------------------\n\nPyTorch \ub3d9\uc801 \uadf8\ub798\ud504\uc758 \uac15\ub825\ud568\uc744 \ubcf4\uc5ec\uc8fc\uae30 \uc704\ud574, \ub9e4\uc6b0 \uc774\uc0c1\ud55c \ubaa8\ub378\uc744 \uad6c\ud604\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4:\n\uac01 \uc21c\uc804\ud30c \ub2e8\uacc4\uc5d0\uc11c \ub9ce\uc740 \uc740\ub2c9 \uacc4\uce35\uc744 \uac16\ub294 \uc644\uc804\ud788 \uc5f0\uacb0(fully-connected)\ub41c ReLU\n\uc2e0\uacbd\ub9dd\uc774 \ubb34\uc791\uc704\ub85c 0 ~ 3 \uc0ac\uc774\uc758 \uc22b\uc790\ub97c \uc120\ud0dd\ud558\uace0, \uac00\uc7a5 \uc548\ucabd(innermost)\uc758 \uc740\ub2c9\uce35\ub4e4\uc744\n\uacc4\uc0b0\ud558\uae30 \uc704\ud574 \ub3d9\uc77c\ud55c \uac00\uc911\uce58\ub97c \uc5ec\ub7ec \ubc88 \uc7ac\uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n" ] }, { @@ -46,7 +46,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/6d8d81d44ebb8c150be6c39ce263fc89/torchtext_translation_tutorial.ipynb b/docs/_downloads/6d8d81d44ebb8c150be6c39ce263fc89/torchtext_translation_tutorial.ipynb index d8c90b6f7..495229feb 100644 --- a/docs/_downloads/6d8d81d44ebb8c150be6c39ce263fc89/torchtext_translation_tutorial.ipynb +++ b/docs/_downloads/6d8d81d44ebb8c150be6c39ce263fc89/torchtext_translation_tutorial.ipynb @@ -15,14 +15,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nTorchText\ub85c \uc5b8\uc5b4 \ubc88\uc5ed\ud558\uae30\n===================================\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 ``torchtext`` \uc758 \uc720\uc6a9\ud55c \uc5ec\ub7ec \ud074\ub798\uc2a4\ub4e4\uacfc \uc2dc\ud000\uc2a4 \ud22c \uc2dc\ud000\uc2a4(sequence-to-sequence, seq2seq)\ubaa8\ub378\uc744 \ud1b5\ud574\n\uc601\uc5b4\uc640 \ub3c5\uc77c\uc5b4 \ubb38\uc7a5\ub4e4\uc774 \ud3ec\ud568\ub41c \uc720\uba85\ud55c \ub370\uc774\ud130 \uc14b\uc744 \uc774\uc6a9\ud574\uc11c \ub3c5\uc77c\uc5b4 \ubb38\uc7a5\uc744 \uc601\uc5b4\ub85c \ubc88\uc5ed\ud574 \ubcfc \uac83\uc785\ub2c8\ub2e4.\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc740 \nPyTorch \ucee4\ubba4\ub2c8\ud2f0 \uba64\ubc84\uc778 `Ben Trevett `__ \uc774 \uc791\uc131\ud55c\n`\ud29c\ud1a0\ub9ac\uc5bc `__ \uc5d0 \uae30\ucd08\ud558\uace0 \uc788\uc73c\uba70\n`Seth Weidman `__ \uc774 Ben\uc758 \ud5c8\ub77d\uc744 \ubc1b\uace0 \ub9cc\ub4e4\uc5c8\uc2b5\ub2c8\ub2e4.\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc744 \ud1b5\ud574 \uc5ec\ub7ec\ubd84\uc740 \ub2e4\uc74c\uacfc \uac19\uc740 \uac83\uc744 \ud560 \uc218 \uc788\uac8c \ub429\ub2c8\ub2e4:\n\n- ``torchtext`` \uc758 \uc544\ub798\uc640 \uac19\uc740 \uc720\uc6a9\ud55c \ud074\ub798\uc2a4\ub4e4\uc744 \ud1b5\ud574 \ubb38\uc7a5\ub4e4\uc744 NLP\ubaa8\ub378\ub9c1\uc5d0 \uc790\uc8fc \uc0ac\uc6a9\ub418\ub294 \ud615\ud0dc\ub85c \uc804\ucc98\ub9ac\ud560 \uc218 \uc788\uac8c \ub429\ub2c8\ub2e4:\n - `TranslationDataset `__\n - `Field `__\n - `BucketIterator `__\n\n" + "\nTorchText\ub85c \uc5b8\uc5b4 \ubc88\uc5ed\ud558\uae30\n===================================\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 ``torchtext`` \uc758 \uc720\uc6a9\ud55c \uc5ec\ub7ec \ud074\ub798\uc2a4\ub4e4\uacfc \uc2dc\ud000\uc2a4 \ud22c \uc2dc\ud000\uc2a4(sequence-to-sequence, seq2seq)\ubaa8\ub378\uc744 \ud1b5\ud574\n\uc601\uc5b4\uc640 \ub3c5\uc77c\uc5b4 \ubb38\uc7a5\ub4e4\uc774 \ud3ec\ud568\ub41c \uc720\uba85\ud55c \ub370\uc774\ud130 \uc14b\uc744 \uc774\uc6a9\ud574\uc11c \ub3c5\uc77c\uc5b4 \ubb38\uc7a5\uc744 \uc601\uc5b4\ub85c \ubc88\uc5ed\ud574 \ubcfc \uac83\uc785\ub2c8\ub2e4.\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc740\nPyTorch \ucee4\ubba4\ub2c8\ud2f0 \uba64\ubc84\uc778 `Ben Trevett `__ \uc774 \uc791\uc131\ud55c\n`\ud29c\ud1a0\ub9ac\uc5bc `__ \uc5d0 \uae30\ucd08\ud558\uace0 \uc788\uc73c\uba70\n`Seth Weidman `__ \uc774 Ben\uc758 \ud5c8\ub77d\uc744 \ubc1b\uace0 \ub9cc\ub4e4\uc5c8\uc2b5\ub2c8\ub2e4.\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc744 \ud1b5\ud574 \uc5ec\ub7ec\ubd84\uc740 \ub2e4\uc74c\uacfc \uac19\uc740 \uac83\uc744 \ud560 \uc218 \uc788\uac8c \ub429\ub2c8\ub2e4:\n\n- ``torchtext`` \uc758 \uc544\ub798\uc640 \uac19\uc740 \uc720\uc6a9\ud55c \ud074\ub798\uc2a4\ub4e4\uc744 \ud1b5\ud574 \ubb38\uc7a5\ub4e4\uc744 NLP\ubaa8\ub378\ub9c1\uc5d0 \uc790\uc8fc \uc0ac\uc6a9\ub418\ub294 \ud615\ud0dc\ub85c \uc804\ucc98\ub9ac\ud560 \uc218 \uc788\uac8c \ub429\ub2c8\ub2e4:\n - `TranslationDataset `__\n - `Field `__\n - `BucketIterator `__\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "`Field` \uc640 `TranslationDataset`\n----------------\n``torchtext`` \uc5d0\ub294 \uc5b8\uc5b4 \ubcc0\ud658 \ubaa8\ub378\uc744 \ub9cc\ub4e4\ub54c \uc27d\uac8c \uc0ac\uc6a9\ud560 \uc218 \uc788\ub294 \ub370\uc774\ud130\uc14b\uc744 \ub9cc\ub4e4\uae30 \uc801\ud569\ud55c \ub2e4\uc591\ud55c \ub3c4\uad6c\uac00 \uc788\uc2b5\ub2c8\ub2e4.\n\uadf8 \uc911\uc5d0\uc11c\ub3c4 \uc911\uc694\ud55c \ud074\ub798\uc2a4 \uc911 \ud558\ub098\uc778 `Field `__ \ub294\n\uac01 \ubb38\uc7a5\uc774 \uc5b4\ub5bb\uac8c \uc804\ucc98\ub9ac\ub418\uc5b4\uc57c \ud558\ub294\uc9c0 \uc9c0\uc815\ud558\uba70, \ub610 \ub2e4\ub978 \uc911\uc694\ud55c \ud074\ub798\uc2a4\ub85c\ub294 `TranslationDataset` \uc774 \uc788\uc2b5\ub2c8\ub2e4. \n``torchtext`` \uc5d0\ub294 \uc774 \uc678\uc5d0\ub3c4 \ube44\uc2b7\ud55c \ub370\uc774\ud130\uc14b\ub4e4\uc774 \uc788\ub294\ub370, \uc774\ubc88 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 `Multi30k dataset `__ \uc744 \uc0ac\uc6a9\ud560 \uac83\uc785\ub2c8\ub2e4.\n\uc774 \ub370\uc774\ud130 \uc14b\uc740 \ud3c9\uade0 \uc57d 13\uac1c\uc758 \ub2e8\uc5b4\ub85c \uad6c\uc131\ub41c \uc57d \uc0bc\ub9cc \uac1c\uc758 \ubb38\uc7a5\uc744 \uc601\uc5b4\uc640 \ub3c5\uc77c\uc5b4 \ub450 \uc5b8\uc5b4\ub85c \ud3ec\ud568\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n\n\ucc38\uace0 : \uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\uc758 \ud1a0\ud070\ud654(tokenization)\uc5d0\ub294 `Spacy `__ \uac00 \ud544\uc694\ud569\ub2c8\ub2e4.\nSpacy\ub294 \uc601\uc5b4 \uc774 \uc678\uc758 \ub2e4\ub978 \uc5b8\uc5b4\uc5d0 \ub300\ud55c \uac15\ub825\ud55c \ud1a0\ud070\ud654 \uae30\ub2a5\uc744 \uc81c\uacf5\ud558\uae30 \ub54c\ubb38\uc5d0 \uc0ac\uc6a9\ud569\ub2c8\ub2e4. ``torchtext`` \ub294\n`basic_english`` \ud1a0\ud06c\ub098\uc774\uc800\ub97c \uc81c\uacf5\ud560 \ubfd0 \uc544\ub2c8\ub77c \uc601\uc5b4\uc5d0 \uc0ac\uc6a9\ud560 \uc218 \uc788\ub294 \ub2e4\ub978 \ud1a0\ud06c\ub098\uc774\uc800\ub4e4(\uc608\ucee8\ub370\n`Moses `__ )\uc744 \uc9c0\uc6d0\ud569\ub2c8\ub2e4\ub9cc, \uc5b8\uc5b4 \ubc88\uc5ed\uc744 \uc704\ud574\uc11c\ub294 \ub2e4\uc591\ud55c \uc5b8\uc5b4\ub97c\n\ub2e4\ub8e8\uc5b4\uc57c \ud558\uae30 \ub54c\ubb38\uc5d0 Spacy\uac00 \uac00\uc7a5 \uc801\ud569\ud569\ub2c8\ub2e4.\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc744 \uc2e4\ud589\ud558\ub824\uba74, \uc6b0\uc120 ``pip`` \ub098 ``conda`` \ub85c ``spacy`` \ub97c \uc124\uce58\ud558\uc138\uc694. \uadf8 \ub2e4\uc74c,\nSpacy \ud1a0\ud06c\ub098\uc774\uc800\uac00 \uc4f8 \uc601\uc5b4\uc640 \ub3c5\uc77c\uc5b4\uc5d0 \ub300\ud55c \ub370\uc774\ud130\ub97c \ub2e4\uc6b4\ub85c\ub4dc \ubc1b\uc2b5\ub2c8\ub2e4.\n\n::\n\n python -m spacy download en\n python -m spacy download de\n\nSpacy\uac00 \uc124\uce58\ub418\uc5b4 \uc788\ub2e4\uba74, \ub2e4\uc74c \ucf54\ub4dc\ub294 ``TranslationDataset`` \uc5d0 \uc788\ub294 \uac01 \ubb38\uc7a5\uc744 ``Field`` \uc5d0 \uc815\uc758\ub41c\n\ub0b4\uc6a9\uc744 \uae30\ubc18\uc73c\ub85c \ud1a0\ud070\ud654\ud560 \uac83\uc785\ub2c8\ub2e4.\n\n" + "`Field` \uc640 `TranslationDataset`\n--------------------------------\n\n``torchtext`` \uc5d0\ub294 \uc5b8\uc5b4 \ubcc0\ud658 \ubaa8\ub378\uc744 \ub9cc\ub4e4\ub54c \uc27d\uac8c \uc0ac\uc6a9\ud560 \uc218 \uc788\ub294 \ub370\uc774\ud130\uc14b\uc744 \ub9cc\ub4e4\uae30 \uc801\ud569\ud55c \ub2e4\uc591\ud55c \ub3c4\uad6c\uac00 \uc788\uc2b5\ub2c8\ub2e4.\n\uadf8 \uc911\uc5d0\uc11c\ub3c4 \uc911\uc694\ud55c \ud074\ub798\uc2a4 \uc911 \ud558\ub098\uc778 `Field `__ \ub294\n\uac01 \ubb38\uc7a5\uc774 \uc5b4\ub5bb\uac8c \uc804\ucc98\ub9ac\ub418\uc5b4\uc57c \ud558\ub294\uc9c0 \uc9c0\uc815\ud558\uba70, \ub610 \ub2e4\ub978 \uc911\uc694\ud55c \ud074\ub798\uc2a4\ub85c\ub294 `TranslationDataset` \uc774 \uc788\uc2b5\ub2c8\ub2e4.\n``torchtext`` \uc5d0\ub294 \uc774 \uc678\uc5d0\ub3c4 \ube44\uc2b7\ud55c \ub370\uc774\ud130\uc14b\ub4e4\uc774 \uc788\ub294\ub370, \uc774\ubc88 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 `Multi30k dataset `__ \uc744 \uc0ac\uc6a9\ud560 \uac83\uc785\ub2c8\ub2e4.\n\uc774 \ub370\uc774\ud130 \uc14b\uc740 \ud3c9\uade0 \uc57d 13\uac1c\uc758 \ub2e8\uc5b4\ub85c \uad6c\uc131\ub41c \uc57d \uc0bc\ub9cc \uac1c\uc758 \ubb38\uc7a5\uc744 \uc601\uc5b4\uc640 \ub3c5\uc77c\uc5b4 \ub450 \uc5b8\uc5b4\ub85c \ud3ec\ud568\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n\n\ucc38\uace0 : \uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\uc758 \ud1a0\ud070\ud654(tokenization)\uc5d0\ub294 `Spacy `__ \uac00 \ud544\uc694\ud569\ub2c8\ub2e4.\nSpacy\ub294 \uc601\uc5b4 \uc774 \uc678\uc758 \ub2e4\ub978 \uc5b8\uc5b4\uc5d0 \ub300\ud55c \uac15\ub825\ud55c \ud1a0\ud070\ud654 \uae30\ub2a5\uc744 \uc81c\uacf5\ud558\uae30 \ub54c\ubb38\uc5d0 \uc0ac\uc6a9\ud569\ub2c8\ub2e4. ``torchtext`` \ub294\n`basic_english`` \ud1a0\ud06c\ub098\uc774\uc800\ub97c \uc81c\uacf5\ud560 \ubfd0 \uc544\ub2c8\ub77c \uc601\uc5b4\uc5d0 \uc0ac\uc6a9\ud560 \uc218 \uc788\ub294 \ub2e4\ub978 \ud1a0\ud06c\ub098\uc774\uc800\ub4e4(\uc608\ucee8\ub370\n`Moses `__ )\uc744 \uc9c0\uc6d0\ud569\ub2c8\ub2e4\ub9cc, \uc5b8\uc5b4 \ubc88\uc5ed\uc744 \uc704\ud574\uc11c\ub294 \ub2e4\uc591\ud55c \uc5b8\uc5b4\ub97c\n\ub2e4\ub8e8\uc5b4\uc57c \ud558\uae30 \ub54c\ubb38\uc5d0 Spacy\uac00 \uac00\uc7a5 \uc801\ud569\ud569\ub2c8\ub2e4.\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc744 \uc2e4\ud589\ud558\ub824\uba74, \uc6b0\uc120 ``pip`` \ub098 ``conda`` \ub85c ``spacy`` \ub97c \uc124\uce58\ud558\uc138\uc694. \uadf8 \ub2e4\uc74c,\nSpacy \ud1a0\ud06c\ub098\uc774\uc800\uac00 \uc4f8 \uc601\uc5b4\uc640 \ub3c5\uc77c\uc5b4\uc5d0 \ub300\ud55c \ub370\uc774\ud130\ub97c \ub2e4\uc6b4\ub85c\ub4dc \ubc1b\uc2b5\ub2c8\ub2e4.\n\n::\n\n python -m spacy download en\n python -m spacy download de\n\nSpacy\uac00 \uc124\uce58\ub418\uc5b4 \uc788\ub2e4\uba74, \ub2e4\uc74c \ucf54\ub4dc\ub294 ``TranslationDataset`` \uc5d0 \uc788\ub294 \uac01 \ubb38\uc7a5\uc744 ``Field`` \uc5d0 \uc815\uc758\ub41c\n\ub0b4\uc6a9\uc744 \uae30\ubc18\uc73c\ub85c \ud1a0\ud070\ud654\ud560 \uac83\uc785\ub2c8\ub2e4.\n\n" ] }, { @@ -65,7 +65,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "``BucketIterator``\n----------------\n\ub9c8\uc9c0\ub9c9\uc73c\ub85c \uc0ac\uc6a9\ud574 \ubcfc ``torchtext`` \uc5d0 \ud2b9\ud654\ub41c \uae30\ub2a5\uc740 \ubc14\ub85c ``BucketIterator`` \uc785\ub2c8\ub2e4.\n\uccab \ubc88\uc9f8 \uc778\uc790\ub85c ``TranslationDataset`` \uc744 \uc804\ub2ec\ubc1b\uae30 \ub54c\ubb38\uc5d0 \uc0ac\uc6a9\ud558\uae30\uac00 \uc27d\uc2b5\ub2c8\ub2e4. \ubb38\uc11c\uc5d0\uc11c\ub3c4 \ubcfc \uc218 \uc788\ub4ef\n\uc774 \uae30\ub2a5\uc740 \ube44\uc2b7\ud55c \uae38\uc774\uc758 \uc608\uc81c\ub4e4\uc744 \ubb36\uc5b4\uc8fc\ub294 \ubc18\ubcf5\uc790(iterator)\ub97c \uc815\uc758\ud569\ub2c8\ub2e4. \uac01\uac01\uc758 \uc0c8\ub85c\uc6b4 \uc5d0\ud3ec\ud06c(epoch)\ub9c8\ub2e4\n\uc0c8\ub85c \uc11e\uc778 \uacb0\uacfc\ub97c \ub9cc\ub4dc\ub294\ub370 \ud544\uc694\ud55c \ud328\ub529\uc758 \uc218\ub97c \ucd5c\uc18c\ud654 \ud569\ub2c8\ub2e4. \ubc84\ucf00\ud305 \uacfc\uc815\uc5d0\uc11c \uc0ac\uc6a9\ub418\ub294 \uc800\uc7a5 \uacf5\uac04\uc744 \ud55c\ubc88 \uc0b4\ud3b4\ubcf4\uc2dc\uae30 \ubc14\ub78d\ub2c8\ub2e4.\n\n" + "``BucketIterator``\n--------------------\n\ub9c8\uc9c0\ub9c9\uc73c\ub85c \uc0ac\uc6a9\ud574 \ubcfc ``torchtext`` \uc5d0 \ud2b9\ud654\ub41c \uae30\ub2a5\uc740 \ubc14\ub85c ``BucketIterator`` \uc785\ub2c8\ub2e4.\n\uccab \ubc88\uc9f8 \uc778\uc790\ub85c ``TranslationDataset`` \uc744 \uc804\ub2ec\ubc1b\uae30 \ub54c\ubb38\uc5d0 \uc0ac\uc6a9\ud558\uae30\uac00 \uc27d\uc2b5\ub2c8\ub2e4. \ubb38\uc11c\uc5d0\uc11c\ub3c4 \ubcfc \uc218 \uc788\ub4ef\n\uc774 \uae30\ub2a5\uc740 \ube44\uc2b7\ud55c \uae38\uc774\uc758 \uc608\uc81c\ub4e4\uc744 \ubb36\uc5b4\uc8fc\ub294 \ubc18\ubcf5\uc790(iterator)\ub97c \uc815\uc758\ud569\ub2c8\ub2e4. \uac01\uac01\uc758 \uc0c8\ub85c\uc6b4 \uc5d0\ud3ec\ud06c(epoch)\ub9c8\ub2e4\n\uc0c8\ub85c \uc11e\uc778 \uacb0\uacfc\ub97c \ub9cc\ub4dc\ub294\ub370 \ud544\uc694\ud55c \ud328\ub529\uc758 \uc218\ub97c \ucd5c\uc18c\ud654 \ud569\ub2c8\ub2e4. \ubc84\ucf00\ud305 \uacfc\uc815\uc5d0\uc11c \uc0ac\uc6a9\ub418\ub294 \uc800\uc7a5 \uacf5\uac04\uc744 \ud55c\ubc88 \uc0b4\ud3b4\ubcf4\uc2dc\uae30 \ubc14\ub78d\ub2c8\ub2e4.\n\n" ] }, { @@ -83,14 +83,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uc774 \ubc18\ubcf5\uc790\ub4e4\uc740 ``DataLoader`` \uc640 \ub9c8\ucc2c\uac00\uc9c0\ub85c \ud638\ucd9c\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc544\ub798 ``train`` \uacfc \n``evaluation`` \ud568\uc218\uc5d0\uc11c \ubcf4\uba74, \ub2e4\uc74c\uacfc \uac19\uc774 \uac04\ub2e8\ud788 \ud638\ucd9c\ud560 \uc218 \uc788\uc74c\uc744 \uc54c \uc218 \uc788\uc2b5\ub2c8\ub2e4 :\n::\n\n for i, batch in enumerate(iterator):\n\n\uac01 ``batch`` \ub294 ``src`` \uc640 ``trg`` \uc18d\uc131\uc744 \uac00\uc9c0\uac8c \ub429\ub2c8\ub2e4.\n\n::\n\n src = batch.src\n trg = batch.trg\n\n" + "\uc774 \ubc18\ubcf5\uc790\ub4e4\uc740 ``DataLoader`` \uc640 \ub9c8\ucc2c\uac00\uc9c0\ub85c \ud638\ucd9c\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc544\ub798 ``train`` \uacfc\n``evaluation`` \ud568\uc218\uc5d0\uc11c \ubcf4\uba74, \ub2e4\uc74c\uacfc \uac19\uc774 \uac04\ub2e8\ud788 \ud638\ucd9c\ud560 \uc218 \uc788\uc74c\uc744 \uc54c \uc218 \uc788\uc2b5\ub2c8\ub2e4 :\n::\n\n for i, batch in enumerate(iterator):\n\n\uac01 ``batch`` \ub294 ``src`` \uc640 ``trg`` \uc18d\uc131\uc744 \uac00\uc9c0\uac8c \ub429\ub2c8\ub2e4.\n\n::\n\n src = batch.src\n trg = batch.trg\n\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "``nn.Module`` \uacfc ``Optimizer`` \uc815\uc758\ud558\uae30\n----------------\n\ub300\ubd80\ubd84\uc740 ``torchtext`` \uac00 \uc54c\uc544\uc11c \ud574\uc90d\ub2c8\ub2e4 : \ub370\uc774\ud130\uc14b\uc774 \ub9cc\ub4e4\uc5b4\uc9c0\uace0 \ubc18\ubcf5\uc790\uac00 \uc815\uc758\ub418\uba74, \uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\n\uc6b0\ub9ac\uac00 \ud574\uc57c \ud560 \uc77c\uc774\ub77c\uace0\ub294 \uadf8\uc800 ``nn.Module`` \uc640 ``Optimizer`` \ub97c \ubaa8\ub378\ub85c\uc11c \uc815\uc758\ud558\uace0 \ud6c8\ub828\uc2dc\ud0a4\ub294 \uac83\uc774 \uc804\ubd80\uc785\ub2c8\ub2e4.\n\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c \uc0ac\uc6a9\ud560 \ubaa8\ub378\uc740 `\uc774\uacf3 `__ \uc5d0\uc11c \uc124\uba85\ud558\uace0 \uc788\ub294 \uad6c\uc870\ub97c \ub530\ub974\uace0 \uc788\uc73c\uba70,\n\ub354 \uc790\uc138\ud55c \ub0b4\uc6a9\uc740 `\uc5ec\uae30 `__ \n\ub97c \ucc38\uace0\ud558\uc2dc\uae30 \ubc14\ub78d\ub2c8\ub2e4.\n\n\ucc38\uace0 : \uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c \uc0ac\uc6a9\ud558\ub294 \ubaa8\ub378\uc740 \uc5b8\uc5b4 \ubc88\uc5ed\uc744 \uc704\ud574 \uc0ac\uc6a9\ud560 \uc608\uc2dc \ubaa8\ub378\uc785\ub2c8\ub2e4. \uc774 \ubaa8\ub378\uc744 \uc0ac\uc6a9\ud558\ub294 \uac83\uc740\n\uc774 \uc791\uc5c5\uc5d0 \uc801\ub2f9\ud55c \ud45c\uc900 \ubaa8\ub378\uc774\uae30 \ub54c\ubb38\uc774\uc9c0, \ubc88\uc5ed\uc5d0 \uc801\ud569\ud55c \ubaa8\ub378\uc774\uae30 \ub54c\ubb38\uc740 \uc544\ub2d9\ub2c8\ub2e4. \uc5ec\ub7ec\ubd84\uc774 \ucd5c\uc2e0 \uae30\uc220 \ud2b8\ub80c\ub4dc\ub97c\n\uc798 \ub530\ub77c\uac00\uace0 \uc788\ub2e4\uba74 \uc798 \uc544\uc2dc\uaca0\uc9c0\ub9cc, \ud604\uc7ac \ubc88\uc5ed\uc5d0\uc11c \uac00\uc7a5 \ub6f0\uc5b4\ub09c \ubaa8\ub378\uc740 Transformers\uc785\ub2c8\ub2e4. PyTorch\uac00\nTransformer \ub808\uc774\uc5b4\ub97c \uad6c\ud604\ud55c \ub0b4\uc6a9\uc740 `\uc5ec\uae30 `__\n\uc5d0\uc11c \ud655\uc778\ud560 \uc218 \uc788\uc73c\uba70 \uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc758 \ubaa8\ub378\uc774 \uc0ac\uc6a9\ud558\ub294 \"attention\" \uc740 Transformer \ubaa8\ub378\uc5d0\uc11c \uc81c\uc548\ud558\ub294\n\uba40\ud2f0 \ud5e4\ub4dc \uc140\ud504 \uc5b4\ud150\uc158(multi-headed self-attention) \uacfc\ub294 \ub2e4\ub974\ub2e4\ub294 \uc810\uc744 \uc54c\ub824\ub4dc\ub9bd\ub2c8\ub2e4.\n\n" + "``nn.Module`` \uacfc ``Optimizer`` \uc815\uc758\ud558\uae30\n------------------------------------------\n\ub300\ubd80\ubd84\uc740 ``torchtext`` \uac00 \uc54c\uc544\uc11c \ud574\uc90d\ub2c8\ub2e4 : \ub370\uc774\ud130\uc14b\uc774 \ub9cc\ub4e4\uc5b4\uc9c0\uace0 \ubc18\ubcf5\uc790\uac00 \uc815\uc758\ub418\uba74, \uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\n\uc6b0\ub9ac\uac00 \ud574\uc57c \ud560 \uc77c\uc774\ub77c\uace0\ub294 \uadf8\uc800 ``nn.Module`` \uc640 ``Optimizer`` \ub97c \ubaa8\ub378\ub85c\uc11c \uc815\uc758\ud558\uace0 \ud6c8\ub828\uc2dc\ud0a4\ub294 \uac83\uc774 \uc804\ubd80\uc785\ub2c8\ub2e4.\n\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c \uc0ac\uc6a9\ud560 \ubaa8\ub378\uc740 `\uc774\uacf3 `__ \uc5d0\uc11c \uc124\uba85\ud558\uace0 \uc788\ub294 \uad6c\uc870\ub97c \ub530\ub974\uace0 \uc788\uc73c\uba70,\n\ub354 \uc790\uc138\ud55c \ub0b4\uc6a9\uc740 `\uc5ec\uae30 `__\n\ub97c \ucc38\uace0\ud558\uc2dc\uae30 \ubc14\ub78d\ub2c8\ub2e4.\n\n\ucc38\uace0 : \uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c \uc0ac\uc6a9\ud558\ub294 \ubaa8\ub378\uc740 \uc5b8\uc5b4 \ubc88\uc5ed\uc744 \uc704\ud574 \uc0ac\uc6a9\ud560 \uc608\uc2dc \ubaa8\ub378\uc785\ub2c8\ub2e4. \uc774 \ubaa8\ub378\uc744 \uc0ac\uc6a9\ud558\ub294 \uac83\uc740\n\uc774 \uc791\uc5c5\uc5d0 \uc801\ub2f9\ud55c \ud45c\uc900 \ubaa8\ub378\uc774\uae30 \ub54c\ubb38\uc774\uc9c0, \ubc88\uc5ed\uc5d0 \uc801\ud569\ud55c \ubaa8\ub378\uc774\uae30 \ub54c\ubb38\uc740 \uc544\ub2d9\ub2c8\ub2e4. \uc5ec\ub7ec\ubd84\uc774 \ucd5c\uc2e0 \uae30\uc220 \ud2b8\ub80c\ub4dc\ub97c\n\uc798 \ub530\ub77c\uac00\uace0 \uc788\ub2e4\uba74 \uc798 \uc544\uc2dc\uaca0\uc9c0\ub9cc, \ud604\uc7ac \ubc88\uc5ed\uc5d0\uc11c \uac00\uc7a5 \ub6f0\uc5b4\ub09c \ubaa8\ub378\uc740 Transformers\uc785\ub2c8\ub2e4. PyTorch\uac00\nTransformer \ub808\uc774\uc5b4\ub97c \uad6c\ud604\ud55c \ub0b4\uc6a9\uc740 `\uc5ec\uae30 `__\n\uc5d0\uc11c \ud655\uc778\ud560 \uc218 \uc788\uc73c\uba70 \uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc758 \ubaa8\ub378\uc774 \uc0ac\uc6a9\ud558\ub294 \"attention\" \uc740 Transformer \ubaa8\ub378\uc5d0\uc11c \uc81c\uc548\ud558\ub294\n\uba40\ud2f0 \ud5e4\ub4dc \uc140\ud504 \uc5b4\ud150\uc158(multi-headed self-attention) \uacfc\ub294 \ub2e4\ub974\ub2e4\ub294 \uc810\uc744 \uc54c\ub824\ub4dc\ub9bd\ub2c8\ub2e4.\n\n" ] }, { @@ -175,7 +175,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/70d2076b0da469ff43a0ec4f6100b39c/numeric_suite_tutorial.ipynb b/docs/_downloads/70d2076b0da469ff43a0ec4f6100b39c/numeric_suite_tutorial.ipynb index 4e6a0f75e..efd2ee5d1 100644 --- a/docs/_downloads/70d2076b0da469ff43a0ec4f6100b39c/numeric_suite_tutorial.ipynb +++ b/docs/_downloads/70d2076b0da469ff43a0ec4f6100b39c/numeric_suite_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nPyTorch Numeric Suite Tutorial\n==============================\n\nIntroduction\n------------\n\nQuantization is good when it works, but it\u2019s difficult to know what's wrong when it doesn't satisfy the accuracy we expect. Debugging the accuracy issue of quantization is not easy and time consuming.\n\nOne important step of debugging is to measure the statistics of the float model and its corresponding quantized model to know where are they differ most. We built a suite of numeric tools called PyTorch Numeric Suite in PyTorch quantization to enable the measurement of the statistics between quantized module and float module to support quantization debugging efforts. Even for the quantized model with good accuracy, PyTorch Numeric Suite can still be used as the profiling tool to better understand the quantization error within the model and provide the guidance for further optimization.\n\nPyTorch Numeric Suite currently supports models quantized through both static quantization and dynamic quantization with unified APIs.\n\nIn this tutorial we will first use ResNet18 as an example to show how to use PyTorch Numeric Suite to measure the statistics between static quantized model and float model in eager mode. Then we will use LSTM based sequence model as an example to show the usage of PyTorch Numeric Suite for dynamic quantized model.\n\nNumeric Suite for Static Quantization\n-------------------------------------\n\nSetup\n^^^^^^\nWe\u2019ll start by doing the necessary imports:\n\n" + "\nPyTorch Numeric Suite Tutorial\n==============================\n\nIntroduction\n------------\n\nQuantization is good when it works, but it\u2019s difficult to know what's wrong when it doesn't satisfy the accuracy we expect. Debugging the accuracy issue of quantization is not easy and time consuming.\n\nOne important step of debugging is to measure the statistics of the float model and its corresponding quantized model to know where are they differ most. We built a suite of numeric tools called PyTorch Numeric Suite in PyTorch quantization to enable the measurement of the statistics between quantized module and float module to support quantization debugging efforts. Even for the quantized model with good accuracy, PyTorch Numeric Suite can still be used as the profiling tool to better understand the quantization error within the model and provide the guidance for further optimization.\n\nPyTorch Numeric Suite currently supports models quantized through both static quantization and dynamic quantization with unified APIs.\n\nIn this tutorial we will first use ResNet18 as an example to show how to use PyTorch Numeric Suite to measure the statistics between static quantized model and float model in eager mode. Then we will use LSTM based sequence model as an example to show the usage of PyTorch Numeric Suite for dynamic quantized model.\n\nNumeric Suite for Static Quantization\n-------------------------------------\n\nSetup\n^^^^^^\nWe\u2019ll start by doing the necessary imports:\n" ] }, { @@ -528,7 +528,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/7126bf7beed4c4c3a05bcc2dac8baa3c/pruning_tutorial.ipynb b/docs/_downloads/7126bf7beed4c4c3a05bcc2dac8baa3c/pruning_tutorial.ipynb index 06f7e21f9..c23b5a461 100644 --- a/docs/_downloads/7126bf7beed4c4c3a05bcc2dac8baa3c/pruning_tutorial.ipynb +++ b/docs/_downloads/7126bf7beed4c4c3a05bcc2dac8baa3c/pruning_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n\uac00\uc9c0\uce58\uae30 \uae30\ubc95(Pruning) \ud29c\ud1a0\ub9ac\uc5bc\n=====================================\n**\uc800\uc790**: `Michela Paganini `_\n**\ubc88\uc5ed** : `\uc548\uc0c1\uc900 `_\n\n\ucd5c\ucca8\ub2e8 \ub525\ub7ec\ub2dd \ubaa8\ub378\ub4e4\uc740 \uad49\uc7a5\ud788 \ub9ce\uc740 \uc218\uc758 \ud30c\ub77c\ubbf8\ud130\uac12\ub4e4\ub85c \uad6c\uc131\ub418\uae30 \ub54c\ubb38\uc5d0, \uc27d\uac8c \ubc30\ud3ec\ub418\uae30 \uc5b4\ub835\uc2b5\ub2c8\ub2e4.\n\uc774\uc640 \ubc18\ub300\ub85c, \uc0dd\ubb3c\ud559\uc801 \uc2e0\uacbd\ub9dd\ub4e4\uc740 \ud6a8\uc728\uc801\uc73c\ub85c \ud76c\uc18c\ud558\uac8c \uc5f0\uacb0\ub41c \uac83\uc73c\ub85c \uc54c\ub824\uc838 \uc788\uc2b5\ub2c8\ub2e4.\n\ubaa8\ub378\uc758 \uc815\ud655\ub3c4\uac00 \uc190\uc0c1\ub418\uc9c0 \uc54a\ub294 \ubc94\uc704\uc5d0\uc11c \uba54\ubaa8\ub9ac, \ubc30\ud130\ub9ac, \ud558\ub4dc\uc6e8\uc5b4 \uc18c\ube44\ub7c9\uc744 \uc904\uc774\uace0, \n\uae30\uae30\uc5d0 \uacbd\ub7c9\ud654\ub41c \ubaa8\ub378\uc744 \ubc30\uce58\ud558\uba70, \uac1c\uc778\uc774 \uc774\uc6a9\ud558\uace0 \uc788\ub294 \uae30\uae30\uc5d0\uc11c \ud504\ub77c\uc774\ubc84\uc2dc\uac00 \ubcf4\uc7a5\ub418\uae30 \uc704\ud574\uc11c\ub294 \n\ubaa8\ub378\uc5d0 \ud3ec\ud568\ub41c \ud30c\ub77c\ubbf8\ud130 \uc218\ub97c \uc904\uc5ec \uc555\ucd95\ud558\ub294 \ucd5c\uc801\uc758 \uae30\ubc95\uc744 \ud30c\uc545\ud558\ub294 \uac83\uc774 \uc911\uc694\ud569\ub2c8\ub2e4.\n\uc5f0\uad6c \uce21\uba74\uc5d0\uc11c\ub294, \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc740 \uad49\uc7a5\ud788 \ub9ce\uc740 \uc218\uc758 \ud30c\ub77c\ubbf8\ud130\uac12\ub4e4\ub85c \uad6c\uc131\ub41c \ubaa8\ub378\uacfc \n\uad49\uc7a5\ud788 \uc801\uc740 \uc218\uc758 \ud30c\ub77c\ubbf8\ud130\uac12\ub4e4\ub85c \uad6c\uc131\ub41c \ubaa8\ub378 \uac04 \ud559\uc2b5 \uc5ed\ud559 \ucc28\uc774\ub97c \uc870\uc0ac\ud558\ub294\ub370 \uc8fc\ub85c \uc774\uc6a9\ub418\uae30\ub3c4 \ud558\uba70,\n\ud558\uc704 \uc2e0\uacbd\ub9dd \ubaa8\ub378\uacfc \ud30c\ub77c\ubbf8\ud130\uac12\ub4e4\uc758 \ucd08\uae30\ud654\uac00 \uc6b4\uc774 \uc88b\uac8c \uc798 \ub41c \ucf00\uc774\uc2a4\ub97c \ubc14\ud0d5\uc73c\ub85c \n(\"`lottery tickets `_\") \uc2e0\uacbd\ub9dd \uad6c\uc870\ub97c \ucc3e\ub294 \uae30\uc220\ub4e4\uc5d0 \ub300\ud574 \ubc18\ub300 \uc758\uacac\uc744 \uc81c\uc2dc\ud558\uae30\ub3c4 \ud569\ub2c8\ub2e4.\n\n\uc774\ubc88 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294, ``torch.nn.utils.prune`` \uc744 \uc774\uc6a9\ud558\uc5ec \uc5ec\ub7ec\ubd84\uc774 \uc124\uacc4\ud55c \ub525\ub7ec\ub2dd \ubaa8\ub378\uc5d0 \ub300\ud574 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud574\ubcf4\ub294 \uac83\uc744 \ubc30\uc6cc\ubcf4\uace0, \n\uc2ec\ud654\uc801\uc73c\ub85c \uc5ec\ub7ec\ubd84\uc758 \ub9de\ucda4\ud615 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uad6c\ud604\ud558\ub294 \ubc29\ubc95\uc5d0 \ub300\ud574 \ubc30\uc6cc\ubcf4\ub3c4\ub85d \ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc694\uad6c\uc0ac\ud56d\n------------\n``\"torch>=1.4\"``\n\n\n" + "\n\uac00\uc9c0\uce58\uae30 \uae30\ubc95(Pruning) \ud29c\ud1a0\ub9ac\uc5bc\n=====================================\n**\uc800\uc790**: `Michela Paganini `_\n**\ubc88\uc5ed** : `\uc548\uc0c1\uc900 `_\n\n\ucd5c\ucca8\ub2e8 \ub525\ub7ec\ub2dd \ubaa8\ub378\ub4e4\uc740 \uad49\uc7a5\ud788 \ub9ce\uc740 \uc218\uc758 \ud30c\ub77c\ubbf8\ud130\uac12\ub4e4\ub85c \uad6c\uc131\ub418\uae30 \ub54c\ubb38\uc5d0, \uc27d\uac8c \ubc30\ud3ec\ub418\uae30 \uc5b4\ub835\uc2b5\ub2c8\ub2e4.\n\uc774\uc640 \ubc18\ub300\ub85c, \uc0dd\ubb3c\ud559\uc801 \uc2e0\uacbd\ub9dd\ub4e4\uc740 \ud6a8\uc728\uc801\uc73c\ub85c \ud76c\uc18c\ud558\uac8c \uc5f0\uacb0\ub41c \uac83\uc73c\ub85c \uc54c\ub824\uc838 \uc788\uc2b5\ub2c8\ub2e4.\n\ubaa8\ub378\uc758 \uc815\ud655\ub3c4\uac00 \uc190\uc0c1\ub418\uc9c0 \uc54a\ub294 \ubc94\uc704\uc5d0\uc11c \uba54\ubaa8\ub9ac, \ubc30\ud130\ub9ac, \ud558\ub4dc\uc6e8\uc5b4 \uc18c\ube44\ub7c9\uc744 \uc904\uc774\uace0,\n\uae30\uae30\uc5d0 \uacbd\ub7c9\ud654\ub41c \ubaa8\ub378\uc744 \ubc30\uce58\ud558\uba70, \uac1c\uc778\uc774 \uc774\uc6a9\ud558\uace0 \uc788\ub294 \uae30\uae30\uc5d0\uc11c \ud504\ub77c\uc774\ubc84\uc2dc\uac00 \ubcf4\uc7a5\ub418\uae30 \uc704\ud574\uc11c\ub294\n\ubaa8\ub378\uc5d0 \ud3ec\ud568\ub41c \ud30c\ub77c\ubbf8\ud130 \uc218\ub97c \uc904\uc5ec \uc555\ucd95\ud558\ub294 \ucd5c\uc801\uc758 \uae30\ubc95\uc744 \ud30c\uc545\ud558\ub294 \uac83\uc774 \uc911\uc694\ud569\ub2c8\ub2e4.\n\uc5f0\uad6c \uce21\uba74\uc5d0\uc11c\ub294, \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc740 \uad49\uc7a5\ud788 \ub9ce\uc740 \uc218\uc758 \ud30c\ub77c\ubbf8\ud130\uac12\ub4e4\ub85c \uad6c\uc131\ub41c \ubaa8\ub378\uacfc\n\uad49\uc7a5\ud788 \uc801\uc740 \uc218\uc758 \ud30c\ub77c\ubbf8\ud130\uac12\ub4e4\ub85c \uad6c\uc131\ub41c \ubaa8\ub378 \uac04 \ud559\uc2b5 \uc5ed\ud559 \ucc28\uc774\ub97c \uc870\uc0ac\ud558\ub294\ub370 \uc8fc\ub85c \uc774\uc6a9\ub418\uae30\ub3c4 \ud558\uba70,\n\ud558\uc704 \uc2e0\uacbd\ub9dd \ubaa8\ub378\uacfc \ud30c\ub77c\ubbf8\ud130\uac12\ub4e4\uc758 \ucd08\uae30\ud654\uac00 \uc6b4\uc774 \uc88b\uac8c \uc798 \ub41c \ucf00\uc774\uc2a4\ub97c \ubc14\ud0d5\uc73c\ub85c\n(\"`lottery tickets `_\") \uc2e0\uacbd\ub9dd \uad6c\uc870\ub97c \ucc3e\ub294 \uae30\uc220\ub4e4\uc5d0 \ub300\ud574 \ubc18\ub300 \uc758\uacac\uc744 \uc81c\uc2dc\ud558\uae30\ub3c4 \ud569\ub2c8\ub2e4.\n\n\uc774\ubc88 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294, ``torch.nn.utils.prune`` \uc744 \uc774\uc6a9\ud558\uc5ec \uc5ec\ub7ec\ubd84\uc774 \uc124\uacc4\ud55c \ub525\ub7ec\ub2dd \ubaa8\ub378\uc5d0 \ub300\ud574 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud574\ubcf4\ub294 \uac83\uc744 \ubc30\uc6cc\ubcf4\uace0,\n\uc2ec\ud654\uc801\uc73c\ub85c \uc5ec\ub7ec\ubd84\uc758 \ub9de\ucda4\ud615 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uad6c\ud604\ud558\ub294 \ubc29\ubc95\uc5d0 \ub300\ud574 \ubc30\uc6cc\ubcf4\ub3c4\ub85d \ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc694\uad6c\uc0ac\ud56d\n------------\n``\"torch>=1.4\"``\n" ] }, { @@ -33,7 +33,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\ub525\ub7ec\ub2dd \ubaa8\ub378 \uc0dd\uc131\n-----------------------\n\uc774\ubc88 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294, \uc580 \ub974\ucfe4 \uad50\uc218\ub2d8\uc758 \uc5f0\uad6c\uc9c4\ub4e4\uc774 1998\ub144\ub3c4\uc5d0 \ubc1c\ud45c\ud55c ``LeNet \n`` \uc758 \ubaa8\ub378 \uad6c\uc870\ub97c \uc774\uc6a9\ud569\ub2c8\ub2e4.\n\n" + "\ub525\ub7ec\ub2dd \ubaa8\ub378 \uc0dd\uc131\n-----------------------\n\uc774\ubc88 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294, \uc580 \ub974\ucfe4 \uad50\uc218\ub2d8\uc758 \uc5f0\uad6c\uc9c4\ub4e4\uc774 1998\ub144\ub3c4\uc5d0 \ubc1c\ud45c\ud55c ``LeNet\n`` \uc758 \ubaa8\ub378 \uad6c\uc870\ub97c \uc774\uc6a9\ud569\ub2c8\ub2e4.\n\n" ] }, { @@ -51,7 +51,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\ubaa8\ub4c8 \uc810\uac80\n-----------------\n\n\uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub418\uc9c0 \uc54a\uc740 LeNet \ubaa8\ub378\uc758 ``conv1`` \uce35\uc744 \uc810\uac80\ud574\ubd05\uc2dc\ub2e4. \n\uc5ec\uae30\uc5d0\ub294 2\uac1c\uc758 \ud30c\ub77c\ubbf8\ud130\uac12\ub4e4\uc778 ``\uac00\uc911\uce58``\uac12\uacfc ``\ud3b8\ud5a5``\uac12\uc744\uc774 \ud3ec\ud568\ub420 \uac83\uc774\uba70, \ubc84\ud37c\ub294 \uc874\uc7ac\ud558\uc9c0 \uc54a\uc744 \uac83\uc785\ub2c8\ub2e4.\n\n" + "\ubaa8\ub4c8 \uc810\uac80\n-----------------\n\n\uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub418\uc9c0 \uc54a\uc740 LeNet \ubaa8\ub378\uc758 ``conv1`` \uce35\uc744 \uc810\uac80\ud574\ubd05\uc2dc\ub2e4.\n\uc5ec\uae30\uc5d0\ub294 2\uac1c\uc758 \ud30c\ub77c\ubbf8\ud130\uac12\ub4e4\uc778 ``\uac00\uc911\uce58`` \uac12\uacfc ``\ud3b8\ud5a5`` \uac12\uc774 \ud3ec\ud568\ub420 \uac83\uc774\uba70, \ubc84\ud37c\ub294 \uc874\uc7ac\ud558\uc9c0 \uc54a\uc744 \uac83\uc785\ub2c8\ub2e4.\n\n\n" ] }, { @@ -80,7 +80,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\ubaa8\ub4c8 \uac00\uc9c0\uce58\uae30 \uae30\ubc95 \uc801\uc6a9 \uc608\uc81c\n-----------------------------------\n\n\ubaa8\ub4c8\uc5d0 \ub300\ud574 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud558\uae30 \uc704\ud574 (\uc774\ubc88 \uc608\uc81c\uc5d0\uc11c\ub294, LeNet \ubaa8\ub378\uc758 ``conv1`` \uce35)\n\uccab \ubc88\uc9f8\ub85c\ub294, ``torch.nn.utils.prune`` (\ub610\ub294 ``BasePruningMethod`` \uc758 \uc11c\ube0c \ud074\ub798\uc2a4\ub85c \uc9c1\uc811 `\uad6c\ud604\n`_ )\n\ub0b4 \uc874\uc7ac\ud558\ub294 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc120\ud0dd\ud569\ub2c8\ub2e4.\n\uadf8 \ud6c4, \ud574\ub2f9 \ubaa8\ub4c8 \ub0b4\uc5d0\uc11c \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud558\uace0\uc790 \ud558\ub294 \ubaa8\ub4c8\uacfc \ud30c\ub77c\ubbf8\ud130\ub97c \uc9c0\uc815\ud569\ub2c8\ub2e4.\n\ub9c8\uc9c0\ub9c9\uc73c\ub85c, \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc5d0 \uc801\ub2f9\ud55c \ud0a4\uc6cc\ub4dc \uc778\uc790\uac12\uc744 \uc774\uc6a9\ud558\uc5ec \uac00\uc9c0\uce58\uae30 \ub9e4\uac1c\ubcc0\uc218\ub97c \uc9c0\uc815\ud569\ub2c8\ub2e4.\n\uc774\ubc88 \uc608\uc81c\uc5d0\uc11c\ub294, ``conv1`` \uce35\uc758 \uac00\uc911\uce58\uc758 30%\uac12\ub4e4\uc744 \ub79c\ub364\uc73c\ub85c \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\ubaa8\ub4c8\uc740 \ud568\uc218\uc5d0 \ub300\ud55c \uccab \ubc88\uc9f8 \uc778\uc790\uac12\uc73c\ub85c \uc804\ub2ec\ub418\uba70, ``name`` \uc740 \ubb38\uc790\uc5f4 \uc2dd\ubcc4\uc790\ub97c \uc774\uc6a9\ud558\uc5ec \ud574\ub2f9 \ubaa8\ub4c8 \ub0b4 \ub9e4\uac1c\ubcc0\uc218\ub97c \uad6c\ubd84\ud569\ub2c8\ub2e4.\n\uadf8\ub9ac\uace0, ``amount`` \ub294 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud558\uae30 \uc704\ud55c \ub300\uc0c1 \uac00\uc911\uce58\uac12\ub4e4\uc758 \ubc31\ubd84\uc728 (0\uacfc 1\uc0ac\uc774\uc758 \uc2e4\uc218\uac12), \n\ud639\uc740 \uac00\uc911\uce58\uac12\uc758 \uc5f0\uacb0\uc758 \uac1c\uc218 (\uc74c\uc218\uac00 \uc544\ub2cc \uc815\uc218) \ub97c \uc9c0\uc815\ud569\ub2c8\ub2e4.\n\n" + "\ubaa8\ub4c8 \uac00\uc9c0\uce58\uae30 \uae30\ubc95 \uc801\uc6a9 \uc608\uc81c\n-----------------------------------\n\n\ubaa8\ub4c8\uc5d0 \ub300\ud574 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud558\uae30 \uc704\ud574 (\uc774\ubc88 \uc608\uc81c\uc5d0\uc11c\ub294, LeNet \ubaa8\ub378\uc758 ``conv1`` \uce35)\n\uccab \ubc88\uc9f8\ub85c\ub294, ``torch.nn.utils.prune`` (\ub610\ub294 ``BasePruningMethod`` \uc758 \uc11c\ube0c \ud074\ub798\uc2a4\ub85c \uc9c1\uc811\n`\uad6c\ud604 `_ )\n\ub0b4 \uc874\uc7ac\ud558\ub294 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc120\ud0dd\ud569\ub2c8\ub2e4.\n\uadf8 \ud6c4, \ud574\ub2f9 \ubaa8\ub4c8 \ub0b4\uc5d0\uc11c \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud558\uace0\uc790 \ud558\ub294 \ubaa8\ub4c8\uacfc \ud30c\ub77c\ubbf8\ud130\ub97c \uc9c0\uc815\ud569\ub2c8\ub2e4.\n\ub9c8\uc9c0\ub9c9\uc73c\ub85c, \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc5d0 \uc801\ub2f9\ud55c \ud0a4\uc6cc\ub4dc \uc778\uc790\uac12\uc744 \uc774\uc6a9\ud558\uc5ec \uac00\uc9c0\uce58\uae30 \ub9e4\uac1c\ubcc0\uc218\ub97c \uc9c0\uc815\ud569\ub2c8\ub2e4.\n\uc774\ubc88 \uc608\uc81c\uc5d0\uc11c\ub294, ``conv1`` \uce35\uc758 \uac00\uc911\uce58\uc758 30%\uac12\ub4e4\uc744 \ub79c\ub364\uc73c\ub85c \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\ubaa8\ub4c8\uc740 \ud568\uc218\uc5d0 \ub300\ud55c \uccab \ubc88\uc9f8 \uc778\uc790\uac12\uc73c\ub85c \uc804\ub2ec\ub418\uba70, ``name`` \uc740 \ubb38\uc790\uc5f4 \uc2dd\ubcc4\uc790\ub97c \uc774\uc6a9\ud558\uc5ec \ud574\ub2f9 \ubaa8\ub4c8 \ub0b4 \ub9e4\uac1c\ubcc0\uc218\ub97c \uad6c\ubd84\ud569\ub2c8\ub2e4.\n\uadf8\ub9ac\uace0, ``amount`` \ub294 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud558\uae30 \uc704\ud55c \ub300\uc0c1 \uac00\uc911\uce58\uac12\ub4e4\uc758 \ubc31\ubd84\uc728 (0\uacfc 1\uc0ac\uc774\uc758 \uc2e4\uc218\uac12),\n\ud639\uc740 \uac00\uc911\uce58\uac12\uc758 \uc5f0\uacb0\uc758 \uac1c\uc218 (\uc74c\uc218\uac00 \uc544\ub2cc \uc815\uc218) \ub97c \uc9c0\uc815\ud569\ub2c8\ub2e4.\n\n" ] }, { @@ -98,7 +98,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc740 \uac00\uc911\uce58\uac12\ub4e4\uc744 \ud30c\ub77c\ubbf8\ud130\uac12\ub4e4\ub85c\ubd80\ud130 \uc81c\uac70\ud558\uace0 ``weight_orig`` (\uc989, \ucd08\uae30 \uac00\uc911\uce58 \uc774\ub984\uc5d0 \"_orig\"\uc744 \ubd99\uc778) \uc774\ub77c\ub294 \n\uc0c8\ub85c\uc6b4 \ud30c\ub77c\ubbf8\ud130\uac12\uc73c\ub85c \ub300\uccb4\ud558\ub294 \uac83\uc73c\ub85c \uc2e4\ud589\ub429\ub2c8\ub2e4.\n``weight_orig`` \uc740 \ud150\uc11c\uac12\uc5d0 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub418\uc9c0 \uc54a\uc740 \uc0c1\ud0dc\ub97c \uc800\uc7a5\ud569\ub2c8\ub2e4. \n``bias`` \uc740 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub418\uc9c0 \uc54a\uc558\uae30 \ub54c\ubb38\uc5d0 \uadf8\ub300\ub85c \ub0a8\uc544 \uc788\uc2b5\ub2c8\ub2e4.\n\n" + "\uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc740 \uac00\uc911\uce58\uac12\ub4e4\uc744 \ud30c\ub77c\ubbf8\ud130\uac12\ub4e4\ub85c\ubd80\ud130 \uc81c\uac70\ud558\uace0 ``weight_orig`` (\uc989, \ucd08\uae30 \uac00\uc911\uce58 \uc774\ub984\uc5d0 \"_orig\"\uc744 \ubd99\uc778) \uc774\ub77c\ub294\n\uc0c8\ub85c\uc6b4 \ud30c\ub77c\ubbf8\ud130\uac12\uc73c\ub85c \ub300\uccb4\ud558\ub294 \uac83\uc73c\ub85c \uc2e4\ud589\ub429\ub2c8\ub2e4.\n``weight_orig`` \uc740 \ud150\uc11c\uac12\uc5d0 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub418\uc9c0 \uc54a\uc740 \uc0c1\ud0dc\ub97c \uc800\uc7a5\ud569\ub2c8\ub2e4.\n``bias`` \uc740 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub418\uc9c0 \uc54a\uc558\uae30 \ub54c\ubb38\uc5d0 \uadf8\ub300\ub85c \ub0a8\uc544 \uc788\uc2b5\ub2c8\ub2e4.\n\n" ] }, { @@ -116,7 +116,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uc704\uc5d0\uc11c \uc120\ud0dd\ud55c \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc5d0 \uc758\ud574 \uc0dd\uc131\ub418\ub294 \uac00\uc9c0\uce58\uae30 \ub9c8\uc2a4\ud06c\ub294 \ucd08\uae30 \ud30c\ub77c\ubbf8\ud130 ``name`` \uc5d0 ``weight_mask`` \n(\uc989, \ucd08\uae30 \uac00\uc911\uce58 \uc774\ub984\uc5d0 \"_mask\"\ub97c \ubd99\uc778) \uc774\ub984\uc758 \ubaa8\ub4c8 \ubc84\ud37c\ub85c \uc800\uc7a5\ub429\ub2c8\ub2e4.\n\n" + "\uc704\uc5d0\uc11c \uc120\ud0dd\ud55c \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc5d0 \uc758\ud574 \uc0dd\uc131\ub418\ub294 \uac00\uc9c0\uce58\uae30 \ub9c8\uc2a4\ud06c\ub294 \ucd08\uae30 \ud30c\ub77c\ubbf8\ud130 ``name`` \uc5d0 ``weight_mask``\n(\uc989, \ucd08\uae30 \uac00\uc911\uce58 \uc774\ub984\uc5d0 \"_mask\"\ub97c \ubd99\uc778) \uc774\ub984\uc758 \ubaa8\ub4c8 \ubc84\ud37c\ub85c \uc800\uc7a5\ub429\ub2c8\ub2e4.\n\n" ] }, { @@ -134,7 +134,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uc218\uc815\uc774 \ub418\uc9c0 \uc54a\uc740 \uc0c1\ud0dc\uc5d0\uc11c \uc21c\uc804\ud30c\ub97c \uc9c4\ud589\ud558\uae30 \uc704\ud574\uc11c\ub294 ``\uac00\uc911\uce58``\uac12 \uc18d\uc131\uc774 \uc874\uc7ac\ud574\uc57c \ud569\ub2c8\ub2e4.\n``torch.nn.utils.prune`` \ub0b4 \uad6c\ud604\ub41c \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc740 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c \uac00\uc911\uce58\uac12\ub4e4\uc744 \uc774\uc6a9\ud558\uc5ec \n(\uae30\uc874\uc758 \uac00\uc911\uce58\uac12\uc5d0 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c) \uc21c\uc804\ud30c\ub97c \uc9c4\ud589\ud558\uace0, ``weight`` \uc18d\uc131\uac12\uc5d0 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c \uac00\uc911\uce58\uac12\ub4e4\uc744 \uc800\uc7a5\ud569\ub2c8\ub2e4.\n\uc774\uc81c \uac00\uc911\uce58\uac12\ub4e4\uc740 ``module`` \uc758 \ub9e4\uac1c\ubcc0\uc218\uac00 \uc544\ub2c8\ub77c \ud558\ub098\uc758 \uc18d\uc131\uac12\uc73c\ub85c \ucde8\uae09\ub418\ub294 \uc810\uc744 \uc8fc\uc758\ud558\uc138\uc694.\n\n" + "\uc218\uc815\uc774 \ub418\uc9c0 \uc54a\uc740 \uc0c1\ud0dc\uc5d0\uc11c \uc21c\uc804\ud30c\ub97c \uc9c4\ud589\ud558\uae30 \uc704\ud574\uc11c\ub294 ``\uac00\uc911\uce58`` \uac12 \uc18d\uc131\uc774 \uc874\uc7ac\ud574\uc57c \ud569\ub2c8\ub2e4.\n``torch.nn.utils.prune`` \ub0b4 \uad6c\ud604\ub41c \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc740 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c \uac00\uc911\uce58\uac12\ub4e4\uc744 \uc774\uc6a9\ud558\uc5ec\n(\uae30\uc874\uc758 \uac00\uc911\uce58\uac12\uc5d0 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c) \uc21c\uc804\ud30c\ub97c \uc9c4\ud589\ud558\uace0, ``weight`` \uc18d\uc131\uac12\uc5d0 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c \uac00\uc911\uce58\uac12\ub4e4\uc744 \uc800\uc7a5\ud569\ub2c8\ub2e4.\n\uc774\uc81c \uac00\uc911\uce58\uac12\ub4e4\uc740 ``module`` \uc758 \ub9e4\uac1c\ubcc0\uc218\uac00 \uc544\ub2c8\ub77c \ud558\ub098\uc758 \uc18d\uc131\uac12\uc73c\ub85c \ucde8\uae09\ub418\ub294 \uc810\uc744 \uc8fc\uc758\ud558\uc138\uc694.\n\n" ] }, { @@ -152,7 +152,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\ucd5c\uc885\uc801\uc73c\ub85c, \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc740 \ud30c\uc774\ud1a0\uce58\uc758 ``forward_pre_hooks`` \ub97c \uc774\uc6a9\ud558\uc5ec \uac01 \uc21c\uc804\ud30c\uac00 \uc9c4\ud589\ub418\uae30 \uc804\uc5d0 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub429\ub2c8\ub2e4.\n\uad6c\uccb4\uc801\uc73c\ub85c, \uc9c0\uae08\uae4c\uc9c0 \uc9c4\ud589\ud55c \uac83 \ucc98\ub7fc, \ubaa8\ub4c8\uc774 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub418\uc5c8\uc744 \ub54c, \n\uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c \uac01 \ud30c\ub77c\ubbf8\ud130\uac12\ub4e4\uc774 ``forward_pre_hook`` \ub97c \uc5bb\uac8c\ub429\ub2c8\ub2e4.\n\uc774\ub7ec\ud55c \uacbd\uc6b0, ``weight`` \uc774\ub984\uc778 \uae30\uc874 \ud30c\ub77c\ubbf8\ud130\uac12\uc5d0 \ub300\ud574\uc11c\ub9cc \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud558\uc600\uae30 \ub54c\ubb38\uc5d0, \n\ud6c5\uc740 \uc624\uc9c1 1\uac1c\ub9cc \uc874\uc7ac\ud560 \uac83\uc785\ub2c8\ub2e4.\n\n" + "\ucd5c\uc885\uc801\uc73c\ub85c, \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc740 \ud30c\uc774\ud1a0\uce58\uc758 ``forward_pre_hooks`` \ub97c \uc774\uc6a9\ud558\uc5ec \uac01 \uc21c\uc804\ud30c\uac00 \uc9c4\ud589\ub418\uae30 \uc804\uc5d0 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub429\ub2c8\ub2e4.\n\uad6c\uccb4\uc801\uc73c\ub85c, \uc9c0\uae08\uae4c\uc9c0 \uc9c4\ud589\ud55c \uac83 \ucc98\ub7fc, \ubaa8\ub4c8\uc774 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub418\uc5c8\uc744 \ub54c,\n\uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c \uac01 \ud30c\ub77c\ubbf8\ud130\uac12\ub4e4\uc774 ``forward_pre_hook`` \ub97c \uc5bb\uac8c\ub429\ub2c8\ub2e4.\n\uc774\ub7ec\ud55c \uacbd\uc6b0, ``weight`` \uc774\ub984\uc778 \uae30\uc874 \ud30c\ub77c\ubbf8\ud130\uac12\uc5d0 \ub300\ud574\uc11c\ub9cc \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud558\uc600\uae30 \ub54c\ubb38\uc5d0,\n\ud6c5\uc740 \uc624\uc9c1 1\uac1c\ub9cc \uc874\uc7ac\ud560 \uac83\uc785\ub2c8\ub2e4.\n\n" ] }, { @@ -170,7 +170,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uc644\uacb0\uc131\uc744 \uc704\ud574, \ud3b8\ud5a5\uac12\uc5d0 \ub300\ud574\uc11c\ub3c4 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud560 \uc218 \uc788\uc73c\uba70, \n\ubaa8\ub4c8\uc758 \ud30c\ub77c\ubbf8\ud130, \ubc84\ud37c, \ud6c5, \uc18d\uc131\uac12\ub4e4\uc774 \uc5b4\ub5bb\uac8c \ubcc0\uacbd\ub418\ub294\uc9c0 \ud655\uc778\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\ub610 \ub2e4\ub978 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud574\ubcf4\uae30 \uc704\ud574, ``l1_unstructured`` \uac00\uc9c0\uce58\uae30 \ud568\uc218\uc5d0\uc11c \uad6c\ud604\ub41c \ub0b4\uc6a9\uacfc \uac19\uc774, \nL1 Norm \uac12\uc774 \uac00\uc7a5 \uc791\uc740 \ud3b8\ud5a5\uac12 3\uac1c\ub97c \uac00\uc9c0\uce58\uae30\ub97c \uc2dc\ub3c4\ud574\ubd05\uc2dc\ub2e4.\n\n" + "\uc644\uacb0\uc131\uc744 \uc704\ud574, \ud3b8\ud5a5\uac12\uc5d0 \ub300\ud574\uc11c\ub3c4 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud560 \uc218 \uc788\uc73c\uba70,\n\ubaa8\ub4c8\uc758 \ud30c\ub77c\ubbf8\ud130, \ubc84\ud37c, \ud6c5, \uc18d\uc131\uac12\ub4e4\uc774 \uc5b4\ub5bb\uac8c \ubcc0\uacbd\ub418\ub294\uc9c0 \ud655\uc778\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\ub610 \ub2e4\ub978 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud574\ubcf4\uae30 \uc704\ud574, ``l1_unstructured`` \uac00\uc9c0\uce58\uae30 \ud568\uc218\uc5d0\uc11c \uad6c\ud604\ub41c \ub0b4\uc6a9\uacfc \uac19\uc774,\nL1 Norm \uac12\uc774 \uac00\uc7a5 \uc791\uc740 \ud3b8\ud5a5\uac12 3\uac1c\ub97c \uac00\uc9c0\uce58\uae30\ub97c \uc2dc\ub3c4\ud574\ubd05\uc2dc\ub2e4.\n\n" ] }, { @@ -293,7 +293,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c \ubaa8\ub378\uc758 \uc9c1\ub82c\ud654\n---------------------------------------------\n\ub9c8\uc2a4\ud06c \ubc84\ud37c\ub4e4\uacfc \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c \ud150\uc11c \uacc4\uc0b0\uc5d0 \uc0ac\uc6a9\ub41c \uae30\uc874\uc758 \ud30c\ub77c\ubbf8\ud130\ub97c \ud3ec\ud568\ud558\uc5ec \uad00\ub828\ub41c \ubaa8\ub4e0 \ud150\uc11c\uac12\ub4e4\uc740 \n\ud544\uc694\ud55c \uacbd\uc6b0 \ubaa8\ub378\uc758 ``state_dict`` \uc5d0 \uc800\uc7a5\ub418\uae30 \ub584\ubb38\uc5d0, \uc27d\uac8c \uc9c1\ub82c\ud654\ud558\uc5ec \uc800\uc7a5\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n" + "\uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c \ubaa8\ub378\uc758 \uc9c1\ub82c\ud654\n---------------------------------------------\n\ub9c8\uc2a4\ud06c \ubc84\ud37c\ub4e4\uacfc \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c \ud150\uc11c \uacc4\uc0b0\uc5d0 \uc0ac\uc6a9\ub41c \uae30\uc874\uc758 \ud30c\ub77c\ubbf8\ud130\ub97c \ud3ec\ud568\ud558\uc5ec \uad00\ub828\ub41c \ubaa8\ub4e0 \ud150\uc11c\uac12\ub4e4\uc740\n\ud544\uc694\ud55c \uacbd\uc6b0 \ubaa8\ub378\uc758 ``state_dict`` \uc5d0 \uc800\uc7a5\ub418\uae30 \ub584\ubb38\uc5d0, \uc27d\uac8c \uc9c1\ub82c\ud654\ud558\uc5ec \uc800\uc7a5\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n" ] }, { @@ -311,14 +311,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc758 \uc7ac-\ud30c\ub77c\ubbf8\ud130\ud654 \uc81c\uac70\n-----------------------------------------\n\n\uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c \uac83\uc744 \uc601\uad6c\uc801\uc73c\ub85c \ub9cc\ub4e4\uae30 \uc704\ud574\uc11c, \uc7ac-\ud30c\ub77c\ubbf8\ud130\ud654 \uad00\uc810\uc758 \n``weight_orig`` \uc640 ``weight_mask`` \uac12\uc744 \uc81c\uac70\ud558\uace0, ``forward_pre_hook`` \uac12\uc744 \uc81c\uac70\ud569\ub2c8\ub2e4.\n\uc81c\uac70\ud558\uae30 \uc704\ud574 ``torch.nn.utils.prune`` \ub0b4 ``remove`` \ud568\uc218\ub97c \uc774\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub418\uc9c0 \uc54a\uc740 \uac83\ucc98\ub7fc \uc2e4\ud589\ub418\ub294 \uac83\uc774 \uc544\ub2cc \uc810\uc744 \uc8fc\uc758\ud558\uc138\uc694.\n\uc774\ub294 \ub2e8\uc9c0 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c \uc0c1\ud0dc\uc5d0\uc11c \uac00\uc911\uce58 \ud30c\ub77c\ubbf8\ud130\uac12\uc744 \ubaa8\ub378 \ud30c\ub77c\ubbf8\ud130\uac12\uc73c\ub85c \uc7ac\ud560\ub2f9\ud558\ub294 \uac83\uc744 \ud1b5\ud574 \uc601\uad6c\uc801\uc73c\ub85c \ub9cc\ub4dc\ub294 \uac83\uc77c \ubfd0\uc785\ub2c8\ub2e4.\n\n" + "\uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc758 \uc7ac-\ud30c\ub77c\ubbf8\ud130\ud654 \uc81c\uac70\n-----------------------------------------\n\n\uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c \uac83\uc744 \uc601\uad6c\uc801\uc73c\ub85c \ub9cc\ub4e4\uae30 \uc704\ud574\uc11c, \uc7ac-\ud30c\ub77c\ubbf8\ud130\ud654 \uad00\uc810\uc758\n``weight_orig`` \uc640 ``weight_mask`` \uac12\uc744 \uc81c\uac70\ud558\uace0, ``forward_pre_hook`` \uac12\uc744 \uc81c\uac70\ud569\ub2c8\ub2e4.\n\uc81c\uac70\ud558\uae30 \uc704\ud574 ``torch.nn.utils.prune`` \ub0b4 ``remove`` \ud568\uc218\ub97c \uc774\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub418\uc9c0 \uc54a\uc740 \uac83\ucc98\ub7fc \uc2e4\ud589\ub418\ub294 \uac83\uc774 \uc544\ub2cc \uc810\uc744 \uc8fc\uc758\ud558\uc138\uc694.\n\uc774\ub294 \ub2e8\uc9c0 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c \uc0c1\ud0dc\uc5d0\uc11c \uac00\uc911\uce58 \ud30c\ub77c\ubbf8\ud130\uac12\uc744 \ubaa8\ub378 \ud30c\ub77c\ubbf8\ud130\uac12\uc73c\ub85c \uc7ac\ud560\ub2f9\ud558\ub294 \uac83\uc744 \ud1b5\ud574 \uc601\uad6c\uc801\uc73c\ub85c \ub9cc\ub4dc\ub294 \uac83\uc77c \ubfd0\uc785\ub2c8\ub2e4.\n\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "\uc7ac-\ud30c\ub77c\ubbf8\ud130\ud654\ub97c \uc81c\uac70\ud558\uae30 \uc804 \uc0c1\ud0dc \n\n" + "\uc7ac-\ud30c\ub77c\ubbf8\ud130\ud654\ub97c \uc81c\uac70\ud558\uae30 \uc804 \uc0c1\ud0dc\n\n" ] }, { @@ -358,7 +358,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uc7ac-\ud30c\ub77c\ubbf8\ud130\ub97c \uc81c\uac70\ud55c \ud6c4 \uc0c1\ud0dc \n\n" + "\uc7ac-\ud30c\ub77c\ubbf8\ud130\ub97c \uc81c\uac70\ud55c \ud6c4 \uc0c1\ud0dc\n\n" ] }, { @@ -387,7 +387,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\ubaa8\ub378 \ub0b4 \uc5ec\ub7ec \ud30c\ub77c\ubbf8\ud130\uac12\ub4e4\uc5d0 \ub300\ud558\uc5ec \uac00\uc9c0\uce58\uae30 \uae30\ubc95 \uc801\uc6a9\n--------------------------------------\n\n\uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud558\uace0 \uc2f6\uc740 \ud30c\ub77c\ubbf8\ud130\uac12\ub4e4\uc744 \uc9c0\uc815\ud568\uc73c\ub85c\uc368, \uc774\ubc88 \uc608\uc81c\uc5d0\uc11c \ubcfc \uc218 \uc788\ub294 \uac83 \ucc98\ub7fc, \n\uc2e0\uacbd\ub9dd \ubaa8\ub378 \ub0b4 \uc5ec\ub7ec \ud150\uc11c\uac12\ub4e4\uc5d0 \ub300\ud574\uc11c \uc27d\uac8c \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n" + "\ubaa8\ub378 \ub0b4 \uc5ec\ub7ec \ud30c\ub77c\ubbf8\ud130\uac12\ub4e4\uc5d0 \ub300\ud558\uc5ec \uac00\uc9c0\uce58\uae30 \uae30\ubc95 \uc801\uc6a9\n----------------------------------------------------------\n\n\uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud558\uace0 \uc2f6\uc740 \ud30c\ub77c\ubbf8\ud130\uac12\ub4e4\uc744 \uc9c0\uc815\ud568\uc73c\ub85c\uc368, \uc774\ubc88 \uc608\uc81c\uc5d0\uc11c \ubcfc \uc218 \uc788\ub294 \uac83 \ucc98\ub7fc,\n\uc2e0\uacbd\ub9dd \ubaa8\ub378 \ub0b4 \uc5ec\ub7ec \ud150\uc11c\uac12\ub4e4\uc5d0 \ub300\ud574\uc11c \uc27d\uac8c \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n" ] }, { @@ -441,7 +441,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "``torch.nn.utils.prune`` \uc5d0\uc11c \ud655\uc7a5\ub41c \ub9de\ucda4\ud615 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\n------------------------------------------------------------------\n\ub9de\ucda4\ud615 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc740, \ub2e4\ub978 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud558\ub294 \uac83\uacfc \uac19\uc740 \ubc29\uc2dd\uc73c\ub85c, \n``BasePruningMethod`` \uc758 \uae30\ubcf8 \ud074\ub798\uc2a4\uc778 ``nn.utils.prune`` \ubaa8\ub4c8\uc744 \ud65c\uc6a9\ud558\uc5ec \uad6c\ud604\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uae30\ubcf8 \ud074\ub798\uc2a4\ub294 ``__call__``, ``apply_mask``, ``apply``, ``prune``, ``remove`` \uba54\uc18c\ub4dc\ub4e4\uc744 \ub0b4\ud3ec\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n\ud2b9\ubcc4\ud55c \ucf00\uc774\uc2a4\uac00 \uc544\ub2cc \uacbd\uc6b0, \uae30\ubcf8\uc801\uc73c\ub85c \uad6c\uc131\ub41c \uba54\uc18c\ub4dc\ub4e4\uc744 \uc7ac\uad6c\uc131\ud560 \ud544\uc694\uac00 \uc5c6\uc2b5\ub2c8\ub2e4.\n\uadf8\ub7ec\ub098, ``__init__`` (\uad6c\uc131\uc694\uc18c), ``compute_mask`` \n(\uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc758 \ub17c\ub9ac\uc5d0 \ub530\ub77c \uc8fc\uc5b4\uc9c4 \ud150\uc11c\uac12\uc5d0 \ub9c8\uc2a4\ud06c\ub97c \uc801\uc6a9\ud558\ub294 \ubc29\ubc95) \uc744 \uace0\ub824\ud558\uc5ec \uad6c\uc131\ud574\uc57c \ud569\ub2c8\ub2e4.\n\uac8c\ub2e4\uac00, \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc5b4\ub5a0\ud55c \ubc29\uc2dd\uc73c\ub85c \uc801\uc6a9\ud558\ub294\uc9c0 \uba85\ud655\ud558\uac8c \uad6c\uc131\ud574\uc57c \ud569\ub2c8\ub2e4. \n(\uc9c0\uc6d0\ub418\ub294 \uc635\uc158\uc740 ``global``, ``structured``, ``unstructured`` \uc785\ub2c8\ub2e4.)\n\uc774\ub7ec\ud55c \ubc29\uc2dd\uc740, \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \ubc18\ubcf5\uc801\uc73c\ub85c \uc801\uc6a9\ud574\uc57c \ud558\ub294 \uacbd\uc6b0 \ub9c8\uc2a4\ud06c\ub97c \uacb0\ud569\ud558\ub294 \ubc29\ubc95\uc744 \uacb0\uc815\ud558\uae30 \uc704\ud574 \ud544\uc694\ud569\ub2c8\ub2e4.\n\uc989, \uc774\ubbf8 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c \ubaa8\ub378\uc5d0 \ub300\ud574\uc11c \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud560 \ub54c, \n\uae30\uc874\uc758 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub418\uc9c0 \uc54a\uc740 \ud30c\ub77c\ubbf8\ud130 \uac12\uc5d0 \ub300\ud574 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc601\ud5a5\uc744 \ubbf8\uce60 \uac83\uc73c\ub85c \uc608\uc0c1\ub429\ub2c8\ub2e4.\n``PRUNING_TYPE``\uc744 \uc9c0\uc815\ud55c\ub2e4\uba74, \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud558\uae30 \uc704\ud574 \ud30c\ub77c\ubbf8\ud130 \uac12\uc744 \uc62c\ubc14\ub974\uac8c \uc81c\uac70\ud558\ub294 \n``PruningContainer`` (\ub9c8\uc2a4\ud06c \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \ubc18\ubcf5\uc801\uc73c\ub85c \uc801\uc6a9\ud558\ub294 \uac83\uc744 \ucc98\ub9ac\ud558\ub294)\ub97c \uac00\ub2a5\ud558\uac8c \ud569\ub2c8\ub2e4.\n\uc608\ub97c \ub4e4\uc5b4, \ub2e4\ub978 \ubaa8\ub4e0 \ud56d\ubaa9\uc774 \uc874\uc7ac\ud558\ub294 \ud150\uc11c\ub97c \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uad6c\ud604\ud558\uace0 \uc2f6\uc744 \ub54c,\n(\ub610\ub294, \ud150\uc11c\uac00 \uc774\uc804\uc5d0 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc5d0 \uc758\ud574 \uc81c\uac70\ub418\uc5c8\uac70\ub098 \ub0a8\uc544\uc788\ub294 \ud150\uc11c\uc5d0 \ub300\ud574)\n\ud55c \uce35\uc758 \uac1c\ubcc4 \uc5f0\uacb0\uc5d0 \uc791\uc6a9\ud558\uba70 \uc804\uccb4 \uc720\ub2db/\ucc44\ub110 (``'structured'``), \ub610\ub294 \ub2e4\ub978 \ud30c\ub77c\ubbf8\ud130 \uac04 \n(``'global'``) \uc5f0\uacb0\uc5d0\ub294 \uc791\uc6a9\ud558\uc9c0 \uc54a\uae30 \ub54c\ubb38\uc5d0 ``PRUNING_TYPE='unstructured'`` \ubc29\uc2dd\uc73c\ub85c \uc9c4\ud589\ub429\ub2c8\ub2e4.\n\n" + "``torch.nn.utils.prune`` \uc5d0\uc11c \ud655\uc7a5\ub41c \ub9de\ucda4\ud615 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\n------------------------------------------------------------------\n\ub9de\ucda4\ud615 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc740, \ub2e4\ub978 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud558\ub294 \uac83\uacfc \uac19\uc740 \ubc29\uc2dd\uc73c\ub85c,\n``BasePruningMethod`` \uc758 \uae30\ubcf8 \ud074\ub798\uc2a4\uc778 ``nn.utils.prune`` \ubaa8\ub4c8\uc744 \ud65c\uc6a9\ud558\uc5ec \uad6c\ud604\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uae30\ubcf8 \ud074\ub798\uc2a4\ub294 ``__call__``, ``apply_mask``, ``apply``, ``prune``, ``remove`` \uba54\uc18c\ub4dc\ub4e4\uc744 \ub0b4\ud3ec\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n\ud2b9\ubcc4\ud55c \ucf00\uc774\uc2a4\uac00 \uc544\ub2cc \uacbd\uc6b0, \uae30\ubcf8\uc801\uc73c\ub85c \uad6c\uc131\ub41c \uba54\uc18c\ub4dc\ub4e4\uc744 \uc7ac\uad6c\uc131\ud560 \ud544\uc694\uac00 \uc5c6\uc2b5\ub2c8\ub2e4.\n\uadf8\ub7ec\ub098, ``__init__`` (\uad6c\uc131\uc694\uc18c), ``compute_mask``\n(\uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc758 \ub17c\ub9ac\uc5d0 \ub530\ub77c \uc8fc\uc5b4\uc9c4 \ud150\uc11c\uac12\uc5d0 \ub9c8\uc2a4\ud06c\ub97c \uc801\uc6a9\ud558\ub294 \ubc29\ubc95) \uc744 \uace0\ub824\ud558\uc5ec \uad6c\uc131\ud574\uc57c \ud569\ub2c8\ub2e4.\n\uac8c\ub2e4\uac00, \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc5b4\ub5a0\ud55c \ubc29\uc2dd\uc73c\ub85c \uc801\uc6a9\ud558\ub294\uc9c0 \uba85\ud655\ud558\uac8c \uad6c\uc131\ud574\uc57c \ud569\ub2c8\ub2e4.\n(\uc9c0\uc6d0\ub418\ub294 \uc635\uc158\uc740 ``global``, ``structured``, ``unstructured`` \uc785\ub2c8\ub2e4.)\n\uc774\ub7ec\ud55c \ubc29\uc2dd\uc740, \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \ubc18\ubcf5\uc801\uc73c\ub85c \uc801\uc6a9\ud574\uc57c \ud558\ub294 \uacbd\uc6b0 \ub9c8\uc2a4\ud06c\ub97c \uacb0\ud569\ud558\ub294 \ubc29\ubc95\uc744 \uacb0\uc815\ud558\uae30 \uc704\ud574 \ud544\uc694\ud569\ub2c8\ub2e4.\n\uc989, \uc774\ubbf8 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c \ubaa8\ub378\uc5d0 \ub300\ud574\uc11c \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud560 \ub54c,\n\uae30\uc874\uc758 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub418\uc9c0 \uc54a\uc740 \ud30c\ub77c\ubbf8\ud130 \uac12\uc5d0 \ub300\ud574 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc601\ud5a5\uc744 \ubbf8\uce60 \uac83\uc73c\ub85c \uc608\uc0c1\ub429\ub2c8\ub2e4.\n``PRUNING_TYPE`` \uc744 \uc9c0\uc815\ud55c\ub2e4\uba74, \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud558\uae30 \uc704\ud574 \ud30c\ub77c\ubbf8\ud130 \uac12\uc744 \uc62c\ubc14\ub974\uac8c \uc81c\uac70\ud558\ub294\n``PruningContainer`` (\ub9c8\uc2a4\ud06c \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \ubc18\ubcf5\uc801\uc73c\ub85c \uc801\uc6a9\ud558\ub294 \uac83\uc744 \ucc98\ub9ac\ud558\ub294)\ub97c \uac00\ub2a5\ud558\uac8c \ud569\ub2c8\ub2e4.\n\uc608\ub97c \ub4e4\uc5b4, \ub2e4\ub978 \ubaa8\ub4e0 \ud56d\ubaa9\uc774 \uc874\uc7ac\ud558\ub294 \ud150\uc11c\ub97c \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uad6c\ud604\ud558\uace0 \uc2f6\uc744 \ub54c,\n(\ub610\ub294, \ud150\uc11c\uac00 \uc774\uc804\uc5d0 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc5d0 \uc758\ud574 \uc81c\uac70\ub418\uc5c8\uac70\ub098 \ub0a8\uc544\uc788\ub294 \ud150\uc11c\uc5d0 \ub300\ud574)\n\ud55c \uce35\uc758 \uac1c\ubcc4 \uc5f0\uacb0\uc5d0 \uc791\uc6a9\ud558\uba70 \uc804\uccb4 \uc720\ub2db/\ucc44\ub110 (``'structured'``), \ub610\ub294 \ub2e4\ub978 \ud30c\ub77c\ubbf8\ud130 \uac04\n(``'global'``) \uc5f0\uacb0\uc5d0\ub294 \uc791\uc6a9\ud558\uc9c0 \uc54a\uae30 \ub54c\ubb38\uc5d0 ``PRUNING_TYPE='unstructured'`` \ubc29\uc2dd\uc73c\ub85c \uc9c4\ud589\ub429\ub2c8\ub2e4.\n\n" ] }, { @@ -452,7 +452,7 @@ }, "outputs": [], "source": [ - "class FooBarPruningMethod(prune.BasePruningMethod):\n \"\"\"\n \ud150\uc11c \ub0b4 \ub2e4\ub978 \ud56d\ubaa9\ub4e4\uc5d0 \ub300\ud574 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\n \"\"\"\n PRUNING_TYPE = 'unstructured'\n\n def compute_mask(self, t, default_mask):\n mask = default_mask.clone()\n mask.view(-1)[::2] = 0 \n return mask" + "class FooBarPruningMethod(prune.BasePruningMethod):\n \"\"\"\n \ud150\uc11c \ub0b4 \ub2e4\ub978 \ud56d\ubaa9\ub4e4\uc5d0 \ub300\ud574 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\n \"\"\"\n PRUNING_TYPE = 'unstructured'\n\n def compute_mask(self, t, default_mask):\n mask = default_mask.clone()\n mask.view(-1)[::2] = 0\n return mask" ] }, { @@ -470,7 +470,7 @@ }, "outputs": [], "source": [ - "def foobar_unstructured(module, name):\n \"\"\"\n \ud150\uc11c \ub0b4 \ub2e4\ub978 \ubaa8\ub4e0 \ud56d\ubaa9\ub4e4\uc744 \uc81c\uac70\ud558\uc5ec `module` \uc5d0\uc11c `name` \uc774\ub77c\ub294 \ud30c\ub77c\ubbf8\ud130\uc5d0 \ub300\ud574 \uac00\uc790\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\n \ub2e4\uc74c \ub0b4\uc6a9\uc5d0 \ub530\ub77c \ubaa8\ub4c8\uc744 \uc218\uc815 (\ub610\ub294 \uc218\uc815\ub41c \ubaa8\ub4c8\uc744 \ubc18\ud658):\n 1) \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc5d0 \uc758\ud574 \ub9e4\uac1c\ubcc0\uc218 `name` \uc5d0 \uc801\uc6a9\ub41c \uc774\uc9c4 \ub9c8\uc2a4\ud06c\uc5d0 \ud574\ub2f9\ud558\ub294 \uba85\uba85\ub41c \ubc84\ud37c `name+'_mask'` \ub97c \ucd94\uac00\ud569\ub2c8\ub2e4.\n `name` \ud30c\ub77c\ubbf8\ud130\ub294 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c \uac83\uc73c\ub85c \ub300\uccb4\ub418\uba70, \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub418\uc9c0 \uc54a\uc740 \n \uae30\uc874\uc758 \ud30c\ub77c\ubbf8\ud130\ub294 `name+'_orig'` \ub77c\ub294 \uc774\ub984\uc758 \uc0c8\ub85c\uc6b4 \ub9e4\uac1c\ubcc0\uc218\uc5d0 \uc800\uc7a5\ub429\ub2c8\ub2e4.\n\n \uc778\uc790\uac12:\n module (nn.Module): \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud574\uc57c\ud558\ub294 \ud150\uc11c\ub97c \ud3ec\ud568\ud558\ub294 \ubaa8\ub4c8\n name (string): \ubaa8\ub4c8 \ub0b4 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub420 \ud30c\ub77c\ubbf8\ud130\uc758 \uc774\ub984\n\n \ubc18\ud658\uac12:\n module (nn.Module): \uc785\ub825 \ubaa8\ub4c8\uc5d0 \ub300\ud574\uc11c \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c \ubaa8\ub4c8 \n \n \uc608\uc2dc:\n >>> m = nn.Linear(3, 4)\n >>> foobar_unstructured(m, name='bias')\n \"\"\"\n FooBarPruningMethod.apply(module, name)\n return module" + "def foobar_unstructured(module, name):\n \"\"\"\n \ud150\uc11c \ub0b4 \ub2e4\ub978 \ubaa8\ub4e0 \ud56d\ubaa9\ub4e4\uc744 \uc81c\uac70\ud558\uc5ec `module` \uc5d0\uc11c `name` \uc774\ub77c\ub294 \ud30c\ub77c\ubbf8\ud130\uc5d0 \ub300\ud574 \uac00\uc790\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\n \ub2e4\uc74c \ub0b4\uc6a9\uc5d0 \ub530\ub77c \ubaa8\ub4c8\uc744 \uc218\uc815 (\ub610\ub294 \uc218\uc815\ub41c \ubaa8\ub4c8\uc744 \ubc18\ud658):\n 1) \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc5d0 \uc758\ud574 \ub9e4\uac1c\ubcc0\uc218 `name` \uc5d0 \uc801\uc6a9\ub41c \uc774\uc9c4 \ub9c8\uc2a4\ud06c\uc5d0 \ud574\ub2f9\ud558\ub294 \uba85\uba85\ub41c \ubc84\ud37c `name+'_mask'` \ub97c \ucd94\uac00\ud569\ub2c8\ub2e4.\n `name` \ud30c\ub77c\ubbf8\ud130\ub294 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c \uac83\uc73c\ub85c \ub300\uccb4\ub418\uba70, \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub418\uc9c0 \uc54a\uc740\n \uae30\uc874\uc758 \ud30c\ub77c\ubbf8\ud130\ub294 `name+'_orig'` \ub77c\ub294 \uc774\ub984\uc758 \uc0c8\ub85c\uc6b4 \ub9e4\uac1c\ubcc0\uc218\uc5d0 \uc800\uc7a5\ub429\ub2c8\ub2e4.\n\n \uc778\uc790\uac12:\n module (nn.Module): \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc744 \uc801\uc6a9\ud574\uc57c\ud558\ub294 \ud150\uc11c\ub97c \ud3ec\ud568\ud558\ub294 \ubaa8\ub4c8\n name (string): \ubaa8\ub4c8 \ub0b4 \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub420 \ud30c\ub77c\ubbf8\ud130\uc758 \uc774\ub984\n\n \ubc18\ud658\uac12:\n module (nn.Module): \uc785\ub825 \ubaa8\ub4c8\uc5d0 \ub300\ud574\uc11c \uac00\uc9c0\uce58\uae30 \uae30\ubc95\uc774 \uc801\uc6a9\ub41c \ubaa8\ub4c8\n\n \uc608\uc2dc:\n >>> m = nn.Linear(3, 4)\n >>> foobar_unstructured(m, name='bias')\n \"\"\"\n FooBarPruningMethod.apply(module, name)\n return module" ] }, { @@ -508,7 +508,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/74249e7f9f1f398f57ccd094a4f3021b/transfer_learning_tutorial.ipynb b/docs/_downloads/74249e7f9f1f398f57ccd094a4f3021b/transfer_learning_tutorial.ipynb index 4b540878d..074facaf0 100644 --- a/docs/_downloads/74249e7f9f1f398f57ccd094a4f3021b/transfer_learning_tutorial.ipynb +++ b/docs/_downloads/74249e7f9f1f398f57ccd094a4f3021b/transfer_learning_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n\ucef4\ud4e8\ud130 \ube44\uc804(Vision)\uc744 \uc704\ud55c \uc804\uc774\ud559\uc2b5(Transfer Learning)\n=======================================================\n**Author**: `Sasank Chilamkurthy `_\n **\ubc88\uc5ed**: `\ubc15\uc815\ud658 `_\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 \uc804\uc774\ud559\uc2b5(Transfer Learning)\uc744 \uc774\uc6a9\ud558\uc5ec \uc774\ubbf8\uc9c0 \ubd84\ub958\ub97c \uc704\ud55c\n\ud569\uc131\uacf1 \uc2e0\uacbd\ub9dd\uc744 \uc5b4\ub5bb\uac8c \ud559\uc2b5\uc2dc\ud0a4\ub294\uc9c0 \ubc30\uc6cc\ubcf4\uaca0\uc2b5\ub2c8\ub2e4. \uc804\uc774\ud559\uc2b5\uc5d0 \ub300\ud574\uc11c\ub294\n`CS231n \ub178\ud2b8 `__ \uc5d0\uc11c \ub354 \ub9ce\uc740 \ub0b4\uc6a9\uc744\n\uc77d\uc5b4\ubcf4\uc2e4 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc704 \ub178\ud2b8\ub97c \uc778\uc6a9\ud574\ubcf4\uba74,\n\n \uc2e4\uc81c\ub85c \ucda9\ubd84\ud55c \ud06c\uae30\uc758 \ub370\uc774\ud130\uc14b\uc744 \uac16\ucd94\uae30\ub294 \uc0c1\ub300\uc801\uc73c\ub85c \ub4dc\ubb3c\uae30 \ub54c\ubb38\uc5d0,\n (\ubb34\uc791\uc704 \ucd08\uae30\ud654\ub97c \ud1b5\ud574) \ub9e8 \ucc98\uc74c\ubd80\ud130 \ud569\uc131\uacf1 \uc2e0\uacbd\ub9dd(Convolutional\n Network) \uc804\uccb4\ub97c \ud559\uc2b5\ud558\ub294 \uc0ac\ub78c\uc740 \ub9e4\uc6b0 \uc801\uc2b5\ub2c8\ub2e4. \ub300\uc2e0, \ub9e4\uc6b0 \ud070 \ub370\uc774\ud130\uc14b(\uc608.\n 100\uac00\uc9c0 \ubd84\ub958\uc5d0 \ub300\ud574 120\ub9cc\uac1c\uc758 \uc774\ubbf8\uc9c0\uac00 \ud3ec\ud568\ub41c ImageNet)\uc5d0\uc11c \ud569\uc131\uacf1\n \uc2e0\uacbd\ub9dd(ConvNet)\uc744 \ubbf8\ub9ac \ud559\uc2b5\ud55c \ud6c4, \uc774 \ud569\uc131\uacf1 \uc2e0\uacbd\ub9dd\uc744 \uad00\uc2ec\uc788\ub294 \uc791\uc5c5\n \uc744 \uc704\ud55c \ucd08\uae30 \uc124\uc815 \ub610\ub294 \uace0\uc815\ub41c \ud2b9\uc9d5 \ucd94\ucd9c\uae30(fixed feature extractor)\ub85c \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n\n\uc774\ub7ec\ud55c \uc804\uc774\ud559\uc2b5 \uc2dc\ub098\ub9ac\uc624\uc758 \uc8fc\uc694\ud55c 2\uac00\uc9c0\ub294 \ub2e4\uc74c\uacfc \uac19\uc2b5\ub2c8\ub2e4:\n\n- **\ud569\uc131\uacf1 \uc2e0\uacbd\ub9dd\uc758 \ubbf8\uc138\uc870\uc815(finetuning)**: \ubb34\uc791\uc704 \ucd08\uae30\ud654 \ub300\uc2e0, \uc2e0\uacbd\ub9dd\uc744\n ImageNet 1000 \ub370\uc774\ud130\uc14b \ub4f1\uc73c\ub85c \ubbf8\ub9ac \ud559\uc2b5\ud55c \uc2e0\uacbd\ub9dd\uc73c\ub85c \ucd08\uae30\ud654\ud569\ub2c8\ub2e4. \ud559\uc2b5\uc758 \ub098\uba38\uc9c0\n \uacfc\uc815\ub4e4\uc740 \ud3c9\uc0c1\uc2dc\uc640 \uac19\uc2b5\ub2c8\ub2e4.\n- **\uace0\uc815\ub41c \ud2b9\uc9d5 \ucd94\ucd9c\uae30\ub85c\uc368\uc758 \ud569\uc131\uacf1 \uc2e0\uacbd\ub9dd**: \uc5ec\uae30\uc11c\ub294 \ub9c8\uc9c0\ub9c9\uc5d0 \uc644\uc804\ud788 \uc5f0\uacb0\n \ub41c \uacc4\uce35\uc744 \uc81c\uc678\ud55c \ubaa8\ub4e0 \uc2e0\uacbd\ub9dd\uc758 \uac00\uc911\uce58\ub97c \uace0\uc815\ud569\ub2c8\ub2e4. \uc774 \ub9c8\uc9c0\ub9c9\uc758 \uc644\uc804\ud788 \uc5f0\uacb0\ub41c\n \uacc4\uce35\uc740 \uc0c8\ub85c\uc6b4 \ubb34\uc791\uc704\uc758 \uac00\uc911\uce58\ub97c \uac16\ub294 \uacc4\uce35\uc73c\ub85c \ub300\uccb4\ub418\uc5b4 \uc774 \uacc4\uce35\ub9cc \ud559\uc2b5\ud569\ub2c8\ub2e4.\n\n\n" + "\n\ucef4\ud4e8\ud130 \ube44\uc804(Vision)\uc744 \uc704\ud55c \uc804\uc774\ud559\uc2b5(Transfer Learning)\n=======================================================\n**Author**: `Sasank Chilamkurthy `_\n **\ubc88\uc5ed**: `\ubc15\uc815\ud658 `_\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 \uc804\uc774\ud559\uc2b5(Transfer Learning)\uc744 \uc774\uc6a9\ud558\uc5ec \uc774\ubbf8\uc9c0 \ubd84\ub958\ub97c \uc704\ud55c\n\ud569\uc131\uacf1 \uc2e0\uacbd\ub9dd\uc744 \uc5b4\ub5bb\uac8c \ud559\uc2b5\uc2dc\ud0a4\ub294\uc9c0 \ubc30\uc6cc\ubcf4\uaca0\uc2b5\ub2c8\ub2e4. \uc804\uc774\ud559\uc2b5\uc5d0 \ub300\ud574\uc11c\ub294\n`CS231n \ub178\ud2b8 `__ \uc5d0\uc11c \ub354 \ub9ce\uc740 \ub0b4\uc6a9\uc744\n\uc77d\uc5b4\ubcf4\uc2e4 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc704 \ub178\ud2b8\ub97c \uc778\uc6a9\ud574\ubcf4\uba74,\n\n \uc2e4\uc81c\ub85c \ucda9\ubd84\ud55c \ud06c\uae30\uc758 \ub370\uc774\ud130\uc14b\uc744 \uac16\ucd94\uae30\ub294 \uc0c1\ub300\uc801\uc73c\ub85c \ub4dc\ubb3c\uae30 \ub54c\ubb38\uc5d0,\n (\ubb34\uc791\uc704 \ucd08\uae30\ud654\ub97c \ud1b5\ud574) \ub9e8 \ucc98\uc74c\ubd80\ud130 \ud569\uc131\uacf1 \uc2e0\uacbd\ub9dd(Convolutional\n Network) \uc804\uccb4\ub97c \ud559\uc2b5\ud558\ub294 \uc0ac\ub78c\uc740 \ub9e4\uc6b0 \uc801\uc2b5\ub2c8\ub2e4. \ub300\uc2e0, \ub9e4\uc6b0 \ud070 \ub370\uc774\ud130\uc14b(\uc608.\n 100\uac00\uc9c0 \ubd84\ub958\uc5d0 \ub300\ud574 120\ub9cc\uac1c\uc758 \uc774\ubbf8\uc9c0\uac00 \ud3ec\ud568\ub41c ImageNet)\uc5d0\uc11c \ud569\uc131\uacf1\n \uc2e0\uacbd\ub9dd(ConvNet)\uc744 \ubbf8\ub9ac \ud559\uc2b5\ud55c \ud6c4, \uc774 \ud569\uc131\uacf1 \uc2e0\uacbd\ub9dd\uc744 \uad00\uc2ec\uc788\ub294 \uc791\uc5c5\n \uc744 \uc704\ud55c \ucd08\uae30 \uc124\uc815 \ub610\ub294 \uace0\uc815\ub41c \ud2b9\uc9d5 \ucd94\ucd9c\uae30(fixed feature extractor)\ub85c \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n\n\uc774\ub7ec\ud55c \uc804\uc774\ud559\uc2b5 \uc2dc\ub098\ub9ac\uc624\uc758 \uc8fc\uc694\ud55c 2\uac00\uc9c0\ub294 \ub2e4\uc74c\uacfc \uac19\uc2b5\ub2c8\ub2e4:\n\n- **\ud569\uc131\uacf1 \uc2e0\uacbd\ub9dd\uc758 \ubbf8\uc138\uc870\uc815(finetuning)**: \ubb34\uc791\uc704 \ucd08\uae30\ud654 \ub300\uc2e0, \uc2e0\uacbd\ub9dd\uc744\n ImageNet 1000 \ub370\uc774\ud130\uc14b \ub4f1\uc73c\ub85c \ubbf8\ub9ac \ud559\uc2b5\ud55c \uc2e0\uacbd\ub9dd\uc73c\ub85c \ucd08\uae30\ud654\ud569\ub2c8\ub2e4. \ud559\uc2b5\uc758 \ub098\uba38\uc9c0\n \uacfc\uc815\ub4e4\uc740 \ud3c9\uc0c1\uc2dc\uc640 \uac19\uc2b5\ub2c8\ub2e4.\n- **\uace0\uc815\ub41c \ud2b9\uc9d5 \ucd94\ucd9c\uae30\ub85c\uc368\uc758 \ud569\uc131\uacf1 \uc2e0\uacbd\ub9dd**: \uc5ec\uae30\uc11c\ub294 \ub9c8\uc9c0\ub9c9\uc5d0 \uc644\uc804\ud788 \uc5f0\uacb0\n \ub41c \uacc4\uce35\uc744 \uc81c\uc678\ud55c \ubaa8\ub4e0 \uc2e0\uacbd\ub9dd\uc758 \uac00\uc911\uce58\ub97c \uace0\uc815\ud569\ub2c8\ub2e4. \uc774 \ub9c8\uc9c0\ub9c9\uc758 \uc644\uc804\ud788 \uc5f0\uacb0\ub41c\n \uacc4\uce35\uc740 \uc0c8\ub85c\uc6b4 \ubb34\uc791\uc704\uc758 \uac00\uc911\uce58\ub97c \uac16\ub294 \uacc4\uce35\uc73c\ub85c \ub300\uccb4\ub418\uc5b4 \uc774 \uacc4\uce35\ub9cc \ud559\uc2b5\ud569\ub2c8\ub2e4.\n" ] }, { @@ -219,7 +219,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/786469bd4d28fe2528b92a6d12fb189e/flask_rest_api_tutorial.ipynb b/docs/_downloads/786469bd4d28fe2528b92a6d12fb189e/flask_rest_api_tutorial.ipynb index bd22f62de..909dd0e66 100644 --- a/docs/_downloads/786469bd4d28fe2528b92a6d12fb189e/flask_rest_api_tutorial.ipynb +++ b/docs/_downloads/786469bd4d28fe2528b92a6d12fb189e/flask_rest_api_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nFlask\ub97c \uc774\uc6a9\ud558\uc5ec Python\uc5d0\uc11c PyTorch\ub97c REST API\ub85c \ubc30\ud3ec\ud558\uae30\n===========================================================\n**Author**: `Avinash Sajjanshetty `_\n **\ubc88\uc5ed**: `\ubc15\uc815\ud658 `_\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 Flask\ub97c \uc774\uc6a9\ud558\uc5ec PyTorch \ubaa8\ub378\uc744 \ubc30\ud3ec\ud558\uace0 \ubaa8\ub378 \ucd94\ub860(inference)\uc744\n\ud560 \uc218 \uc788\ub294 REST API\ub97c \ub9cc\ub4e4\uc5b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4. \ubbf8\ub9ac \ud6c8\ub828\ub41c DenseNet 121 \ubaa8\ub378\uc744 \ubc30\ud3ec\ud558\uc5ec\n\uc774\ubbf8\uc9c0\ub97c \uc778\uc2dd\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n.. tip:: \uc5ec\uae30\uc11c \uc0ac\uc6a9\ud55c \ubaa8\ub4e0 \ucf54\ub4dc\ub294 MIT \ub77c\uc774\uc120\uc2a4\ub85c \ubc30\ud3ec\ub418\uba70,\n `GitHub `_ \uc5d0\uc11c \ud655\uc778\ud558\uc2e4 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc774\uac83\uc740 PyTorch \ubaa8\ub378\uc744 \uc0c1\uc6a9(production)\uc73c\ub85c \ubc30\ud3ec\ud558\ub294 \ud29c\ud1a0\ub9ac\uc5bc \uc2dc\ub9ac\uc988\uc758 \uccab\ubc88\uc9f8\n\ud3b8\uc785\ub2c8\ub2e4. Flask\ub97c \uc5ec\uae30\uc5d0 \uc18c\uac1c\ub41c \uac83\ucc98\ub7fc \uc0ac\uc6a9\ud558\ub294 \uac83\uc774 PyTorch \ubaa8\ub378\uc744 \uc81c\uacf5\ud558\ub294\n\uac00\uc7a5 \uc26c\uc6b4 \ubc29\ubc95\uc774\uc9c0\ub9cc, \uace0\uc131\ub2a5\uc744 \uc694\uad6c\ud558\ub294 \ub54c\uc5d0\ub294 \uc801\ud569\ud558\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4. \uadf8\uc5d0 \ub300\ud574\uc11c\ub294:\n\n - TorchScript\uc5d0 \uc774\ubbf8 \uc775\uc219\ud558\ub2e4\uba74, \ubc14\ub85c `Loading a TorchScript Model in C++ `_\n \ub97c \uc77d\uc5b4\ubcf4\uc138\uc694.\n\n - TorchScript\uac00 \ubb34\uc5c7\uc778\uc9c0 \uc54c\uc544\ubcf4\ub294 \uac83\uc774 \ud544\uc694\ud558\ub2e4\uba74 `TorchScript \uc18c\uac1c `_\n \ubd80\ud130 \ubcf4\uc2dc\uae38 \ucd94\ucc9c\ud569\ub2c8\ub2e4.\n\n" + "\nFlask\ub97c \uc774\uc6a9\ud558\uc5ec Python\uc5d0\uc11c PyTorch\ub97c REST API\ub85c \ubc30\ud3ec\ud558\uae30\n===========================================================\n**Author**: `Avinash Sajjanshetty `_\n **\ubc88\uc5ed**: `\ubc15\uc815\ud658 `_\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 Flask\ub97c \uc774\uc6a9\ud558\uc5ec PyTorch \ubaa8\ub378\uc744 \ubc30\ud3ec\ud558\uace0 \ubaa8\ub378 \ucd94\ub860(inference)\uc744\n\ud560 \uc218 \uc788\ub294 REST API\ub97c \ub9cc\ub4e4\uc5b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4. \ubbf8\ub9ac \ud6c8\ub828\ub41c DenseNet 121 \ubaa8\ub378\uc744 \ubc30\ud3ec\ud558\uc5ec\n\uc774\ubbf8\uc9c0\ub97c \uc778\uc2dd\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n.. tip:: \uc5ec\uae30\uc11c \uc0ac\uc6a9\ud55c \ubaa8\ub4e0 \ucf54\ub4dc\ub294 MIT \ub77c\uc774\uc120\uc2a4\ub85c \ubc30\ud3ec\ub418\uba70,\n `GitHub `_ \uc5d0\uc11c \ud655\uc778\ud558\uc2e4 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc774\uac83\uc740 PyTorch \ubaa8\ub378\uc744 \uc0c1\uc6a9(production)\uc73c\ub85c \ubc30\ud3ec\ud558\ub294 \ud29c\ud1a0\ub9ac\uc5bc \uc2dc\ub9ac\uc988\uc758 \uccab\ubc88\uc9f8\n\ud3b8\uc785\ub2c8\ub2e4. Flask\ub97c \uc5ec\uae30\uc5d0 \uc18c\uac1c\ub41c \uac83\ucc98\ub7fc \uc0ac\uc6a9\ud558\ub294 \uac83\uc774 PyTorch \ubaa8\ub378\uc744 \uc81c\uacf5\ud558\ub294\n\uac00\uc7a5 \uc26c\uc6b4 \ubc29\ubc95\uc774\uc9c0\ub9cc, \uace0\uc131\ub2a5\uc744 \uc694\uad6c\ud558\ub294 \ub54c\uc5d0\ub294 \uc801\ud569\ud558\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4. \uadf8\uc5d0 \ub300\ud574\uc11c\ub294:\n\n - TorchScript\uc5d0 \uc774\ubbf8 \uc775\uc219\ud558\ub2e4\uba74, \ubc14\ub85c `Loading a TorchScript Model in C++ `_\n \ub97c \uc77d\uc5b4\ubcf4\uc138\uc694.\n\n - TorchScript\uac00 \ubb34\uc5c7\uc778\uc9c0 \uc54c\uc544\ubcf4\ub294 \uac83\uc774 \ud544\uc694\ud558\ub2e4\uba74 `TorchScript \uc18c\uac1c `_\n \ubd80\ud130 \ubcf4\uc2dc\uae38 \ucd94\ucc9c\ud569\ub2c8\ub2e4.\n" ] }, { @@ -29,7 +29,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uc758\uc874\uc131(Dependencies)\n-------------------------\n\n\uc544\ub798 \uba85\ub839\uc5b4\ub97c \uc2e4\ud589\ud558\uc5ec \ud544\uc694\ud55c \ud328\ud0a4\uc9c0\ub4e4\uc744 \uc124\uce58\ud569\ub2c8\ub2e4:\n\n::\n\n $ pip install Flask==1.0.3 torchvision-0.3.0\n\n" + "\uc758\uc874\uc131(Dependencies)\n-------------------------\n\n\uc544\ub798 \uba85\ub839\uc5b4\ub97c \uc2e4\ud589\ud558\uc5ec \ud544\uc694\ud55c \ud328\ud0a4\uc9c0\ub4e4\uc744 \uc124\uce58\ud569\ub2c8\ub2e4:\n\n::\n\n $ pip install Flask==1.0.3 torchvision==0.3.0\n\n" ] }, { @@ -104,7 +104,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\ucd94\ub860(Inference)\n-----------------\n\n\ub2e4\uc74c \uc139\uc158\uc5d0\uc11c\ub294 \ucd94\ub860 \ucf54\ub4dc \uc791\uc131\uc5d0 \uc9d1\uc911\ud558\uaca0\uc2b5\ub2c8\ub2e4. \uba3c\uc800 \uc774\ubbf8\uc9c0\ub97c DenseNet\uc5d0 \uacf5\uae09(feed)\ud560 \uc218 \n\uc788\ub3c4\ub85d \uc900\ube44\ud558\ub294 \ubc29\ubc95\uc744 \uc0b4\ud3b4\ubcf8 \ub4a4, \ubaa8\ub378\ub85c\ubd80\ud130 \uc608\uce21 \uacb0\uacfc\ub97c \uc5bb\ub294 \ubc29\ubc95\uc744 \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc774\ubbf8\uc9c0 \uc900\ube44\ud558\uae30\n~~~~~~~~~~~~~~~~~~~\n\nDenseNet \ubaa8\ub378\uc740 224 x 224\uc758 3\ucc44\ub110 RGB \uc774\ubbf8\uc9c0\ub97c \ud544\uc694\ub85c \ud569\ub2c8\ub2e4.\n\ub610\ud55c \uc774\ubbf8\uc9c0 \ud150\uc11c\ub97c \ud3c9\uaddc \ubc0f \ud45c\uc900\ud3b8\ucc28 \uac12\uc73c\ub85c \uc815\uaddc\ud654\ud569\ub2c8\ub2e4. \uc790\uc138\ud55c \ub0b4\uc6a9\uc740\n`\uc5ec\uae30 `_ \ub97c \ucc38\uace0\ud558\uc138\uc694.\n\n``torchvision`` \ub77c\uc774\ube0c\ub7ec\ub9ac\uc758 ``transforms`` \uc744 \uc774\uc6a9\ud558\uc5ec \ubcc0\ud658 \ud30c\uc774\ud504\ub77c\uc778\n(transform pipeline)\uc744 \uad6c\ucd95\ud569\ub2c8\ub2e4. Transforms\uc640 \uad00\ub828\ud55c \ub354 \uc790\uc138\ud55c \ub0b4\uc6a9\uc740\n`\uc5ec\uae30 `_ \uc5d0\uc11c\n\uc77d\uc5b4\ubcfc \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n" + "\ucd94\ub860(Inference)\n-----------------\n\n\ub2e4\uc74c \uc139\uc158\uc5d0\uc11c\ub294 \ucd94\ub860 \ucf54\ub4dc \uc791\uc131\uc5d0 \uc9d1\uc911\ud558\uaca0\uc2b5\ub2c8\ub2e4. \uba3c\uc800 \uc774\ubbf8\uc9c0\ub97c DenseNet\uc5d0 \uacf5\uae09(feed)\ud560 \uc218\n\uc788\ub3c4\ub85d \uc900\ube44\ud558\ub294 \ubc29\ubc95\uc744 \uc0b4\ud3b4\ubcf8 \ub4a4, \ubaa8\ub378\ub85c\ubd80\ud130 \uc608\uce21 \uacb0\uacfc\ub97c \uc5bb\ub294 \ubc29\ubc95\uc744 \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc774\ubbf8\uc9c0 \uc900\ube44\ud558\uae30\n~~~~~~~~~~~~~~~~~~~\n\nDenseNet \ubaa8\ub378\uc740 224 x 224\uc758 3\ucc44\ub110 RGB \uc774\ubbf8\uc9c0\ub97c \ud544\uc694\ub85c \ud569\ub2c8\ub2e4.\n\ub610\ud55c \uc774\ubbf8\uc9c0 \ud150\uc11c\ub97c \ud3c9\uaddc \ubc0f \ud45c\uc900\ud3b8\ucc28 \uac12\uc73c\ub85c \uc815\uaddc\ud654\ud569\ub2c8\ub2e4. \uc790\uc138\ud55c \ub0b4\uc6a9\uc740\n`\uc5ec\uae30 `_ \ub97c \ucc38\uace0\ud558\uc138\uc694.\n\n``torchvision`` \ub77c\uc774\ube0c\ub7ec\ub9ac\uc758 ``transforms`` \uc744 \uc774\uc6a9\ud558\uc5ec \ubcc0\ud658 \ud30c\uc774\ud504\ub77c\uc778\n(transform pipeline)\uc744 \uad6c\ucd95\ud569\ub2c8\ub2e4. Transforms\uc640 \uad00\ub828\ud55c \ub354 \uc790\uc138\ud55c \ub0b4\uc6a9\uc740\n`\uc5ec\uae30 `_ \uc5d0\uc11c\n\uc77d\uc5b4\ubcfc \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n" ] }, { @@ -274,7 +274,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/78c2efd84ea24eb52e6d415e69187a36/two_layer_net_autograd.ipynb b/docs/_downloads/78c2efd84ea24eb52e6d415e69187a36/two_layer_net_autograd.ipynb index 5491d34f2..4ff6c6895 100644 --- a/docs/_downloads/78c2efd84ea24eb52e6d415e69187a36/two_layer_net_autograd.ipynb +++ b/docs/_downloads/78c2efd84ea24eb52e6d415e69187a36/two_layer_net_autograd.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nPyTorch: Tensor\uc640 autograd\n-------------------------------\n\n\ud558\ub098\uc758 \uc740\ub2c9\uce35(hidden layer)\uacfc \ud3b8\ud5a5(bias)\uc774 \uc5c6\ub294 \uc644\uc804\ud788 \uc5f0\uacb0\ub41c ReLU \uc2e0\uacbd\ub9dd\uc744,\n\uc720\ud074\ub9ac\ub4dc \uac70\ub9ac(Euclidean distance) \uc81c\uacf1\uc744 \ucd5c\uc18c\ud654\ud558\ub294 \uc2dd\uc73c\ub85c x\ub85c\ubd80\ud130 y\ub97c \uc608\uce21\ud558\ub3c4\ub85d\n\ud559\uc2b5\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc774\ubc88\uc5d0\ub294 PyTorch Tensor \uc5f0\uc0b0\uc744 \uc0ac\uc6a9\ud558\uc5ec \uc21c\uc804\ud30c \ub2e8\uacc4\ub97c \uacc4\uc0b0\ud558\uace0, PyTorch autograd\ub97c\n\uc0ac\uc6a9\ud558\uc5ec \ubcc0\ud654\ub3c4(gradient)\ub97c \uacc4\uc0b0\ud558\ub294 \uac83\uc744 \uad6c\ud604\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\nPyTorch Tensor\ub294 \uc5f0\uc0b0 \uadf8\ub798\ud504\uc5d0\uc11c \ub178\ub4dc(node)\ub85c \ud45c\ud604\ub429\ub2c8\ub2e4. \ub9cc\uc57d ``x`` \uac00\n``x.requires_grad=True`` \uc778 Tensor\uba74 ``x.grad`` \ub294 \uc5b4\ub5a4 \uc2a4\uce7c\ub77c \uac12\uc5d0 \ub300\ud55c ``x`` \uc758\n\ubcc0\ud654\ub3c4\ub97c \uac16\ub294 \ub610 \ub2e4\ub978 Tensor\uc785\ub2c8\ub2e4.\n\n" + "\nPyTorch: Tensor\uc640 autograd\n-------------------------------\n\n\ud558\ub098\uc758 \uc740\ub2c9\uce35(hidden layer)\uacfc \ud3b8\ud5a5(bias)\uc774 \uc5c6\ub294 \uc644\uc804\ud788 \uc5f0\uacb0\ub41c ReLU \uc2e0\uacbd\ub9dd\uc744,\n\uc720\ud074\ub9ac\ub4dc \uac70\ub9ac(Euclidean distance) \uc81c\uacf1\uc744 \ucd5c\uc18c\ud654\ud558\ub294 \uc2dd\uc73c\ub85c x\ub85c\ubd80\ud130 y\ub97c \uc608\uce21\ud558\ub3c4\ub85d\n\ud559\uc2b5\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc774\ubc88\uc5d0\ub294 PyTorch Tensor \uc5f0\uc0b0\uc744 \uc0ac\uc6a9\ud558\uc5ec \uc21c\uc804\ud30c \ub2e8\uacc4\ub97c \uacc4\uc0b0\ud558\uace0, PyTorch autograd\ub97c\n\uc0ac\uc6a9\ud558\uc5ec \ubcc0\ud654\ub3c4(gradient)\ub97c \uacc4\uc0b0\ud558\ub294 \uac83\uc744 \uad6c\ud604\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\nPyTorch Tensor\ub294 \uc5f0\uc0b0 \uadf8\ub798\ud504\uc5d0\uc11c \ub178\ub4dc(node)\ub85c \ud45c\ud604\ub429\ub2c8\ub2e4. \ub9cc\uc57d ``x`` \uac00\n``x.requires_grad=True`` \uc778 Tensor\uba74 ``x.grad`` \ub294 \uc5b4\ub5a4 \uc2a4\uce7c\ub77c \uac12\uc5d0 \ub300\ud55c ``x`` \uc758\n\ubcc0\ud654\ub3c4\ub97c \uac16\ub294 \ub610 \ub2e4\ub978 Tensor\uc785\ub2c8\ub2e4.\n" ] }, { @@ -46,7 +46,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/7eba5f912646e2d2239d4e1ce08de5ec/deploy_seq2seq_hybrid_frontend_tutorial.ipynb b/docs/_downloads/7eba5f912646e2d2239d4e1ce08de5ec/deploy_seq2seq_hybrid_frontend_tutorial.ipynb index 9556abad6..8a6c3bcea 100644 --- a/docs/_downloads/7eba5f912646e2d2239d4e1ce08de5ec/deploy_seq2seq_hybrid_frontend_tutorial.ipynb +++ b/docs/_downloads/7eba5f912646e2d2239d4e1ce08de5ec/deploy_seq2seq_hybrid_frontend_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nDeploying a Seq2Seq Model with TorchScript\n==================================================\n**Author:** `Matthew Inkawhich `_\n\n" + "\nDeploying a Seq2Seq Model with TorchScript\n==================================================\n**Author:** `Matthew Inkawhich `_\n" ] }, { @@ -272,7 +272,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/7f37028fb3517ca10f3388f4bb4889b8/data_parallel_tutorial.ipynb b/docs/_downloads/7f37028fb3517ca10f3388f4bb4889b8/data_parallel_tutorial.ipynb index 918ff092c..990bbdbaf 100644 --- a/docs/_downloads/7f37028fb3517ca10f3388f4bb4889b8/data_parallel_tutorial.ipynb +++ b/docs/_downloads/7f37028fb3517ca10f3388f4bb4889b8/data_parallel_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n\uc120\ud0dd \uc0ac\ud56d: \ub370\uc774\ud130 \ubcd1\ub82c \ucc98\ub9ac (Data Parallelism)\n==========================\n**\uae00\uc4f4\uc774**: `Sung Kim `_ and `Jenny Kang `_\n**\ubc88\uc5ed**: '\uc815\uc544\uc9c4 '\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 ``DataParallel`` (\ub370\uc774\ud130 \ubcd1\ub82c) \uc744 \uc0ac\uc6a9\ud558\uc5ec \uc5ec\ub7ec GPU\ub97c \uc0ac\uc6a9\ud558\ub294 \ubc95\uc744 \ubc30\uc6b0\uaca0\uc2b5\ub2c8\ub2e4.\n\nPyTorch\ub97c \ud1b5\ud574 GPU\ub97c \uc0ac\uc6a9\ud558\ub294 \uac83\uc740 \ub9e4\uc6b0 \uc27d\uc2b5\ub2c8\ub2e4. \uba3c\uc800, \ubaa8\ub378\uc744 GPU\uc5d0 \ub123\uc2b5\ub2c8\ub2e4:\n\n.. code:: python\n\n device = torch.device(\"cuda:0\")\n model.to(device)\n\n\uadf8 \ub2e4\uc74c\uc73c\ub85c\ub294 \ubaa8\ub4e0 Tensors \ub97c GPU\ub85c \ubcf5\uc0ac\ud569\ub2c8\ub2e4:\n\n.. code:: python\n\n mytensor = my_tensor.to(device)\n\n''my_tensor.to(device)'' \ub97c \ud638\ucd9c \uc2dc \uc5d0\ub294 ''my_tensor'' \ub97c \ub2e4\uc2dc\uc4f0\ub294 \ub300\uc2e0 ''my_tensor'' \uc758 \ub610\ub2e4\ub978 \ubcf5\uc0ac\ubcf8\uc774 \uc0dd\uae34\ub2e4\ub294 \uc0ac\uc2e4\uc744 \uae30\uc5b5\ud558\uc2ed\uc2dc\uc624. \n\ub2f9\uc2e0\uc740 \uadf8\uac83\uc744 \uc0c8\ub85c\uc6b4 tensor \uc5d0 \uc18c\uc18d\uc2dc\ud0a4\uace0 GPU\uc5d0 \uadf8 tensor\ub97c \uc368\uc57c\ud569\ub2c8\ub2e4.\n\n\uc5ec\ub7ec GPU\ub97c \ud1b5\ud574 \uc55e\uacfc \ub4a4\uc758 \uc804\ud30c\ub97c \uc2e4\ud589\ud558\ub294 \uac83\uc740 \ub2f9\uc5f0\ud55c \uc77c \uc785\ub2c8\ub2e4.\n\uadf8\ub7ec\ub098 PyTorch\ub294 \uae30\ubcf8\uc801 \ud558\ub098\uc758 GPU\ub9cc \uc0ac\uc6a9\ud569\ub2c8\ub2e4. ``DataParallel`` \uc744 \uc774\uc6a9\ud558\uc5ec \ubaa8\ub378\uc744 \ubcd1\ub82c\ub85c \uc2e4\ud589\ud558\uc5ec \ub2e4\uc218\uc758 GPU \uc5d0\uc11c \uc27d\uac8c \uc791\uc5c5\uc744 \uc2e4\ud589\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4:\n\n.. code:: python\n\n model = nn.DataParallel(model)\n\n\uc774\uac83\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc758 \ud575\uc2ec\uc785\ub2c8\ub2e4. \uc790\uc138\ud55c \uc0ac\ud56d\ub4e4\uc5d0 \ub300\ud574\uc11c\ub294 \uc544\ub798\uc5d0\uc11c \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n" + "\n\uc120\ud0dd \uc0ac\ud56d: \ub370\uc774\ud130 \ubcd1\ub82c \ucc98\ub9ac (Data Parallelism)\n====================================================\n\n**\uae00\uc4f4\uc774**: `Sung Kim `_ and `Jenny Kang `_\n**\ubc88\uc5ed**: '\uc815\uc544\uc9c4 '\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 ``DataParallel`` (\ub370\uc774\ud130 \ubcd1\ub82c) \uc744 \uc0ac\uc6a9\ud558\uc5ec \uc5ec\ub7ec GPU\ub97c \uc0ac\uc6a9\ud558\ub294 \ubc95\uc744 \ubc30\uc6b0\uaca0\uc2b5\ub2c8\ub2e4.\n\nPyTorch\ub97c \ud1b5\ud574 GPU\ub97c \uc0ac\uc6a9\ud558\ub294 \uac83\uc740 \ub9e4\uc6b0 \uc27d\uc2b5\ub2c8\ub2e4. \uba3c\uc800, \ubaa8\ub378\uc744 GPU\uc5d0 \ub123\uc2b5\ub2c8\ub2e4:\n\n.. code:: python\n\n device = torch.device(\"cuda:0\")\n model.to(device)\n\n\uadf8 \ub2e4\uc74c\uc73c\ub85c\ub294 \ubaa8\ub4e0 Tensors \ub97c GPU\ub85c \ubcf5\uc0ac\ud569\ub2c8\ub2e4:\n\n.. code:: python\n\n mytensor = my_tensor.to(device)\n\n''my_tensor.to(device)'' \ub97c \ud638\ucd9c \uc2dc \uc5d0\ub294 ''my_tensor'' \ub97c \ub2e4\uc2dc\uc4f0\ub294 \ub300\uc2e0 ''my_tensor'' \uc758 \ub610\ub2e4\ub978 \ubcf5\uc0ac\ubcf8\uc774 \uc0dd\uae34\ub2e4\ub294 \uc0ac\uc2e4\uc744 \uae30\uc5b5\ud558\uc2ed\uc2dc\uc624.\n\ub2f9\uc2e0\uc740 \uadf8\uac83\uc744 \uc0c8\ub85c\uc6b4 tensor \uc5d0 \uc18c\uc18d\uc2dc\ud0a4\uace0 GPU\uc5d0 \uadf8 tensor\ub97c \uc368\uc57c\ud569\ub2c8\ub2e4.\n\n\uc5ec\ub7ec GPU\ub97c \ud1b5\ud574 \uc55e\uacfc \ub4a4\uc758 \uc804\ud30c\ub97c \uc2e4\ud589\ud558\ub294 \uac83\uc740 \ub2f9\uc5f0\ud55c \uc77c \uc785\ub2c8\ub2e4.\n\uadf8\ub7ec\ub098 PyTorch\ub294 \uae30\ubcf8\uc801 \ud558\ub098\uc758 GPU\ub9cc \uc0ac\uc6a9\ud569\ub2c8\ub2e4. ``DataParallel`` \uc744 \uc774\uc6a9\ud558\uc5ec \ubaa8\ub378\uc744 \ubcd1\ub82c\ub85c \uc2e4\ud589\ud558\uc5ec \ub2e4\uc218\uc758 GPU \uc5d0\uc11c \uc27d\uac8c \uc791\uc5c5\uc744 \uc2e4\ud589\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4:\n\n.. code:: python\n\n model = nn.DataParallel(model)\n\n\uc774\uac83\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc758 \ud575\uc2ec\uc785\ub2c8\ub2e4. \uc790\uc138\ud55c \uc0ac\ud56d\ub4e4\uc5d0 \ub300\ud574\uc11c\ub294 \uc544\ub798\uc5d0\uc11c \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n" ] }, { @@ -58,7 +58,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\ub354\ubbf8(Dummy) \ub370\uc774\ud130\uc14b\n-------------\n\n\ub354\ubbf8(ramdom) \ub370\uc774\ud130\uc14b\uc744 \ub9cc\ub4e4\uc5b4 \ubd05\uc2dc\ub2e4. Getitem \ub9cc \uad6c\ud604\ud558\uba74 \ub429\ub2c8\ub2e4.\n\n\n" + "\ub354\ubbf8(Dummy) \ub370\uc774\ud130\uc14b\n-----------------------\n\n\ub354\ubbf8(ramdom) \ub370\uc774\ud130\uc14b\uc744 \ub9cc\ub4e4\uc5b4 \ubd05\uc2dc\ub2e4. Getitem \ub9cc \uad6c\ud604\ud558\uba74 \ub429\ub2c8\ub2e4.\n\n\n" ] }, { @@ -76,7 +76,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uac04\ub2e8\ud55c \ubaa8\ub378\n ------------\n\n \ub370\ubaa8\ub97c \uc704\ud574 \ubaa8\ub378\uc740 \uc785\ub825\uc744 \ubc1b\uace0 \uc120\ud615 \uc5f0\uc0b0\uc744 \uc218\ud589\ud558\uba70 \ucd9c\ub825\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4. \uadf8\ub7ec\ub098 ``DataParallel`` \uc758 \uc5b4\ub5a4 \ubaa8\ub378 (CNN, RNN, Capsule Net \ub4f1) \uc5d0\uc11c\ub4e0 \uc0ac\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc6b0\ub9ac\ub294 input\uacfc output\uc758 \ud06c\uae30\ub97c \ubaa8\ub2c8\ud130\ub9c1\ud558\uae30 \uc704\ud574 \ubaa8\ub378\uc548\uc5d0 print \ubb38\uc744 \ub123\uc5c8\uc2b5\ub2c8\ub2e4.\n \ubb34\uc5c7\uc774 \ubc30\uce58 \uc21c\uc704 (batch rank) 0 \uc5d0 \ud504\ub9b0\ud2b8 \ub418\ub294\uc9c0 \uc8fc\uc758 \uae4a\uac8c \ubd10\uc8fc\uc2dc\uae38 \ubc14\ub78d\ub2c8\ub2e4.\n\n\n" + "\uac04\ub2e8\ud55c \ubaa8\ub378\n------------\n\n\ub370\ubaa8\ub97c \uc704\ud574 \ubaa8\ub378\uc740 \uc785\ub825\uc744 \ubc1b\uace0 \uc120\ud615 \uc5f0\uc0b0\uc744 \uc218\ud589\ud558\uba70 \ucd9c\ub825\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4. \uadf8\ub7ec\ub098 ``DataParallel`` \uc758 \uc5b4\ub5a4 \ubaa8\ub378 (CNN, RNN, Capsule Net \ub4f1) \uc5d0\uc11c\ub4e0 \uc0ac\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc6b0\ub9ac\ub294 input\uacfc output\uc758 \ud06c\uae30\ub97c \ubaa8\ub2c8\ud130\ub9c1\ud558\uae30 \uc704\ud574 \ubaa8\ub378\uc548\uc5d0 print \ubb38\uc744 \ub123\uc5c8\uc2b5\ub2c8\ub2e4.\n\ubb34\uc5c7\uc774 \ubc30\uce58 \uc21c\uc704 (batch rank) 0 \uc5d0 \ud504\ub9b0\ud2b8 \ub418\ub294\uc9c0 \uc8fc\uc758 \uae4a\uac8c \ubd10\uc8fc\uc2dc\uae38 \ubc14\ub78d\ub2c8\ub2e4.\n\n\n" ] }, { @@ -94,7 +94,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\ubaa8\ub378\uacfc \ub370\uc774\ud130 \ubcd1\ub82c\uc758 \uad6c\ud604\n-----------------------------\n\n\uc774\uac83\uc740 \uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc758 \ud575\uc2ec \ubd80\ubd84\uc785\ub2c8\ub2e4. \uba3c\uc800, model instance \ub97c \ub9cc\ub4e4\uace0 \uac00\uc9c0\uace0 \uc788\ub294 GPU\uac00 \uc5ec\ub7ec\uac1c\uc778\uc9c0 \ud655\uc778\ud574\uc57c\ud569\ub2c8\ub2e4.\n\ub9cc\uc57d \ub2e4\uc218\uc758 GPU\ub97c \ubcf4\uc720\uc911\uc774\ub77c\uba74, ``nn.DataParallel`` \uc744 \uc0ac\uc6a9\ud558\uc5ec \ubaa8\ub378\uc744 \ub798\ud551 (wrapping) \ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \n\uadf8\ub7f0 \ub2e4\uc74c ``model.to(device)`` \ub97c \ud1b5\ud558\uc5ec \ubaa8\ub378\uc744 GPU\uc5d0 \ub123\uc744 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\n" + "\ubaa8\ub378\uacfc \ub370\uc774\ud130 \ubcd1\ub82c\uc758 \uad6c\ud604\n-----------------------------\n\n\uc774\uac83\uc740 \uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc758 \ud575\uc2ec \ubd80\ubd84\uc785\ub2c8\ub2e4. \uba3c\uc800, model instance \ub97c \ub9cc\ub4e4\uace0 \uac00\uc9c0\uace0 \uc788\ub294 GPU\uac00 \uc5ec\ub7ec\uac1c\uc778\uc9c0 \ud655\uc778\ud574\uc57c\ud569\ub2c8\ub2e4.\n\ub9cc\uc57d \ub2e4\uc218\uc758 GPU\ub97c \ubcf4\uc720\uc911\uc774\ub77c\uba74, ``nn.DataParallel`` \uc744 \uc0ac\uc6a9\ud558\uc5ec \ubaa8\ub378\uc744 \ub798\ud551 (wrapping) \ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uadf8\ub7f0 \ub2e4\uc74c ``model.to(device)`` \ub97c \ud1b5\ud558\uc5ec \ubaa8\ub378\uc744 GPU\uc5d0 \ub123\uc744 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\n" ] }, { @@ -157,7 +157,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/813e9f58ce3b4f2283d6c6cee9fee5a7/custom_dataset_transforms_loader.ipynb b/docs/_downloads/813e9f58ce3b4f2283d6c6cee9fee5a7/custom_dataset_transforms_loader.ipynb index a6f129046..d07cd2073 100644 --- a/docs/_downloads/813e9f58ce3b4f2283d6c6cee9fee5a7/custom_dataset_transforms_loader.ipynb +++ b/docs/_downloads/813e9f58ce3b4f2283d6c6cee9fee5a7/custom_dataset_transforms_loader.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n\uc0ac\uc6a9\uc790 \uc815\uc758 PyTorch Dataloader \uc791\uc131\ud558\uae30\n===========================================================\n\n\uba38\uc2e0\ub7ec\ub2dd \uc54c\uace0\ub9ac\uc998\uc744 \uac1c\ubc1c\ud558\uae30 \uc704\ud574\uc11c\ub294 \ub370\uc774\ud130 \uc804\ucc98\ub9ac\uc5d0 \ub9ce\uc740 \ub178\ub825\uc774 \ud544\uc694\ud569\ub2c8\ub2e4. PyTorch\ub294 \ub370\uc774\ud130\ub97c \ub85c\ub4dc\ud558\ub294\ub370 \uc27d\uace0 \uac00\ub2a5\ud558\ub2e4\uba74\n\ub354 \uc88b\uc740 \uac00\ub3c5\uc131\uc744 \uac00\uc9c4 \ucf54\ub4dc\ub97c \ub9cc\ub4e4\uae30\uc704\ud574 \ub9ce\uc740 \ub3c4\uad6c\ub4e4\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4. \uc774 \ub808\uc2dc\ud53c\uc5d0\uc11c\ub294 \ub2e4\uc74c \uc138 \uac00\uc9c0\ub97c \ubc30\uc6b8 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n 1. PyTorch \ub370\uc774\ud130\uc14b API\ub4e4\uc744 \uc774\uc6a9\ud558\uc5ec \uc0ac\uc6a9\uc790 \uc815\uc758 \ub370\uc774\ud130\uc14b \ub9cc\ub4e4\uae30.\n 2. \uad6c\uc131\uac00\ub2a5\ud558\uba70 \ud638\ucd9c \ub420 \uc218 \uc788\ub294 \uc0ac\uc6a9\uc790 \uc815\uc758 transform \ub9cc\ub4e4\uae30.\n 3. \uc774\ub7ec\ud55c \ucef4\ud3ec\ub10c\ud2b8\ub4e4\uc744 \ud569\uccd0\uc11c \uc0ac\uc6a9\uc790 \uc815\uc758 dataloader \ub9cc\ub4e4\uae30.\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc744 \uc2e4\ud589\ud558\uae30 \uc704\ud574\uc11c\ub294 \ub2e4\uc74c\uc758 \ud328\ud0a4\uc9c0\ub4e4\uc774 \uc124\uce58 \ub418\uc5c8\ub294\uc9c0 \ud655\uc778\ud574 \uc8fc\uc138\uc694.\n - ``scikit-image``: \uc774\ubbf8\uc9c0 I/O\uc640 \uc774\ubbf8\uc9c0 \ubcc0\ud615\uc5d0 \ud544\uc694\ud569\ub2c8\ub2e4.\n - ``pandas``: CSV\ub97c \ub354 \uc27d\uac8c \ud30c\uc2f1\ud558\uae30 \uc704\ud574 \ud544\uc694\ud569\ub2c8\ub2e4.\n\n\uc791\uc131\ub418\uace0 \uc788\ub294 \uc774 \uc2dc\uc810\uc5d0\uc11c, \uc774 \ub808\uc2dc\ud53c\ub294 `Sasank Chilamkurthy `__ \uc758 \uc624\ub9ac\uc9c0\ub110 \ud29c\ud1a0\ub9ac\uc5bc\uc744 \ubc14\ud0d5\uc73c\ub85c \ud558\uba70\n\ub098\uc911\uc5d0\ub294 `Joe Spisak `__ \uc5d0 \uc758\ud574 \uc218\uc815\ub418\uc5c8\uc2b5\ub2c8\ub2e4.\n\ud55c\uad6d\uc5b4\ub85c `Jae Joong Lee `__ \uc5d0 \uc758\ud574 \ubc88\uc5ed\ub418\uc5c8\uc2b5\ub2c8\ub2e4.\n\n" + "\n\uc0ac\uc6a9\uc790 \uc815\uc758 PyTorch Dataloader \uc791\uc131\ud558\uae30\n===========================================================\n\n\uba38\uc2e0\ub7ec\ub2dd \uc54c\uace0\ub9ac\uc998\uc744 \uac1c\ubc1c\ud558\uae30 \uc704\ud574\uc11c\ub294 \ub370\uc774\ud130 \uc804\ucc98\ub9ac\uc5d0 \ub9ce\uc740 \ub178\ub825\uc774 \ud544\uc694\ud569\ub2c8\ub2e4. PyTorch\ub294 \ub370\uc774\ud130\ub97c \ub85c\ub4dc\ud558\ub294\ub370 \uc27d\uace0 \uac00\ub2a5\ud558\ub2e4\uba74\n\ub354 \uc88b\uc740 \uac00\ub3c5\uc131\uc744 \uac00\uc9c4 \ucf54\ub4dc\ub97c \ub9cc\ub4e4\uae30\uc704\ud574 \ub9ce\uc740 \ub3c4\uad6c\ub4e4\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4. \uc774 \ub808\uc2dc\ud53c\uc5d0\uc11c\ub294 \ub2e4\uc74c \uc138 \uac00\uc9c0\ub97c \ubc30\uc6b8 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n 1. PyTorch \ub370\uc774\ud130\uc14b API\ub4e4\uc744 \uc774\uc6a9\ud558\uc5ec \uc0ac\uc6a9\uc790 \uc815\uc758 \ub370\uc774\ud130\uc14b \ub9cc\ub4e4\uae30.\n 2. \uad6c\uc131\uac00\ub2a5\ud558\uba70 \ud638\ucd9c \ub420 \uc218 \uc788\ub294 \uc0ac\uc6a9\uc790 \uc815\uc758 transform \ub9cc\ub4e4\uae30.\n 3. \uc774\ub7ec\ud55c \ucef4\ud3ec\ub10c\ud2b8\ub4e4\uc744 \ud569\uccd0\uc11c \uc0ac\uc6a9\uc790 \uc815\uc758 dataloader \ub9cc\ub4e4\uae30.\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc744 \uc2e4\ud589\ud558\uae30 \uc704\ud574\uc11c\ub294 \ub2e4\uc74c\uc758 \ud328\ud0a4\uc9c0\ub4e4\uc774 \uc124\uce58 \ub418\uc5c8\ub294\uc9c0 \ud655\uc778\ud574 \uc8fc\uc138\uc694.\n - ``scikit-image``: \uc774\ubbf8\uc9c0 I/O\uc640 \uc774\ubbf8\uc9c0 \ubcc0\ud615\uc5d0 \ud544\uc694\ud569\ub2c8\ub2e4.\n - ``pandas``: CSV\ub97c \ub354 \uc27d\uac8c \ud30c\uc2f1\ud558\uae30 \uc704\ud574 \ud544\uc694\ud569\ub2c8\ub2e4.\n\n\uc791\uc131\ub418\uace0 \uc788\ub294 \uc774 \uc2dc\uc810\uc5d0\uc11c, \uc774 \ub808\uc2dc\ud53c\ub294 `Sasank Chilamkurthy `__ \uc758 \uc624\ub9ac\uc9c0\ub110 \ud29c\ud1a0\ub9ac\uc5bc\uc744 \ubc14\ud0d5\uc73c\ub85c \ud558\uba70\n\ub098\uc911\uc5d0\ub294 `Joe Spisak `__ \uc5d0 \uc758\ud574 \uc218\uc815\ub418\uc5c8\uc2b5\ub2c8\ub2e4.\n\ud55c\uad6d\uc5b4\ub85c `Jae Joong Lee `__ \uc5d0 \uc758\ud574 \ubc88\uc5ed\ub418\uc5c8\uc2b5\ub2c8\ub2e4.\n" ] }, { @@ -147,7 +147,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uc6b0\ub9ac\ub294 \uc9c0\uae08\uae4c\uc9c0 \uc5b4\ub290\uc815\ub3c4 \uc0ac\uc6a9\uc790 \uc815\uc758 \ub370\uc774\ud130\uc14b\uc744 \ub9cc\ub4e4\uc5b4 \ubcf4\uc558\ub294\ub370 \uc774\uc81c\ub294 \uc0ac\uc6a9\uc790 \uc815\uc758 \ubcc0\ud615\uc744 \ub9cc\ub4e4 \ucc28\ub840 \uc785\ub2c8\ub2e4.\n\ucef4\ud4e8\ud130 \ube44\uc804\uc5d0\uc11c\ub294 \uc0ac\uc6a9\uc790 \uc815\uc758 \ubcc0\ud615\uc740 \uc54c\uace0\ub9ac\uc998\uc744 \uc77c\ubc18\ud654\uc2dc\ud0a4\uace0 \uc815\ud655\ub3c4\ub97c \uc62c\ub9ac\ub294\ub370 \ub3c4\uc6c0\uc744 \uc90d\ub2c8\ub2e4.\n\ubcc0\ud615\ub4e4\uc740 \ud6c8\ub828\uc2dc\uc5d0 \uc0ac\uc6a9\uc774 \ub418\uba70 \uc8fc\ub85c \ub370\uc774\ud130 \uc99d\uac15\uc73c\ub85c \ucc38\uc870\ub418\uba70 \ucd5c\uadfc\uc758 \ubaa8\ub378 \uac1c\ubc1c\uc5d0\uc120 \ud754\ud788 \uc0ac\uc6a9\ub429\ub2c8\ub2e4.\n\n\ub370\uc774\ud130\uc14b\uc744 \ub2e4\ub8f0\ub54c \uc790\uc8fc \uc77c\uc5b4\ub098\ub294 \ubb38\uc81c\uc911 \ud558\ub098\ub294 \ubaa8\ub4e0 \uc0d8\ud50c\ub4e4\uc774 \uac19\uc740 \ud06c\uae30\ub97c \uac00\uc9c0\uace0 \uc788\uc9c0 \uc54a\uc744 \uacbd\uc6b0\uc785\ub2c8\ub2e4.\n\ub300\ubd80\ubd84\uc758 \uc2e0\uacbd\ub9dd\ub4e4\uc740 \ubbf8\ub9ac \uc815\ud574\uc9c4 \ud06c\uae30\uc758 \uc774\ubbf8\uc9c0\ub4e4\uc744 \ubc1b\uc544\ub4dc\ub9bd\ub2c8\ub2e4.\n\uadf8\ub807\uae30 \ub54c\ubb38\uc5d0 \uc6b0\ub9ac\ub294 \uc804\ucc98\ub9ac \ucf54\ub4dc\ub97c \uc791\uc131\ud574\uc57c\ud560 \ud544\uc694\uac00 \uc788\uc2b5\ub2c8\ub2e4.\n\uc774\uc81c \uc138\uac1c\uc758 \ubcc0\ud615\uc744 \ub9cc\ub4e4\uc5b4 \ubd05\uc2dc\ub2e4.\n\n- ``Rescale``: \uc774\ubbf8\uc9c0 \ud06c\uae30\ub97c \ubcc0\uacbd\ud560\ub54c \uc0ac\uc6a9\ub429\ub2c8\ub2e4.\n- ``RandomCrop``: \ubb34\uc791\uc704\ub85c \uc774\ubbf8\uc9c0\ub97c \uc798\ub77c\ub0b4\uba70 \ub370\uc774\ud130 \uc99d\uac15\uc5d0 \uc4f0\uc785\ub2c8\ub2e4.\n- ``ToTensor``: Numpy \uc774\ubbf8\uc9c0\ub4e4\uc744 \ud30c\uc774\ud1a0\uce58 \uc774\ubbf8\uc9c0\ub85c \ubcc0\ud658\ud560\ub54c \uc0ac\uc6a9\ub429\ub2c8\ub2e4. (\uadf8\ub7ec\uae30 \uc704\ud574\uc11c\ub294 \uc774\ubbf8\uc9c0 \ucc28\uc6d0\uc758 \uc21c\uc11c\ub97c \ubc14\uafd4\uc57c\ud569\ub2c8\ub2e4.)\n\n\uc6b0\ub9ac\ub294 \uc704\uc758 \uc138\uac1c\uc758 \ubcc0\ud615\ub4e4\uc744 \ub2e8\uc21c\ud55c \ud568\uc218 \ub300\uc2e0\uc5d0 \ud638\ucd9c\uac00\ub2a5\ud55c \ud074\ub798\uc2a4\ub85c \ub9cc\ub4e4\uc5b4\uc11c \ub9e4\ubc88 \ubcc0\ud615\uc774 \ud638\ucd9c\ub420\ub54c \ud56d\uc0c1 \ub9e4\uac1c\ubcc0\uc218\uac00 \ub118\uaca8\uc9c0\uc9c0 \uc54a\ub3c4\ub85d \ud560\uac81\ub2c8\ub2e4.\n\uadf8\ub7ec\uae30 \uc704\ud574\uc11c\ub294 \uc6b0\ub9ac\ub294 \ub2e8\uc21c\ud788 ``__call__`` \uba54\uc18c\ub4dc\ub97c \ub9cc\ub4e4\uace0 \ud544\uc694\ud558\ub2e4\uba74 ``__init__`` \ub97c \ub9cc\ub4e4\uba74 \ub429\ub2c8\ub2e4.\n\uadf8\ub7ec\uba74 \uc6b0\ub9ac\ub294 \ubcc0\ud615\uc744 \uc774\ub7f0\uc2dd\uc73c\ub85c \uc0ac\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n::\n\n tsfm = Transform(params)\n transformed_sample = tsfm(sample)\n\n\uc5b4\ub5bb\uac8c \uc774\ub7f0 \ubcc0\ud615\ub4e4\uc774 \uc774\ubbf8\uc9c0\uc640 \ub79c\ub4dc\ub9c8\ud06c\uc5d0 \uc801\uc6a9\uc774 \ub418\uc5c8\ub294\uc9c0 \uc544\ub798\ub97c \ubd10\uc8fc\uc2dc\uae38 \ubc14\ub78d\ub2c8\ub2e4.\n\n\n" + "\uc6b0\ub9ac\ub294 \uc9c0\uae08\uae4c\uc9c0 \uc5b4\ub290\uc815\ub3c4 \uc0ac\uc6a9\uc790 \uc815\uc758 \ub370\uc774\ud130\uc14b\uc744 \ub9cc\ub4e4\uc5b4 \ubcf4\uc558\ub294\ub370 \uc774\uc81c\ub294 \uc0ac\uc6a9\uc790 \uc815\uc758 \ubcc0\ud615\uc744 \ub9cc\ub4e4 \ucc28\ub840 \uc785\ub2c8\ub2e4.\n\ucef4\ud4e8\ud130 \ube44\uc804\uc5d0\uc11c\ub294 \uc0ac\uc6a9\uc790 \uc815\uc758 \ubcc0\ud615\uc740 \uc54c\uace0\ub9ac\uc998\uc744 \uc77c\ubc18\ud654\uc2dc\ud0a4\uace0 \uc815\ud655\ub3c4\ub97c \uc62c\ub9ac\ub294\ub370 \ub3c4\uc6c0\uc744 \uc90d\ub2c8\ub2e4.\n\ubcc0\ud615\ub4e4\uc740 \ud6c8\ub828\uc2dc\uc5d0 \uc0ac\uc6a9\uc774 \ub418\uba70 \uc8fc\ub85c \ub370\uc774\ud130 \uc99d\uac15\uc73c\ub85c \ucc38\uc870\ub418\uba70 \ucd5c\uadfc\uc758 \ubaa8\ub378 \uac1c\ubc1c\uc5d0\uc120 \ud754\ud788 \uc0ac\uc6a9\ub429\ub2c8\ub2e4.\n\n\ub370\uc774\ud130\uc14b\uc744 \ub2e4\ub8f0\ub54c \uc790\uc8fc \uc77c\uc5b4\ub098\ub294 \ubb38\uc81c\uc911 \ud558\ub098\ub294 \ubaa8\ub4e0 \uc0d8\ud50c\ub4e4\uc774 \uac19\uc740 \ud06c\uae30\ub97c \uac00\uc9c0\uace0 \uc788\uc9c0 \uc54a\uc744 \uacbd\uc6b0\uc785\ub2c8\ub2e4.\n\ub300\ubd80\ubd84\uc758 \uc2e0\uacbd\ub9dd\ub4e4\uc740 \ubbf8\ub9ac \uc815\ud574\uc9c4 \ud06c\uae30\uc758 \uc774\ubbf8\uc9c0\ub4e4\uc744 \ubc1b\uc544\ub4e4\uc785\ub2c8\ub2e4.\n\uadf8\ub807\uae30 \ub54c\ubb38\uc5d0 \uc6b0\ub9ac\ub294 \uc804\ucc98\ub9ac \ucf54\ub4dc\ub97c \uc791\uc131\ud574\uc57c\ud560 \ud544\uc694\uac00 \uc788\uc2b5\ub2c8\ub2e4.\n\uc774\uc81c \uc138\uac1c\uc758 \ubcc0\ud615\uc744 \ub9cc\ub4e4\uc5b4 \ubd05\uc2dc\ub2e4.\n\n- ``Rescale``: \uc774\ubbf8\uc9c0 \ud06c\uae30\ub97c \ubcc0\uacbd\ud560\ub54c \uc0ac\uc6a9\ub429\ub2c8\ub2e4.\n- ``RandomCrop``: \ubb34\uc791\uc704\ub85c \uc774\ubbf8\uc9c0\ub97c \uc798\ub77c\ub0b4\uba70 \ub370\uc774\ud130 \uc99d\uac15\uc5d0 \uc4f0\uc785\ub2c8\ub2e4.\n- ``ToTensor``: Numpy \uc774\ubbf8\uc9c0\ub4e4\uc744 \ud30c\uc774\ud1a0\uce58 \uc774\ubbf8\uc9c0\ub85c \ubcc0\ud658\ud560\ub54c \uc0ac\uc6a9\ub429\ub2c8\ub2e4. (\uadf8\ub7ec\uae30 \uc704\ud574\uc11c\ub294 \uc774\ubbf8\uc9c0 \ucc28\uc6d0\uc758 \uc21c\uc11c\ub97c \ubc14\uafd4\uc57c\ud569\ub2c8\ub2e4.)\n\n\uc6b0\ub9ac\ub294 \uc704\uc758 \uc138\uac1c\uc758 \ubcc0\ud615\ub4e4\uc744 \ub2e8\uc21c\ud55c \ud568\uc218 \ub300\uc2e0\uc5d0 \ud638\ucd9c\uac00\ub2a5\ud55c \ud074\ub798\uc2a4\ub85c \ub9cc\ub4e4\uc5b4\uc11c \ub9e4\ubc88 \ubcc0\ud615\uc774 \ud638\ucd9c\ub420\ub54c \ud56d\uc0c1 \ub9e4\uac1c\ubcc0\uc218\uac00 \ub118\uaca8\uc9c0\uc9c0 \uc54a\ub3c4\ub85d \ud560\uac81\ub2c8\ub2e4.\n\uadf8\ub7ec\uae30 \uc704\ud574\uc11c\ub294 \uc6b0\ub9ac\ub294 \ub2e8\uc21c\ud788 ``__call__`` \uba54\uc18c\ub4dc\ub97c \ub9cc\ub4e4\uace0 \ud544\uc694\ud558\ub2e4\uba74 ``__init__`` \ub97c \ub9cc\ub4e4\uba74 \ub429\ub2c8\ub2e4.\n\uadf8\ub7ec\uba74 \uc6b0\ub9ac\ub294 \ubcc0\ud615\uc744 \uc774\ub7f0\uc2dd\uc73c\ub85c \uc0ac\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n::\n\n tsfm = Transform(params)\n transformed_sample = tsfm(sample)\n\n\uc5b4\ub5bb\uac8c \uc774\ub7f0 \ubcc0\ud615\ub4e4\uc774 \uc774\ubbf8\uc9c0\uc640 \ub79c\ub4dc\ub9c8\ud06c\uc5d0 \uc801\uc6a9\uc774 \ub418\uc5c8\ub294\uc9c0 \uc544\ub798\ub97c \ubd10\uc8fc\uc2dc\uae38 \ubc14\ub78d\ub2c8\ub2e4.\n\n\n" ] }, { @@ -253,7 +253,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/84ab670fda2216116ac8e3ecd5805f0b/model_parallel_tutorial.py b/docs/_downloads/84ab670fda2216116ac8e3ecd5805f0b/model_parallel_tutorial.py index be56e9ec9..a99563c8a 100644 --- a/docs/_downloads/84ab670fda2216116ac8e3ecd5805f0b/model_parallel_tutorial.py +++ b/docs/_downloads/84ab670fda2216116ac8e3ecd5805f0b/model_parallel_tutorial.py @@ -5,10 +5,10 @@ **저자** : `Shen Li `_ **번역** : `안상준 `_ -모델 병렬 처리는 분산 학습 기술에 범용적으로 사용되고 있습니다. +모델 병렬 처리는 분산 학습 기술에 범용적으로 사용되고 있습니다. 이전 튜토리얼들에서는 'DataParallel' ``_ 여러 GPU를 사용하여 신경망 모델을 학습 시킬 때 어떻게 DataParallel을 사용하는지에 대해서 살펴보았습니다. -이 방법은 각 GPU에 입력 데이터를 부분적으로 할당하고 동일한 신경망 모델을 복제하여 이용하는 방식이었습니다. +이 방법은 각 GPU에 입력 데이터를 부분적으로 할당하고 동일한 신경망 모델을 복제하여 이용하는 방식이었습니다. 이 방법은 신경망 모델을 상당히 빠르게 학습시킬 수 있는 장점이 있지만, 신경망 모델이 너무 커서 단일 GPU에 할당이 되지 않는 경우에는 동작하지 않습니다. 이번 튜토리얼에서는 ``데이터 병렬 처리`` 가 아닌 **모델 병렬 처리** 문제를 해결하는 방법을 소개합니다. @@ -16,10 +16,10 @@ 구체적으로, 10개의 층으로 구성된 ``m`` 신경망 모델에 대해서 ``데이터 병렬 처리`` 방법은 10개의 층을 전부 복제하여 각 GPU에 할당하여 처리하지만, 이와 반대로 2개의 GPU에 모델을 병렬 처리한다면, 각 GPU에 5개의 층씩 각각 할당하여 호스팅할 수 있습니다. -모델 병렬 처리의 전반적인 아이디어는 모델의 서브 네트워크들을 각각 다른 GPU에 할당하고, -각 장비 별로 순전파를 진행하여 계산되는 출력값들을 각 장비 간 공유하여 이용하는 것입니다. +모델 병렬 처리의 전반적인 아이디어는 모델의 서브 네트워크들을 각각 다른 GPU에 할당하고, +각 장비 별로 순전파를 진행하여 계산되는 출력값들을 각 장비 간 공유하여 이용하는 것입니다. 이용하고자 하는 신경망 모델을 부분적으로 각 GPU에 할당하는 것이기 때문에, 여러 GPU를 이용하여 더 큰 신경망 모델을 할당하고 학습시킬 수 있습니다. -이번 튜토리얼은 거대한 모델을 제한된 수의 GPU에 분할하여 할당하지 않고, 그 대신, 모델 병렬 처리의 아이디어를 이해하는 목적으로 작성되었습니다. +이번 튜토리얼은 거대한 모델을 제한된 수의 GPU에 분할하여 할당하지 않고, 그 대신, 모델 병렬 처리의 아이디어를 이해하는 목적으로 작성되었습니다. 모델 병렬 처리의 아이디어를 활용하여 실제 어플리케이션에 적용하는 것은 여러분의 몫입니다. .. note:: @@ -49,7 +49,7 @@ def __init__(self): self.net2 = torch.nn.Linear(10, 5).to('cuda:1') # 두 번째 층을 두 번째 GPU에 할당 def forward(self, x): - x = self.relu(self.net1(x.to('cuda:0'))) + x = self.relu(self.net1(x.to('cuda:0'))) return self.net2(x.to('cuda:1')) # 첫 번째 층의 산출물을 두 번째 GPU에 할당하여 진행 ###################################################################### @@ -71,10 +71,10 @@ def forward(self, x): ###################################################################### # 기존에 존재하는 모듈에 모델 병렬 처리 적용해보기 -# ---------------------------------------- +# --------------------------------------------------- # -# 기존에 단일 GPU에 존재하는 모듈을 여러 GPU에 할당하는 것은 단지 몇 줄의 코드를 수정하는 것으로도 쉽게 가능합니다. -# 아래에 있는 코드들은 ResNet50 모델을 분할하는 방법입니다. 이 아이디어는, 기존에 존재하는 ResNet 모듈을 상속받아 설계할 때, 2개의 GPU에 층을 나누어 설계하는 방식으로 진행됩니다. +# 기존에 단일 GPU에 존재하는 모듈을 여러 GPU에 할당하는 것은 단지 몇 줄의 코드를 수정하는 것으로도 쉽게 가능합니다. +# 아래에 있는 코드들은 ResNet50 모델을 분할하는 방법입니다. 이 아이디어는, 기존에 존재하는 ResNet 모듈을 상속받아 설계할 때, 2개의 GPU에 층을 나누어 설계하는 방식으로 진행됩니다. # 그 후, 2개 GPU에서 계산되는 중간 산출물 텐서값을 적절히 배치하기 위헤 순전파 메소드를 수정합니다. @@ -113,12 +113,12 @@ def forward(self, x): ###################################################################### -# 위의 예제에서는 단일 GPU에 신경망 모델을 할당하여 학습시키기에는 모델 크기가 너무 클 때 발생하는 문제를 해결하는 방법입니다. -# 하지만, 여러분은 단일 GPU를 이용할 때보다 학습 과정이 오래걸리며, 이는 여러분들이 이미 알고 있는 내용이었을 수 있습니다. +# 위의 예제에서는 단일 GPU에 신경망 모델을 할당하여 학습시키기에는 모델 크기가 너무 클 때 발생하는 문제를 해결하는 방법입니다. +# 하지만, 여러분은 단일 GPU를 이용할 때보다 학습 과정이 오래걸리며, 이는 여러분들이 이미 알고 있는 내용이었을 수 있습니다. # 그 이유는, 두 개의 GPU가 동시에 계산하는 것이 아니라 1개의 GPU는 계산하지 않고 대기하고 있기 때문입니다. # 또한, 두 번째 층 (layer2)이 할당된 첫 번째 GPU에서 계산된 결과를 세 번째 층 (layer3)이 할당된 두 번째 GPU로 텐서값을 복사하기 때문에 계산 과정이 더 길어지게 됩니다. # -# 코드 실행 시간을 정량적으로 살펴보기 위해 실험을 하나 해봅시다. 입력 텐서값과 레이블값을 랜덤으로 설정한 후, +# 코드 실행 시간을 정량적으로 살펴보기 위해 실험을 하나 해봅시다. 입력 텐서값과 레이블값을 랜덤으로 설정한 후, # 이미 존재하는 torchvision.models.reset50() 과, 모델 병렬 처리를 진행한 ``ModelParallelResNet50`` 을 통해 학습을 진행합니다. # 학습 진행을 완료한 후, 두 모델들은 랜덤으로 생성된 데이터로 학습을 진행했기 때문에 실용적인 예측을 하진 못하지만, 학습 진행 시간을 실용적으로 비교하여 할 수 있습니다. @@ -157,8 +157,8 @@ def train(model): ###################################################################### -# 위에서 정의한 ``train(model)`` 메소드는 nn.MSELoss (Mean Squared Error ; 평균 제곱 오차) 로 손실 함수를 정의하여 신경망 모델을 학습하는 것을 의미합니다. -# 그리고, ``optim.SGD`` 메소드는 최적화 방식을 의미합니다. 위 방식은 128 * 128 크기의 이미지가 120개로 구성된 배치 데이터가 3개 존재하는 상황을 모방하기 위해 랜덤으로 생성하였습니다. +# 위에서 정의한 ``train(model)`` 메소드는 nn.MSELoss (Mean Squared Error ; 평균 제곱 오차) 로 손실 함수를 정의하여 신경망 모델을 학습하는 것을 의미합니다. +# 그리고, ``optim.SGD`` 메소드는 최적화 방식을 의미합니다. 위 방식은 128 * 128 크기의 이미지가 120개로 구성된 배치 데이터가 3개 존재하는 상황을 모방하기 위해 랜덤으로 생성하였습니다. # 그리고나서, 우리는 ``timeit`` 을 이용하여 ``train(model)`` 메소드를 10회 실행하여 학습을 진행하고, 학습 실행 시간에 대해서 표준 편차값을 반영하는 이미지를 생성하여 저장합니다. @@ -217,7 +217,7 @@ def plot(means, stds, labels, fig_name): # 실험 결과, 모델 병렬 철리하여 학습하는 시간이 단일 GPU로 학습하는 시간보다 약 7% ``4.02/3.75-1=7%``정도 # 오래 걸리는 것을 확인할 수 있습니다. 그러므로, 순전파와 역전파를 진행하면서 GPU 간 텐서값들이 # 복제되어 이용하는 시간이 약 7%정도 소요되는 것으로 결론지을 수 있습니다. 학습하는 과정 속에서 -# 2개의 GPU 중 1개의 GPU가 계산하지 않고 대기하고 있기 때문에, 이를 해결하여 +# 2개의 GPU 중 1개의 GPU가 계산하지 않고 대기하고 있기 때문에, 이를 해결하여 # 학습 시간을 빠르게 개선시킬 수 있습니다. 그 중 한 가지 방법은, 학습 단위인 미니 배치 1개의 데이터를 # 2개로 분할하는 파이프라인을 생성하여, 분할된 첫 번째 데이터가 첫 번째 층을 통과하여 두 번째 층으로 # 복제되고, 두 번째 층을 통과할 때, 두번재로 분할된 데이터가 첫 번쨰 층을 통해 계산되는 방식으로 설정하는 것입니다. @@ -225,10 +225,10 @@ def plot(means, stds, labels, fig_name): ###################################################################### # 입력 텐서값을 분할하는 파이프라인을 설계하여 학습 시간을 단축하는 방법에 대한 예제 -# ----------------------------- +# --------------------------------------------------------------------------------------- # -# 아래에 있는 실험은, 120개의 이미지로 구성된 1개의 미니 배치 데이터를 20개씩 나누어 진행하는 -# 과정입니다. 아래의 과정을 실행할 때, PyTorch가 CUDA 연산을 비동기적으로 이용하기 때문에, +# 아래에 있는 실험은, 120개의 이미지로 구성된 1개의 미니 배치 데이터를 20개씩 나누어 진행하는 +# 과정입니다. 아래의 과정을 실행할 때, PyTorch가 CUDA 연산을 비동기적으로 이용하기 때문에, # 프로세스를 실행하는 스레드를 여러개 생성할 필요가 없습니다. @@ -248,7 +248,7 @@ def forward(self, x): s_prev = self.seq2(s_prev) ret.append(self.fc(s_prev.view(s_prev.size(0), -1))) - # B. s_next는 A.와 동시에 진행되면서 첫 번째 GPU에서 실행됩니다. + # B. s_next는 A.와 동시에 진행되면서 첫 번째 GPU에서 실행됩니다. s_prev = self.seq1(s_next).to('cuda:1') s_prev = self.seq2(s_prev) @@ -271,11 +271,11 @@ def forward(self, x): # GPU 간 텐서값이 복사되는 것은 현재 계산되고 있는 소스값과, 소스값의 목적지 GPU 간 연산되고 있는 # 스트림과 동기화되는 것을 주의하세요. 만약 여러 스트림을 생성하여 진행하고 있다면, GPU 간 텐서값이 # 정상적으로 복사되어 계산되고 있는지 꼭 확인해야 합니다. 만약 복사되는 과정 중에 소스값을 이용하거나, -# GPU의 텐서값을 읽거나 쓰는 것은 올바르게 계산되지 않을 수 있습니다. 위의 예제에서는 소스값 및 GPU +# GPU의 텐서값을 읽거나 쓰는 것은 올바르게 계산되지 않을 수 있습니다. 위의 예제에서는 소스값 및 GPU # 텐서값을 기본 스트림만 이용하여 진행하므로 추가적인 동기화 과정을 진행할 필요는 없습니다. ###################################################################### -# +# # .. figure:: /_static/img/model-parallel-images/mp_vs_rn_vs_pp.png # :alt: # @@ -284,7 +284,7 @@ def forward(self, x): # 학습 시간이 2배 단축되는 것에 비해 다소 적게 학습 시간이 단축되었습니다. 파이프라인을 이용할 때, # ``split_sizes`` 매개변수를 도입하였기 때문에, 파이프라인을 이용하는 것이 학습 시간 단축에 얼마나 # 영향을 미쳤는지 불분명합니다. 직관적으로 생각하였을 때, ``split_sizes`` 매개변수 값을 작게 설정한다면, -# 아주 소규모의 CUDA 연산이 많이 진행되고, ``split_sizes`` 매개변수 값을 크게 설정한다면, 첫 번째와 +# 아주 소규모의 CUDA 연산이 많이 진행되고, ``split_sizes`` 매개변수 값을 크게 설정한다면, 첫 번째와 # 마지막 분리될 때 비교적 긴 시간 동안 CUDA 연산이 이루어지게 됩니다. 둘 다 최적의 설정이 아닙니다. # 따라서, ``split_sizes`` 매개변수 값을 최적으로 설정하였을 때, 학습 시간 과정이 단축될 수 있을 것이라 # 기대됩니다. ``split_sizes`` 매개변수 값을 조정하여 실험하면서 최적의 값을 찾아봅시다. @@ -317,20 +317,20 @@ def forward(self, x): # .. figure:: /_static/img/model-parallel-images/split_size_tradeoff.png # :alt: # -# 실험 결과, ``split_size`` 매개변수값을 12로 설정하였을 때, 학습 시간이 54% 수준으로 +# 실험 결과, ``split_size`` 매개변수값을 12로 설정하였을 때, 학습 시간이 54% 수준으로 # 가장 많이 단축되었습니다. 아직 학습 시간을 더 단축시킬 수 있는 방법은 다양하게 존재합니다. # 예를 들어, 첫 번째 GPU에서 모든 연산과정이 기본으로 설정되어 진행됩니다. 이는 미니배치 분할 과정 중, # 현재 진행되는 과정의 다음 단계는 현재 진행되는 과정과 동시에 복제가 이루어질 수 없는 것을 의미합니다. # 그러나, 이전과 다음 단계의 분할과정이 다른 텐서값을 이용하기 때문에, 다른 계산과 중복되어 진행되어도 -# 문제가 없습니다. 이에 대해서, 2개 GPU에 여러개의 스트림을 사용하는 것이 필요하며, 서로 다른 서브 네트워크 -# 구조가 서로 다른 스트림을 관리하는 전략이 요구됩니다. 모델 병렬 처리에 대해서 여러 스트림을 사용하는 방법이 +# 문제가 없습니다. 이에 대해서, 2개 GPU에 여러개의 스트림을 사용하는 것이 필요하며, 서로 다른 서브 네트워크 +# 구조가 서로 다른 스트림을 관리하는 전략이 요구됩니다. 모델 병렬 처리에 대해서 여러 스트림을 사용하는 방법이 # 일반적을로 존재하지 않기 때문에 이번 튜토리얼에서는 설명하지 않습니다. """ .. note:: 이번 게시물에서는 다양한 성능 측정값을 확인할 수 있습니다. 여러분은 위의 예제를 실행할 때 마다 매번 - 다른 결과를 확인할 수 있습니다. 그 이유는, 이용하는 소프트웨어 및 하드웨어에 따라 결과가 + 다른 결과를 확인할 수 있습니다. 그 이유는, 이용하는 소프트웨어 및 하드웨어에 따라 결과가 다르게 나타나기 때문입니다. 여러분이 이용하고 있는 환경 내에서 가장 좋은 성능을 얻기 위해서는, 곡선을 그려서 - 최적의 ``split_size`` 값을 도출한 후, 해당 값을 이용하여 미니 배치 내 데이터를 분리하는 파이프라인을 + 최적의 ``split_size`` 값을 도출한 후, 해당 값을 이용하여 미니 배치 내 데이터를 분리하는 파이프라인을 생성하는 것입니다. """ \ No newline at end of file diff --git a/docs/_downloads/8eed7e178f8fa30798f280ea82ff468b/autograd_tutorial.ipynb b/docs/_downloads/8eed7e178f8fa30798f280ea82ff468b/autograd_tutorial.ipynb index a2a75f0d5..92ac81646 100644 --- a/docs/_downloads/8eed7e178f8fa30798f280ea82ff468b/autograd_tutorial.ipynb +++ b/docs/_downloads/8eed7e178f8fa30798f280ea82ff468b/autograd_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nAutograd: \uc790\ub3d9 \ubbf8\ubd84\n===================================\n\nPyTorch\uc758 \ubaa8\ub4e0 \uc2e0\uacbd\ub9dd\uc758 \uc911\uc2ec\uc5d0\ub294 ``autograd`` \ud328\ud0a4\uc9c0\uac00 \uc788\uc2b5\ub2c8\ub2e4.\n\uba3c\uc800 \uc774\uac83\uc744 \uac00\ubccd\uac8c \uc0b4\ud3b4\ubcf8 \ub4a4, \uccab\ubc88\uc9f8 \uc2e0\uacbd\ub9dd\uc744 \ud559\uc2b5\uc2dc\ucf1c\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n``autograd`` \ud328\ud0a4\uc9c0\ub294 Tensor\uc758 \ubaa8\ub4e0 \uc5f0\uc0b0\uc5d0 \ub300\ud574 \uc790\ub3d9 \ubbf8\ubd84\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.\n\uc774\ub294 \uc2e4\ud589-\uae30\ubc18-\uc815\uc758(define-by-run) \ud504\ub808\uc784\uc6cc\ud06c\ub85c, \uc774\ub294 \ucf54\ub4dc\ub97c \uc5b4\ub5bb\uac8c \uc791\uc131\ud558\uc5ec\n\uc2e4\ud589\ud558\ub290\ub0d0\uc5d0 \ub530\ub77c \uc5ed\uc804\ud30c\uac00 \uc815\uc758\ub41c\ub2e4\ub294 \ub73b\uc774\uba70, \uc5ed\uc804\ud30c\ub294 \ud559\uc2b5 \uacfc\uc815\uc758 \ub9e4 \ub2e8\uacc4\ub9c8\ub2e4\n\ub2ec\ub77c\uc9d1\ub2c8\ub2e4.\n\n\ub354 \uac04\ub2e8\ud55c \uc6a9\uc5b4\ub85c \uba87 \uac00\uc9c0 \uc608\ub97c \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\nTensor\n--------\n\n\ud328\ud0a4\uc9c0\uc758 \uc911\uc2ec\uc5d0\ub294 ``torch.Tensor`` \ud074\ub798\uc2a4\uac00 \uc788\uc2b5\ub2c8\ub2e4. \ub9cc\uc57d ``.requires_grad``\n\uc18d\uc131\uc744 ``True`` \ub85c \uc124\uc815\ud558\uba74, \uadf8 tensor\uc5d0\uc11c \uc774\ub904\uc9c4 \ubaa8\ub4e0 \uc5f0\uc0b0\ub4e4\uc744 \ucd94\uc801(track)\ud558\uae30\n\uc2dc\uc791\ud569\ub2c8\ub2e4. \uacc4\uc0b0\uc774 \uc644\ub8cc\ub41c \ud6c4 ``.backward()`` \ub97c \ud638\ucd9c\ud558\uc5ec \ubaa8\ub4e0 \ubcc0\ud654\ub3c4(gradient)\ub97c\n\uc790\ub3d9\uc73c\ub85c \uacc4\uc0b0\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc774 Tensor\uc758 \ubcc0\ud654\ub3c4\ub294 ``.grad`` \uc18d\uc131\uc5d0 \ub204\uc801\ub429\ub2c8\ub2e4.\n\nTensor\uac00 \uae30\ub85d\uc744 \ucd94\uc801\ud558\ub294 \uac83\uc744 \uc911\ub2e8\ud558\uac8c \ud558\ub824\uba74, ``.detach()`` \ub97c \ud638\ucd9c\ud558\uc5ec \uc5f0\uc0b0\n\uae30\ub85d\uc73c\ub85c\ubd80\ud130 \ubd84\ub9ac(detach)\ud558\uc5ec \uc774\ud6c4 \uc5f0\uc0b0\ub4e4\uc774 \ucd94\uc801\ub418\ub294 \uac83\uc744 \ubc29\uc9c0\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uae30\ub85d\uc744 \ucd94\uc801\ud558\ub294 \uac83(\uacfc \uba54\ubaa8\ub9ac\ub97c \uc0ac\uc6a9\ud558\ub294 \uac83)\uc744 \ubc29\uc9c0\ud558\uae30 \uc704\ud574, \ucf54\ub4dc \ube14\ub7ed\uc744\n``with torch.no_grad():`` \ub85c \uac10\uc300 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc774\ub294 \ud2b9\ud788 \ubcc0\ud654\ub3c4(gradient)\ub294\n\ud544\uc694\uc5c6\uc9c0\ub9cc, `requires_grad=True` \uac00 \uc124\uc815\ub418\uc5b4 \ud559\uc2b5 \uac00\ub2a5\ud55c \ub9e4\uac1c\ubcc0\uc218\ub97c \uac16\ub294 \ubaa8\ub378\uc744\n\ud3c9\uac00(evaluate)\ud560 \ub54c \uc720\uc6a9\ud569\ub2c8\ub2e4.\n\nAutograd \uad6c\ud604\uc5d0\uc11c \ub9e4\uc6b0 \uc911\uc694\ud55c \ud074\ub798\uc2a4\uac00 \ud558\ub098 \ub354 \uc788\ub294\ub370, \uc774\uac83\uc740 \ubc14\ub85c ``Function``\n\ud074\ub798\uc2a4\uc785\ub2c8\ub2e4.\n\n``Tensor`` \uc640 ``Function`` \uc740 \uc11c\ub85c \uc5f0\uacb0\ub418\uc5b4 \uc788\uc73c\uba70, \ubaa8\ub4e0 \uc5f0\uc0b0 \uacfc\uc815\uc744\n\ubd80\ud638\ud654(encode)\ud558\uc5ec \uc21c\ud658\ud558\uc9c0 \uc54a\ub294 \uadf8\ub798\ud504(acyclic graph)\ub97c \uc0dd\uc131\ud569\ub2c8\ub2e4. \uac01 tensor\ub294\n``.grad_fn`` \uc18d\uc131\uc744 \uac16\uace0 \uc788\ub294\ub370, \uc774\ub294 ``Tensor`` \ub97c \uc0dd\uc131\ud55c ``Function`` \uc744\n\ucc38\uc870\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4. (\ub2e8, \uc0ac\uc6a9\uc790\uac00 \ub9cc\ub4e0 Tensor\ub294 \uc608\uc678\ub85c, \uc774 \ub54c ``grad_fn`` \uc740\n``None`` \uc785\ub2c8\ub2e4.)\n\n\ub3c4\ud568\uc218\ub97c \uacc4\uc0b0\ud558\uae30 \uc704\ud574\uc11c\ub294 ``Tensor`` \uc758 ``.backward()`` \ub97c \ud638\ucd9c\ud558\uba74\n\ub429\ub2c8\ub2e4. \ub9cc\uc57d ``Tensor`` \uac00 \uc2a4\uce7c\ub77c(scalar)\uc778 \uacbd\uc6b0(\uc608. \ud558\ub098\uc758 \uc694\uc18c \uac12\ub9cc \uac16\ub294 \ub4f1)\uc5d0\ub294\n``backward`` \uc5d0 \uc778\uc790\ub97c \uc815\ud574\uc904 \ud544\uc694\uac00 \uc5c6\uc2b5\ub2c8\ub2e4. \ud558\uc9c0\ub9cc \uc5ec\ub7ec \uac1c\uc758 \uc694\uc18c\ub97c \uac16\uace0 \uc788\uc744\n\ub54c\ub294 tensor\uc758 \ubaa8\uc591\uc744 ``gradient`` \uc758 \uc778\uc790\ub85c \uc9c0\uc815\ud560 \ud544\uc694\uac00 \uc788\uc2b5\ub2c8\ub2e4.\n\n" + "\nAutograd: \uc790\ub3d9 \ubbf8\ubd84\n===================================\n\nPyTorch\uc758 \ubaa8\ub4e0 \uc2e0\uacbd\ub9dd\uc758 \uc911\uc2ec\uc5d0\ub294 ``autograd`` \ud328\ud0a4\uc9c0\uac00 \uc788\uc2b5\ub2c8\ub2e4.\n\uba3c\uc800 \uc774\uac83\uc744 \uac00\ubccd\uac8c \uc0b4\ud3b4\ubcf8 \ub4a4, \uccab\ubc88\uc9f8 \uc2e0\uacbd\ub9dd\uc744 \ud559\uc2b5\uc2dc\ucf1c\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n``autograd`` \ud328\ud0a4\uc9c0\ub294 Tensor\uc758 \ubaa8\ub4e0 \uc5f0\uc0b0\uc5d0 \ub300\ud574 \uc790\ub3d9 \ubbf8\ubd84\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.\n\uc774\ub294 \uc2e4\ud589-\uae30\ubc18-\uc815\uc758(define-by-run) \ud504\ub808\uc784\uc6cc\ud06c\ub85c, \uc774\ub294 \ucf54\ub4dc\ub97c \uc5b4\ub5bb\uac8c \uc791\uc131\ud558\uc5ec\n\uc2e4\ud589\ud558\ub290\ub0d0\uc5d0 \ub530\ub77c \uc5ed\uc804\ud30c\uac00 \uc815\uc758\ub41c\ub2e4\ub294 \ub73b\uc774\uba70, \uc5ed\uc804\ud30c\ub294 \ud559\uc2b5 \uacfc\uc815\uc758 \ub9e4 \ub2e8\uacc4\ub9c8\ub2e4\n\ub2ec\ub77c\uc9d1\ub2c8\ub2e4.\n\n\ub354 \uac04\ub2e8\ud55c \uc6a9\uc5b4\ub85c \uba87 \uac00\uc9c0 \uc608\ub97c \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\nTensor\n--------\n\n\ud328\ud0a4\uc9c0\uc758 \uc911\uc2ec\uc5d0\ub294 ``torch.Tensor`` \ud074\ub798\uc2a4\uac00 \uc788\uc2b5\ub2c8\ub2e4. \ub9cc\uc57d ``.requires_grad``\n\uc18d\uc131\uc744 ``True`` \ub85c \uc124\uc815\ud558\uba74, \uadf8 tensor\uc5d0\uc11c \uc774\ub904\uc9c4 \ubaa8\ub4e0 \uc5f0\uc0b0\ub4e4\uc744 \ucd94\uc801(track)\ud558\uae30\n\uc2dc\uc791\ud569\ub2c8\ub2e4. \uacc4\uc0b0\uc774 \uc644\ub8cc\ub41c \ud6c4 ``.backward()`` \ub97c \ud638\ucd9c\ud558\uc5ec \ubaa8\ub4e0 \ubcc0\ud654\ub3c4(gradient)\ub97c\n\uc790\ub3d9\uc73c\ub85c \uacc4\uc0b0\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc774 Tensor\uc758 \ubcc0\ud654\ub3c4\ub294 ``.grad`` \uc18d\uc131\uc5d0 \ub204\uc801\ub429\ub2c8\ub2e4.\n\nTensor\uac00 \uae30\ub85d\uc744 \ucd94\uc801\ud558\ub294 \uac83\uc744 \uc911\ub2e8\ud558\uac8c \ud558\ub824\uba74, ``.detach()`` \ub97c \ud638\ucd9c\ud558\uc5ec \uc5f0\uc0b0\n\uae30\ub85d\uc73c\ub85c\ubd80\ud130 \ubd84\ub9ac(detach)\ud558\uc5ec \uc774\ud6c4 \uc5f0\uc0b0\ub4e4\uc774 \ucd94\uc801\ub418\ub294 \uac83\uc744 \ubc29\uc9c0\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uae30\ub85d\uc744 \ucd94\uc801\ud558\ub294 \uac83(\uacfc \uba54\ubaa8\ub9ac\ub97c \uc0ac\uc6a9\ud558\ub294 \uac83)\uc744 \ubc29\uc9c0\ud558\uae30 \uc704\ud574, \ucf54\ub4dc \ube14\ub7ed\uc744\n``with torch.no_grad():`` \ub85c \uac10\uc300 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc774\ub294 \ud2b9\ud788 \ubcc0\ud654\ub3c4(gradient)\ub294\n\ud544\uc694\uc5c6\uc9c0\ub9cc, `requires_grad=True` \uac00 \uc124\uc815\ub418\uc5b4 \ud559\uc2b5 \uac00\ub2a5\ud55c \ub9e4\uac1c\ubcc0\uc218\ub97c \uac16\ub294 \ubaa8\ub378\uc744\n\ud3c9\uac00(evaluate)\ud560 \ub54c \uc720\uc6a9\ud569\ub2c8\ub2e4.\n\nAutograd \uad6c\ud604\uc5d0\uc11c \ub9e4\uc6b0 \uc911\uc694\ud55c \ud074\ub798\uc2a4\uac00 \ud558\ub098 \ub354 \uc788\ub294\ub370, \uc774\uac83\uc740 \ubc14\ub85c ``Function``\n\ud074\ub798\uc2a4\uc785\ub2c8\ub2e4.\n\n``Tensor`` \uc640 ``Function`` \uc740 \uc11c\ub85c \uc5f0\uacb0\ub418\uc5b4 \uc788\uc73c\uba70, \ubaa8\ub4e0 \uc5f0\uc0b0 \uacfc\uc815\uc744\n\ubd80\ud638\ud654(encode)\ud558\uc5ec \uc21c\ud658\ud558\uc9c0 \uc54a\ub294 \uadf8\ub798\ud504(acyclic graph)\ub97c \uc0dd\uc131\ud569\ub2c8\ub2e4. \uac01 tensor\ub294\n``.grad_fn`` \uc18d\uc131\uc744 \uac16\uace0 \uc788\ub294\ub370, \uc774\ub294 ``Tensor`` \ub97c \uc0dd\uc131\ud55c ``Function`` \uc744\n\ucc38\uc870\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4. (\ub2e8, \uc0ac\uc6a9\uc790\uac00 \ub9cc\ub4e0 Tensor\ub294 \uc608\uc678\ub85c, \uc774 \ub54c ``grad_fn`` \uc740\n``None`` \uc785\ub2c8\ub2e4.)\n\n\ub3c4\ud568\uc218\ub97c \uacc4\uc0b0\ud558\uae30 \uc704\ud574\uc11c\ub294 ``Tensor`` \uc758 ``.backward()`` \ub97c \ud638\ucd9c\ud558\uba74\n\ub429\ub2c8\ub2e4. \ub9cc\uc57d ``Tensor`` \uac00 \uc2a4\uce7c\ub77c(scalar)\uc778 \uacbd\uc6b0(\uc608. \ud558\ub098\uc758 \uc694\uc18c \uac12\ub9cc \uac16\ub294 \ub4f1)\uc5d0\ub294\n``backward`` \uc5d0 \uc778\uc790\ub97c \uc815\ud574\uc904 \ud544\uc694\uac00 \uc5c6\uc2b5\ub2c8\ub2e4. \ud558\uc9c0\ub9cc \uc5ec\ub7ec \uac1c\uc758 \uc694\uc18c\ub97c \uac16\uace0 \uc788\uc744\n\ub54c\ub294 tensor\uc758 \ubaa8\uc591\uc744 ``gradient`` \uc758 \uc778\uc790\ub85c \uc9c0\uc815\ud560 \ud544\uc694\uac00 \uc788\uc2b5\ub2c8\ub2e4.\n" ] }, { @@ -265,7 +265,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/988d3733a780db0912efb0d6f34d0ec7/two_layer_net_nn.ipynb b/docs/_downloads/988d3733a780db0912efb0d6f34d0ec7/two_layer_net_nn.ipynb index a19e21e72..661c99589 100644 --- a/docs/_downloads/988d3733a780db0912efb0d6f34d0ec7/two_layer_net_nn.ipynb +++ b/docs/_downloads/988d3733a780db0912efb0d6f34d0ec7/two_layer_net_nn.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nPyTorch: nn\n-----------\n\n\ud558\ub098\uc758 \uc740\ub2c9\uce35(hidden layer)\uacfc \ud3b8\ud5a5(bias)\uc774 \uc5c6\ub294 \uc644\uc804\ud788 \uc5f0\uacb0\ub41c ReLU \uc2e0\uacbd\ub9dd\uc744,\n\uc720\ud074\ub9ac\ub4dc \uac70\ub9ac(Euclidean distance) \uc81c\uacf1\uc744 \ucd5c\uc18c\ud654\ud558\ub294 \uc2dd\uc73c\ub85c x\ub85c\ubd80\ud130 y\ub97c \uc608\uce21\ud558\ub3c4\ub85d\n\ud559\uc2b5\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc774\ubc88\uc5d0\ub294 PyTorch\uc758 nn \ud328\ud0a4\uc9c0\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc2e0\uacbd\ub9dd\uc744 \uad6c\ud604\ud558\uaca0\uc2b5\ub2c8\ub2e4.\nPyTorch autograd\ub294 \uc5f0\uc0b0 \uadf8\ub798\ud504\ub97c \uc815\uc758\ud558\uace0 \ubcc0\ud654\ub3c4\ub97c \uacc4\uc0b0\ud558\ub294 \uac83\uc744 \uc190\uc27d\uac8c \ub9cc\ub4e4\uc5b4\uc8fc\uc9c0\ub9cc,\nautograd \uadf8 \uc790\uccb4\ub9cc\uc73c\ub85c\ub294 \ubcf5\uc7a1\ud55c \uc2e0\uacbd\ub9dd\uc744 \uc815\uc758\ud558\uae30\uc5d0\ub294 \ub108\ubb34 \uc800\uc218\uc900(low-level)\uc77c \uc218\n\uc788\uc2b5\ub2c8\ub2e4; \uc774\uac83\uc774 nn \ud328\ud0a4\uc9c0\uac00 \ud544\uc694\ud55c \uc774\uc720\uc785\ub2c8\ub2e4. nn \ud328\ud0a4\uc9c0\ub294 Module\uc758 \uc9d1\ud569\uc744\n\uc815\uc758\ud558\ub294\ub370, \uc774\ub294 \uc785\ub825\uc73c\ub85c\ubd80\ud130 \ucd9c\ub825\uc744 \uc0dd\uc131\ud558\uace0 \ud559\uc2b5 \uac00\ub2a5\ud55c \uac00\uc911\uce58\ub97c \uac16\ub294\n\uc2e0\uacbd\ub9dd \uacc4\uce35(neural network layer)\uc774\ub77c\uace0 \uc0dd\uac01\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n" + "\nPyTorch: nn\n-----------\n\n\ud558\ub098\uc758 \uc740\ub2c9\uce35(hidden layer)\uacfc \ud3b8\ud5a5(bias)\uc774 \uc5c6\ub294 \uc644\uc804\ud788 \uc5f0\uacb0\ub41c ReLU \uc2e0\uacbd\ub9dd\uc744,\n\uc720\ud074\ub9ac\ub4dc \uac70\ub9ac(Euclidean distance) \uc81c\uacf1\uc744 \ucd5c\uc18c\ud654\ud558\ub294 \uc2dd\uc73c\ub85c x\ub85c\ubd80\ud130 y\ub97c \uc608\uce21\ud558\ub3c4\ub85d\n\ud559\uc2b5\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc774\ubc88\uc5d0\ub294 PyTorch\uc758 nn \ud328\ud0a4\uc9c0\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc2e0\uacbd\ub9dd\uc744 \uad6c\ud604\ud558\uaca0\uc2b5\ub2c8\ub2e4.\nPyTorch autograd\ub294 \uc5f0\uc0b0 \uadf8\ub798\ud504\ub97c \uc815\uc758\ud558\uace0 \ubcc0\ud654\ub3c4\ub97c \uacc4\uc0b0\ud558\ub294 \uac83\uc744 \uc190\uc27d\uac8c \ub9cc\ub4e4\uc5b4\uc8fc\uc9c0\ub9cc,\nautograd \uadf8 \uc790\uccb4\ub9cc\uc73c\ub85c\ub294 \ubcf5\uc7a1\ud55c \uc2e0\uacbd\ub9dd\uc744 \uc815\uc758\ud558\uae30\uc5d0\ub294 \ub108\ubb34 \uc800\uc218\uc900(low-level)\uc77c \uc218\n\uc788\uc2b5\ub2c8\ub2e4; \uc774\uac83\uc774 nn \ud328\ud0a4\uc9c0\uac00 \ud544\uc694\ud55c \uc774\uc720\uc785\ub2c8\ub2e4. nn \ud328\ud0a4\uc9c0\ub294 Module\uc758 \uc9d1\ud569\uc744\n\uc815\uc758\ud558\ub294\ub370, \uc774\ub294 \uc785\ub825\uc73c\ub85c\ubd80\ud130 \ucd9c\ub825\uc744 \uc0dd\uc131\ud558\uace0 \ud559\uc2b5 \uac00\ub2a5\ud55c \uac00\uc911\uce58\ub97c \uac16\ub294\n\uc2e0\uacbd\ub9dd \uacc4\uce35(neural network layer)\uc774\ub77c\uace0 \uc0dd\uac01\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n" ] }, { @@ -46,7 +46,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/99fdc80b1d3c74fcdb1432b4e7df20f7/advanced_tutorial.ipynb b/docs/_downloads/99fdc80b1d3c74fcdb1432b4e7df20f7/advanced_tutorial.ipynb index d29aff0a0..14b7ef70f 100644 --- a/docs/_downloads/99fdc80b1d3c74fcdb1432b4e7df20f7/advanced_tutorial.ipynb +++ b/docs/_downloads/99fdc80b1d3c74fcdb1432b4e7df20f7/advanced_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nAdvanced: Making Dynamic Decisions and the Bi-LSTM CRF\n======================================================\n\nDynamic versus Static Deep Learning Toolkits\n--------------------------------------------\n\nPytorch is a *dynamic* neural network kit. Another example of a dynamic\nkit is `Dynet `__ (I mention this because\nworking with Pytorch and Dynet is similar. If you see an example in\nDynet, it will probably help you implement it in Pytorch). The opposite\nis the *static* tool kit, which includes Theano, Keras, TensorFlow, etc.\nThe core difference is the following:\n\n* In a static toolkit, you define\n a computation graph once, compile it, and then stream instances to it.\n* In a dynamic toolkit, you define a computation graph *for each\n instance*. It is never compiled and is executed on-the-fly\n\nWithout a lot of experience, it is difficult to appreciate the\ndifference. One example is to suppose we want to build a deep\nconstituent parser. Suppose our model involves roughly the following\nsteps:\n\n* We build the tree bottom up\n* Tag the root nodes (the words of the sentence)\n* From there, use a neural network and the embeddings\n of the words to find combinations that form constituents. Whenever you\n form a new constituent, use some sort of technique to get an embedding\n of the constituent. In this case, our network architecture will depend\n completely on the input sentence. In the sentence \"The green cat\n scratched the wall\", at some point in the model, we will want to combine\n the span $(i,j,r) = (1, 3, \\text{NP})$ (that is, an NP constituent\n spans word 1 to word 3, in this case \"The green cat\").\n\nHowever, another sentence might be \"Somewhere, the big fat cat scratched\nthe wall\". In this sentence, we will want to form the constituent\n$(2, 4, NP)$ at some point. The constituents we will want to form\nwill depend on the instance. If we just compile the computation graph\nonce, as in a static toolkit, it will be exceptionally difficult or\nimpossible to program this logic. In a dynamic toolkit though, there\nisn't just 1 pre-defined computation graph. There can be a new\ncomputation graph for each instance, so this problem goes away.\n\nDynamic toolkits also have the advantage of being easier to debug and\nthe code more closely resembling the host language (by that I mean that\nPytorch and Dynet look more like actual Python code than Keras or\nTheano).\n\nBi-LSTM Conditional Random Field Discussion\n-------------------------------------------\n\nFor this section, we will see a full, complicated example of a Bi-LSTM\nConditional Random Field for named-entity recognition. The LSTM tagger\nabove is typically sufficient for part-of-speech tagging, but a sequence\nmodel like the CRF is really essential for strong performance on NER.\nFamiliarity with CRF's is assumed. Although this name sounds scary, all\nthe model is is a CRF but where an LSTM provides the features. This is\nan advanced model though, far more complicated than any earlier model in\nthis tutorial. If you want to skip it, that is fine. To see if you're\nready, see if you can:\n\n- Write the recurrence for the viterbi variable at step i for tag k.\n- Modify the above recurrence to compute the forward variables instead.\n- Modify again the above recurrence to compute the forward variables in\n log-space (hint: log-sum-exp)\n\nIf you can do those three things, you should be able to understand the\ncode below. Recall that the CRF computes a conditional probability. Let\n$y$ be a tag sequence and $x$ an input sequence of words.\nThen we compute\n\n\\begin{align}P(y|x) = \\frac{\\exp{(\\text{Score}(x, y)})}{\\sum_{y'} \\exp{(\\text{Score}(x, y')})}\\end{align}\n\nWhere the score is determined by defining some log potentials\n$\\log \\psi_i(x,y)$ such that\n\n\\begin{align}\\text{Score}(x,y) = \\sum_i \\log \\psi_i(x,y)\\end{align}\n\nTo make the partition function tractable, the potentials must look only\nat local features.\n\nIn the Bi-LSTM CRF, we define two kinds of potentials: emission and\ntransition. The emission potential for the word at index $i$ comes\nfrom the hidden state of the Bi-LSTM at timestep $i$. The\ntransition scores are stored in a $|T|x|T|$ matrix\n$\\textbf{P}$, where $T$ is the tag set. In my\nimplementation, $\\textbf{P}_{j,k}$ is the score of transitioning\nto tag $j$ from tag $k$. So:\n\n\\begin{align}\\text{Score}(x,y) = \\sum_i \\log \\psi_\\text{EMIT}(y_i \\rightarrow x_i) + \\log \\psi_\\text{TRANS}(y_{i-1} \\rightarrow y_i)\\end{align}\n\n\\begin{align}= \\sum_i h_i[y_i] + \\textbf{P}_{y_i, y_{i-1}}\\end{align}\n\nwhere in this second expression, we think of the tags as being assigned\nunique non-negative indices.\n\nIf the above discussion was too brief, you can check out\n`this `__ write up from\nMichael Collins on CRFs.\n\nImplementation Notes\n--------------------\n\nThe example below implements the forward algorithm in log space to\ncompute the partition function, and the viterbi algorithm to decode.\nBackpropagation will compute the gradients automatically for us. We\ndon't have to do anything by hand.\n\nThe implementation is not optimized. If you understand what is going on,\nyou'll probably quickly see that iterating over the next tag in the\nforward algorithm could probably be done in one big operation. I wanted\nto code to be more readable. If you want to make the relevant change,\nyou could probably use this tagger for real tasks.\n\n" + "\n\uc2ec\ud654 \uacfc\uc815 : Bi-LSTM CRF\uc640 \ub3d9\uc801 \uacb0\uc815\n======================================================\n\n\ub3d9\uc801, \uc815\uc801 \ub525 \ub7ec\ub2dd \ud234\ud0b7(toolkits) \ube44\uad50\n--------------------------------------------\n\nPytorch\ub294 *\ub3d9\uc801* \uc2e0\uacbd\ub9dd \ud234\ud0b7\uc785\ub2c8\ub2e4. \ub2e4\ub978 \ub3d9\uc801 \uc2e0\uacbd\ub9dd \ud234\ud0b7\uc73c\ub85c\ub294\n`Dynet `_ \uc774 \uc788\uc2b5\ub2c8\ub2e4.(\uc774 \ud234\ud0b7\uc744\n\uc608\ub85c \ub4e0 \uc774\uc720\ub294 \uc0ac\uc6a9\ud558\ub294 \ubc95\uc774 Pytorch\uc640 \ube44\uc2b7\ud558\uae30 \ub54c\ubb38\uc785\ub2c8\ub2e4. Dynet\uc758 \uc608\uc81c\ub97c \ubcf4\uba74\nPytorch\ub85c \uad6c\ud604\ud560 \ub54c\ub3c4 \ub3c4\uc6c0\uc774 \ub420 \uac83\uc785\ub2c8\ub2e4.) \ubc18\ub300\ub85c *\uc815\uc801* \ud234\ud0b7\ub4e4\ub85c\nTheano, Keras, TensorFlow \ub4f1\uc774 \uc788\uc2b5\ub2c8\ub2e4. \uc8fc\uc694 \ucc28\uc774\uc810\uc740 \ub2e4\uc74c\uacfc \uac19\uc2b5\ub2c8\ub2e4:\n\n* \uc815\uc801 \ud234\ud0b7\uc744 \uc4f8 \ub54c\ub294 \uacc4\uc0b0 \uadf8\ub798\ud504\ub97c \ud55c \ubc88\ub9cc \uc815\uc758\ud558\uace0, \ucef4\ud30c\uc77c \ud55c \ud6c4,\n \ub370\uc774\ud130\ub97c \uacc4\uc0b0 \uadf8\ub798\ud504\uc5d0 \ub118\uae41\ub2c8\ub2e4.\n* \ub3d9\uc801 \ud234\ud0b7\uc5d0\uc11c\ub294 *\uac01 \ub370\uc774\ud130* \uc758 \uacc4\uc0b0 \uadf8\ub798\ud504\ub97c \uc815\uc758\ud558\uba70 \ucef4\ud30c\uc77c\ud558\uc9c0\n \uc54a\uace0 \uc989\uac01 \uc2e4\ud589\ub429\ub2c8\ub2e4.\n\n\uacbd\ud5d8\uc774 \ub9ce\uc9c0 \uc54a\ub2e4\uba74 \ub450 \ubc29\uc2dd\uc758 \ucc28\uc774\ub97c \uc54c\uae30 \uc5b4\ub835\uc2b5\ub2c8\ub2e4. \ub525 \ub7ec\ub2dd \uae30\ubc18\uc758\n\uad6c\uad6c\uc870 \ubd84\uc11d\uae30(constituent parser)\ub97c \uc608\ub85c \ub4e4\uc5b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4. \ubaa8\ub378\uc740 \ub300\ub7b5\n\ub2e4\uc74c\uacfc \uac19\uc740 \uacfc\uc815\uc744 \uc218\ud589\ud569\ub2c8\ub2e4:\n\n* \ud2b8\ub9ac\ub97c \uc0c1\ud5a5\uc2dd(bottom-up)\uc73c\ub85c \ub9cc\ub4e4\uc5b4 \ub098\uac11\ub2c8\ub2e4.\n* \ucd5c\uc0c1\uc704 \ub178\ub4dc\ub97c \ud0dc\uae45\ud569\ub2c8\ub2e4. (\ubb38\uc7a5\uc758 \uac01 \ub2e8\uc5b4)\n* \uac70\uae30\uc11c\ubd80\ud130 \uc2e0\uacbd\ub9dd\uacfc \ub2e8\uc5b4\ub4e4\uc758 \uc784\ubca0\ub529\uc744 \uc774\uc6a9\ud574 \uad6c\uad6c\uc870\ub97c \uc774\ub8e8\ub294 \uc870\ud569\uc744\n \ucc3e\uc544\ub0c5\ub2c8\ub2e4. \uc0c8\ub85c\uc6b4 \uad6c\uad6c\uc870\ub97c \uc0dd\uc131\ud560 \ub54c\ub9c8\ub2e4 \uad6c\uad6c\uc870\uc758 \uc784\ubca0\ub529\uc744 \uc5bb\uae30 \uc704\ud55c\n \uc5b4\ub5a4 \uae30\uc220\uc774 \ud544\uc694\ud569\ub2c8\ub2e4. \uc9c0\uae08\uc740 \uc2e0\uacbd\ub9dd\uc774 \uc624\uc9c1 \uc785\ub825 \ubb38\uc7a5\ub9cc \ucc38\uace0\ud560\n \uac83\uc785\ub2c8\ub2e4. \"The green cat scratched the wall\"\uc774\ub780 \ubb38\uc7a5\uc5d0\uc11c, \ubaa8\ub378\uc758 \uc5b4\ub290 \uc2dc\uc810\uc5d0\n $(i,j,r) = (1, 3, \\text{NP})$ \ubc94\uc704 (\ub2e8\uc5b4 1\uc5d0\uc11c\ubd80\ud130 \ub2e8\uc5b4 3\uae4c\uc9c0\uac00\n NP \uad6c\uad6c\uc870\ub77c\ub294 \ub73b\uc774\uba70, \uc774 \ubb38\uc7a5\uc5d0\uc11c\ub294 \"The green cat\") \ub97c \uacb0\ud569\ud558\uae38 \uc6d0\ud560\n \uac83\uc785\ub2c8\ub2e4.\n\n\uadf8\ub7f0\ub370, \ub610\ub2e4\ub978 \ubb38\uc7a5 \"Somewhere, the big fat cat scratched the wall\" \uc5d0\uc11c\ub294\n\uc5b4\ub290 \uc2dc\uc810\uc5d0 $(2, 4, NP)$ \uad6c\uad6c\uc870\ub97c \ub9cc\ub4e4\uae30\ub97c \uc6d0\ud560 \uac83\uc785\ub2c8\ub2e4. \uc6b0\ub9ac\uac00\n\ub9cc\ub4e4\uae30 \uc6d0\ud558\ub294 \uad6c\uad6c\uc870\ub4e4\uc740 \ubb38\uc7a5\uc5d0 \ub530\ub77c \ub2e4\ub985\ub2c8\ub2e4. \ub9cc\uc57d \uc815\uc801 \ud234\ud0b7\uc5d0\uc11c\ucc98\ub7fc\n\uacc4\uc0b0 \uadf8\ub798\ud504\ub97c \ud55c \ubc88\ub9cc \ucef4\ud30c\uc77c\ud55c\ub2e4\uba74, \uc774 \uacfc\uc815\uc744 \ud504\ub85c\uadf8\ub798\ubc0d\ud558\uae30 \ub9e4\uc6b0 \uc5b4\ub835\uac70\ub098\n\ubd88\uac00\ub2a5\ud560 \uac83\uc785\ub2c8\ub2e4. \ud558\uc9c0\ub9cc \ub3d9\uc801 \ud234\ud0b7\uc5d0\uc11c\ub294 \ud558\ub098\uc758 \uacc4\uc0b0 \uadf8\ub798\ud504\ub9cc \uc788\uc9c0\n\uc54a\uc2b5\ub2c8\ub2e4. \uac01 \ubb38\uc7a5\ub4e4\ub9c8\ub2e4 \uc0c8\ub85c\uc6b4 \uacc4\uc0b0 \uadf8\ub798\ud504\uac00 \uc788\uc744 \uc218 \uc788\uae30 \ub54c\ubb38\uc5d0 \uc774\ub7f0\n\ubb38\uc81c\uac00 \uc5c6\uc2b5\ub2c8\ub2e4.\n\n\ub3d9\uc801 \ud2c0\ud0b7\uc740 \ub514\ubc84\uae45 \ud558\uae30 \ub354 \uc27d\uace0, \ucf54\ub4dc\uac00 \uae30\ubc18 \uc5b8\uc5b4\uc640 \ub354 \ube44\uc2b7\ud569\ub2c8\ub2e4\n(Pytorch\uc640 Dynet\uc774 Keras \ub610\ub294 Theano \ubcf4\ub2e4 Python \ucf54\ub4dc\uc640 \ub354 \ube44\uc2b7\ud569\ub2c8\ub2e4).\n\nBi-LSTM Conditional Rnadom Field \uc124\uba85\n-------------------------------------------\n\n\uc774 \uc601\uc5ed\uc5d0\uc11c\ub294 \uac1c\uccb4\uba85 \uc778\uc2dd\uc744 \uc218\ud589\ud558\ub294 \uc644\uc131\ub41c Bi-LSTM Conditional Random\nField \uc608\uc2dc\ub97c \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4. \uc704\uc5d0 \ub098\uc628 LSTM \ud0dc\uac70(tagger)\ub294 \uc77c\ubc18\uc801\uc73c\ub85c\n\ud488\uc0ac \ud0dc\uae45\uc744 \ud558\uae30\uc5d0 \ucda9\ubd84\ud569\ub2c8\ub2e4. \ud558\uc9c0\ub9cc CRF \uac19\uc740 \uc5f0\uc18d\ub41c \ub370\uc774\ud130\ub97c \ub2e4\ub8e8\ub294\n\ubaa8\ub378\uc740 \uc88b\uc740 \uac1c\uccb4\uba85 \uc778\uc2dd \ubaa8\ub378(NER)\uc5d0 \uaf2d \ud544\uc694\ud569\ub2c8\ub2e4. \uc5ec\ub7ec\ubd84\uc774 CRF\ub97c \uc798 \uc54c\uace0\n\uc788\ub2e4\uace0 \uac00\uc815\ud558\uaca0\uc2b5\ub2c8\ub2e4. \uc774\ub984\uc774 \ubb34\uc12d\uac8c \ub4e4\ub9b4 \uc218\ub3c4 \uc788\uc9c0\ub9cc, LSTM\uc774 \ud2b9\uc9d5\uc744\n\uc81c\uacf5\ud558\ub294 \uc810\uc744 \uc81c\uc678\ud558\uba74 \uc774 \ubaa8\ub378\uc740 CRF \uc785\ub2c8\ub2e4. \ud558\uc9c0\ub9cc \ub354 \ubc1c\uc804\ub41c \ubaa8\ub378\uc774\uba70,\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc758 \uc55e\ubd80\ubd84\uc5d0 \ub098\uc654\ub358 \ubaa8\ub378\ubcf4\ub2e4 \ud6e8\uc52c \ubcf5\uc7a1\ud569\ub2c8\ub2e4. \ub118\uc5b4\uac00\uace0 \uc2f6\ub2e4\uba74\n\ub118\uc5b4\uac00\ub3c4 \uad1c\ucc2e\uc2b5\ub2c8\ub2e4. \uc774\ud574\ud560 \uc218 \uc788\ub2e4\uace0 \uc0dd\uac01\ud55c\ub2e4\uba74, \uc544\ub798\ub97c \uc77d\uc5b4\ubcf4\uc138\uc694:\n\n- \ud0dc\uadf8 k\uc5d0 \ub300\ud55c i\ubc88\uc9f8 \ub2e8\uacc4\uc758 \ube44\ud130\ube44(viterbi) \ubcc0\uc218\ub97c \uc704\ud574 \uc21c\ud658 \ud750\ub984\uc744 \ub9cc\ub4e0\ub2e4.\n- \uc21c\ubc29\ud5a5 \ubcc0\uc218\ub97c \uacc4\uc0b0\ud558\uae30 \uc704\ud574 \uc704\uc758 \uc21c\ud55c \ud750\ub984\uc744 \uc870\uc815\ud55c\ub2e4.\n- \uc21c\ubc29\ud5a5 \ubcc0\uc218\ub97c \ub85c\uadf8 \uacf5\uac04\uc5d0\uc11c \uacc4\uc0b0\ud558\uae30 \uc704\ud574 \ub2e4\uc2dc \ud55c \ubc88 \uc870\uc815\ud55c\ub2e4.\n (\ud78c\ud2b8 : \ub85c\uadf8-\ud569-\uc9c0\uc218\uc2b9)\n\n\uc704\uc758 \uc138\uac00\uc9c0\ub97c \ud560 \uc218 \uc788\ub2e4\uba74, \uc544\ub798\uc758 \ucf54\ub4dc\ub97c \uc774\ud574\ud560 \uc218 \uc788\uc744 \uac83\uc785\ub2c8\ub2e4.\nCRF\ub294 \uc870\uac74\ubd80 \ud655\ub960\uc744 \uacc4\uc0b0\ud55c\ub2e4\ub294 \uc810\uc744 \uae30\uc5b5\ud558\uc138\uc694. $y$ \ub97c \uc5f0\uc18d\ub41c\n\ud0dc\uadf8\ub77c \ud558\uace0, $x$ \ub97c \uc5f0\uc18d\ub41c \uc785\ub825 \ub2e8\uc5b4\ub77c \ud558\uaca0\uc2b5\ub2c8\ub2e4. \uadf8\ub7ec\uba74 \uc544\ub798\uc758\n\uc2dd\uc744 \uacc4\uc0b0\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\\begin{align}P(y|x) = \\frac{\\exp{(\\text{Score}(x, y)})}{\\sum_{y'} \\exp{(\\text{Score}(x, y')})}\\end{align}\n\n\uc810\uc218(score) \ud568\uc218\ub294 \uc544\ub798\uc640 \uac19\uc774 \uc815\uc758\ub41c \ub85c\uadf8 \ud3ec\ud150\uc15c(potential) $\\log \\psi_i(x,y)$\n\ud568\uc218\uc5d0 \uc758\ud574 \uacb0\uc815\ub429\ub2c8\ub2e4.\n\n\\begin{align}\\text{Score}(x,y) = \\sum_i \\log \\psi_i(x,y)\\end{align}\n\n\ubd84\ubc30 \ud568\uc218(partition function)\ub97c \ub2e8\uc21c\ud654\ud558\uae30 \uc704\ud574\uc11c, \ud3ec\ud150\uc15c\uc774 \uc8fc\ubcc0\uc758\n\ud2b9\uc9d5\ub4e4\ub9cc \ubc18\uc601\ud55c\ub2e4\uace0 \ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\nBi-LSTM CRF \uc548\uc5d0 \ubc30\ucd9c(emission), \uc804\uc774(transition) \ub450 \uc885\ub958\uc758 \ud3ec\ud150\uc15c\uc744\n\uc815\uc758\ud569\ub2c8\ub2e4. $i$ \ubc88\uc9f8 \ub2e8\uc5b4\uc5d0 \ub300\ud55c \ubc30\ucd9c \ud3ec\ud150\uc15c\uc740 Bi-LSTM\uc758\n$i$ \ubc88\uc9f8 \uc2dc\uc810\uc758 \uc740\ub2c9 \uc0c1\ud0dc\uac00 \uacb0\uc815\ud569\ub2c8\ub2e4. \uc804\uc774 \uc810\uc218\ub294 $|T|x|T|$\n\ud615\ud0dc\uc778 \ud589\ub82c $\\textbf{P}$ \uc5d0 \uc800\uc7a5\ub418\uc5b4 \uc788\uc2b5\ub2c8\ub2e4. $T$ \ub294\n\ud0dc\uadf8\uc758 \uc9d1\ud569\uc785\ub2c8\ub2e4. \uc774 \uad6c\ud604\uc5d0\uc11c, $\\textbf{P}_{j,k}$ \ub294 tag $j$ \uc5d0\uc11c\ntag $k$ \ub85c\uc758 \uc804\uc774 \uc810\uc218\ub97c \uc758\ubbf8\ud569\ub2c8\ub2e4. \ub530\ub77c\uc11c:\n\n\\begin{align}\\text{Score}(x,y) = \\sum_i \\log \\psi_\\text{EMIT}(y_i \\rightarrow x_i) + \\log \\psi_\\text{TRANS}(y_{i-1} \\rightarrow y_i)\\end{align}\n\n\\begin{align}= \\sum_i h_i[y_i] + \\textbf{P}_{y_i, y_{i-1}}\\end{align}\n\n\ub450 \ubc88\uc9f8 \uc2dd\uc5d0\uc11c \uace0\uc720\ud558\uace0 \uc74c\uc218\uac00 \uc544\ub2cc \uc778\ub371\uc2a4\uc5d0 \uc758\ud574 \ud0dc\uadf8\uac00 \ubd80\uc5ec\ub410\ub2e4\uace0\n\uac04\uc8fc\ud569\ub2c8\ub2e4.\n\n\uc704\uc758 \uc124\uba85\uc774 \ub108\ubb34 \uac04\ub2e8\ud558\ub2e4\uace0 \uc0dd\uac01\ud55c\ub2e4\uba74 CRF\uc5d0 \ub300\ud55c Michael Collins\uc758\n\uae00\uc744 `\uc5ec\uae30 `__ \uc5d0\uc11c\n\uc77d\uc5b4\ubcf4\uc138\uc694.\n\n\uad6c\ud604 \ubb38\uc11c\n--------------------\n\n\uc544\ub798\uc758 \uc608\uc2dc\ub294 \ub85c\uadf8 \uacf5\uac04\uc5d0\uc11c \ubd84\ubc30 \ud568\uc218\ub97c \uacc4\uc0b0\ud558\uae30 \uc704\ud55c \uc21c\ubc29\ud5a5 \uc54c\uace0\ub9ac\uc998\uacfc\n\ubcf5\ud638\ud654\ud558\uae30 \uc704\ud55c \ube44\ud130\ube44 \uc54c\uace0\ub9ac\uc998\uc744 \uad6c\ud604\ud55c \uac83\uc785\ub2c8\ub2e4. \uc5ed\uc804\ud30c\n\ub2e8\uacc4\uc5d0\uc11c \ubcc0\ud654\ub3c4\ub294 \uc790\ub3d9\uc73c\ub85c \uacc4\uc0b0\ub420 \uac83\uc785\ub2c8\ub2e4. \uc6b0\ub9ac\uac00 \uc9c1\uc811 \ud560 \uc77c\uc740\n\uc5c6\uc2b5\ub2c8\ub2e4.\n\n\uc774 \uad6c\ud604\uc740 \ucd5c\uc801\uc758 \uc0c1\ud0dc\uac00 \uc544\ub2d9\ub2c8\ub2e4. \uacfc\uc815\uc744 \uc774\ud574\ud588\ub2e4\uba74, \uc21c\ubc29\ud5a5 \uc54c\uace0\ub9ac\uc998\n\uc0c1\uc5d0\uc11c \ub2e4\uc74c \ud0dc\uadf8\ub97c \uc21c\ucc28\uc801\uc73c\ub85c \ucc98\ub9ac\ud558\ub294 \uacfc\uc815\uc744 \ud558\ub098\uc758 \ud070 \uc5f0\uc0b0\uc73c\ub85c \uc904\uc77c\n\uc218 \uc788\ub2e4\ub294 \uac83\uc744 \uc544\ub9c8 \ube60\ub974\uac8c \uc54c \uc218 \uc788\uc744 \uac83\uc785\ub2c8\ub2e4. \uc774 \ucf54\ub4dc\ub294 \uac00\ub2a5\ud55c \uc77d\uae30\n\uc27d\uac8c \uc791\uc131\ud588\uc2b5\ub2c8\ub2e4. \uc801\uc808\ud558\uac8c \uc218\uc815\ud558\uba74, \uc774 \ud0dc\uac70\ub97c \uc2e4\uc81c \ubb38\uc81c\ub4e4\uc5d0 \uc0ac\uc6a9\ud560\n\uc218\ub3c4 \uc788\uc744 \uac83\uc785\ub2c8\ub2e4.\n" ] }, { @@ -26,14 +26,14 @@ }, "outputs": [], "source": [ - "# Author: Robert Guthrie\n\nimport torch\nimport torch.autograd as autograd\nimport torch.nn as nn\nimport torch.optim as optim\n\ntorch.manual_seed(1)" + "# \uc791\uc131\uc790: Robert Guthrie\n\nimport torch\nimport torch.autograd as autograd\nimport torch.nn as nn\nimport torch.optim as optim\n\ntorch.manual_seed(1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Helper functions to make the code more readable.\n\n" + "\ucf54\ub4dc \uac00\ub3c5\uc131\uc744 \ub192\uc5ec\uc8fc\ub294 \ubcf4\uc870 \ud568\uc218\ub4e4\n\n" ] }, { @@ -44,14 +44,14 @@ }, "outputs": [], "source": [ - "def argmax(vec):\n # return the argmax as a python int\n _, idx = torch.max(vec, 1)\n return idx.item()\n\n\ndef prepare_sequence(seq, to_ix):\n idxs = [to_ix[w] for w in seq]\n return torch.tensor(idxs, dtype=torch.long)\n\n\n# Compute log sum exp in a numerically stable way for the forward algorithm\ndef log_sum_exp(vec):\n max_score = vec[0, argmax(vec)]\n max_score_broadcast = max_score.view(1, -1).expand(1, vec.size()[1])\n return max_score + \\\n torch.log(torch.sum(torch.exp(vec - max_score_broadcast)))" + "def argmax(vec):\n # argmax\ub97c \ud30c\uc774\uc36c \uc815\uc218\ud615\uc73c\ub85c \ubc18\ud658\ud569\ub2c8\ub2e4.\n _, idx = torch.max(vec, 1)\n return idx.item()\n\n\ndef prepare_sequence(seq, to_ix):\n idxs = [to_ix[w] for w in seq]\n return torch.tensor(idxs, dtype=torch.long)\n\n\n# \uc21c\ubc29\ud5a5 \uc54c\uace0\ub9ac\uc998\uc744 \uc704\ud574 \uc218\uce58\uc801\uc73c\ub85c \uc548\uc815\uc801\uc778 \ubc29\ubc95\uc73c\ub85c \ub85c\uadf8 \ud569 \uc9c0\uc218\uc2b9\uc744 \uacc4\uc0b0\ud569\ub2c8\ub2e4.\ndef log_sum_exp(vec):\n max_score = vec[0, argmax(vec)]\n max_score_broadcast = max_score.view(1, -1).expand(1, vec.size()[1])\n return max_score + \\\n torch.log(torch.sum(torch.exp(vec - max_score_broadcast)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Create model\n\n" + "\ubaa8\ub378 \uc0dd\uc131\n\n" ] }, { @@ -62,14 +62,14 @@ }, "outputs": [], "source": [ - "class BiLSTM_CRF(nn.Module):\n\n def __init__(self, vocab_size, tag_to_ix, embedding_dim, hidden_dim):\n super(BiLSTM_CRF, self).__init__()\n self.embedding_dim = embedding_dim\n self.hidden_dim = hidden_dim\n self.vocab_size = vocab_size\n self.tag_to_ix = tag_to_ix\n self.tagset_size = len(tag_to_ix)\n\n self.word_embeds = nn.Embedding(vocab_size, embedding_dim)\n self.lstm = nn.LSTM(embedding_dim, hidden_dim // 2,\n num_layers=1, bidirectional=True)\n\n # Maps the output of the LSTM into tag space.\n self.hidden2tag = nn.Linear(hidden_dim, self.tagset_size)\n\n # Matrix of transition parameters. Entry i,j is the score of\n # transitioning *to* i *from* j.\n self.transitions = nn.Parameter(\n torch.randn(self.tagset_size, self.tagset_size))\n\n # These two statements enforce the constraint that we never transfer\n # to the start tag and we never transfer from the stop tag\n self.transitions.data[tag_to_ix[START_TAG], :] = -10000\n self.transitions.data[:, tag_to_ix[STOP_TAG]] = -10000\n\n self.hidden = self.init_hidden()\n\n def init_hidden(self):\n return (torch.randn(2, 1, self.hidden_dim // 2),\n torch.randn(2, 1, self.hidden_dim // 2))\n\n def _forward_alg(self, feats):\n # Do the forward algorithm to compute the partition function\n init_alphas = torch.full((1, self.tagset_size), -10000.)\n # START_TAG has all of the score.\n init_alphas[0][self.tag_to_ix[START_TAG]] = 0.\n\n # Wrap in a variable so that we will get automatic backprop\n forward_var = init_alphas\n\n # Iterate through the sentence\n for feat in feats:\n alphas_t = [] # The forward tensors at this timestep\n for next_tag in range(self.tagset_size):\n # broadcast the emission score: it is the same regardless of\n # the previous tag\n emit_score = feat[next_tag].view(\n 1, -1).expand(1, self.tagset_size)\n # the ith entry of trans_score is the score of transitioning to\n # next_tag from i\n trans_score = self.transitions[next_tag].view(1, -1)\n # The ith entry of next_tag_var is the value for the\n # edge (i -> next_tag) before we do log-sum-exp\n next_tag_var = forward_var + trans_score + emit_score\n # The forward variable for this tag is log-sum-exp of all the\n # scores.\n alphas_t.append(log_sum_exp(next_tag_var).view(1))\n forward_var = torch.cat(alphas_t).view(1, -1)\n terminal_var = forward_var + self.transitions[self.tag_to_ix[STOP_TAG]]\n alpha = log_sum_exp(terminal_var)\n return alpha\n\n def _get_lstm_features(self, sentence):\n self.hidden = self.init_hidden()\n embeds = self.word_embeds(sentence).view(len(sentence), 1, -1)\n lstm_out, self.hidden = self.lstm(embeds, self.hidden)\n lstm_out = lstm_out.view(len(sentence), self.hidden_dim)\n lstm_feats = self.hidden2tag(lstm_out)\n return lstm_feats\n\n def _score_sentence(self, feats, tags):\n # Gives the score of a provided tag sequence\n score = torch.zeros(1)\n tags = torch.cat([torch.tensor([self.tag_to_ix[START_TAG]], dtype=torch.long), tags])\n for i, feat in enumerate(feats):\n score = score + \\\n self.transitions[tags[i + 1], tags[i]] + feat[tags[i + 1]]\n score = score + self.transitions[self.tag_to_ix[STOP_TAG], tags[-1]]\n return score\n\n def _viterbi_decode(self, feats):\n backpointers = []\n\n # Initialize the viterbi variables in log space\n init_vvars = torch.full((1, self.tagset_size), -10000.)\n init_vvars[0][self.tag_to_ix[START_TAG]] = 0\n\n # forward_var at step i holds the viterbi variables for step i-1\n forward_var = init_vvars\n for feat in feats:\n bptrs_t = [] # holds the backpointers for this step\n viterbivars_t = [] # holds the viterbi variables for this step\n\n for next_tag in range(self.tagset_size):\n # next_tag_var[i] holds the viterbi variable for tag i at the\n # previous step, plus the score of transitioning\n # from tag i to next_tag.\n # We don't include the emission scores here because the max\n # does not depend on them (we add them in below)\n next_tag_var = forward_var + self.transitions[next_tag]\n best_tag_id = argmax(next_tag_var)\n bptrs_t.append(best_tag_id)\n viterbivars_t.append(next_tag_var[0][best_tag_id].view(1))\n # Now add in the emission scores, and assign forward_var to the set\n # of viterbi variables we just computed\n forward_var = (torch.cat(viterbivars_t) + feat).view(1, -1)\n backpointers.append(bptrs_t)\n\n # Transition to STOP_TAG\n terminal_var = forward_var + self.transitions[self.tag_to_ix[STOP_TAG]]\n best_tag_id = argmax(terminal_var)\n path_score = terminal_var[0][best_tag_id]\n\n # Follow the back pointers to decode the best path.\n best_path = [best_tag_id]\n for bptrs_t in reversed(backpointers):\n best_tag_id = bptrs_t[best_tag_id]\n best_path.append(best_tag_id)\n # Pop off the start tag (we dont want to return that to the caller)\n start = best_path.pop()\n assert start == self.tag_to_ix[START_TAG] # Sanity check\n best_path.reverse()\n return path_score, best_path\n\n def neg_log_likelihood(self, sentence, tags):\n feats = self._get_lstm_features(sentence)\n forward_score = self._forward_alg(feats)\n gold_score = self._score_sentence(feats, tags)\n return forward_score - gold_score\n\n def forward(self, sentence): # dont confuse this with _forward_alg above.\n # Get the emission scores from the BiLSTM\n lstm_feats = self._get_lstm_features(sentence)\n\n # Find the best path, given the features.\n score, tag_seq = self._viterbi_decode(lstm_feats)\n return score, tag_seq" + "class BiLSTM_CRF(nn.Module):\n\n def __init__(self, vocab_size, tag_to_ix, embedding_dim, hidden_dim):\n super(BiLSTM_CRF, self).__init__()\n self.embedding_dim = embedding_dim\n self.hidden_dim = hidden_dim\n self.vocab_size = vocab_size\n self.tag_to_ix = tag_to_ix\n self.tagset_size = len(tag_to_ix)\n\n self.word_embeds = nn.Embedding(vocab_size, embedding_dim)\n self.lstm = nn.LSTM(embedding_dim, hidden_dim // 2,\n num_layers=1, bidirectional=True)\n\n # LSTM\uc758 \ucd9c\ub825\uc744 \ud0dc\uadf8 \uacf5\uac04\uc73c\ub85c \ub300\uc751\uc2dc\ud0b5\ub2c8\ub2e4.\n self.hidden2tag = nn.Linear(hidden_dim, self.tagset_size)\n\n # \uc804\uc774 \ub9e4\uac1c\ubcc0\uc218 \ud589\ub82c. i, j \uc131\ubd84\uc740 i\uc5d0\uc11c j\ub85c \ubcc0\ud560 \ub54c\uc758 \uc810\uc218\uc785\ub2c8\ub2e4.\n self.transitions = nn.Parameter(\n torch.randn(self.tagset_size, self.tagset_size))\n\n # \uc774 \ub450 \ucf54\ub4dc\ub294 \uc2dc\uc791 \ud0dc\uadf8\ub85c \uc804\uc774\ud558\uc9c0 \uc54a\uace0, \uc815\uc9c0 \ud0dc\uadf8\uc5d0\uc11c\ubd80\ud130\n # \uc804\uc774\ud558\uc9c0 \uc54a\ub3c4\ub85d \uac15\uc81c\ud569\ub2c8\ub2e4.\n self.transitions.data[tag_to_ix[START_TAG], :] = -10000\n self.transitions.data[:, tag_to_ix[STOP_TAG]] = -10000\n\n self.hidden = self.init_hidden()\n\n def init_hidden(self):\n return (torch.randn(2, 1, self.hidden_dim // 2),\n torch.randn(2, 1, self.hidden_dim // 2))\n\n def _forward_alg(self, feats):\n # \ubd84\ubc30 \ud568\uc218\ub97c \uacc4\uc0b0\ud558\uae30 \uc704\ud574 \uc21c\ubc29\ud5a5 \uc54c\uace0\ub9ac\uc998\uc744 \uc218\ud589\ud569\ub2c8\ub2e4.\n init_alphas = torch.full((1, self.tagset_size), -10000.)\n # START_TAG\ub294 \ubaa8\ub4e0 \uc810\uc218\ub97c \uac16\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n init_alphas[0][self.tag_to_ix[START_TAG]] = 0.\n\n # \uc790\ub3d9\uc73c\ub85c \uc5ed\uc804\ud30c \ub418\ub3c4\ub85d \ubcc0\uc218\ub85c \uac10\uc309\ub2c8\ub2e4.\n forward_var = init_alphas\n\n # \ubb38\uc7a5\uc758 \uac01 \uc131\ubd84\uc744 \ubc18\ubcf5 \ucc98\ub9ac\ud569\ub2c8\ub2e4.\n for feat in feats:\n alphas_t = [] # \ud604\uc7ac \uc2dc\uc810\uc758 \uc21c\ubc29\ud5a5 \ud150\uc11c\n for next_tag in range(self.tagset_size):\n # \uc774\uc804\uc758 \ud0dc\uadf8\uc640 \uc0c1\uad00\uc5c6\uc774 \ubc30\ucd9c \uc810\uc218\ub97c \uc804\ud30c\ud569\ub2c8\ub2e4.\n emit_score = feat[next_tag].view(\n 1, -1).expand(1, self.tagset_size)\n # trans_score\uc758 i\ubc88\uc9f8 \uc131\ubd84\uc740 i\ub85c\ubd80\ud130 next_tag\ub85c \uc804\uc774\ud560 \uc810\uc218\uc785\ub2c8\ub2e4.\n trans_score = self.transitions[next_tag].view(1, -1)\n # next_tag_var\uc758 i\ubc88\uc9f8 \uc131\ubd84\uc740 \ub85c\uadf8-\ud569-\uc9c0\uc218\uc2b9\uc744 \uacc4\uc0b0\ud558\uae30 \uc804\n # i\uc5d0\uc11c next_tag\ub85c \uac00\ub294 \uac04\uc120\uc758 \uac12\uc785\ub2c8\ub2e4.\n next_tag_var = forward_var + trans_score + emit_score\n # \uc774 \ud0dc\uadf8\uc758 \uc21c\ubc29\ud5a5 \ubcc0\uc218\ub294 \ubaa8\ub4e0 \uc810\uc218\ub4e4\uc758 \ub85c\uadf8-\ud569-\uc9c0\uc218\uc2b9 \uacc4\uc0b0\n # \uacb0\uacfc\uc785\ub2c8\ub2e4.\n alphas_t.append(log_sum_exp(next_tag_var).view(1))\n forward_var = torch.cat(alphas_t).view(1, -1)\n terminal_var = forward_var + self.transitions[self.tag_to_ix[STOP_TAG]]\n alpha = log_sum_exp(terminal_var)\n return alpha\n\n def _get_lstm_features(self, sentence):\n self.hidden = self.init_hidden()\n embeds = self.word_embeds(sentence).view(len(sentence), 1, -1)\n lstm_out, self.hidden = self.lstm(embeds, self.hidden)\n lstm_out = lstm_out.view(len(sentence), self.hidden_dim)\n lstm_feats = self.hidden2tag(lstm_out)\n return lstm_feats\n\n def _score_sentence(self, feats, tags):\n # \uc8fc\uc5b4\uc9c4 \ud0dc\uadf8 \uc21c\uc5f4\uc5d0 \uc810\uc218\ub97c \ub9e4\uae41\ub2c8\ub2e4.\n score = torch.zeros(1)\n tags = torch.cat([torch.tensor([self.tag_to_ix[START_TAG]], dtype=torch.long), tags])\n for i, feat in enumerate(feats):\n score = score + \\\n self.transitions[tags[i + 1], tags[i]] + feat[tags[i + 1]]\n score = score + self.transitions[self.tag_to_ix[STOP_TAG], tags[-1]]\n return score\n\n def _viterbi_decode(self, feats):\n backpointers = []\n\n # \ube44\ud130\ube44 \ubcc0\uc218\ub97c \ub85c\uadf8 \uacf5\uac04 \uc0c1\uc5d0 \ucd08\uae30\ud654\ud569\ub2c8\ub2e4.\n init_vvars = torch.full((1, self.tagset_size), -10000.)\n init_vvars[0][self.tag_to_ix[START_TAG]] = 0\n\n # i \ub2e8\uacc4\uc758 forward_var\ub294 i-1 \ub2e8\uacc4\uc758 \ube44\ud130\ube44 \ubcc0\uc218\ub97c \uac16\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n forward_var = init_vvars\n for feat in feats:\n bptrs_t = [] # \ud604\uc7ac \ub2e8\uacc4\uc758 backpointer\ub97c \uac16\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n viterbivars_t = [] # \ud604\uc7ac \ub2e8\uacc4\uc758 \ube44\ud130\ube44 \ubcc0\uc218\ub97c \uac16\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n\n for next_tag in range(self.tagset_size):\n # next_tag_var[i]\ub294 \uc774\uc804 \ub2e8\uacc4\uc758 \ud0dc\uadf8 i\uc5d0 \ub300\ud55c \ube44\ud130\ube44 \ubcc0\uc218\uc640,\n # \ud0dc\uadf8 i\uc5d0\uc11c next_tag\ub85c \uc804\uc774\ud560 \uc810\uc218\ub97c \ub354\ud55c \uac12\uc744 \uac16\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n # \ubc30\ucd9c \uc810\uc218\ub294 argmax\uc640 \uc0c1\uad00 \uc5c6\uae30 \ub54c\ubb38\uc5d0(\uc544\ub798 \ucf54\ub4dc\uc5d0\uc11c \ucd94\uac00\ud560 \uac83\uc785\ub2c8\ub2e4)\n # \uc5ec\uae30\uc5d0 \ud3ec\ud568\ub418\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4.\n next_tag_var = forward_var + self.transitions[next_tag]\n best_tag_id = argmax(next_tag_var)\n bptrs_t.append(best_tag_id)\n viterbivars_t.append(next_tag_var[0][best_tag_id].view(1))\n # \uc774\uc81c \ubc30\ucd9c \uc810\uc218\ub97c \ub354\ud569\ub2c8\ub2e4. \uadf8\ub9ac\uace0 \ubc29\uae08 \uacc4\uc0b0\ud55c \ube44\ud130\ube44 \ubcc0\uc218\uc758\n # \uc9d1\ud569\uc744 forward_var\uc5d0 \ud560\ub2f9\ud569\ub2c8\ub2e4.\n forward_var = (torch.cat(viterbivars_t) + feat).view(1, -1)\n backpointers.append(bptrs_t)\n\n # STAP_TAG\ub85c\uc758 \uc804\uc774\n terminal_var = forward_var + self.transitions[self.tag_to_ix[STOP_TAG]]\n best_tag_id = argmax(terminal_var)\n path_score = terminal_var[0][best_tag_id]\n\n # \ucd5c\uc801\uc758 \uacbd\ub85c\ub97c \uad6c\ud558\uae30 \uc704\ud574 back pointer\ub97c \ub530\ub77c\uac11\ub2c8\ub2e4.\n best_path = [best_tag_id]\n for bptrs_t in reversed(backpointers):\n best_tag_id = bptrs_t[best_tag_id]\n best_path.append(best_tag_id)\n # \uc2dc\uc791 \ud0dc\uadf8\ub97c \ube7c\ub0c5\ub2c8\ub2e4 (\uc2dc\uc791 \ud0dc\uadf8\ub294 \ubc18\ud658\ub41c \ud544\uc694\uac00 \uc5c6\uc2b5\ub2c8\ub2e4)\n start = best_path.pop()\n assert start == self.tag_to_ix[START_TAG] # \uc644\uacb0\uc131 \uac80\uc0ac (Sanity check)\n best_path.reverse()\n return path_score, best_path\n\n def neg_log_likelihood(self, sentence, tags):\n feats = self._get_lstm_features(sentence)\n forward_score = self._forward_alg(feats)\n gold_score = self._score_sentence(feats, tags)\n return forward_score - gold_score\n\n def forward(self, sentence): # \uc774 \ud568\uc218\uc640 \uc704\uc758 _forward_alg\ub97c \ud5f7\uac08\ub9ac\uc9c0 \ub9c8\uc138\uc694.\n # Bi-LSTM\uc73c\ub85c\ubd80\ud130 \ubc30\ucd9c \uc810\uc218\ub97c \uc5bb\uc2b5\ub2c8\ub2e4.\n lstm_feats = self._get_lstm_features(sentence)\n\n # \uc8fc\uc5b4\uc9c4 \ud2b9\uc9d5(\ubc30\ucd9c \uc810\uc218)\ub4e4\ub85c \ucd5c\uc801\uc758 \uacbd\ub85c\ub97c \ucc3e\uc544\ub0c5\ub2c8\ub2e4.\n score, tag_seq = self._viterbi_decode(lstm_feats)\n return score, tag_seq" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Run training\n\n" + "\ud6c8\ub828 \uc2e4\ud589\n\n" ] }, { @@ -80,14 +80,14 @@ }, "outputs": [], "source": [ - "START_TAG = \"\"\nSTOP_TAG = \"\"\nEMBEDDING_DIM = 5\nHIDDEN_DIM = 4\n\n# Make up some training data\ntraining_data = [(\n \"the wall street journal reported today that apple corporation made money\".split(),\n \"B I I I O O O B I O O\".split()\n), (\n \"georgia tech is a university in georgia\".split(),\n \"B I O O O O B\".split()\n)]\n\nword_to_ix = {}\nfor sentence, tags in training_data:\n for word in sentence:\n if word not in word_to_ix:\n word_to_ix[word] = len(word_to_ix)\n\ntag_to_ix = {\"B\": 0, \"I\": 1, \"O\": 2, START_TAG: 3, STOP_TAG: 4}\n\nmodel = BiLSTM_CRF(len(word_to_ix), tag_to_ix, EMBEDDING_DIM, HIDDEN_DIM)\noptimizer = optim.SGD(model.parameters(), lr=0.01, weight_decay=1e-4)\n\n# Check predictions before training\nwith torch.no_grad():\n precheck_sent = prepare_sequence(training_data[0][0], word_to_ix)\n precheck_tags = torch.tensor([tag_to_ix[t] for t in training_data[0][1]], dtype=torch.long)\n print(model(precheck_sent))\n\n# Make sure prepare_sequence from earlier in the LSTM section is loaded\nfor epoch in range(\n 300): # again, normally you would NOT do 300 epochs, it is toy data\n for sentence, tags in training_data:\n # Step 1. Remember that Pytorch accumulates gradients.\n # We need to clear them out before each instance\n model.zero_grad()\n\n # Step 2. Get our inputs ready for the network, that is,\n # turn them into Tensors of word indices.\n sentence_in = prepare_sequence(sentence, word_to_ix)\n targets = torch.tensor([tag_to_ix[t] for t in tags], dtype=torch.long)\n\n # Step 3. Run our forward pass.\n loss = model.neg_log_likelihood(sentence_in, targets)\n\n # Step 4. Compute the loss, gradients, and update the parameters by\n # calling optimizer.step()\n loss.backward()\n optimizer.step()\n\n# Check predictions after training\nwith torch.no_grad():\n precheck_sent = prepare_sequence(training_data[0][0], word_to_ix)\n print(model(precheck_sent))\n# We got it!" + "START_TAG = \"\"\nSTOP_TAG = \"\"\nEMBEDDING_DIM = 5\nHIDDEN_DIM = 4\n\n# \ud6c8\ub828\uc6a9 \ub370\uc774\ud130\ub97c \ub9cc\ub4ed\ub2c8\ub2e4.\ntraining_data = [(\n \"the wall street journal reported today that apple corporation made money\".split(),\n \"B I I I O O O B I O O\".split()\n), (\n \"georgia tech is a university in georgia\".split(),\n \"B I O O O O B\".split()\n)]\n\nword_to_ix = {}\nfor sentence, tags in training_data:\n for word in sentence:\n if word not in word_to_ix:\n word_to_ix[word] = len(word_to_ix)\n\ntag_to_ix = {\"B\": 0, \"I\": 1, \"O\": 2, START_TAG: 3, STOP_TAG: 4}\n\nmodel = BiLSTM_CRF(len(word_to_ix), tag_to_ix, EMBEDDING_DIM, HIDDEN_DIM)\noptimizer = optim.SGD(model.parameters(), lr=0.01, weight_decay=1e-4)\n\n# \ud6c8\ub828 \uc804 \uc608\uce21 \uacb0\uacfc\ub97c \ud655\uc778\ud569\ub2c8\ub2e4.\nwith torch.no_grad():\n precheck_sent = prepare_sequence(training_data[0][0], word_to_ix)\n precheck_tags = torch.tensor([tag_to_ix[t] for t in training_data[0][1]], dtype=torch.long)\n print(model(precheck_sent))\n\n# Make sure prepare_sequence from earlier in the LSTM section is loaded\n# \uc704\uc758 \ubcf4\uc870 \ud568\uc218 \uc601\uc5ed\uc5d0 \uc788\ub294 prepare_sequence \ud568\uc218\uac00 \ubd88\ub7ec\uc640 \uc84c\ub294\uc9c0 \ud655\uc778\ud569\ub2c8\ub2e4.\nfor epoch in range(\n 300): # \ub2e4\uc2dc \ub9d0\ud558\uc9c0\ub9cc, \uc544\ub9c8 300 \uc5d0\ud3ed\uc744 \uc2e4\ud589\ud558\uc9c4 \uc54a\uc744 \uac83\uc785\ub2c8\ub2e4. \uc774\uac83\uc740 \uc5f0\uc2b5\uc6a9 \ub370\uc774\ud130\uc785\ub2c8\ub2e4.\n for sentence, tags in training_data:\n # 1\ub2e8\uacc4. Pytorch\uac00 \ubcc0\ud654\ub3c4\ub97c \ub204\uc801\ud55c\ub2e4\ub294 \uac83\uc744 \uae30\uc5b5\ud558\uc138\uc694.\n # \uadf8\uac83\ub4e4\uc744 \uc81c\uac70\ud569\ub2c8\ub2e4.\n model.zero_grad()\n\n # 2\ub2e8\uacc4. \uc785\ub825 \ub370\uc774\ud130\ub97c \uc2e0\uacbd\ub9dd\uc5d0 \uc0ac\uc6a9\ub420 \uc218 \uc788\ub3c4\ub85d \ub2e8\uc5b4\n # \uc778\ub371\uc2a4\ub4e4\uc758 \ud150\uc11c\ub85c \ubcc0\ud658\ud569\ub2c8\ub2e4.\n sentence_in = prepare_sequence(sentence, word_to_ix)\n targets = torch.tensor([tag_to_ix[t] for t in tags], dtype=torch.long)\n\n # 3\ub2e8\uacc4. \uc21c\ubc29\ud5a5 \uacc4\uc0b0\uc744 \uc218\ud589\ud569\ub2c8\ub2e4.\n loss = model.neg_log_likelihood(sentence_in, targets)\n\n # 4\ub2e8\uacc4. \uc190\uc2e4\uac12, \ubcc0\ud654\ub3c4\ub97c \uacc4\uc0b0\ud558\uace0 optimizer.step()\uc744 \ud638\ucd9c\ud558\uc5ec\n # \ub9e4\uac1c\ubcc0\uc218\ub4e4\uc744 \uac31\uc2e0\ud569\ub2c8\ub2e4.\n loss.backward()\n optimizer.step()\n\n# \ud6c8\ub828\uc774 \ub05d\ub09c \ud6c4 \uc608\uce21 \uacb0\uacfc\ub97c \ud655\uc778\ud569\ub2c8\ub2e4.\nwith torch.no_grad():\n precheck_sent = prepare_sequence(training_data[0][0], word_to_ix)\n print(model(precheck_sent))\n# \ub2e4 \ud588\uc2b5\ub2c8\ub2e4!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Exercise: A new loss function for discriminative tagging\n--------------------------------------------------------\n\nIt wasn't really necessary for us to create a computation graph when\ndoing decoding, since we do not backpropagate from the viterbi path\nscore. Since we have it anyway, try training the tagger where the loss\nfunction is the difference between the Viterbi path score and the score\nof the gold-standard path. It should be clear that this function is\nnon-negative and 0 when the predicted tag sequence is the correct tag\nsequence. This is essentially *structured perceptron*.\n\nThis modification should be short, since Viterbi and score\\_sentence are\nalready implemented. This is an example of the shape of the computation\ngraph *depending on the training instance*. Although I haven't tried\nimplementing this in a static toolkit, I imagine that it is possible but\nmuch less straightforward.\n\nPick up some real data and do a comparison!\n\n\n" + "\uc5f0\uc2b5 : \ud310\ubcc4\uc801(discriminative) \ud0dc\uae45\uc744 \uc704\ud55c \uc0c8\ub85c\uc6b4 \uc190\uc2e4 \ud568\uc218\n------------------------------------------------------------\n\n\uc0ac\uc2e4 \ubcf5\ud638\ud654 \ud560 \ub54c\ub294 \ube44\ud130\ube44 \uacbd\ub85c \uc810\uc218\ub85c \uc5ed\uc804\ud30c\ub97c \ud558\uc9c0 \uc54a\uae30 \ub54c\ubb38\uc5d0 \uacc4\uc0b0\n\uadf8\ub798\ud504\ub97c \ub9cc\ub4e4 \ud544\uc694\uac00 \uc5c6\uc5c8\uc2b5\ub2c8\ub2e4. \uadf8\ub7ec\ub098 \uc774\ubbf8 \ub9cc\ub4e4\uc5c8\uc73c\ub2c8, \ube44\ud130\ube44 \uacbd\ub85c\n\uc810\uc218\uc640 \uc2e4\uc81c \uc815\ub2f5 \uacbd\ub85c \uc810\uc218\uc758 \ucc28\uc774\ub97c \uc190\uc2e4 \ud568\uc218\ub85c \uc0ac\uc6a9\ud574\uc11c \ud0dc\uac70\ub97c\n\ud559\uc2b5\uc2dc\ucf1c \ubcf4\uc138\uc694. \uc190\uc2e4 \ud568\uc218\uc758 \uac12\uc740 \uc74c\uc218\uac00 \uc544\ub2c8\uc5b4\uc57c \ud558\uba70, \uc608\uce21\ub41c \ud0dc\uadf8\n\uc21c\uc5f4\uc774 \uc815\ub2f5\uc774\ub77c\uba74 \uc190\uc2e4 \ud568\uc218\uc758 \uac12\uc740 0\uc774\uc5b4\uc57c \ud569\ub2c8\ub2e4. \uc774\uac83\uc740 \ubcf8\uc9c8\uc801\uc73c\ub85c\n*\uad6c\uc870\ud654\ub41c \ud37c\uc149\ud2b8\ub860* \uc785\ub2c8\ub2e4.\n\n\uc774\ubbf8 \ube44\ud130\ube44\uc640 score_sentence \ud568\uc218\uac00 \uad6c\ud604\ub418\uc5b4 \uc788\uae30 \ub54c\ubb38\uc5d0 \uac04\ub2e8\ud788 \uc218\uc815\ud560\n\uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc774 \ubaa8\ub378\uc740 *\ud559\uc2b5 \ub370\uc774\ud130\uc5d0 \ub530\ub77c \ubcc0\ud558\ub294* \uacc4\uc0b0 \uadf8\ub798\ud504\uc758 \ud55c\n\uc608\uc2dc\uc785\ub2c8\ub2e4. \uc774 \ubaa8\ub378\uc744 \uc815\uc801 \ud234\ud0b7\uc5d0\uc11c \uad6c\ud604\ud574 \ubcf4\uc9c0\ub294 \uc54a\uc558\ub294\ub370, \uad6c\ud604\uc774\n\uac00\ub2a5\ud558\uc9c0\ub9cc \ub35c \uc9c1\uad00\uc801\uc77c \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc2e4\uc81c \ub370\uc774\ud130\ub97c \uc0ac\uc6a9\ud574\ubcf4\uace0 \ube44\uad50\ud574\ubcf4\uc138\uc694!\n\n" ] } ], @@ -107,7 +107,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/9a0e851f5cb70c78bfde07b9bd268569/dynamic_quantization_tutorial.ipynb b/docs/_downloads/9a0e851f5cb70c78bfde07b9bd268569/dynamic_quantization_tutorial.ipynb index f708f3859..4d0efb1b0 100644 --- a/docs/_downloads/9a0e851f5cb70c78bfde07b9bd268569/dynamic_quantization_tutorial.ipynb +++ b/docs/_downloads/9a0e851f5cb70c78bfde07b9bd268569/dynamic_quantization_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n(beta) Dynamic Quantization on an LSTM Word Language Model\n==================================================================\n\n**Author**: `James Reed `_\n\n**Edited by**: `Seth Weidman `_\n\nIntroduction\n------------\n\nQuantization involves converting the weights and activations of your model from float\nto int, which can result in smaller model size and faster inference with only a small\nhit to accuracy.\n\nIn this tutorial, we'll apply the easiest form of quantization -\n`dynamic quantization `_ -\nto an LSTM-based next word-prediction model, closely following the\n`word language model `_\nfrom the PyTorch examples.\n\n" + "\n(\ubca0\ud0c0) LSTM \uae30\ubc18 \ub2e8\uc5b4 \ub2e8\uc704 \uc5b8\uc5b4 \ubaa8\ub378\uc758 \ub3d9\uc801 \uc591\uc790\ud654\n==================================================================\n\n**Author**: `James Reed `_\n\n**Edited by**: `Seth Weidman `_\n\n**\ubc88\uc5ed**: `\ubc15\uacbd\ub9bc `_ `Myungha Kwon `_ \n\n\uc2dc\uc791\ud558\uae30\n------------\n\n\uc591\uc790\ud654\ub294 \ubaa8\ub378\uc758 \ud06c\uae30\ub97c \uc904\uc774\uace0 \ucd94\ub860 \uc18d\ub3c4\ub97c \ub192\uc774\uba74\uc11c\ub3c4 \uc815\ud655\ub3c4\ub294 \ubcc4\ub85c \ub0ae\uc544\uc9c0\uc9c0 \uc54a\ub3c4\ub85d,\n\ubaa8\ub378\uc758 \uac00\uc911\uce58\uc640 \ud65c\uc131 \ud568\uc218\ub97c \uc2e4\uc218\ud615\uc5d0\uc11c \uc815\uc218\ud615\uc73c\ub85c \ubcc0\ud658\ud569\ub2c8\ub2e4.\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 PyTorch\uc758 `\ub2e8\uc5b4 \ub2e8\uc704 \uc5b8\uc5b4 \ubaa8\ub378 `_\n\uc608\uc81c\ub97c \ub530\ub77c\ud558\uba74\uc11c, LSTM \uae30\ubc18\uc758 \ub2e8\uc5b4 \uc608\uce21 \ubaa8\ub378\uc5d0 \uac00\uc7a5 \uac04\ub2e8\ud55c \uc591\uc790\ud654 \uae30\ubc95\uc778\n`\ub3d9\uc801 \uc591\uc790\ud654 `_\n\ub97c \uc801\uc6a9\ud574 \ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n" ] }, { @@ -26,14 +26,14 @@ }, "outputs": [], "source": [ - "# imports\nimport os\nfrom io import open\nimport time\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F" + "# \ubd88\ub7ec\uc624\uae30\nimport os\nfrom io import open\nimport time\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "1. Define the model\n-------------------\n\nHere we define the LSTM model architecture, following the\n`model `_\nfrom the word language model example.\n\n" + "1. \ubaa8\ub378 \uc815\uc758\ud558\uae30\n-------------------\n\n\ub2e8\uc5b4 \ub2e8\uc704 \uc5b8\uc5b4 \ubaa8\ub378 \uc608\uc81c\uc5d0\uc11c \uc0ac\uc6a9\ub41c `\ubaa8\ub378 `_ \uc744\n\ub530\ub77c LSTM \ubaa8\ub378 \uc544\ud0a4\ud14d\ucc98\ub97c \uc815\uc758\ud569\ub2c8\ub2e4.\n\n" ] }, { @@ -44,14 +44,14 @@ }, "outputs": [], "source": [ - "class LSTMModel(nn.Module):\n \"\"\"Container module with an encoder, a recurrent module, and a decoder.\"\"\"\n\n def __init__(self, ntoken, ninp, nhid, nlayers, dropout=0.5):\n super(LSTMModel, self).__init__()\n self.drop = nn.Dropout(dropout)\n self.encoder = nn.Embedding(ntoken, ninp)\n self.rnn = nn.LSTM(ninp, nhid, nlayers, dropout=dropout)\n self.decoder = nn.Linear(nhid, ntoken)\n\n self.init_weights()\n\n self.nhid = nhid\n self.nlayers = nlayers\n\n def init_weights(self):\n initrange = 0.1\n self.encoder.weight.data.uniform_(-initrange, initrange)\n self.decoder.bias.data.zero_()\n self.decoder.weight.data.uniform_(-initrange, initrange)\n\n def forward(self, input, hidden):\n emb = self.drop(self.encoder(input))\n output, hidden = self.rnn(emb, hidden)\n output = self.drop(output)\n decoded = self.decoder(output)\n return decoded, hidden\n\n def init_hidden(self, bsz):\n weight = next(self.parameters())\n return (weight.new_zeros(self.nlayers, bsz, self.nhid),\n weight.new_zeros(self.nlayers, bsz, self.nhid))" + "class LSTMModel(nn.Module):\n \"\"\"\uc778\ucf54\ub354, \ubc18\ubcf5 \ubaa8\ub4c8 \ubc0f \ub514\ucf54\ub354\uac00 \uc788\ub294 \ucee8\ud14c\uc774\ub108 \ubaa8\ub4c8.\"\"\"\n\n def __init__(self, ntoken, ninp, nhid, nlayers, dropout=0.5):\n super(LSTMModel, self).__init__()\n self.drop = nn.Dropout(dropout)\n self.encoder = nn.Embedding(ntoken, ninp)\n self.rnn = nn.LSTM(ninp, nhid, nlayers, dropout=dropout)\n self.decoder = nn.Linear(nhid, ntoken)\n\n self.init_weights()\n\n self.nhid = nhid\n self.nlayers = nlayers\n\n def init_weights(self):\n initrange = 0.1\n self.encoder.weight.data.uniform_(-initrange, initrange)\n self.decoder.bias.data.zero_()\n self.decoder.weight.data.uniform_(-initrange, initrange)\n\n def forward(self, input, hidden):\n emb = self.drop(self.encoder(input))\n output, hidden = self.rnn(emb, hidden)\n output = self.drop(output)\n decoded = self.decoder(output)\n return decoded, hidden\n\n def init_hidden(self, bsz):\n weight = next(self.parameters())\n return (weight.new_zeros(self.nlayers, bsz, self.nhid),\n weight.new_zeros(self.nlayers, bsz, self.nhid))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "2. Load in the text data\n------------------------\n\nNext, we load the\n`Wikitext-2 dataset `_ into a `Corpus`,\nagain following the\n`preprocessing `_\nfrom the word language model example.\n\n" + "2. \ud14d\uc2a4\ud2b8 \ub370\uc774\ud130 \ubd88\ub7ec\uc624\uae30\n------------------------\n\n\ub2e4\uc74c\uc73c\ub85c, \ub2e8\uc5b4 \ub2e8\uc704 \uc5b8\uc5b4 \ubaa8\ub378 \uc608\uc81c\uc758 `\uc804\ucc98\ub9ac `_\n\uacfc\uc815\uc744 \ub530\ub77c `Wikitext-2 \ub370\uc774\ud130\uc14b `_ \uc744 `Corpus` \uc778\uc2a4\ud134\uc2a4\uc5d0 \ubd88\ub7ec\uc635\ub2c8\ub2e4.\n\n" ] }, { @@ -62,14 +62,14 @@ }, "outputs": [], "source": [ - "class Dictionary(object):\n def __init__(self):\n self.word2idx = {}\n self.idx2word = []\n\n def add_word(self, word):\n if word not in self.word2idx:\n self.idx2word.append(word)\n self.word2idx[word] = len(self.idx2word) - 1\n return self.word2idx[word]\n\n def __len__(self):\n return len(self.idx2word)\n\n\nclass Corpus(object):\n def __init__(self, path):\n self.dictionary = Dictionary()\n self.train = self.tokenize(os.path.join(path, 'train.txt'))\n self.valid = self.tokenize(os.path.join(path, 'valid.txt'))\n self.test = self.tokenize(os.path.join(path, 'test.txt'))\n\n def tokenize(self, path):\n \"\"\"Tokenizes a text file.\"\"\"\n assert os.path.exists(path)\n # Add words to the dictionary\n with open(path, 'r', encoding=\"utf8\") as f:\n for line in f:\n words = line.split() + ['']\n for word in words:\n self.dictionary.add_word(word)\n\n # Tokenize file content\n with open(path, 'r', encoding=\"utf8\") as f:\n idss = []\n for line in f:\n words = line.split() + ['']\n ids = []\n for word in words:\n ids.append(self.dictionary.word2idx[word])\n idss.append(torch.tensor(ids).type(torch.int64))\n ids = torch.cat(idss)\n\n return ids\n\nmodel_data_filepath = 'data/'\n\ncorpus = Corpus(model_data_filepath + 'wikitext-2')" + "class Dictionary(object):\n def __init__(self):\n self.word2idx = {}\n self.idx2word = []\n\n def add_word(self, word):\n if word not in self.word2idx:\n self.idx2word.append(word)\n self.word2idx[word] = len(self.idx2word) - 1\n return self.word2idx[word]\n\n def __len__(self):\n return len(self.idx2word)\n\n\nclass Corpus(object):\n def __init__(self, path):\n self.dictionary = Dictionary()\n self.train = self.tokenize(os.path.join(path, 'train.txt'))\n self.valid = self.tokenize(os.path.join(path, 'valid.txt'))\n self.test = self.tokenize(os.path.join(path, 'test.txt'))\n\n def tokenize(self, path):\n assert os.path.exists(path)\n \"\"\"\ud14d\uc2a4\ud2b8 \ud30c\uc77c \ud1a0\ud070\ud654\"\"\"\n assert os.path.exists(path)\n # \uc0ac\uc804\uc5d0 \ub2e8\uc5b4 \ucd94\uac00\n with open(path, 'r', encoding=\"utf8\") as f:\n for line in f:\n words = line.split() + ['']\n for word in words:\n self.dictionary.add_word(word)\n\n # \ud30c\uc77c \ub0b4\uc6a9 \ud1a0\ud070\ud654\n with open(path, 'r', encoding=\"utf8\") as f:\n idss = []\n for line in f:\n words = line.split() + ['']\n ids = []\n for word in words:\n ids.append(self.dictionary.word2idx[word])\n idss.append(torch.tensor(ids).type(torch.int64))\n ids = torch.cat(idss)\n\n return ids\n\nmodel_data_filepath = 'data/'\n\ncorpus = Corpus(model_data_filepath + 'wikitext-2')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "3. Load the pre-trained model\n-----------------------------\n\nThis is a tutorial on dynamic quantization, a quantization technique\nthat is applied after a model has been trained. Therefore, we'll simply load some\npre-trained weights into this model architecture; these weights were obtained\nby training for five epochs using the default settings in the word language model\nexample.\n\n" + "3. \uc0ac\uc804 \ud559\uc2b5\ub41c \ubaa8\ub378 \ubd88\ub7ec\uc624\uae30\n-----------------------------\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc740 \ubaa8\ub378\uc774 \ud559\uc2b5\ub41c \ud6c4 \uc801\uc6a9\ub418\ub294 \uc591\uc790\ud654 \uae30\uc220\uc778 \ub3d9\uc801 \uc591\uc790\ud654\uc5d0 \ub300\ud55c \ud29c\ud1a0\ub9ac\uc5bc\uc785\ub2c8\ub2e4.\n\ub530\ub77c\uc11c \uc6b0\ub9ac\ub294 \ubbf8\ub9ac \ud559\uc2b5\ub41c \uac00\uc911\uce58\ub97c \ubaa8\ub378 \uc544\ud0a4\ud14d\ucc98\uc5d0 \ub85c\ub4dc\ud560 \uac83 \uc785\ub2c8\ub2e4. \uc774 \uac00\uc911\uce58\ub294 word \nlanguage \ubaa8\ub378 \uc608\uc81c\uc758 \uae30\ubcf8 \uc124\uc815\uc744 \uc0ac\uc6a9\ud558\uc5ec 5\uac1c\uc758 epoch \ub3d9\uc548 \ud559\uc2b5\ud558\uc5ec \uc5bb\uc740 \uac83\uc785\ub2c8\ub2e4.\n\n\n" ] }, { @@ -87,7 +87,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now let's generate some text to ensure that the pre-trained model is working\nproperly - similarly to before, we follow\n`here `_\n\n" + "\uc774\uc81c \uc0ac\uc804 \ud559\uc2b5\ub41c \ubaa8\ub378\uc774 \uc798 \ub3d9\uc791\ud558\ub294\uc9c0 \ud655\uc778\ud574\ubcf4\uae30 \uc704\ud574 \ud14d\uc2a4\ud2b8\ub97c \uc0dd\uc131\ud574 \ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\uc9c0\uae08\uae4c\uc9c0 \ud29c\ud1a0\ub9ac\uc5bc\uc744 \uc9c4\ud589\ud588\ub358 \ubc29\uc2dd\ucc98\ub7fc `\uc774 \uc608\uc81c `_ \ub97c \ub530\ub77c \ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n" ] }, { @@ -98,14 +98,14 @@ }, "outputs": [], "source": [ - "input_ = torch.randint(ntokens, (1, 1), dtype=torch.long)\nhidden = model.init_hidden(1)\ntemperature = 1.0\nnum_words = 1000\n\nwith open(model_data_filepath + 'out.txt', 'w') as outf:\n with torch.no_grad(): # no tracking history\n for i in range(num_words):\n output, hidden = model(input_, hidden)\n word_weights = output.squeeze().div(temperature).exp().cpu()\n word_idx = torch.multinomial(word_weights, 1)[0]\n input_.fill_(word_idx)\n\n word = corpus.dictionary.idx2word[word_idx]\n\n outf.write(str(word.encode('utf-8')) + ('\\n' if i % 20 == 19 else ' '))\n\n if i % 100 == 0:\n print('| Generated {}/{} words'.format(i, 1000))\n\nwith open(model_data_filepath + 'out.txt', 'r') as outf:\n all_output = outf.read()\n print(all_output)" + "input_ = torch.randint(ntokens, (1, 1), dtype=torch.long)\nhidden = model.init_hidden(1)\ntemperature = 1.0\nnum_words = 1000\n\nwith open(model_data_filepath + 'out.txt', 'w') as outf:\n with torch.no_grad(): # \uae30\ub85d\uc744 \ucd94\uc801\ud558\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4.\n for i in range(num_words):\n output, hidden = model(input_, hidden)\n word_weights = output.squeeze().div(temperature).exp().cpu()\n word_idx = torch.multinomial(word_weights, 1)[0]\n input_.fill_(word_idx)\n\n word = corpus.dictionary.idx2word[word_idx]\n\n outf.write(str(word.encode('utf-8')) + ('\\n' if i % 20 == 19 else ' '))\n\n if i % 100 == 0:\n print('| Generated {}/{} words'.format(i, 1000))\n\nwith open(model_data_filepath + 'out.txt', 'r') as outf:\n all_output = outf.read()\n print(all_output)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "It's no GPT-2, but it looks like the model has started to learn the structure of\nlanguage!\n\nWe're almost ready to demonstrate dynamic quantization. We just need to define a few more\nhelper functions:\n\n" + "\uc774 \ubaa8\ub378\uc774 GPT-2\ub294 \uc544\ub2c8\uc9c0\ub9cc, \uc5b8\uc5b4\uc758 \uad6c\uc870\ub97c \ubc30\uc6b0\uae30 \uc2dc\uc791\ud55c \uac83\ucc98\ub7fc \ubcf4\uc785\ub2c8\ub2e4!\n\n\n\ub3d9\uc801 \uc591\uc790\ud654\ub97c \uc2dc\uc5f0\ud560 \uc900\ube44\uac00 \uac70\uc758 \ub05d\ub0ac\uc2b5\ub2c8\ub2e4. \uba87 \uac00\uc9c0 helper \ud568\uc218\ub97c \uc815\uc758\ud558\uae30\ub9cc \ud558\uba74 \ub429\ub2c8\ub2e4:\n\n" ] }, { @@ -116,14 +116,14 @@ }, "outputs": [], "source": [ - "bptt = 25\ncriterion = nn.CrossEntropyLoss()\neval_batch_size = 1\n\n# create test data set\ndef batchify(data, bsz):\n # Work out how cleanly we can divide the dataset into bsz parts.\n nbatch = data.size(0) // bsz\n # Trim off any extra elements that wouldn't cleanly fit (remainders).\n data = data.narrow(0, 0, nbatch * bsz)\n # Evenly divide the data across the bsz batches.\n return data.view(bsz, -1).t().contiguous()\n\ntest_data = batchify(corpus.test, eval_batch_size)\n\n# Evaluation functions\ndef get_batch(source, i):\n seq_len = min(bptt, len(source) - 1 - i)\n data = source[i:i+seq_len]\n target = source[i+1:i+1+seq_len].view(-1)\n return data, target\n\ndef repackage_hidden(h):\n \"\"\"Wraps hidden states in new Tensors, to detach them from their history.\"\"\"\n\n if isinstance(h, torch.Tensor):\n return h.detach()\n else:\n return tuple(repackage_hidden(v) for v in h)\n\ndef evaluate(model_, data_source):\n # Turn on evaluation mode which disables dropout.\n model_.eval()\n total_loss = 0.\n hidden = model_.init_hidden(eval_batch_size)\n with torch.no_grad():\n for i in range(0, data_source.size(0) - 1, bptt):\n data, targets = get_batch(data_source, i)\n output, hidden = model_(data, hidden)\n hidden = repackage_hidden(hidden)\n output_flat = output.view(-1, ntokens)\n total_loss += len(data) * criterion(output_flat, targets).item()\n return total_loss / (len(data_source) - 1)" + "bptt = 25\ncriterion = nn.CrossEntropyLoss()\neval_batch_size = 1\n\n# \ud14c\uc2a4\ud2b8 \ub370\uc774\ud130\uc14b \ub9cc\ub4e4\uae30\ndef batchify(data, bsz):\n # \ub370\uc774\ud130\uc14b\uc744 bsz \ubd80\ubd84\uc73c\ub85c \uc5bc\ub9c8\ub098 \uae54\ub054\ud558\uac8c \ub098\ub20c \uc218 \uc788\ub294\uc9c0 \uacc4\uc0b0\ud569\ub2c8\ub2e4.\n nbatch = data.size(0) // bsz\n # \uae54\ub054\ud558\uac8c \ub9de\uc9c0 \uc54a\ub294 \ucd94\uac00\uc801\uc778 \ubd80\ubd84(\ub098\uba38\uc9c0\ub4e4)\uc744 \uc798\ub77c\ub0c5\ub2c8\ub2e4.\n data = data.narrow(0, 0, nbatch * bsz)\n # \ub370\uc774\ud130\uc5d0 \ub300\ud558\uc5ec bsz \ubc30\uce58\ub4e4\ub85c \ub3d9\ub4f1\ud558\uac8c \ub098\ub215\ub2c8\ub2e4.\n return data.view(bsz, -1).t().contiguous()\n\ntest_data = batchify(corpus.test, eval_batch_size)\n\n# \ud3c9\uac00 \ud568\uc218\ub4e4\ndef get_batch(source, i):\n seq_len = min(bptt, len(source) - 1 - i)\n data = source[i:i+seq_len]\n target = source[i+1:i+1+seq_len].view(-1)\n return data, target\n\ndef repackage_hidden(h):\n \"\"\"\uc740\ub2c9 \uc0c1\ud0dc\ub97c \ubcc0\ud654\ub3c4 \uae30\ub85d\uc5d0\uc11c \uc81c\uac70\ub41c \uc0c8\ub85c\uc6b4 tensor\ub85c \ub9cc\ub4ed\ub2c8\ub2e4.\"\"\"\n\n if isinstance(h, torch.Tensor):\n return h.detach()\n else:\n return tuple(repackage_hidden(v) for v in h)\n\ndef evaluate(model_, data_source):\n # Dropout\uc744 \uc911\uc9c0\uc2dc\ud0a4\ub294 \ud3c9\uac00 \ubaa8\ub4dc\ub85c \uc2e4\ud589\ud569\ub2c8\ub2e4.\n model_.eval()\n total_loss = 0.\n hidden = model_.init_hidden(eval_batch_size)\n with torch.no_grad():\n for i in range(0, data_source.size(0) - 1, bptt):\n data, targets = get_batch(data_source, i)\n output, hidden = model_(data, hidden)\n hidden = repackage_hidden(hidden)\n output_flat = output.view(-1, ntokens)\n total_loss += len(data) * criterion(output_flat, targets).item()\n return total_loss / (len(data_source) - 1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "4. Test dynamic quantization\n----------------------------\n\nFinally, we can call ``torch.quantization.quantize_dynamic`` on the model!\nSpecifically,\n\n- We specify that we want the ``nn.LSTM`` and ``nn.Linear`` modules in our\n model to be quantized\n- We specify that we want weights to be converted to ``int8`` values\n\n" + "4. \ub3d9\uc801 \uc591\uc790\ud654 \ud14c\uc2a4\ud2b8\ud558\uae30\n----------------------------\n\n\ub9c8\uc9c0\ub9c9\uc73c\ub85c \ubaa8\ub378\uc5d0\uc11c ``torch.quantization.quantize_dynamic`` \uc744 \ud638\ucd9c \ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4!\n\uad6c\uccb4\uc801\uc73c\ub85c,\n\n- \ubaa8\ub378\uc758 ``nn.LSTM`` \uacfc ``nn.Linear`` \ubaa8\ub4c8\uc744 \uc591\uc790\ud654 \ud558\ub3c4\ub85d \uba85\uc2dc\ud569\ub2c8\ub2e4.\n- \uac00\uc911\uce58\ub4e4\uc774 ``int8`` \uac12\uc73c\ub85c \ubcc0\ud658\ub418\ub3c4\ub85d \uba85\uc2dc\ud569\ub2c8\ub2e4.\n\n" ] }, { @@ -141,7 +141,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The model looks the same; how has this benefited us? First, we see a\nsignificant reduction in model size:\n\n" + "\ubaa8\ub378\uc740 \ub3d9\uc77c\ud558\uac8c \ubcf4\uc785\ub2c8\ub2e4. \uc774\uac83\uc774 \uc5b4\ub5bb\uac8c \uc774\ub4dd\uc744 \uc8fc\ub294 \uac83\uc77c\uae4c\uc694? \uccab\uc9f8, \ubaa8\ub378 \ud06c\uae30\uac00\n\uc0c1\ub2f9\ud788 \uc904\uc5b4 \ub4ed\ub2c8\ub2e4:\n\n" ] }, { @@ -159,7 +159,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Second, we see faster inference time, with no difference in evaluation loss:\n\nNote: we number of threads to one for single threaded comparison, since quantized\nmodels run single threaded.\n\n" + "\ub450 \ubc88\uc9f8\ub85c, \ud3c9\uac00 \uc190\uc2e4\uac12\uc740 \uac19\uc73c\ub098 \ucd94\ub860(inference) \uc18d\ub3c4\uac00 \ube68\ub77c\uc84c\uc2b5\ub2c8\ub2e4.\n\n" ] }, { @@ -170,14 +170,14 @@ }, "outputs": [], "source": [ - "torch.set_num_threads(1)\n\ndef time_model_evaluation(model, test_data):\n s = time.time()\n loss = evaluate(model, test_data)\n elapsed = time.time() - s\n print('''loss: {0:.3f}\\nelapsed time (seconds): {1:.1f}'''.format(loss, elapsed))\n\ntime_model_evaluation(model, test_data)\ntime_model_evaluation(quantized_model, test_data)" + "# \uba54\ubaa8: \uc591\uc790\ud654 \ub41c \ubaa8\ub378\uc740 \ub2e8\uc77c \uc2a4\ub808\ub4dc\ub85c \uc2e4\ud589\ub418\uae30 \ub54c\ubb38\uc5d0 \ub2e8\uc77c \uc2a4\ub808\ub4dc \ube44\uad50\ub97c \uc704\ud574\n# \uc2a4\ub808\ub4dc \uc218\ub97c 1\ub85c \uc124\uc815\ud588\uc2b5\ub2c8\ub2e4.\n\ntorch.set_num_threads(1)\n\ndef time_model_evaluation(model, test_data):\n s = time.time()\n loss = evaluate(model, test_data)\n elapsed = time.time() - s\n print('''loss: {0:.3f}\\nelapsed time (seconds): {1:.1f}'''.format(loss, elapsed))\n\ntime_model_evaluation(model, test_data)\ntime_model_evaluation(quantized_model, test_data)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Running this locally on a MacBook Pro, without quantization, inference takes about 200 seconds,\nand with quantization it takes just about 100 seconds.\n\nConclusion\n----------\n\nDynamic quantization can be an easy way to reduce model size while only\nhaving a limited effect on accuracy.\n\nThanks for reading! As always, we welcome any feedback, so please create an issue\n`here `_ if you have any.\n\n" + "MacBook Pro\uc5d0\uc11c \ub85c\uceec\ub85c \uc2e4\ud589\ud558\ub294 \uacbd\uc6b0, \uc591\uc790\ud654 \uc5c6\uc774\ub294 \ucd94\ub860(inference)\uc5d0 \uc57d 200\ucd08\uac00 \uac78\ub9ac\uace0 \n\uc591\uc790\ud654\ub97c \uc0ac\uc6a9\ud558\uba74 \uc57d 100\ucd08\uac00 \uac78\ub9bd\ub2c8\ub2e4.\n\n\ub9c8\uce58\uba70\n----------\n\n\ub3d9\uc801 \uc591\uc790\ud654\ub294 \uc815\ud655\ub3c4\uc5d0 \uc81c\ud55c\uc801\uc778 \uc601\ud5a5\uc744 \ubbf8\uce58\uba74\uc11c \ubaa8\ub378 \ud06c\uae30\ub97c \uc904\uc774\ub294\n\uc26c\uc6b4 \ubc29\ubc95\uc774 \ub420 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc77d\uc5b4\uc8fc\uc154\uc11c \uac10\uc0ac\ud569\ub2c8\ub2e4. \uc5b8\uc81c\ub098\ucc98\ub7fc \uc5b4\ub5a0\ud55c \ud53c\ub4dc\ubc31\ub3c4 \ud658\uc601\uc774\ub2c8, \uc758\uacac\uc774 \uc788\ub2e4\uba74\n`\uc5ec\uae30 `_ \uc5d0 \uc774\uc288\ub97c \ub0a8\uaca8 \uc8fc\uc138\uc694.\n\n" ] } ], @@ -197,7 +197,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/9a7fec783882a14243d1253bf1335d36/warmstarting_model_using_parameters_from_a_different_model.ipynb b/docs/_downloads/9a7fec783882a14243d1253bf1335d36/warmstarting_model_using_parameters_from_a_different_model.ipynb index 3e24d821a..45f69f96b 100644 --- a/docs/_downloads/9a7fec783882a14243d1253bf1335d36/warmstarting_model_using_parameters_from_a_different_model.ipynb +++ b/docs/_downloads/9a7fec783882a14243d1253bf1335d36/warmstarting_model_using_parameters_from_a_different_model.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nWarmstarting model using parameters from a different model in PyTorch\n=====================================================================\nPartially loading a model or loading a partial model are common\nscenarios when transfer learning or training a new complex model.\nLeveraging trained parameters, even if only a few are usable, will help\nto warmstart the training process and hopefully help your model converge\nmuch faster than training from scratch.\n\nIntroduction\n------------\nWhether you are loading from a partial ``state_dict``, which is missing\nsome keys, or loading a ``state_dict`` with more keys than the model\nthat you are loading into, you can set the strict argument to ``False``\nin the ``load_state_dict()`` function to ignore non-matching keys.\nIn this recipe, we will experiment with warmstarting a model using\nparameters of a different model.\n\nSetup\n-----\nBefore we begin, we need to install ``torch`` if it isn\u2019t already\navailable.\n\n::\n\n pip install torch\n \n\n" + "\nWarmstarting model using parameters from a different model in PyTorch\n=====================================================================\nPartially loading a model or loading a partial model are common\nscenarios when transfer learning or training a new complex model.\nLeveraging trained parameters, even if only a few are usable, will help\nto warmstart the training process and hopefully help your model converge\nmuch faster than training from scratch.\n\nIntroduction\n------------\nWhether you are loading from a partial ``state_dict``, which is missing\nsome keys, or loading a ``state_dict`` with more keys than the model\nthat you are loading into, you can set the strict argument to ``False``\nin the ``load_state_dict()`` function to ignore non-matching keys.\nIn this recipe, we will experiment with warmstarting a model using\nparameters of a different model.\n\nSetup\n-----\nBefore we begin, we need to install ``torch`` if it isn\u2019t already\navailable.\n\n::\n\n pip install torch\n \n" ] }, { @@ -114,7 +114,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/9b89023ea3fb5bf2511a9c08a4311cce/saving_multiple_models_in_one_file.ipynb b/docs/_downloads/9b89023ea3fb5bf2511a9c08a4311cce/saving_multiple_models_in_one_file.ipynb index 97c6dcffb..9a56153c9 100644 --- a/docs/_downloads/9b89023ea3fb5bf2511a9c08a4311cce/saving_multiple_models_in_one_file.ipynb +++ b/docs/_downloads/9b89023ea3fb5bf2511a9c08a4311cce/saving_multiple_models_in_one_file.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nPyTorch\uc5d0\uc11c \uc5ec\ub7ec \ubaa8\ub378\uc744 \ud558\ub098\uc758 \ud30c\uc77c\uc5d0 \uc800\uc7a5\ud558\uae30 & \ubd88\ub7ec\uc624\uae30\n============================================================\n\uc5ec\ub7ec \ubaa8\ub378\uc744 \uc800\uc7a5\ud558\uace0 \ubd88\ub7ec\uc624\ub294 \uac83\uc740 \uc774\uc804\uc5d0 \ud559\uc2b5\ud588\ub358 \ubaa8\ub378\ub4e4\uc744 \uc7ac\uc0ac\uc6a9\ud558\ub294\ub370 \ub3c4\uc6c0\uc774 \ub429\ub2c8\ub2e4.\n\n\uac1c\uc694\n------------\nGAN\uc774\ub098 \uc2dc\ud000\uc2a4-\ud22c-\uc2dc\ud000\uc2a4(sequence-to-sequence model), \uc559\uc0c1\ube14 \ubaa8\ub378(ensemble of models)\uacfc\n\uac19\uc774 \uc5ec\ub7ec ``torch.nn.Modules`` \ub85c \uad6c\uc131\ub41c \ubaa8\ub378\uc744 \uc800\uc7a5\ud560 \ub54c\ub294 \uac01 \ubaa8\ub378\uc758 state_dict\uc640\n\ud574\ub2f9 \uc635\ud2f0\ub9c8\uc774\uc800(optimizer)\uc758 \uc0ac\uc804\uc744 \uc800\uc7a5\ud574\uc57c \ud569\ub2c8\ub2e4. \ub610\ud55c, \ud559\uc2b5 \ud559\uc2b5\uc744 \uc7ac\uac1c\ud558\ub294\ub370\n\ud544\uc694\ud55c \ub2e4\ub978 \ud56d\ubaa9\ub4e4\uc744 \uc0ac\uc804\uc5d0 \ucd94\uac00\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \ubaa8\ub378\ub4e4\uc744 \ubd88\ub7ec\uc62c \ub54c\uc5d0\ub294, \uba3c\uc800\n\ubaa8\ub378\ub4e4\uacfc \uc635\ud2f0\ub9c8\uc774\uc800\ub97c \ucd08\uae30\ud654\ud558\uace0, ``torch.load()`` \ub97c \uc0ac\uc6a9\ud558\uc5ec \uc0ac\uc804\uc744 \ubd88\ub7ec\uc635\ub2c8\ub2e4.\n\uc774\ud6c4 \uc6d0\ud558\ub294\ub300\ub85c \uc800\uc7a5\ud55c \ud56d\ubaa9\ub4e4\uc744 \uc0ac\uc804\uc5d0 \uc870\ud68c\ud558\uc5ec \uc811\uadfc\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uc774 \ub808\uc2dc\ud53c\uc5d0\uc11c\ub294 PyTorch\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc5ec\ub7ec \ubaa8\ub378\ub4e4\uc744 \ud558\ub098\uc758 \ud30c\uc77c\uc5d0 \uc5b4\ub5bb\uac8c \uc800\uc7a5\ud558\uace0\n\ubd88\ub7ec\uc624\ub294\uc9c0 \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc124\uc815\n---------\n\uc2dc\uc791\ud558\uae30 \uc804\uc5d0 ``torch`` \uac00 \uc5c6\ub2e4\uba74 \uc124\uce58\ud574\uc57c \ud569\ub2c8\ub2e4.\n\n::\n\n pip install torch\n\n\n" + "\nPyTorch\uc5d0\uc11c \uc5ec\ub7ec \ubaa8\ub378\uc744 \ud558\ub098\uc758 \ud30c\uc77c\uc5d0 \uc800\uc7a5\ud558\uae30 & \ubd88\ub7ec\uc624\uae30\n============================================================\n\uc5ec\ub7ec \ubaa8\ub378\uc744 \uc800\uc7a5\ud558\uace0 \ubd88\ub7ec\uc624\ub294 \uac83\uc740 \uc774\uc804\uc5d0 \ud559\uc2b5\ud588\ub358 \ubaa8\ub378\ub4e4\uc744 \uc7ac\uc0ac\uc6a9\ud558\ub294\ub370 \ub3c4\uc6c0\uc774 \ub429\ub2c8\ub2e4.\n\n\uac1c\uc694\n------------\nGAN\uc774\ub098 \uc2dc\ud000\uc2a4-\ud22c-\uc2dc\ud000\uc2a4(sequence-to-sequence model), \uc559\uc0c1\ube14 \ubaa8\ub378(ensemble of models)\uacfc\n\uac19\uc774 \uc5ec\ub7ec ``torch.nn.Modules`` \ub85c \uad6c\uc131\ub41c \ubaa8\ub378\uc744 \uc800\uc7a5\ud560 \ub54c\ub294 \uac01 \ubaa8\ub378\uc758 state_dict\uc640\n\ud574\ub2f9 \uc635\ud2f0\ub9c8\uc774\uc800(optimizer)\uc758 \uc0ac\uc804\uc744 \uc800\uc7a5\ud574\uc57c \ud569\ub2c8\ub2e4. \ub610\ud55c, \ud559\uc2b5 \ud559\uc2b5\uc744 \uc7ac\uac1c\ud558\ub294\ub370\n\ud544\uc694\ud55c \ub2e4\ub978 \ud56d\ubaa9\ub4e4\uc744 \uc0ac\uc804\uc5d0 \ucd94\uac00\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \ubaa8\ub378\ub4e4\uc744 \ubd88\ub7ec\uc62c \ub54c\uc5d0\ub294, \uba3c\uc800\n\ubaa8\ub378\ub4e4\uacfc \uc635\ud2f0\ub9c8\uc774\uc800\ub97c \ucd08\uae30\ud654\ud558\uace0, ``torch.load()`` \ub97c \uc0ac\uc6a9\ud558\uc5ec \uc0ac\uc804\uc744 \ubd88\ub7ec\uc635\ub2c8\ub2e4.\n\uc774\ud6c4 \uc6d0\ud558\ub294\ub300\ub85c \uc800\uc7a5\ud55c \ud56d\ubaa9\ub4e4\uc744 \uc0ac\uc804\uc5d0 \uc870\ud68c\ud558\uc5ec \uc811\uadfc\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uc774 \ub808\uc2dc\ud53c\uc5d0\uc11c\ub294 PyTorch\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc5ec\ub7ec \ubaa8\ub378\ub4e4\uc744 \ud558\ub098\uc758 \ud30c\uc77c\uc5d0 \uc5b4\ub5bb\uac8c \uc800\uc7a5\ud558\uace0\n\ubd88\ub7ec\uc624\ub294\uc9c0 \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc124\uc815\n---------\n\uc2dc\uc791\ud558\uae30 \uc804\uc5d0 ``torch`` \uac00 \uc5c6\ub2e4\uba74 \uc124\uce58\ud574\uc57c \ud569\ub2c8\ub2e4.\n\n::\n\n pip install torch\n" ] }, { @@ -143,7 +143,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/9cf2d4ead514e661e20d2070c9bf7324/transformer_tutorial.ipynb b/docs/_downloads/9cf2d4ead514e661e20d2070c9bf7324/transformer_tutorial.ipynb index bf784604b..459b02d27 100644 --- a/docs/_downloads/9cf2d4ead514e661e20d2070c9bf7324/transformer_tutorial.ipynb +++ b/docs/_downloads/9cf2d4ead514e661e20d2070c9bf7324/transformer_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nnn.Transformer \uc640 TorchText \ub85c \uc2dc\ud000\uc2a4-\ud22c-\uc2dc\ud000\uc2a4(Sequence-to-Sequence) \ubaa8\ub378\ub9c1\ud558\uae30\n==========================================================================\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294\n`nn.Transformer `__ \ubaa8\ub4c8\uc744\n\uc774\uc6a9\ud558\ub294 \uc2dc\ud000\uc2a4-\ud22c-\uc2dc\ud000\uc2a4(Sequence-to-Sequence) \ubaa8\ub378\uc744 \ud559\uc2b5\ud558\ub294 \ubc29\ubc95\uc744 \ubc30\uc6cc\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\nPyTorch 1.2 \ubc84\uc83c\uc5d0\ub294\n`Attention is All You Need `__ \ub17c\ubb38\uc5d0\n\uae30\ubc18\ud55c \ud45c\uc900 \ud2b8\ub79c\uc2a4\ud3ec\uba38(transformer) \ubaa8\ub4c8\uc744 \ud3ec\ud568\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n\ud2b8\ub79c\uc2a4\ud3ec\uba38 \ubaa8\ub378\uc740 \ub354 \ub192\uc740 \uc218\uc900\uc758 \ubcd1\ub82c\ud654(parallelization)\uac00 \uac00\ub2a5\ud558\uba74\uc11c\ub3c4\n\ub9ce\uc740 \uc2dc\ud000\uc2a4-\ud22c-\uc2dc\ud000\uc2a4 \ubb38\uc81c\ub4e4\uc5d0\uc11c \ud488\uc9c8\uc774 \uc6b0\uc218\ud568\uc774 \uc785\uc99d\ub418\uc5c8\uc2b5\ub2c8\ub2e4.\n``nn.Transformer`` \ubaa8\ub4c8\uc740 \uc785\ub825(input) \uacfc \ucd9c\ub825(output) \uc0ac\uc774\uc758 \uc804\uc5ed\uc801\uc778 \uc758\uc874\uc131(global dependencies)\n\uc744 \ub098\ud0c0\ub0b4\uae30 \uc704\ud558\uc5ec \uc804\uc801\uc73c\ub85c \uc5b4\ud150\uc158(attention) \uba54\ucee4\ub2c8\uc998\uc5d0 \uc758\uc874\ud569\ub2c8\ub2e4.\n(\ucd5c\uadfc\uc5d0 \ub610 \ub2e4\ub978 \ubaa8\ub4c8\uc774 `nn.MultiheadAttention `__ \uc73c\ub85c \uad6c\ud604\ub418\uc5c8\uc2b5\ub2c8\ub2e4.)\n``nn.Transformer`` \ubaa8\ub4c8\uc740 \ud604\uc7ac \ubaa8\ub4c8\ud654\uac00 \ub9e4\uc6b0 \uc798 \ub418\uc5b4 \uc788\uc5b4,\n\ub2e4\uc74c\uacfc \uac19\uc740 \ub2e8\uc77c \ucef4\ud3ec\ub10c\ud2b8 (\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc758 `nn.TransformerEncoder `__ \uc640 \uac19\uc740)\n\ub294 \uc27d\uac8c \uc801\uc6a9 \ubc0f \uad6c\uc131\ub420 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n![](../_static/img/transformer_architecture.jpg)\n\n\n\n" + "\nnn.Transformer \uc640 TorchText \ub85c \uc2dc\ud000\uc2a4-\ud22c-\uc2dc\ud000\uc2a4(Sequence-to-Sequence) \ubaa8\ub378\ub9c1\ud558\uae30\n=================================================================================\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294\n`nn.Transformer `__ \ubaa8\ub4c8\uc744\n\uc774\uc6a9\ud558\ub294 \uc2dc\ud000\uc2a4-\ud22c-\uc2dc\ud000\uc2a4(Sequence-to-Sequence) \ubaa8\ub378\uc744 \ud559\uc2b5\ud558\ub294 \ubc29\ubc95\uc744 \ubc30\uc6cc\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\nPyTorch 1.2 \ubc84\uc83c\uc5d0\ub294\n`Attention is All You Need `__ \ub17c\ubb38\uc5d0\n\uae30\ubc18\ud55c \ud45c\uc900 \ud2b8\ub79c\uc2a4\ud3ec\uba38(transformer) \ubaa8\ub4c8\uc744 \ud3ec\ud568\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n\ud2b8\ub79c\uc2a4\ud3ec\uba38 \ubaa8\ub378\uc740 \ub354 \ub192\uc740 \uc218\uc900\uc758 \ubcd1\ub82c\ud654(parallelization)\uac00 \uac00\ub2a5\ud558\uba74\uc11c\ub3c4\n\ub9ce\uc740 \uc2dc\ud000\uc2a4-\ud22c-\uc2dc\ud000\uc2a4 \ubb38\uc81c\ub4e4\uc5d0\uc11c \ud488\uc9c8\uc774 \uc6b0\uc218\ud568\uc774 \uc785\uc99d\ub418\uc5c8\uc2b5\ub2c8\ub2e4.\n``nn.Transformer`` \ubaa8\ub4c8\uc740 \uc785\ub825(input) \uacfc \ucd9c\ub825(output) \uc0ac\uc774\uc758 \uc804\uc5ed\uc801\uc778 \uc758\uc874\uc131(global dependencies)\n\uc744 \ub098\ud0c0\ub0b4\uae30 \uc704\ud558\uc5ec \uc804\uc801\uc73c\ub85c \uc5b4\ud150\uc158(attention) \uba54\ucee4\ub2c8\uc998\uc5d0 \uc758\uc874\ud569\ub2c8\ub2e4.\n(\ucd5c\uadfc\uc5d0 \ub610 \ub2e4\ub978 \ubaa8\ub4c8\uc774 `nn.MultiheadAttention `__ \uc73c\ub85c \uad6c\ud604\ub418\uc5c8\uc2b5\ub2c8\ub2e4.)\n``nn.Transformer`` \ubaa8\ub4c8\uc740 \ud604\uc7ac \ubaa8\ub4c8\ud654\uac00 \ub9e4\uc6b0 \uc798 \ub418\uc5b4 \uc788\uc5b4,\n\ub2e4\uc74c\uacfc \uac19\uc740 \ub2e8\uc77c \ucef4\ud3ec\ub10c\ud2b8 (\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc758 `nn.TransformerEncoder `__ \uc640 \uac19\uc740)\n\ub294 \uc27d\uac8c \uc801\uc6a9 \ubc0f \uad6c\uc131\ub420 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n![](../_static/img/transformer_architecture.jpg)\n\n" ] }, { @@ -65,7 +65,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\ub370\uc774\ud130 \ub85c\ub4dc\ud558\uace0 \ubc30\uce58 \ub9cc\ub4e4\uae30\n-------------------\n\n\n" + "\ub370\uc774\ud130 \ub85c\ub4dc\ud558\uace0 \ubc30\uce58 \ub9cc\ub4e4\uae30\n-----------------------------\n\n\n" ] }, { @@ -90,7 +90,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uc785\ub825(input) \uacfc \ud0c0\uac9f(target) \uc2dc\ud000\uc2a4\ub97c \uc0dd\uc131\ud558\uae30 \uc704\ud55c \ud568\uc218\ub4e4\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n\n" + "\uc785\ub825(input) \uacfc \ud0c0\uac9f(target) \uc2dc\ud000\uc2a4\ub97c \uc0dd\uc131\ud558\uae30 \uc704\ud55c \ud568\uc218\ub4e4\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n\n" ] }, { @@ -115,7 +115,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uc778\uc2a4\ud134\uc2a4(instance) \ucd08\uae30\ud654\ud558\uae30\n--------------------\n\n\n" + "\uc778\uc2a4\ud134\uc2a4(instance) \ucd08\uae30\ud654\ud558\uae30\n--------------------------------\n\n\n" ] }, { @@ -183,7 +183,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\ud3c9\uac00 \ub370\uc774\ud130\uc14b(test dataset) \uc73c\ub85c \ubaa8\ub378\uc744 \ud3c9\uac00\ud558\uae30\n-------------------------------------\n\n\ud3c9\uac00 \ub370\uc774\ud130\uc14b\uc5d0 \ub300\ud55c \uacb0\uacfc\ub97c \ud655\uc778\ud558\uae30 \uc704\ud574\uc11c \ucd5c\uace0\uc758 \ubaa8\ub378\uc744 \uc801\uc6a9\ud569\ub2c8\ub2e4.\n\n" + "\ud3c9\uac00 \ub370\uc774\ud130\uc14b(test dataset)\uc73c\ub85c \ubaa8\ub378\uc744 \ud3c9\uac00\ud558\uae30\n-------------------------------------------------\n\n\ud3c9\uac00 \ub370\uc774\ud130\uc14b\uc5d0 \ub300\ud55c \uacb0\uacfc\ub97c \ud655\uc778\ud558\uae30 \uc704\ud574\uc11c \ucd5c\uace0\uc758 \ubaa8\ub378\uc744 \uc801\uc6a9\ud569\ub2c8\ub2e4.\n\n" ] }, { @@ -214,7 +214,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/9da0471a9eeb2351a488cd4b44fc6bbf/reinforcement_q_learning.ipynb b/docs/_downloads/9da0471a9eeb2351a488cd4b44fc6bbf/reinforcement_q_learning.ipynb index 7bcda26b6..cc37362b0 100644 --- a/docs/_downloads/9da0471a9eeb2351a488cd4b44fc6bbf/reinforcement_q_learning.ipynb +++ b/docs/_downloads/9da0471a9eeb2351a488cd4b44fc6bbf/reinforcement_q_learning.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n\uac15\ud654 \ud559\uc2b5 (DQN) \ud29c\ud1a0\ub9ac\uc5bc\n=====================================\n**Author**: `Adam Paszke `_\n **\ubc88\uc5ed**: `\ud669\uc131\uc218 `_\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 `OpenAI Gym `__ \uc758\nCartPole-v0 \ud0dc\uc2a4\ud06c\uc5d0\uc11c DQN (Deep Q Learning) \uc5d0\uc774\uc804\ud2b8\ub97c \ud559\uc2b5\ud558\ub294\ub370\nPyTorch\ub97c \uc0ac\uc6a9\ud558\ub294 \ubc29\ubc95\uc744 \ubcf4\uc5ec\ub4dc\ub9bd\ub2c8\ub2e4.\n\n**\ud0dc\uc2a4\ud06c**\n\n\uc5d0\uc774\uc804\ud2b8\ub294 \uc5f0\uacb0\ub41c \ub9c9\ub300\uac00 \ub611\ubc14\ub85c \uc11c \uc788\ub3c4\ub85d \uce74\ud2b8\ub97c \uc67c\ucabd\uc774\ub098 \uc624\ub978\ucabd\uc73c\ub85c\n\uc6c0\uc9c1\uc774\ub294 \ub450 \uac00\uc9c0 \ub3d9\uc791 \uc911 \ud558\ub098\ub97c \uc120\ud0dd\ud574\uc57c \ud569\ub2c8\ub2e4.\n\ub2e4\uc591\ud55c \uc54c\uace0\ub9ac\uc998\uacfc \uc2dc\uac01\ud654 \uae30\ub2a5\uc744 \uac16\ucd98 \uacf5\uc2dd \uc21c\uc704\ud45c\ub97c\n`Gym \uc6f9\uc0ac\uc774\ud2b8 `__ \uc5d0\uc11c \ucc3e\uc744 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n.. figure:: /_static/img/cartpole.gif\n :alt: cartpole\n\n cartpole\n\n\uc5d0\uc774\uc804\ud2b8\uac00 \ud604\uc7ac \ud658\uacbd \uc0c1\ud0dc\ub97c \uad00\ucc30\ud558\uace0 \ud589\ub3d9\uc744 \uc120\ud0dd\ud558\uba74,\n\ud658\uacbd\uc774 \uc0c8\ub85c\uc6b4 \uc0c1\ud0dc\ub85c *\uc804\ud658* \ub418\uace0 \uc791\uc5c5\uc758 \uacb0\uacfc\ub97c \ub098\ud0c0\ub0b4\ub294 \ubcf4\uc0c1\ub3c4 \ubc18\ud658\ub429\ub2c8\ub2e4.\n\uc774 \ud0dc\uc2a4\ud06c\uc5d0\uc11c \ub9e4 \ud0c0\uc784\uc2a4\ud15d \uc99d\uac00\ub9c8\ub2e4 \ubcf4\uc0c1\uc774 +1\uc774 \ub418\uace0, \ub9cc\uc57d \ub9c9\ub300\uac00 \ub108\ubb34 \uba40\ub9ac\n\ub5a8\uc5b4\uc9c0\uac70\ub098 \uce74\ud2b8\uac00 \uc911\uc2ec\uc5d0\uc11c 2.4 \uc720\ub2db \uc774\uc0c1 \uba40\uc5b4\uc9c0\uba74 \ud658\uacbd\uc774 \uc911\ub2e8\ub429\ub2c8\ub2e4.\n\uc774\uac83\uc740 \ub354 \uc88b\uc740 \uc2dc\ub098\ub9ac\uc624\uac00 \ub354 \uc624\ub7ab\ub3d9\uc548 \ub354 \ub9ce\uc740 \ubcf4\uc0c1\uc744 \ucd95\uc801\ud558\ub294 \uac83\uc744 \uc758\ubbf8\ud569\ub2c8\ub2e4.\n\n\uce74\ud2b8\ud3f4 \ud0dc\uc2a4\ud06c\ub294 \uc5d0\uc774\uc804\ud2b8\uc5d0 \ub300\ud55c \uc785\ub825\uc774 \ud658\uacbd \uc0c1\ud0dc(\uc704\uce58, \uc18d\ub3c4 \ub4f1)\ub97c \ub098\ud0c0\ub0b4\ub294\n4\uac1c\uc758 \uc2e4\uc81c \uac12\uc774 \ub418\ub3c4\ub85d \uc124\uacc4\ub418\uc5c8\uc2b5\ub2c8\ub2e4. \uadf8\ub7ec\ub098 \uc2e0\uacbd\ub9dd\uc740 \uc21c\uc218\ud558\uac8c \uadf8 \uc7a5\uba74\uc744 \ubcf4\uace0\n\ud0dc\uc2a4\ud06c\ub97c \ud574\uacb0\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4 \ub530\ub77c\uc11c \uce74\ud2b8 \uc911\uc2ec\uc758 \ud654\uba74 \ud328\uce58\ub97c \uc785\ub825\uc73c\ub85c \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n\uc774 \ub54c\ubb38\uc5d0 \uc6b0\ub9ac\uc758 \uacb0\uacfc\ub294 \uacf5\uc2dd \uc21c\uc704\ud45c\uc758 \uacb0\uacfc\uc640 \uc9c1\uc811\uc801\uc73c\ub85c \ube44\uad50\ud560 \uc218 \uc5c6\uc2b5\ub2c8\ub2e4.\n\uc6b0\ub9ac\uc758 \ud0dc\uc2a4\ud06c\ub294 \ud6e8\uc52c \ub354 \uc5b4\ub835\uc2b5\ub2c8\ub2e4.\n\ubd88\ud589\ud788\ub3c4 \ubaa8\ub4e0 \ud504\ub808\uc784\uc744 \ub80c\ub354\ub9c1\ud574\uc57c\ub418\ubbc0\ub85c \uc774\uac83\uc740 \ud559\uc2b5 \uc18d\ub3c4\ub97c \ub2a6\ucd94\uac8c\ub429\ub2c8\ub2e4.\n\n\uc5c4\ubc00\ud788 \ub9d0\ud558\uba74, \ud604\uc7ac \uc2a4\ud06c\ub9b0 \ud328\uce58\uc640 \uc774\uc804 \uc2a4\ud06c\ub9b0 \ud328\uce58 \uc0ac\uc774\uc758 \ucc28\uc774\ub85c \uc0c1\ud0dc\ub97c \ud45c\uc2dc\ud560 \uac83\uc785\ub2c8\ub2e4.\n\uc774\ub807\uac8c\ud558\uba74 \uc5d0\uc774\uc804\ud2b8\uac00 \ub9c9\ub300\uc758 \uc18d\ub3c4\ub97c \ud55c \uc774\ubbf8\uc9c0\uc5d0\uc11c \uace0\ub824\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n**\ud328\ud0a4\uc9c0**\n\n\uba3c\uc800 \ud544\uc694\ud55c \ud328\ud0a4\uc9c0\ub97c \uac00\uc838\uc635\ub2c8\ub2e4. \uccab\uc9f8, \ud658\uacbd\uc744 \uc704\ud574\n`gym `__ \uc774 \ud544\uc694\ud569\ub2c8\ub2e4.\n(`pip install gym` \uc744 \uc0ac\uc6a9\ud558\uc5ec \uc124\uce58\ud558\uc2ed\uc2dc\uc624).\n\ub610\ud55c PyTorch\uc5d0\uc11c \ub2e4\uc74c\uc744 \uc0ac\uc6a9\ud569\ub2c8\ub2e4:\n\n- \uc2e0\uacbd\ub9dd (``torch.nn``)\n- \ucd5c\uc801\ud654 (``torch.optim``)\n- \uc790\ub3d9 \ubbf8\ubd84 (``torch.autograd``)\n- \uc2dc\uac01 \ud0dc\uc2a4\ud06c\ub97c \uc704\ud55c \uc720\ud2f8\ub9ac\ud2f0\ub4e4 (``torchvision`` - `a separate\n package `__).\n\n\n" + "\n\uac15\ud654 \ud559\uc2b5 (DQN) \ud29c\ud1a0\ub9ac\uc5bc\n=====================================\n**Author**: `Adam Paszke `_\n **\ubc88\uc5ed**: `\ud669\uc131\uc218 `_\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 `OpenAI Gym `__ \uc758\nCartPole-v0 \ud0dc\uc2a4\ud06c\uc5d0\uc11c DQN (Deep Q Learning) \uc5d0\uc774\uc804\ud2b8\ub97c \ud559\uc2b5\ud558\ub294\ub370\nPyTorch\ub97c \uc0ac\uc6a9\ud558\ub294 \ubc29\ubc95\uc744 \ubcf4\uc5ec\ub4dc\ub9bd\ub2c8\ub2e4.\n\n**\ud0dc\uc2a4\ud06c**\n\n\uc5d0\uc774\uc804\ud2b8\ub294 \uc5f0\uacb0\ub41c \ub9c9\ub300\uac00 \ub611\ubc14\ub85c \uc11c \uc788\ub3c4\ub85d \uce74\ud2b8\ub97c \uc67c\ucabd\uc774\ub098 \uc624\ub978\ucabd\uc73c\ub85c\n\uc6c0\uc9c1\uc774\ub294 \ub450 \uac00\uc9c0 \ub3d9\uc791 \uc911 \ud558\ub098\ub97c \uc120\ud0dd\ud574\uc57c \ud569\ub2c8\ub2e4.\n\ub2e4\uc591\ud55c \uc54c\uace0\ub9ac\uc998\uacfc \uc2dc\uac01\ud654 \uae30\ub2a5\uc744 \uac16\ucd98 \uacf5\uc2dd \uc21c\uc704\ud45c\ub97c\n`Gym \uc6f9\uc0ac\uc774\ud2b8 `__ \uc5d0\uc11c \ucc3e\uc744 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n.. figure:: /_static/img/cartpole.gif\n :alt: cartpole\n\n cartpole\n\n\uc5d0\uc774\uc804\ud2b8\uac00 \ud604\uc7ac \ud658\uacbd \uc0c1\ud0dc\ub97c \uad00\ucc30\ud558\uace0 \ud589\ub3d9\uc744 \uc120\ud0dd\ud558\uba74,\n\ud658\uacbd\uc774 \uc0c8\ub85c\uc6b4 \uc0c1\ud0dc\ub85c *\uc804\ud658* \ub418\uace0 \uc791\uc5c5\uc758 \uacb0\uacfc\ub97c \ub098\ud0c0\ub0b4\ub294 \ubcf4\uc0c1\ub3c4 \ubc18\ud658\ub429\ub2c8\ub2e4.\n\uc774 \ud0dc\uc2a4\ud06c\uc5d0\uc11c \ub9e4 \ud0c0\uc784\uc2a4\ud15d \uc99d\uac00\ub9c8\ub2e4 \ubcf4\uc0c1\uc774 +1\uc774 \ub418\uace0, \ub9cc\uc57d \ub9c9\ub300\uac00 \ub108\ubb34 \uba40\ub9ac\n\ub5a8\uc5b4\uc9c0\uac70\ub098 \uce74\ud2b8\uac00 \uc911\uc2ec\uc5d0\uc11c 2.4 \uc720\ub2db \uc774\uc0c1 \uba40\uc5b4\uc9c0\uba74 \ud658\uacbd\uc774 \uc911\ub2e8\ub429\ub2c8\ub2e4.\n\uc774\uac83\uc740 \ub354 \uc88b\uc740 \uc2dc\ub098\ub9ac\uc624\uac00 \ub354 \uc624\ub7ab\ub3d9\uc548 \ub354 \ub9ce\uc740 \ubcf4\uc0c1\uc744 \ucd95\uc801\ud558\ub294 \uac83\uc744 \uc758\ubbf8\ud569\ub2c8\ub2e4.\n\n\uce74\ud2b8\ud3f4 \ud0dc\uc2a4\ud06c\ub294 \uc5d0\uc774\uc804\ud2b8\uc5d0 \ub300\ud55c \uc785\ub825\uc774 \ud658\uacbd \uc0c1\ud0dc(\uc704\uce58, \uc18d\ub3c4 \ub4f1)\ub97c \ub098\ud0c0\ub0b4\ub294\n4\uac1c\uc758 \uc2e4\uc81c \uac12\uc774 \ub418\ub3c4\ub85d \uc124\uacc4\ub418\uc5c8\uc2b5\ub2c8\ub2e4. \uadf8\ub7ec\ub098 \uc2e0\uacbd\ub9dd\uc740 \uc21c\uc218\ud558\uac8c \uadf8 \uc7a5\uba74\uc744 \ubcf4\uace0\n\ud0dc\uc2a4\ud06c\ub97c \ud574\uacb0\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4 \ub530\ub77c\uc11c \uce74\ud2b8 \uc911\uc2ec\uc758 \ud654\uba74 \ud328\uce58\ub97c \uc785\ub825\uc73c\ub85c \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n\uc774 \ub54c\ubb38\uc5d0 \uc6b0\ub9ac\uc758 \uacb0\uacfc\ub294 \uacf5\uc2dd \uc21c\uc704\ud45c\uc758 \uacb0\uacfc\uc640 \uc9c1\uc811\uc801\uc73c\ub85c \ube44\uad50\ud560 \uc218 \uc5c6\uc2b5\ub2c8\ub2e4.\n\uc6b0\ub9ac\uc758 \ud0dc\uc2a4\ud06c\ub294 \ud6e8\uc52c \ub354 \uc5b4\ub835\uc2b5\ub2c8\ub2e4.\n\ubd88\ud589\ud788\ub3c4 \ubaa8\ub4e0 \ud504\ub808\uc784\uc744 \ub80c\ub354\ub9c1\ud574\uc57c\ub418\ubbc0\ub85c \uc774\uac83\uc740 \ud559\uc2b5 \uc18d\ub3c4\ub97c \ub2a6\ucd94\uac8c\ub429\ub2c8\ub2e4.\n\n\uc5c4\ubc00\ud788 \ub9d0\ud558\uba74, \ud604\uc7ac \uc2a4\ud06c\ub9b0 \ud328\uce58\uc640 \uc774\uc804 \uc2a4\ud06c\ub9b0 \ud328\uce58 \uc0ac\uc774\uc758 \ucc28\uc774\ub85c \uc0c1\ud0dc\ub97c \ud45c\uc2dc\ud560 \uac83\uc785\ub2c8\ub2e4.\n\uc774\ub807\uac8c\ud558\uba74 \uc5d0\uc774\uc804\ud2b8\uac00 \ub9c9\ub300\uc758 \uc18d\ub3c4\ub97c \ud55c \uc774\ubbf8\uc9c0\uc5d0\uc11c \uace0\ub824\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n**\ud328\ud0a4\uc9c0**\n\n\uba3c\uc800 \ud544\uc694\ud55c \ud328\ud0a4\uc9c0\ub97c \uac00\uc838\uc635\ub2c8\ub2e4. \uccab\uc9f8, \ud658\uacbd\uc744 \uc704\ud574\n`gym `__ \uc774 \ud544\uc694\ud569\ub2c8\ub2e4.\n(`pip install gym` \uc744 \uc0ac\uc6a9\ud558\uc5ec \uc124\uce58\ud558\uc2ed\uc2dc\uc624).\n\ub610\ud55c PyTorch\uc5d0\uc11c \ub2e4\uc74c\uc744 \uc0ac\uc6a9\ud569\ub2c8\ub2e4:\n\n- \uc2e0\uacbd\ub9dd (``torch.nn``)\n- \ucd5c\uc801\ud654 (``torch.optim``)\n- \uc790\ub3d9 \ubbf8\ubd84 (``torch.autograd``)\n- \uc2dc\uac01 \ud0dc\uc2a4\ud06c\ub97c \uc704\ud55c \uc720\ud2f8\ub9ac\ud2f0\ub4e4 (``torchvision`` - `a separate\n package `__).\n" ] }, { @@ -161,7 +161,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/9dfafb625d821ccfaa723af12eed7ce4/nnft_tutorial.ipynb b/docs/_downloads/9dfafb625d821ccfaa723af12eed7ce4/nnft_tutorial.ipynb index b5703bfb0..1761dcb29 100644 --- a/docs/_downloads/9dfafb625d821ccfaa723af12eed7ce4/nnft_tutorial.ipynb +++ b/docs/_downloads/9dfafb625d821ccfaa723af12eed7ce4/nnft_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nnn \ud328\ud0a4\uc9c0\n==========\n\nautograd\uc5d0 \uc644\ubcbd\ud788 \ud1b5\ud569\ub418\uac8c \ud558\uae30 \uc704\ud574 nn \ud328\ud0a4\uc9c0\ub97c \uc7ac\uc124\uacc4\ud558\uc600\uc2b5\ub2c8\ub2e4.\n\uc774\uc81c \ubb34\uc5c7\uc774 \ubcc0\uacbd\ub418\uc5c8\ub294\uc9c0 \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n**\ucee8\ud14c\uc774\ub108\ub97c autograd\ub85c \uad50\uccb4:**\n\n \uc774\uc81c \ub354\uc774\uc0c1 ``ConcatTable`` \uac19\uc740 \ucee8\ud14c\uc774\ub108\ub098 ``CAddTable`` \uac19\uc740 \ubaa8\ub4c8, \ub610\ub294\n nngraph\ub97c \uc774\uc6a9\ud558\uac70\ub098 \ub514\ubc84\uae45\ud558\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4. \ub300\uc2e0 autograd\ub97c \uc774\uc6a9\ud558\uc5ec \ub354 \uae54\ub054\ud558\uac8c\n \uc2e0\uacbd\ub9dd\uc744 \uc815\uc758\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4. \uc608\ub97c \ub4e4\uc5b4,\n\n * ``output = nn.CAddTable():forward({input1, input2})`` \ub300\uc2e0,\n ``output = input1 + input2`` \ub97c \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n * ``output = nn.MulConstant(0.5):forward(input)`` \ub300\uc2e0,\n ``output = input * 0.5`` \ub97c \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n\n**\uc0c1\ud0dc(state)\ub294 \ubaa8\ub4c8 \ub0b4\uc5d0 \uc800\uc7a5\ub418\uc9c0 \uc54a\uace0, \uc2e0\uacbd\ub9dd \uadf8\ub798\ud504 \uc0c1\uc5d0 \uc874\uc7ac\ud569\ub2c8\ub2e4:**\n\n \ub355\ubd84\uc5d0 \uc21c\ud658\uc2e0\uacbd\ub9dd\uc744 \uc0ac\uc6a9\ud558\ub294 \ubc29\ubc95\uc774 \ub354 \uac04\ub2e8\ud574\uc84c\uc2b5\ub2c8\ub2e4. \uc774\uc81c \uc21c\ud658\uc2e0\uacbd\ub9dd\uc744 \ub9cc\ub4e4\n \ub54c, \ub354 \uc774\uc0c1 \uac00\uc911\uce58(weight)\ub97c \uacf5\uc720\ud558\ub294 \uac83\uc5d0 \ub300\ud574\uc11c\ub294 \uc0dd\uac01\ud560 \ud544\uc694\uc5c6\uc774 \ub3d9\uc77c\ud55c\n Linear \uacc4\uce35\uc744 \uc5ec\ub7ec \ucc28\ub840 \ud638\ucd9c\ud558\uba74 \ub429\ub2c8\ub2e4.\n\n .. figure:: /_static/img/torch-nn-vs-pytorch-nn.png\n :alt: torch-nn-vs-pytorch-nn\n\n torch-nn-vs-pytorch-nn\n\n**\uac04\uc18c\ud654\ub41c \ub514\ubc84\uae45:**\n\n \ub514\ubc84\uae45\uc740 Python\uc758 pdb \ub514\ubc84\uac70\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc9c1\uad00\uc801\uc774\uba70,\n **\ub514\ubc84\uac70\uc640 \uc2a4\ud0dd \ucd94\uc801(stack trace)\uc740 \uc5d0\ub7ec\uac00 \ubc1c\uc0dd\ud55c \uacf3\uc5d0\uc11c \uc815\ud655\ud788 \uba48\ucda5\ub2c8\ub2e4.**\n \uc774\uc81c \ubcf4\uc774\ub294\ub300\ub85c \uc5bb\uc744 \uac83\uc785\ub2c8\ub2e4. (What you see is what you get.)\n\n\uc608\uc81c1: \ud569\uc131\uacf1 \uc2e0\uacbd\ub9dd(ConvNet)\n-----------------------------\n\n\uc774\uc81c \uc5b4\ub5bb\uac8c \uc791\uc740 \ud569\uc131\uacf1 \uc2e0\uacbd\ub9dd\uc744 \ub9cc\ub4dc\ub294\uc9c0 \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\ubaa8\ub4e0 \uc2e0\uacbd\ub9dd\uc740 \uae30\ubcf8 \ud074\ub798\uc2a4\uc778 ``nn.Module`` \ub85c\ubd80\ud130 \ud30c\uc0dd\ub429\ub2c8\ub2e4:\n\n- \uc0dd\uc131\uc790\uc5d0\uc11c\ub294 \uc0ac\uc6a9\ud560 \ubaa8\ub4e0 \uacc4\uce35\uc744 \uc120\uc5b8\ud569\ub2c8\ub2e4.\n- \uc21c\uc804\ud30c \ud568\uc218\uc5d0\uc11c\ub294 \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc774 \uc785\ub825\uc5d0\uc11c \ucd9c\ub825\uae4c\uc9c0 \uc5b4\ub5bb\uac8c \uc2e4\ud589\ub418\ub294\uc9c0\ub97c \uc815\uc758\ud569\ub2c8\ub2e4.\n\n" + "\nnn \ud328\ud0a4\uc9c0\n==========\n\nautograd\uc5d0 \uc644\ubcbd\ud788 \ud1b5\ud569\ub418\uac8c \ud558\uae30 \uc704\ud574 nn \ud328\ud0a4\uc9c0\ub97c \uc7ac\uc124\uacc4\ud558\uc600\uc2b5\ub2c8\ub2e4.\n\uc774\uc81c \ubb34\uc5c7\uc774 \ubcc0\uacbd\ub418\uc5c8\ub294\uc9c0 \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n**\ucee8\ud14c\uc774\ub108\ub97c autograd\ub85c \uad50\uccb4:**\n\n \uc774\uc81c \ub354\uc774\uc0c1 ``ConcatTable`` \uac19\uc740 \ucee8\ud14c\uc774\ub108\ub098 ``CAddTable`` \uac19\uc740 \ubaa8\ub4c8, \ub610\ub294\n nngraph\ub97c \uc774\uc6a9\ud558\uac70\ub098 \ub514\ubc84\uae45\ud558\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4. \ub300\uc2e0 autograd\ub97c \uc774\uc6a9\ud558\uc5ec \ub354 \uae54\ub054\ud558\uac8c\n \uc2e0\uacbd\ub9dd\uc744 \uc815\uc758\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4. \uc608\ub97c \ub4e4\uc5b4,\n\n * ``output = nn.CAddTable():forward({input1, input2})`` \ub300\uc2e0,\n ``output = input1 + input2`` \ub97c \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n * ``output = nn.MulConstant(0.5):forward(input)`` \ub300\uc2e0,\n ``output = input * 0.5`` \ub97c \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n\n**\uc0c1\ud0dc(state)\ub294 \ubaa8\ub4c8 \ub0b4\uc5d0 \uc800\uc7a5\ub418\uc9c0 \uc54a\uace0, \uc2e0\uacbd\ub9dd \uadf8\ub798\ud504 \uc0c1\uc5d0 \uc874\uc7ac\ud569\ub2c8\ub2e4:**\n\n \ub355\ubd84\uc5d0 \uc21c\ud658\uc2e0\uacbd\ub9dd\uc744 \uc0ac\uc6a9\ud558\ub294 \ubc29\ubc95\uc774 \ub354 \uac04\ub2e8\ud574\uc84c\uc2b5\ub2c8\ub2e4. \uc774\uc81c \uc21c\ud658\uc2e0\uacbd\ub9dd\uc744 \ub9cc\ub4e4\n \ub54c, \ub354 \uc774\uc0c1 \uac00\uc911\uce58(weight)\ub97c \uacf5\uc720\ud558\ub294 \uac83\uc5d0 \ub300\ud574\uc11c\ub294 \uc0dd\uac01\ud560 \ud544\uc694\uc5c6\uc774 \ub3d9\uc77c\ud55c\n Linear \uacc4\uce35\uc744 \uc5ec\ub7ec \ucc28\ub840 \ud638\ucd9c\ud558\uba74 \ub429\ub2c8\ub2e4.\n\n .. figure:: /_static/img/torch-nn-vs-pytorch-nn.png\n :alt: torch-nn-vs-pytorch-nn\n\n torch-nn-vs-pytorch-nn\n\n**\uac04\uc18c\ud654\ub41c \ub514\ubc84\uae45:**\n\n \ub514\ubc84\uae45\uc740 Python\uc758 pdb \ub514\ubc84\uac70\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc9c1\uad00\uc801\uc774\uba70,\n **\ub514\ubc84\uac70\uc640 \uc2a4\ud0dd \ucd94\uc801(stack trace)\uc740 \uc5d0\ub7ec\uac00 \ubc1c\uc0dd\ud55c \uacf3\uc5d0\uc11c \uc815\ud655\ud788 \uba48\ucda5\ub2c8\ub2e4.**\n \uc774\uc81c \ubcf4\uc774\ub294\ub300\ub85c \uc5bb\uc744 \uac83\uc785\ub2c8\ub2e4. (What you see is what you get.)\n\n\uc608\uc81c1: \ud569\uc131\uacf1 \uc2e0\uacbd\ub9dd(ConvNet)\n-----------------------------\n\n\uc774\uc81c \uc5b4\ub5bb\uac8c \uc791\uc740 \ud569\uc131\uacf1 \uc2e0\uacbd\ub9dd\uc744 \ub9cc\ub4dc\ub294\uc9c0 \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\ubaa8\ub4e0 \uc2e0\uacbd\ub9dd\uc740 \uae30\ubcf8 \ud074\ub798\uc2a4\uc778 ``nn.Module`` \ub85c\ubd80\ud130 \ud30c\uc0dd\ub429\ub2c8\ub2e4:\n\n- \uc0dd\uc131\uc790\uc5d0\uc11c\ub294 \uc0ac\uc6a9\ud560 \ubaa8\ub4e0 \uacc4\uce35\uc744 \uc120\uc5b8\ud569\ub2c8\ub2e4.\n- \uc21c\uc804\ud30c \ud568\uc218\uc5d0\uc11c\ub294 \uc2e0\uacbd\ub9dd \ubaa8\ub378\uc774 \uc785\ub825\uc5d0\uc11c \ucd9c\ub825\uae4c\uc9c0 \uc5b4\ub5bb\uac8c \uc2e4\ud589\ub418\ub294\uc9c0\ub97c \uc815\uc758\ud569\ub2c8\ub2e4.\n" ] }, { @@ -201,7 +201,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/a1687058579b7f1d970855ee56beb272/autograd_tutorial_old.ipynb b/docs/_downloads/a1687058579b7f1d970855ee56beb272/autograd_tutorial_old.ipynb index f6b6945d6..1083c2376 100644 --- a/docs/_downloads/a1687058579b7f1d970855ee56beb272/autograd_tutorial_old.ipynb +++ b/docs/_downloads/a1687058579b7f1d970855ee56beb272/autograd_tutorial_old.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nAutograd\n========\n\nAutograd\ub294 \uc790\ub3d9 \ubbf8\ubd84\uc744 \uc218\ud589\ud558\ub294 torch\uc758 \ud575\uc2ec \ud328\ud0a4\uc9c0\ub85c, \uc790\ub3d9 \ubbf8\ubd84\uc744 \uc704\ud574\n\ud14c\uc78e(tape) \uae30\ubc18 \uc2dc\uc2a4\ud15c\uc744 \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n\n\uc21c\uc804\ud30c(forward) \ub2e8\uacc4\uc5d0\uc11c autograd \ud14c\uc78e\uc740 \uc218\ud589\ud558\ub294 \ubaa8\ub4e0 \uc5f0\uc0b0\uc744 \uae30\uc5b5\ud569\ub2c8\ub2e4.\n\uadf8\ub9ac\uace0, \uc5ed\uc804\ud30c(backward) \ub2e8\uacc4\uc5d0\uc11c \uc5f0\uc0b0\ub4e4\uc744 \uc7ac\ud604(replay)\ud569\ub2c8\ub2e4.\n\n\uc5f0\uc0b0 \uae30\ub85d\uc744 \ucd94\uc801\ud558\ub294 Tensor\n----------------------------\n\nAutograd\uc5d0\uc11c ``requires_grad=True`` \ub85c \uc124\uc815\ub41c \uc785\ub825 ``Tensor`` \uc758 \uc5f0\uc0b0\uc740\n\uae30\ub85d\ub429\ub2c8\ub2e4. \uc5ed\uc804\ud30c \ub2e8\uacc4 \uc5f0\uc0b0 \ud6c4\uc5d0, \uc774 Tensor\uc5d0 \ub300\ud55c \ubcc0\ud654\ub3c4(grdient)\ub294 ``.grad``\n\uc18d\uc131\uc5d0 \ub204\uc801\ub429\ub2c8\ub2e4.\n\nAutograd \uad6c\ud604\uc5d0\uc11c \ub9e4\uc6b0 \uc911\uc694\ud55c \ud074\ub798\uc2a4\uac00 \ud558\ub098 \ub354 \uc788\ub294\ub370, \uc774\uac83\uc740 \ubc14\ub85c ``Function``\n\ud074\ub798\uc2a4\uc785\ub2c8\ub2e4. ``Tensor`` \uc640 ``Function`` \uc740 \uc11c\ub85c \uc5f0\uacb0\ub418\uc5b4 \uc788\uc73c\uba70, \ubaa8\ub4e0 \uc5f0\uc0b0 \uacfc\uc815\uc744\n\ubd80\ud638\ud654(encode)\ud558\uc5ec \uc21c\ud658\ud558\uc9c0 \uc54a\ub294 \uadf8\ub798\ud504(acyclic graph)\ub97c \uc0dd\uc131\ud569\ub2c8\ub2e4. \uac01 \ubcc0\uc218\ub294\n``.grad_fn`` \uc18d\uc131\uc744 \uac16\uace0 \uc788\ub294\ub370, \uc774\ub294 ``Tensor`` \ub97c \uc0dd\uc131\ud55c ``Function`` \uc744\n\ucc38\uc870\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4. (\ub2e8, \uc0ac\uc6a9\uc790\uac00 \ub9cc\ub4e0 Tensor\ub294 \uc608\uc678\ub85c, \uc774 \ub54c ``grad_fn`` \uc740\n``None`` \uc785\ub2c8\ub2e4.)\n\n\ub3c4\ud568\uc218\ub97c \uacc4\uc0b0\ud558\uae30 \uc704\ud574\uc11c\ub294 ``Tensor`` \uc758 ``.backward()`` \ub97c \ud638\ucd9c\ud558\uba74 \ub429\ub2c8\ub2e4.\n``Tensor`` \uac00 \uc2a4\uce7c\ub77c(scalar)\uc778 \uacbd\uc6b0(\uc608. \ud558\ub098\uc758 \uc694\uc18c\ub9cc \uac16\ub294 \ub4f1)\uc5d0\ub294, ``backward`` \uc5d0\n\uc778\uc790\ub97c \uc815\ud574\uc904 \ud544\uc694\uac00 \uc5c6\uc2b5\ub2c8\ub2e4. \ud558\uc9c0\ub9cc \uc5ec\ub7ec \uac1c\uc758 \uc694\uc18c\ub97c \uac16\uace0 \uc788\uc744\n\ub54c\ub294 tensor\uc758 \ubaa8\uc591\uc744 ``gradient`` \uc758 \uc778\uc790\ub85c \uc9c0\uc815\ud560 \ud544\uc694\uac00 \uc788\uc2b5\ub2c8\ub2e4.\n\n" + "\nAutograd\n========\n\nAutograd\ub294 \uc790\ub3d9 \ubbf8\ubd84\uc744 \uc218\ud589\ud558\ub294 torch\uc758 \ud575\uc2ec \ud328\ud0a4\uc9c0\ub85c, \uc790\ub3d9 \ubbf8\ubd84\uc744 \uc704\ud574\n\ud14c\uc78e(tape) \uae30\ubc18 \uc2dc\uc2a4\ud15c\uc744 \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n\n\uc21c\uc804\ud30c(forward) \ub2e8\uacc4\uc5d0\uc11c autograd \ud14c\uc78e\uc740 \uc218\ud589\ud558\ub294 \ubaa8\ub4e0 \uc5f0\uc0b0\uc744 \uae30\uc5b5\ud569\ub2c8\ub2e4.\n\uadf8\ub9ac\uace0, \uc5ed\uc804\ud30c(backward) \ub2e8\uacc4\uc5d0\uc11c \uc5f0\uc0b0\ub4e4\uc744 \uc7ac\ud604(replay)\ud569\ub2c8\ub2e4.\n\n\uc5f0\uc0b0 \uae30\ub85d\uc744 \ucd94\uc801\ud558\ub294 Tensor\n----------------------------\n\nAutograd\uc5d0\uc11c ``requires_grad=True`` \ub85c \uc124\uc815\ub41c \uc785\ub825 ``Tensor`` \uc758 \uc5f0\uc0b0\uc740\n\uae30\ub85d\ub429\ub2c8\ub2e4. \uc5ed\uc804\ud30c \ub2e8\uacc4 \uc5f0\uc0b0 \ud6c4\uc5d0, \uc774 Tensor\uc5d0 \ub300\ud55c \ubcc0\ud654\ub3c4(grdient)\ub294 ``.grad``\n\uc18d\uc131\uc5d0 \ub204\uc801\ub429\ub2c8\ub2e4.\n\nAutograd \uad6c\ud604\uc5d0\uc11c \ub9e4\uc6b0 \uc911\uc694\ud55c \ud074\ub798\uc2a4\uac00 \ud558\ub098 \ub354 \uc788\ub294\ub370, \uc774\uac83\uc740 \ubc14\ub85c ``Function``\n\ud074\ub798\uc2a4\uc785\ub2c8\ub2e4. ``Tensor`` \uc640 ``Function`` \uc740 \uc11c\ub85c \uc5f0\uacb0\ub418\uc5b4 \uc788\uc73c\uba70, \ubaa8\ub4e0 \uc5f0\uc0b0 \uacfc\uc815\uc744\n\ubd80\ud638\ud654(encode)\ud558\uc5ec \uc21c\ud658\ud558\uc9c0 \uc54a\ub294 \uadf8\ub798\ud504(acyclic graph)\ub97c \uc0dd\uc131\ud569\ub2c8\ub2e4. \uac01 \ubcc0\uc218\ub294\n``.grad_fn`` \uc18d\uc131\uc744 \uac16\uace0 \uc788\ub294\ub370, \uc774\ub294 ``Tensor`` \ub97c \uc0dd\uc131\ud55c ``Function`` \uc744\n\ucc38\uc870\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4. (\ub2e8, \uc0ac\uc6a9\uc790\uac00 \ub9cc\ub4e0 Tensor\ub294 \uc608\uc678\ub85c, \uc774 \ub54c ``grad_fn`` \uc740\n``None`` \uc785\ub2c8\ub2e4.)\n\n\ub3c4\ud568\uc218\ub97c \uacc4\uc0b0\ud558\uae30 \uc704\ud574\uc11c\ub294 ``Tensor`` \uc758 ``.backward()`` \ub97c \ud638\ucd9c\ud558\uba74 \ub429\ub2c8\ub2e4.\n``Tensor`` \uac00 \uc2a4\uce7c\ub77c(scalar)\uc778 \uacbd\uc6b0(\uc608. \ud558\ub098\uc758 \uc694\uc18c\ub9cc \uac16\ub294 \ub4f1)\uc5d0\ub294, ``backward`` \uc5d0\n\uc778\uc790\ub97c \uc815\ud574\uc904 \ud544\uc694\uac00 \uc5c6\uc2b5\ub2c8\ub2e4. \ud558\uc9c0\ub9cc \uc5ec\ub7ec \uac1c\uc758 \uc694\uc18c\ub97c \uac16\uace0 \uc788\uc744\n\ub54c\ub294 tensor\uc758 \ubaa8\uc591\uc744 ``gradient`` \uc758 \uc778\uc790\ub85c \uc9c0\uc815\ud560 \ud544\uc694\uac00 \uc788\uc2b5\ub2c8\ub2e4.\n" ] }, { @@ -252,7 +252,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/a4f07fecba75b5e84fe9e56cac0c7b71/spatial_transformer_tutorial.py b/docs/_downloads/a4f07fecba75b5e84fe9e56cac0c7b71/spatial_transformer_tutorial.py index 9fa3317aa..f17bd6b61 100644 --- a/docs/_downloads/a4f07fecba75b5e84fe9e56cac0c7b71/spatial_transformer_tutorial.py +++ b/docs/_downloads/a4f07fecba75b5e84fe9e56cac0c7b71/spatial_transformer_tutorial.py @@ -1,31 +1,27 @@ # -*- coding: utf-8 -*- """ -Spatial Transformer Networks Tutorial +공간 변형 네트워크(Spatial Transformer Networks) 튜토리얼 ===================================== -**Author**: `Ghassen HAMROUNI `_ - +**저자**: `Ghassen HAMROUNI `_ +**번역**: `황성수 `_ , `정신유 `_ .. figure:: /_static/img/stn/FSeq.png - -In this tutorial, you will learn how to augment your network using -a visual attention mechanism called spatial transformer -networks. You can read more about the spatial transformer -networks in the `DeepMind paper `__ - -Spatial transformer networks are a generalization of differentiable -attention to any spatial transformation. Spatial transformer networks -(STN for short) allow a neural network to learn how to perform spatial -transformations on the input image in order to enhance the geometric -invariance of the model. -For example, it can crop a region of interest, scale and correct -the orientation of an image. It can be a useful mechanism because CNNs -are not invariant to rotation and scale and more general affine -transformations. - -One of the best things about STN is the ability to simply plug it into -any existing CNN with very little modification. +이 튜토리얼에서는 공간 변형 네트워크(spatial transformer networks, 이하 STN)이라 +불리는 비주얼 어텐션 메커니즘을 이용해 신경망을 증강(augment)시키는 방법에 대해 +학습합니다. 이 방법에 대한 자세한 내용은 `DeepMind paper `__ 에서 +확인할 수 있습니다. +STN은 어떠한 공간적 변형(spatial transformation)에도 적용할 수 있는 미분 가능한 +어텐션의 일반화입니다. 따라서 STN은 신경망의 기하학적 불변성(geometric invariance)을 +강화하기 위해 입력 이미지를 대상으로 어떠한 공간적 변형을 수행해야 하는지 학습하도록 +합니다. +예를 들어 이미지의 관심 영역을 잘라내거나, 크기를 조정하거나, 방향(orientation)을 +수정할 수 있습니다. CNN은 이러한 회전, 크기 조정 등의 일반적인 아핀(affine) 변환된 +입력에 대해 결과의 변동이 크기 때문에 (민감하기 때문에), STN은 이를 극복하는데 매우 +유용한 메커니즘이 될 수 있습니다. +STN이 가진 장점 중 하나는 아주 작은 수정만으로 기존에 사용하던 CNN에 간단하게 연결 시킬 +수 있다는 것입니다. """ -# License: BSD -# Author: Ghassen Hamrouni +# 라이센스: BSD +# 저자: Ghassen Hamrouni from __future__ import print_function import torch @@ -37,26 +33,25 @@ import matplotlib.pyplot as plt import numpy as np -plt.ion() # interactive mode +plt.ion() # 대화형 모드 ###################################################################### -# Loading the data +# 데이터 불러오기 # ---------------- # -# In this post we experiment with the classic MNIST dataset. Using a -# standard convolutional network augmented with a spatial transformer -# network. +# 이 튜토리얼에서는 MNIST 데이터셋을 이용해 실험합니다. 실험에는 STN으로 +# 증강된 일반적인 CNN을 사용합니다. device = torch.device("cuda" if torch.cuda.is_available() else "cpu") -# Training dataset +# 학습용 데이터셋 train_loader = torch.utils.data.DataLoader( datasets.MNIST(root='.', train=True, download=True, transform=transforms.Compose([ transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,)) ])), batch_size=64, shuffle=True, num_workers=4) -# Test dataset +# 테스트용 데이터셋 test_loader = torch.utils.data.DataLoader( datasets.MNIST(root='.', train=False, transform=transforms.Compose([ transforms.ToTensor(), @@ -64,25 +59,22 @@ ])), batch_size=64, shuffle=True, num_workers=4) ###################################################################### -# Depicting spatial transformer networks +# Spatial Transformer Networks(STN) 구성하기 # -------------------------------------- # -# Spatial transformer networks boils down to three main components : +# STN은 다음의 세 가지 주요 구성 요소로 요약됩니다. # -# - The localization network is a regular CNN which regresses the -# transformation parameters. The transformation is never learned -# explicitly from this dataset, instead the network learns automatically -# the spatial transformations that enhances the global accuracy. -# - The grid generator generates a grid of coordinates in the input -# image corresponding to each pixel from the output image. -# - The sampler uses the parameters of the transformation and applies -# it to the input image. +# - 위치 결정 네트워크(localization network)는 공간 변환 파라미터를 예측(regress) +# 하는 일반적인 CNN 입니다. 공간 변환은 데이터 셋으로부터 명시적으로 학습되지 않고, +# 신경망이 전체 정확도를 향상 시키도록 공간 변환을 자동으로 학습합니다. +# - 그리드 생성기(grid generator)는 출력 이미지로부터 각 픽셀에 대응하는 입력 +# 이미지 내 좌표 그리드를 생성합니다. +# - 샘플러(sampler)는 공간 변환 파라미터를 입력 이미지에 적용합니다. # # .. figure:: /_static/img/stn/stn-arch.png # # .. Note:: -# We need the latest version of PyTorch that contains -# affine_grid and grid_sample modules. +# affine_grid 및 grid_sample 모듈이 포함된 최신 버전의 PyTorch가 필요합니다. # @@ -95,7 +87,7 @@ def __init__(self): self.fc1 = nn.Linear(320, 50) self.fc2 = nn.Linear(50, 10) - # Spatial transformer localization-network + # 공간 변환을 위한 위치 결정 네트워크 (localization-network) self.localization = nn.Sequential( nn.Conv2d(1, 8, kernel_size=7), nn.MaxPool2d(2, stride=2), @@ -105,18 +97,18 @@ def __init__(self): nn.ReLU(True) ) - # Regressor for the 3 * 2 affine matrix + # [3 * 2] 크기의 아핀(affine) 행렬에 대해 예측 self.fc_loc = nn.Sequential( nn.Linear(10 * 3 * 3, 32), nn.ReLU(True), nn.Linear(32, 3 * 2) ) - # Initialize the weights/bias with identity transformation + # 항등 변환(identity transformation)으로 가중치/바이어스 초기화 self.fc_loc[2].weight.data.zero_() self.fc_loc[2].bias.data.copy_(torch.tensor([1, 0, 0, 0, 1, 0], dtype=torch.float)) - # Spatial transformer network forward function + # STN의 forward 함수 def stn(self, x): xs = self.localization(x) xs = xs.view(-1, 10 * 3 * 3) @@ -129,10 +121,10 @@ def stn(self, x): return x def forward(self, x): - # transform the input + # 입력을 변환 x = self.stn(x) - # Perform the usual forward pass + # 일반적인 forward pass를 수행 x = F.relu(F.max_pool2d(self.conv1(x), 2)) x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2)) x = x.view(-1, 320) @@ -145,12 +137,12 @@ def forward(self, x): model = Net().to(device) ###################################################################### -# Training the model +# 모델 학습하기 # ------------------ # -# Now, let's use the SGD algorithm to train the model. The network is -# learning the classification task in a supervised way. In the same time -# the model is learning STN automatically in an end-to-end fashion. +# 이제 SGD 알고리즘을 이용해 모델을 학습시켜 봅시다. 앞서 구성한 신경망은 +# 감독 학습 방식(supervised way)으로 분류 문제를 학습합니다. 또한 이 모델은 +# end-to-end 방식으로 STN을 자동으로 학습합니다. optimizer = optim.SGD(model.parameters(), lr=0.01) @@ -171,7 +163,7 @@ def train(epoch): epoch, batch_idx * len(data), len(train_loader.dataset), 100. * batch_idx / len(train_loader), loss.item())) # -# A simple test procedure to measure STN the performances on MNIST. +# MNIST 데이터셋에서 STN의 성능을 측정하기 위한 간단한 테스트 절차 # @@ -184,9 +176,9 @@ def test(): data, target = data.to(device), target.to(device) output = model(data) - # sum up batch loss + # 배치 손실 합하기 test_loss += F.nll_loss(output, target, size_average=False).item() - # get the index of the max log-probability + # 로그-확률의 최대값에 해당하는 인덱스 가져오기 pred = output.max(1, keepdim=True)[1] correct += pred.eq(target.view_as(pred)).sum().item() @@ -196,14 +188,12 @@ def test(): 100. * correct / len(test_loader.dataset))) ###################################################################### -# Visualizing the STN results +# STN 결과 시각화하기 # --------------------------- # -# Now, we will inspect the results of our learned visual attention -# mechanism. +# 이제 학습한 비주얼 어텐션 메커니즘의 결과를 살펴보겠습니다. # -# We define a small helper function in order to visualize the -# transformations while training. +# 학습하는 동안 변환된 결과를 시각화하기 위해 작은 도움(helper) 함수를 정의합니다. def convert_image_np(inp): @@ -215,14 +205,13 @@ def convert_image_np(inp): inp = np.clip(inp, 0, 1) return inp -# We want to visualize the output of the spatial transformers layer -# after the training, we visualize a batch of input images and -# the corresponding transformed batch using STN. +# 학습 후 공간 변환 계층의 출력을 시각화하고, 입력 이미지 배치 데이터 및 +# STN을 사용해 변환된 배치 데이터를 시각화 합니다. def visualize_stn(): with torch.no_grad(): - # Get a batch of training data + # 학습 데이터의 배치 가져오기 data = next(iter(test_loader))[0].to(device) input_tensor = data.cpu() @@ -234,7 +223,7 @@ def visualize_stn(): out_grid = convert_image_np( torchvision.utils.make_grid(transformed_input_tensor)) - # Plot the results side-by-side + # 결과를 나란히 표시하기 f, axarr = plt.subplots(1, 2) axarr[0].imshow(in_grid) axarr[0].set_title('Dataset Images') @@ -246,8 +235,8 @@ def visualize_stn(): train(epoch) test() -# Visualize the STN transformation on some input batch +# 일부 입력 배치 데이터에서 STN 변환 결과를 시각화 visualize_stn() plt.ioff() -plt.show() +plt.show() \ No newline at end of file diff --git a/docs/_downloads/a5513958454950ed22df8da4c47f6429/spatial_transformer_tutorial.ipynb b/docs/_downloads/a5513958454950ed22df8da4c47f6429/spatial_transformer_tutorial.ipynb index 9a9b699cb..9ad8df15a 100644 --- a/docs/_downloads/a5513958454950ed22df8da4c47f6429/spatial_transformer_tutorial.ipynb +++ b/docs/_downloads/a5513958454950ed22df8da4c47f6429/spatial_transformer_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nSpatial Transformer Networks Tutorial\n=====================================\n**Author**: `Ghassen HAMROUNI `_\n\n.. figure:: /_static/img/stn/FSeq.png\n\nIn this tutorial, you will learn how to augment your network using\na visual attention mechanism called spatial transformer\nnetworks. You can read more about the spatial transformer\nnetworks in the `DeepMind paper `__\n\nSpatial transformer networks are a generalization of differentiable\nattention to any spatial transformation. Spatial transformer networks\n(STN for short) allow a neural network to learn how to perform spatial\ntransformations on the input image in order to enhance the geometric\ninvariance of the model.\nFor example, it can crop a region of interest, scale and correct\nthe orientation of an image. It can be a useful mechanism because CNNs\nare not invariant to rotation and scale and more general affine\ntransformations.\n\nOne of the best things about STN is the ability to simply plug it into\nany existing CNN with very little modification.\n\n" + "\n\uacf5\uac04 \ubcc0\ud615 \ub124\ud2b8\uc6cc\ud06c(Spatial Transformer Networks) \ud29c\ud1a0\ub9ac\uc5bc\n=====================================\n**\uc800\uc790**: `Ghassen HAMROUNI `_ \n**\ubc88\uc5ed**: `\ud669\uc131\uc218 `_ , `\uc815\uc2e0\uc720 `_\n.. figure:: /_static/img/stn/FSeq.png\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 \uacf5\uac04 \ubcc0\ud615 \ub124\ud2b8\uc6cc\ud06c(spatial transformer networks, \uc774\ud558 STN)\uc774\ub77c \n\ubd88\ub9ac\ub294 \ube44\uc8fc\uc5bc \uc5b4\ud150\uc158 \uba54\ucee4\ub2c8\uc998\uc744 \uc774\uc6a9\ud574 \uc2e0\uacbd\ub9dd\uc744 \uc99d\uac15(augment)\uc2dc\ud0a4\ub294 \ubc29\ubc95\uc5d0 \ub300\ud574 \n\ud559\uc2b5\ud569\ub2c8\ub2e4. \uc774 \ubc29\ubc95\uc5d0 \ub300\ud55c \uc790\uc138\ud55c \ub0b4\uc6a9\uc740 `DeepMind paper `__ \uc5d0\uc11c\n\ud655\uc778\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\nSTN\uc740 \uc5b4\ub5a0\ud55c \uacf5\uac04\uc801 \ubcc0\ud615(spatial transformation)\uc5d0\ub3c4 \uc801\uc6a9\ud560 \uc218 \uc788\ub294 \ubbf8\ubd84 \uac00\ub2a5\ud55c \n\uc5b4\ud150\uc158\uc758 \uc77c\ubc18\ud654\uc785\ub2c8\ub2e4. \ub530\ub77c\uc11c STN\uc740 \uc2e0\uacbd\ub9dd\uc758 \uae30\ud558\ud559\uc801 \ubd88\ubcc0\uc131(geometric invariance)\uc744\n\uac15\ud654\ud558\uae30 \uc704\ud574 \uc785\ub825 \uc774\ubbf8\uc9c0\ub97c \ub300\uc0c1\uc73c\ub85c \uc5b4\ub5a0\ud55c \uacf5\uac04\uc801 \ubcc0\ud615\uc744 \uc218\ud589\ud574\uc57c \ud558\ub294\uc9c0 \ud559\uc2b5\ud558\ub3c4\ub85d\n\ud569\ub2c8\ub2e4.\n\uc608\ub97c \ub4e4\uc5b4 \uc774\ubbf8\uc9c0\uc758 \uad00\uc2ec \uc601\uc5ed\uc744 \uc798\ub77c\ub0b4\uac70\ub098, \ud06c\uae30\ub97c \uc870\uc815\ud558\uac70\ub098, \ubc29\ud5a5(orientation)\uc744 \n\uc218\uc815\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. CNN\uc740 \uc774\ub7ec\ud55c \ud68c\uc804, \ud06c\uae30 \uc870\uc815 \ub4f1\uc758 \uc77c\ubc18\uc801\uc778 \uc544\ud540(affine) \ubcc0\ud658\ub41c \n\uc785\ub825\uc5d0 \ub300\ud574 \uacb0\uacfc\uc758 \ubcc0\ub3d9\uc774 \ud06c\uae30 \ub54c\ubb38\uc5d0 (\ubbfc\uac10\ud558\uae30 \ub54c\ubb38\uc5d0), STN\uc740 \uc774\ub97c \uadf9\ubcf5\ud558\ub294\ub370 \ub9e4\uc6b0\n\uc720\uc6a9\ud55c \uba54\ucee4\ub2c8\uc998\uc774 \ub420 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\nSTN\uc774 \uac00\uc9c4 \uc7a5\uc810 \uc911 \ud558\ub098\ub294 \uc544\uc8fc \uc791\uc740 \uc218\uc815\ub9cc\uc73c\ub85c \uae30\uc874\uc5d0 \uc0ac\uc6a9\ud558\ub358 CNN\uc5d0 \uac04\ub2e8\ud558\uac8c \uc5f0\uacb0 \uc2dc\ud0ac \n\uc218 \uc788\ub2e4\ub294 \uac83\uc785\ub2c8\ub2e4.\n" ] }, { @@ -26,14 +26,14 @@ }, "outputs": [], "source": [ - "# License: BSD\n# Author: Ghassen Hamrouni\n\nfrom __future__ import print_function\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.optim as optim\nimport torchvision\nfrom torchvision import datasets, transforms\nimport matplotlib.pyplot as plt\nimport numpy as np\n\nplt.ion() # interactive mode" + "# \ub77c\uc774\uc13c\uc2a4: BSD\n# \uc800\uc790: Ghassen Hamrouni\n\nfrom __future__ import print_function\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.optim as optim\nimport torchvision\nfrom torchvision import datasets, transforms\nimport matplotlib.pyplot as plt\nimport numpy as np\n\nplt.ion() # \ub300\ud654\ud615 \ubaa8\ub4dc" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Loading the data\n----------------\n\nIn this post we experiment with the classic MNIST dataset. Using a\nstandard convolutional network augmented with a spatial transformer\nnetwork.\n\n" + "\ub370\uc774\ud130 \ubd88\ub7ec\uc624\uae30\n----------------\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 MNIST \ub370\uc774\ud130\uc14b\uc744 \uc774\uc6a9\ud574 \uc2e4\ud5d8\ud569\ub2c8\ub2e4. \uc2e4\ud5d8\uc5d0\ub294 STN\uc73c\ub85c \n\uc99d\uac15\ub41c \uc77c\ubc18\uc801\uc778 CNN\uc744 \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n\n" ] }, { @@ -44,14 +44,14 @@ }, "outputs": [], "source": [ - "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n# Training dataset\ntrain_loader = torch.utils.data.DataLoader(\n datasets.MNIST(root='.', train=True, download=True,\n transform=transforms.Compose([\n transforms.ToTensor(),\n transforms.Normalize((0.1307,), (0.3081,))\n ])), batch_size=64, shuffle=True, num_workers=4)\n# Test dataset\ntest_loader = torch.utils.data.DataLoader(\n datasets.MNIST(root='.', train=False, transform=transforms.Compose([\n transforms.ToTensor(),\n transforms.Normalize((0.1307,), (0.3081,))\n ])), batch_size=64, shuffle=True, num_workers=4)" + "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n# \ud559\uc2b5\uc6a9 \ub370\uc774\ud130\uc14b\ntrain_loader = torch.utils.data.DataLoader(\n datasets.MNIST(root='.', train=True, download=True,\n transform=transforms.Compose([\n transforms.ToTensor(),\n transforms.Normalize((0.1307,), (0.3081,))\n ])), batch_size=64, shuffle=True, num_workers=4)\n# \ud14c\uc2a4\ud2b8\uc6a9 \ub370\uc774\ud130\uc14b\ntest_loader = torch.utils.data.DataLoader(\n datasets.MNIST(root='.', train=False, transform=transforms.Compose([\n transforms.ToTensor(),\n transforms.Normalize((0.1307,), (0.3081,))\n ])), batch_size=64, shuffle=True, num_workers=4)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Depicting spatial transformer networks\n--------------------------------------\n\nSpatial transformer networks boils down to three main components :\n\n- The localization network is a regular CNN which regresses the\n transformation parameters. The transformation is never learned\n explicitly from this dataset, instead the network learns automatically\n the spatial transformations that enhances the global accuracy.\n- The grid generator generates a grid of coordinates in the input\n image corresponding to each pixel from the output image.\n- The sampler uses the parameters of the transformation and applies\n it to the input image.\n\n.. figure:: /_static/img/stn/stn-arch.png\n\n.. Note::\n We need the latest version of PyTorch that contains\n affine_grid and grid_sample modules.\n\n\n" + "Spatial Transformer Networks(STN) \uad6c\uc131\ud558\uae30\n--------------------------------------\n\nSTN\uc740 \ub2e4\uc74c\uc758 \uc138 \uac00\uc9c0 \uc8fc\uc694 \uad6c\uc131 \uc694\uc18c\ub85c \uc694\uc57d\ub429\ub2c8\ub2e4.\n\n- \uc704\uce58 \uacb0\uc815 \ub124\ud2b8\uc6cc\ud06c(localization network)\ub294 \uacf5\uac04 \ubcc0\ud658 \ud30c\ub77c\ubbf8\ud130\ub97c \uc608\uce21(regress)\n \ud558\ub294 \uc77c\ubc18\uc801\uc778 CNN \uc785\ub2c8\ub2e4. \uacf5\uac04 \ubcc0\ud658\uc740 \ub370\uc774\ud130 \uc14b\uc73c\ub85c\ubd80\ud130 \uba85\uc2dc\uc801\uc73c\ub85c \ud559\uc2b5\ub418\uc9c0 \uc54a\uace0,\n \uc2e0\uacbd\ub9dd\uc774 \uc804\uccb4 \uc815\ud655\ub3c4\ub97c \ud5a5\uc0c1 \uc2dc\ud0a4\ub3c4\ub85d \uacf5\uac04 \ubcc0\ud658\uc744 \uc790\ub3d9\uc73c\ub85c \ud559\uc2b5\ud569\ub2c8\ub2e4.\n- \uadf8\ub9ac\ub4dc \uc0dd\uc131\uae30(grid generator)\ub294 \ucd9c\ub825 \uc774\ubbf8\uc9c0\ub85c\ubd80\ud130 \uac01 \ud53d\uc140\uc5d0 \ub300\uc751\ud558\ub294 \uc785\ub825\n \uc774\ubbf8\uc9c0 \ub0b4 \uc88c\ud45c \uadf8\ub9ac\ub4dc\ub97c \uc0dd\uc131\ud569\ub2c8\ub2e4.\n- \uc0d8\ud50c\ub7ec(sampler)\ub294 \uacf5\uac04 \ubcc0\ud658 \ud30c\ub77c\ubbf8\ud130\ub97c \uc785\ub825 \uc774\ubbf8\uc9c0\uc5d0 \uc801\uc6a9\ud569\ub2c8\ub2e4.\n\n.. figure:: /_static/img/stn/stn-arch.png\n\n.. Note::\n affine_grid \ubc0f grid_sample \ubaa8\ub4c8\uc774 \ud3ec\ud568\ub41c \ucd5c\uc2e0 \ubc84\uc804\uc758 PyTorch\uac00 \ud544\uc694\ud569\ub2c8\ub2e4.\n\n\n" ] }, { @@ -62,14 +62,14 @@ }, "outputs": [], "source": [ - "class Net(nn.Module):\n def __init__(self):\n super(Net, self).__init__()\n self.conv1 = nn.Conv2d(1, 10, kernel_size=5)\n self.conv2 = nn.Conv2d(10, 20, kernel_size=5)\n self.conv2_drop = nn.Dropout2d()\n self.fc1 = nn.Linear(320, 50)\n self.fc2 = nn.Linear(50, 10)\n\n # Spatial transformer localization-network\n self.localization = nn.Sequential(\n nn.Conv2d(1, 8, kernel_size=7),\n nn.MaxPool2d(2, stride=2),\n nn.ReLU(True),\n nn.Conv2d(8, 10, kernel_size=5),\n nn.MaxPool2d(2, stride=2),\n nn.ReLU(True)\n )\n\n # Regressor for the 3 * 2 affine matrix\n self.fc_loc = nn.Sequential(\n nn.Linear(10 * 3 * 3, 32),\n nn.ReLU(True),\n nn.Linear(32, 3 * 2)\n )\n\n # Initialize the weights/bias with identity transformation\n self.fc_loc[2].weight.data.zero_()\n self.fc_loc[2].bias.data.copy_(torch.tensor([1, 0, 0, 0, 1, 0], dtype=torch.float))\n\n # Spatial transformer network forward function\n def stn(self, x):\n xs = self.localization(x)\n xs = xs.view(-1, 10 * 3 * 3)\n theta = self.fc_loc(xs)\n theta = theta.view(-1, 2, 3)\n\n grid = F.affine_grid(theta, x.size())\n x = F.grid_sample(x, grid)\n\n return x\n\n def forward(self, x):\n # transform the input\n x = self.stn(x)\n\n # Perform the usual forward pass\n x = F.relu(F.max_pool2d(self.conv1(x), 2))\n x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))\n x = x.view(-1, 320)\n x = F.relu(self.fc1(x))\n x = F.dropout(x, training=self.training)\n x = self.fc2(x)\n return F.log_softmax(x, dim=1)\n\n\nmodel = Net().to(device)" + "class Net(nn.Module):\n def __init__(self):\n super(Net, self).__init__()\n self.conv1 = nn.Conv2d(1, 10, kernel_size=5)\n self.conv2 = nn.Conv2d(10, 20, kernel_size=5)\n self.conv2_drop = nn.Dropout2d()\n self.fc1 = nn.Linear(320, 50)\n self.fc2 = nn.Linear(50, 10)\n\n # \uacf5\uac04 \ubcc0\ud658\uc744 \uc704\ud55c \uc704\uce58 \uacb0\uc815 \ub124\ud2b8\uc6cc\ud06c (localization-network)\n self.localization = nn.Sequential(\n nn.Conv2d(1, 8, kernel_size=7),\n nn.MaxPool2d(2, stride=2),\n nn.ReLU(True),\n nn.Conv2d(8, 10, kernel_size=5),\n nn.MaxPool2d(2, stride=2),\n nn.ReLU(True)\n )\n\n # [3 * 2] \ud06c\uae30\uc758 \uc544\ud540(affine) \ud589\ub82c\uc5d0 \ub300\ud574 \uc608\uce21\n self.fc_loc = nn.Sequential(\n nn.Linear(10 * 3 * 3, 32),\n nn.ReLU(True),\n nn.Linear(32, 3 * 2)\n )\n\n # \ud56d\ub4f1 \ubcc0\ud658(identity transformation)\uc73c\ub85c \uac00\uc911\uce58/\ubc14\uc774\uc5b4\uc2a4 \ucd08\uae30\ud654\n self.fc_loc[2].weight.data.zero_()\n self.fc_loc[2].bias.data.copy_(torch.tensor([1, 0, 0, 0, 1, 0], dtype=torch.float))\n\n # STN\uc758 forward \ud568\uc218\n def stn(self, x):\n xs = self.localization(x)\n xs = xs.view(-1, 10 * 3 * 3)\n theta = self.fc_loc(xs)\n theta = theta.view(-1, 2, 3)\n\n grid = F.affine_grid(theta, x.size())\n x = F.grid_sample(x, grid)\n\n return x\n\n def forward(self, x):\n # \uc785\ub825\uc744 \ubcc0\ud658\n x = self.stn(x)\n\n # \uc77c\ubc18\uc801\uc778 forward pass\ub97c \uc218\ud589\n x = F.relu(F.max_pool2d(self.conv1(x), 2))\n x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))\n x = x.view(-1, 320)\n x = F.relu(self.fc1(x))\n x = F.dropout(x, training=self.training)\n x = self.fc2(x)\n return F.log_softmax(x, dim=1)\n\n\nmodel = Net().to(device)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Training the model\n------------------\n\nNow, let's use the SGD algorithm to train the model. The network is\nlearning the classification task in a supervised way. In the same time\nthe model is learning STN automatically in an end-to-end fashion.\n\n" + "\ubaa8\ub378 \ud559\uc2b5\ud558\uae30\n------------------\n\n\uc774\uc81c SGD \uc54c\uace0\ub9ac\uc998\uc744 \uc774\uc6a9\ud574 \ubaa8\ub378\uc744 \ud559\uc2b5\uc2dc\ucf1c \ubd05\uc2dc\ub2e4. \uc55e\uc11c \uad6c\uc131\ud55c \uc2e0\uacbd\ub9dd\uc740\n\uac10\ub3c5 \ud559\uc2b5 \ubc29\uc2dd(supervised way)\uc73c\ub85c \ubd84\ub958 \ubb38\uc81c\ub97c \ud559\uc2b5\ud569\ub2c8\ub2e4. \ub610\ud55c \uc774 \ubaa8\ub378\uc740\nend-to-end \ubc29\uc2dd\uc73c\ub85c STN\uc744 \uc790\ub3d9\uc73c\ub85c \ud559\uc2b5\ud569\ub2c8\ub2e4. \n\n" ] }, { @@ -80,14 +80,14 @@ }, "outputs": [], "source": [ - "optimizer = optim.SGD(model.parameters(), lr=0.01)\n\n\ndef train(epoch):\n model.train()\n for batch_idx, (data, target) in enumerate(train_loader):\n data, target = data.to(device), target.to(device)\n\n optimizer.zero_grad()\n output = model(data)\n loss = F.nll_loss(output, target)\n loss.backward()\n optimizer.step()\n if batch_idx % 500 == 0:\n print('Train Epoch: {} [{}/{} ({:.0f}%)]\\tLoss: {:.6f}'.format(\n epoch, batch_idx * len(data), len(train_loader.dataset),\n 100. * batch_idx / len(train_loader), loss.item()))\n#\n# A simple test procedure to measure STN the performances on MNIST.\n#\n\n\ndef test():\n with torch.no_grad():\n model.eval()\n test_loss = 0\n correct = 0\n for data, target in test_loader:\n data, target = data.to(device), target.to(device)\n output = model(data)\n\n # sum up batch loss\n test_loss += F.nll_loss(output, target, size_average=False).item()\n # get the index of the max log-probability\n pred = output.max(1, keepdim=True)[1]\n correct += pred.eq(target.view_as(pred)).sum().item()\n\n test_loss /= len(test_loader.dataset)\n print('\\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\\n'\n .format(test_loss, correct, len(test_loader.dataset),\n 100. * correct / len(test_loader.dataset)))" + "optimizer = optim.SGD(model.parameters(), lr=0.01)\n\n\ndef train(epoch):\n model.train()\n for batch_idx, (data, target) in enumerate(train_loader):\n data, target = data.to(device), target.to(device)\n\n optimizer.zero_grad()\n output = model(data)\n loss = F.nll_loss(output, target)\n loss.backward()\n optimizer.step()\n if batch_idx % 500 == 0:\n print('Train Epoch: {} [{}/{} ({:.0f}%)]\\tLoss: {:.6f}'.format(\n epoch, batch_idx * len(data), len(train_loader.dataset),\n 100. * batch_idx / len(train_loader), loss.item()))\n#\n# MNIST \ub370\uc774\ud130\uc14b\uc5d0\uc11c STN\uc758 \uc131\ub2a5\uc744 \uce21\uc815\ud558\uae30 \uc704\ud55c \uac04\ub2e8\ud55c \ud14c\uc2a4\ud2b8 \uc808\ucc28\n#\n\n\ndef test():\n with torch.no_grad():\n model.eval()\n test_loss = 0\n correct = 0\n for data, target in test_loader:\n data, target = data.to(device), target.to(device)\n output = model(data)\n\n # \ubc30\uce58 \uc190\uc2e4 \ud569\ud558\uae30\n test_loss += F.nll_loss(output, target, size_average=False).item()\n # \ub85c\uadf8-\ud655\ub960\uc758 \ucd5c\ub300\uac12\uc5d0 \ud574\ub2f9\ud558\ub294 \uc778\ub371\uc2a4 \uac00\uc838\uc624\uae30\n pred = output.max(1, keepdim=True)[1]\n correct += pred.eq(target.view_as(pred)).sum().item()\n\n test_loss /= len(test_loader.dataset)\n print('\\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\\n'\n .format(test_loss, correct, len(test_loader.dataset),\n 100. * correct / len(test_loader.dataset)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Visualizing the STN results\n---------------------------\n\nNow, we will inspect the results of our learned visual attention\nmechanism.\n\nWe define a small helper function in order to visualize the\ntransformations while training.\n\n" + "STN \uacb0\uacfc \uc2dc\uac01\ud654\ud558\uae30\n---------------------------\n\n\uc774\uc81c \ud559\uc2b5\ud55c \ube44\uc8fc\uc5bc \uc5b4\ud150\uc158 \uba54\ucee4\ub2c8\uc998\uc758 \uacb0\uacfc\ub97c \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\ud559\uc2b5\ud558\ub294 \ub3d9\uc548 \ubcc0\ud658\ub41c \uacb0\uacfc\ub97c \uc2dc\uac01\ud654\ud558\uae30 \uc704\ud574 \uc791\uc740 \ub3c4\uc6c0(helper) \ud568\uc218\ub97c \uc815\uc758\ud569\ub2c8\ub2e4.\n\n" ] }, { @@ -98,7 +98,7 @@ }, "outputs": [], "source": [ - "def convert_image_np(inp):\n \"\"\"Convert a Tensor to numpy image.\"\"\"\n inp = inp.numpy().transpose((1, 2, 0))\n mean = np.array([0.485, 0.456, 0.406])\n std = np.array([0.229, 0.224, 0.225])\n inp = std * inp + mean\n inp = np.clip(inp, 0, 1)\n return inp\n\n# We want to visualize the output of the spatial transformers layer\n# after the training, we visualize a batch of input images and\n# the corresponding transformed batch using STN.\n\n\ndef visualize_stn():\n with torch.no_grad():\n # Get a batch of training data\n data = next(iter(test_loader))[0].to(device)\n\n input_tensor = data.cpu()\n transformed_input_tensor = model.stn(data).cpu()\n\n in_grid = convert_image_np(\n torchvision.utils.make_grid(input_tensor))\n\n out_grid = convert_image_np(\n torchvision.utils.make_grid(transformed_input_tensor))\n\n # Plot the results side-by-side\n f, axarr = plt.subplots(1, 2)\n axarr[0].imshow(in_grid)\n axarr[0].set_title('Dataset Images')\n\n axarr[1].imshow(out_grid)\n axarr[1].set_title('Transformed Images')\n\nfor epoch in range(1, 20 + 1):\n train(epoch)\n test()\n\n# Visualize the STN transformation on some input batch\nvisualize_stn()\n\nplt.ioff()\nplt.show()" + "def convert_image_np(inp):\n \"\"\"Convert a Tensor to numpy image.\"\"\"\n inp = inp.numpy().transpose((1, 2, 0))\n mean = np.array([0.485, 0.456, 0.406])\n std = np.array([0.229, 0.224, 0.225])\n inp = std * inp + mean\n inp = np.clip(inp, 0, 1)\n return inp\n\n# \ud559\uc2b5 \ud6c4 \uacf5\uac04 \ubcc0\ud658 \uacc4\uce35\uc758 \ucd9c\ub825\uc744 \uc2dc\uac01\ud654\ud558\uace0, \uc785\ub825 \uc774\ubbf8\uc9c0 \ubc30\uce58 \ub370\uc774\ud130 \ubc0f \n# STN\uc744 \uc0ac\uc6a9\ud574 \ubcc0\ud658\ub41c \ubc30\uce58 \ub370\uc774\ud130\ub97c \uc2dc\uac01\ud654 \ud569\ub2c8\ub2e4.\n\n\ndef visualize_stn():\n with torch.no_grad():\n # \ud559\uc2b5 \ub370\uc774\ud130\uc758 \ubc30\uce58 \uac00\uc838\uc624\uae30\n data = next(iter(test_loader))[0].to(device)\n\n input_tensor = data.cpu()\n transformed_input_tensor = model.stn(data).cpu()\n\n in_grid = convert_image_np(\n torchvision.utils.make_grid(input_tensor))\n\n out_grid = convert_image_np(\n torchvision.utils.make_grid(transformed_input_tensor))\n\n # \uacb0\uacfc\ub97c \ub098\ub780\ud788 \ud45c\uc2dc\ud558\uae30\n f, axarr = plt.subplots(1, 2)\n axarr[0].imshow(in_grid)\n axarr[0].set_title('Dataset Images')\n\n axarr[1].imshow(out_grid)\n axarr[1].set_title('Transformed Images')\n\nfor epoch in range(1, 20 + 1):\n train(epoch)\n test()\n\n# \uc77c\ubd80 \uc785\ub825 \ubc30\uce58 \ub370\uc774\ud130\uc5d0\uc11c STN \ubcc0\ud658 \uacb0\uacfc\ub97c \uc2dc\uac01\ud654\nvisualize_stn()\n\nplt.ioff()\nplt.show()" ] } ], @@ -118,7 +118,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/a75cfadf4fa84dd594874d4c53b62820/char_rnn_generation_tutorial.ipynb b/docs/_downloads/a75cfadf4fa84dd594874d4c53b62820/char_rnn_generation_tutorial.ipynb index 2f016f083..7f080987b 100644 --- a/docs/_downloads/a75cfadf4fa84dd594874d4c53b62820/char_rnn_generation_tutorial.ipynb +++ b/docs/_downloads/a75cfadf4fa84dd594874d4c53b62820/char_rnn_generation_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n\uae30\ucd08\ubd80\ud130 \uc2dc\uc791\ud558\ub294 NLP: \ubb38\uc790-\ub2e8\uc704 RNN\uc73c\ub85c \uc774\ub984 \uc0dd\uc131\ud558\uae30\n********************************************************************************\n**Author**: `Sean Robertson `_\n **\ubc88\uc5ed**: `\ud669\uc131\uc218 `_\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc740 3\uac1c\ub85c \uc774\ub904\uc9c4 \"\uae30\ucd08\ubd80\ud130 \uc2dc\uc791\ud558\ub294 NLP\"\uc758 2\ubc88\uc9f8 \ud29c\ud1a0\ub9ac\uc5bc\uc785\ub2c8\ub2e4.\n`\uccab\ubc88\uc9f8 \ud29c\ud1a0\ub9ac\uc5bc `\n\uc5d0\uc11c\ub294 \uc774\ub984\uc758 \uc5b8\uc5b4\ub97c \ubd84\ub958\ud558\uae30 \uc704\ud574 RNN\uc744 \uc0ac\uc6a9\ud588\uc2b5\ub2c8\ub2e4.\n\uc774\ubc88\uc5d0\ub294 \ubc18\ub300\ub85c \uc5b8\uc5b4\ub85c \uc774\ub984\uc744 \uc0dd\uc131\ud560 \uc608\uc815\uc785\ub2c8\ub2e4.\n\n::\n\n > python sample.py Russian RUS\n Rovakov\n Uantov\n Shavakov\n\n > python sample.py German GER\n Gerren\n Ereng\n Rosher\n\n > python sample.py Spanish SPA\n Salla\n Parer\n Allan\n\n > python sample.py Chinese CHI\n Chan\n Hang\n Iun\n\n\uc6b0\ub9ac\ub294 \uba87 \uac1c\uc758 \uc120\ud615 \uacc4\uce35\uc73c\ub85c \uc791\uc740 RNN\uc744 \uc9c1\uc811 \ub9cc\ub4e4\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n\uc774\uc804 \ud29c\ud1a0\ub9ac\uc5bc\uc778 \uc774\ub984\uc744 \uc77d\uc740 \ud6c4 \uadf8 \uc5b8\uc5b4\ub97c \uc608\uce21\ud558\ub294 \uac83\uacfc\uc758 \ud070 \ucc28\uc774\uc810\uc740\n\uc5b8\uc5b4\ub97c \uc785\ub825\ud558\uace0 \ud55c \ubc88\uc5d0 \ud55c \uae00\uc790\ub97c \uc0dd\uc131\ud558\uc5ec \ucd9c\ub825\ud558\ub294 \uac83\uc785\ub2c8\ub2e4.\n\uc5b8\uc5b4 \ud615\uc131(\ub2e8\uc5b4 \ub610\ub294 \ub2e4\ub978 \uace0\ucc28\uc6d0 \uad6c\uc870\ub85c\ub3c4 \uc218\ud589 \ub420 \uc218 \uc788\uc74c)\uc744 \uc704\ud574\n\ubb38\uc790\ub97c \ubc18\ubcf5\uc801\uc73c\ub85c \uc608\uce21\ud558\ub294 \uac83\uc744 \"\uc5b8\uc5b4 \ubaa8\ub378\" \uc774\ub77c\uace0 \ud569\ub2c8\ub2e4.\n\n**\ucd94\ucc9c \uc790\ub8cc:**\n\nPytorch\ub97c \uc124\uce58\ud588\uace0, Python\uc744 \uc54c\uace0, Tensor\ub97c \uc774\ud574\ud55c\ub2e4\uace0 \uac00\uc815\ud569\ub2c8\ub2e4:\n\n- https://pytorch.org/ \uc124\uce58 \uc548\ub0b4\n- :doc:`/beginner/deep_learning_60min_blitz` PyTorch \uc2dc\uc791\ud558\uae30\n- :doc:`/beginner/pytorch_with_examples` \ub113\uace0 \uae4a\uc740 \ud1b5\ucc30\uc744 \uc704\ud55c \uc790\ub8cc\n- :doc:`/beginner/former_torchies_tutorial` \uc774\uc804 Lua Torch \uc0ac\uc6a9\uc790\ub97c \uc704\ud55c \uc790\ub8cc\n\nRNN\uacfc \uc791\ub3d9 \ubc29\uc2dd\uc744 \uc544\ub294 \uac83 \ub610\ud55c \uc720\uc6a9\ud569\ub2c8\ub2e4:\n\n- `The Unreasonable Effectiveness of Recurrent Neural\n Networks `__\n \uc2e4\uc0dd\ud65c \uc608\uc81c\ub97c \ubcf4\uc5ec \uc90d\ub2c8\ub2e4.\n- `Understanding LSTM\n Networks `__\n LSTM\uc5d0 \uad00\ud55c \uac83\uc774\uc9c0\ub9cc RNN\uc5d0 \uad00\ud574\uc11c\ub3c4 \uc720\uc775\ud569\ub2c8\ub2e4.\n\n\uc774\uc804 \ud29c\ud1a0\ub9ac\uc5bc\ub3c4 \ucd94\ucc9c\ud569\ub2c8\ub2e4. :doc:`/intermediate/char_rnn_classification_tutorial`\n\n\n\ub370\uc774\ud130 \uc900\ube44\n==================\n\n.. Note::\n `\uc5ec\uae30 `_\n \uc5d0\uc11c \ub370\uc774\ud130\ub97c \ub2e4\uc6b4 \ubc1b\uace0, \ud604\uc7ac \ub514\ub809\ud1a0\ub9ac\uc5d0 \uc555\ucd95\uc744 \ud478\uc2ed\uc2dc\uc624.\n\n\uc774 \uacfc\uc815\uc758 \ub354 \uc790\uc138\ud55c \uc0ac\ud56d\uc740 \uc9c0\ub09c \ud29c\ud1a0\ub9ac\uc5bc\uc744 \ubcf4\uc2ed\uc2dc\uc624.\n\uc694\uc57d\ud558\uba74, \uc904\ub9c8\ub2e4 \uc774\ub984\uc774 \uc801\ud78c \ud14d\uc2a4\ud2b8 \ud30c\uc77c ``data/names/[Language].txt`` \uc788\uc2b5\ub2c8\ub2e4.\n\uc774\uac83\uc744 \uc5b4\ub808\uc774\ub85c \ubd84\ub9ac\ud558\uace0, Unicode\ub97c ASCII\ub85c \ubcc0\uacbd\ud558\uace0,\n\uc0ac\uc804 ``{language: [names ...]}`` \uc744 \ub9cc\ub4e4\uc5b4\uc11c \ub9c8\ubb34\ub9ac\ud569\ub2c8\ub2e4.\n\n\n" + "\n\uae30\ucd08\ubd80\ud130 \uc2dc\uc791\ud558\ub294 NLP: \ubb38\uc790-\ub2e8\uc704 RNN\uc73c\ub85c \uc774\ub984 \uc0dd\uc131\ud558\uae30\n********************************************************************************\n**Author**: `Sean Robertson `_\n **\ubc88\uc5ed**: `\ud669\uc131\uc218 `_\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc740 3\uac1c\ub85c \uc774\ub904\uc9c4 \"\uae30\ucd08\ubd80\ud130 \uc2dc\uc791\ud558\ub294 NLP\"\uc758 2\ubc88\uc9f8 \ud29c\ud1a0\ub9ac\uc5bc\uc785\ub2c8\ub2e4.\n`\uccab\ubc88\uc9f8 \ud29c\ud1a0\ub9ac\uc5bc `\n\uc5d0\uc11c\ub294 \uc774\ub984\uc758 \uc5b8\uc5b4\ub97c \ubd84\ub958\ud558\uae30 \uc704\ud574 RNN\uc744 \uc0ac\uc6a9\ud588\uc2b5\ub2c8\ub2e4.\n\uc774\ubc88\uc5d0\ub294 \ubc18\ub300\ub85c \uc5b8\uc5b4\ub85c \uc774\ub984\uc744 \uc0dd\uc131\ud560 \uc608\uc815\uc785\ub2c8\ub2e4.\n\n::\n\n > python sample.py Russian RUS\n Rovakov\n Uantov\n Shavakov\n\n > python sample.py German GER\n Gerren\n Ereng\n Rosher\n\n > python sample.py Spanish SPA\n Salla\n Parer\n Allan\n\n > python sample.py Chinese CHI\n Chan\n Hang\n Iun\n\n\uc6b0\ub9ac\ub294 \uba87 \uac1c\uc758 \uc120\ud615 \uacc4\uce35\uc73c\ub85c \uc791\uc740 RNN\uc744 \uc9c1\uc811 \ub9cc\ub4e4\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n\uc774\uc804 \ud29c\ud1a0\ub9ac\uc5bc\uc778 \uc774\ub984\uc744 \uc77d\uc740 \ud6c4 \uadf8 \uc5b8\uc5b4\ub97c \uc608\uce21\ud558\ub294 \uac83\uacfc\uc758 \ud070 \ucc28\uc774\uc810\uc740\n\uc5b8\uc5b4\ub97c \uc785\ub825\ud558\uace0 \ud55c \ubc88\uc5d0 \ud55c \uae00\uc790\ub97c \uc0dd\uc131\ud558\uc5ec \ucd9c\ub825\ud558\ub294 \uac83\uc785\ub2c8\ub2e4.\n\uc5b8\uc5b4 \ud615\uc131(\ub2e8\uc5b4 \ub610\ub294 \ub2e4\ub978 \uace0\ucc28\uc6d0 \uad6c\uc870\ub85c\ub3c4 \uc218\ud589 \ub420 \uc218 \uc788\uc74c)\uc744 \uc704\ud574\n\ubb38\uc790\ub97c \ubc18\ubcf5\uc801\uc73c\ub85c \uc608\uce21\ud558\ub294 \uac83\uc744 \"\uc5b8\uc5b4 \ubaa8\ub378\" \uc774\ub77c\uace0 \ud569\ub2c8\ub2e4.\n\n**\ucd94\ucc9c \uc790\ub8cc:**\n\nPytorch\ub97c \uc124\uce58\ud588\uace0, Python\uc744 \uc54c\uace0, Tensor\ub97c \uc774\ud574\ud55c\ub2e4\uace0 \uac00\uc815\ud569\ub2c8\ub2e4:\n\n- https://pytorch.org/ \uc124\uce58 \uc548\ub0b4\n- :doc:`/beginner/deep_learning_60min_blitz` PyTorch \uc2dc\uc791\ud558\uae30\n- :doc:`/beginner/pytorch_with_examples` \ub113\uace0 \uae4a\uc740 \ud1b5\ucc30\uc744 \uc704\ud55c \uc790\ub8cc\n- :doc:`/beginner/former_torchies_tutorial` \uc774\uc804 Lua Torch \uc0ac\uc6a9\uc790\ub97c \uc704\ud55c \uc790\ub8cc\n\nRNN\uacfc \uc791\ub3d9 \ubc29\uc2dd\uc744 \uc544\ub294 \uac83 \ub610\ud55c \uc720\uc6a9\ud569\ub2c8\ub2e4:\n\n- `The Unreasonable Effectiveness of Recurrent Neural\n Networks `__\n \uc2e4\uc0dd\ud65c \uc608\uc81c\ub97c \ubcf4\uc5ec \uc90d\ub2c8\ub2e4.\n- `Understanding LSTM\n Networks `__\n LSTM\uc5d0 \uad00\ud55c \uac83\uc774\uc9c0\ub9cc RNN\uc5d0 \uad00\ud574\uc11c\ub3c4 \uc720\uc775\ud569\ub2c8\ub2e4.\n\n\uc774\uc804 \ud29c\ud1a0\ub9ac\uc5bc\ub3c4 \ucd94\ucc9c\ud569\ub2c8\ub2e4. :doc:`/intermediate/char_rnn_classification_tutorial`\n\n\n\ub370\uc774\ud130 \uc900\ube44\n==================\n\n.. Note::\n `\uc5ec\uae30 `_\n \uc5d0\uc11c \ub370\uc774\ud130\ub97c \ub2e4\uc6b4 \ubc1b\uace0, \ud604\uc7ac \ub514\ub809\ud1a0\ub9ac\uc5d0 \uc555\ucd95\uc744 \ud478\uc2ed\uc2dc\uc624.\n\n\uc774 \uacfc\uc815\uc758 \ub354 \uc790\uc138\ud55c \uc0ac\ud56d\uc740 \uc9c0\ub09c \ud29c\ud1a0\ub9ac\uc5bc\uc744 \ubcf4\uc2ed\uc2dc\uc624.\n\uc694\uc57d\ud558\uba74, \uc904\ub9c8\ub2e4 \uc774\ub984\uc774 \uc801\ud78c \ud14d\uc2a4\ud2b8 \ud30c\uc77c ``data/names/[Language].txt`` \uc788\uc2b5\ub2c8\ub2e4.\n\uc774\uac83\uc744 \uc5b4\ub808\uc774\ub85c \ubd84\ub9ac\ud558\uace0, Unicode\ub97c ASCII\ub85c \ubcc0\uacbd\ud558\uace0,\n\uc0ac\uc804 ``{language: [names ...]}`` \uc744 \ub9cc\ub4e4\uc5b4\uc11c \ub9c8\ubb34\ub9ac\ud569\ub2c8\ub2e4.\n" ] }, { @@ -215,7 +215,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/aa3898eb04d468790e00cb42405b1c23/transformer_tutorial.py b/docs/_downloads/aa3898eb04d468790e00cb42405b1c23/transformer_tutorial.py index 7de36bbc0..8fbe8946d 100644 --- a/docs/_downloads/aa3898eb04d468790e00cb42405b1c23/transformer_tutorial.py +++ b/docs/_downloads/aa3898eb04d468790e00cb42405b1c23/transformer_tutorial.py @@ -1,6 +1,6 @@ """ nn.Transformer 와 TorchText 로 시퀀스-투-시퀀스(Sequence-to-Sequence) 모델링하기 -========================================================================== +================================================================================= 이 튜토리얼에서는 `nn.Transformer `__ 모듈을 @@ -114,7 +114,7 @@ def forward(self, x): ###################################################################### # 데이터 로드하고 배치 만들기 -# ------------------- +# ----------------------------- # @@ -168,7 +168,7 @@ def batchify(data, bsz): ###################################################################### # 입력(input) 과 타겟(target) 시퀀스를 생성하기 위한 함수들 -# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ # @@ -194,7 +194,7 @@ def get_batch(source, i): ###################################################################### # 인스턴스(instance) 초기화하기 -# -------------------- +# -------------------------------- # @@ -304,8 +304,8 @@ def evaluate(eval_model, data_source): ###################################################################### -# 평가 데이터셋(test dataset) 으로 모델을 평가하기 -# ------------------------------------- +# 평가 데이터셋(test dataset)으로 모델을 평가하기 +# ------------------------------------------------- # # 평가 데이터셋에 대한 결과를 확인하기 위해서 최고의 모델을 적용합니다. diff --git a/docs/_downloads/b45d95b1d38fe556c77d1ee548809d28/flask_rest_api_tutorial.py b/docs/_downloads/b45d95b1d38fe556c77d1ee548809d28/flask_rest_api_tutorial.py index ce061fa7d..b21cdeeaf 100644 --- a/docs/_downloads/b45d95b1d38fe556c77d1ee548809d28/flask_rest_api_tutorial.py +++ b/docs/_downloads/b45d95b1d38fe556c77d1ee548809d28/flask_rest_api_tutorial.py @@ -47,7 +47,7 @@ # # :: # -# $ pip install Flask==1.0.3 torchvision-0.3.0 +# $ pip install Flask==1.0.3 torchvision==0.3.0 ###################################################################### @@ -104,7 +104,7 @@ def predict(): # 추론(Inference) # ----------------- # -# 다음 섹션에서는 추론 코드 작성에 집중하겠습니다. 먼저 이미지를 DenseNet에 공급(feed)할 수 +# 다음 섹션에서는 추론 코드 작성에 집중하겠습니다. 먼저 이미지를 DenseNet에 공급(feed)할 수 # 있도록 준비하는 방법을 살펴본 뒤, 모델로부터 예측 결과를 얻는 방법을 살펴보겠습니다. # # 이미지 준비하기 diff --git a/docs/_downloads/b5fa995b1432ebc93ea7bfe7ec9daed1/text_sentiment_ngrams_tutorial.ipynb b/docs/_downloads/b5fa995b1432ebc93ea7bfe7ec9daed1/text_sentiment_ngrams_tutorial.ipynb index 7097f7e3a..6fe2d679e 100644 --- a/docs/_downloads/b5fa995b1432ebc93ea7bfe7ec9daed1/text_sentiment_ngrams_tutorial.ipynb +++ b/docs/_downloads/b5fa995b1432ebc93ea7bfe7ec9daed1/text_sentiment_ngrams_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nTorchText\ub85c \ud14d\uc2a4\ud2b8 \ubd84\ub958\ud558\uae30\n==================================\n**\ubc88\uc5ed**: `\uae40\uac15\ubbfc `_ , `\uae40\uc9c4\ud604 `_\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 ``torchtext`` \uc5d0 \ud3ec\ud568\ub418\uc5b4 \uc788\ub294 \ud14d\uc2a4\ud2b8 \ubd84\ub958\n\ub370\uc774\ud130\uc14b\uc758 \uc0ac\uc6a9 \ubc29\ubc95\uc744 \uc0b4\ud3b4 \ubd05\ub2c8\ub2e4. \ub370\uc774\ud130\uc14b\uc740 \ub2e4\uc74c\uc744 \ud3ec\ud568\ud569\ub2c8\ub2e4.\n\n::\n\n - AG_NEWS,\n - SogouNews,\n - DBpedia,\n - YelpReviewPolarity,\n - YelpReviewFull,\n - YahooAnswers,\n - AmazonReviewPolarity,\n - AmazonReviewFull\n\n\uc774 \uc608\uc81c\uc5d0\uc11c\ub294 ``TextClassification`` \uc758 \ub370\uc774\ud130\uc14b\ub4e4 \uc911 \ud558\ub098\ub97c \uc774\uc6a9\ud574 \ubd84\ub958\ub97c \uc704\ud55c\n \uc9c0\ub3c4 \ud559\uc2b5 \uc54c\uace0\ub9ac\uc998\uc744 \ud6c8\ub828\ud558\ub294 \ubc29\ubc95\uc744 \ubcf4\uc5ec\uc90d\ub2c8\ub2e4.\n\nngrams\ub97c \uc774\uc6a9\ud558\uc5ec \ub370\uc774\ud130 \ubd88\ub7ec\uc624\uae30\n-----------------------------------\n\nBag of ngrams \ud53c\uccd0\ub294 \uc9c0\uc5ed(local) \ub2e8\uc5b4 \uc21c\uc11c\uc5d0 \ub300\ud55c \ubd80\ubd84\uc801\uc778 \uc815\ubcf4\ub97c \ud3ec\ucc29\ud558\uae30 \uc704\ud574 \uc801\uc6a9\ud569\ub2c8\ub2e4.\n\uc2e4\uc81c \uc0c1\ud669\uc5d0\uc11c\ub294 bi-gram\uc774\ub098 tri-gram\uc740 \ub2e8 \ud558\ub098\uc758 \ub2e8\uc5b4\ub97c \uc774\uc6a9\ud558\ub294 \uac83\ubcf4\ub2e4 \ub354 \ub9ce\uc740 \uc774\uc775\uc744 \uc8fc\uae30 \ub54c\ubb38\uc5d0 \uc801\uc6a9\ub429\ub2c8\ub2e4.\n\uc608\ub97c \ub4e4\uba74 \ub2e4\uc74c\uacfc \uac19\uc2b5\ub2c8\ub2e4.\n\n::\n\n \"load data with ngrams\"\n Bi-grams \uacb0\uacfc: \"load data\", \"data with\", \"with ngrams\"\n Tri-grams \uacb0\uacfc: \"load data with\", \"data with ngrams\"\n\n``TextClassification`` \ub370\uc774\ud130\uc14b\uc740 ngrams method\uc744 \uc9c0\uc6d0\ud569\ub2c8\ub2e4. ngrams\uc744 2\ub85c \uc124\uc815\ud558\uba74,\n\ub370\uc774\ud130\uc14b \uc548\uc758 \uc608\uc81c \ud14d\uc2a4\ud2b8\ub294 \uac01\uac01\uc758(single) \ub2e8\uc5b4\ub4e4\uc5d0 bi-grams \ubb38\uc790\uc5f4\uc774 \ub354\ud574\uc9c4 \ub9ac\uc2a4\ud2b8\uac00 \ub420 \uac83\uc785\ub2c8\ub2e4.\n\n\n" + "\nTorchText\ub85c \ud14d\uc2a4\ud2b8 \ubd84\ub958\ud558\uae30\n==================================\n**\ubc88\uc5ed**: `\uae40\uac15\ubbfc `_ , `\uae40\uc9c4\ud604 `_\n\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 ``torchtext`` \uc5d0 \ud3ec\ud568\ub418\uc5b4 \uc788\ub294 \ud14d\uc2a4\ud2b8 \ubd84\ub958\n\ub370\uc774\ud130\uc14b\uc758 \uc0ac\uc6a9 \ubc29\ubc95\uc744 \uc0b4\ud3b4 \ubd05\ub2c8\ub2e4. \ub370\uc774\ud130\uc14b\uc740 \ub2e4\uc74c\uc744 \ud3ec\ud568\ud569\ub2c8\ub2e4.\n\n::\n\n - AG_NEWS,\n - SogouNews,\n - DBpedia,\n - YelpReviewPolarity,\n - YelpReviewFull,\n - YahooAnswers,\n - AmazonReviewPolarity,\n - AmazonReviewFull\n\n\uc774 \uc608\uc81c\uc5d0\uc11c\ub294 ``TextClassification`` \uc758 \ub370\uc774\ud130\uc14b\ub4e4 \uc911 \ud558\ub098\ub97c \uc774\uc6a9\ud574 \ubd84\ub958\ub97c \uc704\ud55c\n \uc9c0\ub3c4 \ud559\uc2b5 \uc54c\uace0\ub9ac\uc998\uc744 \ud6c8\ub828\ud558\ub294 \ubc29\ubc95\uc744 \ubcf4\uc5ec\uc90d\ub2c8\ub2e4.\n\nngrams\ub97c \uc774\uc6a9\ud558\uc5ec \ub370\uc774\ud130 \ubd88\ub7ec\uc624\uae30\n-----------------------------------\n\nBag of ngrams \ud53c\uccd0\ub294 \uc9c0\uc5ed(local) \ub2e8\uc5b4 \uc21c\uc11c\uc5d0 \ub300\ud55c \ubd80\ubd84\uc801\uc778 \uc815\ubcf4\ub97c \ud3ec\ucc29\ud558\uae30 \uc704\ud574 \uc801\uc6a9\ud569\ub2c8\ub2e4.\n\uc2e4\uc81c \uc0c1\ud669\uc5d0\uc11c\ub294 bi-gram\uc774\ub098 tri-gram\uc740 \ub2e8 \ud558\ub098\uc758 \ub2e8\uc5b4\ub97c \uc774\uc6a9\ud558\ub294 \uac83\ubcf4\ub2e4 \ub354 \ub9ce\uc740 \uc774\uc775\uc744 \uc8fc\uae30 \ub54c\ubb38\uc5d0 \uc801\uc6a9\ub429\ub2c8\ub2e4.\n\uc608\ub97c \ub4e4\uba74 \ub2e4\uc74c\uacfc \uac19\uc2b5\ub2c8\ub2e4.\n\n::\n\n \"load data with ngrams\"\n Bi-grams \uacb0\uacfc: \"load data\", \"data with\", \"with ngrams\"\n Tri-grams \uacb0\uacfc: \"load data with\", \"data with ngrams\"\n\n``TextClassification`` \ub370\uc774\ud130\uc14b\uc740 ngrams method\uc744 \uc9c0\uc6d0\ud569\ub2c8\ub2e4. ngrams\uc744 2\ub85c \uc124\uc815\ud558\uba74,\n\ub370\uc774\ud130\uc14b \uc548\uc758 \uc608\uc81c \ud14d\uc2a4\ud2b8\ub294 \uac01\uac01\uc758(single) \ub2e8\uc5b4\ub4e4\uc5d0 bi-grams \ubb38\uc790\uc5f4\uc774 \ub354\ud574\uc9c4 \ub9ac\uc2a4\ud2b8\uac00 \ub420 \uac83\uc785\ub2c8\ub2e4.\n" ] }, { @@ -225,7 +225,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/c029676472d90691aa145c6fb97a61c3/neural_networks_tutorial.ipynb b/docs/_downloads/c029676472d90691aa145c6fb97a61c3/neural_networks_tutorial.ipynb index c3d25ac72..efb1eadbb 100644 --- a/docs/_downloads/c029676472d90691aa145c6fb97a61c3/neural_networks_tutorial.ipynb +++ b/docs/_downloads/c029676472d90691aa145c6fb97a61c3/neural_networks_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n\uc2e0\uacbd\ub9dd(Neural Networks)\n=======================\n\n\uc2e0\uacbd\ub9dd\uc740 ``torch.nn`` \ud328\ud0a4\uc9c0\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc0dd\uc131\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc9c0\uae08\uae4c\uc9c0 ``autograd`` \ub97c \uc0b4\ud3b4\ubd24\ub294\ub370\uc694, ``nn`` \uc740 \ubaa8\ub378\uc744 \uc815\uc758\ud558\uace0 \ubbf8\ubd84\ud558\ub294\ub370\n``autograd`` \ub97c \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n``nn.Module`` \uc740 \uacc4\uce35(layer)\uacfc ``output`` \uc744 \ubc18\ud658\ud558\ub294 ``forward(input)``\n\uba54\uc11c\ub4dc\ub97c \ud3ec\ud568\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc22b\uc790 \uc774\ubbf8\uc9c0\ub97c \ubd84\ub958\ud558\ub294 \uc2e0\uacbd\ub9dd\uc744 \uc608\uc81c\ub85c \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4:\n\n.. figure:: /_static/img/mnist.png\n :alt: convnet\n\n convnet\n\n\uc774\ub294 \uac04\ub2e8\ud55c \uc21c\uc804\ud30c \ub124\ud2b8\uc6cc\ud06c(Feed-forward network)\uc785\ub2c8\ub2e4. \uc785\ub825(input)\uc744 \ubc1b\uc544\n\uc5ec\ub7ec \uacc4\uce35\uc5d0 \ucc28\ub840\ub85c \uc804\ub2ec\ud55c \ud6c4, \ucd5c\uc885 \ucd9c\ub825(output)\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.\n\n\uc2e0\uacbd\ub9dd\uc758 \uc77c\ubc18\uc801\uc778 \ud559\uc2b5 \uacfc\uc815\uc740 \ub2e4\uc74c\uacfc \uac19\uc2b5\ub2c8\ub2e4:\n\n- \ud559\uc2b5 \uac00\ub2a5\ud55c \ub9e4\uac1c\ubcc0\uc218(\ub610\ub294 \uac00\uc911\uce58(weight))\ub97c \uac16\ub294 \uc2e0\uacbd\ub9dd\uc744 \uc815\uc758\ud569\ub2c8\ub2e4.\n- \ub370\uc774\ud130\uc14b(dataset) \uc785\ub825\uc744 \ubc18\ubcf5\ud569\ub2c8\ub2e4.\n- \uc785\ub825\uc744 \uc2e0\uacbd\ub9dd\uc5d0\uc11c \uc804\ud30c(process)\ud569\ub2c8\ub2e4.\n- \uc190\uc2e4(loss; \ucd9c\ub825\uc774 \uc815\ub2f5\uc73c\ub85c\ubd80\ud130 \uc5bc\ub9c8\ub098 \ub5a8\uc5b4\uc838\uc788\ub294\uc9c0)\uc744 \uacc4\uc0b0\ud569\ub2c8\ub2e4.\n- \ubcc0\ud654\ub3c4(gradient)\ub97c \uc2e0\uacbd\ub9dd\uc758 \ub9e4\uac1c\ubcc0\uc218\ub4e4\uc5d0 \uc5ed\uc73c\ub85c \uc804\ud30c\ud569\ub2c8\ub2e4.\n- \uc2e0\uacbd\ub9dd\uc758 \uac00\uc911\uce58\ub97c \uac31\uc2e0\ud569\ub2c8\ub2e4. \uc77c\ubc18\uc801\uc73c\ub85c \ub2e4\uc74c\uacfc \uac19\uc740 \uac04\ub2e8\ud55c \uaddc\uce59\uc744 \uc0ac\uc6a9\ud569\ub2c8\ub2e4:\n ``\uc0c8\ub85c\uc6b4 \uac00\uc911\uce58(weight) = \uac00\uc911\uce58(weight) - \ud559\uc2b5\ub960(learning rate) * \ubcc0\ud654\ub3c4(gradient)``\n\n\uc2e0\uacbd\ub9dd \uc815\uc758\ud558\uae30\n------------------\n\n\uc774\uc81c \uc2e0\uacbd\ub9dd\uc744 \uc815\uc758\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4:\n\n" + "\n\uc2e0\uacbd\ub9dd(Neural Networks)\n=======================\n\n\uc2e0\uacbd\ub9dd\uc740 ``torch.nn`` \ud328\ud0a4\uc9c0\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc0dd\uc131\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc9c0\uae08\uae4c\uc9c0 ``autograd`` \ub97c \uc0b4\ud3b4\ubd24\ub294\ub370\uc694, ``nn`` \uc740 \ubaa8\ub378\uc744 \uc815\uc758\ud558\uace0 \ubbf8\ubd84\ud558\ub294\ub370\n``autograd`` \ub97c \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n``nn.Module`` \uc740 \uacc4\uce35(layer)\uacfc ``output`` \uc744 \ubc18\ud658\ud558\ub294 ``forward(input)``\n\uba54\uc11c\ub4dc\ub97c \ud3ec\ud568\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc22b\uc790 \uc774\ubbf8\uc9c0\ub97c \ubd84\ub958\ud558\ub294 \uc2e0\uacbd\ub9dd\uc744 \uc608\uc81c\ub85c \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4:\n\n.. figure:: /_static/img/mnist.png\n :alt: convnet\n\n convnet\n\n\uc774\ub294 \uac04\ub2e8\ud55c \uc21c\uc804\ud30c \ub124\ud2b8\uc6cc\ud06c(Feed-forward network)\uc785\ub2c8\ub2e4. \uc785\ub825(input)\uc744 \ubc1b\uc544\n\uc5ec\ub7ec \uacc4\uce35\uc5d0 \ucc28\ub840\ub85c \uc804\ub2ec\ud55c \ud6c4, \ucd5c\uc885 \ucd9c\ub825(output)\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.\n\n\uc2e0\uacbd\ub9dd\uc758 \uc77c\ubc18\uc801\uc778 \ud559\uc2b5 \uacfc\uc815\uc740 \ub2e4\uc74c\uacfc \uac19\uc2b5\ub2c8\ub2e4:\n\n- \ud559\uc2b5 \uac00\ub2a5\ud55c \ub9e4\uac1c\ubcc0\uc218(\ub610\ub294 \uac00\uc911\uce58(weight))\ub97c \uac16\ub294 \uc2e0\uacbd\ub9dd\uc744 \uc815\uc758\ud569\ub2c8\ub2e4.\n- \ub370\uc774\ud130\uc14b(dataset) \uc785\ub825\uc744 \ubc18\ubcf5\ud569\ub2c8\ub2e4.\n- \uc785\ub825\uc744 \uc2e0\uacbd\ub9dd\uc5d0\uc11c \uc804\ud30c(process)\ud569\ub2c8\ub2e4.\n- \uc190\uc2e4(loss; \ucd9c\ub825\uc774 \uc815\ub2f5\uc73c\ub85c\ubd80\ud130 \uc5bc\ub9c8\ub098 \ub5a8\uc5b4\uc838\uc788\ub294\uc9c0)\uc744 \uacc4\uc0b0\ud569\ub2c8\ub2e4.\n- \ubcc0\ud654\ub3c4(gradient)\ub97c \uc2e0\uacbd\ub9dd\uc758 \ub9e4\uac1c\ubcc0\uc218\ub4e4\uc5d0 \uc5ed\uc73c\ub85c \uc804\ud30c\ud569\ub2c8\ub2e4.\n- \uc2e0\uacbd\ub9dd\uc758 \uac00\uc911\uce58\ub97c \uac31\uc2e0\ud569\ub2c8\ub2e4. \uc77c\ubc18\uc801\uc73c\ub85c \ub2e4\uc74c\uacfc \uac19\uc740 \uac04\ub2e8\ud55c \uaddc\uce59\uc744 \uc0ac\uc6a9\ud569\ub2c8\ub2e4:\n ``\uc0c8\ub85c\uc6b4 \uac00\uc911\uce58(weight) = \uac00\uc911\uce58(weight) - \ud559\uc2b5\ub960(learning rate) * \ubcc0\ud654\ub3c4(gradient)``\n\n\uc2e0\uacbd\ub9dd \uc815\uc758\ud558\uae30\n------------------\n\n\uc774\uc81c \uc2e0\uacbd\ub9dd\uc744 \uc815\uc758\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4:\n" ] }, { @@ -179,7 +179,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/c4bf1a4ba1714ace73ad54fe5c6d9d00/pytorch_tutorial.ipynb b/docs/_downloads/c4bf1a4ba1714ace73ad54fe5c6d9d00/pytorch_tutorial.ipynb index f53e3b294..ca4f7dbd8 100644 --- a/docs/_downloads/c4bf1a4ba1714ace73ad54fe5c6d9d00/pytorch_tutorial.ipynb +++ b/docs/_downloads/c4bf1a4ba1714ace73ad54fe5c6d9d00/pytorch_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nIntroduction to PyTorch\n***********************\n\nIntroduction to Torch's tensor library\n======================================\n\nAll of deep learning is computations on tensors, which are\ngeneralizations of a matrix that can be indexed in more than 2\ndimensions. We will see exactly what this means in-depth later. First,\nlets look what we can do with tensors.\n\n" + "\nIntroduction to PyTorch\n***********************\n\nIntroduction to Torch's tensor library\n======================================\n\nAll of deep learning is computations on tensors, which are\ngeneralizations of a matrix that can be indexed in more than 2\ndimensions. We will see exactly what this means in-depth later. First,\nlets look what we can do with tensors.\n" ] }, { @@ -258,7 +258,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/c531680c9d28951567d472785ed937cd/two_layer_net_optim.ipynb b/docs/_downloads/c531680c9d28951567d472785ed937cd/two_layer_net_optim.ipynb index 56f84b05b..0b2cc624e 100644 --- a/docs/_downloads/c531680c9d28951567d472785ed937cd/two_layer_net_optim.ipynb +++ b/docs/_downloads/c531680c9d28951567d472785ed937cd/two_layer_net_optim.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nPyTorch: optim\n--------------\n\n\ud558\ub098\uc758 \uc740\ub2c9\uce35(hidden layer)\uacfc \ud3b8\ud5a5(bias)\uc774 \uc5c6\ub294 \uc644\uc804\ud788 \uc5f0\uacb0\ub41c ReLU \uc2e0\uacbd\ub9dd\uc744,\n\uc720\ud074\ub9ac\ub4dc \uac70\ub9ac(Euclidean distance) \uc81c\uacf1\uc744 \ucd5c\uc18c\ud654\ud558\ub294 \uc2dd\uc73c\ub85c x\ub85c\ubd80\ud130 y\ub97c \uc608\uce21\ud558\ub3c4\ub85d\n\ud559\uc2b5\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc774\ubc88\uc5d0\ub294 PyTorch\uc758 nn \ud328\ud0a4\uc9c0\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc2e0\uacbd\ub9dd\uc744 \uad6c\ud604\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc9c0\uae08\uae4c\uc9c0 \ud574\uc654\ub358 \uac83\ucc98\ub7fc \uc9c1\uc811 \ubaa8\ub378\uc758 \uac00\uc911\uce58\ub97c \uac31\uc2e0\ud558\ub294 \ub300\uc2e0, optim \ud328\ud0a4\uc9c0\ub97c \uc0ac\uc6a9\ud558\uc5ec\n\uac00\uc911\uce58\ub97c \uac31\uc2e0\ud560 Optimizer\ub97c \uc815\uc758\ud569\ub2c8\ub2e4. optim \ud328\ud0a4\uc9c0\ub294 \uc77c\ubc18\uc801\uc73c\ub85c \ub525\ub7ec\ub2dd\uc5d0 \uc0ac\uc6a9\ud558\ub294\nSGD+momentum, RMSProp, Adam \ub4f1\uacfc \uac19\uc740 \ub2e4\uc591\ud55c \ucd5c\uc801\ud654(Optimization) \uc54c\uace0\ub9ac\uc998\uc744\n\uc815\uc758\ud569\ub2c8\ub2e4.\n\n" + "\nPyTorch: optim\n--------------\n\n\ud558\ub098\uc758 \uc740\ub2c9\uce35(hidden layer)\uacfc \ud3b8\ud5a5(bias)\uc774 \uc5c6\ub294 \uc644\uc804\ud788 \uc5f0\uacb0\ub41c ReLU \uc2e0\uacbd\ub9dd\uc744,\n\uc720\ud074\ub9ac\ub4dc \uac70\ub9ac(Euclidean distance) \uc81c\uacf1\uc744 \ucd5c\uc18c\ud654\ud558\ub294 \uc2dd\uc73c\ub85c x\ub85c\ubd80\ud130 y\ub97c \uc608\uce21\ud558\ub3c4\ub85d\n\ud559\uc2b5\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc774\ubc88\uc5d0\ub294 PyTorch\uc758 nn \ud328\ud0a4\uc9c0\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc2e0\uacbd\ub9dd\uc744 \uad6c\ud604\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc9c0\uae08\uae4c\uc9c0 \ud574\uc654\ub358 \uac83\ucc98\ub7fc \uc9c1\uc811 \ubaa8\ub378\uc758 \uac00\uc911\uce58\ub97c \uac31\uc2e0\ud558\ub294 \ub300\uc2e0, optim \ud328\ud0a4\uc9c0\ub97c \uc0ac\uc6a9\ud558\uc5ec\n\uac00\uc911\uce58\ub97c \uac31\uc2e0\ud560 Optimizer\ub97c \uc815\uc758\ud569\ub2c8\ub2e4. optim \ud328\ud0a4\uc9c0\ub294 \uc77c\ubc18\uc801\uc73c\ub85c \ub525\ub7ec\ub2dd\uc5d0 \uc0ac\uc6a9\ud558\ub294\nSGD+momentum, RMSProp, Adam \ub4f1\uacfc \uac19\uc740 \ub2e4\uc591\ud55c \ucd5c\uc801\ud654(Optimization) \uc54c\uace0\ub9ac\uc998\uc744\n\uc815\uc758\ud569\ub2c8\ub2e4.\n" ] }, { @@ -46,7 +46,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/cad5020cab595c3bf83a518b7e4d4125/neural_style_tutorial.ipynb b/docs/_downloads/cad5020cab595c3bf83a518b7e4d4125/neural_style_tutorial.ipynb index 858fc4f20..0e5a59512 100644 --- a/docs/_downloads/cad5020cab595c3bf83a518b7e4d4125/neural_style_tutorial.ipynb +++ b/docs/_downloads/cad5020cab595c3bf83a518b7e4d4125/neural_style_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nNeural Transfer Using PyTorch\n=============================\n\n\n**Author**: `Alexis Jacq `_\n\n**Edited by**: `Winston Herring `_\n\nIntroduction\n------------\n\nThis tutorial explains how to implement the `Neural-Style algorithm `__\ndeveloped by Leon A. Gatys, Alexander S. Ecker and Matthias Bethge.\nNeural-Style, or Neural-Transfer, allows you to take an image and\nreproduce it with a new artistic style. The algorithm takes three images,\nan input image, a content-image, and a style-image, and changes the input\nto resemble the content of the content-image and the artistic style of the style-image.\n\n\n.. figure:: /_static/img/neural-style/neuralstyle.png\n :alt: content1\n\n" + "\nNeural Transfer Using PyTorch\n=============================\n\n\n**Author**: `Alexis Jacq `_\n\n**Edited by**: `Winston Herring `_\n\nIntroduction\n------------\n\nThis tutorial explains how to implement the `Neural-Style algorithm `__\ndeveloped by Leon A. Gatys, Alexander S. Ecker and Matthias Bethge.\nNeural-Style, or Neural-Transfer, allows you to take an image and\nreproduce it with a new artistic style. The algorithm takes three images,\nan input image, a content-image, and a style-image, and changes the input\nto resemble the content of the content-image and the artistic style of the style-image.\n\n\n.. figure:: /_static/img/neural-style/neuralstyle.png\n :alt: content1\n" ] }, { @@ -294,7 +294,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/cd38eeeacb38da1eaaab2cc8c419d0e5/loading_data_recipe.ipynb b/docs/_downloads/cd38eeeacb38da1eaaab2cc8c419d0e5/loading_data_recipe.ipynb index cd9d45cb7..6370465f0 100644 --- a/docs/_downloads/cd38eeeacb38da1eaaab2cc8c419d0e5/loading_data_recipe.ipynb +++ b/docs/_downloads/cd38eeeacb38da1eaaab2cc8c419d0e5/loading_data_recipe.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nPyTorch\uc5d0\uc11c \ub370\uc774\ud130 \ubd88\ub7ec\uc624\uae30\n=======================\nPyTorch\ub294 \uc778\uacf5\uc2e0\uacbd\ub9dd\uc744 \ub9cc\ub4dc\ub294\ub370 \ud544\uc694\ud55c \ub2e4\uc591\ud55c \uae30\ubcf8 \uc694\uc18c\ub97c \uac04\ub2e8\ud558\uace0 \uc9c1\uad00\uc801\uc774\uba70\n\uc548\uc815\uc801\uc778 API\ub85c \uc81c\uacf5\ud569\ub2c8\ub2e4. PyTorch\ub294 \uacf5\uc6a9 \ub370\uc774\ud130\uc14b\uc744 \uc27d\uac8c \uc0ac\uc6a9\ud560 \uc218 \uc788\ub3c4\ub85d\n\ub3c4\uc640\uc8fc\ub294 \ud328\ud0a4\uc9c0\ub97c \ud3ec\ud568\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uac1c\uc694\n------------\nPyTorch \ub370\uc774\ud130 \ubd88\ub7ec\uc624\uae30 \uae30\ub2a5\uc758 \ud575\uc2ec\uc740\n`torch.utils.data.DataLoader `__\n\ud074\ub798\uc2a4\uc785\ub2c8\ub2e4. \ub370\uc774\ud130\ub97c \ud30c\uc774\uc36c iterable\ub85c\uc368 \uc811\uadfc\ud560 \uc218 \uc788\uac8c \ud574\uc8fc\ub294 \ud074\ub798\uc2a4\uc785\ub2c8\ub2e4.\n\ub610\ud55c, `torch.utils.data.Dataset `__\n\ud074\ub798\uc2a4\ub97c \ud1b5\ud574 PyTorch\uc5d0 \ub0b4\uc7a5\ub41c \ub2e4\uc591\ud55c \uace0\ud488\uc9c8 \ub370\uc774\ud130\uc14b\uc744 \uc774\uc6a9\ud558\uc2e4 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\n\uac1c\ubcc4 \ub370\uc774\ud130\uc14b\uc740 \uc544\ub798 \ud328\ud0a4\uc9c0\uc5d0\uc11c \ud655\uc778\ud558\uc2e4 \uc218 \uc788\uc73c\uba70, \ub370\uc774\ud130\uc14b\uc740 \uacc4\uc18d\ud574\uc11c \ucd94\uac00\ub420 \uc608\uc815\uc785\ub2c8\ub2e4.\n\n* `torchvision `__\n* `torchaudio `__\n* `torchtext `__\n\n\n\uc774\ubc88 \ub808\uc2dc\ud53c\uc5d0\uc11c\ub294 ``torchaudio.datasets.YESNO`` \ub370\uc774\ud130\uc14b\uc744 \uc0b4\ud3b4\ubcf4\uba74\uc11c,\nPyTorch ``Dataset`` \uc5d0\uc11c PyTorch ``DataLoader`` \ub85c \ub370\uc774\ud130\ub97c \ud6a8\uacfc\uc801\uc774\uace0 \ud6a8\uc728\uc801\uc73c\ub85c\n\ubd88\ub7ec\uc624\ub294 \ubc29\ubc95\uc744 \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\ucd08\uae30 \uc124\uc815\n-----\n\uc2dc\uc791\ud558\uae30 \uc804\uc5d0, \ub370\uc774\ud130\uc14b\uc774 \ud3ec\ud568\ub41c ``torchaudio`` \ud328\ud0a4\uc9c0\ub97c \uc124\uce58\ud569\ub2c8\ub2e4.\n\n::\n\n pip install torchaudio\n\n\n\n" + "\nPyTorch\uc5d0\uc11c \ub370\uc774\ud130 \ubd88\ub7ec\uc624\uae30\n=======================\nPyTorch\ub294 \uc778\uacf5\uc2e0\uacbd\ub9dd\uc744 \ub9cc\ub4dc\ub294\ub370 \ud544\uc694\ud55c \ub2e4\uc591\ud55c \uae30\ubcf8 \uc694\uc18c\ub97c \uac04\ub2e8\ud558\uace0 \uc9c1\uad00\uc801\uc774\uba70\n\uc548\uc815\uc801\uc778 API\ub85c \uc81c\uacf5\ud569\ub2c8\ub2e4. PyTorch\ub294 \uacf5\uc6a9 \ub370\uc774\ud130\uc14b\uc744 \uc27d\uac8c \uc0ac\uc6a9\ud560 \uc218 \uc788\ub3c4\ub85d\n\ub3c4\uc640\uc8fc\ub294 \ud328\ud0a4\uc9c0\ub97c \ud3ec\ud568\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uac1c\uc694\n------------\nPyTorch \ub370\uc774\ud130 \ubd88\ub7ec\uc624\uae30 \uae30\ub2a5\uc758 \ud575\uc2ec\uc740\n`torch.utils.data.DataLoader `__\n\ud074\ub798\uc2a4\uc785\ub2c8\ub2e4. \ub370\uc774\ud130\ub97c \ud30c\uc774\uc36c iterable\ub85c\uc368 \uc811\uadfc\ud560 \uc218 \uc788\uac8c \ud574\uc8fc\ub294 \ud074\ub798\uc2a4\uc785\ub2c8\ub2e4.\n\ub610\ud55c, `torch.utils.data.Dataset `__\n\ud074\ub798\uc2a4\ub97c \ud1b5\ud574 PyTorch\uc5d0 \ub0b4\uc7a5\ub41c \ub2e4\uc591\ud55c \uace0\ud488\uc9c8 \ub370\uc774\ud130\uc14b\uc744 \uc774\uc6a9\ud558\uc2e4 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\n\uac1c\ubcc4 \ub370\uc774\ud130\uc14b\uc740 \uc544\ub798 \ud328\ud0a4\uc9c0\uc5d0\uc11c \ud655\uc778\ud558\uc2e4 \uc218 \uc788\uc73c\uba70, \ub370\uc774\ud130\uc14b\uc740 \uacc4\uc18d\ud574\uc11c \ucd94\uac00\ub420 \uc608\uc815\uc785\ub2c8\ub2e4.\n\n* `torchvision `__\n* `torchaudio `__\n* `torchtext `__\n\n\n\uc774\ubc88 \ub808\uc2dc\ud53c\uc5d0\uc11c\ub294 ``torchaudio.datasets.YESNO`` \ub370\uc774\ud130\uc14b\uc744 \uc0b4\ud3b4\ubcf4\uba74\uc11c,\nPyTorch ``Dataset`` \uc5d0\uc11c PyTorch ``DataLoader`` \ub85c \ub370\uc774\ud130\ub97c \ud6a8\uacfc\uc801\uc774\uace0 \ud6a8\uc728\uc801\uc73c\ub85c\n\ubd88\ub7ec\uc624\ub294 \ubc29\ubc95\uc744 \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\ucd08\uae30 \uc124\uc815\n-----\n\uc2dc\uc791\ud558\uae30 \uc804\uc5d0, \ub370\uc774\ud130\uc14b\uc774 \ud3ec\ud568\ub41c ``torchaudio`` \ud328\ud0a4\uc9c0\ub97c \uc124\uce58\ud569\ub2c8\ub2e4.\n\n::\n\n pip install torchaudio\n" ] }, { @@ -132,7 +132,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/ce0d3d42f46408ae910553ca370054c0/graph_mode_static_quantization_tutorial.ipynb b/docs/_downloads/ce0d3d42f46408ae910553ca370054c0/graph_mode_static_quantization_tutorial.ipynb index 79edcee36..f2f04e268 100644 --- a/docs/_downloads/ce0d3d42f46408ae910553ca370054c0/graph_mode_static_quantization_tutorial.ipynb +++ b/docs/_downloads/ce0d3d42f46408ae910553ca370054c0/graph_mode_static_quantization_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n(prototype) Graph Mode Post Training Static Quantization in PyTorch\n=========================================================\n\n**Author**: `Jerry Zhang `_\n\nThis tutorial introduces the steps to do post training static quantization in graph mode. \nThe advantage of graph mode quantization is that as long as the model can be scripted or traced, \nwe can perform quantization fully automatically on the model. \nRight now we can do post training static and post training dynamic quantization \nand quantization aware training support will come later. \nWe have a separate tutorial for `Graph Mode Post Training Dynamic Quantization `_.\n\ntldr; The graph mode API looks like the following:\n\n.. code:: python\n\n import torch\n from torch.quantization import get_default_qconfig, quantize_jit\n \n ts_model = torch.jit.script(float_model.eval()) # or torch.jit.trace(float_model, input)\n qconfig = get_default_qconfig('fbgemm')\n def calibrate(model, data_loader):\n model.eval()\n with torch.no_grad():\n for image, target in data_loader:\n model(image)\n quantized_model = quantize_jit(\n ts_model, # TorchScript model\n {'': qconfig}, # qconfig dict\n calibrate, # calibration function\n [data_loader_test]) # positional arguments to calibration function, typically some sample dataset\n\n\n" + "\n(prototype) Graph Mode Post Training Static Quantization in PyTorch\n=========================================================\n\n**Author**: `Jerry Zhang `_\n\nThis tutorial introduces the steps to do post training static quantization in graph mode. \nThe advantage of graph mode quantization is that as long as the model can be scripted or traced, \nwe can perform quantization fully automatically on the model. \nRight now we can do post training static and post training dynamic quantization \nand quantization aware training support will come later. \nWe have a separate tutorial for `Graph Mode Post Training Dynamic Quantization `_.\n\ntldr; The graph mode API looks like the following:\n\n.. code:: python\n\n import torch\n from torch.quantization import get_default_qconfig, quantize_jit\n \n ts_model = torch.jit.script(float_model.eval()) # or torch.jit.trace(float_model, input)\n qconfig = get_default_qconfig('fbgemm')\n def calibrate(model, data_loader):\n model.eval()\n with torch.no_grad():\n for image, target in data_loader:\n model(image)\n quantized_model = quantize_jit(\n ts_model, # TorchScript model\n {'': qconfig}, # qconfig dict\n calibrate, # calibration function\n [data_loader_test]) # positional arguments to calibration function, typically some sample dataset\n" ] }, { @@ -258,7 +258,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/d493dae89f8804b07cdf678f7d0c2dc6/tensorboard_with_pytorch.ipynb b/docs/_downloads/d493dae89f8804b07cdf678f7d0c2dc6/tensorboard_with_pytorch.ipynb index 55bbb9cd4..fa43e8748 100644 --- a/docs/_downloads/d493dae89f8804b07cdf678f7d0c2dc6/tensorboard_with_pytorch.ipynb +++ b/docs/_downloads/d493dae89f8804b07cdf678f7d0c2dc6/tensorboard_with_pytorch.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nPyTorch\ub85c TensorBoard \uc0ac\uc6a9\ud558\uae30\n===================================\nTensorBoard\ub294 \uba38\uc2e0\ub7ec\ub2dd \uc2e4\ud5d8\uc744 \uc704\ud55c \uc2dc\uac01\ud654 \ud234\ud0b7(toolkit)\uc785\ub2c8\ub2e4.\nTensorBoard\ub97c \uc0ac\uc6a9\ud558\uba74 \uc190\uc2e4 \ubc0f \uc815\ud655\ub3c4\uc640 \uac19\uc740 \uce21\uc815 \ud56d\ubaa9\uc744 \ucd94\uc801 \ubc0f \uc2dc\uac01\ud654\ud558\ub294 \uac83,\n\ubaa8\ub378 \uadf8\ub798\ud504\ub97c \uc2dc\uac01\ud654\ud558\ub294 \uac83, \ud788\uc2a4\ud1a0\uadf8\ub7a8\uc744 \ubcf4\ub294 \uac83, \uc774\ubbf8\uc9c0\ub97c \ucd9c\ub825\ud558\ub294 \uac83 \ub4f1\uc774 \uac00\ub2a5\ud569\ub2c8\ub2e4.\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 TensorBoard \uc124\uce58, PyTorch\uc758 \uae30\ubcf8 \uc0ac\uc6a9\ubc95,\nTensorBoard UI\uc5d0 \uae30\ub85d\ud55c \ub370\uc774\ud130\ub97c \uc2dc\uac01\ud654 \ud558\ub294 \ubc29\ubc95\uc744 \ub2e4\ub8f0 \uac83\uc785\ub2c8\ub2e4.\n\n\uc124\uce58\ud558\uae30\n----------------------\n\ubaa8\ub378\uacfc \uce21\uc815 \ud56d\ubaa9\uc744 TensorBoard \ub85c\uadf8 \ub514\ub809\ud130\ub9ac\uc5d0 \uae30\ub85d\ud558\ub824\uba74 PyTorch\ub97c \uc124\uce58\ud574\uc57c \ud569\ub2c8\ub2e4.\nAnaconda\ub97c \ud1b5\ud574 PyTorch 1.4 \uc774\uc0c1\uc744 \uc124\uce58\ud558\ub294 \ubc29\ubc95\uc740 \ub2e4\uc74c\uacfc \uac19\uc2b5\ub2c8\ub2e4.(\uad8c\uc7a5):\n::\n\n $ conda install pytorch torchvision -c pytorch \n \n\n\ub610\ub294 pip\ub97c \uc0ac\uc6a9\ud560 \uc218\ub3c4 \uc788\uc2b5\ub2c8\ub2e4.\n\n::\n\n $ pip install torch torchvision\n\n\n" + "\nPyTorch\ub85c TensorBoard \uc0ac\uc6a9\ud558\uae30\n===================================\nTensorBoard\ub294 \uba38\uc2e0\ub7ec\ub2dd \uc2e4\ud5d8\uc744 \uc704\ud55c \uc2dc\uac01\ud654 \ud234\ud0b7(toolkit)\uc785\ub2c8\ub2e4.\nTensorBoard\ub97c \uc0ac\uc6a9\ud558\uba74 \uc190\uc2e4 \ubc0f \uc815\ud655\ub3c4\uc640 \uac19\uc740 \uce21\uc815 \ud56d\ubaa9\uc744 \ucd94\uc801 \ubc0f \uc2dc\uac01\ud654\ud558\ub294 \uac83,\n\ubaa8\ub378 \uadf8\ub798\ud504\ub97c \uc2dc\uac01\ud654\ud558\ub294 \uac83, \ud788\uc2a4\ud1a0\uadf8\ub7a8\uc744 \ubcf4\ub294 \uac83, \uc774\ubbf8\uc9c0\ub97c \ucd9c\ub825\ud558\ub294 \uac83 \ub4f1\uc774 \uac00\ub2a5\ud569\ub2c8\ub2e4.\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 TensorBoard \uc124\uce58, PyTorch\uc758 \uae30\ubcf8 \uc0ac\uc6a9\ubc95,\nTensorBoard UI\uc5d0 \uae30\ub85d\ud55c \ub370\uc774\ud130\ub97c \uc2dc\uac01\ud654 \ud558\ub294 \ubc29\ubc95\uc744 \ub2e4\ub8f0 \uac83\uc785\ub2c8\ub2e4.\n\n\uc124\uce58\ud558\uae30\n----------------------\n\ubaa8\ub378\uacfc \uce21\uc815 \ud56d\ubaa9\uc744 TensorBoard \ub85c\uadf8 \ub514\ub809\ud130\ub9ac\uc5d0 \uae30\ub85d\ud558\ub824\uba74 PyTorch\ub97c \uc124\uce58\ud574\uc57c \ud569\ub2c8\ub2e4.\nAnaconda\ub97c \ud1b5\ud574 PyTorch 1.4 \uc774\uc0c1\uc744 \uc124\uce58\ud558\ub294 \ubc29\ubc95\uc740 \ub2e4\uc74c\uacfc \uac19\uc2b5\ub2c8\ub2e4.(\uad8c\uc7a5):\n::\n\n $ conda install pytorch torchvision -c pytorch \n \n\n\ub610\ub294 pip\ub97c \uc0ac\uc6a9\ud560 \uc218\ub3c4 \uc788\uc2b5\ub2c8\ub2e4.\n\n::\n\n $ pip install torch torchvision\n" ] }, { @@ -117,7 +117,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/d90127b0deeb355be3a350521d770206/chatbot_tutorial.py b/docs/_downloads/d90127b0deeb355be3a350521d770206/chatbot_tutorial.py index 6db571cf6..bcce89918 100644 --- a/docs/_downloads/d90127b0deeb355be3a350521d770206/chatbot_tutorial.py +++ b/docs/_downloads/d90127b0deeb355be3a350521d770206/chatbot_tutorial.py @@ -1331,7 +1331,7 @@ def evaluateInput(encoder, decoder, searcher, voc): for k, v in state.items(): if isinstance(v, torch.Tensor): state[k] = v.cuda() - + # 학습 단계를 수행합니다 print("Starting Training!") trainIters(model_name, voc, pairs, encoder, decoder, encoder_optimizer, decoder_optimizer, diff --git a/docs/_downloads/d9398fce39ca80dc4bb8b8ea55b575a8/nn_tutorial.ipynb b/docs/_downloads/d9398fce39ca80dc4bb8b8ea55b575a8/nn_tutorial.ipynb index 3e34c6157..90d5d838c 100644 --- a/docs/_downloads/d9398fce39ca80dc4bb8b8ea55b575a8/nn_tutorial.ipynb +++ b/docs/_downloads/d9398fce39ca80dc4bb8b8ea55b575a8/nn_tutorial.ipynb @@ -15,14 +15,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n`torch.nn` \uc774 *\uc2e4\uc81c\ub85c* \ubb34\uc5c7\uc778\uac00\uc694?\n===============================\n\uc800\uc790: Jeremy Howard, `fast.ai `_.\n\n\ub3c4\uc6c0: Rachel Thomas, Francisco Ingham.\n\n\ubc88\uc5ed: `\ub0a8\uc0c1\ud638 `_\n\n" + "\n`torch.nn` \uc774 *\uc2e4\uc81c\ub85c* \ubb34\uc5c7\uc778\uac00\uc694?\n=====================================\n\uc800\uc790: Jeremy Howard, `fast.ai `_.\n\n\ub3c4\uc6c0: Rachel Thomas, Francisco Ingham.\n\n\ubc88\uc5ed: `\ub0a8\uc0c1\ud638 `_\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc744 \uc2a4\ud06c\ub9bd\ud2b8\uac00 \uc544\ub2cc \ub178\ud2b8\ubd81\uc73c\ub85c \uc2e4\ud589\ud558\uae30\ub97c \uad8c\uc7a5\ud569\ub2c8\ub2e4. \ub178\ud2b8\ubd81 (.ipynb) \ud30c\uc77c\uc744 \ub2e4\uc6b4 \ubc1b\uc73c\uc2dc\ub824\uba74,\n\ud398\uc774\uc9c0 \uc0c1\ub2e8\uc5d0 \uc788\ub294 \ub9c1\ud06c\ub97c \ud074\ub9ad\ud574\uc8fc\uc138\uc694.\n\nPyTorch \ub294 \uc5ec\ub7ec\ubd84\uc774 \uc2e0\uacbd\ub9dd(neural network)\ub97c \uc0dd\uc131\ud558\uace0 \ud559\uc2b5\uc2dc\ud0a4\ub294 \uac83\uc744 \ub3c4\uc640\uc8fc\uae30 \uc704\ud574\uc11c\n`torch.nn `_ ,\n`torch.optim `_ ,\n`Dataset `_ ,\n\uadf8\ub9ac\uace0 `DataLoader `_\n\uc640 \uac19\uc740 \uc798 \ub514\uc790\uc778\ub41c \ubaa8\ub4c8\uacfc \ud074\ub798\uc2a4\ub4e4\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.\n\uc774\ub4e4\uc758 \uc131\ub2a5\uc744 \ucd5c\ub300\ud55c \ud65c\uc6a9\ud558\uace0 \uc5ec\ub7ec\ubd84\uc758 \ubb38\uc81c\uc5d0 \ub9de\uac8c \ucee4\uc2a4\ud130\ub9c8\uc774\uc988\ud558\uae30 \uc704\ud574\uc11c,\n\uc815\ud655\ud788 \uc774\ub4e4\uc774 \uc5b4\ub5a4 \uc791\uc5c5\uc744 \uc218\ud589\ud558\ub294\uc9c0 \uc774\ud574\ud560 \ud544\uc694\uac00 \uc788\uc2b5\ub2c8\ub2e4.\n\uc774\ud574\ub97c \uc99d\uc9c4\ud558\uae30 \uc704\ud574\uc11c, \uc6b0\ub9ac\ub294 \uba3c\uc800 \uc774\ub4e4 \ubaa8\ub378\ub4e4\ub85c \ubd80\ud130 \uc544\ubb34 \ud53c\uccd0\ub3c4 \uc0ac\uc6a9\ud558\uc9c0 \uc54a\uace0\nMNIST \ub370\uc774\ud130\uc14b\uc5d0 \ub300\ud574 \uae30\ucd08\uc801\uc778 \uc2e0\uacbd\ub9dd\uc744 \ud559\uc2b5\uc2dc\ud0ac \uac83\uc785\ub2c8\ub2e4;\n\uc6b0\ub9ac\ub294 \ucc98\uc74c\uc5d0\ub294 \uac00\uc7a5 \uae30\ucd08\uc801\uc778 PyTorch \ud150\uc11c(tensor) \uae30\ub2a5\ub9cc\uc744 \uc0ac\uc6a9\ud560 \uac83\uc785\ub2c8\ub2e4.\n\uadf8\ub9ac\uace0\ub098\uc11c \uc6b0\ub9ac\ub294 \uc810\ucc28\uc801\uc73c\ub85c ``torch.nn``, ``torch.optim``, ``Dataset``, \ub610\ub294\n``DataLoader`` \ub85c\ubd80\ud130 \ud55c\ubc88\uc5d0 \ud558\ub098\uc529 \ud53c\uccd0\ub97c \ucd94\uac00\ud558\uba74\uc11c, \uc815\ud655\ud788 \uac01 \ubd80\ubd84\uc774 \uc5b4\ub5a4 \uc77c\uc744 \ud558\ub294\uc9c0 \uadf8\ub9ac\uace0\n\uc774\uac83\uc774 \uc5b4\ub5bb\uac8c \ucf54\ub4dc\ub97c \ub354 \uc815\ud655\ud558\uace0 \uc720\uc5f0\ud558\uac8c \ub9cc\ub4dc\ub294\uc9c0 \ubcf4\uc5ec\uc904 \uac83\uc785\ub2c8\ub2e4.\n\n**\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc740 \uc5ec\ub7ec\ubd84\uc774 \uc774\ubbf8 PyTorch\ub97c \uc124\uce58\ud558\uc600\uace0, \uadf8\ub9ac\uace0 \ud150\uc11c \uc5f0\uc0b0\uc758 \uae30\ucd08\uc5d0 \ub300\ud574 \uc775\uc219\ud558\ub2e4\uace0 \uac00\uc815\ud569\ub2c8\ub2e4.**\n(\ub9cc\uc57d \uc5ec\ub7ec\ubd84\uc774 Numpy \ubc30\uc5f4(array) \uc5f0\uc0b0\uc5d0 \uc775\uc219\ud558\ub2e4\uba74, \uc5ec\uae30\uc5d0\uc11c \uc0ac\uc6a9\ub418\ub294 PyTorch \ud150\uc11c \uc5f0\uc0b0\ub3c4\n\uac70\uc758 \ub3d9\uc77c\ud558\ub2e4\ub294 \uac83\uc744 \uc54c\uac8c \ub420 \uac83\uc785\ub2c8\ub2e4).\n\nMNIST \ub370\uc774\ud130 \uc900\ube44\n----------------\n\n\uc6b0\ub9ac\ub294 \uc190\uc73c\ub85c \uc4f4 \uc22b\uc790(0\uc5d0\uc11c 9 \uc0ac\uc774)\uc758 \ud751\ubc31 \uc774\ubbf8\uc9c0\ub85c \uad6c\uc131\ub41c \ud074\ub798\uc2dd\n`MNIST `_ \ub370\uc774\ud130\uc14b\uc744 \uc0ac\uc6a9\ud560 \uac83 \uc785\ub2c8\ub2e4.\n\n\uc6b0\ub9ac\ub294 \uacbd\ub85c \uc124\uc815\uc744 \ub2f4\ub2f9\ud558\ub294 (Python3 \ud45c\uc900 \ub77c\uc774\ube0c\ub7ec\ub9ac\uc758 \uc77c\ubd80\uc778)\n`pathlib `_ \uc744 \uc0ac\uc6a9\ud560 \uac83\uc774\uace0,\n`requests `_ \ub97c \uc774\uc6a9\ud558\uc5ec\n\ub370\uc774\ud130\uc14b\uc744 \ub2e4\uc6b4\ub85c\ub4dc \ud560 \uac83\uc785\ub2c8\ub2e4. \uc6b0\ub9ac\ub294 \ubaa8\ub4c8\uc744 \uc0ac\uc6a9\ud560 \ub54c\ub9cc \uc784\ud3ec\ud2b8(import) \ud560 \uac83\uc774\ubbc0\ub85c,\n\uc5ec\ub7ec\ubd84\uc740 \ub9e4 \ud3ec\uc778\ud2b8\ub9c8\ub2e4 \uc815\ud655\ud788 \uc5b4\ub5a4 \uac83\uc774 \uc0ac\uc6a9\ub418\ub294\uc9c0 \ud655\uc778\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n" + "\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc744 \uc2a4\ud06c\ub9bd\ud2b8\uac00 \uc544\ub2cc \ub178\ud2b8\ubd81\uc73c\ub85c \uc2e4\ud589\ud558\uae30\ub97c \uad8c\uc7a5\ud569\ub2c8\ub2e4. \ub178\ud2b8\ubd81 (.ipynb) \ud30c\uc77c\uc744 \ub2e4\uc6b4 \ubc1b\uc73c\uc2dc\ub824\uba74,\n\ud398\uc774\uc9c0 \uc0c1\ub2e8\uc5d0 \uc788\ub294 \ub9c1\ud06c\ub97c \ud074\ub9ad\ud574\uc8fc\uc138\uc694.\n\nPyTorch \ub294 \uc5ec\ub7ec\ubd84\uc774 \uc2e0\uacbd\ub9dd(neural network)\ub97c \uc0dd\uc131\ud558\uace0 \ud559\uc2b5\uc2dc\ud0a4\ub294 \uac83\uc744 \ub3c4\uc640\uc8fc\uae30 \uc704\ud574\uc11c\n`torch.nn `_ ,\n`torch.optim `_ ,\n`Dataset `_ ,\n\uadf8\ub9ac\uace0 `DataLoader `_\n\uc640 \uac19\uc740 \uc798 \ub514\uc790\uc778\ub41c \ubaa8\ub4c8\uacfc \ud074\ub798\uc2a4\ub4e4\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.\n\uc774\ub4e4\uc758 \uc131\ub2a5\uc744 \ucd5c\ub300\ud55c \ud65c\uc6a9\ud558\uace0 \uc5ec\ub7ec\ubd84\uc758 \ubb38\uc81c\uc5d0 \ub9de\uac8c \ucee4\uc2a4\ud130\ub9c8\uc774\uc988\ud558\uae30 \uc704\ud574\uc11c,\n\uc815\ud655\ud788 \uc774\ub4e4\uc774 \uc5b4\ub5a4 \uc791\uc5c5\uc744 \uc218\ud589\ud558\ub294\uc9c0 \uc774\ud574\ud560 \ud544\uc694\uac00 \uc788\uc2b5\ub2c8\ub2e4.\n\uc774\ud574\ub97c \uc99d\uc9c4\ud558\uae30 \uc704\ud574\uc11c, \uc6b0\ub9ac\ub294 \uba3c\uc800 \uc774\ub4e4 \ubaa8\ub378\ub4e4\ub85c \ubd80\ud130 \uc544\ubb34 \ud53c\uccd0\ub3c4 \uc0ac\uc6a9\ud558\uc9c0 \uc54a\uace0\nMNIST \ub370\uc774\ud130\uc14b\uc5d0 \ub300\ud574 \uae30\ucd08\uc801\uc778 \uc2e0\uacbd\ub9dd\uc744 \ud559\uc2b5\uc2dc\ud0ac \uac83\uc785\ub2c8\ub2e4;\n\uc6b0\ub9ac\ub294 \ucc98\uc74c\uc5d0\ub294 \uac00\uc7a5 \uae30\ucd08\uc801\uc778 PyTorch \ud150\uc11c(tensor) \uae30\ub2a5\ub9cc\uc744 \uc0ac\uc6a9\ud560 \uac83\uc785\ub2c8\ub2e4.\n\uadf8\ub9ac\uace0\ub098\uc11c \uc6b0\ub9ac\ub294 \uc810\ucc28\uc801\uc73c\ub85c ``torch.nn``, ``torch.optim``, ``Dataset``, \ub610\ub294\n``DataLoader`` \ub85c\ubd80\ud130 \ud55c\ubc88\uc5d0 \ud558\ub098\uc529 \ud53c\uccd0\ub97c \ucd94\uac00\ud558\uba74\uc11c, \uc815\ud655\ud788 \uac01 \ubd80\ubd84\uc774 \uc5b4\ub5a4 \uc77c\uc744 \ud558\ub294\uc9c0 \uadf8\ub9ac\uace0\n\uc774\uac83\uc774 \uc5b4\ub5bb\uac8c \ucf54\ub4dc\ub97c \ub354 \uc815\ud655\ud558\uace0 \uc720\uc5f0\ud558\uac8c \ub9cc\ub4dc\ub294\uc9c0 \ubcf4\uc5ec\uc904 \uac83\uc785\ub2c8\ub2e4.\n\n**\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc740 \uc5ec\ub7ec\ubd84\uc774 \uc774\ubbf8 PyTorch\ub97c \uc124\uce58\ud558\uc600\uace0, \uadf8\ub9ac\uace0 \ud150\uc11c \uc5f0\uc0b0\uc758 \uae30\ucd08\uc5d0 \ub300\ud574 \uc775\uc219\ud558\ub2e4\uace0 \uac00\uc815\ud569\ub2c8\ub2e4.**\n(\ub9cc\uc57d \uc5ec\ub7ec\ubd84\uc774 Numpy \ubc30\uc5f4(array) \uc5f0\uc0b0\uc5d0 \uc775\uc219\ud558\ub2e4\uba74, \uc5ec\uae30\uc5d0\uc11c \uc0ac\uc6a9\ub418\ub294 PyTorch \ud150\uc11c \uc5f0\uc0b0\ub3c4\n\uac70\uc758 \ub3d9\uc77c\ud558\ub2e4\ub294 \uac83\uc744 \uc54c\uac8c \ub420 \uac83\uc785\ub2c8\ub2e4).\n\nMNIST \ub370\uc774\ud130 \uc900\ube44\n-------------------\n\n\uc6b0\ub9ac\ub294 \uc190\uc73c\ub85c \uc4f4 \uc22b\uc790(0\uc5d0\uc11c 9 \uc0ac\uc774)\uc758 \ud751\ubc31 \uc774\ubbf8\uc9c0\ub85c \uad6c\uc131\ub41c \ud074\ub798\uc2dd\n`MNIST `_ \ub370\uc774\ud130\uc14b\uc744 \uc0ac\uc6a9\ud560 \uac83 \uc785\ub2c8\ub2e4.\n\n\uc6b0\ub9ac\ub294 \uacbd\ub85c \uc124\uc815\uc744 \ub2f4\ub2f9\ud558\ub294 (Python3 \ud45c\uc900 \ub77c\uc774\ube0c\ub7ec\ub9ac\uc758 \uc77c\ubd80\uc778)\n`pathlib `_ \uc744 \uc0ac\uc6a9\ud560 \uac83\uc774\uace0,\n`requests `_ \ub97c \uc774\uc6a9\ud558\uc5ec\n\ub370\uc774\ud130\uc14b\uc744 \ub2e4\uc6b4\ub85c\ub4dc \ud560 \uac83\uc785\ub2c8\ub2e4. \uc6b0\ub9ac\ub294 \ubaa8\ub4c8\uc744 \uc0ac\uc6a9\ud560 \ub54c\ub9cc \uc784\ud3ec\ud2b8(import) \ud560 \uac83\uc774\ubbc0\ub85c,\n\uc5ec\ub7ec\ubd84\uc740 \ub9e4 \ud3ec\uc778\ud2b8\ub9c8\ub2e4 \uc815\ud655\ud788 \uc5b4\ub5a4 \uac83\uc774 \uc0ac\uc6a9\ub418\ub294\uc9c0 \ud655\uc778\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n" ] }, { @@ -33,7 +33,7 @@ }, "outputs": [], "source": [ - "from pathlib import Path\nimport requests\n\nDATA_PATH = Path(\"data\")\nPATH = DATA_PATH / \"mnist\"\n\nPATH.mkdir(parents=True, exist_ok=True)\n\nURL = \"http://deeplearning.net/data/mnist/\"\nFILENAME = \"mnist.pkl.gz\"\n\nif not (PATH / FILENAME).exists():\n content = requests.get(URL + FILENAME).content\n (PATH / FILENAME).open(\"wb\").write(content)" + "from pathlib import Path\nimport requests\n\nDATA_PATH = Path(\"data\")\nPATH = DATA_PATH / \"mnist\"\n\nPATH.mkdir(parents=True, exist_ok=True)\n\nURL = \"https://github.com/pytorch/tutorials/raw/master/_static/\"\nFILENAME = \"mnist.pkl.gz\"\n\nif not (PATH / FILENAME).exists():\n content = requests.get(URL + FILENAME).content\n (PATH / FILENAME).open(\"wb\").write(content)" ] }, { @@ -292,7 +292,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "nn.Module \uc744 \uc774\uc6a9\ud558\uc5ec \ub9ac\ud329\ud1a0\ub9c1 \ud558\uae30\n------------------------------\n\ub2e4\uc74c\uc73c\ub85c, \ub354 \uba85\ud655\ud558\uace0 \uac04\uacb0\ud55c \ud6c8\ub828 \ub8e8\ud504\ub97c \uc704\ud574 ``nn.Module`` \ubc0f ``nn.Parameter`` \ub97c \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n\uc6b0\ub9ac\ub294 ``nn.Module`` (\uc790\uccb4\uac00 \ud074\ub798\uc2a4\uc774\uace0 \uc0c1\ud0dc\ub97c \ucd94\ucc99\ud560 \uc218 \uc788\ub294) \ud558\uc704 \ud074\ub798\uc2a4(subclass)\ub97c \ub9cc\ub4ed\ub2c8\ub2e4.\n\uc774 \uacbd\uc6b0\uc5d0\ub294, \ud3ec\uc6cc\ub4dc(forward) \ub2e8\uacc4\uc5d0 \ub300\ud55c \uac00\uc911\uce58, \uc808\ud3b8, \uadf8\ub9ac\uace0 \uba54\uc18c\ub4dc(method) \ub4f1\uc744 \uc720\uc9c0\ud558\ub294\n\ud074\ub798\uc2a4\ub97c \ub9cc\ub4e4\uace0\uc790 \ud569\ub2c8\ub2e4.\n``nn.Module`` \uc740 \uc6b0\ub9ac\uac00 \uc0ac\uc6a9\ud560 \uba87 \uac00\uc9c0 \uc18d\uc131(attribute)\uacfc \uba54\uc18c\ub4dc\ub97c (``.parameters()`` \uc640\n``.zero_grad()`` \uac19\uc740) \uac00\uc9c0\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n\n

Note

``nn.Module`` (\ub300\ubb38\uc790 M) \uc740 PyTorch \uc758 \ud2b9\uc815 \uac1c\ub150\uc774\uace0, \uc6b0\ub9ac\ub294 \uc774 \ud074\ub798\uc2a4\ub97c\n \ub9ce\uc774 \uc0ac\uc6a9\ud560 \uac83\uc785\ub2c8\ub2e4. ``nn.Module`` \ub97c Python \uc758 \ucf54\ub4dc\ub97c \uc784\ud3ec\ud2b8\ud558\uae30 \uc704\ud55c \ucf54\ub4dc \ud30c\uc77c\uc778\n `module `_ (\uc18c\ubb38\uc790 ``m``)\n \uc758 \uac1c\ub150\uacfc \ud5f7\uac08\ub9ac\uc9c0 \ub9d0\uc544\uc8fc\uc138\uc694.

\n\n" + "nn.Module \uc744 \uc774\uc6a9\ud558\uc5ec \ub9ac\ud329\ud1a0\ub9c1 \ud558\uae30\n--------------------------------------\n\ub2e4\uc74c\uc73c\ub85c, \ub354 \uba85\ud655\ud558\uace0 \uac04\uacb0\ud55c \ud6c8\ub828 \ub8e8\ud504\ub97c \uc704\ud574 ``nn.Module`` \ubc0f ``nn.Parameter`` \ub97c \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n\uc6b0\ub9ac\ub294 ``nn.Module`` (\uc790\uccb4\uac00 \ud074\ub798\uc2a4\uc774\uace0 \uc0c1\ud0dc\ub97c \ucd94\ucc99\ud560 \uc218 \uc788\ub294) \ud558\uc704 \ud074\ub798\uc2a4(subclass)\ub97c \ub9cc\ub4ed\ub2c8\ub2e4.\n\uc774 \uacbd\uc6b0\uc5d0\ub294, \ud3ec\uc6cc\ub4dc(forward) \ub2e8\uacc4\uc5d0 \ub300\ud55c \uac00\uc911\uce58, \uc808\ud3b8, \uadf8\ub9ac\uace0 \uba54\uc18c\ub4dc(method) \ub4f1\uc744 \uc720\uc9c0\ud558\ub294\n\ud074\ub798\uc2a4\ub97c \ub9cc\ub4e4\uace0\uc790 \ud569\ub2c8\ub2e4.\n``nn.Module`` \uc740 \uc6b0\ub9ac\uac00 \uc0ac\uc6a9\ud560 \uba87 \uac00\uc9c0 \uc18d\uc131(attribute)\uacfc \uba54\uc18c\ub4dc\ub97c (``.parameters()`` \uc640\n``.zero_grad()`` \uac19\uc740) \uac00\uc9c0\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n\n

Note

``nn.Module`` (\ub300\ubb38\uc790 M) \uc740 PyTorch \uc758 \ud2b9\uc815 \uac1c\ub150\uc774\uace0, \uc6b0\ub9ac\ub294 \uc774 \ud074\ub798\uc2a4\ub97c\n \ub9ce\uc774 \uc0ac\uc6a9\ud560 \uac83\uc785\ub2c8\ub2e4. ``nn.Module`` \ub97c Python \uc758 \ucf54\ub4dc\ub97c \uc784\ud3ec\ud2b8\ud558\uae30 \uc704\ud55c \ucf54\ub4dc \ud30c\uc77c\uc778\n `module `_ (\uc18c\ubb38\uc790 ``m``)\n \uc758 \uac1c\ub150\uacfc \ud5f7\uac08\ub9ac\uc9c0 \ub9d0\uc544\uc8fc\uc138\uc694.

\n\n" ] }, { @@ -346,7 +346,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uc774\uc804\uc5d0\ub294 \ud6c8\ub828 \ub8e8\ud504\ub97c \uc704\ud574 \uc774\ub984 \ubcc4\ub85c \uac01 \ub9e4\uac1c\ubcc0\uc218(parameter)\uc758 \uac12\uc744 \uc5c5\ub370\uc774\ud2b8\ud558\uace0 \ub2e4\uc74c\uacfc \uac19\uc774\n\uac01 \ub9e4\uac1c \ubcc0\uc218\uc5d0 \ub300\ud55c \uae30\uc6b8\uae30\ub4e4\uc744 \uac1c\ubcc4\uc801\uc73c\ub85c \uc218\ub3d9\uc73c\ub85c 0\uc73c\ub85c \uc81c\uac70\ud574\uc57c \ud588\uc2b5\ub2c8\ub2e4:\n::\n with torch.no_grad():\n weights -= weights.grad * lr\n bias -= bias.grad * lr\n weights.grad.zero_()\n bias.grad.zero_()\n\n\n\uc774\uc81c \uc6b0\ub9ac\ub294 model.parameters() \ubc0f model.zero_grad() (\ubaa8\ub450\n``nn.Module`` \uc5d0 \ub300\ud574 PyTorch\uc5d0 \uc758\ud574 \uc815\uc758\ub428)\ub97c \ud65c\uc6a9\ud558\uc5ec \uc774\ub7ec\ud55c \ub2e8\uacc4\ub97c \ub354 \uac04\uacb0\ud558\uac8c\n\ub9cc\ub4e4\uace0, \ud2b9\ud788 \ub354 \ubcf5\uc7a1\ud55c \ubaa8\ub378\uc5d0 \ub300\ud574\uc11c \uc77c\ubd80 \ub9e4\uac1c\ubcc0\uc218\ub97c \uc78a\uc5b4 \ubc84\ub9ac\ub294 \uc624\ub958\ub97c \ub35c \ubc1c\uc0dd\uc2dc\ud0ac \uc218 \uc788\uc2b5\ub2c8\ub2e4:\n::\n with torch.no_grad():\n for p in model.parameters(): p -= p.grad * lr\n model.zero_grad()\n\n\n\uc774\uc81c \uc774\uac83\uc744 \ub098\uc911\uc5d0 \ub2e4\uc2dc \uc2e4\ud589\ud560 \uc218 \uc788\ub3c4\ub85d ``fit`` \ud568\uc218\ub85c \uc791\uc740 \ud6c8\ub828 \ub8e8\ud504\ub97c \uac10\uc300 \uac83\uc785\ub2c8\ub2e4.\n\n" + "\uc774\uc804\uc5d0\ub294 \ud6c8\ub828 \ub8e8\ud504\ub97c \uc704\ud574 \uc774\ub984 \ubcc4\ub85c \uac01 \ub9e4\uac1c\ubcc0\uc218(parameter)\uc758 \uac12\uc744 \uc5c5\ub370\uc774\ud2b8\ud558\uace0 \ub2e4\uc74c\uacfc \uac19\uc774\n\uac01 \ub9e4\uac1c \ubcc0\uc218\uc5d0 \ub300\ud55c \uae30\uc6b8\uae30\ub4e4\uc744 \uac1c\ubcc4\uc801\uc73c\ub85c \uc218\ub3d9\uc73c\ub85c 0\uc73c\ub85c \uc81c\uac70\ud574\uc57c \ud588\uc2b5\ub2c8\ub2e4:\n\n::\n\n with torch.no_grad():\n weights -= weights.grad * lr\n bias -= bias.grad * lr\n weights.grad.zero_()\n bias.grad.zero_()\n\n\n\uc774\uc81c \uc6b0\ub9ac\ub294 model.parameters() \ubc0f model.zero_grad() (\ubaa8\ub450\n``nn.Module`` \uc5d0 \ub300\ud574 PyTorch\uc5d0 \uc758\ud574 \uc815\uc758\ub428)\ub97c \ud65c\uc6a9\ud558\uc5ec \uc774\ub7ec\ud55c \ub2e8\uacc4\ub97c \ub354 \uac04\uacb0\ud558\uac8c\n\ub9cc\ub4e4\uace0, \ud2b9\ud788 \ub354 \ubcf5\uc7a1\ud55c \ubaa8\ub378\uc5d0 \ub300\ud574\uc11c \uc77c\ubd80 \ub9e4\uac1c\ubcc0\uc218\ub97c \uc78a\uc5b4 \ubc84\ub9ac\ub294 \uc624\ub958\ub97c \ub35c \ubc1c\uc0dd\uc2dc\ud0ac \uc218 \uc788\uc2b5\ub2c8\ub2e4:\n\n::\n\n with torch.no_grad():\n for p in model.parameters(): p -= p.grad * lr\n model.zero_grad()\n\n\n\uc774\uc81c \uc774\uac83\uc744 \ub098\uc911\uc5d0 \ub2e4\uc2dc \uc2e4\ud589\ud560 \uc218 \uc788\ub3c4\ub85d ``fit`` \ud568\uc218\ub85c \uc791\uc740 \ud6c8\ub828 \ub8e8\ud504\ub97c \uac10\uc300 \uac83\uc785\ub2c8\ub2e4.\n\n" ] }, { @@ -382,7 +382,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "nn.Linear \ub97c \uc774\uc6a9\ud558\uc5ec \ub9ac\ud329\ud1a0\ub9c1 \ud558\uae30\n------------------------------\n\n\uacc4\uc18d\ud574\uc11c \ucf54\ub4dc\ub97c \ub9ac\ud329\ud1a0\ub9c1 \ud569\ub2c8\ub2e4. ``self.weights`` \ubc0f ``self.bias`` \ub97c \uc218\ub3d9\uc73c\ub85c \uc815\uc758 \ubc0f\n\ucd08\uae30\ud654\ud558\uace0, ``xb @ self.weights + self.bias`` \ub97c \uacc4\uc0b0\ud558\ub294 \ub300\uc2e0\uc5d0,\n\uc704\uc758 \ubaa8\ub4e0 \uac83\uc744 \ud574\uc904 Pytorch \ud074\ub798\uc2a4\uc778\n`nn.Linear `_ \ub97c \uc120\ud615\n\ub808\uc774\uc5b4\ub85c \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\nPytorch \uc5d0\ub294 \ub2e4\uc591\ud55c \uc720\ud615\uc758 \ucf54\ub4dc\ub97c \ud06c\uac8c \ub2e8\uc21c\ud654 \ud560 \uc218 \uc788\ub294 \ubbf8\ub9ac \uc815\uc758\ub41c \ub808\uc774\uc5b4\uac00 \uc788\uace0 \uc774\ub294 \ub610\ud55c\n\uc885\uc885 \uae30\uc874 \ucf54\ub4dc\ubcf4\ub2e4 \uc18d\ub3c4\ub97c \ube60\ub974\uac8c \ud569\ub2c8\ub2e4.\n\n" + "nn.Linear \ub97c \uc774\uc6a9\ud558\uc5ec \ub9ac\ud329\ud1a0\ub9c1 \ud558\uae30\n------------------------------------\n\n\uacc4\uc18d\ud574\uc11c \ucf54\ub4dc\ub97c \ub9ac\ud329\ud1a0\ub9c1 \ud569\ub2c8\ub2e4. ``self.weights`` \ubc0f ``self.bias`` \ub97c \uc218\ub3d9\uc73c\ub85c \uc815\uc758 \ubc0f\n\ucd08\uae30\ud654\ud558\uace0, ``xb @ self.weights + self.bias`` \ub97c \uacc4\uc0b0\ud558\ub294 \ub300\uc2e0\uc5d0,\n\uc704\uc758 \ubaa8\ub4e0 \uac83\uc744 \ud574\uc904 Pytorch \ud074\ub798\uc2a4\uc778\n`nn.Linear `_ \ub97c \uc120\ud615\n\ub808\uc774\uc5b4\ub85c \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\nPytorch \uc5d0\ub294 \ub2e4\uc591\ud55c \uc720\ud615\uc758 \ucf54\ub4dc\ub97c \ud06c\uac8c \ub2e8\uc21c\ud654 \ud560 \uc218 \uc788\ub294 \ubbf8\ub9ac \uc815\uc758\ub41c \ub808\uc774\uc5b4\uac00 \uc788\uace0 \uc774\ub294 \ub610\ud55c\n\uc885\uc885 \uae30\uc874 \ucf54\ub4dc\ubcf4\ub2e4 \uc18d\ub3c4\ub97c \ube60\ub974\uac8c \ud569\ub2c8\ub2e4.\n\n" ] }, { @@ -436,7 +436,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "optim \uc744 \uc774\uc6a9\ud558\uc5ec \ub9ac\ud329\ud1a0\ub9c1 \ud558\uae30\n------------------------------\n\nPytorch\uc5d0\ub294 \ub2e4\uc591\ud55c \ucd5c\uc801\ud654(optimization) \uc54c\uace0\ub9ac\uc998\uc744 \uac00\uc9c4 \ud328\ud0a4\uc9c0\uc778 ``torch.optim`` \ub3c4 \uc788\uc2b5\ub2c8\ub2e4.\n\uac01 \ub9e4\uac1c\ubcc0\uc218\ub97c \uc218\ub3d9\uc73c\ub85c \uc5c5\ub370\uc774\ud2b8 \ud558\ub294 \ub300\uc2e0, \uc635\ud2f0\ub9c8\uc774\uc800(optimizer)\uc758 ``step`` \uba54\uc18c\ub4dc\ub97c \uc0ac\uc6a9\ud558\uc5ec\n\uc5c5\ub370\uc774\ud2b8\ub97c \uc9c4\ud589\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc774\ub807\uac8c \ud558\uba74 \uc774\uc804\uc5d0 \uc218\ub3d9\uc73c\ub85c \ucf54\ub529\ud55c \ucd5c\uc801\ud654 \ub2e8\uacc4\ub97c \ub300\uccb4\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4:\n::\n with torch.no_grad():\n for p in model.parameters(): p -= p.grad * lr\n model.zero_grad()\n\n\ub300\uc2e0\uc5d0 \uc774\ub807\uac8c \ub9d0\uc774\uc8e0:\n::\n opt.step()\n opt.zero_grad()\n\n(``optim.zero_grad()`` \ub294 \uae30\uc6b8\uae30\ub97c 0\uc73c\ub85c \uc7ac\uc124\uc815 \ud574\uc90d\ub2c8\ub2e4. \ub2e4\uc74c \ubbf8\ub2c8 \ubc30\uce58\uc5d0 \ub300\ud55c\n\uae30\uc6b8\uae30\ub97c \uacc4\uc0b0\ud558\uae30 \uc804\uc5d0 \ud638\ucd9c\ud574\uc57c \ud569\ub2c8\ub2e4.)\n\n" + "optim \uc744 \uc774\uc6a9\ud558\uc5ec \ub9ac\ud329\ud1a0\ub9c1 \ud558\uae30\n---------------------------------\n\nPytorch\uc5d0\ub294 \ub2e4\uc591\ud55c \ucd5c\uc801\ud654(optimization) \uc54c\uace0\ub9ac\uc998\uc744 \uac00\uc9c4 \ud328\ud0a4\uc9c0\uc778 ``torch.optim`` \ub3c4 \uc788\uc2b5\ub2c8\ub2e4.\n\uac01 \ub9e4\uac1c\ubcc0\uc218\ub97c \uc218\ub3d9\uc73c\ub85c \uc5c5\ub370\uc774\ud2b8 \ud558\ub294 \ub300\uc2e0, \uc635\ud2f0\ub9c8\uc774\uc800(optimizer)\uc758 ``step`` \uba54\uc18c\ub4dc\ub97c \uc0ac\uc6a9\ud558\uc5ec\n\uc5c5\ub370\uc774\ud2b8\ub97c \uc9c4\ud589\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc774\ub807\uac8c \ud558\uba74 \uc774\uc804\uc5d0 \uc218\ub3d9\uc73c\ub85c \ucf54\ub529\ud55c \ucd5c\uc801\ud654 \ub2e8\uacc4\ub97c \ub300\uccb4\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4:\n\n::\n\n with torch.no_grad():\n for p in model.parameters(): p -= p.grad * lr\n model.zero_grad()\n\n\ub300\uc2e0\uc5d0 \uc774\ub807\uac8c \ub9d0\uc774\uc8e0:\n\n::\n\n opt.step()\n opt.zero_grad()\n\n(``optim.zero_grad()`` \ub294 \uae30\uc6b8\uae30\ub97c 0\uc73c\ub85c \uc7ac\uc124\uc815 \ud574\uc90d\ub2c8\ub2e4. \ub2e4\uc74c \ubbf8\ub2c8 \ubc30\uce58\uc5d0 \ub300\ud55c\n\uae30\uc6b8\uae30\ub97c \uacc4\uc0b0\ud558\uae30 \uc804\uc5d0 \ud638\ucd9c\ud574\uc57c \ud569\ub2c8\ub2e4.)\n\n" ] }, { @@ -472,7 +472,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Dataset \uc744 \uc774\uc6a9\ud558\uc5ec \ub9ac\ud329\ud1a0\ub9c1\ud558\uae30\n------------------------------\n\nPyTorch \uc5d0\ub294 \ucd94\uc0c1 Dataset \ud074\ub798\uc2a4\uac00 \uc788\uc2b5\ub2c8\ub2e4. Dataset \uc740\n``__len__`` \ud568\uc218 (Python\uc758 \ud45c\uc900 ``len`` \ud568\uc218\uc5d0 \uc758\ud574 \ud638\ucd9c\ub428) \ubc0f\n``__getitem__`` \ud568\uc218\ub97c \uac00\uc9c4 \uc5b4\ub5a4 \uac83\uc774\ub77c\ub3c4 \ub420 \uc218 \uc788\uc73c\uba70, \uc774 \ud568\uc218\ub4e4\uc744 \uc778\ub371\uc2f1(indexing)\ud558\uae30\n\uc704\ud55c \ubc29\ubc95\uc73c\ub85c \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n`\uc774 \ud29c\ud1a0\ub9ac\uc5bc `_\n\uc740 ``Dataset`` \uc758 \ud558\uc704 \ud074\ub798\uc2a4\ub85c\uc368, \uc0ac\uc6a9\uc790 \uc9c0\uc815 ``FacialLandmarkDataset`` \ud074\ub798\uc2a4\ub97c \ub9cc\ub4dc\ub294\n\uc88b\uc740 \uc608\ub97c \uc81c\uc2dc\ud569\ub2c8\ub2e4.\n\nPyTorch \uc758 `TensorDataset `_\n\uc740 \ud150\uc11c\ub97c \uac10\uc2f8\ub294(wrapping) Dataset \uc785\ub2c8\ub2e4.\n\uae38\uc774\uc640 \uc778\ub371\uc2f1 \ubc29\uc2dd\uc744 \uc815\uc758\ud568\uc73c\ub85c\uc368 \ud150\uc11c\uc758 \uccab \ubc88\uc9f8 \ucc28\uc6d0\uc744 \ub530\ub77c \ubc18\ubcf5, \uc778\ub371\uc2f1 \ubc0f \uc2ac\ub77c\uc774\uc2a4(slice)\ud558\ub294 \ubc29\ubc95\ub3c4 \uc81c\uacf5\ud569\ub2c8\ub2e4.\n\uc774\ub807\uac8c\ud558\uba74 \ud6c8\ub828 \ud560 \ub54c \ub3d9\uc77c\ud55c \ub77c\uc778\uc5d0\uc11c \ub3c5\ub9bd(independent) \ubcc0\uc218\uc640 \uc885\uc18d(dependent) \ubcc0\uc218\uc5d0 \uc27d\uac8c \uc561\uc138\uc2a4 \ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n" + "Dataset \uc744 \uc774\uc6a9\ud558\uc5ec \ub9ac\ud329\ud1a0\ub9c1\ud558\uae30\n----------------------------------\n\nPyTorch \uc5d0\ub294 \ucd94\uc0c1 Dataset \ud074\ub798\uc2a4\uac00 \uc788\uc2b5\ub2c8\ub2e4. Dataset \uc740\n``__len__`` \ud568\uc218 (Python\uc758 \ud45c\uc900 ``len`` \ud568\uc218\uc5d0 \uc758\ud574 \ud638\ucd9c\ub428) \ubc0f\n``__getitem__`` \ud568\uc218\ub97c \uac00\uc9c4 \uc5b4\ub5a4 \uac83\uc774\ub77c\ub3c4 \ub420 \uc218 \uc788\uc73c\uba70, \uc774 \ud568\uc218\ub4e4\uc744 \uc778\ub371\uc2f1(indexing)\ud558\uae30\n\uc704\ud55c \ubc29\ubc95\uc73c\ub85c \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n`\uc774 \ud29c\ud1a0\ub9ac\uc5bc `_\n\uc740 ``Dataset`` \uc758 \ud558\uc704 \ud074\ub798\uc2a4\ub85c\uc368, \uc0ac\uc6a9\uc790 \uc9c0\uc815 ``FacialLandmarkDataset`` \ud074\ub798\uc2a4\ub97c \ub9cc\ub4dc\ub294\n\uc88b\uc740 \uc608\ub97c \uc81c\uc2dc\ud569\ub2c8\ub2e4.\n\nPyTorch \uc758 `TensorDataset `_\n\uc740 \ud150\uc11c\ub97c \uac10\uc2f8\ub294(wrapping) Dataset \uc785\ub2c8\ub2e4.\n\uae38\uc774\uc640 \uc778\ub371\uc2f1 \ubc29\uc2dd\uc744 \uc815\uc758\ud568\uc73c\ub85c\uc368 \ud150\uc11c\uc758 \uccab \ubc88\uc9f8 \ucc28\uc6d0\uc744 \ub530\ub77c \ubc18\ubcf5, \uc778\ub371\uc2f1 \ubc0f \uc2ac\ub77c\uc774\uc2a4(slice)\ud558\ub294 \ubc29\ubc95\ub3c4 \uc81c\uacf5\ud569\ub2c8\ub2e4.\n\uc774\ub807\uac8c\ud558\uba74 \ud6c8\ub828 \ud560 \ub54c \ub3d9\uc77c\ud55c \ub77c\uc778\uc5d0\uc11c \ub3c5\ub9bd(independent) \ubcc0\uc218\uc640 \uc885\uc18d(dependent) \ubcc0\uc218\uc5d0 \uc27d\uac8c \uc561\uc138\uc2a4 \ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n" ] }, { @@ -508,7 +508,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uc774\uc804\uc5d0\ub294 x \ubc0f y \uac12\uc758 \ubbf8\ub2c8 \ubc30\uce58\ub97c \ubcc4\ub3c4\ub85c \ubc18\ubcf5\ud574\uc57c\ud588\uc2b5\ub2c8\ub2e4:\n::\n xb = x_train[start_i:end_i]\n yb = y_train[start_i:end_i]\n\n\n\uc774\uc81c \uc774 \ub450 \ub2e8\uacc4\ub97c \ud568\uaed8 \uc218\ud589 \ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4:\n::\n xb,yb = train_ds[i*bs : i*bs+bs]\n\n\n" + "\uc774\uc804\uc5d0\ub294 x \ubc0f y \uac12\uc758 \ubbf8\ub2c8 \ubc30\uce58\ub97c \ubcc4\ub3c4\ub85c \ubc18\ubcf5\ud574\uc57c\ud588\uc2b5\ub2c8\ub2e4:\n\n::\n\n xb = x_train[start_i:end_i]\n yb = y_train[start_i:end_i]\n\n\n\uc774\uc81c \uc774 \ub450 \ub2e8\uacc4\ub97c \ud568\uaed8 \uc218\ud589 \ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4:\n\n::\n\n xb,yb = train_ds[i*bs : i*bs+bs]\n\n\n" ] }, { @@ -526,7 +526,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "DataLoader \ub97c \uc774\uc6a9\ud558\uc5ec \ub9ac\ud329\ud1a0\ub9c1\ud558\uae30\n-------------------------------\n\nPytorch \uc758 ``DataLoader`` \ub294 \ubc30\uce58 \uad00\ub9ac\ub97c \ub2f4\ub2f9\ud569\ub2c8\ub2e4.\n\uc5ec\ub7ec\ubd84\ub4e4\uc740 \ubaa8\ub4e0 ``Dataset`` \uc73c\ub85c\ubd80\ud130 ``DataLoader`` \ub97c \uc0dd\uc131\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n``DataLoader`` \ub294 \ubc30\uce58\ub4e4\uc5d0 \ub300\ud574\uc11c \ubc18\ubcf5\ud558\uae30 \uc27d\uac8c \ub9cc\ub4e4\uc5b4\uc90d\ub2c8\ub2e4.\n``train_ds[i*bs : i*bs+bs]`` \ub97c \uc0ac\uc6a9\ud558\ub294 \ub300\uc2e0,\nDataLoader \ub294 \ub9e4 \ubbf8\ub2c8\ubc30\uce58\ub97c \uc790\ub3d9\uc801\uc73c\ub85c \uc81c\uacf5\ud569\ub2c8\ub2e4.\n\n" + "DataLoader \ub97c \uc774\uc6a9\ud558\uc5ec \ub9ac\ud329\ud1a0\ub9c1\ud558\uae30\n-----------------------------------\n\nPytorch \uc758 ``DataLoader`` \ub294 \ubc30\uce58 \uad00\ub9ac\ub97c \ub2f4\ub2f9\ud569\ub2c8\ub2e4.\n\uc5ec\ub7ec\ubd84\ub4e4\uc740 \ubaa8\ub4e0 ``Dataset`` \uc73c\ub85c\ubd80\ud130 ``DataLoader`` \ub97c \uc0dd\uc131\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n``DataLoader`` \ub294 \ubc30\uce58\ub4e4\uc5d0 \ub300\ud574\uc11c \ubc18\ubcf5\ud558\uae30 \uc27d\uac8c \ub9cc\ub4e4\uc5b4\uc90d\ub2c8\ub2e4.\n``train_ds[i*bs : i*bs+bs]`` \ub97c \uc0ac\uc6a9\ud558\ub294 \ub300\uc2e0,\nDataLoader \ub294 \ub9e4 \ubbf8\ub2c8\ubc30\uce58\ub97c \uc790\ub3d9\uc801\uc73c\ub85c \uc81c\uacf5\ud569\ub2c8\ub2e4.\n\n" ] }, { @@ -544,7 +544,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uc774\uc804\uc5d0\ub294 \ub8e8\ud504\uac00 \ub2e4\uc74c\uacfc \uac19\uc774 \ubc30\uce58 (xb, yb)\ub97c \ubc18\ubcf5\ud588\uc2b5\ub2c8\ub2e4:\n::\n for i in range((n-1)//bs + 1):\n xb,yb = train_ds[i*bs : i*bs+bs]\n pred = model(xb)\n\n\uc774\uc81c (xb, yb)\uac00 DataLoader \uc5d0\uc11c \uc790\ub3d9\uc73c\ub85c \ub85c\ub4dc\ub418\ubbc0\ub85c \ub8e8\ud504\uac00 \ud6e8\uc52c \uae68\ub057\ud574\uc84c\uc2b5\ub2c8\ub2e4:\n::\n for xb,yb in train_dl:\n pred = model(xb)\n\n" + "\uc774\uc804\uc5d0\ub294 \ub8e8\ud504\uac00 \ub2e4\uc74c\uacfc \uac19\uc774 \ubc30\uce58 (xb, yb)\ub97c \ubc18\ubcf5\ud588\uc2b5\ub2c8\ub2e4:\n\n::\n\n for i in range((n-1)//bs + 1):\n xb,yb = train_ds[i*bs : i*bs+bs]\n pred = model(xb)\n\n\uc774\uc81c (xb, yb)\uac00 DataLoader \uc5d0\uc11c \uc790\ub3d9\uc73c\ub85c \ub85c\ub4dc\ub418\ubbc0\ub85c \ub8e8\ud504\uac00 \ud6e8\uc52c \uae68\ub057\ud574\uc84c\uc2b5\ub2c8\ub2e4:\n\n::\n\n for xb,yb in train_dl:\n pred = model(xb)\n\n" ] }, { @@ -562,7 +562,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Pytorch\uc758 nn.Module, nn.Parameter, Dataset \ubc0f DataLoader \ub355\ubd84\uc5d0 \uc774\uc81c \ud6c8\ub828 \ub8e8\ud504\uac00\n\ud6e8\uc52c \ub354 \uc791\uc544\uc9c0\uace0 \uc774\ud574\ud558\uae30 \uc26c\uc6cc\uc84c\uc2b5\ub2c8\ub2e4.\n\uc774\uc81c \uc2e4\uc81c\ub85c \ud6a8\uacfc\uc801\uc778 \ubaa8\ub378\uc744 \ub9cc\ub4dc\ub294 \ub370 \ud544\uc694\ud55c \uae30\ubcf8 \uae30\ub2a5\uc744 \ucd94\uac00\ud574 \ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uac80\uc99d(validation) \ucd94\uac00\ud558\uae30\n-----------------------\n\n\uc139\uc158 1\uc5d0\uc11c, \uc6b0\ub9ac\ub294 \ud6c8\ub828 \ub370\uc774\ud130\uc5d0 \uc0ac\uc6a9\ud558\uae30 \uc704\ud574 \ud569\ub9ac\uc801\uc778 \ud6c8\ub828 \ub8e8\ud504\ub97c \uc124\uc815\ud558\ub824\uace0\ud588\uc2b5\ub2c8\ub2e4.\n\uc2e4\uc804\uc5d0\uc11c, \uc5ec\ub7ec\ubd84\ub4e4\uc740 \uacfc\uc801\ud569(overfitting)\uc744 \ud655\uc778\ud558\uae30 \uc704\ud574\uc11c **\ud56d\uc0c1**\n`\uac80\uc99d \ub370\uc774\ud130\uc14b(validation set) `_ \uc774\n\uc788\uc5b4\uc57c \ud569\ub2c8\ub2e4.\n\n\ud6c8\ub828 \ub370\uc774\ud130\ub97c \uc11e\ub294(shuffling) \uac83\uc740 \ubc30\uce58\uc640 \uacfc\uc801\ud569 \uc0ac\uc774\uc758 \uc0c1\uad00\uad00\uacc4\ub97c \ubc29\uc9c0\ud558\uae30 \uc704\ud574\n`\uc911\uc694\ud569\ub2c8\ub2e4. `_\n\ubc18\uba74\uc5d0, \uac80\uc99d \uc190\uc2e4(validation loss)\uc740 \uac80\uc99d \ub370\uc774\ud130\uc14b\uc744 \uc11e\ub4e0 \uc548\uc11e\ub4e0 \ub3d9\uc77c\ud569\ub2c8\ub2e4.\n\ub370\uc774\ud130\ub97c \uc11e\ub294 \uac83\uc740 \ucd94\uac00 \uc2dc\uac04\uc774 \uac78\ub9ac\ubbc0\ub85c, \uac80\uc99d \ub370\uc774\ud130\ub97c \uc11e\ub294 \uac83\uc740 \uc758\ubbf8\uac00 \uc5c6\uc2b5\ub2c8\ub2e4.\n\n\uac80\uc99d \ub370\uc774\ud130\uc14b\uc5d0 \ub300\ud55c \ubc30\uce58 \uc0ac\uc774\uc988\ub294 \ud559\uc2b5 \ub370\uc774\ud130\uc14b \ubc30\uce58 \ud06c\uae30\uc758 2\ubc30\ub97c \uc0ac\uc6a9\ud560 \uac83\uc785\ub2c8\ub2e4.\n\uc774\ub294 \uac80\uc99d \ub370\uc774\ud130\uc14b\uc5d0 \ub300\ud574\uc11c\ub294 \uc5ed\uc804\ud30c(backpropagation)\uac00 \ud544\uc694\ud558\uc9c0 \uc54a\uc73c\ubbc0\ub85c \uba54\ubaa8\ub9ac\ub97c\n\ub35c \uc0ac\uc6a9\ud558\uae30 \ub54c\ubb38\uc785\ub2c8\ub2e4 (\uae30\uc6b8\uae30\ub97c \uc800\uc7a5\ud560 \ud544\uc694\uac00 \uc5c6\uc74c).\n\ub354 \ud070 \ubc30\uce58 \ud06c\uae30\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc190\uc2e4\uc744 \ub354 \ube68\ub9ac \uacc4\uc0b0\ud558\uae30 \uc704\ud574 \uc774\ub807\uac8c \ud569\ub2c8\ub2e4.\n\n" + "Pytorch\uc758 nn.Module, nn.Parameter, Dataset \ubc0f DataLoader \ub355\ubd84\uc5d0 \uc774\uc81c \ud6c8\ub828 \ub8e8\ud504\uac00\n\ud6e8\uc52c \ub354 \uc791\uc544\uc9c0\uace0 \uc774\ud574\ud558\uae30 \uc26c\uc6cc\uc84c\uc2b5\ub2c8\ub2e4.\n\uc774\uc81c \uc2e4\uc81c\ub85c \ud6a8\uacfc\uc801\uc778 \ubaa8\ub378\uc744 \ub9cc\ub4dc\ub294 \ub370 \ud544\uc694\ud55c \uae30\ubcf8 \uae30\ub2a5\uc744 \ucd94\uac00\ud574 \ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uac80\uc99d(validation) \ucd94\uac00\ud558\uae30\n---------------------------\n\n\uc139\uc158 1\uc5d0\uc11c, \uc6b0\ub9ac\ub294 \ud6c8\ub828 \ub370\uc774\ud130\uc5d0 \uc0ac\uc6a9\ud558\uae30 \uc704\ud574 \ud569\ub9ac\uc801\uc778 \ud6c8\ub828 \ub8e8\ud504\ub97c \uc124\uc815\ud558\ub824\uace0\ud588\uc2b5\ub2c8\ub2e4.\n\uc2e4\uc804\uc5d0\uc11c, \uc5ec\ub7ec\ubd84\ub4e4\uc740 \uacfc\uc801\ud569(overfitting)\uc744 \ud655\uc778\ud558\uae30 \uc704\ud574\uc11c **\ud56d\uc0c1**\n`\uac80\uc99d \ub370\uc774\ud130\uc14b(validation set) `_ \uc774\n\uc788\uc5b4\uc57c \ud569\ub2c8\ub2e4.\n\n\ud6c8\ub828 \ub370\uc774\ud130\ub97c \uc11e\ub294(shuffling) \uac83\uc740 \ubc30\uce58\uc640 \uacfc\uc801\ud569 \uc0ac\uc774\uc758 \uc0c1\uad00\uad00\uacc4\ub97c \ubc29\uc9c0\ud558\uae30 \uc704\ud574\n`\uc911\uc694\ud569\ub2c8\ub2e4. `_\n\ubc18\uba74\uc5d0, \uac80\uc99d \uc190\uc2e4(validation loss)\uc740 \uac80\uc99d \ub370\uc774\ud130\uc14b\uc744 \uc11e\ub4e0 \uc548\uc11e\ub4e0 \ub3d9\uc77c\ud569\ub2c8\ub2e4.\n\ub370\uc774\ud130\ub97c \uc11e\ub294 \uac83\uc740 \ucd94\uac00 \uc2dc\uac04\uc774 \uac78\ub9ac\ubbc0\ub85c, \uac80\uc99d \ub370\uc774\ud130\ub97c \uc11e\ub294 \uac83\uc740 \uc758\ubbf8\uac00 \uc5c6\uc2b5\ub2c8\ub2e4.\n\n\uac80\uc99d \ub370\uc774\ud130\uc14b\uc5d0 \ub300\ud55c \ubc30\uce58 \uc0ac\uc774\uc988\ub294 \ud559\uc2b5 \ub370\uc774\ud130\uc14b \ubc30\uce58 \ud06c\uae30\uc758 2\ubc30\ub97c \uc0ac\uc6a9\ud560 \uac83\uc785\ub2c8\ub2e4.\n\uc774\ub294 \uac80\uc99d \ub370\uc774\ud130\uc14b\uc5d0 \ub300\ud574\uc11c\ub294 \uc5ed\uc804\ud30c(backpropagation)\uac00 \ud544\uc694\ud558\uc9c0 \uc54a\uc73c\ubbc0\ub85c \uba54\ubaa8\ub9ac\ub97c\n\ub35c \uc0ac\uc6a9\ud558\uae30 \ub54c\ubb38\uc785\ub2c8\ub2e4 (\uae30\uc6b8\uae30\ub97c \uc800\uc7a5\ud560 \ud544\uc694\uac00 \uc5c6\uc74c).\n\ub354 \ud070 \ubc30\uce58 \ud06c\uae30\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc190\uc2e4\uc744 \ub354 \ube68\ub9ac \uacc4\uc0b0\ud558\uae30 \uc704\ud574 \uc774\ub807\uac8c \ud569\ub2c8\ub2e4.\n\n" ] }, { @@ -670,7 +670,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uc774\ub7ec\ud55c \uae30\ubcf8 3\uc904\uc758 \ucf54\ub4dc\ub97c \uc0ac\uc6a9\ud558\uc5ec \ub2e4\uc591\ud55c \ubaa8\ub378\uc744 \ud6c8\ub828\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\ucee8\ubcfc\ub8e8\uc158 \uc2e0\uacbd\ub9dd(CNN)\uc744 \ud6c8\ub828\ud558\ub294 \ub370 \uc0ac\uc6a9\ud560 \uc218 \uc788\ub294\uc9c0 \uc0b4\ud3b4 \ubcf4\uaca0\uc2b5\ub2c8\ub2e4!\n\nCNN \uc73c\ub85c \ub118\uc5b4\uac00\uae30\n---------------\n\n\uc774\uc81c 3\uac1c\uc758 \ucee8\ubcfc\ub8e8\uc158 \ub808\uc774\uc5b4\ub85c \uc2e0\uacbd\ub9dd\uc744 \uad6c\ucd95\ud560 \uac83\uc785\ub2c8\ub2e4.\n\uc774\uc804 \uc139\uc158\uc758 \uc5b4\ub5a4 \ud568\uc218\ub3c4 \ubaa8\ub378\uc758 \ud615\uc2dd\uc5d0 \ub300\ud574 \uac00\uc815\ud558\uc9c0 \uc54a\uae30 \ub54c\ubb38\uc5d0,\n\ubcc4\ub3c4\uc758 \uc218\uc815\uc5c6\uc774 CNN\uc744 \ud559\uc2b5\ud558\ub294 \ub370 \uc0ac\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\nPytorch \uc758 \uc0ac\uc804\uc815\uc758\ub41c\n`Conv2d `_ \ud074\ub798\uc2a4\ub97c\n\ucee8\ubcfc\ub8e8\uc158 \ub808\uc774\uc5b4\ub85c \uc0ac\uc6a9\ud569\ub2c8\ub2e4. 3\uac1c\uc758 \ucee8\ubcfc\ub8e8\uc158 \ub808\uc774\uc5b4\ub85c CNN\uc744 \uc815\uc758\ud569\ub2c8\ub2e4.\n\uac01 \ucee8\ubcfc\ub8e8\uc158 \ub4a4\uc5d0\ub294 ReLU\uac00 \uc788\uc2b5\ub2c8\ub2e4. \ub9c8\uc9c0\ub9c9\uc73c\ub85c \ud3c9\uade0 \ud480\ub9c1(average pooling)\uc744 \uc218\ud589\ud569\ub2c8\ub2e4.\n(``view`` \ub294 PyTorch\uc758 numpy ``reshape`` \ubc84\uc804\uc785\ub2c8\ub2e4.)\n\n" + "\uc774\ub7ec\ud55c \uae30\ubcf8 3\uc904\uc758 \ucf54\ub4dc\ub97c \uc0ac\uc6a9\ud558\uc5ec \ub2e4\uc591\ud55c \ubaa8\ub378\uc744 \ud6c8\ub828\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\ucee8\ubcfc\ub8e8\uc158 \uc2e0\uacbd\ub9dd(CNN)\uc744 \ud6c8\ub828\ud558\ub294 \ub370 \uc0ac\uc6a9\ud560 \uc218 \uc788\ub294\uc9c0 \uc0b4\ud3b4 \ubcf4\uaca0\uc2b5\ub2c8\ub2e4!\n\nCNN \uc73c\ub85c \ub118\uc5b4\uac00\uae30\n--------------------\n\n\uc774\uc81c 3\uac1c\uc758 \ucee8\ubcfc\ub8e8\uc158 \ub808\uc774\uc5b4\ub85c \uc2e0\uacbd\ub9dd\uc744 \uad6c\ucd95\ud560 \uac83\uc785\ub2c8\ub2e4.\n\uc774\uc804 \uc139\uc158\uc758 \uc5b4\ub5a4 \ud568\uc218\ub3c4 \ubaa8\ub378\uc758 \ud615\uc2dd\uc5d0 \ub300\ud574 \uac00\uc815\ud558\uc9c0 \uc54a\uae30 \ub54c\ubb38\uc5d0,\n\ubcc4\ub3c4\uc758 \uc218\uc815\uc5c6\uc774 CNN\uc744 \ud559\uc2b5\ud558\ub294 \ub370 \uc0ac\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\nPytorch \uc758 \uc0ac\uc804\uc815\uc758\ub41c\n`Conv2d `_ \ud074\ub798\uc2a4\ub97c\n\ucee8\ubcfc\ub8e8\uc158 \ub808\uc774\uc5b4\ub85c \uc0ac\uc6a9\ud569\ub2c8\ub2e4. 3\uac1c\uc758 \ucee8\ubcfc\ub8e8\uc158 \ub808\uc774\uc5b4\ub85c CNN\uc744 \uc815\uc758\ud569\ub2c8\ub2e4.\n\uac01 \ucee8\ubcfc\ub8e8\uc158 \ub4a4\uc5d0\ub294 ReLU\uac00 \uc788\uc2b5\ub2c8\ub2e4. \ub9c8\uc9c0\ub9c9\uc73c\ub85c \ud3c9\uade0 \ud480\ub9c1(average pooling)\uc744 \uc218\ud589\ud569\ub2c8\ub2e4.\n(``view`` \ub294 PyTorch\uc758 numpy ``reshape`` \ubc84\uc804\uc785\ub2c8\ub2e4.)\n\n" ] }, { @@ -906,7 +906,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/dbcf5e6a5e95bf9f7a0e49e123c13b60/saving_and_loading_a_general_checkpoint.ipynb b/docs/_downloads/dbcf5e6a5e95bf9f7a0e49e123c13b60/saving_and_loading_a_general_checkpoint.ipynb index cd1c072c7..892f41e34 100644 --- a/docs/_downloads/dbcf5e6a5e95bf9f7a0e49e123c13b60/saving_and_loading_a_general_checkpoint.ipynb +++ b/docs/_downloads/dbcf5e6a5e95bf9f7a0e49e123c13b60/saving_and_loading_a_general_checkpoint.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nPyTorch\uc5d0\uc11c \uc77c\ubc18\uc801\uc778 \uccb4\ud06c\ud3ec\uc778\ud2b8(checkpoint) \uc800\uc7a5\ud558\uae30 & \ubd88\ub7ec\uc624\uae30\n===================================================================\n\ucd94\ub860(inference) \ub610\ub294 \ud559\uc2b5(training)\uc758 \uc7ac\uac1c\ub97c \uc704\ud574 \uccb4\ud06c\ud3ec\uc778\ud2b8(checkpoint) \ubaa8\ub378\uc744\n\uc800\uc7a5\ud558\uace0 \ubd88\ub7ec\uc624\ub294 \uac83\uc740 \ub9c8\uc9c0\ub9c9\uc73c\ub85c \uc911\ub2e8\ud588\ub358 \ubd80\ubd84\uc744 \uc120\ud0dd\ud558\ub294\ub370 \ub3c4\uc6c0\uc744 \uc904 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uccb4\ud06c\ud3ec\uc778\ud2b8\ub97c \uc800\uc7a5\ud560 \ub54c\ub294 \ub2e8\uc21c\ud788 \ubaa8\ub378\uc758 state_dict \uc774\uc0c1\uc758 \uac83\uc744 \uc800\uc7a5\ud574\uc57c \ud569\ub2c8\ub2e4.\n\ubaa8\ub378 \ud559\uc2b5 \uc911\uc5d0 \uac31\uc2e0\ub418\ub294 \ud37c\ubc84\uc640 \ub9e4\uac1c\ubcc0\uc218\ub4e4\uc744 \ud3ec\ud568\ud558\ub294 \uc635\ud2f0\ub9c8\uc774\uc800(Optimizer)\uc758\nstate_dict\ub97c \ud568\uaed8 \uc800\uc7a5\ud558\ub294 \uac83\uc774 \uc911\uc694\ud569\ub2c8\ub2e4. \uc774 \uc678\uc5d0\ub3c4 \uc911\ub2e8 \uc2dc\uc810\uc758 \uc5d0\ud3ec\ud06c(epoch),\n\ub9c8\uc9c0\ub9c9\uc73c\ub85c \uae30\ub85d\ub41c \ud559\uc2b5 \uc624\ucc28(training loss), \uc678\ubd80 ``torch.nn.Embedding`` \uacc4\uce35 \ub4f1,\n\uc54c\uace0\ub9ac\uc998\uc5d0 \ub530\ub77c \uc800\uc7a5\ud558\uace0 \uc2f6\uc740 \ud56d\ubaa9\ub4e4\uc774 \uc788\uc744 \uac83\uc785\ub2c8\ub2e4.\n\n\uac1c\uc694\n------------\n\uc5ec\ub7ec \uccb4\ud06c\ud3ec\uc778\ud2b8\ub4e4\uc744 \uc800\uc7a5\ud558\uae30 \uc704\ud574\uc11c\ub294 \uc0ac\uc804(dictionary)\uc5d0 \uccb4\ud06c\ud3ec\uc778\ud2b8\ub4e4\uc744 \uad6c\uc131\ud558\uace0\n``torch.save()`` \ub97c \uc0ac\uc6a9\ud558\uc5ec \uc0ac\uc804\uc744 \uc9c1\ub82c\ud654(serialize)\ud574\uc57c \ud569\ub2c8\ub2e4. \uc77c\ubc18\uc801\uc778\nPyTorch\uc5d0\uc11c\ub294 \uc774\ub7ec\ud55c \uc5ec\ub7ec \uccb4\ud06c\ud3ec\uc778\ud2b8\ub4e4\uc744 \uc800\uc7a5\ud560 \ub54c ``.tar`` \ud655\uc7a5\uc790\ub97c \uc0ac\uc6a9\ud558\ub294 \uac83\uc774\n\uc77c\ubc18\uc801\uc778 \uaddc\uce59\uc785\ub2c8\ub2e4. \ud56d\ubaa9\ub4e4\uc744 \ubd88\ub7ec\uc62c \ub54c\uc5d0\ub294, \uba3c\uc800 \ubaa8\ub378\uacfc \uc635\ud2f0\ub9c8\uc774\uc800\ub97c \ucd08\uae30\ud654\ud558\uace0,\ntorch.load()\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc0ac\uc804\uc744 \ubd88\ub7ec\uc635\ub2c8\ub2e4. \uc774\ud6c4 \uc6d0\ud558\ub294\ub300\ub85c \uc800\uc7a5\ud55c \ud56d\ubaa9\ub4e4\uc744 \uc0ac\uc804\uc5d0\n\uc870\ud68c\ud558\uc5ec \uc811\uadfc\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc774 \ub808\uc2dc\ud53c\uc5d0\uc11c\ub294 \uc5ec\ub7ec \uccb4\ud06c\ud3ec\uc778\ud2b8\ub4e4\uc744 \uc5b4\ub5bb\uac8c \uc800\uc7a5\ud558\uace0 \ubd88\ub7ec\uc624\ub294\uc9c0 \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc124\uc815\n-----\n\uc2dc\uc791\ud558\uae30 \uc804\uc5d0 ``torch`` \uac00 \uc5c6\ub2e4\uba74 \uc124\uce58\ud574\uc57c \ud569\ub2c8\ub2e4.\n\n\n::\n\n pip install torch\n\n\n\n" + "\nPyTorch\uc5d0\uc11c \uc77c\ubc18\uc801\uc778 \uccb4\ud06c\ud3ec\uc778\ud2b8(checkpoint) \uc800\uc7a5\ud558\uae30 & \ubd88\ub7ec\uc624\uae30\n===================================================================\n\ucd94\ub860(inference) \ub610\ub294 \ud559\uc2b5(training)\uc758 \uc7ac\uac1c\ub97c \uc704\ud574 \uccb4\ud06c\ud3ec\uc778\ud2b8(checkpoint) \ubaa8\ub378\uc744\n\uc800\uc7a5\ud558\uace0 \ubd88\ub7ec\uc624\ub294 \uac83\uc740 \ub9c8\uc9c0\ub9c9\uc73c\ub85c \uc911\ub2e8\ud588\ub358 \ubd80\ubd84\uc744 \uc120\ud0dd\ud558\ub294\ub370 \ub3c4\uc6c0\uc744 \uc904 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uccb4\ud06c\ud3ec\uc778\ud2b8\ub97c \uc800\uc7a5\ud560 \ub54c\ub294 \ub2e8\uc21c\ud788 \ubaa8\ub378\uc758 state_dict \uc774\uc0c1\uc758 \uac83\uc744 \uc800\uc7a5\ud574\uc57c \ud569\ub2c8\ub2e4.\n\ubaa8\ub378 \ud559\uc2b5 \uc911\uc5d0 \uac31\uc2e0\ub418\ub294 \ud37c\ubc84\uc640 \ub9e4\uac1c\ubcc0\uc218\ub4e4\uc744 \ud3ec\ud568\ud558\ub294 \uc635\ud2f0\ub9c8\uc774\uc800(Optimizer)\uc758\nstate_dict\ub97c \ud568\uaed8 \uc800\uc7a5\ud558\ub294 \uac83\uc774 \uc911\uc694\ud569\ub2c8\ub2e4. \uc774 \uc678\uc5d0\ub3c4 \uc911\ub2e8 \uc2dc\uc810\uc758 \uc5d0\ud3ec\ud06c(epoch),\n\ub9c8\uc9c0\ub9c9\uc73c\ub85c \uae30\ub85d\ub41c \ud559\uc2b5 \uc624\ucc28(training loss), \uc678\ubd80 ``torch.nn.Embedding`` \uacc4\uce35 \ub4f1,\n\uc54c\uace0\ub9ac\uc998\uc5d0 \ub530\ub77c \uc800\uc7a5\ud558\uace0 \uc2f6\uc740 \ud56d\ubaa9\ub4e4\uc774 \uc788\uc744 \uac83\uc785\ub2c8\ub2e4.\n\n\uac1c\uc694\n------------\n\uc5ec\ub7ec \uccb4\ud06c\ud3ec\uc778\ud2b8\ub4e4\uc744 \uc800\uc7a5\ud558\uae30 \uc704\ud574\uc11c\ub294 \uc0ac\uc804(dictionary)\uc5d0 \uccb4\ud06c\ud3ec\uc778\ud2b8\ub4e4\uc744 \uad6c\uc131\ud558\uace0\n``torch.save()`` \ub97c \uc0ac\uc6a9\ud558\uc5ec \uc0ac\uc804\uc744 \uc9c1\ub82c\ud654(serialize)\ud574\uc57c \ud569\ub2c8\ub2e4. \uc77c\ubc18\uc801\uc778\nPyTorch\uc5d0\uc11c\ub294 \uc774\ub7ec\ud55c \uc5ec\ub7ec \uccb4\ud06c\ud3ec\uc778\ud2b8\ub4e4\uc744 \uc800\uc7a5\ud560 \ub54c ``.tar`` \ud655\uc7a5\uc790\ub97c \uc0ac\uc6a9\ud558\ub294 \uac83\uc774\n\uc77c\ubc18\uc801\uc778 \uaddc\uce59\uc785\ub2c8\ub2e4. \ud56d\ubaa9\ub4e4\uc744 \ubd88\ub7ec\uc62c \ub54c\uc5d0\ub294, \uba3c\uc800 \ubaa8\ub378\uacfc \uc635\ud2f0\ub9c8\uc774\uc800\ub97c \ucd08\uae30\ud654\ud558\uace0,\ntorch.load()\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc0ac\uc804\uc744 \ubd88\ub7ec\uc635\ub2c8\ub2e4. \uc774\ud6c4 \uc6d0\ud558\ub294\ub300\ub85c \uc800\uc7a5\ud55c \ud56d\ubaa9\ub4e4\uc744 \uc0ac\uc804\uc5d0\n\uc870\ud68c\ud558\uc5ec \uc811\uadfc\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc774 \ub808\uc2dc\ud53c\uc5d0\uc11c\ub294 \uc5ec\ub7ec \uccb4\ud06c\ud3ec\uc778\ud2b8\ub4e4\uc744 \uc5b4\ub5bb\uac8c \uc800\uc7a5\ud558\uace0 \ubd88\ub7ec\uc624\ub294\uc9c0 \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc124\uc815\n-----\n\uc2dc\uc791\ud558\uae30 \uc804\uc5d0 ``torch`` \uac00 \uc5c6\ub2e4\uba74 \uc124\uce58\ud574\uc57c \ud569\ub2c8\ub2e4.\n\n\n::\n\n pip install torch\n" ] }, { @@ -132,7 +132,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/dd1c511de656ab48216de2866264b28f/deep_learning_tutorial.ipynb b/docs/_downloads/dd1c511de656ab48216de2866264b28f/deep_learning_tutorial.ipynb index 148227faf..ca01a55fd 100644 --- a/docs/_downloads/dd1c511de656ab48216de2866264b28f/deep_learning_tutorial.ipynb +++ b/docs/_downloads/dd1c511de656ab48216de2866264b28f/deep_learning_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nPyTorch\ub97c \uc774\uc6a9\ud55c \ub525\ub7ec\ub2dd\n**************************\n**\ubc88\uc5ed**: `\ud669\uc131\uc218 `_\n\n\ub525\ub7ec\ub2dd \ube14\ub85d \uad6c\ucd95 : \uc544\ud540 \ub9f5(affine maps), \ube44\uc120\ud615\uc131, \uac1d\uccb4\n==========================================================================\n\n\ub525\ub7ec\ub2dd\uc740 \uc601\ub9ac\ud55c \ubc29\ubc95\uc73c\ub85c \ube44\uc120\ud615\uc131\uc744 \uac00\uc9c4 \uc120\ud615\uc131\uc744 \uad6c\uc131\ud558\ub294 \uac83\uc73c\ub85c\n\uc774\ub8e8\uc5b4\uc9d1\ub2c8\ub2e4. \ube44\uc120\ud615\uc131\uc758 \ub3c4\uc785\uc740 \uac15\ub825\ud55c \ubaa8\ub378\uc744 \uac00\ub2a5\ud558\uac8c \ud569\ub2c8\ub2e4.\n\uc774 \uc139\uc158\uc5d0\uc11c \uc774 \ud575\uc2ec \uad6c\uc131 \uc694\uc18c\ub97c \ub2e4\ub8e8\uace0, \uac1d\uccb4 \ud568\uc218\ub97c \ub9cc\ub4e4\uace0, \uc5b4\ub5bb\uac8c\n\ubaa8\ub378\uc774 \ud559\uc2b5\ub418\uc9c0\ub294 \uc0b4\ud3b4\ubd05\uc2dc\ub2e4.\n\n\n\uc544\ud540 \ub9f5\n~~~~~~~~~~~\n\n\ub525\ub7ec\ub2dd\uc758 \ud575\uc2ec \uc791\uc5c5\uc790 \uc911 \ud558\ub098\ub294 \uc544\ud540 \ub9f5 \uc785\ub2c8\ub2e4.\n\uc774 \ud568\uc218 $f(x)$ \ub294 \ub2e4\uc74c\uacfc \uac19\uc2b5\ub2c8\ub2e4.\n\n\\begin{align}f(x) = Ax + b\\end{align}\n\n\uc5ec\uae30\uc11c $A$ \ub294 \ud589\ub82c, $x, b$ \ub294 \ubca1\ud130 \uc785\ub2c8\ub2e4.\n\uc5ec\uae30\uc11c \ud559\uc2b5\ub418\ub294 \ubcc0\uc218\ub294 $A$ \uc640 $b$ \uc785\ub2c8\ub2e4.\n\uc885\uc885 $b$ \ub294 *\ud3b8\ud5a5(Bias)* \uc774\ub77c \ubd88\ub9bd\ub2c8\ub2e4.\n\n\nPyTorch \uc640 \ub300\ubd80\ubd84\uc758 \ub2e4\ub978 \ub525\ub7ec\ub2dd \ud504\ub808\uc784\uc6cc\ud06c\ub4e4\uc740 \uace0\uc804\uc801\uc778 \uc120\ud615 \ub300\uc218\ud559\uc640\n\uc870\uae08 \ub2e4\ub974\uac8c \ub3d9\uc791\ud569\ub2c8\ub2e4. \uc785\ub825\uc758 \uc5f4 \ub300\uc2e0\uc5d0 \ud589\uc73c\ub85c \ub9e4\ud551\ud569\ub2c8\ub2e4.\n\uc989 \uc8fc\uc5b4\uc9c4 $A$ \uc5d0\uc11c \ucd9c\ub825\uc758 $i$ \ubc88\uc9f8 \ud589\uc740\n\uc785\ub825\uc758 $i$ \ubc88\uc9f8 \ud589\uc5d0 \ub9e4\ud551\ub418\uace0 \ud3b8\ud5a5(Bias)\uc744 \ub354\ud569\ub2c8\ub2e4.\n\uc544\ub798 \uc608\uc2dc\ub97c \uc0b4\ud3b4\ubcf4\uc2ed\uc2dc\uc624.\n\n\n" + "\nPyTorch\ub97c \uc774\uc6a9\ud55c \ub525\ub7ec\ub2dd\n**************************\n**\ubc88\uc5ed**: `\ud669\uc131\uc218 `_\n\n\ub525\ub7ec\ub2dd \ube14\ub85d \uad6c\ucd95 : \uc544\ud540 \ub9f5(affine maps), \ube44\uc120\ud615\uc131, \uac1d\uccb4\n==========================================================================\n\n\ub525\ub7ec\ub2dd\uc740 \uc601\ub9ac\ud55c \ubc29\ubc95\uc73c\ub85c \ube44\uc120\ud615\uc131\uc744 \uac00\uc9c4 \uc120\ud615\uc131\uc744 \uad6c\uc131\ud558\ub294 \uac83\uc73c\ub85c\n\uc774\ub8e8\uc5b4\uc9d1\ub2c8\ub2e4. \ube44\uc120\ud615\uc131\uc758 \ub3c4\uc785\uc740 \uac15\ub825\ud55c \ubaa8\ub378\uc744 \uac00\ub2a5\ud558\uac8c \ud569\ub2c8\ub2e4.\n\uc774 \uc139\uc158\uc5d0\uc11c \uc774 \ud575\uc2ec \uad6c\uc131 \uc694\uc18c\ub97c \ub2e4\ub8e8\uace0, \uac1d\uccb4 \ud568\uc218\ub97c \ub9cc\ub4e4\uace0, \uc5b4\ub5bb\uac8c\n\ubaa8\ub378\uc774 \ud559\uc2b5\ub418\uc9c0\ub294 \uc0b4\ud3b4\ubd05\uc2dc\ub2e4.\n\n\n\uc544\ud540 \ub9f5\n~~~~~~~~~~~\n\n\ub525\ub7ec\ub2dd\uc758 \ud575\uc2ec \uc791\uc5c5\uc790 \uc911 \ud558\ub098\ub294 \uc544\ud540 \ub9f5 \uc785\ub2c8\ub2e4.\n\uc774 \ud568\uc218 $f(x)$ \ub294 \ub2e4\uc74c\uacfc \uac19\uc2b5\ub2c8\ub2e4.\n\n\\begin{align}f(x) = Ax + b\\end{align}\n\n\uc5ec\uae30\uc11c $A$ \ub294 \ud589\ub82c, $x, b$ \ub294 \ubca1\ud130 \uc785\ub2c8\ub2e4.\n\uc5ec\uae30\uc11c \ud559\uc2b5\ub418\ub294 \ubcc0\uc218\ub294 $A$ \uc640 $b$ \uc785\ub2c8\ub2e4.\n\uc885\uc885 $b$ \ub294 *\ud3b8\ud5a5(Bias)* \uc774\ub77c \ubd88\ub9bd\ub2c8\ub2e4.\n\n\nPyTorch \uc640 \ub300\ubd80\ubd84\uc758 \ub2e4\ub978 \ub525\ub7ec\ub2dd \ud504\ub808\uc784\uc6cc\ud06c\ub4e4\uc740 \uace0\uc804\uc801\uc778 \uc120\ud615 \ub300\uc218\ud559\uc640\n\uc870\uae08 \ub2e4\ub974\uac8c \ub3d9\uc791\ud569\ub2c8\ub2e4. \uc785\ub825\uc758 \uc5f4 \ub300\uc2e0\uc5d0 \ud589\uc73c\ub85c \ub9e4\ud551\ud569\ub2c8\ub2e4.\n\uc989 \uc8fc\uc5b4\uc9c4 $A$ \uc5d0\uc11c \ucd9c\ub825\uc758 $i$ \ubc88\uc9f8 \ud589\uc740\n\uc785\ub825\uc758 $i$ \ubc88\uc9f8 \ud589\uc5d0 \ub9e4\ud551\ub418\uace0 \ud3b8\ud5a5(Bias)\uc744 \ub354\ud569\ub2c8\ub2e4.\n\uc544\ub798 \uc608\uc2dc\ub97c \uc0b4\ud3b4\ubcf4\uc2ed\uc2dc\uc624.\n" ] }, { @@ -175,7 +175,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/dd583ee0bd16160dc842469ec717b9d3/audio_preprocessing_tutorial.ipynb b/docs/_downloads/dd583ee0bd16160dc842469ec717b9d3/audio_preprocessing_tutorial.ipynb index f41b66feb..8eb398339 100644 --- a/docs/_downloads/dd583ee0bd16160dc842469ec717b9d3/audio_preprocessing_tutorial.ipynb +++ b/docs/_downloads/dd583ee0bd16160dc842469ec717b9d3/audio_preprocessing_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\ntorchaudio Tutorial\n===================\n\nPyTorch is an open source deep learning platform that provides a\nseamless path from research prototyping to production deployment with\nGPU support.\n\nSignificant effort in solving machine learning problems goes into data\npreparation. ``torchaudio`` leverages PyTorch\u2019s GPU support, and provides\nmany tools to make data loading easy and more readable. In this\ntutorial, we will see how to load and preprocess data from a simple\ndataset.\n\nFor this tutorial, please make sure the ``matplotlib`` package is\ninstalled for easier visualization.\n\n\n" + "\ntorchaudio Tutorial\n===================\n\nPyTorch is an open source deep learning platform that provides a\nseamless path from research prototyping to production deployment with\nGPU support.\n\nSignificant effort in solving machine learning problems goes into data\npreparation. ``torchaudio`` leverages PyTorch\u2019s GPU support, and provides\nmany tools to make data loading easy and more readable. In this\ntutorial, we will see how to load and preprocess data from a simple\ndataset.\n\nFor this tutorial, please make sure the ``matplotlib`` package is\ninstalled for easier visualization.\n" ] }, { @@ -398,7 +398,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/e139fbbcadcc4d83aab8995db4b9147c/saving_loading_models.ipynb b/docs/_downloads/e139fbbcadcc4d83aab8995db4b9147c/saving_loading_models.ipynb index 6f3e42358..3c8645884 100644 --- a/docs/_downloads/e139fbbcadcc4d83aab8995db4b9147c/saving_loading_models.ipynb +++ b/docs/_downloads/e139fbbcadcc4d83aab8995db4b9147c/saving_loading_models.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n\ubaa8\ub378 \uc800\uc7a5\ud558\uae30 & \ubd88\ub7ec\uc624\uae30\n=========================\n**Author:** `Matthew Inkawhich `_\n **\ubc88\uc5ed**: `\ubc15\uc815\ud658 `_\n\n\uc774 \ubb38\uc11c\uc5d0\uc11c\ub294 PyTorch \ubaa8\ub378\uc744 \uc800\uc7a5\ud558\uace0 \ubd88\ub7ec\uc624\ub294 \ub2e4\uc591\ud55c \ubc29\ubc95\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.\n\uc774 \ubb38\uc11c \uc804\uccb4\ub97c \ub2e4 \uc77d\ub294 \uac83\ub3c4 \uc88b\uc740 \ubc29\ubc95\uc774\uc9c0\ub9cc, \ud544\uc694\ud55c \uc0ac\uc6a9 \uc608\uc758 \ucf54\ub4dc\ub9cc \ucc38\uace0\ud558\ub294\n\uac83\ub3c4 \uace0\ub824\ud574\ubcf4\uc138\uc694.\n\n\ubaa8\ub378\uc744 \uc800\uc7a5\ud558\uac70\ub098 \ubd88\ub7ec\uc62c \ub54c\ub294 3\uac00\uc9c0\uc758 \ud575\uc2ec \ud568\uc218\uc640 \uc775\uc219\ud574\uc9c8 \ud544\uc694\uac00 \uc788\uc2b5\ub2c8\ub2e4:\n\n1) `torch.save `__:\n \uc9c1\ub82c\ud654\ub41c \uac1d\uccb4\ub97c \ub514\uc2a4\ud06c\uc5d0 \uc800\uc7a5\ud569\ub2c8\ub2e4. \uc774 \ud568\uc218\ub294 Python\uc758\n `pickle `__ \uc744 \uc0ac\uc6a9\ud558\uc5ec \uc9c1\ub82c\ud654\ud569\ub2c8\ub2e4\n \uc774 \ud568\uc218\ub97c \uc0ac\uc6a9\ud558\uc5ec \ubaa8\ub4e0 \uc885\ub958\uc758 \uac1d\uccb4\uc758 \ubaa8\ub378, Tensor \ubc0f \uc0ac\uc804\uc744 \uc800\uc7a5\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n2) `torch.load `__:\n `pickle `__\\ \uc744 \uc0ac\uc6a9\ud558\uc5ec\n \uc800\uc7a5\ub41c \uac1d\uccb4 \ud30c\uc77c\ub4e4\uc744 \uc5ed\uc9c1\ub82c\ud654\ud558\uc5ec \uba54\ubaa8\ub9ac\uc5d0 \uc62c\ub9bd\ub2c8\ub2e4. \uc774 \ud568\uc218\ub294 \ub370\uc774\ud130\ub97c \uc7a5\uce58\uc5d0\n \ubd88\ub7ec\uc62c \ub54c\ub3c4 \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n (`\uc7a5\uce58\uac04 \ubaa8\ub378 \uc800\uc7a5\ud558\uae30 & \ubd88\ub7ec\uc624\uae30 <#device>`__ \ucc38\uace0)\n\n3) `torch.nn.Module.load_state_dict `__:\n \uc5ed\uc9c1\ub82c\ud654\ub41c *state_dict* \ub97c \uc0ac\uc6a9\ud558\uc5ec \ubaa8\ub378\uc758 \ub9e4\uac1c\ubcc0\uc218\ub4e4\uc744 \ubd88\ub7ec\uc635\ub2c8\ub2e4.\n *state_dict* \uc5d0 \ub300\ud55c \ub354 \uc790\uc138\ud55c \uc815\ubcf4\ub294 `state_dict\uac00 \ubb34\uc5c7\uc778\uac00\uc694?\n <#state-dict>`__ \ub97c \ucc38\uace0\ud558\uc138\uc694.\n\n\n\n**\ubaa9\ucc28:**\n\n- `state_dict\uac00 \ubb34\uc5c7\uc778\uac00\uc694? <#state-dict>`__\n- `\ucd94\ub860(inference)\ub97c \uc704\ud574 \ubaa8\ub378 \uc800\uc7a5\ud558\uae30 & \ubd88\ub7ec\uc624\uae30 <#inference>`__\n- `\uc77c\ubc18 \uccb4\ud06c\ud3ec\uc778\ud2b8(checkpoint) \uc800\uc7a5\ud558\uae30 & \ubd88\ub7ec\uc624\uae30 <#checkpoint>`__\n- `\uc5ec\ub7ec\uac1c(multiple)\uc758 \ubaa8\ub378\uc744 \ud558\ub098\uc758 \ud30c\uc77c\uc5d0 \uc800\uc7a5\ud558\uae30 <#multiple>`__\n- `\ub2e4\ub978 \ubaa8\ub378\uc758 \ub9e4\uac1c\ubcc0\uc218\ub97c \uc0ac\uc6a9\ud558\uc5ec \ube60\ub974\uac8c \ubaa8\ub378 \uc2dc\uc791\ud558\uae30(warmstart) <#warmstart>`__\n- `\uc7a5\uce58(device)\uac04 \ubaa8\ub378 \uc800\uc7a5\ud558\uae30 & \ubd88\ub7ec\uc624\uae30 <#device>`__\n\n\n" + "\n\ubaa8\ub378 \uc800\uc7a5\ud558\uae30 & \ubd88\ub7ec\uc624\uae30\n=========================\n**Author:** `Matthew Inkawhich `_\n **\ubc88\uc5ed**: `\ubc15\uc815\ud658 `_\n\n\uc774 \ubb38\uc11c\uc5d0\uc11c\ub294 PyTorch \ubaa8\ub378\uc744 \uc800\uc7a5\ud558\uace0 \ubd88\ub7ec\uc624\ub294 \ub2e4\uc591\ud55c \ubc29\ubc95\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.\n\uc774 \ubb38\uc11c \uc804\uccb4\ub97c \ub2e4 \uc77d\ub294 \uac83\ub3c4 \uc88b\uc740 \ubc29\ubc95\uc774\uc9c0\ub9cc, \ud544\uc694\ud55c \uc0ac\uc6a9 \uc608\uc758 \ucf54\ub4dc\ub9cc \ucc38\uace0\ud558\ub294\n\uac83\ub3c4 \uace0\ub824\ud574\ubcf4\uc138\uc694.\n\n\ubaa8\ub378\uc744 \uc800\uc7a5\ud558\uac70\ub098 \ubd88\ub7ec\uc62c \ub54c\ub294 3\uac00\uc9c0\uc758 \ud575\uc2ec \ud568\uc218\uc640 \uc775\uc219\ud574\uc9c8 \ud544\uc694\uac00 \uc788\uc2b5\ub2c8\ub2e4:\n\n1) `torch.save `__:\n \uc9c1\ub82c\ud654\ub41c \uac1d\uccb4\ub97c \ub514\uc2a4\ud06c\uc5d0 \uc800\uc7a5\ud569\ub2c8\ub2e4. \uc774 \ud568\uc218\ub294 Python\uc758\n `pickle `__ \uc744 \uc0ac\uc6a9\ud558\uc5ec \uc9c1\ub82c\ud654\ud569\ub2c8\ub2e4\n \uc774 \ud568\uc218\ub97c \uc0ac\uc6a9\ud558\uc5ec \ubaa8\ub4e0 \uc885\ub958\uc758 \uac1d\uccb4\uc758 \ubaa8\ub378, Tensor \ubc0f \uc0ac\uc804\uc744 \uc800\uc7a5\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n2) `torch.load `__:\n `pickle `__\\ \uc744 \uc0ac\uc6a9\ud558\uc5ec\n \uc800\uc7a5\ub41c \uac1d\uccb4 \ud30c\uc77c\ub4e4\uc744 \uc5ed\uc9c1\ub82c\ud654\ud558\uc5ec \uba54\ubaa8\ub9ac\uc5d0 \uc62c\ub9bd\ub2c8\ub2e4. \uc774 \ud568\uc218\ub294 \ub370\uc774\ud130\ub97c \uc7a5\uce58\uc5d0\n \ubd88\ub7ec\uc62c \ub54c\ub3c4 \uc0ac\uc6a9\ud569\ub2c8\ub2e4.\n (`\uc7a5\uce58\uac04 \ubaa8\ub378 \uc800\uc7a5\ud558\uae30 & \ubd88\ub7ec\uc624\uae30 <#device>`__ \ucc38\uace0)\n\n3) `torch.nn.Module.load_state_dict `__:\n \uc5ed\uc9c1\ub82c\ud654\ub41c *state_dict* \ub97c \uc0ac\uc6a9\ud558\uc5ec \ubaa8\ub378\uc758 \ub9e4\uac1c\ubcc0\uc218\ub4e4\uc744 \ubd88\ub7ec\uc635\ub2c8\ub2e4.\n *state_dict* \uc5d0 \ub300\ud55c \ub354 \uc790\uc138\ud55c \uc815\ubcf4\ub294 `state_dict\uac00 \ubb34\uc5c7\uc778\uac00\uc694?\n <#state-dict>`__ \ub97c \ucc38\uace0\ud558\uc138\uc694.\n\n\n\n**\ubaa9\ucc28:**\n\n- `state_dict\uac00 \ubb34\uc5c7\uc778\uac00\uc694? <#state-dict>`__\n- `\ucd94\ub860(inference)\ub97c \uc704\ud574 \ubaa8\ub378 \uc800\uc7a5\ud558\uae30 & \ubd88\ub7ec\uc624\uae30 <#inference>`__\n- `\uc77c\ubc18 \uccb4\ud06c\ud3ec\uc778\ud2b8(checkpoint) \uc800\uc7a5\ud558\uae30 & \ubd88\ub7ec\uc624\uae30 <#checkpoint>`__\n- `\uc5ec\ub7ec\uac1c(multiple)\uc758 \ubaa8\ub378\uc744 \ud558\ub098\uc758 \ud30c\uc77c\uc5d0 \uc800\uc7a5\ud558\uae30 <#multiple>`__\n- `\ub2e4\ub978 \ubaa8\ub378\uc758 \ub9e4\uac1c\ubcc0\uc218\ub97c \uc0ac\uc6a9\ud558\uc5ec \ube60\ub974\uac8c \ubaa8\ub378 \uc2dc\uc791\ud558\uae30(warmstart) <#warmstart>`__\n- `\uc7a5\uce58(device)\uac04 \ubaa8\ub378 \uc800\uc7a5\ud558\uae30 & \ubd88\ub7ec\uc624\uae30 <#device>`__\n" ] }, { @@ -77,7 +77,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/e3e6e72b3ed19d4dd77f128e92beda52/profiler.ipynb b/docs/_downloads/e3e6e72b3ed19d4dd77f128e92beda52/profiler.ipynb index 61049d068..7ebb86eba 100644 --- a/docs/_downloads/e3e6e72b3ed19d4dd77f128e92beda52/profiler.ipynb +++ b/docs/_downloads/e3e6e72b3ed19d4dd77f128e92beda52/profiler.ipynb @@ -15,14 +15,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nPyTorch Profiler\n====================================\nThis recipe explains how to use PyTorch profiler and measure the time and\nmemory consumption of the model's operators.\n\nIntroduction\n------------\nPyTorch includes a simple profiler API that is useful when user needs\nto determine the most expensive operators in the model.\n\nIn this recipe, we will use a simple Resnet model to demonstrate how to\nuse profiler to analyze model performance.\n\nSetup\n-----\nTo install ``torch`` and ``torchvision`` use the following command:\n\n::\n\n pip install torch torchvision\n\n\n\n" + "\nPyTorch \ud504\ub85c\ud30c\uc77c\ub7ec(Profiler)\n====================================\n\uc774 \ub808\uc2dc\ud53c\uc5d0\uc11c\ub294 \uc5b4\ub5bb\uac8c PyTorch \ud504\ub85c\ud30c\uc77c\ub7ec\ub97c \uc0ac\uc6a9\ud558\ub294\uc9c0, \uadf8\ub9ac\uace0 \ubaa8\ub378\uc758 \uc5f0\uc0b0\uc790\ub4e4\uc774 \uc18c\ube44\ud558\ub294 \uba54\ubaa8\ub9ac\uc640 \uc2dc\uac04\uc744 \uce21\uc815\ud558\ub294 \ubc29\ubc95\uc744 \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uac1c\uc694\n------------\nPyTorch\ub294 \uc0ac\uc6a9\uc790\uac00 \ubaa8\ub378 \ub0b4\uc758 \uc5f0\uc0b0 \ube44\uc6a9\uc774 \ud070(expensive) \uc5f0\uc0b0\uc790\ub4e4\uc774 \ubb34\uc5c7\uc778\uc9c0 \uc54c\uace0\uc2f6\uc744 \ub54c \uc720\uc6a9\ud558\uac8c \uc0ac\uc6a9\ud560 \uc218 \uc788\ub294 \uac04\ub2e8\ud55c \ud504\ub85c\ud30c\uc77c\ub7ec API\ub97c \ud3ec\ud568\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n\n\uc774 \ub808\uc2dc\ud53c\uc5d0\uc11c\ub294 \ubaa8\ub378\uc758 \uc131\ub2a5(performance)\uc744 \ubd84\uc11d\ud558\ub824\uace0 \ud560 \ub54c \uc5b4\ub5bb\uac8c \ud504\ub85c\ud30c\uc77c\ub7ec\ub97c \uc0ac\uc6a9\ud574\uc57c \ud558\ub294\uc9c0\ub97c \ubcf4\uc5ec\uc8fc\uae30 \uc704\ud574 \uac04\ub2e8\ud55c ResNet \ubaa8\ub378\uc744 \uc0ac\uc6a9\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc124\uc815(Setup)\n-------------\n``torch`` \uc640 ``torchvision`` \uc744 \uc124\uce58\ud558\uae30 \uc704\ud574\uc11c \uc544\ub798\uc758 \ucee4\ub9e8\ub4dc\ub97c \uc785\ub825\ud569\ub2c8\ub2e4:\n\n\n::\n\n pip install torch torchvision\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Steps\n-----\n\n1. Import all necessary libraries\n2. Instantiate a simple Resnet model\n3. Use profiler to analyze execution time\n4. Use profiler to analyze memory consumption\n5. Using tracing functionality\n\n1. Import all necessary libraries\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nIn this recipe we will use ``torch``, ``torchvision.models``\nand ``profiler`` modules:\n\n\n" + "\ub2e8\uacc4(Steps)\n-------------\n\n1. \ud544\uc694\ud55c \ub77c\uc774\ube0c\ub7ec\ub9ac\ub4e4 \ubd88\ub7ec\uc624\uae30\n2. \uac04\ub2e8\ud55c ResNet \ubaa8\ub378 \uc778\uc2a4\ud134\uc2a4\ud654 \ud558\uae30\n3. \ud504\ub85c\ud30c\uc77c\ub7ec\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc2e4\ud589\uc2dc\uac04 \ubd84\uc11d\ud558\uae30\n4. \ud504\ub85c\ud30c\uc77c\ub7ec\ub97c \uc0ac\uc6a9\ud558\uc5ec \uba54\ubaa8\ub9ac \uc18c\ube44 \ubd84\uc11d\ud558\uae30\n5. \ucd94\uc801\uae30\ub2a5 \uc0ac\uc6a9\ud558\uae30\n\n1. \ud544\uc694\ud55c \ub77c\uc774\ube0c\ub7ec\ub9ac\ub4e4 \ubd88\ub7ec\uc624\uae30\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n\uc774 \ub808\uc2dc\ud53c\uc5d0\uc11c\ub294 ``torch`` \uc640 ``torchvision.models``,\n\uadf8\ub9ac\uace0 ``profiler`` \ubaa8\ub4c8\uc744 \uc0ac\uc6a9\ud569\ub2c8\ub2e4:\n\n\n" ] }, { @@ -40,7 +40,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "2. Instantiate a simple Resnet model\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nLet's create an instance of a Resnet model and prepare an input\nfor it:\n\n\n" + "2. \uac04\ub2e8\ud55c ResNet \ubaa8\ub378 \uc778\uc2a4\ud134\uc2a4\ud654 \ud558\uae30\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nResNet \ubaa8\ub378 \uc778\uc2a4\ud134\uc2a4\ub97c \ub9cc\ub4e4\uace0 \uc785\ub825\uac12\uc744\n\uc900\ube44\ud569\ub2c8\ub2e4 :\n\n\n" ] }, { @@ -58,7 +58,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "3. Use profiler to analyze execution time\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nPyTorch profiler is enabled through the context manager and accepts\na number of parameters, some of the most useful are:\n\n- ``record_shapes`` - whether to record shapes of the operator inputs;\n- ``profile_memory`` - whether to report amount of memory consumed by\n model's Tensors;\n- ``use_cuda`` - whether to measure execution time of CUDA kernels.\n\nLet's see how we can use profiler to analyze the execution time:\n\n" + "3. \ud504\ub85c\ud30c\uc77c\ub7ec\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc2e4\ud589\uc2dc\uac04 \ubd84\uc11d\ud558\uae30\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nPyTorch \ud504\ub85c\ud30c\uc77c\ub7ec\ub294 \ucee8\ud14d\uc2a4\ud2b8 \uba54\ub2c8\uc800(context manager)\ub97c \ud1b5\ud574 \ud65c\uc131\ud654\ub418\uace0,\n\uc5ec\ub7ec \ub9e4\uac1c\ubcc0\uc218\ub97c \ubc1b\uc744 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc720\uc6a9\ud55c \uba87 \uac00\uc9c0 \ub9e4\uac1c\ubcc0\uc218\ub294 \ub2e4\uc74c\uacfc \uac19\uc2b5\ub2c8\ub2e4:\n\n- ``record_shapes`` - \uc5f0\uc0ac\uc790 \uc785\ub825(input)\uc758 shape\uc744 \uae30\ub85d\ud560\uc9c0 \uc5ec\ubd80;\n- ``profile_memory`` - \ubaa8\ub378\uc758 \ud150\uc11c(Tensor)\ub4e4\uc774 \uc18c\ube44\ud558\ub294 \uba54\ubaa8\ub9ac \uc591\uc744 \ubcf4\uace0(report)\ud560\uc9c0 \uc5ec\ubd80;\n- ``use_cuda`` - CUDA \ucee4\ub110\uc758 \uc2e4\ud589\uc2dc\uac04\uc744 \uce21\uc815\ud560\uc9c0 \uc5ec\ubd80;\n\n\ud504\ub85c\ud30c\uc77c\ub7ec\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc5b4\ub5bb\uac8c \uc2e4\ud589\uc2dc\uac04\uc744 \ubd84\uc11d\ud558\ub294\uc9c0 \ubcf4\uaca0\uc2b5\ub2c8\ub2e4:\n\n" ] }, { @@ -76,7 +76,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Note that we can use ``record_function`` context manager to label\narbitrary code ranges with user provided names\n(``model_inference`` is used as a label in the example above).\nProfiler allows one to check which operators were called during the\nexecution of a code range wrapped with a profiler context manager.\nIf multiple profiler ranges are active at the same time (e.g. in\nparallel PyTorch threads), each profiling context manager tracks only\nthe operators of its corresponding range.\nProfiler also automatically profiles the async tasks launched\nwith ``torch.jit._fork`` and (in case of a backward pass)\nthe backward pass operators launched with ``backward()`` call.\n\nLet's print out the stats for the execution above:\n\n" + "``record_function`` \ucee8\ud14d\uc2a4\ud2b8 \uad00\ub9ac\uc790\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc784\uc758\uc758 \ucf54\ub4dc \ubc94\uc704\uc5d0\n\uc0ac\uc6a9\uc790\uac00 \uc9c0\uc815\ud55c \uc774\ub984\uc73c\ub85c \ub808\uc774\ube14(label)\uc744 \ud45c\uc2dc\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n(\uc704 \uc608\uc81c\uc5d0\uc11c\ub294 ``model_inference`` \ub97c \ub808\uc774\ube14\ub85c \uc0ac\uc6a9\ud588\uc2b5\ub2c8\ub2e4.)\n\ud504\ub85c\ud30c\uc77c\ub7ec\ub97c \uc0ac\uc6a9\ud558\uba74 \ud504\ub85c\ud30c\uc77c\ub7ec \ucee8\ud14d\uc2a4\ud2b8 \uad00\ub9ac\uc790\ub85c \uac10\uc2f8\uc9c4(wrap) \ucf54\ub4dc \ubc94\uc704\ub97c\n\uc2e4\ud589\ud558\ub294 \ub3d9\uc548 \uc5b4\ub5a4 \uc5f0\uc0b0\uc790\ub4e4\uc774 \ud638\ucd9c\ub418\uc5c8\ub294\uc9c0 \ud655\uc778\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\ub9cc\uc57d \uc5ec\ub7ec \ud504\ub85c\ud30c\uc77c\ub7ec\uc758 \ubc94\uc704\uac00 \ub3d9\uc2dc\uc5d0 \ud65c\uc131\ud654\ub41c \uacbd\uc6b0(\uc608. PyTorch \uc4f0\ub808\ub4dc\uac00 \ubcd1\ub82c\ub85c\n\uc2e4\ud589 \uc911\uc778 \uacbd\uc6b0), \uac01 \ud504\ub85c\ud30c\uc77c\ub9c1 \ucee8\ud14d\uc2a4\ud2b8 \uad00\ub9ac\uc790\ub294 \uac01\uac01\uc758 \ubc94\uc704 \ub0b4\uc758 \uc5f0\uc0b0\uc790\ub4e4\ub9cc\n\ucd94\uc801(track)\ud569\ub2c8\ub2e4.\n\ud504\ub85c\ud30c\uc77c\ub7ec\ub294 \ub610\ud55c ``torch.jit._fork`` \ub85c \uc2e4\ud589\ub41c \ube44\ub3d9\uae30 \uc791\uc5c5\uacfc\n(\uc5ed\uc804\ud30c \ub2e8\uacc4\uc758 \uacbd\uc6b0) ``backward()`` \uc758 \ud638\ucd9c\ub85c \uc2e4\ud589\ub41c \uc5ed\uc804\ud30c \uc5f0\uc0b0\uc790\ub4e4\ub3c4\n\uc790\ub3d9\uc73c\ub85c \ud504\ub85c\ud30c\uc77c\ub9c1\ud569\ub2c8\ub2e4.\n\n\uc704 \ucf54\ub4dc\ub97c \uc2e4\ud589\ud55c \ud1b5\uacc4\ub97c \ucd9c\ub825\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4:\n\n" ] }, { @@ -94,7 +94,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The output will look like (omitting some columns):\n\n" + "(\uba87\uba87 \uc5f4\uc744 \uc81c\uc678\ud558\uace0) \ucd9c\ub825\uac12\uc774 \uc774\ub807\uac8c \ubcf4\uc77c \uac83\uc785\ub2c8\ub2e4:\n\n" ] }, { @@ -112,7 +112,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Here we see that, as expected, most of the time is spent in convolution (and specifically in ``mkldnn_convolution``\nfor PyTorch compiled with MKL-DNN support).\nNote the difference between self cpu time and cpu time - operators can call other operators, self cpu time exludes time\nspent in children operator calls, while total cpu time includes it.\n\nTo get a finer granularity of results and include operator input shapes, pass ``group_by_input_shape=True``:\n\n" + "\uc608\uc0c1\ud588\ub358 \ub300\ub85c, \ub300\ubd80\ubd84\uc758 \uc2dc\uac04\uc774 \ud569\uc131\uacf1(convolution) \uc5f0\uc0b0(\ud2b9\ud788 MKL-DNN\uc744 \uc9c0\uc6d0\ud558\ub3c4\ub85d\n\ucef4\ud30c\uc77c\ub41c PyTorch\uc758 \uacbd\uc6b0\uc5d0\ub294 ``mkldnn_convolution`` )\uc5d0\uc11c \uc18c\uc694\ub418\ub294 \uac83\uc744 \ud655\uc778\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n(\uacb0\uacfc \uc5f4\ub4e4 \uc911) Self CPU time\uacfc CPU time\uc758 \ucc28\uc774\uc5d0 \uc720\uc758\ud574\uc57c \ud569\ub2c8\ub2e4 -\n\uc5f0\uc0b0\uc790\ub294 \ub2e4\ub978 \uc5f0\uc0b0\uc790\ub4e4\uc744 \ud638\ucd9c\ud560 \uc218 \uc788\uc73c\uba70, Self CPU time\uc5d0\ub294 \ud558\uc704(child) \uc5f0\uc0b0\uc790 \ud638\ucd9c\uc5d0\uc11c \ubc1c\uc0dd\ud55c\n\uc2dc\uac04\uc744 \uc81c\uc678\ud574\uc11c, Totacl CPU time\uc5d0\ub294 \ud3ec\ud568\ud574\uc11c \ud45c\uc2dc\ud569\ub2c8\ub2e4.\n\n\ubcf4\ub2e4 \uc138\ubd80\uc801\uc778 \uacb0\uacfc \uc815\ubcf4 \ubc0f \uc5f0\uc0b0\uc790\uc758 \uc785\ub825 shape\uc744 \ud568\uaed8 \ubcf4\ub824\uba74 ``group_by_input_shape=True`` \ub97c\n\uc778\uc790\ub85c \uc804\ub2ec\ud558\uba74 \ub429\ub2c8\ub2e4:\n\n" ] }, { @@ -123,14 +123,14 @@ }, "outputs": [], "source": [ - "print(prof.key_averages(group_by_input_shape=True).table(sort_by=\"cpu_time_total\", row_limit=10))\n\n# (omitting some columns)\n# ------------------------- ----------- -------- -------------------------------------\n# Name CPU total # Calls Input Shapes\n# ------------------------- ----------- -------- -------------------------------------\n# model_inference 69.571ms 1 []\n# conv2d 9.019ms 4 [[5, 64, 56, 56], [64, 64, 3, 3], []]\n# convolution 9.006ms 4 [[5, 64, 56, 56], [64, 64, 3, 3], []]\n# _convolution 8.982ms 4 [[5, 64, 56, 56], [64, 64, 3, 3], []]\n# mkldnn_convolution 8.894ms 4 [[5, 64, 56, 56], [64, 64, 3, 3], []]\n# max_pool2d 7.200ms 1 [[5, 64, 112, 112]]\n# conv2d 7.189ms 3 [[5, 512, 7, 7], [512, 512, 3, 3], []]\n# convolution 7.180ms 3 [[5, 512, 7, 7], [512, 512, 3, 3], []]\n# _convolution 7.171ms 3 [[5, 512, 7, 7], [512, 512, 3, 3], []]\n# max_pool2d_with_indices 7.170ms 1 [[5, 64, 112, 112]]\n# ------------------------- ----------- -------- --------------------------------------" + "print(prof.key_averages(group_by_input_shape=True).table(sort_by=\"cpu_time_total\", row_limit=10))\n\n# (\uba87\uba87 \uc5f4\uc740 \uc81c\uc678\ud558\uc600\uc2b5\ub2c8\ub2e4)\n# ------------------------- ----------- -------- -------------------------------------\n# Name CPU total # Calls Input Shapes\n# ------------------------- ----------- -------- -------------------------------------\n# model_inference 69.571ms 1 []\n# conv2d 9.019ms 4 [[5, 64, 56, 56], [64, 64, 3, 3], []]\n# convolution 9.006ms 4 [[5, 64, 56, 56], [64, 64, 3, 3], []]\n# _convolution 8.982ms 4 [[5, 64, 56, 56], [64, 64, 3, 3], []]\n# mkldnn_convolution 8.894ms 4 [[5, 64, 56, 56], [64, 64, 3, 3], []]\n# max_pool2d 7.200ms 1 [[5, 64, 112, 112]]\n# conv2d 7.189ms 3 [[5, 512, 7, 7], [512, 512, 3, 3], []]\n# convolution 7.180ms 3 [[5, 512, 7, 7], [512, 512, 3, 3], []]\n# _convolution 7.171ms 3 [[5, 512, 7, 7], [512, 512, 3, 3], []]\n# max_pool2d_with_indices 7.170ms 1 [[5, 64, 112, 112]]\n# ------------------------- ----------- -------- --------------------------------------" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "4. Use profiler to analyze memory consumption\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nPyTorch profiler can also show the amount of memory (used by the model's tensors)\nthat was allocated (or released) during the execution of the model's operators.\nIn the output below, 'self' memory corresponds to the memory allocated (released)\nby the operator, excluding the children calls to the other operators.\nTo enable memory profiling functionality pass ``profile_memory=True``.\n\n" + "4. \ud504\ub85c\ud30c\uc77c\ub7ec\ub97c \uc0ac\uc6a9\ud558\uc5ec \uba54\ubaa8\ub9ac \uc18c\ube44 \ubd84\uc11d\ud558\uae30\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nPyTorch \ud504\ub85c\ud30c\uc77c\ub7ec\ub294 \ubaa8\ub378\uc758 \uc5f0\uc0b0\uc790\ub4e4\uc744 \uc2e4\ud589\ud558\uba70 (\ubaa8\ub378\uc758 \ud150\uc11c\ub4e4\uc774 \uc0ac\uc6a9\ud558\uba70) \ud560\ub2f9(\ub610\ub294 \ud574\uc81c)\ud55c\n\uba54\ubaa8\ub9ac\uc758 \uc591\ub3c4 \ud45c\uc2dc\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uc544\ub798 \ucd9c\ub825 \uacb0\uacfc\uc5d0\uc11c 'Self' memory\ub294 \ud574\ub2f9 \uc5f0\uc0b0\uc790\uc5d0 \uc758\ud574 \ud638\ucd9c\ub41c \ud558\uc704(child) \uc5f0\uc0b0\uc790\ub4e4\uc744 \uc81c\uc678\ud55c,\n\uc5f0\uc0b0\uc790 \uc790\uccb4\uc5d0 \ud560\ub2f9(\ud574\uc81c)\ub41c \uba54\ubaa8\ub9ac\uc5d0 \ud574\ub2f9\ud569\ub2c8\ub2e4.\n\uba54\ubaa8\ub9ac \ud504\ub85c\ud30c\uc77c\ub9c1 \uae30\ub2a5\uc744 \ud65c\uc131\ud654\ud558\ub824\uba74 ``profile_memory=True`` \ub97c \uc778\uc790\ub85c \uc804\ub2ec\ud558\uba74 \ub429\ub2c8\ub2e4.\n\n" ] }, { @@ -141,14 +141,14 @@ }, "outputs": [], "source": [ - "with profiler.profile(profile_memory=True, record_shapes=True) as prof:\n model(inputs)\n\nprint(prof.key_averages().table(sort_by=\"self_cpu_memory_usage\", row_limit=10))\n\n# (omitting some columns)\n# --------------------------- --------------- --------------- ---------------\n# Name CPU Mem Self CPU Mem Number of Calls\n# --------------------------- --------------- --------------- ---------------\n# empty 94.79 Mb 94.79 Mb 123\n# resize_ 11.48 Mb 11.48 Mb 2\n# addmm 19.53 Kb 19.53 Kb 1\n# empty_strided 4 b 4 b 1\n# conv2d 47.37 Mb 0 b 20\n# --------------------------- --------------- --------------- ---------------\n\nprint(prof.key_averages().table(sort_by=\"cpu_memory_usage\", row_limit=10))\n\n# (omitting some columns)\n# --------------------------- --------------- --------------- ---------------\n# Name CPU Mem Self CPU Mem Number of Calls\n# --------------------------- --------------- --------------- ---------------\n# empty 94.79 Mb 94.79 Mb 123\n# batch_norm 47.41 Mb 0 b 20\n# _batch_norm_impl_index 47.41 Mb 0 b 20\n# native_batch_norm 47.41 Mb 0 b 20\n# conv2d 47.37 Mb 0 b 20\n# convolution 47.37 Mb 0 b 20\n# _convolution 47.37 Mb 0 b 20\n# mkldnn_convolution 47.37 Mb 0 b 20\n# empty_like 47.37 Mb 0 b 20\n# max_pool2d 11.48 Mb 0 b 1\n# max_pool2d_with_indices 11.48 Mb 0 b 1\n# resize_ 11.48 Mb 11.48 Mb 2\n# addmm 19.53 Kb 19.53 Kb 1\n# adaptive_avg_pool2d 10.00 Kb 0 b 1\n# mean 10.00 Kb 0 b 1\n# --------------------------- --------------- --------------- ---------------" + "with profiler.profile(profile_memory=True, record_shapes=True) as prof:\n model(inputs)\n\nprint(prof.key_averages().table(sort_by=\"self_cpu_memory_usage\", row_limit=10))\n\n# (\uba87\uba87 \uc5f4\uc740 \uc81c\uc678\ud558\uc600\uc2b5\ub2c8\ub2e4)\n# --------------------------- --------------- --------------- ---------------\n# Name CPU Mem Self CPU Mem Number of Calls\n# --------------------------- --------------- --------------- ---------------\n# empty 94.79 Mb 94.79 Mb 123\n# resize_ 11.48 Mb 11.48 Mb 2\n# addmm 19.53 Kb 19.53 Kb 1\n# empty_strided 4 b 4 b 1\n# conv2d 47.37 Mb 0 b 20\n# --------------------------- --------------- --------------- ---------------\n\nprint(prof.key_averages().table(sort_by=\"cpu_memory_usage\", row_limit=10))\n\n# (\uba87\uba87 \uc5f4\uc740 \uc81c\uc678\ud558\uc600\uc2b5\ub2c8\ub2e4)\n# --------------------------- --------------- --------------- ---------------\n# Name CPU Mem Self CPU Mem Number of Calls\n# --------------------------- --------------- --------------- ---------------\n# empty 94.79 Mb 94.79 Mb 123\n# batch_norm 47.41 Mb 0 b 20\n# _batch_norm_impl_index 47.41 Mb 0 b 20\n# native_batch_norm 47.41 Mb 0 b 20\n# conv2d 47.37 Mb 0 b 20\n# convolution 47.37 Mb 0 b 20\n# _convolution 47.37 Mb 0 b 20\n# mkldnn_convolution 47.37 Mb 0 b 20\n# empty_like 47.37 Mb 0 b 20\n# max_pool2d 11.48 Mb 0 b 1\n# max_pool2d_with_indices 11.48 Mb 0 b 1\n# resize_ 11.48 Mb 11.48 Mb 2\n# addmm 19.53 Kb 19.53 Kb 1\n# adaptive_avg_pool2d 10.00 Kb 0 b 1\n# mean 10.00 Kb 0 b 1\n# --------------------------- --------------- --------------- ---------------" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "5. Using tracing functionality\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nProfiling results can be outputted as a .json trace file:\n\n" + "5. \ucd94\uc801\uae30\ub2a5 \uc0ac\uc6a9\ud558\uae30\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n\ud504\ub85c\ud30c\uc77c\ub9c1 \uacb0\uacfc\ub294 .json \ud615\ud0dc\uc758 \ucd94\uc801 \ud30c\uc77c(trace file)\ub85c \ucd9c\ub825\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4:\n\n" ] }, { @@ -166,14 +166,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "User can examine the sequence of profiled operators after loading the trace file\nin Chrome (``chrome://tracing``):\n\n![](../../_static/img/trace_img.png)\n\n :scale: 25 %\n\n" + "\uc0ac\uc6a9\uc790\ub294 Chrome \ube0c\ub77c\uc6b0\uc800( ``chrome://tracing`` )\uc5d0\uc11c \ucd94\uc801 \ud30c\uc77c\uc744 \ubd88\ub7ec\uc640\n\ud504\ub85c\ud30c\uc77c\ub41c \uc77c\ub828\uc758 \uc5f0\uc0b0\uc790\ub4e4\uc744 \uac80\ud1a0\ud574\ubcfc \uc218 \uc788\uc2b5\ub2c8\ub2e4:\n\n![](../../_static/img/trace_img.png)\n\n :scale: 25 %\n\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Learn More\n----------\n\nTake a look at the following tutorial to learn how to visualize your model with TensorBoard:\n\n- `Visualizing models, data, and training with TensorBoard `_ tutorial\n\n\n" + "\ub354 \uc54c\uc544\ubcf4\uae30\n-------------\n\n\ub2e4\uc74c\uc758 \ud29c\ud1a0\ub9ac\uc5bc\uc744 \ud1b5\ud574 Tensorboard\ub97c \uc0ac\uc6a9\ud558\uc5ec \ubaa8\ub378 \uc2dc\uac01\ud654\ud558\ub294 \ubc29\ubc95\uc744 \uc0b4\ud3b4\ubcf4\uc138\uc694:\n\n- :doc:`/intermediate/tensorboard_tutorial`\n\n\n" ] } ], @@ -193,7 +193,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/e3f9fff2f30691a0831549aa6b6712f5/two_layer_net_module.ipynb b/docs/_downloads/e3f9fff2f30691a0831549aa6b6712f5/two_layer_net_module.ipynb index c3a50bfe5..e5843b806 100644 --- a/docs/_downloads/e3f9fff2f30691a0831549aa6b6712f5/two_layer_net_module.ipynb +++ b/docs/_downloads/e3f9fff2f30691a0831549aa6b6712f5/two_layer_net_module.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nPyTorch: \uc0ac\uc6a9\uc790 \uc815\uc758 nn Module\n-------------------------------\n\n\ud558\ub098\uc758 \uc740\ub2c9\uce35(hidden layer)\uacfc \ud3b8\ud5a5(bias)\uc774 \uc5c6\ub294 \uc644\uc804\ud788 \uc5f0\uacb0\ub41c ReLU \uc2e0\uacbd\ub9dd\uc744,\n\uc720\ud074\ub9ac\ub4dc \uac70\ub9ac(Euclidean distance) \uc81c\uacf1\uc744 \ucd5c\uc18c\ud654\ud558\ub294 \uc2dd\uc73c\ub85c x\ub85c\ubd80\ud130 y\ub97c \uc608\uce21\ud558\ub3c4\ub85d\n\ud559\uc2b5\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc774\ubc88\uc5d0\ub294 \uc0ac\uc6a9\uc790 \uc815\uc758 Module\uc758 \uc11c\ube0c\ud074\ub798\uc2a4\ub85c \ubaa8\ub378\uc744 \uc815\uc758\ud569\ub2c8\ub2e4. \uae30\uc874 Module\uc758 \uac04\ub2e8\ud55c\n\uad6c\uc131\ubcf4\ub2e4 \ub354 \ubcf5\uc7a1\ud55c \ubaa8\ub378\uc744 \uc6d0\ud55c\ub2e4\uba74, \uc774 \ubc29\ubc95\uc73c\ub85c \ubaa8\ub378\uc744 \uc815\uc758\ud558\uba74 \ub429\ub2c8\ub2e4.\n\n" + "\nPyTorch: \uc0ac\uc6a9\uc790 \uc815\uc758 nn Module\n-------------------------------\n\n\ud558\ub098\uc758 \uc740\ub2c9\uce35(hidden layer)\uacfc \ud3b8\ud5a5(bias)\uc774 \uc5c6\ub294 \uc644\uc804\ud788 \uc5f0\uacb0\ub41c ReLU \uc2e0\uacbd\ub9dd\uc744,\n\uc720\ud074\ub9ac\ub4dc \uac70\ub9ac(Euclidean distance) \uc81c\uacf1\uc744 \ucd5c\uc18c\ud654\ud558\ub294 \uc2dd\uc73c\ub85c x\ub85c\ubd80\ud130 y\ub97c \uc608\uce21\ud558\ub3c4\ub85d\n\ud559\uc2b5\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc774\ubc88\uc5d0\ub294 \uc0ac\uc6a9\uc790 \uc815\uc758 Module\uc758 \uc11c\ube0c\ud074\ub798\uc2a4\ub85c \ubaa8\ub378\uc744 \uc815\uc758\ud569\ub2c8\ub2e4. \uae30\uc874 Module\uc758 \uac04\ub2e8\ud55c\n\uad6c\uc131\ubcf4\ub2e4 \ub354 \ubcf5\uc7a1\ud55c \ubaa8\ub378\uc744 \uc6d0\ud55c\ub2e4\uba74, \uc774 \ubc29\ubc95\uc73c\ub85c \ubaa8\ub378\uc744 \uc815\uc758\ud558\uba74 \ub429\ub2c8\ub2e4.\n" ] }, { @@ -46,7 +46,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/e72ffa33b709a3d3f5b3f4c195c07637/aws_distributed_training_tutorial.ipynb b/docs/_downloads/e72ffa33b709a3d3f5b3f4c195c07637/aws_distributed_training_tutorial.ipynb index 0c830156a..912624eb0 100644 --- a/docs/_downloads/e72ffa33b709a3d3f5b3f4c195c07637/aws_distributed_training_tutorial.ipynb +++ b/docs/_downloads/e72ffa33b709a3d3f5b3f4c195c07637/aws_distributed_training_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n(advanced) PyTorch 1.0 Distributed Trainer with Amazon AWS\n=============================================================\n\n**Author**: `Nathan Inkawhich `_\n\n**Edited by**: `Teng Li `_\n\n\n" + "\n(advanced) PyTorch 1.0 Distributed Trainer with Amazon AWS\n=============================================================\n\n**Author**: `Nathan Inkawhich `_\n\n**Edited by**: `Teng Li `_\n" ] }, { @@ -232,7 +232,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/e733d8cec5d7c07a409a12a4273a4a28/torchtext_translation_tutorial.py b/docs/_downloads/e733d8cec5d7c07a409a12a4273a4a28/torchtext_translation_tutorial.py index 61f2a0a59..68b64682e 100644 --- a/docs/_downloads/e733d8cec5d7c07a409a12a4273a4a28/torchtext_translation_tutorial.py +++ b/docs/_downloads/e733d8cec5d7c07a409a12a4273a4a28/torchtext_translation_tutorial.py @@ -5,7 +5,7 @@ 이 튜토리얼에서는 ``torchtext`` 의 유용한 여러 클래스들과 시퀀스 투 시퀀스(sequence-to-sequence, seq2seq)모델을 통해 영어와 독일어 문장들이 포함된 유명한 데이터 셋을 이용해서 독일어 문장을 영어로 번역해 볼 것입니다. -이 튜토리얼은 +이 튜토리얼은 PyTorch 커뮤니티 멤버인 `Ben Trevett `__ 이 작성한 `튜토리얼 `__ 에 기초하고 있으며 `Seth Weidman `__ 이 Ben의 허락을 받고 만들었습니다. @@ -16,14 +16,16 @@ - `TranslationDataset `__ - `Field `__ - `BucketIterator `__ + """ ###################################################################### # `Field` 와 `TranslationDataset` -# ---------------- +# -------------------------------- +# # ``torchtext`` 에는 언어 변환 모델을 만들때 쉽게 사용할 수 있는 데이터셋을 만들기 적합한 다양한 도구가 있습니다. # 그 중에서도 중요한 클래스 중 하나인 `Field `__ 는 -# 각 문장이 어떻게 전처리되어야 하는지 지정하며, 또 다른 중요한 클래스로는 `TranslationDataset` 이 있습니다. +# 각 문장이 어떻게 전처리되어야 하는지 지정하며, 또 다른 중요한 클래스로는 `TranslationDataset` 이 있습니다. # ``torchtext`` 에는 이 외에도 비슷한 데이터셋들이 있는데, 이번 튜토리얼에서는 `Multi30k dataset `__ 을 사용할 것입니다. # 이 데이터 셋은 평균 약 13개의 단어로 구성된 약 삼만 개의 문장을 영어와 독일어 두 언어로 포함하고 있습니다. # @@ -43,6 +45,7 @@ # # Spacy가 설치되어 있다면, 다음 코드는 ``TranslationDataset`` 에 있는 각 문장을 ``Field`` 에 정의된 # 내용을 기반으로 토큰화할 것입니다. + from torchtext.datasets import Multi30k from torchtext.data import Field, BucketIterator @@ -75,7 +78,7 @@ ###################################################################### # ``BucketIterator`` -# ---------------- +# -------------------- # 마지막으로 사용해 볼 ``torchtext`` 에 특화된 기능은 바로 ``BucketIterator`` 입니다. # 첫 번째 인자로 ``TranslationDataset`` 을 전달받기 때문에 사용하기가 쉽습니다. 문서에서도 볼 수 있듯 # 이 기능은 비슷한 길이의 예제들을 묶어주는 반복자(iterator)를 정의합니다. 각각의 새로운 에포크(epoch)마다 @@ -93,7 +96,7 @@ device = device) ###################################################################### -# 이 반복자들은 ``DataLoader`` 와 마찬가지로 호출할 수 있습니다. 아래 ``train`` 과 +# 이 반복자들은 ``DataLoader`` 와 마찬가지로 호출할 수 있습니다. 아래 ``train`` 과 # ``evaluation`` 함수에서 보면, 다음과 같이 간단히 호출할 수 있음을 알 수 있습니다 : # :: # @@ -108,15 +111,15 @@ ###################################################################### # ``nn.Module`` 과 ``Optimizer`` 정의하기 -# ---------------- +# ------------------------------------------ # 대부분은 ``torchtext`` 가 알아서 해줍니다 : 데이터셋이 만들어지고 반복자가 정의되면, 이 튜토리얼에서 # 우리가 해야 할 일이라고는 그저 ``nn.Module`` 와 ``Optimizer`` 를 모델로서 정의하고 훈련시키는 것이 전부입니다. -# +# # # 이 튜토리얼에서 사용할 모델은 `이곳 `__ 에서 설명하고 있는 구조를 따르고 있으며, -# 더 자세한 내용은 `여기 `__ +# 더 자세한 내용은 `여기 `__ # 를 참고하시기 바랍니다. -# +# # 참고 : 이 튜토리얼에서 사용하는 모델은 언어 번역을 위해 사용할 예시 모델입니다. 이 모델을 사용하는 것은 # 이 작업에 적당한 표준 모델이기 때문이지, 번역에 적합한 모델이기 때문은 아닙니다. 여러분이 최신 기술 트렌드를 # 잘 따라가고 있다면 잘 아시겠지만, 현재 번역에서 가장 뛰어난 모델은 Transformers입니다. PyTorch가 diff --git a/docs/_downloads/e85264945029fe236addcb864bf5f13f/data_parallel_tutorial.py b/docs/_downloads/e85264945029fe236addcb864bf5f13f/data_parallel_tutorial.py index ae3bbe337..82b4fc991 100644 --- a/docs/_downloads/e85264945029fe236addcb864bf5f13f/data_parallel_tutorial.py +++ b/docs/_downloads/e85264945029fe236addcb864bf5f13f/data_parallel_tutorial.py @@ -1,6 +1,7 @@ """ 선택 사항: 데이터 병렬 처리 (Data Parallelism) -========================== +==================================================== + **글쓴이**: `Sung Kim `_ and `Jenny Kang `_ **번역**: '정아진 ' @@ -19,7 +20,7 @@ mytensor = my_tensor.to(device) -''my_tensor.to(device)'' 를 호출 시 에는 ''my_tensor'' 를 다시쓰는 대신 ''my_tensor'' 의 또다른 복사본이 생긴다는 사실을 기억하십시오. +''my_tensor.to(device)'' 를 호출 시 에는 ''my_tensor'' 를 다시쓰는 대신 ''my_tensor'' 의 또다른 복사본이 생긴다는 사실을 기억하십시오. 당신은 그것을 새로운 tensor 에 소속시키고 GPU에 그 tensor를 써야합니다. 여러 GPU를 통해 앞과 뒤의 전파를 실행하는 것은 당연한 일 입니다. @@ -59,7 +60,7 @@ ###################################################################### # 더미(Dummy) 데이터셋 -# ------------- +# ----------------------- # # 더미(ramdom) 데이터셋을 만들어 봅시다. Getitem 만 구현하면 됩니다. # @@ -86,7 +87,7 @@ def __len__(self): # # 데모를 위해 모델은 입력을 받고 선형 연산을 수행하며 출력을 제공합니다. 그러나 ``DataParallel`` 의 어떤 모델 (CNN, RNN, Capsule Net 등) 에서든 사용할 수 있습니다. # -#우리는 input과 output의 크기를 모니터링하기 위해 모델안에 print 문을 넣었습니다. +# 우리는 input과 output의 크기를 모니터링하기 위해 모델안에 print 문을 넣었습니다. # 무엇이 배치 순위 (batch rank) 0 에 프린트 되는지 주의 깊게 봐주시길 바랍니다. # @@ -110,7 +111,7 @@ def forward(self, input): # ----------------------------- # # 이것은 이 튜토리얼의 핵심 부분입니다. 먼저, model instance 를 만들고 가지고 있는 GPU가 여러개인지 확인해야합니다. -# 만약 다수의 GPU를 보유중이라면, ``nn.DataParallel`` 을 사용하여 모델을 래핑 (wrapping) 할 수 있습니다. +# 만약 다수의 GPU를 보유중이라면, ``nn.DataParallel`` 을 사용하여 모델을 래핑 (wrapping) 할 수 있습니다. # 그런 다음 ``model.to(device)`` 를 통하여 모델을 GPU에 넣을 수 있습니다. # diff --git a/docs/_downloads/edd578af5440b92a4d66dcd1d4aa8601/zeroing_out_gradients.ipynb b/docs/_downloads/edd578af5440b92a4d66dcd1d4aa8601/zeroing_out_gradients.ipynb index 14b7509e2..e56dfa1c8 100644 --- a/docs/_downloads/edd578af5440b92a4d66dcd1d4aa8601/zeroing_out_gradients.ipynb +++ b/docs/_downloads/edd578af5440b92a4d66dcd1d4aa8601/zeroing_out_gradients.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nPyTorch\uc5d0\uc11c \ubcc0\ud654\ub3c4\ub97c 0\uc73c\ub85c \ub9cc\ub4e4\uae30\n================================\n\uc2e0\uacbd\ub9dd\uc744 \uad6c\ucd95\ud560 \ub54c\ub294 \ubcc0\ud654\ub3c4\ub97c 0\uc73c\ub85c \ub9cc\ub4e4\uc5b4 \uc8fc\ub294 \uac83\uc774 \uc88b\uc2b5\ub2c8\ub2e4. \uae30\ubcf8\uc801\uc73c\ub85c \n``.backward()`` \ub97c \ud638\ucd9c\ud560 \ub54c\ub9c8\ub2e4 \ubcc0\ud654\ub3c4\uac00 \ubc84\ud37c\uc5d0 \uc313\uc774\uae30 \ub54c\ubb38\uc785\ub2c8\ub2e4. (\ub36e\uc5b4\uc4f0\uc9c0 \uc54a\ub294\ub2e4\ub294 \uc758\ubbf8\uc785\ub2c8\ub2e4.)\n\n\uac1c\uc694\n------------\n\uc2e0\uacbd\ub9dd\uc744 \ud559\uc2b5\uc2dc\ud0ac \ub54c, \uacbd\uc0ac \ud558\uac15\ubc95\uc744 \uac70\uccd0 \ubaa8\ub378 \uc815\ud655\ub3c4\ub97c \ub192\uc77c \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uacbd\uc0ac \ud558\uac15\ubc95\uc740 \uac04\ub2e8\ud788 \n\uc124\uba85\ud574 \ubaa8\ub378\uc758 \uac00\uc911\uce58\uc640 \ud3b8\ud5a5\uc744 \uc57d\uac04\uc529 \uc218\uc815\ud558\uba74\uc11c \uc190\uc2e4(\ub610\ub294 \uc624\ub958)\ub97c \ucd5c\uc18c\ud654\ud558\ub294 \uacfc\uc815\uc785\ub2c8\ub2e4. \n\n``torch.Tensor`` \ub294 PyTorch \uc758 \ud575\uc2ec\uc774 \ub418\ub294 \ud074\ub798\uc2a4 \uc785\ub2c8\ub2e4. \ud150\uc11c\ub97c \uc0dd\uc131\ud560 \ub54c \n``.requires_grad`` \uc18d\uc131\uc744 ``True`` \ub85c \uc124\uc815\ud558\uba74, \ud150\uc11c\uc5d0 \uac00\ud574\uc9c4 \ubaa8\ub4e0 \uc5f0\uc0b0\uc744 \ucd94\uc801\ud569\ub2c8\ub2e4. \n\ub4a4\ub530\ub974\ub294 \ubaa8\ub4e0 \uc5ed\uc804\ud30c \ub2e8\uacc4\uc5d0\uc11c\ub3c4 \ub9c8\ucc2c\uac00\uc9c0\ub85c, \uc774 \ud150\uc11c\uc758 \ubcc0\ud654\ub3c4\ub294 ``.grad`` \uc18d\uc131\uc5d0 \ub204\uc801\ub420 \uac83\uc785\ub2c8\ub2e4.\n\ubaa8\ub4e0 \ubcc0\ud654\ub3c4\uc758 \ucd95\uc801 \ub610\ub294 \ud569\uc740 \uc190\uc2e4 \ud150\uc11c\uc5d0\uc11c ``.backward()`` \ub97c \ud638\ucd9c\ud560 \ub54c \uacc4\uc0b0\ub429\ub2c8\ub2e4.\n\n\ud150\uc11c\uc758 \ubcc0\ud654\ub3c4\ub97c 0\uc73c\ub85c \ub9cc\ub4e4\uc5b4 \uc8fc\uc5b4\uc57c \ud558\ub294 \uacbd\uc6b0\ub3c4 \uc788\uc2b5\ub2c8\ub2e4. \uc608\ub97c \ub4e4\uc5b4 \ud559\uc2b5 \uacfc\uc815 \ubc18\ubcf5\ubb38\uc744 \n\uc2dc\uc791\ud560 \ub54c, \ub204\uc801\ub418\ub294 \ubcc0\ud654\ub3c4\ub97c \uc815\ud655\ud558\uac8c \ucd94\uc801\ud558\uae30 \uc704\ud574\uc11c\ub294 \ubcc0\ud654\ub3c4\ub97c \uc6b0\uc120 0\uc73c\ub85c \ub9cc\ub4e4\uc5b4 \uc8fc\uc5b4\uc57c \ud569\ub2c8\ub2e4. \n\uc774 \ub808\uc2dc\ud53c\uc5d0\uc11c\ub294 PyTorch \ub77c\uc774\ube0c\ub7ec\ub9ac\ub97c \uc0ac\uc6a9\ud558\uc5ec \ubcc0\ud654\ub3c4\ub97c 0\uc73c\ub85c \ub9cc\ub4dc\ub294 \ubc29\ubc95\uc744 \ubc30\uc6cc\ubd05\ub2c8\ub2e4.\nPyTorch\uc5d0 \ub0b4\uc7a5\ub41c ``CIFAR10`` \ub370\uc774\ud130\uc14b\uc5d0 \ub300\ud558\uc5ec \uc2e0\uacbd\ub9dd\uc744 \ud6c8\ub828\uc2dc\ud0a4\ub294 \uacfc\uc815\uc744 \ud1b5\ud574 \uc54c\uc544\ubd05\uc2dc\ub2e4.\n\n\uc124\uc815\n-----\n\uc774 \ub808\uc2dc\ud53c\uc5d0\ub294 \ub370\uc774\ud130\ub97c \ud559\uc2b5\uc2dc\ud0a4\ub294 \ub0b4\uc6a9\uc774 \ud3ec\ud568\ub418\uc5b4 \uc788\uae30 \ub54c\ubb38\uc5d0, \uc2e4\ud589 \uac00\ub2a5\ud55c \ub178\ud2b8\ubd81 \ud30c\uc77c\uc774 \uc788\ub2e4\uba74\n\ub7f0\ud0c0\uc784\uc744 GPU \ub610\ub294 TPU\ub85c \uc804\ud658\ud558\ub294 \uac83\uc774 \uc88b\uc2b5\ub2c8\ub2e4. \uc2dc\uc791\ud558\uae30\uc5d0 \uc55e\uc11c, ``torch`` \uc640 \n``torchvision`` \ud328\ud0a4\uc9c0\uac00 \uc5c6\ub2e4\uba74 \uc124\uce58\ud569\ub2c8\ub2e4.\n\n::\n\n pip install torchvision\n\n\n\n" + "\nPyTorch\uc5d0\uc11c \ubcc0\ud654\ub3c4\ub97c 0\uc73c\ub85c \ub9cc\ub4e4\uae30\n================================\n\uc2e0\uacbd\ub9dd\uc744 \uad6c\ucd95\ud560 \ub54c\ub294 \ubcc0\ud654\ub3c4\ub97c 0\uc73c\ub85c \ub9cc\ub4e4\uc5b4 \uc8fc\ub294 \uac83\uc774 \uc88b\uc2b5\ub2c8\ub2e4. \uae30\ubcf8\uc801\uc73c\ub85c \n``.backward()`` \ub97c \ud638\ucd9c\ud560 \ub54c\ub9c8\ub2e4 \ubcc0\ud654\ub3c4\uac00 \ubc84\ud37c\uc5d0 \uc313\uc774\uae30 \ub54c\ubb38\uc785\ub2c8\ub2e4. (\ub36e\uc5b4\uc4f0\uc9c0 \uc54a\ub294\ub2e4\ub294 \uc758\ubbf8\uc785\ub2c8\ub2e4.)\n\n\uac1c\uc694\n------------\n\uc2e0\uacbd\ub9dd\uc744 \ud559\uc2b5\uc2dc\ud0ac \ub54c, \uacbd\uc0ac \ud558\uac15\ubc95\uc744 \uac70\uccd0 \ubaa8\ub378 \uc815\ud655\ub3c4\ub97c \ub192\uc77c \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uacbd\uc0ac \ud558\uac15\ubc95\uc740 \uac04\ub2e8\ud788 \n\uc124\uba85\ud574 \ubaa8\ub378\uc758 \uac00\uc911\uce58\uc640 \ud3b8\ud5a5\uc744 \uc57d\uac04\uc529 \uc218\uc815\ud558\uba74\uc11c \uc190\uc2e4(\ub610\ub294 \uc624\ub958)\ub97c \ucd5c\uc18c\ud654\ud558\ub294 \uacfc\uc815\uc785\ub2c8\ub2e4. \n\n``torch.Tensor`` \ub294 PyTorch \uc758 \ud575\uc2ec\uc774 \ub418\ub294 \ud074\ub798\uc2a4 \uc785\ub2c8\ub2e4. \ud150\uc11c\ub97c \uc0dd\uc131\ud560 \ub54c \n``.requires_grad`` \uc18d\uc131\uc744 ``True`` \ub85c \uc124\uc815\ud558\uba74, \ud150\uc11c\uc5d0 \uac00\ud574\uc9c4 \ubaa8\ub4e0 \uc5f0\uc0b0\uc744 \ucd94\uc801\ud569\ub2c8\ub2e4. \n\ub4a4\ub530\ub974\ub294 \ubaa8\ub4e0 \uc5ed\uc804\ud30c \ub2e8\uacc4\uc5d0\uc11c\ub3c4 \ub9c8\ucc2c\uac00\uc9c0\ub85c, \uc774 \ud150\uc11c\uc758 \ubcc0\ud654\ub3c4\ub294 ``.grad`` \uc18d\uc131\uc5d0 \ub204\uc801\ub420 \uac83\uc785\ub2c8\ub2e4.\n\ubaa8\ub4e0 \ubcc0\ud654\ub3c4\uc758 \ucd95\uc801 \ub610\ub294 \ud569\uc740 \uc190\uc2e4 \ud150\uc11c\uc5d0\uc11c ``.backward()`` \ub97c \ud638\ucd9c\ud560 \ub54c \uacc4\uc0b0\ub429\ub2c8\ub2e4.\n\n\ud150\uc11c\uc758 \ubcc0\ud654\ub3c4\ub97c 0\uc73c\ub85c \ub9cc\ub4e4\uc5b4 \uc8fc\uc5b4\uc57c \ud558\ub294 \uacbd\uc6b0\ub3c4 \uc788\uc2b5\ub2c8\ub2e4. \uc608\ub97c \ub4e4\uc5b4 \ud559\uc2b5 \uacfc\uc815 \ubc18\ubcf5\ubb38\uc744 \n\uc2dc\uc791\ud560 \ub54c, \ub204\uc801\ub418\ub294 \ubcc0\ud654\ub3c4\ub97c \uc815\ud655\ud558\uac8c \ucd94\uc801\ud558\uae30 \uc704\ud574\uc11c\ub294 \ubcc0\ud654\ub3c4\ub97c \uc6b0\uc120 0\uc73c\ub85c \ub9cc\ub4e4\uc5b4 \uc8fc\uc5b4\uc57c \ud569\ub2c8\ub2e4. \n\uc774 \ub808\uc2dc\ud53c\uc5d0\uc11c\ub294 PyTorch \ub77c\uc774\ube0c\ub7ec\ub9ac\ub97c \uc0ac\uc6a9\ud558\uc5ec \ubcc0\ud654\ub3c4\ub97c 0\uc73c\ub85c \ub9cc\ub4dc\ub294 \ubc29\ubc95\uc744 \ubc30\uc6cc\ubd05\ub2c8\ub2e4.\nPyTorch\uc5d0 \ub0b4\uc7a5\ub41c ``CIFAR10`` \ub370\uc774\ud130\uc14b\uc5d0 \ub300\ud558\uc5ec \uc2e0\uacbd\ub9dd\uc744 \ud6c8\ub828\uc2dc\ud0a4\ub294 \uacfc\uc815\uc744 \ud1b5\ud574 \uc54c\uc544\ubd05\uc2dc\ub2e4.\n\n\uc124\uc815\n-----\n\uc774 \ub808\uc2dc\ud53c\uc5d0\ub294 \ub370\uc774\ud130\ub97c \ud559\uc2b5\uc2dc\ud0a4\ub294 \ub0b4\uc6a9\uc774 \ud3ec\ud568\ub418\uc5b4 \uc788\uae30 \ub54c\ubb38\uc5d0, \uc2e4\ud589 \uac00\ub2a5\ud55c \ub178\ud2b8\ubd81 \ud30c\uc77c\uc774 \uc788\ub2e4\uba74\n\ub7f0\ud0c0\uc784\uc744 GPU \ub610\ub294 TPU\ub85c \uc804\ud658\ud558\ub294 \uac83\uc774 \uc88b\uc2b5\ub2c8\ub2e4. \uc2dc\uc791\ud558\uae30\uc5d0 \uc55e\uc11c, ``torch`` \uc640 \n``torchvision`` \ud328\ud0a4\uc9c0\uac00 \uc5c6\ub2e4\uba74 \uc124\uce58\ud569\ub2c8\ub2e4.\n\n::\n\n pip install torchvision\n" ] }, { @@ -132,7 +132,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/ee55f2537c08fa13e041e479675a6c2c/numpy_extensions_tutorial.py b/docs/_downloads/ee55f2537c08fa13e041e479675a6c2c/numpy_extensions_tutorial.py index afc9a118c..9475ff084 100644 --- a/docs/_downloads/ee55f2537c08fa13e041e479675a6c2c/numpy_extensions_tutorial.py +++ b/docs/_downloads/ee55f2537c08fa13e041e479675a6c2c/numpy_extensions_tutorial.py @@ -1,35 +1,34 @@ # -*- coding: utf-8 -*- """ -Creating Extensions Using numpy and scipy -========================================= +numpy 와 scipy 를 이용한 확장(Extensions) 만들기 +===================================================== **Author**: `Adam Paszke `_ **Updated by**: `Adam Dziedzic `_ -In this tutorial, we shall go through two tasks: +**번역**: `Ajin Jeong `_ -1. Create a neural network layer with no parameters. +이번 튜토리얼에서는 두 가지 작업을 수행할 것입니다: - - This calls into **numpy** as part of its implementation +1. 매개 변수가 없는 신경망 계층(layer) 만들기 + - 이는 구현의 일부로 **numpy** 를 호출합니다. -2. Create a neural network layer that has learnable weights - - - This calls into **SciPy** as part of its implementation +2. 학습 가능한 가중치가 있는 신경망 계층(layer) 만들기 + - 이는 구현의 일부로 **Scipy** 를 호출합니다. """ import torch from torch.autograd import Function ############################################################### -# Parameter-less example -# ---------------------- +# 매개 변수가 없는(Parameter-less) 예시 +# ---------------------------------------- # -# This layer doesn’t particularly do anything useful or mathematically -# correct. +# 이 계층(layer)은 특별히 유용하거나 수학적으로 올바른 작업을 수행하지 않습니다. # -# It is aptly named BadFFTFunction +# 이름은 대충 BadFFTFunction으로 지었습니다. # -# **Layer Implementation** +# **계층(layer) 구현** from numpy.fft import rfft2, irfft2 @@ -47,15 +46,14 @@ def backward(ctx, grad_output): result = irfft2(numpy_go) return grad_output.new(result) -# since this layer does not have any parameters, we can -# simply declare this as a function, rather than as an nn.Module class +# 이 계층에는 매개 변수가 없으므로 nn.Module 클래스가 아닌 함수로 간단히 선언할 수 있습니다. def incorrect_fft(input): return BadFFTFunction.apply(input) ############################################################### -# **Example usage of the created layer:** +# **생성된 계층(layer)의 사용 예시:** input = torch.randn(8, 8, requires_grad=True) result = incorrect_fft(input) @@ -64,18 +62,16 @@ def incorrect_fft(input): print(input) ############################################################### -# Parametrized example -# -------------------- +# 매개 변수가 있는(Parameterized) 예시 +# ---------------------------------------- # -# In deep learning literature, this layer is confusingly referred -# to as convolution while the actual operation is cross-correlation -# (the only difference is that filter is flipped for convolution, -# which is not the case for cross-correlation). +# 딥러닝 문헌에서 이 계층(layer)의 실제 연산은 상호 상관(cross-correlation)이지만 +# 합성곱(convolution)이라고 헷갈리게 부르고 있습니다. +# (합성곱은 필터를 뒤집어서 연산을 하는 반면, 상호 상관은 그렇지 않은 차이가 있습니다) # -# Implementation of a layer with learnable weights, where cross-correlation -# has a filter (kernel) that represents weights. +# 학습 가능한 가중치를 가는 필터(커널)를 갖는 상호 상관 계층을 구현해보겠습니다. # -# The backward pass computes the gradient wrt the input and the gradient wrt the filter. +# 역전파 단계(backward pass)에서는 입력에 대한 기울기(gradient)와 필터에 대한 기울기를 계산합니다. from numpy import flip import numpy as np @@ -87,7 +83,7 @@ def incorrect_fft(input): class ScipyConv2dFunction(Function): @staticmethod def forward(ctx, input, filter, bias): - # detach so we can cast to NumPy + # 분리(detach)하여 NumPy로 변환(cast)할 수 있습니다. input, filter, bias = input.detach(), filter.detach(), bias.detach() result = correlate2d(input.numpy(), filter.numpy(), mode='valid') result += bias.numpy() @@ -101,7 +97,7 @@ def backward(ctx, grad_output): grad_output = grad_output.numpy() grad_bias = np.sum(grad_output, keepdims=True) grad_input = convolve2d(grad_output, filter.numpy(), mode='full') - # the previous line can be expressed equivalently as: + # 윗줄은 다음과 같이 표현할 수도 있습니다: # grad_input = correlate2d(grad_output, flip(flip(filter.numpy(), axis=0), axis=1), mode='full') grad_filter = correlate2d(input.numpy(), grad_output, mode='valid') return torch.from_numpy(grad_input), torch.from_numpy(grad_filter).to(torch.float), torch.from_numpy(grad_bias).to(torch.float) @@ -118,7 +114,7 @@ def forward(self, input): ############################################################### -# **Example usage:** +# **사용 예시:** module = ScipyConv2d(3, 3) print("Filter and bias: ", list(module.parameters())) @@ -129,7 +125,7 @@ def forward(self, input): print("Gradient for the input map: ", input.grad) ############################################################### -# **Check the gradients:** +# **기울기(gradient) 확인:** from torch.autograd.gradcheck import gradcheck diff --git a/docs/_downloads/ef3541eb2ef78e22efa65b3d6f4ba737/pruning_tutorial.py b/docs/_downloads/ef3541eb2ef78e22efa65b3d6f4ba737/pruning_tutorial.py index 30c392c8c..192e06c26 100644 --- a/docs/_downloads/ef3541eb2ef78e22efa65b3d6f4ba737/pruning_tutorial.py +++ b/docs/_downloads/ef3541eb2ef78e22efa65b3d6f4ba737/pruning_tutorial.py @@ -7,15 +7,15 @@ 최첨단 딥러닝 모델들은 굉장히 많은 수의 파라미터값들로 구성되기 때문에, 쉽게 배포되기 어렵습니다. 이와 반대로, 생물학적 신경망들은 효율적으로 희소하게 연결된 것으로 알려져 있습니다. -모델의 정확도가 손상되지 않는 범위에서 메모리, 배터리, 하드웨어 소비량을 줄이고, -기기에 경량화된 모델을 배치하며, 개인이 이용하고 있는 기기에서 프라이버시가 보장되기 위해서는 +모델의 정확도가 손상되지 않는 범위에서 메모리, 배터리, 하드웨어 소비량을 줄이고, +기기에 경량화된 모델을 배치하며, 개인이 이용하고 있는 기기에서 프라이버시가 보장되기 위해서는 모델에 포함된 파라미터 수를 줄여 압축하는 최적의 기법을 파악하는 것이 중요합니다. -연구 측면에서는, 가지치기 기법은 굉장히 많은 수의 파라미터값들로 구성된 모델과 +연구 측면에서는, 가지치기 기법은 굉장히 많은 수의 파라미터값들로 구성된 모델과 굉장히 적은 수의 파라미터값들로 구성된 모델 간 학습 역학 차이를 조사하는데 주로 이용되기도 하며, -하위 신경망 모델과 파라미터값들의 초기화가 운이 좋게 잘 된 케이스를 바탕으로 +하위 신경망 모델과 파라미터값들의 초기화가 운이 좋게 잘 된 케이스를 바탕으로 ("`lottery tickets `_") 신경망 구조를 찾는 기술들에 대해 반대 의견을 제시하기도 합니다. -이번 튜토리얼에서는, ``torch.nn.utils.prune`` 을 이용하여 여러분이 설계한 딥러닝 모델에 대해 가지치기 기법을 적용해보는 것을 배워보고, +이번 튜토리얼에서는, ``torch.nn.utils.prune`` 을 이용하여 여러분이 설계한 딥러닝 모델에 대해 가지치기 기법을 적용해보는 것을 배워보고, 심화적으로 여러분의 맞춤형 가지치기 기법을 구현하는 방법에 대해 배워보도록 하겠습니다. 요구사항 @@ -31,7 +31,7 @@ ###################################################################### # 딥러닝 모델 생성 # ----------------------- -# 이번 튜토리얼에서는, 얀 르쿤 교수님의 연구진들이 1998년도에 발표한 ``LeNet +# 이번 튜토리얼에서는, 얀 르쿤 교수님의 연구진들이 1998년도에 발표한 ``LeNet # `` 의 모델 구조를 이용합니다. @@ -63,9 +63,10 @@ def forward(self, x): ###################################################################### # 모듈 점검 # ----------------- -# -# 가지치기 기법이 적용되지 않은 LeNet 모델의 ``conv1`` 층을 점검해봅시다. -# 여기에는 2개의 파라미터값들인 ``가중치``값과 ``편향``값을이 포함될 것이며, 버퍼는 존재하지 않을 것입니다. +# +# 가지치기 기법이 적용되지 않은 LeNet 모델의 ``conv1`` 층을 점검해봅시다. +# 여기에는 2개의 파라미터값들인 ``가중치`` 값과 ``편향`` 값이 포함될 것이며, 버퍼는 존재하지 않을 것입니다. +# module = model.conv1 print(list(module.named_parameters())) @@ -76,51 +77,51 @@ def forward(self, x): ###################################################################### # 모듈 가지치기 기법 적용 예제 # ----------------------------------- -# +# # 모듈에 대해 가지치기 기법을 적용하기 위해 (이번 예제에서는, LeNet 모델의 ``conv1`` 층) -# 첫 번째로는, ``torch.nn.utils.prune`` (또는 ``BasePruningMethod`` 의 서브 클래스로 직접 `구현 -# `_ ) +# 첫 번째로는, ``torch.nn.utils.prune`` (또는 ``BasePruningMethod`` 의 서브 클래스로 직접 +# `구현 `_ ) # 내 존재하는 가지치기 기법을 선택합니다. # 그 후, 해당 모듈 내에서 가지치기 기법을 적용하고자 하는 모듈과 파라미터를 지정합니다. # 마지막으로, 가지치기 기법에 적당한 키워드 인자값을 이용하여 가지치기 매개변수를 지정합니다. # 이번 예제에서는, ``conv1`` 층의 가중치의 30%값들을 랜덤으로 가지치기 기법을 적용해보겠습니다. # 모듈은 함수에 대한 첫 번째 인자값으로 전달되며, ``name`` 은 문자열 식별자를 이용하여 해당 모듈 내 매개변수를 구분합니다. -# 그리고, ``amount`` 는 가지치기 기법을 적용하기 위한 대상 가중치값들의 백분율 (0과 1사이의 실수값), +# 그리고, ``amount`` 는 가지치기 기법을 적용하기 위한 대상 가중치값들의 백분율 (0과 1사이의 실수값), # 혹은 가중치값의 연결의 개수 (음수가 아닌 정수) 를 지정합니다. -prune.random_unstructured(module, name="weight", amount=0.3) +prune.random_unstructured(module, name="weight", amount=0.3) ###################################################################### -# 가지치기 기법은 가중치값들을 파라미터값들로부터 제거하고 ``weight_orig`` (즉, 초기 가중치 이름에 "_orig"을 붙인) 이라는 +# 가지치기 기법은 가중치값들을 파라미터값들로부터 제거하고 ``weight_orig`` (즉, 초기 가중치 이름에 "_orig"을 붙인) 이라는 # 새로운 파라미터값으로 대체하는 것으로 실행됩니다. -# ``weight_orig`` 은 텐서값에 가지치기 기법이 적용되지 않은 상태를 저장합니다. +# ``weight_orig`` 은 텐서값에 가지치기 기법이 적용되지 않은 상태를 저장합니다. # ``bias`` 은 가지치기 기법이 적용되지 않았기 때문에 그대로 남아 있습니다. print(list(module.named_parameters())) ###################################################################### -# 위에서 선택한 가지치기 기법에 의해 생성되는 가지치기 마스크는 초기 파라미터 ``name`` 에 ``weight_mask`` +# 위에서 선택한 가지치기 기법에 의해 생성되는 가지치기 마스크는 초기 파라미터 ``name`` 에 ``weight_mask`` # (즉, 초기 가중치 이름에 "_mask"를 붙인) 이름의 모듈 버퍼로 저장됩니다. print(list(module.named_buffers())) ###################################################################### -# 수정이 되지 않은 상태에서 순전파를 진행하기 위해서는 ``가중치``값 속성이 존재해야 합니다. -# ``torch.nn.utils.prune`` 내 구현된 가지치기 기법은 가지치기 기법이 적용된 가중치값들을 이용하여 +# 수정이 되지 않은 상태에서 순전파를 진행하기 위해서는 ``가중치`` 값 속성이 존재해야 합니다. +# ``torch.nn.utils.prune`` 내 구현된 가지치기 기법은 가지치기 기법이 적용된 가중치값들을 이용하여 # (기존의 가중치값에 가지치기 기법이 적용된) 순전파를 진행하고, ``weight`` 속성값에 가지치기 기법이 적용된 가중치값들을 저장합니다. # 이제 가중치값들은 ``module`` 의 매개변수가 아니라 하나의 속성값으로 취급되는 점을 주의하세요. print(module.weight) ###################################################################### # 최종적으로, 가지치기 기법은 파이토치의 ``forward_pre_hooks`` 를 이용하여 각 순전파가 진행되기 전에 가지치기 기법이 적용됩니다. -# 구체적으로, 지금까지 진행한 것 처럼, 모듈이 가지치기 기법이 적용되었을 때, +# 구체적으로, 지금까지 진행한 것 처럼, 모듈이 가지치기 기법이 적용되었을 때, # 가지치기 기법이 적용된 각 파라미터값들이 ``forward_pre_hook`` 를 얻게됩니다. -# 이러한 경우, ``weight`` 이름인 기존 파라미터값에 대해서만 가지치기 기법을 적용하였기 때문에, +# 이러한 경우, ``weight`` 이름인 기존 파라미터값에 대해서만 가지치기 기법을 적용하였기 때문에, # 훅은 오직 1개만 존재할 것입니다. print(module._forward_pre_hooks) ###################################################################### -# 완결성을 위해, 편향값에 대해서도 가지치기 기법을 적용할 수 있으며, +# 완결성을 위해, 편향값에 대해서도 가지치기 기법을 적용할 수 있으며, # 모듈의 파라미터, 버퍼, 훅, 속성값들이 어떻게 변경되는지 확인할 수 있습니다. -# 또 다른 가지치기 기법을 적용해보기 위해, ``l1_unstructured`` 가지치기 함수에서 구현된 내용과 같이, +# 또 다른 가지치기 기법을 적용해보기 위해, ``l1_unstructured`` 가지치기 함수에서 구현된 내용과 같이, # L1 Norm 값이 가장 작은 편향값 3개를 가지치기를 시도해봅시다. prune.l1_unstructured(module, name="bias", amount=3) @@ -142,7 +143,7 @@ def forward(self, x): ###################################################################### # 가지치기 기법 반복 적용 # ------------------------------------ -# +# # 모듈 내 같은 파라미터값에 대해 가지치기 기법이 여러번 적용될 수 있으며, 다양한 가지치기 기법의 조합이 적용된 것과 동일하게 적용될 수 있습니다. # 새로운 마스크와 이전의 마스크의 결합은 ``PruningContainer`` 의 ``compute_mask`` 메소드를 통해 처리할 수 있습니다. # @@ -166,7 +167,7 @@ def forward(self, x): ###################################################################### # 가지치기 기법이 적용된 모델의 직렬화 # --------------------------------------------- -# 마스크 버퍼들과 가지치기 기법이 적용된 텐서 계산에 사용된 기존의 파라미터를 포함하여 관련된 모든 텐서값들은 +# 마스크 버퍼들과 가지치기 기법이 적용된 텐서 계산에 사용된 기존의 파라미터를 포함하여 관련된 모든 텐서값들은 # 필요한 경우 모델의 ``state_dict`` 에 저장되기 떄문에, 쉽게 직렬화하여 저장할 수 있습니다. print(model.state_dict().keys()) @@ -175,14 +176,14 @@ def forward(self, x): # 가지치기 기법의 재-파라미터화 제거 # ----------------------------------------- # -# 가지치기 기법이 적용된 것을 영구적으로 만들기 위해서, 재-파라미터화 관점의 +# 가지치기 기법이 적용된 것을 영구적으로 만들기 위해서, 재-파라미터화 관점의 # ``weight_orig`` 와 ``weight_mask`` 값을 제거하고, ``forward_pre_hook`` 값을 제거합니다. # 제거하기 위해 ``torch.nn.utils.prune`` 내 ``remove`` 함수를 이용할 수 있습니다. # 가지치기 기법이 적용되지 않은 것처럼 실행되는 것이 아닌 점을 주의하세요. # 이는 단지 가지치기 기법이 적용된 상태에서 가중치 파라미터값을 모델 파라미터값으로 재할당하는 것을 통해 영구적으로 만드는 것일 뿐입니다. ###################################################################### -# 재-파라미터화를 제거하기 전 상태 +# 재-파라미터화를 제거하기 전 상태 print(list(module.named_parameters())) ###################################################################### print(list(module.named_buffers())) @@ -190,7 +191,7 @@ def forward(self, x): print(module.weight) ###################################################################### -# 재-파라미터를 제거한 후 상태 +# 재-파라미터를 제거한 후 상태 prune.remove(module, 'weight') print(list(module.named_parameters())) ###################################################################### @@ -198,9 +199,9 @@ def forward(self, x): ###################################################################### # 모델 내 여러 파라미터값들에 대하여 가지치기 기법 적용 -# -------------------------------------- +# ---------------------------------------------------------- # -# 가지치기 기법을 적용하고 싶은 파라미터값들을 지정함으로써, 이번 예제에서 볼 수 있는 것 처럼, +# 가지치기 기법을 적용하고 싶은 파라미터값들을 지정함으로써, 이번 예제에서 볼 수 있는 것 처럼, # 신경망 모델 내 여러 텐서값들에 대해서 쉽게 가지치기 기법을 적용할 수 있습니다. new_model = LeNet() @@ -296,22 +297,22 @@ def forward(self, x): ###################################################################### # ``torch.nn.utils.prune`` 에서 확장된 맞춤형 가지치기 기법 # ------------------------------------------------------------------ -# 맞춤형 가지치기 기법은, 다른 가지치기 기법을 적용하는 것과 같은 방식으로, +# 맞춤형 가지치기 기법은, 다른 가지치기 기법을 적용하는 것과 같은 방식으로, # ``BasePruningMethod`` 의 기본 클래스인 ``nn.utils.prune`` 모듈을 활용하여 구현할 수 있습니다. # 기본 클래스는 ``__call__``, ``apply_mask``, ``apply``, ``prune``, ``remove`` 메소드들을 내포하고 있습니다. # 특별한 케이스가 아닌 경우, 기본적으로 구성된 메소드들을 재구성할 필요가 없습니다. -# 그러나, ``__init__`` (구성요소), ``compute_mask`` +# 그러나, ``__init__`` (구성요소), ``compute_mask`` # (가지치기 기법의 논리에 따라 주어진 텐서값에 마스크를 적용하는 방법) 을 고려하여 구성해야 합니다. -# 게다가, 가지치기 기법을 어떠한 방식으로 적용하는지 명확하게 구성해야 합니다. +# 게다가, 가지치기 기법을 어떠한 방식으로 적용하는지 명확하게 구성해야 합니다. # (지원되는 옵션은 ``global``, ``structured``, ``unstructured`` 입니다.) # 이러한 방식은, 가지치기 기법을 반복적으로 적용해야 하는 경우 마스크를 결합하는 방법을 결정하기 위해 필요합니다. -# 즉, 이미 가지치기 기법이 적용된 모델에 대해서 가지치기 기법을 적용할 때, +# 즉, 이미 가지치기 기법이 적용된 모델에 대해서 가지치기 기법을 적용할 때, # 기존의 가지치기 기법이 적용되지 않은 파라미터 값에 대해 가지치기 기법이 영향을 미칠 것으로 예상됩니다. -# ``PRUNING_TYPE``을 지정한다면, 가지치기 기법을 적용하기 위해 파라미터 값을 올바르게 제거하는 +# ``PRUNING_TYPE`` 을 지정한다면, 가지치기 기법을 적용하기 위해 파라미터 값을 올바르게 제거하는 # ``PruningContainer`` (마스크 가지치기 기법을 반복적으로 적용하는 것을 처리하는)를 가능하게 합니다. # 예를 들어, 다른 모든 항목이 존재하는 텐서를 가지치기 기법을 구현하고 싶을 때, # (또는, 텐서가 이전에 가지치기 기법에 의해 제거되었거나 남아있는 텐서에 대해) -# 한 층의 개별 연결에 작용하며 전체 유닛/채널 (``'structured'``), 또는 다른 파라미터 간 +# 한 층의 개별 연결에 작용하며 전체 유닛/채널 (``'structured'``), 또는 다른 파라미터 간 # (``'global'``) 연결에는 작용하지 않기 때문에 ``PRUNING_TYPE='unstructured'`` 방식으로 진행됩니다. class FooBarPruningMethod(prune.BasePruningMethod): @@ -322,7 +323,7 @@ class FooBarPruningMethod(prune.BasePruningMethod): def compute_mask(self, t, default_mask): mask = default_mask.clone() - mask.view(-1)[::2] = 0 + mask.view(-1)[::2] = 0 return mask ###################################################################### @@ -332,7 +333,7 @@ def foobar_unstructured(module, name): 텐서 내 다른 모든 항목들을 제거하여 `module` 에서 `name` 이라는 파라미터에 대해 가자치기 기법을 적용 다음 내용에 따라 모듈을 수정 (또는 수정된 모듈을 반환): 1) 가지치기 기법에 의해 매개변수 `name` 에 적용된 이진 마스크에 해당하는 명명된 버퍼 `name+'_mask'` 를 추가합니다. - `name` 파라미터는 가지치기 기법이 적용된 것으로 대체되며, 가지치기 기법이 적용되지 않은 + `name` 파라미터는 가지치기 기법이 적용된 것으로 대체되며, 가지치기 기법이 적용되지 않은 기존의 파라미터는 `name+'_orig'` 라는 이름의 새로운 매개변수에 저장됩니다. 인자값: @@ -340,8 +341,8 @@ def foobar_unstructured(module, name): name (string): 모듈 내 가지치기 기법이 적용될 파라미터의 이름 반환값: - module (nn.Module): 입력 모듈에 대해서 가지치기 기법이 적용된 모듈 - + module (nn.Module): 입력 모듈에 대해서 가지치기 기법이 적용된 모듈 + 예시: >>> m = nn.Linear(3, 4) >>> foobar_unstructured(m, name='bias') diff --git a/docs/_downloads/f11c58c36c9b8a5daf09d3f9a792ef84/memory_format_tutorial.ipynb b/docs/_downloads/f11c58c36c9b8a5daf09d3f9a792ef84/memory_format_tutorial.ipynb index c13b1af35..bd2000abc 100644 --- a/docs/_downloads/f11c58c36c9b8a5daf09d3f9a792ef84/memory_format_tutorial.ipynb +++ b/docs/_downloads/f11c58c36c9b8a5daf09d3f9a792ef84/memory_format_tutorial.ipynb @@ -15,14 +15,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n(beta) Channels Last Memory Format in PyTorch\n*******************************************************\n**Author**: `Vitaly Fedyunin `_\n\nWhat is Channels Last\n---------------------\n\nChannels Last memory format is an alternative way of ordering NCHW tensors in memory preserving dimensions ordering. Channels Last tensors ordered in such a way that channels become the densest dimension (aka storing images pixel-per-pixel).\n\nFor example, classic (contiguous) storage of NCHW tensor (in our case it is two 2x2 images with 3 color channels) look like this:\n\n.. figure:: /_static/img/classic_memory_format.png\n :alt: classic_memory_format\n\nChannels Last memory format orders data differently:\n\n.. figure:: /_static/img/channels_last_memory_format.png\n :alt: channels_last_memory_format\n\nPytorch supports memory formats (and provides back compatibility with existing models including eager, JIT, and TorchScript) by utilizing existing strides structure.\nFor example, 10x3x16x16 batch in Channels Last format will have strides equal to (768, 1, 48, 3).\n\n" + "\n(\uc2e4\ud5d8\uc6a9) PyTorch\ub97c \uc0ac\uc6a9\ud55c Channels Last Memory Format\n*******************************************************\n\n**Author**: `Vitaly Fedyunin `_\n\n**\ubc88\uc5ed**: `Choi Yoonjeong `_\n\nChannels Last\uac00 \ubb34\uc5c7\uc778\uac00\uc694\n----------------------------\nChannels Last \uba54\ubaa8\ub9ac \ud615\uc2dd(memory format)\uc740 \ucc28\uc6d0 \uc21c\uc11c\ub97c \uc720\uc9c0\ud558\uba74\uc11c \uba54\ubaa8\ub9ac \uc0c1\uc758 NCHW \ud150\uc11c(tensor)\ub97c \uc815\ub82c\ud558\ub294 \ub610 \ub2e4\ub978 \ubc29\uc2dd\uc785\ub2c8\ub2e4.\nChannels Last \ud150\uc11c\ub294 \ucc44\ub110(Channel)\uc774 \uac00\uc7a5 \ubc00\ub3c4\uac00 \ub192\uc740(densest) \ucc28\uc6d0\uc73c\ub85c \uc815\ub82c(\uc608. \uc774\ubbf8\uc9c0\ub97c \ud53d\uc140x\ud53d\uc140\ub85c \uc800\uc7a5)\ub429\ub2c8\ub2e4.\n\n\uc608\ub97c \ub4e4\uc5b4, (2\uac1c\uc758 2 x 2 \uc774\ubbf8\uc9c0\uc5d0 3\uac1c\uc758 \ucc44\ub110\uc774 \uc874\uc7ac\ud558\ub294 \uacbd\uc6b0) \uc804\ud615\uc801\uc778(\uc5f0\uc18d\uc801\uc778) NCHW \ud150\uc11c\uc758 \uc800\uc7a5 \ubc29\uc2dd\uc740 \ub2e4\uc74c\uacfc \uac19\uc2b5\ub2c8\ub2e4:\n\n.. figure:: /_static/img/classic_memory_format.png\n :alt: classic_memory_format\n\nChannels Last \uba54\ubaa8\ub9ac \ud615\uc2dd\uc740 \ub370\uc774\ud130\ub97c \ub2e4\ub974\uac8c \uc815\ub82c\ud569\ub2c8\ub2e4:\n\n.. figure:: /_static/img/channels_last_memory_format.png\n :alt: channels_last_memory_format\n\nPyTorch\ub294 \uae30\uc874\uc758 \uc2a4\ud2b8\ub77c\uc774\ub4dc(strides) \uad6c\uc870\ub97c \uc0ac\uc6a9\ud568\uc73c\ub85c\uc368 \uba54\ubaa8\ub9ac \ud615\uc2dd\uc744 \uc9c0\uc6d0(\ud558\uba70, eager, JIT \ubc0f TorchScript\ub97c \ud3ec\ud568\ud55c\n\uae30\uc874\uc758 \ubaa8\ub378\ub4e4\uacfc \ud558\uc704 \ud638\ud658\uc131\uc744 \uc81c\uacf5)\ud569\ub2c8\ub2e4. \uc608\ub97c \ub4e4\uc5b4, Channels Last \ud615\uc2dd\uc5d0\uc11c 10x3x16x16 \ubc30\uce58(batch)\ub294 (768, 1, 48, 3)\uc640\n\uac19\uc740 \ud3ed(strides)\uc744 \uac00\uc9c0\uace0 \uc788\uac8c \ub429\ub2c8\ub2e4.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Channels Last memory format is implemented for 4D NCWH Tensors only.\n\n\n" + "Channels Last \uba54\ubaa8\ub9ac \ud615\uc2dd\uc740 \uc624\uc9c1 4D NCWH Tensors\uc5d0\uc11c\ub9cc \uc2e4\ud589\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\n" ] }, { @@ -40,14 +40,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Memory Format API\n-----------------------\n\nHere is how to convert tensors between contiguous and channels\nlast memory formats.\n\n" + "\uba54\ubaa8\ub9ac \ud615\uc2dd(Memory Format) API\n---------------------------------\n\n\uc5f0\uc18d \uba54\ubaa8\ub9ac \ud615\uc2dd\uacfc channels last \uba54\ubaa8\ub9ac \ud615\uc2dd \uac04\uc5d0 \ud150\uc11c\ub97c \ubcc0\ud658\ud558\ub294 \ubc29\ubc95\uc740 \ub2e4\uc74c\uacfc \uac19\uc2b5\ub2c8\ub2e4.\n\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Classic PyTorch contiguous tensor\n\n" + "\uc804\ud615\uc801\uc778 PyTorch\uc758 \uc5f0\uc18d\uc801\uc778 \ud150\uc11c(tensor)\n\n" ] }, { @@ -58,14 +58,14 @@ }, "outputs": [], "source": [ - "x = torch.empty(N, C, H, W)\nprint(x.stride()) # Ouputs: (3072, 1024, 32, 1)" + "x = torch.empty(N, C, H, W)\nprint(x.stride()) # \uacb0\uacfc: (3072, 1024, 32, 1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Conversion operator\n\n" + "\ubcc0\ud658 \uc5f0\uc0b0\uc790\n\n" ] }, { @@ -76,14 +76,14 @@ }, "outputs": [], "source": [ - "x = x.contiguous(memory_format=torch.channels_last)\nprint(x.shape) # Outputs: (10, 3, 32, 32) as dimensions order preserved\nprint(x.stride()) # Outputs: (3072, 1, 96, 3)" + "x = x.contiguous(memory_format=torch.channels_last)\nprint(x.shape) # \uacb0\uacfc: (10, 3, 32, 32) \ucc28\uc6d0 \uc21c\uc11c\ub294 \ubcf4\uc874\ud568\nprint(x.stride()) # \uacb0\uacfc: (3072, 1, 96, 3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Back to contiguous\n\n" + "\uc5f0\uc18d\uc801\uc778 \ud615\uc2dd\uc73c\ub85c \ub418\ub3cc\ub9ac\uae30\n\n" ] }, { @@ -94,14 +94,14 @@ }, "outputs": [], "source": [ - "x = x.contiguous(memory_format=torch.contiguous_format)\nprint(x.stride()) # Outputs: (3072, 1024, 32, 1)" + "x = x.contiguous(memory_format=torch.contiguous_format)\nprint(x.stride()) # \uacb0\uacfc: (3072, 1024, 32, 1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Alternative option\n\n" + "\ub2e4\ub978 \ubc29\uc2dd\n\n" ] }, { @@ -112,14 +112,14 @@ }, "outputs": [], "source": [ - "x = x.to(memory_format=torch.channels_last)\nprint(x.stride()) # Ouputs: (3072, 1, 96, 3)" + "x = x.to(memory_format=torch.channels_last)\nprint(x.stride()) # \uacb0\uacfc: (3072, 1, 96, 3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Format checks\n\n" + "\ud615\uc2dd(format) \ud655\uc778\n\n" ] }, { @@ -130,14 +130,14 @@ }, "outputs": [], "source": [ - "print(x.is_contiguous(memory_format=torch.channels_last)) # Ouputs: True" + "print(x.is_contiguous(memory_format=torch.channels_last)) # \uacb0\uacfc: True" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Create as Channels Last\n\n" + "Channels Last \ubc29\uc2dd\uc73c\ub85c \uc0dd\uc131\ud558\uae30\n\n" ] }, { @@ -148,14 +148,14 @@ }, "outputs": [], "source": [ - "x = torch.empty(N, C, H, W, memory_format=torch.channels_last)\nprint(x.stride()) # Ouputs: (3072, 1, 96, 3)" + "x = torch.empty(N, C, H, W, memory_format=torch.channels_last)\nprint(x.stride()) # \uacb0\uacfc: (3072, 1, 96, 3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "``clone`` preserves memory format\n\n" + "``clone`` \uc740 \uba54\ubaa8\ub9ac \ud615\uc2dd\uc744 \ubcf4\uc874\ud569\ub2c8\ub2e4.\n\n" ] }, { @@ -166,14 +166,14 @@ }, "outputs": [], "source": [ - "y = x.clone()\nprint(y.stride()) # Ouputs: (3072, 1, 96, 3)" + "y = x.clone()\nprint(y.stride()) # \uacb0\uacfc: (3072, 1, 96, 3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "``to``, ``cuda``, ``float`` ... preserves memory format\n\n" + "``to``, ``cuda``, ``float`` ... \ub4f1\ub3c4 \uba54\ubaa8\ub9ac \ud615\uc2dd\uc744 \ubcf4\uc874\ud569\ub2c8\ub2e4.\n\n" ] }, { @@ -184,14 +184,14 @@ }, "outputs": [], "source": [ - "if torch.cuda.is_available():\n y = x.cuda()\n print(y.stride()) # Ouputs: (3072, 1, 96, 3)" + "if torch.cuda.is_available():\n y = x.cuda()\n print(y.stride()) # \uacb0\uacfc: (3072, 1, 96, 3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "``empty_like``, ``*_like`` operators preserves memory format\n\n" + "``empty_like``, ``*_like`` \uc5f0\uc0b0\uc790\ub3c4 \uba54\ubaa8\ub9ac \ud615\uc2dd\uc744 \ubcf4\uc874\ud569\ub2c8\ub2e4.\n\n" ] }, { @@ -202,14 +202,14 @@ }, "outputs": [], "source": [ - "y = torch.empty_like(x)\nprint(y.stride()) # Ouputs: (3072, 1, 96, 3)" + "y = torch.empty_like(x)\nprint(y.stride()) # \uacb0\uacfc: (3072, 1, 96, 3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Pointwise operators preserves memory format\n\n" + "Pointwise \uc5f0\uc0b0\uc790\ub3c4 \uba54\ubaa8\ub9ac \ud615\uc2dd\uc744 \ubcf4\uc874\ud569\ub2c8\ub2e4.\n\n" ] }, { @@ -220,14 +220,14 @@ }, "outputs": [], "source": [ - "z = x + y\nprint(z.stride()) # Ouputs: (3072, 1, 96, 3)" + "z = x + y\nprint(z.stride()) # \uacb0\uacfc: (3072, 1, 96, 3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Conv, Batchnorm modules support Channels Last\n(only works for CudNN >= 7.6)\n\n" + "Conv, Batchnorm \ubaa8\ub4c8\uc740 Channels Last\ub97c \uc9c0\uc6d0\ud569\ub2c8\ub2e4. (\ub2e8, CudNN >=7.6 \uc5d0\uc11c\ub9cc \ub3d9\uc791)\n\n" ] }, { @@ -238,14 +238,14 @@ }, "outputs": [], "source": [ - "if torch.backends.cudnn.version() >= 7603:\n input = torch.randint(1, 10, (2, 8, 4, 4), dtype=torch.float32, device=\"cuda\", requires_grad=True)\n model = torch.nn.Conv2d(8, 4, 3).cuda().float()\n\n input = input.contiguous(memory_format=torch.channels_last)\n model = model.to(memory_format=torch.channels_last) # Module parameters need to be Channels Last\n\n out = model(input)\n print(out.is_contiguous(memory_format=torch.channels_last)) # Ouputs: True" + "if torch.backends.cudnn.version() >= 7603:\n input = torch.randint(1, 10, (2, 8, 4, 4), dtype=torch.float32, device=\"cuda\", requires_grad=True)\n model = torch.nn.Conv2d(8, 4, 3).cuda().float()\n\n input = input.contiguous(memory_format=torch.channels_last)\n model = model.to(memory_format=torch.channels_last) # \ubaa8\ub4c8 \uc778\uc790\ub4e4\uc740 Channels Last\ub85c \ubcc0\ud658\uc774 \ud544\uc694\ud569\ub2c8\ub2e4\n\n out = model(input)\n print(out.is_contiguous(memory_format=torch.channels_last)) # \uacb0\uacfc: True" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Performance Gains\n-------------------------------------------------------------------------------------------\nThe most significant performance gains are observed on Nvidia's hardware with\nTensor Cores support. We were able to archive over 22% perf gains while running '\nAMP (Automated Mixed Precision) training scripts supplied by Nvidia https://github.com/NVIDIA/apex.\n\n``python main_amp.py -a resnet50 --b 200 --workers 16 --opt-level O2 ./data``\n\n" + "\uc131\ub2a5 \ud5a5\uc0c1\n-------------------------------------------------------------------------------------------\nTensor Cores\ub97c \uc9c0\uc6d0\ud558\ub294 Nvidia\uc758 \ud558\ub4dc\uc6e8\uc5b4\uc5d0\uc11c \uac00\uc7a5 \uc758\ubbf8\uc2ec\uc7a5\ud55c \uc131\ub2a5 \ud5a5\uc0c1\uc744 \ubcf4\uc600\uc2b5\ub2c8\ub2e4.\nNvidia\uac00 \uc81c\uacf5\ud558\ub294 AMP(Automated Mixed Precision) \ud559\uc2b5 \uc2a4\ud06c\ub9bd\ud2b8\ub85c 22% \uc774\uc0c1\uc758 \uc131\ub2a5 \ud5a5\uc0c1\uc744 \ud655\uc778\ud560 \uc218 \uc788\uc5c8\uc2b5\ub2c8\ub2e4.\nhttps://github.com/NVIDIA/apex\n\n``python main_amp.py -a resnet50 --b 200 --workers 16 --opt-level O2 ./data``\n\n" ] }, { @@ -263,7 +263,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Passing ``--channels-last true`` allows running a model in Channels Last format with observed 22% perf gain.\n\n``python main_amp.py -a resnet50 --b 200 --workers 16 --opt-level O2 --channels-last true ./data``\n\n" + "``--channels-last true`` \uc778\uc790\ub97c \uc804\ub2ec\ud558\uc5ec Channels Last \ud615\uc2dd\uc73c\ub85c \ubaa8\ub378\uc744 \uc2e4\ud589\ud558\uba74 22%\uc758 \uc131\ub2a5 \ud5a5\uc0c1\uc744 \ubcf4\uc785\ub2c8\ub2e4.\n\n``python main_amp.py -a resnet50 --b 200 --workers 16 --opt-level O2 --channels-last true ./data``\n\n" ] }, { @@ -281,14 +281,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The following list of models has the full support of Channels Last and showing 8%-35% perf gains on Volta devices:\n``alexnet``, ``mnasnet0_5``, ``mnasnet0_75``, ``mnasnet1_0``, ``mnasnet1_3``, ``mobilenet_v2``, ``resnet101``, ``resnet152``, ``resnet18``, ``resnet34``, ``resnet50``, ``resnext50_32x4d``, ``shufflenet_v2_x0_5``, ``shufflenet_v2_x1_0``, ``shufflenet_v2_x1_5``, ``shufflenet_v2_x2_0``, ``squeezenet1_0``, ``squeezenet1_1``, ``vgg11``, ``vgg11_bn``, ``vgg13``, ``vgg13_bn``, ``vgg16``, ``vgg16_bn``, ``vgg19``, ``vgg19_bn``, ``wide_resnet101_2``, ``wide_resnet50_2``\n\n\n" + "\uc544\ub798 \ubaa9\ub85d\uc758 \ubaa8\ub378\ub4e4\uc740 Channels Last \ud615\uc2dd\uc744 \uc804\uc801\uc73c\ub85c \uc9c0\uc6d0(full support)\ud558\uba70 Volta \uc7a5\ube44\uc5d0\uc11c 8%-35%\uc758 \uc131\ub2a5 \ud5a5\uc0c1\uc744 \ubcf4\uc785\ub2c8\ub2e4:\n``alexnet``, ``mnasnet0_5``, ``mnasnet0_75``, ``mnasnet1_0``, ``mnasnet1_3``, ``mobilenet_v2``, ``resnet101``, ``resnet152``, ``resnet18``, ``resnet34``, ``resnet50``, ``resnext50_32x4d``, ``shufflenet_v2_x0_5``, ``shufflenet_v2_x1_0``, ``shufflenet_v2_x1_5``, ``shufflenet_v2_x2_0``, ``squeezenet1_0``, ``squeezenet1_1``, ``vgg11``, ``vgg11_bn``, ``vgg13``, ``vgg13_bn``, ``vgg16``, ``vgg16_bn``, ``vgg19``, ``vgg19_bn``, ``wide_resnet101_2``, ``wide_resnet50_2``\n\n\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Converting existing models\n--------------------------\n\nChannels Last support not limited by existing models, as any model can be converted to Channels Last and propagate format through the graph as soon as input formatted correctly.\n\n\n" + "\uae30\uc874 \ubaa8\ub378\ub4e4 \ubcc0\ud658\ud558\uae30\n--------------------------\n\nChannels Last \uc9c0\uc6d0\uc740 \uae30\uc874 \ubaa8\ub378\uc774 \ubb34\uc5c7\uc774\ub0d0\uc5d0 \ub530\ub77c \uc81c\ud55c\ub418\uc9c0 \uc54a\uc73c\uba70, \uc5b4\ub5a0\ud55c \ubaa8\ub378\ub3c4 Channels Last\ub85c \ubcc0\ud658\ud560 \uc218 \uc788\uc73c\uba70\n\uc785\ub825(input)\uc758 \ud615\uc2dd\ub9cc \ub9de\ucdb0\uc8fc\uba74 (\uc2e0\uacbd\ub9dd) \uadf8\ub798\ud504\ub97c \ud1b5\ud574 \ubc14\ub85c \uc804\ud30c(propagate)\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n\n" ] }, { @@ -299,14 +299,14 @@ }, "outputs": [], "source": [ - "# Need to be done once, after model initialization (or load)\nmodel = model.to(memory_format=torch.channels_last) # Replace with your model\n\n# Need to be done for every input\ninput = input.to(memory_format=torch.channels_last) # Replace with your input\noutput = model(input)" + "# \ubaa8\ub378\uc744 \ucd08\uae30\ud654\ud55c(\ub610\ub294 \ubd88\ub7ec\uc628) \uc774\ud6c4, \ud55c \ubc88 \uc2e4\ud589\uc774 \ud544\uc694\ud569\ub2c8\ub2e4.\nmodel = model.to(memory_format=torch.channels_last) # \uc6d0\ud558\ub294 \ubaa8\ub378\ub85c \uad50\uccb4\ud558\uae30\n\n# \ubaa8\ub4e0 \uc785\ub825\uc5d0 \ub300\ud574\uc11c \uc2e4\ud589\uc774 \ud544\uc694\ud569\ub2c8\ub2e4.\ninput = input.to(memory_format=torch.channels_last) # \uc6d0\ud558\ub294 \uc785\ub825\uc73c\ub85c \uad50\uccb4\ud558\uae30\noutput = model(input)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "However, not all operators fully converted to support Channels Last (usually returning\ncontiguous output instead). That means you need to verify the list of used operators\nagainst supported operators list https://github.com/pytorch/pytorch/wiki/Operators-with-Channels-Last-support,\nor introduce memory format checks into eager execution mode and run your model.\n\nAfter running the code below, operators will raise an exception if the output of the\noperator doesn't match the memory format of the input.\n\n\n\n" + "\uadf8\ub7ec\ub098, \ubaa8\ub4e0 \uc5f0\uc0b0\uc790\ub4e4\uc774 Channels Last\ub97c \uc9c0\uc6d0\ud558\ub3c4\ub85d \uc644\uc804\ud788 \ubc14\ub010 \uac83\uc740 \uc544\ub2d9\ub2c8\ub2e4(\uc77c\ubc18\uc801\uc73c\ub85c\ub294 \uc5f0\uc18d\uc801\uc778 \ucd9c\ub825\uc744 \ub300\uc2e0 \ubc18\ud658\ud569\ub2c8\ub2e4).\n\uc989, Channel Last \uc9c0\uc6d0 \uc5f0\uc0b0\uc790 \ubaa9\ub85d https://github.com/pytorch/pytorch/wiki/Operators-with-Channels-Last-support \uc5d0\uc11c \uc0ac\uc6a9\ud55c \uc5f0\uc0b0\uc790\ub4e4\uc774 \uc874\uc7ac\ud558\ub294\uc9c0 \ud655\uc778\ud558\uac70\ub098,\neager \uc2e4\ud589 \ubaa8\ub4dc\uc5d0\uc11c \uba54\ubaa8\ub9ac \ud615\uc2dd \uac80\uc0ac\ub97c \ub3c4\uc785\ud558\uace0 \ubaa8\ub378\uc744 \uc2e4\ud589\ud574\uc57c \ud569\ub2c8\ub2e4.\n\n\uc544\ub798 \ucf54\ub4dc\uc5d0\uc11c, \uc5f0\uc0b0\uc790\ub4e4\uc758 \ucd9c\ub825\uc774 \uc785\ub825\uc758 \uba54\ubaa8\ub9ac \ud615\uc2dd\uacfc \uc77c\uce58\ud558\uc9c0 \uc54a\uc73c\uba74 \uc608\uc678(exception)\ub97c \ubc1c\uc0dd\uc2dc\ud0b5\ub2c8\ub2e4.\n\n\n\n" ] }, { @@ -324,14 +324,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "If you found an operator that doesn't support Channels Last tensors\nand you want to contribute, feel free to use following developers\nguide https://github.com/pytorch/pytorch/wiki/Writing-memory-format-aware-operators.\n\n\n" + "\ub9cc\uc57d Channels Last \ud150\uc11c\ub97c \uc9c0\uc6d0\ud558\uc9c0 \uc54a\ub294 \uc5f0\uc0b0\uc790\ub97c \ubc1c\uacac\ud558\uc600\uace0, \uae30\uc5ec\ud558\uae30\ub97c \uc6d0\ud55c\ub2e4\uba74\n\ub2e4\uc74c \uac1c\ubc1c \ubb38\uc11c\ub97c \ucc38\uace0\ud574\uc8fc\uc138\uc694.\nhttps://github.com/pytorch/pytorch/wiki/Writing-memory-format-aware-operators\n\n\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Work to do\n----------\nThere are still many things to do, such as:\n\n- Resolving ambiguity of N1HW and NC11 Tensors;\n- Testing of Distributed Training support;\n- Improving operators coverage.\n\nIf you have feedback and/or suggestions for improvement, please let us\nknow by creating `an issue `_.\n\n" + "\ud574\uc57c\ud560 \uc77c\n----------\n\ub2e4\uc74c\uacfc \uac19\uc774 \uc5ec\uc804\ud788 \ud574\uc57c \ud560 \uc77c\uc774 \ub9ce\uc774 \ub0a8\uc544\uc788\uc2b5\ub2c8\ub2e4:\n\n- N1HW\uc640 NC11 Tensors\uc758 \ubaa8\ud638\uc131 \ud574\uacb0\ud558\uae30;\n- \ubd84\uc0b0 \ud559\uc2b5\uc744 \uc9c0\uc6d0\ud558\ub294\uc9c0 \ud655\uc778\ud558\uae30;\n- \uc5f0\uc0b0\uc790 \ubc94\uc704(operators coverage) \uac1c\uc120(improve)\ud558\uae30\n\n\uac1c\uc120\ud560 \ubd80\ubd84\uc5d0 \ub300\ud55c \ud53c\ub4dc\ubc31 \ub610\ub294 \uc81c\uc548\uc774 \uc788\ub2e4\uba74 `\uc774\uc288\ub97c \ub9cc\ub4e4\uc5b4 `_ \uc54c\ub824\uc8fc\uc138\uc694.\n\n\n" ] } ], @@ -351,7 +351,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/f16255c783f9e487235b8eff6c8792b9/nn_tutorial.py b/docs/_downloads/f16255c783f9e487235b8eff6c8792b9/nn_tutorial.py index bbce4b7bf..6606bf3cb 100644 --- a/docs/_downloads/f16255c783f9e487235b8eff6c8792b9/nn_tutorial.py +++ b/docs/_downloads/f16255c783f9e487235b8eff6c8792b9/nn_tutorial.py @@ -1,13 +1,15 @@ # -*- coding: utf-8 -*- """ `torch.nn` 이 *실제로* 무엇인가요? -=============================== +===================================== 저자: Jeremy Howard, `fast.ai `_. 도움: Rachel Thomas, Francisco Ingham. 번역: `남상호 `_ + """ + ############################################################################### # 이 튜토리얼을 스크립트가 아닌 노트북으로 실행하기를 권장합니다. 노트북 (.ipynb) 파일을 다운 받으시려면, # 페이지 상단에 있는 링크를 클릭해주세요. @@ -32,7 +34,7 @@ # 거의 동일하다는 것을 알게 될 것입니다). # # MNIST 데이터 준비 -# ---------------- +# ------------------- # # 우리는 손으로 쓴 숫자(0에서 9 사이)의 흑백 이미지로 구성된 클래식 # `MNIST `_ 데이터셋을 사용할 것 입니다. @@ -51,7 +53,7 @@ PATH.mkdir(parents=True, exist_ok=True) -URL = "http://deeplearning.net/data/mnist/" +URL = "https://github.com/pytorch/tutorials/raw/master/_static/" FILENAME = "mnist.pkl.gz" if not (PATH / FILENAME).exists(): @@ -277,7 +279,7 @@ def model(xb): ############################################################################### # nn.Module 을 이용하여 리팩토링 하기 -# ------------------------------ +# -------------------------------------- # 다음으로, 더 명확하고 간결한 훈련 루프를 위해 ``nn.Module`` 및 ``nn.Parameter`` 를 사용합니다. # 우리는 ``nn.Module`` (자체가 클래스이고 상태를 추척할 수 있는) 하위 클래스(subclass)를 만듭니다. # 이 경우에는, 포워드(forward) 단계에 대한 가중치, 절편, 그리고 메소드(method) 등을 유지하는 @@ -317,7 +319,9 @@ def forward(self, xb): ############################################################################### # 이전에는 훈련 루프를 위해 이름 별로 각 매개변수(parameter)의 값을 업데이트하고 다음과 같이 # 각 매개 변수에 대한 기울기들을 개별적으로 수동으로 0으로 제거해야 했습니다: +# # :: +# # with torch.no_grad(): # weights -= weights.grad * lr # bias -= bias.grad * lr @@ -328,7 +332,9 @@ def forward(self, xb): # 이제 우리는 model.parameters() 및 model.zero_grad() (모두 # ``nn.Module`` 에 대해 PyTorch에 의해 정의됨)를 활용하여 이러한 단계를 더 간결하게 # 만들고, 특히 더 복잡한 모델에 대해서 일부 매개변수를 잊어 버리는 오류를 덜 발생시킬 수 있습니다: +# # :: +# # with torch.no_grad(): # for p in model.parameters(): p -= p.grad * lr # model.zero_grad() @@ -361,7 +367,7 @@ def fit(): ############################################################################### # nn.Linear 를 이용하여 리팩토링 하기 -# ------------------------------ +# ------------------------------------ # # 계속해서 코드를 리팩토링 합니다. ``self.weights`` 및 ``self.bias`` 를 수동으로 정의 및 # 초기화하고, ``xb @ self.weights + self.bias`` 를 계산하는 대신에, @@ -394,20 +400,24 @@ def forward(self, xb): ############################################################################### # optim 을 이용하여 리팩토링 하기 -# ------------------------------ +# --------------------------------- # # Pytorch에는 다양한 최적화(optimization) 알고리즘을 가진 패키지인 ``torch.optim`` 도 있습니다. # 각 매개변수를 수동으로 업데이트 하는 대신, 옵티마이저(optimizer)의 ``step`` 메소드를 사용하여 # 업데이트를 진행할 수 있습니다. # # 이렇게 하면 이전에 수동으로 코딩한 최적화 단계를 대체할 수 있습니다: +# # :: +# # with torch.no_grad(): # for p in model.parameters(): p -= p.grad * lr # model.zero_grad() # # 대신에 이렇게 말이죠: +# # :: +# # opt.step() # opt.zero_grad() # @@ -443,7 +453,7 @@ def get_model(): ############################################################################### # Dataset 을 이용하여 리팩토링하기 -# ------------------------------ +# ---------------------------------- # # PyTorch 에는 추상 Dataset 클래스가 있습니다. Dataset 은 # ``__len__`` 함수 (Python의 표준 ``len`` 함수에 의해 호출됨) 및 @@ -468,13 +478,17 @@ def get_model(): ############################################################################### # 이전에는 x 및 y 값의 미니 배치를 별도로 반복해야했습니다: +# # :: +# # xb = x_train[start_i:end_i] # yb = y_train[start_i:end_i] # # # 이제 이 두 단계를 함께 수행 할 수 있습니다: +# # :: +# # xb,yb = train_ds[i*bs : i*bs+bs] # @@ -494,7 +508,7 @@ def get_model(): ############################################################################### # DataLoader 를 이용하여 리팩토링하기 -# ------------------------------- +# ----------------------------------- # # Pytorch 의 ``DataLoader`` 는 배치 관리를 담당합니다. # 여러분들은 모든 ``Dataset`` 으로부터 ``DataLoader`` 를 생성할 수 있습니다. @@ -509,13 +523,17 @@ def get_model(): ############################################################################### # 이전에는 루프가 다음과 같이 배치 (xb, yb)를 반복했습니다: +# # :: +# # for i in range((n-1)//bs + 1): # xb,yb = train_ds[i*bs : i*bs+bs] # pred = model(xb) # # 이제 (xb, yb)가 DataLoader 에서 자동으로 로드되므로 루프가 훨씬 깨끗해졌습니다: +# # :: +# # for xb,yb in train_dl: # pred = model(xb) @@ -538,7 +556,7 @@ def get_model(): # 이제 실제로 효과적인 모델을 만드는 데 필요한 기본 기능을 추가해 보겠습니다. # # 검증(validation) 추가하기 -# ----------------------- +# --------------------------- # # 섹션 1에서, 우리는 훈련 데이터에 사용하기 위해 합리적인 훈련 루프를 설정하려고했습니다. # 실전에서, 여러분들은 과적합(overfitting)을 확인하기 위해서 **항상** @@ -650,7 +668,7 @@ def get_data(train_ds, valid_ds, bs): # 컨볼루션 신경망(CNN)을 훈련하는 데 사용할 수 있는지 살펴 보겠습니다! # # CNN 으로 넘어가기 -# --------------- +# -------------------- # # 이제 3개의 컨볼루션 레이어로 신경망을 구축할 것입니다. # 이전 섹션의 어떤 함수도 모델의 형식에 대해 가정하지 않기 때문에, diff --git a/docs/_downloads/f1992c37a58ca1039b273eaac839f845/custom_dataset_transforms_loader.py b/docs/_downloads/f1992c37a58ca1039b273eaac839f845/custom_dataset_transforms_loader.py index b5e0b93bd..e73039647 100644 --- a/docs/_downloads/f1992c37a58ca1039b273eaac839f845/custom_dataset_transforms_loader.py +++ b/docs/_downloads/f1992c37a58ca1039b273eaac839f845/custom_dataset_transforms_loader.py @@ -222,7 +222,7 @@ def __getitem__(self, idx): # 변형들은 훈련시에 사용이 되며 주로 데이터 증강으로 참조되며 최근의 모델 개발에선 흔히 사용됩니다. # # 데이터셋을 다룰때 자주 일어나는 문제중 하나는 모든 샘플들이 같은 크기를 가지고 있지 않을 경우입니다. -# 대부분의 신경망들은 미리 정해진 크기의 이미지들을 받아드립니다. +# 대부분의 신경망들은 미리 정해진 크기의 이미지들을 받아들입니다. # 그렇기 때문에 우리는 전처리 코드를 작성해야할 필요가 있습니다. # 이제 세개의 변형을 만들어 봅시다. # diff --git a/docs/_downloads/f41f6eb36b8dff464b9cd21b8ea30765/super_resolution_with_onnxruntime.ipynb b/docs/_downloads/f41f6eb36b8dff464b9cd21b8ea30765/super_resolution_with_onnxruntime.ipynb index edbf49ad3..a3010faf3 100644 --- a/docs/_downloads/f41f6eb36b8dff464b9cd21b8ea30765/super_resolution_with_onnxruntime.ipynb +++ b/docs/_downloads/f41f6eb36b8dff464b9cd21b8ea30765/super_resolution_with_onnxruntime.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n(\uc120\ud0dd) PyTorch \ubaa8\ub378\uc744 ONNX\uc73c\ub85c \ubcc0\ud658\ud558\uace0 ONNX \ub7f0\ud0c0\uc784\uc5d0\uc11c \uc2e4\ud589\ud558\uae30\n========================================================================\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 \uc5b4\ub5bb\uac8c PyTorch\uc5d0\uc11c \uc815\uc758\ub41c \ubaa8\ub378\uc744 ONNX \ud615\uc2dd\uc73c\ub85c \ubcc0\ud658\ud558\uace0 \ub610 \uc5b4\ub5bb\uac8c \uadf8 \ubcc0\ud658\ub41c \ubaa8\ub378\uc744\nONNX \ub7f0\ud0c0\uc784\uc5d0\uc11c \uc2e4\ud589\ud560 \uc218 \uc788\ub294\uc9c0\uc5d0 \ub300\ud574 \uc54c\uc544\ubcf4\ub3c4\ub85d \ud558\uaca0\uc2b5\ub2c8\ub2e4.\nONNX \ub7f0\ud0c0\uc784\uc740 ONNX \ubaa8\ub378\uc744 \uc704\ud55c \uc5d4\uc9c4\uc73c\ub85c\uc11c \uc131\ub2a5\uc5d0 \ucd08\uc810\uc744 \ub9de\ucd94\uace0 \uc788\uace0 \uc5ec\ub7ec \ub2e4\uc591\ud55c \ud50c\ub7ab\ud3fc\uacfc \ud558\ub4dc\uc6e8\uc5b4(\uc708\ub3c4\uc6b0,\n\ub9ac\ub205\uc2a4, \ub9e5\uc744 \ube44\ub86f\ud55c \ud50c\ub7ab\ud3fc \ubfd0\ub9cc \uc544\ub2c8\ub77c CPU, GPU \ub4f1\uc758 \ud558\ub4dc\uc6e8\uc5b4)\uc5d0\uc11c \ud6a8\uc728\uc801\uc778 \ucd94\ub860\uc744 \uac00\ub2a5\ud558\uac8c \ud569\ub2c8\ub2e4.\nONNX \ub7f0\ud0c0\uc784\uc740 `\uc5ec\uae30\n`__ \uc5d0\uc11c\n\uc124\uba85\ub41c \uac83\uacfc \uac19\uc774 \uc5ec\ub7ec \ubaa8\ub378\ub4e4\uc758 \uc131\ub2a5\uc744 \uc0c1\ub2f9\ud788 \ub192\uc77c \uc218 \uc788\ub2e4\ub294 \uc810\uc774 \uc99d\uba85\ub418\uc5c8\uc2b5\ub2c8\ub2e4.\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc744 \uc9c4\ud589\ud558\uae30 \uc704\ud574\uc11c\ub294 `ONNX `__\n\uc640 `ONNX Runtime `__ \uc124\uce58\uac00 \ud544\uc694\ud569\ub2c8\ub2e4.\nONNX\uc640 ONNX \ub7f0\ud0c0\uc784\uc758 \ubc14\uc774\ub108\ub9ac \ube4c\ub4dc\ub97c ``pip install onnx onnxruntime`` \ub97c \ud1b5\ud574 \ubc1b\uc744 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\nONNX \ub7f0\ud0c0\uc784\uc740 \ubc84\uc804 3.5\uc5d0\uc11c 3.7\uae4c\uc9c0\uc758 Python\uacfc \ud638\ud658\ub429\ub2c8\ub2e4.\n``\ucc38\uace0``: \ubcf8 \ud29c\ud1a0\ub9ac\uc5bc\uc740 PyTorch\uc758 master \ube0c\ub79c\uce58\ub97c \ud544\uc694\ub85c\ud558\uba70 `\ub9c1\ud06c `__ \uc5d0\uc11c\n\uc124\uce58\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n" + "\n(\uc120\ud0dd) PyTorch \ubaa8\ub378\uc744 ONNX\uc73c\ub85c \ubcc0\ud658\ud558\uace0 ONNX \ub7f0\ud0c0\uc784\uc5d0\uc11c \uc2e4\ud589\ud558\uae30\n========================================================================\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 \uc5b4\ub5bb\uac8c PyTorch\uc5d0\uc11c \uc815\uc758\ub41c \ubaa8\ub378\uc744 ONNX \ud615\uc2dd\uc73c\ub85c \ubcc0\ud658\ud558\uace0 \ub610 \uc5b4\ub5bb\uac8c \uadf8 \ubcc0\ud658\ub41c \ubaa8\ub378\uc744\nONNX \ub7f0\ud0c0\uc784\uc5d0\uc11c \uc2e4\ud589\ud560 \uc218 \uc788\ub294\uc9c0\uc5d0 \ub300\ud574 \uc54c\uc544\ubcf4\ub3c4\ub85d \ud558\uaca0\uc2b5\ub2c8\ub2e4.\nONNX \ub7f0\ud0c0\uc784\uc740 ONNX \ubaa8\ub378\uc744 \uc704\ud55c \uc5d4\uc9c4\uc73c\ub85c\uc11c \uc131\ub2a5\uc5d0 \ucd08\uc810\uc744 \ub9de\ucd94\uace0 \uc788\uace0 \uc5ec\ub7ec \ub2e4\uc591\ud55c \ud50c\ub7ab\ud3fc\uacfc \ud558\ub4dc\uc6e8\uc5b4(\uc708\ub3c4\uc6b0,\n\ub9ac\ub205\uc2a4, \ub9e5\uc744 \ube44\ub86f\ud55c \ud50c\ub7ab\ud3fc \ubfd0\ub9cc \uc544\ub2c8\ub77c CPU, GPU \ub4f1\uc758 \ud558\ub4dc\uc6e8\uc5b4)\uc5d0\uc11c \ud6a8\uc728\uc801\uc778 \ucd94\ub860\uc744 \uac00\ub2a5\ud558\uac8c \ud569\ub2c8\ub2e4.\nONNX \ub7f0\ud0c0\uc784\uc740 `\uc5ec\uae30\n`__ \uc5d0\uc11c\n\uc124\uba85\ub41c \uac83\uacfc \uac19\uc774 \uc5ec\ub7ec \ubaa8\ub378\ub4e4\uc758 \uc131\ub2a5\uc744 \uc0c1\ub2f9\ud788 \ub192\uc77c \uc218 \uc788\ub2e4\ub294 \uc810\uc774 \uc99d\uba85\ub418\uc5c8\uc2b5\ub2c8\ub2e4.\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc744 \uc9c4\ud589\ud558\uae30 \uc704\ud574\uc11c\ub294 `ONNX `__\n\uc640 `ONNX Runtime `__ \uc124\uce58\uac00 \ud544\uc694\ud569\ub2c8\ub2e4.\nONNX\uc640 ONNX \ub7f0\ud0c0\uc784\uc758 \ubc14\uc774\ub108\ub9ac \ube4c\ub4dc\ub97c ``pip install onnx onnxruntime`` \ub97c \ud1b5\ud574 \ubc1b\uc744 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\nONNX \ub7f0\ud0c0\uc784\uc740 \ubc84\uc804 3.5\uc5d0\uc11c 3.7\uae4c\uc9c0\uc758 Python\uacfc \ud638\ud658\ub429\ub2c8\ub2e4.\n``\ucc38\uace0``: \ubcf8 \ud29c\ud1a0\ub9ac\uc5bc\uc740 PyTorch\uc758 master \ube0c\ub79c\uce58\ub97c \ud544\uc694\ub85c\ud558\uba70 `\ub9c1\ud06c `__ \uc5d0\uc11c\n\uc124\uce58\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n" ] }, { @@ -69,7 +69,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\uc774\uc81c Tracing\uc774\ub098 \uc2a4\ud06c\ub9bd\ud305\uc744 \ud1b5\ud574\uc11c PyTorch \ubaa8\ub378\uc744 \ubcc0\ud658\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 tracing\uc744 \ud1b5\ud574 \ubcc0\ud658\ub41c \ubaa8\ub378\uc744 \uc0ac\uc6a9\ud558\ub3c4\ub85d \ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\ubaa8\ub378\uc744 \ubcc0\ud658\ud558\uae30 \uc704\ud574\uc11c\ub294 ``torch.onnx.export()`` \ud568\uc218\ub97c \ud638\ucd9c\ud569\ub2c8\ub2e4.\n\uc774 \ud568\uc218\ub294 \ubaa8\ub378\uc744 \uc2e4\ud589\ud558\uc5ec \uc5b4\ub5a4 \uc5f0\uc0b0\uc790\ub4e4\uc774 \ucd9c\ub825\uac12\uc744 \uacc4\uc0b0\ud558\ub294\ub370 \uc0ac\uc6a9\ub418\uc5c8\ub294\uc9c0\ub97c \uae30\ub85d\ud569\ub2c8\ub2e4.\n``export`` \ud568\uc218\uac00 \ubaa8\ub378\uc744 \uc2e4\ud589\ud558\uae30 \ub54c\ubb38\uc5d0, \uc6b0\ub9ac\uac00 \uc9c1\uc811 \ud150\uc11c\ub97c \uc785\ub825\uac12\uc73c\ub85c \ub118\uaca8\uc8fc\uc5b4\uc57c \ud569\ub2c8\ub2e4.\n\uc774 \ud150\uc11c\uc758 \uac12\uc740 \uc54c\ub9de\uc740 \uc790\ub8cc\ud615\uacfc \ubaa8\uc591\uc774\ub77c\uba74 \ub79c\ub364\ud558\uac8c \uacb0\uc815\ub418\uc5b4\ub3c4 \ubb34\ubc29\ud569\ub2c8\ub2e4.\n\ud2b9\uc815 \ucc28\uc6d0\uc744 \ub3d9\uc801\uc778 \ucc28\uc6d0\uc73c\ub85c \uc9c0\uc815\ud558\uc9c0 \uc54a\ub294 \uc774\uc0c1, ONNX\ub85c \ubcc0\ud658\ub41c \uadf8\ub798\ud504\uc758 \uacbd\uc6b0 \uc785\ub825\uac12\uc758 \uc0ac\uc774\uc988\ub294 \ubaa8\ub4e0 \ucc28\uc6d0\uc5d0 \ub300\ud574 \uace0\uc815\ub429\ub2c8\ub2e4. \n\uc608\uc2dc\uc5d0\uc11c\ub294 \ubaa8\ub378\uc774 \ud56d\uc0c1 \ubc30\uce58 \uc0ac\uc774\uc988 1\uc744 \uc0ac\uc6a9\ud558\ub3c4\ub85d \ubcc0\ud658\ud558\uc600\uc9c0\ub9cc, \uccab\ubc88\uc9f8 \ucc28\uc6d0\uc744 ``torch.onnx.export()`` \uc758\n``dynamic_axes`` \uc778\uc790\uc5d0 \ub3d9\uc801\uc778 \ucc28\uc6d0\uc73c\ub85c \uc9c0\uc815\ud574\uc8fc\uba74 \ub429\ub2c8\ub2e4.\n\nPyTorch\uc758 \ubcc0\ud658 \uc778\ud130\ud398\uc774\uc2a4\uc5d0 \ub300\ud574 \ub354 \uc790\uc138\ud788 \uc54c\uace0 \uc2f6\ub2e4\uba74\n`torch.onnx \ubb38\uc11c `__ \ub97c \ucc38\uace0\ud574\uc8fc\uc138\uc694.\n\n\n" + "\uc774\uc81c Tracing\uc774\ub098 \uc2a4\ud06c\ub9bd\ud305\uc744 \ud1b5\ud574\uc11c PyTorch \ubaa8\ub378\uc744 \ubcc0\ud658\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c\ub294 tracing\uc744 \ud1b5\ud574 \ubcc0\ud658\ub41c \ubaa8\ub378\uc744 \uc0ac\uc6a9\ud558\ub3c4\ub85d \ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\ubaa8\ub378\uc744 \ubcc0\ud658\ud558\uae30 \uc704\ud574\uc11c\ub294 ``torch.onnx.export()`` \ud568\uc218\ub97c \ud638\ucd9c\ud569\ub2c8\ub2e4.\n\uc774 \ud568\uc218\ub294 \ubaa8\ub378\uc744 \uc2e4\ud589\ud558\uc5ec \uc5b4\ub5a4 \uc5f0\uc0b0\uc790\ub4e4\uc774 \ucd9c\ub825\uac12\uc744 \uacc4\uc0b0\ud558\ub294\ub370 \uc0ac\uc6a9\ub418\uc5c8\ub294\uc9c0\ub97c \uae30\ub85d\ud569\ub2c8\ub2e4.\n``export`` \ud568\uc218\uac00 \ubaa8\ub378\uc744 \uc2e4\ud589\ud558\uae30 \ub54c\ubb38\uc5d0, \uc6b0\ub9ac\uac00 \uc9c1\uc811 \ud150\uc11c\ub97c \uc785\ub825\uac12\uc73c\ub85c \ub118\uaca8\uc8fc\uc5b4\uc57c \ud569\ub2c8\ub2e4.\n\uc774 \ud150\uc11c\uc758 \uac12\uc740 \uc54c\ub9de\uc740 \uc790\ub8cc\ud615\uacfc \ubaa8\uc591\uc774\ub77c\uba74 \ub79c\ub364\ud558\uac8c \uacb0\uc815\ub418\uc5b4\ub3c4 \ubb34\ubc29\ud569\ub2c8\ub2e4.\n\ud2b9\uc815 \ucc28\uc6d0\uc744 \ub3d9\uc801\uc778 \ucc28\uc6d0\uc73c\ub85c \uc9c0\uc815\ud558\uc9c0 \uc54a\ub294 \uc774\uc0c1, ONNX\ub85c \ubcc0\ud658\ub41c \uadf8\ub798\ud504\uc758 \uacbd\uc6b0 \uc785\ub825\uac12\uc758 \uc0ac\uc774\uc988\ub294 \ubaa8\ub4e0 \ucc28\uc6d0\uc5d0 \ub300\ud574 \uace0\uc815\ub429\ub2c8\ub2e4.\n\uc608\uc2dc\uc5d0\uc11c\ub294 \ubaa8\ub378\uc774 \ud56d\uc0c1 \ubc30\uce58 \uc0ac\uc774\uc988 1\uc744 \uc0ac\uc6a9\ud558\ub3c4\ub85d \ubcc0\ud658\ud558\uc600\uc9c0\ub9cc, \uccab\ubc88\uc9f8 \ucc28\uc6d0\uc744 ``torch.onnx.export()`` \uc758\n``dynamic_axes`` \uc778\uc790\uc5d0 \ub3d9\uc801\uc778 \ucc28\uc6d0\uc73c\ub85c \uc9c0\uc815\ud574\uc8fc\uba74 \ub429\ub2c8\ub2e4.\n\nPyTorch\uc758 \ubcc0\ud658 \uc778\ud130\ud398\uc774\uc2a4\uc5d0 \ub300\ud574 \ub354 \uc790\uc138\ud788 \uc54c\uace0 \uc2f6\ub2e4\uba74\n`torch.onnx \ubb38\uc11c `__ \ub97c \ucc38\uace0\ud574\uc8fc\uc138\uc694.\n\n\n" ] }, { @@ -130,7 +130,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "ONNX \ub7f0\ud0c0\uc784\uc5d0\uc11c \uc774\ubbf8\uc9c0\ub97c \uc785\ub825\uac12\uc73c\ub85c \ubaa8\ub378\uc744 \uc2e4\ud589\ud558\uae30\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n\n" + "ONNX \ub7f0\ud0c0\uc784\uc5d0\uc11c \uc774\ubbf8\uc9c0\ub97c \uc785\ub825\uac12\uc73c\ub85c \ubaa8\ub378\uc744 \uc2e4\ud589\ud558\uae30\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n\n" ] }, { @@ -205,7 +205,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - ".. figure:: /_static/img/cat_superres_with_ort.jpg\n :alt: output\\_cat\n\n\nONNX \ub7f0\ud0c0\uc784\uc740 \ud06c\ub85c\uc2a4\ud50c\ub7ab\ud3fc \uc5d4\uc9c4\uc73c\ub85c\uc11c CPU\uc640 GPU \ubfd0\ub9cc \uc544\ub2c8\ub77c \uc5ec\ub7ec \ud50c\ub7ab\ud3fc\uc5d0\uc11c \uc2e4\ud589\ub420 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\nONNX \ub7f0\ud0c0\uc784\uc740 Azure Machine Learning Services\uc640 \uac19\uc740 \ud074\ub77c\uc6b0\ub4dc\uc5d0 \ubc30\ud3ec\ub418\uc5b4 \ubaa8\ub378 \ucd94\ub860\uc744 \ud558\ub294\ub370\n\uc0ac\uc6a9\ub420 \uc218\ub3c4 \uc788\uc2b5\ub2c8\ub2e4.\n\ub354 \uc790\uc138\ud55c \ub0b4\uc6a9\uc740 `\ub9c1\ud06c `__ \ub97c\n\ucc38\uc870\ud574\uc8fc\uc2ed\uc2dc\uc624.\n\nONNX \ub7f0\ud0c0\uc784\uc758 \uc131\ub2a5\uc5d0 \uad00\ud55c \uac83\uc740 `\uc5ec\uae30 `__ \uc5d0\uc11c \n\ud655\uc778\ud558\uc2e4 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\nONNX \ub7f0\ud0c0\uc784\uc5d0 \uad00\ud55c \ub354 \uc790\uc138\ud55c \ub0b4\uc6a9\uc740 `\uc774 \uacf3 `__ \uc744 \ucc38\uc870\ud574 \uc8fc\uc138\uc694.\n\n" + ".. figure:: /_static/img/cat_superres_with_ort.jpg\n :alt: output\\_cat\n\n\nONNX \ub7f0\ud0c0\uc784\uc740 \ud06c\ub85c\uc2a4\ud50c\ub7ab\ud3fc \uc5d4\uc9c4\uc73c\ub85c\uc11c CPU\uc640 GPU \ubfd0\ub9cc \uc544\ub2c8\ub77c \uc5ec\ub7ec \ud50c\ub7ab\ud3fc\uc5d0\uc11c \uc2e4\ud589\ub420 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\nONNX \ub7f0\ud0c0\uc784\uc740 Azure Machine Learning Services\uc640 \uac19\uc740 \ud074\ub77c\uc6b0\ub4dc\uc5d0 \ubc30\ud3ec\ub418\uc5b4 \ubaa8\ub378 \ucd94\ub860\uc744 \ud558\ub294\ub370\n\uc0ac\uc6a9\ub420 \uc218\ub3c4 \uc788\uc2b5\ub2c8\ub2e4.\n\ub354 \uc790\uc138\ud55c \ub0b4\uc6a9\uc740 `\ub9c1\ud06c `__ \ub97c\n\ucc38\uc870\ud574\uc8fc\uc2ed\uc2dc\uc624.\n\nONNX \ub7f0\ud0c0\uc784\uc758 \uc131\ub2a5\uc5d0 \uad00\ud55c \uac83\uc740 `\uc5ec\uae30 `__ \uc5d0\uc11c\n\ud655\uc778\ud558\uc2e4 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\nONNX \ub7f0\ud0c0\uc784\uc5d0 \uad00\ud55c \ub354 \uc790\uc138\ud55c \ub0b4\uc6a9\uc740 `\uc774 \uacf3 `__ \uc744 \ucc38\uc870\ud574 \uc8fc\uc138\uc694.\n\n" ] }, { @@ -236,7 +236,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/f43ed4eaa18b84dc854a0e4799225d7c/static_quantization_tutorial.ipynb b/docs/_downloads/f43ed4eaa18b84dc854a0e4799225d7c/static_quantization_tutorial.ipynb index f3dd35c49..167c42562 100644 --- a/docs/_downloads/f43ed4eaa18b84dc854a0e4799225d7c/static_quantization_tutorial.ipynb +++ b/docs/_downloads/f43ed4eaa18b84dc854a0e4799225d7c/static_quantization_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n(beta) Static Quantization with Eager Mode in PyTorch\n=========================================================\n\n**Author**: `Raghuraman Krishnamoorthi `_\n\n**Edited by**: `Seth Weidman `_\n\nThis tutorial shows how to do post-training static quantization, as well as illustrating\ntwo more advanced techniques - per-channel quantization and quantization-aware training -\nto further improve the model's accuracy. Note that quantization is currently only supported\nfor CPUs, so we will not be utilizing GPUs / CUDA in this tutorial.\n\nBy the end of this tutorial, you will see how quantization in PyTorch can result in\nsignificant decreases in model size while increasing speed. Furthermore, you'll see how\nto easily apply some advanced quantization techniques shown\n`here `_ so that your quantized models take much less\nof an accuracy hit than they would otherwise.\n\nWarning: we use a lot of boilerplate code from other PyTorch repos to, for example,\ndefine the ``MobileNetV2`` model archtecture, define data loaders, and so on. We of course\nencourage you to read it; but if you want to get to the quantization features, feel free\nto skip to the \"4. Post-training static quantization\" section.\n\nWe'll start by doing the necessary imports:\n\n" + "\n(beta) Static Quantization with Eager Mode in PyTorch\n=========================================================\n\n**Author**: `Raghuraman Krishnamoorthi `_\n\n**Edited by**: `Seth Weidman `_\n\nThis tutorial shows how to do post-training static quantization, as well as illustrating\ntwo more advanced techniques - per-channel quantization and quantization-aware training -\nto further improve the model's accuracy. Note that quantization is currently only supported\nfor CPUs, so we will not be utilizing GPUs / CUDA in this tutorial.\n\nBy the end of this tutorial, you will see how quantization in PyTorch can result in\nsignificant decreases in model size while increasing speed. Furthermore, you'll see how\nto easily apply some advanced quantization techniques shown\n`here `_ so that your quantized models take much less\nof an accuracy hit than they would otherwise.\n\nWarning: we use a lot of boilerplate code from other PyTorch repos to, for example,\ndefine the ``MobileNetV2`` model archtecture, define data loaders, and so on. We of course\nencourage you to read it; but if you want to get to the quantization features, feel free\nto skip to the \"4. Post-training static quantization\" section.\n\nWe'll start by doing the necessary imports:\n" ] }, { @@ -287,7 +287,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/f498e3bcd9b6159ecfb1a07d6551287d/data_loading_tutorial.ipynb b/docs/_downloads/f498e3bcd9b6159ecfb1a07d6551287d/data_loading_tutorial.ipynb index 12e0e87cf..5ac0c71ff 100644 --- a/docs/_downloads/f498e3bcd9b6159ecfb1a07d6551287d/data_loading_tutorial.ipynb +++ b/docs/_downloads/f498e3bcd9b6159ecfb1a07d6551287d/data_loading_tutorial.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n\uc0ac\uc6a9\uc790 \uc815\uc758 Dataset, Dataloader, Transforms \uc791\uc131\ud558\uae30\n==========================================================\n\n**\uc800\uc790** : Sasank Chilamkurthy \n**\ubc88\uc5ed** : \uc815\uc724\uc131 \n\n\uba38\uc2e0\ub7ec\ub2dd \ubb38\uc81c\ub97c \ud478\ub294 \uacfc\uc815\uc5d0\uc11c \ub370\uc774\ud130\ub97c \uc900\ube44\ud558\ub294\ub370 \ub9ce\uc740 \ub178\ub825\uc774 \ud544\uc694\ud569\ub2c8\ub2e4.\nPyTorch\ub294 \ub370\uc774\ud130\ub97c \ubd88\ub7ec\uc624\ub294 \uacfc\uc815\uc744 \uc27d\uac8c\ud574\uc8fc\uace0, \ub610 \uc798 \uc0ac\uc6a9\ud55c\ub2e4\uba74 \ucf54\ub4dc\uc758 \uac00\ub3c5\uc131\ub3c4 \ubcf4\ub2e4 \ub192\uc5ec\uc904 \uc218 \uc788\ub294 \ub3c4\uad6c\ub4e4\uc744\n\uc81c\uacf5\ud569\ub2c8\ub2e4. \uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c \uc77c\ubc18\uc801\uc774\uc9c0 \uc54a\uc740 \ub370\uc774\ud130\uc14b\uc73c\ub85c\ubd80\ud130 \ub370\uc774\ud130\ub97c \uc77d\uc5b4\uc624\uace0\n\uc804\ucc98\ub9ac\ud558\uace0 \uc99d\uac00\ud558\ub294 \ubc29\ubc95\uc744 \uc54c\uc544\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc774\ubc88 \ud29c\ud1a0\ub9ac\uc5bc\uc744 \uc9c4\ud589\ud558\uae30 \uc704\ud574 \uc544\ub798 \ud328\ud0a4\uc9c0\ub4e4\uc744 \uc124\uce58\ud574\uc8fc\uc138\uc694.\n\n- ``scikit-image``: \uc774\ubbf8\uc9c0 I/O \uc640 \ubcc0\ud615\uc744 \uc704\ud574 \ud544\uc694\ud569\ub2c8\ub2e4.\n- ``pandas``: CSV \ud30c\uc77c \ud30c\uc2f1\uc744 \ubcf4\ub2e4 \uc27d\uac8c \ud574\uc90d\ub2c8\ub2e4.\n\n\n\n" + "\n\uc0ac\uc6a9\uc790 \uc815\uc758 Dataset, Dataloader, Transforms \uc791\uc131\ud558\uae30\n==========================================================\n\n**\uc800\uc790** : Sasank Chilamkurthy \n**\ubc88\uc5ed** : \uc815\uc724\uc131 \n\n\uba38\uc2e0\ub7ec\ub2dd \ubb38\uc81c\ub97c \ud478\ub294 \uacfc\uc815\uc5d0\uc11c \ub370\uc774\ud130\ub97c \uc900\ube44\ud558\ub294\ub370 \ub9ce\uc740 \ub178\ub825\uc774 \ud544\uc694\ud569\ub2c8\ub2e4.\nPyTorch\ub294 \ub370\uc774\ud130\ub97c \ubd88\ub7ec\uc624\ub294 \uacfc\uc815\uc744 \uc27d\uac8c\ud574\uc8fc\uace0, \ub610 \uc798 \uc0ac\uc6a9\ud55c\ub2e4\uba74 \ucf54\ub4dc\uc758 \uac00\ub3c5\uc131\ub3c4 \ubcf4\ub2e4 \ub192\uc5ec\uc904 \uc218 \uc788\ub294 \ub3c4\uad6c\ub4e4\uc744\n\uc81c\uacf5\ud569\ub2c8\ub2e4. \uc774 \ud29c\ud1a0\ub9ac\uc5bc\uc5d0\uc11c \uc77c\ubc18\uc801\uc774\uc9c0 \uc54a\uc740 \ub370\uc774\ud130\uc14b\uc73c\ub85c\ubd80\ud130 \ub370\uc774\ud130\ub97c \uc77d\uc5b4\uc624\uace0\n\uc804\ucc98\ub9ac\ud558\uace0 \uc99d\uac00\ud558\ub294 \ubc29\ubc95\uc744 \uc54c\uc544\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc774\ubc88 \ud29c\ud1a0\ub9ac\uc5bc\uc744 \uc9c4\ud589\ud558\uae30 \uc704\ud574 \uc544\ub798 \ud328\ud0a4\uc9c0\ub4e4\uc744 \uc124\uce58\ud574\uc8fc\uc138\uc694.\n\n- ``scikit-image``: \uc774\ubbf8\uc9c0 I/O \uc640 \ubcc0\ud615\uc744 \uc704\ud574 \ud544\uc694\ud569\ub2c8\ub2e4.\n- ``pandas``: CSV \ud30c\uc77c \ud30c\uc2f1\uc744 \ubcf4\ub2e4 \uc27d\uac8c \ud574\uc90d\ub2c8\ub2e4.\n" ] }, { @@ -105,7 +105,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Transforms\n----------\n\n\uc704\uc5d0\uc11c \ubcfc \uc218 \uc788\uc5c8\ub358 \ud55c\uac00\uc9c0 \ubb38\uc81c\uc810\uc740 \uc0d8\ud50c\ub4e4\uc774 \ub2e4 \uac19\uc740 \uc0ac\uc774\uc988\uac00 \uc544\ub2c8\ub77c\ub294 \uac83\uc785\ub2c8\ub2e4.\n\ub300\ubd80\ubd84\uc758 \uc2e0\uacbd\ub9dd(neural networks)\uc740 \uace0\uc815\ub41c \ud06c\uae30\uc758 \uc774\ubbf8\uc9c0\ub77c\uace0 \uac00\uc815\ud569\ub2c8\ub2e4.\n\uadf8\ub7ec\ubbc0\ub85c \uc6b0\ub9ac\ub294 \uc2e0\uacbd\ub9dd\uc5d0 \uc8fc\uae30 \uc804\uc5d0 \ucc98\ub9ac\ud560 \uacfc\uc815\uc744 \uc791\uc131\ud574\uc57c \ud569\ub2c8\ub2e4.\n\n3\uac00\uc9c0\uc758 transforms \uc744 \ub9cc\ub4e4\uc5b4 \ubd05\uc2dc\ub2e4:\n- ``Rescale``: \uc774\ubbf8\uc9c0\uc758 \ud06c\uae30\ub97c \uc870\uc808\ud569\ub2c8\ub2e4.\n- ``RandomCrop``: \uc774\ubbf8\uc9c0\ub97c \ubb34\uc791\uc704\ub85c \uc790\ub985\ub2c8\ub2e4.\n \uc774\uac83\uc744 data augmentation\uc774\ub77c \ud569\ub2c8\ub2e4.\n- ``ToTensor``: numpy \uc774\ubbf8\uc9c0\uc5d0\uc11c torch \uc774\ubbf8\uc9c0\ub85c \ubcc0\uacbd\ud569\ub2c8\ub2e4.\n (\ucd95\ubcc0\ud658\uc774 \ud544\uc694\ud569\ub2c8\ub2e4)\n\n\uac04\ub2e8\ud55c \ud568\uc218\ub300\uc2e0\uc5d0 \ud638\ucd9c \ud560 \uc218 \uc788\ub294 \ud074\ub798\uc2a4\ub85c \uc791\uc131 \ud569\ub2c8\ub2e4.\n\uc774\ub807\uac8c \ud55c\ub2e4\uba74, \ud074\ub798\uc2a4\uac00 \ud638\ucd9c \ub420 \ub54c\ub9c8\ub2e4 \uc804\uc774(Transform)\uc758 \ub9e4\uac1c\ubcc0\uc218\uac00 \uc804\ub2ec \ub418\uc9c0 \uc54a\uc544\ub3c4 \ub429\ub2c8\ub2e4.\n\uc774\uc640 \uac19\uc774, ``__call__`` \ud568\uc218\ub97c \uad6c\ud604\ud574\uc57c \ud569\ub2c8\ub2e4.\n\ud544\uc694\ud558\ub2e4\uba74, ``__init__`` \ud568\uc218\ub3c4 \uad6c\ud604\ud574\uc57c \ud569\ub2c8\ub2e4. \ub2e4\uc74c\uacfc \uac19\uc774 \uc804\uc774(transform)\ub97c \uc0ac\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n::\n\n tsfm = Transform(params)\n transformed_sample = tsfm(sample)\n\n\uc544\ub798\uc5d0\uc11c\ub294 \uc774\ubbf8\uc9c0\uc640 \ub79c\ub4dc\ub9c8\ud06c(landmark)\ub4e4\uc744 \uc5b4\ub5bb\uac8c \uc801\uc6a9\ud558\ub294\uc9c0 \uc0b4\ud3b4\ubcf4\ub3c4\ub85d \ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n" + "Transforms\n----------\n\n\uc704\uc5d0\uc11c \ubcfc \uc218 \uc788\uc5c8\ub358 \ud55c\uac00\uc9c0 \ubb38\uc81c\uc810\uc740 \uc0d8\ud50c\ub4e4\uc774 \ub2e4 \uac19\uc740 \uc0ac\uc774\uc988\uac00 \uc544\ub2c8\ub77c\ub294 \uac83\uc785\ub2c8\ub2e4.\n\ub300\ubd80\ubd84\uc758 \uc2e0\uacbd\ub9dd(neural networks)\uc740 \uace0\uc815\ub41c \ud06c\uae30\uc758 \uc774\ubbf8\uc9c0\ub77c\uace0 \uac00\uc815\ud569\ub2c8\ub2e4.\n\uadf8\ub7ec\ubbc0\ub85c \uc6b0\ub9ac\ub294 \uc2e0\uacbd\ub9dd\uc5d0 \uc8fc\uae30 \uc804\uc5d0 \ucc98\ub9ac\ud560 \uacfc\uc815\uc744 \uc791\uc131\ud574\uc57c \ud569\ub2c8\ub2e4.\n\n3\uac00\uc9c0\uc758 transforms \uc744 \ub9cc\ub4e4\uc5b4 \ubd05\uc2dc\ub2e4:\n\n- ``Rescale``: \uc774\ubbf8\uc9c0\uc758 \ud06c\uae30\ub97c \uc870\uc808\ud569\ub2c8\ub2e4.\n- ``RandomCrop``: \uc774\ubbf8\uc9c0\ub97c \ubb34\uc791\uc704\ub85c \uc790\ub985\ub2c8\ub2e4.\n \uc774\uac83\uc744 data augmentation\uc774\ub77c \ud569\ub2c8\ub2e4.\n- ``ToTensor``: numpy \uc774\ubbf8\uc9c0\uc5d0\uc11c torch \uc774\ubbf8\uc9c0\ub85c \ubcc0\uacbd\ud569\ub2c8\ub2e4.\n (\ucd95\ubcc0\ud658\uc774 \ud544\uc694\ud569\ub2c8\ub2e4)\n\n\uac04\ub2e8\ud55c \ud568\uc218\ub300\uc2e0\uc5d0 \ud638\ucd9c \ud560 \uc218 \uc788\ub294 \ud074\ub798\uc2a4\ub85c \uc791\uc131 \ud569\ub2c8\ub2e4.\n\uc774\ub807\uac8c \ud55c\ub2e4\uba74, \ud074\ub798\uc2a4\uac00 \ud638\ucd9c \ub420 \ub54c\ub9c8\ub2e4 \uc804\uc774(Transform)\uc758 \ub9e4\uac1c\ubcc0\uc218\uac00 \uc804\ub2ec \ub418\uc9c0 \uc54a\uc544\ub3c4 \ub429\ub2c8\ub2e4.\n\uc774\uc640 \uac19\uc774, ``__call__`` \ud568\uc218\ub97c \uad6c\ud604\ud574\uc57c \ud569\ub2c8\ub2e4.\n\ud544\uc694\ud558\ub2e4\uba74, ``__init__`` \ud568\uc218\ub3c4 \uad6c\ud604\ud574\uc57c \ud569\ub2c8\ub2e4. \ub2e4\uc74c\uacfc \uac19\uc774 \uc804\uc774(transform)\ub97c \uc0ac\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n::\n\n tsfm = Transform(params)\n transformed_sample = tsfm(sample)\n\n\uc544\ub798\uc5d0\uc11c\ub294 \uc774\ubbf8\uc9c0\uc640 \ub79c\ub4dc\ub9c8\ud06c(landmark)\ub4e4\uc744 \uc5b4\ub5bb\uac8c \uc801\uc6a9\ud558\ub294\uc9c0 \uc0b4\ud3b4\ubcf4\ub3c4\ub85d \ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n" ] }, { @@ -197,7 +197,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/f720861441352889ad66f9e2b85807f7/two_layer_net_custom_function.ipynb b/docs/_downloads/f720861441352889ad66f9e2b85807f7/two_layer_net_custom_function.ipynb index d4f9990ba..76201e5c0 100644 --- a/docs/_downloads/f720861441352889ad66f9e2b85807f7/two_layer_net_custom_function.ipynb +++ b/docs/_downloads/f720861441352889ad66f9e2b85807f7/two_layer_net_custom_function.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nPyTorch: \uc0c8 autograd Function \uc815\uc758\ud558\uae30\n----------------------------------------\n\n\ud558\ub098\uc758 \uc740\ub2c9\uce35(hidden layer)\uacfc \ud3b8\ud5a5(bias)\uc774 \uc5c6\ub294 \uc644\uc804\ud788 \uc5f0\uacb0\ub41c ReLU \uc2e0\uacbd\ub9dd\uc744,\n\uc720\ud074\ub9ac\ub4dc \uac70\ub9ac(Euclidean distance) \uc81c\uacf1\uc744 \ucd5c\uc18c\ud654\ud558\ub294 \uc2dd\uc73c\ub85c x\ub85c\ubd80\ud130 y\ub97c \uc608\uce21\ud558\ub3c4\ub85d\n\ud559\uc2b5\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\nPyTorch Variable \uc5f0\uc0b0\uc744 \uc0ac\uc6a9\ud558\uc5ec \uc21c\uc804\ud30c\ub97c \uacc4\uc0b0\ud558\uace0, PyTorch autograd\ub97c \uc0ac\uc6a9\ud558\uc5ec\n\ubcc0\ud654\ub3c4(gradient)\ub97c \uacc4\uc0b0\ud558\ub294 \uac83\uc744 \uad6c\ud604\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc5ec\uae30\uc11c\ub294 \uc0ac\uc6a9\uc790 \uc815\uc758 autograd \ud568\uc218\ub97c \uad6c\ud604\ud558\uc5ec ReLU \uae30\ub2a5\uc744 \uc218\ud589\ud558\ub3c4\ub85d \ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n" + "\nPyTorch: \uc0c8 autograd Function \uc815\uc758\ud558\uae30\n----------------------------------------\n\n\ud558\ub098\uc758 \uc740\ub2c9\uce35(hidden layer)\uacfc \ud3b8\ud5a5(bias)\uc774 \uc5c6\ub294 \uc644\uc804\ud788 \uc5f0\uacb0\ub41c ReLU \uc2e0\uacbd\ub9dd\uc744,\n\uc720\ud074\ub9ac\ub4dc \uac70\ub9ac(Euclidean distance) \uc81c\uacf1\uc744 \ucd5c\uc18c\ud654\ud558\ub294 \uc2dd\uc73c\ub85c x\ub85c\ubd80\ud130 y\ub97c \uc608\uce21\ud558\ub3c4\ub85d\n\ud559\uc2b5\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\nPyTorch Variable \uc5f0\uc0b0\uc744 \uc0ac\uc6a9\ud558\uc5ec \uc21c\uc804\ud30c\ub97c \uacc4\uc0b0\ud558\uace0, PyTorch autograd\ub97c \uc0ac\uc6a9\ud558\uc5ec\n\ubcc0\ud654\ub3c4(gradient)\ub97c \uacc4\uc0b0\ud558\ub294 \uac83\uc744 \uad6c\ud604\ud558\uaca0\uc2b5\ub2c8\ub2e4.\n\n\uc5ec\uae30\uc11c\ub294 \uc0ac\uc6a9\uc790 \uc815\uc758 autograd \ud568\uc218\ub97c \uad6c\ud604\ud558\uc5ec ReLU \uae30\ub2a5\uc744 \uc218\ud589\ud558\ub3c4\ub85d \ud558\uaca0\uc2b5\ub2c8\ub2e4.\n" ] }, { @@ -46,7 +46,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/f84ef04333132b77a0663044e78e7cbb/dynamic_quantization.ipynb b/docs/_downloads/f84ef04333132b77a0663044e78e7cbb/dynamic_quantization.ipynb index f99f5f2f1..18ae5c908 100644 --- a/docs/_downloads/f84ef04333132b77a0663044e78e7cbb/dynamic_quantization.ipynb +++ b/docs/_downloads/f84ef04333132b77a0663044e78e7cbb/dynamic_quantization.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nDynamic Quantization\n====================\n\nIn this recipe you will see how to take advantage of Dynamic\nQuantization to accelerate inference on an LSTM-style recurrent neural\nnetwork. This reduces the size of the model weights and speeds up model\nexecution.\n\nIntroduction\n-------------\n\nThere are a number of trade-offs that can be made when designing neural\nnetworks. During model developmenet and training you can alter the\nnumber of layers and number of parameters in a recurrent neural network\nand trade-off accuracy against model size and/or model latency or\nthroughput. Such changes can take lot of time and compute resources\nbecause you are iterating over the model training. Quantization gives\nyou a way to make a similar trade off between performance and model\naccuracy with a known model after training is completed.\n\nYou can give it a try in a single session and you will certainly reduce\nyour model size significantly and may get a significant latency\nreduction without losing a lot of accuracy.\n\nWhat is dynamic quantization?\n-------------\n\nQuantizing a network means converting it to use a reduced precision\ninteger representation for the weights and/or activations. This saves on\nmodel size and allows the use of higher throughput math operations on\nyour CPU or GPU.\n\nWhen converting from floating point to integer values you are\nessentially multiplying the floating point value by some scale factor\nand rounding the result to a whole number. The various quantization\napproaches differ in the way they approach determining that scale\nfactor.\n\nThe key idea with dynamic quantization as described here is that we are\ngoing to determine the scale factor for activations dynamically based on\nthe data range observed at runtime. This ensures that the scale factor\nis \"tuned\" so that as much signal as possible about each observed\ndataset is preserved.\n\nThe model parameters on the other hand are known during model conversion\nand they are converted ahead of time and stored in INT8 form.\n\nArithmetic in the quantized model is done using vectorized INT8\ninstructions. Accumulation is typically done with INT16 or INT32 to\navoid overflow. This higher precision value is scaled back to INT8 if\nthe next layer is quantized or converted to FP32 for output.\n\nDynamic quantization is relatively free of tuning parameters which makes\nit well suited to be added into production pipelines as a standard part\nof converting LSTM models to deployment.\n\n\n\n

Note

Limitations on the approach taken here\n\n\n This recipe provides a quick introduction to the dynamic quantization\n features in PyTorch and the workflow for using it. Our focus is on\n explaining the specific functions used to convert the model. We will\n make a number of significant simplifications in the interest of brevity\n and clarity

\n\n\n1. You will start with a minimal LSTM network\n2. You are simply going to initialize the network with a random hidden\n state\n3. You are going to test the network with random inputs\n4. You are not going to train the network in this tutorial\n5. You will see that the quantized form of this network is smaller and\n runs faster than the floating point network we started with\n6. You will see that the output values are generally in the same\n ballpark as the output of the FP32 network, but we are not\n demonstrating here the expected accuracy loss on a real trained\n network\n\nYou will see how dynamic quantization is done and be able to see\nsuggestive reductions in memory use and latency times. Providing a\ndemonstration that the technique can preserve high levels of model\naccuracy on a trained LSTM is left to a more advanced tutorial. If you\nwant to move right away to that more rigorous treatment please proceed\nto the `advanced dynamic quantization\ntutorial `__.\n\nSteps\n-------------\n\nThis recipe has 5 steps.\n\n1. Set Up - Here you define a very simple LSTM, import modules, and establish\n some random input tensors.\n\n2. Do the Quantization - Here you instantiate a floating point model and then create quantized\n version of it.\n\n3. Look at Model Size - Here you show that the model size gets smaller.\n\n4. Look at Latency - Here you run the two models and compare model runtime (latency).\n\n5. Look at Accuracy - Here you run the two models and compare outputs.\n\n\n1: Set Up\n~~~~~~~~~~~~~~~\nThis is a straightfoward bit of code to set up for the rest of the\nrecipe.\n\nThe unique module we are importing here is torch.quantization which\nincludes PyTorch's quantized operators and conversion functions. We also\ndefine a very simple LSTM model and set up some inputs.\n\n\n" + "\nDynamic Quantization\n====================\n\nIn this recipe you will see how to take advantage of Dynamic\nQuantization to accelerate inference on an LSTM-style recurrent neural\nnetwork. This reduces the size of the model weights and speeds up model\nexecution.\n\nIntroduction\n-------------\n\nThere are a number of trade-offs that can be made when designing neural\nnetworks. During model developmenet and training you can alter the\nnumber of layers and number of parameters in a recurrent neural network\nand trade-off accuracy against model size and/or model latency or\nthroughput. Such changes can take lot of time and compute resources\nbecause you are iterating over the model training. Quantization gives\nyou a way to make a similar trade off between performance and model\naccuracy with a known model after training is completed.\n\nYou can give it a try in a single session and you will certainly reduce\nyour model size significantly and may get a significant latency\nreduction without losing a lot of accuracy.\n\nWhat is dynamic quantization?\n-------------\n\nQuantizing a network means converting it to use a reduced precision\ninteger representation for the weights and/or activations. This saves on\nmodel size and allows the use of higher throughput math operations on\nyour CPU or GPU.\n\nWhen converting from floating point to integer values you are\nessentially multiplying the floating point value by some scale factor\nand rounding the result to a whole number. The various quantization\napproaches differ in the way they approach determining that scale\nfactor.\n\nThe key idea with dynamic quantization as described here is that we are\ngoing to determine the scale factor for activations dynamically based on\nthe data range observed at runtime. This ensures that the scale factor\nis \"tuned\" so that as much signal as possible about each observed\ndataset is preserved.\n\nThe model parameters on the other hand are known during model conversion\nand they are converted ahead of time and stored in INT8 form.\n\nArithmetic in the quantized model is done using vectorized INT8\ninstructions. Accumulation is typically done with INT16 or INT32 to\navoid overflow. This higher precision value is scaled back to INT8 if\nthe next layer is quantized or converted to FP32 for output.\n\nDynamic quantization is relatively free of tuning parameters which makes\nit well suited to be added into production pipelines as a standard part\nof converting LSTM models to deployment.\n\n\n\n

Note

Limitations on the approach taken here\n\n\n This recipe provides a quick introduction to the dynamic quantization\n features in PyTorch and the workflow for using it. Our focus is on\n explaining the specific functions used to convert the model. We will\n make a number of significant simplifications in the interest of brevity\n and clarity

\n\n\n1. You will start with a minimal LSTM network\n2. You are simply going to initialize the network with a random hidden\n state\n3. You are going to test the network with random inputs\n4. You are not going to train the network in this tutorial\n5. You will see that the quantized form of this network is smaller and\n runs faster than the floating point network we started with\n6. You will see that the output values are generally in the same\n ballpark as the output of the FP32 network, but we are not\n demonstrating here the expected accuracy loss on a real trained\n network\n\nYou will see how dynamic quantization is done and be able to see\nsuggestive reductions in memory use and latency times. Providing a\ndemonstration that the technique can preserve high levels of model\naccuracy on a trained LSTM is left to a more advanced tutorial. If you\nwant to move right away to that more rigorous treatment please proceed\nto the `advanced dynamic quantization\ntutorial `__.\n\nSteps\n-------------\n\nThis recipe has 5 steps.\n\n1. Set Up - Here you define a very simple LSTM, import modules, and establish\n some random input tensors.\n\n2. Do the Quantization - Here you instantiate a floating point model and then create quantized\n version of it.\n\n3. Look at Model Size - Here you show that the model size gets smaller.\n\n4. Look at Latency - Here you run the two models and compare model runtime (latency).\n\n5. Look at Accuracy - Here you run the two models and compare outputs.\n\n\n1: Set Up\n~~~~~~~~~~~~~~~\nThis is a straightfoward bit of code to set up for the rest of the\nrecipe.\n\nThe unique module we are importing here is torch.quantization which\nincludes PyTorch's quantized operators and conversion functions. We also\ndefine a very simple LSTM model and set up some inputs.\n" ] }, { @@ -125,7 +125,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_downloads/f8cb9bf90ebfb30e3e9f9cedbd7ca1d2/defining_a_neural_network.ipynb b/docs/_downloads/f8cb9bf90ebfb30e3e9f9cedbd7ca1d2/defining_a_neural_network.ipynb index 7251e590b..2efae4eb8 100644 --- a/docs/_downloads/f8cb9bf90ebfb30e3e9f9cedbd7ca1d2/defining_a_neural_network.ipynb +++ b/docs/_downloads/f8cb9bf90ebfb30e3e9f9cedbd7ca1d2/defining_a_neural_network.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\nPytorch\ub97c \uc0ac\uc6a9\ud574 \uc2e0\uacbd\ub9dd \uc815\uc758\ud558\uae30\n====================================\n\ub525\ub7ec\ub2dd\uc740 \uc778\uacf5\uc2e0\uacbd\ub9dd(models)\uc744 \uc0ac\uc6a9\ud558\uba70 \uc774\uac83\uc740 \uc0c1\ud638\uc5f0\uacb0\ub41c \uc9d1\ub2e8\uc758 \ub9ce\uc740 \uacc4\uce35\uc73c\ub85c \uad6c\uc131\ub41c \uacc4\uc0b0 \uc2dc\uc2a4\ud15c\uc785\ub2c8\ub2e4. \n\ub370\uc774\ud130\uac00 \uc774 \uc0c1\ud638\uc5f0\uacb0\ub41c \uc9d1\ub2e8\uc744 \ud1b5\uacfc\ud558\uba74\uc11c, \uc2e0\uacbd\ub9dd\uc740 \uc785\ub825\uc744 \ucd9c\ub825\uc73c\ub85c \ubc14\uafb8\uae30 \uc704\ud574 \uc694\uad6c\ub41c \uacc4\uc0b0 \ubc29\ubc95\uc5d0 \uc5b4\ub5bb\uac8c \uadfc\uc811\ud558\ub294 \uc9c0\ub97c \ubc30\uc6b8 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \nPyTorch\uc5d0\uc11c, \uc2e0\uacbd\ub9dd\uc740 ``torch.nn`` \ud328\ud0a4\uc9c0\ub97c \uc0ac\uc6a9\ud574 \uad6c\uc131\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \n\n\uc18c\uac1c\n-----\nPyTorch\ub294 ``torch.nn`` \uc744 \ud3ec\ud568\ud558\uc5ec \uc2e0\uacbd\ub9dd\uc744 \ub9cc\ub4e4\uace0 \ud6c8\ub828\uc2dc\ud0a4\ub294 \uac83\uc744 \ub3c4\uc6b8 \uc218 \uc788\ub3c4\ub85d \uc12c\uc138\ud558\uac8c \ub9cc\ub4e4\uc5b4\uc9c4 \ubaa8\ub4c8\uacfc \ud074\ub798\uc2a4\ub4e4\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4. \n``nn.Moduel`` \uc740 \uacc4\uce35, \uadf8\ub9ac\uace0 ``output`` \uc744 \ubc18\ud658\ud558\ub294 ``forward(input)`` \uba54\uc18c\ub4dc\ub97c \ud3ec\ud568\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4. \n\n\uc774 \ub808\uc2dc\ud53c\uc5d0\uc11c, `MNIST dataset `__ \uc744 \uc0ac\uc6a9\ud558\uc5ec \uc2e0\uacbd\ub9dd\uc744 \uc815\uc758\ud558\uae30 \uc704\ud574 ``torch.nn`` \uc744 \uc0ac\uc6a9\ud560 \uc608\uc815\uc785\ub2c8\ub2e4.\n\n\uc124\uce58\n-----\n\uc2dc\uc791\ud558\uae30 \uc804\uc5d0, \uc900\ube44\uac00 \ub418\uc5b4\uc788\uc9c0 \uc54a\ub2e4\uba74 ``torch`` \ub97c \uc124\uce58\ud574\uc57c \ud569\ub2c8\ub2e4. \n\n::\n\n pip install torchaudio\n\n\n\n" + "\nPytorch\ub97c \uc0ac\uc6a9\ud574 \uc2e0\uacbd\ub9dd \uc815\uc758\ud558\uae30\n====================================\n\ub525\ub7ec\ub2dd\uc740 \uc778\uacf5\uc2e0\uacbd\ub9dd(models)\uc744 \uc0ac\uc6a9\ud558\uba70 \uc774\uac83\uc740 \uc0c1\ud638\uc5f0\uacb0\ub41c \uc9d1\ub2e8\uc758 \ub9ce\uc740 \uacc4\uce35\uc73c\ub85c \uad6c\uc131\ub41c \uacc4\uc0b0 \uc2dc\uc2a4\ud15c\uc785\ub2c8\ub2e4. \n\ub370\uc774\ud130\uac00 \uc774 \uc0c1\ud638\uc5f0\uacb0\ub41c \uc9d1\ub2e8\uc744 \ud1b5\uacfc\ud558\uba74\uc11c, \uc2e0\uacbd\ub9dd\uc740 \uc785\ub825\uc744 \ucd9c\ub825\uc73c\ub85c \ubc14\uafb8\uae30 \uc704\ud574 \uc694\uad6c\ub41c \uacc4\uc0b0 \ubc29\ubc95\uc5d0 \uc5b4\ub5bb\uac8c \uadfc\uc811\ud558\ub294 \uc9c0\ub97c \ubc30\uc6b8 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \nPyTorch\uc5d0\uc11c, \uc2e0\uacbd\ub9dd\uc740 ``torch.nn`` \ud328\ud0a4\uc9c0\ub97c \uc0ac\uc6a9\ud574 \uad6c\uc131\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \n\n\uc18c\uac1c\n-----\nPyTorch\ub294 ``torch.nn`` \uc744 \ud3ec\ud568\ud558\uc5ec \uc2e0\uacbd\ub9dd\uc744 \ub9cc\ub4e4\uace0 \ud6c8\ub828\uc2dc\ud0a4\ub294 \uac83\uc744 \ub3c4\uc6b8 \uc218 \uc788\ub3c4\ub85d \uc12c\uc138\ud558\uac8c \ub9cc\ub4e4\uc5b4\uc9c4 \ubaa8\ub4c8\uacfc \ud074\ub798\uc2a4\ub4e4\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4. \n``nn.Moduel`` \uc740 \uacc4\uce35, \uadf8\ub9ac\uace0 ``output`` \uc744 \ubc18\ud658\ud558\ub294 ``forward(input)`` \uba54\uc18c\ub4dc\ub97c \ud3ec\ud568\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4. \n\n\uc774 \ub808\uc2dc\ud53c\uc5d0\uc11c, `MNIST dataset `__ \uc744 \uc0ac\uc6a9\ud558\uc5ec \uc2e0\uacbd\ub9dd\uc744 \uc815\uc758\ud558\uae30 \uc704\ud574 ``torch.nn`` \uc744 \uc0ac\uc6a9\ud560 \uc608\uc815\uc785\ub2c8\ub2e4.\n\n\uc124\uce58\n-----\n\uc2dc\uc791\ud558\uae30 \uc804\uc5d0, \uc900\ube44\uac00 \ub418\uc5b4\uc788\uc9c0 \uc54a\ub2e4\uba74 ``torch`` \ub97c \uc124\uce58\ud574\uc57c \ud569\ub2c8\ub2e4. \n\n::\n\n pip install torchaudio\n" ] }, { @@ -114,7 +114,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.7" + "version": "3.7.9" } }, "nbformat": 4, diff --git a/docs/_images/sphx_glr_audio_preprocessing_tutorial_001.png b/docs/_images/sphx_glr_audio_preprocessing_tutorial_001.png new file mode 100644 index 000000000..ea278c085 Binary files /dev/null and b/docs/_images/sphx_glr_audio_preprocessing_tutorial_001.png differ diff --git a/docs/_images/sphx_glr_audio_preprocessing_tutorial_002.png b/docs/_images/sphx_glr_audio_preprocessing_tutorial_002.png new file mode 100644 index 000000000..da78bd6c9 Binary files /dev/null and b/docs/_images/sphx_glr_audio_preprocessing_tutorial_002.png differ diff --git a/docs/_images/sphx_glr_audio_preprocessing_tutorial_003.png b/docs/_images/sphx_glr_audio_preprocessing_tutorial_003.png new file mode 100644 index 000000000..b4049e542 Binary files /dev/null and b/docs/_images/sphx_glr_audio_preprocessing_tutorial_003.png differ diff --git a/docs/_images/sphx_glr_audio_preprocessing_tutorial_004.png b/docs/_images/sphx_glr_audio_preprocessing_tutorial_004.png new file mode 100644 index 000000000..268927fae Binary files /dev/null and b/docs/_images/sphx_glr_audio_preprocessing_tutorial_004.png differ diff --git a/docs/_images/sphx_glr_audio_preprocessing_tutorial_005.png b/docs/_images/sphx_glr_audio_preprocessing_tutorial_005.png new file mode 100644 index 000000000..4e60eb5a7 Binary files /dev/null and b/docs/_images/sphx_glr_audio_preprocessing_tutorial_005.png differ diff --git a/docs/_images/sphx_glr_audio_preprocessing_tutorial_006.png b/docs/_images/sphx_glr_audio_preprocessing_tutorial_006.png new file mode 100644 index 000000000..dccb5ade4 Binary files /dev/null and b/docs/_images/sphx_glr_audio_preprocessing_tutorial_006.png differ diff --git a/docs/_images/sphx_glr_audio_preprocessing_tutorial_007.png b/docs/_images/sphx_glr_audio_preprocessing_tutorial_007.png new file mode 100644 index 000000000..4e60eb5a7 Binary files /dev/null and b/docs/_images/sphx_glr_audio_preprocessing_tutorial_007.png differ diff --git a/docs/_images/sphx_glr_audio_preprocessing_tutorial_008.png b/docs/_images/sphx_glr_audio_preprocessing_tutorial_008.png new file mode 100644 index 000000000..5fbfcaf98 Binary files /dev/null and b/docs/_images/sphx_glr_audio_preprocessing_tutorial_008.png differ diff --git a/docs/_images/sphx_glr_audio_preprocessing_tutorial_009.png b/docs/_images/sphx_glr_audio_preprocessing_tutorial_009.png new file mode 100644 index 000000000..412262d44 Binary files /dev/null and b/docs/_images/sphx_glr_audio_preprocessing_tutorial_009.png differ diff --git a/docs/_images/sphx_glr_audio_preprocessing_tutorial_010.png b/docs/_images/sphx_glr_audio_preprocessing_tutorial_010.png new file mode 100644 index 000000000..cebfbe68f Binary files /dev/null and b/docs/_images/sphx_glr_audio_preprocessing_tutorial_010.png differ diff --git a/docs/_images/sphx_glr_audio_preprocessing_tutorial_011.png b/docs/_images/sphx_glr_audio_preprocessing_tutorial_011.png new file mode 100644 index 000000000..91c39becf Binary files /dev/null and b/docs/_images/sphx_glr_audio_preprocessing_tutorial_011.png differ diff --git a/docs/_images/sphx_glr_audio_preprocessing_tutorial_012.png b/docs/_images/sphx_glr_audio_preprocessing_tutorial_012.png new file mode 100644 index 000000000..d36112a89 Binary files /dev/null and b/docs/_images/sphx_glr_audio_preprocessing_tutorial_012.png differ diff --git a/docs/_images/sphx_glr_audio_preprocessing_tutorial_013.png b/docs/_images/sphx_glr_audio_preprocessing_tutorial_013.png new file mode 100644 index 000000000..48654c929 Binary files /dev/null and b/docs/_images/sphx_glr_audio_preprocessing_tutorial_013.png differ diff --git a/docs/_images/sphx_glr_audio_preprocessing_tutorial_014.png b/docs/_images/sphx_glr_audio_preprocessing_tutorial_014.png new file mode 100644 index 000000000..de4bced1e Binary files /dev/null and b/docs/_images/sphx_glr_audio_preprocessing_tutorial_014.png differ diff --git a/docs/_images/sphx_glr_char_rnn_classification_tutorial_001.png b/docs/_images/sphx_glr_char_rnn_classification_tutorial_001.png index bc7e8386d..766e5132e 100644 Binary files a/docs/_images/sphx_glr_char_rnn_classification_tutorial_001.png and b/docs/_images/sphx_glr_char_rnn_classification_tutorial_001.png differ diff --git a/docs/_images/sphx_glr_char_rnn_classification_tutorial_002.png b/docs/_images/sphx_glr_char_rnn_classification_tutorial_002.png index 4cf2292bf..dbd52f1de 100644 Binary files a/docs/_images/sphx_glr_char_rnn_classification_tutorial_002.png and b/docs/_images/sphx_glr_char_rnn_classification_tutorial_002.png differ diff --git a/docs/_images/sphx_glr_char_rnn_generation_tutorial_001.png b/docs/_images/sphx_glr_char_rnn_generation_tutorial_001.png index 8513bfdad..f32ea9b52 100644 Binary files a/docs/_images/sphx_glr_char_rnn_generation_tutorial_001.png and b/docs/_images/sphx_glr_char_rnn_generation_tutorial_001.png differ diff --git a/docs/_images/sphx_glr_cifar10_tutorial_001.png b/docs/_images/sphx_glr_cifar10_tutorial_001.png index a22c980fd..5769ed6a5 100644 Binary files a/docs/_images/sphx_glr_cifar10_tutorial_001.png and b/docs/_images/sphx_glr_cifar10_tutorial_001.png differ diff --git a/docs/_images/sphx_glr_cifar10_tutorial_002.png b/docs/_images/sphx_glr_cifar10_tutorial_002.png index 1c56b7109..8b6a4ee4a 100644 Binary files a/docs/_images/sphx_glr_cifar10_tutorial_002.png and b/docs/_images/sphx_glr_cifar10_tutorial_002.png differ diff --git a/docs/_images/sphx_glr_data_loading_tutorial_001.png b/docs/_images/sphx_glr_data_loading_tutorial_001.png index 65dcb4fa1..45cce021e 100644 Binary files a/docs/_images/sphx_glr_data_loading_tutorial_001.png and b/docs/_images/sphx_glr_data_loading_tutorial_001.png differ diff --git a/docs/_images/sphx_glr_data_loading_tutorial_002.png b/docs/_images/sphx_glr_data_loading_tutorial_002.png index 0c3e7e956..e18fb42f1 100644 Binary files a/docs/_images/sphx_glr_data_loading_tutorial_002.png and b/docs/_images/sphx_glr_data_loading_tutorial_002.png differ diff --git a/docs/_images/sphx_glr_data_loading_tutorial_003.png b/docs/_images/sphx_glr_data_loading_tutorial_003.png index 33d1db846..a4589ae0c 100644 Binary files a/docs/_images/sphx_glr_data_loading_tutorial_003.png and b/docs/_images/sphx_glr_data_loading_tutorial_003.png differ diff --git a/docs/_images/sphx_glr_data_loading_tutorial_004.png b/docs/_images/sphx_glr_data_loading_tutorial_004.png index 3b9cb84a3..233440793 100644 Binary files a/docs/_images/sphx_glr_data_loading_tutorial_004.png and b/docs/_images/sphx_glr_data_loading_tutorial_004.png differ diff --git a/docs/_images/sphx_glr_dcgan_faces_tutorial_001.png b/docs/_images/sphx_glr_dcgan_faces_tutorial_001.png index c5ae53845..9773a343b 100644 Binary files a/docs/_images/sphx_glr_dcgan_faces_tutorial_001.png and b/docs/_images/sphx_glr_dcgan_faces_tutorial_001.png differ diff --git a/docs/_images/sphx_glr_dcgan_faces_tutorial_002.png b/docs/_images/sphx_glr_dcgan_faces_tutorial_002.png index c71c50200..d4aca6303 100644 Binary files a/docs/_images/sphx_glr_dcgan_faces_tutorial_002.png and b/docs/_images/sphx_glr_dcgan_faces_tutorial_002.png differ diff --git a/docs/_images/sphx_glr_dcgan_faces_tutorial_003.png b/docs/_images/sphx_glr_dcgan_faces_tutorial_003.png index 3805bcc58..a90c2ba22 100644 Binary files a/docs/_images/sphx_glr_dcgan_faces_tutorial_003.png and b/docs/_images/sphx_glr_dcgan_faces_tutorial_003.png differ diff --git a/docs/_images/sphx_glr_dcgan_faces_tutorial_004.png b/docs/_images/sphx_glr_dcgan_faces_tutorial_004.png index ed485d14b..145be596c 100644 Binary files a/docs/_images/sphx_glr_dcgan_faces_tutorial_004.png and b/docs/_images/sphx_glr_dcgan_faces_tutorial_004.png differ diff --git a/docs/_images/sphx_glr_fgsm_tutorial_001.png b/docs/_images/sphx_glr_fgsm_tutorial_001.png index 6808fcd27..b430014b8 100644 Binary files a/docs/_images/sphx_glr_fgsm_tutorial_001.png and b/docs/_images/sphx_glr_fgsm_tutorial_001.png differ diff --git a/docs/_images/sphx_glr_fgsm_tutorial_002.png b/docs/_images/sphx_glr_fgsm_tutorial_002.png index fc37a54f4..ba41d3f93 100644 Binary files a/docs/_images/sphx_glr_fgsm_tutorial_002.png and b/docs/_images/sphx_glr_fgsm_tutorial_002.png differ diff --git a/docs/_images/sphx_glr_neural_style_tutorial_001.png b/docs/_images/sphx_glr_neural_style_tutorial_001.png index 2320186f7..0dd2399cc 100644 Binary files a/docs/_images/sphx_glr_neural_style_tutorial_001.png and b/docs/_images/sphx_glr_neural_style_tutorial_001.png differ diff --git a/docs/_images/sphx_glr_neural_style_tutorial_002.png b/docs/_images/sphx_glr_neural_style_tutorial_002.png index ffb199138..34a962643 100644 Binary files a/docs/_images/sphx_glr_neural_style_tutorial_002.png and b/docs/_images/sphx_glr_neural_style_tutorial_002.png differ diff --git a/docs/_images/sphx_glr_neural_style_tutorial_003.png b/docs/_images/sphx_glr_neural_style_tutorial_003.png index 3f93478d3..c2a10ac9d 100644 Binary files a/docs/_images/sphx_glr_neural_style_tutorial_003.png and b/docs/_images/sphx_glr_neural_style_tutorial_003.png differ diff --git a/docs/_images/sphx_glr_neural_style_tutorial_004.png b/docs/_images/sphx_glr_neural_style_tutorial_004.png index 25ac31c98..8811532df 100644 Binary files a/docs/_images/sphx_glr_neural_style_tutorial_004.png and b/docs/_images/sphx_glr_neural_style_tutorial_004.png differ diff --git a/docs/_images/sphx_glr_nn_tutorial_001.png b/docs/_images/sphx_glr_nn_tutorial_001.png index 2149f7d2c..eee54887b 100644 Binary files a/docs/_images/sphx_glr_nn_tutorial_001.png and b/docs/_images/sphx_glr_nn_tutorial_001.png differ diff --git a/docs/_images/sphx_glr_seq2seq_translation_tutorial_001.png b/docs/_images/sphx_glr_seq2seq_translation_tutorial_001.png index f9294f16e..4c424c13d 100644 Binary files a/docs/_images/sphx_glr_seq2seq_translation_tutorial_001.png and b/docs/_images/sphx_glr_seq2seq_translation_tutorial_001.png differ diff --git a/docs/_images/sphx_glr_seq2seq_translation_tutorial_002.png b/docs/_images/sphx_glr_seq2seq_translation_tutorial_002.png index 630720f47..93422262e 100644 Binary files a/docs/_images/sphx_glr_seq2seq_translation_tutorial_002.png and b/docs/_images/sphx_glr_seq2seq_translation_tutorial_002.png differ diff --git a/docs/_images/sphx_glr_seq2seq_translation_tutorial_003.png b/docs/_images/sphx_glr_seq2seq_translation_tutorial_003.png index 97736e1ef..2ee3e96d4 100644 Binary files a/docs/_images/sphx_glr_seq2seq_translation_tutorial_003.png and b/docs/_images/sphx_glr_seq2seq_translation_tutorial_003.png differ diff --git a/docs/_images/sphx_glr_seq2seq_translation_tutorial_004.png b/docs/_images/sphx_glr_seq2seq_translation_tutorial_004.png index b0c0324d6..712db0f67 100644 Binary files a/docs/_images/sphx_glr_seq2seq_translation_tutorial_004.png and b/docs/_images/sphx_glr_seq2seq_translation_tutorial_004.png differ diff --git a/docs/_images/sphx_glr_seq2seq_translation_tutorial_005.png b/docs/_images/sphx_glr_seq2seq_translation_tutorial_005.png index af0b01b18..9dae04711 100644 Binary files a/docs/_images/sphx_glr_seq2seq_translation_tutorial_005.png and b/docs/_images/sphx_glr_seq2seq_translation_tutorial_005.png differ diff --git a/docs/_images/sphx_glr_seq2seq_translation_tutorial_006.png b/docs/_images/sphx_glr_seq2seq_translation_tutorial_006.png index 29e81a7f3..15d941fa6 100644 Binary files a/docs/_images/sphx_glr_seq2seq_translation_tutorial_006.png and b/docs/_images/sphx_glr_seq2seq_translation_tutorial_006.png differ diff --git a/docs/_images/sphx_glr_seq2seq_translation_tutorial_007.png b/docs/_images/sphx_glr_seq2seq_translation_tutorial_007.png index ffee91250..0106e179a 100644 Binary files a/docs/_images/sphx_glr_seq2seq_translation_tutorial_007.png and b/docs/_images/sphx_glr_seq2seq_translation_tutorial_007.png differ diff --git a/docs/_images/sphx_glr_spatial_transformer_tutorial_001.png b/docs/_images/sphx_glr_spatial_transformer_tutorial_001.png index 71a72e9bd..d634b5922 100644 Binary files a/docs/_images/sphx_glr_spatial_transformer_tutorial_001.png and b/docs/_images/sphx_glr_spatial_transformer_tutorial_001.png differ diff --git a/docs/_images/sphx_glr_transfer_learning_tutorial_001.png b/docs/_images/sphx_glr_transfer_learning_tutorial_001.png index 17ad17384..757c53afb 100644 Binary files a/docs/_images/sphx_glr_transfer_learning_tutorial_001.png and b/docs/_images/sphx_glr_transfer_learning_tutorial_001.png differ diff --git a/docs/_images/sphx_glr_transfer_learning_tutorial_002.png b/docs/_images/sphx_glr_transfer_learning_tutorial_002.png index db9a0168b..353ebe64f 100644 Binary files a/docs/_images/sphx_glr_transfer_learning_tutorial_002.png and b/docs/_images/sphx_glr_transfer_learning_tutorial_002.png differ diff --git a/docs/_images/sphx_glr_transfer_learning_tutorial_003.png b/docs/_images/sphx_glr_transfer_learning_tutorial_003.png index 2fd2d77cf..674edf752 100644 Binary files a/docs/_images/sphx_glr_transfer_learning_tutorial_003.png and b/docs/_images/sphx_glr_transfer_learning_tutorial_003.png differ diff --git a/docs/_sources/index.rst.txt b/docs/_sources/index.rst.txt index a1fa70ccd..3cf443f62 100644 --- a/docs/_sources/index.rst.txt +++ b/docs/_sources/index.rst.txt @@ -128,8 +128,7 @@ .. customcarditem:: :header: 기초부터 시작하는 NLP: 문자-단위 RNN으로 이름 분류하기 - :card_description: - torchtext를 사용하지 않고 기본적인 문자-단위 RNN을 사용하여 단어를 분류하는 모델을 기초부터 만들고 학습합니다. 총 3개로 이뤄진 튜토리얼 시리즈의 첫번째 편입니다. + :card_description: torchtext를 사용하지 않고 기본적인 문자-단위 RNN을 사용하여 단어를 분류하는 모델을 기초부터 만들고 학습합니다. 총 3개로 이뤄진 튜토리얼 시리즈의 첫번째 편입니다. :image: _static/img/thumbnails/cropped/NLP-From-Scratch-Classifying-Names-with-a-Character-Level-RNN.png :link: intermediate/char_rnn_classification_tutorial :tags: Text diff --git a/docs/_static/jquery-3.5.1.js b/docs/_static/jquery-3.5.1.js new file mode 100644 index 000000000..50937333b --- /dev/null +++ b/docs/_static/jquery-3.5.1.js @@ -0,0 +1,10872 @@ +/*! + * jQuery JavaScript Library v3.5.1 + * https://jquery.com/ + * + * Includes Sizzle.js + * https://sizzlejs.com/ + * + * Copyright JS Foundation and other contributors + * Released under the MIT license + * https://jquery.org/license + * + * Date: 2020-05-04T22:49Z + */ +( function( global, factory ) { + + "use strict"; + + if ( typeof module === "object" && typeof module.exports === "object" ) { + + // For CommonJS and CommonJS-like environments where a proper `window` + // is present, execute the factory and get jQuery. + // For environments that do not have a `window` with a `document` + // (such as Node.js), expose a factory as module.exports. + // This accentuates the need for the creation of a real `window`. + // e.g. var jQuery = require("jquery")(window); + // See ticket #14549 for more info. + module.exports = global.document ? + factory( global, true ) : + function( w ) { + if ( !w.document ) { + throw new Error( "jQuery requires a window with a document" ); + } + return factory( w ); + }; + } else { + factory( global ); + } + +// Pass this if window is not defined yet +} )( typeof window !== "undefined" ? window : this, function( window, noGlobal ) { + +// Edge <= 12 - 13+, Firefox <=18 - 45+, IE 10 - 11, Safari 5.1 - 9+, iOS 6 - 9.1 +// throw exceptions when non-strict code (e.g., ASP.NET 4.5) accesses strict mode +// arguments.callee.caller (trac-13335). But as of jQuery 3.0 (2016), strict mode should be common +// enough that all such attempts are guarded in a try block. +"use strict"; + +var arr = []; + +var getProto = Object.getPrototypeOf; + +var slice = arr.slice; + +var flat = arr.flat ? function( array ) { + return arr.flat.call( array ); +} : function( array ) { + return arr.concat.apply( [], array ); +}; + + +var push = arr.push; + +var indexOf = arr.indexOf; + +var class2type = {}; + +var toString = class2type.toString; + +var hasOwn = class2type.hasOwnProperty; + +var fnToString = hasOwn.toString; + +var ObjectFunctionString = fnToString.call( Object ); + +var support = {}; + +var isFunction = function isFunction( obj ) { + + // Support: Chrome <=57, Firefox <=52 + // In some browsers, typeof returns "function" for HTML elements + // (i.e., `typeof document.createElement( "object" ) === "function"`). + // We don't want to classify *any* DOM node as a function. + return typeof obj === "function" && typeof obj.nodeType !== "number"; + }; + + +var isWindow = function isWindow( obj ) { + return obj != null && obj === obj.window; + }; + + +var document = window.document; + + + + var preservedScriptAttributes = { + type: true, + src: true, + nonce: true, + noModule: true + }; + + function DOMEval( code, node, doc ) { + doc = doc || document; + + var i, val, + script = doc.createElement( "script" ); + + script.text = code; + if ( node ) { + for ( i in preservedScriptAttributes ) { + + // Support: Firefox 64+, Edge 18+ + // Some browsers don't support the "nonce" property on scripts. + // On the other hand, just using `getAttribute` is not enough as + // the `nonce` attribute is reset to an empty string whenever it + // becomes browsing-context connected. + // See https://github.com/whatwg/html/issues/2369 + // See https://html.spec.whatwg.org/#nonce-attributes + // The `node.getAttribute` check was added for the sake of + // `jQuery.globalEval` so that it can fake a nonce-containing node + // via an object. + val = node[ i ] || node.getAttribute && node.getAttribute( i ); + if ( val ) { + script.setAttribute( i, val ); + } + } + } + doc.head.appendChild( script ).parentNode.removeChild( script ); + } + + +function toType( obj ) { + if ( obj == null ) { + return obj + ""; + } + + // Support: Android <=2.3 only (functionish RegExp) + return typeof obj === "object" || typeof obj === "function" ? + class2type[ toString.call( obj ) ] || "object" : + typeof obj; +} +/* global Symbol */ +// Defining this global in .eslintrc.json would create a danger of using the global +// unguarded in another place, it seems safer to define global only for this module + + + +var + version = "3.5.1", + + // Define a local copy of jQuery + jQuery = function( selector, context ) { + + // The jQuery object is actually just the init constructor 'enhanced' + // Need init if jQuery is called (just allow error to be thrown if not included) + return new jQuery.fn.init( selector, context ); + }; + +jQuery.fn = jQuery.prototype = { + + // The current version of jQuery being used + jquery: version, + + constructor: jQuery, + + // The default length of a jQuery object is 0 + length: 0, + + toArray: function() { + return slice.call( this ); + }, + + // Get the Nth element in the matched element set OR + // Get the whole matched element set as a clean array + get: function( num ) { + + // Return all the elements in a clean array + if ( num == null ) { + return slice.call( this ); + } + + // Return just the one element from the set + return num < 0 ? this[ num + this.length ] : this[ num ]; + }, + + // Take an array of elements and push it onto the stack + // (returning the new matched element set) + pushStack: function( elems ) { + + // Build a new jQuery matched element set + var ret = jQuery.merge( this.constructor(), elems ); + + // Add the old object onto the stack (as a reference) + ret.prevObject = this; + + // Return the newly-formed element set + return ret; + }, + + // Execute a callback for every element in the matched set. + each: function( callback ) { + return jQuery.each( this, callback ); + }, + + map: function( callback ) { + return this.pushStack( jQuery.map( this, function( elem, i ) { + return callback.call( elem, i, elem ); + } ) ); + }, + + slice: function() { + return this.pushStack( slice.apply( this, arguments ) ); + }, + + first: function() { + return this.eq( 0 ); + }, + + last: function() { + return this.eq( -1 ); + }, + + even: function() { + return this.pushStack( jQuery.grep( this, function( _elem, i ) { + return ( i + 1 ) % 2; + } ) ); + }, + + odd: function() { + return this.pushStack( jQuery.grep( this, function( _elem, i ) { + return i % 2; + } ) ); + }, + + eq: function( i ) { + var len = this.length, + j = +i + ( i < 0 ? len : 0 ); + return this.pushStack( j >= 0 && j < len ? [ this[ j ] ] : [] ); + }, + + end: function() { + return this.prevObject || this.constructor(); + }, + + // For internal use only. + // Behaves like an Array's method, not like a jQuery method. + push: push, + sort: arr.sort, + splice: arr.splice +}; + +jQuery.extend = jQuery.fn.extend = function() { + var options, name, src, copy, copyIsArray, clone, + target = arguments[ 0 ] || {}, + i = 1, + length = arguments.length, + deep = false; + + // Handle a deep copy situation + if ( typeof target === "boolean" ) { + deep = target; + + // Skip the boolean and the target + target = arguments[ i ] || {}; + i++; + } + + // Handle case when target is a string or something (possible in deep copy) + if ( typeof target !== "object" && !isFunction( target ) ) { + target = {}; + } + + // Extend jQuery itself if only one argument is passed + if ( i === length ) { + target = this; + i--; + } + + for ( ; i < length; i++ ) { + + // Only deal with non-null/undefined values + if ( ( options = arguments[ i ] ) != null ) { + + // Extend the base object + for ( name in options ) { + copy = options[ name ]; + + // Prevent Object.prototype pollution + // Prevent never-ending loop + if ( name === "__proto__" || target === copy ) { + continue; + } + + // Recurse if we're merging plain objects or arrays + if ( deep && copy && ( jQuery.isPlainObject( copy ) || + ( copyIsArray = Array.isArray( copy ) ) ) ) { + src = target[ name ]; + + // Ensure proper type for the source value + if ( copyIsArray && !Array.isArray( src ) ) { + clone = []; + } else if ( !copyIsArray && !jQuery.isPlainObject( src ) ) { + clone = {}; + } else { + clone = src; + } + copyIsArray = false; + + // Never move original objects, clone them + target[ name ] = jQuery.extend( deep, clone, copy ); + + // Don't bring in undefined values + } else if ( copy !== undefined ) { + target[ name ] = copy; + } + } + } + } + + // Return the modified object + return target; +}; + +jQuery.extend( { + + // Unique for each copy of jQuery on the page + expando: "jQuery" + ( version + Math.random() ).replace( /\D/g, "" ), + + // Assume jQuery is ready without the ready module + isReady: true, + + error: function( msg ) { + throw new Error( msg ); + }, + + noop: function() {}, + + isPlainObject: function( obj ) { + var proto, Ctor; + + // Detect obvious negatives + // Use toString instead of jQuery.type to catch host objects + if ( !obj || toString.call( obj ) !== "[object Object]" ) { + return false; + } + + proto = getProto( obj ); + + // Objects with no prototype (e.g., `Object.create( null )`) are plain + if ( !proto ) { + return true; + } + + // Objects with prototype are plain iff they were constructed by a global Object function + Ctor = hasOwn.call( proto, "constructor" ) && proto.constructor; + return typeof Ctor === "function" && fnToString.call( Ctor ) === ObjectFunctionString; + }, + + isEmptyObject: function( obj ) { + var name; + + for ( name in obj ) { + return false; + } + return true; + }, + + // Evaluates a script in a provided context; falls back to the global one + // if not specified. + globalEval: function( code, options, doc ) { + DOMEval( code, { nonce: options && options.nonce }, doc ); + }, + + each: function( obj, callback ) { + var length, i = 0; + + if ( isArrayLike( obj ) ) { + length = obj.length; + for ( ; i < length; i++ ) { + if ( callback.call( obj[ i ], i, obj[ i ] ) === false ) { + break; + } + } + } else { + for ( i in obj ) { + if ( callback.call( obj[ i ], i, obj[ i ] ) === false ) { + break; + } + } + } + + return obj; + }, + + // results is for internal usage only + makeArray: function( arr, results ) { + var ret = results || []; + + if ( arr != null ) { + if ( isArrayLike( Object( arr ) ) ) { + jQuery.merge( ret, + typeof arr === "string" ? + [ arr ] : arr + ); + } else { + push.call( ret, arr ); + } + } + + return ret; + }, + + inArray: function( elem, arr, i ) { + return arr == null ? -1 : indexOf.call( arr, elem, i ); + }, + + // Support: Android <=4.0 only, PhantomJS 1 only + // push.apply(_, arraylike) throws on ancient WebKit + merge: function( first, second ) { + var len = +second.length, + j = 0, + i = first.length; + + for ( ; j < len; j++ ) { + first[ i++ ] = second[ j ]; + } + + first.length = i; + + return first; + }, + + grep: function( elems, callback, invert ) { + var callbackInverse, + matches = [], + i = 0, + length = elems.length, + callbackExpect = !invert; + + // Go through the array, only saving the items + // that pass the validator function + for ( ; i < length; i++ ) { + callbackInverse = !callback( elems[ i ], i ); + if ( callbackInverse !== callbackExpect ) { + matches.push( elems[ i ] ); + } + } + + return matches; + }, + + // arg is for internal usage only + map: function( elems, callback, arg ) { + var length, value, + i = 0, + ret = []; + + // Go through the array, translating each of the items to their new values + if ( isArrayLike( elems ) ) { + length = elems.length; + for ( ; i < length; i++ ) { + value = callback( elems[ i ], i, arg ); + + if ( value != null ) { + ret.push( value ); + } + } + + // Go through every key on the object, + } else { + for ( i in elems ) { + value = callback( elems[ i ], i, arg ); + + if ( value != null ) { + ret.push( value ); + } + } + } + + // Flatten any nested arrays + return flat( ret ); + }, + + // A global GUID counter for objects + guid: 1, + + // jQuery.support is not used in Core but other projects attach their + // properties to it so it needs to exist. + support: support +} ); + +if ( typeof Symbol === "function" ) { + jQuery.fn[ Symbol.iterator ] = arr[ Symbol.iterator ]; +} + +// Populate the class2type map +jQuery.each( "Boolean Number String Function Array Date RegExp Object Error Symbol".split( " " ), +function( _i, name ) { + class2type[ "[object " + name + "]" ] = name.toLowerCase(); +} ); + +function isArrayLike( obj ) { + + // Support: real iOS 8.2 only (not reproducible in simulator) + // `in` check used to prevent JIT error (gh-2145) + // hasOwn isn't used here due to false negatives + // regarding Nodelist length in IE + var length = !!obj && "length" in obj && obj.length, + type = toType( obj ); + + if ( isFunction( obj ) || isWindow( obj ) ) { + return false; + } + + return type === "array" || length === 0 || + typeof length === "number" && length > 0 && ( length - 1 ) in obj; +} +var Sizzle = +/*! + * Sizzle CSS Selector Engine v2.3.5 + * https://sizzlejs.com/ + * + * Copyright JS Foundation and other contributors + * Released under the MIT license + * https://js.foundation/ + * + * Date: 2020-03-14 + */ +( function( window ) { +var i, + support, + Expr, + getText, + isXML, + tokenize, + compile, + select, + outermostContext, + sortInput, + hasDuplicate, + + // Local document vars + setDocument, + document, + docElem, + documentIsHTML, + rbuggyQSA, + rbuggyMatches, + matches, + contains, + + // Instance-specific data + expando = "sizzle" + 1 * new Date(), + preferredDoc = window.document, + dirruns = 0, + done = 0, + classCache = createCache(), + tokenCache = createCache(), + compilerCache = createCache(), + nonnativeSelectorCache = createCache(), + sortOrder = function( a, b ) { + if ( a === b ) { + hasDuplicate = true; + } + return 0; + }, + + // Instance methods + hasOwn = ( {} ).hasOwnProperty, + arr = [], + pop = arr.pop, + pushNative = arr.push, + push = arr.push, + slice = arr.slice, + + // Use a stripped-down indexOf as it's faster than native + // https://jsperf.com/thor-indexof-vs-for/5 + indexOf = function( list, elem ) { + var i = 0, + len = list.length; + for ( ; i < len; i++ ) { + if ( list[ i ] === elem ) { + return i; + } + } + return -1; + }, + + booleans = "checked|selected|async|autofocus|autoplay|controls|defer|disabled|hidden|" + + "ismap|loop|multiple|open|readonly|required|scoped", + + // Regular expressions + + // http://www.w3.org/TR/css3-selectors/#whitespace + whitespace = "[\\x20\\t\\r\\n\\f]", + + // https://www.w3.org/TR/css-syntax-3/#ident-token-diagram + identifier = "(?:\\\\[\\da-fA-F]{1,6}" + whitespace + + "?|\\\\[^\\r\\n\\f]|[\\w-]|[^\0-\\x7f])+", + + // Attribute selectors: http://www.w3.org/TR/selectors/#attribute-selectors + attributes = "\\[" + whitespace + "*(" + identifier + ")(?:" + whitespace + + + // Operator (capture 2) + "*([*^$|!~]?=)" + whitespace + + + // "Attribute values must be CSS identifiers [capture 5] + // or strings [capture 3 or capture 4]" + "*(?:'((?:\\\\.|[^\\\\'])*)'|\"((?:\\\\.|[^\\\\\"])*)\"|(" + identifier + "))|)" + + whitespace + "*\\]", + + pseudos = ":(" + identifier + ")(?:\\((" + + + // To reduce the number of selectors needing tokenize in the preFilter, prefer arguments: + // 1. quoted (capture 3; capture 4 or capture 5) + "('((?:\\\\.|[^\\\\'])*)'|\"((?:\\\\.|[^\\\\\"])*)\")|" + + + // 2. simple (capture 6) + "((?:\\\\.|[^\\\\()[\\]]|" + attributes + ")*)|" + + + // 3. anything else (capture 2) + ".*" + + ")\\)|)", + + // Leading and non-escaped trailing whitespace, capturing some non-whitespace characters preceding the latter + rwhitespace = new RegExp( whitespace + "+", "g" ), + rtrim = new RegExp( "^" + whitespace + "+|((?:^|[^\\\\])(?:\\\\.)*)" + + whitespace + "+$", "g" ), + + rcomma = new RegExp( "^" + whitespace + "*," + whitespace + "*" ), + rcombinators = new RegExp( "^" + whitespace + "*([>+~]|" + whitespace + ")" + whitespace + + "*" ), + rdescend = new RegExp( whitespace + "|>" ), + + rpseudo = new RegExp( pseudos ), + ridentifier = new RegExp( "^" + identifier + "$" ), + + matchExpr = { + "ID": new RegExp( "^#(" + identifier + ")" ), + "CLASS": new RegExp( "^\\.(" + identifier + ")" ), + "TAG": new RegExp( "^(" + identifier + "|[*])" ), + "ATTR": new RegExp( "^" + attributes ), + "PSEUDO": new RegExp( "^" + pseudos ), + "CHILD": new RegExp( "^:(only|first|last|nth|nth-last)-(child|of-type)(?:\\(" + + whitespace + "*(even|odd|(([+-]|)(\\d*)n|)" + whitespace + "*(?:([+-]|)" + + whitespace + "*(\\d+)|))" + whitespace + "*\\)|)", "i" ), + "bool": new RegExp( "^(?:" + booleans + ")$", "i" ), + + // For use in libraries implementing .is() + // We use this for POS matching in `select` + "needsContext": new RegExp( "^" + whitespace + + "*[>+~]|:(even|odd|eq|gt|lt|nth|first|last)(?:\\(" + whitespace + + "*((?:-\\d)?\\d*)" + whitespace + "*\\)|)(?=[^-]|$)", "i" ) + }, + + rhtml = /HTML$/i, + rinputs = /^(?:input|select|textarea|button)$/i, + rheader = /^h\d$/i, + + rnative = /^[^{]+\{\s*\[native \w/, + + // Easily-parseable/retrievable ID or TAG or CLASS selectors + rquickExpr = /^(?:#([\w-]+)|(\w+)|\.([\w-]+))$/, + + rsibling = /[+~]/, + + // CSS escapes + // http://www.w3.org/TR/CSS21/syndata.html#escaped-characters + runescape = new RegExp( "\\\\[\\da-fA-F]{1,6}" + whitespace + "?|\\\\([^\\r\\n\\f])", "g" ), + funescape = function( escape, nonHex ) { + var high = "0x" + escape.slice( 1 ) - 0x10000; + + return nonHex ? + + // Strip the backslash prefix from a non-hex escape sequence + nonHex : + + // Replace a hexadecimal escape sequence with the encoded Unicode code point + // Support: IE <=11+ + // For values outside the Basic Multilingual Plane (BMP), manually construct a + // surrogate pair + high < 0 ? + String.fromCharCode( high + 0x10000 ) : + String.fromCharCode( high >> 10 | 0xD800, high & 0x3FF | 0xDC00 ); + }, + + // CSS string/identifier serialization + // https://drafts.csswg.org/cssom/#common-serializing-idioms + rcssescape = /([\0-\x1f\x7f]|^-?\d)|^-$|[^\0-\x1f\x7f-\uFFFF\w-]/g, + fcssescape = function( ch, asCodePoint ) { + if ( asCodePoint ) { + + // U+0000 NULL becomes U+FFFD REPLACEMENT CHARACTER + if ( ch === "\0" ) { + return "\uFFFD"; + } + + // Control characters and (dependent upon position) numbers get escaped as code points + return ch.slice( 0, -1 ) + "\\" + + ch.charCodeAt( ch.length - 1 ).toString( 16 ) + " "; + } + + // Other potentially-special ASCII characters get backslash-escaped + return "\\" + ch; + }, + + // Used for iframes + // See setDocument() + // Removing the function wrapper causes a "Permission Denied" + // error in IE + unloadHandler = function() { + setDocument(); + }, + + inDisabledFieldset = addCombinator( + function( elem ) { + return elem.disabled === true && elem.nodeName.toLowerCase() === "fieldset"; + }, + { dir: "parentNode", next: "legend" } + ); + +// Optimize for push.apply( _, NodeList ) +try { + push.apply( + ( arr = slice.call( preferredDoc.childNodes ) ), + preferredDoc.childNodes + ); + + // Support: Android<4.0 + // Detect silently failing push.apply + // eslint-disable-next-line no-unused-expressions + arr[ preferredDoc.childNodes.length ].nodeType; +} catch ( e ) { + push = { apply: arr.length ? + + // Leverage slice if possible + function( target, els ) { + pushNative.apply( target, slice.call( els ) ); + } : + + // Support: IE<9 + // Otherwise append directly + function( target, els ) { + var j = target.length, + i = 0; + + // Can't trust NodeList.length + while ( ( target[ j++ ] = els[ i++ ] ) ) {} + target.length = j - 1; + } + }; +} + +function Sizzle( selector, context, results, seed ) { + var m, i, elem, nid, match, groups, newSelector, + newContext = context && context.ownerDocument, + + // nodeType defaults to 9, since context defaults to document + nodeType = context ? context.nodeType : 9; + + results = results || []; + + // Return early from calls with invalid selector or context + if ( typeof selector !== "string" || !selector || + nodeType !== 1 && nodeType !== 9 && nodeType !== 11 ) { + + return results; + } + + // Try to shortcut find operations (as opposed to filters) in HTML documents + if ( !seed ) { + setDocument( context ); + context = context || document; + + if ( documentIsHTML ) { + + // If the selector is sufficiently simple, try using a "get*By*" DOM method + // (excepting DocumentFragment context, where the methods don't exist) + if ( nodeType !== 11 && ( match = rquickExpr.exec( selector ) ) ) { + + // ID selector + if ( ( m = match[ 1 ] ) ) { + + // Document context + if ( nodeType === 9 ) { + if ( ( elem = context.getElementById( m ) ) ) { + + // Support: IE, Opera, Webkit + // TODO: identify versions + // getElementById can match elements by name instead of ID + if ( elem.id === m ) { + results.push( elem ); + return results; + } + } else { + return results; + } + + // Element context + } else { + + // Support: IE, Opera, Webkit + // TODO: identify versions + // getElementById can match elements by name instead of ID + if ( newContext && ( elem = newContext.getElementById( m ) ) && + contains( context, elem ) && + elem.id === m ) { + + results.push( elem ); + return results; + } + } + + // Type selector + } else if ( match[ 2 ] ) { + push.apply( results, context.getElementsByTagName( selector ) ); + return results; + + // Class selector + } else if ( ( m = match[ 3 ] ) && support.getElementsByClassName && + context.getElementsByClassName ) { + + push.apply( results, context.getElementsByClassName( m ) ); + return results; + } + } + + // Take advantage of querySelectorAll + if ( support.qsa && + !nonnativeSelectorCache[ selector + " " ] && + ( !rbuggyQSA || !rbuggyQSA.test( selector ) ) && + + // Support: IE 8 only + // Exclude object elements + ( nodeType !== 1 || context.nodeName.toLowerCase() !== "object" ) ) { + + newSelector = selector; + newContext = context; + + // qSA considers elements outside a scoping root when evaluating child or + // descendant combinators, which is not what we want. + // In such cases, we work around the behavior by prefixing every selector in the + // list with an ID selector referencing the scope context. + // The technique has to be used as well when a leading combinator is used + // as such selectors are not recognized by querySelectorAll. + // Thanks to Andrew Dupont for this technique. + if ( nodeType === 1 && + ( rdescend.test( selector ) || rcombinators.test( selector ) ) ) { + + // Expand context for sibling selectors + newContext = rsibling.test( selector ) && testContext( context.parentNode ) || + context; + + // We can use :scope instead of the ID hack if the browser + // supports it & if we're not changing the context. + if ( newContext !== context || !support.scope ) { + + // Capture the context ID, setting it first if necessary + if ( ( nid = context.getAttribute( "id" ) ) ) { + nid = nid.replace( rcssescape, fcssescape ); + } else { + context.setAttribute( "id", ( nid = expando ) ); + } + } + + // Prefix every selector in the list + groups = tokenize( selector ); + i = groups.length; + while ( i-- ) { + groups[ i ] = ( nid ? "#" + nid : ":scope" ) + " " + + toSelector( groups[ i ] ); + } + newSelector = groups.join( "," ); + } + + try { + push.apply( results, + newContext.querySelectorAll( newSelector ) + ); + return results; + } catch ( qsaError ) { + nonnativeSelectorCache( selector, true ); + } finally { + if ( nid === expando ) { + context.removeAttribute( "id" ); + } + } + } + } + } + + // All others + return select( selector.replace( rtrim, "$1" ), context, results, seed ); +} + +/** + * Create key-value caches of limited size + * @returns {function(string, object)} Returns the Object data after storing it on itself with + * property name the (space-suffixed) string and (if the cache is larger than Expr.cacheLength) + * deleting the oldest entry + */ +function createCache() { + var keys = []; + + function cache( key, value ) { + + // Use (key + " ") to avoid collision with native prototype properties (see Issue #157) + if ( keys.push( key + " " ) > Expr.cacheLength ) { + + // Only keep the most recent entries + delete cache[ keys.shift() ]; + } + return ( cache[ key + " " ] = value ); + } + return cache; +} + +/** + * Mark a function for special use by Sizzle + * @param {Function} fn The function to mark + */ +function markFunction( fn ) { + fn[ expando ] = true; + return fn; +} + +/** + * Support testing using an element + * @param {Function} fn Passed the created element and returns a boolean result + */ +function assert( fn ) { + var el = document.createElement( "fieldset" ); + + try { + return !!fn( el ); + } catch ( e ) { + return false; + } finally { + + // Remove from its parent by default + if ( el.parentNode ) { + el.parentNode.removeChild( el ); + } + + // release memory in IE + el = null; + } +} + +/** + * Adds the same handler for all of the specified attrs + * @param {String} attrs Pipe-separated list of attributes + * @param {Function} handler The method that will be applied + */ +function addHandle( attrs, handler ) { + var arr = attrs.split( "|" ), + i = arr.length; + + while ( i-- ) { + Expr.attrHandle[ arr[ i ] ] = handler; + } +} + +/** + * Checks document order of two siblings + * @param {Element} a + * @param {Element} b + * @returns {Number} Returns less than 0 if a precedes b, greater than 0 if a follows b + */ +function siblingCheck( a, b ) { + var cur = b && a, + diff = cur && a.nodeType === 1 && b.nodeType === 1 && + a.sourceIndex - b.sourceIndex; + + // Use IE sourceIndex if available on both nodes + if ( diff ) { + return diff; + } + + // Check if b follows a + if ( cur ) { + while ( ( cur = cur.nextSibling ) ) { + if ( cur === b ) { + return -1; + } + } + } + + return a ? 1 : -1; +} + +/** + * Returns a function to use in pseudos for input types + * @param {String} type + */ +function createInputPseudo( type ) { + return function( elem ) { + var name = elem.nodeName.toLowerCase(); + return name === "input" && elem.type === type; + }; +} + +/** + * Returns a function to use in pseudos for buttons + * @param {String} type + */ +function createButtonPseudo( type ) { + return function( elem ) { + var name = elem.nodeName.toLowerCase(); + return ( name === "input" || name === "button" ) && elem.type === type; + }; +} + +/** + * Returns a function to use in pseudos for :enabled/:disabled + * @param {Boolean} disabled true for :disabled; false for :enabled + */ +function createDisabledPseudo( disabled ) { + + // Known :disabled false positives: fieldset[disabled] > legend:nth-of-type(n+2) :can-disable + return function( elem ) { + + // Only certain elements can match :enabled or :disabled + // https://html.spec.whatwg.org/multipage/scripting.html#selector-enabled + // https://html.spec.whatwg.org/multipage/scripting.html#selector-disabled + if ( "form" in elem ) { + + // Check for inherited disabledness on relevant non-disabled elements: + // * listed form-associated elements in a disabled fieldset + // https://html.spec.whatwg.org/multipage/forms.html#category-listed + // https://html.spec.whatwg.org/multipage/forms.html#concept-fe-disabled + // * option elements in a disabled optgroup + // https://html.spec.whatwg.org/multipage/forms.html#concept-option-disabled + // All such elements have a "form" property. + if ( elem.parentNode && elem.disabled === false ) { + + // Option elements defer to a parent optgroup if present + if ( "label" in elem ) { + if ( "label" in elem.parentNode ) { + return elem.parentNode.disabled === disabled; + } else { + return elem.disabled === disabled; + } + } + + // Support: IE 6 - 11 + // Use the isDisabled shortcut property to check for disabled fieldset ancestors + return elem.isDisabled === disabled || + + // Where there is no isDisabled, check manually + /* jshint -W018 */ + elem.isDisabled !== !disabled && + inDisabledFieldset( elem ) === disabled; + } + + return elem.disabled === disabled; + + // Try to winnow out elements that can't be disabled before trusting the disabled property. + // Some victims get caught in our net (label, legend, menu, track), but it shouldn't + // even exist on them, let alone have a boolean value. + } else if ( "label" in elem ) { + return elem.disabled === disabled; + } + + // Remaining elements are neither :enabled nor :disabled + return false; + }; +} + +/** + * Returns a function to use in pseudos for positionals + * @param {Function} fn + */ +function createPositionalPseudo( fn ) { + return markFunction( function( argument ) { + argument = +argument; + return markFunction( function( seed, matches ) { + var j, + matchIndexes = fn( [], seed.length, argument ), + i = matchIndexes.length; + + // Match elements found at the specified indexes + while ( i-- ) { + if ( seed[ ( j = matchIndexes[ i ] ) ] ) { + seed[ j ] = !( matches[ j ] = seed[ j ] ); + } + } + } ); + } ); +} + +/** + * Checks a node for validity as a Sizzle context + * @param {Element|Object=} context + * @returns {Element|Object|Boolean} The input node if acceptable, otherwise a falsy value + */ +function testContext( context ) { + return context && typeof context.getElementsByTagName !== "undefined" && context; +} + +// Expose support vars for convenience +support = Sizzle.support = {}; + +/** + * Detects XML nodes + * @param {Element|Object} elem An element or a document + * @returns {Boolean} True iff elem is a non-HTML XML node + */ +isXML = Sizzle.isXML = function( elem ) { + var namespace = elem.namespaceURI, + docElem = ( elem.ownerDocument || elem ).documentElement; + + // Support: IE <=8 + // Assume HTML when documentElement doesn't yet exist, such as inside loading iframes + // https://bugs.jquery.com/ticket/4833 + return !rhtml.test( namespace || docElem && docElem.nodeName || "HTML" ); +}; + +/** + * Sets document-related variables once based on the current document + * @param {Element|Object} [doc] An element or document object to use to set the document + * @returns {Object} Returns the current document + */ +setDocument = Sizzle.setDocument = function( node ) { + var hasCompare, subWindow, + doc = node ? node.ownerDocument || node : preferredDoc; + + // Return early if doc is invalid or already selected + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( doc == document || doc.nodeType !== 9 || !doc.documentElement ) { + return document; + } + + // Update global variables + document = doc; + docElem = document.documentElement; + documentIsHTML = !isXML( document ); + + // Support: IE 9 - 11+, Edge 12 - 18+ + // Accessing iframe documents after unload throws "permission denied" errors (jQuery #13936) + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( preferredDoc != document && + ( subWindow = document.defaultView ) && subWindow.top !== subWindow ) { + + // Support: IE 11, Edge + if ( subWindow.addEventListener ) { + subWindow.addEventListener( "unload", unloadHandler, false ); + + // Support: IE 9 - 10 only + } else if ( subWindow.attachEvent ) { + subWindow.attachEvent( "onunload", unloadHandler ); + } + } + + // Support: IE 8 - 11+, Edge 12 - 18+, Chrome <=16 - 25 only, Firefox <=3.6 - 31 only, + // Safari 4 - 5 only, Opera <=11.6 - 12.x only + // IE/Edge & older browsers don't support the :scope pseudo-class. + // Support: Safari 6.0 only + // Safari 6.0 supports :scope but it's an alias of :root there. + support.scope = assert( function( el ) { + docElem.appendChild( el ).appendChild( document.createElement( "div" ) ); + return typeof el.querySelectorAll !== "undefined" && + !el.querySelectorAll( ":scope fieldset div" ).length; + } ); + + /* Attributes + ---------------------------------------------------------------------- */ + + // Support: IE<8 + // Verify that getAttribute really returns attributes and not properties + // (excepting IE8 booleans) + support.attributes = assert( function( el ) { + el.className = "i"; + return !el.getAttribute( "className" ); + } ); + + /* getElement(s)By* + ---------------------------------------------------------------------- */ + + // Check if getElementsByTagName("*") returns only elements + support.getElementsByTagName = assert( function( el ) { + el.appendChild( document.createComment( "" ) ); + return !el.getElementsByTagName( "*" ).length; + } ); + + // Support: IE<9 + support.getElementsByClassName = rnative.test( document.getElementsByClassName ); + + // Support: IE<10 + // Check if getElementById returns elements by name + // The broken getElementById methods don't pick up programmatically-set names, + // so use a roundabout getElementsByName test + support.getById = assert( function( el ) { + docElem.appendChild( el ).id = expando; + return !document.getElementsByName || !document.getElementsByName( expando ).length; + } ); + + // ID filter and find + if ( support.getById ) { + Expr.filter[ "ID" ] = function( id ) { + var attrId = id.replace( runescape, funescape ); + return function( elem ) { + return elem.getAttribute( "id" ) === attrId; + }; + }; + Expr.find[ "ID" ] = function( id, context ) { + if ( typeof context.getElementById !== "undefined" && documentIsHTML ) { + var elem = context.getElementById( id ); + return elem ? [ elem ] : []; + } + }; + } else { + Expr.filter[ "ID" ] = function( id ) { + var attrId = id.replace( runescape, funescape ); + return function( elem ) { + var node = typeof elem.getAttributeNode !== "undefined" && + elem.getAttributeNode( "id" ); + return node && node.value === attrId; + }; + }; + + // Support: IE 6 - 7 only + // getElementById is not reliable as a find shortcut + Expr.find[ "ID" ] = function( id, context ) { + if ( typeof context.getElementById !== "undefined" && documentIsHTML ) { + var node, i, elems, + elem = context.getElementById( id ); + + if ( elem ) { + + // Verify the id attribute + node = elem.getAttributeNode( "id" ); + if ( node && node.value === id ) { + return [ elem ]; + } + + // Fall back on getElementsByName + elems = context.getElementsByName( id ); + i = 0; + while ( ( elem = elems[ i++ ] ) ) { + node = elem.getAttributeNode( "id" ); + if ( node && node.value === id ) { + return [ elem ]; + } + } + } + + return []; + } + }; + } + + // Tag + Expr.find[ "TAG" ] = support.getElementsByTagName ? + function( tag, context ) { + if ( typeof context.getElementsByTagName !== "undefined" ) { + return context.getElementsByTagName( tag ); + + // DocumentFragment nodes don't have gEBTN + } else if ( support.qsa ) { + return context.querySelectorAll( tag ); + } + } : + + function( tag, context ) { + var elem, + tmp = [], + i = 0, + + // By happy coincidence, a (broken) gEBTN appears on DocumentFragment nodes too + results = context.getElementsByTagName( tag ); + + // Filter out possible comments + if ( tag === "*" ) { + while ( ( elem = results[ i++ ] ) ) { + if ( elem.nodeType === 1 ) { + tmp.push( elem ); + } + } + + return tmp; + } + return results; + }; + + // Class + Expr.find[ "CLASS" ] = support.getElementsByClassName && function( className, context ) { + if ( typeof context.getElementsByClassName !== "undefined" && documentIsHTML ) { + return context.getElementsByClassName( className ); + } + }; + + /* QSA/matchesSelector + ---------------------------------------------------------------------- */ + + // QSA and matchesSelector support + + // matchesSelector(:active) reports false when true (IE9/Opera 11.5) + rbuggyMatches = []; + + // qSa(:focus) reports false when true (Chrome 21) + // We allow this because of a bug in IE8/9 that throws an error + // whenever `document.activeElement` is accessed on an iframe + // So, we allow :focus to pass through QSA all the time to avoid the IE error + // See https://bugs.jquery.com/ticket/13378 + rbuggyQSA = []; + + if ( ( support.qsa = rnative.test( document.querySelectorAll ) ) ) { + + // Build QSA regex + // Regex strategy adopted from Diego Perini + assert( function( el ) { + + var input; + + // Select is set to empty string on purpose + // This is to test IE's treatment of not explicitly + // setting a boolean content attribute, + // since its presence should be enough + // https://bugs.jquery.com/ticket/12359 + docElem.appendChild( el ).innerHTML = "" + + ""; + + // Support: IE8, Opera 11-12.16 + // Nothing should be selected when empty strings follow ^= or $= or *= + // The test attribute must be unknown in Opera but "safe" for WinRT + // https://msdn.microsoft.com/en-us/library/ie/hh465388.aspx#attribute_section + if ( el.querySelectorAll( "[msallowcapture^='']" ).length ) { + rbuggyQSA.push( "[*^$]=" + whitespace + "*(?:''|\"\")" ); + } + + // Support: IE8 + // Boolean attributes and "value" are not treated correctly + if ( !el.querySelectorAll( "[selected]" ).length ) { + rbuggyQSA.push( "\\[" + whitespace + "*(?:value|" + booleans + ")" ); + } + + // Support: Chrome<29, Android<4.4, Safari<7.0+, iOS<7.0+, PhantomJS<1.9.8+ + if ( !el.querySelectorAll( "[id~=" + expando + "-]" ).length ) { + rbuggyQSA.push( "~=" ); + } + + // Support: IE 11+, Edge 15 - 18+ + // IE 11/Edge don't find elements on a `[name='']` query in some cases. + // Adding a temporary attribute to the document before the selection works + // around the issue. + // Interestingly, IE 10 & older don't seem to have the issue. + input = document.createElement( "input" ); + input.setAttribute( "name", "" ); + el.appendChild( input ); + if ( !el.querySelectorAll( "[name='']" ).length ) { + rbuggyQSA.push( "\\[" + whitespace + "*name" + whitespace + "*=" + + whitespace + "*(?:''|\"\")" ); + } + + // Webkit/Opera - :checked should return selected option elements + // http://www.w3.org/TR/2011/REC-css3-selectors-20110929/#checked + // IE8 throws error here and will not see later tests + if ( !el.querySelectorAll( ":checked" ).length ) { + rbuggyQSA.push( ":checked" ); + } + + // Support: Safari 8+, iOS 8+ + // https://bugs.webkit.org/show_bug.cgi?id=136851 + // In-page `selector#id sibling-combinator selector` fails + if ( !el.querySelectorAll( "a#" + expando + "+*" ).length ) { + rbuggyQSA.push( ".#.+[+~]" ); + } + + // Support: Firefox <=3.6 - 5 only + // Old Firefox doesn't throw on a badly-escaped identifier. + el.querySelectorAll( "\\\f" ); + rbuggyQSA.push( "[\\r\\n\\f]" ); + } ); + + assert( function( el ) { + el.innerHTML = "" + + ""; + + // Support: Windows 8 Native Apps + // The type and name attributes are restricted during .innerHTML assignment + var input = document.createElement( "input" ); + input.setAttribute( "type", "hidden" ); + el.appendChild( input ).setAttribute( "name", "D" ); + + // Support: IE8 + // Enforce case-sensitivity of name attribute + if ( el.querySelectorAll( "[name=d]" ).length ) { + rbuggyQSA.push( "name" + whitespace + "*[*^$|!~]?=" ); + } + + // FF 3.5 - :enabled/:disabled and hidden elements (hidden elements are still enabled) + // IE8 throws error here and will not see later tests + if ( el.querySelectorAll( ":enabled" ).length !== 2 ) { + rbuggyQSA.push( ":enabled", ":disabled" ); + } + + // Support: IE9-11+ + // IE's :disabled selector does not pick up the children of disabled fieldsets + docElem.appendChild( el ).disabled = true; + if ( el.querySelectorAll( ":disabled" ).length !== 2 ) { + rbuggyQSA.push( ":enabled", ":disabled" ); + } + + // Support: Opera 10 - 11 only + // Opera 10-11 does not throw on post-comma invalid pseudos + el.querySelectorAll( "*,:x" ); + rbuggyQSA.push( ",.*:" ); + } ); + } + + if ( ( support.matchesSelector = rnative.test( ( matches = docElem.matches || + docElem.webkitMatchesSelector || + docElem.mozMatchesSelector || + docElem.oMatchesSelector || + docElem.msMatchesSelector ) ) ) ) { + + assert( function( el ) { + + // Check to see if it's possible to do matchesSelector + // on a disconnected node (IE 9) + support.disconnectedMatch = matches.call( el, "*" ); + + // This should fail with an exception + // Gecko does not error, returns false instead + matches.call( el, "[s!='']:x" ); + rbuggyMatches.push( "!=", pseudos ); + } ); + } + + rbuggyQSA = rbuggyQSA.length && new RegExp( rbuggyQSA.join( "|" ) ); + rbuggyMatches = rbuggyMatches.length && new RegExp( rbuggyMatches.join( "|" ) ); + + /* Contains + ---------------------------------------------------------------------- */ + hasCompare = rnative.test( docElem.compareDocumentPosition ); + + // Element contains another + // Purposefully self-exclusive + // As in, an element does not contain itself + contains = hasCompare || rnative.test( docElem.contains ) ? + function( a, b ) { + var adown = a.nodeType === 9 ? a.documentElement : a, + bup = b && b.parentNode; + return a === bup || !!( bup && bup.nodeType === 1 && ( + adown.contains ? + adown.contains( bup ) : + a.compareDocumentPosition && a.compareDocumentPosition( bup ) & 16 + ) ); + } : + function( a, b ) { + if ( b ) { + while ( ( b = b.parentNode ) ) { + if ( b === a ) { + return true; + } + } + } + return false; + }; + + /* Sorting + ---------------------------------------------------------------------- */ + + // Document order sorting + sortOrder = hasCompare ? + function( a, b ) { + + // Flag for duplicate removal + if ( a === b ) { + hasDuplicate = true; + return 0; + } + + // Sort on method existence if only one input has compareDocumentPosition + var compare = !a.compareDocumentPosition - !b.compareDocumentPosition; + if ( compare ) { + return compare; + } + + // Calculate position if both inputs belong to the same document + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + compare = ( a.ownerDocument || a ) == ( b.ownerDocument || b ) ? + a.compareDocumentPosition( b ) : + + // Otherwise we know they are disconnected + 1; + + // Disconnected nodes + if ( compare & 1 || + ( !support.sortDetached && b.compareDocumentPosition( a ) === compare ) ) { + + // Choose the first element that is related to our preferred document + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( a == document || a.ownerDocument == preferredDoc && + contains( preferredDoc, a ) ) { + return -1; + } + + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( b == document || b.ownerDocument == preferredDoc && + contains( preferredDoc, b ) ) { + return 1; + } + + // Maintain original order + return sortInput ? + ( indexOf( sortInput, a ) - indexOf( sortInput, b ) ) : + 0; + } + + return compare & 4 ? -1 : 1; + } : + function( a, b ) { + + // Exit early if the nodes are identical + if ( a === b ) { + hasDuplicate = true; + return 0; + } + + var cur, + i = 0, + aup = a.parentNode, + bup = b.parentNode, + ap = [ a ], + bp = [ b ]; + + // Parentless nodes are either documents or disconnected + if ( !aup || !bup ) { + + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + /* eslint-disable eqeqeq */ + return a == document ? -1 : + b == document ? 1 : + /* eslint-enable eqeqeq */ + aup ? -1 : + bup ? 1 : + sortInput ? + ( indexOf( sortInput, a ) - indexOf( sortInput, b ) ) : + 0; + + // If the nodes are siblings, we can do a quick check + } else if ( aup === bup ) { + return siblingCheck( a, b ); + } + + // Otherwise we need full lists of their ancestors for comparison + cur = a; + while ( ( cur = cur.parentNode ) ) { + ap.unshift( cur ); + } + cur = b; + while ( ( cur = cur.parentNode ) ) { + bp.unshift( cur ); + } + + // Walk down the tree looking for a discrepancy + while ( ap[ i ] === bp[ i ] ) { + i++; + } + + return i ? + + // Do a sibling check if the nodes have a common ancestor + siblingCheck( ap[ i ], bp[ i ] ) : + + // Otherwise nodes in our document sort first + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + /* eslint-disable eqeqeq */ + ap[ i ] == preferredDoc ? -1 : + bp[ i ] == preferredDoc ? 1 : + /* eslint-enable eqeqeq */ + 0; + }; + + return document; +}; + +Sizzle.matches = function( expr, elements ) { + return Sizzle( expr, null, null, elements ); +}; + +Sizzle.matchesSelector = function( elem, expr ) { + setDocument( elem ); + + if ( support.matchesSelector && documentIsHTML && + !nonnativeSelectorCache[ expr + " " ] && + ( !rbuggyMatches || !rbuggyMatches.test( expr ) ) && + ( !rbuggyQSA || !rbuggyQSA.test( expr ) ) ) { + + try { + var ret = matches.call( elem, expr ); + + // IE 9's matchesSelector returns false on disconnected nodes + if ( ret || support.disconnectedMatch || + + // As well, disconnected nodes are said to be in a document + // fragment in IE 9 + elem.document && elem.document.nodeType !== 11 ) { + return ret; + } + } catch ( e ) { + nonnativeSelectorCache( expr, true ); + } + } + + return Sizzle( expr, document, null, [ elem ] ).length > 0; +}; + +Sizzle.contains = function( context, elem ) { + + // Set document vars if needed + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( ( context.ownerDocument || context ) != document ) { + setDocument( context ); + } + return contains( context, elem ); +}; + +Sizzle.attr = function( elem, name ) { + + // Set document vars if needed + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( ( elem.ownerDocument || elem ) != document ) { + setDocument( elem ); + } + + var fn = Expr.attrHandle[ name.toLowerCase() ], + + // Don't get fooled by Object.prototype properties (jQuery #13807) + val = fn && hasOwn.call( Expr.attrHandle, name.toLowerCase() ) ? + fn( elem, name, !documentIsHTML ) : + undefined; + + return val !== undefined ? + val : + support.attributes || !documentIsHTML ? + elem.getAttribute( name ) : + ( val = elem.getAttributeNode( name ) ) && val.specified ? + val.value : + null; +}; + +Sizzle.escape = function( sel ) { + return ( sel + "" ).replace( rcssescape, fcssescape ); +}; + +Sizzle.error = function( msg ) { + throw new Error( "Syntax error, unrecognized expression: " + msg ); +}; + +/** + * Document sorting and removing duplicates + * @param {ArrayLike} results + */ +Sizzle.uniqueSort = function( results ) { + var elem, + duplicates = [], + j = 0, + i = 0; + + // Unless we *know* we can detect duplicates, assume their presence + hasDuplicate = !support.detectDuplicates; + sortInput = !support.sortStable && results.slice( 0 ); + results.sort( sortOrder ); + + if ( hasDuplicate ) { + while ( ( elem = results[ i++ ] ) ) { + if ( elem === results[ i ] ) { + j = duplicates.push( i ); + } + } + while ( j-- ) { + results.splice( duplicates[ j ], 1 ); + } + } + + // Clear input after sorting to release objects + // See https://github.com/jquery/sizzle/pull/225 + sortInput = null; + + return results; +}; + +/** + * Utility function for retrieving the text value of an array of DOM nodes + * @param {Array|Element} elem + */ +getText = Sizzle.getText = function( elem ) { + var node, + ret = "", + i = 0, + nodeType = elem.nodeType; + + if ( !nodeType ) { + + // If no nodeType, this is expected to be an array + while ( ( node = elem[ i++ ] ) ) { + + // Do not traverse comment nodes + ret += getText( node ); + } + } else if ( nodeType === 1 || nodeType === 9 || nodeType === 11 ) { + + // Use textContent for elements + // innerText usage removed for consistency of new lines (jQuery #11153) + if ( typeof elem.textContent === "string" ) { + return elem.textContent; + } else { + + // Traverse its children + for ( elem = elem.firstChild; elem; elem = elem.nextSibling ) { + ret += getText( elem ); + } + } + } else if ( nodeType === 3 || nodeType === 4 ) { + return elem.nodeValue; + } + + // Do not include comment or processing instruction nodes + + return ret; +}; + +Expr = Sizzle.selectors = { + + // Can be adjusted by the user + cacheLength: 50, + + createPseudo: markFunction, + + match: matchExpr, + + attrHandle: {}, + + find: {}, + + relative: { + ">": { dir: "parentNode", first: true }, + " ": { dir: "parentNode" }, + "+": { dir: "previousSibling", first: true }, + "~": { dir: "previousSibling" } + }, + + preFilter: { + "ATTR": function( match ) { + match[ 1 ] = match[ 1 ].replace( runescape, funescape ); + + // Move the given value to match[3] whether quoted or unquoted + match[ 3 ] = ( match[ 3 ] || match[ 4 ] || + match[ 5 ] || "" ).replace( runescape, funescape ); + + if ( match[ 2 ] === "~=" ) { + match[ 3 ] = " " + match[ 3 ] + " "; + } + + return match.slice( 0, 4 ); + }, + + "CHILD": function( match ) { + + /* matches from matchExpr["CHILD"] + 1 type (only|nth|...) + 2 what (child|of-type) + 3 argument (even|odd|\d*|\d*n([+-]\d+)?|...) + 4 xn-component of xn+y argument ([+-]?\d*n|) + 5 sign of xn-component + 6 x of xn-component + 7 sign of y-component + 8 y of y-component + */ + match[ 1 ] = match[ 1 ].toLowerCase(); + + if ( match[ 1 ].slice( 0, 3 ) === "nth" ) { + + // nth-* requires argument + if ( !match[ 3 ] ) { + Sizzle.error( match[ 0 ] ); + } + + // numeric x and y parameters for Expr.filter.CHILD + // remember that false/true cast respectively to 0/1 + match[ 4 ] = +( match[ 4 ] ? + match[ 5 ] + ( match[ 6 ] || 1 ) : + 2 * ( match[ 3 ] === "even" || match[ 3 ] === "odd" ) ); + match[ 5 ] = +( ( match[ 7 ] + match[ 8 ] ) || match[ 3 ] === "odd" ); + + // other types prohibit arguments + } else if ( match[ 3 ] ) { + Sizzle.error( match[ 0 ] ); + } + + return match; + }, + + "PSEUDO": function( match ) { + var excess, + unquoted = !match[ 6 ] && match[ 2 ]; + + if ( matchExpr[ "CHILD" ].test( match[ 0 ] ) ) { + return null; + } + + // Accept quoted arguments as-is + if ( match[ 3 ] ) { + match[ 2 ] = match[ 4 ] || match[ 5 ] || ""; + + // Strip excess characters from unquoted arguments + } else if ( unquoted && rpseudo.test( unquoted ) && + + // Get excess from tokenize (recursively) + ( excess = tokenize( unquoted, true ) ) && + + // advance to the next closing parenthesis + ( excess = unquoted.indexOf( ")", unquoted.length - excess ) - unquoted.length ) ) { + + // excess is a negative index + match[ 0 ] = match[ 0 ].slice( 0, excess ); + match[ 2 ] = unquoted.slice( 0, excess ); + } + + // Return only captures needed by the pseudo filter method (type and argument) + return match.slice( 0, 3 ); + } + }, + + filter: { + + "TAG": function( nodeNameSelector ) { + var nodeName = nodeNameSelector.replace( runescape, funescape ).toLowerCase(); + return nodeNameSelector === "*" ? + function() { + return true; + } : + function( elem ) { + return elem.nodeName && elem.nodeName.toLowerCase() === nodeName; + }; + }, + + "CLASS": function( className ) { + var pattern = classCache[ className + " " ]; + + return pattern || + ( pattern = new RegExp( "(^|" + whitespace + + ")" + className + "(" + whitespace + "|$)" ) ) && classCache( + className, function( elem ) { + return pattern.test( + typeof elem.className === "string" && elem.className || + typeof elem.getAttribute !== "undefined" && + elem.getAttribute( "class" ) || + "" + ); + } ); + }, + + "ATTR": function( name, operator, check ) { + return function( elem ) { + var result = Sizzle.attr( elem, name ); + + if ( result == null ) { + return operator === "!="; + } + if ( !operator ) { + return true; + } + + result += ""; + + /* eslint-disable max-len */ + + return operator === "=" ? result === check : + operator === "!=" ? result !== check : + operator === "^=" ? check && result.indexOf( check ) === 0 : + operator === "*=" ? check && result.indexOf( check ) > -1 : + operator === "$=" ? check && result.slice( -check.length ) === check : + operator === "~=" ? ( " " + result.replace( rwhitespace, " " ) + " " ).indexOf( check ) > -1 : + operator === "|=" ? result === check || result.slice( 0, check.length + 1 ) === check + "-" : + false; + /* eslint-enable max-len */ + + }; + }, + + "CHILD": function( type, what, _argument, first, last ) { + var simple = type.slice( 0, 3 ) !== "nth", + forward = type.slice( -4 ) !== "last", + ofType = what === "of-type"; + + return first === 1 && last === 0 ? + + // Shortcut for :nth-*(n) + function( elem ) { + return !!elem.parentNode; + } : + + function( elem, _context, xml ) { + var cache, uniqueCache, outerCache, node, nodeIndex, start, + dir = simple !== forward ? "nextSibling" : "previousSibling", + parent = elem.parentNode, + name = ofType && elem.nodeName.toLowerCase(), + useCache = !xml && !ofType, + diff = false; + + if ( parent ) { + + // :(first|last|only)-(child|of-type) + if ( simple ) { + while ( dir ) { + node = elem; + while ( ( node = node[ dir ] ) ) { + if ( ofType ? + node.nodeName.toLowerCase() === name : + node.nodeType === 1 ) { + + return false; + } + } + + // Reverse direction for :only-* (if we haven't yet done so) + start = dir = type === "only" && !start && "nextSibling"; + } + return true; + } + + start = [ forward ? parent.firstChild : parent.lastChild ]; + + // non-xml :nth-child(...) stores cache data on `parent` + if ( forward && useCache ) { + + // Seek `elem` from a previously-cached index + + // ...in a gzip-friendly way + node = parent; + outerCache = node[ expando ] || ( node[ expando ] = {} ); + + // Support: IE <9 only + // Defend against cloned attroperties (jQuery gh-1709) + uniqueCache = outerCache[ node.uniqueID ] || + ( outerCache[ node.uniqueID ] = {} ); + + cache = uniqueCache[ type ] || []; + nodeIndex = cache[ 0 ] === dirruns && cache[ 1 ]; + diff = nodeIndex && cache[ 2 ]; + node = nodeIndex && parent.childNodes[ nodeIndex ]; + + while ( ( node = ++nodeIndex && node && node[ dir ] || + + // Fallback to seeking `elem` from the start + ( diff = nodeIndex = 0 ) || start.pop() ) ) { + + // When found, cache indexes on `parent` and break + if ( node.nodeType === 1 && ++diff && node === elem ) { + uniqueCache[ type ] = [ dirruns, nodeIndex, diff ]; + break; + } + } + + } else { + + // Use previously-cached element index if available + if ( useCache ) { + + // ...in a gzip-friendly way + node = elem; + outerCache = node[ expando ] || ( node[ expando ] = {} ); + + // Support: IE <9 only + // Defend against cloned attroperties (jQuery gh-1709) + uniqueCache = outerCache[ node.uniqueID ] || + ( outerCache[ node.uniqueID ] = {} ); + + cache = uniqueCache[ type ] || []; + nodeIndex = cache[ 0 ] === dirruns && cache[ 1 ]; + diff = nodeIndex; + } + + // xml :nth-child(...) + // or :nth-last-child(...) or :nth(-last)?-of-type(...) + if ( diff === false ) { + + // Use the same loop as above to seek `elem` from the start + while ( ( node = ++nodeIndex && node && node[ dir ] || + ( diff = nodeIndex = 0 ) || start.pop() ) ) { + + if ( ( ofType ? + node.nodeName.toLowerCase() === name : + node.nodeType === 1 ) && + ++diff ) { + + // Cache the index of each encountered element + if ( useCache ) { + outerCache = node[ expando ] || + ( node[ expando ] = {} ); + + // Support: IE <9 only + // Defend against cloned attroperties (jQuery gh-1709) + uniqueCache = outerCache[ node.uniqueID ] || + ( outerCache[ node.uniqueID ] = {} ); + + uniqueCache[ type ] = [ dirruns, diff ]; + } + + if ( node === elem ) { + break; + } + } + } + } + } + + // Incorporate the offset, then check against cycle size + diff -= last; + return diff === first || ( diff % first === 0 && diff / first >= 0 ); + } + }; + }, + + "PSEUDO": function( pseudo, argument ) { + + // pseudo-class names are case-insensitive + // http://www.w3.org/TR/selectors/#pseudo-classes + // Prioritize by case sensitivity in case custom pseudos are added with uppercase letters + // Remember that setFilters inherits from pseudos + var args, + fn = Expr.pseudos[ pseudo ] || Expr.setFilters[ pseudo.toLowerCase() ] || + Sizzle.error( "unsupported pseudo: " + pseudo ); + + // The user may use createPseudo to indicate that + // arguments are needed to create the filter function + // just as Sizzle does + if ( fn[ expando ] ) { + return fn( argument ); + } + + // But maintain support for old signatures + if ( fn.length > 1 ) { + args = [ pseudo, pseudo, "", argument ]; + return Expr.setFilters.hasOwnProperty( pseudo.toLowerCase() ) ? + markFunction( function( seed, matches ) { + var idx, + matched = fn( seed, argument ), + i = matched.length; + while ( i-- ) { + idx = indexOf( seed, matched[ i ] ); + seed[ idx ] = !( matches[ idx ] = matched[ i ] ); + } + } ) : + function( elem ) { + return fn( elem, 0, args ); + }; + } + + return fn; + } + }, + + pseudos: { + + // Potentially complex pseudos + "not": markFunction( function( selector ) { + + // Trim the selector passed to compile + // to avoid treating leading and trailing + // spaces as combinators + var input = [], + results = [], + matcher = compile( selector.replace( rtrim, "$1" ) ); + + return matcher[ expando ] ? + markFunction( function( seed, matches, _context, xml ) { + var elem, + unmatched = matcher( seed, null, xml, [] ), + i = seed.length; + + // Match elements unmatched by `matcher` + while ( i-- ) { + if ( ( elem = unmatched[ i ] ) ) { + seed[ i ] = !( matches[ i ] = elem ); + } + } + } ) : + function( elem, _context, xml ) { + input[ 0 ] = elem; + matcher( input, null, xml, results ); + + // Don't keep the element (issue #299) + input[ 0 ] = null; + return !results.pop(); + }; + } ), + + "has": markFunction( function( selector ) { + return function( elem ) { + return Sizzle( selector, elem ).length > 0; + }; + } ), + + "contains": markFunction( function( text ) { + text = text.replace( runescape, funescape ); + return function( elem ) { + return ( elem.textContent || getText( elem ) ).indexOf( text ) > -1; + }; + } ), + + // "Whether an element is represented by a :lang() selector + // is based solely on the element's language value + // being equal to the identifier C, + // or beginning with the identifier C immediately followed by "-". + // The matching of C against the element's language value is performed case-insensitively. + // The identifier C does not have to be a valid language name." + // http://www.w3.org/TR/selectors/#lang-pseudo + "lang": markFunction( function( lang ) { + + // lang value must be a valid identifier + if ( !ridentifier.test( lang || "" ) ) { + Sizzle.error( "unsupported lang: " + lang ); + } + lang = lang.replace( runescape, funescape ).toLowerCase(); + return function( elem ) { + var elemLang; + do { + if ( ( elemLang = documentIsHTML ? + elem.lang : + elem.getAttribute( "xml:lang" ) || elem.getAttribute( "lang" ) ) ) { + + elemLang = elemLang.toLowerCase(); + return elemLang === lang || elemLang.indexOf( lang + "-" ) === 0; + } + } while ( ( elem = elem.parentNode ) && elem.nodeType === 1 ); + return false; + }; + } ), + + // Miscellaneous + "target": function( elem ) { + var hash = window.location && window.location.hash; + return hash && hash.slice( 1 ) === elem.id; + }, + + "root": function( elem ) { + return elem === docElem; + }, + + "focus": function( elem ) { + return elem === document.activeElement && + ( !document.hasFocus || document.hasFocus() ) && + !!( elem.type || elem.href || ~elem.tabIndex ); + }, + + // Boolean properties + "enabled": createDisabledPseudo( false ), + "disabled": createDisabledPseudo( true ), + + "checked": function( elem ) { + + // In CSS3, :checked should return both checked and selected elements + // http://www.w3.org/TR/2011/REC-css3-selectors-20110929/#checked + var nodeName = elem.nodeName.toLowerCase(); + return ( nodeName === "input" && !!elem.checked ) || + ( nodeName === "option" && !!elem.selected ); + }, + + "selected": function( elem ) { + + // Accessing this property makes selected-by-default + // options in Safari work properly + if ( elem.parentNode ) { + // eslint-disable-next-line no-unused-expressions + elem.parentNode.selectedIndex; + } + + return elem.selected === true; + }, + + // Contents + "empty": function( elem ) { + + // http://www.w3.org/TR/selectors/#empty-pseudo + // :empty is negated by element (1) or content nodes (text: 3; cdata: 4; entity ref: 5), + // but not by others (comment: 8; processing instruction: 7; etc.) + // nodeType < 6 works because attributes (2) do not appear as children + for ( elem = elem.firstChild; elem; elem = elem.nextSibling ) { + if ( elem.nodeType < 6 ) { + return false; + } + } + return true; + }, + + "parent": function( elem ) { + return !Expr.pseudos[ "empty" ]( elem ); + }, + + // Element/input types + "header": function( elem ) { + return rheader.test( elem.nodeName ); + }, + + "input": function( elem ) { + return rinputs.test( elem.nodeName ); + }, + + "button": function( elem ) { + var name = elem.nodeName.toLowerCase(); + return name === "input" && elem.type === "button" || name === "button"; + }, + + "text": function( elem ) { + var attr; + return elem.nodeName.toLowerCase() === "input" && + elem.type === "text" && + + // Support: IE<8 + // New HTML5 attribute values (e.g., "search") appear with elem.type === "text" + ( ( attr = elem.getAttribute( "type" ) ) == null || + attr.toLowerCase() === "text" ); + }, + + // Position-in-collection + "first": createPositionalPseudo( function() { + return [ 0 ]; + } ), + + "last": createPositionalPseudo( function( _matchIndexes, length ) { + return [ length - 1 ]; + } ), + + "eq": createPositionalPseudo( function( _matchIndexes, length, argument ) { + return [ argument < 0 ? argument + length : argument ]; + } ), + + "even": createPositionalPseudo( function( matchIndexes, length ) { + var i = 0; + for ( ; i < length; i += 2 ) { + matchIndexes.push( i ); + } + return matchIndexes; + } ), + + "odd": createPositionalPseudo( function( matchIndexes, length ) { + var i = 1; + for ( ; i < length; i += 2 ) { + matchIndexes.push( i ); + } + return matchIndexes; + } ), + + "lt": createPositionalPseudo( function( matchIndexes, length, argument ) { + var i = argument < 0 ? + argument + length : + argument > length ? + length : + argument; + for ( ; --i >= 0; ) { + matchIndexes.push( i ); + } + return matchIndexes; + } ), + + "gt": createPositionalPseudo( function( matchIndexes, length, argument ) { + var i = argument < 0 ? argument + length : argument; + for ( ; ++i < length; ) { + matchIndexes.push( i ); + } + return matchIndexes; + } ) + } +}; + +Expr.pseudos[ "nth" ] = Expr.pseudos[ "eq" ]; + +// Add button/input type pseudos +for ( i in { radio: true, checkbox: true, file: true, password: true, image: true } ) { + Expr.pseudos[ i ] = createInputPseudo( i ); +} +for ( i in { submit: true, reset: true } ) { + Expr.pseudos[ i ] = createButtonPseudo( i ); +} + +// Easy API for creating new setFilters +function setFilters() {} +setFilters.prototype = Expr.filters = Expr.pseudos; +Expr.setFilters = new setFilters(); + +tokenize = Sizzle.tokenize = function( selector, parseOnly ) { + var matched, match, tokens, type, + soFar, groups, preFilters, + cached = tokenCache[ selector + " " ]; + + if ( cached ) { + return parseOnly ? 0 : cached.slice( 0 ); + } + + soFar = selector; + groups = []; + preFilters = Expr.preFilter; + + while ( soFar ) { + + // Comma and first run + if ( !matched || ( match = rcomma.exec( soFar ) ) ) { + if ( match ) { + + // Don't consume trailing commas as valid + soFar = soFar.slice( match[ 0 ].length ) || soFar; + } + groups.push( ( tokens = [] ) ); + } + + matched = false; + + // Combinators + if ( ( match = rcombinators.exec( soFar ) ) ) { + matched = match.shift(); + tokens.push( { + value: matched, + + // Cast descendant combinators to space + type: match[ 0 ].replace( rtrim, " " ) + } ); + soFar = soFar.slice( matched.length ); + } + + // Filters + for ( type in Expr.filter ) { + if ( ( match = matchExpr[ type ].exec( soFar ) ) && ( !preFilters[ type ] || + ( match = preFilters[ type ]( match ) ) ) ) { + matched = match.shift(); + tokens.push( { + value: matched, + type: type, + matches: match + } ); + soFar = soFar.slice( matched.length ); + } + } + + if ( !matched ) { + break; + } + } + + // Return the length of the invalid excess + // if we're just parsing + // Otherwise, throw an error or return tokens + return parseOnly ? + soFar.length : + soFar ? + Sizzle.error( selector ) : + + // Cache the tokens + tokenCache( selector, groups ).slice( 0 ); +}; + +function toSelector( tokens ) { + var i = 0, + len = tokens.length, + selector = ""; + for ( ; i < len; i++ ) { + selector += tokens[ i ].value; + } + return selector; +} + +function addCombinator( matcher, combinator, base ) { + var dir = combinator.dir, + skip = combinator.next, + key = skip || dir, + checkNonElements = base && key === "parentNode", + doneName = done++; + + return combinator.first ? + + // Check against closest ancestor/preceding element + function( elem, context, xml ) { + while ( ( elem = elem[ dir ] ) ) { + if ( elem.nodeType === 1 || checkNonElements ) { + return matcher( elem, context, xml ); + } + } + return false; + } : + + // Check against all ancestor/preceding elements + function( elem, context, xml ) { + var oldCache, uniqueCache, outerCache, + newCache = [ dirruns, doneName ]; + + // We can't set arbitrary data on XML nodes, so they don't benefit from combinator caching + if ( xml ) { + while ( ( elem = elem[ dir ] ) ) { + if ( elem.nodeType === 1 || checkNonElements ) { + if ( matcher( elem, context, xml ) ) { + return true; + } + } + } + } else { + while ( ( elem = elem[ dir ] ) ) { + if ( elem.nodeType === 1 || checkNonElements ) { + outerCache = elem[ expando ] || ( elem[ expando ] = {} ); + + // Support: IE <9 only + // Defend against cloned attroperties (jQuery gh-1709) + uniqueCache = outerCache[ elem.uniqueID ] || + ( outerCache[ elem.uniqueID ] = {} ); + + if ( skip && skip === elem.nodeName.toLowerCase() ) { + elem = elem[ dir ] || elem; + } else if ( ( oldCache = uniqueCache[ key ] ) && + oldCache[ 0 ] === dirruns && oldCache[ 1 ] === doneName ) { + + // Assign to newCache so results back-propagate to previous elements + return ( newCache[ 2 ] = oldCache[ 2 ] ); + } else { + + // Reuse newcache so results back-propagate to previous elements + uniqueCache[ key ] = newCache; + + // A match means we're done; a fail means we have to keep checking + if ( ( newCache[ 2 ] = matcher( elem, context, xml ) ) ) { + return true; + } + } + } + } + } + return false; + }; +} + +function elementMatcher( matchers ) { + return matchers.length > 1 ? + function( elem, context, xml ) { + var i = matchers.length; + while ( i-- ) { + if ( !matchers[ i ]( elem, context, xml ) ) { + return false; + } + } + return true; + } : + matchers[ 0 ]; +} + +function multipleContexts( selector, contexts, results ) { + var i = 0, + len = contexts.length; + for ( ; i < len; i++ ) { + Sizzle( selector, contexts[ i ], results ); + } + return results; +} + +function condense( unmatched, map, filter, context, xml ) { + var elem, + newUnmatched = [], + i = 0, + len = unmatched.length, + mapped = map != null; + + for ( ; i < len; i++ ) { + if ( ( elem = unmatched[ i ] ) ) { + if ( !filter || filter( elem, context, xml ) ) { + newUnmatched.push( elem ); + if ( mapped ) { + map.push( i ); + } + } + } + } + + return newUnmatched; +} + +function setMatcher( preFilter, selector, matcher, postFilter, postFinder, postSelector ) { + if ( postFilter && !postFilter[ expando ] ) { + postFilter = setMatcher( postFilter ); + } + if ( postFinder && !postFinder[ expando ] ) { + postFinder = setMatcher( postFinder, postSelector ); + } + return markFunction( function( seed, results, context, xml ) { + var temp, i, elem, + preMap = [], + postMap = [], + preexisting = results.length, + + // Get initial elements from seed or context + elems = seed || multipleContexts( + selector || "*", + context.nodeType ? [ context ] : context, + [] + ), + + // Prefilter to get matcher input, preserving a map for seed-results synchronization + matcherIn = preFilter && ( seed || !selector ) ? + condense( elems, preMap, preFilter, context, xml ) : + elems, + + matcherOut = matcher ? + + // If we have a postFinder, or filtered seed, or non-seed postFilter or preexisting results, + postFinder || ( seed ? preFilter : preexisting || postFilter ) ? + + // ...intermediate processing is necessary + [] : + + // ...otherwise use results directly + results : + matcherIn; + + // Find primary matches + if ( matcher ) { + matcher( matcherIn, matcherOut, context, xml ); + } + + // Apply postFilter + if ( postFilter ) { + temp = condense( matcherOut, postMap ); + postFilter( temp, [], context, xml ); + + // Un-match failing elements by moving them back to matcherIn + i = temp.length; + while ( i-- ) { + if ( ( elem = temp[ i ] ) ) { + matcherOut[ postMap[ i ] ] = !( matcherIn[ postMap[ i ] ] = elem ); + } + } + } + + if ( seed ) { + if ( postFinder || preFilter ) { + if ( postFinder ) { + + // Get the final matcherOut by condensing this intermediate into postFinder contexts + temp = []; + i = matcherOut.length; + while ( i-- ) { + if ( ( elem = matcherOut[ i ] ) ) { + + // Restore matcherIn since elem is not yet a final match + temp.push( ( matcherIn[ i ] = elem ) ); + } + } + postFinder( null, ( matcherOut = [] ), temp, xml ); + } + + // Move matched elements from seed to results to keep them synchronized + i = matcherOut.length; + while ( i-- ) { + if ( ( elem = matcherOut[ i ] ) && + ( temp = postFinder ? indexOf( seed, elem ) : preMap[ i ] ) > -1 ) { + + seed[ temp ] = !( results[ temp ] = elem ); + } + } + } + + // Add elements to results, through postFinder if defined + } else { + matcherOut = condense( + matcherOut === results ? + matcherOut.splice( preexisting, matcherOut.length ) : + matcherOut + ); + if ( postFinder ) { + postFinder( null, results, matcherOut, xml ); + } else { + push.apply( results, matcherOut ); + } + } + } ); +} + +function matcherFromTokens( tokens ) { + var checkContext, matcher, j, + len = tokens.length, + leadingRelative = Expr.relative[ tokens[ 0 ].type ], + implicitRelative = leadingRelative || Expr.relative[ " " ], + i = leadingRelative ? 1 : 0, + + // The foundational matcher ensures that elements are reachable from top-level context(s) + matchContext = addCombinator( function( elem ) { + return elem === checkContext; + }, implicitRelative, true ), + matchAnyContext = addCombinator( function( elem ) { + return indexOf( checkContext, elem ) > -1; + }, implicitRelative, true ), + matchers = [ function( elem, context, xml ) { + var ret = ( !leadingRelative && ( xml || context !== outermostContext ) ) || ( + ( checkContext = context ).nodeType ? + matchContext( elem, context, xml ) : + matchAnyContext( elem, context, xml ) ); + + // Avoid hanging onto element (issue #299) + checkContext = null; + return ret; + } ]; + + for ( ; i < len; i++ ) { + if ( ( matcher = Expr.relative[ tokens[ i ].type ] ) ) { + matchers = [ addCombinator( elementMatcher( matchers ), matcher ) ]; + } else { + matcher = Expr.filter[ tokens[ i ].type ].apply( null, tokens[ i ].matches ); + + // Return special upon seeing a positional matcher + if ( matcher[ expando ] ) { + + // Find the next relative operator (if any) for proper handling + j = ++i; + for ( ; j < len; j++ ) { + if ( Expr.relative[ tokens[ j ].type ] ) { + break; + } + } + return setMatcher( + i > 1 && elementMatcher( matchers ), + i > 1 && toSelector( + + // If the preceding token was a descendant combinator, insert an implicit any-element `*` + tokens + .slice( 0, i - 1 ) + .concat( { value: tokens[ i - 2 ].type === " " ? "*" : "" } ) + ).replace( rtrim, "$1" ), + matcher, + i < j && matcherFromTokens( tokens.slice( i, j ) ), + j < len && matcherFromTokens( ( tokens = tokens.slice( j ) ) ), + j < len && toSelector( tokens ) + ); + } + matchers.push( matcher ); + } + } + + return elementMatcher( matchers ); +} + +function matcherFromGroupMatchers( elementMatchers, setMatchers ) { + var bySet = setMatchers.length > 0, + byElement = elementMatchers.length > 0, + superMatcher = function( seed, context, xml, results, outermost ) { + var elem, j, matcher, + matchedCount = 0, + i = "0", + unmatched = seed && [], + setMatched = [], + contextBackup = outermostContext, + + // We must always have either seed elements or outermost context + elems = seed || byElement && Expr.find[ "TAG" ]( "*", outermost ), + + // Use integer dirruns iff this is the outermost matcher + dirrunsUnique = ( dirruns += contextBackup == null ? 1 : Math.random() || 0.1 ), + len = elems.length; + + if ( outermost ) { + + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + outermostContext = context == document || context || outermost; + } + + // Add elements passing elementMatchers directly to results + // Support: IE<9, Safari + // Tolerate NodeList properties (IE: "length"; Safari: ) matching elements by id + for ( ; i !== len && ( elem = elems[ i ] ) != null; i++ ) { + if ( byElement && elem ) { + j = 0; + + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( !context && elem.ownerDocument != document ) { + setDocument( elem ); + xml = !documentIsHTML; + } + while ( ( matcher = elementMatchers[ j++ ] ) ) { + if ( matcher( elem, context || document, xml ) ) { + results.push( elem ); + break; + } + } + if ( outermost ) { + dirruns = dirrunsUnique; + } + } + + // Track unmatched elements for set filters + if ( bySet ) { + + // They will have gone through all possible matchers + if ( ( elem = !matcher && elem ) ) { + matchedCount--; + } + + // Lengthen the array for every element, matched or not + if ( seed ) { + unmatched.push( elem ); + } + } + } + + // `i` is now the count of elements visited above, and adding it to `matchedCount` + // makes the latter nonnegative. + matchedCount += i; + + // Apply set filters to unmatched elements + // NOTE: This can be skipped if there are no unmatched elements (i.e., `matchedCount` + // equals `i`), unless we didn't visit _any_ elements in the above loop because we have + // no element matchers and no seed. + // Incrementing an initially-string "0" `i` allows `i` to remain a string only in that + // case, which will result in a "00" `matchedCount` that differs from `i` but is also + // numerically zero. + if ( bySet && i !== matchedCount ) { + j = 0; + while ( ( matcher = setMatchers[ j++ ] ) ) { + matcher( unmatched, setMatched, context, xml ); + } + + if ( seed ) { + + // Reintegrate element matches to eliminate the need for sorting + if ( matchedCount > 0 ) { + while ( i-- ) { + if ( !( unmatched[ i ] || setMatched[ i ] ) ) { + setMatched[ i ] = pop.call( results ); + } + } + } + + // Discard index placeholder values to get only actual matches + setMatched = condense( setMatched ); + } + + // Add matches to results + push.apply( results, setMatched ); + + // Seedless set matches succeeding multiple successful matchers stipulate sorting + if ( outermost && !seed && setMatched.length > 0 && + ( matchedCount + setMatchers.length ) > 1 ) { + + Sizzle.uniqueSort( results ); + } + } + + // Override manipulation of globals by nested matchers + if ( outermost ) { + dirruns = dirrunsUnique; + outermostContext = contextBackup; + } + + return unmatched; + }; + + return bySet ? + markFunction( superMatcher ) : + superMatcher; +} + +compile = Sizzle.compile = function( selector, match /* Internal Use Only */ ) { + var i, + setMatchers = [], + elementMatchers = [], + cached = compilerCache[ selector + " " ]; + + if ( !cached ) { + + // Generate a function of recursive functions that can be used to check each element + if ( !match ) { + match = tokenize( selector ); + } + i = match.length; + while ( i-- ) { + cached = matcherFromTokens( match[ i ] ); + if ( cached[ expando ] ) { + setMatchers.push( cached ); + } else { + elementMatchers.push( cached ); + } + } + + // Cache the compiled function + cached = compilerCache( + selector, + matcherFromGroupMatchers( elementMatchers, setMatchers ) + ); + + // Save selector and tokenization + cached.selector = selector; + } + return cached; +}; + +/** + * A low-level selection function that works with Sizzle's compiled + * selector functions + * @param {String|Function} selector A selector or a pre-compiled + * selector function built with Sizzle.compile + * @param {Element} context + * @param {Array} [results] + * @param {Array} [seed] A set of elements to match against + */ +select = Sizzle.select = function( selector, context, results, seed ) { + var i, tokens, token, type, find, + compiled = typeof selector === "function" && selector, + match = !seed && tokenize( ( selector = compiled.selector || selector ) ); + + results = results || []; + + // Try to minimize operations if there is only one selector in the list and no seed + // (the latter of which guarantees us context) + if ( match.length === 1 ) { + + // Reduce context if the leading compound selector is an ID + tokens = match[ 0 ] = match[ 0 ].slice( 0 ); + if ( tokens.length > 2 && ( token = tokens[ 0 ] ).type === "ID" && + context.nodeType === 9 && documentIsHTML && Expr.relative[ tokens[ 1 ].type ] ) { + + context = ( Expr.find[ "ID" ]( token.matches[ 0 ] + .replace( runescape, funescape ), context ) || [] )[ 0 ]; + if ( !context ) { + return results; + + // Precompiled matchers will still verify ancestry, so step up a level + } else if ( compiled ) { + context = context.parentNode; + } + + selector = selector.slice( tokens.shift().value.length ); + } + + // Fetch a seed set for right-to-left matching + i = matchExpr[ "needsContext" ].test( selector ) ? 0 : tokens.length; + while ( i-- ) { + token = tokens[ i ]; + + // Abort if we hit a combinator + if ( Expr.relative[ ( type = token.type ) ] ) { + break; + } + if ( ( find = Expr.find[ type ] ) ) { + + // Search, expanding context for leading sibling combinators + if ( ( seed = find( + token.matches[ 0 ].replace( runescape, funescape ), + rsibling.test( tokens[ 0 ].type ) && testContext( context.parentNode ) || + context + ) ) ) { + + // If seed is empty or no tokens remain, we can return early + tokens.splice( i, 1 ); + selector = seed.length && toSelector( tokens ); + if ( !selector ) { + push.apply( results, seed ); + return results; + } + + break; + } + } + } + } + + // Compile and execute a filtering function if one is not provided + // Provide `match` to avoid retokenization if we modified the selector above + ( compiled || compile( selector, match ) )( + seed, + context, + !documentIsHTML, + results, + !context || rsibling.test( selector ) && testContext( context.parentNode ) || context + ); + return results; +}; + +// One-time assignments + +// Sort stability +support.sortStable = expando.split( "" ).sort( sortOrder ).join( "" ) === expando; + +// Support: Chrome 14-35+ +// Always assume duplicates if they aren't passed to the comparison function +support.detectDuplicates = !!hasDuplicate; + +// Initialize against the default document +setDocument(); + +// Support: Webkit<537.32 - Safari 6.0.3/Chrome 25 (fixed in Chrome 27) +// Detached nodes confoundingly follow *each other* +support.sortDetached = assert( function( el ) { + + // Should return 1, but returns 4 (following) + return el.compareDocumentPosition( document.createElement( "fieldset" ) ) & 1; +} ); + +// Support: IE<8 +// Prevent attribute/property "interpolation" +// https://msdn.microsoft.com/en-us/library/ms536429%28VS.85%29.aspx +if ( !assert( function( el ) { + el.innerHTML = ""; + return el.firstChild.getAttribute( "href" ) === "#"; +} ) ) { + addHandle( "type|href|height|width", function( elem, name, isXML ) { + if ( !isXML ) { + return elem.getAttribute( name, name.toLowerCase() === "type" ? 1 : 2 ); + } + } ); +} + +// Support: IE<9 +// Use defaultValue in place of getAttribute("value") +if ( !support.attributes || !assert( function( el ) { + el.innerHTML = ""; + el.firstChild.setAttribute( "value", "" ); + return el.firstChild.getAttribute( "value" ) === ""; +} ) ) { + addHandle( "value", function( elem, _name, isXML ) { + if ( !isXML && elem.nodeName.toLowerCase() === "input" ) { + return elem.defaultValue; + } + } ); +} + +// Support: IE<9 +// Use getAttributeNode to fetch booleans when getAttribute lies +if ( !assert( function( el ) { + return el.getAttribute( "disabled" ) == null; +} ) ) { + addHandle( booleans, function( elem, name, isXML ) { + var val; + if ( !isXML ) { + return elem[ name ] === true ? name.toLowerCase() : + ( val = elem.getAttributeNode( name ) ) && val.specified ? + val.value : + null; + } + } ); +} + +return Sizzle; + +} )( window ); + + + +jQuery.find = Sizzle; +jQuery.expr = Sizzle.selectors; + +// Deprecated +jQuery.expr[ ":" ] = jQuery.expr.pseudos; +jQuery.uniqueSort = jQuery.unique = Sizzle.uniqueSort; +jQuery.text = Sizzle.getText; +jQuery.isXMLDoc = Sizzle.isXML; +jQuery.contains = Sizzle.contains; +jQuery.escapeSelector = Sizzle.escape; + + + + +var dir = function( elem, dir, until ) { + var matched = [], + truncate = until !== undefined; + + while ( ( elem = elem[ dir ] ) && elem.nodeType !== 9 ) { + if ( elem.nodeType === 1 ) { + if ( truncate && jQuery( elem ).is( until ) ) { + break; + } + matched.push( elem ); + } + } + return matched; +}; + + +var siblings = function( n, elem ) { + var matched = []; + + for ( ; n; n = n.nextSibling ) { + if ( n.nodeType === 1 && n !== elem ) { + matched.push( n ); + } + } + + return matched; +}; + + +var rneedsContext = jQuery.expr.match.needsContext; + + + +function nodeName( elem, name ) { + + return elem.nodeName && elem.nodeName.toLowerCase() === name.toLowerCase(); + +}; +var rsingleTag = ( /^<([a-z][^\/\0>:\x20\t\r\n\f]*)[\x20\t\r\n\f]*\/?>(?:<\/\1>|)$/i ); + + + +// Implement the identical functionality for filter and not +function winnow( elements, qualifier, not ) { + if ( isFunction( qualifier ) ) { + return jQuery.grep( elements, function( elem, i ) { + return !!qualifier.call( elem, i, elem ) !== not; + } ); + } + + // Single element + if ( qualifier.nodeType ) { + return jQuery.grep( elements, function( elem ) { + return ( elem === qualifier ) !== not; + } ); + } + + // Arraylike of elements (jQuery, arguments, Array) + if ( typeof qualifier !== "string" ) { + return jQuery.grep( elements, function( elem ) { + return ( indexOf.call( qualifier, elem ) > -1 ) !== not; + } ); + } + + // Filtered directly for both simple and complex selectors + return jQuery.filter( qualifier, elements, not ); +} + +jQuery.filter = function( expr, elems, not ) { + var elem = elems[ 0 ]; + + if ( not ) { + expr = ":not(" + expr + ")"; + } + + if ( elems.length === 1 && elem.nodeType === 1 ) { + return jQuery.find.matchesSelector( elem, expr ) ? [ elem ] : []; + } + + return jQuery.find.matches( expr, jQuery.grep( elems, function( elem ) { + return elem.nodeType === 1; + } ) ); +}; + +jQuery.fn.extend( { + find: function( selector ) { + var i, ret, + len = this.length, + self = this; + + if ( typeof selector !== "string" ) { + return this.pushStack( jQuery( selector ).filter( function() { + for ( i = 0; i < len; i++ ) { + if ( jQuery.contains( self[ i ], this ) ) { + return true; + } + } + } ) ); + } + + ret = this.pushStack( [] ); + + for ( i = 0; i < len; i++ ) { + jQuery.find( selector, self[ i ], ret ); + } + + return len > 1 ? jQuery.uniqueSort( ret ) : ret; + }, + filter: function( selector ) { + return this.pushStack( winnow( this, selector || [], false ) ); + }, + not: function( selector ) { + return this.pushStack( winnow( this, selector || [], true ) ); + }, + is: function( selector ) { + return !!winnow( + this, + + // If this is a positional/relative selector, check membership in the returned set + // so $("p:first").is("p:last") won't return true for a doc with two "p". + typeof selector === "string" && rneedsContext.test( selector ) ? + jQuery( selector ) : + selector || [], + false + ).length; + } +} ); + + +// Initialize a jQuery object + + +// A central reference to the root jQuery(document) +var rootjQuery, + + // A simple way to check for HTML strings + // Prioritize #id over to avoid XSS via location.hash (#9521) + // Strict HTML recognition (#11290: must start with <) + // Shortcut simple #id case for speed + rquickExpr = /^(?:\s*(<[\w\W]+>)[^>]*|#([\w-]+))$/, + + init = jQuery.fn.init = function( selector, context, root ) { + var match, elem; + + // HANDLE: $(""), $(null), $(undefined), $(false) + if ( !selector ) { + return this; + } + + // Method init() accepts an alternate rootjQuery + // so migrate can support jQuery.sub (gh-2101) + root = root || rootjQuery; + + // Handle HTML strings + if ( typeof selector === "string" ) { + if ( selector[ 0 ] === "<" && + selector[ selector.length - 1 ] === ">" && + selector.length >= 3 ) { + + // Assume that strings that start and end with <> are HTML and skip the regex check + match = [ null, selector, null ]; + + } else { + match = rquickExpr.exec( selector ); + } + + // Match html or make sure no context is specified for #id + if ( match && ( match[ 1 ] || !context ) ) { + + // HANDLE: $(html) -> $(array) + if ( match[ 1 ] ) { + context = context instanceof jQuery ? context[ 0 ] : context; + + // Option to run scripts is true for back-compat + // Intentionally let the error be thrown if parseHTML is not present + jQuery.merge( this, jQuery.parseHTML( + match[ 1 ], + context && context.nodeType ? context.ownerDocument || context : document, + true + ) ); + + // HANDLE: $(html, props) + if ( rsingleTag.test( match[ 1 ] ) && jQuery.isPlainObject( context ) ) { + for ( match in context ) { + + // Properties of context are called as methods if possible + if ( isFunction( this[ match ] ) ) { + this[ match ]( context[ match ] ); + + // ...and otherwise set as attributes + } else { + this.attr( match, context[ match ] ); + } + } + } + + return this; + + // HANDLE: $(#id) + } else { + elem = document.getElementById( match[ 2 ] ); + + if ( elem ) { + + // Inject the element directly into the jQuery object + this[ 0 ] = elem; + this.length = 1; + } + return this; + } + + // HANDLE: $(expr, $(...)) + } else if ( !context || context.jquery ) { + return ( context || root ).find( selector ); + + // HANDLE: $(expr, context) + // (which is just equivalent to: $(context).find(expr) + } else { + return this.constructor( context ).find( selector ); + } + + // HANDLE: $(DOMElement) + } else if ( selector.nodeType ) { + this[ 0 ] = selector; + this.length = 1; + return this; + + // HANDLE: $(function) + // Shortcut for document ready + } else if ( isFunction( selector ) ) { + return root.ready !== undefined ? + root.ready( selector ) : + + // Execute immediately if ready is not present + selector( jQuery ); + } + + return jQuery.makeArray( selector, this ); + }; + +// Give the init function the jQuery prototype for later instantiation +init.prototype = jQuery.fn; + +// Initialize central reference +rootjQuery = jQuery( document ); + + +var rparentsprev = /^(?:parents|prev(?:Until|All))/, + + // Methods guaranteed to produce a unique set when starting from a unique set + guaranteedUnique = { + children: true, + contents: true, + next: true, + prev: true + }; + +jQuery.fn.extend( { + has: function( target ) { + var targets = jQuery( target, this ), + l = targets.length; + + return this.filter( function() { + var i = 0; + for ( ; i < l; i++ ) { + if ( jQuery.contains( this, targets[ i ] ) ) { + return true; + } + } + } ); + }, + + closest: function( selectors, context ) { + var cur, + i = 0, + l = this.length, + matched = [], + targets = typeof selectors !== "string" && jQuery( selectors ); + + // Positional selectors never match, since there's no _selection_ context + if ( !rneedsContext.test( selectors ) ) { + for ( ; i < l; i++ ) { + for ( cur = this[ i ]; cur && cur !== context; cur = cur.parentNode ) { + + // Always skip document fragments + if ( cur.nodeType < 11 && ( targets ? + targets.index( cur ) > -1 : + + // Don't pass non-elements to Sizzle + cur.nodeType === 1 && + jQuery.find.matchesSelector( cur, selectors ) ) ) { + + matched.push( cur ); + break; + } + } + } + } + + return this.pushStack( matched.length > 1 ? jQuery.uniqueSort( matched ) : matched ); + }, + + // Determine the position of an element within the set + index: function( elem ) { + + // No argument, return index in parent + if ( !elem ) { + return ( this[ 0 ] && this[ 0 ].parentNode ) ? this.first().prevAll().length : -1; + } + + // Index in selector + if ( typeof elem === "string" ) { + return indexOf.call( jQuery( elem ), this[ 0 ] ); + } + + // Locate the position of the desired element + return indexOf.call( this, + + // If it receives a jQuery object, the first element is used + elem.jquery ? elem[ 0 ] : elem + ); + }, + + add: function( selector, context ) { + return this.pushStack( + jQuery.uniqueSort( + jQuery.merge( this.get(), jQuery( selector, context ) ) + ) + ); + }, + + addBack: function( selector ) { + return this.add( selector == null ? + this.prevObject : this.prevObject.filter( selector ) + ); + } +} ); + +function sibling( cur, dir ) { + while ( ( cur = cur[ dir ] ) && cur.nodeType !== 1 ) {} + return cur; +} + +jQuery.each( { + parent: function( elem ) { + var parent = elem.parentNode; + return parent && parent.nodeType !== 11 ? parent : null; + }, + parents: function( elem ) { + return dir( elem, "parentNode" ); + }, + parentsUntil: function( elem, _i, until ) { + return dir( elem, "parentNode", until ); + }, + next: function( elem ) { + return sibling( elem, "nextSibling" ); + }, + prev: function( elem ) { + return sibling( elem, "previousSibling" ); + }, + nextAll: function( elem ) { + return dir( elem, "nextSibling" ); + }, + prevAll: function( elem ) { + return dir( elem, "previousSibling" ); + }, + nextUntil: function( elem, _i, until ) { + return dir( elem, "nextSibling", until ); + }, + prevUntil: function( elem, _i, until ) { + return dir( elem, "previousSibling", until ); + }, + siblings: function( elem ) { + return siblings( ( elem.parentNode || {} ).firstChild, elem ); + }, + children: function( elem ) { + return siblings( elem.firstChild ); + }, + contents: function( elem ) { + if ( elem.contentDocument != null && + + // Support: IE 11+ + // elements with no `data` attribute has an object + // `contentDocument` with a `null` prototype. + getProto( elem.contentDocument ) ) { + + return elem.contentDocument; + } + + // Support: IE 9 - 11 only, iOS 7 only, Android Browser <=4.3 only + // Treat the template element as a regular one in browsers that + // don't support it. + if ( nodeName( elem, "template" ) ) { + elem = elem.content || elem; + } + + return jQuery.merge( [], elem.childNodes ); + } +}, function( name, fn ) { + jQuery.fn[ name ] = function( until, selector ) { + var matched = jQuery.map( this, fn, until ); + + if ( name.slice( -5 ) !== "Until" ) { + selector = until; + } + + if ( selector && typeof selector === "string" ) { + matched = jQuery.filter( selector, matched ); + } + + if ( this.length > 1 ) { + + // Remove duplicates + if ( !guaranteedUnique[ name ] ) { + jQuery.uniqueSort( matched ); + } + + // Reverse order for parents* and prev-derivatives + if ( rparentsprev.test( name ) ) { + matched.reverse(); + } + } + + return this.pushStack( matched ); + }; +} ); +var rnothtmlwhite = ( /[^\x20\t\r\n\f]+/g ); + + + +// Convert String-formatted options into Object-formatted ones +function createOptions( options ) { + var object = {}; + jQuery.each( options.match( rnothtmlwhite ) || [], function( _, flag ) { + object[ flag ] = true; + } ); + return object; +} + +/* + * Create a callback list using the following parameters: + * + * options: an optional list of space-separated options that will change how + * the callback list behaves or a more traditional option object + * + * By default a callback list will act like an event callback list and can be + * "fired" multiple times. + * + * Possible options: + * + * once: will ensure the callback list can only be fired once (like a Deferred) + * + * memory: will keep track of previous values and will call any callback added + * after the list has been fired right away with the latest "memorized" + * values (like a Deferred) + * + * unique: will ensure a callback can only be added once (no duplicate in the list) + * + * stopOnFalse: interrupt callings when a callback returns false + * + */ +jQuery.Callbacks = function( options ) { + + // Convert options from String-formatted to Object-formatted if needed + // (we check in cache first) + options = typeof options === "string" ? + createOptions( options ) : + jQuery.extend( {}, options ); + + var // Flag to know if list is currently firing + firing, + + // Last fire value for non-forgettable lists + memory, + + // Flag to know if list was already fired + fired, + + // Flag to prevent firing + locked, + + // Actual callback list + list = [], + + // Queue of execution data for repeatable lists + queue = [], + + // Index of currently firing callback (modified by add/remove as needed) + firingIndex = -1, + + // Fire callbacks + fire = function() { + + // Enforce single-firing + locked = locked || options.once; + + // Execute callbacks for all pending executions, + // respecting firingIndex overrides and runtime changes + fired = firing = true; + for ( ; queue.length; firingIndex = -1 ) { + memory = queue.shift(); + while ( ++firingIndex < list.length ) { + + // Run callback and check for early termination + if ( list[ firingIndex ].apply( memory[ 0 ], memory[ 1 ] ) === false && + options.stopOnFalse ) { + + // Jump to end and forget the data so .add doesn't re-fire + firingIndex = list.length; + memory = false; + } + } + } + + // Forget the data if we're done with it + if ( !options.memory ) { + memory = false; + } + + firing = false; + + // Clean up if we're done firing for good + if ( locked ) { + + // Keep an empty list if we have data for future add calls + if ( memory ) { + list = []; + + // Otherwise, this object is spent + } else { + list = ""; + } + } + }, + + // Actual Callbacks object + self = { + + // Add a callback or a collection of callbacks to the list + add: function() { + if ( list ) { + + // If we have memory from a past run, we should fire after adding + if ( memory && !firing ) { + firingIndex = list.length - 1; + queue.push( memory ); + } + + ( function add( args ) { + jQuery.each( args, function( _, arg ) { + if ( isFunction( arg ) ) { + if ( !options.unique || !self.has( arg ) ) { + list.push( arg ); + } + } else if ( arg && arg.length && toType( arg ) !== "string" ) { + + // Inspect recursively + add( arg ); + } + } ); + } )( arguments ); + + if ( memory && !firing ) { + fire(); + } + } + return this; + }, + + // Remove a callback from the list + remove: function() { + jQuery.each( arguments, function( _, arg ) { + var index; + while ( ( index = jQuery.inArray( arg, list, index ) ) > -1 ) { + list.splice( index, 1 ); + + // Handle firing indexes + if ( index <= firingIndex ) { + firingIndex--; + } + } + } ); + return this; + }, + + // Check if a given callback is in the list. + // If no argument is given, return whether or not list has callbacks attached. + has: function( fn ) { + return fn ? + jQuery.inArray( fn, list ) > -1 : + list.length > 0; + }, + + // Remove all callbacks from the list + empty: function() { + if ( list ) { + list = []; + } + return this; + }, + + // Disable .fire and .add + // Abort any current/pending executions + // Clear all callbacks and values + disable: function() { + locked = queue = []; + list = memory = ""; + return this; + }, + disabled: function() { + return !list; + }, + + // Disable .fire + // Also disable .add unless we have memory (since it would have no effect) + // Abort any pending executions + lock: function() { + locked = queue = []; + if ( !memory && !firing ) { + list = memory = ""; + } + return this; + }, + locked: function() { + return !!locked; + }, + + // Call all callbacks with the given context and arguments + fireWith: function( context, args ) { + if ( !locked ) { + args = args || []; + args = [ context, args.slice ? args.slice() : args ]; + queue.push( args ); + if ( !firing ) { + fire(); + } + } + return this; + }, + + // Call all the callbacks with the given arguments + fire: function() { + self.fireWith( this, arguments ); + return this; + }, + + // To know if the callbacks have already been called at least once + fired: function() { + return !!fired; + } + }; + + return self; +}; + + +function Identity( v ) { + return v; +} +function Thrower( ex ) { + throw ex; +} + +function adoptValue( value, resolve, reject, noValue ) { + var method; + + try { + + // Check for promise aspect first to privilege synchronous behavior + if ( value && isFunction( ( method = value.promise ) ) ) { + method.call( value ).done( resolve ).fail( reject ); + + // Other thenables + } else if ( value && isFunction( ( method = value.then ) ) ) { + method.call( value, resolve, reject ); + + // Other non-thenables + } else { + + // Control `resolve` arguments by letting Array#slice cast boolean `noValue` to integer: + // * false: [ value ].slice( 0 ) => resolve( value ) + // * true: [ value ].slice( 1 ) => resolve() + resolve.apply( undefined, [ value ].slice( noValue ) ); + } + + // For Promises/A+, convert exceptions into rejections + // Since jQuery.when doesn't unwrap thenables, we can skip the extra checks appearing in + // Deferred#then to conditionally suppress rejection. + } catch ( value ) { + + // Support: Android 4.0 only + // Strict mode functions invoked without .call/.apply get global-object context + reject.apply( undefined, [ value ] ); + } +} + +jQuery.extend( { + + Deferred: function( func ) { + var tuples = [ + + // action, add listener, callbacks, + // ... .then handlers, argument index, [final state] + [ "notify", "progress", jQuery.Callbacks( "memory" ), + jQuery.Callbacks( "memory" ), 2 ], + [ "resolve", "done", jQuery.Callbacks( "once memory" ), + jQuery.Callbacks( "once memory" ), 0, "resolved" ], + [ "reject", "fail", jQuery.Callbacks( "once memory" ), + jQuery.Callbacks( "once memory" ), 1, "rejected" ] + ], + state = "pending", + promise = { + state: function() { + return state; + }, + always: function() { + deferred.done( arguments ).fail( arguments ); + return this; + }, + "catch": function( fn ) { + return promise.then( null, fn ); + }, + + // Keep pipe for back-compat + pipe: function( /* fnDone, fnFail, fnProgress */ ) { + var fns = arguments; + + return jQuery.Deferred( function( newDefer ) { + jQuery.each( tuples, function( _i, tuple ) { + + // Map tuples (progress, done, fail) to arguments (done, fail, progress) + var fn = isFunction( fns[ tuple[ 4 ] ] ) && fns[ tuple[ 4 ] ]; + + // deferred.progress(function() { bind to newDefer or newDefer.notify }) + // deferred.done(function() { bind to newDefer or newDefer.resolve }) + // deferred.fail(function() { bind to newDefer or newDefer.reject }) + deferred[ tuple[ 1 ] ]( function() { + var returned = fn && fn.apply( this, arguments ); + if ( returned && isFunction( returned.promise ) ) { + returned.promise() + .progress( newDefer.notify ) + .done( newDefer.resolve ) + .fail( newDefer.reject ); + } else { + newDefer[ tuple[ 0 ] + "With" ]( + this, + fn ? [ returned ] : arguments + ); + } + } ); + } ); + fns = null; + } ).promise(); + }, + then: function( onFulfilled, onRejected, onProgress ) { + var maxDepth = 0; + function resolve( depth, deferred, handler, special ) { + return function() { + var that = this, + args = arguments, + mightThrow = function() { + var returned, then; + + // Support: Promises/A+ section 2.3.3.3.3 + // https://promisesaplus.com/#point-59 + // Ignore double-resolution attempts + if ( depth < maxDepth ) { + return; + } + + returned = handler.apply( that, args ); + + // Support: Promises/A+ section 2.3.1 + // https://promisesaplus.com/#point-48 + if ( returned === deferred.promise() ) { + throw new TypeError( "Thenable self-resolution" ); + } + + // Support: Promises/A+ sections 2.3.3.1, 3.5 + // https://promisesaplus.com/#point-54 + // https://promisesaplus.com/#point-75 + // Retrieve `then` only once + then = returned && + + // Support: Promises/A+ section 2.3.4 + // https://promisesaplus.com/#point-64 + // Only check objects and functions for thenability + ( typeof returned === "object" || + typeof returned === "function" ) && + returned.then; + + // Handle a returned thenable + if ( isFunction( then ) ) { + + // Special processors (notify) just wait for resolution + if ( special ) { + then.call( + returned, + resolve( maxDepth, deferred, Identity, special ), + resolve( maxDepth, deferred, Thrower, special ) + ); + + // Normal processors (resolve) also hook into progress + } else { + + // ...and disregard older resolution values + maxDepth++; + + then.call( + returned, + resolve( maxDepth, deferred, Identity, special ), + resolve( maxDepth, deferred, Thrower, special ), + resolve( maxDepth, deferred, Identity, + deferred.notifyWith ) + ); + } + + // Handle all other returned values + } else { + + // Only substitute handlers pass on context + // and multiple values (non-spec behavior) + if ( handler !== Identity ) { + that = undefined; + args = [ returned ]; + } + + // Process the value(s) + // Default process is resolve + ( special || deferred.resolveWith )( that, args ); + } + }, + + // Only normal processors (resolve) catch and reject exceptions + process = special ? + mightThrow : + function() { + try { + mightThrow(); + } catch ( e ) { + + if ( jQuery.Deferred.exceptionHook ) { + jQuery.Deferred.exceptionHook( e, + process.stackTrace ); + } + + // Support: Promises/A+ section 2.3.3.3.4.1 + // https://promisesaplus.com/#point-61 + // Ignore post-resolution exceptions + if ( depth + 1 >= maxDepth ) { + + // Only substitute handlers pass on context + // and multiple values (non-spec behavior) + if ( handler !== Thrower ) { + that = undefined; + args = [ e ]; + } + + deferred.rejectWith( that, args ); + } + } + }; + + // Support: Promises/A+ section 2.3.3.3.1 + // https://promisesaplus.com/#point-57 + // Re-resolve promises immediately to dodge false rejection from + // subsequent errors + if ( depth ) { + process(); + } else { + + // Call an optional hook to record the stack, in case of exception + // since it's otherwise lost when execution goes async + if ( jQuery.Deferred.getStackHook ) { + process.stackTrace = jQuery.Deferred.getStackHook(); + } + window.setTimeout( process ); + } + }; + } + + return jQuery.Deferred( function( newDefer ) { + + // progress_handlers.add( ... ) + tuples[ 0 ][ 3 ].add( + resolve( + 0, + newDefer, + isFunction( onProgress ) ? + onProgress : + Identity, + newDefer.notifyWith + ) + ); + + // fulfilled_handlers.add( ... ) + tuples[ 1 ][ 3 ].add( + resolve( + 0, + newDefer, + isFunction( onFulfilled ) ? + onFulfilled : + Identity + ) + ); + + // rejected_handlers.add( ... ) + tuples[ 2 ][ 3 ].add( + resolve( + 0, + newDefer, + isFunction( onRejected ) ? + onRejected : + Thrower + ) + ); + } ).promise(); + }, + + // Get a promise for this deferred + // If obj is provided, the promise aspect is added to the object + promise: function( obj ) { + return obj != null ? jQuery.extend( obj, promise ) : promise; + } + }, + deferred = {}; + + // Add list-specific methods + jQuery.each( tuples, function( i, tuple ) { + var list = tuple[ 2 ], + stateString = tuple[ 5 ]; + + // promise.progress = list.add + // promise.done = list.add + // promise.fail = list.add + promise[ tuple[ 1 ] ] = list.add; + + // Handle state + if ( stateString ) { + list.add( + function() { + + // state = "resolved" (i.e., fulfilled) + // state = "rejected" + state = stateString; + }, + + // rejected_callbacks.disable + // fulfilled_callbacks.disable + tuples[ 3 - i ][ 2 ].disable, + + // rejected_handlers.disable + // fulfilled_handlers.disable + tuples[ 3 - i ][ 3 ].disable, + + // progress_callbacks.lock + tuples[ 0 ][ 2 ].lock, + + // progress_handlers.lock + tuples[ 0 ][ 3 ].lock + ); + } + + // progress_handlers.fire + // fulfilled_handlers.fire + // rejected_handlers.fire + list.add( tuple[ 3 ].fire ); + + // deferred.notify = function() { deferred.notifyWith(...) } + // deferred.resolve = function() { deferred.resolveWith(...) } + // deferred.reject = function() { deferred.rejectWith(...) } + deferred[ tuple[ 0 ] ] = function() { + deferred[ tuple[ 0 ] + "With" ]( this === deferred ? undefined : this, arguments ); + return this; + }; + + // deferred.notifyWith = list.fireWith + // deferred.resolveWith = list.fireWith + // deferred.rejectWith = list.fireWith + deferred[ tuple[ 0 ] + "With" ] = list.fireWith; + } ); + + // Make the deferred a promise + promise.promise( deferred ); + + // Call given func if any + if ( func ) { + func.call( deferred, deferred ); + } + + // All done! + return deferred; + }, + + // Deferred helper + when: function( singleValue ) { + var + + // count of uncompleted subordinates + remaining = arguments.length, + + // count of unprocessed arguments + i = remaining, + + // subordinate fulfillment data + resolveContexts = Array( i ), + resolveValues = slice.call( arguments ), + + // the master Deferred + master = jQuery.Deferred(), + + // subordinate callback factory + updateFunc = function( i ) { + return function( value ) { + resolveContexts[ i ] = this; + resolveValues[ i ] = arguments.length > 1 ? slice.call( arguments ) : value; + if ( !( --remaining ) ) { + master.resolveWith( resolveContexts, resolveValues ); + } + }; + }; + + // Single- and empty arguments are adopted like Promise.resolve + if ( remaining <= 1 ) { + adoptValue( singleValue, master.done( updateFunc( i ) ).resolve, master.reject, + !remaining ); + + // Use .then() to unwrap secondary thenables (cf. gh-3000) + if ( master.state() === "pending" || + isFunction( resolveValues[ i ] && resolveValues[ i ].then ) ) { + + return master.then(); + } + } + + // Multiple arguments are aggregated like Promise.all array elements + while ( i-- ) { + adoptValue( resolveValues[ i ], updateFunc( i ), master.reject ); + } + + return master.promise(); + } +} ); + + +// These usually indicate a programmer mistake during development, +// warn about them ASAP rather than swallowing them by default. +var rerrorNames = /^(Eval|Internal|Range|Reference|Syntax|Type|URI)Error$/; + +jQuery.Deferred.exceptionHook = function( error, stack ) { + + // Support: IE 8 - 9 only + // Console exists when dev tools are open, which can happen at any time + if ( window.console && window.console.warn && error && rerrorNames.test( error.name ) ) { + window.console.warn( "jQuery.Deferred exception: " + error.message, error.stack, stack ); + } +}; + + + + +jQuery.readyException = function( error ) { + window.setTimeout( function() { + throw error; + } ); +}; + + + + +// The deferred used on DOM ready +var readyList = jQuery.Deferred(); + +jQuery.fn.ready = function( fn ) { + + readyList + .then( fn ) + + // Wrap jQuery.readyException in a function so that the lookup + // happens at the time of error handling instead of callback + // registration. + .catch( function( error ) { + jQuery.readyException( error ); + } ); + + return this; +}; + +jQuery.extend( { + + // Is the DOM ready to be used? Set to true once it occurs. + isReady: false, + + // A counter to track how many items to wait for before + // the ready event fires. See #6781 + readyWait: 1, + + // Handle when the DOM is ready + ready: function( wait ) { + + // Abort if there are pending holds or we're already ready + if ( wait === true ? --jQuery.readyWait : jQuery.isReady ) { + return; + } + + // Remember that the DOM is ready + jQuery.isReady = true; + + // If a normal DOM Ready event fired, decrement, and wait if need be + if ( wait !== true && --jQuery.readyWait > 0 ) { + return; + } + + // If there are functions bound, to execute + readyList.resolveWith( document, [ jQuery ] ); + } +} ); + +jQuery.ready.then = readyList.then; + +// The ready event handler and self cleanup method +function completed() { + document.removeEventListener( "DOMContentLoaded", completed ); + window.removeEventListener( "load", completed ); + jQuery.ready(); +} + +// Catch cases where $(document).ready() is called +// after the browser event has already occurred. +// Support: IE <=9 - 10 only +// Older IE sometimes signals "interactive" too soon +if ( document.readyState === "complete" || + ( document.readyState !== "loading" && !document.documentElement.doScroll ) ) { + + // Handle it asynchronously to allow scripts the opportunity to delay ready + window.setTimeout( jQuery.ready ); + +} else { + + // Use the handy event callback + document.addEventListener( "DOMContentLoaded", completed ); + + // A fallback to window.onload, that will always work + window.addEventListener( "load", completed ); +} + + + + +// Multifunctional method to get and set values of a collection +// The value/s can optionally be executed if it's a function +var access = function( elems, fn, key, value, chainable, emptyGet, raw ) { + var i = 0, + len = elems.length, + bulk = key == null; + + // Sets many values + if ( toType( key ) === "object" ) { + chainable = true; + for ( i in key ) { + access( elems, fn, i, key[ i ], true, emptyGet, raw ); + } + + // Sets one value + } else if ( value !== undefined ) { + chainable = true; + + if ( !isFunction( value ) ) { + raw = true; + } + + if ( bulk ) { + + // Bulk operations run against the entire set + if ( raw ) { + fn.call( elems, value ); + fn = null; + + // ...except when executing function values + } else { + bulk = fn; + fn = function( elem, _key, value ) { + return bulk.call( jQuery( elem ), value ); + }; + } + } + + if ( fn ) { + for ( ; i < len; i++ ) { + fn( + elems[ i ], key, raw ? + value : + value.call( elems[ i ], i, fn( elems[ i ], key ) ) + ); + } + } + } + + if ( chainable ) { + return elems; + } + + // Gets + if ( bulk ) { + return fn.call( elems ); + } + + return len ? fn( elems[ 0 ], key ) : emptyGet; +}; + + +// Matches dashed string for camelizing +var rmsPrefix = /^-ms-/, + rdashAlpha = /-([a-z])/g; + +// Used by camelCase as callback to replace() +function fcamelCase( _all, letter ) { + return letter.toUpperCase(); +} + +// Convert dashed to camelCase; used by the css and data modules +// Support: IE <=9 - 11, Edge 12 - 15 +// Microsoft forgot to hump their vendor prefix (#9572) +function camelCase( string ) { + return string.replace( rmsPrefix, "ms-" ).replace( rdashAlpha, fcamelCase ); +} +var acceptData = function( owner ) { + + // Accepts only: + // - Node + // - Node.ELEMENT_NODE + // - Node.DOCUMENT_NODE + // - Object + // - Any + return owner.nodeType === 1 || owner.nodeType === 9 || !( +owner.nodeType ); +}; + + + + +function Data() { + this.expando = jQuery.expando + Data.uid++; +} + +Data.uid = 1; + +Data.prototype = { + + cache: function( owner ) { + + // Check if the owner object already has a cache + var value = owner[ this.expando ]; + + // If not, create one + if ( !value ) { + value = {}; + + // We can accept data for non-element nodes in modern browsers, + // but we should not, see #8335. + // Always return an empty object. + if ( acceptData( owner ) ) { + + // If it is a node unlikely to be stringify-ed or looped over + // use plain assignment + if ( owner.nodeType ) { + owner[ this.expando ] = value; + + // Otherwise secure it in a non-enumerable property + // configurable must be true to allow the property to be + // deleted when data is removed + } else { + Object.defineProperty( owner, this.expando, { + value: value, + configurable: true + } ); + } + } + } + + return value; + }, + set: function( owner, data, value ) { + var prop, + cache = this.cache( owner ); + + // Handle: [ owner, key, value ] args + // Always use camelCase key (gh-2257) + if ( typeof data === "string" ) { + cache[ camelCase( data ) ] = value; + + // Handle: [ owner, { properties } ] args + } else { + + // Copy the properties one-by-one to the cache object + for ( prop in data ) { + cache[ camelCase( prop ) ] = data[ prop ]; + } + } + return cache; + }, + get: function( owner, key ) { + return key === undefined ? + this.cache( owner ) : + + // Always use camelCase key (gh-2257) + owner[ this.expando ] && owner[ this.expando ][ camelCase( key ) ]; + }, + access: function( owner, key, value ) { + + // In cases where either: + // + // 1. No key was specified + // 2. A string key was specified, but no value provided + // + // Take the "read" path and allow the get method to determine + // which value to return, respectively either: + // + // 1. The entire cache object + // 2. The data stored at the key + // + if ( key === undefined || + ( ( key && typeof key === "string" ) && value === undefined ) ) { + + return this.get( owner, key ); + } + + // When the key is not a string, or both a key and value + // are specified, set or extend (existing objects) with either: + // + // 1. An object of properties + // 2. A key and value + // + this.set( owner, key, value ); + + // Since the "set" path can have two possible entry points + // return the expected data based on which path was taken[*] + return value !== undefined ? value : key; + }, + remove: function( owner, key ) { + var i, + cache = owner[ this.expando ]; + + if ( cache === undefined ) { + return; + } + + if ( key !== undefined ) { + + // Support array or space separated string of keys + if ( Array.isArray( key ) ) { + + // If key is an array of keys... + // We always set camelCase keys, so remove that. + key = key.map( camelCase ); + } else { + key = camelCase( key ); + + // If a key with the spaces exists, use it. + // Otherwise, create an array by matching non-whitespace + key = key in cache ? + [ key ] : + ( key.match( rnothtmlwhite ) || [] ); + } + + i = key.length; + + while ( i-- ) { + delete cache[ key[ i ] ]; + } + } + + // Remove the expando if there's no more data + if ( key === undefined || jQuery.isEmptyObject( cache ) ) { + + // Support: Chrome <=35 - 45 + // Webkit & Blink performance suffers when deleting properties + // from DOM nodes, so set to undefined instead + // https://bugs.chromium.org/p/chromium/issues/detail?id=378607 (bug restricted) + if ( owner.nodeType ) { + owner[ this.expando ] = undefined; + } else { + delete owner[ this.expando ]; + } + } + }, + hasData: function( owner ) { + var cache = owner[ this.expando ]; + return cache !== undefined && !jQuery.isEmptyObject( cache ); + } +}; +var dataPriv = new Data(); + +var dataUser = new Data(); + + + +// Implementation Summary +// +// 1. Enforce API surface and semantic compatibility with 1.9.x branch +// 2. Improve the module's maintainability by reducing the storage +// paths to a single mechanism. +// 3. Use the same single mechanism to support "private" and "user" data. +// 4. _Never_ expose "private" data to user code (TODO: Drop _data, _removeData) +// 5. Avoid exposing implementation details on user objects (eg. expando properties) +// 6. Provide a clear path for implementation upgrade to WeakMap in 2014 + +var rbrace = /^(?:\{[\w\W]*\}|\[[\w\W]*\])$/, + rmultiDash = /[A-Z]/g; + +function getData( data ) { + if ( data === "true" ) { + return true; + } + + if ( data === "false" ) { + return false; + } + + if ( data === "null" ) { + return null; + } + + // Only convert to a number if it doesn't change the string + if ( data === +data + "" ) { + return +data; + } + + if ( rbrace.test( data ) ) { + return JSON.parse( data ); + } + + return data; +} + +function dataAttr( elem, key, data ) { + var name; + + // If nothing was found internally, try to fetch any + // data from the HTML5 data-* attribute + if ( data === undefined && elem.nodeType === 1 ) { + name = "data-" + key.replace( rmultiDash, "-$&" ).toLowerCase(); + data = elem.getAttribute( name ); + + if ( typeof data === "string" ) { + try { + data = getData( data ); + } catch ( e ) {} + + // Make sure we set the data so it isn't changed later + dataUser.set( elem, key, data ); + } else { + data = undefined; + } + } + return data; +} + +jQuery.extend( { + hasData: function( elem ) { + return dataUser.hasData( elem ) || dataPriv.hasData( elem ); + }, + + data: function( elem, name, data ) { + return dataUser.access( elem, name, data ); + }, + + removeData: function( elem, name ) { + dataUser.remove( elem, name ); + }, + + // TODO: Now that all calls to _data and _removeData have been replaced + // with direct calls to dataPriv methods, these can be deprecated. + _data: function( elem, name, data ) { + return dataPriv.access( elem, name, data ); + }, + + _removeData: function( elem, name ) { + dataPriv.remove( elem, name ); + } +} ); + +jQuery.fn.extend( { + data: function( key, value ) { + var i, name, data, + elem = this[ 0 ], + attrs = elem && elem.attributes; + + // Gets all values + if ( key === undefined ) { + if ( this.length ) { + data = dataUser.get( elem ); + + if ( elem.nodeType === 1 && !dataPriv.get( elem, "hasDataAttrs" ) ) { + i = attrs.length; + while ( i-- ) { + + // Support: IE 11 only + // The attrs elements can be null (#14894) + if ( attrs[ i ] ) { + name = attrs[ i ].name; + if ( name.indexOf( "data-" ) === 0 ) { + name = camelCase( name.slice( 5 ) ); + dataAttr( elem, name, data[ name ] ); + } + } + } + dataPriv.set( elem, "hasDataAttrs", true ); + } + } + + return data; + } + + // Sets multiple values + if ( typeof key === "object" ) { + return this.each( function() { + dataUser.set( this, key ); + } ); + } + + return access( this, function( value ) { + var data; + + // The calling jQuery object (element matches) is not empty + // (and therefore has an element appears at this[ 0 ]) and the + // `value` parameter was not undefined. An empty jQuery object + // will result in `undefined` for elem = this[ 0 ] which will + // throw an exception if an attempt to read a data cache is made. + if ( elem && value === undefined ) { + + // Attempt to get data from the cache + // The key will always be camelCased in Data + data = dataUser.get( elem, key ); + if ( data !== undefined ) { + return data; + } + + // Attempt to "discover" the data in + // HTML5 custom data-* attrs + data = dataAttr( elem, key ); + if ( data !== undefined ) { + return data; + } + + // We tried really hard, but the data doesn't exist. + return; + } + + // Set the data... + this.each( function() { + + // We always store the camelCased key + dataUser.set( this, key, value ); + } ); + }, null, value, arguments.length > 1, null, true ); + }, + + removeData: function( key ) { + return this.each( function() { + dataUser.remove( this, key ); + } ); + } +} ); + + +jQuery.extend( { + queue: function( elem, type, data ) { + var queue; + + if ( elem ) { + type = ( type || "fx" ) + "queue"; + queue = dataPriv.get( elem, type ); + + // Speed up dequeue by getting out quickly if this is just a lookup + if ( data ) { + if ( !queue || Array.isArray( data ) ) { + queue = dataPriv.access( elem, type, jQuery.makeArray( data ) ); + } else { + queue.push( data ); + } + } + return queue || []; + } + }, + + dequeue: function( elem, type ) { + type = type || "fx"; + + var queue = jQuery.queue( elem, type ), + startLength = queue.length, + fn = queue.shift(), + hooks = jQuery._queueHooks( elem, type ), + next = function() { + jQuery.dequeue( elem, type ); + }; + + // If the fx queue is dequeued, always remove the progress sentinel + if ( fn === "inprogress" ) { + fn = queue.shift(); + startLength--; + } + + if ( fn ) { + + // Add a progress sentinel to prevent the fx queue from being + // automatically dequeued + if ( type === "fx" ) { + queue.unshift( "inprogress" ); + } + + // Clear up the last queue stop function + delete hooks.stop; + fn.call( elem, next, hooks ); + } + + if ( !startLength && hooks ) { + hooks.empty.fire(); + } + }, + + // Not public - generate a queueHooks object, or return the current one + _queueHooks: function( elem, type ) { + var key = type + "queueHooks"; + return dataPriv.get( elem, key ) || dataPriv.access( elem, key, { + empty: jQuery.Callbacks( "once memory" ).add( function() { + dataPriv.remove( elem, [ type + "queue", key ] ); + } ) + } ); + } +} ); + +jQuery.fn.extend( { + queue: function( type, data ) { + var setter = 2; + + if ( typeof type !== "string" ) { + data = type; + type = "fx"; + setter--; + } + + if ( arguments.length < setter ) { + return jQuery.queue( this[ 0 ], type ); + } + + return data === undefined ? + this : + this.each( function() { + var queue = jQuery.queue( this, type, data ); + + // Ensure a hooks for this queue + jQuery._queueHooks( this, type ); + + if ( type === "fx" && queue[ 0 ] !== "inprogress" ) { + jQuery.dequeue( this, type ); + } + } ); + }, + dequeue: function( type ) { + return this.each( function() { + jQuery.dequeue( this, type ); + } ); + }, + clearQueue: function( type ) { + return this.queue( type || "fx", [] ); + }, + + // Get a promise resolved when queues of a certain type + // are emptied (fx is the type by default) + promise: function( type, obj ) { + var tmp, + count = 1, + defer = jQuery.Deferred(), + elements = this, + i = this.length, + resolve = function() { + if ( !( --count ) ) { + defer.resolveWith( elements, [ elements ] ); + } + }; + + if ( typeof type !== "string" ) { + obj = type; + type = undefined; + } + type = type || "fx"; + + while ( i-- ) { + tmp = dataPriv.get( elements[ i ], type + "queueHooks" ); + if ( tmp && tmp.empty ) { + count++; + tmp.empty.add( resolve ); + } + } + resolve(); + return defer.promise( obj ); + } +} ); +var pnum = ( /[+-]?(?:\d*\.|)\d+(?:[eE][+-]?\d+|)/ ).source; + +var rcssNum = new RegExp( "^(?:([+-])=|)(" + pnum + ")([a-z%]*)$", "i" ); + + +var cssExpand = [ "Top", "Right", "Bottom", "Left" ]; + +var documentElement = document.documentElement; + + + + var isAttached = function( elem ) { + return jQuery.contains( elem.ownerDocument, elem ); + }, + composed = { composed: true }; + + // Support: IE 9 - 11+, Edge 12 - 18+, iOS 10.0 - 10.2 only + // Check attachment across shadow DOM boundaries when possible (gh-3504) + // Support: iOS 10.0-10.2 only + // Early iOS 10 versions support `attachShadow` but not `getRootNode`, + // leading to errors. We need to check for `getRootNode`. + if ( documentElement.getRootNode ) { + isAttached = function( elem ) { + return jQuery.contains( elem.ownerDocument, elem ) || + elem.getRootNode( composed ) === elem.ownerDocument; + }; + } +var isHiddenWithinTree = function( elem, el ) { + + // isHiddenWithinTree might be called from jQuery#filter function; + // in that case, element will be second argument + elem = el || elem; + + // Inline style trumps all + return elem.style.display === "none" || + elem.style.display === "" && + + // Otherwise, check computed style + // Support: Firefox <=43 - 45 + // Disconnected elements can have computed display: none, so first confirm that elem is + // in the document. + isAttached( elem ) && + + jQuery.css( elem, "display" ) === "none"; + }; + + + +function adjustCSS( elem, prop, valueParts, tween ) { + var adjusted, scale, + maxIterations = 20, + currentValue = tween ? + function() { + return tween.cur(); + } : + function() { + return jQuery.css( elem, prop, "" ); + }, + initial = currentValue(), + unit = valueParts && valueParts[ 3 ] || ( jQuery.cssNumber[ prop ] ? "" : "px" ), + + // Starting value computation is required for potential unit mismatches + initialInUnit = elem.nodeType && + ( jQuery.cssNumber[ prop ] || unit !== "px" && +initial ) && + rcssNum.exec( jQuery.css( elem, prop ) ); + + if ( initialInUnit && initialInUnit[ 3 ] !== unit ) { + + // Support: Firefox <=54 + // Halve the iteration target value to prevent interference from CSS upper bounds (gh-2144) + initial = initial / 2; + + // Trust units reported by jQuery.css + unit = unit || initialInUnit[ 3 ]; + + // Iteratively approximate from a nonzero starting point + initialInUnit = +initial || 1; + + while ( maxIterations-- ) { + + // Evaluate and update our best guess (doubling guesses that zero out). + // Finish if the scale equals or crosses 1 (making the old*new product non-positive). + jQuery.style( elem, prop, initialInUnit + unit ); + if ( ( 1 - scale ) * ( 1 - ( scale = currentValue() / initial || 0.5 ) ) <= 0 ) { + maxIterations = 0; + } + initialInUnit = initialInUnit / scale; + + } + + initialInUnit = initialInUnit * 2; + jQuery.style( elem, prop, initialInUnit + unit ); + + // Make sure we update the tween properties later on + valueParts = valueParts || []; + } + + if ( valueParts ) { + initialInUnit = +initialInUnit || +initial || 0; + + // Apply relative offset (+=/-=) if specified + adjusted = valueParts[ 1 ] ? + initialInUnit + ( valueParts[ 1 ] + 1 ) * valueParts[ 2 ] : + +valueParts[ 2 ]; + if ( tween ) { + tween.unit = unit; + tween.start = initialInUnit; + tween.end = adjusted; + } + } + return adjusted; +} + + +var defaultDisplayMap = {}; + +function getDefaultDisplay( elem ) { + var temp, + doc = elem.ownerDocument, + nodeName = elem.nodeName, + display = defaultDisplayMap[ nodeName ]; + + if ( display ) { + return display; + } + + temp = doc.body.appendChild( doc.createElement( nodeName ) ); + display = jQuery.css( temp, "display" ); + + temp.parentNode.removeChild( temp ); + + if ( display === "none" ) { + display = "block"; + } + defaultDisplayMap[ nodeName ] = display; + + return display; +} + +function showHide( elements, show ) { + var display, elem, + values = [], + index = 0, + length = elements.length; + + // Determine new display value for elements that need to change + for ( ; index < length; index++ ) { + elem = elements[ index ]; + if ( !elem.style ) { + continue; + } + + display = elem.style.display; + if ( show ) { + + // Since we force visibility upon cascade-hidden elements, an immediate (and slow) + // check is required in this first loop unless we have a nonempty display value (either + // inline or about-to-be-restored) + if ( display === "none" ) { + values[ index ] = dataPriv.get( elem, "display" ) || null; + if ( !values[ index ] ) { + elem.style.display = ""; + } + } + if ( elem.style.display === "" && isHiddenWithinTree( elem ) ) { + values[ index ] = getDefaultDisplay( elem ); + } + } else { + if ( display !== "none" ) { + values[ index ] = "none"; + + // Remember what we're overwriting + dataPriv.set( elem, "display", display ); + } + } + } + + // Set the display of the elements in a second loop to avoid constant reflow + for ( index = 0; index < length; index++ ) { + if ( values[ index ] != null ) { + elements[ index ].style.display = values[ index ]; + } + } + + return elements; +} + +jQuery.fn.extend( { + show: function() { + return showHide( this, true ); + }, + hide: function() { + return showHide( this ); + }, + toggle: function( state ) { + if ( typeof state === "boolean" ) { + return state ? this.show() : this.hide(); + } + + return this.each( function() { + if ( isHiddenWithinTree( this ) ) { + jQuery( this ).show(); + } else { + jQuery( this ).hide(); + } + } ); + } +} ); +var rcheckableType = ( /^(?:checkbox|radio)$/i ); + +var rtagName = ( /<([a-z][^\/\0>\x20\t\r\n\f]*)/i ); + +var rscriptType = ( /^$|^module$|\/(?:java|ecma)script/i ); + + + +( function() { + var fragment = document.createDocumentFragment(), + div = fragment.appendChild( document.createElement( "div" ) ), + input = document.createElement( "input" ); + + // Support: Android 4.0 - 4.3 only + // Check state lost if the name is set (#11217) + // Support: Windows Web Apps (WWA) + // `name` and `type` must use .setAttribute for WWA (#14901) + input.setAttribute( "type", "radio" ); + input.setAttribute( "checked", "checked" ); + input.setAttribute( "name", "t" ); + + div.appendChild( input ); + + // Support: Android <=4.1 only + // Older WebKit doesn't clone checked state correctly in fragments + support.checkClone = div.cloneNode( true ).cloneNode( true ).lastChild.checked; + + // Support: IE <=11 only + // Make sure textarea (and checkbox) defaultValue is properly cloned + div.innerHTML = ""; + support.noCloneChecked = !!div.cloneNode( true ).lastChild.defaultValue; + + // Support: IE <=9 only + // IE <=9 replaces "; + support.option = !!div.lastChild; +} )(); + + +// We have to close these tags to support XHTML (#13200) +var wrapMap = { + + // XHTML parsers do not magically insert elements in the + // same way that tag soup parsers do. So we cannot shorten + // this by omitting or other required elements. + thead: [ 1, "", "
" ], + col: [ 2, "", "
" ], + tr: [ 2, "", "
" ], + td: [ 3, "", "
" ], + + _default: [ 0, "", "" ] +}; + +wrapMap.tbody = wrapMap.tfoot = wrapMap.colgroup = wrapMap.caption = wrapMap.thead; +wrapMap.th = wrapMap.td; + +// Support: IE <=9 only +if ( !support.option ) { + wrapMap.optgroup = wrapMap.option = [ 1, "" ]; +} + + +function getAll( context, tag ) { + + // Support: IE <=9 - 11 only + // Use typeof to avoid zero-argument method invocation on host objects (#15151) + var ret; + + if ( typeof context.getElementsByTagName !== "undefined" ) { + ret = context.getElementsByTagName( tag || "*" ); + + } else if ( typeof context.querySelectorAll !== "undefined" ) { + ret = context.querySelectorAll( tag || "*" ); + + } else { + ret = []; + } + + if ( tag === undefined || tag && nodeName( context, tag ) ) { + return jQuery.merge( [ context ], ret ); + } + + return ret; +} + + +// Mark scripts as having already been evaluated +function setGlobalEval( elems, refElements ) { + var i = 0, + l = elems.length; + + for ( ; i < l; i++ ) { + dataPriv.set( + elems[ i ], + "globalEval", + !refElements || dataPriv.get( refElements[ i ], "globalEval" ) + ); + } +} + + +var rhtml = /<|&#?\w+;/; + +function buildFragment( elems, context, scripts, selection, ignored ) { + var elem, tmp, tag, wrap, attached, j, + fragment = context.createDocumentFragment(), + nodes = [], + i = 0, + l = elems.length; + + for ( ; i < l; i++ ) { + elem = elems[ i ]; + + if ( elem || elem === 0 ) { + + // Add nodes directly + if ( toType( elem ) === "object" ) { + + // Support: Android <=4.0 only, PhantomJS 1 only + // push.apply(_, arraylike) throws on ancient WebKit + jQuery.merge( nodes, elem.nodeType ? [ elem ] : elem ); + + // Convert non-html into a text node + } else if ( !rhtml.test( elem ) ) { + nodes.push( context.createTextNode( elem ) ); + + // Convert html into DOM nodes + } else { + tmp = tmp || fragment.appendChild( context.createElement( "div" ) ); + + // Deserialize a standard representation + tag = ( rtagName.exec( elem ) || [ "", "" ] )[ 1 ].toLowerCase(); + wrap = wrapMap[ tag ] || wrapMap._default; + tmp.innerHTML = wrap[ 1 ] + jQuery.htmlPrefilter( elem ) + wrap[ 2 ]; + + // Descend through wrappers to the right content + j = wrap[ 0 ]; + while ( j-- ) { + tmp = tmp.lastChild; + } + + // Support: Android <=4.0 only, PhantomJS 1 only + // push.apply(_, arraylike) throws on ancient WebKit + jQuery.merge( nodes, tmp.childNodes ); + + // Remember the top-level container + tmp = fragment.firstChild; + + // Ensure the created nodes are orphaned (#12392) + tmp.textContent = ""; + } + } + } + + // Remove wrapper from fragment + fragment.textContent = ""; + + i = 0; + while ( ( elem = nodes[ i++ ] ) ) { + + // Skip elements already in the context collection (trac-4087) + if ( selection && jQuery.inArray( elem, selection ) > -1 ) { + if ( ignored ) { + ignored.push( elem ); + } + continue; + } + + attached = isAttached( elem ); + + // Append to fragment + tmp = getAll( fragment.appendChild( elem ), "script" ); + + // Preserve script evaluation history + if ( attached ) { + setGlobalEval( tmp ); + } + + // Capture executables + if ( scripts ) { + j = 0; + while ( ( elem = tmp[ j++ ] ) ) { + if ( rscriptType.test( elem.type || "" ) ) { + scripts.push( elem ); + } + } + } + } + + return fragment; +} + + +var + rkeyEvent = /^key/, + rmouseEvent = /^(?:mouse|pointer|contextmenu|drag|drop)|click/, + rtypenamespace = /^([^.]*)(?:\.(.+)|)/; + +function returnTrue() { + return true; +} + +function returnFalse() { + return false; +} + +// Support: IE <=9 - 11+ +// focus() and blur() are asynchronous, except when they are no-op. +// So expect focus to be synchronous when the element is already active, +// and blur to be synchronous when the element is not already active. +// (focus and blur are always synchronous in other supported browsers, +// this just defines when we can count on it). +function expectSync( elem, type ) { + return ( elem === safeActiveElement() ) === ( type === "focus" ); +} + +// Support: IE <=9 only +// Accessing document.activeElement can throw unexpectedly +// https://bugs.jquery.com/ticket/13393 +function safeActiveElement() { + try { + return document.activeElement; + } catch ( err ) { } +} + +function on( elem, types, selector, data, fn, one ) { + var origFn, type; + + // Types can be a map of types/handlers + if ( typeof types === "object" ) { + + // ( types-Object, selector, data ) + if ( typeof selector !== "string" ) { + + // ( types-Object, data ) + data = data || selector; + selector = undefined; + } + for ( type in types ) { + on( elem, type, selector, data, types[ type ], one ); + } + return elem; + } + + if ( data == null && fn == null ) { + + // ( types, fn ) + fn = selector; + data = selector = undefined; + } else if ( fn == null ) { + if ( typeof selector === "string" ) { + + // ( types, selector, fn ) + fn = data; + data = undefined; + } else { + + // ( types, data, fn ) + fn = data; + data = selector; + selector = undefined; + } + } + if ( fn === false ) { + fn = returnFalse; + } else if ( !fn ) { + return elem; + } + + if ( one === 1 ) { + origFn = fn; + fn = function( event ) { + + // Can use an empty set, since event contains the info + jQuery().off( event ); + return origFn.apply( this, arguments ); + }; + + // Use same guid so caller can remove using origFn + fn.guid = origFn.guid || ( origFn.guid = jQuery.guid++ ); + } + return elem.each( function() { + jQuery.event.add( this, types, fn, data, selector ); + } ); +} + +/* + * Helper functions for managing events -- not part of the public interface. + * Props to Dean Edwards' addEvent library for many of the ideas. + */ +jQuery.event = { + + global: {}, + + add: function( elem, types, handler, data, selector ) { + + var handleObjIn, eventHandle, tmp, + events, t, handleObj, + special, handlers, type, namespaces, origType, + elemData = dataPriv.get( elem ); + + // Only attach events to objects that accept data + if ( !acceptData( elem ) ) { + return; + } + + // Caller can pass in an object of custom data in lieu of the handler + if ( handler.handler ) { + handleObjIn = handler; + handler = handleObjIn.handler; + selector = handleObjIn.selector; + } + + // Ensure that invalid selectors throw exceptions at attach time + // Evaluate against documentElement in case elem is a non-element node (e.g., document) + if ( selector ) { + jQuery.find.matchesSelector( documentElement, selector ); + } + + // Make sure that the handler has a unique ID, used to find/remove it later + if ( !handler.guid ) { + handler.guid = jQuery.guid++; + } + + // Init the element's event structure and main handler, if this is the first + if ( !( events = elemData.events ) ) { + events = elemData.events = Object.create( null ); + } + if ( !( eventHandle = elemData.handle ) ) { + eventHandle = elemData.handle = function( e ) { + + // Discard the second event of a jQuery.event.trigger() and + // when an event is called after a page has unloaded + return typeof jQuery !== "undefined" && jQuery.event.triggered !== e.type ? + jQuery.event.dispatch.apply( elem, arguments ) : undefined; + }; + } + + // Handle multiple events separated by a space + types = ( types || "" ).match( rnothtmlwhite ) || [ "" ]; + t = types.length; + while ( t-- ) { + tmp = rtypenamespace.exec( types[ t ] ) || []; + type = origType = tmp[ 1 ]; + namespaces = ( tmp[ 2 ] || "" ).split( "." ).sort(); + + // There *must* be a type, no attaching namespace-only handlers + if ( !type ) { + continue; + } + + // If event changes its type, use the special event handlers for the changed type + special = jQuery.event.special[ type ] || {}; + + // If selector defined, determine special event api type, otherwise given type + type = ( selector ? special.delegateType : special.bindType ) || type; + + // Update special based on newly reset type + special = jQuery.event.special[ type ] || {}; + + // handleObj is passed to all event handlers + handleObj = jQuery.extend( { + type: type, + origType: origType, + data: data, + handler: handler, + guid: handler.guid, + selector: selector, + needsContext: selector && jQuery.expr.match.needsContext.test( selector ), + namespace: namespaces.join( "." ) + }, handleObjIn ); + + // Init the event handler queue if we're the first + if ( !( handlers = events[ type ] ) ) { + handlers = events[ type ] = []; + handlers.delegateCount = 0; + + // Only use addEventListener if the special events handler returns false + if ( !special.setup || + special.setup.call( elem, data, namespaces, eventHandle ) === false ) { + + if ( elem.addEventListener ) { + elem.addEventListener( type, eventHandle ); + } + } + } + + if ( special.add ) { + special.add.call( elem, handleObj ); + + if ( !handleObj.handler.guid ) { + handleObj.handler.guid = handler.guid; + } + } + + // Add to the element's handler list, delegates in front + if ( selector ) { + handlers.splice( handlers.delegateCount++, 0, handleObj ); + } else { + handlers.push( handleObj ); + } + + // Keep track of which events have ever been used, for event optimization + jQuery.event.global[ type ] = true; + } + + }, + + // Detach an event or set of events from an element + remove: function( elem, types, handler, selector, mappedTypes ) { + + var j, origCount, tmp, + events, t, handleObj, + special, handlers, type, namespaces, origType, + elemData = dataPriv.hasData( elem ) && dataPriv.get( elem ); + + if ( !elemData || !( events = elemData.events ) ) { + return; + } + + // Once for each type.namespace in types; type may be omitted + types = ( types || "" ).match( rnothtmlwhite ) || [ "" ]; + t = types.length; + while ( t-- ) { + tmp = rtypenamespace.exec( types[ t ] ) || []; + type = origType = tmp[ 1 ]; + namespaces = ( tmp[ 2 ] || "" ).split( "." ).sort(); + + // Unbind all events (on this namespace, if provided) for the element + if ( !type ) { + for ( type in events ) { + jQuery.event.remove( elem, type + types[ t ], handler, selector, true ); + } + continue; + } + + special = jQuery.event.special[ type ] || {}; + type = ( selector ? special.delegateType : special.bindType ) || type; + handlers = events[ type ] || []; + tmp = tmp[ 2 ] && + new RegExp( "(^|\\.)" + namespaces.join( "\\.(?:.*\\.|)" ) + "(\\.|$)" ); + + // Remove matching events + origCount = j = handlers.length; + while ( j-- ) { + handleObj = handlers[ j ]; + + if ( ( mappedTypes || origType === handleObj.origType ) && + ( !handler || handler.guid === handleObj.guid ) && + ( !tmp || tmp.test( handleObj.namespace ) ) && + ( !selector || selector === handleObj.selector || + selector === "**" && handleObj.selector ) ) { + handlers.splice( j, 1 ); + + if ( handleObj.selector ) { + handlers.delegateCount--; + } + if ( special.remove ) { + special.remove.call( elem, handleObj ); + } + } + } + + // Remove generic event handler if we removed something and no more handlers exist + // (avoids potential for endless recursion during removal of special event handlers) + if ( origCount && !handlers.length ) { + if ( !special.teardown || + special.teardown.call( elem, namespaces, elemData.handle ) === false ) { + + jQuery.removeEvent( elem, type, elemData.handle ); + } + + delete events[ type ]; + } + } + + // Remove data and the expando if it's no longer used + if ( jQuery.isEmptyObject( events ) ) { + dataPriv.remove( elem, "handle events" ); + } + }, + + dispatch: function( nativeEvent ) { + + var i, j, ret, matched, handleObj, handlerQueue, + args = new Array( arguments.length ), + + // Make a writable jQuery.Event from the native event object + event = jQuery.event.fix( nativeEvent ), + + handlers = ( + dataPriv.get( this, "events" ) || Object.create( null ) + )[ event.type ] || [], + special = jQuery.event.special[ event.type ] || {}; + + // Use the fix-ed jQuery.Event rather than the (read-only) native event + args[ 0 ] = event; + + for ( i = 1; i < arguments.length; i++ ) { + args[ i ] = arguments[ i ]; + } + + event.delegateTarget = this; + + // Call the preDispatch hook for the mapped type, and let it bail if desired + if ( special.preDispatch && special.preDispatch.call( this, event ) === false ) { + return; + } + + // Determine handlers + handlerQueue = jQuery.event.handlers.call( this, event, handlers ); + + // Run delegates first; they may want to stop propagation beneath us + i = 0; + while ( ( matched = handlerQueue[ i++ ] ) && !event.isPropagationStopped() ) { + event.currentTarget = matched.elem; + + j = 0; + while ( ( handleObj = matched.handlers[ j++ ] ) && + !event.isImmediatePropagationStopped() ) { + + // If the event is namespaced, then each handler is only invoked if it is + // specially universal or its namespaces are a superset of the event's. + if ( !event.rnamespace || handleObj.namespace === false || + event.rnamespace.test( handleObj.namespace ) ) { + + event.handleObj = handleObj; + event.data = handleObj.data; + + ret = ( ( jQuery.event.special[ handleObj.origType ] || {} ).handle || + handleObj.handler ).apply( matched.elem, args ); + + if ( ret !== undefined ) { + if ( ( event.result = ret ) === false ) { + event.preventDefault(); + event.stopPropagation(); + } + } + } + } + } + + // Call the postDispatch hook for the mapped type + if ( special.postDispatch ) { + special.postDispatch.call( this, event ); + } + + return event.result; + }, + + handlers: function( event, handlers ) { + var i, handleObj, sel, matchedHandlers, matchedSelectors, + handlerQueue = [], + delegateCount = handlers.delegateCount, + cur = event.target; + + // Find delegate handlers + if ( delegateCount && + + // Support: IE <=9 + // Black-hole SVG instance trees (trac-13180) + cur.nodeType && + + // Support: Firefox <=42 + // Suppress spec-violating clicks indicating a non-primary pointer button (trac-3861) + // https://www.w3.org/TR/DOM-Level-3-Events/#event-type-click + // Support: IE 11 only + // ...but not arrow key "clicks" of radio inputs, which can have `button` -1 (gh-2343) + !( event.type === "click" && event.button >= 1 ) ) { + + for ( ; cur !== this; cur = cur.parentNode || this ) { + + // Don't check non-elements (#13208) + // Don't process clicks on disabled elements (#6911, #8165, #11382, #11764) + if ( cur.nodeType === 1 && !( event.type === "click" && cur.disabled === true ) ) { + matchedHandlers = []; + matchedSelectors = {}; + for ( i = 0; i < delegateCount; i++ ) { + handleObj = handlers[ i ]; + + // Don't conflict with Object.prototype properties (#13203) + sel = handleObj.selector + " "; + + if ( matchedSelectors[ sel ] === undefined ) { + matchedSelectors[ sel ] = handleObj.needsContext ? + jQuery( sel, this ).index( cur ) > -1 : + jQuery.find( sel, this, null, [ cur ] ).length; + } + if ( matchedSelectors[ sel ] ) { + matchedHandlers.push( handleObj ); + } + } + if ( matchedHandlers.length ) { + handlerQueue.push( { elem: cur, handlers: matchedHandlers } ); + } + } + } + } + + // Add the remaining (directly-bound) handlers + cur = this; + if ( delegateCount < handlers.length ) { + handlerQueue.push( { elem: cur, handlers: handlers.slice( delegateCount ) } ); + } + + return handlerQueue; + }, + + addProp: function( name, hook ) { + Object.defineProperty( jQuery.Event.prototype, name, { + enumerable: true, + configurable: true, + + get: isFunction( hook ) ? + function() { + if ( this.originalEvent ) { + return hook( this.originalEvent ); + } + } : + function() { + if ( this.originalEvent ) { + return this.originalEvent[ name ]; + } + }, + + set: function( value ) { + Object.defineProperty( this, name, { + enumerable: true, + configurable: true, + writable: true, + value: value + } ); + } + } ); + }, + + fix: function( originalEvent ) { + return originalEvent[ jQuery.expando ] ? + originalEvent : + new jQuery.Event( originalEvent ); + }, + + special: { + load: { + + // Prevent triggered image.load events from bubbling to window.load + noBubble: true + }, + click: { + + // Utilize native event to ensure correct state for checkable inputs + setup: function( data ) { + + // For mutual compressibility with _default, replace `this` access with a local var. + // `|| data` is dead code meant only to preserve the variable through minification. + var el = this || data; + + // Claim the first handler + if ( rcheckableType.test( el.type ) && + el.click && nodeName( el, "input" ) ) { + + // dataPriv.set( el, "click", ... ) + leverageNative( el, "click", returnTrue ); + } + + // Return false to allow normal processing in the caller + return false; + }, + trigger: function( data ) { + + // For mutual compressibility with _default, replace `this` access with a local var. + // `|| data` is dead code meant only to preserve the variable through minification. + var el = this || data; + + // Force setup before triggering a click + if ( rcheckableType.test( el.type ) && + el.click && nodeName( el, "input" ) ) { + + leverageNative( el, "click" ); + } + + // Return non-false to allow normal event-path propagation + return true; + }, + + // For cross-browser consistency, suppress native .click() on links + // Also prevent it if we're currently inside a leveraged native-event stack + _default: function( event ) { + var target = event.target; + return rcheckableType.test( target.type ) && + target.click && nodeName( target, "input" ) && + dataPriv.get( target, "click" ) || + nodeName( target, "a" ); + } + }, + + beforeunload: { + postDispatch: function( event ) { + + // Support: Firefox 20+ + // Firefox doesn't alert if the returnValue field is not set. + if ( event.result !== undefined && event.originalEvent ) { + event.originalEvent.returnValue = event.result; + } + } + } + } +}; + +// Ensure the presence of an event listener that handles manually-triggered +// synthetic events by interrupting progress until reinvoked in response to +// *native* events that it fires directly, ensuring that state changes have +// already occurred before other listeners are invoked. +function leverageNative( el, type, expectSync ) { + + // Missing expectSync indicates a trigger call, which must force setup through jQuery.event.add + if ( !expectSync ) { + if ( dataPriv.get( el, type ) === undefined ) { + jQuery.event.add( el, type, returnTrue ); + } + return; + } + + // Register the controller as a special universal handler for all event namespaces + dataPriv.set( el, type, false ); + jQuery.event.add( el, type, { + namespace: false, + handler: function( event ) { + var notAsync, result, + saved = dataPriv.get( this, type ); + + if ( ( event.isTrigger & 1 ) && this[ type ] ) { + + // Interrupt processing of the outer synthetic .trigger()ed event + // Saved data should be false in such cases, but might be a leftover capture object + // from an async native handler (gh-4350) + if ( !saved.length ) { + + // Store arguments for use when handling the inner native event + // There will always be at least one argument (an event object), so this array + // will not be confused with a leftover capture object. + saved = slice.call( arguments ); + dataPriv.set( this, type, saved ); + + // Trigger the native event and capture its result + // Support: IE <=9 - 11+ + // focus() and blur() are asynchronous + notAsync = expectSync( this, type ); + this[ type ](); + result = dataPriv.get( this, type ); + if ( saved !== result || notAsync ) { + dataPriv.set( this, type, false ); + } else { + result = {}; + } + if ( saved !== result ) { + + // Cancel the outer synthetic event + event.stopImmediatePropagation(); + event.preventDefault(); + return result.value; + } + + // If this is an inner synthetic event for an event with a bubbling surrogate + // (focus or blur), assume that the surrogate already propagated from triggering the + // native event and prevent that from happening again here. + // This technically gets the ordering wrong w.r.t. to `.trigger()` (in which the + // bubbling surrogate propagates *after* the non-bubbling base), but that seems + // less bad than duplication. + } else if ( ( jQuery.event.special[ type ] || {} ).delegateType ) { + event.stopPropagation(); + } + + // If this is a native event triggered above, everything is now in order + // Fire an inner synthetic event with the original arguments + } else if ( saved.length ) { + + // ...and capture the result + dataPriv.set( this, type, { + value: jQuery.event.trigger( + + // Support: IE <=9 - 11+ + // Extend with the prototype to reset the above stopImmediatePropagation() + jQuery.extend( saved[ 0 ], jQuery.Event.prototype ), + saved.slice( 1 ), + this + ) + } ); + + // Abort handling of the native event + event.stopImmediatePropagation(); + } + } + } ); +} + +jQuery.removeEvent = function( elem, type, handle ) { + + // This "if" is needed for plain objects + if ( elem.removeEventListener ) { + elem.removeEventListener( type, handle ); + } +}; + +jQuery.Event = function( src, props ) { + + // Allow instantiation without the 'new' keyword + if ( !( this instanceof jQuery.Event ) ) { + return new jQuery.Event( src, props ); + } + + // Event object + if ( src && src.type ) { + this.originalEvent = src; + this.type = src.type; + + // Events bubbling up the document may have been marked as prevented + // by a handler lower down the tree; reflect the correct value. + this.isDefaultPrevented = src.defaultPrevented || + src.defaultPrevented === undefined && + + // Support: Android <=2.3 only + src.returnValue === false ? + returnTrue : + returnFalse; + + // Create target properties + // Support: Safari <=6 - 7 only + // Target should not be a text node (#504, #13143) + this.target = ( src.target && src.target.nodeType === 3 ) ? + src.target.parentNode : + src.target; + + this.currentTarget = src.currentTarget; + this.relatedTarget = src.relatedTarget; + + // Event type + } else { + this.type = src; + } + + // Put explicitly provided properties onto the event object + if ( props ) { + jQuery.extend( this, props ); + } + + // Create a timestamp if incoming event doesn't have one + this.timeStamp = src && src.timeStamp || Date.now(); + + // Mark it as fixed + this[ jQuery.expando ] = true; +}; + +// jQuery.Event is based on DOM3 Events as specified by the ECMAScript Language Binding +// https://www.w3.org/TR/2003/WD-DOM-Level-3-Events-20030331/ecma-script-binding.html +jQuery.Event.prototype = { + constructor: jQuery.Event, + isDefaultPrevented: returnFalse, + isPropagationStopped: returnFalse, + isImmediatePropagationStopped: returnFalse, + isSimulated: false, + + preventDefault: function() { + var e = this.originalEvent; + + this.isDefaultPrevented = returnTrue; + + if ( e && !this.isSimulated ) { + e.preventDefault(); + } + }, + stopPropagation: function() { + var e = this.originalEvent; + + this.isPropagationStopped = returnTrue; + + if ( e && !this.isSimulated ) { + e.stopPropagation(); + } + }, + stopImmediatePropagation: function() { + var e = this.originalEvent; + + this.isImmediatePropagationStopped = returnTrue; + + if ( e && !this.isSimulated ) { + e.stopImmediatePropagation(); + } + + this.stopPropagation(); + } +}; + +// Includes all common event props including KeyEvent and MouseEvent specific props +jQuery.each( { + altKey: true, + bubbles: true, + cancelable: true, + changedTouches: true, + ctrlKey: true, + detail: true, + eventPhase: true, + metaKey: true, + pageX: true, + pageY: true, + shiftKey: true, + view: true, + "char": true, + code: true, + charCode: true, + key: true, + keyCode: true, + button: true, + buttons: true, + clientX: true, + clientY: true, + offsetX: true, + offsetY: true, + pointerId: true, + pointerType: true, + screenX: true, + screenY: true, + targetTouches: true, + toElement: true, + touches: true, + + which: function( event ) { + var button = event.button; + + // Add which for key events + if ( event.which == null && rkeyEvent.test( event.type ) ) { + return event.charCode != null ? event.charCode : event.keyCode; + } + + // Add which for click: 1 === left; 2 === middle; 3 === right + if ( !event.which && button !== undefined && rmouseEvent.test( event.type ) ) { + if ( button & 1 ) { + return 1; + } + + if ( button & 2 ) { + return 3; + } + + if ( button & 4 ) { + return 2; + } + + return 0; + } + + return event.which; + } +}, jQuery.event.addProp ); + +jQuery.each( { focus: "focusin", blur: "focusout" }, function( type, delegateType ) { + jQuery.event.special[ type ] = { + + // Utilize native event if possible so blur/focus sequence is correct + setup: function() { + + // Claim the first handler + // dataPriv.set( this, "focus", ... ) + // dataPriv.set( this, "blur", ... ) + leverageNative( this, type, expectSync ); + + // Return false to allow normal processing in the caller + return false; + }, + trigger: function() { + + // Force setup before trigger + leverageNative( this, type ); + + // Return non-false to allow normal event-path propagation + return true; + }, + + delegateType: delegateType + }; +} ); + +// Create mouseenter/leave events using mouseover/out and event-time checks +// so that event delegation works in jQuery. +// Do the same for pointerenter/pointerleave and pointerover/pointerout +// +// Support: Safari 7 only +// Safari sends mouseenter too often; see: +// https://bugs.chromium.org/p/chromium/issues/detail?id=470258 +// for the description of the bug (it existed in older Chrome versions as well). +jQuery.each( { + mouseenter: "mouseover", + mouseleave: "mouseout", + pointerenter: "pointerover", + pointerleave: "pointerout" +}, function( orig, fix ) { + jQuery.event.special[ orig ] = { + delegateType: fix, + bindType: fix, + + handle: function( event ) { + var ret, + target = this, + related = event.relatedTarget, + handleObj = event.handleObj; + + // For mouseenter/leave call the handler if related is outside the target. + // NB: No relatedTarget if the mouse left/entered the browser window + if ( !related || ( related !== target && !jQuery.contains( target, related ) ) ) { + event.type = handleObj.origType; + ret = handleObj.handler.apply( this, arguments ); + event.type = fix; + } + return ret; + } + }; +} ); + +jQuery.fn.extend( { + + on: function( types, selector, data, fn ) { + return on( this, types, selector, data, fn ); + }, + one: function( types, selector, data, fn ) { + return on( this, types, selector, data, fn, 1 ); + }, + off: function( types, selector, fn ) { + var handleObj, type; + if ( types && types.preventDefault && types.handleObj ) { + + // ( event ) dispatched jQuery.Event + handleObj = types.handleObj; + jQuery( types.delegateTarget ).off( + handleObj.namespace ? + handleObj.origType + "." + handleObj.namespace : + handleObj.origType, + handleObj.selector, + handleObj.handler + ); + return this; + } + if ( typeof types === "object" ) { + + // ( types-object [, selector] ) + for ( type in types ) { + this.off( type, selector, types[ type ] ); + } + return this; + } + if ( selector === false || typeof selector === "function" ) { + + // ( types [, fn] ) + fn = selector; + selector = undefined; + } + if ( fn === false ) { + fn = returnFalse; + } + return this.each( function() { + jQuery.event.remove( this, types, fn, selector ); + } ); + } +} ); + + +var + + // Support: IE <=10 - 11, Edge 12 - 13 only + // In IE/Edge using regex groups here causes severe slowdowns. + // See https://connect.microsoft.com/IE/feedback/details/1736512/ + rnoInnerhtml = /\s*$/g; + +// Prefer a tbody over its parent table for containing new rows +function manipulationTarget( elem, content ) { + if ( nodeName( elem, "table" ) && + nodeName( content.nodeType !== 11 ? content : content.firstChild, "tr" ) ) { + + return jQuery( elem ).children( "tbody" )[ 0 ] || elem; + } + + return elem; +} + +// Replace/restore the type attribute of script elements for safe DOM manipulation +function disableScript( elem ) { + elem.type = ( elem.getAttribute( "type" ) !== null ) + "/" + elem.type; + return elem; +} +function restoreScript( elem ) { + if ( ( elem.type || "" ).slice( 0, 5 ) === "true/" ) { + elem.type = elem.type.slice( 5 ); + } else { + elem.removeAttribute( "type" ); + } + + return elem; +} + +function cloneCopyEvent( src, dest ) { + var i, l, type, pdataOld, udataOld, udataCur, events; + + if ( dest.nodeType !== 1 ) { + return; + } + + // 1. Copy private data: events, handlers, etc. + if ( dataPriv.hasData( src ) ) { + pdataOld = dataPriv.get( src ); + events = pdataOld.events; + + if ( events ) { + dataPriv.remove( dest, "handle events" ); + + for ( type in events ) { + for ( i = 0, l = events[ type ].length; i < l; i++ ) { + jQuery.event.add( dest, type, events[ type ][ i ] ); + } + } + } + } + + // 2. Copy user data + if ( dataUser.hasData( src ) ) { + udataOld = dataUser.access( src ); + udataCur = jQuery.extend( {}, udataOld ); + + dataUser.set( dest, udataCur ); + } +} + +// Fix IE bugs, see support tests +function fixInput( src, dest ) { + var nodeName = dest.nodeName.toLowerCase(); + + // Fails to persist the checked state of a cloned checkbox or radio button. + if ( nodeName === "input" && rcheckableType.test( src.type ) ) { + dest.checked = src.checked; + + // Fails to return the selected option to the default selected state when cloning options + } else if ( nodeName === "input" || nodeName === "textarea" ) { + dest.defaultValue = src.defaultValue; + } +} + +function domManip( collection, args, callback, ignored ) { + + // Flatten any nested arrays + args = flat( args ); + + var fragment, first, scripts, hasScripts, node, doc, + i = 0, + l = collection.length, + iNoClone = l - 1, + value = args[ 0 ], + valueIsFunction = isFunction( value ); + + // We can't cloneNode fragments that contain checked, in WebKit + if ( valueIsFunction || + ( l > 1 && typeof value === "string" && + !support.checkClone && rchecked.test( value ) ) ) { + return collection.each( function( index ) { + var self = collection.eq( index ); + if ( valueIsFunction ) { + args[ 0 ] = value.call( this, index, self.html() ); + } + domManip( self, args, callback, ignored ); + } ); + } + + if ( l ) { + fragment = buildFragment( args, collection[ 0 ].ownerDocument, false, collection, ignored ); + first = fragment.firstChild; + + if ( fragment.childNodes.length === 1 ) { + fragment = first; + } + + // Require either new content or an interest in ignored elements to invoke the callback + if ( first || ignored ) { + scripts = jQuery.map( getAll( fragment, "script" ), disableScript ); + hasScripts = scripts.length; + + // Use the original fragment for the last item + // instead of the first because it can end up + // being emptied incorrectly in certain situations (#8070). + for ( ; i < l; i++ ) { + node = fragment; + + if ( i !== iNoClone ) { + node = jQuery.clone( node, true, true ); + + // Keep references to cloned scripts for later restoration + if ( hasScripts ) { + + // Support: Android <=4.0 only, PhantomJS 1 only + // push.apply(_, arraylike) throws on ancient WebKit + jQuery.merge( scripts, getAll( node, "script" ) ); + } + } + + callback.call( collection[ i ], node, i ); + } + + if ( hasScripts ) { + doc = scripts[ scripts.length - 1 ].ownerDocument; + + // Reenable scripts + jQuery.map( scripts, restoreScript ); + + // Evaluate executable scripts on first document insertion + for ( i = 0; i < hasScripts; i++ ) { + node = scripts[ i ]; + if ( rscriptType.test( node.type || "" ) && + !dataPriv.access( node, "globalEval" ) && + jQuery.contains( doc, node ) ) { + + if ( node.src && ( node.type || "" ).toLowerCase() !== "module" ) { + + // Optional AJAX dependency, but won't run scripts if not present + if ( jQuery._evalUrl && !node.noModule ) { + jQuery._evalUrl( node.src, { + nonce: node.nonce || node.getAttribute( "nonce" ) + }, doc ); + } + } else { + DOMEval( node.textContent.replace( rcleanScript, "" ), node, doc ); + } + } + } + } + } + } + + return collection; +} + +function remove( elem, selector, keepData ) { + var node, + nodes = selector ? jQuery.filter( selector, elem ) : elem, + i = 0; + + for ( ; ( node = nodes[ i ] ) != null; i++ ) { + if ( !keepData && node.nodeType === 1 ) { + jQuery.cleanData( getAll( node ) ); + } + + if ( node.parentNode ) { + if ( keepData && isAttached( node ) ) { + setGlobalEval( getAll( node, "script" ) ); + } + node.parentNode.removeChild( node ); + } + } + + return elem; +} + +jQuery.extend( { + htmlPrefilter: function( html ) { + return html; + }, + + clone: function( elem, dataAndEvents, deepDataAndEvents ) { + var i, l, srcElements, destElements, + clone = elem.cloneNode( true ), + inPage = isAttached( elem ); + + // Fix IE cloning issues + if ( !support.noCloneChecked && ( elem.nodeType === 1 || elem.nodeType === 11 ) && + !jQuery.isXMLDoc( elem ) ) { + + // We eschew Sizzle here for performance reasons: https://jsperf.com/getall-vs-sizzle/2 + destElements = getAll( clone ); + srcElements = getAll( elem ); + + for ( i = 0, l = srcElements.length; i < l; i++ ) { + fixInput( srcElements[ i ], destElements[ i ] ); + } + } + + // Copy the events from the original to the clone + if ( dataAndEvents ) { + if ( deepDataAndEvents ) { + srcElements = srcElements || getAll( elem ); + destElements = destElements || getAll( clone ); + + for ( i = 0, l = srcElements.length; i < l; i++ ) { + cloneCopyEvent( srcElements[ i ], destElements[ i ] ); + } + } else { + cloneCopyEvent( elem, clone ); + } + } + + // Preserve script evaluation history + destElements = getAll( clone, "script" ); + if ( destElements.length > 0 ) { + setGlobalEval( destElements, !inPage && getAll( elem, "script" ) ); + } + + // Return the cloned set + return clone; + }, + + cleanData: function( elems ) { + var data, elem, type, + special = jQuery.event.special, + i = 0; + + for ( ; ( elem = elems[ i ] ) !== undefined; i++ ) { + if ( acceptData( elem ) ) { + if ( ( data = elem[ dataPriv.expando ] ) ) { + if ( data.events ) { + for ( type in data.events ) { + if ( special[ type ] ) { + jQuery.event.remove( elem, type ); + + // This is a shortcut to avoid jQuery.event.remove's overhead + } else { + jQuery.removeEvent( elem, type, data.handle ); + } + } + } + + // Support: Chrome <=35 - 45+ + // Assign undefined instead of using delete, see Data#remove + elem[ dataPriv.expando ] = undefined; + } + if ( elem[ dataUser.expando ] ) { + + // Support: Chrome <=35 - 45+ + // Assign undefined instead of using delete, see Data#remove + elem[ dataUser.expando ] = undefined; + } + } + } + } +} ); + +jQuery.fn.extend( { + detach: function( selector ) { + return remove( this, selector, true ); + }, + + remove: function( selector ) { + return remove( this, selector ); + }, + + text: function( value ) { + return access( this, function( value ) { + return value === undefined ? + jQuery.text( this ) : + this.empty().each( function() { + if ( this.nodeType === 1 || this.nodeType === 11 || this.nodeType === 9 ) { + this.textContent = value; + } + } ); + }, null, value, arguments.length ); + }, + + append: function() { + return domManip( this, arguments, function( elem ) { + if ( this.nodeType === 1 || this.nodeType === 11 || this.nodeType === 9 ) { + var target = manipulationTarget( this, elem ); + target.appendChild( elem ); + } + } ); + }, + + prepend: function() { + return domManip( this, arguments, function( elem ) { + if ( this.nodeType === 1 || this.nodeType === 11 || this.nodeType === 9 ) { + var target = manipulationTarget( this, elem ); + target.insertBefore( elem, target.firstChild ); + } + } ); + }, + + before: function() { + return domManip( this, arguments, function( elem ) { + if ( this.parentNode ) { + this.parentNode.insertBefore( elem, this ); + } + } ); + }, + + after: function() { + return domManip( this, arguments, function( elem ) { + if ( this.parentNode ) { + this.parentNode.insertBefore( elem, this.nextSibling ); + } + } ); + }, + + empty: function() { + var elem, + i = 0; + + for ( ; ( elem = this[ i ] ) != null; i++ ) { + if ( elem.nodeType === 1 ) { + + // Prevent memory leaks + jQuery.cleanData( getAll( elem, false ) ); + + // Remove any remaining nodes + elem.textContent = ""; + } + } + + return this; + }, + + clone: function( dataAndEvents, deepDataAndEvents ) { + dataAndEvents = dataAndEvents == null ? false : dataAndEvents; + deepDataAndEvents = deepDataAndEvents == null ? dataAndEvents : deepDataAndEvents; + + return this.map( function() { + return jQuery.clone( this, dataAndEvents, deepDataAndEvents ); + } ); + }, + + html: function( value ) { + return access( this, function( value ) { + var elem = this[ 0 ] || {}, + i = 0, + l = this.length; + + if ( value === undefined && elem.nodeType === 1 ) { + return elem.innerHTML; + } + + // See if we can take a shortcut and just use innerHTML + if ( typeof value === "string" && !rnoInnerhtml.test( value ) && + !wrapMap[ ( rtagName.exec( value ) || [ "", "" ] )[ 1 ].toLowerCase() ] ) { + + value = jQuery.htmlPrefilter( value ); + + try { + for ( ; i < l; i++ ) { + elem = this[ i ] || {}; + + // Remove element nodes and prevent memory leaks + if ( elem.nodeType === 1 ) { + jQuery.cleanData( getAll( elem, false ) ); + elem.innerHTML = value; + } + } + + elem = 0; + + // If using innerHTML throws an exception, use the fallback method + } catch ( e ) {} + } + + if ( elem ) { + this.empty().append( value ); + } + }, null, value, arguments.length ); + }, + + replaceWith: function() { + var ignored = []; + + // Make the changes, replacing each non-ignored context element with the new content + return domManip( this, arguments, function( elem ) { + var parent = this.parentNode; + + if ( jQuery.inArray( this, ignored ) < 0 ) { + jQuery.cleanData( getAll( this ) ); + if ( parent ) { + parent.replaceChild( elem, this ); + } + } + + // Force callback invocation + }, ignored ); + } +} ); + +jQuery.each( { + appendTo: "append", + prependTo: "prepend", + insertBefore: "before", + insertAfter: "after", + replaceAll: "replaceWith" +}, function( name, original ) { + jQuery.fn[ name ] = function( selector ) { + var elems, + ret = [], + insert = jQuery( selector ), + last = insert.length - 1, + i = 0; + + for ( ; i <= last; i++ ) { + elems = i === last ? this : this.clone( true ); + jQuery( insert[ i ] )[ original ]( elems ); + + // Support: Android <=4.0 only, PhantomJS 1 only + // .get() because push.apply(_, arraylike) throws on ancient WebKit + push.apply( ret, elems.get() ); + } + + return this.pushStack( ret ); + }; +} ); +var rnumnonpx = new RegExp( "^(" + pnum + ")(?!px)[a-z%]+$", "i" ); + +var getStyles = function( elem ) { + + // Support: IE <=11 only, Firefox <=30 (#15098, #14150) + // IE throws on elements created in popups + // FF meanwhile throws on frame elements through "defaultView.getComputedStyle" + var view = elem.ownerDocument.defaultView; + + if ( !view || !view.opener ) { + view = window; + } + + return view.getComputedStyle( elem ); + }; + +var swap = function( elem, options, callback ) { + var ret, name, + old = {}; + + // Remember the old values, and insert the new ones + for ( name in options ) { + old[ name ] = elem.style[ name ]; + elem.style[ name ] = options[ name ]; + } + + ret = callback.call( elem ); + + // Revert the old values + for ( name in options ) { + elem.style[ name ] = old[ name ]; + } + + return ret; +}; + + +var rboxStyle = new RegExp( cssExpand.join( "|" ), "i" ); + + + +( function() { + + // Executing both pixelPosition & boxSizingReliable tests require only one layout + // so they're executed at the same time to save the second computation. + function computeStyleTests() { + + // This is a singleton, we need to execute it only once + if ( !div ) { + return; + } + + container.style.cssText = "position:absolute;left:-11111px;width:60px;" + + "margin-top:1px;padding:0;border:0"; + div.style.cssText = + "position:relative;display:block;box-sizing:border-box;overflow:scroll;" + + "margin:auto;border:1px;padding:1px;" + + "width:60%;top:1%"; + documentElement.appendChild( container ).appendChild( div ); + + var divStyle = window.getComputedStyle( div ); + pixelPositionVal = divStyle.top !== "1%"; + + // Support: Android 4.0 - 4.3 only, Firefox <=3 - 44 + reliableMarginLeftVal = roundPixelMeasures( divStyle.marginLeft ) === 12; + + // Support: Android 4.0 - 4.3 only, Safari <=9.1 - 10.1, iOS <=7.0 - 9.3 + // Some styles come back with percentage values, even though they shouldn't + div.style.right = "60%"; + pixelBoxStylesVal = roundPixelMeasures( divStyle.right ) === 36; + + // Support: IE 9 - 11 only + // Detect misreporting of content dimensions for box-sizing:border-box elements + boxSizingReliableVal = roundPixelMeasures( divStyle.width ) === 36; + + // Support: IE 9 only + // Detect overflow:scroll screwiness (gh-3699) + // Support: Chrome <=64 + // Don't get tricked when zoom affects offsetWidth (gh-4029) + div.style.position = "absolute"; + scrollboxSizeVal = roundPixelMeasures( div.offsetWidth / 3 ) === 12; + + documentElement.removeChild( container ); + + // Nullify the div so it wouldn't be stored in the memory and + // it will also be a sign that checks already performed + div = null; + } + + function roundPixelMeasures( measure ) { + return Math.round( parseFloat( measure ) ); + } + + var pixelPositionVal, boxSizingReliableVal, scrollboxSizeVal, pixelBoxStylesVal, + reliableTrDimensionsVal, reliableMarginLeftVal, + container = document.createElement( "div" ), + div = document.createElement( "div" ); + + // Finish early in limited (non-browser) environments + if ( !div.style ) { + return; + } + + // Support: IE <=9 - 11 only + // Style of cloned element affects source element cloned (#8908) + div.style.backgroundClip = "content-box"; + div.cloneNode( true ).style.backgroundClip = ""; + support.clearCloneStyle = div.style.backgroundClip === "content-box"; + + jQuery.extend( support, { + boxSizingReliable: function() { + computeStyleTests(); + return boxSizingReliableVal; + }, + pixelBoxStyles: function() { + computeStyleTests(); + return pixelBoxStylesVal; + }, + pixelPosition: function() { + computeStyleTests(); + return pixelPositionVal; + }, + reliableMarginLeft: function() { + computeStyleTests(); + return reliableMarginLeftVal; + }, + scrollboxSize: function() { + computeStyleTests(); + return scrollboxSizeVal; + }, + + // Support: IE 9 - 11+, Edge 15 - 18+ + // IE/Edge misreport `getComputedStyle` of table rows with width/height + // set in CSS while `offset*` properties report correct values. + // Behavior in IE 9 is more subtle than in newer versions & it passes + // some versions of this test; make sure not to make it pass there! + reliableTrDimensions: function() { + var table, tr, trChild, trStyle; + if ( reliableTrDimensionsVal == null ) { + table = document.createElement( "table" ); + tr = document.createElement( "tr" ); + trChild = document.createElement( "div" ); + + table.style.cssText = "position:absolute;left:-11111px"; + tr.style.height = "1px"; + trChild.style.height = "9px"; + + documentElement + .appendChild( table ) + .appendChild( tr ) + .appendChild( trChild ); + + trStyle = window.getComputedStyle( tr ); + reliableTrDimensionsVal = parseInt( trStyle.height ) > 3; + + documentElement.removeChild( table ); + } + return reliableTrDimensionsVal; + } + } ); +} )(); + + +function curCSS( elem, name, computed ) { + var width, minWidth, maxWidth, ret, + + // Support: Firefox 51+ + // Retrieving style before computed somehow + // fixes an issue with getting wrong values + // on detached elements + style = elem.style; + + computed = computed || getStyles( elem ); + + // getPropertyValue is needed for: + // .css('filter') (IE 9 only, #12537) + // .css('--customProperty) (#3144) + if ( computed ) { + ret = computed.getPropertyValue( name ) || computed[ name ]; + + if ( ret === "" && !isAttached( elem ) ) { + ret = jQuery.style( elem, name ); + } + + // A tribute to the "awesome hack by Dean Edwards" + // Android Browser returns percentage for some values, + // but width seems to be reliably pixels. + // This is against the CSSOM draft spec: + // https://drafts.csswg.org/cssom/#resolved-values + if ( !support.pixelBoxStyles() && rnumnonpx.test( ret ) && rboxStyle.test( name ) ) { + + // Remember the original values + width = style.width; + minWidth = style.minWidth; + maxWidth = style.maxWidth; + + // Put in the new values to get a computed value out + style.minWidth = style.maxWidth = style.width = ret; + ret = computed.width; + + // Revert the changed values + style.width = width; + style.minWidth = minWidth; + style.maxWidth = maxWidth; + } + } + + return ret !== undefined ? + + // Support: IE <=9 - 11 only + // IE returns zIndex value as an integer. + ret + "" : + ret; +} + + +function addGetHookIf( conditionFn, hookFn ) { + + // Define the hook, we'll check on the first run if it's really needed. + return { + get: function() { + if ( conditionFn() ) { + + // Hook not needed (or it's not possible to use it due + // to missing dependency), remove it. + delete this.get; + return; + } + + // Hook needed; redefine it so that the support test is not executed again. + return ( this.get = hookFn ).apply( this, arguments ); + } + }; +} + + +var cssPrefixes = [ "Webkit", "Moz", "ms" ], + emptyStyle = document.createElement( "div" ).style, + vendorProps = {}; + +// Return a vendor-prefixed property or undefined +function vendorPropName( name ) { + + // Check for vendor prefixed names + var capName = name[ 0 ].toUpperCase() + name.slice( 1 ), + i = cssPrefixes.length; + + while ( i-- ) { + name = cssPrefixes[ i ] + capName; + if ( name in emptyStyle ) { + return name; + } + } +} + +// Return a potentially-mapped jQuery.cssProps or vendor prefixed property +function finalPropName( name ) { + var final = jQuery.cssProps[ name ] || vendorProps[ name ]; + + if ( final ) { + return final; + } + if ( name in emptyStyle ) { + return name; + } + return vendorProps[ name ] = vendorPropName( name ) || name; +} + + +var + + // Swappable if display is none or starts with table + // except "table", "table-cell", or "table-caption" + // See here for display values: https://developer.mozilla.org/en-US/docs/CSS/display + rdisplayswap = /^(none|table(?!-c[ea]).+)/, + rcustomProp = /^--/, + cssShow = { position: "absolute", visibility: "hidden", display: "block" }, + cssNormalTransform = { + letterSpacing: "0", + fontWeight: "400" + }; + +function setPositiveNumber( _elem, value, subtract ) { + + // Any relative (+/-) values have already been + // normalized at this point + var matches = rcssNum.exec( value ); + return matches ? + + // Guard against undefined "subtract", e.g., when used as in cssHooks + Math.max( 0, matches[ 2 ] - ( subtract || 0 ) ) + ( matches[ 3 ] || "px" ) : + value; +} + +function boxModelAdjustment( elem, dimension, box, isBorderBox, styles, computedVal ) { + var i = dimension === "width" ? 1 : 0, + extra = 0, + delta = 0; + + // Adjustment may not be necessary + if ( box === ( isBorderBox ? "border" : "content" ) ) { + return 0; + } + + for ( ; i < 4; i += 2 ) { + + // Both box models exclude margin + if ( box === "margin" ) { + delta += jQuery.css( elem, box + cssExpand[ i ], true, styles ); + } + + // If we get here with a content-box, we're seeking "padding" or "border" or "margin" + if ( !isBorderBox ) { + + // Add padding + delta += jQuery.css( elem, "padding" + cssExpand[ i ], true, styles ); + + // For "border" or "margin", add border + if ( box !== "padding" ) { + delta += jQuery.css( elem, "border" + cssExpand[ i ] + "Width", true, styles ); + + // But still keep track of it otherwise + } else { + extra += jQuery.css( elem, "border" + cssExpand[ i ] + "Width", true, styles ); + } + + // If we get here with a border-box (content + padding + border), we're seeking "content" or + // "padding" or "margin" + } else { + + // For "content", subtract padding + if ( box === "content" ) { + delta -= jQuery.css( elem, "padding" + cssExpand[ i ], true, styles ); + } + + // For "content" or "padding", subtract border + if ( box !== "margin" ) { + delta -= jQuery.css( elem, "border" + cssExpand[ i ] + "Width", true, styles ); + } + } + } + + // Account for positive content-box scroll gutter when requested by providing computedVal + if ( !isBorderBox && computedVal >= 0 ) { + + // offsetWidth/offsetHeight is a rounded sum of content, padding, scroll gutter, and border + // Assuming integer scroll gutter, subtract the rest and round down + delta += Math.max( 0, Math.ceil( + elem[ "offset" + dimension[ 0 ].toUpperCase() + dimension.slice( 1 ) ] - + computedVal - + delta - + extra - + 0.5 + + // If offsetWidth/offsetHeight is unknown, then we can't determine content-box scroll gutter + // Use an explicit zero to avoid NaN (gh-3964) + ) ) || 0; + } + + return delta; +} + +function getWidthOrHeight( elem, dimension, extra ) { + + // Start with computed style + var styles = getStyles( elem ), + + // To avoid forcing a reflow, only fetch boxSizing if we need it (gh-4322). + // Fake content-box until we know it's needed to know the true value. + boxSizingNeeded = !support.boxSizingReliable() || extra, + isBorderBox = boxSizingNeeded && + jQuery.css( elem, "boxSizing", false, styles ) === "border-box", + valueIsBorderBox = isBorderBox, + + val = curCSS( elem, dimension, styles ), + offsetProp = "offset" + dimension[ 0 ].toUpperCase() + dimension.slice( 1 ); + + // Support: Firefox <=54 + // Return a confounding non-pixel value or feign ignorance, as appropriate. + if ( rnumnonpx.test( val ) ) { + if ( !extra ) { + return val; + } + val = "auto"; + } + + + // Support: IE 9 - 11 only + // Use offsetWidth/offsetHeight for when box sizing is unreliable. + // In those cases, the computed value can be trusted to be border-box. + if ( ( !support.boxSizingReliable() && isBorderBox || + + // Support: IE 10 - 11+, Edge 15 - 18+ + // IE/Edge misreport `getComputedStyle` of table rows with width/height + // set in CSS while `offset*` properties report correct values. + // Interestingly, in some cases IE 9 doesn't suffer from this issue. + !support.reliableTrDimensions() && nodeName( elem, "tr" ) || + + // Fall back to offsetWidth/offsetHeight when value is "auto" + // This happens for inline elements with no explicit setting (gh-3571) + val === "auto" || + + // Support: Android <=4.1 - 4.3 only + // Also use offsetWidth/offsetHeight for misreported inline dimensions (gh-3602) + !parseFloat( val ) && jQuery.css( elem, "display", false, styles ) === "inline" ) && + + // Make sure the element is visible & connected + elem.getClientRects().length ) { + + isBorderBox = jQuery.css( elem, "boxSizing", false, styles ) === "border-box"; + + // Where available, offsetWidth/offsetHeight approximate border box dimensions. + // Where not available (e.g., SVG), assume unreliable box-sizing and interpret the + // retrieved value as a content box dimension. + valueIsBorderBox = offsetProp in elem; + if ( valueIsBorderBox ) { + val = elem[ offsetProp ]; + } + } + + // Normalize "" and auto + val = parseFloat( val ) || 0; + + // Adjust for the element's box model + return ( val + + boxModelAdjustment( + elem, + dimension, + extra || ( isBorderBox ? "border" : "content" ), + valueIsBorderBox, + styles, + + // Provide the current computed size to request scroll gutter calculation (gh-3589) + val + ) + ) + "px"; +} + +jQuery.extend( { + + // Add in style property hooks for overriding the default + // behavior of getting and setting a style property + cssHooks: { + opacity: { + get: function( elem, computed ) { + if ( computed ) { + + // We should always get a number back from opacity + var ret = curCSS( elem, "opacity" ); + return ret === "" ? "1" : ret; + } + } + } + }, + + // Don't automatically add "px" to these possibly-unitless properties + cssNumber: { + "animationIterationCount": true, + "columnCount": true, + "fillOpacity": true, + "flexGrow": true, + "flexShrink": true, + "fontWeight": true, + "gridArea": true, + "gridColumn": true, + "gridColumnEnd": true, + "gridColumnStart": true, + "gridRow": true, + "gridRowEnd": true, + "gridRowStart": true, + "lineHeight": true, + "opacity": true, + "order": true, + "orphans": true, + "widows": true, + "zIndex": true, + "zoom": true + }, + + // Add in properties whose names you wish to fix before + // setting or getting the value + cssProps: {}, + + // Get and set the style property on a DOM Node + style: function( elem, name, value, extra ) { + + // Don't set styles on text and comment nodes + if ( !elem || elem.nodeType === 3 || elem.nodeType === 8 || !elem.style ) { + return; + } + + // Make sure that we're working with the right name + var ret, type, hooks, + origName = camelCase( name ), + isCustomProp = rcustomProp.test( name ), + style = elem.style; + + // Make sure that we're working with the right name. We don't + // want to query the value if it is a CSS custom property + // since they are user-defined. + if ( !isCustomProp ) { + name = finalPropName( origName ); + } + + // Gets hook for the prefixed version, then unprefixed version + hooks = jQuery.cssHooks[ name ] || jQuery.cssHooks[ origName ]; + + // Check if we're setting a value + if ( value !== undefined ) { + type = typeof value; + + // Convert "+=" or "-=" to relative numbers (#7345) + if ( type === "string" && ( ret = rcssNum.exec( value ) ) && ret[ 1 ] ) { + value = adjustCSS( elem, name, ret ); + + // Fixes bug #9237 + type = "number"; + } + + // Make sure that null and NaN values aren't set (#7116) + if ( value == null || value !== value ) { + return; + } + + // If a number was passed in, add the unit (except for certain CSS properties) + // The isCustomProp check can be removed in jQuery 4.0 when we only auto-append + // "px" to a few hardcoded values. + if ( type === "number" && !isCustomProp ) { + value += ret && ret[ 3 ] || ( jQuery.cssNumber[ origName ] ? "" : "px" ); + } + + // background-* props affect original clone's values + if ( !support.clearCloneStyle && value === "" && name.indexOf( "background" ) === 0 ) { + style[ name ] = "inherit"; + } + + // If a hook was provided, use that value, otherwise just set the specified value + if ( !hooks || !( "set" in hooks ) || + ( value = hooks.set( elem, value, extra ) ) !== undefined ) { + + if ( isCustomProp ) { + style.setProperty( name, value ); + } else { + style[ name ] = value; + } + } + + } else { + + // If a hook was provided get the non-computed value from there + if ( hooks && "get" in hooks && + ( ret = hooks.get( elem, false, extra ) ) !== undefined ) { + + return ret; + } + + // Otherwise just get the value from the style object + return style[ name ]; + } + }, + + css: function( elem, name, extra, styles ) { + var val, num, hooks, + origName = camelCase( name ), + isCustomProp = rcustomProp.test( name ); + + // Make sure that we're working with the right name. We don't + // want to modify the value if it is a CSS custom property + // since they are user-defined. + if ( !isCustomProp ) { + name = finalPropName( origName ); + } + + // Try prefixed name followed by the unprefixed name + hooks = jQuery.cssHooks[ name ] || jQuery.cssHooks[ origName ]; + + // If a hook was provided get the computed value from there + if ( hooks && "get" in hooks ) { + val = hooks.get( elem, true, extra ); + } + + // Otherwise, if a way to get the computed value exists, use that + if ( val === undefined ) { + val = curCSS( elem, name, styles ); + } + + // Convert "normal" to computed value + if ( val === "normal" && name in cssNormalTransform ) { + val = cssNormalTransform[ name ]; + } + + // Make numeric if forced or a qualifier was provided and val looks numeric + if ( extra === "" || extra ) { + num = parseFloat( val ); + return extra === true || isFinite( num ) ? num || 0 : val; + } + + return val; + } +} ); + +jQuery.each( [ "height", "width" ], function( _i, dimension ) { + jQuery.cssHooks[ dimension ] = { + get: function( elem, computed, extra ) { + if ( computed ) { + + // Certain elements can have dimension info if we invisibly show them + // but it must have a current display style that would benefit + return rdisplayswap.test( jQuery.css( elem, "display" ) ) && + + // Support: Safari 8+ + // Table columns in Safari have non-zero offsetWidth & zero + // getBoundingClientRect().width unless display is changed. + // Support: IE <=11 only + // Running getBoundingClientRect on a disconnected node + // in IE throws an error. + ( !elem.getClientRects().length || !elem.getBoundingClientRect().width ) ? + swap( elem, cssShow, function() { + return getWidthOrHeight( elem, dimension, extra ); + } ) : + getWidthOrHeight( elem, dimension, extra ); + } + }, + + set: function( elem, value, extra ) { + var matches, + styles = getStyles( elem ), + + // Only read styles.position if the test has a chance to fail + // to avoid forcing a reflow. + scrollboxSizeBuggy = !support.scrollboxSize() && + styles.position === "absolute", + + // To avoid forcing a reflow, only fetch boxSizing if we need it (gh-3991) + boxSizingNeeded = scrollboxSizeBuggy || extra, + isBorderBox = boxSizingNeeded && + jQuery.css( elem, "boxSizing", false, styles ) === "border-box", + subtract = extra ? + boxModelAdjustment( + elem, + dimension, + extra, + isBorderBox, + styles + ) : + 0; + + // Account for unreliable border-box dimensions by comparing offset* to computed and + // faking a content-box to get border and padding (gh-3699) + if ( isBorderBox && scrollboxSizeBuggy ) { + subtract -= Math.ceil( + elem[ "offset" + dimension[ 0 ].toUpperCase() + dimension.slice( 1 ) ] - + parseFloat( styles[ dimension ] ) - + boxModelAdjustment( elem, dimension, "border", false, styles ) - + 0.5 + ); + } + + // Convert to pixels if value adjustment is needed + if ( subtract && ( matches = rcssNum.exec( value ) ) && + ( matches[ 3 ] || "px" ) !== "px" ) { + + elem.style[ dimension ] = value; + value = jQuery.css( elem, dimension ); + } + + return setPositiveNumber( elem, value, subtract ); + } + }; +} ); + +jQuery.cssHooks.marginLeft = addGetHookIf( support.reliableMarginLeft, + function( elem, computed ) { + if ( computed ) { + return ( parseFloat( curCSS( elem, "marginLeft" ) ) || + elem.getBoundingClientRect().left - + swap( elem, { marginLeft: 0 }, function() { + return elem.getBoundingClientRect().left; + } ) + ) + "px"; + } + } +); + +// These hooks are used by animate to expand properties +jQuery.each( { + margin: "", + padding: "", + border: "Width" +}, function( prefix, suffix ) { + jQuery.cssHooks[ prefix + suffix ] = { + expand: function( value ) { + var i = 0, + expanded = {}, + + // Assumes a single number if not a string + parts = typeof value === "string" ? value.split( " " ) : [ value ]; + + for ( ; i < 4; i++ ) { + expanded[ prefix + cssExpand[ i ] + suffix ] = + parts[ i ] || parts[ i - 2 ] || parts[ 0 ]; + } + + return expanded; + } + }; + + if ( prefix !== "margin" ) { + jQuery.cssHooks[ prefix + suffix ].set = setPositiveNumber; + } +} ); + +jQuery.fn.extend( { + css: function( name, value ) { + return access( this, function( elem, name, value ) { + var styles, len, + map = {}, + i = 0; + + if ( Array.isArray( name ) ) { + styles = getStyles( elem ); + len = name.length; + + for ( ; i < len; i++ ) { + map[ name[ i ] ] = jQuery.css( elem, name[ i ], false, styles ); + } + + return map; + } + + return value !== undefined ? + jQuery.style( elem, name, value ) : + jQuery.css( elem, name ); + }, name, value, arguments.length > 1 ); + } +} ); + + +function Tween( elem, options, prop, end, easing ) { + return new Tween.prototype.init( elem, options, prop, end, easing ); +} +jQuery.Tween = Tween; + +Tween.prototype = { + constructor: Tween, + init: function( elem, options, prop, end, easing, unit ) { + this.elem = elem; + this.prop = prop; + this.easing = easing || jQuery.easing._default; + this.options = options; + this.start = this.now = this.cur(); + this.end = end; + this.unit = unit || ( jQuery.cssNumber[ prop ] ? "" : "px" ); + }, + cur: function() { + var hooks = Tween.propHooks[ this.prop ]; + + return hooks && hooks.get ? + hooks.get( this ) : + Tween.propHooks._default.get( this ); + }, + run: function( percent ) { + var eased, + hooks = Tween.propHooks[ this.prop ]; + + if ( this.options.duration ) { + this.pos = eased = jQuery.easing[ this.easing ]( + percent, this.options.duration * percent, 0, 1, this.options.duration + ); + } else { + this.pos = eased = percent; + } + this.now = ( this.end - this.start ) * eased + this.start; + + if ( this.options.step ) { + this.options.step.call( this.elem, this.now, this ); + } + + if ( hooks && hooks.set ) { + hooks.set( this ); + } else { + Tween.propHooks._default.set( this ); + } + return this; + } +}; + +Tween.prototype.init.prototype = Tween.prototype; + +Tween.propHooks = { + _default: { + get: function( tween ) { + var result; + + // Use a property on the element directly when it is not a DOM element, + // or when there is no matching style property that exists. + if ( tween.elem.nodeType !== 1 || + tween.elem[ tween.prop ] != null && tween.elem.style[ tween.prop ] == null ) { + return tween.elem[ tween.prop ]; + } + + // Passing an empty string as a 3rd parameter to .css will automatically + // attempt a parseFloat and fallback to a string if the parse fails. + // Simple values such as "10px" are parsed to Float; + // complex values such as "rotate(1rad)" are returned as-is. + result = jQuery.css( tween.elem, tween.prop, "" ); + + // Empty strings, null, undefined and "auto" are converted to 0. + return !result || result === "auto" ? 0 : result; + }, + set: function( tween ) { + + // Use step hook for back compat. + // Use cssHook if its there. + // Use .style if available and use plain properties where available. + if ( jQuery.fx.step[ tween.prop ] ) { + jQuery.fx.step[ tween.prop ]( tween ); + } else if ( tween.elem.nodeType === 1 && ( + jQuery.cssHooks[ tween.prop ] || + tween.elem.style[ finalPropName( tween.prop ) ] != null ) ) { + jQuery.style( tween.elem, tween.prop, tween.now + tween.unit ); + } else { + tween.elem[ tween.prop ] = tween.now; + } + } + } +}; + +// Support: IE <=9 only +// Panic based approach to setting things on disconnected nodes +Tween.propHooks.scrollTop = Tween.propHooks.scrollLeft = { + set: function( tween ) { + if ( tween.elem.nodeType && tween.elem.parentNode ) { + tween.elem[ tween.prop ] = tween.now; + } + } +}; + +jQuery.easing = { + linear: function( p ) { + return p; + }, + swing: function( p ) { + return 0.5 - Math.cos( p * Math.PI ) / 2; + }, + _default: "swing" +}; + +jQuery.fx = Tween.prototype.init; + +// Back compat <1.8 extension point +jQuery.fx.step = {}; + + + + +var + fxNow, inProgress, + rfxtypes = /^(?:toggle|show|hide)$/, + rrun = /queueHooks$/; + +function schedule() { + if ( inProgress ) { + if ( document.hidden === false && window.requestAnimationFrame ) { + window.requestAnimationFrame( schedule ); + } else { + window.setTimeout( schedule, jQuery.fx.interval ); + } + + jQuery.fx.tick(); + } +} + +// Animations created synchronously will run synchronously +function createFxNow() { + window.setTimeout( function() { + fxNow = undefined; + } ); + return ( fxNow = Date.now() ); +} + +// Generate parameters to create a standard animation +function genFx( type, includeWidth ) { + var which, + i = 0, + attrs = { height: type }; + + // If we include width, step value is 1 to do all cssExpand values, + // otherwise step value is 2 to skip over Left and Right + includeWidth = includeWidth ? 1 : 0; + for ( ; i < 4; i += 2 - includeWidth ) { + which = cssExpand[ i ]; + attrs[ "margin" + which ] = attrs[ "padding" + which ] = type; + } + + if ( includeWidth ) { + attrs.opacity = attrs.width = type; + } + + return attrs; +} + +function createTween( value, prop, animation ) { + var tween, + collection = ( Animation.tweeners[ prop ] || [] ).concat( Animation.tweeners[ "*" ] ), + index = 0, + length = collection.length; + for ( ; index < length; index++ ) { + if ( ( tween = collection[ index ].call( animation, prop, value ) ) ) { + + // We're done with this property + return tween; + } + } +} + +function defaultPrefilter( elem, props, opts ) { + var prop, value, toggle, hooks, oldfire, propTween, restoreDisplay, display, + isBox = "width" in props || "height" in props, + anim = this, + orig = {}, + style = elem.style, + hidden = elem.nodeType && isHiddenWithinTree( elem ), + dataShow = dataPriv.get( elem, "fxshow" ); + + // Queue-skipping animations hijack the fx hooks + if ( !opts.queue ) { + hooks = jQuery._queueHooks( elem, "fx" ); + if ( hooks.unqueued == null ) { + hooks.unqueued = 0; + oldfire = hooks.empty.fire; + hooks.empty.fire = function() { + if ( !hooks.unqueued ) { + oldfire(); + } + }; + } + hooks.unqueued++; + + anim.always( function() { + + // Ensure the complete handler is called before this completes + anim.always( function() { + hooks.unqueued--; + if ( !jQuery.queue( elem, "fx" ).length ) { + hooks.empty.fire(); + } + } ); + } ); + } + + // Detect show/hide animations + for ( prop in props ) { + value = props[ prop ]; + if ( rfxtypes.test( value ) ) { + delete props[ prop ]; + toggle = toggle || value === "toggle"; + if ( value === ( hidden ? "hide" : "show" ) ) { + + // Pretend to be hidden if this is a "show" and + // there is still data from a stopped show/hide + if ( value === "show" && dataShow && dataShow[ prop ] !== undefined ) { + hidden = true; + + // Ignore all other no-op show/hide data + } else { + continue; + } + } + orig[ prop ] = dataShow && dataShow[ prop ] || jQuery.style( elem, prop ); + } + } + + // Bail out if this is a no-op like .hide().hide() + propTween = !jQuery.isEmptyObject( props ); + if ( !propTween && jQuery.isEmptyObject( orig ) ) { + return; + } + + // Restrict "overflow" and "display" styles during box animations + if ( isBox && elem.nodeType === 1 ) { + + // Support: IE <=9 - 11, Edge 12 - 15 + // Record all 3 overflow attributes because IE does not infer the shorthand + // from identically-valued overflowX and overflowY and Edge just mirrors + // the overflowX value there. + opts.overflow = [ style.overflow, style.overflowX, style.overflowY ]; + + // Identify a display type, preferring old show/hide data over the CSS cascade + restoreDisplay = dataShow && dataShow.display; + if ( restoreDisplay == null ) { + restoreDisplay = dataPriv.get( elem, "display" ); + } + display = jQuery.css( elem, "display" ); + if ( display === "none" ) { + if ( restoreDisplay ) { + display = restoreDisplay; + } else { + + // Get nonempty value(s) by temporarily forcing visibility + showHide( [ elem ], true ); + restoreDisplay = elem.style.display || restoreDisplay; + display = jQuery.css( elem, "display" ); + showHide( [ elem ] ); + } + } + + // Animate inline elements as inline-block + if ( display === "inline" || display === "inline-block" && restoreDisplay != null ) { + if ( jQuery.css( elem, "float" ) === "none" ) { + + // Restore the original display value at the end of pure show/hide animations + if ( !propTween ) { + anim.done( function() { + style.display = restoreDisplay; + } ); + if ( restoreDisplay == null ) { + display = style.display; + restoreDisplay = display === "none" ? "" : display; + } + } + style.display = "inline-block"; + } + } + } + + if ( opts.overflow ) { + style.overflow = "hidden"; + anim.always( function() { + style.overflow = opts.overflow[ 0 ]; + style.overflowX = opts.overflow[ 1 ]; + style.overflowY = opts.overflow[ 2 ]; + } ); + } + + // Implement show/hide animations + propTween = false; + for ( prop in orig ) { + + // General show/hide setup for this element animation + if ( !propTween ) { + if ( dataShow ) { + if ( "hidden" in dataShow ) { + hidden = dataShow.hidden; + } + } else { + dataShow = dataPriv.access( elem, "fxshow", { display: restoreDisplay } ); + } + + // Store hidden/visible for toggle so `.stop().toggle()` "reverses" + if ( toggle ) { + dataShow.hidden = !hidden; + } + + // Show elements before animating them + if ( hidden ) { + showHide( [ elem ], true ); + } + + /* eslint-disable no-loop-func */ + + anim.done( function() { + + /* eslint-enable no-loop-func */ + + // The final step of a "hide" animation is actually hiding the element + if ( !hidden ) { + showHide( [ elem ] ); + } + dataPriv.remove( elem, "fxshow" ); + for ( prop in orig ) { + jQuery.style( elem, prop, orig[ prop ] ); + } + } ); + } + + // Per-property setup + propTween = createTween( hidden ? dataShow[ prop ] : 0, prop, anim ); + if ( !( prop in dataShow ) ) { + dataShow[ prop ] = propTween.start; + if ( hidden ) { + propTween.end = propTween.start; + propTween.start = 0; + } + } + } +} + +function propFilter( props, specialEasing ) { + var index, name, easing, value, hooks; + + // camelCase, specialEasing and expand cssHook pass + for ( index in props ) { + name = camelCase( index ); + easing = specialEasing[ name ]; + value = props[ index ]; + if ( Array.isArray( value ) ) { + easing = value[ 1 ]; + value = props[ index ] = value[ 0 ]; + } + + if ( index !== name ) { + props[ name ] = value; + delete props[ index ]; + } + + hooks = jQuery.cssHooks[ name ]; + if ( hooks && "expand" in hooks ) { + value = hooks.expand( value ); + delete props[ name ]; + + // Not quite $.extend, this won't overwrite existing keys. + // Reusing 'index' because we have the correct "name" + for ( index in value ) { + if ( !( index in props ) ) { + props[ index ] = value[ index ]; + specialEasing[ index ] = easing; + } + } + } else { + specialEasing[ name ] = easing; + } + } +} + +function Animation( elem, properties, options ) { + var result, + stopped, + index = 0, + length = Animation.prefilters.length, + deferred = jQuery.Deferred().always( function() { + + // Don't match elem in the :animated selector + delete tick.elem; + } ), + tick = function() { + if ( stopped ) { + return false; + } + var currentTime = fxNow || createFxNow(), + remaining = Math.max( 0, animation.startTime + animation.duration - currentTime ), + + // Support: Android 2.3 only + // Archaic crash bug won't allow us to use `1 - ( 0.5 || 0 )` (#12497) + temp = remaining / animation.duration || 0, + percent = 1 - temp, + index = 0, + length = animation.tweens.length; + + for ( ; index < length; index++ ) { + animation.tweens[ index ].run( percent ); + } + + deferred.notifyWith( elem, [ animation, percent, remaining ] ); + + // If there's more to do, yield + if ( percent < 1 && length ) { + return remaining; + } + + // If this was an empty animation, synthesize a final progress notification + if ( !length ) { + deferred.notifyWith( elem, [ animation, 1, 0 ] ); + } + + // Resolve the animation and report its conclusion + deferred.resolveWith( elem, [ animation ] ); + return false; + }, + animation = deferred.promise( { + elem: elem, + props: jQuery.extend( {}, properties ), + opts: jQuery.extend( true, { + specialEasing: {}, + easing: jQuery.easing._default + }, options ), + originalProperties: properties, + originalOptions: options, + startTime: fxNow || createFxNow(), + duration: options.duration, + tweens: [], + createTween: function( prop, end ) { + var tween = jQuery.Tween( elem, animation.opts, prop, end, + animation.opts.specialEasing[ prop ] || animation.opts.easing ); + animation.tweens.push( tween ); + return tween; + }, + stop: function( gotoEnd ) { + var index = 0, + + // If we are going to the end, we want to run all the tweens + // otherwise we skip this part + length = gotoEnd ? animation.tweens.length : 0; + if ( stopped ) { + return this; + } + stopped = true; + for ( ; index < length; index++ ) { + animation.tweens[ index ].run( 1 ); + } + + // Resolve when we played the last frame; otherwise, reject + if ( gotoEnd ) { + deferred.notifyWith( elem, [ animation, 1, 0 ] ); + deferred.resolveWith( elem, [ animation, gotoEnd ] ); + } else { + deferred.rejectWith( elem, [ animation, gotoEnd ] ); + } + return this; + } + } ), + props = animation.props; + + propFilter( props, animation.opts.specialEasing ); + + for ( ; index < length; index++ ) { + result = Animation.prefilters[ index ].call( animation, elem, props, animation.opts ); + if ( result ) { + if ( isFunction( result.stop ) ) { + jQuery._queueHooks( animation.elem, animation.opts.queue ).stop = + result.stop.bind( result ); + } + return result; + } + } + + jQuery.map( props, createTween, animation ); + + if ( isFunction( animation.opts.start ) ) { + animation.opts.start.call( elem, animation ); + } + + // Attach callbacks from options + animation + .progress( animation.opts.progress ) + .done( animation.opts.done, animation.opts.complete ) + .fail( animation.opts.fail ) + .always( animation.opts.always ); + + jQuery.fx.timer( + jQuery.extend( tick, { + elem: elem, + anim: animation, + queue: animation.opts.queue + } ) + ); + + return animation; +} + +jQuery.Animation = jQuery.extend( Animation, { + + tweeners: { + "*": [ function( prop, value ) { + var tween = this.createTween( prop, value ); + adjustCSS( tween.elem, prop, rcssNum.exec( value ), tween ); + return tween; + } ] + }, + + tweener: function( props, callback ) { + if ( isFunction( props ) ) { + callback = props; + props = [ "*" ]; + } else { + props = props.match( rnothtmlwhite ); + } + + var prop, + index = 0, + length = props.length; + + for ( ; index < length; index++ ) { + prop = props[ index ]; + Animation.tweeners[ prop ] = Animation.tweeners[ prop ] || []; + Animation.tweeners[ prop ].unshift( callback ); + } + }, + + prefilters: [ defaultPrefilter ], + + prefilter: function( callback, prepend ) { + if ( prepend ) { + Animation.prefilters.unshift( callback ); + } else { + Animation.prefilters.push( callback ); + } + } +} ); + +jQuery.speed = function( speed, easing, fn ) { + var opt = speed && typeof speed === "object" ? jQuery.extend( {}, speed ) : { + complete: fn || !fn && easing || + isFunction( speed ) && speed, + duration: speed, + easing: fn && easing || easing && !isFunction( easing ) && easing + }; + + // Go to the end state if fx are off + if ( jQuery.fx.off ) { + opt.duration = 0; + + } else { + if ( typeof opt.duration !== "number" ) { + if ( opt.duration in jQuery.fx.speeds ) { + opt.duration = jQuery.fx.speeds[ opt.duration ]; + + } else { + opt.duration = jQuery.fx.speeds._default; + } + } + } + + // Normalize opt.queue - true/undefined/null -> "fx" + if ( opt.queue == null || opt.queue === true ) { + opt.queue = "fx"; + } + + // Queueing + opt.old = opt.complete; + + opt.complete = function() { + if ( isFunction( opt.old ) ) { + opt.old.call( this ); + } + + if ( opt.queue ) { + jQuery.dequeue( this, opt.queue ); + } + }; + + return opt; +}; + +jQuery.fn.extend( { + fadeTo: function( speed, to, easing, callback ) { + + // Show any hidden elements after setting opacity to 0 + return this.filter( isHiddenWithinTree ).css( "opacity", 0 ).show() + + // Animate to the value specified + .end().animate( { opacity: to }, speed, easing, callback ); + }, + animate: function( prop, speed, easing, callback ) { + var empty = jQuery.isEmptyObject( prop ), + optall = jQuery.speed( speed, easing, callback ), + doAnimation = function() { + + // Operate on a copy of prop so per-property easing won't be lost + var anim = Animation( this, jQuery.extend( {}, prop ), optall ); + + // Empty animations, or finishing resolves immediately + if ( empty || dataPriv.get( this, "finish" ) ) { + anim.stop( true ); + } + }; + doAnimation.finish = doAnimation; + + return empty || optall.queue === false ? + this.each( doAnimation ) : + this.queue( optall.queue, doAnimation ); + }, + stop: function( type, clearQueue, gotoEnd ) { + var stopQueue = function( hooks ) { + var stop = hooks.stop; + delete hooks.stop; + stop( gotoEnd ); + }; + + if ( typeof type !== "string" ) { + gotoEnd = clearQueue; + clearQueue = type; + type = undefined; + } + if ( clearQueue ) { + this.queue( type || "fx", [] ); + } + + return this.each( function() { + var dequeue = true, + index = type != null && type + "queueHooks", + timers = jQuery.timers, + data = dataPriv.get( this ); + + if ( index ) { + if ( data[ index ] && data[ index ].stop ) { + stopQueue( data[ index ] ); + } + } else { + for ( index in data ) { + if ( data[ index ] && data[ index ].stop && rrun.test( index ) ) { + stopQueue( data[ index ] ); + } + } + } + + for ( index = timers.length; index--; ) { + if ( timers[ index ].elem === this && + ( type == null || timers[ index ].queue === type ) ) { + + timers[ index ].anim.stop( gotoEnd ); + dequeue = false; + timers.splice( index, 1 ); + } + } + + // Start the next in the queue if the last step wasn't forced. + // Timers currently will call their complete callbacks, which + // will dequeue but only if they were gotoEnd. + if ( dequeue || !gotoEnd ) { + jQuery.dequeue( this, type ); + } + } ); + }, + finish: function( type ) { + if ( type !== false ) { + type = type || "fx"; + } + return this.each( function() { + var index, + data = dataPriv.get( this ), + queue = data[ type + "queue" ], + hooks = data[ type + "queueHooks" ], + timers = jQuery.timers, + length = queue ? queue.length : 0; + + // Enable finishing flag on private data + data.finish = true; + + // Empty the queue first + jQuery.queue( this, type, [] ); + + if ( hooks && hooks.stop ) { + hooks.stop.call( this, true ); + } + + // Look for any active animations, and finish them + for ( index = timers.length; index--; ) { + if ( timers[ index ].elem === this && timers[ index ].queue === type ) { + timers[ index ].anim.stop( true ); + timers.splice( index, 1 ); + } + } + + // Look for any animations in the old queue and finish them + for ( index = 0; index < length; index++ ) { + if ( queue[ index ] && queue[ index ].finish ) { + queue[ index ].finish.call( this ); + } + } + + // Turn off finishing flag + delete data.finish; + } ); + } +} ); + +jQuery.each( [ "toggle", "show", "hide" ], function( _i, name ) { + var cssFn = jQuery.fn[ name ]; + jQuery.fn[ name ] = function( speed, easing, callback ) { + return speed == null || typeof speed === "boolean" ? + cssFn.apply( this, arguments ) : + this.animate( genFx( name, true ), speed, easing, callback ); + }; +} ); + +// Generate shortcuts for custom animations +jQuery.each( { + slideDown: genFx( "show" ), + slideUp: genFx( "hide" ), + slideToggle: genFx( "toggle" ), + fadeIn: { opacity: "show" }, + fadeOut: { opacity: "hide" }, + fadeToggle: { opacity: "toggle" } +}, function( name, props ) { + jQuery.fn[ name ] = function( speed, easing, callback ) { + return this.animate( props, speed, easing, callback ); + }; +} ); + +jQuery.timers = []; +jQuery.fx.tick = function() { + var timer, + i = 0, + timers = jQuery.timers; + + fxNow = Date.now(); + + for ( ; i < timers.length; i++ ) { + timer = timers[ i ]; + + // Run the timer and safely remove it when done (allowing for external removal) + if ( !timer() && timers[ i ] === timer ) { + timers.splice( i--, 1 ); + } + } + + if ( !timers.length ) { + jQuery.fx.stop(); + } + fxNow = undefined; +}; + +jQuery.fx.timer = function( timer ) { + jQuery.timers.push( timer ); + jQuery.fx.start(); +}; + +jQuery.fx.interval = 13; +jQuery.fx.start = function() { + if ( inProgress ) { + return; + } + + inProgress = true; + schedule(); +}; + +jQuery.fx.stop = function() { + inProgress = null; +}; + +jQuery.fx.speeds = { + slow: 600, + fast: 200, + + // Default speed + _default: 400 +}; + + +// Based off of the plugin by Clint Helfers, with permission. +// https://web.archive.org/web/20100324014747/http://blindsignals.com/index.php/2009/07/jquery-delay/ +jQuery.fn.delay = function( time, type ) { + time = jQuery.fx ? jQuery.fx.speeds[ time ] || time : time; + type = type || "fx"; + + return this.queue( type, function( next, hooks ) { + var timeout = window.setTimeout( next, time ); + hooks.stop = function() { + window.clearTimeout( timeout ); + }; + } ); +}; + + +( function() { + var input = document.createElement( "input" ), + select = document.createElement( "select" ), + opt = select.appendChild( document.createElement( "option" ) ); + + input.type = "checkbox"; + + // Support: Android <=4.3 only + // Default value for a checkbox should be "on" + support.checkOn = input.value !== ""; + + // Support: IE <=11 only + // Must access selectedIndex to make default options select + support.optSelected = opt.selected; + + // Support: IE <=11 only + // An input loses its value after becoming a radio + input = document.createElement( "input" ); + input.value = "t"; + input.type = "radio"; + support.radioValue = input.value === "t"; +} )(); + + +var boolHook, + attrHandle = jQuery.expr.attrHandle; + +jQuery.fn.extend( { + attr: function( name, value ) { + return access( this, jQuery.attr, name, value, arguments.length > 1 ); + }, + + removeAttr: function( name ) { + return this.each( function() { + jQuery.removeAttr( this, name ); + } ); + } +} ); + +jQuery.extend( { + attr: function( elem, name, value ) { + var ret, hooks, + nType = elem.nodeType; + + // Don't get/set attributes on text, comment and attribute nodes + if ( nType === 3 || nType === 8 || nType === 2 ) { + return; + } + + // Fallback to prop when attributes are not supported + if ( typeof elem.getAttribute === "undefined" ) { + return jQuery.prop( elem, name, value ); + } + + // Attribute hooks are determined by the lowercase version + // Grab necessary hook if one is defined + if ( nType !== 1 || !jQuery.isXMLDoc( elem ) ) { + hooks = jQuery.attrHooks[ name.toLowerCase() ] || + ( jQuery.expr.match.bool.test( name ) ? boolHook : undefined ); + } + + if ( value !== undefined ) { + if ( value === null ) { + jQuery.removeAttr( elem, name ); + return; + } + + if ( hooks && "set" in hooks && + ( ret = hooks.set( elem, value, name ) ) !== undefined ) { + return ret; + } + + elem.setAttribute( name, value + "" ); + return value; + } + + if ( hooks && "get" in hooks && ( ret = hooks.get( elem, name ) ) !== null ) { + return ret; + } + + ret = jQuery.find.attr( elem, name ); + + // Non-existent attributes return null, we normalize to undefined + return ret == null ? undefined : ret; + }, + + attrHooks: { + type: { + set: function( elem, value ) { + if ( !support.radioValue && value === "radio" && + nodeName( elem, "input" ) ) { + var val = elem.value; + elem.setAttribute( "type", value ); + if ( val ) { + elem.value = val; + } + return value; + } + } + } + }, + + removeAttr: function( elem, value ) { + var name, + i = 0, + + // Attribute names can contain non-HTML whitespace characters + // https://html.spec.whatwg.org/multipage/syntax.html#attributes-2 + attrNames = value && value.match( rnothtmlwhite ); + + if ( attrNames && elem.nodeType === 1 ) { + while ( ( name = attrNames[ i++ ] ) ) { + elem.removeAttribute( name ); + } + } + } +} ); + +// Hooks for boolean attributes +boolHook = { + set: function( elem, value, name ) { + if ( value === false ) { + + // Remove boolean attributes when set to false + jQuery.removeAttr( elem, name ); + } else { + elem.setAttribute( name, name ); + } + return name; + } +}; + +jQuery.each( jQuery.expr.match.bool.source.match( /\w+/g ), function( _i, name ) { + var getter = attrHandle[ name ] || jQuery.find.attr; + + attrHandle[ name ] = function( elem, name, isXML ) { + var ret, handle, + lowercaseName = name.toLowerCase(); + + if ( !isXML ) { + + // Avoid an infinite loop by temporarily removing this function from the getter + handle = attrHandle[ lowercaseName ]; + attrHandle[ lowercaseName ] = ret; + ret = getter( elem, name, isXML ) != null ? + lowercaseName : + null; + attrHandle[ lowercaseName ] = handle; + } + return ret; + }; +} ); + + + + +var rfocusable = /^(?:input|select|textarea|button)$/i, + rclickable = /^(?:a|area)$/i; + +jQuery.fn.extend( { + prop: function( name, value ) { + return access( this, jQuery.prop, name, value, arguments.length > 1 ); + }, + + removeProp: function( name ) { + return this.each( function() { + delete this[ jQuery.propFix[ name ] || name ]; + } ); + } +} ); + +jQuery.extend( { + prop: function( elem, name, value ) { + var ret, hooks, + nType = elem.nodeType; + + // Don't get/set properties on text, comment and attribute nodes + if ( nType === 3 || nType === 8 || nType === 2 ) { + return; + } + + if ( nType !== 1 || !jQuery.isXMLDoc( elem ) ) { + + // Fix name and attach hooks + name = jQuery.propFix[ name ] || name; + hooks = jQuery.propHooks[ name ]; + } + + if ( value !== undefined ) { + if ( hooks && "set" in hooks && + ( ret = hooks.set( elem, value, name ) ) !== undefined ) { + return ret; + } + + return ( elem[ name ] = value ); + } + + if ( hooks && "get" in hooks && ( ret = hooks.get( elem, name ) ) !== null ) { + return ret; + } + + return elem[ name ]; + }, + + propHooks: { + tabIndex: { + get: function( elem ) { + + // Support: IE <=9 - 11 only + // elem.tabIndex doesn't always return the + // correct value when it hasn't been explicitly set + // https://web.archive.org/web/20141116233347/http://fluidproject.org/blog/2008/01/09/getting-setting-and-removing-tabindex-values-with-javascript/ + // Use proper attribute retrieval(#12072) + var tabindex = jQuery.find.attr( elem, "tabindex" ); + + if ( tabindex ) { + return parseInt( tabindex, 10 ); + } + + if ( + rfocusable.test( elem.nodeName ) || + rclickable.test( elem.nodeName ) && + elem.href + ) { + return 0; + } + + return -1; + } + } + }, + + propFix: { + "for": "htmlFor", + "class": "className" + } +} ); + +// Support: IE <=11 only +// Accessing the selectedIndex property +// forces the browser to respect setting selected +// on the option +// The getter ensures a default option is selected +// when in an optgroup +// eslint rule "no-unused-expressions" is disabled for this code +// since it considers such accessions noop +if ( !support.optSelected ) { + jQuery.propHooks.selected = { + get: function( elem ) { + + /* eslint no-unused-expressions: "off" */ + + var parent = elem.parentNode; + if ( parent && parent.parentNode ) { + parent.parentNode.selectedIndex; + } + return null; + }, + set: function( elem ) { + + /* eslint no-unused-expressions: "off" */ + + var parent = elem.parentNode; + if ( parent ) { + parent.selectedIndex; + + if ( parent.parentNode ) { + parent.parentNode.selectedIndex; + } + } + } + }; +} + +jQuery.each( [ + "tabIndex", + "readOnly", + "maxLength", + "cellSpacing", + "cellPadding", + "rowSpan", + "colSpan", + "useMap", + "frameBorder", + "contentEditable" +], function() { + jQuery.propFix[ this.toLowerCase() ] = this; +} ); + + + + + // Strip and collapse whitespace according to HTML spec + // https://infra.spec.whatwg.org/#strip-and-collapse-ascii-whitespace + function stripAndCollapse( value ) { + var tokens = value.match( rnothtmlwhite ) || []; + return tokens.join( " " ); + } + + +function getClass( elem ) { + return elem.getAttribute && elem.getAttribute( "class" ) || ""; +} + +function classesToArray( value ) { + if ( Array.isArray( value ) ) { + return value; + } + if ( typeof value === "string" ) { + return value.match( rnothtmlwhite ) || []; + } + return []; +} + +jQuery.fn.extend( { + addClass: function( value ) { + var classes, elem, cur, curValue, clazz, j, finalValue, + i = 0; + + if ( isFunction( value ) ) { + return this.each( function( j ) { + jQuery( this ).addClass( value.call( this, j, getClass( this ) ) ); + } ); + } + + classes = classesToArray( value ); + + if ( classes.length ) { + while ( ( elem = this[ i++ ] ) ) { + curValue = getClass( elem ); + cur = elem.nodeType === 1 && ( " " + stripAndCollapse( curValue ) + " " ); + + if ( cur ) { + j = 0; + while ( ( clazz = classes[ j++ ] ) ) { + if ( cur.indexOf( " " + clazz + " " ) < 0 ) { + cur += clazz + " "; + } + } + + // Only assign if different to avoid unneeded rendering. + finalValue = stripAndCollapse( cur ); + if ( curValue !== finalValue ) { + elem.setAttribute( "class", finalValue ); + } + } + } + } + + return this; + }, + + removeClass: function( value ) { + var classes, elem, cur, curValue, clazz, j, finalValue, + i = 0; + + if ( isFunction( value ) ) { + return this.each( function( j ) { + jQuery( this ).removeClass( value.call( this, j, getClass( this ) ) ); + } ); + } + + if ( !arguments.length ) { + return this.attr( "class", "" ); + } + + classes = classesToArray( value ); + + if ( classes.length ) { + while ( ( elem = this[ i++ ] ) ) { + curValue = getClass( elem ); + + // This expression is here for better compressibility (see addClass) + cur = elem.nodeType === 1 && ( " " + stripAndCollapse( curValue ) + " " ); + + if ( cur ) { + j = 0; + while ( ( clazz = classes[ j++ ] ) ) { + + // Remove *all* instances + while ( cur.indexOf( " " + clazz + " " ) > -1 ) { + cur = cur.replace( " " + clazz + " ", " " ); + } + } + + // Only assign if different to avoid unneeded rendering. + finalValue = stripAndCollapse( cur ); + if ( curValue !== finalValue ) { + elem.setAttribute( "class", finalValue ); + } + } + } + } + + return this; + }, + + toggleClass: function( value, stateVal ) { + var type = typeof value, + isValidValue = type === "string" || Array.isArray( value ); + + if ( typeof stateVal === "boolean" && isValidValue ) { + return stateVal ? this.addClass( value ) : this.removeClass( value ); + } + + if ( isFunction( value ) ) { + return this.each( function( i ) { + jQuery( this ).toggleClass( + value.call( this, i, getClass( this ), stateVal ), + stateVal + ); + } ); + } + + return this.each( function() { + var className, i, self, classNames; + + if ( isValidValue ) { + + // Toggle individual class names + i = 0; + self = jQuery( this ); + classNames = classesToArray( value ); + + while ( ( className = classNames[ i++ ] ) ) { + + // Check each className given, space separated list + if ( self.hasClass( className ) ) { + self.removeClass( className ); + } else { + self.addClass( className ); + } + } + + // Toggle whole class name + } else if ( value === undefined || type === "boolean" ) { + className = getClass( this ); + if ( className ) { + + // Store className if set + dataPriv.set( this, "__className__", className ); + } + + // If the element has a class name or if we're passed `false`, + // then remove the whole classname (if there was one, the above saved it). + // Otherwise bring back whatever was previously saved (if anything), + // falling back to the empty string if nothing was stored. + if ( this.setAttribute ) { + this.setAttribute( "class", + className || value === false ? + "" : + dataPriv.get( this, "__className__" ) || "" + ); + } + } + } ); + }, + + hasClass: function( selector ) { + var className, elem, + i = 0; + + className = " " + selector + " "; + while ( ( elem = this[ i++ ] ) ) { + if ( elem.nodeType === 1 && + ( " " + stripAndCollapse( getClass( elem ) ) + " " ).indexOf( className ) > -1 ) { + return true; + } + } + + return false; + } +} ); + + + + +var rreturn = /\r/g; + +jQuery.fn.extend( { + val: function( value ) { + var hooks, ret, valueIsFunction, + elem = this[ 0 ]; + + if ( !arguments.length ) { + if ( elem ) { + hooks = jQuery.valHooks[ elem.type ] || + jQuery.valHooks[ elem.nodeName.toLowerCase() ]; + + if ( hooks && + "get" in hooks && + ( ret = hooks.get( elem, "value" ) ) !== undefined + ) { + return ret; + } + + ret = elem.value; + + // Handle most common string cases + if ( typeof ret === "string" ) { + return ret.replace( rreturn, "" ); + } + + // Handle cases where value is null/undef or number + return ret == null ? "" : ret; + } + + return; + } + + valueIsFunction = isFunction( value ); + + return this.each( function( i ) { + var val; + + if ( this.nodeType !== 1 ) { + return; + } + + if ( valueIsFunction ) { + val = value.call( this, i, jQuery( this ).val() ); + } else { + val = value; + } + + // Treat null/undefined as ""; convert numbers to string + if ( val == null ) { + val = ""; + + } else if ( typeof val === "number" ) { + val += ""; + + } else if ( Array.isArray( val ) ) { + val = jQuery.map( val, function( value ) { + return value == null ? "" : value + ""; + } ); + } + + hooks = jQuery.valHooks[ this.type ] || jQuery.valHooks[ this.nodeName.toLowerCase() ]; + + // If set returns undefined, fall back to normal setting + if ( !hooks || !( "set" in hooks ) || hooks.set( this, val, "value" ) === undefined ) { + this.value = val; + } + } ); + } +} ); + +jQuery.extend( { + valHooks: { + option: { + get: function( elem ) { + + var val = jQuery.find.attr( elem, "value" ); + return val != null ? + val : + + // Support: IE <=10 - 11 only + // option.text throws exceptions (#14686, #14858) + // Strip and collapse whitespace + // https://html.spec.whatwg.org/#strip-and-collapse-whitespace + stripAndCollapse( jQuery.text( elem ) ); + } + }, + select: { + get: function( elem ) { + var value, option, i, + options = elem.options, + index = elem.selectedIndex, + one = elem.type === "select-one", + values = one ? null : [], + max = one ? index + 1 : options.length; + + if ( index < 0 ) { + i = max; + + } else { + i = one ? index : 0; + } + + // Loop through all the selected options + for ( ; i < max; i++ ) { + option = options[ i ]; + + // Support: IE <=9 only + // IE8-9 doesn't update selected after form reset (#2551) + if ( ( option.selected || i === index ) && + + // Don't return options that are disabled or in a disabled optgroup + !option.disabled && + ( !option.parentNode.disabled || + !nodeName( option.parentNode, "optgroup" ) ) ) { + + // Get the specific value for the option + value = jQuery( option ).val(); + + // We don't need an array for one selects + if ( one ) { + return value; + } + + // Multi-Selects return an array + values.push( value ); + } + } + + return values; + }, + + set: function( elem, value ) { + var optionSet, option, + options = elem.options, + values = jQuery.makeArray( value ), + i = options.length; + + while ( i-- ) { + option = options[ i ]; + + /* eslint-disable no-cond-assign */ + + if ( option.selected = + jQuery.inArray( jQuery.valHooks.option.get( option ), values ) > -1 + ) { + optionSet = true; + } + + /* eslint-enable no-cond-assign */ + } + + // Force browsers to behave consistently when non-matching value is set + if ( !optionSet ) { + elem.selectedIndex = -1; + } + return values; + } + } + } +} ); + +// Radios and checkboxes getter/setter +jQuery.each( [ "radio", "checkbox" ], function() { + jQuery.valHooks[ this ] = { + set: function( elem, value ) { + if ( Array.isArray( value ) ) { + return ( elem.checked = jQuery.inArray( jQuery( elem ).val(), value ) > -1 ); + } + } + }; + if ( !support.checkOn ) { + jQuery.valHooks[ this ].get = function( elem ) { + return elem.getAttribute( "value" ) === null ? "on" : elem.value; + }; + } +} ); + + + + +// Return jQuery for attributes-only inclusion + + +support.focusin = "onfocusin" in window; + + +var rfocusMorph = /^(?:focusinfocus|focusoutblur)$/, + stopPropagationCallback = function( e ) { + e.stopPropagation(); + }; + +jQuery.extend( jQuery.event, { + + trigger: function( event, data, elem, onlyHandlers ) { + + var i, cur, tmp, bubbleType, ontype, handle, special, lastElement, + eventPath = [ elem || document ], + type = hasOwn.call( event, "type" ) ? event.type : event, + namespaces = hasOwn.call( event, "namespace" ) ? event.namespace.split( "." ) : []; + + cur = lastElement = tmp = elem = elem || document; + + // Don't do events on text and comment nodes + if ( elem.nodeType === 3 || elem.nodeType === 8 ) { + return; + } + + // focus/blur morphs to focusin/out; ensure we're not firing them right now + if ( rfocusMorph.test( type + jQuery.event.triggered ) ) { + return; + } + + if ( type.indexOf( "." ) > -1 ) { + + // Namespaced trigger; create a regexp to match event type in handle() + namespaces = type.split( "." ); + type = namespaces.shift(); + namespaces.sort(); + } + ontype = type.indexOf( ":" ) < 0 && "on" + type; + + // Caller can pass in a jQuery.Event object, Object, or just an event type string + event = event[ jQuery.expando ] ? + event : + new jQuery.Event( type, typeof event === "object" && event ); + + // Trigger bitmask: & 1 for native handlers; & 2 for jQuery (always true) + event.isTrigger = onlyHandlers ? 2 : 3; + event.namespace = namespaces.join( "." ); + event.rnamespace = event.namespace ? + new RegExp( "(^|\\.)" + namespaces.join( "\\.(?:.*\\.|)" ) + "(\\.|$)" ) : + null; + + // Clean up the event in case it is being reused + event.result = undefined; + if ( !event.target ) { + event.target = elem; + } + + // Clone any incoming data and prepend the event, creating the handler arg list + data = data == null ? + [ event ] : + jQuery.makeArray( data, [ event ] ); + + // Allow special events to draw outside the lines + special = jQuery.event.special[ type ] || {}; + if ( !onlyHandlers && special.trigger && special.trigger.apply( elem, data ) === false ) { + return; + } + + // Determine event propagation path in advance, per W3C events spec (#9951) + // Bubble up to document, then to window; watch for a global ownerDocument var (#9724) + if ( !onlyHandlers && !special.noBubble && !isWindow( elem ) ) { + + bubbleType = special.delegateType || type; + if ( !rfocusMorph.test( bubbleType + type ) ) { + cur = cur.parentNode; + } + for ( ; cur; cur = cur.parentNode ) { + eventPath.push( cur ); + tmp = cur; + } + + // Only add window if we got to document (e.g., not plain obj or detached DOM) + if ( tmp === ( elem.ownerDocument || document ) ) { + eventPath.push( tmp.defaultView || tmp.parentWindow || window ); + } + } + + // Fire handlers on the event path + i = 0; + while ( ( cur = eventPath[ i++ ] ) && !event.isPropagationStopped() ) { + lastElement = cur; + event.type = i > 1 ? + bubbleType : + special.bindType || type; + + // jQuery handler + handle = ( + dataPriv.get( cur, "events" ) || Object.create( null ) + )[ event.type ] && + dataPriv.get( cur, "handle" ); + if ( handle ) { + handle.apply( cur, data ); + } + + // Native handler + handle = ontype && cur[ ontype ]; + if ( handle && handle.apply && acceptData( cur ) ) { + event.result = handle.apply( cur, data ); + if ( event.result === false ) { + event.preventDefault(); + } + } + } + event.type = type; + + // If nobody prevented the default action, do it now + if ( !onlyHandlers && !event.isDefaultPrevented() ) { + + if ( ( !special._default || + special._default.apply( eventPath.pop(), data ) === false ) && + acceptData( elem ) ) { + + // Call a native DOM method on the target with the same name as the event. + // Don't do default actions on window, that's where global variables be (#6170) + if ( ontype && isFunction( elem[ type ] ) && !isWindow( elem ) ) { + + // Don't re-trigger an onFOO event when we call its FOO() method + tmp = elem[ ontype ]; + + if ( tmp ) { + elem[ ontype ] = null; + } + + // Prevent re-triggering of the same event, since we already bubbled it above + jQuery.event.triggered = type; + + if ( event.isPropagationStopped() ) { + lastElement.addEventListener( type, stopPropagationCallback ); + } + + elem[ type ](); + + if ( event.isPropagationStopped() ) { + lastElement.removeEventListener( type, stopPropagationCallback ); + } + + jQuery.event.triggered = undefined; + + if ( tmp ) { + elem[ ontype ] = tmp; + } + } + } + } + + return event.result; + }, + + // Piggyback on a donor event to simulate a different one + // Used only for `focus(in | out)` events + simulate: function( type, elem, event ) { + var e = jQuery.extend( + new jQuery.Event(), + event, + { + type: type, + isSimulated: true + } + ); + + jQuery.event.trigger( e, null, elem ); + } + +} ); + +jQuery.fn.extend( { + + trigger: function( type, data ) { + return this.each( function() { + jQuery.event.trigger( type, data, this ); + } ); + }, + triggerHandler: function( type, data ) { + var elem = this[ 0 ]; + if ( elem ) { + return jQuery.event.trigger( type, data, elem, true ); + } + } +} ); + + +// Support: Firefox <=44 +// Firefox doesn't have focus(in | out) events +// Related ticket - https://bugzilla.mozilla.org/show_bug.cgi?id=687787 +// +// Support: Chrome <=48 - 49, Safari <=9.0 - 9.1 +// focus(in | out) events fire after focus & blur events, +// which is spec violation - http://www.w3.org/TR/DOM-Level-3-Events/#events-focusevent-event-order +// Related ticket - https://bugs.chromium.org/p/chromium/issues/detail?id=449857 +if ( !support.focusin ) { + jQuery.each( { focus: "focusin", blur: "focusout" }, function( orig, fix ) { + + // Attach a single capturing handler on the document while someone wants focusin/focusout + var handler = function( event ) { + jQuery.event.simulate( fix, event.target, jQuery.event.fix( event ) ); + }; + + jQuery.event.special[ fix ] = { + setup: function() { + + // Handle: regular nodes (via `this.ownerDocument`), window + // (via `this.document`) & document (via `this`). + var doc = this.ownerDocument || this.document || this, + attaches = dataPriv.access( doc, fix ); + + if ( !attaches ) { + doc.addEventListener( orig, handler, true ); + } + dataPriv.access( doc, fix, ( attaches || 0 ) + 1 ); + }, + teardown: function() { + var doc = this.ownerDocument || this.document || this, + attaches = dataPriv.access( doc, fix ) - 1; + + if ( !attaches ) { + doc.removeEventListener( orig, handler, true ); + dataPriv.remove( doc, fix ); + + } else { + dataPriv.access( doc, fix, attaches ); + } + } + }; + } ); +} +var location = window.location; + +var nonce = { guid: Date.now() }; + +var rquery = ( /\?/ ); + + + +// Cross-browser xml parsing +jQuery.parseXML = function( data ) { + var xml; + if ( !data || typeof data !== "string" ) { + return null; + } + + // Support: IE 9 - 11 only + // IE throws on parseFromString with invalid input. + try { + xml = ( new window.DOMParser() ).parseFromString( data, "text/xml" ); + } catch ( e ) { + xml = undefined; + } + + if ( !xml || xml.getElementsByTagName( "parsererror" ).length ) { + jQuery.error( "Invalid XML: " + data ); + } + return xml; +}; + + +var + rbracket = /\[\]$/, + rCRLF = /\r?\n/g, + rsubmitterTypes = /^(?:submit|button|image|reset|file)$/i, + rsubmittable = /^(?:input|select|textarea|keygen)/i; + +function buildParams( prefix, obj, traditional, add ) { + var name; + + if ( Array.isArray( obj ) ) { + + // Serialize array item. + jQuery.each( obj, function( i, v ) { + if ( traditional || rbracket.test( prefix ) ) { + + // Treat each array item as a scalar. + add( prefix, v ); + + } else { + + // Item is non-scalar (array or object), encode its numeric index. + buildParams( + prefix + "[" + ( typeof v === "object" && v != null ? i : "" ) + "]", + v, + traditional, + add + ); + } + } ); + + } else if ( !traditional && toType( obj ) === "object" ) { + + // Serialize object item. + for ( name in obj ) { + buildParams( prefix + "[" + name + "]", obj[ name ], traditional, add ); + } + + } else { + + // Serialize scalar item. + add( prefix, obj ); + } +} + +// Serialize an array of form elements or a set of +// key/values into a query string +jQuery.param = function( a, traditional ) { + var prefix, + s = [], + add = function( key, valueOrFunction ) { + + // If value is a function, invoke it and use its return value + var value = isFunction( valueOrFunction ) ? + valueOrFunction() : + valueOrFunction; + + s[ s.length ] = encodeURIComponent( key ) + "=" + + encodeURIComponent( value == null ? "" : value ); + }; + + if ( a == null ) { + return ""; + } + + // If an array was passed in, assume that it is an array of form elements. + if ( Array.isArray( a ) || ( a.jquery && !jQuery.isPlainObject( a ) ) ) { + + // Serialize the form elements + jQuery.each( a, function() { + add( this.name, this.value ); + } ); + + } else { + + // If traditional, encode the "old" way (the way 1.3.2 or older + // did it), otherwise encode params recursively. + for ( prefix in a ) { + buildParams( prefix, a[ prefix ], traditional, add ); + } + } + + // Return the resulting serialization + return s.join( "&" ); +}; + +jQuery.fn.extend( { + serialize: function() { + return jQuery.param( this.serializeArray() ); + }, + serializeArray: function() { + return this.map( function() { + + // Can add propHook for "elements" to filter or add form elements + var elements = jQuery.prop( this, "elements" ); + return elements ? jQuery.makeArray( elements ) : this; + } ) + .filter( function() { + var type = this.type; + + // Use .is( ":disabled" ) so that fieldset[disabled] works + return this.name && !jQuery( this ).is( ":disabled" ) && + rsubmittable.test( this.nodeName ) && !rsubmitterTypes.test( type ) && + ( this.checked || !rcheckableType.test( type ) ); + } ) + .map( function( _i, elem ) { + var val = jQuery( this ).val(); + + if ( val == null ) { + return null; + } + + if ( Array.isArray( val ) ) { + return jQuery.map( val, function( val ) { + return { name: elem.name, value: val.replace( rCRLF, "\r\n" ) }; + } ); + } + + return { name: elem.name, value: val.replace( rCRLF, "\r\n" ) }; + } ).get(); + } +} ); + + +var + r20 = /%20/g, + rhash = /#.*$/, + rantiCache = /([?&])_=[^&]*/, + rheaders = /^(.*?):[ \t]*([^\r\n]*)$/mg, + + // #7653, #8125, #8152: local protocol detection + rlocalProtocol = /^(?:about|app|app-storage|.+-extension|file|res|widget):$/, + rnoContent = /^(?:GET|HEAD)$/, + rprotocol = /^\/\//, + + /* Prefilters + * 1) They are useful to introduce custom dataTypes (see ajax/jsonp.js for an example) + * 2) These are called: + * - BEFORE asking for a transport + * - AFTER param serialization (s.data is a string if s.processData is true) + * 3) key is the dataType + * 4) the catchall symbol "*" can be used + * 5) execution will start with transport dataType and THEN continue down to "*" if needed + */ + prefilters = {}, + + /* Transports bindings + * 1) key is the dataType + * 2) the catchall symbol "*" can be used + * 3) selection will start with transport dataType and THEN go to "*" if needed + */ + transports = {}, + + // Avoid comment-prolog char sequence (#10098); must appease lint and evade compression + allTypes = "*/".concat( "*" ), + + // Anchor tag for parsing the document origin + originAnchor = document.createElement( "a" ); + originAnchor.href = location.href; + +// Base "constructor" for jQuery.ajaxPrefilter and jQuery.ajaxTransport +function addToPrefiltersOrTransports( structure ) { + + // dataTypeExpression is optional and defaults to "*" + return function( dataTypeExpression, func ) { + + if ( typeof dataTypeExpression !== "string" ) { + func = dataTypeExpression; + dataTypeExpression = "*"; + } + + var dataType, + i = 0, + dataTypes = dataTypeExpression.toLowerCase().match( rnothtmlwhite ) || []; + + if ( isFunction( func ) ) { + + // For each dataType in the dataTypeExpression + while ( ( dataType = dataTypes[ i++ ] ) ) { + + // Prepend if requested + if ( dataType[ 0 ] === "+" ) { + dataType = dataType.slice( 1 ) || "*"; + ( structure[ dataType ] = structure[ dataType ] || [] ).unshift( func ); + + // Otherwise append + } else { + ( structure[ dataType ] = structure[ dataType ] || [] ).push( func ); + } + } + } + }; +} + +// Base inspection function for prefilters and transports +function inspectPrefiltersOrTransports( structure, options, originalOptions, jqXHR ) { + + var inspected = {}, + seekingTransport = ( structure === transports ); + + function inspect( dataType ) { + var selected; + inspected[ dataType ] = true; + jQuery.each( structure[ dataType ] || [], function( _, prefilterOrFactory ) { + var dataTypeOrTransport = prefilterOrFactory( options, originalOptions, jqXHR ); + if ( typeof dataTypeOrTransport === "string" && + !seekingTransport && !inspected[ dataTypeOrTransport ] ) { + + options.dataTypes.unshift( dataTypeOrTransport ); + inspect( dataTypeOrTransport ); + return false; + } else if ( seekingTransport ) { + return !( selected = dataTypeOrTransport ); + } + } ); + return selected; + } + + return inspect( options.dataTypes[ 0 ] ) || !inspected[ "*" ] && inspect( "*" ); +} + +// A special extend for ajax options +// that takes "flat" options (not to be deep extended) +// Fixes #9887 +function ajaxExtend( target, src ) { + var key, deep, + flatOptions = jQuery.ajaxSettings.flatOptions || {}; + + for ( key in src ) { + if ( src[ key ] !== undefined ) { + ( flatOptions[ key ] ? target : ( deep || ( deep = {} ) ) )[ key ] = src[ key ]; + } + } + if ( deep ) { + jQuery.extend( true, target, deep ); + } + + return target; +} + +/* Handles responses to an ajax request: + * - finds the right dataType (mediates between content-type and expected dataType) + * - returns the corresponding response + */ +function ajaxHandleResponses( s, jqXHR, responses ) { + + var ct, type, finalDataType, firstDataType, + contents = s.contents, + dataTypes = s.dataTypes; + + // Remove auto dataType and get content-type in the process + while ( dataTypes[ 0 ] === "*" ) { + dataTypes.shift(); + if ( ct === undefined ) { + ct = s.mimeType || jqXHR.getResponseHeader( "Content-Type" ); + } + } + + // Check if we're dealing with a known content-type + if ( ct ) { + for ( type in contents ) { + if ( contents[ type ] && contents[ type ].test( ct ) ) { + dataTypes.unshift( type ); + break; + } + } + } + + // Check to see if we have a response for the expected dataType + if ( dataTypes[ 0 ] in responses ) { + finalDataType = dataTypes[ 0 ]; + } else { + + // Try convertible dataTypes + for ( type in responses ) { + if ( !dataTypes[ 0 ] || s.converters[ type + " " + dataTypes[ 0 ] ] ) { + finalDataType = type; + break; + } + if ( !firstDataType ) { + firstDataType = type; + } + } + + // Or just use first one + finalDataType = finalDataType || firstDataType; + } + + // If we found a dataType + // We add the dataType to the list if needed + // and return the corresponding response + if ( finalDataType ) { + if ( finalDataType !== dataTypes[ 0 ] ) { + dataTypes.unshift( finalDataType ); + } + return responses[ finalDataType ]; + } +} + +/* Chain conversions given the request and the original response + * Also sets the responseXXX fields on the jqXHR instance + */ +function ajaxConvert( s, response, jqXHR, isSuccess ) { + var conv2, current, conv, tmp, prev, + converters = {}, + + // Work with a copy of dataTypes in case we need to modify it for conversion + dataTypes = s.dataTypes.slice(); + + // Create converters map with lowercased keys + if ( dataTypes[ 1 ] ) { + for ( conv in s.converters ) { + converters[ conv.toLowerCase() ] = s.converters[ conv ]; + } + } + + current = dataTypes.shift(); + + // Convert to each sequential dataType + while ( current ) { + + if ( s.responseFields[ current ] ) { + jqXHR[ s.responseFields[ current ] ] = response; + } + + // Apply the dataFilter if provided + if ( !prev && isSuccess && s.dataFilter ) { + response = s.dataFilter( response, s.dataType ); + } + + prev = current; + current = dataTypes.shift(); + + if ( current ) { + + // There's only work to do if current dataType is non-auto + if ( current === "*" ) { + + current = prev; + + // Convert response if prev dataType is non-auto and differs from current + } else if ( prev !== "*" && prev !== current ) { + + // Seek a direct converter + conv = converters[ prev + " " + current ] || converters[ "* " + current ]; + + // If none found, seek a pair + if ( !conv ) { + for ( conv2 in converters ) { + + // If conv2 outputs current + tmp = conv2.split( " " ); + if ( tmp[ 1 ] === current ) { + + // If prev can be converted to accepted input + conv = converters[ prev + " " + tmp[ 0 ] ] || + converters[ "* " + tmp[ 0 ] ]; + if ( conv ) { + + // Condense equivalence converters + if ( conv === true ) { + conv = converters[ conv2 ]; + + // Otherwise, insert the intermediate dataType + } else if ( converters[ conv2 ] !== true ) { + current = tmp[ 0 ]; + dataTypes.unshift( tmp[ 1 ] ); + } + break; + } + } + } + } + + // Apply converter (if not an equivalence) + if ( conv !== true ) { + + // Unless errors are allowed to bubble, catch and return them + if ( conv && s.throws ) { + response = conv( response ); + } else { + try { + response = conv( response ); + } catch ( e ) { + return { + state: "parsererror", + error: conv ? e : "No conversion from " + prev + " to " + current + }; + } + } + } + } + } + } + + return { state: "success", data: response }; +} + +jQuery.extend( { + + // Counter for holding the number of active queries + active: 0, + + // Last-Modified header cache for next request + lastModified: {}, + etag: {}, + + ajaxSettings: { + url: location.href, + type: "GET", + isLocal: rlocalProtocol.test( location.protocol ), + global: true, + processData: true, + async: true, + contentType: "application/x-www-form-urlencoded; charset=UTF-8", + + /* + timeout: 0, + data: null, + dataType: null, + username: null, + password: null, + cache: null, + throws: false, + traditional: false, + headers: {}, + */ + + accepts: { + "*": allTypes, + text: "text/plain", + html: "text/html", + xml: "application/xml, text/xml", + json: "application/json, text/javascript" + }, + + contents: { + xml: /\bxml\b/, + html: /\bhtml/, + json: /\bjson\b/ + }, + + responseFields: { + xml: "responseXML", + text: "responseText", + json: "responseJSON" + }, + + // Data converters + // Keys separate source (or catchall "*") and destination types with a single space + converters: { + + // Convert anything to text + "* text": String, + + // Text to html (true = no transformation) + "text html": true, + + // Evaluate text as a json expression + "text json": JSON.parse, + + // Parse text as xml + "text xml": jQuery.parseXML + }, + + // For options that shouldn't be deep extended: + // you can add your own custom options here if + // and when you create one that shouldn't be + // deep extended (see ajaxExtend) + flatOptions: { + url: true, + context: true + } + }, + + // Creates a full fledged settings object into target + // with both ajaxSettings and settings fields. + // If target is omitted, writes into ajaxSettings. + ajaxSetup: function( target, settings ) { + return settings ? + + // Building a settings object + ajaxExtend( ajaxExtend( target, jQuery.ajaxSettings ), settings ) : + + // Extending ajaxSettings + ajaxExtend( jQuery.ajaxSettings, target ); + }, + + ajaxPrefilter: addToPrefiltersOrTransports( prefilters ), + ajaxTransport: addToPrefiltersOrTransports( transports ), + + // Main method + ajax: function( url, options ) { + + // If url is an object, simulate pre-1.5 signature + if ( typeof url === "object" ) { + options = url; + url = undefined; + } + + // Force options to be an object + options = options || {}; + + var transport, + + // URL without anti-cache param + cacheURL, + + // Response headers + responseHeadersString, + responseHeaders, + + // timeout handle + timeoutTimer, + + // Url cleanup var + urlAnchor, + + // Request state (becomes false upon send and true upon completion) + completed, + + // To know if global events are to be dispatched + fireGlobals, + + // Loop variable + i, + + // uncached part of the url + uncached, + + // Create the final options object + s = jQuery.ajaxSetup( {}, options ), + + // Callbacks context + callbackContext = s.context || s, + + // Context for global events is callbackContext if it is a DOM node or jQuery collection + globalEventContext = s.context && + ( callbackContext.nodeType || callbackContext.jquery ) ? + jQuery( callbackContext ) : + jQuery.event, + + // Deferreds + deferred = jQuery.Deferred(), + completeDeferred = jQuery.Callbacks( "once memory" ), + + // Status-dependent callbacks + statusCode = s.statusCode || {}, + + // Headers (they are sent all at once) + requestHeaders = {}, + requestHeadersNames = {}, + + // Default abort message + strAbort = "canceled", + + // Fake xhr + jqXHR = { + readyState: 0, + + // Builds headers hashtable if needed + getResponseHeader: function( key ) { + var match; + if ( completed ) { + if ( !responseHeaders ) { + responseHeaders = {}; + while ( ( match = rheaders.exec( responseHeadersString ) ) ) { + responseHeaders[ match[ 1 ].toLowerCase() + " " ] = + ( responseHeaders[ match[ 1 ].toLowerCase() + " " ] || [] ) + .concat( match[ 2 ] ); + } + } + match = responseHeaders[ key.toLowerCase() + " " ]; + } + return match == null ? null : match.join( ", " ); + }, + + // Raw string + getAllResponseHeaders: function() { + return completed ? responseHeadersString : null; + }, + + // Caches the header + setRequestHeader: function( name, value ) { + if ( completed == null ) { + name = requestHeadersNames[ name.toLowerCase() ] = + requestHeadersNames[ name.toLowerCase() ] || name; + requestHeaders[ name ] = value; + } + return this; + }, + + // Overrides response content-type header + overrideMimeType: function( type ) { + if ( completed == null ) { + s.mimeType = type; + } + return this; + }, + + // Status-dependent callbacks + statusCode: function( map ) { + var code; + if ( map ) { + if ( completed ) { + + // Execute the appropriate callbacks + jqXHR.always( map[ jqXHR.status ] ); + } else { + + // Lazy-add the new callbacks in a way that preserves old ones + for ( code in map ) { + statusCode[ code ] = [ statusCode[ code ], map[ code ] ]; + } + } + } + return this; + }, + + // Cancel the request + abort: function( statusText ) { + var finalText = statusText || strAbort; + if ( transport ) { + transport.abort( finalText ); + } + done( 0, finalText ); + return this; + } + }; + + // Attach deferreds + deferred.promise( jqXHR ); + + // Add protocol if not provided (prefilters might expect it) + // Handle falsy url in the settings object (#10093: consistency with old signature) + // We also use the url parameter if available + s.url = ( ( url || s.url || location.href ) + "" ) + .replace( rprotocol, location.protocol + "//" ); + + // Alias method option to type as per ticket #12004 + s.type = options.method || options.type || s.method || s.type; + + // Extract dataTypes list + s.dataTypes = ( s.dataType || "*" ).toLowerCase().match( rnothtmlwhite ) || [ "" ]; + + // A cross-domain request is in order when the origin doesn't match the current origin. + if ( s.crossDomain == null ) { + urlAnchor = document.createElement( "a" ); + + // Support: IE <=8 - 11, Edge 12 - 15 + // IE throws exception on accessing the href property if url is malformed, + // e.g. http://example.com:80x/ + try { + urlAnchor.href = s.url; + + // Support: IE <=8 - 11 only + // Anchor's host property isn't correctly set when s.url is relative + urlAnchor.href = urlAnchor.href; + s.crossDomain = originAnchor.protocol + "//" + originAnchor.host !== + urlAnchor.protocol + "//" + urlAnchor.host; + } catch ( e ) { + + // If there is an error parsing the URL, assume it is crossDomain, + // it can be rejected by the transport if it is invalid + s.crossDomain = true; + } + } + + // Convert data if not already a string + if ( s.data && s.processData && typeof s.data !== "string" ) { + s.data = jQuery.param( s.data, s.traditional ); + } + + // Apply prefilters + inspectPrefiltersOrTransports( prefilters, s, options, jqXHR ); + + // If request was aborted inside a prefilter, stop there + if ( completed ) { + return jqXHR; + } + + // We can fire global events as of now if asked to + // Don't fire events if jQuery.event is undefined in an AMD-usage scenario (#15118) + fireGlobals = jQuery.event && s.global; + + // Watch for a new set of requests + if ( fireGlobals && jQuery.active++ === 0 ) { + jQuery.event.trigger( "ajaxStart" ); + } + + // Uppercase the type + s.type = s.type.toUpperCase(); + + // Determine if request has content + s.hasContent = !rnoContent.test( s.type ); + + // Save the URL in case we're toying with the If-Modified-Since + // and/or If-None-Match header later on + // Remove hash to simplify url manipulation + cacheURL = s.url.replace( rhash, "" ); + + // More options handling for requests with no content + if ( !s.hasContent ) { + + // Remember the hash so we can put it back + uncached = s.url.slice( cacheURL.length ); + + // If data is available and should be processed, append data to url + if ( s.data && ( s.processData || typeof s.data === "string" ) ) { + cacheURL += ( rquery.test( cacheURL ) ? "&" : "?" ) + s.data; + + // #9682: remove data so that it's not used in an eventual retry + delete s.data; + } + + // Add or update anti-cache param if needed + if ( s.cache === false ) { + cacheURL = cacheURL.replace( rantiCache, "$1" ); + uncached = ( rquery.test( cacheURL ) ? "&" : "?" ) + "_=" + ( nonce.guid++ ) + + uncached; + } + + // Put hash and anti-cache on the URL that will be requested (gh-1732) + s.url = cacheURL + uncached; + + // Change '%20' to '+' if this is encoded form body content (gh-2658) + } else if ( s.data && s.processData && + ( s.contentType || "" ).indexOf( "application/x-www-form-urlencoded" ) === 0 ) { + s.data = s.data.replace( r20, "+" ); + } + + // Set the If-Modified-Since and/or If-None-Match header, if in ifModified mode. + if ( s.ifModified ) { + if ( jQuery.lastModified[ cacheURL ] ) { + jqXHR.setRequestHeader( "If-Modified-Since", jQuery.lastModified[ cacheURL ] ); + } + if ( jQuery.etag[ cacheURL ] ) { + jqXHR.setRequestHeader( "If-None-Match", jQuery.etag[ cacheURL ] ); + } + } + + // Set the correct header, if data is being sent + if ( s.data && s.hasContent && s.contentType !== false || options.contentType ) { + jqXHR.setRequestHeader( "Content-Type", s.contentType ); + } + + // Set the Accepts header for the server, depending on the dataType + jqXHR.setRequestHeader( + "Accept", + s.dataTypes[ 0 ] && s.accepts[ s.dataTypes[ 0 ] ] ? + s.accepts[ s.dataTypes[ 0 ] ] + + ( s.dataTypes[ 0 ] !== "*" ? ", " + allTypes + "; q=0.01" : "" ) : + s.accepts[ "*" ] + ); + + // Check for headers option + for ( i in s.headers ) { + jqXHR.setRequestHeader( i, s.headers[ i ] ); + } + + // Allow custom headers/mimetypes and early abort + if ( s.beforeSend && + ( s.beforeSend.call( callbackContext, jqXHR, s ) === false || completed ) ) { + + // Abort if not done already and return + return jqXHR.abort(); + } + + // Aborting is no longer a cancellation + strAbort = "abort"; + + // Install callbacks on deferreds + completeDeferred.add( s.complete ); + jqXHR.done( s.success ); + jqXHR.fail( s.error ); + + // Get transport + transport = inspectPrefiltersOrTransports( transports, s, options, jqXHR ); + + // If no transport, we auto-abort + if ( !transport ) { + done( -1, "No Transport" ); + } else { + jqXHR.readyState = 1; + + // Send global event + if ( fireGlobals ) { + globalEventContext.trigger( "ajaxSend", [ jqXHR, s ] ); + } + + // If request was aborted inside ajaxSend, stop there + if ( completed ) { + return jqXHR; + } + + // Timeout + if ( s.async && s.timeout > 0 ) { + timeoutTimer = window.setTimeout( function() { + jqXHR.abort( "timeout" ); + }, s.timeout ); + } + + try { + completed = false; + transport.send( requestHeaders, done ); + } catch ( e ) { + + // Rethrow post-completion exceptions + if ( completed ) { + throw e; + } + + // Propagate others as results + done( -1, e ); + } + } + + // Callback for when everything is done + function done( status, nativeStatusText, responses, headers ) { + var isSuccess, success, error, response, modified, + statusText = nativeStatusText; + + // Ignore repeat invocations + if ( completed ) { + return; + } + + completed = true; + + // Clear timeout if it exists + if ( timeoutTimer ) { + window.clearTimeout( timeoutTimer ); + } + + // Dereference transport for early garbage collection + // (no matter how long the jqXHR object will be used) + transport = undefined; + + // Cache response headers + responseHeadersString = headers || ""; + + // Set readyState + jqXHR.readyState = status > 0 ? 4 : 0; + + // Determine if successful + isSuccess = status >= 200 && status < 300 || status === 304; + + // Get response data + if ( responses ) { + response = ajaxHandleResponses( s, jqXHR, responses ); + } + + // Use a noop converter for missing script + if ( !isSuccess && jQuery.inArray( "script", s.dataTypes ) > -1 ) { + s.converters[ "text script" ] = function() {}; + } + + // Convert no matter what (that way responseXXX fields are always set) + response = ajaxConvert( s, response, jqXHR, isSuccess ); + + // If successful, handle type chaining + if ( isSuccess ) { + + // Set the If-Modified-Since and/or If-None-Match header, if in ifModified mode. + if ( s.ifModified ) { + modified = jqXHR.getResponseHeader( "Last-Modified" ); + if ( modified ) { + jQuery.lastModified[ cacheURL ] = modified; + } + modified = jqXHR.getResponseHeader( "etag" ); + if ( modified ) { + jQuery.etag[ cacheURL ] = modified; + } + } + + // if no content + if ( status === 204 || s.type === "HEAD" ) { + statusText = "nocontent"; + + // if not modified + } else if ( status === 304 ) { + statusText = "notmodified"; + + // If we have data, let's convert it + } else { + statusText = response.state; + success = response.data; + error = response.error; + isSuccess = !error; + } + } else { + + // Extract error from statusText and normalize for non-aborts + error = statusText; + if ( status || !statusText ) { + statusText = "error"; + if ( status < 0 ) { + status = 0; + } + } + } + + // Set data for the fake xhr object + jqXHR.status = status; + jqXHR.statusText = ( nativeStatusText || statusText ) + ""; + + // Success/Error + if ( isSuccess ) { + deferred.resolveWith( callbackContext, [ success, statusText, jqXHR ] ); + } else { + deferred.rejectWith( callbackContext, [ jqXHR, statusText, error ] ); + } + + // Status-dependent callbacks + jqXHR.statusCode( statusCode ); + statusCode = undefined; + + if ( fireGlobals ) { + globalEventContext.trigger( isSuccess ? "ajaxSuccess" : "ajaxError", + [ jqXHR, s, isSuccess ? success : error ] ); + } + + // Complete + completeDeferred.fireWith( callbackContext, [ jqXHR, statusText ] ); + + if ( fireGlobals ) { + globalEventContext.trigger( "ajaxComplete", [ jqXHR, s ] ); + + // Handle the global AJAX counter + if ( !( --jQuery.active ) ) { + jQuery.event.trigger( "ajaxStop" ); + } + } + } + + return jqXHR; + }, + + getJSON: function( url, data, callback ) { + return jQuery.get( url, data, callback, "json" ); + }, + + getScript: function( url, callback ) { + return jQuery.get( url, undefined, callback, "script" ); + } +} ); + +jQuery.each( [ "get", "post" ], function( _i, method ) { + jQuery[ method ] = function( url, data, callback, type ) { + + // Shift arguments if data argument was omitted + if ( isFunction( data ) ) { + type = type || callback; + callback = data; + data = undefined; + } + + // The url can be an options object (which then must have .url) + return jQuery.ajax( jQuery.extend( { + url: url, + type: method, + dataType: type, + data: data, + success: callback + }, jQuery.isPlainObject( url ) && url ) ); + }; +} ); + +jQuery.ajaxPrefilter( function( s ) { + var i; + for ( i in s.headers ) { + if ( i.toLowerCase() === "content-type" ) { + s.contentType = s.headers[ i ] || ""; + } + } +} ); + + +jQuery._evalUrl = function( url, options, doc ) { + return jQuery.ajax( { + url: url, + + // Make this explicit, since user can override this through ajaxSetup (#11264) + type: "GET", + dataType: "script", + cache: true, + async: false, + global: false, + + // Only evaluate the response if it is successful (gh-4126) + // dataFilter is not invoked for failure responses, so using it instead + // of the default converter is kludgy but it works. + converters: { + "text script": function() {} + }, + dataFilter: function( response ) { + jQuery.globalEval( response, options, doc ); + } + } ); +}; + + +jQuery.fn.extend( { + wrapAll: function( html ) { + var wrap; + + if ( this[ 0 ] ) { + if ( isFunction( html ) ) { + html = html.call( this[ 0 ] ); + } + + // The elements to wrap the target around + wrap = jQuery( html, this[ 0 ].ownerDocument ).eq( 0 ).clone( true ); + + if ( this[ 0 ].parentNode ) { + wrap.insertBefore( this[ 0 ] ); + } + + wrap.map( function() { + var elem = this; + + while ( elem.firstElementChild ) { + elem = elem.firstElementChild; + } + + return elem; + } ).append( this ); + } + + return this; + }, + + wrapInner: function( html ) { + if ( isFunction( html ) ) { + return this.each( function( i ) { + jQuery( this ).wrapInner( html.call( this, i ) ); + } ); + } + + return this.each( function() { + var self = jQuery( this ), + contents = self.contents(); + + if ( contents.length ) { + contents.wrapAll( html ); + + } else { + self.append( html ); + } + } ); + }, + + wrap: function( html ) { + var htmlIsFunction = isFunction( html ); + + return this.each( function( i ) { + jQuery( this ).wrapAll( htmlIsFunction ? html.call( this, i ) : html ); + } ); + }, + + unwrap: function( selector ) { + this.parent( selector ).not( "body" ).each( function() { + jQuery( this ).replaceWith( this.childNodes ); + } ); + return this; + } +} ); + + +jQuery.expr.pseudos.hidden = function( elem ) { + return !jQuery.expr.pseudos.visible( elem ); +}; +jQuery.expr.pseudos.visible = function( elem ) { + return !!( elem.offsetWidth || elem.offsetHeight || elem.getClientRects().length ); +}; + + + + +jQuery.ajaxSettings.xhr = function() { + try { + return new window.XMLHttpRequest(); + } catch ( e ) {} +}; + +var xhrSuccessStatus = { + + // File protocol always yields status code 0, assume 200 + 0: 200, + + // Support: IE <=9 only + // #1450: sometimes IE returns 1223 when it should be 204 + 1223: 204 + }, + xhrSupported = jQuery.ajaxSettings.xhr(); + +support.cors = !!xhrSupported && ( "withCredentials" in xhrSupported ); +support.ajax = xhrSupported = !!xhrSupported; + +jQuery.ajaxTransport( function( options ) { + var callback, errorCallback; + + // Cross domain only allowed if supported through XMLHttpRequest + if ( support.cors || xhrSupported && !options.crossDomain ) { + return { + send: function( headers, complete ) { + var i, + xhr = options.xhr(); + + xhr.open( + options.type, + options.url, + options.async, + options.username, + options.password + ); + + // Apply custom fields if provided + if ( options.xhrFields ) { + for ( i in options.xhrFields ) { + xhr[ i ] = options.xhrFields[ i ]; + } + } + + // Override mime type if needed + if ( options.mimeType && xhr.overrideMimeType ) { + xhr.overrideMimeType( options.mimeType ); + } + + // X-Requested-With header + // For cross-domain requests, seeing as conditions for a preflight are + // akin to a jigsaw puzzle, we simply never set it to be sure. + // (it can always be set on a per-request basis or even using ajaxSetup) + // For same-domain requests, won't change header if already provided. + if ( !options.crossDomain && !headers[ "X-Requested-With" ] ) { + headers[ "X-Requested-With" ] = "XMLHttpRequest"; + } + + // Set headers + for ( i in headers ) { + xhr.setRequestHeader( i, headers[ i ] ); + } + + // Callback + callback = function( type ) { + return function() { + if ( callback ) { + callback = errorCallback = xhr.onload = + xhr.onerror = xhr.onabort = xhr.ontimeout = + xhr.onreadystatechange = null; + + if ( type === "abort" ) { + xhr.abort(); + } else if ( type === "error" ) { + + // Support: IE <=9 only + // On a manual native abort, IE9 throws + // errors on any property access that is not readyState + if ( typeof xhr.status !== "number" ) { + complete( 0, "error" ); + } else { + complete( + + // File: protocol always yields status 0; see #8605, #14207 + xhr.status, + xhr.statusText + ); + } + } else { + complete( + xhrSuccessStatus[ xhr.status ] || xhr.status, + xhr.statusText, + + // Support: IE <=9 only + // IE9 has no XHR2 but throws on binary (trac-11426) + // For XHR2 non-text, let the caller handle it (gh-2498) + ( xhr.responseType || "text" ) !== "text" || + typeof xhr.responseText !== "string" ? + { binary: xhr.response } : + { text: xhr.responseText }, + xhr.getAllResponseHeaders() + ); + } + } + }; + }; + + // Listen to events + xhr.onload = callback(); + errorCallback = xhr.onerror = xhr.ontimeout = callback( "error" ); + + // Support: IE 9 only + // Use onreadystatechange to replace onabort + // to handle uncaught aborts + if ( xhr.onabort !== undefined ) { + xhr.onabort = errorCallback; + } else { + xhr.onreadystatechange = function() { + + // Check readyState before timeout as it changes + if ( xhr.readyState === 4 ) { + + // Allow onerror to be called first, + // but that will not handle a native abort + // Also, save errorCallback to a variable + // as xhr.onerror cannot be accessed + window.setTimeout( function() { + if ( callback ) { + errorCallback(); + } + } ); + } + }; + } + + // Create the abort callback + callback = callback( "abort" ); + + try { + + // Do send the request (this may raise an exception) + xhr.send( options.hasContent && options.data || null ); + } catch ( e ) { + + // #14683: Only rethrow if this hasn't been notified as an error yet + if ( callback ) { + throw e; + } + } + }, + + abort: function() { + if ( callback ) { + callback(); + } + } + }; + } +} ); + + + + +// Prevent auto-execution of scripts when no explicit dataType was provided (See gh-2432) +jQuery.ajaxPrefilter( function( s ) { + if ( s.crossDomain ) { + s.contents.script = false; + } +} ); + +// Install script dataType +jQuery.ajaxSetup( { + accepts: { + script: "text/javascript, application/javascript, " + + "application/ecmascript, application/x-ecmascript" + }, + contents: { + script: /\b(?:java|ecma)script\b/ + }, + converters: { + "text script": function( text ) { + jQuery.globalEval( text ); + return text; + } + } +} ); + +// Handle cache's special case and crossDomain +jQuery.ajaxPrefilter( "script", function( s ) { + if ( s.cache === undefined ) { + s.cache = false; + } + if ( s.crossDomain ) { + s.type = "GET"; + } +} ); + +// Bind script tag hack transport +jQuery.ajaxTransport( "script", function( s ) { + + // This transport only deals with cross domain or forced-by-attrs requests + if ( s.crossDomain || s.scriptAttrs ) { + var script, callback; + return { + send: function( _, complete ) { + script = jQuery( " - - - - - + + + diff --git a/docs/advanced/cpp_autograd.html b/docs/advanced/cpp_autograd.html index 861cd19e3..6748714cf 100644 --- a/docs/advanced/cpp_autograd.html +++ b/docs/advanced/cpp_autograd.html @@ -174,7 +174,7 @@

프론트엔드 API