Add Japanese language dependencies

docs/examples/huggingface/README.ipynb

@@ -424,6 +424,92 @@
     "requests.post(\"http://localhost:8080/v2/models/transformer/infer\", json=inference_request).json()"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "f1d4b24a-4c09-4743-a086-6f8b143711ad",
+   "metadata": {},
+   "source": [
+    "### Masked Language Modeling\n",
+    "\n",
+    "We can also serve a masked language model. In the following example, we also build the `huggingface` runtime with the `-E japanese` flag to enable support for Japanese tokenizers."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "63e368ee-a5ef-44b7-aab8-cafd30ab227a",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Overwriting ./model-settings.json\n"
+     ]
+    }
+   ],
+   "source": [
+    "%%writefile ./model-settings.json\n",
+    "{\n",
+    "  \"name\": \"transformer\",\n",
+    "  \"implementation\": \"mlserver_huggingface.runtime.HuggingFaceRuntime\",\n",
+    "  \"parameters\": {\n",
+    "    \"extra\": {\n",
+    "      \"task\": \"fill-mask\",\n",
+    "      \"pretrained_model\": \"cl-tohoku/bert-base-japanese\",\n",
+    "      \"pretrained_tokenizer\": \"cl-tohoku/bert-base-japanese\"\n",
+    "    }\n",
+    "  }"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "72b05244-f52f-41c0-a1bc-a5d11d1c75a0",
+   "metadata": {},
+   "source": [
+    "Using the shell to start mlserver like so,\n",
+    "\n",
+    "```shell\n",
+    "mlserver start .\n",
+    "```\n",
+    "we can pass inferences like this. Note the `[MASK]` token. The mask token can be different for different models, so check the HuggingFace model config for special tokens."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "b619c852-f35d-4506-91e1-0a1e3bcb7b8b",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'実際 に 空 が 見える の か?'"
+      ]
+     },
+     "execution_count": 6,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from mlserver_huggingface.codecs import HuggingfaceRequestCodec\n",
+    "import json\n",
+    "\n",
+    "# Test sentence: Is the sky really [MASK]?\n",
+    "test_sentence = \"実際に空が[MASK]のか？\"\n",
+    "# [MASK] = visible\n",
+    "expected_output = \"見える\"\n",
+    "\n",
+    "inference_request = HuggingfaceRequestCodec.encode_request(\n",
+    "    {\"inputs\": [test_sentence]},\n",
+    "    use_bytes=False,\n",
+    ")\n",
+    "json.dumps(inference_request.dict())\n",
+    "response = requests.post(\"http://localhost:8080/v2/models/transformer/infer\", json=inference_request.dict()).json()\n",
+    "json.loads(response['outputs'][0]['data'][0])[\"sequence\"]"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "fe6655d9",
@@ -721,7 +807,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.8"
+   "version": "3.9.5"
   }
  },
  "nbformat": 4,

docs/examples/huggingface/README.md

@@ -199,6 +199,46 @@ inference_request = {
 requests.post("http://localhost:8080/v2/models/transformer/infer", json=inference_request).json()
 ```
 
+### Masked Language Modeling
+
+We can also serve a masked language model. In the following example, we also build the `huggingface` runtime with the `-E japanese` flag to enable support for Japanese tokenizers.
+
+```python
+%%writefile ./model-settings.json
+{
+  "name": "model",
+  "implementation": "mlserver_huggingface.runtime.HuggingFaceRuntime",
+  "parameters": {
+    "extra": {
+      "task": "fill-mask",
+      "pretrained_model": "cl-tohoku/bert-base-japanese",
+      "pretrained_tokenizer": "cl-tohoku/bert-base-japanese"
+    }
+  }
+}
+```
+Using the shell to start mlserver like so,
+```shell
+mlserver start .
+```
+we can pass inferences like this. Note the `[MASK]` token. The mask token can be different for different models, so check the HuggingFace model config for special tokens.
+```python
+from mlserver_huggingface.codecs import HuggingfaceRequestCodec
+import json
+
+# Test sentence: Is the sky really [MASK]?
+test_sentence = "実際に空が[MASK]のか？"
+# [MASK] = visible
+expected_output = "見える"
+
+inference_request = HuggingfaceRequestCodec.encode_request(
+    {"inputs": [test_sentence]},
+    use_bytes=False,
+)
+json.dumps(inference_request.dict())
+response = requests.post("http://localhost:8080/v2/models/transformer/infer", json=inference_request.dict()).json()
+json.loads(response['outputs'][0]['data'][0])["sequence"]
+```
 ## GPU Acceleration
 
 We can also evaluate GPU acceleration, we can test the speed on CPU vs GPU using the following parameters