Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MR for 2.5.2 feature additions #32

Merged
merged 1 commit into from
Jul 5, 2024
Merged

MR for 2.5.2 feature additions #32

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
225 changes: 132 additions & 93 deletions docs/getting-started/q_magic_command.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,31 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "5c1e9b66",
"metadata": {},
"source": [
"# Jupyter q Magic Command Notebook\n",
"\n",
"_The purpose of this notebook is to exemplify how to use the q Magic command in a Jupyter notebook._\n",
"\n",
"\n",
"The Jupyter q magic command in PyKX allows you to execute q code within a Jupyter notebook. It provides seamless integration with the q programming language.\n",
"\n",
"This example Notebook has the following sections:\n",
"\n",
"1. [Import PyKX](#1-import-pykx)\n",
"1. [Create the external q process](#2-create-the-external-q-process)\n",
"1. [Execute against Embedded q](#3-execute-against-embedded-q)\n",
"1. [SQL interface](#4-sql-interface)\n",
"1. [q namespaces](#5-q-namespaces)\n",
"1. [(Advanced) q over IPC](#6-advanced-q-over-ipc)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2f25482a",
"metadata": {
"tags": [
"hide_code"
Expand All @@ -15,6 +38,16 @@
"os.environ['PYKX_Q_LOADED_MARKER'] = '' # Only used here for running Notebook under mkdocs-jupyter during document generation."
]
},
{
"cell_type": "markdown",
"id": "688b9ed0",
"metadata": {},
"source": [
"## 1. Import PyKX\n",
"\n",
"To run this example, first import the PyKX library:"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand All @@ -30,12 +63,9 @@
"id": "9c520c21",
"metadata": {},
"source": [
"#### Create the external q process\n",
"To run this example, the Python code in the following cell will do the equivalent to executing the following in a terminal:\n",
"## 2. Create the external q process\n",
"\n",
"```\n",
"$ q -p 5001\n",
"```"
"You can run an external q process by using the following Python code:"
]
},
{
Expand All @@ -58,13 +88,27 @@
" raise kx.QError('Unable to create q process on port 5000')"
]
},
{
"cell_type": "markdown",
"id": "1b318ba2",
"metadata": {},
"source": [
"\n",
"Or execute this command in a terminal:\n",
"\n",
"```sh\n",
"$ q -p 5000\n",
"```\n"
]
},
{
"cell_type": "markdown",
"id": "bc7219fb",
"metadata": {},
"source": [
"#### Executing against Embedded q\n",
"A cell beginning with `%%q` will execute q within `PyKX`'s `EmbeddedQ` module."
"## 3. Execute against Embedded q\n",
"\n",
"To execute q code within PyKX's `EmbeddedQ` module, type `%%q` at the beginning of the cell:"
]
},
{
Expand All @@ -83,23 +127,18 @@
"id": "89ec26e4",
"metadata": {},
"source": [
"#### Execution options\n",
"\n",
"Execution options can also be included after `%%q`.\n",
"After `%%q` you can further add two execution options:\n",
"\n",
"Here is the list of currently supported execution options.\n",
"\n",
"```\n",
"--debug: prints the q backtrace before raising a QError\n",
" if the cell errors\n",
"--display: calls display rather than the default print\n",
" on returned objects\n",
"```\n"
"| **Execution option** | **Description** |\n",
"|---------------|----------------------------------------------------|\n",
"| --debug | Prints the q backtrace before raising a QError if the cell gives an error.|\n",
"| --display | Calls display rather than the default print on returned objects.|"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f9ed8310",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -110,6 +149,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "daa9a196",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -119,156 +159,155 @@
},
{
"cell_type": "markdown",
"id": "89ec26e4",
"id": "2905895e",
"metadata": {},
"source": [
"#### Executing against an external q process over IPC\n",
"## 4. SQL interface\n",
"\n",
"Connection information can also be included after the `%%q` to connect to a remote `q` process over\n",
"IPC.\n",
"The `s)` syntax runs SQL queries against local tables within the `q` process.\n",
"\n",
"Here is the list of currently supported connection parameters.\n",
"If they specify a type a second value is expected to follow them to be used as the parameter.\n",
"If no type follows them they can be used as a stand alone flag.\n",
"\n",
"```\n",
"--host: A string object denoting the host to connect to\n",
"--port: An int object denoting the port to connect over\n",
"--user: A str object denoting the username to use when connecting\n",
"--password: A str object denoting the password to use when connecting\n",
"--timeout: A float object denoting the time in seconds before the query\n",
" times out, defaults to no timeout\n",
"--nolarge: Disable messages over 2GB being sent / received\n",
"--tls: Use a tls connection\n",
"--unix: Use a unix connection\n",
"--reconnection_attempts: An int object denoting how many\n",
" reconnection attempts to make\n",
"--noctx: Disable the context interface\n",
"```\n",
"\n",
"Connect to a q server running on `localhost` at port `5001` as `user` using password `password`\n",
"and disable the context interface."
"Note: To use the SQL interface, first you need to load the `s.k_` library."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1faca1e1",
"metadata": {},
"id": "56220bb5",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"%%q --host localhost --port 5000 --user user --pass password --noctx\n",
"til 10"
"%%q\n",
"\\l s.k_\n",
"tab:([]a:1000?1000; b:1000?500.0; c:1000?`AAPL`MSFT`GOOG);\n",
"s) select * from tab where a>500 and b<250.0 limit 5"
]
},
{
"cell_type": "markdown",
"id": "f046ebb6",
"id": "da906296",
"metadata": {},
"source": [
"All connection arguments are optional with the exception of the `--port` argument. If `--host` is not provided `localhost` will be used as the default host."
"## 5. q namespaces\n",
"\n",
"You can use `q` namespaces, and switch between them with `\\d`.\n",
"\n",
"Note: The namespace is reset back to the base namespace `.` between cells."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "615d7d2e",
"id": "502af937",
"metadata": {},
"outputs": [],
"source": [
"%%q --port 5000\n",
"tab:([]a:1000?1000; b:1000?500.0; c:1000?`AAPL`MSFT`GOOG);"
]
},
{
"cell_type": "markdown",
"id": "d756f342",
"metadata": {},
"source": [
"It is possible to execute `q` code spanning multiple lines."
"%%q\n",
"\\d .example\n",
"f: {[x] til x};"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c739a80a",
"metadata": {
"scrolled": true
},
"id": "58d0c7c9",
"metadata": {},
"outputs": [],
"source": [
"%%q --port 5000\n",
"afunc: {[x; y]\n",
" x + y \n",
" };\n",
"afunc[0; 1]\n",
"afunc[2; 3]"
"%%q\n",
"\\d\n",
".example.f[10]"
]
},
{
"cell_type": "markdown",
"id": "2905895e",
"id": "52ca850e",
"metadata": {},
"source": [
"#### Using the SQL interface\n",
"The `s)` syntax to run SQL queries against local tables within the `q` process.\n",
"## 6. (Advanced) q over IPC\n",
"\n",
"After `%%q` you can include connection information, if you wish to connect to a remote `q` process over IPC. \n",
"\n",
"Note: The `s.k_` library must be loaded first to use the SQL interface"
"The list of supported connection parameters is below. The rule is:\n",
"\n",
"- If they have a type, it must be followed by a second value/parameter.\n",
"- If there's no type after them, you can use them as a standalone flag.\n",
"\n",
"| **Parameter** &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| **Object type and description**|\n",
"|-----------------------|-----------------------------------------------|\n",
"|--host | (string) The host to connect to. |\n",
"|--port | (integer) The port to connect over. |\n",
"|--user | (string) The username to use when connecting. |\n",
"|--password | (string) The password to use when connecting. |\n",
"|--timeout | (float) The time in seconds before the query times out. Defaults to no timeout.|\n",
"|--nolarge | Disable messages over 2GB being sent / received. |\n",
"|--tls | Use a tls connection. |\n",
"|--unix | Use a unix connection. |\n",
"|--reconnection_attempts| (integer) How many reconnection attempts to make.|\n",
"|--noctx | Disable the context interface. |\n",
"\n",
"Connect to a q server running on `localhost` at port `5000` as `user` using password `password`\n",
"and disable the context interface."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "56220bb5",
"metadata": {
"scrolled": true
},
"id": "a282e069",
"metadata": {},
"outputs": [],
"source": [
"%%q --port 5000\n",
"\\l s.k_\n",
"s) select * from tab where a>500 and b<250.0 limit 5"
"%%q --host localhost --port 5000 --user user --pass password --noctx\n",
"til 10"
]
},
{
"cell_type": "markdown",
"id": "da906296",
"id": "a1fe3b8e",
"metadata": {},
"source": [
"#### Using namespaces\n",
"You can also use `q` namespaces, and switch between them using `\\d`.\n",
"\n",
"Note: The namespace is reset back to the base namespace `.` between cells."
"All connection arguments are optional, except the `--port` argument. If `--host` is not provided `localhost` is the default host."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "502af937",
"id": "18d8416b",
"metadata": {},
"outputs": [],
"source": [
"%%q --port 5000\n",
"\\d .example\n",
"f: {[x] til x};"
"tab:([]a:1000?1000; b:1000?500.0; c:1000?`AAPL`MSFT`GOOG);"
]
},
{
"cell_type": "markdown",
"id": "e143c382",
"metadata": {},
"source": [
"Note that it's possible to execute `q` code spanning multiple lines:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "58d0c7c9",
"id": "ccb197e8",
"metadata": {},
"outputs": [],
"source": [
"%%q --port 5000\n",
"\\d\n",
".example.f[10]"
"afunc: {[x; y]\n",
" x + y \n",
" };\n",
"afunc[0; 1]\n",
"afunc[2; 3]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "334d0ac5",
"id": "c12a7d38",
"metadata": {},
"outputs": [],
"source": [
Expand Down
Loading
Loading