00_Pytorch_Basics_p2.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "### **Precision in Computing**\n",
        "\n",
        "It refers to the level of detail or exactness in representing numerical values.\n",
        "\n",
        "It determines how accurately a number is stored and manipulated within a computer system. **Precision** is often characterized by the number of digits used to represent a value.\n",
        "\n",
        "Higher precision allows for more detailed and accurate representation, reducing rounding errors and enhancing calculations.\n",
        "\n",
        "For example:\n",
        "\n",
        "- `Single precision`: Typically uses `32 bits` to represent a floating-point number, offering about `7` decimal digits of accuracy.\n",
        "\n",
        "- `Double precision`: Uses `64 bits`, providing about `15-16` decimal digits of accuracy.\n",
        "\n",
        "Precision is crucial in applications requiring high accuracy, such as scientific computations, graphics, and financial modeling."
      ],
      "metadata": {
        "id": "tb47skY5X6Ju"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **Tensor datatypes**\n",
        "\n",
        "Note: Tensor datatypes is one of the 3 big errors you'll run into with PyTorch & deep learning:\n",
        "\n",
        "- Tensors not right `datatype`.\n",
        "- Tensors not right `shape`.\n",
        "- Tensors not on the right `device`."
      ],
      "metadata": {
        "id": "tO7PZfi7OiiP"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import torch"
      ],
      "metadata": {
        "id": "nQTpT8atO47e"
      },
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Float 32 tensor\n",
        "float_32_tensor = torch.tensor([3.0, 6.0, 9.0],\n",
        "                               dtype=None,          # what datatype is the tensor (e.g. float32 or float16)\n",
        "                               device=None,         # What device is your tensor on\n",
        "                               requires_grad=False) # whether or not to track gradients with this tensors operations\n",
        "\n",
        "print(float_32_tensor.shape)\n",
        "print(float_32_tensor.ndim)\n",
        "print(float_32_tensor.dtype)\n",
        "print(float_32_tensor.device)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZVc3c3htOvjt",
        "outputId": "cb2a2e64-1bed-4255-8b81-ce40b2be0dcc"
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "torch.Size([3])\n",
            "1\n",
            "torch.float32\n",
            "cpu\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "float_16_tensor = float_32_tensor.type(torch.float16)\n",
        "\n",
        "print(float_16_tensor.dtype)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NRR6Z97dPFkG",
        "outputId": "2d42f571-692c-4bda-a510-c54541a44dd8"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "torch.float16\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "float_16_tensor * float_32_tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Xkn-gJXuPPvm",
        "outputId": "173f90f8-cd96-40c8-e0c2-5d2bd6b48a89"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([ 9., 36., 81.])"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "In PyTorch, torch.long is a data type representing `64-bit` integer (or `int64`). It is used when you need to store and manipulate integer values that require a larger range than `32-bit` integers can provide. Common use cases include:\n",
        "\n",
        "- Indexing tensors\n",
        "- Storing large integer values\n",
        "- Handling operations where 32-bit integers might overflow\n",
        "\n",
        "Using `torch.long` ensures that computations involving large integers are accurate and do not result in overflow errors."
      ],
      "metadata": {
        "id": "rT8EhY0xSrA0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "\n",
        "# Create a tensor with float32 type\n",
        "float32_tensor = torch.tensor([3, 6, 9], dtype=torch.float32)\n",
        "print(\"Original tensor:\", float32_tensor)\n",
        "print(\"Original tensor dtype:\", float32_tensor.dtype)\n",
        "\n",
        "# Convert the float32 tensor to int64 (torch.long) type\n",
        "int64_tensor = float32_tensor.to(torch.long)\n",
        "print(\"Converted tensor:\", int64_tensor)\n",
        "print(\"Converted tensor dtype:\", int64_tensor.dtype)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ju2y5xhNPjaW",
        "outputId": "015bcc0f-d4e5-4dea-99a1-5767ccc8f46f"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Original tensor: tensor([3., 6., 9.])\n",
            "Original tensor dtype: torch.float32\n",
            "Converted tensor: tensor([3, 6, 9])\n",
            "Converted tensor dtype: torch.int64\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Manipulating Tensors (tensor operations)\n",
        "Tensor opertions include:\n",
        "\n",
        "- `Addition`\n",
        "- `Subtraction`\n",
        "- `Multiplication` (element-wise)\n",
        "- `Division`\n",
        "- `Matrix multiplication`"
      ],
      "metadata": {
        "id": "JFbHvgYVVVsL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "tensor = torch.tensor([1, 2, 3])\n",
        "\n",
        "print(tensor+10)\n",
        "print(tensor-10)\n",
        "print(tensor*10)\n",
        "print(tensor/10)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dOSzBEFCPzId",
        "outputId": "2f756733-3a83-4248-de20-4f565fc2885b"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([11, 12, 13])\n",
            "tensor([-9, -8, -7])\n",
            "tensor([10, 20, 30])\n",
            "tensor([0.1000, 0.2000, 0.3000])\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**PyTorch in-built functions**"
      ],
      "metadata": {
        "id": "Tw--cE5CWcVb"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print(torch.add(tensor, 10))\n",
        "print(torch.sub(tensor, 10))\n",
        "print(torch.mul(tensor, 10))\n",
        "print(torch.div(tensor, 10))\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TwXwDazsV5fc",
        "outputId": "807da9b7-746d-4cab-e34d-e2546b19faff"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([11, 12, 13])\n",
            "tensor([-9, -8, -7])\n",
            "tensor([10, 20, 30])\n",
            "tensor([0.1000, 0.2000, 0.3000])\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **Matrix multiplication**\n",
        "\n",
        "Two main ways of performing multiplication in neural networks and deep learning:\n",
        "\n",
        "- **Element-wise multiplication**\n",
        "- **Matrix mutliplication** (`dot product`)\n",
        "\n",
        "The inner dimensions must match:\n",
        "- `(3, 2) @ (3, 2)` won't work\n",
        "- `(2, 3) @ (3, 2)` will work\n",
        "- `(3, 2) @ (2, 3)` will work\n",
        "\n",
        "The resulting matrix has the shape of the outer dimensions:\n",
        "\n",
        "- `(2, 3) @ (3, 2) -> (2, 2)`\n",
        "- `(3, 2) @ (2, 3) -> (3, 3)`"
      ],
      "metadata": {
        "id": "N2XKGsjxWmWm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print(tensor*tensor)\n",
        "print(torch.mul(tensor, tensor))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ke7N7SSfWDxz",
        "outputId": "6c8ca02a-fd8b-4a64-d1d2-7ec2765e2c9b"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([1, 4, 9])\n",
            "tensor([1, 4, 9])\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "\n",
        "tensor = torch.tensor([1, 2, 3])\n",
        "\n",
        "value = 0\n",
        "for i in range(len(tensor)):\n",
        "  value += tensor[i] * tensor[i]\n",
        "print(value)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-A-WWwfZY86x",
        "outputId": "721f50f9-199a-4515-c4b8-2a6b87caaca0"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor(14)\n",
            "CPU times: user 1.73 ms, sys: 0 ns, total: 1.73 ms\n",
            "Wall time: 1.84 ms\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Faster due to optimized PyTorch operation.\n",
        "%%time\n",
        "torch.mul(tensor, tensor)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SsuC2UxLZvA6",
        "outputId": "b965fbb3-59a6-4e42-8d07-88dedc55ed2e"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 28 µs, sys: 6 µs, total: 34 µs\n",
            "Wall time: 36.5 µs\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([1, 4, 9])"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "from timeit import default_timer as timer\n",
        "\n",
        "# Function to time the first snippet (for loop)\n",
        "def time_for_loop():\n",
        "    tensor = torch.tensor([1, 2, 3])\n",
        "    value = 0\n",
        "    start_time = timer()\n",
        "    for i in range(len(tensor)):\n",
        "        value += tensor[i] * tensor[i]\n",
        "    end_time = timer()\n",
        "    elapsed_time = end_time - start_time\n",
        "    print(f\"Result of for loop: {value}\")\n",
        "    print(f\"Time taken by for loop: {elapsed_time:.8f} seconds\")\n",
        "    return elapsed_time\n",
        "\n",
        "# Function to time the second snippet (torch.mul)\n",
        "def time_torch_mul():\n",
        "    tensor = torch.tensor([1, 2, 3])\n",
        "    start_time = timer()\n",
        "    result = torch.mul(tensor, tensor)\n",
        "    end_time = timer()\n",
        "    elapsed_time = end_time - start_time\n",
        "    print(f\"Result of torch.mul: {result}\")\n",
        "    print(f\"Time taken by torch.mul: {elapsed_time:.8f} seconds\")\n",
        "    return elapsed_time\n",
        "\n",
        "# Time both snippets\n",
        "time_for_loop_elapsed = time_for_loop()\n",
        "time_torch_mul_elapsed = time_torch_mul()\n",
        "\n",
        "# Compare and print which is faster\n",
        "if time_for_loop_elapsed < time_torch_mul_elapsed:\n",
        "    print(f\"For loop is faster by {time_torch_mul_elapsed - time_for_loop_elapsed:.8f} seconds.\")\n",
        "else:\n",
        "    print(f\"torch.mul is faster by {time_for_loop_elapsed - time_torch_mul_elapsed:.8f} seconds.\")\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EhzuIi1pgMYg",
        "outputId": "545915e7-f6b3-4f34-e375-1591b5dc8085"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Result of for loop: 14\n",
            "Time taken by for loop: 0.00010150 seconds\n",
            "Result of torch.mul: tensor([1, 4, 9])\n",
            "Time taken by torch.mul: 0.00001485 seconds\n",
            "torch.mul is faster by 0.00008665 seconds.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "`matmal()`"
      ],
      "metadata": {
        "id": "NBSk664ko0ji"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "tensor_A = torch.tensor([[1, 2],\n",
        "                         [3, 4],\n",
        "                         [5, 6]])\n",
        "\n",
        "tensor_B = torch.tensor([[7, 10],\n",
        "                         [8, 11],\n",
        "                         [9, 12]])\n",
        "\n",
        "# torch.matmul(tensor_A, tensor_B)   # RuntimeError: mat1 and mat2 shapes cannot be multiplied (3x2 and 3x2)\n",
        "result = torch.matmul(tensor_A, tensor_B.T)  # Transpose tensor_B before multiplication\n",
        "print(tensor_A.shape, tensor_B.shape)\n",
        "print(result)\n",
        "print(result.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bqSukb2zkv2G",
        "outputId": "466543cc-7161-457c-b174-fa6f1100255a"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "torch.Size([3, 2]) torch.Size([3, 2])\n",
            "tensor([[ 27,  30,  33],\n",
            "        [ 61,  68,  75],\n",
            "        [ 95, 106, 117]])\n",
            "torch.Size([3, 3])\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Finding the `min`, `max`, `mean`, `sum`, etc (tensor aggregation)"
      ],
      "metadata": {
        "id": "pZjDuoJ5pmHW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "\n",
        "# Create a tensor using torch.arange\n",
        "# tensor = torch.randn(3, 3)\n",
        "tensor = torch.arange(1, 10, 1).reshape(3, 3).to(torch.float32)\n",
        "print(\"Original Tensor:\")\n",
        "print(tensor)\n",
        "\n",
        "# Finding the minimum value\n",
        "min_value = torch.min(tensor)\n",
        "print(\"\\nMinimum Value:\")\n",
        "print(min_value.item())\n",
        "\n",
        "# Finding the maximum value\n",
        "max_value = torch.max(tensor)\n",
        "print(\"\\nMaximum Value:\")\n",
        "print(max_value.item())\n",
        "\n",
        "# Finding the mean\n",
        "mean_value = torch.mean(tensor)\n",
        "print(\"\\nMean Value:\")\n",
        "print(mean_value.item())\n",
        "\n",
        "# Finding the sum\n",
        "sum_value = torch.sum(tensor)\n",
        "print(\"\\nSum Value:\")\n",
        "print(sum_value.item())\n",
        "\n",
        "# Finding the indices of the minimum and maximum values\n",
        "min_indices = torch.argmin(tensor)\n",
        "max_indices = torch.argmax(tensor)\n",
        "print(\"\\nIndices of Minimum and Maximum Values:\")\n",
        "print(\"Index of Minimum:\", min_indices.item())\n",
        "print(\"Index of Maximum:\", max_indices.item())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7qCTmYjLk3oF",
        "outputId": "ad7df1d5-cc95-411b-edf6-45b424e72686"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Original Tensor:\n",
            "tensor([[1., 2., 3.],\n",
            "        [4., 5., 6.],\n",
            "        [7., 8., 9.]])\n",
            "\n",
            "Minimum Value:\n",
            "1.0\n",
            "\n",
            "Maximum Value:\n",
            "9.0\n",
            "\n",
            "Mean Value:\n",
            "5.0\n",
            "\n",
            "Sum Value:\n",
            "45.0\n",
            "\n",
            "Indices of Minimum and Maximum Values:\n",
            "Index of Minimum: 0\n",
            "Index of Maximum: 8\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **Reshaping, stacking, squeezing and unsqueezing tensors**"
      ],
      "metadata": {
        "id": "hD4wzOdAsNKt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "\n",
        "# Create a tensor using torch.arange\n",
        "tensor = torch.arange(1, 13).reshape(3, 4)\n",
        "print(\"Original Tensor:\")\n",
        "print(tensor)\n",
        "\n",
        "# Reshape the tensor\n",
        "reshaped_tensor = tensor.reshape(2, 6)\n",
        "print(\"\\nReshaped Tensor (2x6):\")\n",
        "print(reshaped_tensor)\n",
        "\n",
        "# Stack tensors along a new dimension\n",
        "tensor_A = torch.arange(1, 5).reshape(2, 2)\n",
        "tensor_B = torch.arange(5, 9).reshape(2, 2)\n",
        "stacked_tensor = torch.stack((tensor_A, tensor_B), dim=0)\n",
        "print(\"\\nStacked Tensor along dimension 0:\")\n",
        "print(stacked_tensor)\n",
        "\n",
        "# Squeeze the tensor to remove single-dimensional entries\n",
        "unsqueeze_tensor = torch.unsqueeze(tensor, dim=0)\n",
        "print(\"\\nUnsqueezed Tensor:\")\n",
        "print(unsqueeze_tensor.shape)  # Note the shape change after unsqueezing\n",
        "\n",
        "# Unsqueeze the tensor to add a single-dimensional entry\n",
        "squeezed_tensor = torch.squeeze(unsqueeze_tensor, dim=0)\n",
        "print(\"\\nSqueezed Tensor:\")\n",
        "print(squeezed_tensor.shape)  # Note the shape change after squeezing"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3p8rG4W9qxxk",
        "outputId": "f3899252-30c1-49d5-9872-b55270bf6d70"
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Original Tensor:\n",
            "tensor([[ 1,  2,  3,  4],\n",
            "        [ 5,  6,  7,  8],\n",
            "        [ 9, 10, 11, 12]])\n",
            "\n",
            "Reshaped Tensor (2x6):\n",
            "tensor([[ 1,  2,  3,  4,  5,  6],\n",
            "        [ 7,  8,  9, 10, 11, 12]])\n",
            "\n",
            "Stacked Tensor along dimension 0:\n",
            "tensor([[[1, 2],\n",
            "         [3, 4]],\n",
            "\n",
            "        [[5, 6],\n",
            "         [7, 8]]])\n",
            "\n",
            "Unsqueezed Tensor:\n",
            "torch.Size([1, 3, 4])\n",
            "\n",
            "Squeezed Tensor:\n",
            "torch.Size([3, 4])\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "x = torch.arange(1., 10.)\n",
        "print(x, x.shape)\n",
        "\n",
        "x_reshaped = x.reshape(1, 9)\n",
        "print(x_reshaped, x_reshaped.shape)\n",
        "\n",
        "# Remove extra dimensions from x_reshaped\n",
        "x_squeezed = x_reshaped.squeeze()\n",
        "print(f\"\\nNew tensor: {x_squeezed}\")\n",
        "print(f\"New shape: {x_squeezed.shape}\")\n",
        "\n",
        "# Add an extra dimension with unsqueeze\n",
        "x_unsqueezed_0 = x_squeezed.unsqueeze(dim=0)\n",
        "x_unsqueezed_1 = x_squeezed.unsqueeze(dim=1)\n",
        "\n",
        "print(f\"\\nNew tensor: {x_unsqueezed_0}\")\n",
        "print(f\"New shape: {x_unsqueezed_0.shape}\")\n",
        "\n",
        "print(f\"\\nNew tensor: {x_unsqueezed_1}\")\n",
        "print(f\"New shape: {x_unsqueezed_1.shape}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yr2tPi4utEuy",
        "outputId": "b63c2fa9-435d-4f04-b67f-4e5ca748514e"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([1., 2., 3., 4., 5., 6., 7., 8., 9.]) torch.Size([9])\n",
            "tensor([[1., 2., 3., 4., 5., 6., 7., 8., 9.]]) torch.Size([1, 9])\n",
            "\n",
            "New tensor: tensor([1., 2., 3., 4., 5., 6., 7., 8., 9.])\n",
            "New shape: torch.Size([9])\n",
            "\n",
            "New tensor: tensor([[1., 2., 3., 4., 5., 6., 7., 8., 9.]])\n",
            "New shape: torch.Size([1, 9])\n",
            "\n",
            "New tensor: tensor([[1.],\n",
            "        [2.],\n",
            "        [3.],\n",
            "        [4.],\n",
            "        [5.],\n",
            "        [6.],\n",
            "        [7.],\n",
            "        [8.],\n",
            "        [9.]])\n",
            "New shape: torch.Size([9, 1])\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### `dim=0`, `dim=1`"
      ],
      "metadata": {
        "id": "cXRtMJRJu2ka"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "\n",
        "tensor_A = torch.tensor([[1, 2], [3, 4]])\n",
        "tensor_B = torch.tensor([[5, 6]])\n",
        "\n",
        "print(tensor_A.shape, tensor_B.shape)\n",
        "print(tensor_A.ndim, tensor_B.ndim)\n",
        "\n",
        "# Concatenate tensor_B as a new row to tensor_A\n",
        "concatenated_tensor = torch.cat((tensor_A, tensor_B), dim=0)\n",
        "print(concatenated_tensor)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "24DWSbD4tGc0",
        "outputId": "e95063a1-246b-4b56-f5d0-ad8abd98e0d5"
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "torch.Size([2, 2]) torch.Size([1, 2])\n",
            "2 2\n",
            "tensor([[1, 2],\n",
            "        [3, 4],\n",
            "        [5, 6]])\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "\n",
        "tensor_A = torch.tensor([[1, 2], [3, 4]])\n",
        "tensor_B = torch.tensor([[5], [6]])\n",
        "\n",
        "print(tensor_A.shape, tensor_B.shape)\n",
        "print(tensor_A.ndim, tensor_B.ndim)\n",
        "\n",
        "stacked_tensor = torch.cat((tensor_A, tensor_B), dim=1)\n",
        "print(stacked_tensor)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9bjwoUrCufT_",
        "outputId": "4af09432-df9d-4f68-cb87-ff8725961fff"
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "torch.Size([2, 2]) torch.Size([2, 1])\n",
            "2 2\n",
            "tensor([[1, 2, 5],\n",
            "        [3, 4, 6]])\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **Indexing (selecting data from tensors)**"
      ],
      "metadata": {
        "id": "K6tkBWzHvA-s"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "\n",
        "# Create a tensor and reshape it\n",
        "x = torch.arange(1, 10).reshape(1, 3, 3)\n",
        "\n",
        "print(\"Original Tensor and Shape:\")\n",
        "print(x, x.shape)\n",
        "\n",
        "# Print the first (and only) batch\n",
        "print(\"\\nPrint First (and Only) Batch: x[0]\")\n",
        "print(x[0])\n",
        "\n",
        "# Print the first row of the first batch\n",
        "print(\"\\nPrint First Row of First Batch: x[0][0]\")\n",
        "print(x[0][0])\n",
        "\n",
        "# Print the first element of the first row of the first batch\n",
        "print(\"\\nPrint First Element of First Row of First Batch: x[0][0][0]\")\n",
        "print(x[0][0][0])\n",
        "\n",
        "# Print the second element of the first row of the first batch\n",
        "print(\"\\nPrint Second Element of First Row of First Batch: x[0][0][1]\")\n",
        "print(x[0][0][1])\n",
        "\n",
        "# Print the third element of the second row of the first batch\n",
        "print(\"\\nPrint Third Element of Second Row of First Batch: x[0][1][2]\")\n",
        "print(x[0][1][2])\n",
        "\n",
        "# Print the third element of the third row of the first batch\n",
        "print(\"\\nPrint Third Element of Third Row of First Batch: x[0][2][2]\")\n",
        "print(x[0][2][2])\n",
        "\n",
        "# Print all batches, first row elements\n",
        "print(\"\\nPrint All Batches, First Row Elements: x[:, 0]\")\n",
        "print(x[:, 0])\n",
        "\n",
        "# Print all batches, all rows, second column elements\n",
        "print(\"\\nPrint All Batches, All Rows, Second Column Elements: x[:, :, 1]\")\n",
        "print(x[:, :, 1])\n",
        "\n",
        "# Print all batches, second row, second column element\n",
        "print(\"\\nPrint All Batches, Second Row, Second Column Element: x[:, 1, 1]\")\n",
        "print(x[:, 1, 1])\n",
        "\n",
        "# Print the first (and only) batch, first row elements\n",
        "print(\"\\nPrint First (and Only) Batch, First Row Elements: x[0, 0, :]\")\n",
        "print(x[0, 0, :])\n",
        "\n",
        "# Print all batches, all rows, third column elements\n",
        "print(\"\\nPrint All Batches, All Rows, Third Column Elements: x[:, :, 2]\")\n",
        "print(x[:, :, 2])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zGrlN-MZwN9b",
        "outputId": "c597a4f8-2372-4a5a-f256-e613d569c7a0"
      },
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Original Tensor and Shape:\n",
            "tensor([[[1, 2, 3],\n",
            "         [4, 5, 6],\n",
            "         [7, 8, 9]]]) torch.Size([1, 3, 3])\n",
            "\n",
            "Print First (and Only) Batch: x[0]\n",
            "tensor([[1, 2, 3],\n",
            "        [4, 5, 6],\n",
            "        [7, 8, 9]])\n",
            "\n",
            "Print First Row of First Batch: x[0][0]\n",
            "tensor([1, 2, 3])\n",
            "\n",
            "Print First Element of First Row of First Batch: x[0][0][0]\n",
            "tensor(1)\n",
            "\n",
            "Print Second Element of First Row of First Batch: x[0][0][1]\n",
            "tensor(2)\n",
            "\n",
            "Print Third Element of Second Row of First Batch: x[0][1][2]\n",
            "tensor(6)\n",
            "\n",
            "Print Third Element of Third Row of First Batch: x[0][2][2]\n",
            "tensor(9)\n",
            "\n",
            "Print All Batches, First Row Elements: x[:, 0]\n",
            "tensor([[1, 2, 3]])\n",
            "\n",
            "Print All Batches, All Rows, Second Column Elements: x[:, :, 1]\n",
            "tensor([[2, 5, 8]])\n",
            "\n",
            "Print All Batches, Second Row, Second Column Element: x[:, 1, 1]\n",
            "tensor([5])\n",
            "\n",
            "Print First (and Only) Batch, First Row Elements: x[0, 0, :]\n",
            "tensor([1, 2, 3])\n",
            "\n",
            "Print All Batches, All Rows, Third Column Elements: x[:, :, 2]\n",
            "tensor([[3, 6, 9]])\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "torch.arange(1, 19).reshape(2, 1, 3, 3) # 4D Tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MMFJV6lbwavD",
        "outputId": "8e1c87a1-c4ab-47df-df2a-9e877801f5e0"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[[[ 1,  2,  3],\n",
              "          [ 4,  5,  6],\n",
              "          [ 7,  8,  9]]],\n",
              "\n",
              "\n",
              "        [[[10, 11, 12],\n",
              "          [13, 14, 15],\n",
              "          [16, 17, 18]]]])"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "- `2` matrices (or batches)\n",
        "- Each matrix has `1` channel (color channel)\n",
        "- Each channel has a shape of `(3, 3)` (3 rows and 3 columns).\n",
        "\n",
        "To calculate the total number of elements, you multiply these sizes together:\n",
        "`2x1x3x3=18`\n"
      ],
      "metadata": {
        "id": "NrltTlePzJCv"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "torch.arange(1, 55).reshape(2, 3, 3, 3) # 4D Tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HrVUf3TLydM7",
        "outputId": "a382cab5-ab96-4d0f-8362-74fef17480a1"
      },
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[[[ 1,  2,  3],\n",
              "          [ 4,  5,  6],\n",
              "          [ 7,  8,  9]],\n",
              "\n",
              "         [[10, 11, 12],\n",
              "          [13, 14, 15],\n",
              "          [16, 17, 18]],\n",
              "\n",
              "         [[19, 20, 21],\n",
              "          [22, 23, 24],\n",
              "          [25, 26, 27]]],\n",
              "\n",
              "\n",
              "        [[[28, 29, 30],\n",
              "          [31, 32, 33],\n",
              "          [34, 35, 36]],\n",
              "\n",
              "         [[37, 38, 39],\n",
              "          [40, 41, 42],\n",
              "          [43, 44, 45]],\n",
              "\n",
              "         [[46, 47, 48],\n",
              "          [49, 50, 51],\n",
              "          [52, 53, 54]]]])"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "3AAif8JG23vv"
      },
      "execution_count": 20,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **PyTorch tensors & NumPy**"
      ],
      "metadata": {
        "id": "bZC5wh3u0FJL"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- NumPy $\\rightarrow$ PyTorch tensor $\\rangle$ `torch.from_numpy(ndarray)`\n",
        "- PyTorch tensor $\\rightarrow$ NumPy $\\rangle$ `torch.Tensor.numpy()`"
      ],
      "metadata": {
        "id": "YNinDivb2yeY"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "import torch\n",
        "\n",
        "# Create a NumPy array\n",
        "numpy_array = np.array([[1, 2, 3], [4, 5, 6]])\n",
        "\n",
        "# Convert NumPy array to PyTorch tensor\n",
        "torch_tensor = torch.from_numpy(numpy_array)\n",
        "print(torch_tensor)\n",
        "print(type(torch_tensor))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EaicTi3c4I_C",
        "outputId": "b11dea5f-b802-483f-dd79-d25136d9490f"
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([[1, 2, 3],\n",
            "        [4, 5, 6]])\n",
            "<class 'torch.Tensor'>\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "\n",
        "# Create a PyTorch tensor\n",
        "torch_tensor = torch.tensor([[1, 2, 3], [4, 5, 6]])\n",
        "\n",
        "# Convert PyTorch tensor to NumPy array\n",
        "numpy_array = torch_tensor.numpy()\n",
        "print(numpy_array)\n",
        "print(type(numpy_array))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jvoqUuuQ3lMY",
        "outputId": "e2dde3ad-e0f7-4648-87a0-1cca6f0c1539"
      },
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[[1 2 3]\n",
            " [4 5 6]]\n",
            "<class 'numpy.ndarray'>\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **device**"
      ],
      "metadata": {
        "id": "hB8QApuo4xhQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
        "print(device)\n",
        "\n",
        "# Count number of devices\n",
        "print(torch.cuda.device_count())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7cLfkPy84jfh",
        "outputId": "1b9d0b1f-c820-494f-ec6c-4894ac7bf6f3"
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "cuda\n",
            "1\n"
          ]
        }
      ]
    }
  ]
}