13_unit_test.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ChatGPT写的功能代码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def format_time(seconds):\n",
    "    minutes, seconds = divmod(seconds, 60)\n",
    "    hours, minutes = divmod(minutes, 60)\n",
    "\n",
    "    if hours > 0:\n",
    "        return f\"{hours}h{minutes}min{seconds}s\"\n",
    "    elif minutes > 0:\n",
    "        return f\"{minutes}min{seconds}s\"\n",
    "    else:\n",
    "        return f\"{seconds}s\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import openai, os\n",
    "\n",
    "openai.api_key = os.getenv(\"OPENAI_API_KEY\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ChatGPT写的简单的单元测试"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pytest\n",
    "\n",
    "def test_format_time():\n",
    "    assert format_time(1) == \"1s\"\n",
    "    assert format_time(59) == \"59s\"\n",
    "    assert format_time(60) == \"1min0s\"\n",
    "    assert format_time(61) == \"1min1s\"\n",
    "    assert format_time(3600) == \"1h0min0s\"\n",
    "    assert format_time(3661) == \"1h1min1s\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 第一步，解释功能代码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " we use the `divmod` built-in function to get the quotient and remainder of `seconds` divided by 60. This is assigned to the variables `minutes` and `seconds`, respectively.\n",
      "- Next, we do the same thing with `minutes` and 60, assigning the results to `hours` and `minutes`.\n",
      "- Finally, we use string interpolation to return a string formatted according to how many hours/minutes/seconds are left.\n"
     ]
    }
   ],
   "source": [
    "def gpt35(prompt, model=\"text-davinci-002\", temperature=0.4, max_tokens=1000, \n",
    "          top_p=1, stop=[\"\\n\\n\", \"\\n\\t\\n\", \"\\n    \\n\"]):\n",
    "    response = openai.Completion.create(\n",
    "        model=model,\n",
    "        prompt = prompt,\n",
    "        temperature = temperature,\n",
    "        max_tokens = max_tokens,\n",
    "        top_p = top_p,\n",
    "        stop = stop\n",
    "        )\n",
    "    message = response[\"choices\"][0][\"text\"]\n",
    "    return message\n",
    "\n",
    "code = \"\"\"\n",
    "def format_time(seconds):\n",
    "    minutes, seconds = divmod(seconds, 60)\n",
    "    hours, minutes = divmod(minutes, 60)\n",
    "\n",
    "    if hours > 0:\n",
    "        return f\"{hours}h{minutes}min{seconds}s\"\n",
    "    elif minutes > 0:\n",
    "        return f\"{minutes}min{seconds}s\"\n",
    "    else:\n",
    "        return f\"{seconds}s\"\n",
    "\"\"\"\n",
    "\n",
    "def explain_code(function_to_test, unit_test_package=\"pytest\"):\n",
    "    prompt = f\"\"\"\"# How to write great unit tests with {unit_test_package}\n",
    "\n",
    "In this advanced tutorial for experts, we'll use Python 3.10 and `{unit_test_package}` to write a suite of unit tests to verify the behavior of the following function.\n",
    "```python\n",
    "{function_to_test}\n",
    "```\n",
    "\n",
    "Before writing any unit tests, let's review what each element of the function is doing exactly and what the author's intentions may have been.\n",
    "- First,\"\"\"\n",
    "    response = gpt35(prompt)\n",
    "    return response, prompt\n",
    "\n",
    "code_explaination, prompt_to_explain_code = explain_code(code)\n",
    "print(code_explaination)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 第二步，设计测试用例"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Normal behavior:\n",
      "    - `format_time(0)` should return `\"0s\"`\n",
      "    - `format_time(59)` should return `\"59s\"`\n",
      "    - `format_time(60)` should return `\"1min0s\"`\n",
      "    - `format_time(119)` should return `\"1min59s\"`\n",
      "    - `format_time(3600)` should return `\"1h0min0s\"`\n",
      "    - `format_time(3601)` should return `\"1h0min1s\"`\n",
      "    - `format_time(3660)` should return `\"1h1min0s\"`\n",
      "    - `format_time(7200)` should return `\"2h0min0s\"`\n",
      "- Invalid inputs:\n",
      "    - `format_time(None)` should raise a `TypeError`\n",
      "    - `format_time(\"abc\")` should raise a `TypeError`\n",
      "    - `format_time(-1)` should raise a `ValueError`\n"
     ]
    }
   ],
   "source": [
    "def generate_a_test_plan(full_code_explaination, unit_test_package=\"pytest\"):\n",
    "    prompt_to_explain_a_plan = f\"\"\"\n",
    "    \n",
    "A good unit test suite should aim to:\n",
    "- Test the function's behavior for a wide range of possible inputs\n",
    "- Test edge cases that the author may not have foreseen\n",
    "- Take advantage of the features of `{unit_test_package}` to make the tests easy to write and maintain\n",
    "- Be easy to read and understand, with clean code and descriptive names\n",
    "- Be deterministic, so that the tests always pass or fail in the same way\n",
    "\n",
    "`{unit_test_package}` has many convenient features that make it easy to write and maintain unit tests. We'll use them to write unit tests for the function above.\n",
    "\n",
    "For this particular function, we'll want our unit tests to handle the following diverse scenarios (and under each scenario, we include a few examples as sub-bullets):\n",
    "-\"\"\"\n",
    "    prompt = full_code_explaination + prompt_to_explain_a_plan\n",
    "    response = gpt35(prompt)\n",
    "    return response, prompt\n",
    "\n",
    "\n",
    "test_plan, prompt_to_get_test_plan = generate_a_test_plan(prompt_to_explain_code + code_explaination)\n",
    "print(test_plan)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 第二步补充，补充更多用例"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " The function is called with a valid number of seconds\n",
      "    - `format_time(1)` should return `\"1s\"`\n",
      "    - `format_time(59)` should return `\"59s\"`\n",
      "    - `format_time(60)` should return `\"1min\"`\n",
      "- The function is called with an invalid number of seconds\n",
      "    - `format_time(-1)` should raise a `ValueError`\n",
      "    - `format_time(\"60\")` should raise a `TypeError`\n",
      "- The function is called with a `None` value\n",
      "    - `format_time(None)` should raise a `TypeError`\n"
     ]
    }
   ],
   "source": [
    "not_enough_test_plan = \"\"\"The function is called with a valid number of seconds\n",
    "    - `format_time(1)` should return `\"1s\"`\n",
    "    - `format_time(59)` should return `\"59s\"`\n",
    "    - `format_time(60)` should return `\"1min\"`\n",
    "\"\"\"\n",
    "\n",
    "approx_min_cases_to_cover = 7\n",
    "elaboration_needed = test_plan.count(\"\\n-\") +1 < approx_min_cases_to_cover \n",
    "if elaboration_needed:\n",
    "        prompt_to_elaborate_on_the_plan = f\"\"\"\n",
    "\n",
    "In addition to the scenarios above, we'll also want to make sure we don't forget to test rare or unexpected edge cases (and under each edge case, we include a few examples as sub-bullets):\n",
    "-\"\"\"\n",
    "        more_test_plan, prompt_to_get_test_plan = generate_a_test_plan(prompt_to_explain_code + code_explaination + not_enough_test_plan + prompt_to_elaborate_on_the_plan)\n",
    "        print(more_test_plan)\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 第三步，撰写单元测试"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".\n",
      "#The first element of the tuple is the name of the test case, and the second element is the value to be passed to the format_time() function.\n",
      "@pytest.mark.parametrize('test_input,expected', [\n",
      "    ('0', '0s'),\n",
      "    ('59', '59s'),\n",
      "    ('60', '1min0s'),\n",
      "    ('119', '1min59s'),\n",
      "    ('3600', '1h0min0s'),\n",
      "    ('3601', '1h0min1s'),\n",
      "    ('3660', '1h1min0s'),\n",
      "    ('7200', '2h0min0s'),\n",
      "])\n",
      "def test_format_time(test_input, expected):\n",
      "    #For each test case, we call the format_time() function and compare the returned value to the expected value.\n",
      "    assert format_time(int(test_input)) == expected\n",
      "\n",
      "\n",
      "#We use the @pytest.mark.parametrize decorator again to test the invalid inputs.\n",
      "@pytest.mark.parametrize('test_input', [\n",
      "    None,\n",
      "    'abc',\n",
      "    -1\n",
      "])\n",
      "def test_format_time_invalid_inputs(test_input):\n",
      "    #For each invalid input, we expect a TypeError or ValueError to be raised.\n",
      "    with pytest.raises((TypeError, ValueError)):\n",
      "        format_time(test_input)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "def generate_test_cases(function_to_test, unit_test_package=\"pytest\"):\n",
    "    starter_comment = \"Below, each test case is represented by a tuple passed to the @pytest.mark.parametrize decorator\"\n",
    "    prompt_to_generate_the_unit_test = f\"\"\"\n",
    "\n",
    "Before going into the individual tests, let's first look at the complete suite of unit tests as a cohesive whole. We've added helpful comments to explain what each line does.\n",
    "```python\n",
    "import {unit_test_package}  # used for our unit tests\n",
    "\n",
    "{function_to_test}\n",
    "\n",
    "#{starter_comment}\"\"\"\n",
    "    full_unit_test_prompt = prompt_to_explain_code + code_explaination + test_plan + prompt_to_generate_the_unit_test\n",
    "    return gpt35(model=\"text-davinci-003\", prompt=full_unit_test_prompt, stop=\"```\"), prompt_to_generate_the_unit_test\n",
    "\n",
    "unit_test_response, prompt_to_generate_the_unit_test = generate_test_cases(code)\n",
    "print(unit_test_response)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 检查代码语法合法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "import ast\n",
    "\n",
    "code_start_index = prompt_to_generate_the_unit_test.find(\"```python\\n\") + len(\"```python\\n\")\n",
    "code_output = prompt_to_generate_the_unit_test[code_start_index:] + unit_test_response\n",
    "try:\n",
    "    ast.parse(code_output)\n",
    "except SyntaxError as e:\n",
    "    print(f\"Syntax error in generated code: {e}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "import pytest  # used for our unit tests\n",
      "\n",
      "\n",
      "def format_time(seconds):\n",
      "    minutes, seconds = divmod(seconds, 60)\n",
      "    hours, minutes = divmod(minutes, 60)\n",
      "\n",
      "    if hours > 0:\n",
      "        return f\"{hours}h{minutes}min{seconds}s\"\n",
      "    elif minutes > 0:\n",
      "        return f\"{minutes}min{seconds}s\"\n",
      "    else:\n",
      "        return f\"{seconds}s\"\n",
      "\n",
      "\n",
      "#Below, each test case is represented by a tuple passed to the @pytest.mark.parametrize decorator.\n",
      "#The first element of the tuple is the name of the test case, and the second element is the value to be passed to the format_time() function.\n",
      "@pytest.mark.parametrize('test_input,expected', [\n",
      "    ('0', '0s'),\n",
      "    ('59', '59s'),\n",
      "    ('60', '1min0s'),\n",
      "    ('119', '1min59s'),\n",
      "    ('3600', '1h0min0s'),\n",
      "    ('3601', '1h0min1s'),\n",
      "    ('3660', '1h1min0s'),\n",
      "    ('7200', '2h0min0s'),\n",
      "])\n",
      "def test_format_time(test_input, expected):\n",
      "    #For each test case, we call the format_time() function and compare the returned value to the expected value.\n",
      "    assert format_time(int(test_input)) == expected\n",
      "\n",
      "\n",
      "#We use the @pytest.mark.parametrize decorator again to test the invalid inputs.\n",
      "@pytest.mark.parametrize('test_input', [\n",
      "    None,\n",
      "    'abc',\n",
      "    -1\n",
      "])\n",
      "def test_format_time_invalid_inputs(test_input):\n",
      "    #For each invalid input, we expect a TypeError or ValueError to be raised.\n",
      "    with pytest.raises((TypeError, ValueError)):\n",
      "        format_time(test_input)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(code_output)\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 实验Bad Case"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'59min59s'"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "format_time(-1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "py310",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}