Skip to content

Latest commit

 

History

History
611 lines (400 loc) · 18.7 KB

hw03.md

File metadata and controls

611 lines (400 loc) · 18.7 KB

 


 

🌀 CSC510: Software Engineering
NC State, Spring '25

Testing is the process of running a program to try and ascertain whether it works as intended. The purpose of testing is to show that bugs exist, not to show that a program is bug-free.

Debugging is the process of trying to fix a program that you already know does not work as intended.

Importance of Testing and Debugging

  • Testing and debugging should not be an afterthought.
  • Good programmers design programs for easier testing and debugging.
  • Break programs into separate components for independent testing.

Testing vs. Debugging

Testing/Validation Debugging
Compare input/output pairs to specifications. Study events leading up to an error.
"It's not working!" Ask “Why is it not working?”
Ask “How can I break my program?” Find ways to fix the program.

The Origin of the Term

  • The term "debugging" is linked to a 1947 event when a moth was found in the Mark II Aiken Relay Calculator at Harvard.

  • The phrase "First actual case of bug being found" was recorded in a lab notebook.

  • Grace Murray Hopper clarified that "bug" was already in use to describe electronic system issues.

  • The term dates back to an 1896 electrical handbook mentioning "bug" for faults in electric apparatus.

    image

    Image sourse

Understanding Bugs in Software

  • Bugs are programmer mistakes: Bugs do not appear randomly; they result from coding errors.
  • Intermittent vs. persistent bugs:
    • Persistent bugs appear every time under the same conditions.
    • Intermittent bugs appear inconsistently, making them harder to detect.
  • Overt vs. covert bugs:
    • Overt bugs are obvious (crashes, slow execution, errors).
    • Covert bugs are hidden and may provide incorrect outputs silently.

The Danger of Intermittent Bugs

  • Bugs that appear inconsistently can be misleading and dangerous.
  • Example: An air traffic control system that mostly works but occasionally miscalculates positions can be catastrophic.
  • Harder-to-reproduce bugs are difficult to identify and fix.

Defensive Programming

  • Aim for bugs that are overt and persistent to ensure they are detected and fixed early.
  • Defensive programming strategies:
    • Use assertions and logging to catch bugs early.
    • Design with fail-fast principles to make errors clear.
    • Encourage clear and modular code for easier debugging.

The Risks of Covert Bugs

  • Covert bugs can go undetected for long periods:
    • Financial software miscalculations can impact entire economies.
    • Aviation software errors can cause flight malfunctions.
    • Medical software inaccuracies can affect patient treatment.
  • Such issues highlight the critical importance of rigorous software testing.

Learning to Debug

Debugging as a Skill

  • Debugging is a learned skill, not an instinct.
  • Transferable skill applicable to other complex systems (e.g., lab experiments, medical diagnosis).
  • Debugging tools exist but mindset and approach matter more.
  • Many experienced programmers rely on print statements over debugging tools.

First Steps in Debugging

  • Ensure the language system can run the program.
  • Eliminate syntax errors and static semantic errors.
  • If struggling with syntax, practice with small programs first.

Debugging Process

1. Study Program Code

  • Debugging starts when testing reveals unexpected behavior.
  • Ask “How did I get the unexpected result?”
  • The goal is to find an explanation for the problem and identify if it's part of a pattern.
  • A systematic approach is key to effective debugging.

2. Debug Using the Scientific Method

Study Available Data

  • Analyze test results and inspect the program code.
  • Compare failing test cases with successful ones to find patterns.
  • Accept that if a bug exists, there’s something you don’t fully understand—your goal is to uncover it.

Form a Hypothesis

  • Develop a hypothesis that explains all observed data.
  • Hypotheses can be:
    • Narrow: “Changing x < y to x <= y will fix the bug.”
    • Broad: “Aliasing issues are causing unexpected behavior.”

Design and Run Experiments

  • Conduct repeatable experiments to validate or refute your hypothesis.
  • Use the simplest possible input to isolate the problem.
  • Example: Add print statements before and after loops to track infinite loops.
  • Plan ahead how to interpret results and avoid confirmation bias (seeing only what you expect).

Keep Records of Experiments

  • Document changes, tests, and results to avoid repeating the same ineffective solutions.
  • Prevent wasting hours debugging the same issue multiple times.
  • Remember: "Insanity is doing the same thing over and over again and expecting different results."

Debug systematically, not randomly.
Question assumptions and verify findings.
Maintain a logical approach and document progress.

Debugging Tools

  • Python Tutor
  • Print Statements
  • Breakpoints & Debugging Tools in IDLE & Anaconda
  • Just be systematic

Debugging with Print Statements

Benefits

  • A simple and effective way to test hypotheses about code behavior.

When to Use Print Statements

  • At the beginning of a function to indicate entry.
  • To display function parameters.
  • To check function results.

Bisection Method for Debugging

  • Insert a print statement at the midpoint of the code.
  • Use the output to determine which half contains the potential issue.
  • Continue narrowing down until the bug is located.

Common Error Messages & Their Causes

Error Type Cause Example
IndexError Trying to access an element at an invalid index. numbers = [10, 20, 30] then numbers[5]
TypeError Using an operation on an incompatible data type. len(42) , int(text)
NameError Attempting to use a variable that has not been defined. print(result) (when result was never assigned)
TypeError Performing operations between mismatched data types. 5 / "hello"
SyntaxError Writing code with incorrect syntax, such as missing brackets or quotes. message = "Hello
print(message)

Debugging Tips & Avoiding Logic Errors

Strategies to Prevent Logical Mistakes

  • Plan before coding: A structured approach minimizes errors.
  • Use visual aids: Diagrams and flowcharts help clarify logic.
  • Explain the problem: Talking through the issue can expose blind spots.
  • Debug systematically: Justifying why the bug *cannot* be in certain places can reveal insights.
  • Clarify thoughts with an external perspective:
    • Discuss the issue with another person.
    • Use "Rubber Duck Debugging" (explain the code to an inanimate object).

Best Practices for Debugging

Step-by-Step Approach

❌ Don't ✅ Do
Write the entire program before testing. Develop and test one function at a time.
Test the entire program at once. Test and debug each function separately.
Debug the entire program at once. Perform integration testing after verifying individual functions.

When You Have Found "The" Bug

Pause Before Fixing

  • Resist the temptation to immediately start coding a fix.
  • The goal is not just to fix one bug but to work towards a bug-free program.
  • Consider whether this bug explains all observed issues or if it’s part of a larger problem.

Evaluate the Fix

  • Does the bug result from mutating a list?
    • Consider making a copy of the list or using a tuple instead.
  • Will the fix introduce new issues or excessive complexity?
  • Can the fix improve or tidy up other parts of the code?

Managing Code Changes During Debugging

❌ Don't ✅ Do
Modify code without tracking changes. Create a backup before making changes.
Try to remember where the bug was. Add comments noting potential issues.
Test the code without documenting changes. Keep track of modifications and compare versions.
Forget what changes were made. Use version control or save checkpoints.
Panic when things break. Stay calm and follow a structured debugging process.

Backup and Reassess

Backup Your Work

  • Ensure you can revert changes if needed.
  • Store old versions of the program to avoid losing progress.

Reassess the Approach

  • If many unexplained errors persist:
    • Consider restructuring the program.
    • Evaluate if a simpler algorithm could be more reliable.

When the Going Gets Tough: Debugging Strategies

Look for the Usual Suspects

  • Common Mistakes to Check First:
    • Passed arguments in the wrong order.
    • Misspelled a variable or function name.
    • Failed to reinitialize a variable.
    • Used == to compare floating-point values instead of checking for near equality.
    • Tested for value equality instead of object equality (L1 == L2 vs. id(L1) == id(L2)).
    • Ignored side effects of built-in functions.
    • Forgot to add () when calling a function.
    • Created an unintentional alias.
    • Made any mistake that is typical for you—track recurring errors!

Change Your Debugging Approach

  • Shift Your Mindset:
    • Stop asking why the program isn’t working as expected.
    • Instead, ask why the program is doing what it is doing.
    • The bug is likely not where you think it is.
    • Eliminate possibilities systematically (Sherlock Holmes' method:
      “Eliminate all other factors, and the one which remains must be the truth.”)

Additional Debugging Best Practices

  • Don’t trust everything you read

    • Documentation may be outdated or incorrect.
    • Code comments might not accurately reflect behavior.

  • Write documentation instead of debugging

    • Describing the issue in words can bring clarity and a fresh perspective.

  • Step away and revisit later

    • Taking breaks can make debugging more efficient.
    • Start debugging early to allow time for breaks when needed.

Finding the Location of a Bug Using Print Statements

def find_max(numbers):
    max_value = numbers[0]
    for num in numbers:
        print(f"Checking: {num}, Current Max: {max_value}")  # Debug print
        if num > max_value:
            max_value = num
    return max_value


print(find_max([3, 1, 7, 2, 8, 5]))

Output:

Checking: 3, Current Max: 3
Checking: 1, Current Max: 3
Checking: 7, Current Max: 3
Checking: 2, Current Max: 7
Checking: 8, Current Max: 7
Checking: 5, Current Max: 8
8

Example: A function to divide two numbers

def divide(a, b):
    return a / b  # This may cause a ZeroDivisionError


# Testing: Finding potential issues
print(divide(10, 2))  # Expected output: 5.0
print(divide(5, 0))   # This causes a ZeroDivisionError

Output:

5.0


......, line 3, in divide
    return a / b  # This may cause a ZeroDivisionError
ZeroDivisionError: division by zero

Debugging: Fixing the issue

def safe_divide(a, b):
    if b == 0:
        print("Error: Division by zero!")
        return None
    return a / b


print(safe_divide(5, 0))  # Debugged version, handles division by zero

Output:

Error: Division by zero!
None

Using assertions to catch unexpected behavior early

def square_root(x):
    assert x >= 0, "Cannot compute square root of a negative number"
    return x ** 0.5


print(square_root(16))  # Works
print(square_root(-4))  # AssertionError

Output:

4.0


AssertionError: Cannot compute square root of a negative number

Debugging Loops with the Bisection Method

# Finding a bug in a loop using print statements
def faulty_loop():
    for i in range(5):
        print(f"Before update: i = {i}")
        i += 2  # This won't affect the loop iteration
        print(f"After update: i = {i}")


faulty_loop()

Output:

Before update: i = 0
After update: i = 2
Before update: i = 1
After update: i = 3
Before update: i = 2
After update: i = 4
Before update: i = 3
After update: i = 5
Before update: i = 4
After update: i = 6

Handling Unexpected Behavior with Logging

import logging


logging.basicConfig(level=logging.DEBUG)


def add(a, b):
    logging.debug(f"Adding {a} and {b}")
    return a + b


result = add(5, 7)
print(result)

Output:

DEBUG:root:Adding 5 and 7
12

Homework 2: Debugging, Static Analysis, and Automated Testing

Objective

In this assignment, you will apply debugging techniques, use static analysis tools, and implement automated testing to improve a faulty merge sort implementation. By the end of this assignment, you will:

  • Identify and fix bugs in hw2_debugging.py, which uses helper functions from rand.py.

  • Use static analysis tools to detect and correct coding issues before execution.

  • Write unit tests with pytest to validate the correctness of your merge sort.

  • Automate code formatting, static analysis, and testing in your GitHub repository.

Submitting Your Work

  • Create a PDF file containing:

    1. Your repository link
    2. Group number
    3. Names of all participants (Name - github user id)

  • Only one group member submits the work.

  • Submit the PDF file.

1. Getting Started

  • Download and unzip hw3.zip.
  • Inside hw3.zip, you will find:
    • hw2_debugging.py (contains a faulty merge sort implementation).
    • rand.py (provides a function to generate a random array).

You can modify files.

  • Your goal is to debug, test, and improve the merge sort function. Make sure to fix logic errors to ensure the sorting algorithm functions correctly.

2. Run Static Analysis Tools

  • Choose "number of group members" of the following:
    • pylint (general static analysis and code quality)
    • pyflakes (error detection)
    • bandit (security vulnerabilities)
    • radon (complexity analysis)

If you have 3 group members, that means that you need to choose 3 tools. Each group member should be responsible for installing 1 tool to the repository. In step 3, you need to run all tools for your code.

The goal is to have tools running in GitHub Actions as part of the workflow.

3. Create your own branch.

  • Add extra code (any algorithm with issues; the algorithm should be different from your group members’). It is recommended that students implement the algorithm by hand rather than copying an example online. This approach will help reinforce the understanding of debugging techniques, which are key learning objectives of this assignment.

  • Run tools on all files:

    pylint hw2_debugging.py rand.py > pylint_report.txt
pyflakes hw2_debugging.py rand.py > pyflakes_report.txt

4. Fix issues reported by static analysis.

  • Correct coding style violations, security concerns, and potential logical errors.
  • Ensure that all fixes maintain the intended functionality.

5. Re-run the Tools

  • After fixing issues, rerun each tool and save the updated results inside a post_traces folder(tools output).

6. Resolve all code issues and commit the changes to your repository.

Make sure to address all static analysis issues.

7. Write 3 test cases for merge sort using pytest.

8. Merge all branches.

9. Configure automated testing to run on every commit.

Important Links

References

  1. Guttag, J. V. Introduction to Computation and Programming Using Python. http://repo.darmajaya.ac.id/5070/1/Introduction%20to%20Computation%20and%20Programming%20Using%20Python%20by%20John%20V.%20Guttag%20%28z-lib.org%29.pdf
  2. Brian Marick, "How to Misuse Code Coverage"
  3. Ayewah, Pugh, Hovemeyer, Morgenthaler, Penix, "Using Static Analysis to Find Bugs" =======

🎯 Grading Scale

  • Total: 1 point.
  • Each step is worth 0.11 points.
  • Ensure all steps are completed to receive full credit.