Deep-AST

Parse the AST for any Call nodes encountered.
Report Bug · Request Feature ·

About The Project

deep-ast is a Python library that attempts to parse all AST that would be encountered when invoking a callable in python. It does this by extending the ast.NodeVistor and ast.NodeTransformer classes, so that when an ast.Call node is encountered, it's source code is parsed into an ast.Node and then passes the node to the visit() method.

We say attempts because currently there are some limitations. It cant parse any arbitrary code like exec('rorrEeulaV esiar'[::-1]) which raises a ValueError, functions that aren't written in Python and all python internals like print().

If you have a way around these limitations then a PR would be greatly appreciated.

Getting Started

Prerequisites

deep-ast requires python >= 3.7

Installation

deep-ast is available as an easy to install pip package.

pip install deep-ast

Usage

deep-ast offers drop in replacements for the ast.NodeVistor and ast.NodeTransformer classes.

from deep_ast import DeepVisitor, DeepTransformer

To start the deep processing you the deep_visit() method. Each function takes an optional module arugment. This is used if the function/method is not defined in the same file that you calling deep_visit() method in.

Example to get all exceptions

This example shows how you might deeply parse the AST to get all exceptions that might be thrown.

# Custom NodeVisitor to visit Raise nodes and record them
class ParseExceptions(DeepVisitor):
    def __init__(self) -> None:
        self.raw_exceptions = []
        self.found_exceptions = []
        super().__init__()

    def _add_exception(self, name: str):
        self.raw_exceptions.append(name)

        if name not in self.found_exceptions:
            self.found_exceptions.append(name)

    def visit_Raise(self, node):

        exception_obj = node.exc

        if isinstance(exception_obj, (ast.Call, ast.Name):
            name = (
                exception_obj.id
                if isinstance(exception_obj, ast.Name)
                else exception_obj.func.id
            )

            self._add_exception(name)
            return self.generic_visit(node)

        self._add_exception("EmptyRaise")
        return self.generic_visit(node)

# Test functions to visit
def foo():
    bar()
    raise TypeError()


def bar():
    raise ValueError()

parser = ParseExceptions()

parser.deep_visit(foo)

print(parser.found_exceptions) # prints ['ValueError', 'TypeError']

Roadmap

• Parsing of deeply nested attribute calls like foo().bar().bazz()

• Tracking of variable assignment:

  print_func = print
  print_func()

See the open issues for a list of proposed features (and known issues).

Contributing

Contributions are what make the open-source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

This project uses poetry to manage dependencies and pre-commit to run formatting, linting and tests. You will need to have both installed to your system as well as python 3.9.

1. Fork the Project

2. Setup the environment.
   This project uses vscode devcontainer to provide a completly configured development environment. If you are using vscode and have the remote container extension installed, you should be asked to use the devcontainer when you open this project inside of vscode.

   If you are not using devcontainers then you will need to have python installed. Install the poetry, nox, nox_poetry and pre-commit packages. Then run poetry install and pre-commit install commands.

   Most of the steps can be found in the Dockerfile.

3. Create your Feature Branch (git checkout -b feature/AmazingFeature)

4. Commit your Changes (git commit -m 'Add some AmazingFeature')

5. Push to the Branch (git push origin feature/AmazingFeature)

6. Open a Pull Request

Contact

Levi - @shady_cuz

Acknowledgements

• Stackoverflow for getting me started down this road.
• @sethmlarson for asking me which exceptions can be raised.