Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Python package module for Python+Chapel interop #26156

Open
wants to merge 52 commits into
base: main
Choose a base branch
from

Conversation

jabraham17
Copy link
Member

@jabraham17 jabraham17 commented Oct 29, 2024

Adds a new package module, Python, which supports calling Python code from Chapel.

For example, the following Chapel progam use the python interface to BeautifulSoup to parse HTML

import Python;
use URL;
use List;

proc main() {

  const url = 'https://chapel-lang.org/docs/main/language/spec/interoperability.html';
  var htmlReader = openUrlReader(url);
  var html = htmlReader.readAll(string);

  var interp = new Python.Interpreter();
  var mod = new Python.Module(interp, "bs4");

  var cls = new Python.Class(mod, "BeautifulSoup");
  var soup = cls(html, 'html.parser');

  var res: list(owned Python.ClassObject?);
  res = soup.callMethod(res.type, "find_all", "h3");
  for c in res {
    writeln(c!.getAttr(string, "text"));
  }
}

As an another example, this simple program compiles an runs a Python lambda from Chapel code

import Python;

config const n = 10;
config const func = "lambda x,: x + 1 if x % 2 != 0 else x";

proc apply(interp: borrowed, type t, arr, l) {
  var lambdaFunc = new Python.Function(interp, l);
  var res: [arr.domain] t;
  for i in arr.domain {
    res(i) = lambdaFunc(t, arr(i));
  }
  return res;
}
proc main() {
  var interp = new Python.Interpreter();

  var data: [1..#n] int = 1..#n;
  writeln(" data: ", data);

  var res = apply(interp, int, data, func);
  writeln("res: ", res);
}

Future work:

  • Convert classes to a hierarchy under Value so that more implementation can be shared and the Chapel interface can better imitate python
    • Add special support for common python types as sub-classes of value, to avoid round tripping values through chapel in many cases
    • Support common operators like add. These can be called today as dunder methods (.call("__add__", ...)), but it would be nice to support native operators like +
  • Add the ability to compile arbitrary chapel strings, beyond just lambdas
  • Setup python to use Chapel stdout/stderr
  • Add custom adapters for Chapel arrays to allow python functions to operate on Chapel arrays, without copying
  • Support python context managers as Chapel context managers
  • Fully support Python sets, Python dictionaries, Chapel sets, Chapel maps, Chapel associative arrays, and Chapel associative domains
  • Make sure all python objects are properly reference counted
  • Resolve TODO with LLVM IR verification

Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
@jabraham17 jabraham17 marked this pull request as ready for review November 6, 2024 19:19
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Signed-off-by: Jade Abraham <jade.abraham@hpe.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant