Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor dialect macro input #655

Open
wants to merge 22 commits into
base: main
Choose a base branch
from
Open
114 changes: 16 additions & 98 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion macro/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ proc-macro2 = "1"
quote = "1"
regex = "1.11.1"
syn = { version = "2", features = ["full"] }
tblgen = { version = "0.4.0", features = [
tblgen = { version = "0.5.0", features = [
"llvm19-0",
], default-features = false, package = "tblgen-alt" }
unindent = "0.2.3"
Expand Down
56 changes: 35 additions & 21 deletions macro/src/dialect.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,39 +16,53 @@ use operation::Operation;
use proc_macro::TokenStream;
use proc_macro2::Span;
use quote::quote;
use std::{env, fmt::Display, path::Path, str};
use std::{
env,
fmt::Display,
path::{Component, Path},
str,
};
use tblgen::{record::Record, record_keeper::RecordKeeper, TableGenParser};

const LLVM_INCLUDE_DIRECTORY: &str = env!("LLVM_INCLUDE_DIRECTORY");

pub fn generate_dialect(input: DialectInput) -> Result<TokenStream, Box<dyn std::error::Error>> {
let mut parser = TableGenParser::new();

if let Some(source) = input.table_gen() {
let base = Path::new(env!("LLVM_INCLUDE_DIRECTORY"));

// Here source looks like `include "foo.td" include "bar.td"`.
for (index, path) in source.split_ascii_whitespace().enumerate() {
if index % 2 == 0 {
continue; // skip "include"
}

let path = &path[1..(path.len() - 1)]; // remove ""
let path = Path::new(path).parent().unwrap();
let path = base.join(path);
parser = parser.add_include_path(&path.to_string_lossy());
}

parser = parser.add_source(source).map_err(create_syn_error)?;
}

if let Some(file) = input.td_file() {
parser = parser.add_source_file(file).map_err(create_syn_error)?;
parser = parser.add_source_file(file);
}

for path in input.include_directories().chain([LLVM_INCLUDE_DIRECTORY]) {
parser = parser.add_include_directory(path);
}

let llvm_include_directory = Path::new(LLVM_INCLUDE_DIRECTORY);

for path in input.directories() {
let path = if matches!(
Path::new(path).components().next(),
Some(Component::CurDir | Component::ParentDir)
) {
path.into()
} else {
llvm_include_directory.join(path).display().to_string()
};

parser = parser.add_include_directory(&path);
}

for path in input
.include_directories()
.chain([env!("LLVM_INCLUDE_DIRECTORY")])
{
parser = parser.add_include_path(path);
if input.files().count() > 0 {
parser = parser.add_source(
&input
.files()
.map(|path| format!(r#"include "{path}""#))
Copy link
Member Author

@raviqqe raviqqe Jan 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that the llvm::SourceMgr class doens't actually handle "multiple files" but there is only one main file and other complementary files in there. So we didn't have to fix the TableGenParser::add_source_file. method...

For now, we fall back to the original approach of defining the main file on the fly with a bunch of include statements.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, now I wonder if we should expose the add_source and add_source_file methods of TableGenParser in the current way because it's confusing and only the first call of the method makes sense in most cases.

.collect::<String>(),
)?;
}

let keeper = parser.parse().map_err(Error::Parse)?;
Expand Down
23 changes: 23 additions & 0 deletions macro/src/dialect/input.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,14 @@ use syn::{parse::Parse, punctuated::Punctuated, Token};

pub struct DialectInput {
name: String,
// TODO Remove this field.
table_gen: Option<String>,
// TODO Remove this field.
td_file: Option<String>,
// TODO Remove this field.
include_directories: Vec<String>,
files: Vec<String>,
directories: Vec<String>,
}

impl DialectInput {
Expand All @@ -27,6 +32,14 @@ impl DialectInput {
pub fn include_directories(&self) -> impl Iterator<Item = &str> {
self.include_directories.iter().map(Deref::deref)
}

pub fn files(&self) -> impl Iterator<Item = &str> {
self.files.iter().map(Deref::deref)
}

pub fn directories(&self) -> impl Iterator<Item = &str> {
self.directories.iter().map(Deref::deref)
}
}

impl Parse for DialectInput {
Expand All @@ -35,6 +48,8 @@ impl Parse for DialectInput {
let mut table_gen = None;
let mut td_file = None;
let mut includes = vec![];
let mut files = vec![];
let mut directories = vec![];

for item in Punctuated::<InputField, Token![,]>::parse_terminated(input)? {
match item {
Expand All @@ -44,6 +59,12 @@ impl Parse for DialectInput {
InputField::IncludeDirectories(field) => {
includes = field.into_iter().map(|literal| literal.value()).collect()
}
InputField::Files(field) => {
files = field.into_iter().map(|literal| literal.value()).collect()
}
InputField::Directories(field) => {
directories = field.into_iter().map(|literal| literal.value()).collect()
}
}
}

Expand All @@ -52,6 +73,8 @@ impl Parse for DialectInput {
table_gen,
td_file,
include_directories: includes,
files,
directories,
})
}
}
14 changes: 14 additions & 0 deletions macro/src/dialect/input/input_field.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ pub enum InputField {
TableGen(LitStr),
TdFile(LitStr),
IncludeDirectories(Punctuated<LitStr, Token![,]>),
Files(Punctuated<LitStr, Token![,]>),
Directories(Punctuated<LitStr, Token![,]>),
}

impl Parse for InputField {
Expand All @@ -27,6 +29,18 @@ impl Parse for InputField {
Ok(Self::IncludeDirectories(
Punctuated::<LitStr, Token![,]>::parse_terminated(&content)?,
))
} else if ident == format_ident!("files") {
let content;
bracketed!(content in input);
Ok(Self::Files(
Punctuated::<LitStr, Token![,]>::parse_terminated(&content)?,
))
} else if ident == format_ident!("include_directories") {
let content;
bracketed!(content in input);
Ok(Self::Directories(
Punctuated::<LitStr, Token![,]>::parse_terminated(&content)?,
))
} else {
Err(input.error(format!("invalid field {}", ident)))
}
Expand Down
Loading
Loading