[RFC] TileOPs Architecture Refactoring

[zhen8838]

Description

This RFC proposes a comprehensive refactoring to reduce development barriers, optimize architecture design, and improve development efficiency.

Type of Change

• Bug fix
• New operator implementation
• Performance improvement
• Documentation update
• Infrastructure/CI

Checklist

• I have run pre-commit run --all-files and fixed all linting issues.
• I have verified that my changes pass local unit tests.
• (For new ops) I have added the corresponding Benchmark class in benchmarks/.
• (For new ops) I have reported benchmark results in the tracking issue.

[gemini-code-assist]

Summary of Changes

Hello @zhen8838, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a Request for Comments (RFC) document detailing a significant architectural refactoring for the TileOPs project. The primary motivation is to simplify the system, making it easier for developers to contribute new operators, improve overall code quality, and accelerate development cycles by addressing current issues like excessive abstraction layers and coupled testing/benchmarking practices.

Highlights

• Comprehensive Refactoring Proposal: Introduces an RFC for a major architectural overhaul of TileOPs to enhance development efficiency and reduce barriers.
• Addressing Abstraction Redundancy: Identifies and proposes to eliminate a four-layer abstraction (Layer → Function → Op → Kernel) that currently causes high cognitive load, code duplication, and maintenance overhead.
• Decoupling Tests and Benchmarks: Outlines a new test framework design to separate correctness verification from performance profiling, aiming to reduce test execution time by 80% and simplify test code.
• New Directory Structure: Proposes a streamlined file organization, including the removal of layers/ and functions/, and the introduction of experimental/ for prototyping and microbenchmarks/ for GPU hardware analysis.
• Improved Developer Experience: Targets significant improvements in key metrics such as reducing the number of files required for a new operator from 7 to 2-3 and decreasing overall code redundancy from ~60% to ~10%.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• docs/rfcs/001_tileops_architecture_refactor_rfc.md
  • Added a new RFC document outlining the proposed architecture refactoring for TileOPs.

Activity

• The author, zhen8838, has created this RFC to propose a major architectural refactoring for TileOPs.
• The author has ensured that pre-commit hooks have been run and local unit tests pass, indicating initial code quality checks for the documentation.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This is a well-structured and detailed RFC for refactoring the TileOPs architecture. The proposal effectively communicates the current problems and the planned solutions. My review includes a few minor suggestions to correct typos, resolve an inconsistency in the proposed test structure, and improve the accuracy of a performance metric calculation in an example. Overall, the proposed changes seem well-considered and should significantly improve the developer experience.

[Copilot]

Pull request overview

This RFC proposes a comprehensive architectural refactoring of the TileOPs codebase to reduce development complexity and improve efficiency. The document outlines a plan to eliminate redundant abstraction layers (layers/ and functions/ directories), separate test and benchmark responsibilities, and introduce an experimental directory for rapid prototyping.

Changes:

• Proposes eliminating the four-layer architecture (layers → functions → ops → kernels) down to two layers (ops → kernels)
• Introduces separation of concerns between correctness tests and performance benchmarks through new TestBase and BenchmarkBase classes
• Adds new experimental/ directory for prototyping and microbenchmarks/ directory for GPU micro-benchmarks

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[xZacky]

Maybe ops/experimental/ and kernels/experimental where we can place experimental operator interfaces and experimental tilelang kernels.

Directory	Responsibility	Status
ops/	Stable operator implementations	Core
ops/experimental/	Experimental operator implementations	New
kernels/	High-performance kernel implementations	Core
kernels/experimental/	Experimental kernel implementations	New
tests/	Correctness tests	Core
tests/experimental/	Experimental operator tests	New
benchmarks/	Performance benchmarks	Core
benchmarks/microbenchmarks/	GPU micro-benchmarks	New

[zhen8838]

Good Idea

[RMLYC]

If you want to completely separate benchmarks from tests, it is recommended to add a utility module to uniformly manage test parameters. Managing pytest parameters separately for tests and benchmarks can lead to redundancy and potential inconsistencies in parameters. Using a set of parameters that have not been verified for correctness to test performance seems meaningless.

[xZacky]

I think the current benchmarks don't need to be executed automatically via pytest. They can be run manually as needed during performance tuning—including installing baseline dependencies, autotuning, and profiling.

[lcy-seso]

After reading through this RFC, my takeaway is that we should first focus on making the high-level decisions and then move on to the “how.” There are four key points:

1. Delete layers and functions.

2. Separate benchmarks and unit tests.

3. Add an experimental area for unstable features.

4. Add a macro benchmark category.

I think the first two can be agreed on quickly and moved into action. The latter two can be introduced first, and then we can decide what content should go into each category.

[lcy-seso]

If you want to completely separate benchmarks from tests, it is recommended to add a utility module to uniformly manage test parameters. Managing pytest parameters separately for tests and benchmarks can lead to redundancy and potential inconsistencies in parameters. Using a set of parameters that have not been verified for correctness to test performance seems meaningless.

I think this falls into the “how” category. That's it to say we all agree that unit tests should be separated from benchmarks, as they are two very different things.