Capture DSL definitions and support Class|Module.new

[st0012]

DSL Support for Class.new and Module.new

1. Overview

Supported Ruby Patterns

This implementation adds support for dynamically created classes and modules:

# Named dynamic class with inheritance and mixins
Foo = Class.new(Parent) do
  include SomeMixin

  def instance_method
    @ivar = 1
  end

  def self.class_method; end

  CONST = "value"
end

# Named dynamic module
Bar = Module.new do
  def helper; end
end

# Nested dynamic classes
Outer = Class.new do
  Inner = Class.new { def foo; end }
end

# Anonymous (indexed but no declaration)
Class.new { def bar; end }


# Singleton class inside anonymous classes
Baz = Class.new do
  class << self
    def self.qux; end
  end
end

High-Level Approach

┌────────────────────────────────────┐      ┌────────────────────────────────────┐
│             INDEXING               │      │            RESOLUTION              │
├────────────────────────────────────┤      ├────────────────────────────────────┤
│                                    │      │                                    │
│  Foo = Class.new(Parent) { ... }   │      │  1. Resolve receiver "Class"       │
│            │                       │      │     → CLASS_ID                     │
│            ▼                       │      │                                    │
│  ┌──────────────────────────────┐  │      │  2. Match DSL processor            │
│  │ ConstantDefinition(Foo)      │  │      │     class_new_matches() = true     │
│  └──────────────────────────────┘  │      │                                    │
│  ┌──────────────────────────────┐  │      │  3. Create DynamicClassDefinition  │
│  │ DslDefinition                │  │ ───► │     - name_id from Constant        │
│  │   receiver: Class (NameId)   │  │      │     - superclass from args(Parent) │
│  │   method: "new"              │  │      │     - mixins from block            │
│  │   args: [Parent]             │  │      │                                    │
│  │   assigned_to: Foo.id        │  │      │  4. Remove ConstantDefinition      │
│  └──────────────────────────────┘  │      │                                    │
│  ┌──────────────────────────────┐  │      │  5. Resolve DynamicClassDefinition │
│  │ Block contents assigned with │  │      │     → ClassDeclaration(Foo)        │
│  │ lexical_nesting_id = DSL     │  │      │     → Members resolved with        │
│  └──────────────────────────────┘  │      │       owner = Foo                  │
│                                    │      │                                    │
└────────────────────────────────────┘      └────────────────────────────────────┘

2. Class.new and Module.new

Named vs Anonymous

┌─────────────────────────────────────────────────────────────────────────────┐
│                              NAMED                                          │
│  Foo = Class.new { ... }                                                    │
│                                                                             │
│  Creates ClassDeclaration(Foo)                                              │
│  Creates SingletonClassDeclaration(Foo::<Foo>) when applicable              │
│  Members get declarations (Foo#method, Foo::CONST)                          │
│  Inheritance and mixins tracked                                             │
└─────────────────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────────────────┐
│                            ANONYMOUS                                        │
│  Class.new { ... }                                                          │
│                                                                             │
│  No declaration created (can't be referenced by name)                       │
│  Members exist at definition level only (under DyanmicClassDefinition)      │
└─────────────────────────────────────────────────────────────────────────────┘

Superclass and Mixins

Foo = Class.new(Parent) do
  include M1
  prepend M2
  extend E1
end

• First positional argument to Class.new is captured as superclass reference
• include/prepend/extend calls inside the block are attached to the DSL
• During resolution, these are transferred to the DynamicClassDefinition
• Inheritance linearization works the same as class Foo < Parent

Nested Dynamic Classes

Outer = Class.new do
  Inner = Class.new { def foo; end }
end

Challenge: Inner's name must be "Outer::Inner", not just "Inner"

Solution: Depth-first processing with name reparenting

┌────────────────────────────────────────────────────────────────────────┐
│  1. Sort DSLs by name depth (shallower first)                          │
│     → Outer (depth 1) processed before Inner (depth 2)                 │
│                                                                        │
│  2. When processing Inner:                                             │
│     → Check: is parent DSL (Outer) already processed?                  │
│     → If no: re-enqueue Inner, wait for Outer                          │
│     → If yes: reparent name from "Inner" to "Outer::Inner"             │
│                                                                        │
│  Final declarations:                                                   │
│     • Outer                                                            │
│     • Outer::Inner                                                     │
│     • Outer::Inner#foo                                                 │
└────────────────────────────────────────────────────────────────────────┘

3. Elements Inside DSL Blocks

Methods

Foo = Class.new do
  def instance_method; end      # → Foo#instance_method
  def self.class_method; end    # → Foo::<Foo>#class_method (singleton method)
end

• Instance methods: owned by the dynamic class (Foo)
• Singleton methods (def self.x): owned by the class's singleton class (Foo::<Foo>)
• self inside the block resolves to the constant name being assigned

Instance Variables

# Case 1: Named Class.new/Module.new
# @x owned by Foo class
Foo = Class.new do
  @x = 1
end

# Case 2: Anonymous Class.new/Module.new
# @x belongs to anonymous class, no declaration created
Class.new do
  @x = 1
end

# Case 3: Non-namespace DSL (e.g., some_gem's DSL)
# @x falls back to enclosing class Outer
class Outer
  Bar.new do
    @x = 1
  end
end

Constants

Constants inside Class.new/Module.new blocks follow lexical scope, not the dynamic class:

# CONST goes to enclosing lexical scope, NOT Foo
Foo = Class.new do
  CONST = 1           # → ::CONST (top-level)
end

# Inside a lexical class, constants go to that class
class Outer
  Bar = Class.new do
    CONST = 1         # → Outer::CONST
  end
end

Singleton Classes

Foo = Class.new do
  class << self
    def class_method; end
  end
end

Indexing:
  • SingletonClassDefinition created with lexical_nesting_id = DSL
  • parent_for_singleton_class() recognizes DSL assigned to constant

Resolution:
  • DSL processed → DynamicClass(Foo) created
  • Singleton class resolves owner via get_dsl_namespace_declaration()
  • Method declared in Foo::<Foo>

4. Implementation Details

DSL Indexing

When the indexer encounters a .new call that matches a DSL target:

┌───────────────────────────────────────────────────────────────────────────────┐
│  Ruby: Foo = Class.new(Parent) { include M; def bar; end }                    │
│                                                                               │
│  RubyIndexer:                                                                 │
│  1. Encounters constant assignment `Foo = ...`                                │
│  2. Sees RHS is a call to `.new` (a DSL target method)                        │
│  3. Creates ConstantDefinition for `Foo`                                      │
│  4. Creates DslDefinition:                                                    │
│     - receiver: `Class` (as NameId)                                           │
│     - method_name: "new"                                                      │
│     - arguments: [Parent]                                                     │
│     - assigned_to: ConstantDefinition.id                                      │
│  5. Pushes Nesting::Dsl onto nesting stack                                    │
│  6. Visits block → members get lexical_nesting_id = DSL                       │
│  7. Pops DSL from nesting stack                                               │
└───────────────────────────────────────────────────────────────────────────────┘

Nesting Stack Types:
┌───────────────────────────────┬─────────────────────────────────────────────┐
│  Nesting::LexicalScope(id)    │  class/module keywords - Ruby lexical scope │
│  Nesting::Method(id)          │  method definitions - owns instance vars    │
│  Nesting::Dsl(id)             │  DSL blocks - owns members, no lexical scope│
└───────────────────────────────┴─────────────────────────────────────────────┘

Resolution Loop

Resolution processes definitions, references, ancestors, and DSLs in a unified loop:

                                    ┌─────────────────────────────────────────────────────┐
                                    │              OUTER LOOP: loop { ... }               │
                                    │  Continues until: !made_progress || queue.is_empty()│
                                    └─────────────────────────────┬───────────────────────┘
                                                                  │
                                                                  ▼
┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│  START OF PASS                                                                                                  │
│  made_progress = false                                                                                          │
│  pass_length = queue.len()  ◄─── snapshot length (retries go to back, next pass)                                │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
                                                                  │
                                                                  ▼
┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│  INNER LOOP: for _ in 0..pass_length { pop_front and process }                                                  │
│                                                                                                                 │
│    ┌────────────────────────┬────────────────────────┬────────────────────────┬────────────────────────┐        │
│    │ Unit::Definition       │ Unit::ConstantRef      │ Unit::Dsl              │ Unit::Ancestors        │        │
│    └───────────┬────────────┴───────────┬────────────┴───────────┬────────────┴───────────┬────────────┘        │
│                │                        │                        │                        │                     │
│                ▼                        ▼                        ▼                        ▼                     │
│    ┌────────────────────────┐ ┌────────────────────────┐ ┌────────────────────────┐ ┌────────────────────────┐  │
│    │ handle_definition_unit │ │ handle_reference_unit  │ │ handle_dsl_unit        │ │ handle_ancestor_unit   │  │
│    │                        │ │                        │ │                        │ │                        │  │
│    │ • maybe_handle_        │ │ • resolve_constant_    │ │ • resolve receiver     │ │ • linearize ancestors  │  │
│    │   constant_inside_dsl  │ │   internal             │ │ • match processor      │ │                        │  │
│    │ • handle_constant_     │ │                        │ │ • call handler         │ │                        │  │
│    │   declaration          │ │                        │ │                        │ │                        │  │
│    └───────────┬────────────┘ └───────────┬────────────┘ └───────────┬────────────┘ └───────────┬────────────┘  │
│                │                          │                          │                          │               │
│                └──────────────────────────┴─────────────┬────────────┴──────────────────────────┘               │
│                                                         ▼                                                       │
│    ┌────────────────────────────────────────────────────────────────────────────────────────────────────────┐   │
│    │  OUTCOMES                                                                                              │   │
│    │  Resolved ─────► made_progress = true, may enqueue Unit::Ancestors                                     │   │
│    │  Retry ────────► push_back(unit), processed in NEXT pass                                               │   │
│    │  DSL matched ──► creates DynamicClass/Module, enqueues Unit::Definition                                │   │
│    └────────────────────────────────────────────────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
                                                                  │
                                                                  ▼
┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│  END OF PASS: if !made_progress || queue.is_empty() → EXIT                                                      │
│               else → CONTINUE (retries might succeed now)                                                       │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

Example: Processing Foo = Class.new { def bar; end }

Pass 1: Queue = [Def(Foo/Const), Dsl(Class.new)]
  • Def(Foo) → resolves, creates ConstantDeclaration
  • Dsl → receiver "Class" not resolved → Retry
  made_progress = true → continue

Pass 2: Queue = [Dsl(Class.new)]
  • Dsl → receiver resolved to CLASS_ID
       → class_new_matches() = true
       → handle_class_new() creates DynamicClassDefinition
       → Removes ConstantDefinition, enqueues Unit::Definition(DynamicClass)
  made_progress = true → continue

Pass 3: Queue = [Def(DynamicClass)]
  • DynamicClass → creates ClassDeclaration(Foo)
  made_progress = true → continue

... (method resolution, ancestors) ...

Pass N: Queue = [] → EXIT

Key Functions Reference

Function	Location	Purpose
try_index_dsl_call	ruby_indexer.rs	Index DSL call as DslDefinition
handle_dsl_unit	resolution.rs	Process DSL, match processors
handle_class_new	dsl_processors.rs	Create DynamicClassDefinition
handle_module_new	dsl_processors.rs	Create DynamicModuleDefinition
extract_dsl_context	dsl_processors.rs	Get name, comments, flags from DSL+Constant
is_parent_dsl_processed	dsl_processors.rs	Check if parent DSL is processed
get_dsl_namespace_declaration	resolution.rs	Get declaration for processed DSL
maybe_handle_constant_inside_dsl	resolution.rs	Handle constants nested in DSL
resolve_lexical_owner	resolution.rs	Resolve owner, handles DSL nesting

Definition Type Summary

Definition Type	When Created	Declaration Created?
DslDefinition	Indexing (for any .new call)	No (captured for processing)
ConstantDefinition	Indexing (Foo = ...)	Removed if DSL matches
DynamicClassDefinition	Resolution (Class.new matches)	Yes if named, No if anonymous
DynamicModuleDefinition	Resolution (Module.new matches)	Yes if named, No if anonymous

Benchmark Diff

Benchmark diff against Core compared to main

                                      implement-497          main              diff                                                    
  ─────────────────────────────────────────────────────────────────────────────────────
  QUERY STATISTICS                                                                                                                     
    Total declarations:                   858,592         862,281           -3,689
    With documentation:                   145,908         146,000              -92
    Without documentation:                712,684         716,281           -3,597
    Multi-definition names:                16,654          16,716              -62

  DECLARATION BREAKDOWN
    Method                                409,543         410,329             -786
    InstanceVariable                      190,112         190,227             -115
    Class                                 107,185         107,981             -796
    SingletonClass                         69,310          70,175             -865
    Constant                               64,382          65,468           -1,086
    Module                                 13,881          13,924              -43
    ConstantAlias                           3,746           3,744               +2
    ClassVariable                             429             429                0
    GlobalVariable                              4               4                0

  DEFINITION BREAKDOWN
    Method                                359,263         360,083             -820
    InstanceVariable                      214,997         215,139             -142
    Module                                197,167         197,217              -50
    Class                                 110,793         112,063           -1,270
    Constant                               64,394          65,478           -1,084
    AttrReader                             36,851          36,902              -51
    SingletonClass                         15,982          15,993              -11
    AliasMethod                             7,165           7,166               -1
    AttrAccessor                            6,411           6,425              -14
    ConstantAlias                           3,746           3,744               +2
    ClassVariable                             856             856                0
    DynamicClass                              469             N/A             +469  ← NEW
    AttrWriter                                418             419               -1
    GlobalVariable                             36              36                0
    DynamicModule                               1             N/A               +1  ← NEW

  TIMING
    Listing                                0.92s           0.90s           +0.02s
    Indexing                              11.73s          10.99s           +0.74s
    Resolution                            21.00s          65.84s          -44.84s  ← 68% FASTER
    Querying                               1.33s           1.09s           +0.24s
    Total                                 34.97s          78.81s          -43.84s  ← 56% FASTER

  MEMORY
    Maximum RSS                         3,510 MB        2,885 MB          +625 MB  (+22%)

  SUMMARY
    Indexed files                         93,096          93,096                0
    Total definitions                  1,021,363       1,021,561             -198
    Total URIs                            93,096          93,096                0

[st0012]

Currently we eagerly retrieve nesting's name id as the receiver's name id:

class Foo
  def self.bar; end # uses `NameId(Foo)`
end

But this doesn't work in anonymous class/module's case with dsl processing:

module Namespace
  # Foo may or may not be a class, depending on whether Namespace defines a Class class or not
  # So at indexing we only know this is a DslDefinition that's assigned to a constant Foo
  Foo = Class.new
    def self.bar; end # self can't be resolved to a NameId just yet. we need to associate it with the DslDefinition directly
  end
end

[st0012]

This allows us to index Class.new(Foo) and have the Foo reference resolved for DSL processing.

[st0012]

Ideally, DSL definition should know nothing about what it's assigned to. But having this field makes Class.new and Module.new's processing much more efficient.

[vinistock]

Some other examples worth testing:

module Foo
  Bar = Class.new do
    def Foo.my_method; end
  end
end

module Foo
  module Bar; end
end

module Baz
  include Foo

  Bar::Qux = Class.new do
  end
end

Foo = Class.new do
  attr_reader :bar
end

module Foo; end

Bar = Module.new do
  extend Foo
end

Foo = Class.new do
end

class Bar < Foo
end

Foo = Module.new do
end

class Bar
  prepend Foo
end