Hitting usage limits too fast

[ezyang]

Something has changed about the rate limit algorithm where lots of tool calls are causing us to hit rate limit faster than we used to. I need to check my logs to confirm that this is indeed what is happening. Not sure if there is anything we can do.

[bobrenjc93]

FWIW I've noticed that the String replace tool fails quite often, wasting lots of tokens:

{
  `path`: `/Users/bobren/projects/snapfill/main.js`,
  `chat_id`: `7-feat-generate-meaningful-form-field-placeholders`,
  `subtool`: `EditFile`,
  `new_string`: `    // First, collect all fields to analyze patterns
    if (debugFieldInfo) {
      console.log('------- FORM FIELDS ANALYSIS -------');
      console.log(`Total fields: ${fields.length}`);
      
      // Group fields by type
      const fieldsByType = {};
      for (const field of fields) {
        const type = field.constructor.name;
        if (!fieldsByType[type]) {
          fieldsByType[type] = [];
        }
        fieldsByType[type].push(field.getName());
      }
      
      // Log field types and examples
      console.log('Field types:');
      for (const type in fieldsByType) {
        console.log(`  ${type}: ${fieldsByType[type].length} fields`);
        console.log(`  Examples: ${fieldsByType[type].slice(0, 5).join(', ')}${fieldsByType[type].length > 5 ? '...' : ''}`);
      }
      
      // Look for sequential numeric patterns in field names
      const numericPatterns = fields.map(f => f.getName()).filter(name => /^[a-zA-Z]+\\d+$/.test(name));
      if (numericPatterns.length > 0) {
        console.log('Detected sequential field pattern:');
        const sortedPatterns = numericPatterns.sort((a, b) => {
          const numA = parseInt(a.replace(/[^\\d]/g, ''));
          const numB = parseInt(b.replace(/[^\\d]/g, ''));
          return numA - numB;
        });
        console.log(`  Sorted sequence: ${sortedPatterns.slice(0, 10).join(', ')}${sortedPatterns.length > 10 ? '...' : ''}`);
      }
      
      // Extract potential field labels based on annotations
      console.log('Potential field labels from PDF:');
      
      // Collect all fields and their positions
      const fieldPositions = [];
      for (const field of fields) {
        try {
          const rect = field.acroField?.dict?.lookup('Rect');
          if (rect && rect.array) {
            fieldPositions.push({
              name: field.getName(),
              rect: rect.array,
              type: field.constructor.name
            });
          }
        } catch (err) {
          // Ignore errors
        }
      }
      
      // Collect all annotations that might be labels
      const pages = pdfDoc.getPages();
      console.log(`PDF has ${pages.length} pages`);
      
      // Extract page level information
      for (let pageIndex = 0; pageIndex < pages.length; pageIndex++) {
        const page = pages[pageIndex];
        console.log(`Page ${pageIndex + 1} - Size: [${page.getWidth().toFixed(2)}, ${page.getHeight().toFixed(2)}]`);
        
        // Get all fields on this page
        const pageFields = fieldPositions.filter(field => {
          // This is a simplistic approach since we can't directly determine which page a field is on
          // In a real implementation, we'd use more sophisticated methods
          return true; // Assume all fields are on all pages for now
        });
        
        if (pageFields.length > 0) {
          console.log(`  Fields on page ${pageIndex + 1}: ${pageFields.map(f => f.name).join(', ')}`);
        }
      }
      
      // Print out relationships between fields
      if (fieldPositions.length > 0) {
        console.log('Field positions and potential relationships:');
        
        // Sort fields by vertical position (top to bottom) to simulate reading order
        const sortedFields = [...fieldPositions].sort((a, b) => {
          // Rect format is typically [left, bottom, right, top]
          // Using the bottom value for vertical sorting
          return b.rect[3] - a.rect[3]; // Descending by top position
        });
        
        // Print sorted fields
        sortedFields.forEach((field, index) => {
          console.log(`  ${index + 1}. ${field.name} (${field.type}) - Position: [${field.rect.join(', ')}]`);
          
          // Look for nearby fields that might be related
          for (const otherField of fieldPositions) {
            if (field.name !== otherField.name) {
              // Check if fields are close to each other horizontally or vertically
              // This is a simplistic approach - a real implementation would have better spatial analysis
              const horizDist = Math.min(
                Math.abs(field.rect[0] - otherField.rect[0]), // left edges
                Math.abs(field.rect[2] - otherField.rect[2])  // right edges
              );
              
              const vertDist = Math.min(
                Math.abs(field.rect[1] - otherField.rect[1]), // bottom edges
                Math.abs(field.rect[3] - otherField.rect[3])  // top edges
              );
              
              // If fields are close to each other, they might be related
              const proximityThreshold = 50; // Arbitrary threshold
              if (horizDist < proximityThreshold || vertDist < proximityThreshold) {
                console.log(`    Possible relationship with: ${otherField.name} (Horizontal distance: ${horizDist.toFixed(2)}, Vertical distance: ${vertDist.toFixed(2)})`);
              }
            }
          }
        });
      }
      
      console.log('-----------------------------------');
    }`,
  `old_string`: `    // First, collect all fields to analyze patterns
    if (debugFieldInfo) {
      console.log('------- FORM FIELDS ANALYSIS -------');
      console.log(`Total fields: ${fields.length}`);
      
      // Group fields by type
      const fieldsByType = {};
      for (const field of fields) {
        const type = field.constructor.name;
        if (!fieldsByType[type]) {
          fieldsByType[type] = [];
        }
        fieldsByType[type].push(field.getName());
      }
      
      // Log field types and examples
      console.log('Field types:');
      for (const type in fieldsByType) {
        console.log(`  ${type}: ${fieldsByType[type].length} fields`);
        console.log(`  Examples: ${fieldsByType[type].slice(0, 5).join(', ')}${fieldsByType[type].length > 5 ? '...' : ''}`);
      }
      
      // Look for sequential numeric patterns in field names
      const numericPatterns = fields.map(f => f.getName()).filter(name => /^[a-zA-Z]+\\d+$/.test(name));
      if (numericPatterns.length > 0) {
        console.log('Detected sequential field pattern:');
        const sortedPatterns = numericPatterns.sort((a, b) => {
          const numA = parseInt(a.replace(/[^\\d]/g, ''));
          const numB = parseInt(b.replace(/[^\\d]/g, ''));
          return numA - numB;
        });
        console.log(`  Sorted sequence: ${sortedPatterns.slice(0, 10).join(', ')}${sortedPatterns.length > 10 ? '...' : ''}`);
      }
      
      console.log('-----------------------------------');
    }`,
  `description`: `Add field position analysis to identify potential labels`
}
Error executing tool codemcp: String to replace not found in file.

and in the same session

{
  `path`: `/Users/bobren/projects/snapfill/main.js`,
  `chat_id`: `7-feat-generate-meaningful-form-field-placeholders`,
  `subtool`: `EditFile`,
  `new_string`: `      // Look for sequential numeric patterns in field names
      const numericPatterns = fields.map(f => f.getName()).filter(name => /^[a-zA-Z]+\\d+$/.test(name));
      if (numericPatterns.length > 0) {
        console.log('Detected sequential field pattern:');
        const sortedPatterns = numericPatterns.sort((a, b) => {
          const numA = parseInt(a.replace(/[^\\d]/g, ''));
          const numB = parseInt(b.replace(/[^\\d]/g, ''));
          return numA - numB;
        });
        console.log(`  Sorted sequence: ${sortedPatterns.slice(0, 10).join(', ')}${sortedPatterns.length > 10 ? '...' : ''}`);
      }
      
      // Extract potential field labels based on annotations
      console.log('Potential field labels from PDF:');
      
      // Collect all fields and their positions
      const fieldPositions = [];
      for (const field of fields) {
        try {
          const rect = field.acroField?.dict?.lookup('Rect');
          if (rect && rect.array) {
            fieldPositions.push({
              name: field.getName(),
              rect: rect.array,
              type: field.constructor.name
            });
          }
        } catch (err) {
          // Ignore errors
        }
      }
      
      // Print out fields sorted by position (approximate reading order)
      if (fieldPositions.length > 0) {
        console.log('Field positions and potential relationships:');
        
        // Sort fields by vertical position (top to bottom) to simulate reading order
        const sortedFields = [...fieldPositions].sort((a, b) => {
          // Rect format is typically [left, bottom, right, top]
          // Using the top value for vertical sorting
          return b.rect[3] - a.rect[3]; // Descending by top position (PDF coordinates)
        });
        
        // Print sorted fields with their positions
        sortedFields.forEach((field, index) => {
          console.log(`  ${index + 1}. ${field.name} (${field.type}) - Position: [${field.rect.join(', ')}]`);
          
          // Look for nearby fields that might be related
          const nearbyFields = fieldPositions.filter(otherField => {
            if (field.name === otherField.name) return false;
            
            // Check vertical proximity - fields on same line or just above
            const verticallyClose = Math.abs(field.rect[3] - otherField.rect[3]) < 30;
            
            // Check horizontal proximity - fields to the left might be labels
            const horizontallyBefore = otherField.rect[2] < field.rect[0];
            
            return verticallyClose && horizontallyBefore;
          });
          
          // Log potential label fields
          if (nearbyFields.length > 0) {
            console.log(`    Potential labels: ${nearbyFields.map(f => f.name).join(', ')}`);
          }
        });
      }

      console.log('-----------------------------------');`,
  `old_string`: `      // Look for sequential numeric patterns in field names
      const numericPatterns = fields.map(f => f.getName()).filter(name => /^[a-zA-Z]+\\d+$/.test(name));
      if (numericPatterns.length > 0) {
        console.log('Detected sequential field pattern:');
        const sortedPatterns = numericPatterns.sort((a, b) => {
          const numA = parseInt(a.replace(/[^\\d]/g, ''));
          const numB = parseInt(b.replace(/[^\\d]/g, ''));
          return numA - numB;
        });
        console.log(`  Sorted sequence: ${sortedPatterns.slice(0, 10).join(', ')}${sortedPatterns.length > 10 ? '...' : ''}`);
      }
      
      console.log('-----------------------------------');`,
  `description`: `Add field position analysis to identify potential labels`
}
Error executing tool codemcp: String to replace not found in file.

[ezyang]

I fixed one major bug re this on main

[bobrenjc93]

What's the best way to pull main? Does it happen automatically? Do I just delete the tool and re-add it?

[Sattva-108]

What's the best way to pull main? Does it happen automatically? Do I just delete the tool and re-add it?

Afaik you just change prod to main, if you like to live dangerously! 🤠🖲️💥

Before

uvx --from git+https://github.com/ezyang/codemcp@prod codemcp serve

AFTER

uvx --from git+https://github.com/ezyang/codemcp@main codemcp serve

[Sattva-108]

I could confirm, the MAIN vs PROD. Is a huge difference atm.

With PROD I usually hit limit in 1.5-2hr of careful work.
If not careful, especially if Thinking Mode + Opus (not 100% sure) is on, I could hit limit in 1-2 messages (15-30 minutes).

And in MAIN I work 4-5 hours with no problem.

I would appreciate if you guys share your experience 🥰

[ezyang]

I will cut a release soon