Skip to content

A powerful and efficient URL resolution web app service that tracks campaign parameters, handles complex redirects, and validates marketing URLs.

Notifications You must be signed in to change notification settings

thequick10/TraceToEnd

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

77 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

🎯 TraceToEnd

Node.js Version Express Version PRs Welcome

A powerful and efficient URL resolution service that tracks campaign parameters, handles complex redirects, and validates marketing URLs.

Key Features β€’ Installation β€’ API Endpoints Usage β€’ Deployment β€’ Dependencies β€’ Frontend Usage β€’ Sample JSON Response β€’ Import/Export Capabilities

πŸš€ Key Features

  • 🌐 Browser Emulation: Uses Puppeteer for JavaScript-heavy redirects
  • 🌏 Region Based URL Resolutions: Region-based URL resolution with proxy support for multiple geographic zones
  • πŸ–‡ Multiple Endpoint Support: Single and multiple region URL resolution endpoints
  • πŸ“Š Data Management: Campaign management frontend with URL tracking, tagging, and status display
  • πŸ“₯ File Import/Export Support: Import and export campaigns via CSV and XLSX file formats
  • πŸ“ Location Detection: Auto-detection of user location and manual country selection
  • πŸ” Security: Rate limiting, security headers, and basic authentication for enhanced security
  • πŸ“Š API Usage Analytics: Analytics page for usage tracking and monitoring
  • πŸ”— URL Resolution Stats URL Resolutions stat page to track failed and success url in respect of regions
  • βœ… Health Check Support: System health and region listing API endpoints

πŸ“ Import/Export Capabilities

  • πŸ“₯ Import Support:
    • CSV file import
    • XLSX file import
    • Drag & drop file support
    • Smart column detection
    • Batch processing
  • πŸ“€ Export Options:
    • Export to CSV
    • Complete campaign history
    • Formatted date and time

🌍 Location Detection

  • πŸ” Auto-Detection: Automatic country detection
  • πŸ”„ Multiple Services: Fallback to multiple geolocation services
  • πŸš₯ Status Indicators: Visual feedback for detection process
  • πŸ”„ Manual Refresh: Option to refresh location detection

πŸ” Search & Filter

  • πŸ”Ž Real-time Search: Instant search across all fields
  • πŸ“… Date Range Filter:
    • Built-in date range picker
    • Clear filter option
    • Support for custom date formats
  • πŸ“Š Sorting Options:
    • Sort by newest/oldest
    • Persistent sorting preferences
    • Sort by File Import Order

πŸ”„ URL Refresh Features

  • πŸ”„ Individual Refresh: Refresh single URLs
  • πŸ“Š Batch Refresh: Refresh all URLs with progress tracking
  • ⚠️ Error Handling:
    • Automatic retry mechanism
    • Error status indicators
    • Restore previous URL on failure
  • πŸ“ˆ Progress Tracking: Visual progress indicators

πŸ§‘β€πŸ’» User Agent Support

  • Desktop: Emulates a desktop browser user agent.
  • Mobile: Emulates a mobile browser user agent.
  • Random (Rotating): Randomly selects a desktop or mobile user agent for each request. This is the default option and helps simulate diverse real-world traffic.

You can select the user agent type from the frontend dropdown. The selected type is sent to the backend and used for all URL resolutions and analytics.

πŸ’« Additional Features

  • πŸ“‹ Clipboard Support: One-click URL copying
  • ✏️ Inline Editing: Edit campaign URLs and tags directly
  • πŸ—‘οΈ Data Management: Delete individual or all campaigns
  • πŸ“± Responsive Design: Works on desktop and mobile
  • πŸ”” Notifications: Beautiful toast notifications for all actions
  • πŸ–₯ User Agent Selection: Choose between Desktop, Mobile, or Random (rotating) user agents for each request. The Random option will select a new user agent for every request, simulating real-world browsing patterns.
  • ⏱️ Timing Statistics Dashboard: Dedicated dashboard to track and analyze the time taken to resolve URLs, with filtering, sorting, and CSV export capabilities.

πŸ› οΈ Installation

  1. Clone the repository
git clone https://github.com/thequick10/TraceToEnd.git
cd TraceToEnd
  1. Install dependencies
npm install
  1. DotENV Config
Configure your dotenv file in your local server and add all variable values in dotenv

For Instance:

BRIGHTDATA_API_KEY=<YOUR_BRIGHTDATA_API_KEY>
BRIGHTDATA_US_PROXY=brd-customer-<CUSTOMER_ID>-zone-<YOUR_ZONE_ID>-country-<COUNTRY>

Add all variables and their value just like above

Make sure you use 2-letter country code like for united states use only - US
  1. Start the server
# Development mode
npm run dev

# Production mode
npm start

πŸ’» API Endpoints Usage

The server provides multiple endpoints for URL resolution with different levels of depth and speed:

  • GET /resolve?url=<URL>&region=<REGION_CODE>
    Resolve a single URL for a specified region.

  • GET /resolve?url=<URL>&region=<REGION_CODE>&uaType=desktop|mobile Resolve a single URL for a specified region with a specific user agent type.

  • GET /resolve-multiple?url=<URL>&regions=us,ca,ae
    Resolve a URL across multiple regions simultaneously.

  • GET /resolve-multiple?url=<URL>&regions=us,ca,ae&uaTyp=desktop|mobile Resolve a URL across multiple regions simultaneously with a specific user agent type.

  • GET /zone-usage?from=YYYY-MM-DD&to=YYYY-MM-DD
    Retrieve BrightData API usage statistics for the configured zone.

  • GET /regions
    List all supported proxy regions.

  • GET /system-info
    Get system health and resource usage information.

  • GET /ip
    Returns the client IP address.

Campaign link generator

  • Dashboard: Access the dashboard at https://your-campaign-url.com/ (replace with your actual campaign URL).
  • Add Campaigns: Enter campaign URL, tags/notes, and select country, then click "Add Campaign".
  • Refresh URLs: Refresh individual or all campaign URLs to get updated final resolved URLs.
  • Import/Export: Import campaigns from CSV or XLSX files; export current campaigns to CSV.
  • Filtering and Sorting: Search campaigns, filter by date range, and sort by newest, oldest, or import order.
  • Delete Campaigns: Delete individual or all campaigns.
  • Edit Campaigns: Edit campaign details, including URL, tags, and notes.
  • User Agent Selection: Select user agent type (desktop or mobile) for each campaign.

API Request Analytics

  • Access the dashboard at /analytics/stats.html
  • View campaign statistics, including date, total requests and bandwidth etc.
  • Filter by date range & search requests by date.
  • Export data to CSV.

URL Resolution Stats

  • Access the dashboard at /resolution-stats/resolutions.html
  • View URL resolution statistics, including total requests, successful and failed requests, and bandwidth etc.
  • Filter by date range & search requests by date.
  • Export data to CSV.
  • View the stats in a table format.
  • View stats by regional performance

⏱️ Timing Statistics Dashboard

  • Access the dashboard at /timing-stat/timing-stat.html.
  • View daily statistics for total time, average time per URL, and request counts.
  • Filter by date range, sort results, and export timing data as CSV.
  • The dashboard is fully responsive and mobile-friendly.

Sample Response

{
  "originalUrl": "https://your-campaign-url.com",
  "finalUrl": "https://final-destination.com?clickid=123&utm_source=campaign",
  "method": "browser-api",
  "hasClickId": true,
  "hasUtmSource": true,
  "hasClickRef": false
  //more paramenters
}

πŸš€ Deployment

Prerequisites

  • Node.js >= 14.0.0
  • NPM or Yarn
  • 512MB RAM minimum

Environment Variables

PORT=8080  # Server port (optional)

Deployment Platforms

  • 🌐 Any Node.js compatible hosting

Dependencies

Key npm packages used:

  • express
  • cors
  • dotenv
  • helmet
  • express-basic-auth
  • express-rate-limit
  • puppeteer-core
  • https
  • os
  • path
  • url

License and Author

Author: Rupesh Shah
License: MIT (or specify your license) This project is licensed under the MIT License - see the LICENSE file for details.

πŸ™ Acknowledgments


About

A powerful and efficient URL resolution web app service that tracks campaign parameters, handles complex redirects, and validates marketing URLs.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published