Skip to content

shivamgupta1112/Data-Extraction

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

MongoDB to Excel Data Exporter

A Node.js script to fetch user data from MongoDB and export it to Excel format.

Features

  • 🔌 Connect to MongoDB (local or cloud)
  • 📊 Fetch data from any collection
  • 📋 List all available collections
  • 📈 Export data to Excel with formatting
  • 🎨 Auto-sized columns and styled headers
  • 📁 Automatic file naming with timestamps
  • ⚡ Support for large datasets
  • 🔒 Secure connection handling

Prerequisites

  • Node.js (version 14 or higher)
  • MongoDB database (local or cloud)
  • MongoDB connection string

Installation

  1. Clone or download this project

  2. Install dependencies:

    npm install
  3. Configure your MongoDB connection:

    • Copy env.example to .env
    • Edit .env and add your MongoDB connection string:
    MONGODB_URL=mongodb://username:password@host:port/database_name

Usage

Basic Usage

Run the script:

npm start

The script will:

  1. Connect to your MongoDB database
  2. List all available collections
  3. Fetch data from the first collection (or you can modify to select specific)
  4. Export the data to an Excel file in the exports/ folder

Advanced Usage

You can modify the index.js file to:

  • Select a specific collection
  • Add data filtering
  • Limit the number of records
  • Customize the Excel filename

Example modifications:

// Select specific collection
const selectedCollection = 'users'; // Change this to your collection name

// Limit records (add to fetchData call)
const data = await exporter.fetchData(selectedCollection, 1000); // Limit to 1000 records

// Custom filename
const excelPath = await exporter.exportToExcel(data, 'my_custom_filename.xlsx');

MongoDB Connection String Examples

Local MongoDB

mongodb://localhost:27017/your_database

MongoDB Atlas (Cloud)

mongodb+srv://username:password@cluster.mongodb.net/database_name

MongoDB with Authentication

mongodb://username:password@host:port/database_name?authSource=admin

Output

The script creates an exports/ folder and saves Excel files with:

  • Timestamped filenames (e.g., user_data_export_2024-01-15.xlsx)
  • Formatted headers with gray background
  • Auto-sized columns
  • All data fields from your MongoDB documents

Error Handling

The script includes comprehensive error handling for:

  • Connection failures
  • Invalid MongoDB URLs
  • Missing collections
  • Empty datasets
  • File write errors

Troubleshooting

Connection Issues

  • Verify your MongoDB connection string
  • Check if MongoDB is running
  • Ensure network connectivity (for cloud databases)
  • Verify username/password credentials

Permission Issues

  • Ensure you have read access to the database
  • Check if the collection exists
  • Verify database name in the connection string

Memory Issues

  • For large datasets, consider adding limits
  • Monitor system memory usage
  • Use pagination for very large collections

Customization

Adding Filters

// In the fetchData method, modify the query
let query = { status: 'active' }; // Add your filter conditions
const data = await this.collection.find(query, options).toArray();

Custom Excel Formatting

// In the exportToExcel method, add custom styling
worksheet.getRow(1).font = { bold: true, size: 14 };
worksheet.getRow(1).fill = {
    type: 'pattern',
    pattern: 'solid',
    fgColor: { argb: 'FF4F81BD' }
};

Dependencies

  • mongodb: MongoDB driver for Node.js
  • exceljs: Excel file generation library
  • dotenv: Environment variable management

License

ISC License

Support

For issues or questions:

  1. Check the error messages in the console
  2. Verify your MongoDB connection string
  3. Ensure all dependencies are installed
  4. Check the troubleshooting section above

About

Data Extraction is the customized script written in Node js, to extract the dataset from the MongoDB database in single csv file using one terminal command "npm start"

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors