MongoDB Database Setup Guide

This guide helps you set up and import multiple MongoDB databases for the Great AI Hackathon project. The setup includes 5 separate databases with Malaysian government service data.

Prerequisites

MongoDB installed and running
MongoDB command line tools (mongoimport) available
Database connection access

Database Structure

databases/
├── accounts.json        # Government service accounts database
├── idcards.json         # Malaysian identity cards database
├── licenses.json        # Malaysian driving license database
├── tnb-bills.json       # TNB electricity bills database  
└── transactions.json    # Payment transactions database

chats/                   # Chat messages database (user-specific collections)
├── (default)            # Default collection for general chat
└── (userId)             # Individual user chat collections (e.g., YYMMDD-XX-XXXX)

Data Format

All JSON files contain structured data ready for MongoDB import. Some files use array format while others contain individual documents.

Database Import Commands

1. Import Accounts Collections

mongoimport --db databases --collection accounts --file databases/accounts.json --jsonArray

2. Import ID Cards Collections

mongoimport --db databases --collection idcards --file databases/idcards.json --jsonArray

3. Import Licenses Collections

mongoimport --db databases --collection licenses --file databases/licenses.json --jsonArray

4. Import TNB Bills Collections

mongoimport --db databases --collection tnb-bills --file databases/tnb-bills.json --jsonArray

5. Import Transactions Collections

mongoimport --db databases --collection transactions --file databases/transactions.json --jsonArray

6. Chats Database Setup

The chats database is special - it contains user-specific collections:

# Create the chats database (will be created automatically when first collection is added)
# Each user will have their own collection named by userId
# Collections: chats.userId (e.g., chats.YYMMDD-XX-XXXX) and chats.(default)
# Example collections: "YYMMDD-XX-XXXX", "(default)"

mongoimport -db chats --collection (default) --file sample/chat-empty.json

Environment Variables (.env file)

Create a .env file based on .env.example to store your MongoDB connection details:

ATLAS_URI=mongodb+srv://<username>:<password>@<cluster>.mongodb.net
ATLAS_DB_NAME=databases

Import All Databases (Batch Commands)

Windows Command Prompt

mongoimport --db databases --collection accounts --file databases\accounts.json --jsonArray
mongoimport --db databases --collection idcards --file databases\idcards.json --jsonArray
mongoimport --db databases --collection licenses --file databases\licenses.json --jsonArray
mongoimport --db databases --collection tnb-bills --file databases\tnb-bills.json --jsonArray
mongoimport --db databases --collection transactions --file databases\transactions.json --jsonArray
mongoimport -db chats --collection (default) --file sample/chat-empty.json

MongoDB Atlas (Cloud) - Using Environment Variable

# Set your Atlas URI as environment variable (Windows CMD)
SET ATLAS_URI=mongodb+srv://<username>:<password>@<cluster>.mongodb.net
SET ATLAS_DB_NAME=databases

# Import all databases using environment variable
mongoimport --uri "%ATLAS_URI%" --db "%ATLAS_DB_NAME%" --collection accounts --file databases\accounts.json --jsonArray
mongoimport --uri "%ATLAS_URI%" --db "%ATLAS_DB_NAME%" --collection idcards --file databases\idcards.json --jsonArray
mongoimport --uri "%ATLAS_URI%" --db "%ATLAS_DB_NAME%" --collection licenses --file databases\licenses.json --jsonArray
mongoimport --uri "%ATLAS_URI%" --db "%ATLAS_DB_NAME%" --collection tnb-bills --file databases\tnb-bills.json --jsonArray
mongoimport --uri "%ATLAS_URI%" --db "%ATLAS_DB_NAME%" --collection transactions --file databases\transactions.json --jsonArray
mongoimport --uri "%ATLAS_URI%" --db chats --collection (default) --file sample/chat-empty.json

MongoDB Atlas (Direct URI)

mongoimport --uri "mongodb+srv://username:password@cluster.mongodb.net" --db "%ATLAS_DB_NAME%" --collection accounts --file databases/accounts.json --jsonArray
mongoimport --uri "mongodb+srv://username:password@cluster.mongodb.net" --db "%ATLAS_DB_NAME%" --collection idcards --file databases/idcards.json --jsonArray
mongoimport --uri "mongodb+srv://username:password@cluster.mongodb.net" --db "%ATLAS_DB_NAME%" --collection licenses --file databases/licenses.json --jsonArray
mongoimport --uri "mongodb+srv://username:password@cluster.mongodb.net" --db "%ATLAS_DB_NAME%" --collection tnb-bills --file databases/tnb-bills.json --jsonArray
mongoimport --uri "mongodb+srv://username:password@cluster.mongodb.net" --db "%ATLAS_DB_NAME%" --collection transactions --file databases/transactions.json --jsonArray
mongoimport --uri "mongodb+srv://username:password@cluster.mongodb.net" --db chats --collection (default) --file sample/chat-empty.json

Use import-all.bat for Windows users

import-all

Use clean-all.bat for Windows users

clean-all

Create unique index

create-unique-index

Common Connection Parameters

Parameter	Description	Example
`--host`	MongoDB host and port	`localhost:27017`
`--uri`	Complete connection string	`mongodb://user:pass@host:port/db`
`--username`	Authentication username	`myuser`
`--password`	Authentication password	`mypassword`
`--authenticationDatabase`	Auth database	`admin`
`--ssl`	Enable SSL/TLS	`--ssl`
`--db`	Target database name	`accounts`
`--collection`	Target collection name	`accounts`
`--file`	Input JSON file path	`databases/accounts.json`
`--jsonArray`	Import JSON array format	`--jsonArray`

Quick Connection Test

# Test MongoDB connection
mongo --eval "db.runCommand('ping')"

# Test with authentication
mongo --username your_username --password your_password --authenticationDatabase admin --eval "db.runCommand('ping')"

# Test Atlas connection
mongo "mongodb+srv://username:password@cluster.mongodb.net" --eval "db.runCommand('ping')"

Database Relationships

Cross-Database Relationships

TNB Bills ↔ Transactions

tnb-bills.bill.akaun.no_invois matches transactions.metadata.tnbInvoiceId
tnb-bills.pembayaran.rujukan matches transactions._id
transactions.billplz.billId matches accounts.billplz_collection_id

Accounts ↔ Transactions

Government service accounts store beneficiary information
Transactions reference service types (LICENSE_RENEWAL, PAY_TNB_BILL)

ID Cards ↔ Licenses

idcards.userId matches licenses.userId

Licenses ↔ Transactions

License renewals generate transactions with serviceType "LICENSE_RENEWAL"
License numbers stored in transaction metadata

Chats Database

User-specific collections store chat history
Collections named by userId (e.g., YYMMDD-XX-XXXX)
Default collection contains conversation messages

Sample Database Queries

Accounts Database

// Connect to accounts database
use accounts

// Find all active government service accounts
db.accounts.find({"active": true})

// Find TNB service accounts
db.accounts.find({"service": "TNB"})

ID Cards Database

// Connect to idcards database
use idcards

// Find all valid ID cards
db.idcards.find({"status": "valid"})

// Find ID cards by userId
db.idcards.find({"userId": "YYMMDD-XX-XXXX"})

Licenses Database

// Connect to licenses database
use licenses

// Find valid licenses
db.licenses.find({"status": "valid"})

// Find licenses expiring soon
db.licenses.find({"valid_to": {$lt: "2026-01-01"}})

TNB Bills Database

// Connect to tnb-bills database
use tnb-bills

// Find paid bills
db['tnb-bills'].find({"status": "paid"})

// Find overdue bills
db['tnb-bills'].find({"status": "overdue"})

// Find residential tariff bills
db['tnb-bills'].find({"bill.tarif.jenis": "A: Kediaman"})

Transactions Database

// Connect to transactions database
use transactions

// Find all paid transactions
db.transactions.find({"status": "paid"})

// Find TNB bill payments
db.transactions.find({"serviceType": "PAY_TNB_BILL"})

// Find license renewals
db.transactions.find({"serviceType": "LICENSE_RENEWAL"})

Chats Database

// Connect to chats database
use chats

// List all user collections (user chat histories)
show collections

// Access specific user's chat history
db['YYMMDD-XX-XXXX'].find()

Data Validation

After import, verify database and collection counts:

// Check all databases
show dbs

// Verify each database
use accounts
db.accounts.countDocuments()

use idcards
db.idcards.countDocuments()

use licenses  
db.licenses.countDocuments()

use tnb-bills
db['tnb-bills'].countDocuments()      // Should return 5 (1 original + 4 new)

use transactions
db.transactions.countDocuments() // Should return 4 (2 original + 2 new)

use chats
show collections              // Will show user-specific collections

Database Schemas

Accounts Database

{
  agency: "String",               // JPJ, TNB, etc.
  billplz_collection_id: "String"
}

ID Cards Database

{
  full_name: "String",            // Full name in ID card
  userId: "String",               // Malaysian IC format, e.g., YYMMDD-XX-XXXX
  gender: "String",               // Male, Female, etc.
  address: "String",              // Address in ID card
}

Licenses Database

{
  full_name: "String",         // Full name in license, e.g. JOHN DOE
  userId: "String",            // Malaysian IC format, e.g., YYMMDD-XX-XXXX
  dob: "String",               // YYYY-MM-DD
  nationality: "String",       // MALAYSIA, etc.
  license_number: "String",    // Malaysian license format, e.g., 0107011 abc123Xy
  license_classes: ["String"], // D, DA, B2, etc.
  valid_from: "String",
  valid_to: "String",
  address: "String",
  status: "String"             // valid, expired, suspended
}

TNB Bills Database

{
  bill: {
    akaun: {
      no_akaun: "String",      // TNB account number
      no_kontrak: "String",    // Contract number
      deposit: Number,         // Deposit amount
      no_invois: "String",     // Invoice number
      name: "String",
      address: "String"
    },
    meta: {
      bil_semasa: Object,      // Current bill details
      butiran: Object,         // Bill breakdown
      bil_terdahulu: Object,   // Previous bill
      bayaran_akhir: Object    // Last payment
      jenis_bacaan: "String"   // Reading type
    },
    tarif: {
      tempoh: Object,          // Tariff period
      jenis: "String",         // A: Kediaman (Residential)
      faktor_prorata: Number,  // Tariff factor
      blok: [Object],          // Tariff blocks
      jumlah_kWh: Number,
      jumlah_amaun: Number
    },
    caj: Object,               // Charges breakdown
    meter: Object              // Meter readings
  },
  status: "String",            // paid, unpaid, overdue
  pembayaran: {                // Payment details (if paid)
    jumlah: Number,
    tarikh_bayar: "ISO Date",
    kaedah: "String",          // MyGovHub
    rujukan: "String"          // Transaction reference
  }
}

Transactions Database

{
  _id: "String",               // Unique transaction ID
  userId: "String",            // Malaysian IC format
  serviceType: "String",       // LICENSE_RENEWAL, PAY_TNB_BILL
  description: "String",
  amount: Number,              // Amount in MYR
  currency: "String",          // MYR
  status: "String",            // paid, pending, failed
  createdAt: "ISO Date",
  updatedAt: "ISO Date",
  billplz: {                   // Payment gateway details
    billId: "String",          // bill_xyz123 (TNB), bill_xyz789 (JPJ)
    url: "String",
    paidAt: "ISO Date",
    transactionId: "String",
    webhookPayload: Object
  },
  metadata: {
    sessionId: "String",
    // Service-specific metadata
    tnbAccountId: "String",    // For TNB bills
    tnbInvoiceId: "String",
    renewalYears: Number,      // For license renewals
    licenseNumber: "String"
  }
}

Chats Database

// Collection name: userId (e.g., "YYMMDD-XX-XXXX") or "default"
{
  sessionId: "String",         // Unique session identifier
  createdAt: "ISO Date",       // Session creation timestamp
  
  messages: [                  // Array of chat messages
    {
      messageId: "String",     // Unique message identifier
      message: "String",       // Message content
      timestamp: "ISO Date",   // Message timestamp
      type: "String",          // user, assistant
      intent: "String",        // Optional: renew_license, document_processing, etc.
      attachment: [            // Optional: file attachments
        {
          url: "String",       // File URL
          type: "String",      // MIME type (image/jpeg, etc.)
          name: "String"       // Original filename
        }
      ]
    }
  ],
  
  status: "String",            // active, completed, archived
  topic: "String",             // Main conversation topic
  
  context: {                   // User context information
    full_name: "String",       // User's full name
    userId: "String",          // Malaysian IC format (YYMMDD-XX-XXXX)
    gender: "String",          // LELAKI, PE#PUAN
    address: "String"          // User's address
  },
  
  ekyc: {                      // eKYC verification data (if applicable)
    tnb_account_no: ["String"], // Array of TNB account numbers (if any)
    iwk_account_no: ["String"]  // Array of IWK account numbers (if any)
  }
}

Troubleshooting

Common Issues

File Path Issues (Windows)
- Use backslashes \ for Windows paths
- Ensure you're in the correct directory
JSON Array Import
- Use --jsonArray flag for array-formatted files
- Each database file may have different formats
Database vs Collection Confusion
- This setup uses separate databases, not collections
- Each database contains its own collections
Chat Database Collections
- Chat collections are created dynamically per user
- Collection names follow userId format

Verification Commands

# Check MongoDB connection
mongo --eval "db.runCommand('ping')"

# List all databases
mongo --eval "show dbs"

# Check specific database collections
mongo accounts --eval "show collections"
mongo idcards --eval "show collections"
mongo licenses --eval "show collections" 
mongo tnb-bills --eval "show collections"
mongo transactions --eval "show collections"
mongo chats --eval "show collections"

Notes

Database Count: 5 separate databases
Data Type: Malaysian Government Services (TNB, JPJ, etc.)
Currency: Malaysian Ringgit (MYR)
Date Format: ISO 8601 where applicable
Special Database: Chats uses dynamic user-based collections
Payment Gateway: Billplz integration with specific bill IDs
Service Types: LICENSE_RENEWAL (RM30/year), PAY_TNB_BILL (variable amounts)

Project: Great AI Hackathon
Total Databases: 5 (accounts, licenses, tnb-bills, transactions, chats)
Data Source: Malaysian Government Services
Last Updated: October 2025

Name		Name	Last commit message	Last commit date
Latest commit History 2 Commits
.env.example		.env.example
.gitignore		.gitignore
README.md		README.md
clean-all.bat		clean-all.bat
create-unique-index.bat		create-unique-index.bat
deleteExpiredOtpVerifications-trigger.js		deleteExpiredOtpVerifications-trigger.js
import-all.bat		import-all.bat
markOverdueTnbBills-trigger.js		markOverdueTnbBills-trigger.js

MyGovHub-Goodbye-World/mygovhub-mongodb

Folders and files

Latest commit

History

Repository files navigation

MongoDB Database Setup Guide

Prerequisites

Database Structure

Data Format

Database Import Commands

1. Import Accounts Collections

2. Import ID Cards Collections

3. Import Licenses Collections

4. Import TNB Bills Collections

5. Import Transactions Collections

6. Chats Database Setup

Environment Variables (.env file)

Import All Databases (Batch Commands)

Windows Command Prompt

MongoDB Atlas (Cloud) - Using Environment Variable

MongoDB Atlas (Direct URI)

Use import-all.bat for Windows users

Use clean-all.bat for Windows users

Create unique index

Common Connection Parameters

Quick Connection Test

Database Relationships

Cross-Database Relationships

TNB Bills ↔ Transactions

Accounts ↔ Transactions

ID Cards ↔ Licenses

Licenses ↔ Transactions

Chats Database

Sample Database Queries

Accounts Database

ID Cards Database

Licenses Database

TNB Bills Database

Transactions Database

Chats Database

Data Validation

Database Schemas

Accounts Database

ID Cards Database

Licenses Database

TNB Bills Database

Transactions Database

Chats Database

Troubleshooting

Common Issues

Verification Commands

Notes

About

Topics

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Languages

Packages