Skip to content

Latest commit

 

History

History

google-cloud

README.md

RuVector Cloud Run GPU Deployment

High-performance vector database benchmarks and deployment on Google Cloud Run with GPU acceleration (NVIDIA L4).

Table of Contents

Overview

This example provides:

  • GPU-Accelerated Benchmarks: SIMD (AVX-512, AVX2, NEON) and CUDA optimized operations
  • Cloud Run Deployment: Scalable, serverless deployment with GPU support
  • Multiple Deployment Models:
    • Single-node benchmark service
    • Attention/GNN inference service
    • Raft consensus cluster (3+ nodes)
    • Primary-replica replication

Supported RuVector Capabilities

Capability Description Cloud Run Support
Core Vector Search HNSW indexing, k-NN search ✅ Full GPU
Attention Mechanisms Multi-head attention layers ✅ Full GPU
GNN Inference Graph neural network forward pass ✅ Full GPU
Raft Consensus Distributed consensus protocol ✅ Multi-service
Replication Primary-replica data replication ✅ Multi-service
Quantization INT8/PQ compression ✅ GPU optimized

Prerequisites

Required Tools

# Google Cloud CLI
curl https://sdk.cloud.google.com | bash
gcloud init

# Docker
# Install from: https://docs.docker.com/get-docker/

# Rust (for local development)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

GCP Setup

# Authenticate
gcloud auth login

# Set project
gcloud config set project YOUR_PROJECT_ID

# Enable required APIs
gcloud services enable \
    run.googleapis.com \
    containerregistry.googleapis.com \
    cloudbuild.googleapis.com \
    compute.googleapis.com

Quick Start

1. One-Command Deployment

cd examples/google-cloud

# Setup and deploy
./deploy.sh setup
./deploy.sh build Dockerfile.gpu latest
./deploy.sh push latest
./deploy.sh deploy latest true  # true = GPU enabled

# Run benchmark
./deploy.sh benchmark ruvector-benchmark quick

2. View Results

# Get service URL
gcloud run services describe ruvector-benchmark \
    --region=us-central1 \
    --format='value(status.url)'

# Test endpoints
curl $URL/health
curl $URL/info
curl -X POST $URL/benchmark/quick

Step-by-Step Tutorial

Step 1: Project Setup

# Clone the repository
git clone https://github.com/ruvnet/ruvector.git
cd ruvector/examples/google-cloud

# Set environment variables
export GCP_PROJECT_ID="your-project-id"
export GCP_REGION="us-central1"

# Run setup
./deploy.sh setup

Step 2: Build the Docker Image

Option A: Local Build (faster iteration)

# Build locally
./deploy.sh build Dockerfile.gpu latest

# Push to Container Registry
./deploy.sh push latest

Option B: Cloud Build (no local Docker required)

# Build in the cloud
./deploy.sh build-cloud Dockerfile.gpu latest

Step 3: Deploy to Cloud Run

Basic Deployment (with GPU)

./deploy.sh deploy latest true

Custom Configuration

# High-memory configuration for large vector sets
MEMORY=16Gi CPU=8 ./deploy.sh deploy latest true

# Scale settings
MIN_INSTANCES=1 MAX_INSTANCES=20 ./deploy.sh deploy latest true

Step 4: Run Benchmarks

# Quick benchmark (128d, 10k vectors)
./deploy.sh benchmark ruvector-benchmark quick

# Distance computation benchmark
./deploy.sh benchmark ruvector-benchmark distance

# HNSW index benchmark
./deploy.sh benchmark ruvector-benchmark hnsw

# Full benchmark suite
./deploy.sh benchmark ruvector-benchmark full

Step 5: View Results

# Get all results
./deploy.sh results ruvector-benchmark

# View logs
./deploy.sh logs ruvector-benchmark

# Check service status
./deploy.sh status

Deployment Options

1. Single-Node Benchmark Service

Best for: Development, testing, single-user benchmarks

./deploy.sh deploy latest true

2. Attention/GNN Service

Best for: Neural network inference, embedding generation

./deploy.sh deploy-attention latest

Features:

  • 16GB memory for large models
  • 3-layer GNN with 8 attention heads
  • Optimized for batch inference

3. Raft Consensus Cluster

Best for: High availability, consistent distributed state

# Deploy 3-node cluster
CLUSTER_SIZE=3 ./deploy.sh deploy-raft

# Deploy 5-node cluster for higher fault tolerance
CLUSTER_SIZE=5 ./deploy.sh deploy-raft

Architecture:

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Node 1    │◄───►│   Node 2    │◄───►│   Node 3    │
│  (Leader)   │     │  (Follower) │     │  (Follower) │
└─────────────┘     └─────────────┘     └─────────────┘
       │                  │                   │
       └──────────────────┴───────────────────┘
                    Raft Consensus

Configuration:

# Environment variables for Raft nodes
RUVECTOR_NODE_ID=0              # Node identifier (0, 1, 2, ...)
RUVECTOR_CLUSTER_SIZE=3         # Total cluster size
RUVECTOR_RAFT_ELECTION_TIMEOUT=150  # Election timeout (ms)
RUVECTOR_RAFT_HEARTBEAT_INTERVAL=50 # Heartbeat interval (ms)

4. Primary-Replica Replication

Best for: Read scaling, geographic distribution

# Deploy with 3 replicas
./deploy.sh deploy-replication 3

Architecture:

                    ┌─────────────┐
          Writes───►│   Primary   │
                    └──────┬──────┘
                           │ Replication
          ┌────────────────┼────────────────┐
          ▼                ▼                ▼
    ┌─────────────┐  ┌─────────────┐  ┌─────────────┐
    │  Replica 1  │  │  Replica 2  │  │  Replica 3  │
    └─────────────┘  └─────────────┘  └─────────────┘
          │                │                │
          └────────────────┴────────────────┘
                      Reads (load balanced)

Configuration:

# Primary node
RUVECTOR_MODE=primary
RUVECTOR_REPLICATION_FACTOR=3
RUVECTOR_SYNC_MODE=async  # or "sync" for strong consistency

# Replica nodes
RUVECTOR_MODE=replica
RUVECTOR_PRIMARY_URL=https://ruvector-primary-xxx.run.app

Benchmarking

Available Benchmarks

Benchmark Description Dimensions Vector Count
quick Fast sanity check 128 10,000
distance Distance computation configurable configurable
hnsw HNSW index search configurable configurable
gnn GNN forward pass 256 10,000 nodes
cuda CUDA kernel perf - -
quantization INT8/PQ compression configurable configurable

Running Benchmarks via API

# Quick benchmark
curl -X POST https://YOUR-SERVICE-URL/benchmark/quick

# Custom distance benchmark
curl -X POST "https://YOUR-SERVICE-URL/benchmark/distance?dims=768&num_vectors=100000&batch_size=64"

# Custom HNSW benchmark
curl -X POST "https://YOUR-SERVICE-URL/benchmark/hnsw?dims=768&num_vectors=100000&k=10"

# Full custom benchmark
curl -X POST https://YOUR-SERVICE-URL/benchmark \
    -H "Content-Type: application/json" \
    -d '{
        "dims": 768,
        "num_vectors": 100000,
        "num_queries": 1000,
        "k": 10,
        "benchmark_type": "hnsw"
    }'

Expected Performance

NVIDIA L4 GPU (Cloud Run default):

Operation Dimensions Vectors P99 Latency QPS
L2 Distance 128 10k 0.5ms 2,000
L2 Distance 768 100k 5ms 200
HNSW Search 128 100k 1ms 1,000
HNSW Search 768 1M 10ms 100
GNN Forward 256 10k nodes 15ms 66

SIMD Capabilities

The benchmark automatically detects and uses:

Architecture SIMD Vector Width Speedup
x86_64 AVX-512 16 floats 8-16x
x86_64 AVX2 8 floats 4-8x
x86_64 SSE4.1 4 floats 2-4x
ARM64 NEON 4 floats 2-4x

Architecture

System Components

┌─────────────────────────────────────────────────────────────────┐
│                        Cloud Run                                 │
├─────────────────────────────────────────────────────────────────┤
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────────┐ │
│  │ HTTP Server │  │  Benchmark  │  │    SIMD/GPU Runtime     │ │
│  │   (Axum)    │  │   Engine    │  │  AVX-512 │ CUDA │ NEON  │ │
│  └──────┬──────┘  └──────┬──────┘  └────────────────┬────────┘ │
│         │                │                          │          │
│  ┌──────┴────────────────┴──────────────────────────┴────────┐ │
│  │                    RuVector Core                          │ │
│  │  ┌────────┐  ┌────────┐  ┌────────┐  ┌────────────────┐  │ │
│  │  │  HNSW  │  │  GNN   │  │ Quant  │  │  Attention     │  │ │
│  │  │ Index  │  │ Layers │  │  INT8  │  │  Multi-Head    │  │ │
│  │  └────────┘  └────────┘  └────────┘  └────────────────┘  │ │
│  └───────────────────────────────────────────────────────────┘ │
├─────────────────────────────────────────────────────────────────┤
│                      NVIDIA L4 GPU                              │
└─────────────────────────────────────────────────────────────────┘

File Structure

examples/google-cloud/
├── Cargo.toml              # Rust dependencies
├── Dockerfile.gpu          # GPU-optimized Docker image
├── cloudrun.yaml           # Cloud Run service configs
├── deploy.sh               # Deployment automation
├── README.md               # This file
└── src/
    ├── main.rs             # CLI entry point
    ├── benchmark.rs        # Benchmark implementations
    ├── simd.rs             # SIMD-optimized operations
    ├── cuda.rs             # GPU/CUDA operations
    ├── report.rs           # Report generation
    └── server.rs           # HTTP server for Cloud Run

API Reference

Endpoints

Method Endpoint Description
GET / API info and available endpoints
GET /health Health check
GET /info System information (GPU, SIMD, memory)
POST /benchmark Run custom benchmark
POST /benchmark/quick Run quick benchmark
POST /benchmark/distance Run distance benchmark
POST /benchmark/hnsw Run HNSW benchmark
GET /results Get all benchmark results
POST /results/clear Clear stored results

Health Check Response

{
    "status": "healthy",
    "version": "0.1.0",
    "gpu_available": true,
    "gpu_name": "NVIDIA L4",
    "simd_capability": "AVX2",
    "uptime_secs": 3600
}

Benchmark Request

{
    "dims": 768,
    "num_vectors": 100000,
    "num_queries": 1000,
    "k": 10,
    "benchmark_type": "hnsw"
}

Benchmark Response

{
    "status": "success",
    "message": "Benchmark completed",
    "result": {
        "name": "hnsw_768d_100000v",
        "operation": "hnsw_search",
        "dimensions": 768,
        "num_vectors": 100000,
        "mean_time_ms": 2.5,
        "p50_ms": 2.1,
        "p95_ms": 3.8,
        "p99_ms": 5.2,
        "qps": 400.0,
        "memory_mb": 585.9,
        "gpu_enabled": true
    }
}

Troubleshooting

Common Issues

1. GPU not detected

# Check GPU availability
gcloud run services describe ruvector-benchmark \
    --region=us-central1 \
    --format='yaml(spec.template.metadata.annotations)'

# Ensure GPU annotations are present:
# run.googleapis.com/gpu-type: nvidia-l4
# run.googleapis.com/gpu-count: "1"

2. Container fails to start

# Check logs
./deploy.sh logs ruvector-benchmark 200

# Common causes:
# - Missing CUDA libraries (use nvidia/cuda base image)
# - Memory limit too low (increase MEMORY env var)
# - Health check failing (check /health endpoint)

3. Slow cold starts

# Set minimum instances
MIN_INSTANCES=1 ./deploy.sh deploy latest true

# Enable startup CPU boost (already in cloudrun.yaml)

4. Out of memory

# Increase memory allocation
MEMORY=16Gi ./deploy.sh deploy latest true

# Or reduce vector count in benchmark
curl -X POST "$URL/benchmark?num_vectors=50000"

Performance Optimization

  1. Enable CPU boost for cold starts

    run.googleapis.com/startup-cpu-boost: "true"

  2. Disable CPU throttling

    run.googleapis.com/cpu-throttling: "false"

  3. Use Gen2 execution environment

    run.googleapis.com/execution-environment: gen2

  4. Tune concurrency based on workload

    • CPU-bound: Lower concurrency (10-20)
    • Memory-bound: Medium concurrency (50-80)
    • I/O-bound: Higher concurrency (100+)

Cleanup

# Remove all RuVector services
./deploy.sh cleanup

# Remove specific service
gcloud run services delete ruvector-benchmark --region=us-central1

# Remove container images
gcloud container images delete gcr.io/PROJECT_ID/ruvector-benchmark

Cost Estimation

Configuration vCPU Memory GPU Cost/hour
Basic 2 4GB None ~$0.10
GPU Standard 4 8GB L4 ~$0.80
GPU High-Mem 8 16GB L4 ~$1.20
Raft Cluster (3) 6 12GB None ~$0.30

Costs are approximate and vary by region. See Cloud Run Pricing.

Contributing

  1. Fork the repository
  2. Create a feature branch
  3. Make your changes
  4. Run benchmarks to verify performance
  5. Submit a pull request

License

MIT License - see LICENSE for details.