Skip to content

Latest commit

 

History

History
450 lines (368 loc) · 12 KB

clakwdo5q000y08jo1ymp7kgf.md

File metadata and controls

450 lines (368 loc) · 12 KB
title datePublished cuid slug cover ogImage tags
Getting started with Google Compute Engine
Thu Nov 17 2022 09:56:18 GMT+0000 (Coordinated Universal Time)
clakwdo5q000y08jo1ymp7kgf
getting-started-with-google-compute-engine
linux, aws, google-cloud, kubernetes, devops

Google Compute Engine

When we need to deploy applications we need servers and when we need to deploy applications in the cloud we need virtual servers.

  • In corporate data centers, applications are deployed to physical servers.
  • Where do we deploy applications in the cloud?
    • Rent virtual servers
    • Virtual Machines- Virtual servers in GCP
    • Google Compute Engine(GCE) - Provision and manage virtual machines.

    Google compute engine - Features

    Compute Engine Persistent Disk Cloud Load Balancing
  • Create and manage lifecycle of virtual machine instances.
  • Load balancing and autoscaling for multiple VM instances.
  • Attach storage (& network storage) to your VM instances.
  • Manage network connectivity and configuration for your VM instances.
  • TODO:
    • Setup VM instances as HTTP(Web) server.
    • Distribute load with load balancer.

    Creating our first VM in GCP

  • Create a few VM instances and play with them.
  • Checkout the lifecycle of VM instances.
  • Use SSH to connect to VM instances.

    Create a virtual machine using the GCP Console

    1. In the Navigation menu (Navigation menu), click Compute Engine > VM instances.
    2. Click Create.
    3. On the Create an Instance page, for Name, type your VM name my-vm-1.
    4. For Region and Zone, select the region and zone .
    5. For Machine type, accept the default.
    6. For Boot disk, if the Image shown is not Debian GNU/Linux 10 (Buster), click Change and select Debian GNU/Linux 10 (Buster).
    7. Leave the defaults for Identity and API access unmodified.
    8. For Firewall, click Allow HTTP traffic.
    9. Leave all other defaults unmodified.
    10. To create and launch the VM, click Create.

    Compute Engine Machine types and images

    Compute engine machine family

  • What type of hardware do you want to run your worklaods on?
  • Different machine families for different workloads:
    • General purpose(E2,N2,N2D,N1): Best price-performance ratio
      • Web and application servers, small-medium databases, dev environments.
    • Memory optimized(M2,M1): Ultra high memory workloads
      • Large in-memory databases and in-memory analytics.
    • Compute Optimized(C2): Compute intensive workloads
      • Gaming Applications

    Compute Engine Machine Types

    Machine Name vCPUs^1 Memory(GB) Max number of persistent disks(PDs)^2 Max total PD size(TB) Local SSD Maximum egress bandwidth(Gpps)^3
    e2-standard-2 2 8 128 257 no 4
    e2-standard-4 4 16 128 257 no 8
    e2-standard-8 8 32 128 257 no 16
    e2-standard-16 16 64 128 257 no 16
    e2-standard-32 32 128 128 257 no 16

  • How much CPU, memory or disk do you want?
    • variety of machine types are available for each machine family.
    • Let's take an example: e2-standard-2:
      • e2- machine type family
      • standard- Type of workload
      • 2- number of CPUs

  • Memory, disk and networing capabilities increase along with vCPUs
  • Image

  • What operating system and what software do you want on the instance?
  • Types of Images:
    1. Public images: Provided & maintained by google or open source communities or third party vendors.
    2. Custom Images: Created by you for your projects.

    Some commands we can use inside our debian VM

    sudo su
    apt update 
    apt install apache2
    ls /var/www/html
    echo "Hello World!"
    echo "Hello World!" > /var/www/html/index.html
    echo $(hostname)
    echo $(hostname -i)
    echo "Hello World from $(hostname)"
    echo "Hello World from $(hostname) $(hostname -i)"
    echo "Hello world from $(hostname) $(hostname -i)" > /var/www/html/index.html
    sudo service apache2 start

    Installing HTTP webserver on GCE VM

  • Install apache web server
    sudo su
    apt install apache2

    Now if I click on external IP of the virual machine I can see the apache debian webpage

    Now lets go to apache html dir

    ls /var/www/html
    # output- index.html

    we can put our content inside our index.html file

    echo "Hi I'm Tushar" > /var/www/html/index.html

    Internal and External IPs

  • External (Public) IP addresses are internet addressable.
  • Internal(Private) IP addresses are internal to a corporate network.
  • You can't have two resource with same public(external) IP address
    • However, two different corporate networks can have resources with same internal (private) IP address.
  • Al VM instances are assigned to at least one internal IP address.
  • Creation of External IP addresses ca be enables for VM instances.
    • (Remember) When you stop an VM instance, External IP address is lost.

    Static IP addresses

  • Scenario: How do you get a constant external IP address for a VM instance?
    1. Go to Search bar and search External IP addresses
    2. Click on external ip address(Part of VPC)
    3. We can see our External IP address which is assigned to our VM.
    4. Click on reserve Static address
    5. Add a name for your static IP address.
    6. click on reserve.
    7. We can see- one static address with type- static, and in VM was ephameral.
    8. Check a button- change in VM row click on that, and attach IP address to VM.
  • Static IP can be switched to another VM instance in same project.
  • Static IP remains attached even if you stop the instance. You have to manually detach it.
  • REMEMBER: You are billed for an static IP when you are not using it as well!
    • Make sure that you explicitly release an static IP when you're not using it.

    Startup Script

    Simplify VM HTTP server setup

  • How do you reduce the number of steps in creating an VM instance and setting up a HTTP server?
  • Options:
    • Startup script
    • instance template
    • custom image

    Bootstrapping with startup script

    #!/bin/bash
    apt update 
    apt -y install apache2
    echo "Hello world from $(hostname) $(hostname -I)" > /var/www/html/index.html
  • Bootstrapping: Install OS patches or software when an VM instance is launched.
  • In VM, you can configure startup script to bootstrap.

    How to use startup script

  • While creating a VM
  • Click on automation step
  • Add you startup script

    Simplify VM creation with Instance templates

  • Why do you need to specify all the VM instance details(Image, instance type etc) every time you launch an instance?
    • How about creating an instance template?
    • Define machine type, image, labels, startup script and other properties.
  • Used to create VM instances and manage instance groups
    • Provides a convenient way to create similar instances.
  • CANNOT be updated.
    • To make a change, copy an existing template and modify it.
  • (Optional)Image family can be specified(Example- debian-9)
    • Latest non-depprecated version of the family is used.
    1. Go to instance template (under Compute engine)
    2. Click on create instance template
    3. Name it
    4. Add all configurations(region, zone, machine family, machine type etc.)
    5. Fire wall - allow http traffic
    6. Add startup script
    7. Create it
    8. No cost associated with instance template.
  • Create instance from instance template

    Reducing Launch time with a custom image

  • Installing OS patches and software at launch of VM instances increases bootup time.
  • How about creating custom image with OS patches and software pre-installed.
    • Can be created from an instance, a persistent disk, a snapshot, another image, or a file in cloud storage.
    • Can be shared across projects.
    • (Recommendation) Deprecate old images($ specify replacement image)
    • (Recomendation) Hardering an image- customize images to your corporate security standards.
  • Prefer using custom image to startup script

    always stop the instance and create image from it.

    Troubleshooting launch of Apache web server on VM

  • Make sure we're using the external IP of the running VM instance.
  • SSH into it, make sure that apache web server is perfectly installed.
    ls /var/www/html
    cat /var/www/html/index.html
  • Make sure that apache service is startedup fine.
    sudo su
    service apache2 start
  • check url is working or not
  • if not click on VM instance, check firewalls allow http traffic

    Reducing costs - Compute engine Virtual Machines

    Feature GCE
    Sustained use discounts Automatic discounts for using resources for long periods of time.
    Ex: if you use N1, N2, machine types for more than 25% of a month, you get a 20% to 50% discount on every incremental minute.
    Committed use discounts Reserve compute instances ahead of time, commit for 1 year or 3 years, Up to 70% discount based on machine types and base images.
    Preemptible VMs Cheaper, temporary instances for non critical workloads(Fixed pricing, Max 24 hrs, cheapest)

    Achieving High availability with live migration and automatic restart

  • How do you keep your VM instances running when a host system needs to be updated( a software or a hardware update needs to be performed?)
  • Live migration
    • Your running instance is migrated to another host in the same zone.
    • does not change any attributes or properties of the VM.
    • Supported for instances with local SSDs.
    • Not supported for GPUs and preemptible instances.
  • Important configuration- Availabality policy:
    • On Host maintainance: What should happen during periodic infrastructure maintenance?
      • Migrate(default): Migrate VM instance to other hardware/
      • Terminate: Stop the VM instance
    • Automatic restart- Restart VM Instance if they are terminated due to non-user-initiated reasons(maintenance event, hardware failure etc)

    VMs - Best practices

  • Choose Zone and region based on:
    • Cost, regulations, availability needs, latency and specific hardware needs.
    • distribute instances in multiple zones and regions for high availability.
  • Choose right machine type for you needs:
    • Play with them to find out the right machine type.
    • Use GPUs for Math and graphical intensive applications.
  • Reserve for "committed use discounts" for constant workloads.
  • Use preemptible instances for fault-tolerant, NON time critial workloads.
  • Use labels to indicate enviroment, team, business unit etc.

    Compute Engine Scenarios

    Scenario Solution
    What are the pre-requisite to be able to create a VM instance?
  • Project
  • Billing account
  • Compute engine APIs should be enabled.
  • You want dedicated hardware for your compliance, licensing, and management needs Sole-tenant nodes
    I've 1000s of VM and I want to automate OS patch management, OS inventory management and OS configuration management(manage software installed) Use VM Manager
    You want to login to your VM instance to install software SSH into it
    Don't want to expose a VM to internet Don't assign an external IP address
    You want to allow http traffic to your VM Configure firewall rules