Skip to content

Home

Jump to bottom
Alberto Galan edited this page Jan 12, 2017 · 27 revisions

Introduction

Based on:

Course outline:

Pre-reqs

  • Github account
  • Reasonable package management on the host machine: on MacOS X, http://brew.sh
  • Able to use vim (finish vimtutor) and shell/terminal (bash and host shell)

Environment setup (Virtual Machine)

We've selected Ubuntu 14.04 as our preferred Linux distribution.

We recommend vagrant to create VM:

On MacOS X:

brew cask install virtualbox
brew cask install vagrant
brew cask install vagrant-manager

  • vagrant box add ubuntu/trusty64 (this installs the image for Ubuntu 14.04)

  • in a new directory:

vagrant init ubuntu/trusty64
vagrant up 
vagrant ssh

Project

Overall goal:

  • Setup 3-tier WebApp architecture and optimize:
  • Webserver: (NGINX)
  • Application server (NodeJS)
  • database (PostgreSQL)

...also, add data analysis (Jupyter).

Meta: collect, analysis, report on data generated by the system itself (logs and monitoring information)

Overview of planned learnings

  • optimize for China (packages from Chinese mirrors)
  • configure timezone for China (make sure to use NTP)
  • security
  • automate setup and installation
  • test different workloads
  • benchmark and tune operating system and each tier
  • depending on simulated load and hardware profile, select:
  • sysctl settings tweaks
  • service configuration and optimization
  • filesystem / configuration
  • VPN setup
  • services monitoring
  • load monitoring (memory, disk, cpu)
  • log monitoring
  • networking configuration
  • firewall configuration

Exercises

Week 0: installing Linux

Read chapters 1 to 5 of the online course

  • Preface
  • System Startup and Shutdown
  • GRUB
  • init: SystemV, Upstart, systemd
  • Linux Filesystem Tree Layout

Although vagrant and Ubuntu provide a default image (ubuntu/trusty64) it's also possible to create one from scratch and it is a good exercise to do so.

An example tutorial: http://aruizca.com/steps-to-create-a-vagrant-base-box-with-ubuntu-14-04-desktop-gui-and-virtualbox/

Project

  • setup and install an instance of Ubuntu 14.04 LTS Server with Internet connectivity
  • list all services running

Week 1: installing/configuring the webserver

Read chapters 6 to 10 of the online course

  • Kernel Services and Configuration
  • Kernel Modules
  • Devices and udev
  • Partitioning and Formatting Disks
  • Encrypting Disks

Project

  • update packages source to use local country mirrors
  • install webserver: nginx
  • list all services running
  • list files installed by nginx using the package manager
  • make sure that the webserver is up, running and listening to port 80
  • from your own host and using your browser, access http://localhost and make sure the above serves the nginx welcome page
  • change the webserver configuration to serve your own index.html file from your home directory

Week 2: installing/configuring the database server and data source

Project

$ModLoad ompgsql
$WorkDirectory /var/tmp/rsyslog/work
$MainMsgQueueFileName mainq
*.* :ompgsql:localhost,rsyslogpgsql,rsyslogpgsql,<this is the generated password>
  • create directory from the config file above: mkdir -p /var/tmp/rsyslog/work
  • restart rsyslog service
  • query database rsyslogpgsql and table systemevents for count of events
  • create syslog events at each priority level
  • query systemevents table for last ten messages

Week 3: installing the app server

We'll install and run: https://github.com/coderbunker/logsws

  • install vm nodejs
  • create appserver user (can test with vagrant at first)
  • checkout code (/vagrant/logsws, which is mapped to the directory containing your vagrant file)
  • configure nginx to point to appserver (make sure nginx is configured to proxy Websocket connections!)
  • test from a browser that you can connect to the ap

Week 4: install an app server service with upstart/systemd

  • see which management service you have installed in you machine
  • use the appserver created on week 3 to install as service
  • use daemon-start-stop to handle the app
  • create a log file to follow up the service
  • this service is always enabled so test if you stop how the service start again.

look into this documentation install app node.js as a service

Week 5: Monitor the app server with monit

  • install monit
  • create an instance to monitor app server
  • test that is monitor and visualice log

Week 6: separating the app into separate machine instances

  • use centralized logging for all instances

Week 7: automation

  • use ansible to automate the installation so far
Clone this wiki locally