Cloud-computing

Concepts of cloud

Cloud Computing:-

Cloud computing is the delivery of computing services—including servers, storage, databases, networking, software, analytics, and intelligence—over the Internet ("the cloud"). This allows for faster innovation, flexible resources, and economies of scale.

Docker:-

Docker is a software platform that allows you to build, test, and deploy applications quickly. Docker packages software into standardized units called containers that have everything the software needs to run including libraries, system tools, code, and runtime. Using Docker, you can quickly deploy and scale applications into any environment and know your code will run.

Key features of Docker include:-;

1.Containerization:

Applications run in isolated environments, which makes them portable and easy to manage.

2. Images and Dockerfile:

Docker uses images as blueprints for containers. A Dockerfile is a script that contains instructions to build an image.

3. Docker Hub:

A cloud-based registry where users can share and distribute Docker image

How Docker works:-

Docker works by providing a standard way to run your code. Docker is an operating system for containers. Similar to how a virtual machine virtualizes .containers virtualize the operating system of a server. Docker is installed on each server and provides simple commands you can use to build, start, or stop containers.

What is the structure of a Docker image?:-

A Docker image is composed of multiple layers stacked on top of each other. Each layer represents a specific modification to the file system (inside the container), such as adding a new file or modifying an existing one. Once a layer is created, it becomes immutable, meaning it can't be changed.

HOW TO INSTALL DOCKER:-

To install Docker on a remote server using PuTTY, you'll first need to ensure you have access to a Linux server (like Ubuntu, CentOS, etc.) via SSH. Here's a step-by-step guide:

Step 1:Connect to Your Server

Step 2: Update Your Package Index

Before installing Docker, it’s a good idea to update the package index:

sudo apt update

Step 3:Install Prerequisites

For Ubuntu, install the required packages:

sudo apt install apt-transport-https ca-certificates curl software-properties-common

For CentOS, run:

sudo yum install -y yum-utils

Step 4: Add Docker’s Official GPG Key

For Ubuntu:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

For CentOS:

sudo rpm --import https://download.docker.com/linux/centos/gpg

Step 5: Set Up the Stable Repository

For Ubuntu:

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

For CentOS:

sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

Step 6: Install Docker

For Ubuntu:

sudo apt update

sudo apt install docker-ce

For CentOS:

sudo yum install docker-ce

Step 7: Start Docker

Enable and start the Docker service:

sudo systemctl start docker

sudo systemctl enable docker

Step 8: Verify the Installation

Check if Docker is running:

sudo systemctl status docker

You can also run a test container:

sudo docker run hello-world

Step 9: (Optional) Manage Docker as a Non-Root User

If you want to run Docker commands without sudo, add your user to the

Docker group:

sudo usermod -aG docker $USER

After running this command, log out and back in for the changes to take effect.

Conclusion

You’ve successfully installed Docker using PuTTY! If you have any questions or run into issues, feel free to ask.

container:-

A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another. A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries and settings.

(in other and simple words )

Containers are an abstraction at the app layer that packages code and dependencies together. Multiple containers can run on the same machine and share the OS kernel with other containers, each running as isolated processes in user space. Containers take up less space than VMs (container images are typically tens of MBs in size), can handle more applications and require fewer VMs and Operating systems

NGINX:-

NGINX is a high-performance web server and reverse proxy server that is widely used for serving web applications, handling HTTP and HTTPS requests, and load balancing traffic. It is known for its speed, efficiency, and ability to handle a large number of concurrent connections with low resource usage.

HOW TO INSTALL NGINX :-

In this tutorial, we’ll show you how to install NGINX on Linux.

Open your Linux machine and run an update using the command below:

sudo apt-get update

Next, run this command:

sudo apt-get install nginx

Then, enable your firewall with the following:

sudo ufw enable

To verify NGINX is installed, run the following:

nginx -v

You can run the command below to find out if NGINX is running:

sudo ufw status

After running this command, you should see the following:

status: active

To check whether your NGINX server is working fine, run the following:

sudo systemctl status nginx

using EC2 in aws:-

First open aws search EC2 then Launch Instance and there select keypair in putty then download it

after that Launch it and run putty and paste public id on HOST NAME and open that downloaded key pair for putty in SSH then Auth then Credentials and open there

after that run it and write username as ubuntu as selected os and then type following commands

sudo apt update

sudo apt install apache2

to install a web server on ip then

sudo su

for convert $ into # for getting admin role then

cd /var/www/html/

##then

ls

for list of html file in it

then copy that html file name and write

rm index.html

rm means remove command

vi index.html

this will open a notepad like and write html code there like (vi is editor) -

then press ctrl+c then shift+colon then write wq and enter

now copy your public ip and paste it on browser you will see the texts written by you (by using html above)

congratulations you got it 👏🏻 🎉

USING CONTAINER IN VM and adding nginx server by Docker:-

First open aws search EC2 then Launch Instance and there select keypair in putty then download it

after that edit network setting and click on add security group rule and select TCP,UDP,ALL TRAFFIC AND SELECT EVERYWHERE SOURCE TYPE IN THEM then Launch it and run putty and paste public id on HOST NAME and open that downloaded key pair for putty in SSH

(SSH (Secure Shell) is a way to securely connect to another computer over a network. It's like a safe tunnel that allows you to control a remote computer and transfer files to it, without anyone else being able to listen in or interfere with the connection.)

then Auth then Credentials and open there

after that run it and write username as ubuntu as selected os and then type following commands

curl -sL https://github.com/ShubhamTatvamasi/docker-install/raw/master/docker-install.sh | bash

this will install and run docker in your vm

newgrp docker

this command will help us to use docker

docker ps

this will list docker

docker --version

this will display the version of docker installed

now installing nginx

docker pull nginx

You can download Nginx from a pre-built Docker image, with a default Nginx configuration, by above command. This downloads all the necessary components for the container.

docker run --name docker-nginx -p 80:80 nginx

Nginx installed, you can configure the container so that it’s publicly accessible as a web server.

run is the command to create a new container

The --name flag is how you specify the name of the container. If left blank, a generated name like nostalgic_hopper will be assigned.

-p specifies the port you are exposing in the format of -p local-machine-port:internal-container-port. In this case, you are mapping port :80 in the container to port :80 on the server.

nginx is the name of the image on Docker Hub.

now this will show this on your public ip

In your terminal, enter CTRL+C to stop the container from running.

docker ps -a

verify the container status with this command

docker rm docker-nginx

Remove the existing container

docker run --name docker-nginx -p 80:80 -d nginx

Create a new, detached Nginx container,By attaching the -d flag, you are running this container in the background.

docker ps

this will obtain info about your container

docker stop docker-nginx

Stop the container

docker rm docker-nginx

remove the container

Building a Web Page to Serve on Nginx

mkdir -p ~/docker-nginx/html

Create a new directory for your website content within the home directory

cd ~/docker-nginx/html

by this you navigate into this

vi index.html

now press i and write your code in html like

then press ctrl+c then shift+colon then write wq and enter

docker run --name docker-nginx -p 80:80 -d -v ~/docker-nginx/html:/usr/share/nginx/html nginx

Linking the Container to the Local Filesystem

open your public ip in browser you will see as the content as your html code

here you go 👏🏻

Using Your Own Nginx Configuration File

cd ~/docker-nginx

docker cp docker-nginx:/etc/nginx/conf.d/default.conf default.conf

Copy the Nginx config directory into your project folder

docker stop docker-nginx

docker rm docker-nginx

to rebuild the container stop the container then remove it

docker run --name docker-nginx -p 80:80 -v ~/docker-nginx/html:/usr/share/nginx/html -v ~/docker-nginx/default.conf:/etc/nginx/conf.d/default.conf -d nginx

This command links the custom website pages to the container.

docker restart docker-nginx

you need to restart your container to reflect changes on the associated pages.

Orchestration:-

In cloud computing, orchestration is the process of coordinating and automating the management of applications, tools, and infrastructure across multiple clouds

Kubernestes :-

Kubernetes (sometimes shortened to K8s with the 8 standing for the number of letters between the “K” and the “s”) is an open source system to deploy, scale, and manage containerized applications anywhere.

Kubernetes has built-in commands to handle a lot of the heavy lifting that goes into application management, allowing you to automate day-to-day operations. You can make sure applications are always running the way you intended them to run.

Minikube:-

Minikube is a tool that sets up a Kubernetes environment on a local PC or laptop. This tool provides an easy means of creating a local Kubernetes environment on any Linux, Mac, or Windows system, where you can experiment with and test Kubernetes deployments.

USING MINIKUBE IN AWS

##First open aws search EC2 ,and download your passkey then Launch Instance and select 22.04 AMI then select t2.xlarge instance type then select keypair then configure storage to 30 GB then enable all traffic in network and Launch.

##Launch PuTTY and add the IP address from instance and add key pair file and open the PuTTY terminal.

Now connect it with putty and login into it by writing ubuntu

Now put some commands

curl -sL https://github.com/ShubhamTatvamasi/docker-install/raw/master/docker-install.sh | bash

it will install docker

sudo usermod -aG docker $USER

newgrp docker

it will Add your local user to docker group so that your local user run docker commands

sudo snap install kubectl --classic

it will intall kubernetes

kubectl version --client

it checks the version

Installing Minikube

curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64

sudo install minikube-linux-amd64 /usr/local/bin/minikube

minikube version

it checks its version

Starting Minikube with Docker Driver

minikube start --driver=docker

If you encounter root privileges error, run:

minikube start --driver=docker --force

minikube status

it checks its status

kubectl cluster-info

it checks cluster info

kubectl config view

it will show the config

kubectl get nodes

it will display nodes in it

kubectl get pods

it will show pods in it

kubectl create deployment nginx-web --image=nginx

kubectl expose deployment nginx-web --type NodePort --port=80

kubectl get deployment,pod,svc

it will deploy a sample nginx deployment

minikube addons list

It will display all addons

minikube addons enable dashboard

minikube addons enable ingress

this enables these addons

minikube dashboard --url**

it will get the url and run the dashboard of MiniKube

kubectl proxy --address='0.0.0.0' --disable-filter=true &

This will enable port :8001 to access it on your public ip

http://server_ip:8001/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/#/workloads?namespace=default

in browser replace server ip with public ip

Now go to above link and replace server_ip with your public ip and it will show you like :

KVM:- Kernel-based Virtual Machine

It is a technology that allows you to run multiple operating systems on a single physical machine. It turns the Linux kernel into a hypervisor, enabling virtual machines (VMs) to operate as if they were separate computers.

Openstack :-

OpenStack is an open-source platform that lets you create and manage cloud computing services. It allows users to control computing power, storage, and networking in a data center through a web interface. Essentially, it helps organizations build their own private or public clouds, making it easier to deploy and manage applications.

QEMU

QEMU is an open-source emulator and virtualization tool that allows you to run different operating systems on a host machine.

VPC

Amazon Virtual Private Cloud (VPC) is a virtual network that allows users to launch AWS resources in a logically isolated environment. It's a foundational service of AWS that gives users complete control over their virtual networking environment.

What is the difference between EC2 and VPC?

•EC2 is a virtual server that you can run your software on. VPC is a virtual network that you use to connect your virtual servers, and other resources.

Go to VPC and create a VPC then we have to create 4 subnets , where 2 subnets are private and other two are public .

Creating launch template in auto scaling group

Creating auto scaling group with ec2 instances in private subnet

Creating bastion host

copying pem file to bashtion host

log in to private instance

putting simple html file in private instance

Creating load balancer

creating target group with port 8000

creating load balancer

viewing our project through DNS name in browser

NETWORKING

what is client and server

( below image shows this )

types of network

Network Devices:-

1. network adaptor/ interface

• connects a device to network.

• has a Mac address by manufacturer

• second layer device

2. switch

•it is a very multiport network bridge that uses MAC address to forward data

•link layer device

3. Router

•it is a device that forwards data between computer networks

• 3rd layer device

4. HUB

•network device that used to connect multiple computers in a network

• all information send to the hub is automatically send to each port to every device

5. TAP

(WORKS ON LAYER 2 -- ethernet frame )

used to create a user space network bridge.

6. TUN

( WORKS ON LAYER 3 -- IP packets )

create a tunnel network to reach another network .

Basic terms for understanding networking better:-

1.NAT :-

it is a process in which one or more local IP addresses are translated into one or more global IP addresses and vice versa

2. veth :-

these are pair of virtual network interfaces that are used to connect network namespaces together.

3.DPDK :- Data Plane Development Kit

it is a set of libraries and drivers that accelerates packet processing and their ability to create packet forwarders without the need of costly custom switches and routers

4. NIC :- network interface card

It allows one device to connect to network

5. DPU :- Data Processing Unit

it is a new programmable processor that helps move data around data centres. it ensures right data goes to right place in right format quickly .

6.CSI :- Container Storage Interface

7. OVS :- Open virtual switch

it is used with hypervisors to interconnect virtual machines within a host and between different hosts accross networks.

8. QEMU :- Quick emulator

free open source machine it can run various guest operating systems (OS's ) and architecture on a single host system.

9. Docker :-

it is like a container that holds everything your application needs to run including the code, libraries.

10. Kubernets :-

it is like a manager for containers .it helps to deploy, scale and manage a group of containers making sure they run smoothly.

11. web assembly:- (WASM)

technology that allows you to run code written in different programming languages. It is a way to build high speed, responsive web applications that can handle data and communicate over networks.

12. Firewall :-

security system that controls incoming and outgoing network traffic based on pre- determined security rules.

13. DMZ :-

14. VxLAN :-

It is tunneling report that tunnel Ethernet traffic (layer 2) over an IP network(layer 3).

14.1 VTEP:-

it is a device that's responsible for encapsulating and de-capsulating layer 2 traffic.

15. LINUX Bridge :-

it is a kernel module that behaves like a network switch .It is usually used for forwarding packages on routes or gateways or between virtual machines.

16. Pktgen :-

it is a tool for high speed package generation and testing. It in the Linux kernel.

netns :- network namespace

feature of Linux kernel that provides a way to create isolated network environment.

TAP is often used to connect VM or containers to a physical network

KVM :- kernel virtual machine

( typer 1 hypervisor)

CNI :- Container Networking Bridge

it is responsible for setting up the network ( assigning IP address, create network bridge) for containers ,enabling communication between containers and outside world

examples VLAN , IPvLAN , CALICO , FLANNEL , VMware. etc

Flannel :-

it acts as a layer that allows containers to send and receive data seamlessly accross various hosts .

working:-

it runs a small single binary agent on every host. this networking tool gives every host an IP subnet.

Packet Switching:-

when we send email or web page the data does not travel as single continuous stream instead is broken down into smaller chunks called packets .

key functions of network core :-

1. Forwarding :-

• it is a local action of moving and arriving packets from a router's input to appropriate router output link.

2. Routing :-

• global process of determining the full paths packets take from source to destination

Network Protocols:-

set of rules that determine how the packet is to be transferred and received and in which format .

1. TCP

it ensures reliable order delivery of data between applications . It handles things like breaking data into packets

2. IP

responsible for addressing and routing packets across the internet.

3. HTTP

it is a protocol that powers the world wide web defining how messages are formatted and transmitted between web browsers and servers.

NETWORK STACK

TCP /IP

provide reliable transmission of data.

UDP

provides faster but less reliable transmission of data.

.

Packet flow, also known as network flow or traffic flow:-

it is the sequence of packets that travel from a source computer to a destination.

•Packets are segments of data that are routed through a network of interconnected devices, such as switches and routers, before reaching their destination.

--below images shows network setup for packet flow