Skip to content

Commit

Permalink
some network update
Browse files Browse the repository at this point in the history
  • Loading branch information
@dingxiong
dingxiong committed Dec 16, 2024
1 parent 9f46769 commit 8756197
Show file tree
Hide file tree
Showing 3 changed files with 309 additions and 2 deletions.
36 changes: 36 additions & 0 deletions _posts/2024-02-07-network.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -106,3 +106,39 @@ TODO: I do not see Kafka protocol in the list. Maybe make a contribution to it?
A small note about RESP: it uses `\r\n` for separator. So the sample response
above has a lot of `0d0a`. I may need to remember this code as it is quite
common. :)

## Private network

I am curious why my home wifi address is always 192.168.0.1. Also, AWS EKS
services all have address 10.100.x.x

```
$ k get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
emails ClusterIP 10.100.217.45 <none> 8887/TCP 397d
redis-parameter-id-headless ClusterIP None <none> 6379/TCP 14d
redis-parameter-id-master ClusterIP 10.100.255.63 <none> 6379/TCP 14d
redis-parameter-id-replicas ClusterIP 10.100.250.215 <none> 6379/TCP 14d
server ClusterIP 10.100.180.177 <none> 443/TCP,8000/TCP,80/TCP,8001/TCP,8003/TCP 7d5h
```

pod address is always 172.31.x.x

```
$ k get pods -o wide | head -5
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
debug-86dc584667-q27gd 1/1 Running 0 78d 172.31.91.48 ip-172-31-85-107.us-east-2.compute.internal <none> <none>
kafka-binlog-consumer-58c54bcb8-88sqm 1/1 Running 0 101m 172.31.50.98 ip-172-31-49-98.us-east-2.compute.internal <none> <none>
kafka-example-consumer-6fc86dc7f9-q6ml8 1/1 Running 0 101m 172.31.61.141 ip-172-31-49-98.us-east-2.compute.internal <none> <none>
kafka-ip-webhook-consumer-bc5bf8c8d-m5psn 1/1 Running 0 99m 172.31.94.200 ip-172-31-94-125.us-east-2.compute.internal <none> <none>
```

After reading [rfc1918](https://datatracker.ietf.org/doc/html/rfc1918), you
will know why. Quote from it:

> The Internet Assigned Numbers Authority (IANA) has reserved the following
> three blocks of the IP address space for private Internets:
>
> 10.0.0.0 - 10.255.255.255 (10/8 prefix)
> 172.16.0.0 - 172.31.255.255 (172.16/12 prefix)
> 192.168.0.0 - 192.168.255.255 (192.168/16 prefix)
39 changes: 37 additions & 2 deletions _posts/2024-10-06-tcp.md
View file
Original file line number Diff line number Diff line change
@@ -1,11 +1,46 @@
---
layout: post
title: TCP
title: Socket, TCP, and etc
date: 2024-10-06 23:12 -0700
categories: [network]
tags: [network, tcp]
---

## Keepalive
## TCP

### Keepalive

<https://tldp.org/HOWTO/html_single/TCP-Keepalive-HOWTO/>

## Socket in general

The first newbie question when doing socket programming is how to tell a socket
has received all bytes from the other party. If you do not handle it well, you
may get stuck forever at `recv`. The python
[Socket Programming HOWTO](https://docs.python.org/3/howto/sockets.html)
summarizes this issue as the fundamental truth of sockets:

> messages must either be fixed length (yuck), or be delimited (shrug), or
> indicate how long they are (much better), or end by shutting down the
> connection
There are only 4 ways! Joash Xu illustrates the first three in his
[blog](https://enzircle.com/handling-message-boundaries-in-socket-programming).
For the last option, you can read Ivan Velichko's
[blog](https://iximiuz.com/en/posts/writing-web-server-in-python-sockets/).
Also, I want to mention that HTTP protocol uses both delimited method and
length method. It uses `\r\n` to separate the title line and header lines. Then
it uses content-length header to determine the size of data section.

## Netlink socket

The best tutorial online is from
[Kevin He](https://www.linuxjournal.com/article/7356).

> Side note: Kevin He built a company call Deepmotion. It sounds interesting.
## Epoll

TOD: read this https://gist.github.com/dtoma/564375673b354397efc5 This is
almost the same implementation as Java's default
[http server implementation](https://github.com/openjdk/jdk/blob/dc08216f0ef55970c96df43bcc86ebd5792d486e/src/jdk.httpserver/share/classes/sun/net/httpserver/ServerImpl.java)
236 changes: 236 additions & 0 deletions _posts/2024-12-16-istio.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,236 @@
---
layout: post
title: Istio
date: 2024-12-16 13:10 -0800
categories: [network]
tags: [network, istio, envoy]
---

## Pilot

Pilot has two components: `pilot-agent` and `pilot-discovery`. If you have read
my [Envoy notes](envoy.md), you know that the xDS protocol has two parts: proxy
and management server. `pilot-agent` is the proxy part, and `pilot-discovery`
is the management server part.

### Pilot-agent

It runs in the istio-proxy container. Ex below:

```
/usr/local/bin/pilot-agent proxy sidecar --domain filebeat.svc.cluster.local --proxyLogLevel=warning --proxyComponentLogLevel=misc:error --log_output_level=default:info --concurrency 2
```

It is mainly responsible for managing envoy proxy sidecar.

### Pilot-discovery

It runs in `istiod` pod. Ex below:

```
/usr/local/bin/pilot-discovery discovery --monitoringAddr=:15014 --log_output_level=default:info --domain cluster.local --keepaliveMaxServerConnectionAge 30m
```

Pilot-discovery is responsible for reacting to k8s events, converting these
events to xDS responses and sending them to pilot-agents. We use two examples
to illustrate how this part works.

**Example 1: When a new service is deployed in a k8s cluster**

As you can imagine, pilot-discovery subscribes service create/update/delete
events in a k8s cluster. When an event happens, pilot-discovery will update its
internal service registry and push this change to all envoy proxies.

How does this process work? The answer is Kubernetes controllers. The
controllers for all types of resources inside Kubernetes such as namespace,
node, service, pod and etc are defined in
[this file](https://github.com/istio/istio/blob/3ea5695508654a77c4c1638134c0a8aca7e9996b/pilot/pkg/serviceregistry/kube/controller/controller.go#L203).
Especially for services, it register a handler for service event:

```
c.registerHandlers(c.serviceInformer, "Services", c.onServiceEvent, nil)
```

Inside the handler, it calls `xdsUpdater` to push the change to remote Envoy
proxies.

**Example 2: What happens when setting service mesh across two k8s clusters**

Istio is able to set up a service mesh across multiple k8s clusters. How is
this possible? To set it up,
[One step](https://istio.io/latest/docs/setup/install/multicluster/multi-primary/#enable-endpoint-discovery)
of the installation process is to run below command:

```
istioctl x create-remote-secret --context="${CTX_CLUSTER1}" --name=cluster1 | kubectl apply -f - --context="${CTX_CLUSTER2}"
```

It generates a yaml file like following

```
apiVersion: v1
kind: Secret
metadata:
annotations:
networking.istio.io/cluster: cluster1
creationTimestamp: null
labels:
istio/multiCluster: "true"
name: istio-remote-secret-cluster1
namespace: istio-system
stringData:
cluster1: |
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: <...>
server: https://369A1EB0F8100069D5C96281F5BD706D.gr7.us-east-2.eks.amazonaws.com
name: cluster1
contexts:
- context:
cluster: cluster1
user: cluster1
name: cluster1
current-context: cluster1
kind: Config
preferences: {}
users:
- name: cluster1
user:
token: <...>
```

and then applies it to cluster2. Basically, cluster2 now has credentials to
operate on cluster1. See
[this code](https://github.com/istio/istio/blob/44de8ccfe4d25bb658fb8f1efec9da314db4bd8b/istioctl/pkg/multicluster/remote_secret.go#L86)
for more details about how this command works. The secret creation event will
be captured by `secretcontroller` because it listens all events on secrets with
label
[istio/multiCluster](https://github.com/istio/istio/blob/3fcff36ddc5e69ed20755c6385d44eb1a0e50505/pkg/kube/multicluster/secretcontroller.go#L50),
which registers a set of `ClusterHandler`s to handle secret
create/update/delete events.

#### How to use pilot-discovery debug endpoints

Pilot-discovery provides a set of
[debug endpoints](https://github.com/istio/istio/blob/cf3b4fdcba365825a1754ce2dca7a32544d6dfbd/pilot/pkg/xds/debug.go#L147-L194).
Here, we show some example use cases.

**Get all connected istioy-proxy instances**

This endpoint will return all pods that have istio-proxy injected.

```
$ ke istiod-64fb77c48d-x8cjj -- curl localhost:15014/debug/connections
{
"totalClients": 21,
"clients": [
{
"connectionId": "sidecar~172.31.90.176~airflow-scheduler-7bc4cf9fb8-bnb4r.data~data.svc.cluster.local-65372",
"connectedAt": "2022-12-30T07:56:30.843188241Z",
"address": "172.31.90.176:45502",
"watches": null
},
...
```

**Get Envoy config dump**

Envoy admin API has an endpoint
[GET /config_dump](https://www.envoyproxy.io/docs/envoy/latest/operations/admin#get--config_dump).
Istio has a debug endpoint to forward request to this `config_dump` endpoint.

```
ke istiod-64fb77c48d-x8cjj -- curl localhost:15014/debug/config_dump?proxyID=sidecar~172.31.90.176~airflow-scheduler-7bc4cf9fb8-bnb4r.data~data.svc.cluster.local-6537
```

The output is huge. Note the `proxyID` query parameter is required and it is
the `connnectionID` above.

**Get all services**

```
$ ke istiod-64fb77c48d-x8cjj -- curl localhost:15014/debug/registryz
[
{
"Attributes": {
"ServiceRegistry": "Kubernetes",
"Name": "airflow-datadog-cluster-agent-admission-controller",
"Namespace": "data",
"UID": "istio://data/services/airflow-datadog-cluster-agent-admission-controller",
"ExportTo": null,
"LabelSelectors": {
"app": "airflow-datadog-cluster-agent"
},
"ClusterExternalAddresses": null,
"ClusterExternalPorts": null
},
"ports": [
{
"port": 443,
"protocol": "UnsupportedProtocol"
}
],
"creationTime": "2022-11-23T05:46:05Z",
"hostname": "airflow-datadog-cluster-agent-admission-controller.data.svc.cluster.local",
"address": "10.100.94.238",
"Mutex": {},
"cluster-vips": {
"cluster1": "10.100.94.238"
},
"Resolution": 0,
"MeshExternal": false
},
```

The `cluster-vips` displays the virtual IP of the service in the corresponding
cluster. If you have the same service set up in two clusters and in the same
namespace, the you will see two records in `cluster-vips`, and istio will
round-robin requests to these two clusters, namely, this service is a
cross-cluster service.

#### How does injection work

Istio installation will install a MutatingWebhookConfiguration
`istio-sidecar-injector` See blow sample output. As you can see that istiod has
an endpoint `/inject` does then injection work.

```
$ k describe mutatingwebhookconfiguration istio-sidecar-injector
...
Service:
Name: istiod
Namespace: istio-system
Path: /inject
Port: 443
...
```

Checkout
[code](https://github.com/istio/istio/blob/c36c5a5d36d6f77eed830fc13875b10459bfe5d6/pkg/kube/inject/webhook.go#L157)
for more details.

## Useful commands

- `istioctl version`: quickly get all running versions.

## Envoy

Envoy is a proxy solution. There are so many proxies in the market: nginx,
haproxy, trafik and etc, but why Envoy stands out? It is because the xDS APIs,
or a fancier name, xDS protocol. Envoy project incubated a protocol that allows
dynamically changing proxy configurations using a channel between proxy and the
management server.

Alex Burnos has a
[great article](https://medium.com/@aburnos/data-plane-control-plane-and-their-apis-explained-d0a3fa7291f3)
talking about data plane, control plane and universal data plan API. Just to
copy his explanation here:

- data plane: proxies
- control plane: management server
- data plan API: the API for proxies and management server to talk to each
other, i.e., xDS APIs.

See Envoy's golang implementation of data-plane-api:
https://github.com/envoyproxy/go-control-plane.

0 comments on commit 8756197

Please sign in to comment.