-
Notifications
You must be signed in to change notification settings - Fork 28
Tutorial Part 2B: Add the new principal type
Dan Barrett edited this page Feb 8, 2017
·
1 revision
Prerequisites: A running XIA network described in the previous part.
In this part we will show you how to add a new principal type, Load Balancing ID (LID), and a new forwarding semantic associated with it.
The goal of LID is to balance requests between service replicas. In this tutorial, we will just implement load balancing between two replicas of a simple UDP-based service.
We will go through the following steps
- Start the service on both server0 and server1. Run the client on host0 to see the unicast output.
- Add a new forwarding semantic on the forwarding engine. It randomly selects next hop out of two candidates
- Add the new principal type: LID. Associate it with the new load balancing forwarding engine.
- Manually add a route of that type on router1 as we have no routing protocol for it yet.
- Run the client on host0. Contact the service via LID to see the load balancing behavior.
- Bring up two replicas of a simple UDP based service.
- Start the simple GetTimeOfDay service in
server0
window./xia-core/tutorial/lb_server &
- executing this in
host0
window./xia-core/tutorial/lb_client
The client side will print the response of date and time:
server0: Wed Oct 15 20:27:12 2014 GMT
The server side will print the DAG (address) of the client:
request from: DAG 0 - AD:1000000000000000000000000000000000000000 1 - HID:0000000000000000000000000000000000000000 2 - SID:30000ff0000000000000000000000000c8c1f026 (This SID is random)
- Bring up another replica of that service on
server1
in a new windowssh -A server1
Then/xia-core/tutorial/lb_server &
Now you can see fromhost0
the service replica on server1 responds all the request because it overwrite the service ID record on the name server.
The name,
lb.tutorial.xia
, recorded on the name server is
- Start the simple GetTimeOfDay service in
- Adding the new principal type LID.
All the updates should be done inrouter1
.
- Login to
router1
- First we fix some permissions.
sudo chown -R `whoami` /xia-core
Which XID uses which forwarding engine is defined in/xia-core/etc/click/xia_router_lib.click
.
- Open
/xia-core/etc/click/xia_router_lib.click
to upgrade to the new engine.- At line 44:
c :: XIAXIDTypeClassifier(next AD, next HID, next SID, next CID, next IP, -);
Change it by insertingnext LID
, toThis step basically tells XIA to identify the new XID type.c :: XIAXIDTypeClassifier(next AD, next HID, next SID, next CID, next IP, next LID, -);
- At line 70 and 71:
rt_AD, rt_HID, rt_SID, rt_CID, rt_IP :: XIAXIDRouteTable($local_addr, $num_ports);
c => rt_AD, rt_HID, rt_SID, rt_CID, rt_IP, [2]output;
Change the code by insertingrt_LID :: XIANEWXIDRouteTable($local_addr, $num_ports);
in the middle line and addrt_LID
to the last line.This code basically tell XIA to use a new routing,rt_AD, rt_HID, rt_SID, rt_CID, rt_IP :: XIAXIDRouteTable($local_addr, $num_ports); rt_LID :: XIANEWXIDRouteTable($local_addr, $num_ports); c => rt_AD, rt_HID, rt_SID, rt_CID, rt_IP, rt_LID, [2]output;
XIANEWXIDRouteTable
, table for LID. We will define the table later. - At line 80 or 81
rt_AD[0], rt_HID[0], rt_SID[0], rt_CID[0], rt_IP[0] -> GPRP;
rt_AD[1], rt_HID[1], rt_CID[1], rt_IP[1] -> XIANextHop -> check_dest;
rt_SID[1] -> XIANextHop -> XIAPaint($DESTINED_FOR_LOCALHOST) -> [1]output;
rt_AD[2], rt_HID[2], rt_SID[2], rt_CID[2], rt_IP[2] -> consider_next_path;
rt_AD[3], rt_HID[3], rt_CID[3], rt_IP[3] -> Discard;
rt_SID[3] -> [3]output;
rt_AD[4], rt_HID[4], rt_SID[4], rt_CID[4], rt_IP[4] -> x; // xcmp redirect message
Addedrt_LID[0,1,2,3,4]
to the first, second, fourth, fifth, and seventh line. Change it tort_AD[0], rt_HID[0], rt_SID[0], rt_CID[0], rt_IP[0], rt_LID[0] -> GPRP; rt_AD[1], rt_HID[1], rt_LID[1], rt_CID[1], rt_IP[1] -> XIANextHop -> check_dest; rt_SID[1] -> XIANextHop -> XIAPaint($DESTINED_FOR_LOCALHOST) -> [1]output; rt_AD[2], rt_HID[2], rt_SID[2], rt_CID[2], rt_IP[2], rt_LID[2] -> consider_next_path; rt_AD[3], rt_HID[3], rt_LID[3], rt_CID[3], rt_IP[3] -> Discard; rt_SID[3] -> [3]output; rt_AD[4], rt_HID[4], rt_SID[4], rt_CID[4], rt_IP[4], rt_LID[4] -> x; // xcmp redirect message
- At line 278, add a line of code
Script(write n/proc/rt_LID.add - $FALLBACK);
cp /xia-core/tutorial/xia_router_lib.click /xia-core/click/conf/xia_router_lib.click
to use the file we prepared for you.) - At line 44:
- Finally, give the new type a unique type number by
echo "0x50 LID" > /xia-core/etc/xids
- Login to
- Add a new forwarding engine: the binary load balancer
-
We already prepared skeleton codes of a new forwarding engine for you. This source file is identical to the table based forwarding engine we've been using in XIA.
- Design
- The goal of the new engine is to balance the load between two service replicas.
- The forwarding engine looks up the next hop port number for a XID. The port number is stored in a integer but only the several lowest bits are use because there are number of physical ports are usually less than 100.
int port 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |C| Unused | HextHop port| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ C= Control bit
- We could use the rest bits to store a second port number. We use the lowest 8 bits to store the port number of the service replica on server0 and the next lowest 8 bit for port number of server1.
int port 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |C| |I| HextHop port 2| HextHop port 1| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ C= Control bit, I= Indicator bit
- We randomly pick one port out of two when the engine looks up a LID from the table.
for example, if we want to load balance between port 1 and port 2, the port number should be:int port = 00000000 00000001 00000010 00000001 = 66049 two ports port2 = 2 port1 = 1
- We open the file
/xia-core/click/elements/xia/xianewxidroutetable.cc
for editing. Go to line 582 in that source file
/* GENI TUTORIAL: EDIT BEGIN
* route entry found
* start forwarding
*/
XIARouteData *xrd = (*it).second;
int port = xrd->port;
// check if outgoing packet
if(xrd->port != DESTINED_FOR_LOCALHOST && xrd->port != FALLBACK && xrd->nexthop != NULL) {
p->set_nexthop_neighbor_xid_anno(*(xrd->nexthop));
}
return port;
/* GENI TUTORIAL: EDIT END
*/
- We assume the hybrid port variable is set by some routing protocol and here we just try to interpret it.
- The basic logic goes like this
IF port < (1<<16) THEN only portA is used, just select it ELSE both portA and portB are used, randomly choose one from them
- The C code of such logic is
That is all for the forwarding engine of the new load balancing LID. Save that file. (If you have no time updating this file you can just
XIARouteData *xrd = (*it).second; int port = xrd->port; // check if outgoing packet if(port < (1<<16) && xrd->port != DESTINED_FOR_LOCALHOST && xrd->port != FALLBACK && xrd->nexthop != NULL) { p->set_nexthop_neighbor_xid_anno(*(xrd->nexthop)); } else if (port >= (1<<16)){ click_chatter("Load balancing"); int random = rand(); port = port & 0xFFFF; //clean high 16 bits if (random%2 == 0){ //random selection port = port & 0xFF; //use lowest 8 bits } else{ port = port >> 8; //use lowest 9-16 bits } p->set_nexthop_neighbor_xid_anno(_bcast_xid); // next hop could be any host click_chatter("go to port %d", port); } return port;
cp /xia-core/tutorial/xianewxidroutetable.cc /xia-core/click/elements/xia/xianewxidroutetable.cc
to use the file we prepared for you.)
- Design
- Compile and restart the network
On router1cd /xia-core make tutorial/xnode restart
- Use this new engine: add route
Now the new LID is deployed. We need to add routes telling router1 there are two service replicas.
If we know server0 connects to port0
of router1 and server1 connects to port1
. Then our hybrid port number should beint port = 00000000 00000001 00000001 00000000 = 65792 0s two ports port2 = 1 port1 = 0
Usebin/xroute
to tell which two ports of server0 and server1 should use.In this case we should use port 1 and port 2.router1:/xia-core$ bin/xroute XIA Route version 0.9 router1 TYPE XID PORT FLAGS NEXT HOP ------------------------------------------------------------------------------------------------------------------------ ... AD AD_NAMESERVER 0 0000ffff HID_ROUTER0 ... HID HID_SERVER1 2 0000ffff HID_SERVER1 HID HID_SERVER0 1 0000ffff HID_SERVER0 ...
port (1|2) = 66049 port (1|0) = 65792 port (2|0) = 66048
- Use one of the three value above according to the ports you get. You can use any LID for the time being. We just pick a handy one for you.
Then we will see the route is added into router1
/xia-core/bin/xroute -a router1,LID,LID:0e00000000000000000000000000009876543210,65792
/xia-core/bin/xroute
It should showThatrouter1 TYPE XID PORT FLAGS NEXT HOP --------------------------------------------------------------------------------------- ... AD AD_NAMESERVER 0 0000ffff HID_ROUTER0 ... LID (default) -7 (fallback) 00000000 LID lid:0e00000000000000000000000000009876543210 65792 (1|0) 00000000
(1|0)
means your LIDlid:0e00000000000000000000000000009876543210
is balancing the load between port 0 and 1. We already prepared the script to print both ports for you.
IfAD_NAMESERVER
is not shown in the routing table. you need to login torouter0
to restart that nodessh router0 /xia-core/tutorial/xnode restart
- Use one of the three value above according to the ports you get. You can use any LID for the time being. We just pick a handy one for you.
- Using the LID to access your service
- Now we login to host0.
- Declear the new LID number.
cd /xia-core echo "0x50 LID" > /xia-core/etc/xids
- Restart the local network stack on host0 to make it effective.
tutorial/xnode restart
- We want the client to reach the service replicas via the load balancing LID. So we update the name
lb.tutorial.xia
, associating it with a new DAGThis DAG basically means that to reach that Service ID (SID), one needs to go to router1 first (AD,HID), then use Load balancing ID (LID) to choose which service replica (SID) to go to and if LID is not supported, go to server0 as a fallback/xia-core/tutorial/addname lb.tutorial.xia "DAG 4 0 - AD:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb 4 2 - LID:0e00000000000000000000000000009876543210 4 - HID:6666666666666666666666666666666666666666 4 1 3 - HID:3333333333333333333333333333333333333333 4 - SID:0f00000000000000000000000000000123456789"
If the servers want clients to use the LID, they should use this DAG at the first place. But in this tutorial, we just update it for them.
- Finally we verify the LID by running the lb_client multiple times
tutorial/lb_client
You should see that about 50% of the requests are served by server0, the other 50% are served by server1.