Wednesday, January 28, 2015

Solaris 11 - Network Administration / Link Aggregation / IPMP


In this post we will be targeting some Advance features of Network administrations as like Link Aggregation, IPMP (IP Multipathing) features. Before going in deep we can see some basic concept of networking.

 
Basic Concepts :
  • Bandwidth : Bandwidth is bit rate for sending and receiving data over the network.
  • Failover     : Capability to have a backup component for the one becomes unavailable.
  • Load balancing : Distribution of workload to achieve best utilization.   
Link Aggregation:

Link Aggregation is combination of two or three physical interface on the system which are consider as a single logical unit (aggr) that provides best bandwidth, automatic fail over features, load balancing and redundancy benefits. Link Aggregation's Link should be in same Speed and same mode. We can administrate the link aggregation with using dladmcommand by adding, modifying, deleting links.

 
Before starting the aggregation, we have to make it sure physical interface should be have same mode as full-duplex  and same speed. By executing dladm show-link 

Creating aggregartion link with using two interface net1 and net2


root@unixrock:~# dladm create-aggr -l net1 -l net2 aggr0
root@unixrock:~#
root@unixrock:~# dladm show-link
LINK CLASS MTU STATE OVER
net0 phys 1500 up --
net1 phys 1500 up --
net3 phys 1500 up --
net2 phys 1500 up --
aggr0 aggr 1500 up net1 net2
root@unixrock:~#
root@unixrock:~# dladm show-aggr
LINK MODE POLICY ADDRPOLICY LACPACTIVITY LACPTIMER
aggr0 trunk L4 auto off short
root@unixrock:~#

To Add a another Link (net3) to an Aggregation (aggr0)

root@unixrock:~# dladm add-aggr -l net3 aggr0
root@unixrock:~# dladm show-link
LINK CLASS MTU STATE OVER
net0 phys 1500 up --
net1 phys 1500 up --
net3 phys 1500 up --
net2 phys 1500 up --
aggr0 aggr 1500 up net1 net2 net3
root@unixrock:~#

To assigning IP to the aggregation link

root@unixrock:~# ipadm create-addr -T static -a 192.168.113.142 aggr0
ipadm: cannot create address: No such interface
root@unixrock:~#

Opps......getting error says that No such interface, that means we have to probe the interface first

root@unixrock:~# ipadm create-ip aggr0
root@unixrock:~# ipadm create-addr -T static -a 192.168.113.142 aggr0
aggr0/v4
root@unixrock:~#

Yes...we have done the aggr0 aggregation link with using net1 net2 net3 physical link and assigned Ip address too.

root@unixrock:~# dladm show-link
LINK CLASS MTU STATE OVER
net0 phys 1500 up --
net1 phys 1500 up --
net3 phys 1500 up --
net2 phys 1500 up --
aggr0 aggr 1500 up net1 net2 net3
root@unixrock:~#
root@unixrock:~# ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
net0/v4test static ok 192.168.113.139/24
aggr0/v4 static ok 192.168.113.142/24
lo0/v6 static ok ::1/128
root@unixrock:~#

To delete/remove the link from an Aggregation link

root@unixrock:~# dladm show-link
LINK CLASS MTU STATE OVER
net0 phys 1500 up --
net1 phys 1500 up --
net3 phys 1500 up --
net2 phys 1500 up --
aggr0 aggr 1500 up net1 net2 net3
root@unixrock:~# dladm remove-aggr -l net3 aggr0
root@unixrock:~#
root@unixrock:~# dladm show-link
LINK CLASS MTU STATE OVER
net0 phys 1500 up --
net1 phys 1500 up --
net3 phys 1500 unknown --
net2 phys 1500 up --
aggr0 aggr 1500 up net1 net2
root@unixrock:~#

 

IP Multipathing (IPMP)

IP Multipathing (IPMP) provides physical interface failure detection, Load balancing of packets, and transparent redundancy  features. IPMP which makes possible to assign the IP address to a group of network interface. if any one of the interface fails, the group can bind the IP address to another interface from the same IPMP group as transparent redundancy. IPMP can be configured for both IPv4 and IPv6. IPMP group will come up with ACTIVE - ACTIVE and ACTIVE- STANDBY
Advantage of IPMP
  • Fault tolerance
  • Spreading the load
  • Increased bandwidth
  • Transparent redundancy
  • Works even if its different switches
Components of IPMP
  • IPMP daemon : in.mapthd
  • IPMP service : svc:/network/ipmp:default
  • IPMP conf file : /etc/default/mpathd
  • IPMP Commands : ipadm, ipmpstat
Let we start Creating an IPMP group and Adding IP address to an IPMP group and Deleting and disable the IPMP group.
Creating ipmp0 group with using "ipadm create-ipmp" command

root@unixrock:/#
root@unixrock:/# ipadm create-ip net1
root@unixrock:/# ipadm create-ip net2
root@unixrock:/# ipadm create-ip net3
root@unixrock:/# ipadm create-ipmp ipmp0
root@unixrock:/# ipadm add-ipmp -i net1 -i net3 ipmp0
root@unixrock:/# ipmpstat -g
GROUP GROUPNAME STATE FDT INTERFACES
ipmp0 ipmp0 ok -- net3 net1
root@unixrock:/#

Assigning IP to that ipmp0 group

root@unixrock:~# ipadm create-addr -T static -a 192.168.113.145 ipmp0/v4addr1
root@unixrock:~# ipadm create-addr -T static -a 192.168.113.146 ipmp0/v4addr2
root@unixrock:~# ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
net0/v4test static ok 192.168.113.139/24
ipmp0/v4addr1 static ok 192.168.113.145/24
ipmp0/v4addr2 static ok 192.168.113.146/24
lo0/v6 static ok ::1/128
root@unixrock:~#

Remove the interface from IPMP group

root@unixrock:~# ipmpstat -g
GROUP GROUPNAME STATE FDT INTERFACES
ipmp0 ipmp0 ok -- net1 net3
root@unixrock:~#
root@unixrock:~# ipadm remove-ipmp -i net1 ipmp0
root@unixrock:~# ipmpstat -g
GROUP GROUPNAME STATE FDT INTERFACES
ipmp0 ipmp0 ok -- net3
root@unixrock:~#

Adding one Interface to the IPMP group

root@unixrock:~# ipmpstat -g
GROUP GROUPNAME STATE FDT INTERFACES
ipmp0 ipmp0 ok -- net3
root@unixrock:~# ipadm add-ipmp -i net2 ipmp0
root@unixrock:~# ipmpstat -g
GROUP GROUPNAME STATE FDT INTERFACES
ipmp0 ipmp0 ok -- net2 net3
root@unixrock:~#

To disable an IPMP group

root@unixrock:~# ipadm disable-if -t ipmp0
root@unixrock:~# ipmpstat -g
root@unixrock:~# ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
net0/v4test static ok 192.168.113.139/24
lo0/v6 static ok ::1/128
ipmp0/v4addr1 static disabled 192.168.113.145
ipmp0/v4addr2 static disabled 192.168.113.146
root@unixrock:~#

To delete the IPMP group

root@unixrock:~# ipadm delete-ipmp ipmp0
root@unixrock:~# ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
net0/v4test static ok 192.168.113.139/24
lo0/v6 static ok ::1/128
root@unixrock:~#

Encountered some issues and fixes
ERROR : ipadm: cannot delete IPMP interface ipmp0: IPMP group is not empty 

root@unixrock:~# ipadm delete-ipmp ipmp0
ipadm: cannot delete IPMP interface ipmp0: IPMP group is not empty
root@unixrock:~#
root@unixrock:~# ipmpstat -g
GROUP GROUPNAME STATE FDT INTERFACES
ipmp0 ipmp0 ok -- net1
root@unixrock:~# ipmpstat -i
INTERFACE ACTIVE GROUP FLAGS LINK PROBE STATE
net1 yes ipmp0 --mbM-- up disabled ok
root@unixrock:~# ipadm delete-ip net1
root@unixrock:~# ipadm delete-ipmp ipmp0
root@unixrock:~# ipadm show-if
IFNAME CLASS STATE ACTIVE OVER
lo0 loopback ok yes --
net0 ip ok yes --
net3 ip disabled no --
net2 ip disabled no --
root@unixrock:~#

ERROR: ipadm: cannot create interface net2: Operation not supported on disabled object 
ERROR: ipadm: persistent operation not supported for enable-if

root@unixrock:~# ipadm create-ip net2
ipadm: cannot create interface net2: Operation not supported on disabled object
root@unixrock:~#
root@unixrock:~# ipadm show-if
IFNAME CLASS STATE ACTIVE OVER
lo0 loopback ok yes --
net0 ip ok yes --
net3 ip disabled no --
net2 ip disabled no --
root@unixrock:~#
root@unixrock:~# ipadm enable-if net3
ipadm: persistent operation not supported for enable-if
root@unixrock:~#
root@unixrock:~# ipadm enable-if -t net3
root@unixrock:~# ipadm enable-if -t net2
root@unixrock:~#
root@unixrock:~# ipadm show-if
IFNAME CLASS STATE ACTIVE OVER
lo0 loopback ok yes --
net0 ip ok yes --
net2 ip down no --
net3 ip down no --
root@unixrock:~#

Thanks for reading this post. Please leave your valuable comments and queries; I will get back to you at earliest possible.

No comments:

Post a Comment