Docker Networking
Tutorial – CoreOS Flannel
Srini Seetharaman
srini@sdnhub.org
November, 2014
CoreOS Flannel
• Lightweight OS based on Gentoo Linux
• Has a distributed key-value store at the core
• Read-only rootfs. Writeable /etc
o All services are in containers
CoreOS
• One CIDR subnet per machine, like Kubernetes
o Host 1: 10.10.10.0/24
o Host 2: 10.10.11.0/24
• No Docker port-based mapping
• Containers reach each other through IP
• Peer network configs exchanged over etcd
• Packets encapsulated using UDP, and soon VxLAN
Flannel Basic
4
1. Build flannel on each host
2. Set key in etcd for network config
Instructions to Run Flannel
6
$ curl -L http://127.0.0.1:4001/v2/keys/coreos.com/network/config
-XPUT -d value='{
"Network": "10.0.0.0/8",
"SubnetLen": 20,
"SubnetMin": "10.10.0.0",
"SubnetMax": "10.99.0.0",
"Backend": {"Type": "udp",
"Port": 7890}}
$ git clone https://github.com/coreos/flannel.git
$ cd flannel
$ docker run -v `pwd`:/opt/flannel -i -t google/golang /bin/bash -c "cd
/opt/flannel && ./build"
3. Start flannel.
o flanneld port created and route is set for the full flat IP range.
Instructions to Run Flannel (contd.)
7
$ sudo ./bin/flanneld &
Output:
I1219 17:34:41.159822 00809 main.go:247] Installing signal handlers
I1219 17:34:41.160030 00809 main.go:118] Determining IP address of default interface
I1219 17:34:41.160579 00809 main.go:205] Using 192.168.111.14 as external interface
I1219 17:34:41.212157 00809 subnet.go:83] Subnet lease acquired: 10.12.224.0/20
I1219 17:34:41.217829 00809 main.go:215] UDP mode initialized
I1219 17:34:41.218953 00809 udp.go:239] Watching for new subnet leases
I1219 17:34:41.219349 00809 udp.go:264] Subnet added: 10.13.128.0/20
core@coreos-05 ~ $ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.111.1 0.0.0.0 UG 1024 0 0 eth0
10.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 flannel0
10.12.224.0 0.0.0.0 255.255.240.0 U 0 0 0 docker0
192.168.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
4. Restart docker daemon with appropriate bridge IP
Instructions to Run Flannel (contd.)
8
$ source /run/flannel/subnet.env
$ sudo ifconfig docker0 ${FLANNEL_SUBNET}
$ sudo docker -d --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} &
• Ping between two bash containers on two different hosts
succeeds. The traffic on wire is encapsulated by flanneld
Testing Flannel Networking
9
192.168.111.14
Docker0
10.12.224.1
bash
192.168.111.13
Docker0
10.13.128.1
bash
$ docker run -i -t ubuntu /bin/bash
root@36484def3b03:/# ifconfig eth0
eth0 Link encap:Ethernet
HWaddr 02:42:0a:0c:e0:02
inet addr:10.12.224.2
Bcast:0.0.0.0
Mask:255.255.240.0
root@36484def3b03:/# ping 10.13.128.2
Success!
$ docker run -i -t ubuntu /bin/bash
root@e0b9dd20d146:/# ifconfig eth0
eth0 Link encap:Ethernet
HWaddr 02:42:0a:0d:80:02
inet addr:10.13.128.2
Bcast:0.0.0.0
Mask:255.255.240.0
Packet on the Wire
10
Original ICMP
packet between
the two containers
Flannel
introduced encap
UDP header
• IP address overlap not possible
o VxLAN not used to create container groups
• User-space encapsulation and forwarding
o Potential performance bottleneck
Limitations
11
Thank you.
https://github.com/sdnhub/lorispack
© 2015 Copyright Reserved

Tutorial on using CoreOS Flannel for Docker networking

  • 1.
    Docker Networking Tutorial –CoreOS Flannel Srini Seetharaman srini@sdnhub.org November, 2014
  • 2.
  • 3.
    • Lightweight OSbased on Gentoo Linux • Has a distributed key-value store at the core • Read-only rootfs. Writeable /etc o All services are in containers CoreOS
  • 4.
    • One CIDRsubnet per machine, like Kubernetes o Host 1: 10.10.10.0/24 o Host 2: 10.10.11.0/24 • No Docker port-based mapping • Containers reach each other through IP • Peer network configs exchanged over etcd • Packets encapsulated using UDP, and soon VxLAN Flannel Basic 4
  • 6.
    1. Build flannelon each host 2. Set key in etcd for network config Instructions to Run Flannel 6 $ curl -L http://127.0.0.1:4001/v2/keys/coreos.com/network/config -XPUT -d value='{ "Network": "10.0.0.0/8", "SubnetLen": 20, "SubnetMin": "10.10.0.0", "SubnetMax": "10.99.0.0", "Backend": {"Type": "udp", "Port": 7890}} $ git clone https://github.com/coreos/flannel.git $ cd flannel $ docker run -v `pwd`:/opt/flannel -i -t google/golang /bin/bash -c "cd /opt/flannel && ./build"
  • 7.
    3. Start flannel. oflanneld port created and route is set for the full flat IP range. Instructions to Run Flannel (contd.) 7 $ sudo ./bin/flanneld & Output: I1219 17:34:41.159822 00809 main.go:247] Installing signal handlers I1219 17:34:41.160030 00809 main.go:118] Determining IP address of default interface I1219 17:34:41.160579 00809 main.go:205] Using 192.168.111.14 as external interface I1219 17:34:41.212157 00809 subnet.go:83] Subnet lease acquired: 10.12.224.0/20 I1219 17:34:41.217829 00809 main.go:215] UDP mode initialized I1219 17:34:41.218953 00809 udp.go:239] Watching for new subnet leases I1219 17:34:41.219349 00809 udp.go:264] Subnet added: 10.13.128.0/20 core@coreos-05 ~ $ route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.111.1 0.0.0.0 UG 1024 0 0 eth0 10.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 flannel0 10.12.224.0 0.0.0.0 255.255.240.0 U 0 0 0 docker0 192.168.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
  • 8.
    4. Restart dockerdaemon with appropriate bridge IP Instructions to Run Flannel (contd.) 8 $ source /run/flannel/subnet.env $ sudo ifconfig docker0 ${FLANNEL_SUBNET} $ sudo docker -d --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} &
  • 9.
    • Ping betweentwo bash containers on two different hosts succeeds. The traffic on wire is encapsulated by flanneld Testing Flannel Networking 9 192.168.111.14 Docker0 10.12.224.1 bash 192.168.111.13 Docker0 10.13.128.1 bash $ docker run -i -t ubuntu /bin/bash root@36484def3b03:/# ifconfig eth0 eth0 Link encap:Ethernet HWaddr 02:42:0a:0c:e0:02 inet addr:10.12.224.2 Bcast:0.0.0.0 Mask:255.255.240.0 root@36484def3b03:/# ping 10.13.128.2 Success! $ docker run -i -t ubuntu /bin/bash root@e0b9dd20d146:/# ifconfig eth0 eth0 Link encap:Ethernet HWaddr 02:42:0a:0d:80:02 inet addr:10.13.128.2 Bcast:0.0.0.0 Mask:255.255.240.0
  • 10.
    Packet on theWire 10 Original ICMP packet between the two containers Flannel introduced encap UDP header
  • 11.
    • IP addressoverlap not possible o VxLAN not used to create container groups • User-space encapsulation and forwarding o Potential performance bottleneck Limitations 11
  • 12.

Editor's Notes