内部负载均衡和 VIP

创建一个mynet的overlay网络,创建一个service

  1. vagrant@swarm-manager:~$ docker network ls
  2. NETWORK ID NAME DRIVER SCOPE
  3. afc8f54c1d07 bridge bridge local
  4. 128fd1cb0fae docker_gwbridge bridge local
  5. 0ea68b0d28b9 host host local
  6. 14fy2l7a4mci ingress overlay swarm
  7. lpirdge00y3j mynet overlay swarm
  8. a8edf1804fb6 none null local
  9. vagrant@swarm-manager:~$ docker service create --name web --network mynet --replicas 2 containous/whoami
  10. jozc1x1c1zpyjl9b5j5abzm0g
  11. overall progress: 2 out of 2 tasks
  12. 1/2: running [==================================================>]
  13. 2/2: running [==================================================>]
  14. verify: Service converged
  15. vagrant@swarm-manager:~$ docker service ls
  16. ID NAME MODE REPLICAS IMAGE PORTS
  17. jozc1x1c1zpy web replicated 2/2 containous/whoami:latest
  18. vagrant@swarm-manager:~$ docker service ps web
  19. ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
  20. pwi87g86kbxd web.1 containous/whoami:latest swarm-worker1 Running Running 47 seconds ago
  21. xbri2akxy2e8 web.2 containous/whoami:latest swarm-worker2 Running Running 44 seconds ago
  22. vagrant@swarm-manager:~$

创建一个client

  1. vagrant@swarm-manager:~$ docker service create --name client --network mynet xiaopeng163/net-box:latest ping 8.8.8.8
  2. skbcdfvgidwafbm4nciq82env
  3. overall progress: 1 out of 1 tasks
  4. 1/1: running [==================================================>]
  5. verify: Service converged
  6. vagrant@swarm-manager:~$ docker service ls
  7. ID NAME MODE REPLICAS IMAGE PORTS
  8. skbcdfvgidwa client replicated 1/1 xiaopeng163/net-box:latest
  9. jozc1x1c1zpy web replicated 2/2 containous/whoami:latest
  10. vagrant@swarm-manager:~$ docker service ps client
  11. ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
  12. sg9b3dqrgru4 client.1 xiaopeng163/net-box:latest swarm-manager Running Running 28 seconds ago
  13. vagrant@swarm-manager:~$

尝试进入client这个容器,去ping web这个service name, 获取到的IP 10.0.1.30,称之为VIP(虚拟IP)

  1. vagrant@swarm-manager:~$ docker container ls
  2. CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
  3. 36dce35d56e8 xiaopeng163/net-box:latest "ping 8.8.8.8" 19 minutes ago Up 19 minutes client.1.sg9b3dqrgru4f14k2tpxzg2ei
  4. vagrant@swarm-manager:~$ docker container exec -it 36dc sh
  5. /omd # curl web
  6. Hostname: 6039865a1e5d
  7. IP: 127.0.0.1
  8. IP: 10.0.1.32
  9. IP: 172.18.0.3
  10. RemoteAddr: 10.0.1.37:40972
  11. GET / HTTP/1.1
  12. Host: web
  13. User-Agent: curl/7.69.1
  14. Accept: */*
  15. /omd # curl web
  16. Hostname: c3b3e99b9bb1
  17. IP: 127.0.0.1
  18. IP: 10.0.1.31
  19. IP: 172.18.0.3
  20. RemoteAddr: 10.0.1.37:40974
  21. GET / HTTP/1.1
  22. Host: web
  23. User-Agent: curl/7.69.1
  24. Accept: */*
  25. /omd # curl web
  26. Hostname: 6039865a1e5d
  27. IP: 127.0.0.1
  28. IP: 10.0.1.32
  29. IP: 172.18.0.3
  30. RemoteAddr: 10.0.1.37:40976
  31. GET / HTTP/1.1
  32. Host: web
  33. User-Agent: curl/7.69.1
  34. Accept: */*
  35. /omd #
  36. /omd # ping web -c 2
  37. PING web (10.0.1.30): 56 data bytes
  38. 64 bytes from 10.0.1.30: seq=0 ttl=64 time=0.044 ms
  39. 64 bytes from 10.0.1.30: seq=1 ttl=64 time=0.071 ms
  40. --- web ping statistics ---
  41. 2 packets transmitted, 2 packets received, 0% packet loss
  42. round-trip min/avg/max = 0.044/0.057/0.071 ms
  43. /omd #

这个虚拟IP在一个特殊的网络命令空间里,这个空间连接在我们的mynet这个overlay的网络上

通过 docker network inspect mynet 可以看到这个命名空间,叫lb-mynet

  1. "Containers": {
  2. "36dce35d56e87d43d08c5b9a94678fe789659cb3b1a5c9ddccd7de4b26e8d588": {
  3. "Name": "client.1.sg9b3dqrgru4f14k2tpxzg2ei",
  4. "EndpointID": "e8972d0091afaaa091886799aca164b742ca93408377d9ee599bdf91188416c1",
  5. "MacAddress": "02:42:0a:00:01:24",
  6. "IPv4Address": "10.0.1.36/24",
  7. "IPv6Address": ""
  8. },
  9. "lb-mynet": {
  10. "Name": "mynet-endpoint",
  11. "EndpointID": "e299d083b25a1942f6e0f7989436c3c3e8d79c7395a80dd50b7709825022bfac",
  12. "MacAddress": "02:42:0a:00:01:25",
  13. "IPv4Address": "10.0.1.37/24",
  14. "IPv6Address": ""
  15. }

通过下面的命令,找到这个命名空间的名字

  1. vagrant@swarm-manager:~$ sudo ls /var/run/docker/netns/
  2. 1-14fy2l7a4m 1-lpirdge00y dfb766d83076 ingress_sbox lb_lpirdge00
  3. vagrant@swarm-manager:~$

名字叫 lb_lpirdge00

通过nsenter进入到这个命名空间的sh里, 可以看到刚才的VIP地址10.0.1.30

  1. vagrant@swarm-manager:~$ sudo nsenter --net=/var/run/docker/netns/lb_lpirdge00 sh
  2. #
  3. # ip a
  4. 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
  5. link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
  6. inet 127.0.0.1/8 scope host lo
  7. valid_lft forever preferred_lft forever
  8. 50: eth0@if51: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
  9. link/ether 02:42:0a:00:01:25 brd ff:ff:ff:ff:ff:ff link-netnsid 0
  10. inet 10.0.1.37/24 brd 10.0.1.255 scope global eth0
  11. valid_lft forever preferred_lft forever
  12. inet 10.0.1.30/32 scope global eth0
  13. valid_lft forever preferred_lft forever
  14. inet 10.0.1.35/32 scope global eth0
  15. valid_lft forever preferred_lft forever
  16. #

和ingress网络一样,可以查看iptables,ipvs的负载均衡, 基本就可以理解负载均衡是怎么一回事了。 Mark=0x106, 也就是262(十进制),会轮询把请求发给10.0.1.31 和 10.0.1.32

  1. # iptables -nvL -t mangle
  2. Chain PREROUTING (policy ACCEPT 128 packets, 11198 bytes)
  3. pkts bytes target prot opt in out source destination
  4. Chain INPUT (policy ACCEPT 92 packets, 6743 bytes)
  5. pkts bytes target prot opt in out source destination
  6. 72 4995 MARK all -- * * 0.0.0.0/0 10.0.1.30 MARK set 0x106
  7. 0 0 MARK all -- * * 0.0.0.0/0 10.0.1.35 MARK set 0x107
  8. Chain FORWARD (policy ACCEPT 36 packets, 4455 bytes)
  9. pkts bytes target prot opt in out source destination
  10. Chain OUTPUT (policy ACCEPT 101 packets, 7535 bytes)
  11. pkts bytes target prot opt in out source destination
  12. Chain POSTROUTING (policy ACCEPT 128 packets, 11198 bytes)
  13. pkts bytes target prot opt in out source destination
  14. # ipvsadm
  15. IP Virtual Server version 1.2.1 (size=4096)
  16. Prot LocalAddress:Port Scheduler Flags
  17. -> RemoteAddress:Port Forward Weight ActiveConn InActConn
  18. FWM 262 rr
  19. -> 10.0.1.31:0 Masq 1 0 0
  20. -> 10.0.1.32:0 Masq 1 0 0
  21. FWM 263 rr
  22. -> 10.0.1.36:0 Masq 1 0 0
  23. #

这个流量会走我们的mynet这个overlay网络。