Examples to understand the support of MPTCP in NeST

This directory contains the examples to demonstrate how MPTCP can be used to augment TCP flows in NeST.

MPTCP is a transport protocol built on top of TCP that allows TCP connections to use multiple paths to maximize resource usage and increase redundancy.

REFERENCES

IMPORTANT

  • MPTCP is enabled by default on all Linux kernals > 5.6.

  • For older systems, we recommend upgrading to a newer version. The development team has not tested their changes against the out-of-kernel implementation of MPTCP available here

  • The current implementation utilizes the mptcpize apt-package available on all kernels above 5.15. It is a program that enables multipath TCP on existing legacy services. Link to the man-page.

  • As MPTCP gains traction in the Linux community, and tools natively support MPTCP, Nest will begin their usage. At the time of writing, no public release of any tool (that NeST uses) does so.

PRE-REQUISITE
To install mptcpize, run

sudo apt install mptcpize

USGAGE TIPS AND DEBUGGING TOOLS FOR MPTCP IN NeST

  • Node.add_mptcp_monitor(self)

    monitor displays creation and deletion of MPTCP connections as well as addition or removal of remote addresses and subflows. As far as the topology is concerned, the monitor is the ultimate source of truth regarding which subflows were setup and used during the experiment.
    Reference: ip-mptcp.

  • config.set_value("show_mptcp_checklist", True)

    This config option is useful when experimenting with NeST. When set to True, NeST will match the topology against a checklist. This checklist defines the necessary checks that must pass if an MPTCP behaviour is to be expected from the topology. Note that passing all checks is not a confirmation that MPTCP behaviour will be seen in the experiment for sure. Following is the exhaustive list of checks in the checklist.

    • At least one MPTCP-enabled Flow must exist in the experiment.

    • For every MPTCP-enabled flow, the following checks are performed.

      • Source Node is MPTCP enabled.

      • Destination Node is MPTCP enabled.

      • Either of the end-nodes is multi-homed and multi-addressed.
        See Example 1 for definitions.

      • Atleast 1 interface on Source Node has a subflow/fullmesh MPTCP endpoint.

      • Atleast 1 interface on Destination Node has a signal MPTCP endpoint.

  • config.set_value("log_level", "trace")

    This config option crates a new file called \verb|commands.sh|, listing all the bash commands that are executed by NeST during the runtime of the experiment. To evaluate the validity of our work, and to also ensure that all APIs work as expected, this config is the absolute source of truth as it reveals the exact commands run by the NeST APIs, internally.

  • Be aware of the limitations of iproute2’s ip route add ... API.
    See Example 3 for more info.

1. mptcp-default-configuration.py

This program emulates a network that connects two hosts h1 and h2 via two routers R1, R2. Here, H2 is multihomed (has >=2 interfaces) and also multiaddressed (the 2 interfaces on H2 belong to different subnets). Hence, this topology can exhibit MPTCP behaviour if there are multiple paths between H1 and H2.

The primary gain due to MPTCP is in throughput. This will be noticed due to aggregation of the two subflows between R2 and H2. Due to no other congestion zones in the network, if MPTCP works, we should see a throughput >10mbit.

Source code

  1# SPDX-License-Identifier: GPL-2.0-only
  2# Copyright (c) 2019-2023 NITK Surathkal
  3
  4########################
  5# SHOULD BE RUN AS ROOT
  6########################
  7from nest.topology import *
  8from nest.experiment import *
  9from nest.topology.network import Network
 10from nest.topology.address_helper import AddressHelper
 11
 12from nest import config
 13
 14"""
 15# This config option is useful when experimenting with NeST.
 16# When set to True, NeST will match the topology against a checklist.
 17# This checklist defines the necessary checks that must pass if an MPTCP
 18# behaviour is to be expected from the topology.
 19
 20# Note that passing all checks is not a confirmation that MPTCP behaviour
 21# will be seen in the experiment for sure.
 22"""
 23config.set_value("show_mptcp_checklist", True)
 24
 25"""
 26# This program emulates a network that connects two hosts `h1` and `h2` via two
 27# routers R1, R2. Here, H2 is multihomed (has >=2 interfaces) and also
 28# multiaddressed (the 2 interfaces on H2 belong to different subnets). Hence,
 29# this topology can exhibit MPTCP behaviour if there are multiple paths between
 30# H1 and H2.
 31
 32# The primary gain due to MPTCP is in throughput. This will be noticed due to
 33# aggregation of the two subflows between R2 and H2. Due to no other congestion
 34# zones in the network, if MPTCP works, we should see a throughput >10mbit.
 35
 36################################################################################
 37#                                                                              #
 38#                               Network Topology                               #
 39#      ____                 ____                 ____                ____      #
 40#     |    |               |    |               |    |  5mbit,10ms  |    |     #
 41#     |    |  15mbit,10ms  |    |  10mbit,10ms  |    | ------------ |    |     #
 42#     | H1 | ------------- | R1 | ------------- | R2 |              | H2 |     #
 43#     |    |               |    |               |    | ------------ |    |     #
 44#     |____|               |____|               |____|  5mbit,10ms  |____|     #
 45#                                                                              #
 46################################################################################
 47"""
 48
 49# In the topology, we can see 4 different subnets.
 50# * One between H1-R1
 51# * One between R1-R2
 52# * Two between R2-H2
 53network1 = Network("10.0.0.0/24")
 54network2 = Network("12.0.0.0/24")
 55network3 = Network("192.168.10.0/24")
 56network4 = Network("192.168.11.0/24")
 57
 58# Create two MPTCP enabled hosts `h1` and `h2`, and the routers 'r1' and 'r2'.
 59h1 = Node("h1")
 60h2 = Node("h2")
 61r1 = Router("r1")
 62r2 = Router("r2")
 63
 64# Even though MPTCP is enabled by default, here is the API to do so manually.
 65h1.enable_mptcp()
 66h2.enable_mptcp()
 67
 68"""
 69# add_mptcp_monitor() will run the MPTCP monitor on the specified node,
 70# during the experiment. The monitor shows the MPTCP connections and subflows
 71# created during the experiment with their flow information.
 72
 73# The output is stored as a part of the experiment dump itself.
 74"""
 75h2.add_mptcp_monitor()
 76
 77# Connect `h1` to `h2` via `r1` and `r2` as per topology
 78(eth1a, etr1a) = connect(h1, r1, network=network1)
 79(etr1b, etr2a) = connect(r1, r2, network=network2)
 80(etr2b, eth2a) = connect(r2, h2, network=network3)
 81(etr2c, eth2b) = connect(r2, h2, network=network4)
 82
 83# Assign IPv4 addresses to all the interfaces in the network.
 84AddressHelper.assign_addresses()
 85
 86h1.add_route("DEFAULT", eth1a)
 87r1.add_route(eth2a.get_address(), etr1b)
 88r1.add_route(eth2b.get_address(), etr1b)
 89r2.add_route(eth1a.get_address(), etr2a)
 90h2.add_route(eth1a.get_address(), eth2a)
 91h2.add_route(eth1a.get_address(), eth2b)
 92
 93# Shape all links as specified in the topology
 94eth1a.set_attributes("100mbit", "10ms")
 95etr1a.set_attributes("100mbit", "10ms")
 96etr1b.set_attributes("100mbit", "10ms")
 97etr2a.set_attributes("100mbit", "10ms")
 98etr2b.set_attributes("10mbit", "10ms")
 99etr2c.set_attributes("10mbit", "10ms")
100eth2a.set_attributes("10mbit", "10ms")
101eth2b.set_attributes("10mbit", "10ms")
102
103# Configure MPTCP parameters and flow for the topology.
104
105## 1. Create a flow from eth1a to eth2a
106flow = Flow(
107    h1,
108    h2,
109    eth2a.get_address(),
110    0,
111    60,
112    1,
113    source_address=eth1a.get_address(),  # notice this new parameter
114)
115
116## 2. Allow H1 and H2 to allow 1 subflow creation each.
117## 3. Enable eth1a on H1 as MPTCP subflow endpoint, to initiate suflow creation.
118"""
119`max_subflows` specifies the maximum number of destiadditional subflows allowed
120for each MPTCP connection. Additional subflows can be created due to: incoming
121accepted ADD_ADDR sub-option, local subflow endpoints, additional subflows
122started by the peer.
123
124`max_add_addr_accepted` specifies the maximum number of
125incoming ADD_ADDR sub-options accepted for each MPTCP connection. After
126receiving the specified number of ADD_ADDR sub-options, any other incoming one
127will be ignored for the MPTCP connection lifetime.
128"""
129h1.set_mptcp_parameters(max_subflows=1, max_add_addr_accepted=1)
130h2.set_mptcp_parameters(max_subflows=1, max_add_addr_accepted=1)
131eth1a.enable_mptcp_endpoint(flags=["subflow"])
132eth2b.enable_mptcp_endpoint(flags=["signal"])
133
134# Run the experiment with the flow we just created.
135exp = Experiment("example-mptcp")
136exp.add_mptcp_flow(flow)
137exp.run()

2. mptcp-helper.py

The topology for this example is borrowed from mptcp-default-configuration.py.

The primary purpose of this example is to show the usage of the MPTCP helper in NeST, and how it drastically reduces the setup effort for its users, while still delivering the required configuratrion and results.

Source code

  1# SPDX-License-Identifier: GPL-2.0-only
  2# Copyright (c) 2019-2023 NITK Surathkal
  3
  4########################
  5# SHOULD BE RUN AS ROOT
  6########################
  7from nest.topology import *
  8from nest.experiment import *
  9from nest.topology.network import Network
 10from nest.topology.address_helper import AddressHelper
 11
 12from nest import config
 13
 14config.set_value("show_mptcp_checklist", True)
 15
 16"""
 17# The topology for this example is borrowed from `mptcp-default-configuration.py`
 18
 19# This program emulates a network that connects two hosts `h1` and `h2` via two
 20# routers R1, R2. Here, H2 is multihomed (has >=2 interfaces) and also
 21# multiaddressed (the 2 interfaces on H2 belong to different subnets). Hence,
 22# this topology can exhibit MPTCP behaviour if there are multiple paths between
 23# H1 and H2.
 24
 25# The primary gain due to MPTCP is in throughput. This will be noticed due to
 26# aggregation of the two subflows between R2 and H2. Due to no other congestion
 27# zones in the network, if MPTCP works, we should see a throughput >10mbit.
 28
 29# The primary purpose of this example is to show the usage of the MPTCP helper in
 30# NeST, and how it drastically reduces the setup effort for its users, while still
 31# delivering the required configuratrion and results.
 32
 33################################################################################
 34#                                                                              #
 35#                               Network Topology                               #
 36#      ____                 ____                 ____                ____      #
 37#     |    |               |    |               |    |  5mbit,10ms  |    |     #
 38#     |    |  15mbit,10ms  |    |  10mbit,10ms  |    | ------------ |    |     #
 39#     | H1 | ------------- | R1 | ------------- | R2 |              | H2 |     #
 40#     |    |               |    |               |    | ------------ |    |     #
 41#     |____|               |____|               |____|  5mbit,10ms  |____|     #
 42#                                                                              #
 43################################################################################
 44"""
 45
 46"""
 47Starting standard topology setup
 48"""
 49
 50# In the topology, we can see 4 different subnets.
 51# * One between H1-R1
 52# * One between R1-R2
 53# * Two between R2-H2
 54network1 = Network("10.0.0.0/24")
 55network2 = Network("12.0.0.0/24")
 56network3 = Network("192.168.10.0/24")
 57network4 = Network("192.168.11.0/24")
 58
 59# Create two MPTCP enabled hosts `h1` and `h2`, and the routers 'r1' and 'r2'.
 60h1 = Node("h1")
 61h2 = Node("h2")
 62r1 = Router("r1")
 63r2 = Router("r2")
 64
 65# Connect `h1` to `h2` via `r1` and `r2` as per topology
 66(eth1a, etr1a) = connect(h1, r1, network=network1)
 67(etr1b, etr2a) = connect(r1, r2, network=network2)
 68(etr2b, eth2a) = connect(r2, h2, network=network3)
 69(etr2c, eth2b) = connect(r2, h2, network=network4)
 70
 71# Assign IPv4 addresses to all the interfaces in the network.
 72AddressHelper.assign_addresses()
 73
 74h1.add_route("DEFAULT", eth1a)
 75r1.add_route(eth2a.get_address(), etr1b)
 76r1.add_route(eth2b.get_address(), etr1b)
 77r2.add_route(eth1a.get_address(), etr2a)
 78h2.add_route(eth1a.get_address(), eth2a)
 79h2.add_route(eth1a.get_address(), eth2b)
 80
 81# Shape all links as specified in the topology
 82eth1a.set_attributes("100mbit", "10ms")
 83etr1a.set_attributes("100mbit", "10ms")
 84etr1b.set_attributes("100mbit", "10ms")
 85etr2a.set_attributes("100mbit", "10ms")
 86etr2b.set_attributes("10mbit", "10ms")
 87etr2c.set_attributes("10mbit", "10ms")
 88eth2a.set_attributes("10mbit", "10ms")
 89eth2b.set_attributes("10mbit", "10ms")
 90
 91"""
 92End of standard topology setup
 93"""
 94
 95# Initialize a monitor to verify the MPTCP connection
 96# parameters after the experiment.
 97h2.add_mptcp_monitor()
 98
 99# Run the experiment with the mptcp helper.
100exp = Experiment("example-mptcp")
101
102flow = Flow.setup_mptcp_connection(
103    source_interface=eth1a,
104    destination_interface=eth2a,
105    start_time=0,
106    stop_time=60,
107    number_of_streams=1,
108)
109
110exp.add_mptcp_flow(flow)
111exp.run()

3. mptcp-mega-dumbbell.py

This program contains a bunch of hosts. Some of these (H1, H5) are not multihomed and multiaddressed and they cannot participate in MPTCP. Other hosts are MPTCP capable. As a test, 2 MPTCP flows are setup as follows.

  • H1 to H6

  • H2 to H6

The experiment must show a notable bandwidth improvement on these flows (> 10 mbits). The key idea of this example is to illustrate what makes a flow MPTCP capable. This is shown by considering the 2 flows stated here, and reasoning out whether or not they will exhibit an MPTCP behaviour. The same is then verified through experimentation.

Source code

  1# SPDX-License-Identifier: GPL-2.0-only
  2# Copyright (c) 2019-2023 NITK Surathkal
  3
  4########################
  5# SHOULD BE RUN AS ROOT
  6########################
  7from nest.topology import *
  8from nest.experiment import *
  9from nest.topology.network import Network
 10from nest.topology.address_helper import AddressHelper
 11
 12from nest import config
 13
 14config.set_value("show_mptcp_checklist", True)
 15config.set_value("assign_random_names", False)
 16
 17"""
 18# This program contains a bunch of hosts. Some of these (H1, H5)
 19# are not multihomed and multiaddressed and they cannot participate in MPTCP.
 20# Other hosts are MPTCP capable. As a test, 2 MPTCP flows are setup as follows.
 21# * H1 to H6
 22# * H2 to H6
 23
 24# The experiment must show a notable bandwidth improvement on these flows (> 10 mbits).
 25# The key idea of this example is to illustrate what makes a flow MPTCP capable. This
 26# is shown by considering the 2 flows stated here, and reasoning out whether or not
 27# they will exhibit an MPTCP behaviour. The same is then verified through experimentation.
 28
 29##################################################################################################################
 30#                                                                                                                #
 31#                                              Network Topology                                                  #
 32#                                                                                                                #
 33##################################################################################################################
 34#       ______                                                                                                   #
 35#      |      |                                                                                                  #
 36#      |  H1  | ______________________                                                                           #
 37#      |______|                       |                                                                          #
 38#                                     |                                                                          #
 39#       ______                        |                                        ______                            #
 40#      |      | ______________________|                                       |      |                           #
 41#      |  H2  | ____|____             |                               _______ |  R4  | _______                   #
 42#      |______|     |    |          ______                           |        |______|        |                  #
 43#                   |    |         |      |                          |                        |                  #
 44#       ______      |    |         |  R1  | ______       ______  ____|         ______         |____  ______      #
 45#      |      | ____|    |         |______|       |____ |      |              |      |              |      |     #
 46#      |  H3  | ____|____|          ______         ____ |  R3  | ------------ |  R5  | ------------ |  H6  |     #
 47#      |______|     |    |         |      | ______|     |______| ____         |______|         ____ |______|     #
 48#                   |    |         |  R2  |                          |                        |                  #
 49#       ______      |    |         |______|                          |         ______         |                  #
 50#      |      | ____|    |                                           |_______ |      | _______|                  #
 51#      |  H4  | _________|____________|                                       |  R6  |                           #
 52#      |______|                       |                                       |______|                           #
 53#                                     |                                                                          #
 54#       ______                        |                                                                          #
 55#      |      | ______________________|                                                                          #
 56#      |  H5  |                                                                                                  #
 57#      |______|                                                                                                  #
 58#                                                                                                                #
 59##################################################################################################################
 60#                                                                                                                #
 61#      Link shaping info:                                                                                        #
 62#                                                                                                                #
 63#      * H{1-5} - R{1-2} are all 1000 mbits / 1 ms links                                                         #
 64#      * R{1-6} - R{1-6} are all 10 mbits / 10 ms links                                                          #
 65#      * R{4-6} - H6     are all 1000 mbits / 1 ms links                                                          #
 66#                                                                                                                #
 67##################################################################################################################
 68"""
 69
 70# Setup topology
 71
 72subnets = [Network(f"10.0.{index}.0/24") for index in range(16)]
 73
 74h1 = Node("h1")
 75h2 = Node("h2")
 76h3 = Node("h3")
 77h4 = Node("h4")
 78h5 = Node("h5")
 79
 80r1 = Router("r1")
 81r2 = Router("r2")
 82r3 = Router("r3")
 83r4 = Router("r4")
 84r5 = Router("r5")
 85r6 = Router("r6")
 86
 87h6 = Node("h6")
 88h6.add_mptcp_monitor()
 89h1.add_mptcp_monitor()
 90h2.add_mptcp_monitor()
 91
 92(eth1a, etr1a) = connect(h1, r1, network=subnets[0])
 93(eth2a, etr1b) = connect(h2, r1, network=subnets[1])
 94(eth2b, etr2a) = connect(h2, r2, network=subnets[2])
 95(eth3a, etr1c) = connect(h3, r1, network=subnets[3])
 96(eth3b, etr2b) = connect(h3, r2, network=subnets[4])
 97(eth4a, etr1d) = connect(h4, r1, network=subnets[5])
 98(eth4b, etr2c) = connect(h4, r2, network=subnets[6])
 99(eth5a, etr2d) = connect(h5, r2, network=subnets[7])
100(etr1e, etr3a) = connect(r1, r3, network=subnets[8])
101(etr2e, etr3b) = connect(r2, r3, network=subnets[9])
102(etr3c, etr4a) = connect(r3, r4, network=subnets[10])
103(etr3d, etr5a) = connect(r3, r5, network=subnets[11])
104(etr3e, etr6a) = connect(r3, r6, network=subnets[12])
105(etr4b, eth6a) = connect(r4, h6, network=subnets[13])
106(etr5b, eth6b) = connect(r5, h6, network=subnets[14])
107(etr6b, eth6c) = connect(r6, h6, network=subnets[15])
108
109AddressHelper.assign_addresses()
110
111h1.add_route(eth6a.get_address(), eth1a)
112h1.add_route(eth6b.get_address(), eth1a)
113h1.add_route(eth6c.get_address(), eth1a)
114
115h2.add_route(eth6a.get_address(), eth2a)
116h2.add_route(eth6b.get_address(), eth2b)
117h2.add_route(eth6c.get_address(), eth2b)
118
119h3.add_route(eth6a.get_address(), eth3a)
120h3.add_route(eth6b.get_address(), eth3b)
121h3.add_route(eth6c.get_address(), eth3b)
122
123h4.add_route(eth6a.get_address(), eth4a)
124h4.add_route(eth6b.get_address(), eth4b)
125h4.add_route(eth6c.get_address(), eth4b)
126
127h5.add_route(eth6a.get_address(), eth5a)
128h5.add_route(eth6b.get_address(), eth5a)
129h5.add_route(eth6c.get_address(), eth5a)
130
131r1.add_route(eth6a.get_address(), etr1e)
132r1.add_route(eth6b.get_address(), etr1e)
133r1.add_route(eth6c.get_address(), etr1e)
134
135r2.add_route(eth6a.get_address(), etr2e)
136r2.add_route(eth6b.get_address(), etr2e)
137r2.add_route(eth6c.get_address(), etr2e)
138
139r3.add_route(eth6a.get_address(), etr3c)
140r3.add_route(eth6b.get_address(), etr3d)
141r3.add_route(eth6c.get_address(), etr3e)
142
143r3.add_route(eth1a.get_address(), etr3a)
144r3.add_route(eth2a.get_address(), etr3a)
145r3.add_route(eth3a.get_address(), etr3a)
146r3.add_route(eth4a.get_address(), etr3a)
147r3.add_route(eth2b.get_address(), etr3b)
148r3.add_route(eth3b.get_address(), etr3b)
149r3.add_route(eth4b.get_address(), etr3b)
150r3.add_route(eth5a.get_address(), etr3b)
151
152r4.add_route(eth1a.get_address(), etr4a)
153r4.add_route(eth2a.get_address(), etr4a)
154r4.add_route(eth2b.get_address(), etr4a)
155r4.add_route(eth3a.get_address(), etr4a)
156r4.add_route(eth3b.get_address(), etr4a)
157r4.add_route(eth4a.get_address(), etr4a)
158r4.add_route(eth4b.get_address(), etr4a)
159r4.add_route(eth5a.get_address(), etr4a)
160
161r5.add_route(eth1a.get_address(), etr5a)
162r5.add_route(eth2a.get_address(), etr5a)
163r5.add_route(eth2b.get_address(), etr5a)
164r5.add_route(eth3a.get_address(), etr5a)
165r5.add_route(eth3b.get_address(), etr5a)
166r5.add_route(eth4a.get_address(), etr5a)
167r5.add_route(eth4b.get_address(), etr5a)
168r5.add_route(eth5a.get_address(), etr5a)
169
170r6.add_route(eth1a.get_address(), etr6a)
171r6.add_route(eth2a.get_address(), etr6a)
172r6.add_route(eth2b.get_address(), etr6a)
173r6.add_route(eth3a.get_address(), etr6a)
174r6.add_route(eth3b.get_address(), etr6a)
175r6.add_route(eth4a.get_address(), etr6a)
176r6.add_route(eth4b.get_address(), etr6a)
177r6.add_route(eth5a.get_address(), etr6a)
178
179h6.add_route(eth1a.get_address(), eth6a)
180h6.add_route(eth2a.get_address(), eth6a)
181h6.add_route(eth2b.get_address(), eth6c)
182h6.add_route(eth3a.get_address(), eth6a)
183h6.add_route(eth3b.get_address(), eth6c)
184h6.add_route(eth4a.get_address(), eth6a)
185h6.add_route(eth4b.get_address(), eth6c)
186h6.add_route(eth5a.get_address(), eth6c)
187
188for interface in [
189    eth1a,
190    eth2a,
191    eth2b,
192    eth3a,
193    eth3b,
194    eth4a,
195    eth4b,
196    eth5a,
197    etr1a,
198    etr1b,
199    etr1c,
200    etr1d,
201    etr2a,
202    etr2b,
203    etr2c,
204    etr2d,
205    etr4b,
206    etr5b,
207    etr6b,
208    eth6a,
209    eth6b,
210    eth6c,
211]:
212    interface.set_attributes("1000mbit", "1ms")
213
214for interface in [etr1e, etr2e, etr3a, etr3b, etr3c, etr3d, etr3e, etr4a, etr5a, etr6a]:
215    interface.set_attributes("10mbit", "10ms")
216
217"""
218# This flow from H1 to H6 will not show MPTCP behaviour.
219# The reason for this is that there is only one path from H6 to H1.
220
221# One might argue that packets from H1 can take multiple routes to reach H6.
222# However, there is only one return path and hence, a new subflow is not created.
223
224# In order to have multiple paths visible from H6 to H1, a simple routing
225# (using `ip route add ...`) does not suffice. One must write more controlled
226# `ip rule add ...` commands to dispatch packets from H6 along different routes,
227# based on the originating interface. Such capability is not provided by `ip route`
228# suite and is currently not supported in NeST.
229
230# This can be verified by looking at the output of the MPTCP monitor on H1.
231# H6 signals H1 that 2 of its other interfaces are available for new MPTCP subflows.
232# However, H1 is unable to create a new subflow to H6.
233"""
234flow1 = Flow(
235    h1,
236    h6,
237    eth6a.get_address(),
238    0,
239    60,
240    1,
241    eth1a.get_address(),
242)
243
244"""
245# This flow from H2 to H6 is a standard MPTCP example.
246# It exhibits MPTCP behaviour.
247
248# Note the difference between the 2 flows in this experiment.
249# H1 has only 1 interface, whereas H2 has 2.
250# Also, notice the routing from H6 to H2. There are 2 very distinct paths defined.
251
252# This can be verified by looking at the output of the MPTCP monitor on H2.
253# H6 signals H2 that 2 of its other interfaces are available for new MPTCP subflows.
254# H2 is able to create a new subflow with this information.
255"""
256flow2 = Flow(
257    h2,
258    h6,
259    eth6a.get_address(),
260    0,
261    60,
262    1,
263    eth2a.get_address(),
264)
265
266for host in [h1, h2, h3, h4, h5, h6]:
267    host.set_mptcp_parameters(5, 5)
268
269for interface in [eth1a, eth2a, eth2b, eth3a, eth3b, eth4a, eth4b, eth5a]:
270    interface.enable_mptcp_endpoint(flags=["subflow"])
271
272for interface in [eth6a, eth6b, eth6c]:
273    interface.enable_mptcp_endpoint(flags=["signal"])
274
275exp = Experiment("example-mptcp")
276exp.add_mptcp_flow(flow1)
277exp.add_mptcp_flow(flow2)
278exp.run()