Last updated: 3 Jan 2014

• Overview
• Our Goals
• Open Source Virtual Networking Tools
• Using VDE
  • Configure a VM for VDE
  • Start a VDE Virtual Network
  • Control a VDE Virtual Network
  • Show a VDE Virtual Network
  • Stop a VDE Virtual Network

Overview

A virtual network attempts to mimic what real networking equipment does. It can do this either with or without real/physical networking hardware, but most often there is an underlying physical network supporting the virtual network.

While there are other use cases for virtual networks, the most common one is to use virtual networks in conjunction with virtual machines (VMs), in order to connect the VMs without necessarily involving the physical network hardware. A layer 2 switch supporting ethernet is very common, as most VMs run virtual NIC cards that use the ethernet protocol. The virtual machine manager (VMM) usually supports this as a simple switch, but may offer connectivity to other virtual networking equipment. Some VMMs also offer NAT and DHCP services as part of their virtual networking.

Inevitably, one wants to connect a VM in one host computer to a VM in another host computer. VMMs often handle this by controlling the host's physical NIC, putting it in "promiscuous" mode in order to intercept packets coming from and destined to VMs. A VMM can also insert packets into the real network. Consequently, communication between two VMs on different hosts works as if the VMs were real.

Once a packet leaves a VM, it is under control of the VMM, until the VMM decides the packet must go beyond its boundaries. A broadcast packet is put on the physical network; external broadcast packets must be examined to determine if they apply to the VMs. Packets are sent as is over the network, which means that they can be sniffed and can contain malware that could affect the destination. There may be little control over the configuration of the network services such as DHCP and NAT. VLANs may not be supported by the virtual switch. Duplicate MAC addresses from cloned VMs on different hosts could cause problems that are difficult to diagnose.

To address some of these problems, virtual network software that is external to the VMM has been developed. Some of it is commercial, and some is open source. The quality, features and usability of such software varies, sometimes widely. Some software is tied to a particular VMM vendor (e.g., VMware NSX, Microsoft's Hyper-V). Some mimic layer 2 switches, while others include layer 3 routers.

Our Goals

Network isolation was the primary driver. For computer security courses, we wanted to make sure the packets sent by malware under investigation did not escape onto the real network. In addition, if a VM running on one host was providing dynamic IPv4 addresses, often from a private subnet, we did not want real computers on the same physical subnet to get one of those addresses. We also did want packets from VMs used by one class to be inspected ("sniffable") by another class, as well as packets from VMs used by one student in a class to be inspected by another student in the same class.

Another goal was to allow VMs in different virtual computing labs (VCL) to be on the same virtual subnet, even though the VCLs are in different real subnets. That would allow more efficient use of the VCLs, including splitting a large class across different VCLs or allowing a class with multiple VMs per student to transparently use more than one VCL.

Open Source Virtual Networking Tools

We started with openvswitch, but were unable to connect VMs running on different hosts; basically, we weren't able to create a distributed virtual switch.

There was support in VirtualBox for Virtual Distributed Ethernet (VDE), and we found that we could make it work as desired. This is what we describe in this document for our virtual networking software.

At some point, we may investigate Juniper's OpenContrail project.

Using VDE

VDE is controlled from within a manage_vc window, by issuing start_vde and stop_vde commands with various arguments.

But before we get into the details, there is one very important thing to know: a VirtualBox VM will not start until the VDE network has been started. And, in order to connect a VM to the VDE, one must configure the VM for using VDE.

• Configure a VM for VDE
• Start a VDE Virtual Network
• Control a VDE Virtual Network
• Show a VDE Virtual Network
• Stop a VDE Virtual Network

Configure a VM for using VDE

VirtualBox allows the specification of various virtual NICs, which are identified by NIC ids, starting with 1 (e.g., "nic1"). NIC id 1 is the only one we will focus on.

manage_vc allows you to change the virtual computer or VM via the drop-down box to the right of the Do cmd: button, at the bottom of the tabbed window.

1. To make any permanent changes to the VM, it must be powered off.
2. From the drop-down box, select config_vde vmname, where "vmname" is the name of your VM.
3. You can either leave it as is for simple configuration, or specify options to use another NIC id and/or switch name.

     config_vde vmname [id] [switchname]
     

   Often you want to use a different switchname (default is your UW Net Id) if you want to share the switch with others. For example, if "kali" is the vmname, the kali VM will use the switch named "tinfo250a" when it sends packets out its first NIC:

     config_vde kali 1 tinfo250a
     

4. Once you have the config_vde command options desired, you must click on Do cmd: to execute the command, and respond to the prompts.
5. Don't start the VM until you after you Start a VDE Virtual Network.

Start a VDE Virtual Network

You may start the virtual network before you configure any VMs to use it, but the VMs can't be started until the network is started.

Before you start the network, you need to decide what you want to do. Do you want to:

• connect all VMs in a single switch on the same host?

  Use the defaults mentioned below.

• connect all VMs in a single hub on the same host?

  Use an option ("-h") to make the virtual switch behave as a hub instead.

• allow VMs not owned by me to plug into the switch?

  Use an option ("-a") to specify allow any.

• name the switch?

  Use an option ("-n") to specify the name.

• set the number of ports on the switch?

  Use an option ("-m") to specify the number of ports.

• provide DHCP and NAT services to the VMs?

  Use an option ("-d") to specify the starting IPv4 address.

• create a distributed virtual switch across multiple hosts?

  Use an option ("-c") to specify the the remote host and port.

1. From the drop-down box, select or type start_vde
2. You can either leave it as is for simple configuration using your UW Net Id as the switch name, or specify more options.

     start_vde [-a] [-c c_ipv4_addr|file] [-p port] [-d d_ipv4_addr] [-f on|off] [-i public_ipv4_addr -g public_ipv4_gateway] [-h on|off] [-j] [-m num_ports] [-n name] [-r]
          where -a means allow anyone to connect; default is only uwnetid
                -c means create a distributed switch; this host computer
                   is the "server" and the remote host IPv4 address specified
                   (or list of hosts in file) are clients
                -d means start dhcp server using first three
                   octets of d_ipv4_addr
                -f means spanning tree on or off -- the default
                -g means public IPv4 gateway
                -h means hub (on) or switch (off -- the default)
                -i means public IPv4 address
                -j means join/connect main switch to NAT/DHCP switch
                -m means the maximum number of VDE ports on this switch
                -n means name the switch; default name is uwnetid
                -p means port number for this distributed switch server
                -r means reconnect the DHCP/NAT switch and main switch
     

   You can specify multiple options at once.

   For example, if you want to:

   1. connect all VMs in a single hub on the same host:

          start_vde -h on
          

   2. allow VMs not owned by me to plug into the switch:

          start_vde -a
          

   3. name the switch:

      For example, to use the switch named "tinfo250a":

          start_vde -n tinfo250a
          

      At some point before the VMs that use the tinfo250a network are started, they need to be configured to use this named switch, using exactly the same name. For example:

          config_vde kali 1 tinfo250a
          

      and after the start_vde command is executed, all of those VMs using "tinfo250a" can be started.

   4. set the number of ports on the switch:

          start_vde -m 32
          

   5. provide DHCP and NAT services to the VMs:
      • slirpvde/slirpvde6 (buggy)

        You need to select an IPv4 subnet, a private one, to use as the basis for your virtual network. A /24 subnet is the only kind allowed, and the IPv4 address provided is the starting IPv4 address to be assigned to a requester. Start no lower than .5:

              start_vde -d 192.168.62.101
              

        The tool used (slirpvde) also provides network address translation (NAT) using the host's presumed public IPv4 address. Consequently, any packets originating from a VM, destined for the outside world, and able to undergo NAT will be sent outside the virtual network.

      • vyatta (preferred)

        As of December 2013, a Vyatta VM needs to be manually configured for DHCP and NAT service, routing, and firewalling.

      To stop providing DHCP and NAT services in a way you can resume them later (retaining DHCP leases), disconnect the services:

          stop_vde -d
          

      To resume providing DHCP and NAT services previously disconnected:

          start_vde -r
          

      If you find that the main switch is NOT connected to the NAT/DHCP switch, you can fix that as follows:

          start_vde -j
          

   6. create a distributed virtual switch across two hosts:

      You will need to know the host IPv4 address and the port on the host to use. This information is supplied by lab staff.

      For example, if the host IPv4 address is "128.208.244.101":

          start_vde -c 128.208.244.101
          

      You only start the network from one VM, never from each VM that will use the distributed virtual network.

   7. create a distributed virtual switch across multiple hosts:

      You will need to know the host IPv4 addresses of the hosts involved. This information is supplied by lab staff, and placed in a file, usually called client_hosts.

      For example:

          start_vde -n tinfo504 -c client_hosts
          

      You only start the network from one VM, never from each VM that will use the distributed virtual network.

3. Once you have the start_vde command options desired, you must click on Do cmd: to execute the command, and respond to the prompts.
4. You can now start any VMs that use that virtual network.

Control a VDE Virtual Network

Controlling an already-started virtual network uses start_vde to start, reconnect or change an option, and stop_vde to stop or disconnect something already started. See the respective commands for examples.

Show a VDE Virtual Network

You may want to try to visualize the virtual network, to see who is connected to your virtual switches. This can be accomplished by the show_vde command.

1. From the drop-down box, select or type show_vde
2. If you have a simple virtual network, you can either leave the command as is to show the virtual switch. In more complicated virtual networks, you will need to specify more options.

     show_vde [-c c_ipv4_addr|file] [-n name] [-s] [-p]
            where -c means show the distributed switch on this host computer
                     and the remote host IPv4 address specified
                     or the list of hosts in file
                  -n means name the switch; default name is uwnetid
                  -p means show progress information
                  -s means summarize
     

   For example, if you want to:

   1. show a named local virtual network:

          show_vde -n tinfo504
          

   2. show a named distributed virtual network (just the active switches):

          show_vde -n tinfo504 -c client_hosts -s
          

   3. show a named distributed virtual network (all switches):

          show_vde -n tinfo504 -c client_hosts
          

   4. show a named distributed virtual network, with progress while contacting remote switches:

          show_vde -n tinfo504 -c client_hosts -p -s
          

3. Once you have the show_vde command options desired, you must click on Do cmd: to execute the command, and respond to the prompts. If you have many client hosts, it can take some time to contact each one for the switch information.

Stop a VDE Virtual Network

1. Optionally stop the VMs on the virtual network

   Since the VMs participating in the virtual network won't be able to use the network any more, you may decide to shutdown all VMs in the network prior to stopping the network. If you choose the start the virtual network again, prior to shutting down those VMs, virtual networking will not work. You must always start the network first, then the VMs using it.

2. From the drop-down box, select or type stop_vde
3. You can either leave it as is to stop the entire virtual network, or specify more options.

     stop_vde [-d] [-c c_ipv4_addr|file] [-n name] [-s]
            where -c means delete the distributed switch on this host computer
                     and the remote host IPv4 address specified
                     or the list of hosts in file
                  -d means disconnect slirpvde from main switch,
                     thereby preventing access to DHCP and NAT services
                  -n means name the switch; default name is uwnetid
                  -s means stop just the DHCP/NAT service (slirpvde)
     

   For example, if you want to:

   1. disconnect from the DHCP/NAT services to be able to add new VMs to the network later, or renew leases later:

          stop_vde -d
          

      To resume DHCP/NAT services later:

          start_vde -r
          

   2. stop just the DHCP/NAT services:

          stop_vde -s
          

      You cannot use start_vde -d ... after this point to start new DHCP/NAT services, without shutting down all VMs and the virtual network.

   3. name the local switch to stop:

          stop_vde -n tinfo250a
          

   4. name the distributed switch to stop:

          stop_vde -n tinfo250a -c client_hosts
          

4. Once you have the stop_vde command options desired, you must click on Do cmd: to execute the command, and respond to the prompts.

Change Log