Try vNet with Docker

This tutorial runs a whole vNet deployment on one machine with Docker Compose: one conductor, three switches and two agents, with TLS on every link. It takes about five minutes, most of it building the images. At the end you will have seen a machine join, the access list at work, a rule changed live, a switch fail and the conductor restart.

What you need

  • A Linux machine with Docker Engine and the Compose plugin. The switches and agents need /dev/net/tun and the NET_ADMIN capability, which Docker provides on Linux.

  • git and openssl.

1. Get the code and make the certificates

$ git clone https://github.com/polympiads/vNet.git
$ cd vNet
$ ./docker/certs.sh
wrote /home/you/vNet/docker/certs/ca.pem and /home/you/vNet/docker/certs/ca.key
...
certificates are in /home/you/vNet/docker/certs

docker/certs.sh makes a certificate authority and one certificate per container, each with its role (conductor, switch or agent) in the certificate. The directory is ignored by git.

2. Start everything

$ cd docker
$ docker compose up -d --build
$ docker compose ps --format 'table {{.Name}}\t{{.Status}}'
NAME             STATUS
vnet-agent       Up 8 seconds
vnet-agent2      Up 8 seconds
vnet-conductor   Up 19 seconds (healthy)
vnet-switch1     Up 13 seconds (healthy)
vnet-switch2     Up 13 seconds (healthy)
vnet-switch3     Up 13 seconds (healthy)

The setup is described by three files in docker/:

auth.txt

The machines the conductor accepts, with their keys.

switch.txt

The access list: agent1 is 10.0.1.1, agent2 is 10.0.1.2, the two may talk to each other, and each may reach 8.8.8.8.

docker-compose.yml

The six containers and their environment.

Look at how the first agent joined:

$ docker logs vnet-agent
[TLS] Enabled, peer certificates required, role in OU checked
[Agent] Connecting to conductor vnet-conductor:5000 ...
[Agent] Assigned to switch switch2 @ 172.18.0.4:6001
[Agent] Authenticated with switch switch2, virtual IP: 10.0.1.1
[TUN] Interface vnet-agent1 up with IP 10.0.1.1/32
[Agent] Entering event loop.

The agent asked the conductor for a switch, received one together with a one-time token, presented the token to the switch and got its virtual address. How a machine joins explains why it happens in that order. Your switch names and addresses may differ: the conductor hands out switches in turn.

Note

The conductor and switch logs show [TLS] handshake failed every few seconds. Those are Docker’s health checks, which open a TCP connection to check the port and close it without speaking TLS. They are harmless. The sysctl ... Read-only file system lines of the switches are harmless too: Compose sets those values from outside the container.

3. Send traffic through the overlay

The two agents may talk to each other, so a ping crosses the overlay, from agent1 on one switch to agent2 on another:

$ docker exec vnet-agent ping -c 3 10.0.1.2
64 bytes from 10.0.1.2: icmp_seq=1 ttl=64 time=0.782 ms
64 bytes from 10.0.1.2: icmp_seq=2 ttl=64 time=1.32 ms
64 bytes from 10.0.1.2: icmp_seq=3 ttl=64 time=1.27 ms

The agent replaced the container’s default route with its TUN interface, so every IPv4 packet of the container now goes through the switch:

$ docker exec vnet-agent ip route
default dev vnet-agent1 scope link
10.0.0.0/8 dev vnet-agent1 scope link
172.18.0.0/16 dev eth0 proto kernel scope link src 172.18.0.7

8.8.8.8 is allowed, so the switch forwards it to the internet; whether the reply arrives depends on whether your machine can reach that address. 1.1.1.1 is not in the access list, so the switch drops it:

$ docker exec vnet-agent ping -c 2 -W 2 1.1.1.1
2 packets transmitted, 0 received, 100% packet loss, time 1028ms

Ask the switch what it did. SIGUSR1 prints its counters and one line per agent:

$ docker kill -s USR1 vnet-switch2
$ docker logs --since 5s vnet-switch2 2>&1 | grep -v TLS
[Switch] Stats: agents=1 routes=1 local=3 to-switch=3 to-internet=2 waited-for-route=0 dropped=4 acl=4 link-full-drops=0 tun-write-errors=0 peers=2 pending-tokens=0 conductor=up
[Switch] Agents:
agent1 10.0.1.1 fd=7 in=7/588B out=3/252B dropped=2

acl=4 counts the packets the access list refused. Log messages explains every counter.

4. Change a rule while everything runs

Allow agent1 to reach 1.1.1.1 and reload the conductor:

$ echo 'agent1  ::internet:1.1.1.1' >> switch.txt
$ docker kill -s HUP vnet-conductor
$ docker logs --since 5s vnet-conductor 2>&1 | grep -v TLS
[Conductor] Auth file reloaded: 5 record(s)
[Config] Loaded 2 agent(s), 2 with agent ACLs, 2 with internet ACLs
[Conductor] Config reloaded, fingerprint fee95658ab07; sent to 3 switch(es)
[Conductor] Switch switch2 applied config fee95658ab07
[Conductor] Switch switch3 applied config fee95658ab07
[Conductor] Switch switch1 applied config fee95658ab07

The conductor sent the new access list to every switch, and each switch confirmed it with the same fingerprint. The rule applies to the next packet; no machine reconnected.

Note

Append to the file, or edit it in place. Compose mounts switch.txt as a single file, and an editor that saves by writing a new file and renaming it leaves the container looking at the old one.

5. See who is connected

The conductor writes its state to a file, and vnet-state prints it. vnet-state is not in the container images, so build it once on your machine (this needs cmake, g++, protobuf and libssl development packages):

$ cmake -S .. -B ../build -DCMAKE_BUILD_TYPE=Release
$ cmake --build ../build --target vnet-state
$ docker kill -s USR1 vnet-conductor
$ docker cp vnet-conductor:/var/vnet-dump.bin .
$ ../build/vnet-state vnet-dump.bin auth.txt
dump written 2026-11-11 18:53:32

switches (3)
  switch3  172.18.0.3:6002  network=::internet  since 2026-11-11 18:52:43
  switch2  172.18.0.4:6001  network=::internet  since 2026-11-11 18:52:43
  switch1  172.18.0.5:6000  network=::internet  since 2026-11-11 18:52:43

agents (2)
  agent1  10.0.1.1  on switch2  since 2026-11-11 18:52:48
  agent2  10.0.1.2  on switch3  since 2026-11-11 18:52:48

expected but not connected (0)

With the auth file as second argument, vnet-state also lists the machines that should be connected and are not. During an event, that list is the first thing to look at.

6. Lose a switch

Stop the switch that holds agent2 (switch3 above; use yours):

$ docker stop vnet-switch3
$ docker logs --since 10s vnet-agent2
[Agent] Connection closed: fd=5 (switch)
[Agent] Switch link lost; joining again
[Agent] Connection closed: fd=4 (conductor)
[Agent] Connecting to conductor vnet-conductor:5000 ...
[Agent] Assigned to switch switch1 @ 172.18.0.5:6000
[Agent] Authenticated with switch switch1, virtual IP: 10.0.1.2
[Agent] Entering event loop.
$ docker exec vnet-agent ping -c 2 10.0.1.2
2 packets transmitted, 2 received, 0% packet loss, time 1030ms

The agent asked the conductor for another switch and kept its address. Start the switch again with docker start vnet-switch3.

7. Lose the conductor

$ docker stop vnet-conductor
$ docker exec vnet-agent ping -c 2 10.0.1.2
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
$ docker start vnet-conductor
$ docker logs --since 10s vnet-conductor 2>&1 | grep -v TLS
...
[Conductor] Previous dump had 3 switch(es) and 2 agent(s)
[Conductor] Listening on port 5000
[Conductor] Switch registered: switch3 @ 172.18.0.3:6002 (network=::internet)
[Conductor] Switch registered: switch1 @ 172.18.0.5:6000 (network=::internet)
[Conductor] Agent agent2 registered with IP 10.0.1.2 on switch switch1
...
[Conductor] Agent reconnected: agent1 on switch2
[Conductor] Agent reconnected: agent2 on switch1

Machines kept talking while the conductor was down. When it came back, the switches registered again and told it which machines they hold, so it rebuilt its state without any action from you. Failures and recovery describes every failure case.

8. Clean up

$ docker compose down

Next steps