vnet::netqueue

namespace netqueue

Enums

enum FiniteStateMachine

This finite state machine represents the state in which the file descriptor is currently.

In particular, there are two distinct finite state machines depending on the type of socket.

Anything in TUN_* represents a device that operates as a queue of packets, under which reading yields the entire IP packet directly and on which you should only write a entire IP packet.

Anything in SCK_* represents a device that follows the specific protobuf format in the specifications. In particular, it contains first a 6 byte header containing the payload size and payload type.

Values:

enumerator TUN_RECEIVE

enumerator TUN_READY

enumerator TUN_ERROR

enumerator TUN_EOF

enumerator SCK_HEADER

enumerator SCK_PAYLOAD

enumerator SCK_READY

enumerator SCK_ERROR

enumerator SCK_EOF

enumerator SCK_CONNECTING

enumerator SCK_TLS_ACCEPT

enumerator SCK_TLS_CONNECT

enum close_reason

Values:

enumerator CLOSE_ERROR

enumerator CLOSE_EPOLL

enumerator CLOSE_HANGUP

Functions

void doNothingOnClose(void *ptr_data, close_data close_content)

void doNothingOnSocketReady(void *ptr_data, socket_data data)

void doNothingOnTunReady(void *ptr_data, tun_data data)

void doNothingOnConnected(void *ptr_data, socket_data data)

Variables

static size_t WRITE_BUFFER_MAX = 2 * 1024 * 1024

const int EPOLL_MAX_EVENTS = 256

Maximum number of events returned by a single call to epoll_wait.

A switch with hundreds of agents gets most of them in one turn.

const int PACKETS_PER_TURN = 64

Packets one element may deliver in one turn of wait_and_process.

A socket that is fed without pause is put down after this many and taken up again next turn, so it cannot starve the other elements.

struct NetworkElement

#include <element.hpp>

Public Functions

NetworkElement(int fd, void *ptr, FiniteStateMachine state)

Public Members

FiniteStateMachine state

int fd

void *ptr

size_t net_buffer_used

uint8_t net_buffer[NET_BUFFER_SIZE]

std::vector<uint8_t> write_buffer

size_t write_buffer_offset = 0

bool dead = false

ssl_st *ssl = nullptr

bool want_write = false

bool has_backlog = false

uint64_t last_turn = 0

struct close_data

#include <handler.hpp>

Public Members

NetworkElement *net_element

void *ptr_data

int fd

close_reason reason

struct socket_data

#include <handler.hpp>

Public Members

NetworkElement *net_element

void *ptr_data

int fd

protocol::PacketType packet_type

unsigned char *packet_buffer

size_t payload_size

unsigned char *full_buffer

size_t full_buffer_size

struct tun_data

#include <handler.hpp>

Public Members

NetworkElement *net_element

void *ptr_data

int fd

unsigned char *ip_buffer

size_t ip_buffer_size

struct NetworkQueueHandler

#include <handler.hpp>

Public Members

void (*onClose)(void *ptr_data, close_data close_content) = &doNothingOnClose

void (*onSocketReady)(void *ptr_data, socket_data data) = &doNothingOnSocketReady

void (*onTunReady)(void *ptr_data, tun_data data) = &doNothingOnTunReady

void (*onConnected)(void *ptr_data, socket_data data) = &doNothingOnConnected

void *ptr_data = nullptr

struct NetQueue

#include <netqueue.hpp>

Instances of this class represents a general purpose pool of file descriptors intended to represent a queue for incoming network packets.

The process must ignore SIGPIPE: a write to a peer that hung up, including the TLS alert OpenSSL sends on a failed handshake, is reported as an error on that socket instead of ending the process.

Public Functions

bool send(int fd, const void *data, size_t n)

Send data to a file descriptor managed by this queue.

If the kernel send buffer is full (EAGAIN), the remaining bytes are saved in the element’s write buffer and sent when epoll signals the fd is writable (EPOLLOUT).

If the write buffer exceeds WRITE_BUFFER_MAX the connection is closed and onClose is fired.

Returns:

true if all bytes were sent or buffered successfully. false if the fd is not in the queue or the buffer limit was exceeded (connection closed).

bool send(int fd, protocol::PacketType type, const google::protobuf::Message &msg)

Send a protobuf message with the 6-byte packet header through the queue’s write path, as one write.

bool send_data(int fd, protocol::PacketType type, const void *data, size_t n)

Send a data packet: a 6-byte header of the given type followed by the bytes as they are.

When the peer’s write buffer is full the packet is dropped and false is returned; the connection stays open, since the overlay’s own transport retransmits.

bool send_heartbeat(int fd)

Send a heartbeat (header-only) through the queue’s write path.

NetworkElement *put(int fd, void *ptr, FiniteStateMachine start)

Put a file descriptor into the queue with a pointer to some data the user wants to store.

The user GIVES OWNERSHIP of the file descriptor to the network queue. One should never close it manually. If the user wishes to close the file descriptor, it should use the close method of the network queue.

The user still HAS OWNERSHIP over the pointer of the data it wanted to store and will be responsible for freeing it or handling it upon receival of an error or a close through the NetworkQueueHandler.

The user DOES NOT HAVE OWNERSHIP over the resulting network element and should in no case free it, close the internal file descriptor or manipulate the state, buffer and number of bytes used in the buffer.

The user MAY modify the network’s element internal pointer to some new data it wants to use to handle for example a client reconnecting.

Returns:

the generated network element. If anything fails whilst trying to put the file descriptor, the method will return NULL

NetworkElement *get_network_element_from_fd(int fd)

Get the network element for the given file descriptor.

Parameters:

fd – the file descriptor for the lookup

Returns:

the network element for that given file descriptor, or NULL if the file descriptor isn’t inside the queue.

void close(int fd)

Close the current file descriptor and remove it from the queue.

void close(NetworkElement *element, close_reason reason = CLOSE_ERROR)

Close the current network element and remove it from the queue.

Fires onClose once, closes the file descriptor, and frees the element after the current epoll batch.

void process(NetworkElement *element)

Take a network element (that is an object containing the file descriptor, data of the element), and read from it until it outputs EAGAIN.

The element must be owned by the network queue

Parameters:

element – the network element to be processed

void wait_and_process()

Calls epoll_wait to get all the available events and their respective network elements, before calling process on them.

NetworkElement *put_tun(int fd, void *ptr)

Put a TUN file descriptor into the queue with a pointer to some data the user wants to store.

The user GIVES OWNERSHIP of the file descriptor to the network queue. One should never close it manually. If the user wishes to close the file descriptor, it should use the close method of the network queue.

The user still HAS OWNERSHIP over the pointer of the data it wanted to store and will be responsible for freeing it or handling it upon receival of an error or a close through the NetworkQueueHandler.

The user DOES NOT HAVE OWNERSHIP over the resulting network element and should in no case free it, close the internal file descriptor or manipulate the state, buffer and number of bytes used in the buffer.

The user MAY modify the network’s element internal pointer to some new data it wants to use to handle for example a client reconnecting.

Returns:

the generated network element. If anything fails whilst trying to put the file descriptor, the method will return NULL

NetworkElement *put_sck(int fd, void *ptr)

Put a file descriptor for a socket into the queue with a pointer to some data the user wants to store.

The user GIVES OWNERSHIP of the file descriptor to the network queue. One should never close it manually. If the user wishes to close the file descriptor, it should use the close method of the network queue.

The user still HAS OWNERSHIP over the pointer of the data it wanted to store and will be responsible for freeing it or handling it upon receival of an error or a close through the NetworkQueueHandler.

The user DOES NOT HAVE OWNERSHIP over the resulting network element and should in no case free it, close the internal file descriptor or manipulate the state, buffer and number of bytes used in the buffer.

The user MAY modify the network’s element internal pointer to some new data it wants to use to handle for example a client reconnecting.

Returns:

the generated network element. If anything fails whilst trying to put the file descriptor, the method will return NULL

NetworkElement *put_sck_connecting(int fd, void *ptr, ssl_st *ssl)

Put a socket whose connect() is in progress (EINPROGRESS) or just completed.

When the connection is up, the TLS handshake runs on it if ssl is given (a client session from tls::new_session), and onConnected fires once packets can flow. The queue owns ssl from here on.

NetworkElement *put_sck_tls_server(int fd, void *ptr, ssl_st *ssl)

Put an accepted socket that must complete a TLS handshake as the server before any packet is read.

ssl is a fresh session from tls::new_session; the queue owns it. onConnected fires when the handshake is done.

NetworkElement *put_sck_tls_client(int fd, void *ptr, ssl_st *ssl)

Put a connected socket that must complete a TLS handshake as the client before any packet is read.

Same ownership and callback as put_sck_tls_server.

inline void adopt_tls(int fd, ssl_st *ssl)

Attach a finished TLS session to a socket already in the queue.

Later reads and writes go through SSL_read / SSL_write.

void drain_tls(int fd)

Handshake can pull the first application record into OpenSSL.

Epoll will not see those bytes, so read them now.

NetQueue(NetworkQueueHandler handler)

Create a network queue with the given handler for the events.

Returns:

the network queue created. If anything fails, it will return NULL.

NetQueue(NetworkQueueHandler handler, int epoll_timeout)

Create a network queue with the given handler for the events and given wait timeout.

Returns:

the network queue created. If anything fails, it will return NULL.

~NetQueue()

Destroy the network queue.

This will free all of the network elements and cleanup the EPOLL. This won’t handle destruction of the user data.