[Work In Progress] Lorax - reliable communication with broadcast messages

.github/workflows/distribute.yml

@@ -13,12 +13,14 @@ jobs:
 
     steps:
     - uses: actions/checkout@v2
-    - name: Autotools autoreconf 
+    - name: Autotools autoreconf
       run: ./autogen.sh
     - name: configure
       run: ./configure
     - name: make
       run: make
+    - name: interfaces
+      run: ip address
     - name: make check
       run: make check
     - name: make distcheck

.gitignore

@@ -51,8 +51,24 @@ src/Makefile
 config.h
 tags
 e32
+e32-*\.*\.\.
+lorax
+lorax_message
+lorax_control
+lorax_echo
+src/*.gdb
+*.log
+*.trs
 \#*#
 \.\#*
 .vscode
+test/test_list
+test/test_message
+test/test_misc
+test/test_options
+test/test_options_lorax
+test/test_packet
+test/test_control
+test/test_convert
 
-/e32-*.tar.gz
+/e32-*.tar.gz

README.md

@@ -1,14 +1,29 @@
 # EByte E32 SX1276 Software for the Raspberry Pi
 
-See this [Blog Post](https://lloydrochester.com/post/hardware/e32-sx1276-lora/) for details.
+This software interacts with the EByte E32 Lora Transceiver. We can interact with it using two methods:
+1. Low Level - writing unstructured bytes to sockets that are sent and received directly to the e32 via the UART
+2. High Level - uses message structures over sockets to implement reliable communications over the low level interface
+
+# Low Level Interface
+
+See this [Blog Post](https://lloydrochester.com/post/hardware/ebyte-e32-lora-getting-started/) for details.
 
 This repository contains the source code, as well as, the source code to distribute the tool which requres GNU Autotools to build. If you just want to run the tool I recommend just getting the tarball below where you can build from source.
 
 This code has also been run on a Pine64 and Orange Pi Zero.
 
+# High Level Interface - Lorax
+
+The high level interface is called lorax. These goals of this high level interface is to send structured data over Lora that is reliable. This data is reliable from the criteria that we have checksums for the packets that are sent and will retry when messages are not responded to. Here are the details of the lorax interface:
+
+1. Interaction over sockets uses a message structure similar to the Internet Protocol
+2. The message is processed and converted to a packet that is sent over the air with Lora
+3. For reliability we will retry failed packets we don't get responses for, or have failed checksums
+4. Broadcasts are sent out and neighbors are stored. This allows us to discover who our neighbors are
+
 # Getting Started
 
-We're going to assume you have 2 E32 Modules attached to two Raspberry PIs. Thus, one can transmit and the other receive and vice-versa. Details for each step in the [Blog Post](https://lloydrochester.com/post/hardware/e32-sx1276-lora/).
+We're going to assume you have 2 E32 Modules attached to two Raspberry PIs. Thus, one can transmit and the other receive and vice-versa. Details for each step in the [Blog Post](https://lloydrochester.com/post/hardware/ebyte-e32-lora-getting-started/).
 
 1. Wire up your E32 module. We require 3 pins. Two for the Mode pins and 1 for the Aux Pin. See section below to change wiring if needed.
 2. Using `raspi-config` configure your Serial Port, Unix groups and UART File Permissions.
@@ -19,8 +34,8 @@ We're going to assume you have 2 E32 Modules attached to two Raspberry PIs. Thus
 ## Install the `e32` command line tool and get status
 
 ```
-wget http://lloydrochester.com/code/e32-1.10.0.tar.gz
-tar zxf e32-1.10.0.tar.gz
+wget http://lloydrochester.com/code/e32-2.0.0.tar.gz
+tar zxf e32-2.0.0.tar.gz
 cd e32-1.10.0
 ./configure
 make
@@ -53,7 +68,7 @@ Run `e32` on both at the same time, no options are needed. In one terminal type
 
 # Advanced Features
 
-The tool offers more than just taking input from a keyboard. It's meant to run as a daemon and run in the background. If, however, you don't run it as a daemon you can send files and/or save to a file.
+The tool offers more than just taking input from a keyboard. It's meant to run as a daemon in the background. If, however, you don't run it as a daemon you can send files and/or save to a file.
 
 When running as a daemon communication to and from the `e32` is via Unix Domain Socket. This allows other tools written an any language to communicate wirelessly by just sending and receiving from a socket. See the blog post for an example in Python.
 

configure.ac

@@ -1,4 +1,4 @@
-AC_INIT([e32], [1.10.0], [[email protected]])
+AC_INIT([e32], [2.0.0], [[email protected]])
 AM_INIT_AUTOMAKE([-Wall -Werror foreign])
 
 AC_CONFIG_SRCDIR([src/config.h.in])

scripts/e32rx

@@ -22,7 +22,7 @@ parser.add_argument('--clientsock',
 args = parser.parse_args()
 
 # register a signal handler so when clean up the socket on ^C for instance
-def handler(signum):
+def handler(signum, frame):
     """ Handle signals by closing the socket """
     if signum == signal.SIGINT:
         close_sock()

src/Makefile.am

@@ -1,2 +1,6 @@
-bin_PROGRAMS = e32
-e32_SOURCES = main.c options.h options.c e32.h e32.c gpio.c gpio.h uart.h uart.c error.h error.c become_daemon.h become_daemon.c list.h list.c
+bin_PROGRAMS = e32 lorax lorax_control lorax_message lorax_echo
+e32_SOURCES = main.c options.h options.c e32.h e32.c gpio.c gpio.h uart.h uart.c error.h error.c become_daemon.h become_daemon.c list.h list.c socket.h socket.c misc.h misc.c
+lorax_SOURCES = main_lorax.c misc.h misc.c lorax.h lorax.c convert.h convert.c message.h message.c neighbor.h neighbor.c connection.h connection.c packet.h packet.c error.h error.c options_lorax.h options_lorax.c socket.h socket.c list.h list.c become_daemon.h become_daemon.c control.h control.c
+lorax_control_SOURCES = lorax_control.c error.h error.c control.h control.c socket.h socket.c misc.h misc.c list.h list.c
+lorax_message_SOURCES = lorax_message.c error.h error.c message.h message.c socket.h socket.c misc.h misc.c
+lorax_echo_SOURCES = lorax_echo.c error.h error.c message.h message.c socket.h socket.c misc.h misc.c become_daemon.h become_daemon.c

src/connection.c

@@ -0,0 +1,100 @@
+#include "connection.h"
+
+struct Connection*
+connection_make_uninitialzed(uint8_t state, uint8_t source_port, uint8_t destination_port, int (*message_match)(void *,void*), void (*message_destroy)(void *data))
+{
+    struct Connection* connection;
+    connection = malloc(sizeof(struct Connection));
+    memset(connection, 0, sizeof(struct Connection));
+    connection->connection_state = state;
+    clock_gettime(CLOCK_REALTIME, &connection->state_time);
+    connection->source_port = source_port;
+    connection->destination_port = destination_port;
+    connection->client = false;
+    connection->messages = malloc(sizeof(struct List));
+    list_init(connection->messages, message_match, message_destroy);
+
+    return connection;
+}
+
+int
+connection_invalid(uint8_t* con, size_t len)
+{
+    if(len < sizeof(struct Connection))
+        return 1;
+
+    return 0;
+}
+
+int
+connection_match(void *p1, void *p2)
+{
+    int t1, t2;
+    struct Connection *c1, *c2;
+    c1 = (struct Connection *) p1;
+    c2 = (struct Connection *) p2;
+
+    t1 = c1->source_port != c2->source_port;
+    if(t1)
+        return t1;
+
+    t2 = c1->destination_port != c2->destination_port;
+    if(t2)
+        return t2;
+
+    return 0;
+}
+
+void
+connection_destroy(void *data)
+{
+    struct Connection *connection;
+    connection = (struct Connection*) data;
+
+    if(connection->sock_client)
+        free(connection->sock_client);
+
+    list_destroy(connection->messages);
+    free(data);
+}
+
+void
+connection_print(struct Connection *connection)
+{
+    char fmt[] = "%s %d %d %d -> %d %s\n";
+    char *rfc8601, *client_sock;
+
+    client_sock = malloc(sizeof(struct sockaddr_un));
+
+    if(connection->client)
+        strncpy(client_sock, connection->sock_client->sun_path, sizeof(struct sockaddr_un));
+    else
+        sprintf(client_sock, "client_socket_undefined");
+
+    rfc8601 = rfc8601_timespec(&connection->state_time);
+    debug_output(fmt,
+        client_sock,
+        list_size(connection->messages),
+        connection->connection_state,
+        connection->source_port,
+        connection->destination_port,
+        rfc8601
+    );
+
+    free(client_sock);
+    free(rfc8601);
+}
+
+struct Connection*
+connection_lookup(struct List* connections, uint8_t source_port, uint8_t destination_port)
+{
+    struct Connection connection_needle;
+    if(connections == NULL)
+    {
+        return NULL;
+    }
+    connection_needle.source_port = source_port;
+    connection_needle.destination_port = destination_port;
+
+    return list_get(connections, &connection_needle);
+}

src/connection.h

@@ -0,0 +1,69 @@
+#ifndef CONNECTION_H
+#define CONNECTION_H
+
+#include <sys/socket.h>
+#include "error.h"
+#include "message.h"
+#include "socket.h"
+#include "list.h"
+#include "misc.h"
+
+/* A connection can be in two states:
+   1) waiting for a message
+   2) waiting for a packet
+   For example we create a new connection by receiving a
+   message to a neighbor. We will convert the message to
+   a packet and then send it off. We'll put this connection
+   in the waiting packet state since we're waiting on a packet
+   back from the destination.
+   On the other end if we receive a packet that with the destination
+   that matches our lorax we will send this packet to the server socket
+   and wait to hear back. Thus, we'd be waiting for a message back from
+   the server. Once we hear back from the server we'll put the connection
+   back in the state of waiting for a packet.
+   */
+enum STATE_CONNECTION
+{
+    STATE_WAITING_MESSAGE,
+    STATE_WAITING_PACKET
+};
+
+/*
+A connection holds the source and destination address and port.
+We create a connection when a message is sent to a neighbor, or,
+a packet with the destination address of our lorax.
+When created we will set the source to ourselves and the destination
+to the other end. However, when looking up a connection from a
+received packet we need to swap the source and destination as we
+hold the connection to other end.
+*/
+struct Connection
+{
+    enum STATE_CONNECTION connection_state;
+    struct sockaddr_un *sock_client;
+    struct timespec state_time;
+    uint8_t source_port;
+    uint8_t destination_port;
+    bool client;
+    struct List *messages;
+};
+
+struct Connection*
+connection_make_uninitialzed(uint8_t state, uint8_t source_port, uint8_t destination_port, int (*message_match)(void *,void*), void (*message_destroy)(void *data));
+
+int
+connection_invalid(uint8_t* con, size_t len);
+
+int
+connection_match(void *p1, void *p2);
+
+void
+connection_destroy(void *data);
+
+void
+connection_print(struct Connection *connection);
+
+struct Connection*
+connection_lookup(struct List* connections, uint8_t source_port, uint8_t destination_port);
+
+#endif

src/control.c

@@ -0,0 +1,52 @@
+#include "control.h"
+
+int
+control_request_invalid(uint8_t *control, size_t len)
+{
+    if(len != 1)
+        return 1;
+
+    switch(control[0])
+    {
+        case CONTROL_REQUEST_GET_NEIGHBORS:
+        case CONTROL_REQUEST_GET_MY_ADDRESS:
+            break;
+        default:
+            return 2;
+    }
+
+    return 0;
+}
+
+void
+control_get_neighbors(struct List *neighbors, uint8_t **neighbor_buffer, size_t *buffer_len)
+{
+    uint8_t num_neighbors;
+    uint8_t *buf_ptr;
+    struct Neighbor *neighbor;
+    num_neighbors = list_size(neighbors);
+    *buffer_len = num_neighbors*NEIGHBOR_ADDRESS_SIZE+3;
+    buf_ptr = malloc(*buffer_len);
+    *neighbor_buffer = buf_ptr;
+    buf_ptr[0] = CONTROL_RESPONSE_OK;
+    buf_ptr[1] = CONTROL_REQUEST_GET_NEIGHBORS;
+    buf_ptr[2] = num_neighbors;
+
+    for(int i=0; i<num_neighbors; i++)
+    {
+        neighbor = (struct Neighbor *) list_get_index(neighbors, i);
+        memcpy(buf_ptr+i*NEIGHBOR_ADDRESS_SIZE+3, neighbor->address, NEIGHBOR_ADDRESS_SIZE);
+    }
+}
+
+void
+control_get_my_address(uint8_t my_address[], uint8_t **neighbor_buffer, size_t *buffer_len)
+{
+    uint8_t *buf_ptr;
+    *buffer_len = NEIGHBOR_ADDRESS_SIZE+2;
+    buf_ptr = malloc(*buffer_len);
+    *neighbor_buffer = buf_ptr;
+    buf_ptr[0] = CONTROL_RESPONSE_OK;
+    buf_ptr[1] = CONTROL_REQUEST_GET_MY_ADDRESS;
+    memcpy(buf_ptr+2, my_address, NEIGHBOR_ADDRESS_SIZE);
+}

src/control.h

@@ -0,0 +1,24 @@
+#ifndef CONTROL_H
+#define CONTROL_H
+
+#include <stdint.h>
+#include <stdlib.h>
+#include "list.h"
+#include "neighbor.h"
+
+#define CONTROL_REQUEST_GET_NEIGHBORS 1
+#define CONTROL_REQUEST_GET_MY_ADDRESS 2
+
+#define CONTROL_RESPONSE_OK 0
+#define CONTROL_RESPONSE_ERROR 255
+
+int
+control_request_invalid(uint8_t *control, size_t len);
+
+void
+control_get_neighbors(struct List *neighbors, uint8_t **neighbor_buffer, size_t *buffer_len);
+
+void
+control_get_my_address(uint8_t my_address[], uint8_t **neighbor_buffer, size_t *buffer_len);
+
+#endif

src/convert.c

@@ -0,0 +1,27 @@
+#include "convert.h"
+
+int
+message_to_packet(struct Message *message, struct PacketHeader** packet)
+{
+    packet_make_uninitialized_packet(packet, message->data_length);
+    packet_make_partial(*packet, message->type, message->source_address, message->destination_address, message->source_port, message->destination_port);
+    memcpy(packet_get_data_pointer(*packet), message->data, message->data_length);
+    packet_compute_checksum(*packet, (*packet)->total_length);
+
+    return 0;
+}
+
+int
+packet_to_message(struct PacketHeader *packet, struct Message** message)
+{
+    uint8_t* data_ptr;
+    uint8_t data_size;
+
+    data_ptr = packet_get_data_pointer(packet);
+    data_size = packet_get_data_size(packet);
+
+    *message = message_make_uninitialized_message(data_ptr, data_size);
+    message_make_partial(*message, packet->type, packet->source_address, packet->destination_address, packet->source_port, packet->destination_port);
+
+    return 0;
+}