Skip to content

Latest commit

 

History

History
221 lines (150 loc) · 13.8 KB

README.md

File metadata and controls

221 lines (150 loc) · 13.8 KB
Raw

Real-time Map

Real-time Map displays real-time positions of public transport vehicles in Helsinki. It's a showcase for Proto.Actor - an ultra-fast distributed actors solution for Go, C#, and Java/Kotlin.

🔥 Check out the live demo! 🔥

The app features:

  • Real-time positions of vehicles.
  • Vehicle trails.
  • Geofencing notifications (vehicle entering and exiting the area).
  • Vehicles in geofencing areas per public transport company.
  • Horizontal scaling.

The goals of this app are:

  1. Showing what Proto.Actor can do.
  2. Presenting a semi-real-world use case of the distributed actor model.
  3. Aiding people in learning how to use Proto.Actor.

Find more about Proto.Actor here.

image

Running the app

Configure Mapbox:

  1. Create an account on Mapbox.
  2. Copy a token from: main dashboard / access tokens / default public token.
  3. Paste the token in Frontend\src\config.ts.

Start frontend (requires node.js 17.x):

cd Frontend
npm install
npm run serve

Start Backend:

cd Backend
dotnet run

The app is available on localhost:8080.

Also check out the proto.actor dashboard (alpha) by navigating to localhost:5000.

Kubernetes

In order to deploy to Kubernetes and use the Kubernetes cluster provider, see Deploying to Kubernetes

How does it work?

Prerequisites

To understand how this app works, it's highly recommended to understand the basics of the following technologies:

  1. .NET / C#
  2. ASP.NET
  3. SignalR

It would also be great, if you knew the basics the of actor model, Proto.Actor, and virtual actors (also called grains). If you don't you can try reading this document anyway: we'll try to explain the essential parts as we go.

Learn more about Proto.Actor here.

Learn more about virtual actors here.

Also, since this app aims to provide horizontal scalability, this document will assume, that we're running a cluster with two nodes.

One last note: this document is not a tutorial, but rather a documentation of this app. If you're learning Proto.Actor, you'll benefit the most by jumping between code and this document.

Data source

Since this app is all about tracking vehicles, we need to get their positions from somewhere. In this app, the positions are received from high-frequency vehicle positioning MQTT broker from Helsinki Region Transport. More info on data:

This data is licensed under © Helsinki Region Transport 2021, Creative Commons BY 4.0 International

In our app, Ingress is a hosted service responsible for subscribing to Helsinki Region Transport MQTT server and handling vehicle position updates.

Vehicles

When creating a system with actors, is common to model real-world physical objects as actors. We'll start by modelling a vehicle since all the features depend on it. It will be implemented as a virtual actor (VehicleActor). It will be responsible for handling events related to that vehicle and remembering its state, e.g. its current position and position history.

Quick info on virtual actors:

  1. Each virtual actor is of a specific kind, in this case, it's Vehicle.
  2. Each virtual actor has an ID, in this case, it's an actual vehicle's number.
  3. Virtual actors can have state, in this case, vehicle's current position and position history.
  4. Virtual actors handle messages sent to them one by one, meaning we don't have to worry about synchronization.
  5. Virtual actors are distributed in the cluster. In normal circumstances, a single virtual actor (in this case a specific vehicle) will be hosted on a single node.
  6. We don't need to spawn (create) virtual actors explicitly. They will be spawned automatically on one of the nodes the first time a message is sent to them.
  7. To communicate with a virtual actor we only need to know its kind and ID. We don't have to care about which node it's hosted on.

Note: virtual actors are sometimes referred to as "grains" - terminology originating in the MS Orleans project.

The workflow looks like this:

  1. Ingress receives an event from Helsinki Region Transport MQTT server.
  2. Ingress reads vehicle's ID and its new position from the event and sends it to a vehicle in question.
  3. VehicleActor processes the message.

vehicles only in a cluster drawio

Notice, that in the above diagram, vehicles (virtual actors) are distributed between two nodes, however, Ingress is present in both of the nodes.

Organizations

Let's consider the following feature: in this app, each vehicle belongs to an organization. Each organization has a specified list of geofences. In our case, a geofence is simply a circular area somewhere in Helsinki, e.g. airport, railway square, or downtown. Users should receive notifications when a vehicle enters or leaves a geofence (from that vehicle's organization).

For that purpose, we'll implement a virtual actor to model an organization (OrganizationActor). When activated, OrganizationActor will spawn (create) a child actor for each configured geofence (GeofenceActor).

Quick info on child actors:

  1. In contrast to virtual actors, we need to manually spawn a child actor.
  2. Child actor is hosted on the same node as the virtual actor that spawned it. Communication in the same node is quite fast, as neither serialization nor remote calls are needed.
  3. After spawning a child actor, its PID is stored in the parent actor. Think of it as a reference to an actor: you can use it to communicate with it.
  4. Since parent actor has PIDs of their child actors, it can easily communicate with them.
  5. Technically, we can communicate with a child actor using their PID from anywhere within the cluster. This functionality is not utilized in this app, though.
  6. Child actor lifecycle is bound to the parent that spawned it. It will be stopped when parent gets stopped.

The workflow looks like this:

  1. VehicleActor receives a position update.
  2. VehicleActor forwards position update to its organization.
  3. OrganizationActor forwards position update to all its geofences.
  4. Each GeofenceActor keeps track of which vehicles are already inside the zone. Thanks to this, GeofenceActor can detect if a vehicle entered or left the geofence.

organization geofences only drawio

Viewport, positions and notifications

So far we've only sent messages to and between actors. However, if we want to send notifications to actual users, we need to find a way to communicate between the actor system and the outside world. In this case, we'll use:

  1. SignalR to push notifications to users.
  2. Proto.Actor's (experimental) pub-sub feature to broadcast positions cluster-wide.

Learn more about PubSub here.

First we'll introduce the UserActor. It models the user's ability to view positions of vehicles and geofencing notifications. UserActor will be implemented as an actor (i.e. non-virtual actor).

Quick info on actors:

  1. An actor's lifecycle is not managed, meaning we have to manually spawn and stop them.
  2. An actor will be hosted on the same node that spawned it. Like with child actors, this gives us performance benefits when communicating with an actor. It also enables the actor to manage a local resource (SignalR connection).
  3. Since an actor lives in the same node (i.e. the same process), we can pass .NET references to it. It will come in handy in this feature.
  4. After spawning an actor, we receive its PID. Think of it as a reference to an actor: you can use it to communicate with it. Don't lose it!
  5. Technically, we can communicate with an actor using their PID from anywhere within the cluster. This functionality is not utilized in this app, though.

The workflow looks like this:

  1. When user connects to SignalR hub, user actor is spawned to represent the connection. Delegates to send the positions and notifications back to the user are provided to the actor upon creation.
  2. When starting, the user actor subscribes to Position and Notification events being broadcasted through the Pub-Sub mechanism. It also makes sure to unsubscribe when stopping.
  3. When user pans the map, updates of the viewport (south-west and north-east coords of the visible area on the map) are sent to the user actor through SignalR connection. The actor keeps track of the viewport to filter out positions.
  4. Geofence actor detects vehicle entering or leaving the geofencing zone. These events are broadcasted as Notification message to all cluster members and available through PubSub. Same goes for Position events broadcasted from vehicle actor.
  5. When receiving a Notification message, user actor will push it to the user via SignalR connection. It will do the same with Position message provided it is within currently visible area on the map. The positions are sent in batches to improve performance.

viewport and pub sub drawio

Map grid

In the Realtime map sample we also showcase an optimization, which is possible thanks to the virtual actors and the pub-sub mechanism. Instead of broadcasting all positions to all users, we can limit the amount of positions sent to just the ones a particular user has interest in. For this purpose we overlay a "virtual grid" on top of the map.

map grid drawio

Each cell in the map becomes a topic in the pub sub system. Topics are implemented as virtual actors so we don't need to create them in an explicit way. We can just start sending messages to them. In the example above we would have a topic for grid cell A1, B1, etc.

Depending on the size and position of the user's viewport, it covers some of the grid cells. This means the user actor needs to subscribe to topics corresponding to the covered grid cells. In the example above, the user is subscribed to topics A1, B1, A2 and B2. When the viewport moves to cover different set of grid cells, the topics corresponding to grid cells that are no longer relevant need to be unsubscribed.

On publishing side, the vehicle calculates what topic to publish to based on current position. Only the topic corresponding to the grid cell the vehicle is currently in receives the position.

This way we're removing a bottleneck in the system that would otherwise force us to send all the messages to all the users, which is not a scalable approach. You could argue, that if the viewport covers all the grid cells, the problem occurs anyway. This is true, but we could prevent users from doing so, for example by stopping the real time updates when the viewport is over some size threshold.

Getting vehicles currently in an organization's geofences

Let's consider the following feature: when a user requests it, we want to list all vehicles currently present in a selected organization's geofences.

This one is quite easy, as actors support request/response pattern.

The workflow looks like that:

  1. A user calls API to get organization's details (including geofences and vehicles in them).
  2. OrganizationController asks OrganizationActor for the details.
  3. OrganizationActor asks each of its geofences (GeofenceActor) for a list of vehicles in these geofences.
  4. Each GeofenceActor returns that information.
  5. OrganizationActor combines that information into a response and returns it to OrganizationController.
  6. OrganizationController maps and returns the results to the user.

getting vehicles in the org drawio

Deploying to Kubernetes

Prerequisites:

The Realtime Map sample can be configured to use Kubernetes as cluster provider. The attached chart contains definition for the deployment. You will need to supply some additional configuration, so create a new values file, e.g. my-values.yaml in the root directory of the sample. Example contents:

frontend:
  config:
    mapboxToken: SOME_TOKEN # provide your MapBox token here

backend:
  config:
    # since all backend pods need to share MQTT subscription, 
    # generate a new guid and provide it here (no dashes)
    RealtimeMap__SharedSubscriptionGroupName: SOME_GUID
    ProtoActor__PubSub__RedisConnectionString: realtimemap-redis-master:6379

ingress:
  # specify localhost if on Docker Desktop, 
  # otherwise add a domain for the sample to your DNS and specify it here
  host: localhost

Create the namespace and deploy the sample:

helm upgrade --install -f my-values.yml --namespace realtimemap --create-namespace realtimemap ./chart

NOTE: the chart creates a new role in the cluster with permissions to access Kubernetes API required for the cluster provider.

The above config will deploy the sample without TLS on the ingress, additional configuration is required to secure the connection.