Part 1 - Project Setup and First Steps with Agents

Introduction

In this segment, we'll lay the foundation for Nectar. We'll kick things off with the initialization of a basic Elixir mix project, then segue into some essential theoretical concepts. As we progress, we'll harness these theories to develop the PIR sensor module. By the end, we'll leverage IEX to evaluate our app's current functionality, allowing us to both read and modify the state of a PIR sensor.

As we begin with this device manager, the PIR sensor module, it's important to note that the architectural principles being introduced are universal. Using a similar approach, we'll later accommodate other devices, such as the lamp and the camera. By focusing on one device manager initially, we can streamline the presentation of subsequent managers.

Processes and Agents

Process

When we talk about concurrency in Elixir & OTP, processes are invariably the first topic that springs to mind. Processes in this context are not operating system processes; rather, they are lightweight threads of execution managed by the Erlang Virtual Machine.

One of the immediate benefits of using processes, especially in the early stages of our IoT backend's development, is their ability to hold state. For now, we'll leverage processes to manage and store the state of our application.

Processes are fundamental to Elixir applications, especially due to their capability to encapsulate state. In the context of our IoT backend's development, we're leveraging these processes as the primary state holders.

While processes guard our state, it's crucial to note that in our initial implementation, the state will not be saved to a persistent storage. Our design, focusing on the fundamental functions of our IoT gateway, retains the flexibility to add such storage as needed.

Here are some key attributes and advantages of processes:

Self-contained State Management: Each process manages its own state. This implies that every piece of data associated with a process is stored within that process, eliminating the need for shared memory.

Independence and Efficiency: Processes operate independently. They don't share memory, which makes them resilient. Moreover, their lightweight nature means that thousands, even millions, can run concurrently without bogging down the system.

Scheduling by the VM: The Erlang VM isn't just a passive entity; it actively schedules these processes, ensuring smooth operations without manual interference.

Garbage Collection Perks: Unlike many other languages that use a 'stop-the-world' approach to garbage collection, affecting all threads and often leading to performance lags, the Erlang VM performs garbage collection per process. This localized approach drastically reduces the observable effects and system pauses.

Latency Reduction: By bypassing frequent database queries, processes facilitate quicker response times. This will be especially beneficial in our IoT gateway, where timely responses are paramount.

Communication via Messages: Processes don't communicate directly. Instead, they send and receive messages through a mailbox mechanism. These messages are processed asynchronously, in the order they arrive, maintaining the integrity of data flow.

As we conclude this section, it's apt to introduce the next topic that we'll be delving into: Agents. While processes provide a foundation, Agents, in Elixir, offer a higher-level abstraction to manage state. They simplify the process-centric approach, making state management more intuitive.

Agents

Elixir's concurrency model, built on the robust shoulders of the Erlang Virtual Machine, introduces the notion of processes. When we talk about state management in these processes, the straightforward path would be to implement our own mailbox-reading loops to handle incoming messages, should there be no feature like Agents in place.

Thankfully, Elixir provides us with Agents, which act as a simple abstraction over stateful processes. Their introduction into the ecosystem has been a game-changer for several reasons:

Avoiding Boilerplate: Without Agents, developers would end up writing repetitive boilerplate code to manage state within processes. Agents streamline this, offering a well-tested, uniform approach to managing the state.

State Management: Agents simplify the act of starting a process in a particular state and provide functionalities to query or update this state. This makes it easier to maintain and manipulate the process state without getting lost in the complexity.

Process Identifier: All functions that query or update the state of an agent require a PID (Process Identifier) as their first parameter. This PID acts as an address for the process, ensuring the right messages reach the right destinations.

Flexibility in State Definition: The beauty of Agents is the flexibility they offer in defining state. Any Elixir data type can be used as a state, which means you can mold it to fit your specific requirements.

Agents Relationships: One of the features of Agents is their ability to hold references to other agents, allowing for nested relationships. This capability provides a way to model intricate relationships between different entities in an application, laying the foundation for a more sophisticated system where agents form the core building blocks.

Setting Up a Basic Elixir Mix Project

Erlang and Elixir setup

If you haven't set your environment yet to work with Elixir: https://elixir-lang.org/install.html

I personally prefer the asdf because it let me switch different versions.

This guide is mainly based on Linux, but asdf can be also installed in macOS and in Windows but through WSL (Subsystem for Linux)

This a brief guide to do it,

$ git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch v0.13.1
Cloning into '/home/username/.asdf'...

$ asdf --version
v0.13.1-0586b37

$ asdf plugin-add erlang https://github.com/asdf-vm/asdf-erlang.git
$ asdf plugin-add elixir https://github.com/asdf-vm/asdf-elixir.git

$ asdf list-all erlang
...
26.0.1  
26.0.2  
26.1  
26.1.1

$ asdf install erlang 26.1.1
Downloading 26.1.1 to /home/adrian/.asdf/downloads/erlang/26.1.1...
Erlang/OTP 26.1.1 (asdf_26.1.1) has been successfully built  
  
Cleaning up compilation products for 26.1.1  
Cleaned up compilation products for 26.1.1 under /home/username/.asdf/plugins/erlang/kerl-home/builds

$ asdf list-all elixir
1.15.6  
1.15.6-otp-24  
1.15.6-otp-25  
1.15.6-otp-26  
main

$ asdf install elixir 1.15.6-otp-26
==> Checking whether specified Elixir release exists...  
==> Downloading 1.15.6-otp-26 to /home/adrian/.asdf/downloads/elixir/1.15.6-otp-26/elixir-precompiled-1.15.6-otp-26.zip

$ erl
erlang 26.1.1

$ elixir --version
elixir 1.15.6-otp-26

$ asdf global erlang 26.1.1
$ asdf global elixir 1.15.6-otp-26

Note that some command outputs were omitted for clarity.

Project Init

Initialize your project using the Mix build tool:

In your projects directory

$ mix new nectar --sup

This `--sup` flag indicates that we want to generate a project that includes a supervision tree. Let's put a pin in that for now; however, as a teaser, know that a supervision tree in Elixir ensures fault tolerance by monitoring and restarting processes as needed.

You should get something like this

$ mix new nectar --sup
* creating README.md
* creating .formatter.exs
* creating .gitignore
* creating mix.exs
* creating lib
* creating lib/nectar.ex
* creating lib/nectar/application.ex
* creating test
* creating test/test_helper.exs
* creating test/nectar_test.exs

Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more:

    cd nectar
    mix test

Run "mix help" for more commands.

Navigate to the project directory

$ git init
Initialized empty Git repository in /home/username/Projects/nectar/.git/

$ git add .
$ git commit -m "Initial commit"
$ git remote add origin git@username/nectar.git
$ git push origin main

Let's start working in a new branch

$ git branch feature/agents

Starting with the PIR Sensor Module

Base structures

First of all, let's draft the initial structure for the modules, we are going to work with the PIR sensor module first but is a good idea to know a bit about where we are going instead of throwing pieces of code with having a general overview of the organization.

This represents the desired initial structure of our project:

nectar/
├── lib/
│   └── nectar/
│       ├── application.ex
│       ├── device/
│       │   └── device.ex
│       ├── pir_sensor/
│       │   ├── pir_sensor.ex
│       │   └── agent.ex
│       ├── lamp/
│       │   ├── lamp.ex
│       │   └── agent.ex
│       ├── camera/
│       │   ├── camera.ex
│       │   └── agent.ex
│       ├── types/
│       │   └── geo_coordinate.ex
├── test/
└── ...

So basically all of our device messages will have a set of common properties, this is why, as you can see in the tree above, we defined a device module. This device module defines a timestamp property and geocoordinate that is defined also as a struct in the Types module.

Having gained a bird's-eye view of the project, let's now adopt a bottom-up approach and begin coding the coordinate struct in the Types module.

In the Nectar.Types.Types.GeoCoordinate module, we have a structure that acts as a container for the position of smart devices in the world:

defmodule Nectar.Types.GeoCoordinate do
  @type t :: %{
    lat: float,
    lng: float,
    alt: float
  }

  defstruct [:lat, :lng, :alt]
end

Understanding a device's position is crucial for security. Ensuring pairing only occurs when devices, like phones or tablets, are nearby helps prevent potential threats. If a smart device tries to pair with a phone that isn't in close proximity, it might indicate suspicious activity.

In Elixir, the @type module attribute helps define custom types and type specifications. These not only clarify the expected data structures but also guide tools like Dialyzer for static type checking.

To elaborate a bit, while "type" refers to fundamental data categories, "type specification" describes the structure and expected types within more intricate data constructs. We'll delve deeper into its technicalities later on.

Next, let's shift our focus to DeviceInfo. Much like our previous code, its purpose and functionality should be largely self-evident.

defmodule Nectar.DeviceInfo do
  @type geo_coordinate :: Nectar.Types.geo_coordinate
  @type current_time :: integer

  defstruct [:geo_coordinate, :current_time]
end

The Nectar.DeviceInfo module is defined with two key fields: geo_coordinate and current_time.

• geo_coordinate: This field holds the geographic coordinates of a device's location, typically encompassing three main values: latitude, longitude, and altitude.
• current_time: Representing the device's current time, this field signifies the timestamp of when the device's data was last recorded or updated. Stored as an integer, it's typically in the Unix timestamp format representing the elapsed seconds since January 1, 1970. Such a timestamp will be important for tasks like synchronization and event logging.

Let's define now the PirSensor module following the same approach.

defmodule Nectar.PIRSensor do
  alias Nectar.Types.GeoCoordinate
  alias Nectar.DeviceInfo

  defstruct [:movement_detected, :tamper_detected, :device_info]

  def new do
    %__MODULE__{
      movement_detected: false,
      tamper_detected: false,
      device_info: %DeviceInfo{
        geo_coordinate: %GeoCoordinate{
          lat: 50.09431362821687,
          lng: 14.449474425948397,
          alt: 0
        },
        current_time: :os.system_time(:second)
      }
    }
  end
end

The Nectar.PIRSensor module defines a structure encompassing the state and details of a PIR sensor:

• movement_detected: As the name suggests, this field reflects whether the PIR sensor has sensed any movement.
• tamper_detected: This flag indicates if the PIR sensor has perceived any tampering or unauthorized interventions.
• device_info: An embedded reference to Nectar.DeviceInfo, this field consolidates general device-related details with the specifics of the PIR sensor. As outlined earlier, it incorporates the device's geographical position (geo_coordinate) and the instant the data was last noted (current_time). Essentially, this field bridges the sensor's specific events with overarching device information.
• The new function creates a new instance of the Nectar.PIRSensor struct with default values for its fields, including a device_info containing a geo_coordinate with specific latitude, longitude, and altitude values; %__MODULE__ refers to the current module (Nectar.PIRSensor) and is used for struct initialization within the same module.

Agents

Now, let's turn this PirSensor struct into a full-fledged agent to unlock a range of powerful capabilities that go beyond its simple structure.

By doing this we gain the ability to:

1. Manage State Over Time: Agents enable us to maintain and manage mutable state over time. This means we can capture its current state and track changes and updates to its properties as events unfold.

2. Concurrency Done Right: Elixir is built for concurrent and parallel processing, and agents are designed to thrive in such environments. With an agent, we can safely handle interactions with the PIR sensor from multiple processes, eliminating the risks of data races and conflicts.

3. Handle Asynchronous Events: PIR sensors will eventually detect events like movement or tampering. With an agent, we can seamlessly integrate event handling. When the sensor detects an event, it can send a message to the agent, triggering actions or updates in response.

4. Ensure Synchronization: Agents enforce synchronization, ensuring that only one process accesses and updates the agent's state at any given time. This also eliminates the potential for race conditions and guarantees data consistency.

5. Encapsulate Logic: An agent encapsulates state and behavior within a single entity. This isolation makes it easier to understand and reason about the behavior of our sensor within the broader application context.

6. Centralized Control: Agents can serve as a centralized point of control for managing and coordinating the behavior of multiple components within our application. This simplifies the logic for handling and responding to various sensor events.

7. Real-Time Updates: When our PIR sensor detects movement or tampering, it can instantly update its state within the agent. This real-time capability allows other parts of our application to respond promptly to changes in the sensor's status.

By transforming our PirSensor into an agent, we empower it to become a dynamic and responsive component in our Elixir application, capable of handling complex state management, concurrency challenges, and event-driven interactions with grace and efficiency.

We can draw an analogy between structs in Elixir and classes/prototypes in Object-Oriented Programming. Structs serve as blueprints, defining the structure and initial properties, much like classes describe properties and behaviors in object-oriented contexts.

Agents can be thought of as instances of these structs, akin to how objects are instances of classes. Just as multiple objects can be created from a single class, Elixir allows for multiple agents, each with its unique state, encapsulating state and behavior within individual units.

While recognizing the differences between Elixir's approach and traditional object-oriented paradigms, this analogy is intended to bridge understanding, especially for those familiar with object-oriented concepts.

Here is the code:

defmodule Netch.PIRSensor.Agent do
  use Agent

  def start_link(initial_state) do
    Agent.start_link(__MODULE__, fn -> initial_state end)
  end

  def init(initial_state) do
    {:ok, initial_state}
  end

  # Set individual properties
  def set_movement_detected(agent, value) do
    Agent.update(agent, fn state -> %{state | movement_detected: value} end)
  end

  def set_tamper_detected(agent, value) do
    Agent.update(agent, fn state -> %{state | tamper_detected: value} end)
  end

  def set_geo_coordinate(agent, geo_coord) do
    Agent.update(agent, fn state -> %{state | device: %{state.device | geo_coordinate: geo_coord}} end)
  end

  def set_current_time(agent, timestamp) do
    Agent.update(agent, fn state -> %{state | device: %{state.device | current_time: timestamp}} end)
  end

  # Get individual properties
  def get_movement_detected(agent) do
    Agent.get(agent, fn state -> state.movement_detected end)
  end

  def get_tamper_detected(agent) do
    Agent.get(agent, fn state -> state.tamper_detected end)
  end

  def get_geo_coordinate(agent) do
    Agent.get(agent, fn state -> state.device.geo_coordinate end)
  end

  def get_current_time(agent) do
    Agent.get(agent, fn state -> state.device.current_time end)
  end
end

Let's break down the code to make it easy what is happening here:

defmodule Netch.PIRSensor.Agent do

As we did before, we define a module named Netch.PIRSensor.Agent where we'll encapsulate the behavior and state management for our PIR sensor agent.

use Agent

We include the Agent module in our module, which provides us with the necessary functionality to create and manage an Elixir Agent. In the context of an Elixir module, the use keyword is used to inject functionality and behavior from a specified module into the current module.

  def start_link() do
    Agent.start_link(&PIRSensor.new/0)
  end

The start_link/0 function initiates an agent using Agent.start_link/1 the new/0 function from the Nectar.PIRSensor module, effectively creating a new agent process with default initial values.

&PIRSensor.new/0 is a function reference:

• The & symbol is used to create a function reference.
• PIRSensor refers to the module where the function is defined.
• new is the name of the function.
• /0 indicates the arity of the function, which is 0.
• In this context, the ampersand & is used to create a reference to a function, and the slash zero /0 specifies that the function takes zero arguments (i.e., it's arity 0).

def init(initial_state) do
	{:ok, initial_state}
end

• This function is a callback required by agents. It initializes the agent's state. It takes the initial_state as a parameter and returns a tuple {:ok, initial_state} to indicate successful initialization.

def set_geo_coordinate(agent, geo_coord) do
	Agent.update(agent, fn state -> %{state | device: %{state.device | geo_coordinate: geo_coord}} end)
end

def set_current_time(agent, timestamp) do
	Agent.update(agent, fn state -> %{state | device: %{state.device | current_time: timestamp}} end)
end

These functions allow us to set individual properties of the agent's state. For example, set_geo_coordinate/2 takes the agent and a value, then uses Agent.update/3 to update the agent's state by merging the current state with the new geo_coordinate location value.

• fn state -> %{state | device: %{state.device | geo_coordinate: geo_coord}} end s an example of an anonymous function in Elixir. It takes one argument, state, and returns a modified version of that state.
• %{state | ...} in this anonymous function is a map update operation in Elixir. It creates a new map based on the state map but with certain keys updated or added. We assign this new map to the state value. Remember in Elixir, data, once created, is immutable.

# Get individual properties
def get_movement_detected(agent) do
	Agent.get(agent, fn state -> state.movement_detected end)
end

def get_tamper_detected(agent) do
	Agent.get(agent, fn state -> state.tamper_detected end)
end

def get_geo_coordinate(agent) do
	Agent.get(agent, fn state -> state.device.geo_coordinate end)
end

def get_current_time(agent) do
	Agent.get(agent, fn state -> state.device.current_time end)
end

These functions allow you to get individual properties of the agent's state. For example, get_movement_detected/1 retrieves the movement_detected property from the agent's state using Agent.get/2.

Interactive Exploration with iex

$ iex -S mix
Erlang/OTP 26 [erts-14.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]  
  
Interactive Elixir (1.15.6) - press Ctrl+C to exit (type h() ENTER for help)  
iex(1)>

When you enter iex -S mix the terminal, you're launching Elixir's interactive shell, iex, integrated with your Mix project environment. The -S flag ensures any relevant scripts, like environment settings, are invoked, and mix, Elixir's build tool, loads your project. This combination offers a real-time development experience, letting you interact directly with your project's modules and functions within the interactive shell.

We can use an alias for Nectar.PIRSensor.Agent to simplify our code, eliminating the need to repeatedly write out the full module name.

iex(1)> alias Nectar.PIRSensor.Agent, as: PIRSensorAgent

Now we can create a new Agent.

iex(2)> PIRSensorAgent.start_link()  
{:ok, #PID<0.139.0>}

We can also bind it at the same time to some variable.

iex(3)> {ok, pir} = PIRSensorAgent.start_link()  
{:ok, #PID<0.140.0>}

The start_link/0 function returns a tuple, which is why we destructure it into two variables. The ok value indicates successful initialization and pir holds the process ID (PID) of the agent. For those with an Object-Oriented Programming (OOP) background, it's vital to note that pir isn't the agent itself, but rather its address. We'll use this PID later to send messages to the agent's mailbox.

Here, for example, to obtain some value

iex(4)> PIRSensorAgent.get_movement_detected(pir)

We are invoking the get_movement_detected/1 function from the (aliased) Nectar.PIRSensor.Agent module, passing in the PID (pir) of our agent as an argument. This function, internally, communicates with the actual agent identified by the provided PID. It queries the agent about the status of movement detection.

The agent then processes this request and sends back a response. In this case, the returned value false indicates that no movement has been detected by the PIR sensor. The function acts as a bridge between us and the agent, allowing us to query and interact with the agent's state and behavior.

We can try other fields too, of course

iex(5)> PIRSensorAgent.get_geo_coordinate(pir)  
%Nectar.Types.GeoCoordinate{  
 lat: 50.09431362821687,  
 lng: 14.449474425948397,  
 alt: 0  
}

iex(6)> PIRSensorAgent.get_current_time(pir)  
1696599184

But also update them

iex(7)> new_coords = %Nectar.Types.GeoCoordinate{lat: -14.031501779103623, lng: -47.61969792383484, alt: 0}    
%Nectar.Types.GeoCoordinate{  
 lat: -14.031501779103623,  
 lng: -47.61969792383484,  
 alt: 0  
}  
iex(8)> PIRSensorAgent.set_geo_coordinate(pir, new_coords)  
:ok

And verify that the value was updated

iex(32)> updated_coords = PIRSensorAgent.get_geo_coordinate(pir)  
%Nectar.Types.GeoCoordinate{  
 lat: -14.031501779103623,  
 lng: -47.61969792383484,  
 alt: 0  
}

Up Next: Part 2 - Expanding Our Toolkit: Implementing Agents for Additional Entities and Diving Deeper

In this next chapter, we'll take a practical step forward. We'll implement agents for our remaining entities and delve further into Agents. As we proceed, we'll continue to build upon the foundational knowledge, ensuring a thorough grasp of the concepts.

Until next time, keep coding!

References

• Elixir Install
• asdf Install
• Nectar GitHub Repo