Arduino programming with CrewAI

Table of Contents

If you prefer to go directly into the code, there’s a GitHub repo available!!

Although I don’t consider myself an expert (far from it!) in Arduino programming, I really enjoy building electronic projects in my spare time. So, the other day an idea popped into my head: I know a few things about AI and I know a few things about Arduino so…. what if I make them work together? 🤔

And since I’ve been working with crewAI for the last few weeks, I didn’t hesitate: let’s connect crewAI with Arduino.

Sounds exciting? What if I tell you that, in addition, I’ll also use Ollama and Llama3 (local LLMs, oh yes 😎)?

But, not so fast, as you may never have heard of what an Arduino is. Let’s calm down, and start with the basics.

Enjoy the ride! 🔥

Warning! ⚠️⚠️⚠️ In this article I’m assuming you know crewAI, and by that I mean that you know what is a Task, an Agent or a Tool. If you don’t know what I’m talking about, I strongly recommend you to check this article.

What’s an Arduino?

You can think of an Arduino as a small, programmable computer you can use to create your own electronic projects, from simple circuits that make lights blink to fully functional robots capable of moving. The possibilities are endless if you use your imagination 💭

Simplifying (a lot 😅), when working with the Arduino platform, you have to differentiate between two “parts”.

1️⃣ The Board

The Arduino board is the physical hardware that contains the microcontroller chip. This chip is the heart of the Arduino and is responsible for executing the instructions you give it through your code.

2️⃣ Programming

You write code (called sketches) using Arduino programming language (which is based on C / C++). These sketches tell the Arduino what to do, such as turning on a light, reading the sensor data or controlling a servomotor.

For example, this is the sketch for turning on a LED for one second, then off for one second, repeatedly.

void setup() {
  pinMode(11, OUTPUT);
}

void loop() {
  digitalWrite(11, HIGH);   // turn the LED on (HIGH is the voltage level)
  delay(1000);                       // wait for a second
  digitalWrite(11, LOW);    // turn the LED off by making the voltage LOW
  delay(1000);                       // wait for a second
}

I won’t get into the details of sketch programming, as it is beyond the scope of this simple tutorial, but, if you are interested in this fascinating world, I recommend you to go through the Arduino official tutorials. It contains a lot of examples and detailed explanations (that’s where I started my Arduino journey 😝)

Three blinking LEDs

To validate the connection between crewAI and my Arduino, I chose a simple Arduino project: making three LEDs blink repeatedly. You can check the circuit we are going to implement in the circuit.io diagram below.

And here’s my “real world” implementation (much, much uglier than circuit.io, I know 🥲)

The next “logical” step would be to manually write a sketch to control de LEDs. But … that’s exactly what we are going to automate!!

So, instead of doing the programming ourselves, we’ll have a crewAI take care of it. Similarly, instead of compiling and uploading the code to the Arduino, another crewAI agent will handle that task for us 😎

Building the Crew

Let’s focus on the diagram below, which shows the crewAI application we are about to build.

As you can see, the crew is not complex. It consists of two agents: the Sketch Programmer Agent and the Arduino Uploader Agent. The first agent will receive the circuit description and its expected behaviour, generating, in return, a sketch file (the extension of a sketch is .ino by the way). The second agent will take the generated sketch, compile it and upload the instructions into the Arduino.

So, if everything works as expected, we should end up with three beautiful blinking LEDs without writing one single line of C 😍

Sketch Programmer Agent

I promised you at the beginning of this article that we were going to use local LLMs, and I’m a man of my word. In this case, I’m going to use a local (quantised) version of Llama 3. How?

Simple, using the almighty Ollama.

If you don’t know Ollama, Matthew Berman has a very good video about it.

To use Ollama within crewAI, we just need to add the following lines before defining the agents.

from langchain_openai import ChatOpenAI

llama3 = ChatOpenAI(
    model="llama3",
    base_url="http://localhost:11434/v1")

Notice that I’m using the llama3 model. If you want to do the same, make sure you’ve downloaded it in Ollama!

ollama pull llama3

And remember that, if you want to try the model, you just need to run this command.

ollama run llama3

So now it’s time to show you the agent.

sketch_programmer_agent = Agent(
    role="Sketch Programmer",
    goal=dedent(
        """Write a Sketch script for Arduino that lights a red led in digital pin 11, 
        a blue led in digital pin 10 and a green led in digital pin 9 with a time
        between the three of 1 second."""),
    backstory=dedent(
        """
        You are an experienced Sketch programmer who really enjoys programming Arduinos
        """
    ),
    verbose=True,
    allow_delegation=False,
    llm=llama3,
)

The agent’s goal has information about the circuit we have implemented as well as the intended behaviour. In addition, you can see that the agent is using llama3 as LLM.

But, where is the sketch file generated? 🤔

Good question! We can control this from the Task assigned to the previous agent. Let me show you.

sketch_programming_task = Task(
    description=dedent(
        "Write a Sketch script that can be immediately uploaded to an Arduino. Just the Arduino code, nothing else."),
    expected_output=dedent("A plain Sketch script that can be copy and pasted directly into the Arduino CLI"),
    agent=sketch_programmer_agent,
    output_file="./tmp/tmp.ino",
)

Note the output_file attribute. That’s exactly what we need; the first agent will dump the generated code into the ./tmp/tmp.ino file.

Arduino Uploader Agent

Ok, so now that the previous agent has generated the sketch file, this agent needs to compile it and load it into the Arduino. Sounds difficult, doesn’t it? Well, … far from it!

The solution is pretty easy: a custom tool. 🛠️

import re
import subprocess

from crewai_tools import BaseTool


class CompileAndUploadToArduinoTool(BaseTool):
    name: str = "CompileAndUploadToArduinoTool"
    description: str = "Compiles and Uploads an Arduino Sketch script to an Arduino"
    ino_file_dir: str = "The directory that contains the ino file"
    board_fqbn: str = "The board type, e.g. 'arduino:avr:uno'"
    port: str = "The port where the Arduino is connected"

    def __init__(self, ino_file_dir: str, board_fqbn: str, port: str, **kwargs):
        super().__init__(**kwargs)
        self.ino_file_dir = ino_file_dir
        self.board_fqbn = board_fqbn
        self.port = port

    def _fix_ino_file(self):
        """
        This is a helper method for fixing the output .ino file when Llama3 adds some unintended text
        that invalidates the compilation.
        """
        with open(f"{self.ino_file_dir}/tmp.ino", "r") as f:
            content = f.read()

        pattern = r'```.*?\n(.*?)```'
        match = re.search(pattern, content, re.DOTALL).group(1).strip()

        with open(f"{self.ino_file_dir}/tmp.ino", "w") as f:
            f.write(match)

    def _run(self):
        self._fix_ino_file()

        try:
            subprocess.check_call([
                "arduino-cli", "compile", "--fqbn", self.board_fqbn, self.ino_file_dir
            ])
            subprocess.check_call([
                "arduino-cli", "upload", "--port", self.port, "--fqbn", self.board_fqbn, self.ino_file_dir
            ])
        except subprocess.CalledProcessError:
            return "Compilation failed"

        return "Code successfully uploaded to the board"

This tool expects three arguments:

ino_file_dir: The directory containing the sketch. Remember we were using the tmp dir.
board_fqbn: The board type. I’m using an Arduino UNO, so my board type is arduino:avr:uno, but it may be different in your case.
port: The port where the Arduino is connected. Mine is connected to port /dev/cu.usbmodem1201.

When the agent uses the tool (by accessing the _run method), it will compile the code (arduino-cli compile ...) and then upload it to the Arduino (arduino-cli upload ...). I have defined this agent like this (in this case I’ve used GPT4 since it works much better when using tools):

tool = CompileAndUploadToArduinoTool(
    ino_file_dir="./tmp",
    board_fqbn="arduino:avr:uno",
    port="/dev/cu.usbmodem1201"
)

arduino_uploader_agent = Agent(
    role="Arduino Uploader Agent",
    goal="Your goal is to compile and upload the received arduino script using a tool",
    backstory=dedent(
        """
        You are a hardware geek.
        """
    ),
    verbose=True,
    allow_delegation=False,
    tools=[tool]
)

And this is the task assigned to the agent.

arduino_uploading_task = Task(
    description=dedent(
        "Compile and Upload the the Sketch script into the Arduino"),
    expected_output=dedent("Just compile the code and upload it into the Arduino"),
    agent=arduino_uploader_agent,
)

Results

Phew 🥵, it took a while, but we have now built all the pieces needed to build our application. It’s time to put it all together!

from crewai import Crew
from dotenv import load_dotenv

from agents import sketch_programmer_agent, arduino_uploader_agent
from tasks import sketch_programming_task, arduino_uploading_task

load_dotenv()


crew = Crew(
            agents=[sketch_programmer_agent, arduino_uploader_agent],
            tasks=[sketch_programming_task, arduino_uploading_task],
        )

result = crew.kickoff()

print(result)