in arduino, c#, tutorial, Unity3D

How to integrate Arduino with Unity

Share Button

In this tutorial you will learn how Unity and Arduino can communicate using the serial port. This tutorial requires both C# and Arduinoi scripts; the labels  Unity  and  Arduino  will be used to avoid confusion.

You can download the final scripts here:

Step 0: Configuring…

The communication between Arduino and the PC is mediated using the serial port. This method is the default one to upload sketches, so we’ll be using it also to exchange messages with Unity. If this is the first time you are doing it, you’ll need to follow these extra steps.

bbb Unity  The class SerialPort is the one that mediates such communication in C#. However, Unity doesn’t usually include the necessary libraries to use it. To compensate for this, we need to force Unity to include the full .NET 2.0 library in its executables:

  1. Go on Edit | Player Settings to open the PlayerSettings in the inspector;
  2. From Optimization, look for Api Compatibility Level and select .NET 2.0.

 Arduino  Arduino doesn’t come with a sophisticated library for the serial port. While C# has all the expected functions to send strings, Arduino simply doesn’t. This is very frustrating and is often a strong limitation for beginners. To compensate this, I’ll be using Steven Cogswell‘s ArduinoSerialCommand library. Follow these steps to install it:

  1. Download the entire repository by clicking “Download ZIP” from its GIT page;
  2. Extract the ZIP file and place its content in C:\Users\<username>\Documents\Arduino\SerialCommand\ (make sure that folder contains the SerialCommand.cpp file);
  3. Restart the Arduino IDE.

Step 1: Opening…

 Unity  To initialise the serial port in C#, we need its address (or port) and speed (also called baud rate).

While the baud rate is determined by the Arduino code, we cannot chose the name for the serial port. It is automatically assigned by the OS depending on which device and port you are using.

 Arduino  Now that SerialCommand has been installed, we can use it in our sketch. The library allows to specify commands that can be received on the serial port. For this toy example, we want to define a command called “PING”. When we receive such string from Unity, we’ll send a “PONG” back. Let’s start by defining the command; its code will be stored in the pingHandler function.

The 9600 used to initialise the serial port represents its baud rate. This value must match the one used in the C# script.

Step 2: Writing…

 Unity  Writing a string to the serial port in C# is relatively easy.

If there is a problem, WriteLine will throw a IOException (read more here). We flush the stream to make sure the data it sent to the Arduino, without any buffering. Following our toy protocol, we should send a “PING” to Arduino.

 Arduino  As discussed before, the SerialCommand library takes care of reading strings from the serial port for us. To do this, we need to update the loop function.

The function readSerial is the one where the magic happens; it reads strings from the serial port and invokes the right handler. If it receives a “PING”, it will execute  pingHandler:

This will write “PONG” on the serial port. You can also use Serial.println to send data from Arduino to Unity at any time.

Step 3: Reading…

 Unity  Reading from the serial port is theoretically as easy as writing, since C# offers a very intuitive  stream.ReadLine() function.

However, there’s a catch. How long do you want to wait for before considering the read failed? If you are waiting indefinitely for Arduino to send data, this might block the execution of your program. Reading from the serial port is, essentially, a system call and can introduce lag. A lot of lag. To avoid this, we should do very quick reads alternated by quick waits. In order to implement an asynchronous waiting mechanism, we have to use coroutines.

Which can be invoked like this:

The code below starts the AsynchronousReadFromArduino coroutine, passing three arguments. The first one is a function (created on the spot) which get the string read from Arduino and logs it. The second one is a callback if the reading fails, and the third one is the timeout (10 seconds).

 Arduino  There are cases in which you might want to send parameters from Unity to Arduino. Let’s do this with an echo function.

We can use the function next to get the next argument provided on the serial command. For instance, if we send from Unity “ECHO message”, “message” will be the first parameter. If next returns NULL, it means there are no more parameters.

Step 4: Communicating…

Now that all the pieces are here, you just have to implement your own communication protocol. Arduinos are usually used as sensors, constantly sending updates to Unity. If this is your case, you should not make the mistake of doing something like this:

Is very likely that this will overflow the serial port buffer, causing either a crash or a severe lag. What you should do instead is creating a protocol in which Arduino is sending data only when is queried by a “PING” from Unity. This ensures data is always fresh and minimises lags.

Conclusion

Last year at GDC a guy approached me and asked my opinion about UNIDUINO, an extension to connect Arduino with Unity. It was only after I said something on the line of “I would never pay so much to use it” that he introduced himself as the creator. Despite this not-so-great start, our conversation highlighted an important issue when it comes to development: how much time are you willing to invest into something? Many software developers have little to no experience with hardware, and they rather spend €32 than a week of headaches to connect Unity to Arduino. To misquote Mike Bithell during his talk at Develop Brighton a couple of years ago, some developers have “more money than time“, If you are going to use an Arduino for your project, you definitely need to invest some money in it. But if you think paying for an extension is out of your budget, this tutorial is definitely here to help. Despite not providing the full range of functions of UNIDUINO, it is surely a good starting point which will suit the majority of applications.

How to start with Arduino

The most annoying part of working with hardware is that you’ll constantly need new components to build things. If you are new to Arduino, my advice is to start with the Arduino Starter Kit. It has a lot (I mean… a lot!) of components and it comes with some great instructions.  If you have a little bit more experience, you might want to buy the components you need separately. In this case, the Arduino Uno is the “default” option you should go for.

If you have already experience with hardware, I advise you to start using Teensy 3.1 instead. It’s a micro-controller fully compatible with the Arduino IDE. It’s compact and more powerful, although it usually doesn’t come with pins. There’s also a Teensy 3.1 Starter Kit which comes with a very bright Color LCD screen. I also suggest a book I particularly liked: 30 Arduino Projects for the Evil Genius. I got it few years ago and it helped me to understand how to use all the most common components.



Don't miss the next tutorial!

There's a new post every Wednesday: leave your email to be notified!


Ways to Support

In the past months I've been dedicating more and more of my time to the creation of quality tutorials, mainly about game development and machine learning. If you think these posts have either helped or inspired you, please consider supporting this blog.

PatreonBecome a Patron Oo4th_patreon_name
PaypalDonate on PayPal
Twitter_logoFollow on Twitter

Write a Comment

Comment

26 Comments

  1. hey, with your code i have problem in unity 3d and arduino, and i dont know how could i resolve them??

    i am using the code that you gave us for download in both and I change de port for COM4, bacause in my pc it is the port that use arduino.

    unity problem:
    default parameter specifiers are not permited.

    arduino problem:
    Arduino: 1.6.5 (Windows 8.1), Board: “Arduino/Genuino Uno”

    In file included from UnityConnector.ino:1:0:
    C:\Users\Sebastian\Documents\Arduino\libraries\ArduinoSerialCommand-master/SerialCommand.h:60:30: fatal error: SoftwareSerial.h: No such file or directory
    #include
    ^
    compilation terminated.
    Error compiling.

  2. Hi, Alan:
    Thanks for your work!
    I tried your solution but there are error message in both side.
    In Unity:
    ” Assets/Script/ArduinoConnector.cs(31,16): error CS1061: Type System.IO.Ports.SerialPort' does not contain a definition for WriteLine’ and no extension method WriteLine' of type System.IO.Ports.SerialPort’ could be found (are you missing a using directive or an assembly reference?)”
    In Arduino:
    ‘arg’ does not name a type.

    Do you have any idea on what going on?

      • Hi, Alan:

        Glad to have your reply!
        I had put the ArduinoSerialCommand library under the normal library path, and its example code can be correctly executed, thus I think the library part should be no problem.
        In Unity, I have change to .NET 2.0. Some stream commend such as stream.timeout, stream.open can be corrected interpret, but only stream.WriteLine(message), stream.BaseStream.Flush(), stream.ReadLine() can’t. I check the error message in google, no explicit answer but point to security protection.
        Any idea on it?

      • Hi, Alan:
        Few updates.
        I test your code also in windows (in before it was in Mac).
        In Unity, when choosing the .NET 2.0, all the error messages gone!
        In Arduino, even though all the example code can be corrected executed, I get the same error message as Sebastian. May I ask which version of arduino are you using? and any special way to place those files?
        Thank you!

          • Hi, Alan:
            I just try your sample code UnityConnector.ino which already included the #include , and get the error message mentioned above.
            In my understanding, I must able to run the sample code before adding any new lines.

        • Hi Carol,

          Did you find a solution for the ” ‘arg’ does not name a type” issue? I’m running into the same problem.

          • Hey!
            Which version of Unity / Arduino / Board are you using?

            This tutorial is few months old. So some things might have changed.

  3. Hello Alan,
    Thank you very much for your tutorial, I have been trying to make this work for a while, but I am not sure on how to implement it properly as I keep on getting errors and lag in the connection.
    I wonder if you would have an example for reading a single analog sensor from the arduino? I have been checking but didn´t find anything.

  4. Hi Alan I follow your tutorial step by step, but when I tried to execute the first part, about writing data, my arduino IDE throws me the following error:

    invalid conversion from ‘void (*)(const char*)’ to ‘void (*)()’ [-fpermissive]
    sCmd.addCommand(“PING”, pingHandler);

    Do you know why? My code it’s exactly what you have:

    #include
    #include
    SerialCommand sCmd;

    void setup() {
    Serial.begin(9600);
    while (!Serial);

    sCmd.addCommand(“PING”, pingHandler);
    }

    void loop () {
    if (Serial.available() > 0)
    sCmd.readSerial();
    }

    void pingHandler (const char *command) {
    Serial.println(“PONG”);
    }

    Thanks in advance.

    • I had the same issue. While I don’t know the details, it would appear the error message indicates a mismatch in the function pointer passed into the second argument of the sCmd.addCommand method. You can fix it by removing the “const char *command” pointer argument from the pingHandler function declaration. It should just read:

      void pingHandler()
      {
      //your code here
      }

  5. **For Mac USERS** arduino and unity 5.3.5 it works!!!
    Use your terminal of mac and type ls /dev/tty.*
    and you will get your address to your arduino port.
    when reading in unity, use this following line of code instead of your “comX”

    SerialPort sp = new SerialPort(“/dev/tty.usbmodem1421”,9600);

    and voila! remember to run in arduino first, and close the serial monitor. Now in unity run your scene…

  6. how do you test the scripts once you’re done writing them?
    I’ve downloaded them and uploaded the arduino code to my Uno and attached the c# script to an empty object in a game scene in Unity and nothing happens in the command windows.

  7. This was great, Thank you!

    However there is one thing that I can’t get to work.. I’m only going to use this to have Unity send information to the arduino, so I only did the writing part. And it seems to work fine except for that there seems to be a collision when I want both Arduino and Unity to use the same COM port? Is the COM port in Unity supposed to be the same as the COM port that we select in Arduino Sketch? Because if I do that I get an error in which ever software that I try to have use the port last, telling me that the COM port is occupied. If I choose another COM port then nothing happens when I do the WriteToArduino (“PING”) thing.

    In this tutorial you write that we cannot chose the name of our COM port in our Unity script, and I don’t quite understand that? In you example it still looks to me like you chose COM4 when you write:
    stream = new SerialPort(“COM4”, 9600);

    Also you write that the COM port will be chosen automatically, which I’m not experiencing that it does. Maybe I’m doing something wrong I don’t know.

    I would very much appreciate some guidance here 🙂

    Thanks for the effort put into this!

    • Hey!

      Sorry if there has been any part that was not super clear. When you plug a device, it is automatically assigned a COM port. You cannot decide directly whether that is going be COM1, COM2 or COM10. This is what I meant. Once the device is connected and you know it’s on (let’s say) COM4, then you can safely write COM4.

      The serial port, in your example, becomes a shared resource. It’s important that you architect your communication protocol so that every time there is a WRITE on one end, there is a READ on the other. If BOTH Arduino and Unity are waiting on a read operation, they’ll both get stuck in a deadlock. Asynchronous communications like those ones are tricky. Is best to have the communication ONE way. For instance… Unity sends commands to Arduino. Arduino NEVER initiates a communication itself. If you want Arduino to send some data, Unity can query it periodically. Then Arduino will have to reply THIS IS THE DATA or NO THERE IS NO DATA. This might not be the most efficient protocol, but is guaranteed you’ll never get stuck in a deadlock.

Webmentions

  • Asynchronous Serial Communication - Alan Zucconi December 14, 2016

    Hey!

    Sorry if there has been any part that was not super clear. When you plug a device, it is automatically assigned a COM port. You cannot decide directly whether that is going be COM1, COM2 or COM10. This is what I meant. Once the device is connected and you know it’s on (let’s say) COM4, then you can safely write COM4.

    The serial port, in your example, becomes a shared resource. It’s important that you architect your communication protocol so that every time there is a WRITE on one end, there is a READ on the other. If BOTH Arduino and Unity are waiting on a read operation, they’ll both get stuck in a deadlock. Asynchronous communications like those ones are tricky. Is best to have the communication ONE way. For instance… Unity sends commands to Arduino. Arduino NEVER initiates a communication itself. If you want Arduino to send some data, Unity can query it periodically. Then Arduino will have to reply THIS IS THE DATA or NO THERE IS NO DATA. This might not be the most efficient protocol, but is guaranteed you’ll never get stuck in a deadlock.

  • Process Blog 014 | Activation December 14, 2016

    Hey!

    Sorry if there has been any part that was not super clear. When you plug a device, it is automatically assigned a COM port. You cannot decide directly whether that is going be COM1, COM2 or COM10. This is what I meant. Once the device is connected and you know it’s on (let’s say) COM4, then you can safely write COM4.

    The serial port, in your example, becomes a shared resource. It’s important that you architect your communication protocol so that every time there is a WRITE on one end, there is a READ on the other. If BOTH Arduino and Unity are waiting on a read operation, they’ll both get stuck in a deadlock. Asynchronous communications like those ones are tricky. Is best to have the communication ONE way. For instance… Unity sends commands to Arduino. Arduino NEVER initiates a communication itself. If you want Arduino to send some data, Unity can query it periodically. Then Arduino will have to reply THIS IS THE DATA or NO THERE IS NO DATA. This might not be the most efficient protocol, but is guaranteed you’ll never get stuck in a deadlock.

  • How to Write Libraries for Arduino – Alan Zucconi December 14, 2016

    Hey!

    Sorry if there has been any part that was not super clear. When you plug a device, it is automatically assigned a COM port. You cannot decide directly whether that is going be COM1, COM2 or COM10. This is what I meant. Once the device is connected and you know it’s on (let’s say) COM4, then you can safely write COM4.

    The serial port, in your example, becomes a shared resource. It’s important that you architect your communication protocol so that every time there is a WRITE on one end, there is a READ on the other. If BOTH Arduino and Unity are waiting on a read operation, they’ll both get stuck in a deadlock. Asynchronous communications like those ones are tricky. Is best to have the communication ONE way. For instance… Unity sends commands to Arduino. Arduino NEVER initiates a communication itself. If you want Arduino to send some data, Unity can query it periodically. Then Arduino will have to reply THIS IS THE DATA or NO THERE IS NO DATA. This might not be the most efficient protocol, but is guaranteed you’ll never get stuck in a deadlock.

  • Personalized Joystick with arduino (for Unity) | Raspi Game December 14, 2016

    Hey!

    Sorry if there has been any part that was not super clear. When you plug a device, it is automatically assigned a COM port. You cannot decide directly whether that is going be COM1, COM2 or COM10. This is what I meant. Once the device is connected and you know it’s on (let’s say) COM4, then you can safely write COM4.

    The serial port, in your example, becomes a shared resource. It’s important that you architect your communication protocol so that every time there is a WRITE on one end, there is a READ on the other. If BOTH Arduino and Unity are waiting on a read operation, they’ll both get stuck in a deadlock. Asynchronous communications like those ones are tricky. Is best to have the communication ONE way. For instance… Unity sends commands to Arduino. Arduino NEVER initiates a communication itself. If you want Arduino to send some data, Unity can query it periodically. Then Arduino will have to reply THIS IS THE DATA or NO THERE IS NO DATA. This might not be the most efficient protocol, but is guaranteed you’ll never get stuck in a deadlock.

  • Joystick personalizado con Arduino (para Unity) | Raspi Game December 14, 2016

    Hey!

    Sorry if there has been any part that was not super clear. When you plug a device, it is automatically assigned a COM port. You cannot decide directly whether that is going be COM1, COM2 or COM10. This is what I meant. Once the device is connected and you know it’s on (let’s say) COM4, then you can safely write COM4.

    The serial port, in your example, becomes a shared resource. It’s important that you architect your communication protocol so that every time there is a WRITE on one end, there is a READ on the other. If BOTH Arduino and Unity are waiting on a read operation, they’ll both get stuck in a deadlock. Asynchronous communications like those ones are tricky. Is best to have the communication ONE way. For instance… Unity sends commands to Arduino. Arduino NEVER initiates a communication itself. If you want Arduino to send some data, Unity can query it periodically. Then Arduino will have to reply THIS IS THE DATA or NO THERE IS NO DATA. This might not be the most efficient protocol, but is guaranteed you’ll never get stuck in a deadlock.

  • A Bestiary of Alternative Game Controllers - Alan Zucconi December 14, 2016

    Hey!

    Sorry if there has been any part that was not super clear. When you plug a device, it is automatically assigned a COM port. You cannot decide directly whether that is going be COM1, COM2 or COM10. This is what I meant. Once the device is connected and you know it’s on (let’s say) COM4, then you can safely write COM4.

    The serial port, in your example, becomes a shared resource. It’s important that you architect your communication protocol so that every time there is a WRITE on one end, there is a READ on the other. If BOTH Arduino and Unity are waiting on a read operation, they’ll both get stuck in a deadlock. Asynchronous communications like those ones are tricky. Is best to have the communication ONE way. For instance… Unity sends commands to Arduino. Arduino NEVER initiates a communication itself. If you want Arduino to send some data, Unity can query it periodically. Then Arduino will have to reply THIS IS THE DATA or NO THERE IS NO DATA. This might not be the most efficient protocol, but is guaranteed you’ll never get stuck in a deadlock.

  • Everything You Need to Know About LEDs - Alan Zucconi December 14, 2016

    Hey!

    Sorry if there has been any part that was not super clear. When you plug a device, it is automatically assigned a COM port. You cannot decide directly whether that is going be COM1, COM2 or COM10. This is what I meant. Once the device is connected and you know it’s on (let’s say) COM4, then you can safely write COM4.

    The serial port, in your example, becomes a shared resource. It’s important that you architect your communication protocol so that every time there is a WRITE on one end, there is a READ on the other. If BOTH Arduino and Unity are waiting on a read operation, they’ll both get stuck in a deadlock. Asynchronous communications like those ones are tricky. Is best to have the communication ONE way. For instance… Unity sends commands to Arduino. Arduino NEVER initiates a communication itself. If you want Arduino to send some data, Unity can query it periodically. Then Arduino will have to reply THIS IS THE DATA or NO THERE IS NO DATA. This might not be the most efficient protocol, but is guaranteed you’ll never get stuck in a deadlock.

  • How to build a distance sensor with Arduino - Alan Zucconi December 14, 2016

    Hey!

    Sorry if there has been any part that was not super clear. When you plug a device, it is automatically assigned a COM port. You cannot decide directly whether that is going be COM1, COM2 or COM10. This is what I meant. Once the device is connected and you know it’s on (let’s say) COM4, then you can safely write COM4.

    The serial port, in your example, becomes a shared resource. It’s important that you architect your communication protocol so that every time there is a WRITE on one end, there is a READ on the other. If BOTH Arduino and Unity are waiting on a read operation, they’ll both get stuck in a deadlock. Asynchronous communications like those ones are tricky. Is best to have the communication ONE way. For instance… Unity sends commands to Arduino. Arduino NEVER initiates a communication itself. If you want Arduino to send some data, Unity can query it periodically. Then Arduino will have to reply THIS IS THE DATA or NO THERE IS NO DATA. This might not be the most efficient protocol, but is guaranteed you’ll never get stuck in a deadlock.