in arduino, c#, tutorial, Unity3D

How to integrate Arduino with Unity

Share Button

In this tutorial you will learn how Unity and Arduino can communicate using the serial port. This tutorial requires both C# and Arduino scripts; the labels  Unity  and  Arduino  will be used to avoid confusion.

You can download the final scripts here:

Step 0: Configuring…

The communication between Arduino and the PC is mediated using the serial port. This method is the default one to upload sketches, so we’ll be using it also to exchange messages with Unity. If this is the first time you are doing it, you’ll need to follow these extra steps.

bbb Unity  The class SerialPort is the one that mediates such communication in C#. However, Unity doesn’t usually include the necessary libraries to use it. To compensate for this, we need to force Unity to include the full .NET 2.0 library in its executables:

  1. Go on Edit | Player Settings to open the PlayerSettings in the inspector;
  2. From Optimization, look for Api Compatibility Level and select .NET 2.0.

 Arduino  Arduino doesn’t come with a sophisticated library for the serial port. While C# has all the expected functions to send strings, Arduino simply doesn’t. This is very frustrating and is often a strong limitation for beginners. To compensate this, I’ll be using Steven Cogswell‘s ArduinoSerialCommand library. Follow these steps to install it:

  1. Download the entire repository by clicking “Download ZIP” from its GIT page;
  2. Extract the ZIP file and place its content in C:\Users\<username>\Documents\Arduino\SerialCommand\ (make sure that folder contains the SerialCommand.cpp file);
  3. Restart the Arduino IDE.

Step 1: Opening…

 Unity  To initialise the serial port in C#, we need its address (or port) and speed (also called baud rate).

While the baud rate is determined by the Arduino code, we cannot chose the name for the serial port. It is automatically assigned by the OS depending on which device and port you are using.

 Arduino  Now that SerialCommand has been installed, we can use it in our sketch. The library allows to specify commands that can be received on the serial port. For this toy example, we want to define a command called “PING”. When we receive such string from Unity, we’ll send a “PONG” back. Let’s start by defining the command; its code will be stored in the pingHandler function.

The 9600 used to initialise the serial port represents its baud rate. This value must match the one used in the C# script.

Step 2: Writing…

 Unity  Writing a string to the serial port in C# is relatively easy.

If there is a problem, WriteLine will throw a IOException (read more here). We flush the stream to make sure the data it sent to the Arduino, without any buffering. Following our toy protocol, we should send a “PING” to Arduino.

 Arduino  As discussed before, the SerialCommand library takes care of reading strings from the serial port for us. To do this, we need to update the loop function.

The function readSerial is the one where the magic happens; it reads strings from the serial port and invokes the right handler. If it receives a “PING”, it will execute  pingHandler:

This will write “PONG” on the serial port. You can also use Serial.println to send data from Arduino to Unity at any time.

Step 3: Reading…

 Unity  Reading from the serial port is theoretically as easy as writing, since C# offers a very intuitive  stream.ReadLine() function.

However, there’s a catch. How long do you want to wait for before considering the read failed? If you are waiting indefinitely for Arduino to send data, this might block the execution of your program. Reading from the serial port is, essentially, a system call and can introduce lag. A lot of lag. To avoid this, we should do very quick reads alternated by quick waits. In order to implement an asynchronous waiting mechanism, we have to use coroutines.

Which can be invoked like this:

The code below starts the AsynchronousReadFromArduino coroutine, passing three arguments. The first one is a function (created on the spot) which get the string read from Arduino and logs it. The second one is a callback if the reading fails, and the third one is the timeout (10 seconds).

 Arduino  There are cases in which you might want to send parameters from Unity to Arduino. Let’s do this with an echo function.

We can use the function next to get the next argument provided on the serial command. For instance, if we send from Unity “ECHO message”, “message” will be the first parameter. If next returns NULL, it means there are no more parameters.

Step 4: Communicating…

Now that all the pieces are here, you just have to implement your own communication protocol. Arduinos are usually used as sensors, constantly sending updates to Unity. If this is your case, you should not make the mistake of doing something like this:

Is very likely that this will overflow the serial port buffer, causing either a crash or a severe lag. What you should do instead is creating a protocol in which Arduino is sending data only when is queried by a “PING” from Unity. This ensures data is always fresh and minimises lags.

Conclusion

Last year at GDC a guy approached me and asked my opinion about UNIDUINO, an extension to connect Arduino with Unity. It was only after I said something on the line of “I would never pay so much to use it” that he introduced himself as the creator. Despite this not-so-great start, our conversation highlighted an important issue when it comes to development: how much time are you willing to invest into something? Many software developers have little to no experience with hardware, and they rather spend €32 than a week of headaches to connect Unity to Arduino. To misquote Mike Bithell during his talk at Develop Brighton a couple of years ago, some developers have “more money than time“, If you are going to use an Arduino for your project, you definitely need to invest some money in it. But if you think paying for an extension is out of your budget, this tutorial is definitely here to help. Despite not providing the full range of functions of UNIDUINO, it is surely a good starting point which will suit the majority of applications.

How to start with Arduino

The most annoying part of working with hardware is that you’ll constantly need new components to build things. If you are new to Arduino, my advice is to start with the Arduino Starter Kit. It has a lot (I mean… a lot!) of components and it comes with some great instructions.  If you have a little bit more experience, you might want to buy the components you need separately. In this case, the Arduino Uno is the “default” option you should go for.

If you have already experience with hardware, I advise you to start using Teensy 3.1 instead. It’s a micro-controller fully compatible with the Arduino IDE. It’s compact and more powerful, although it usually doesn’t come with pins. There’s also a Teensy 3.1 Starter Kit which comes with a very bright Color LCD screen. I also suggest a book I particularly liked: 30 Arduino Projects for the Evil Genius. I got it few years ago and it helped me to understand how to use all the most common components.


Support this blog! ♥

For the past three years I've been dedicating more and more of my time to the creation of quality tutorials, mainly about game development and machine learning. If you think these posts have either helped or inspired you, please consider supporting this blog.

Paypal
Twitter_logo

Don't miss the next tutorial!

There's a new post every Wednesday: leave your email to be notified!


Write a Comment

Comment

37 Comments

  1. hey, with your code i have problem in unity 3d and arduino, and i dont know how could i resolve them??

    i am using the code that you gave us for download in both and I change de port for COM4, bacause in my pc it is the port that use arduino.

    unity problem:
    default parameter specifiers are not permited.

    arduino problem:
    Arduino: 1.6.5 (Windows 8.1), Board: “Arduino/Genuino Uno”

    In file included from UnityConnector.ino:1:0:
    C:\Users\Sebastian\Documents\Arduino\libraries\ArduinoSerialCommand-master/SerialCommand.h:60:30: fatal error: SoftwareSerial.h: No such file or directory
    #include
    ^
    compilation terminated.
    Error compiling.

  2. Hi, Alan:
    Thanks for your work!
    I tried your solution but there are error message in both side.
    In Unity:
    ” Assets/Script/ArduinoConnector.cs(31,16): error CS1061: Type System.IO.Ports.SerialPort' does not contain a definition for WriteLine’ and no extension method WriteLine' of type System.IO.Ports.SerialPort’ could be found (are you missing a using directive or an assembly reference?)”
    In Arduino:
    ‘arg’ does not name a type.

    Do you have any idea on what going on?

      • Hi, Alan:

        Glad to have your reply!
        I had put the ArduinoSerialCommand library under the normal library path, and its example code can be correctly executed, thus I think the library part should be no problem.
        In Unity, I have change to .NET 2.0. Some stream commend such as stream.timeout, stream.open can be corrected interpret, but only stream.WriteLine(message), stream.BaseStream.Flush(), stream.ReadLine() can’t. I check the error message in google, no explicit answer but point to security protection.
        Any idea on it?

      • Hi, Alan:
        Few updates.
        I test your code also in windows (in before it was in Mac).
        In Unity, when choosing the .NET 2.0, all the error messages gone!
        In Arduino, even though all the example code can be corrected executed, I get the same error message as Sebastian. May I ask which version of arduino are you using? and any special way to place those files?
        Thank you!

          • Hi, Alan:
            I just try your sample code UnityConnector.ino which already included the #include , and get the error message mentioned above.
            In my understanding, I must able to run the sample code before adding any new lines.

        • Hi Carol,

          Did you find a solution for the ” ‘arg’ does not name a type” issue? I’m running into the same problem.

          • Hey!
            Which version of Unity / Arduino / Board are you using?

            This tutorial is few months old. So some things might have changed.

  3. Hello Alan,
    Thank you very much for your tutorial, I have been trying to make this work for a while, but I am not sure on how to implement it properly as I keep on getting errors and lag in the connection.
    I wonder if you would have an example for reading a single analog sensor from the arduino? I have been checking but didn´t find anything.

  4. Hi Alan I follow your tutorial step by step, but when I tried to execute the first part, about writing data, my arduino IDE throws me the following error:

    invalid conversion from ‘void (*)(const char*)’ to ‘void (*)()’ [-fpermissive]
    sCmd.addCommand(“PING”, pingHandler);

    Do you know why? My code it’s exactly what you have:

    #include
    #include
    SerialCommand sCmd;

    void setup() {
    Serial.begin(9600);
    while (!Serial);

    sCmd.addCommand(“PING”, pingHandler);
    }

    void loop () {
    if (Serial.available() > 0)
    sCmd.readSerial();
    }

    void pingHandler (const char *command) {
    Serial.println(“PONG”);
    }

    Thanks in advance.

    • I had the same issue. While I don’t know the details, it would appear the error message indicates a mismatch in the function pointer passed into the second argument of the sCmd.addCommand method. You can fix it by removing the “const char *command” pointer argument from the pingHandler function declaration. It should just read:

      void pingHandler()
      {
      //your code here
      }

  5. **For Mac USERS** arduino and unity 5.3.5 it works!!!
    Use your terminal of mac and type ls /dev/tty.*
    and you will get your address to your arduino port.
    when reading in unity, use this following line of code instead of your “comX”

    SerialPort sp = new SerialPort(“/dev/tty.usbmodem1421”,9600);

    and voila! remember to run in arduino first, and close the serial monitor. Now in unity run your scene…

  6. how do you test the scripts once you’re done writing them?
    I’ve downloaded them and uploaded the arduino code to my Uno and attached the c# script to an empty object in a game scene in Unity and nothing happens in the command windows.

  7. This was great, Thank you!

    However there is one thing that I can’t get to work.. I’m only going to use this to have Unity send information to the arduino, so I only did the writing part. And it seems to work fine except for that there seems to be a collision when I want both Arduino and Unity to use the same COM port? Is the COM port in Unity supposed to be the same as the COM port that we select in Arduino Sketch? Because if I do that I get an error in which ever software that I try to have use the port last, telling me that the COM port is occupied. If I choose another COM port then nothing happens when I do the WriteToArduino (“PING”) thing.

    In this tutorial you write that we cannot chose the name of our COM port in our Unity script, and I don’t quite understand that? In you example it still looks to me like you chose COM4 when you write:
    stream = new SerialPort(“COM4”, 9600);

    Also you write that the COM port will be chosen automatically, which I’m not experiencing that it does. Maybe I’m doing something wrong I don’t know.

    I would very much appreciate some guidance here 🙂

    Thanks for the effort put into this!

    • Hey!

      Sorry if there has been any part that was not super clear. When you plug a device, it is automatically assigned a COM port. You cannot decide directly whether that is going be COM1, COM2 or COM10. This is what I meant. Once the device is connected and you know it’s on (let’s say) COM4, then you can safely write COM4.

      The serial port, in your example, becomes a shared resource. It’s important that you architect your communication protocol so that every time there is a WRITE on one end, there is a READ on the other. If BOTH Arduino and Unity are waiting on a read operation, they’ll both get stuck in a deadlock. Asynchronous communications like those ones are tricky. Is best to have the communication ONE way. For instance… Unity sends commands to Arduino. Arduino NEVER initiates a communication itself. If you want Arduino to send some data, Unity can query it periodically. Then Arduino will have to reply THIS IS THE DATA or NO THERE IS NO DATA. This might not be the most efficient protocol, but is guaranteed you’ll never get stuck in a deadlock.

  8. Hey Allen,

    Thanks for the Tutorial.
    I’m just diving into Unity and C# coding, though, I’ve done a bit of Arduino/C in the past.

    Super beginner question: how do I use the ArduinoConnector.cs code in unity?
    Do I attach it as a script to an object?
    Do I use another script to call it?

    For what it’s worth I have a super basic read protocol that simply calls sp.readline() from an Object’s Update() function and parses the line into variables – but for reasons you pointed it out there are big flaws.

    I would like to try your code, but I’m not sure how/where to use it in unity…

    Thanks for your tutorial and for your patience.

    • Okay – So I added the StartCoroutine(AsynchronousReadFromArduino(..)) function inside of a start() function in the ArduinoConenctor.cs file your provided.
      Then I attached the script to an object.
      Then set comport and Baudrate to match my arduino.
      But when I run, I see nothing printed to the console.
      I’d appreciate any suggestions to get this running.

      • Correction: I see an error printed to the console:

        NullReferenceException: Object reference not set to an instance of an object
        ArduinoConnector+c__Iterator0.MoveNext () (at Assets/Scripts/ArduinoConnector.cs:75)
        UnityEngine.SetupCoroutine.InvokeMoveNext (IEnumerator enumerator, IntPtr returnValueAddress) (at C:/buildslave/unity/build/Runtime/Export/Coroutines.cs:17)
        UnityEngine.MonoBehaviour:StartCoroutine(IEnumerator)
        ArduinoConnector:Start() (at Assets/Scripts/ArduinoConnector.cs:22)

        • Okay, so now I call Open(); in my Start() function before StartCoroutine(..).
          Now when I run game, I get only a single Error to the console:

          Error!
          UnityEngine.Debug:LogError(Object)
          ArduinoConnector:m__1() (at Assets/Scripts/ArduinoConnector.cs:27)
          c__Iterator0:MoveNext() (at Assets/Scripts/ArduinoConnector.cs:102)
          UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)

          I tried calling ReadFromArduino() in my Update() function. But I just get a string of errors on each call to Update:
          ArgumentOutOfRangeException: Argument is out of range.
          Parameter name: value
          at System.IO.Ports.SerialPort.set_ReadTimeout (Int32 value) [0x00000] in :0
          at (wrapper remoting-invoke-with-check) System.IO.Ports.SerialPort:set_ReadTimeout (int)
          at ArduinoConnector.ReadFromArduino (Int32 timeout) [0x00008] in C:\Users\Lewis\Documents\Unity\Roll a Ball\Assets\Scripts\ArduinoConnector.cs:57
          at ArduinoConnector.Update () [0x00003] in C:\Users\Lewis\Documents\Unity\Roll a Ball\Assets\Scripts\ArduinoConnector.cs:35

          Any guidance really appreciated. Perhaps a zipped file of a unity project/scene that correctly uses the ArduinoConnector.cs file.

          Thanks again for your patience.

  9. I don’t know if this forum is still active, but I’m getting a lot of Unity errors using this code. Missing namespaces, undeclared variables, etc. Someone who can help me with this?

    • Hey!
      I made a couple of corrections to few types in the code, sorry about that!

      Is there any specific issue you’re experiencing?
      The code is provided in snippet, which I know is know ideal.
      But missing namespaces and undeclared variables should be pretty easy to include and add! Sorry about that! :\

  10. I’m going to second Lewis’s question. Your code is extremely readable (thank you for that), but it’s unclear how I’m supposed to call your Open(), WriteToArduino(), Close() functions. Do I call it from another script? Attach it to an object?

    When I try to create an ArduinoConnector in another script (where I intend to send different messages based on different conditionals), I too get the “NullReferenceException.” How do I reference ArduinoConnector in another script?

    Right now, in a separate script, I create a “public ArduinoConnector connector;” Then in Update(), under certain conditions, I call connector.Open(), connector.WriteToArduino(“specific condition trigger”), connector.Close().

    Thank you for your help!

    • Hi Emily!

      Apologies if this tutorial has not been very clear!
      It was intended for relatively experienced devs. I might re-view it soon, to make sure is still accessible to most developers!

      Yes, what you are doing is indeed correct. You should create a separate script that runs the logic of your game. The “connector” variable is ok. But obviously you will get a “NullReferenceException”, as that is not linked to anything in particular.

      To make the connection, you have to drag your AdruinoConnector component inside the slot of your “public ArduinoConnector connector”. I hope that makes it clear!

Webmentions

  • Asynchronous Serial Communication - Alan Zucconi August 22, 2017

    There’s a slight change to pingHandler function

    https://stackoverflow.com/questions/45811909/void-const-char-to-void-fpermissive/45812041#45812041

  • Process Blog 014 | Activation August 22, 2017

    There’s a slight change to pingHandler function

    https://stackoverflow.com/questions/45811909/void-const-char-to-void-fpermissive/45812041#45812041

  • How to Write Libraries for Arduino – Alan Zucconi August 22, 2017

    There’s a slight change to pingHandler function

    https://stackoverflow.com/questions/45811909/void-const-char-to-void-fpermissive/45812041#45812041

  • Personalized Joystick with arduino (for Unity) | Raspi Game August 22, 2017

    There’s a slight change to pingHandler function

    https://stackoverflow.com/questions/45811909/void-const-char-to-void-fpermissive/45812041#45812041

  • Joystick personalizado con Arduino (para Unity) | Raspi Game August 22, 2017

    There’s a slight change to pingHandler function

    https://stackoverflow.com/questions/45811909/void-const-char-to-void-fpermissive/45812041#45812041

  • A Bestiary of Alternative Game Controllers - Alan Zucconi August 22, 2017

    There’s a slight change to pingHandler function

    https://stackoverflow.com/questions/45811909/void-const-char-to-void-fpermissive/45812041#45812041

  • Everything You Need to Know About LEDs - Alan Zucconi August 22, 2017

    There’s a slight change to pingHandler function

    https://stackoverflow.com/questions/45811909/void-const-char-to-void-fpermissive/45812041#45812041

  • How to build a distance sensor with Arduino - Alan Zucconi August 22, 2017

    There’s a slight change to pingHandler function

    https://stackoverflow.com/questions/45811909/void-const-char-to-void-fpermissive/45812041#45812041