Serial

Tkinter with Serial

Evan Boldt — Sat, 04 May 2013 04:46:07 +0000

Tkinter with Serial

Topics

Python

Serial

GUI

Evan Boldt Fri, 05/03/2013 - 23:46

Introduction

To use Python as a graphical interface for an Arduino powered robot, programmatically read the USB with the pySerial library. However, waiting for input from pySerial's Serial object is blocking, which means that it will prevent your GUI from being responsive. The process cannot update buttons or react to input because it is busy waiting for the serial to say something.

The first key is to use the root.after(milliseconds) method to run a non-blocking version of read in the tkinter main loop. Keep in mind that when TkInter gets to the root.mainloop() method, it is running its own while loop. It needs the things in there to run every now and then in order to make the interface respond to interactions. If you are running your own infinite loop anywhere in the code, the GUI will freeze up. Alternatively, you could write your own infinite loop, and call root.update() yourself occasionally. Both methods achieve basically the same goal of updating the GUI.

However, the real issue is making sure that reading from serial is non-blocking. Normally, the Serial.read() and Serial.readline() will hold up the whole program until it has enough information to give. For example, a Serial.readline() won't print anything until there is a whole line to return, which in some cases might be never! Even using the after() and update() methods will still not allow the UI to be updated in this case, since the function never ends. This problem can be avoided with the timeout=0 option when enitializing the Serial object, which will cause it to return nothing unless something is already waiting in the Serial object's buffer.

Code

from serial import *
from Tkinter import *

serialPort = "/dev/ttyACM0"
baudRate = 9600
ser = Serial(serialPort , baudRate, timeout=0, writeTimeout=0) #ensure non-blocking

#make a TkInter Window
root = Tk()
root.wm_title("Reading Serial")

# make a scrollbar
scrollbar = Scrollbar(root)
scrollbar.pack(side=RIGHT, fill=Y)

# make a text box to put the serial output
log = Text ( root, width=30, height=30, takefocus=0)
log.pack()

# attach text box to scrollbar
log.config(yscrollcommand=scrollbar.set)
scrollbar.config(command=log.yview)

#make our own buffer
#useful for parsing commands
#Serial.readline seems unreliable at times too
serBuffer = ""

def readSerial():
    while True:
        c = ser.read() # attempt to read a character from Serial
        
        #was anything read?
        if len(c) == 0:
            break
        
        # get the buffer from outside of this function
        global serBuffer
        
        # check if character is a delimeter
        if c == '\r':
            c = '' # don't want returns. chuck it
            
        if c == '\n':
            serBuffer += "\n" # add the newline to the buffer
            
            #add the line to the TOP of the log
            log.insert('0.0', serBuffer)
            serBuffer = "" # empty the buffer
        else:
            serBuffer += c # add to the buffer
    
    root.after(10, readSerial) # check serial again soon


# after initializing serial, an arduino may need a bit of time to reset
root.after(100, readSerial)

root.mainloop()

Disqus

Reading Numbers From Serial

Evan Boldt — Sun, 17 Feb 2013 23:32:48 +0000

Reading Numbers From Serial

Topics

Evan Boldt Sun, 02/17/2013 - 17:32

Introduction

Reading numbers from serial on an Arduino is needed surprisingly commonly. Exactly what is happening might be kind of hard to figure out.

Serial

Serial communication is digital, which means all data is transmitted in 1's and 0's. Typically, serial communication is done using ASCII letters. This means that, to send a number to the Arduino, the data sent is not the binary version of the number in base 2 (as an integer), but instead a sequence of characters for each digit in base 10 (which is human-readable). It is not an efficient use of bandwidth, but bandwidth is not usually a problem with Arduinos connect by USB.

Arduino Code

char commandLetter;  // the delineator / command chooser
char numStr[4];      // the number characters and null
long speed;          // the number stored as a long integer

void serialEvent() {
  // Parse the string input once enough characters are available
  if(Serial.available() >= 4) {
    commandLetter = Serial.read();
    
    //dump the buffer if the first character was not a letter
    if ( ! isalpha( commandLetter ) {
      // dump until a letter is found or nothing remains
      while(( ! isalpha( Serial.peak() ) && Serial.available()) {
        Serial.read(); // throw out the letter
      }
      
      //not enough letters left, quit
      if ( Serial.available() < 3 ) {
        return;
      }
    }
    
    // read the characters from the buffer into a character array
    for( int i = 0; i < 3; ++i ) {
      numStr[i] = Serial.read();
    }
    
    //terminate the string with a null prevents atol reading too far
    numStr[i] = '\0';
    
    //read character array until non-number, convert to long int
    speed = atol(numStr);
    Serial.println(speed);
  }
}

Code Explanation

serialEvent

serialEvent is a function that is called by Arduino before each loop. So, you can separate out your serial handling from the rest of your code. You could also put the contents of this function in the loop() function and it would be similar.

Fixed-Width

Commands need to be delineated in some way. Fixed-width commands are convenient on Arduino because the Serial.available function tells you how many characters are waiting. It is also possible to delineate by special characters. An extremely common and simple version of this is a Comma Separated Value file. However, since it is simplest to use C strings, the character arrays which the serial is read into is necessarily fixed-width anyway. To get around this issue, a vector or linked list could be used, so the serial string can dynamically grow and shrink.

Command Differentiation

In many cases, a robot takes multiple kinds of inputs. For example, there might be multiple speeds, or a direction, or some kind of trigger. Prefixing each command with a unique letter can make it easy to differentiate what the following string is supposed to do.

C Strings

C strings are arrays of char data types. They must be terminated by a NULL character '\textbackslash 0'. Without the NULL character at the end, there is no way for any function to know how long the string is. For example, a print function would print the string and then a bunch of garbage characters until it gets to a NULL or the end of the memory available. In this case, it's less likely for that to happen since we are only using atol, which will only read until there is a non-numeric character. A non-numeric byte is more likely to randomly be at the end of the string than a NULL character.

ASCII Table
Decimal	Character
48	'0'
49	'1'
50	'2'
51	'3'
52	'4'
53	'5'
54	'6'
55	'7'
56	'8'
57	'9'

Char to Integer

Converting a single character to a number takes advantage of how characters are stored, which is typically in a format called ASCII. Think of a char of ASCII like an integer data type. Except that when printed, it is not a decimal number. The number in the int is translated into a letter on the screen according to a table. For example, the decimal number 97 is translated into the letter 'a'. Each letter is really stored as a 'number'. I'm showing it as a decimal (base 10) number, but it is really stored in binary (base 2). The C++ compile takes care of translating our base 10 decimal numbers into base 2 binary.

Here is how to take advantage of this:

int number = (int)charInput - (int)48;
// or:
int number = charInput - '0';

Since ASCII stores the letters sequentially, starting at 0, just subtract the value where 0 begins from each of the characters to convert it to its numeric form. This works regardless of the base of the character storage. So, it converts the ASCII character representing the number in base 10 into an integer in base 2.

Char Array (C-string) to Long Integer

Now, each of the letters from right to left need to be multiplied by a power of 10. Here is the basic idea of what atol does:

char charArray[5] = "5432"; // double quotes adds the \0 at the end

long longnum = 0;  //the converted number
long exp = 1;      //start with 10^0
int index = 0;     //the current place in the array

//find the end of the string - when character is not a number
while ( isdigit( charArray[idx] ) ) {
    ++idx;
}

//begin base multiplication and number conversion
while( idx >= 0 ) {
    //convert digit character to a long
    long currentDigit = (long) charArray[idx] - (long)48
    
    currentDigit *= exponent; //number multiplied by 10^(len-idx)
    exponent *= 10; // next number to left is another power of ten
    
    longnum += currentDigit;
    
    --idx;
}

Disqus

Python Web UI with Tornado

Evan Boldt — Fri, 08 Feb 2013 18:10:44 +0000

Python Web UI with Tornado

Topics

Evan Boldt Fri, 02/08/2013 - 12:10

Introduction

So, you want a GUI for your robot. Sure, you could make a nice GUI in Python with Tkinter, but there are some really good reasons to try it as a website instead:

A web GUI can be accessed from almost any location
A well-designed web GUI can be used on almost any device like a tablet or a phone
HTML, CSS, and Javascript are well documented, powerful, and very flexible
HTML can be easily made to look very nice

There are some drawbacks to implementing your GUI as a website though:

It is only able to start communication one-way (browser to server, server responds)

Great for buttons and other input
Requres continuous polling or a commet to get data to be pushed to the browser

It adds another "layer" of complexity to your code

Several different languages (C++ on Arduino, Python Server, and HTML+CSS+Javascript in browser)
The server has to relay information from the browser to the robot

Installing Python Tornado

I used the Tornado web server for my projects. It is better than just using CGI, since the environment is always loaded while the server is running instead of loading it every time a request is made. CGI would be problematic then, since serial.begin would be run every time a request was made, which simply wouldn't work since the Arduino would also restart all the time.

On Ubuntu, the installation is as easy as:

sudo apt-get install python-tornado python-serial

Using Tornado with the Serial Library

So one of the challenges with this is that reading serial input in python is usually blocking, so the webserver becomes unresponsive to browsers' requests while waiting for serial input.

#! /usr/bin/python2.7
import os
import json
import tornado.ioloop
import tornado.web
from serial import *


tornadoPort = 8888
cwd = os.getcwd() # used by static file server


# Make a Serial object
serialPort = '/dev/ttyACM0'
serialBaud = 9600
ser = Serial( serialPort, serialBaud, timeout=0, writeTimeout=0 )



# gets serial input in a non-blocking way
serialPending = ''
def checkSerial():
    try:
        s = ser.read( ser.inWaiting() )
    except:
        print("Error reading from %s " % serialPort )
        return
    
    if len(s):
        serialPending += s
        paseSerial()
    

#called whenever there is new input to check
serialHistory = ''
mostRecentLine = ''
def parseSerial():
    split = serialPending.split("\r\n")
    if len( split ) > 1:
        for line in split[0:-1]:
            print( line )
            
            #do some stuff with the line, if necessary
            #example:
            mostRecentLine = line
            # in this example, status will show the most recent line

            serialHistory += line

        pending = split[-1]


# send the index file
class IndexHandler(tornado.web.RequestHandler):
    def get(self, url = '/'):
        self.render('index.html')
    def post(self, url ='/'):
        self.render('index.html')


# handle commands sent from the web browser
class CommandHandler(tornado.web.RequestHandler):
    #both GET and POST requests have the same responses
    def get(self, url = '/'):
        print "get"
        self.handleRequest()
        
    def post(self, url = '/'):
        print 'post'
        self.handleRequest()
    
    # handle both GET and POST requests with the same function
    def handleRequest( self ):
        # is op to decide what kind of command is being sent
        op = self.get_argument('op',None)
        
        #received a "checkup" operation command from the browser:
        if op == "checkup":
            #make a dictionary
            status = {"server": True, "mostRecentSerial": mostRecentLine }
            #turn it to JSON and send it to the browser
            self.write( json.dumps(status) )
        
        #operation was not one of the ones that we know how to handle
        else:
            print op
            print self.request
            raise tornado.web.HTTPError(404, "Missing argument 'op' or not recognized")


# adds event handlers for commands and file requests
application = tornado.web.Application([
    #all commands are sent to http://*:port/com
    #each command is differentiated by the "op" (operation) JSON parameter
    (r"/(com.*)", CommandHandler ),
    (r"/", IndexHandler),
    (r"/(index\.html)", tornado.web.StaticFileHandler,{"path": cwd}),
    (r"/(.*\.png)", tornado.web.StaticFileHandler,{"path": cwd }),
    (r"/(.*\.jpg)", tornado.web.StaticFileHandler,{"path": cwd }),
    (r"/(.*\.js)", tornado.web.StaticFileHandler,{"path": cwd }),
    (r"/(.*\.css)", tornado.web.StaticFileHandler,{"path": cwd }),
])


if __name__ == "__main__":
    
    #tell tornado to run checkSerial every 10ms
    serial_loop = tornado.ioloop.PeriodicCallback(checkSerial, 10)
    serial_loop.start()
    
    #start tornado
    application.listen(tornadoPort)
    print("Starting server on port number %i..." % tornadoPort )
    print("Open at http://127.0.0.1:%i/index.html" % tornadoPort )
    tornado.ioloop.IOLoop.instance().start()

Setting up the Browser

To make it easy to send requests to the server, it's really a good idea to use jquery, and the easiest way to use jquery is to use the Google API for it. To add jquery to the webpage, just add this script:

<script src="//ajax.googleapis.com/ajax/libs/jquery/1.9.0/jquery.min.js"></script>

So to start, we'll just have a simple webpage that has a div. When the div is clicked, jquery will ask the status of the tornado webserver.

<!DOCTYPE html>
<html>
<head>
<title>Python Tornado Test Page</title>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js"></script>
<script>
function serverResponded( data ) {
    /* 
    log the event data, so you can see what's going on.
    Shows up in the console on your browser. (Chrome: Tools > Developer Tools > Console)
    */
    console.log( data );
    
    // check the server status, and report it on the screen
    if ( data.server === true ) {
        $('#status .value').html("OK");
    }
    else {
        $('#status .value').html("NOT OK");
    }
    
    // add the last serial to the div on the screen
    $('#serial .value').html( data.mostRecentSerial );
}

$(document).ready( function() {
    /* handle the click event on the clickme */
    $('#clickme').click( function() {
        params = { op: "checkup" };
        $.getJSON( 'http://localhost:8888/com' , params, serverResponded );
    });
});
</script>
</head>
<body>
    <div id="clickme" style="cursor: pointer;">CLICK ME</div>
    
    <div id="status">
        Server Status: <span class="value">?</span>
    </div>
    
    <div id="serial">
        Last Serial Input: <span class="value"></span>
    </div>
</body>

Disqus

Arduino to Arduino Serial Communication

Jenn Case — Wed, 06 Feb 2013 19:10:03 +0000

Arduino to Arduino Serial Communication

Topics

Arduino

Serial

Jenn Case Wed, 02/06/2013 - 13:10

Introduction

It is possible to chain Arduinos together in such a way as to get communication between the two. Having Arduino-Arduino communication can be useful for many projects, such as having one Arduino to run motors and having another sense the surroundings and then relay commands to the other Arduino. This can be done in several methods, using I2C and Serial, to list a few.

This tutorial will focus on Arduino-Arduino communication through the serial ports (RX and TX).

Schematic

The schematic below shows how to connect the two Arduinos together. This shows two Unos, but if a Mega is used, it can be connected to any of the Serial ports on the Mega as long as that is accounted for in the code.

There has to be a common ground between the two or else it will not function properly. Also, note that TX goes to RX and RX goes to TX.

Coding

When sending things through serial, everything is sent in bytes. These bytes are then read one byte at a time by the other Arduino. When it is just characters being sent through the serial, it is relatively easy to convert from characters to bytes. However, if there are both characters and numbers are going through, this can lead to messing up the data because a number and a character can have the same byte value, but that does not make them the same. Numbers are also tricky because they may not actually fit in the byte.

Simple Code

The easiest way to get around this is to try to avoid using characters and numbers at the same time. This can be done by sending one character across, each with a different meaning. A good example of this comes from the Arduino Physical Pixel tutorial.

Upload the Physical Pixel code, which can be found in the Arduino IDE under: File >> Examples >> Communication, onto one Arduino.

On the other Arduino, upload:

void setup() {
  Serial.begin(9600);
}

void loop() {
  Serial.print('H');
  delay(1000);
  Serial.print('L');
  delay(1000);
}

When this is run, the LED attached to Pin 13 on the Arduino with the Physical Pixel code should flash on and off at a frequency of 0.5 Hz. To make sure this is actually the code doing that, the delays can always be changed in the above code.

In this code the job of 'H' was to turn an LED on and the job of 'L' was to turn the LED off. This can easily be applicable to getting various characters triggering more reactions.

However, depending on the application, this may not be enough and more drastic code may be required.

Complex Code

For sending data from one Arduino to another in a form that cannot be simplified, there are other options. One option is to turn everything sent from the Sender Arduino into characters and then have the Receiver Arduino read in the characters. The data is actually sent as bytes, but the Arduino can convert from characters to bytes and vice versa.

Sender Code

The sender code changes characters into bytes and, if necessary, it changes number values into characters before turning it into bytes. Below is a sample of the Sender code:

//Sender Code

char str[4];

void setup() {
  Serial.begin(9600);
}

void loop() {
  int value=1234; //this would be much more exciting if it was a sensor value
  
  itoa(value, str, 10); //Turn value into a character array
  Serial.write(str, 4);
}

Receiver Code

The receiver will then receive the byte array from the other Arduino and interpret it there. Below is the code for the receiver. Note that this code is intended for a Mega since it will interpret the data received from the other Arduino and then print to the Serial Monitor what it received so that the user can check it. This debugging can be avoided by using an Uno and then printing what was found onto an LCD screen that uses I2C communication.

//Receiver Code

char str[4];

void setup() {
  Serial.begin(9600);
  Serial1.begin(9600);
}

void loop() {
  int i=0;

  if (Serial1.available()) {
    delay(100); //allows all serial sent to be received together
    while(Serial1.available() && i<4) {
      str[i++] = Serial1.read();
    }
    str[i++]='\0';
  }

  if(i>0) {
    Serial.println(str,4);
  }
}

There is one flaw with this program as it is now. It results in only a character array, so if the integers were desired, they are lost without further work on the data.

Further tutorials have been made to show how this data may be manipulated by splitting strings or getting floats from the character arrays.

Disqus

Serial Commands

Evan Boldt — Wed, 30 Jan 2013 23:11:35 +0000

Serial Commands

Topics

Evan Boldt Wed, 01/30/2013 - 17:11

Introduction

Serial communication through USB can be used to communicate between a host computer (like a laptop, or Raspberry Pi), and an Arduino. Doing so allows:

Use of the faster hardware of the host computer for calculations
Synchronization of multiple Arduinos
Use of other information provided by the host computer, like internet connections or larger data files
Creation of a graphical interface using the host's display or web server

Communication

Communication is easy. Arduino and Python have simple methods for establishing serial communication.

Arduino

void setup() {
    Serial.begin(9600); //Initialize serial communication at 9600 baud
}
void loop() {
    Serial.println('hello');
}

Python

from serial import *
ser = Serial( "/dev/ttyACM0" , 9600 )

while 1:
    print ser.readline()

Incomplete Messages

One of the most difficult issues with this sort of serial communication is ensuring that commands are received in entirety. If you send "1000" from Python, and want Arduino to read it as shown below, Arduino might read "10" and then in another loop() right afterward read "00".

char incoming[11] = "          ";
int i = 0;
while (Serial.available() > 0) {
    // read the incoming byte:
    char[i] = Serial.read();
    i++;
}
//Convert the incoming string into an integer!
int newNumber = atoi(incoming);

Fixed Length Packets

One way to solve this in an efficient manner is to use a fixed length for commands and wait until there are enough bytes waiting in the buffer. The downside is that no commands may be issued with a length longer than the fixed length, and requires remaining length be wasted.

Python

ser.write('%10d' % 1000 ) #pad the number so it is 10 characters long

Now just add a condition so that no reading is done unless a full command is waiting in the serial buffer.

Arduino

if ( Serial.available() >= 10 ) {
    char incoming[11] = "          ";
    int i = 0;
    while (Serial.available() > 0) {
        // read the incoming byte:
        char[i] = Serial.read();
        i++;
    }
    //Convert the incoming string into an integer!
    newNumber = atoi(incoming);
}

Delineation

Another way to solve this problem is to delineate commands by placing a special character at the end of each command to signify that the command is complete. The problem with this way is that the length of the incoming string is unknown, and a long or improperly formed command could overflow a fixed length array. To overcome this limitation, a dynamic class is usually used. The String object is a good solution as it can be easily converted back into a character array. Maintaining data types with C++ strings can quickly become confusing, and it is extremely important that the delineation character is never used in the command. There are some special command characters in ASCII that would be good choices.

Multiple Command Types

You can send different commands over the same communication line by attaching a prefix to the command. For example, speed and position could be sent as "p000001000" and "s000000200" to tell the Arduino to go to position 1000 at a speed of 200.

Python

ser.write('p%9d' % 1000 ) #send position, padded to 9 (plus 1 command char)
ser.write('s%9d' % 200 )  #send speed, padded to 9 (plus 1 command char)

Arduino

int position, speed;
if ( Serial.available() >= 10 ) {
    //read the first character as the command
    char command = Serial.read();
    
    char incoming[10] = "         ";
    int i = 0;
    // read the incoming byte:
        char[i] = Serial.read();
        i++;
    }
    //Convert the incoming string into an integer, based on command
    if ( command == 'p' ) {
        position = atoi(incoming);
    }
    else if ( command == 's' ) {
        speed = atoi(incoming);
    }
    else {
        //The command was not one of the recognized characters
        Serial.println("COMMAND ERROR - Not recognized");
    }
}

Now that your communication is established, it is up to you to figure out what to do with the data on either end.

Disqus