Exploring how mobile web browsers can be used to interact with physical objects.

From the Blog

Aug
10

Explorations in mobile web connected physical devices

Posted by ajfisher on August 10th, 2011 at 12:43 pm

Smart phones are becoming more sophisticated devices by the month it seems – you can even make phone calls with them I’m told. I’ve played around quite a bit with using bluetooth as a connection method between arduinos and mobile phones and whilst it works fairly well there’s a lot of overhead and your battery life is terrible for both phone and device.

One of the areas I’m looking into at the moment though is how mobile web browsers, particularly using the device APIs and web sockets can create interesting interaction points with nothing more than a browser on one side and then some interpretation of messages on the other.

Presently to achieve this I’m using Django Socket IO – a great little web sockets server and because it’s written in Python allows you full access to a serial connection to then talk to the arduino.

On the browser side it’s simply a case of hooking the relevant Device Event – devicemotion or deviceorientation then providing a call back to do something with it – in this case sending the data to a web socket connection.

// assumes some socketio code here to register to a room.
 
window.addEventListener("deviceorientation", update_gyro, true);
 
function update_gyro(e) {
	// gets the gyro position
    var x, y, z = 0;
    var o = deviceOrientation(e);
 
	update_text(o.gamma, o.beta, o.alpha);
    if ((new Date()).getTime() - last_sent > threshold) {
		socket.send({room: room, action: 'movement', x: o.gamma, y: o.beta, z: o.alpha, method: 'orientation'});
    }
}

After this we write a handler to take the data and do something with it. In this case a Django view that is a web socket message handler.

import serial
 
from django.shortcuts import get_object_or_404, render, redirect
from django_socketio import events
 
SERIAL_INTERFACE = "/dev/ttyUSB0"
SERIAL_BAUD = 115200
 
try:
    ser = serial.Serial(SERIAL_INTERFACE, SERIAL_BAUD, timeout=60)
except:
    print "Can't get a serial connection"
 
@events.on_message(channel="^light")
def message(request, socket, message):
    message = message[0]
    if message["action"] == "movement":
        socket.send({"action": "ack"})
 
        # pick up the values from the socket message
        x = int(message["x"])
        y = int(message["y"])
        z = int(message["z"])
 
        # now normalise the values as needed
        if message["method"] == "orientation":
            # put the vals back into +ive integer range as needed
            x += 90 #normalise 0-180
            y += 360 # normalise 0-360
 
        if x > 180:
            x = 180
        if y > 360:
            y = 360
        if z >= 360:
            z = 0
 
        # work out the y bytes
        # leaving this like this even though it's normalised back to 180 deg.
        # just in case there's any more changes to firefox.
        yh = y >> 8
        if y > 255:
            yl = y - 256
        else:
            yl = y
        # work out the z bytes
        zh = z >> 8
        if z > 255 :
            zl = z - 256
        else:
            zl = z
 
        #print "x: %s y: %s z: %s" % (x, y, z)
        try:
            ser.write("%s%s%s%s%s%s%s" % (chr(255), chr(255), chr(x), chr(yh), chr(yl), chr(zh), chr(zl)))
        except:
            #do  nothing - this is a good test anyway.
            print "Doing nothing as no serial"
 
    elif message["action"] == "test":
        # this is a test of the socket
        print "Test of the socket"
        socket.send({"action": "bcast", "message": "got a test"})

 

So this is really really early stage at this point and I’ve got a couple of demos that I’m building at the moment so will post them once they’ve solidified.