One of the major challenges of ubiquitous computing is the context - What is being used? Who is using it? Where is it being used? When was it last used? How is it being used?
Getting answers to these questions will enable innovative applications and formidable use cases for this data. Everything from finding your keys to locating an unused workspace in a shared environment relies on this kind of data, adding tracking and inventorying capabilities that do not exist yet.
The What and the When are relatively easier problems to solve - former being a reference to self, and the latter being a timestamp pulled from the Internet. However, the Who is a relatively harder problem - with NFC or RFID that are more power-hungry and size-heavy protocols. The Where is an even harder problem - reliable indoor localization does not adapt well and is time consuming to configure. The How, of course, is up to the device, and will be an optional metadata field that a smart device will fill in.
We want to build such a system, keeping track of all these fields for different objects. We want to create an embeddable system that can provide context awareness to the device it is attached to, that stores the history of whenever an object is moved, and logs this data into a data store, opening it up for use by other applications in multi-modal interfaces.
Type Class, Team project Role Architecture, Component Sourcing, Hardware Design, Use Case Demonstration Duration 4 months (Jan 2018 - Apr 2018) Tools Arduino, Raspberry Pi, RF Protocols, Audio Interfacing, Conversational UIs, Databases Objectives System and Solution Architecture, Context Gathering, Ubiquitous Computing, Signal Processing, Indoor Localization, Power Optimization
The research publication Charting Past, Present, and Future Research in Ubiquitous Computing by Gregory Abowd and Elizabeth Maynatt talked about the importance, representation and ubiquity of context, and coupling it with awareness, natural interaction and automated capture. Reading this paper got us deeply interested in creating a system that gathers this kind of context, because often, that is the first step to making context-aware solutions.
For our design to be practical as a ubiquitous system, we had to work around some constraints.
My first form was a small cube with electrical contacts on one side, attaching magnetically to whatever device it was talking to. After playing with some mockups, I discovered that the form is not stable enough, and would be easy to lose. I gathered a few other objects around my desk to see if any of them would serve as a good starting point for design. These included a Mini Wireless Router, a Screen Fluid spray bottle, a bike light with rubber loops at the back, a remote control and so on.
I had once found this discarded Fitbit Ultra and thought it would be an interesting design to explore. This second-generation device was created after complaints from users about losing their fitbits, so it was supposed to be a tighter hinge. It also contained several sensors inside, along with a rechargeable battery and a radio that talked to the base station using the ANT protocol.
Therefore, while creating the recommendation for the final form, we would keep this as the reference design, with the wireless radio, battery and accelerometer-based wakeup mechanism while redesigning the contacts for power and data, as well as interaction mechanisms to have it as close to zero-workflow as possible.
The What, a self-referential ID:
The When, a timestamp obtained from the Internet or maintained locally:
The Where, a localized coordinate with respect to some reference point:
The Who, the person or entity making use of this object:
The How, metadata about usage:
The Why, to infer intent:
When to Wake up:
We plan to implement a voice interface and a Slack based bot as example applications that can answer simple questions like "Where is my ****" or "Who used this **** last?"
Context-sensing, in architecture terminology, is a "cross-cutting" concern within and across most, if not all, IoT applications. Hence the solution can be designed as a generalized component that will serve all such applications, perhaps with some configuration capability.
For example, a VR program could use this data to place things in a virtual world as it is in the real world. A store can keep track of who picked what, similar to Amazon Go. We will add more details as we progress through the project.