1. We are looking forward to your feedback!

Hi __FORENAME__, this lab is quite new. Therefore we are even more looking forward to your comments and suggestions than in the other labs!

We tested all components intensively but still we may have overlooked some things. So keep in mind that not everything in this lab must necessarily work as intended.

Whenever you have suggestions on how we can improve the lab (e.g. by adding specific information, giving this or that additional instruction) please let us know via the comment field. Thank you!

The more precise your feedback is and the clearer your suggestion are, the better we can implement them... Thank you!

Have fun with the Smart Space Lab!

What is this lab about?

This lab is about the Distributed Smart Space Orchestration System (DS2OS). The aim of DS2OS is to enable the emergence of Smart Spaces from research labs into everyday life.

But what is a Smart Space? A Smart Space is a physical environment that is enriched with networked Smart Devices.

But what are Smart Devices? Smart Devices are hardware devices that contain an embedded system with a microprocessor and a network interface and sensors and actuators the can be used to interact with the physical world.

From their interaction, the systems we look at in this lab, are also often called Cyber-Physical-Systems (CPS). Another term that is common in the domain we are looking at is Internet of Things (IoT). It describes that entities are connected to the Internet and can be controlled from remote.

As you will learn in this lab, there are two domains of high relevance when creating smart spaces:

• Context management.
• Service management.

DS2OS covers these two domains with its parts:

• Virtual State Layer (VSL) -- a context-aware µ-middleware.
• Smart Space Store (S2S) -- a service management framework that provides similar functionality to a smartphone App Store to smart spaces.

In the practical part of this lab you will cover everything that is necessary for creating your own Smart Spaces in a Do-It-Yourself (DIY) way:

• You will create your own smart device based-on the Arduino platform, and write your firmware to make it remote controllable.
• You will write a device driver for your device to make it accessible in a more abstract way.
• You will write an Advanced Reasoning Service to derive information from your device and other context sources, and to make it available for other services. This kind of service helps modularizing functionality in DS2OS.
• Finally you will write an Orchestration Service that realizes a complex work flow in a simple way.

This lab has several learning goals, including:

• Reading and understanding research papers.
• Learning something about the history of computing.
• Getting familiar with
  • Ubiquitous Computing
  • Pervasive Computing
  • Middleware
  • Service Oriented Architecture
  • Distributed Systems
  • A complex distributed peer-to-peer system
• Evaluating a complex system.
• Statistical analysis of measurement results.

Befor we continue, here some funny things we did with the system in 2011:

As the system has grown since then you can easily reproduce this demo after having done this lab. Think about what you could do if such a system would run everywhere you are...

In this lab you will find some research papers. Take the opportunity not only to get information out of them but also to get familiar with the style such papers are written. For your bachelor thesis, master thesis, or maybe a Phd it is important that you can communicate with your scientific peers in an appropriate way. Reading such written documents is a good exercise to get familiar with the scientific world.

The numbers in brackets [1] are links to research papers as you find them in scientific papers. Like in a paper you find the list of references at the end of this prelab.

This lab only contains a small selection of papers. If you are interested in certain aspects you can use the references of the papers listed here as a starting point.

An Evolution of Computing

Let us start with a look at computing [3]:

In the 1960s the computing world was dominated by mainframe computers. Many people shared one mainframe that was operated by professional administrators.

In the 1980s Perconal Computers (PC) emerged. Over the years people had PCs at home. The relation between users and PCs shifted to few users that shared a PC. PCs were typically administrated by their users.

In the 2000s Mobile Computing became reality. Smartphones and Pads became common computing devices. Now one user typically had multiple devices on her own.

In the 2010s it is likely that sharing devices becomes more popular again. Public devices such as public screens will be available to all passing humans resulting in a many-to-many device-individual relationship.

Looking at the development from a spatial point-of-view, computing moved from dedicated computing centers to offices, to homes, to everywhere. The trend of public devices will make this ubiquity even more reality.

An evolution of computing [Pahl2013]

The Beginning of Ubiquitous Computing

Did you ever hear about Xerox? It is the company that holds the patent on Xerography which is used in many of the copy machines. The company used the money it made to found research in its famous Xerox Palo Alto Research Center (PARC). Many of the technologies that are common to you today found their origin at this lab.

While doing brillant research, Xerox Parc did not manage to commercialize their ground-braking results. Other companies like Apple did that job.

In 1991, a group at Xerox Parc that was led by Mark Weiser worked on a vision they called Ubiquitous Computing. When looking at the picture, reading the papers [1,2], and watching the video keep in mind that this was 1991.

Fig. 1: M. Weiser, “The Computer for the 21st Century,”
Scientific American, Sep. 1991.

Take a look at Fig. 1. Would you have guessed that this picture was taken in 1991? The picture shows a demonstrator at Xerox Palo Alto Research Center (PARC) in California. The following video gives an impression from the demo:

Inform yourself about Xerox PARC and the inventions made there.

The paper [1] marks the beginning of a new research field: ubiquitous computing.

Read the papers [1,2,3,4].

According to [4], UbiquitousComputing can be divided into the areas mobile computing and pervasive computing. See Fig. 2.

Fig. 2: According to K. Lyytinen and Y. Yoo, “Issues and Challenges in Ubiquitous Computing,” Commun. ACM, vol. 45, no. 12, Dec. 2002.

Mobile computing can be paraphrased as "computing everywhere", pervasive computing can be described as "computing in everything".

Mobile computing is reality today:

Even in developing countries cellular networks that are capable of transporting data are available today. Smart devices like smartphones, tablet, laptops, etc. offer network access from almost everywhere [5].

Cloud computing offers the remote use of literally unlimited computing ressources via the devices named before [6].

Pervasive computing is not reality yet [4].

Devices that contain processors are ubiquitous. Even networked sensors and actuators could be bought off-the-shelf at affordable prices in regular shops in 2013 [7].

Heterogeneity is a big problem for realizing pervasive computing scenarios today. Think on the user story about Sal that you read in [1]. The technology is available. You could probably also program a solution. But it would be custom built. You could not run it at your friends' places, or with different hardware.

This lab is about overcoming this heterogeneity.

To realize scenarios where devices interact it is necessary that the involved devices can interact. For different reasons this is not given. Instead hardware devices are often heterogeneous in their interfaces which makes controlling them remotely complex.

Reasons for Heterogeneity

There are several reasons why the communication protocols used by smart space hardware devices are heterogeneous. We will present some of them now.

• Historically the different domains for orchestrating a space like heating, ventilation, air conditioning, lighting were separated. Different vendors offered (usually closed) solutions for the domains. Connecting the formerly separate domains is still new. The domains were not build for such a functional merge.
• Market tactical considerations are a reason for heterogeneity. When hardware of vendor A is not interoperable with hardware of vendor B consumers lock in to a vendor. A consumer that has several devices of vendor A cannot get tempted to buy hardware of vendor B as it would not extend the smartness of the space but create another functional island.
• Technically different device one can buy to sense and actuate (orchestrate) a space serve heterogeneous purposes. According to their intended use they bring different ressources (like CPU, memory, network interface, power supply) and they are mounted in different ways. Extremes are PC-like embedded devices like Raspberry PIs on one end and integrated, battery-powered, wirelessly connected devices on another end.

The historic dimension may disappear with time. The market tactical reason could disappear with standardisation. The technical reasons for heterogeneous communication interfaces will not disappear as they make sense to take best advantage of the requirements of a device.

There are different automation systems that are used in commercial buildings. The following text gives a brief overview on some of the technologies used.

Devices are the interface between software and the physical world. Many devices that are installed today are parts of so-called Building Automation Systems.

There are several wide spread technologies and standards that are used in (commercial) buildings all over the world today.

Often those technologies focus on the core domains heating, ventilation, air conditioning, and lighting (HVACL).

We have a quick look at some of the technologies to give you an overview.

European Installation Bus (EIB/KNX)

Typical devices that are connected using KNX are light sensors, motion detectors, switches, relays, motors, and heating controls.

The FMI building of Technische Universität München in Garching is using KNX for controlling the shutters and the lights for instance.

KNX communication is standardised over twisted pair, power line, radio, and Ethernet.

Data is transmitted in KNX frames as follows:

KNX LPDU standard frame structure (long frames allow N < 255) (source: KNX standard).

Some ressources if you are interested in more details:

• KNX specification here (local copy version 3.0).
• www.konnex.org

Local Operating Network (LON)

Similar to KNX LON is a normed bus system to connect entities within a building. It is a proprietary technology but it is widespread especially in the US.

Short description.

Typical domains where LON is used are heating, air conditioning, lighting, shutter control, security, multimedia.

LON is used in many commercial buildings, especially in the US.

For physical connectivity LON supports twisted pair, power line, radio, coax, fiber, infrared.

The LON topology can be hierarchical while the KNX is primarily a flat bus structure. LON sells special chips that do the communication. They are called Neuron chips.

LON communicates over LONtalk as follows:

LON NPDU/TPDU/SPDU Addressing (source: protocol specification)

Some ressources if you are interested in more details:

• Protocol Specification (local copy)
• Device Interface File Reference Guide (local copy)
• Layer 1 – 6 Interoperability Guidelines (local copy)
• Resource File API Reference Guide Revision 4 (local copy)
• www.echelon.com

Let's have a look at some more protocols that are used for managing and controlling elements within a spaces.

REpresational State Transfer (REST)

You all know REpresational State Transfer (REST) interfaces. The World Wide Web has a REST interface. The HTTP protocol contains methods like:

• GET
• POST
• PUT
• PATCH
• DELETE

They are used to retrieve or manipulate the representation of the state of a resource (e.g. a website). One property of REST communication is that it is usually stateless. The client and the server do not establish a connection and store context about each other.

REST interfaces in form of HTTP accessible websites are offered for many consumer devices today. They are often primarily intended for direct user interaction like the following interface of a remote control capable power plug:

Such interfaces can be scripted. In our example we can make the following call to set the state of the switch of the power plug:

http://192.168.4.1/r?b=1&r=0&s=[0|1]

With the following command we can retrieve the state of the switch of the device:

http://192.168.4.1/r?b=1&r=0&s=1

The returned HTML output is:

The web interface of our power plug.

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"><HTML><HEAD><TITLE>ALL3076</TITLE><STYLE TYPE="text/css">body { font-family: helvetica, sans-serif; color: black}</STYLE></HEAD><BODY LEFTMARGIN=0 TOPMARGIN=0 MARGINWIDTH=0 MARGINHEIGHT=0 BGCOLOR="#E0E0E0"><TABLE BORDER=0 WIDTH="100%" CELLSPACING=0 CELLPADDING=10><TR><TD WIDTH="100%" ALIGN="CENTER" BGCOLOR="#0000C0"><H1><FONT COLOR="#FFFFFF">ALL3076</FONT></H1></TD></TR></TABLE><H4><BR>&nbsp;<BR></H4><CENTER><TABLE BORDER=1 WIDTH="80%" CELLPADDING=5 CELLSPACING=5><TR><TD BGCOLOR="#FFFFFF"><CENTER><TABLE BORDER=1 WIDTH=110 HEIGHT=110><TR>
<TD WIDTH=110 HEIGHT=110 BGCOLOR="#404040" ALIGN=CENTER><H1><A HREF="/r?b=1&amp;r=0&amp;s=1">OFF</A></H1>Output</TD>
</TR></TABLE></CENTER><H4>&nbsp;Dimmstufe = 128, <A HREF=r?d=0>dunkler</A> <A HREF=r?d=1>heller</A><BR></H4></TD></TR></TABLE><BR><BR><HR><H4><A HREF="r">Relay</A>&nbsp;&nbsp;<A HREF="config.html">Configuration</A>&nbsp;&nbsp;</H4><HR><BR></CENTER></BODY></HTML>

It can be parsed with a regex (see multiple-choice question).

Similar RESTful APIs are available for many devices today. The popularity of smartphones is an economic factor that motivates vendors to provide REST HTTP gateways to their proprietary devices to allow remote control via the vendor's smartphone apps...

In the lab we will see how we can make use of this.

Web Services

Web services (WS) are applications that are accessible to other web services over the web (HTTP). Web services are a convenient way to realize a distributed Service Oriented Architecture (SOA).

Web services use the Simple Object Access Protocol (SOAP) for communication over HTTP, SMTP, and other transports. The messages exchanged between different services are usually encoded in XML.

To allow the remote use and the reuse of services their interfaces have to be standardised. This happens using the Web Service Description Language (WSDL). An important part of the WSDL description of a service are the signatures of the programming interface of the service (API).

To share the information about an interface the so-called Universal Description Discovery and Integration (UDDI). Besides other information the UDDI contains the WSDL descriptions of services.

Even though the web service architecture has promising features it is not widely used in devices today.

Simple Network Management Protocol (SNMP)

The Simple Network Management Protocol (SNMP) is a protocol that comes from the network management community. It is used to remotely manage and control network entities like routers in the Internet. Even though SNMP was meant as a provisory solution at the beginning it was improved and is still widely use today. A reason for that is its simplicity.

For communication over SNMP devices run so-called agents. Managing entities (manager) connect to the agents to obtain and change information on the devices.

Information is provided as key-value pairs under hierarchically structured addresses. The structure of the information is defined in the so-called Management Information Base (MIB). The syntax used for specifying the MIB of a device is called Structure of Management Information (SMI). SMI is a subset of the Abstract Syntax Notation 1 (ASN.1).

A MIB entry looks as follows:

NET-SNMP-EXAMPLES-MIB DEFINITIONS ::= BEGIN
...
--
-- Example scalars
--

netSnmpExampleInteger OBJECT-TYPE
    SYNTAX      Integer32
    MAX-ACCESS  read-write
    STATUS      current
    DESCRIPTION
        "This is a simple object which merely houses a writable
         integer.  It's only purposes is to hold the value of a single
         integer.  Writing to it will simply change the value for
         subsequent GET/GETNEXT/GETBULK retrievals.

         This example object is implemented in the
         agent/mibgroup/examples/scalar_int.c file."
    DEFVAL { 42 }
    ::= { netSnmpExampleScalars 1 }
...

SNMP agents usually listen on the well-known ports:

• 161/udp SNMP
• 162/udp SNMPTRAP

A get-request to port 161 of a device is used to obtain information at a certain address in the device's MIB tree. Such a request could be snmpget -c public localhost system.sysDescr.0. With the getnext-command it is possible to walk through MIBs as is simply returns the values of the next node in the MIB tree.

Additional to the synchronous get-request SNMP provides an asynchronous publish-subscribe mechanism. It is called SNMP trap. SNMP agents can subscribe to a trap (e.g. an alarm). When the trap event occurs the SNMP agent sends a notification to port 162 of the subscribing agent (e.g. snmptrap -c community1 -h 2000:1:1:1:209:6bff:feae:6d67 -m hello world).

In the automation domain which we are looking at in this lab reprogramming of components is seldom. Usually the devices are programmed once (or the functionality is even hard wired). Installing software updates on a shutter or a climate control is no regularly done operation for instance.

Especially in the Heating, Ventilation, Air Conditioning, and Lighting domains it is common that each system is closed and contains a separate control logic like an A/C control panel in the room the A/C is mounted. The peripherals (e.g. temperature sensors) are connected to the control element and no information leaves the domain.

In order to realise flexible control scenarios like the one described in the Weiser article it is desired to have access to the values of the sensors and actuators in a space. In this lab we will see how this is possible.

Orchestrating spaces with software is desired. Stored-program architectures allow flexible usage as the programs can easily be changed and thereby the purpose of a device can be changed. To realize smart space scenarios it is necessary to have software-based orchestration. Only this way the intended flexible use can be realised.

To realise software-based orchestration the common approach is to write drivers for the specific devices used in the setup. The drivers are run on the machine that contains the program that realises the orchestration logic / work flows. Often a monolithic approach (one control and management PC) is used.

The described approach works well for research projects with relatively fixed setups and little changes. For real world setups it does not scale. As we have seen in the previous section there are lots of different devices with lots of different protocols. Each combination needs a driver.

It is very likely that real world smart spaces will differ in the available functionality they offer, the devices that are used for each functionality, and thereby the communication protocols.

Therefore we developed a framework that enables and facilitates distributed development of software services like drivers, or orchestration logic for smart spaces.

Pervasive computing contains Distributed Computing. A distributed computing system consists of distributed computing nodes that are connected over a network. The distributed nodes can be heterogeneous in their hardware and software [8].

Covering the heterogeneity and the distribution are tasks done by Middleware.

A Middleware Architecture [8]

Read paper [8].

Software service need input to orchestrate a smart space. This input is called context.

Context can contain different information such as spatial context, temporal context, device context, network and communication context, environmental context, individuality and user profile context, activity context, mental context, or interaction context [9].

A typical context processing cycle is [9]:

• collect
• analyze & aggregate
• decide
• act & adapt

We will create such a control cycle in the lab by ourself.

Have a look at the sections II and III of [9].

8.1. Middleware for Smart Space Orchestration

As described before, software orchestration allows a flexible use of installed hardware of a smart space. There are lots of pervasive computing [9] scenarios for smart spaces from domains such as health, sustainability, security, or comfort. To realize the scenarios, software orchestration is necessary [11,12,14].

Middleware aims to provide a unified interface to a distributed system. It typically provides management and communication functionality that makes the challenging characteristics of a distributed system transparent to software developer. This facilitates the development for distributed systems significantly.

Besides bridging the heterogeneity between computation nodes that execute programs, a middleware for smart spaces must bridge device heterogeneities. A way to do so is providing unified interfaces to heterogeneous devices.

The bridging of the computing environment and device heterogeneities allows to create portable service for smart spaces that can run in different smart spaces with the different hardware installed there. For realizing portability, it is necessary to provide a representation for the abstract interface to a certain device class.

In this section, three different design approaches for middleware for smart spaces are presented:

• The first approach is a centralized middleware that is based on function calls. It targets homes with devices such as gaming consoles or cameras.
• The second approach is a distributed system. It is based on function calls and context representing tuples that can be queried. The system targets commercial buildings with professional Building Automation Systems.
• The third solution is the one we will have a closer look at in this lab. It is fully distributed and solely based on the exchange of context tuples. It targets both areas, the home and commercial or public spaces.

Microsoft Research develops a solution that is called HomeOS.

Their proposal is to have a central computer inside a home that controls and manages all devices. Peripherals are connected to this computer and presented to the locally running orchestration services via locally running drivers.

Drivers provide abstract interfaces to device classes. To realize the adaption, two layer concept with a lower level and a higher level component is presented. One layer is closer to the hardware's protocol (DCL) and one provides the abstract interface towards the software (DFL). See Fig. 3.

Fig. 3: The HomeOS layers [source]

The HomeOS middleware uses so-called roles as abstraction. A role is a collection of method signatures. Roles are intended for device drivers. To implement a certain role, a device driver has to offer the methods specified in its role (see Device Functionality Layer in Fig. 3). A role defines, which functions can be called on the instance that offers the role (e.g. a lamp).

Get an overview of HomeOS with the papers [10,11].

At the University of California, Berkeley, the Building Application Stack (BAS) is developed.

The main goal of the solution is to make orchestration services portable. The application domain the solution is targeted for are Building Automation Systems like the ones we had a look at before.

Get an overview on the so-called BAS middleware looking at [12]. Fig. 4 shows the main components of the middleware. The device driver concept is again divided into an abstract part on top and a device-specific part on the bottom.

Fig. 4: Layers of the BAS system [source]

Like in HomeOS the system abstraction is method based. So-called classes bundle the methods that a driver for a certain device type must offer.

Look at the basic services for BAS in [13].

Since 2008, I develop the Distributed Smart Space Orchestration System (DS2OS) together with many students. DS2OS is a middleware framework that consists of two parts:

• The Smart Space Store (S2Store) comprises functionality to deploy and manage services on the distributed heterogeneous nodes of a future smart space.
• The Virtual State Layer (VSL) middleware provides a distributed context store with transparent access and security mechanisms for services.

In this lab we will focus on the VSL.

DS2OS follows a different approach than the previous middleware designs: the Application Programming Interface (API) to all services is fixed. I provides functionality to manipulate context. All service-specific functionality is stored in the context Model of a service.

DS2OS does not differentiate between driver or smart device adaptation functionality and other functionality such as orchestration. All services are equal and all functionality is provided via services in DS2OS.
This facilitates the service design, as developers only have to learn programming the system once and can create drivers and orchestration workflows then in a similar way. It also allows to extend the VSL middleware core via regular services similar to the extension of µ-kernels in modular operating systems. Because of this property, the VSL is called a µ-middleware.

The fixed API and the local context that contains the instantiations from the service models from the global repository [Pahl2013].

The following poster shows the components of DS2OS. The level of abstraction raises from bottom to top. Smart devices may represent a state as a simple bit mask on the lowest layer.

• Smart Gateway Services do the bidirectional adaptation between the smart device and its context Model in the VSL.

The VSL provides context storage and makes the retrieval from and access to so-called Knowledge (context instances) on the distributed nodes of a Smart Space transparent to services. On top of the VSL different services such as:

• Advanced Reasoning Services that consume context and derive new knowledge from it (e.g. "5000 Lux outside" leads to the inference "it is day")
• Orchestration Services that realize pervasive computing workflows.
• User Interface Services that offer user interfaces on behalf of the distributed devices that often do not have user interfaces, e.g. as they are embedded systems.

For exchanging context Models and services, DS2OS has a central Smart Space Store that contains a repository for the context Models (Model Repository) and for services and their meta data.

Have a look at the poster by clicking on it.

Poster explaining the layers of DS2OS [source]

Following the main entities of the poster are explained more detailed.

Get an overview of the VSL concepts in [14].

VSL context Models consist of hierarchically structured tuples. A VSL tuple consists of a key-value pair and metadata such as the access rights, the version number or the time, the value was set.

Shared Abstraction (Model Repository)

Models are the abstract interfaces of DS2OS services. A Model specifies the interaction nodes of a service.

To realise a service oriented architecture (SOA) and to allow distributed development of services it is necessary to share the abstract interface definitions of the different services that run in a space. As devices are connected via services this includes the interfaces to the devices.

The shared abstraction of DS2OS is called Model Repository.

The Model Repository contains typed (named) trees, called Model, that represent the interfaces of the services.

Data Types / Multi-Inheritance

The data type of a DS2OS node indicates, which functionality is available in the subtree.

A model for a light is defined as follows:

lamp.xml:
<lamp>
  <isOn type="/lighting/lightState" reader="*" writer="*"></isOn>
</lamp>

lighting/lightState.xml:
<lightState type="/derived/boolean"></lightState>

derived/boolean.xml:
<boolean type="/basic/number" lowerBound="0" upperBound="1">0</boolean>

basic/number.xml:
<number lowerBound="" upperBound=""></number>

The reader and writer attributes in the lamp Model specify who may access the node. By default only the creator of the instance (your Gateway Service) has full access to the nodes. Defining additional access rights for * means that any ID can access the node.

Note that the boolean type is initialized with 0 in the listing above. The rule is that later includes overwrite earlier ones. So if the lamp Model would set a default value 1 for isOn, an instance of it would have the default value 1 set.

Types are used as primary context of the VSL, meaning that instances are identified by their type. When a node is of the shown type lamp, a service accessing the node knows automatically that the node will have a subnode isOn of type /lighting/lightState.

The sharing of Models over the global Model Repository allows service developers to look up which Models they want to use, and how they can interface them. Using types as primary context realizes a locator-id-split, meaning that searching for the identifier which is the type returns a pointer to the locator which is the instance in the concrete smart space. The access to the lamp functionality over the lamp interface makes the used context independent of the service implementation that realizes the adaptation. This way Smart Gateway Services from different developers and for different physical lamps can be used transparently as they all offer the same interface. This realizes portability in DS2OS.

Types have two purposes in DS2OS:

1. Tagging a node with a data type such as /basic/number.
2. Adding a semantic type to a node such as adding /lighting/lightState to the derived type /derived/boolean.

This multi-use make multi inheritance necessary. Multi inheritance additionally helps converging functional interfaces as different sub types of a thing can inherit from a base type. A dimmableLamp inherits the type lamp for instance and extends it with additional fields. Tis way, services that are programmed to access lamps can automatically access dimmable lamps as well. feature is also useful when a Model as should represent aggregated

Virtual State Layer

The Virtual State Layer (VSL) is the middleware of DS2OS.

It is a distributed tuple space combined with a publish-subscribe system that notifies subscribing services on changes of a tuple.

The VSL is provided by so-called Knowledge Agents (KA). They offer the DS2OS programming interface (API) to service developers.

The VSL offers the functionality storage, communication, security, and access control to all services using it.

Asynchronous access: get/ set/ subscribe

Many tasks when orchestrating a smart space are asynchronous: a producer produces information, e.g. a sensor value is read by a Smart Gateway Service from a smart device, and stored in the corresponding node in the service's model in the VSL.

Another service can now access the knowledge using the VSL get and set methods to the address of the node. The address can be identified using the search command and the type of the node.

Additionally, a service can subscribe to a node. To do so it registers a callback on an address. The soon a node in the subtree changes, the callback function is invoked and the address of the node is passed so that it can be queried using the get method. This two-step value update mechanism is chosen as service may only sparsely be interested in the actual values and the values may be big, causing unnecessary traffic.

The callback is registered using the connector c as follows:

c.subscribe(myKnowledgeRoot + "/tickle", new ISubscriber() {
    @Override
    public void notificationCallback(String address) {
      // read the node's data
      if (c.get(myKnowledgeRoot + "/tickle/value").equals("1")) {
          // Someone is tickling us
          h.tickleHandler();
      }
    }
});

As the example shows, adding /value can be used to access the plain value. If it is not added, the XML representation of the node with enclosing tags is returned.

All data in the VSL is passed as String.

Synchronous Access: Virtual Nodes

Time critical service functionality, or functionality that provides external items on demand require synchronous interaction. Therefore, the VSL offers so-called Virtual Nodes.

The Model of a Virtual Node is as follows:

<virtualNode>
    <serviceId type="/system/serviceID"></serviceId>
</virtualNode>

The type /system/serviceID gets automatically added to a node when it is declared as virtual by registering the Virtual Node callbacks as described below. So you do not have to declare a node as virtual in the model of your services by adding the /system/serviceID type.

Virtual Nodes are not served from the VSL context store but directly passed to the service they belong to. Though the handling mechanism is different, the interface remains the same: Virtual Nodes are accessed via get and set.

For making a node in its context Model a Virtual Node, a program registers a callback that is called synchronously (blocking call) on access.

The Virtual Node callbacks can be registered and implemented as follows:

c.registerVirtualNode("/address/where/the/virtualNode/is",
        new IVirtualNodeHandler() {
                @Override
                public void set(String address, String value, String writerID) {
                        // will be called when a set-request for the virtual node is received
                }

                @Override
                public String get(String address, String readerID) {
                        // will be called when a get-request for the virtual node is received
                        return null;
                }
        }
);

The callbacks and the addresses they are registered for are stored in the Knowledge Agent. The callback is called as described when a get- or set-access to a Virtual Node happens.

Virtual Subtrees

The VSL handles Virtual Nodes like virtual subtrees. If additional information is added in the path after the Virtual Node, the full address is passed to the subtree. As an example one could call get "/greet/Luke Skywalker" to access the Virtual Node /greet. In the following example, in the get handler implementation everything behind the node name is extracted and passed as argument to the handler:

        // The registration of the /greet node.
        c.registerVirtualNode(myKnowledgeRoot + "/greet", new IVirtualNodeHandler() {
            // Greet the posted name to the std::out. Ignore added suffixes.
            @Override
            public void set(String address, String value, String writerID) {
                h.greetConsole(value + "!");
            }

            // Greet the suffixed name by returning the greeting.
            @Override
            public String get(String address, String readerID) {
                String suffix = address.substring((myKnowledgeRoot + "/greet/").length());
                return h.greetBack(suffix);
            }
        });

The goal of the DS2OS architecture is to facilitate the creation of smart space services. A major methodology applied is creating a Service Oriented Architecture (SOA). Complex functionality is modularized into small and simple parts that get connected over their standardized VSL interfaces.

A DS2OS service package consists of three elements:

• The VSL context Model that describes the context nodes, a service offers as its abstract interface.
• The service executable that contains the service implementation.
• A manifest that contains access rights and dependencies of the service.

In this lab we will have a look at the former two.

Services

Services are all software components that can be plugged to the VSL. Each entity in DS2OS is a service and has a model. This applies for smart device hardware too as it is connected to the VSL via Smart Gateway Services.

DS2OS services can be categorised into

• driver services, that connect hardware to the VSL abstraction,
• orchestration services, containing orchestration logic,
• advanced reasoning services, containing functionality to extend information in the VSL,
• user interface services that interact with users, and
• other services.

The special property of driver services is that they have two interfaces: a proprietary one to the hardware and the regular interface to the middleware. The unification of the middleware interface of device driver services facilitates their programming.

Smart spaces need many services to support as many devices (smart gateway) and pervasive computing work flows (orchestration).

Smart Space Store (S2Store)

Similar to the App Store for smartphones, the Smart Space Store (S2Store) offers services for smart spaces. It provides a global repository for services, their abstract models, and their manifests.

The S2Store is not part of this lab.

Smart Gateway Services (SGW) connect smart devices with their virtual representation in the VSL. This adaptation is done in both directions, changes on the device are reflected to the Model instance and changes in the context of the device in the VSL are reflected to the device.

As we have seen in the architecture, services typically communicate in an asynchronous way. An orchestration service sets the context node /agentId/serviceId/isOn to 1, the SGW has subscribed to the node, gets a notification, queries the value and tries to reflect it to the remote controlled lamp. As there is no synchronous connection between the SGW and other services, the SGWs have to operate autonomously. This makes the overall design of a DS2OS site less complex again as it encapsulates all device related functionality in the SGW.

The lose coupling of services makes it necessary to have Autonomous Smart Gateways. For realizing autonomous behaviour it becomes necessary to distinguish between a desired and a running state as follows:

• For each node that results in a change on the connected device, the Smart Gateway creates a sub node /desired of the same type as its parent. E.g. /lamp/isOn/desired.
• The Smart Gateway subscribes all its desired nodes.
• A Service sets the desired state of a desired node in the abstraction in the VSL.
• The Smart Gateway receives a callback. The callback method handles the desired state change by trying to reflect the value of the desired node to the device.
• Afterwards the current state of the device is reflected onto the parent node.

For example:

1. A service sets /lamp/isOn/desired to TRUE. The value of the node /lamp/isOn is currently FALSE.
2. The Smart Gateway tries to switch the lamp on.
3. Afterwards it checks the current state of the physical lamp and stores the result into /lamp/isOn.
4. If it was successful /lamp/isOn and /lamp/isOn/desired are both TRUE now.

The Smart Gateway tries to keep the virtual representation of the lamp and its physical reality consistent.

Connectors

The VSL uses XML RPC for communication. To facilitate the development of services, native connectors to different programming languages are provided. The connectors encapsulate the communication with the VSL so that service developer so not have to care about it.

Java Connector

To facilitate the creation of Java services, a so-called Java-connector and a service template are provided. See figure 7 for the architecture of a DS2OS Java service.

Fig. 7: The service template classes, the connector, and a Knowledge Agent that offers the interface to the VSL.

The connector offers native Java methods to communicate with the VSL. The proposed design is to use one class (ServiceTemplate.java) for the "wiring", and one class (ServiceTemplateHandler.java) for implementing the functionality. Wiring means registering the callbacks from the VSL and sending the data to the handler.

Advantage of separating the two functionalities are that you have a smaller file that shows you all handlers, and that you can write Unit tests for your handler class without having to care about the VSL functionality.

The Java connector is invoked as shown in Fig. 8. On registration, the VSL creates a subtree with the service's ID and initializes the subtree with the given model that is loaded from the Model Repository. The figure shows how the full address of the service's subtree can be obtained.

Fig. 8: Invoking the Connector.

Application Programming Interface

For service programmers connectors to different programming languages are offered. We will use the Java connector.

You can find the API documentation of the connector to the KA here.

Following the wiring code of the tamplate is shown:

public class ServiceTemplate {

    /**
     * The local Id that is used for your service:
     */
    private static final String myServiceIdentifier = "demoService";

    /**
     * The model that is used for your service.
     */
    private static final String myServiceModelId = "/services/template/demoService";

    /**
     * The connector instance used by this service.
     */
    private Connector c;

    /**
     * The class where your functionality is implemented.
     */
    private ServiceTemplateHandler h;

    /**
     * Stores the sub tree root that is belonging to this service.
     */
    private String myKnowledgeRoot;

    /**
     * You can use the logger for logs.
     */
    private static final Logger LOGGER = LoggerFactory.getLogger(ServiceTemplate.class);

    /**
     * The constructor that creates a connector instance and tries to register to a local agent
     * instance. TODO: Give your service a speaking name.
     */
    public ServiceTemplate() {
        try {
            c = new Connector();
            c.registerService(myServiceIdentifier, myServiceModelId);
            myKnowledgeRoot = c.getKORSubtree();
            h = new ServiceTemplateHandler(c, myKnowledgeRoot);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    /**
     * To know which nodes exist have a look at the model specified with the Id in myServiceModelId.
     * TODO: Change to connect to your subscription handlers
     * @throws AgentErrorException
     * @throws AgentCommunicationException
     */
    public void registerSubscriptions() throws AgentCommunicationException, AgentErrorException {
      // TODO: add your subscription handlers here
    }

    /**
     * To know which nodes exist have a look at the model specified with the Id in myServiceModelId.
     * TODO: Change to connect to your subscription handlers
     * @throws AgentErrorException
     * @throws AgentCommunicationException
     */
    public void registerVirtualNodeHandlers() throws AgentCommunicationException,
            AgentErrorException {
      // TODO: add your Virtual Node handlers here
    }

    /**
     * This method disconnects your service from the local agent.
     */
    public void shutdown() {
        LOGGER.info("Shutting down...");
        c.shutdown();
    }

    /**
     * The main method executes your service.
     */
    public static void main(String[] args) {
        ServiceTemplate s = new ServiceTemplate();
        try {
            s.registerSubscriptions();
            s.registerVirtualNodeHandlers();
        } catch (AgentCommunicationException e1) {
            LOGGER.error("We caught a communication error: {}", e1.getLocalizedMessage());
            e1.printStackTrace();
        } catch (AgentErrorException e1) {
            LOGGER.error("We caught an agent error: {}", e1.getLocalizedMessage());
            e1.printStackTrace();
        }

        // We will just wait here until the user sends us a "q" [return]
        BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
        String quitString;
        try {
            do {
                System.out.println("Type q to end this service.");
            } while ((quitString = in.readLine()) != null && !quitString.equals("q"));
        } catch (IOException e) {
            e.printStackTrace();
        }
        s.shutdown();
    }
}

9.1. Do-It-Yourself Movement

From the early times of software people are creating and sharing software in a crowdsourced way. Since 1998 the open source movement exists as a way to channel the creation on open or free software. With the emergence of smartphones and the opening of the App Store to third party applications (Apps) in 2008, crowdsourced software development was pushed and commercialized more.

Similar to crowdsource software development, crowdsourced hardware development exists. In 2005 the Arduino project was started. Similar projects are the Raspberry Pi, the Beaglebone, and others. The cheap hardware with its development kits allows hobby developers to create their own Do-It-Yourself (DIY) hardware.

DIY hardware is very interesting for creating smart spaces. The platforms allow to make non-smart hardware smart by adding the embedded device that brings a network connection.

In this lab we will not only look at the software part of smart space orchestration but also at the hardware part. The service oriented design of DS2OS in combination with the abstract interfaces provided by the VSL models allow you to build and integrate your own DIY smart devices easily as you will see in the lab.

The focus of the lab is on DS2OS and not on the Arduino. We will connect some sensors and actuators to the Arduino in the lab and make them accessible from remote. As you may be interested in more background about the Arduino project, we provide you with further information here.

What is the Arduino platform and what can I do with it?

General Information

" Arduino is an open source hardware platform with a focus on easy prototyping. The first Arduino - the Arduino Uno - was developed in 2005 as an easy to use microcontroller board that is less expensive than the available alternatives at that time. Since then several other boards widened the Arduino family. Except for the Arduino Due, all Arduinos use an 8-bit Atmel AVR microcontroller, the Due uses a 32-bit micorcontroller.

Typical for microcontroller boards like the Arduino is a relatively low power consumption and low resources in regards of CPU power and memory. The Arduino also provides digital IO-pins, some of which are PWM capable, as well as analog input pins. Some pins can be used to fire interrupts. The Arduino also supports some common protocols, for example SPI, i2c and RS232.

A key feature of the Arduino boards are easy to access IO-pins, which on most boards have a similar layout. This also allows the functionality of the boards to be extended by add-on modules called 'shields'. Typical tasks for such shields are to provide connectivity (ethernet, wifi, zig-bee), motor control, and GSM. Given the open platform and fixed layout, a lot of third-party shields are available. Most software libraries for these shields are open source too.

Example applications

• Mechanical bass: Youtube (1), Youtube (2)
• M&M sorting machine: Youtube
• Lightning controller: Youtube
• Hexapod robot: Youtube
• HTML5 canvas plotter: Youtube
• Bird Buggy: Youtube
• Gardening system: www.instructables.com

" In the lab course we use the Arduino Mega 2560 with an ethernet shield. The Arduino Mega is clocked at 16 MHz, has 8 kB of SRAM, 258 kB flash memory for storing code and 4 kB EEPROM for persistent data storage. It is also pre-loaded with a bootloader to allow easy upload of code via the usb cable.

The official specifications for the Arduino Mega can be found at arduino.cc/​en/​Main/​Arduino​Board​Mega2560.

The analog pins are located on the left side, the digital ones on the right and at the bottom. Power, USB and Ethernet are connected from the top. The red button in the top left corner is a reset button. Pressing it will cause the Arduino to reset and startup the application again. Doing so does not harm the Arduino, neither does removing the power supply or ethernet cable. When connected via USB, an additional power supply is usually not needed.

The digital pins provide 5V (when configured as digital output in a HIGH state) and support up to 40 mA per pin.

The analog pins support a 10bit resolution between 0V (analog read value: 0) and +5V (analog read value 1023) input.

Close to the top left of digital pin 13 is an on-board LED which is connected to pin 13. We will use it a few times as an easy-to-use LED in our demo programs. This LED will also blink rapidly while uploading the compiled code.

What will be done in the lab?

In the lab we will create our own smart device hardware, and we will write a "firmware" that runs on the Arduino and makes its functionality accessible from remote. Your smart device may look as shown in the figure.

A smart device built by an iLab2 team in 2013.

We will give you some information about the hardware prototyping next. You will get all the requires equipment at the secretaries office. Please by cautious not to damage the hardware and bring it back to the secretaries office after use. Thank you!

Prototyping with a Breadboard

Prototyping with Arduinos is typically done with breadboards. A breadboard is solderless plugboard that allows to set up and change the wiring of the project by using jumper wires. That way projects can be tested without any soldering, enabling to try out multiple configurations in a short time.

The breadboard that we use in the lab.

Layout of the breadboard

An example of a prototype setup that mixes solid core wires and flexible jumper wires.

Collection on tutorials and projects

• How to make a Physical Gmail Notifier: Jamie Matthews
• Breadboard and LED tutorial: www.ladyada.net
• Analog read tutorial: arduino.cc
• Read commands from serial connection: arduino.cc
• TCP chat server: arduino.cc
• Rebuild an Arduino on a breadboard: arduino.cc

What do I find here?

On the following pages you find pictures and instruction on how to connect the different sensors and actuators to your arduino Smart Device. You can use and combine those components in the lab part later as you like.

The following pages are also meant as reference manual for you during the lab later. So come back when you do the lab.

Resistors

Resistors are used to restrict the current and/or voltage in a circuit, following the formula U = R * I. The resistors that we use in the lab have the typical color coding that indicates their resistance. The colors are printed on the resistor as rings. There can be three, four, five or six rings, the resistors in the lab have either four or five. With four rings, the first two rings define the digits, the third ring a multiplier, and the last ring the tolerance. If the resistor has five rings, than the first three rings are used for the digits, the other follow as for the four-ringed resistors. The values that are associated with the colors can be looked up in tables. Often one of the rings is a bit thicker or has a greater distance to the other rings. In that case this is the right-most ring, indicating the order at which the rings should be read.

Numerous smartphone apps and web-applications exist that help to calculate the resistance for a given color-coding. One example is found at www.hobby-hour.com/electronics/resistorcalculator.php. Feel free to use it if you're unsure about the resitance of a given resistor.

• 150Ω (brown, green, brown, gold)
• 3240Ω (orange, red, yellow, brown, brown)
• The resistor from the question below...

Light Emitting Diode

LEDs have a relatively high initial resistance until a certain voltage is met, typically around 2 to 3V. Above that voltage resistance drops rapidly, which is part of the reason why we add a resistor in series for every LED that we use in the lab. This characteristic also makes it hard to dim the LED via voltage changes. However LEDs can be turned on and off very quickly, allowing to use pulse width modulation (PWM) to control the brightness by turning it on (to full brightness) for short or longer time intervals.

Plugging the hardware on the breadboard

Required parts

• Breadboard
• LED
• 150 Ω Resistor (colors: brown, green, brown and gold)
• Jumper wires
• Arduino Board (used as power source) + USB cable

Circuit diagram

Connecting the device to the Arduino

To connect the Arduino and the breadboard, use the flexible jumper wires. To create circuits on the breadboard, we recommend to use the solid core wires.

Be careful to not create a short circuit. Always have a resistor between a power source and a current sink.

Temperature Sensor

As an analog input source we provide a temperature sensor for the lab exercises. In our case it is the KTY81 220, a very simple kind of temperature sensor - a temperature-dependent resistor. Since we cannot measure the resistance directly, we instead have to measure the voltage via an analog input.

Since we are only interested in a rather small temperature range (from 20 to 30°C), we use a "voltage divider" circuit to increase the difference in voltage in that range. The circuit diagram below shows the setup, including the voltage divider.

Plugging the hardware on the breadboard

Required parts

• Breadboard
• Temperature Sensor (KTY81 220)
• 3.42kΩ Resistor
• Jumper wires
• Arduino

Circuit diagram

Click to enlarge

Photo of the setup

Connecting the device to the Arduino

To connect the Arduino and the breadboard, use the flexible jumper wires. To create circuits on the breadboard, we recommend to use the solid core wires.

Be careful to not create a short circuit. Always have a resistor between a power source and a current sink.

Light Sensitive Sensor

The photo-sensitive sensor used in the lab exercise is a phototransistor, i.e. a transistor that is controlled by a build-in photodiode. For measuring the light intensity, we connect the analog input from the Arduino with a pull-up resistor to +5V power supply, and have the phototransistor to sink the current to the ground. That way, if no light is shining, the phototransistor does not allow any current to pass and we can meassure +5V on the input pin (assuming a perfect circuit). If there is a lot of light shining, the current flows through the phototransistor to the ground, and we measure close to 0V on the input pin. Since the amount of current that can flow thorough the transistor (i.e. its resistance) depends on the amount of light collected by the sensor, we can determine how much light is shining by measuring the voltage at the input pin.

Plugging the hardware on the breadboard

Required parts

• Breadboard
• Phototransistor (SFH 309)
• ~100kΩ Resistor
• Jumper wires
• Arduino

Circuit diagram

Click to enlarge

Photo of the setup

Click to enlarge

Connecting the device to the Arduino

To connect the Arduino and the breadboard, use the flexible jumper wires. To create circuits on the breadboard, we recommend to use the solid core wires.

Be careful to not create a short circuit. Always have a resistor between a power source and a current sink.

Button

In order to not create a short circuit when the button is closed, the circuit should always contain a resistor between the power source and the input pin. To create a defined state when the button is open, we need an resistor, as shown in the circuit diagram below. This resistor is called a pull-up resistor, if it sets the default state to HIGH (+5V) and pull-down, if it sets the default state to LOW (grounded). Pull-up and pull-down resistors are typically around 10kΩ or higher. The Arduino provides internal pull-up resistors for several of its pins (turned off by default), we use an external pull-up resistor in this example.

Required parts

• button
• resistor: ~10kΩ (we provide a 18.7kΩ resistor that will work)
• jumper wires
• Arduino Mega

Circuit diagram

Photo of the setup

Photo of the setup with an additional LED added.

Connecting the device to the Arduino

To connect the Arduino and the breadboard, use the flexible jumper wires. To create circuits on the breadboard, we recommend to use the solid core wires.

Be careful to not create a short circuit. Always have a resistor between a power source and a current sink.

After having connected the hardware sensors and actuators, we will write a "firmware" for our smart device that makes it remote controllable over network over a proprietary protocol we will develop.

To facilitate software development, the Arduino platform brings an Integrated Development Environment (IDE). If you want to play around with it already, you can download the IDE from the Arduino website (local copy of arduino-1.0.5-linux64.tgz).

Outline - Software

The Arduino IDE

You can find an overview of the basic language features and the provided libraries at arduino.cc/​en/​Reference/​HomePage. The only library that we use is the Ehternet library (and the SPI library as a dependence of the Ethernet library).

To open one of the demo applications that comes with the IDE, use the open button instead of the menu: file->open because these two open different dialogs.

The Arduino homepage also hosts a forum at forum.​arduino.​cc. There you can find additional information about projects done with arduino, known problems of hard- and software and so on.

10.1. What will we do in the Lab?

Besides looking at the DS2OS peer-to-peer framework for Smart Space Orchestration, and building your own DIY smart device, we will have a look at how software can be evaluated.

We will focus on the performance evaluation and more specifically on latencies and throughput measurements in this lab. Besides other thing, in the lab you should learn, how measurement setups can be done, how data can be collected, how measurements can be evaluated and plotted with gnuplot, and how important it is to have a sufficiently big sample size.

This prelab part will give you an quick reminder on some statistic background.

Minimum and Maximum values

Assume you obtained an array of N samples. Then the minimum and maximum values are defined as

When measuring delays, e.g. RTTs, you are usually interested in the minimum value. The reason is that buffering and processing of packets along a path adds a stochastic delay. The figure below shows RTTs in a scatter plot together with the lines marking the minimum and maximum values observed.

Mean value

The mean value of N values is given by

The mean value is used to make a statement about the average execution time. The figure below shows the mean RTT of the previous example.

Median

The median x_m of N values is given by the index m such that

In other words, x_m is chosen such that half of the values is larger and the other half smaller than x_m. Depending on whether N is odd or even, the index sets of the values smaller and larger are equal or differ by one.

The median value is particularly useful when there a few but severe outliers in your data set. In case of the preceding example, the median and mean values would be roughly the same. However, when we assume a single outlier of 2000ms (which is not unusual for RTT measurements), the mean value increases significantly while the median remains stable. The figure below shows this situation. The dashed line denotes the mean value while the solid line indicates the median.

Quantiles

You already know one special quantile: the 0.5-quantile which is nothing else than the median. A parameter α indicates how many values are smaller than the α-quantile x_m^α.

The figure below shows the 0.25-quantile, 0.5-quantile (median), and 0.75-quantile respectively.

In the practical part of this lab you will create such candlestick plots to compare the performance of DS2OS. We provide you with a GNUPlot script which creates these plots.

Candlestick plots are suitable to graphically represent parameters like minimum/maximum values, median, and quantiles at once. The figure below shows an example for a candlestick plot.

The plot shows data for ten different data sets. The minimum and maximum values are represented by the lower and upper horizontal bars. The 0.25-quantiles correspond to the lower ends of the vertical rectangles while the 0.75-quantiles correspond to the upper ends. The horizontal black bars within the rectangles denote the median values.

10.3. GNUplot

A GNUplot of a measurement result about the latency of DS2OS depending on the amount of clients sending 100 requests in parallel to a single service (this is a worst case scenario).

In this lab we will use GNUplot to draw our graphs. There are lots of tutorials in the web. Have a look at one of them such as the one you can find here

12. List of References

[1] M. Weiser, “Some computer science issues in ubiquitous computing,” Commun. ACM, vol. 36, no. 7, Jul. 1993.

[2] M. Weiser, “The Computer for the 21st Century,” Scientific American, Sep. 1991.

[3] G. D. Abowd, “What next, ubicomp?: celebrating an intellectual disappearing act,” presented at the UbiComp '12: Proceedings of the 2012 ACM Conference on Ubiquitous Computing, 2012.

[4] K. Lyytinen and Y. Yoo, “Issues and Challenges in Ubiquitous Computing ,” Commun. ACM, vol. 45, no. 12, Dec. 2002.

[5] I. T. U. ITU, “ICT Facts and Figures,” ICT Data and Statistics Division Telecommunication Development Bureau International Telecommunication Union Place des Nations 1211 Geneva 20 - Switzerland, Geneva, Feb. 2013.

[6] F. Liu, J. Tong, J. Mao, R. Bohn, J. Messina, L. Badger, and D. Leaf, “NIST Cloud Computing Reference Architecture,” NIST Special Publication, 2011.

[7] A. J. B. Brush, B. Lee, R. Mahajan, S. Agarwal, S. Saroiu, and C. Dixon, “Home Automation in the Wild: Challenges and Opportunities,” presented at the the 2011 annual conference, New York, New York, USA, 2011, p. 2115.

[8] P. A. Bernstein, “Middleware: a model for distributed system services,” Commun. ACM, vol. 39, no. 2, pp. 86–98, Feb. 1996.

[9] M. Knappmeyer, S. L. Kiani, E. S. Reetz, N. Baker, and R. Tonjes, “Survey of Context Provisioning Middleware,” Communications Surveys & Tutorials, IEEE, vol. 15, no. 3, pp. 1492–1519, 2013.

[10] C. Dixon, R. Mahajan, S. Agarwal, A. J. Brush, B. Lee, S. Saroiu, and V. Bahl, “The Home Needs an Operating System (and an App Store),” presented at the the Ninth ACM SIGCOMM HotNets Workshop, New York, New York, USA, 2010, pp. 1–6.

[11] C. Dixon, R. Mahajan, S. Agarwal, A. J. Brush, B. Lee, S. Saroiu, and P. Bahl, “An operating system for the home,” presented at the NSDI'12: Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation, 2012.

[12] A. Krioukov, G. Fierro, N. Kitaev, and D. Culler, “Building application stack (BAS),” presented at the the Fourth ACM Workshop, New York, New York, USA, 2012, p. 72.

[13] S. Dawson-Haggerty, A. Krioukov, J. Taneja, S. Karandikar, G. Fierro, N. Kitaev, and D. Culler, “BOSS: Building Operating System Services,” Proceedings of the 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2013.

[14] M.-O. Pahl and G. Carle, “The Missing Layer - Virtualizing Smart Spaces,” presented at the 10th IEEE International Workshop on Managing Ubiquitous Communications and Services 2013 (MUCS 2013), San Diego, USA, 2013, pp. 139–144.

You find this page at the end of every preLab and lab. It is shared between all team members and between preLab and lab. So all of you see the same files and information.

Sometimes you might want to save some text snippets or files to have them available when you restart with the lab or on another computer.

This section is exactly intended for this. You can temporarily store your configurations, commande, etc. here.

What did you (dis-)like most about this lab? Do you have suggestions on what could be improved? Did you find any errors? If you have any suggestions or comments about the prelab or lab please let us know! This question has no bearing on your prelab completion.

Outline of the DS2OS lab

In the practical part of this exercise we will first see the whole DS2OS system working by looking at a web-based interface service and manipulating some context. Then we will build our own Smart Device based on the Arduino platform. Then we will design and implement your own firmware for the device with a communication protocol enabling remote control of the device over network. Next we will design and implement a Gateway to connect our Smart Device with the DS2OS VSL middleware by providing an abstract interface following the conventions of DS2OS. Having the device connected to the abstraction we will experience how easy it is to implement our own orchestration logic in an Orchestration Service. Finally we will assess our solution with three performance measurements from different points of view.

Setup

Fig. 1: DS2OS lab setup [source]

Roles of the different computers

• PC6 will run the web-based User Interface to your Smart Space. Using the basic data types of DS2OS it can provide access to all context nodes that are available in your space.
• PC5 will provide the physical connectivity to your Smart Device. It will assign an IP to the Arduino via DHCP. It will route the Arduino network traffic to PC4 and vice-versa.
• PC4 will host the Advanced Reasoning Service that helps modularizing functionality.
• PC3 will act as Model Repository. So it will store your abstract interface descriptions (Models).
• PC2 will host the Orchestration Service that runs your orchestration logic.
• PC1 will act as DS2OS Gateway. The role of the gateway is to connect your Smart Device with the DS2OS middleware to make it accessible from other services.

Later on each machine but PC5 a Knowledge Agent will be started to create the distributed VSL middleware.

For connectivity between the Knowledge Agents on the different machines we will use the management network so you do not have to cable the ring shown in the setup above.

2.1. Playing around with the system

We start the lab by playing around with the system a little bit. Later on we will have a closer look at the mechanisms that are used to do the first steps we do now.

In this part we will:

• Set up the VSL peer-to-peer context network.
• Create context nodes in the VSL.
• Set up a web-based user interface to the VSL.
• Manipulate the context nodes using the interface.

As you know from the prelab, the Virtual State Layer (VSL) is the context-aware µ-middleware core of DS2OS. It makes the information exchange transparent for services.

We will set up a VSL with two nodes now. Use PC3 and PC6.

Preparing the Downloads

DS2OS is written in Java. We will download the jar archives to PC3 and PC6:

If you want to use wget for downloading make sure you are able to reach the Internet by setting up the default GW on the PCs. You can use the following command to do so:

ip r a default via $(dig +short ilab-server)

Make sure that the right GW is set by issuing route.

What does the command do and how does it work?

Deploying DS2OS

Now download the following archives to the two PCs:

• ds2os.jar
• models.tar.gz

Extract the models.tar.gz into a ~/ds2os/ directory. Move the ds2os.jar into ~/ds2os/.

Create a configuration file for the agent as follows and save it ~/ds2os/config.txt:

network.interface=eth-man
transport.stream.port=6666
transport.datagram.port=6666
transport.compress=false
local.agentid=[myAgentId] ; <-- put your desired AgentID here, e.g. agent4

Make sure to use different agent IDs.

Start the agents on both PCs by issuing

java -jar ds2os.jar --console

Now issue a netstat -tulpen.

Make sure that you do not mix up the DS2OS console and the local shell. OS commands cannot be executed in the DS2OS console...

This exercise part is symmetric. That means you can both play with your agent on your screen and you will see symmetric results. Nobody has to watch only...

Go to the agent console and issue ls agentRegister on both PCs. On both PCs you should see something like:

system@agent6: / % ls agentRegister

<agentRegister type="/system/clientRegister" version="0" timeStamp="2013-12-08 17:35:24.652" subscriber="">
.<agent3 type="/system/clientMetaData" version="0" timeStamp="2013-12-08 17:40:28.002" subscriber="">
.</agent3>
.<agent6 type="/system/clientMetaData" version="0" timeStamp="2013-12-08 17:35:24.666" subscriber="">
.</agent6>
</agentRegister>

Adding Context Nodes

Start wireshark listening to the interface you use on both PCs.

Now change into the context subtree of your agent by issuing

cd [tab] [return]

The agent console should prompt as follows on PC6 now (on PC3 the 6s are replaced by 3s):

system@agent6: /agent6 %

Now add some nodes of type /basic/text on both PCs by issuing:

system@agent6: /agent6 % add fooBar /basic/text

Replace fooBar with the desired names of your new nodes.

Test that the nodes are there by using get [address].

Now assign some values to some nodes, e.g.

system@agent6: /agent6 % set fooBar "Hello world!"

get the node again and look at the data.

Now get only the value:

system@agent6: /agent6 % get fooBar/value

Make sure you added multiple nodes on both PCs.

Synchronization

Now change into the root directory of your agent and list the knowledge tree of the other agent by issuing:

system@agent6: / % ls agent3

Get the value of a node from the other agent.

Now have a look at your wireshark capture. Use follow TCP stream to have a look at the exchanged data.

If you do not see TCP streams, restart one of the agents to enforce synchronization.

Alive Pings

Now follow the UDP stream.

What is the purpose of the hash value in the message? You can also find it in the agentRegister subtree of an agent in your local KOR:

system@agentId: /agentRegister % get agentId/korVersionId

<korVersionId type="/basic/text" version="3" timeStamp="2013-12-13 19:03:02.605" subscriber="">
<![CDATA[16]]>
</korVersionId>

To answer the next question properly it might be interesting to start a new agent and to look how it synchronizes with the existing knowledge... in the console, and in wireshark.

Compare the outputs of lsL [localAgentId]/* and lsL [remoteAgentId]/*.

Adding an "L" to the command tells the console to interact with the local context repository, the Knowledge Object Register (KOR), only. Requests without trailing L are sent to the normal processing queue, resulting in remotely executed queries.

Creating Composed Nodes with Geolocations

The interface we will set up next is based on a spatial representation of context nodes. Therefore add at least one composed node with a geolocation to on each PC as follows:

system@agent6: /agent6 % add myGeoText /basic/composed
system@agent6: /agent6 % add myGeoText/text /basic/text
system@agent6: /agent6 % add myGeoText/running /basic/composed
system@agent6: /agent6 % add myGeoText/running/geolocation /demo/geolocation

Listing the geolocation node

Now list your modified node:

system@agent3: /agent3 % ls myGeoText/*

<myGeoText type="/basic/composed" version="0" timeStamp="2013-12-09 14:51:14.79" subscriber="">
.<running type="/basic/composed" version="0" timeStamp="2013-12-09 14:51:14.81" subscriber="">
..<geolocation type="/demo/geolocation" version="0" timeStamp="2013-12-09 14:51:15.808" subscriber="">
...<alt type="/derived/decimal" version="0" timeStamp="2013-12-09 14:51:15.844" subscriber="">
....<post-decimal type="/basic/number" version="0" timeStamp="2013-12-09 14:51:15.85" subscriber="">
....</post-decimal>
....<pre-decimal type="/basic/number" version="0" timeStamp="2013-12-09 14:51:15.847" subscriber="">
....</pre-decimal>
...</alt>
...<lat type="/derived/decimal" version="0" timeStamp="2013-12-09 14:51:15.814" subscriber="">
....<post-decimal type="/basic/number" version="0" timeStamp="2013-12-09 14:51:15.823" subscriber="">
....</post-decimal>
....<pre-decimal type="/basic/number" version="0" timeStamp="2013-12-09 14:51:15.818" subscriber="">
....</pre-decimal>
...</lat>
...<lon type="/derived/decimal" version="0" timeStamp="2013-12-09 14:51:15.829" subscriber="">
....<post-decimal type="/basic/number" version="0" timeStamp="2013-12-09 14:51:15.841" subscriber="">
....</post-decimal>
....<pre-decimal type="/basic/number" version="0" timeStamp="2013-12-09 14:51:15.835" subscriber="">
....</pre-decimal>
...</lon>
..</geolocation>
.</running>
.<text type="/basic/text" version="0" timeStamp="2013-12-09 14:51:14.801" subscriber="">
.</text>
</myGeoText>

The result shows you how the VSL automatically loaded the inherited type's definitions, e.g. /demo/geolocation is defined as:

grml@pc6 ~/ds2os/models/basic % cat ../demo/geolocation.xml
<model>
      <lat type="/derived/decimal"></lat>
  <lon type="/derived/decimal"></lon>
  <alt type="/derived/decimal"></alt>
</model>

At the multiple types you can also see the multi-inheritance.

Type-based Search

Finally use the search command to look for nodes of type /demo/geolocation. Your result should include all nodes that you edited:

system@agent6: /agent6/fooBar % search /demo/geolocation
Result for type /demo/geolocation in address /
1       /agent3/myGeoText/running/geolocation
2       /agent6/myGeoText/running/geolocation

Now we have at least four nodes in the VSL, and two of them have a /running/geolocation node.

After this preparation we can set up the interface on PC6 now.

REST connector

The web interface wants to communicate with the VSL over a REST interface. Therefore we start a REST DS2OS servlet connector on PC6

For being able to connenct to the VSL, make sure the Knowledge Agent is still running on PC6.

service tomcat7 restart

Installation Required Linux Packets

The REST API is implemented as Java servlet. It will be run inside a Tomcat.

In this lab we need the web server apache, the Java servlet runtime engine tomcat, and the mod-jk that connects the apache with the tomcat.

If some of the required software is missing, you can install it by issuing:

apt-get install apache2 tomcat7 libapache2-mod-jk

Do not forget to add the default gateway first as described here.

setting mod-jk up

Now configure apache to forward requests to the /ds2os sub directory to the tomcat. As we will only host one web site we will use the default configuration for this lab.

Add the following line to the default webserver configuration of apache2 (e.g. 000-default) inside the VirtualHost container:

JkMount /ds2os* ajp13_worker

In the tomcat configuration (e.g. /etc/tomcat7/server.xml) enable AJP by uncommenting the line:

    <!-- Define an AJP 1.3 Connector on port 8009 -->
    <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />

Restart tomcat and reload the apache config by issuing:

service tomcat7 restart
service apache2 reload

Now use a browser that is able to access the localhost without the proxy apache (e.g. firefox) and access http://localhost/ds2os. You should not get an apache error page if tomcat and apache are connected correctly.

Installing the DS2OS Servlet Connector

Download the DS2OS servlet from here. Move it into the tomcat webapps directory (e.g. /var/lib/tomcat7/webapps).

Configure the servlet by editing its config (/var/lib/tomcat7/webapps/ds2os/WEB-INF/lib/config.txt) so that it reflects your setup, e.g.

transport.datagram.port=6666
transport.stream.port=6666
transport.compress=false

If the config is missing, access the servlet first, so that it gets unpacked.

You can also create a symlink to ~/ds2os/config.txt but do not forget to reset this link later again if it is not working anymore as we will delete and recreate the ds2os directory later which will destroy the symlink.

You can test the functionality of the servlet connector by calling its REST interface at http://localhost/ds2os. You should see the help page now.

Query the VSL context directory "/" using the servlet connector.

Installing the Web Site

Download the website from here.

Extract the archive into your web servers root directory as specified in the VirtualHost container you edited before.

Now you should be able to access the graphical user interface by calling the local web server, e.g. http://192.168.X.X/

You will most probably not see any nodes.

Adding Access Rights

You can see the reason when you change the log level in the console to debug by issuing

ll debug

and reloading the interface.

Do not forget to set the log level back to warn (ll warn), not to get irritated by the log messages.

You will get access rights errors such as

[D] [NodeTree       ] Node /clientRegister is not readable by /agent6/servlet-connector

To give the servlet connector access to our nodes we have to set the rights. To do so issue

addReaderR fooBar /agent6/servlet-connector
addWriterR fooBar /agent6/servlet-connector

Add the read and write rights at least to one node that you added a geolocation node to and one where you did not add it.

Now list the gelocation data of the node where you added it

/agent6/servlet-connector@agent6: /agent6 % ls fooBar/running/geolocation/*

<geolocation type="/demo/geolocation" version="0" timeStamp="2013-12-08 20:14:31.472" subscriber="">
.<alt type="/derived/decimal" version="0" timeStamp="2013-12-08 20:14:31.512" subscriber="">
..<post-decimal type="/basic/number" version="0" timeStamp="2013-12-08 20:14:31.521" subscriber="">
..</post-decimal>
..<pre-decimal type="/basic/number" version="0" timeStamp="2013-12-08 20:14:31.516" subscriber="">
..</pre-decimal>
.</alt>
.<lat type="/derived/decimal" version="0" timeStamp="2013-12-08 20:14:31.482" subscriber="">
..<post-decimal type="/basic/number" version="0" timeStamp="2013-12-08 20:14:31.493" subscriber="">
..</post-decimal>
..<pre-decimal type="/basic/number" version="0" timeStamp="2013-12-08 20:14:31.488" subscriber="">
..</pre-decimal>
.</lat>
.<lon type="/derived/decimal" version="0" timeStamp="2013-12-08 20:14:31.497" subscriber="">
..<post-decimal type="/basic/number" version="0" timeStamp="2013-12-08 20:14:31.507" subscriber="">
..</post-decimal>
..<pre-decimal type="/basic/number" version="0" timeStamp="2013-12-08 20:14:31.503" subscriber="">
..</pre-decimal>
.</lon>
</geolocation>

Next drag and drop the nodes on the interface.

Have a look at the geolocation node again.

Manipulating data over the UI

Change the text value of your node with the GUI.

Graphical Elements

To show you where the interface goes add another node and set the access rights so that the servlet connector can read and write the entire subtree.

User the type

add myVirtualLight /demo/virtuallight

Make your virtual light accessible to the servlet connector.

Now play with the new GUI element and observe the changes in

get myVlight/running/isOn/value

To see how the bidirectional reflection works, change the VSL context and see how the interface changes:

/agent6/servlet-connector@agent6: /agent6 % set myVlight/running/isOn TRUE
/agent6/servlet-connector@agent6: /agent6 % set myVlight/running/isOn FALSE

The interface is only updated in intervals, so wait some seconds until it updates

The web user interface with the virtual light and the geo-text node.

Stop all Services

Now you have some experience in using DS2OS. Next we will start some performance measurements.

• Stop the TomCat
• Stop the Apache
• Quit ("q") the agents
• Close Wireshark

Do you want to have a break?

The following parts are not connected to what you did so far. So if you want to stop now or have a break, now is a good time.

If you only want to have a break you could continue for some more minutes and do the setup for the performance measurement and start it before having the break.

The measurement will take some time (>45min) that we will use for something else. So if you set it up now, you will have to continue with the lab or to wait until it finishes before going home.

Setup

For the measurement we will start 250 services, one server, and one control service. As it would take time to start all services and to setup all PCs we will do it via shell scripts.

On PC1 set the default gateway so that you can reach the Internet:

ip r a default via $(dig +short ilab-server)

Make sure that the right GW is set by issuing route.

Now download the following deployment script deployDS2OS.tar.gz with wget to any PC you like and expand it.

Now call the following script with root rights:

./deployAll.sh

The script deploys DS2OS including some services. The partner script is called killAll.sh and as the name says, it kills all processes belonging to DS2OS.

Have a quick look at what both scripts are roughly doing.

Now issue with root rights:

screen -r

It shows you a list of screens you can attach to on each PC.

Now we connect to the VSL by issuing:

screen -r ds2os

Get familiar with the VSL console shell. Start by issuing "help" or simply pressing return to get the help displayed.

Now list the agentRegister that contains the neighboring Knowledge Agents:

  ls agentRegister

Services

Now we will have a look at the services and their directories.

List the subtree of PC4.

What could you do with the search command?

Measurement Control Service

For the measurement services with different purpose are running now:

• A measurementServer that serves the client requests and is used as measurement target.
• A measurementControl service (mcs) that controls the measurement by configuring the clients and triggering different scenarios.
• Multiple clientServices (mCl) that act as client in the measurement.
• A ModelRepository that serves the VSL context models (Model) of the running services.

Remember that the VSL provides location transparency.

Change into the knowledge tree of the measurement control service (mcs). In case you are interested you can find its source code here: MeasurementControlService.java.

When you look at the nodes of the control service, you will see that they are virtual. As accessing nodes is always transparent in the VSL you can read values from Virtual Nodes with the regular get. To set Virtual Nodes you can use the normal set.

You can set different values there:

• amIparticipating wit added "/clientId" returns a 1 if the clientId is participating in the current measurement.
• fullRunStartAmount is the amount of clients the measurement starts with.
• fullRunStepping is the stepping the current client amount is decreased each round.
• fullRunTrigger triggers a measurment.
• minimumLogsToCollectPerTest contains the amount of client logs that must at least be collected before the next test s triggered.
• remainingClientIds contains a list of IDs that did not return yet from the last measurement.
• singleRunTrigger starts a single test.
• testModes contains all available test modes.
• testMode contains the index of the mode to be used for the next test run.
• testRunsClient contains the runs each client should do in a test run.
• timeOfNextRun contains the unix timestamp when the next measurement will start.
• waitUntilNextStart contains the seconds to wait per client until until the measurement starts. The overall waiting time is calculated as #participatingClients * waitUntilNextStart

Before triggering the measurement do the following setup:

• Set the start amount to 126.
• Set the stepping to 25.
• Set the minimumLogsToCollectPerTest to 50

Now trigger the measurement by issueing:

set fullRunTrigger 1

Now detach from the screen by pressing [Ctrl a] [d] (First [Ctrl+a] and then [d]).

To see what the measurement control is doing attach to its screen with screen -r measurmentControl on PC1.

In a screen session you can enter the scroll mode with [Ctrl+a] [. You can exit the scroll mode by pressing [ESC].

Measurement Monitoring

During the measurement you can monitor the actions in the screen of the measurement control service (measureControl). If you are interested in the CPU usage, you can monitor it with htop in parallel as shown in the figure.

The console output of the measurement console and htop during an active measurement session with 2600 parallel requests towards the measurement server service, running on the machine.

The measurement will take some time that we will use for something else. Make sure that the measurement service is still active and doing work from time to time (e.g. by keeping its console open). We will come back to the collected logs later. The last run will be with 1 client and it will be repeated 50 times according to our setup.

If you find out that one agent is apparently not reachable in the ping part of the control service, you can simply restart it by copying the deployMe.sh script to it and executing it. It will shut down everything on the local machine and restart it. (Do not do this on PC1 as it will delete all collected logs.)

The result to this measurement will be collected later. If you want to leave before reaching the part "Collecting the Logs of the First Performance Measurement", consider Collecting the Logs of the First Performance Measurement anyways before leaving if this measurement finished already.

What we will do in this part...

In this part we will build and test our own Smart Device. As base we use the Arduino platform. We will first add one item of each component we have available. Then you can add more sensors and actuators if you want.

To make sure you connect the sensors and actuators correctly we provided you with their setups in the PreLab already.

4.2.1. Quick description of arduino board

Fig. 2: Arduino board with the ethernet shield on top.

This page gives you a quick start to prototyping with the Arduino platform.

On the right you see a photo of the Arduino that we use in the lab. The analog pins are located on the left side, the digital ones on the right and at the bottom. Power, USB and Ethernet are connected from the top. The red button in the top left corner is a reset button. Pressing it will cause the Arduino to reset and startup the application again. Doing so does not harm the Arduino, neither does removing the power supply or ethernet cable. When connected via USB, an additional power supply is usually not needed.

The digital pins provide 5V (when configured as digital output in a HIGH state) and support up to 40 mA per pin.

The analog pins support a 10bit resolution between 0V and +5V input.

Close to the top left of digital pin 13 is an on-board LED which is connected to pin 13. We will use it a few times as an easy-to-use LED in our demo programs. This LED will also blink rapidly while uploading the compiled code.

When prototyping, the pins on the Arduino are typically connected to a solderless breadboard via flexible jumper wires. This allows quick changes to the setup.

We will build the setup iteratively. So keep all components you connect next on the board. We will use them later again.

Setting up the LED hardware

The first task is to set up a simple circuit from the power line of the arduino to power a single LED. This will help you to get familiar with the electric components and the Integrated Development Environment (IDE).

The following circuit scheme (Fig. 3) shows the starting layout.

• Start with connecting the beadboard power lines (red and blue) with the arduino +5V and ground
• Remember the layout of the breadboard!
• Add the resistor
• Add the LED and the jumper wire as shown. Keep in mind, that the longer pin of the LED is facing the positive charge. The head also has a flattened side, which has to face the negative charge,
• Check your setup
• When everything is fine, power up the Arduino by connecting it to your PC via the usb cable

Now the LED should light up.

Fig. 3: LED setup.

Please use the circuit diagrams (above) for the setup. The picture (below) is only for illustration on how it may look like.

Fig. 4: Photo LED demo.

The Integrated Development Environment

The Arduino platform brings a comfortable Integrated Development Environment (IDE). We will use it in this lab to program the device.

You read about it in the PreLab already.

• Download the IDE from the Arduino website (local copy of arduino-1.0.5-linux64.tgz).
• Extract it to your home directory.
• Start it.

LED blink demo

The next step is to control the LED with the Arduino. The IDE comes with multiple demo applications that can be used.

Fig. 5: LED setup.

1. Configure the IDE for the right Arduino model (Arduino Mega 2560)
   • Tools
   • Board
2. Load the "Blink" demo from the IDE (open --> Basics -> Blink)
3. Review the demo sketch. Which physical pin is used in the program?
4. Make sure to select the right Serial Port ( Tools > Serial Port > /dev/ttyACM0 )
5. Click on the upload button to compile and upload the sketch (it may be necessary to set the right port).

One of the on-board LEDs should blink now. This LED is also connected to the digital pin #13. Take a quick look at the program: it is quite easy to turn digital pins on and off.

Change the breadboard wiring from the first tutorial to have the LED circuit be powered from the digital pin #13, instead of the +5V output.

The digital output pin will also provide +5V when set to "HIGH". When done, both LEDs should blink simultaneously.

Circuit Diagramm

Fig. 6: Blink Demo.

We will build the setup iteratively. So keep all components you connect next on the board. We will use them later again.

Temperature Resistor Hardware and Driver

Fig. 7: Voltage divider.

For this we build a small circuit on the breadboard: a "voltage divider", consisting of

• a fixed resistor (R1), and
• a temperature-sensitive resistor (R2)

as shown on the circuit diagram.

A voltage divider takes an input Voltage V_in and separates it to two (or more) output voltages. In our example one of the outputs is connected to the ground. The output voltages depend on the input voltage V_in and on the resistor values. For more information on voltage dividers, see here.

In our case the input Voltage V_in is +5V, R1 is 3240Ω and R2 depends on the current temperature.

We connect V_out with the Arduino analog input A0 and measure V_out.

The higher the temperature is, the higher the resistance at R2 is and as a result the voltage at V_out is also higher. So we can map the measured voltage at V_out with the current temperature.

The temperature sensor in use here is the KTY81 220.

It has a typical resistance of 980Ω at -55°C, 2000Ω at 25°C and 4280Ω at 150°C.

The Voltage divider will make our measurements more sensetive in the 20°C to 30°C range. In this spreadsheet you can see the (expected) voltages (marked [V]) at V_out, and the (expected) value from an analogRead call (marked [A_in]) for different resistors, including the 3240Ω resistor for this exercise.

Adding the Temperature Sensor to the Breadboard

Extend the circuit on you breadboard to match the circuit diagram shown below. Make sure that you use the right resistor.

Fig. 8: LED setup.

Fig. 9: LED setup.

Load the Smoothing Demo

Now load the "Smoothing" Demo (Open -> Analog -> Smoothing). Compile and upload it onto your Arduino. Open the serial console in the IDE. You should see the current value of the analog sensor, a value very roughly around 400. Touch the sensor with your finger and notice how the measured value raises.

We will build the setup iteratively. So keep all components you connect next on the board. We will use them later again.

Reacting to buttons

In this part the task is to set up a button on the breadboard and having the Arduino to react when the button gets pushed.

Plugging the hardware on the breadboard

Required parts

• Breadboard
• LED
• 150 Ω Resistor for the LED (colors: brown, green, brown and gold)
• mechanical switch (blue push button)
• ~10kΩ Resistor (pull-up resistor)
• Jumper wires
• Arduino Board

Circuit diagram

Fig. 10: Switch Demo.

Photo of the setup

Fig. 11: Switch Demo.

Despite the picture the LED should be connected to 13.

Load demo code

Now you should have a basic understanding on how the Arduino platform is programmed using the IDE.

We will build the setup iteratively. So keep all components you connect next on the board. We will use them later again.

Light Sensor (Photo Diode)

Click to enlarge

The light sensor works similar to the temperature sensor: the resistance of the sensor depends on the physical property that we want to measure. Consequently, the circuit is very similar to the one use in conjunction with the temperature sensor. Just the resistance values are different.

Required parts

• Arduino
• Breadboard
• Phototransistor (SFH 309)
• Jumper Wires
• 113kΩ resistor (if this one is not available use a 100kΩ resistor instead)

The circuit can be tested with the "Smoothing" demo application (open -> analog -> smoothing) from the Arduino IDE. Make sure that the application and the circuit are using the same pin configuration. Just as it was with the temperature resistor you should see how the value changes, depending on how much light the sensor measures.

Circuit diagram

Click to enlarge

Photo of the setup

Click to enlarge

We will build the setup iteratively. So keep all components you connect next on the board. We will use them later again.

From the previous section your Smart Device contains:

• 1x LED
• 1x Temperature Sensor
• 1x Button
• 1x Light Sensor

Adding a Traffic Light

The provided demo program uses - additional to the current setup - three more LEDs. Use a red, a yellow, and a green LED to realize a traffic light. Connect each LED to a separate PIN of the Arduino. To keep the setup consistent with the symbolic names in the code, we recommend using: pin 5 for the red LED, pin 6 for the yellow LED and pin 7 for the green LED.

Fig. 12: Traffic Light

Circuit Diagramm

Fig. 13: Full demo setup

We will orchestrate the traffic light with software later. Optionally you can make it running directly on the Arduino on the next page...

Optionally add more components

In the remainder of the lab we will create a high level orchestration workflow that is based on our DIY Arduino-based smart device.

One workflow is fixed. It is setting the traffic light indicators based on the current system load. The requirement from the other workflow on you is only that it includes all remaining sensors and actuators on the board.

Imagine, the Arduino sensors are distributed in your room and the LEDs are real lamps or devices like a heater, or a fan.

Do you have ideas for cool orchestration workflows? We will write them down soon but let us first complete the hardware "manufacturing". Do you need more peripherals than we connected already?

Maybe you have some scenarios in mind that make use of more buttons, more LEDs, more temperature sensors, the light sensor, ... if so add some more items...

As you know now how to connect hardware to the Arduino board you are free to customize your Smart Device with additional peripherals.

Add what you want in the quantity you like.

You will be able to realize more cool functionality when you add some more things here. The minimum necessary components are on the board now.

When you added your additional (optional) components the hardware part of this lab is done...

Do you want to have a break?

For the following parts you will need your hardware setup so you should only break here or you have to setup the hardware next time again.

If you leave, do not forget to collect the logs for the performance measurement.

5.1. Our Own "Firmware"

So far we used the demo programs for testing the functionality of the Arduino. Maybe you looked into the code a bit already. In this part we will have a closer look at a more complex code. We will write our own firmware that allows us to remotely control our smart device over the Ethernet port of the Arduino.

Our goal in this part is to have our smart device that is accessible over the Ethernet port using our own protocol. Therefore we will continue with our own code in the IDE.

Download the following template in the IDE:

• tum_ilab_smart_device.ino

In the IDE, set the Baud rate of the serial monitor according to the rate set in the template, if they do not match already.

Deploy the our template to the Arduino.

If the Arduino seems crashed when uploading the you should reboot it. It typically blinks on upload.

The Arduino will auto configure its address via DHCP. Let's quickly set up a dhcp on PC5 on eth2 for that purpose.

Edit the /etc/dhcp/dhcpd.conf as follows:

• Add the smallest subnet with the address 10.1.2.3 in it.
• Use the highest IP in the subnet as gateway (running the dhcp server).
• Specify that dhcpd should use all available addresses.

If you encounter problems with the DHCP check if there are firewall (iptables) rules that prevent to happen what you want to do...

If you use more than one Arduino use the following 10.1.2... subnet for the next Arduinos.

Starting the dhcp server

The framework for the arduino given to you obtains the IP address via dhcp. So start the dhcpd on the machine you connected the network interface of the arduino to.

Do not forget to bring the interface configured with the highest IP in your subnet configured up before starting dhcpd...

To find the dhcpd just enter which dhcpd in the console.

Connecting the Arduino

When you connect the Arduino to the Ethernet port now you should see in the Arduino console and in the syslog that the Arduino obtains an address.

Press the red reset button to restart the Arduino and make it obtain a new IP if needed.

The console output of the Arduino IDE contains debug output...

Discovering your Smart Device

In a real world setup it is realistic that you do not have access to the logs of your dhcp server and also not to the Smart Device itself as it has no user interface and is reachable via network only.

Let's assume you would not know the IP of the device but you would know that it will listen on port 15002. Can you think of a way to find the IP with an nmap portscan?

Hint: the more you limit the scope of the scan, the faster it is...

Now use grep to let the shell return you only the relevant information of the host that has port 15002 open.

Test the connection

Now test the connection to the arduino using telnet. When you enter "@hello" the arduino should respond...

Do not forget to use the right port...

We will walk through the code now, so take a look at it while reading here.

Like every Arduino sketch, this code has a setup and a loop function. Both are automatically executed at startup.

The Setup Function

The setup function takes care of initializing the I/O pins that are used by the program. Modify the code to meet your needs and hardware layout.

The interrupt handler for the button is also registered in the setup function.

The Loop Function

As the name says the loop function is looping during run time of the Arduino. In our case it only calls the message handler when the "@" that is used as identifier for our protocol is the starting character.

The "@" is used as protocol identifier here. Even though it is on the application layer it has the same functionality like the next header field in IPv6 for instance...

Our Button Implementation (different from the demo)

We make use of this feature for two reasons. First, polling the button state in our main loop is a bad idea, because in general we don't know how long one loop iteration may take. So in worst case we might not register a button state change if it chages back to its original state quick enough. Second, polling the button state in a busy loop is bad design and not needed at all in our case.

Theoretically, we could use the same technique to react to incoming TCP message, but the provided library does not support this feature, as the Arduino website mentions:

The interupt is registered to react on "CHANGE" events, i.e. when the input signal changes from HIGH to LOW or vice versa. Every time the board detects a signal change, the function switchInterruptHandler is called. A static counter is increased, to keep track of the button state. Assuming that the button is not pushed at startup, pushing the button will set pin 8 to HIGH and pushing it again will set pin 8 back to LOW, while releasing the button does not change the state of pin 8 at all. The function also "de-bounces" the button. Bouncing in this context refers to a short burst of state changes when the button is pushed or released. See wikipedia: Contact bounce for more information about bouncing.

If you want to see the state change of pin 8, you can add another LED to your circuit that receives its current from pin 8.

Now it is time to extend the network communication protocol on your device to give access to all peripherals over the network.

Providing Access to the Peripherals over the Protocol

The template provides you with basic functionality:

• It obtains an IP address.
• It has message handlers for timers that we will use later.

All functionality except the one for the yellow and the green LED and your added peripherals is included already.

The access to most peripherals is straight forward. For all entity classes you find an implementation already.

Check that you connected all peripherals to the right pins as listed in the code.

Call the @hello command and try the returned commands. Everything should work except the commands for the yellow and green LED and you additional peripherals.

The goal of this part is to make the smart device offer all functionality advertised by the hello message plus the functionality needed for your peripherals.

Have a Look at the Code

Have a look at the code. Get an idea what is realized how in the code.

When providing remote access, the communication protocol is fundamental. In this lab we will use a simple communication protocol that only consists of extracting bytes from a TCP stream. For resource limited devices such a protocol is realistic.

The communication protocol is parsed with a call to parseInput, which uses a simple state machine. The protocol is further parsed with the rest of the parse* functions like, for example, parseReadGet.

Other approaches for protocol parsing could be lookup tables or regular expressions. Our chosen approach fits the tight memory restrictions, in particular in terms of RAM.

The implemented protocol supports the following commands already:

• @hello: returns info about the device
• @set:timer_{0,1}: sets the timer to the current time (value returned by a call to millis)
• @get:timer_{0,1}: returns the value of the resource

The "@" is used as identifier for the protocol. It helps distinguishing sent and received data in the telnet window as a side effect...

Parsing Function

The message parsing happens here:

/* reads from the parse buffer, expecting a command token or line break. valid command tokens
 * are: "hello", "get" and "set"
 * 
 * parameter:
 *              ptr                     pointer to a parser_t structure
 * 
 * note (1): reads from the global 'client' variable
 * note (2): this function guarantees that the buffer length on returning will be shorter than before
 * 
 * returns:
 *              1               read successfull
 *              0               error (e.g. token incomplete)
 */
int parserReadCmd( void *ptr ) {
        parser_t        *parser = (parser_t*)ptr;               // dealing with the automatic function declaration of the IDE
        char            token[32];                                              // used for printing a debug message
        
        if( parser->length == 0 )
                return 0;
        
        parser->buffer[parser->length] = 0;
        Serial.println( F("parser buffer:") );          // debug message to the serial console
        Serial.println( parser->buffer );
        
        if( !parserStrcmp(parser, "get:") ) {
                parser->state = PARSER_GET;
                parser->length -= 4;
                memmove( parser->buffer, parser->buffer+4, parser->length );
                return 1;
        } else if( !parserStrcmp(parser, "set:") ) {
                parser->state = PARSER_SET;
                parser->length -= 4;
                memmove( parser->buffer, parser->buffer+4, parser->length );
                return 1;
        } else if( !parserStrcmp(parser, "hello") ) {
                parser->state = PARSER_INIT;
                parser->length -= 5;
                memmove( parser->buffer, parser->buffer+5, parser->length );
                client.println( F("Possible commands: @hello;@get:{timer_0,timer_1,switch,temp,led_red,led_yellow,led_green};@set:{timer_0,timer_1,led_red,led_yellow,led_green}=[value]") );
                return 1;
        } else if( !parserStrcmp(parser, "\n")  ||  !parserStrcmp(parser, "\r") ) {
                // line break and carriage return are just used as end-of-command tokens and are ignored here
                parser->length -= 1;
                memmove( parser->buffer, parser->buffer+1, parser->length );
                return 1;
        } else {
                Serial.print( F("error - unrecognized command token:") );
                memset( token, 0, sizeof(token) );
                memmove( token, parser->buffer, sizeof(token) < parser->length ? sizeof(token)-1 : parser->length );
                Serial.println( token );
                
                parser->length = 0;             // error handling: buffer is completly purged
        }
        
        return 0;
}

Add identifiers for all your peripherals to be returned by the hello message.

Call the adapted @hello function on your smart device.

Your Protocol Message Handlers

Now it is time to implement the message handlers for the yellow and the green LED and your peripherals.

Some peripherals can be read out directly like the temperature sensor.

Others need more programming logic as their state is not directly retrievable for some reasons.

GET Handler

The GET handler handles the get protocol messages.

/* reads from the parse buffer, expecting a resource identifier
 * 
 * parameter:
 *              ptr                     pointer to a parser_t structure
 * 
 * returns:
 *              1               success
 *              0               error
 */
int parserReadGet( void *ptr ) {
        parser_t        *parser = (parser_t*)ptr;               // dealing with the automatic function declaration of the IDE
        char            token[32];
        int                     id;
        char            *end_ptr;
        
        parser->state = PARSER_INIT;
        Serial.println( F("parserReadGet") );
        if( !parserStrcmp(parser, "timer_") ) {
                id = strtol( parser->buffer+6, &end_ptr, 10 );
                if( end_ptr == parser->buffer+6 ) {
                        client.println( "error: invalid timer index");
                        return 0;
                }
                memmove( parser->buffer, end_ptr, parser->length - (end_ptr - parser->buffer) );
                sendTimer( id );
                return 1;
        }
        if( !parserStrcmp(parser, "switch") ) {
                parser->length -= 6;
                memmove( parser->buffer, parser->buffer+6, parser->length );
                sendValue( SWITCH );
                return 1;
        }
        if( !parserStrcmp(parser, "temp") ) {
                parser->length -= 4;
                memmove( parser->buffer, parser->buffer+4, parser->length );
                sendValue( SENSOR_TEMP );
                return 1;
        }
        if( !parserStrcmp(parser, "led_red") ) {
                parser->length -= 7;
                memmove( parser->buffer, parser->buffer+7, parser->length );
                sendValue( LED_RED );
                return 1;
        }

/**
 * =======================================================================================
 * TODO (iLab2):
 *      (3) Implement the message handling to get and set the state of your peripherals.
 *          ! Make sure the buttons and the LEDs have a useful implementation to retrieve
 *          their state.
 *   Implement the GET message handlers to access the state of your devices.
 * =======================================================================================
 */
 
        client.println( "error: unrecognized resource identifier" );
        return 0;
}

Add code to get the states of all your peripherals via the protocol.

SET handler

The SET handler handles the set protocol messages.

/* reads from the parse buffer, expecting a resource identifier
 * 
 * parameter:
 *              ptr                     pointer to a parser_t structure
 * 
 * returns:
 *              1               success
 *              0               error
 */
int parserReadSet( void *ptr ) {
        parser_t        *parser = (parser_t*)ptr;               // dealing with the automatic function declaration of the IDE
        char            token[32];
        int                     id;
        char            *end_ptr;
        
        parser->state = PARSER_INIT;
        Serial.println( "parserReadSet" );
        
        if( !parserStrcmp(parser, "timer_") ) {
                id = strtol( parser->buffer+6, &end_ptr, 10 );
                if( end_ptr == parser->buffer+6 ) {
                        client.println( "error: invalid timer index");
                        return 0;
                }
                parser->length -= (end_ptr - parser->buffer);
                memmove( parser->buffer, end_ptr, parser->length );
                setTimer( id );
                return 1;
        }
        if( !parserStrcmp(parser, "led_red=") ) {
                parser->length -= 8;
                memmove( parser->buffer, parser->buffer+8, parser->length );
                return setValue( ptr, LED_RED );
        }

/**
 * =======================================================================================
 * TODO (iLab2):
 *      (3) Implement the message handling to get and set the state of your peripherals.
 *          ! Make sure the buttons and the LEDs have a useful implementation to retrieve
 *          their state.
 *   Implement the SET message handlers to access the state of your devices.
 * =======================================================================================
 */

        client.println( "error: unrecognized resource identifier" );
        return 0;
}

Add code to set the states of all your peripherals via the protocol.

Make sure that all getter and setter are implemented and everything works.

All of your peripherals should be supported. You find them as reminder in the following list again:

Your hello message should return the commands to interact with all peripherals, and all the commands your hello message returns should work:

Now make a diff between your implementation and the original template.

Make sure that you do not overwrite your code accidentally when downloading the template again.