Tag: IoE

“Latent Data” – the secret sauce of the Internet of Things

Posted on by ewand

This is part 2 of a series of articles exploring the Internet of Things, starting with the first post, What is this “Internet of Things” thing, anyway?

clip_image002The secret sauce, the Holy Grail, the raison d’être for Internet of Things is data. That much is pretty obvious to anyone with more than a passing interest in the field – why would you go to the bother of deploying a load of sensing devices and the infrastructure to manage and communicate with them, unless the data they provide is particularly interesting?

At Microsoft, we work with lots of partner companies who use our technology to build their own products and solutions. This often puts us into contact with people and organisations who are doing things we’d never expected or even imagined they’d do, and that is one of the reasons why it’s such a great place to work and an amazing ecosystem to be part of.

As part of this working with companies that are beginning to inhabit this growing Internet of Things niche, a special interest group sitting in Microsoft UK has drawn a few interesting, and sometime controversial, observations:

  • No one technology or technology provider will own the IoT, and a lot of systems will use a smorgasbord of standards and components
  • Scaling a system that manages a few hundred gadgets to one dealing with hundreds of thousands of sensors is very hard, as is managing and analysing a massive quantity of data
  • There are “stacks” within IoT
    • Sensors: the “things” in the IoT, massive in number but small in compute power
    • Hubs: the concentrators which harvest data from sensors, provide some degree of control, logic and processing and ultimately pass the information up the chain
    • Comms: many incompatible but functionally similar wireless standards will connect sensors to hubs, and hubs to the…
    • Cloud: the place where the data is brought back to, where analysis can take place on it and where insights can be passed on to other systems or even back to the devices
  • The real value will come from “latent data”

What is “latent data”?

To a large degree, the IoT is an emerging set of technologies, protocols and patterns for the collection, aggregation, analysis and actioning of intrinsic, latent data, and the management of this process.

Data is ubiquitous and inherent is all environments, be it an outside space, an ecosystem, a manufacturing complex, a supply chain or a city. This data can be regarded as “latent data” or “potential data” in the physical world – the data exists but is not accessible, or if it is accessible then it is of limited use since it is not combined with other, relevant data (such as historical readings, or data from complementary systems). Maybe the data is being accessed by some silo’ed system which uses that data for its own purposes but was never designed to provide any wider access to it.

Every physical thing has properties and attributes which may be discernable but are probably not being measured. A mechanical thermostat has intrinsic data on the temperature of a room and its own state, but this data remains in the physical world. A light bulb could be measured to see if it’s on or off, but this only becomes truly interesting when we could measure all the bulbs in a building, or a facility, or a city. If we can sense when all the bulbs need replacing, or alter their individual brightness depending on other conditions, that’s even more interesting.

For the avoidance of doubt, “Latent Data” is also a legal term applied to deleted files that need special forensic tools to extract… we’re talking about a more ethereal concept here, that there is data all around us in everything, but it’s untapped – and therefore, latent – unless we specifically decide to measure it and do something with it.

We believe that IoT is fundamentally about bringing this latent intrinsic data into the digital world in a way that allows the creation of value. This value is due to the aggregation of collected data, its analysis, and the use of that insight to drive decision-making and actions. The Cloud is the place where this data will be collected, where the data is likely to be stored in the long term, and where data aggregation and analysis will (mostly) occur.

image

The IoT patterns, the technologies and protocols that allow for this aggregation of latent data, are similar in a way to the OSI 7-layer networking model – the stacks which encompass devices, communications and Cloud. There are differing degrees of abstraction between these stacks and their constituent layers which means the IoT is inherently (and to the benefit of everyone within it and using it) a heterogeneous world.

Microsoft’s role in the Internet of Things

Microsoft has developed embedded systems that run in billions of devices already, and some of these could be considered part of an “intelligent system” that forms part of the IoT. Microsoft also has a hugely scalable and low cost cloud computing system in the Microsoft Azure cloud platform, where IoT applications can be quickly deployed and where the data that results from them can be securely kept and worked on.

Almost all IoT applications are likely to generate large volumes – petabytes, even more – of data, which will only become valuable when it is cost effective to keep it for a period of time and to perform large-scale computational analysis, both of which are difficult to do or economically unviable without the availability of public cloud computing.

If you’re an IoT developer building devices with Arduino or using systems like Raspberry Pi, and you’re writing your code in Python or Java and storing your data in some form of NoSQL database… that’s just fine by us. We think we have just the cloud service you need to let you concentrate on doing the stuff you started in this business for in the first place – writing your applications, building your devices and consuming your data.

With Microsoft Azure providing the backplane for these billions of devices to communicate – whether they are running Microsoft software or not – and to store and analyse their data, there is an opportunity for us and our partners to enable and monetize far-reaching change.

Some further reading:

  • If you’ve not seen much around Azure before, see an overview here.
  • For developer resources, including Node.JS, PHP, Python, Java and iOS & Android resources, see here.
  • For the more technically minded – The Microsoft Azure Service Bus and the Internet of Things (part 1 and part 2).

What is this “Internet of Things” thing, anyway?

Posted on by ewand

I’m going to start the odd bit of blogging aside from the weekly Tips that will carry on making their way onto this blog. Here is the first in a short series looking at the Internet of Things.

The term “Internet of Things” (or IoT) has achieved buzzword fever pitch in 2014, thanks in part to a slew of product announcements at the Consumer Electronics Show in January. Combined with high-profile acquisitions (such as Google’s purchase of smart home technology company, Nest), there have been many news stories which associated the subject of the piece with the Internet of Things.

imageEven people who work in the IoT world sometime struggle to articulate what it actually is. There are several ways of looking at IoT, however, and some of the scenarios are only being developed now and will become both significant and disruptive in ways that we probably don’t yet understand. There are, in fact, numerous types of “Internet of Things” application.

The consumer market provides plenty of examples of devices measuring and reporting data back to some kind of service that allows users to use that data for some purpose that would otherwise be difficult or impossible without this technology. “Wearable technology” is a category that typifies this approach.

Industrial Internet of Things applications have often existed for years, just under different names – M2M, SCADA, telemetry of numerous sorts – though are being combined in new ways and with new variants of technology, to open up new scenarios such as telematics. Industrial uses could mean using IoT technology to control a manufacturing process, to monitor complex machines in the field, extending even to remotely monitoring cars for the purposes of insurance, road tolling, safety and performance improvements.

Finally, companies will find a way to use IoT technology inside their own environments, offering up data that is consolidated from other systems and collected using sensors, to be combined with customer relationship management systems, building control systems and a host of other uses.

The interesting thing is, the majority of these examples won’t connect the many things to the Internet at all – maybe the devices and sensors at the very edge of the system will be individually addressable, but they almost certainly won’t be directly connected to the internet. Other groups have tried to establish alternative definitions – some talk or a “Sensor mesh” or a “Network of sensors", and Cisco, for example, talks about the “Internet of Everything” (and has some other, intriguing ideas such as Fog Computing… it’s like Cloud Computing but nearer the ground). It looks like the term IoT has stuck, at least until we stop talking about it as if it’s something special or something different, rather than just the normal way that these things work.

The definitive definition of the Internet of Things

The term “Internet of Things” was coined in 1999 by Kevin Ashton, from Proctor & Gamble, then at MIT. He later wrote, in 2009:

“Nearly all of the data available on the Internet were first captured and created by human beings—by typing, pressing a record button, taking a digital picture or scanning a bar code. The problem is, people have limited time, attention and accuracy—all of which means they are not very good at capturing data about things in the real world. If we had computers that knew everything there was to know about things—using data they gathered without any help from us—we would be able to track and count everything, and greatly reduce waste, loss and cost. We would know when things needed replacing, repairing or recalling, and whether they were fresh or past their best.

The Internet of Things has the potential to change the world, just as the Internet did. Maybe even more so.”

Analysts express differing views as to the exact scale (IDC reckons 212 billion devices with a market value of nearly $9trillion in only 6 years), but all estimates of the future size of the Internet of Things business are extraordinarily large. If even the lower end forecasts are out by a factor of 10, there will still be billions of connected devices within a few years, and the reason those devices are connected is because they have something to communicate.

What’s Microsoft got to say about this Internet of Things? Well, there are lots of things out there which read data and send it for further processing, and those things are probably not going to be running anything like a Windows OS. There are going to be devices which these sensors talk to, which collect the information and process it to some degree (validating it, packaging it for onward delivery, maybe issuing commands back to the sensors on the basis of what the data says). These devices may well be running a more functional OS, something akin to Windows Embedded and forming part of what we could call an Intelligent System. Then there’s the cloud platform that will tie everything together – boy do we have one of those… called Windows Azure. 

Next time, I’ll explore a few thoughts developed by some Microsoft experts and collected by talking with partner companies active in this area.

For further reading in the meantime, check out a couple of articles on the MSDN Magazine site (here and here), which shows some practical examples of how to use the Azure Service Bus to build an IoT application.