Tip o’ the Week 483 – mobile OCR and Office

clip_image002Optical Character Recognition is one of those technologies which has gone from being just-about-possible at great expense and hassle, to so mainstream that people just assume it will work flawlessly, all in a relatively few years. Numerous companies offer OCR services or addins to line-of-business systems which help to prepare printed data for easier consumption – scanning invoices for example.

clip_image004Consumers tend to use OCR in other ways; combined with language translation, you can point your phone at a foreign menu or sign and it may be able to help you understand. clip_image006In OneNote, if you have captured an image (maybe through the clipper addin from your browser), then it can extract the text from that picture – not always perfectly, and not necessarily well-formatted, but it’s probably quicker than re-typing everything.

Near OCR functionality is also pervading the slew of freely available Office apps for Android tablets, phones and even Chromebooks, and similar versions for iPad and iPhone.

A recent addition to the iOS version of Excel is the ability to scan a table of printed data and use OCR plus a bit of tweaking, to import the data into the spreadsheet. See more here. The same functionality was first made available on Android a couple of months earlier

clip_image001Start with the grid capture icon on the toolbar of a new spreadsheet, and then use the camera to highlight the area of a document that you’re interested in – the UI will be familiar to anyone who uses Office Lens, as the same anti-skewing technology is used to prepare the “document” for importing.

clip_image007clip_image008clip_image009

Then the OCR goes to work and tries to lay out the data as closely as possible to its source – obviously, your accuracy will be improved by having a well-lit and clear original document, and you’ll get to tweak the contents in context of seeing the OCR’d data and the scan at the same time, before committing to insert it.

Tip o’ the Week 459 – Building a better phone UI

clip_image002Microsoft fangrlz and fanbois, shed a tear for the Windows Phone platform, which relaunched with some fanfare just over 8 years ago as “Windows Phone 7 Series” (recalling the Microsoft redesigns the iPod packaging spoof?). The original idea with the new platform was that you didn’t need to jump in and out of apps all the time, since apps surfaced their info on the home screen and to a series of Hubs. Check out the original 2010 advert that painted the vision (fairly) clearly…

The hastily-renamed Windows Phone 7 showed up in November 2010, and came with a comparatively lavish marketing budget, bring some quite edgy and memorable adverts – like the Season of the Witch, or Really? (try not to boke at the scene where the guy drops his phone…)

A year later, and almost 7 years ago to this day, Canadian DJ and electro-music producer Deadmau5 played an amazing light show in London to celebrate the launch of the first Nokia Lumia phone; the fact that his track “Bad Selection” was the one that showcased what the phone looked like did raise a snigger at the time. He was back a year later with another event to celebrate the launch of the Lumia 820 with Windows Phone 8.

Now that Windows Phone has been in the ground for more than a year, it’s worth celebrating its somewhat spiritualclip_image004 successor – the Microsoft Launcher for Android (see ToWs passim, #345 and #438). One of the upsides of the Android platform is the fact you can effectively re-write the main UI, and most phone manufacturers ship their own variants of common apps (like Contacts, Phone, Messaging etc), so it’s ripe for customizing.

clip_image006The Launcher brings some of the design elements of Windows Phone to Android, while building in great new ideas – like the swipe right to the “Glance” screen, Bing visual search, Timeline integration with Windows PCs and more.

The Microsoft Launcher has had more than 10 million downloads and has a rating of 4.6 / 5, with over 750,000 reviews – and it’s recognised by many commentators as one of the best Android launchers, even in such a crowded market.

If you’re up for trying out a new release, sign up to be a tester for the Microsoft Launcher Beta – currently offering a major update (5.1) that includes better Cortana functionality, To-Do and Sticky Notes synch from PCs and more. See details here. Join the community here (Google+ is still a thing – who knew?)

The beta even has a new “Screen time” function that promises to tell you how often and how long you use the phone, and with which apps. Google has shipped a “Digital Wellbeing” feature for its latest Android release (v9 aka Android “Pie”), but many phones won’t get that release for ages, if at all. Microsoft Launcher works on Android 4.2 and later.

Tip o’ the Week 442 – Whose phone? Your phone…

clip_image002Back at the May 2018 Build conference, an app for Windows was previewed, which would allow you to manage content from your phone, on your PC – “Your Phone”.

If you’re an Android user, and a Windows Insider, then you can get a preview version of the Your Phone app for the PC; after starting the app on the clip_image004PC, it will ask for your mobile number and text you a link to download an updated version of the Microsoft Apps app (ya falla?). Download the update, sign in as appropriate, and suddenly your photos on the phone will start appearing in near real-time on your PC.

The Your Phone app actually uses a Wi-Fi connection on the phone to sync content with the PC – they don’t need to be on the same network but they do need to be able to talk to the back end service that coordinates things. For now, it just does photos (and only on Android), but in time, more services will be added. See more details here. And here.

clip_image006The photo sharing capability is pretty cool – if you ever find yourself taking a photo on your phone and then immediately wanting to use it on your PC, then your alternatives are either to wait for OneDrive to sync your new pic from phone to cloud (and then back down to your PC)… or plug the phone in on a USB cable and root about in its file system to find the photo. Or the worst, but probably most used: you email the photo from your phone, to yourself…

Some features of Your Phone will be tied to particular preview versions of Windows 10 – such as the recent latest build, 17228.

Tip o’ the Week 345 – Android in the Garage

clip_image002For some time now, there’s been a collective of off-the-wall projects and experiments (which may or may not become part of more fully-fledged Microsoft products) called the Microsoft Garage. In an homage to the semi-stereotypical Silicon Valley startup idea (a few techies hacking away in the garage to make stuff, as gave life to both Apple and HP among many others), the Garage has some projects that will live fast and die young, while others persist as long-term experiments on the periphery of maturity.

One of the more interesting developments – for Microsoft, at least – over the last year or two, has been the number of Garage projects which not only come out first for non-Microsoft platforms, but which may even be exclusive to other environments. The Word Flow Keyboard for iPhone, for example, has been getting rave reviews, yet doesn’t have an immediate equivalent for Windows or any other OS.

clip_image004If you’re an existing – or aspiring – Android user, you may want to take a look at a recently-updated Garage project, called Arrow.

There’s a concept in Android of the “launcher”, the software that presents the main UI to the end user for navigating installed apps. In desktop OSes, it might be called the shell or GUI; with Windows and Mac, you basically get what you’re given, but with Linux, there’s usually a choice of shells and UIs that you can pick from.

Well, Microsoft’s Arrow is a launcher (said to be the best, no less) aimed at the kind of user who wants to put their most useful content at the fore; whether that’s the apps you use the most, or widgets that summarise information that you care about. It’s nowhere near as good as the Start screen and live tiles environment on Windows Mobile, but it’s a step in the right direction.

Arrow has recently been updated (to v2.3.x.xx…) and now, as well as showing you Bing wallpaper of the day, lets you quickly search using Bing, from the home screen (just swipe up from the panel at the bottom of the screen as if you wanted to expand it vertically, and the Bing search box appears at the top).

The new release includes Wunderlist integration too, as well as some abilities to dial-down the animation if you’re a particular speed freak or you have an old device that struggles with modern window movements.

As with iOS, there is a plethora of other Microsoft apps available, too – from OneNote, OneDrive, Cortana, various MSN content-rich apps… and the Arrow launcher provides a neat and fast way of navigating the stuff you care about. Check it out on the Google Play store.

“Latent Data” – the secret sauce of the Internet of Things

This is part 2 of a series of articles exploring the Internet of Things, starting with the first post, What is this “Internet of Things” thing, anyway?

clip_image002The secret sauce, the Holy Grail, the raison d’être for Internet of Things is data. That much is pretty obvious to anyone with more than a passing interest in the field – why would you go to the bother of deploying a load of sensing devices and the infrastructure to manage and communicate with them, unless the data they provide is particularly interesting?

At Microsoft, we work with lots of partner companies who use our technology to build their own products and solutions. This often puts us into contact with people and organisations who are doing things we’d never expected or even imagined they’d do, and that is one of the reasons why it’s such a great place to work and an amazing ecosystem to be part of.

As part of this working with companies that are beginning to inhabit this growing Internet of Things niche, a special interest group sitting in Microsoft UK has drawn a few interesting, and sometime controversial, observations:

  • No one technology or technology provider will own the IoT, and a lot of systems will use a smorgasbord of standards and components
  • Scaling a system that manages a few hundred gadgets to one dealing with hundreds of thousands of sensors is very hard, as is managing and analysing a massive quantity of data
  • There are “stacks” within IoT
    • Sensors: the “things” in the IoT, massive in number but small in compute power
    • Hubs: the concentrators which harvest data from sensors, provide some degree of control, logic and processing and ultimately pass the information up the chain
    • Comms: many incompatible but functionally similar wireless standards will connect sensors to hubs, and hubs to the…
    • Cloud: the place where the data is brought back to, where analysis can take place on it and where insights can be passed on to other systems or even back to the devices
  • The real value will come from “latent data”

What is “latent data”?

To a large degree, the IoT is an emerging set of technologies, protocols and patterns for the collection, aggregation, analysis and actioning of intrinsic, latent data, and the management of this process.

Data is ubiquitous and inherent is all environments, be it an outside space, an ecosystem, a manufacturing complex, a supply chain or a city. This data can be regarded as “latent data” or “potential data” in the physical world – the data exists but is not accessible, or if it is accessible then it is of limited use since it is not combined with other, relevant data (such as historical readings, or data from complementary systems). Maybe the data is being accessed by some silo’ed system which uses that data for its own purposes but was never designed to provide any wider access to it.

Every physical thing has properties and attributes which may be discernable but are probably not being measured. A mechanical thermostat has intrinsic data on the temperature of a room and its own state, but this data remains in the physical world. A light bulb could be measured to see if it’s on or off, but this only becomes truly interesting when we could measure all the bulbs in a building, or a facility, or a city. If we can sense when all the bulbs need replacing, or alter their individual brightness depending on other conditions, that’s even more interesting.

For the avoidance of doubt, “Latent Data” is also a legal term applied to deleted files that need special forensic tools to extract… we’re talking about a more ethereal concept here, that there is data all around us in everything, but it’s untapped – and therefore, latent – unless we specifically decide to measure it and do something with it.

We believe that IoT is fundamentally about bringing this latent intrinsic data into the digital world in a way that allows the creation of value. This value is due to the aggregation of collected data, its analysis, and the use of that insight to drive decision-making and actions. The Cloud is the place where this data will be collected, where the data is likely to be stored in the long term, and where data aggregation and analysis will (mostly) occur.

image

The IoT patterns, the technologies and protocols that allow for this aggregation of latent data, are similar in a way to the OSI 7-layer networking model – the stacks which encompass devices, communications and Cloud. There are differing degrees of abstraction between these stacks and their constituent layers which means the IoT is inherently (and to the benefit of everyone within it and using it) a heterogeneous world.

Microsoft’s role in the Internet of Things

Microsoft has developed embedded systems that run in billions of devices already, and some of these could be considered part of an “intelligent system” that forms part of the IoT. Microsoft also has a hugely scalable and low cost cloud computing system in the Microsoft Azure cloud platform, where IoT applications can be quickly deployed and where the data that results from them can be securely kept and worked on.

Almost all IoT applications are likely to generate large volumes – petabytes, even more – of data, which will only become valuable when it is cost effective to keep it for a period of time and to perform large-scale computational analysis, both of which are difficult to do or economically unviable without the availability of public cloud computing.

If you’re an IoT developer building devices with Arduino or using systems like Raspberry Pi, and you’re writing your code in Python or Java and storing your data in some form of NoSQL database… that’s just fine by us. We think we have just the cloud service you need to let you concentrate on doing the stuff you started in this business for in the first place – writing your applications, building your devices and consuming your data.

With Microsoft Azure providing the backplane for these billions of devices to communicate – whether they are running Microsoft software or not – and to store and analyse their data, there is an opportunity for us and our partners to enable and monetize far-reaching change.

Some further reading:

  • If you’ve not seen much around Azure before, see an overview here.
  • For developer resources, including Node.JS, PHP, Python, Java and iOS & Android resources, see here.
  • For the more technically minded – The Microsoft Azure Service Bus and the Internet of Things (part 1 and part 2).