Microsoft has done a lot to encourage use of machine learning models in Azure, but there’s scope for their use in many other parts of its ecosystem. One increasingly important use is on Windows PCs, bringing trained models onto your desktop or onto embedded devices.
With local machine learning support, there’s no need to worry about shipping data to cloud services, especially where bandwidth can be an issue. You also can protect data where privacy can be an issue, and where regulations control data transfers and storage. It’s a model that’s likely to be important in financial services or in health care, where strong government regulations around privacy have a significant impact with what data you can process and how you process it.
I took an initial look at Windows ML a few months ago, but it’s worth a deeper dive now the release nears of a new Windows 10 version that will bring the Windows ML APIs to your code. If you’re using a Windows Insider build, you can start using the new APIs with the current preview release of the Windows 10 SDK. It adds support for managing models and for working with machine learning data types, including tensors. While you won’t be able to ship code until the Windows 10 update itself is released, now is a good time to start exploring how to add machine learning support to your code.
Bring your own machine learning model to Windows 10
The heart of a Windows 10 machine learning application is a pretrained model. It doesn’t matter where you trained it, or how you trained it, as long as it’s been exported as an ONNX file. With ONNX, you can treat a model that’s come from Microsoft’s own Azure ML service in the same way as one that’s been trained on the Google-backed TensorFlow, or on any of a wide range of open source machine learning platforms. Using the Windows 10 SDK, you can hook the model into a PC camera, a microphone, or a stream of sensor values.
Microsoft is putting Windows ML into the upcoming release of the Windows 10 SDK, for use in UWP apps. Most of the Windows ML APIs are found in the Windows.AI.MachineLearning namespace and are supported by its associated DLLs.
You start by using the LearningModel class to load an ONNX model into your app. Usually your code ships with an associated model so you can load it from a local file path, but you might want to load a model from remote storage if you’re expecting models to be updated regularly. There’s also the option of loading models that are being delivered over a stream, so you can work with encrypted models using the stream to handle decryption.
Once you’ve loaded a model, you create a local file to contain it and then attach an evaluation session to the model. It’s a relatively simple process, and you can have a model ready to use in only a few lines of code. The LearningModel class properties include details of its input and output features, as well as access to model metadata. Once a model has been loaded, you can call methods to close it, as well as dispose of resources that aren’t being used. Once it's been closed, you can’t work with the model any more, so use this option only when finishing up.
LearningModels can be bound to a specific device in your PC, letting Windows ML take advantage of features like GPU acceleration via DirectX (with the option of choosing high or low power) or via a CPU. It’s worth considering your options, because new CPU hardware is adding machine-learning-specific instruction sets that makes it a useful alternative to installing expensive and power-hungry GPU cards. The default is to use the CPU, though this may change in the future.
Getting data in and out of a machine learning model
Once you’ve loaded a LearningModel, you can use .Net’s reflection tools to get its InputFeatures and its OutputFeatures. These define the types of data a model can handle, with four options: tensor, sequence, map, and image. Tensors are multidimensional arrays, and they can hold a considerable amount of data.
Similarly, images are handled as tensors, adding batch and number of channels, as well as the size. So, you need to convert any image you’re processing with Windows ML to the appropriate tensor format before processing it. Microsoft provides the ImageFeatureValue class to handle conversions, though if you prefer you can write your own code to convert an image to a tensor.
Once converted into the appropriate format, you can bind your data to the appropriate input, before calling LearningModelSession’s Evaluate method to get the result, which is available from the LearningModelEvaluationResult class. It’s worth looking at the sample code for this class to see how you deliver data to a model, and read back the result. The process is relatively simple, though as with all complex tasks it’s a good idea to implement it as an asynchronous function, because loading and processing models on PCs and IoT hardware can take time.
Azure ML on Windows 10: Democratizing data science
Microsoft has done a lot to simplify the process of using machine learning models in Windows code. The Windows ML APIs move it into the familiar world of .Net and give you the tools you need to convert your data into model-friendly formats.
While you do still need to consider the complexities of tensors, much of what you’ll be doing with Windows ML will be around working with images, sending frames from video, or delivering streams of values from sensors. In many cases, where you may be using tensor formats, the underlying data will be familiar arrays of, for example, financial data when using machine learning for mortgage or loan approval.
The result is democratized data science. Models can be built using specialized data science tools, like Anaconda Python or Jupyter Notebooks, before being trained on Azure ML. Once data science specialists are happy with the model, it can be exported and made available for anyone to use. With familiar code wrapping that model, you can take it and use it wherever it’s needed.