- Cloud Service Components
In the third and last blogpost in this series I talk about Cloud Service Components
The cloud services you choose is an essential part of your overall solution. In fact, the cloud services used in your solution constitutes the ‘I’ in IoT. There are options from many of the larger companies participating in this space as well as offerings from startups and medium-sized businesses. Now we have to look at categories of services these companies offer to give you an idea of how cloud services fit into an overall IoT architecture.
The concept of a field gateway–a piece of hardware that brokers communication between IoT devices and cloud services. Cloud gateways do more than broker communication. They provide a set of services that devices can run either locally or in the cloud. Cloud gateways can provide workloads such as (among others):
– Authentication and authorization
– Message brokering
– Data storage and filtering
– Data analytics Data Storage Options
Given the centrality of data in an IoT solution, figuring out the right cloud-based data storage and retrieval options ranks high on the list in terms of importance. IoT devices can generate enormous amounts of data very quickly and storing high volumes of data in the cloud cannot become expensive but also unwieldy–you must be able to do something with the data and too much of it can make analytics and decision-making harder.
Data is often time series data and is required to be stored where it can be used in visualization and reporting as well as later accessed for additional processing. It is common to have data split into “warm” and “cold” data stores. The warm data store holds recent data that needs to be accessed with low latency. Data stored in cold storage is typically historical data. Most often the cold storage database solution chosen will be cheaper in cost but offer fewer query and reporting features than the warm database solution.
A common implementation for storage is to keep a recent range (e.g. the last day, week, or month) of telemetry data in warm storage and to store historical data in cold storage. With this implementation, the application has access to the most recent data and can quickly observe recent telemetry data and trends. Retrieving historical information for devices can be accomplished using cold storage, generally with higher latency than if the data were in warm storage.
Analytics Services and Data Visualization
Once data is captured and stored, it only becomes useful when it provides insights into the physical world from which your IoT devices have captured the data. This is where analytic services come into play.
Azure Analysis Services, for example, enable architects to use advanced mashup and modeling features to combine data from multiple data sources, define metrics, and secure data in a single, trusted tabular semantic data model. The data model provides an easier and faster way for users to browse massive amounts of data for ad-hoc data analysis.
Without analytics, data collected from IoT would be too voluminous and unstructured to visualize or gain insights. Analytic services enable architects to build meaningful relationships between sets of data to make it easier to manage. For example, Azure Stream Analytics can take stream data from IoT devices and engineers can specify a transformation query that defines how to look for data, patterns, or relationships. The transformation query leverages a SQL-like query language that is used to filter, sort, aggregate, and join streaming data over a period of time.
Stream analytics can help condition data so it´s easier to manage and provides models that give insight into what you need to understand or learn. Once the data is conditioned and you’ve created the right models, the data can be visualized using tools like Microsoft’s PowerBI so it can be acted upon.
Data visualization tools can take input from various data streams and combine them into “dashboards” that can be used to tell a story about the data that was collected. Ultimately, this is the goal of IoT.
Machine Learning (ML) is one of the more exciting developments in modern computer science. It’s a complex field but one that is producing significant positive results with large datasets. IoT devices produces large large volumes of data. Analytic systems help engineers to model the existing data in meaningful ways. Machine learning takes this a step further and can actually make predictions about what new data will show and provide insights that would not be possible without the machine learning algorithms.
As the name states, the technology gives computers the ability to “learn” (predict) from data by expressing trends or a direction future data will take. This can provide engineers with a powerful mechanism for enabling a wide variety of scenarios.
Using big data and machine learning to predict purchasing decisions is one simple example. Suppose a retailer has warehouse space in various cities and needs to determine which items to stock in those cities in order to be able to get products to customers in the most efficient and timely way. Using machine learning the retailer can predict, for example, that a given set of users that purchase a specific television tend to buy a particular type of cable and other accessories like tv stands and audio equipment. This would allow the retailer to keep those items in the warehouse near where those television sales are popular so that if a customer orders the cable or other accessory, the item can be shipped more and get to the customer more quickly.