There was a very interesting article from Dan Kusnetzky in ZDNET about Big Data. Telco has always been at the forefront of producing, processing, and analyzing large amount data. What I'm witnessing today is another order of magnitude of data volume. When items such as razor blades, racks of fresh produce, jeans, and pallets are "sending" updates about their locations, status, and temperature through a network of automated sensors, we are talking "big data". That data tells a lot about habits and trends at a micro (privacy) and macro (economy) scale.
If you were to send all this raw data right to a central server, you would make Big Data simply Huge Data, and immediately lose the value of real-time automated data. We believe a large amount of information generated at the Edge should be used at the Edge. Only a fraction of that data should make it to a central server and become Big Data.
When a forklift picks up a pallet full of fresh berries from a loading dock, then drives through a multitude of portals equipped with sensors to finally put away the fruits in a cold room, thousands of records have been created. They include very important real-time information such as: "Is this the right truck and loading dock?", "Is this the right pallet?", "Are these the right berries?", "How many workers are around the forklift while it is backing out?", "Are all the pallets in the right cold room?". This data is critical during the actual movement of the assets but unless exception occurs, we are only interested in a small subset for the long term.
