Big Data – Big Waste?

Big Data addresses two topics: the storage of large volumes of unstructured data and the evaluation of large volumes of data. In the eANV, among other places, we have both:

Where is my consignment note? Did I sent it with this content?  Why didn't it arrive at the authorities?

These and similar questions are fielded on an almost daily basis by people involved in the eANV. And the answers? In this complex, asynchronous process, answers aren't so easy to find. The consignment notes are stored in their transmitted and received versions, and there is a nearly infinite number of procedural and technical receipts and log information. If the case is a current one, most of the data is still available and somebody can search through it.

But if the transaction is older, then the log information in particular and many of the electronic receipts will already have been deleted. In both cases, the Big Data approach helps: The storage capacity and designs can store all the data, so nothing has to be deleted and the analytical process can deliver the answers needed across a wide variety of documents and data.

Governikus ITU has based their Big Data solution on open source – the well-known MongoDB and the Hadoop ecosystem. Access is provided through our eANV-Waste API.

If the storage of as much information as possible is in question, then it can be stored in the built-in GridFS (FS=filesystem) of MongoDB. Access to the systems is provided through Modawi or, if Modawi is not available, the ERP software can also call up the functions of the eANV-Waste API directly.

In the extended version (called here through Modawi-Extended or a visualization), in addition to the CRUD functionality, complex reports based on Hadoop are also available. In this scenario, the eANV-Waste API stores the data and documents in the HDFS to make them available to Hadoop.

Of course, it is quite possible to transition from the GridFS to the HDFS scenario. Your own Big Data application can also gradually be phased in. Modawi users only need a small computer cluster with MongoDB to take the first step. All functions to use them are available in the Modawi API as usual.

Not only eANV data, but also any arbitrary information such as log files, data flows from scales, telematics data, in fact documents of any kind can be stored through Modawi or the eANV-Waste API.