With recent advances in communication technology, words such as Big Data, Smart Cities, and Connectivity are frequently tossed around. However, the ways that big data should be transformed to accurate and reliable information and knowledge and how this information should be used in different applications is rather insufficient.
In the transportation industry, the effectiveness of major ITS projects that use big data heavily depends on how the traffic sensors are installed and how the collected raw data are filtered and analysed. For example, the accuracy and reliability of traffic data generated from Bluetooth and Wi-Fi signals (e.g. travel time) depends on many factors such as:
- Sensor detection range
- Relative distance of the sensor and its antenna to the road
- Type of antenna used i.e. Omnidirectional vs. directional
- Scanning technique, active vs. passive
- Source of MAC address detection Wi-Fi vs. Bluetooth
- Filtering algorithms applied to raw data
Among the above-mentioned factors, data filtering is the most important factor in extracting accurate and reliable travel time data from Bluetooth and Wi-Fi signals mainly because the nature of this data collection method involves a “range” rather than “point” detection. For example, Wi-Fi access points and multi-modal traffic are some of the sources of error that can be identified and eliminated only if the right filtering algorithms are employed. These algorithms should involve several layers of filtering to ensure that the remaining data is in fact the target data. Figure below illustrates an example of a weak vs. strong travel time data filtering algorithm under different traffic conditions:
To sum up, data is the starting points of most ITS solutions. Simply because different providers use a similar data collection technology, it doesn’t mean that their output information are equally good!