About the research
The Iowa Department of Transportation (DOT) consumes data from multiple streams that are stored to assist in better decision-making. Despite access to unprecedented amounts of data, decision-makers are often restricted in their ability to explore multiple data sets. This research demonstrated a simple proof-of-concept architecture that addresses some of the constraints on decision-makers but also opens up additional data sets for the Iowa DOT or other researchers to explore without the additional time and effort needed to integrate the data. Users can spend additional time analyzing the data rather than interpreting and processing the raw data. Three data sets used for this demonstration include crash, INRIX probe, and weather data for the past five years.
The data integration methodology developed for both the weather and probe data is a multistep process with intermediate outputs created along the way that are needed for later steps in the integration. Seven outputs were created that are all related back to the crash data. The research team considered any potential output that may be beneficial for future integration efforts and outputted those as separate tables for future use. The primary outputs of the integration process are the weather and probe data at the time of the crash, which allow for the data to be joined directly with the crash data similar to other attributes collected or derived within the crash data. For advanced analysis, a Python script was also developed that allows the probe and weather data to be extracted for a configurable amount of time before and after each crash.
The Iowa DOT views this project as an initial effort to develop a system that enhances crash data reports by integrating additional data sources. The ultimate goal is to have a system that allows any pertinent data sets to be readily available when evaluating crashes and to be utilized within any safety and mobility decision-making. The work in this study has established a foundation to simplify the efforts to integrate additional data sources by associating the crash data to the Iowa DOT’s linear referencing system (LRS).