InTrans / Jul 18, 2019

REACTOR Lab conducts evaluation of open data portals and their transportation data

Improvements in technology and government initiatives toward open data policies are motivating transportation agencies to publish their data openly.

Open data are data that can be used and reused freely, often with attribution, and are commonly defined by three key features: availability, redistribution, and universal participation. The US government moved to implement more open data policies starting in 2013, which, in turn, led many state government agencies to try to implement similar policies.

However, with little guidance, many agencies, including departments of transportation (DOTs), are left with questions about what an open data portal is, what transportation data can or should they offer, how user-friendly does the web portal need to be, and whether this openness is leading toward targeted innovation, transparency, and discoveries. There is an imminent need to assess the quality of open data portals and to provide agencies with a yardstick to measure performance and to be inspired by higher ranking portals.

To achieve this, researchers at the Institute for Transportation (InTrans) at Iowa State University focused on evaluating the quality of open transportation data portals including data content. Through these efforts, researcher’s developed the Data Portal Evaluation Rubric (DPER) to evaluate the open data portals based on their portal usability, data content, and relevance of content to transportation.

The DPER is a weighted rubric used to calculate a score out of 100 for each portal studied. For this research, the team – led by doctoral candidate Archana Venkatachalapathy — studied 43 open data portals (39 state portals and 4 national portals) with open transportation datasets.

Open data portals selected for study
Open data portals selected for study

The rubric consists of three levels of evaluation. At the lowest level, researchers examined the features present in the portal and data content, such as search bars, data descriptions, etc. These features were categorized into specific parameters that highlight their essence. For example, “ease of usage” is a parameter described by features such as search bar, categorization of data, tutorial videos, and access to portals. Several parameters are grouped into one of the three main categories across which the portal is evaluated by DPER. The weights of the rubric were calculated based on an analytical hierarchy process survey conducted among open data publishers and users.

Hierarchy structure of DPER
Hierarchy structure of DPER

The visualization of portal ranking chart shows the state of New York’s open data portal securing the highest score of 72.30 (out of 100). It offers the largest number of transportation datasets covering 19 of the 20 topics listed in the rubric.

The DPER also indicates the areas of variability, thereby highlighting the need to define a uniform format for publishing open data that can lead to beneficial results. Further, the DPER helps identify areas that need to be standardized. This uniformity can also benefit developers and researchers who want to obtain data across multiple agencies without the barrier of inconsistencies in data content.

Visualization of portal ranking
Visualization of portal ranking

With the advent of DOTs launching their repository of transportation datasets, the time is right to explore the idea of standardizing both open data and the design of its portals. To share the valuable information learned through this research, the team created a visualization tool using the Tableau platform. This tool helps visualize the different aspects of the study, highlighting key findings while also serving as a repository for open transportation data in the US.

The rubric serves as a beginning point toward a more clear perspective of the different open data portals in transportation. Using the rubric, individual agencies can not only identify their own ranking but can also observe noteworthy practices of higher scoring agencies. The rubric becomes a yardstick in guiding DOTs and any agency considering publication of open transportation data.

This work was supported by the Iowa DOT Operations Division as part of their focused efforts to encourage innovation through sharing of data and in preparing for higher levels of transportation connectivity and autonomy.

The interactive open data portal evaluation is available here.