New York City Transit Authority
Traditionally, the NYCT employed traffic checkers to collect bus and subway performance data, then reports were created with Microsoft® Excel® and presented quarterly. New technologies now generate data automatically.
Subway movement information is provided via NYCT’s General Transit Feed Specification (GTFS), while location intelligence from Google Maps integration is transmitted from up to 5,000 buses every 30 seconds, 24 hours a day. In addition, the Operations Planning department gathers diagnostic and event data from its Intelligent Vehicle Network sensors, which transmit data when something of note happens, such as movement of doors, wheelchair ramps, etc. In total, tens of thousands of records per minute—or millions of rows of data per day—must be recorded and distilled into useable and helpful information.
Despite this wealth of data, there was no way of analyzing and distributing all of that information in a quick and effective manner. To create true performance change, the department knew it had to offer data on a daily basis to enable timely performance and service adjustments. To accomplish that, it needed a robust reporting and analytics application.
Meeting realtime needs
The solution would have to support NYCT’s existing data collection technology. It would also need to keep up with big data demands and allow interested parties to examine data when and how they wanted, accessing more detail as required. The goal was to stay on top of performance concerns in realtime—from malfunctioning lifts to route change needs or gaps in service. “With the right tool, they could become much more proactive in how they’re monitoring the service and how they’re improving it,” said Anthony Cramer, director of System Data Analysis with NYCT’s Operations Planning department.
To accomplish that, the reporting technology needed to meet the following key criteria:
- Be cost effective, flexible and scalable enough to keep up with the millions of data points coming in each day
- Connect to multiple data sources, including data accessed directly from NYCT vehicles, such as vehicle sensors and location intelligence. One example is NYCT’s Bus Time Automatic Vehicle Location tracking program, which follows GPS data on every bus
- Work seamlessly with open source database MongoDB for big data and Google Maps for location intelligence
- Be easy to use and accessible for the many departments looking to access data daily, without the need for extensive IT assistance
“The goal was to make it a self-service shop, to let the people who want the reports get them when they need them, and be able to run them and customize them as needed,” said Cramer. “We felt that the email distribution route was not efficient—if some people were out, it just became a distribution bottleneck. We wanted to eliminate that.”
Converting to OpenText Magellan BI & Reporting
With those goals in mind, NYCT sought a reporting platform that was powerful enough to keep up with these demands, but could also fit within their budget and resource constraints. “We had a pretty good feel as to what we needed to deliver. What we didn’t know were the different mechanisms for delivering it,” Cramer said. “We’d never got to the point of presenting it in a visually arresting manner. We just didn’t have the technology.”
The team began looking at software vendors for out-of-the-box reporting. Most of the offerings were either not flexible enough to customize to transit reporting needs—as they were designed for more finance-oriented reporting—or exceeded both their budget and infrastructure capabilities.
The team chose OpenText™ Magellan™ BI & Reporting, branding it as Operations Research Computational Analysis (ORCA). Able to scale to the organization’s needs, BI & Reporting could easily plug into all of NYCT’s data sources, both present and those planned for future use. Virtualization and Google Maps apps expanded its capabilities, with mobile access enabling employees to view reports while on the go. Overall, the team has been impressed by the solution’s ease of use.
“One area where BI & Reporting really shines is the integrated development environment. That’s a prime reason why we chose OpenText,” said Tuan Huynh, senior developer with NYCT. “Now we’re integrating our systems seamlessly. It gave us flexibility, scalability and visualization.”
Illy Popovits, NYCT’s development manager, added: “BI & Reporting could integrate with our different data sources. None of the other solutions we looked at, or that we could develop, were as flexible or as user friendly.”
Initially, NYCT focused largely on the solution’s reporting functionality, creating and distributing approximately 80 different reports, both in realtime and on a nightly basis. They have also created dashboards to offer snapshots of performance to senior management. The team plans to expand dashboards even more, empowering field personnel to drill down further to get to the details they need.
With the help of BI & Reporting, their goal is to create data transparency, where key information is easily accessible both internally and externally, driving continual performance improvement. “NYCT is committed to an open data environment,” Cramer said. “That begins with making as much data as possible available to our customers, from countdown clocks at subway stations to realtime arrival apps, all powered by an open, cloud-based system.”
The OpenText solution is doing its part to help NYCT achieve data transparency, while also enabling the seamless flow of data through the transit department. Transit schedules can now be adjusted more rapidly, improving service for commuters throughout the city. “We’re providing them the tools to respond to customer complaints,” said Cramer.
