July 9, 2007 By David Raths
Legacy systems can usually do what they were designed to do; the challenge is opening them up to constituents via the Web or allowing staff members to do analysis. "Internal staff may ask questions such as, 'How many taxpayers in our jurisdiction paid more than $20,000 in taxes last year?" said Grant Brodie, president and chief architect of Arbutus Software, a Canadian-based company specializing in legacy data access solutions. "With a traditional legacy system, there's no way to answer such what-if type questions."
Traditionally a city government would have a programmer write a COBOL program to extract that information. Very often, however, once personnel get answers to their initial questions, they ask follow-up inquiries. "Then another program would have to be written to answer that," Brodie explained. "COBOL is not very flexible."
Programmed special-purpose solutions are low-cost, but they are best suited for nonrecurring needs rather than ongoing ones.
To allow for more robust analysis, many organizations use ETL technology to create large data warehouses that can combine data from multiple internal sources. One benefit is that it balances the processing load by off-loading the data analysis to a separate server. Another is that once the ETL process is established, you can conduct as many different types of queries as needed.
Yet despite their advantages, data warehouses usually involve high front-end and maintenance costs, and the data soon grows out of sync with what's happening on the mainframe, so they're not well suited for real-time analysis.
The U.S. Food and Drug Administration (FDA) turned to ETL technology and a data warehouse to let human resources staff members integrate data from disparate systems and analyze trends using Business Objects' business intelligence software.
The FDA, which is a U.S. Department of Health and Human Services (HHS) agency, kept records on its approximately 9,000 civilian employees in a system that was custom-developed for the HHS, while data on uniformed personnel in its U.S. Public Health Service Commission Corps resided in a different system. Time and attendance records were in payroll software, and records on contractors were in another system.
"Pulling all that together and getting reports out was a frustrating, cumbersome process," remembered Ray Russo, director of the Office of Business Enterprise Solutions in the FDA's Office of the Chief Information Officer.
Russo's office created a data mart that pulls together data from all those sources. "This allows us to create historical views, as well as a snapshot for each pay period," Russo explained. "Now we can ask questions such as which employees are going to be eligible for retirement during certain time horizons. We can study attrition rates over certain time frames. There's no personnel-type question that we can't answer."
If the FDA's IT analysts still have concerns about legacy systems, Russo said, it has nothing to do with combining data and doing sophisticated analysis.
Otto Doll said South Dakota is moving away from the data warehouse approach to more just-in-time information. "We'd rather have more tightly integrated applications where we're dealing with the real thing."
Doll cites the federal government's sex offender databases as a model. The federal government could have created a huge data warehouse, pulling down information from all 50 states and refreshed it regularly, Doll said. "Instead they chose to ping all 50 state databases whenever there's a request."
If creating a data warehouse is time-consuming and expensive, rewriting an application or porting it to a newer platform can be even more troublesome. If you have unlimited resources, Arbutus' Brodie said, rewriting the application is a viable option. Yet most legacy systems still work well. "Replacing it for that reason," he added, "would be like killing a fly with a sledgehammer."
Eliminating the legacy system is a more drastic approach for CIOs to contemplate, admitted Federico Zoufaly, executive vice president of business development for ArtinSoft, a Costa Rica-based company that automates the translation of legacy code into more modern languages, such as Java. "It depends on your long-term strategy," he said. "If you're looking to gain flexibility, you may want to move to a newer architecture and off-load your mainframe applications gradually."
But Brodie said switching from a COBOL program to something like an Oracle database is a daunting task. "It's horribly expensive and problematic," he said. "There's no automatic button to push to do the conversion, so you're counting on people, and there's always the opportunity for errors to creep in. You don't want to do that unless you absolutely have to."
Integrating Justice Data in Illinois
Most people probably assume that if a burglary is committed in a metropolitan area, and the next day a similar burglary happens in the adjacent town, the police investigating the first burglary are aware of the second one. But more likely they aren't, said Kirk Lonbom, assistant deputy director of the Information and Technology Command of the Illinois State Police (ISP) in Springfield.
"Each jurisdiction has its own computer system and its own incident reporting software," he said, "and they don't talk to each other."
To break down those barriers, the ISP is creating the Illinois Citizen Law Enforcement Analysis and Reporting System (ICLEAR), a common data warehouse and standardized police incident reporting system that the state's 400 local police agencies and 40,000 officers can use to share information.
Based on a system used by the Chicago Police Department, ICLEAR is designed to help officers identify trends and allocate resources to drive down crime.
The ISP has its own legacy system issues to deal with, but Lonbom's problem is bigger than getting the 40 ISP applications to work together. "The real legacy data is not under our roof, but under 400 local agency roofs," he said. "Our challenge is to allow those local agencies to continue to use their systems but share information by using a common data format -- the Global Justice XML data model. Criminal justice systems are really coming together with specifications for data exchanges."
This year the ISP is working with iWay Software, a subsidiary of Information Builders Inc., to create specifications for the transfer of data between the ISP and the Chicago Police Department.
With funding from the Department of Homeland Security, the ISP will roll out an incident reporting system pilot this year. The ISP plans to provide the system free to jurisdictions in the state, with the hope of eventual statewide adoption. "The pilot projects we're starting with are small," Lonbom said, "but their potential is not."
600 Billion Lines of Legacy
Yet most IT leaders eager to replace their legacy systems admit that the process is expensive and will take years to accomplish. In a 2006 white paper, EDS estimated that mainframes execute 75 percent of all business logic at the enterprise level, and that there are approximately 600 billion lines of legacy code still in use.
While CIOs are waiting to replace their legacy systems, many realize that they have a responsibility to pursue greater flexibility through integration strategies because they are held to a higher standard of service than ever before. For instance, it's no longer deemed acceptable for a social services caseworker to scribble down information about a caller, access several systems and then call the person back.
Instead, that caseworker is expected to get a snapshot of the caller from an application that instantly queries all state systems. And whatever integration strategy is pursued, the CIO's job is to deliver that connectivity.
South Dakota's Doll is regularly reminded of the impact of EAI on his agency's clients. "When we meet to discuss something complex they need to accomplish," he said, "it's much less likely that the technology is going to get in the way."