November 24, 2008 By Leslie Friesen
Gone are the days when an agency could store all of its data on a small server. Industry experts report that storage needs are increasing between 30 percent and 50 percent annually. This trend will only soar as the size of multimedia and other file types continues to explode.
Fortunately as the demand for storage space rises, the cost of storage media declines. Samsung, for example, currently offers a 1-terabyte hard drive for less than $200. Western Digital, Seagate and Hitachi offer comparable disks for similar prices. A standard 500 GB hard drive typically costs about $100.
The prevailing thought used to be that there's no such thing as having too much storage. So with the cost of storage dirt-cheap these days, why isn't the public sector jumping at the opportunity to store everything and anything? Jim Burns, CIO of Alabama, said cheap storage comes with its own problems.
"It's a bad idea to offer unlimited storage because management costs skyrocket and backup and retrieval becomes unmanageable," Burns said. "Cost of storage media, such as disk, is a very small part of the overall costs associated with an enterprise storage solution. The industry agrees that disk is cheap, but for every dollar spent on disk, it takes about $5 to $7 to manage it."
The challenge isn't storing massive amounts of data -- the real problem lies in managing that data. IT experts and analysts point to a number of information-management issues facing government agencies: How is data secured, backed up and retrieved? How often are tape backups corrupted? How much staff time is spent on manual restores? How do you access and verify the data?
Duplicate data is a huge issue for the public and private sectors. For example, both human resources and payroll might utilize a particular document at the same time, yet the legal requirements for data retention of that particular document may be vastly different based on the department and the governing entity.
Even worse, individual users may keep multiple copies of a particular file, perhaps in various stages of development. These files are shared with other staff via e-mail, which in turn also automatically saves each version of the file. Before you know it, there can be dozens, if not hundreds, of copies of the exact same file being backed up every night. Multiply this process across hundreds or thousands of employees and files, and you get the idea. Each file's footprint may be small, but when replicated and spread across the spectrum of daily business, the issue of duplicate data can become a serious problem.
E-mail is another tricky area. Can an agency realistically archive all e-mails? Should it? If so, backups may include huge amounts of spam that has slipped through firewall precautions, newsletters, personal dialogs and files, etc. Are policies and procedures in place requiring users to self-manage their e-mail archives, or is that an agency-enforced process?
A wide variety exists in government processes regarding e-mail limits and management. On one end of the spectrum, Stanislaus County, Calif., doesn't impose size limits on e-mail folders at all; instead, a retention time of 45 days is implemented. Any items in the e-mail system after 45 days are automatically deleted. If a longer retention time is required, users are expected to archive those e-mails on other file servers.
At the opposing end of the spectrum, Alabama offers four levels of personal mailboxes at different costs, based strictly on physical size. The largest mailbox offered is 2 GB. Burns stresses that e-mail is a messaging system and not a file-storage system. The responsibility of deciding which e-mails need to be archived on a different network file storage location falls on the particular agency and user, along with help