Development of a Custom Environmental Data Management System for the National Archives and Records Administration

Custom Internet-based Environmental Data Management System for the National Archives and Records AdministrationThe Image Permanence Institute recently developed and implemented an Environmental Data Management System for the National Archives and Records Administration (NARA). This system, based on IPI’s eClimateNotebook™ and incorporating custom features and functions designed specifically for NARA, is currently in use by all of their facilities.

Stage One

In early 2007, close to sixty NARA regional facilities and presidential libraries were using IPI’s original PEM dataloggers and Climate Notebook desktop software. NARA began using IPI’s environmental management tools in 2000 and had over 500 PEMs in the field and over 600 monitored locations. At this point, the tools available from IPI were not meeting their needs. Climate Notebook files could not be used on a shared drive, requiring extensive emailing of data files or physical mailing of files on a disk in order to get problems solved. Lack of access to current data from each location also made it difficult to create quarterly reports.

Linda Blaser, former program officer for NARA regional record centers, contacted IPI as her responsibility for management of data collection and reporting became increasingly difficult. Linda cited issues related to NARA IT security measures, changing IT personnel, aging monitoring equipment, archival staff changes and reassignment of duties, computer upgrades and related concerns. Managing problems occurring at local sites from the central preservation office, particularly when IT staff was not available at the local level, was too difficult and time consuming. Security blocked email files of data, and mailing discs of data to illustrate problems took much too long. With a growing number of dataloggers in the field, the workload was overwhelming. Linda was interested in IPI’s help in resolving these issues through improvements in the technology currently available.

At the same time IPI was working on a web application based on a prototype recently designed for the National Museums of Denmark. IPI had applied for and received a National Leadership Grant from the Institute of Museum and Library Services (IMLS) to further develop and field test a web-based data management system, a two year project that would be completed by October 2009. The goal was to develop a program to efficiently move large volumes of environmental data directly to the web for automated analysis and reporting. The project included a pilot group of ten museums and five libraries to test and contribute to the design of the application. Unfortunately the final product wouldn’t be ready for a few years.

NARA central office staff was interested in the concept however, and wanted to determine if the system could work for them and save a great deal of staff time and effort. After some discussion, IPI suggested a project that would look into the feasibility of designing and implementing an intranet based environmental management system that could work with NARA’s recently developed Holding Maintenance System[1]. In the fall of 2007 IPI was granted a one-year contract to provide a cost/benefit analysis for NARA, outline training requirements, and address data management concerns such as migration of existing data to the web and long-term data storage.

NARA needed a system that would permit staff to securely store, manage, and remotely access data documenting the preservation impact of temperature and relative humidity conditions across NARA’s facilities as well as historical data documenting water events, fires, and pest infestations. It should allow staff in various locations to consult with one another in “real-time” regarding environmental conditions at NARA’s facilities, and address the limitations of the current system based on PEMs, SRAM cards, card readers and Climate Notebook desktop software.

Incorporating program development already underway, IPI proposed a system that included a hierarchical location tree, default lists and standardized terminology developed specifically for NARA, analysis based on compliance with existing NARA standards, and secure data backup systems. The 2008 proposal estimated the time commitment required from NARA staff and suggested a three year development and implementation plan. Although the concept was well-received and promoted to the NARA administration, funding was not available at the time.

Stage Two

In February 2011, NARA requested a new cost estimate for the project. Since the last estimate IPI had developed two web-based data management systems – MyClimateData and PEMdata. The MyClimateData site evolved from IPI’s partnership with the National Museum of Denmark, work with partners in the IMLS-funded National Leadership Grant, and data management projects with the Library of Congress. MyClimateData was a robust tool, particularly for large institutions, because it centralized data within a searchable database and made it easier for staff throughout an institution to access up-to-date and complete environmental data. PEMdata had developed to accompany the 2007 release of the PEM2® datalogger. This free, streamlined website gave users a simple way to store and graph data and to understand the effect of the environment on their collections.  By 2011 IPI was deeply involved in developing eClimateNotebook™, a web application that would assimilate the strengths of Climate Notebook, MyClimateData and PEMdata into a single, unified platform.

In May 2011, IPI proposed a parallel data management system for NARA. Using this system, NARA central administration in College Park, Maryland would have access to accounts in use at all other NARA sites in the US. Individual sites could enter and manage their own environmental data and collection information but would not have access to any other site. Each site could upload their existing data and manage their own naming schemes. Specialized analysis features would be based on NARA’s environmental standards and compliance issues.

The proposal outlined a five year system license and support contract with one year of development, customization and training, and four years of ongoing operation and support, along with a plan to replace all existing original PEM loggers with PEM2s. The proposal was accepted and work began in September of the same year.

Stage Three

During the first year, IPI designed a data management website for NARA that included several customized elements such as front page and report design, custom reports, and statistics based on compliance with several NARA standards for storage environments. IPI provided a unique domain name, secure data storage, and a dedicated server with daily backups. A test site populated with data from NARA facilities was made available for review.

Training took place at the NARA central office in September 2012 and was attended by preservation staff both in person and via webcast. The new system was launched for all NARA facilities shortly after.

The program developed by IPI allows NARA to expand its data management, organization, and analysis capabilities through a centralized and easily accessible system. The site is managed by a NARA site administrator and supports an unlimited number of NARA facilities and location datasets. This web-based platform allows universal access for NARA administrative staff to view data and generate reports across all NARA facilities, and allows general users to view data and generate reports for their own facility. NARA staff will continue to incorporate several years of data from hundreds of dataloggers installed at all NARA facilities into the primary site.



[1] The Holding Maintenance System was designed to permit NARA staff to record information relating to the physical holdings, their storage containers, and the facilities that house them.