DIII-D data management

AbstractThe DIII-D tokamak at the DIII-D National Fusion Facility routinely acquires ∼500MB of raw data per pulse of the experiment through a centralized data management system. It is expected that in FY01 (October 2001 to September 2002), nearly 1TB of data will be acquired. In addition there are several diagnostics, which are not part of the centralized system, which acquire hundreds of MB of raw data per pulse. There is also a growing suite of codes running between pulses that produce analyzed data, which add ∼10MB per pulse with total disk usage of about 100GB. A relational database system has been introduced which further adds to the overall data load. In recent years there has been an order of magnitude increase in magnetic disk space devoted to raw data and a hierarchical storage management system was implemented to allow 24 h/day, 7 days/week unattended access to raw data. The management of all of the data is a significant and growing challenge as the quantities of both raw and analyzed data are expected to continue to increase in the future. This paper will examine the experiences of the approaches that have been taken in management of the data and plans for the continued growth of the data quantity.