From the research index
The paper comes from the research index. It wasn’t submitted here.
Journal article
Versioning Complex Data
Matt Macduff, Benno Lee, Sherman Beus
1 Jun 2014
Versioning Complex Data
Matt Macduff, Benno Lee, Sherman Beus · 1 Jun 2014
Abstract
The full text is at the publisher
You can read the abstract here, or open it at the publisher.
The Atmospheric Radiation Measurement (ARM) program is collecting and storing data daily since 1992. New and updated data are continuously added to the data store resulting in a history of data changes that are not readily apparent or easy to document. Users are notified when updated data is available but the reason for the update is difficult to find. The ability to assign a version to the data provides a simple handle for adding further documentation. The software versioning processes did not appear to be a good fit because of the volume and frequency of changes. But by assigning a version number to a set of files and establishing conditions for changing the sets, we created a model that appears to be manageable. As a test, we applied it to the historical ARM data set. Our results showed that with some domain specific adjustments we were able to automatically generate a reasonable set of versions for ARM data. Further, we believe this method could be applied in other complex data domains.
Identifiers
Discussion
Citations
Topics
Details
- Published
- 1 Jun 2014
- Issue
- Pages 788–791
- Language
- EN