Chapter 7: Data Preservation and Archiving

7.1 The Preservation Imperative

Research data must remain accessible not just for years, but for decades. Longitudinal studies may span generations. Medical discoveries require validation and replication over time. Regulatory requirements mandate data retention long after studies conclude. Yet digital preservation is fraught with challenges that threaten long-term access.

Without deliberate preservation planning, research data becomes inaccessible as formats become obsolete, storage media degrade, documentation is lost, and the researchers who understood the data move on. The scientific record becomes incomplete, irreproducible, and ultimately untrustworthy.

7.2 Preservation Challenges

Digital preservation faces multiple threats to long-term accessibility:

Preservation Risks

Risk Category	Description	Mitigation
Format Obsolescence	Software to read format unavailable	Open formats, migration planning
Media Degradation	Storage media fails over time	Refreshment, redundancy
Hardware Obsolescence	No devices to read old media	Migration before obsolescence
Documentation Loss	Context needed to interpret data lost	Comprehensive, embedded documentation
Organizational Change	Custodian organization ceases	Trusted repositories, succession plans
Disaster	Loss through disaster or attack	Geographic redundancy, backup

7.3 Preservation-Friendly Formats

Format selection significantly impacts long-term accessibility:

Format Recommendations

Data Type	Recommended Formats	Avoid
Tabular Data	CSV, TSV, Parquet	Excel (.xlsx), proprietary databases
Text	Plain text, PDF/A, XML	Word (.docx), proprietary formats
Images	TIFF, PNG, JPEG 2000	Proprietary camera RAW
Genomics	FASTQ, VCF, BAM/CRAM	Instrument-specific formats
Medical Imaging	DICOM, NIfTI	Proprietary viewer formats
Statistical	CSV + code, Parquet	.sav, .sas7bdat without export

7.4 Preservation Strategies

Multiple strategies ensure long-term accessibility:

OAIS Reference Model

The Open Archival Information System (OAIS) provides a framework for digital preservation:

• Ingest: Accept data into archive with validation
• Archival Storage: Long-term storage with integrity monitoring
• Data Management: Maintain metadata and administrative information
• Preservation Planning: Monitor technology, plan migrations
• Access: Provide data to users on request
• Administration: Manage overall archive operations

Preservation Actions

Strategy	Description	When to Use
Bit Preservation	Maintain exact byte sequence	Baseline for all preservation
Refreshment	Copy to new storage media	Before media degradation
Migration	Convert to newer format	Before format obsolescence
Emulation	Recreate original environment	Complex software dependencies
Normalization	Convert to preservation format on ingest	Standardize diverse inputs

7.5 Trusted Digital Repositories

Repositories certified for long-term preservation provide essential infrastructure:

Repository Certification

• CoreTrustSeal: Core certification for data repositories
• ISO 16363: Comprehensive audit criteria for digital repositories
• NDSA Levels: Graduated preservation maturity model

7.6 Retention Requirements

Different contexts impose different retention obligations:

Context	Typical Requirement	Authority
FDA Regulated	2 years after approval/termination	21 CFR Part 11, 312.62
NIH Funded	3 years after final report (minimum)	NIH Grants Policy
ICH-GCP	15 years or per regulatory	ICH E6(R2)
GDPR	Only as long as necessary	Article 5(1)(e)
Genomic Data	Indefinite for reference	Community practice

7.7 Sustainability Planning

Long-term preservation requires sustainable funding and organizational models:

Sustainability Considerations

• Funding Models: Institutional, grant-funded, fee-based, endowment
• Cost Factors: Storage, curation, format migration, access provision
• Succession Planning: Data disposition if repository closes
• Community Governance: Stakeholder involvement in repository direction