Data Archiving: the nitty-gritty

The joint data archiving policy has been in place for a year now, and it’s been very encouraging to see how receptive everyone has been to this change. One question we get asked a lot is ‘which data should I make available?’, so I thought it would be helpful to put together a checklist for the broad types of data we receive at Mol Ecol.

The theme throughout is that you generally want to be preparing a tab delimited text file with one row per individual/sample, and then giving all the corresponding data as columns (where it came from, any phenotypic data, the genotype data). The other data that went into the analyses that can’t be given on an individual by individual basis (e.g. environmental data for each site) can be provided in separate files.

Next, we really encourage people to upload all the input files used in the data analysis- the final data set can be many steps removed from the raw data (especially DNA sequence data), and providing these files makes your paper that much more useful. (This in turn will makes it more likely that other people will ask to include it in their research, which equals more citations and more publications for you.) In a similar vein, including all your analysis scripts is also encouraged, as these allow others to see exactly what you’ve done without spending a huge amount of time trying to explain it in words in the methods; the same applies to parameter input files for e.g. Structure and IMa2.

Lastly, a readme file is a really useful thing to include with your archived data- this can explain things like the meaning of column headers in the data file, units, precise localities, indicators for missing data, codes for categorical variables etc (see Whitlock 2010 for more on this). You don’t need to specify the location of the readme file in your data accessibility paragraph (it should be in the same place as the data!), but I’ve included it in the lists below as a reminder…

I’ve organized the lists below in terms of the different data types, with the suggested contents of the files themselves in square brackets. Each entry of the list should appear as an element of the Data Accessibility paragraph (with the exception of the readme file).

a) DNA sequence data:

• Genbank/EMBL/DDBJ accession numbers for all unique DNA sequences
  
• a file with data on each individual [indiv ID, sampling location, accession number for sequence at gene 1, gene 2 etc]
  
• readme.txt file explaining the contents of the above file

OR

• a POPSET accession number from NCBI- this should have one accession number for each individual in the sample, and the accession should explicitly state which individual and population of origin the sequence came from.

optional:
- TreeBASE study number
- DNA sequence alignments (although these are really useful)
- Analysis input files for programs like IMa2, as there’s a lot of steps between raw sequence and phased and aligned haplotypes
- Any scripts used to analyse the data (please check the copyright status before making these available)

b) microsatellite data:

• a file with data on each individual [indiv ID, sampling location, genotype at locus 1, genotype at locus 2 etc]
• readme.txt file explaining the contents of the above file
• Analysis input files for e.g. Structure (including parameter files)
• Any scripts used to analyse the data (please check the copyright status before making these available)

optional:
- Trace files and an accompanying readme explaining the scoring decisions (these would need to go onto Dryad)

c) SNP data:

• a file with data on each individual [indiv ID, sampling location, genotype at locus 1, genotype at locus 2 etc]
• readme.txt file explaining the contents of the above file
• TreeBASE study number, if applicable (this is useful but not essential).
• Analysis input files for e.g. Structure (including parameter files)
• Any scripts used to analyse the data (please check the copyright status before making these available)

d) microarray data:

• all the information required under the MIAME 2.0 protocols should be available on a public archive.
• Analysis input files
• Any scripts used to analyse the data (please check the copyright status before making these available)

e) Next Generation Sequencing data:

The scale and complexity of NGS data means that extra care is needed when archiving it for future generations of researchers. In particular, the steps taken to convert the raw reads into the final dataset may not be fully reproducible even with exactly the same methods, and hence it makes sense to archive the data at several stages of its analysis. This area is worthy of an entire post of its own, but I’ve tried to summarise the main types of data worth archiving below (thanks to Nolan Kane and Sebastien Renaut for advice on this).

• raw read data from next generation sequencing is an important resource, but it’s hard to tell whether public archives will continue to accept this type of data. NCBI’s Short Read Archive has said that it will stop accepting new data at the start of September 2011, whereas the ENA’s Sequence Read Archive currently says that it will continue taking NGS data (see here). If there is no publicly-funded place to store raw read data they can kept on institutional or personal servers; in this case archiving is desirable but not essential under the policy.

• the sequence alignment (e.g. .sam/.bam/.ace file) should be publicly archived whenever possible, although this file can be very large (e.g. if the reads are aligned to an existing genome). The reference genome/transcriptome/gene of interest will also need to be available.

• at the very least, the final data file that the analyses were based on should be publicly available (eg. SNP calls, indel calls, expression values for each sample).

• the scripts used to generate the final dataset from the raw reads (very useful but not absolutely necessary)

f) other data types:

• information about sampling locations and their related variables (e.g. site level environmental data) should be made available in a separate file