Practical Data Management - ACRL DCIG Webinar

download Practical Data Management - ACRL DCIG Webinar

of 56

Embed Size (px)

description

Slides from an ACRL DCIG webinar from 30 April 2014 discussing basic data management practices in file organization and naming, documentation, storage and backup, and making files usable in the future.

Transcript of Practical Data Management - ACRL DCIG Webinar

  • Practical Data Management ACRL DCIG Webinar 30 April 2014 Kristin Briney, PhD
  • andrius.v, https://www.flickr.com/photos/banditaz/6823875954 (CC BY-NC-SA)
  • Mr.TinDC, https://www.flickr.com/photos/mr_t_in_dc/5940438148 (CC BY-ND)
  • International Institute of Tropical Agriculture, https://www.flickr.com/photos/iita-media-library/8160877379 (CC BY-NC) Musgo Dumio_Momio, https://www.flickr.com/photos/30976576@N07/2903662286 (CC BY-NC-SA)
  • Jen Doty and Rob O'Reilly, Learning to Curate @ Emory. RDAP 2014
  • Data Management Basics Introduction to a few topics in data management File organization and naming Documentation Storage and backups Future file usability
  • Data Management Basics Introduction to a few topics in data management File organization and naming Documentation Storage and backups Future file usability Teach & Use
  • For each minute of planning at beginning of a project, you will save 10 minutes of headache later
  • FILE ORGANIZATION & NAMING Dan Zen, http://www.flickr.com/photos/danzen/5551831155/ (CC BY)
  • File Organization What? Keeping your files in order
  • File Organization Why? Easier to find and use data Tell, at a glance, what is done and what you have yet to do Can still find and use files in the future
  • File Organization When? Always! Get in the habit of putting files in the right place
  • File Organization How? Any system is better than none Make your system logical for your data 80/20 Rule Possibilities By project By analysis type By date
  • Example Thesis By chapter By file type (draft, figure, table, etc.) Data By researcher By analysis type By date
  • File Naming Conventions What? Consistent naming for files
  • http://retractionwatch.com/2014/01/07/doing-the-right-thing-authors-retract-brain-paper-with-systematic-human-error- in-coding/
  • File Naming Conventions Why? Make it easier to find files Avoid duplicates Make it easier to wrap up a project because you know which files belong to it
  • File Naming Conventions When? For a group of related files (3 to 1000+) May need different conventions for different groups
  • File Naming Conventions How? Pick what is most important for your name Date Site Analysis Sample Short description
  • File Naming Conventions How? Files should be named consistently Files names should be descriptive but short ( [ ] & $ Use the dating convention: YYYY-MM-DD
  • Example YYYYMMDD_site_sampleNum 20140422_PikeLake_03 20140424_EastLake_12 Analysis-sample-concentration UVVis-stilbene-10mM IR-benzene-pure
  • DOCUMENTATION Brady, https://www.flickr.com/photos/freddyfromutah/4424199420 (CC BY)
  • What would someone unfamiliar with your data need in order to find, evaluate, understand, and reuse them?
  • Documentation Why? Data without notes are unusable Because you wont remember everything For others who may need to use your files
  • Documentation When? Always Documentation needs will vary between files
  • Documentation How? Take good notes Metadata schemas http://www.dcc.ac.uk/resources/metadata-standards
  • Documentation How? Methods Protocols Code Survey Codebook Data dictionary Anything that lets someone reproduce your results
  • Documentation How? Templates Like structured metadata but easier Decide on a list of information before you collect data Make sure you record all necessary details Takes a few minutes upfront, easy to use later Print and post in prominent place or use as worksheet
  • Example I need to collect: Date Experiment Scan number Powers Wavelengths Concentration (or sample weight) Calibration factors, like timing and beam size
  • Documentation How? README.txt For digital information, address the questions What the heck am I looking at? Where do I find X? Use for project description in main folder Use to document conventions Use where ever you need extra clarity
  • Example Project-wide README.txt Basic project information Title Contributors Grant info etc. Contact information for at least one person All locations where data live, including backups
  • Example Talk_v1: rough outline of talk Talk_v2: draft of talk Talk_v3: updated 2014-01-15 after feedback Data folder contains all raw data files by date Analysis has analyzed data and plots Paper has drafts of article on this work
  • grover_net, http://www.flickr.com/photos/9246159@N06/599820538/ (CC BY-ND) STORAGE AND BACKUPS
  • Storage Why? Need good storage practices to prevent loss Keep data secure
  • Storage How? Library motto: Lots of Copies Keeps Stuff Safe! Rule of 3: 2 onsite, 1 offsite
  • Storage How? Computer External hard drive Shared drives/servers Tape backup Cloud storage* CDs/DVDs USB flash drive Erica Wheelan, https://www.flickr.com/photos/reinventedwheel/5985479866 (CC BY)
  • *Cloud Storage Read the Terms of Service! Eg. Google Drive When you upload or otherwise submit content to our Services, you give Google (and those we work with) a worldwide license to use, host, store, reproduce, modify, create derivative works (such as those resulting from translations, adaptations or other changes we make so that your content works better with our Services), communicate, publish, publicly perform, publicly display and distribute such content. The rights you grant in this license are for the limited purpose of operating, promoting, and improving our Services, and to develop new ones
  • Backups http://toystory.disney.com/
  • Backups How? Any backup is better than none Automatic backup is better than manual Your work is only as safe as your backup plan
  • Backups How? Check your backups Backups only as good as ability to recover data Test your backups periodically Preferably a fixed schedule 1 or 2 times a year may be enough Bigger/more complex backups should be checked more often Test your backup whenever you change things
  • Example I keep my data On my computer Backed up manually on shared drive I set a weekly reminder to do this Backed up automatically via SpiderOak cloud storage
  • FUTURE FILE USABILITY Ian, http://www.flickr.com/photos/ian-s/2152798588/ (CC BY-NC-ND)
  • Future File Usability What? Can you read your files from 10 years ago? Data needs to be Accessible Interpretable Readable
  • lukasbenc, https://www.flickr.com/photos/lukasbenc/3493808772 (CC BY-NC-SA)
  • Future File Usability Why? You may want to use the data in 5 years PI sometimes keeps data and notes Prep for data sharing Per OMB Circular A-110, must retain data at least 3 years post-project Better to retain for >6 years
  • Future File Usability When? When you wrap up a project (As you work on a project)
  • Future File Usability How? Back up written notes People always forget this one Difficult to interpret data without notes Options Digitally scan (recommended with digital data) Photocopies