arrow
Return

DVID: Distributed Versioned Image-Oriented Dataservice

delete2019-02-05
delete16
delete
OA
AI
W
William T. Katz *
S
Stephen M. Plaza
DOI:10.3389/fncir.2019.00005delete
deleteOriginal
deleteShare
deleteSave
View PDF
Abstract

Abstract

En 中文
Open-source software development has skyrocketed in part due to community tools like github.com , which allows publication of code as well as the ability to create branches and push accepted modifications back to the original repository. As the number and size of EM-based datasets increases, the connectomics community faces similar issues when we publish snapshot data corresponding to a publication. Ideally, there would be a mechanism where remote collaborators could modify branches of the data and then flexibly reintegrate results via moderated acceptance of changes. The DVID system provides a web-based connectomics API and the first steps toward such a distributed versioning approach to EM-based connectomics datasets. Through its use as the central data resource for Janelia's FlyEM team, we have integrated the concepts of distributed versioning into reconstruction workflows, allowing support for proofreader training and segmentation experiments through branched, versioned data. DVID also supports persistence to a variety of storage systems from high-speed local SSDs to cloud-based object stores, which allows its deployment on laptops as well as large servers. The tailoring of the backend storage to each type of connectomics data leads to efficient storage and fast queries. DVID is freely available as open-source software with an increasing number of supported storage options.
Keywords:
versioning
connectomics
EM reconstruction
dataservice
big data
datastore
collaboration
distributed version control
AI Summary

AI Summary

Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.

Journal

Frontiers in Neural Circuits cover
Frontiers in Neural Circuits
IF:
3
Papers:
1.7K
Citations:
4.6K

Organization

H
Howard Hughes Medical Institute
Scholars:
1.2W
Papers: 7.8K
Citations: 6.0W
Cited Papers

Cited Papers

errShare
errSave
err
IF0
err
err0
PREAI
err
errShare
errSave
BigDataViewer: visualization and processing for large image data sets
err2015-05-28
err187
PREAI
errPietzsch, Tobias; Saalfeld, Stephan; Preibisch, Stephan; Tomancak, Pavel
errShare
errSave
A data model and data structures for moving objects databases
err2000-05-16
err0
PREAI
errLuca Forlizzi; Ralf Hartmut Güting; Enrico Nardelli; Markus Schneider
errShare
errSave
High-precision automated reconstruction of neurons with flood-filling networks
err2018-07-16
err217
errOAAI
errJanuszewski, Michal; Kornfeld, Joergen; Li, Peter H.; Pope, Art; Blakely, Tim; Lindsey, Larry; Maitin-Shepard, Jeremy; Tyka, Mike; Denk, Winfried; Jain, Viren
errShare
errSave
researcher View more