Skip to content

baggepinnen/DiskDataProviders.jl

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

58 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Build Status codecov

DiskDataProviders.jl

This package implements datastructures that are iterable and backed by a buffer that is fed by data from disk. The reading is done on a separate thread, so make sure Julia is started with at least two threads.

Intended usage: To buffer data reading from disk when training convolutional neural networks (1d or 2d) using Flux.jl. This allows the CPU to work with the disk and data while the GPU is working on the training. This package might be useful for other things as well.

For usage example, see the documentation
For lower-level buffered iterators, see LengthChannels.jl