CCES Unicamp

ChunkSplitters.jl: easy split the elements or indices of a collection into chunks.

https://juliafolds2.github.io/ChunkSplitters.jl/stable/#ChunkSplitters.jl

ChunkSplitters.jl makes it easy to split the elements or indices of a collection into chunks:

julia> using ChunkSplitters

julia> x = [1.2, 3.4, 5.6, 7.8, 9.1, 10.11, 11.12];

julia> for inds in index_chunks(x; n=3)
           @show inds
       end
inds = 1:3
inds = 4:5
inds = 6:7

julia> for c in chunks(x; n=3)
           @show c
       end
c = [1.2, 3.4, 5.6]
c = [7.8, 9.1]
c = [10.11, 11.12]

This can be useful in many areas, one of which is multithreading, where we can use chunking to control the number of spawned tasks:

function parallel_sum(x; ntasks=nthreads())
    tasks = map(chunks(x; n=ntasks)) do chunk_of_x
        @spawn sum(chunk_of_x)
    end
    return sum(fetch, tasks)
end

Working with chunks and their respective indices also improves thread-safety compared to a naive parallelisation approach based on threadid(). 

 

Related posts

CCES researcher elected for the board of the main computing association in the world

escience

Transition, intermittency and phase interference effects in airfoil secondary tones and acoustic feedback loop

cces cces

Combining Compression and Prefetching to Improve Checkpointing for Inverse Seismic Problems in GPUs

cces cces