Chunks¶
Header file: <libs/zarr/chunks.hpp>
[source]
-
inline size_t vec_product(const std::vector<size_t> &vec)¶
Calculates the product of all elements in a vector of size_t numbers.
- Parameters:
vec – The vector of size_t numbers.
- Returns:
The product of all the elements in the vector.
-
inline size_t vec_product(const std::vector<size_t> &vec, const size_t aa)¶
Calculates the product of elements in a vector of size_t numbers starting from the aa’th index of the vector.
- Parameters:
vec – The vector of size_t numbers.
aa – The starting index from which to calculate the product.
- Returns:
The product of elements from the aa’th index in the vector.
-
class Chunks¶
A class template for managing and writing chunks of an array.
This class provides functionality for writing chunks of an array to a store.
Public Functions
-
inline Chunks(const std::vector<size_t> &chunkshape, const std::vector<size_t> &reduced_arrayshape)¶
Constructor for the Chunks class.
Initializes the Chunks with the provided chunk shape and shape of reduced array. Reduced array shape is the shape of the array excluding its outermost (0th) dimension. The chunkshape along all the inner dimensions must be factor of the reduced array shape (completely divisible) to ensure good chunking.
- Parameters:
chunkshape – The shape of chunks along each dimension.
reduced_arrayshape – The shape of the reduced array along each dimension.
-
inline std::vector<size_t> get_chunkshape() const¶
Gets the shape of a chunk.
- Returns:
A vector containing the shape (number of data elements) of a chunk along each dimension.
-
inline std::vector<size_t> get_reducedarray_nchunks() const¶
Gets the number of chunks of the reduced array.
- Returns:
A vector containing the number of chunks of an array along its dimensions except for its outermost one.
-
inline std::vector<size_t> get_reduced_arrayshape() const¶
Gets complete shape of the array excluding its outermost dimension.
- Returns:
A vector containing the shape (number of data elements) of the array when complete along all but its outermost dimension.
-
template<typename Store, typename T>
inline size_t write_chunk(Store &store, const std::string_view name, const size_t chunk_num, Buffer<T> &buffer) const¶ Writes a chunk to the store and increments the total number of chunks written.
This function writes the data held in a buffer in the specified store to a chunk identified by “chunk_label” of an array called “name” given the number of chunks of the array already existing. After writing the chunk, the total number of chunks is incremented.
- Template Parameters:
Store – The type of the store.
T – The type of the data elements stored in the buffer.
- Parameters:
store – Reference to the store where the chunk will be written.
name – Name of the array in the store where the chunk will be written.
chunk_num – The total number of chunks of the array already written.
buffer – The buffer containing the data to be written to the chunk.
- Returns:
The updated total number of chunks after writing.
-
template<typename Store, typename T>
inline size_t write_chunk(Store &store, const std::string_view name, const size_t chunk_num, const Buffer<T>::subviewh_buffer h_data_chunk) const¶ Writes a chunk to the store and increments the total number of chunks written.
This function writes the data stored in the Kokkos view (in host memory) in the specified store to a chunk identified by “chunk_label” of an array called “name” given the number of chunks of the array already existing. After writing the chunk, the total number of chunks is incremented.
- Template Parameters:
Store – The type of the store.
T – The type of the data elements stored in the buffer.
- Parameters:
store – Reference to the store where the chunk will be written.
name – Name of the array in the store where the chunk will be written.
chunk_num – The total number of chunks of the array already written.
h_data_chunk – The view containing the data in host memory to be written to the chunk.
- Returns:
The updated total number of chunks after writing.
-
template<typename Store, typename T>
inline size_t write_chunks(Store &store, const std::string_view name, const Buffer<T>::subviewh_buffer h_data, const size_t totnchunks, const size_t chunksize, const size_t nchunks) const¶ Writes multiple chunks to the store and increments the total number of chunks written.
This function writes “nchunks” whole number of chunks from the data stored in the Kokkos view (in host memory) in an array called “name” in the specified store given the total number of chunks of the array already existing.After writing all the chunks, the total number of chunks is updated accordingly.
TODO(all) parallelise this for loop for writing chunks
- Template Parameters:
Store – The type of the store.
T – The type of the data elements stored in the buffer.
- Parameters:
store – Reference to the store where the chunks will be written.
name – Name of the array in the store where the chunks will be written.
h_data – The view containing the data in host memory to be written to the chunks.
totnchunks – The total number of chunks of the array already written.
chunksize – The size of each chunk.
nchunks – The number of chunks to write.
- Returns:
The updated total number of chunks after writing.
Private Functions
-
inline std::string chunk_label(const size_t chunk_num) const¶
Create label for a chunk given current number of chunks written to array.
This function creates a vector of integers for the number of a chunk along each dimension of an array given the chunk is the n’th chunk to be written to the store (starting at n=0 and incrementing along the innermost dimensions first). The vector is then converted into a string which can be used to label the chunk.
- Parameters:
chunk_num – The number of the chunk to write to the array.
- Returns:
A string representing the label of the current chunk to write.
-
inline Chunks(const std::vector<size_t> &chunkshape, const std::vector<size_t> &reduced_arrayshape)¶