API Reference
The API reference is autogenerated directly from docstrings in the source code using sphinx.
The omfiles package provides access to the following classes and modules:
omfiles
Provides classes and utilities for reading, writing, and manipulating OM files.
- class omfiles.OmFileReader(source)
Bases:
objectAn OmFileReader class for reading .om files synchronously.
An OmFileReader object can represent a multidimensional array variable, a scalar variable (an attribute), or a group. An OmFileReader can have an arbitrary number of child readers, each representing a child node in a tree-hierarchy. Supports reading from local files via memory mapping or from remote files through fsspec compatibility.
Variables in OM-Files do not have named dimensions! That means you have to know what the dimensions represent in advance or you need to explicitly encode them as some kind of attribute.
- __annotations__ = {}
- __enter__()
Enter a context manager block.
- Returns:
Self for use in context manager.
- Return type:
- __exit__(_exc_type=None, _exc_value=None, _traceback=None)
Exit a context manager block, closing the reader.
- Parameters:
- Returns:
False (exceptions are not suppressed).
- Return type:
- __getitem__(key, /)
Return self[key].
- __module__ = 'builtins'
- classmethod __new__(*args, **kwargs)
- chunks
The chunk shape of the variable.
- close()
Close the reader and release resources.
This method releases all resources associated with the reader. After closing, any operation on the reader will raise a ValueError.
It is safe to call this method multiple times.
- closed
Check if the reader is closed.
- Returns:
True if the reader is closed, False otherwise.
- Return type:
- compression_name
Get the compression type of the variable.
- Returns:
Compression type of the variable.
- Return type:
- dtype
Get the data type of the data stored in the .om file.
- Returns:
Data type of the data.
- Return type:
- static from_fsspec(fs_obj, path)
Create an OmFileReader from a fsspec fs object.
- Parameters:
fs_obj (fsspec.spec.AbstractFileSystem) – A fsspec file system object which needs to have the methods cat_file and size.
path (str) – The path to the file within the file system.
- Returns:
A new reader instance.
- Return type:
- static from_path(file_path)
Create an OmFileReader from a file path.
- Parameters:
file_path (str) – Path to the .om file to read.
- Returns:
OmFileReader instance.
- Return type:
- get_child_by_index(index)
Get a child reader at the specified index.
- Returns:
Child reader at the specified index if exists.
- Return type:
- get_child_by_name(name)
Get a child reader by name.
- Returns:
Child reader with the specified name if exists.
- Return type:
- is_array
Check if the variable is an array.
- Returns:
True if the variable is an array, False otherwise.
- Return type:
- is_group
Check if the variable is a group (a variable with data type None).
- Returns:
True if the variable is a group, False otherwise.
- Return type:
- is_scalar
Check if the variable is a scalar.
- Returns:
True if the variable is a scalar, False otherwise.
- Return type:
- name
Get the name of the variable stored in the .om file.
- Returns:
Name of the variable or an empty string if not available.
- Return type:
- num_children
Number of children of the variable.
- Returns:
Number of children of the variable.
- Return type:
- read_array(ranges)
Read data from the open variable.om file using numpy-style indexing.
Currently only slices with step 1 are supported.
Follows NumPy indexing semantics: - Integer indices remove that dimension - Slice indices (even of length 1) preserve the dimension
- Parameters:
ranges (
omfiles.types.BasicSelection) – Index expression to select data from the array. Supports basic numpy indexing.- Returns:
NDArray containing the requested data with squeezed singleton dimensions.
- Return type:
numpy.typing.NDArray[numpy.int8 | numpy.int16 | numpy.int32 | numpy.int64 | numpy.uint8 | numpy.uint16 | numpy.uint32 | numpy.uint64 | numpy.float32 | numpy.float64]
- Raises:
ValueError – If the requested ranges are invalid or if there’s an error reading the data.
- read_scalar()
Read the scalar value of the variable.
- Returns:
The scalar value as a numpy scalar or a Python string.
- Return type:
- Raises:
ValueError – If the variable is not a scalar.
- class omfiles.OmFileReaderAsync
Bases:
objectAn OmFileReaderAsync class for reading .om files asynchronously.
An OmFileReaderAsync object can represent a multidimensional array variable, a scalar variable (an attribute), or a group. An OmFileReaderAsync can have an arbitrary number of child readers, each representing a child node in a tree-hierarchy. Supports reading from local files via memory mapping or from remote files through fsspec compatibility.
Variables in OM-Files do not have named dimensions! That means you have to know what the dimensions represent in advance or you need to explicitly encode them as some kind of attribute.
- __enter__()
Enter a context manager block.
- Returns:
Self for use in context manager.
- Return type:
- __exit__(_exc_type=None, _exc_value=None, _traceback=None)
Exit a context manager block, closing the reader.
- Parameters:
- Returns:
False (exceptions are not suppressed).
- Return type:
- __module__ = 'builtins'
- chunks
The chunk shape of the variable.
- close()
Close the reader and release any resources.
Properly closes the underlying file resources.
- Returns:
None
- Raises:
RuntimeError – If the reader cannot be closed due to concurrent access.
- closed
Check if the reader is closed.
- Returns:
True if the reader is closed, False otherwise.
- Return type:
- compression_name
Get the compression type of the variable.
- Returns:
Compression type of the variable.
- Return type:
- dtype
Get the data type of the data stored in the .om file.
- Returns:
Data type of the data.
- Return type:
- static from_fsspec(fs_obj, path)
Create a new async reader from an fsspec fs object.
- Parameters:
fs_obj (fsspec.spec.AbstractFileSystem) – A fsspec file system object which needs to have the async methods _cat_file and _size.
path (str) – The path to the file within the file system.
- Returns:
A new reader instance.
- Return type:
- Raises:
- static from_path(file_path)
Create a new async reader from a local file path.
- get_child_by_index(index)
Get a child reader at the specified index.
- Returns:
Child reader at the specified index if exists.
- Return type:
- get_child_by_name(name)
Get a child reader by name.
- Returns:
Child reader with the specified name if exists.
- Return type:
- is_array
Check if the variable is an array.
- Returns:
True if the variable is an array, False otherwise.
- Return type:
- is_group
Check if the variable is a group (a variable with data type None).
- Returns:
True if the variable is a group, False otherwise.
- Return type:
- is_scalar
Check if the variable is a scalar.
- Returns:
True if the variable is a scalar, False otherwise.
- Return type:
- name
Get the name of the variable stored in the .om file.
- Returns:
Name of the variable or an empty string if not available.
- Return type:
- num_children
Number of children of the variable.
- Returns:
Number of children of the variable.
- Return type:
- read_array(ranges)
Read data from the array concurrently based on specified ranges.
- Parameters:
ranges (
omfiles.types.BasicSelection) – Index or slice object specifying the ranges to read.- Returns:
Array data of the appropriate numpy type.
- Return type:
OmFileTypedArray
- Raises:
ValueError – If the reader is closed.
TypeError – If the data type is not supported.
- read_scalar()
Read the scalar value of the variable.
- Returns:
The scalar value as a numpy scalar or a Python string.
- Return type:
- Raises:
ValueError – If the variable is not a scalar.
- class omfiles.OmFileWriter(file_path)
Bases:
objectA Python wrapper for the Rust OmFileWriter implementation.
- __annotations__ = {}
- __module__ = 'builtins'
- classmethod __new__(*args, **kwargs)
- static at_path(path)
Initialize an OmFileWriter to write to a file at the specified path.
- Parameters:
path – Path where the .om file will be created
- Returns:
A new writer instance
- Return type:
- close(root_variable)
Finalize and close the .om file by writing the trailer with the root variable.
- Parameters:
root_variable (
omfiles.OmVariable) – The OmVariable that serves as the root/entry point of the file hierarchy. All other variables should be accessible through this root variable.- Returns:
None on success.
- Raises:
ValueError – If the writer has already been closed
RuntimeError – If a thread lock error occurs or if there’s an error writing to the file
- closed
Check if the writer is closed.
- static from_fsspec(fs_obj, path)
Create an OmFileWriter from a fsspec filesystem object.
- Parameters:
fs_obj – A fsspec filesystem object that supports write operations
path – The path to the file within the file system
- Returns:
A new writer instance
- Return type:
- write_array(chunks, scale_factor=1.0, add_offset=0.0, compression='pfor_delta_2d', name='data', children=[])
Write a numpy array to the .om file with specified chunking and scaling parameters.
scale_factorandadd_offsetare only respected and required for float32 and float64 data types. Recommended compression is “pfor_delta_2d” as it achieves best compression ratios (on spatio-temporally correlated data), but it will be lossy when applied to floating-point data types because of the scale-offset encoding applied to convert float values to integer values.- Parameters:
data – Input array to be written. Supported dtypes are: float32, float64, int8, uint8, int16, uint16, int32, uint32, int64, uint64,
chunks – Chunk sizes for each dimension of the array
scale_factor – Scale factor for data compression (default: 1.0)
add_offset – Offset value for data compression (default: 0.0)
compression – Compression algorithm to use (default: “pfor_delta_2d”) Supported values: “pfor_delta_2d”, “fpx_xor_2d”, “pfor_delta_2d_int16”, “pfor_delta_2d_int16_logarithmic”
name – Name of the variable to be written (default: “data”)
children – List of child variables (default: [])
- Returns:
omfiles.OmVariablerepresenting the written group in the file structure- Raises:
ValueError – If the data type is unsupported or if parameters are invalid
- write_array_streaming(chunks, chunk_iterator, dtype, scale_factor=1.0, add_offset=0.0, compression='pfor_delta_2d', name='data', children=[])
Write an array to the .om file by streaming chunks from a Python iterator.
This method is designed for writing large arrays that do not fit in memory. Instead of providing the full array, you provide the full array dimensions and an iterator that yields numpy array chunks.
Chunks MUST be yielded in row-major order (C-order) of the chunk grid. Each chunk’s shape determines how many internal file chunks it covers.
- Parameters:
dimensions – Shape of the full array (e.g., [1000, 2000])
chunks – Chunk sizes for each dimension (e.g., [100, 200])
chunk_iterator – Python iterable yielding numpy arrays, one per chunk region
dtype – Numpy dtype of the array (e.g., np.dtype(np.float32))
scale_factor – Scale factor for data compression (default: 1.0)
add_offset – Offset value for data compression (default: 0.0)
compression – Compression algorithm to use (default: “pfor_delta_2d”)
name – Name of the variable (default: “data”)
children – List of child variables (default: [])
- Returns:
omfiles.OmVariablerepresenting the written array in the file structure- Raises:
ValueError – If the dtype is unsupported or parameters are invalid
RuntimeError – If there’s an error during compression or I/O
- write_group(name, children)
Create a new group in the .om file.
This is essentially a variable with no data, which serves as a container for other variables.
- Parameters:
name – Name of the group
children – List of child variables
- Returns:
omfiles.OmVariablerepresenting the written group in the file structure- Raises:
RuntimeError – If there’s an error writing to the file
- write_scalar(name, children=None)
Write a scalar value to the .om file.
- Parameters:
value – Scalar value to write. Supported types are: int8, int16, int32, int64, uint8, uint16, uint32, uint64, float32, float64, String
name – Name of the scalar variable
children – List of child variables (default: None)
- Returns:
omfiles.OmVariablerepresenting the written scalar in the file structure- Raises:
ValueError – If the value type is unsupported (e.g., booleans)
RuntimeError – If there’s an error writing to the file
omfiles.types
Types used throughout the library.
- class omfiles.types.LatLon(lat: float, lon: float)
Bases:
NamedTupleRepresents a latitude and longitude pair.
omfiles.xarray
OmFileReader backend for Xarray.
- class omfiles.xarray.OmBackendArray(reader)
Bases:
BackendArrayOmBackendArray is an xarray backend implementation for the OmFileReader.
- property dtype
- property shape
- class omfiles.xarray.OmDataStore(root_variable)
Bases:
AbstractDataStore- close()
- get_attrs()
- get_variables()
- root_variable: OmFileReader
- variables_store: dict[str, OmVariable]
- class omfiles.xarray.OmXarrayEntrypoint
Bases:
BackendEntrypoint- guess_can_open(filename_or_obj)
Backend open_dataset method used by Xarray in
open_dataset().
- open_dataset(filename_or_obj, *, drop_variables=None)
Backend open_dataset method used by Xarray in
open_dataset().- Return type: