API Reference

The API reference is autogenerated directly from docstrings in the source code using sphinx.

The omfiles package provides access to the following classes and modules:

omfiles

Provides classes and utilities for reading, writing, and manipulating OM files.

class omfiles.OmFileReader(source)

Bases: object

An OmFileReader class for reading .om files synchronously.

An OmFileReader object can represent a multidimensional array variable, a scalar variable (an attribute), or a group. An OmFileReader can have an arbitrary number of child readers, each representing a child node in a tree-hierarchy. Supports reading from local files via memory mapping or from remote files through fsspec compatibility.

Variables in OM-Files do not have named dimensions! That means you have to know what the dimensions represent in advance or you need to explicitly encode them as some kind of attribute.

__annotations__ = {}

__enter__()

Enter a context manager block.

Returns:: Self for use in context manager.
Return type:: OmFileReader

__exit__(_exc_type=None, _exc_value=None, _traceback=None)

Exit a context manager block, closing the reader.

Parameters:

_exc_type (type, optional) – The exception type, if an exception was raised.
_exc_value (Exception, optional) – The exception value, if an exception was raised.
_traceback (traceback, optional) – The traceback, if an exception was raised.

Returns:

False (exceptions are not suppressed).

Return type:

bool

__getitem__(key, /): Return self[key].

__module__ = 'builtins'

classmethod __new__(*args, **kwargs)

add_offset

Get the add offset of the variable.

Returns:: Add offset stored in the array metadata.
Return type:: float

chunks

The chunk shape of the variable.

Returns:: The chunk shape of the variable as a tuple.
Return type:: tuple[int, …]

close()

Close the reader and release resources.

This method releases all resources associated with the reader. After closing, any operation on the reader will raise a ValueError.

It is safe to call this method multiple times.

closed

Check if the reader is closed.

Returns:: True if the reader is closed, False otherwise.
Return type:: bool

compression_name

Get the compression type of the variable.

Returns:: Compression type of the variable.
Return type:: str

dtype

Get the data type of the data stored in the .om file.

Returns:: Data type of the data.
Return type:: numpy.dtype | type

static from_fsspec(fs_obj, path)

Create an OmFileReader from a fsspec fs object.

Parameters:

fs_obj (fsspec.spec.AbstractFileSystem) – A fsspec file system object which needs to have the methods cat_file and size.
path (str) – The path to the file within the file system.

Returns:

A new reader instance.

Return type:

OmFileReader

static from_path(file_path)

Create an OmFileReader from a file path.

Parameters:: file_path (str) – Path to the .om file to read.
Returns:: OmFileReader instance.
Return type:: OmFileReader

get_child_by_index(index)

Get a child reader at the specified index.

Returns:: Child reader at the specified index if exists.
Return type:: OmFileReader

get_child_by_name(name)

Get a child reader by name.

Returns:: Child reader with the specified name if exists.
Return type:: OmFileReader

is_array

Check if the variable is an array.

Returns:: True if the variable is an array, False otherwise.
Return type:: bool

is_group

Check if the variable is a group (a variable with data type None).

Returns:: True if the variable is a group, False otherwise.
Return type:: bool

is_scalar

Check if the variable is a scalar.

Returns:: True if the variable is a scalar, False otherwise.
Return type:: bool

name

Get the name of the variable stored in the .om file.

Returns:: Name of the variable or an empty string if not available.
Return type:: str

num_children

Number of children of the variable.

Returns:: Number of children of the variable.
Return type:: int

read_array(ranges)

Read data from the open variable.om file using numpy-style indexing.

Currently only slices with step 1 are supported.

Follows NumPy indexing semantics: - Integer indices remove that dimension - Slice indices (even of length 1) preserve the dimension

Parameters:: ranges (omfiles.types.BasicSelection) – Index expression to select data from the array. Supports basic numpy indexing.
Returns:: NDArray containing the requested data with squeezed singleton dimensions.
Return type:: numpy.typing.NDArray[numpy.int8 | numpy.int16 | numpy.int32 | numpy.int64 | numpy.uint8 | numpy.uint16 | numpy.uint32 | numpy.uint64 | numpy.float32 | numpy.float64]
Raises:: ValueError – If the requested ranges are invalid or if there’s an error reading the data.

read_scalar()

Read the scalar value of the variable.

Returns:: The scalar value as a numpy scalar or a Python string.
Return type:: object
Raises:: ValueError – If the variable is not a scalar.

scale_factor

Get the scale factor of the variable.

Returns:: Scale factor stored in the array metadata.
Return type:: float

shape

The shape of the variable.

Returns:: The shape of the variable as a tuple.
Return type:: tuple[int, …]

class omfiles.OmFileReaderAsync

Bases: object

An OmFileReaderAsync class for reading .om files asynchronously.

An OmFileReaderAsync object can represent a multidimensional array variable, a scalar variable (an attribute), or a group. An OmFileReaderAsync can have an arbitrary number of child readers, each representing a child node in a tree-hierarchy. Supports reading from local files via memory mapping or from remote files through fsspec compatibility.

Variables in OM-Files do not have named dimensions! That means you have to know what the dimensions represent in advance or you need to explicitly encode them as some kind of attribute.

__enter__()

Enter a context manager block.

Returns:: Self for use in context manager.
Return type:: OmFileReaderAsync

__exit__(_exc_type=None, _exc_value=None, _traceback=None)

Exit a context manager block, closing the reader.

Parameters:

_exc_type (type, optional) – The exception type, if an exception was raised.
_exc_value (Exception, optional) – The exception value, if an exception was raised.
_traceback (traceback, optional) – The traceback, if an exception was raised.

Returns:

False (exceptions are not suppressed).

Return type:

bool

__module__ = 'builtins'

add_offset

Get the add offset of the variable.

Returns:: Add offset stored in the array metadata.
Return type:: float

chunks

The chunk shape of the variable.

Returns:: The chunk shape of the variable as a tuple.
Return type:: tuple[int, …]

close()

Close the reader and release any resources.

Properly closes the underlying file resources.

Returns:: None
Raises:: RuntimeError – If the reader cannot be closed due to concurrent access.

closed

Check if the reader is closed.

Returns:: True if the reader is closed, False otherwise.
Return type:: bool

compression_name

Get the compression type of the variable.

Returns:: Compression type of the variable.
Return type:: str

dtype

Get the data type of the data stored in the .om file.

Returns:: Data type of the data.
Return type:: numpy.dtype | type

static from_fsspec(fs_obj, path)

Create a new async reader from an fsspec fs object.

Parameters:

fs_obj (fsspec.spec.AbstractFileSystem) – A fsspec file system object which needs to have the async methods _cat_file and _size.
path (str) – The path to the file within the file system.

Returns:

A new reader instance.

Return type:

OmFileReaderAsync

Raises:

TypeError – If the provided file object is not a valid fsspec file.
IOError – If there’s an error reading the file.

static from_path(file_path)

Create a new async reader from a local file path.

Parameters:: file_path (str) – Path to the OM file to read.
Returns:: A new reader instance.
Return type:: OmFileReaderAsync
Raises:: IOError – If the file cannot be opened or read.

get_child_by_index(index)

Get a child reader at the specified index.

Returns:: Child reader at the specified index if exists.
Return type:: OmFileReaderAsync

get_child_by_name(name)

Get a child reader by name.

Returns:: Child reader with the specified name if exists.
Return type:: OmFileReaderAsync

is_array

Check if the variable is an array.

Returns:: True if the variable is an array, False otherwise.
Return type:: bool

is_group

Check if the variable is a group (a variable with data type None).

Returns:: True if the variable is a group, False otherwise.
Return type:: bool

is_scalar

Check if the variable is a scalar.

Returns:: True if the variable is a scalar, False otherwise.
Return type:: bool

name

Get the name of the variable stored in the .om file.

Returns:: Name of the variable or an empty string if not available.
Return type:: str

num_children

Number of children of the variable.

Returns:: Number of children of the variable.
Return type:: int

read_array(ranges)

Read data from the array concurrently based on specified ranges.

Parameters:

ranges (omfiles.types.BasicSelection) – Index or slice object specifying the ranges to read.

Returns:

Array data of the appropriate numpy type.

Return type:

OmFileTypedArray

Raises:

ValueError – If the reader is closed.
TypeError – If the data type is not supported.

read_scalar()

Read the scalar value of the variable.

Returns:: The scalar value as a numpy scalar or a Python string.
Return type:: object
Raises:: ValueError – If the variable is not a scalar.

scale_factor

Get the scale factor of the variable.

Returns:: Scale factor stored in the array metadata.
Return type:: float

shape

The shape of the variable.

Returns:: The shape of the variable as a tuple.
Return type:: tuple[int, …]

class omfiles.OmFileWriter(file_path, metadata_placement='tail')

Bases: object

A Python wrapper for the Rust OmFileWriter implementation.

__annotations__ = {}

__module__ = 'builtins'

classmethod __new__(*args, **kwargs)

static at_path(path, metadata_placement='tail')

Initialize an OmFileWriter to write to a file at the specified path.

Parameters:

path – Path where the .om file will be created
metadata_placement – (optional) Where to emit metadata; either “inline” or “tail” (see description in __new__).

Returns:

A new writer instance

Return type:

OmFileWriter

close(root_variable)

Finalize and close the .om file by writing the trailer with the resolved root variable.

In metadata_placement="tail" mode, metadata for arrays, scalars, and groups is resolved and emitted during close() so that metadata is consolidated near the end of the file. In metadata_placement="inline" mode, metadata is written immediately and child handles must already refer to resolved variables from the same writer.

Parameters:

root_variable (omfiles.OmWriterVariable) – The writer handle that serves as the root/entry point of the file hierarchy.

Returns:

None on success.

Raises:

ValueError – If the writer has already been closed or the handle belongs to a different writer.
RuntimeError – If there is an error resolving deferred metadata or writing the trailer.

closed: Check if the writer is closed.

discard()

Discard the writer without writing a trailer.

This intentionally abandons the current output and suppresses the warning that would otherwise be emitted when dropping an unclosed writer. It is primarily useful for cleanup in error paths where no valid root variable exists and close() cannot be called.

Returns:: None on success.
Raises:: ValueError – If the writer has already been closed or discarded.

static from_fsspec(fs_obj, path, metadata_placement='tail')

Create an OmFileWriter from a fsspec filesystem object.

Parameters:

fs_obj – A fsspec filesystem object that supports write operations
path – The path to the file within the file system
metadata_placement – (optional) Where to emit metadata; either “inline” or “tail” (see description in __new__).

Returns:

A new writer instance

Return type:

OmFileWriter

write_array(chunks, scale_factor=1.0, add_offset=0.0, compression='pfor_delta_2d', name='data', children=None)

Write a numpy array to the .om file with specified chunking and scaling parameters.

data must be a NumPy ndarray with one of the supported dtypes: float32, float64, int8, uint8, int16, uint16, int32, uint32, int64, or uint64.

scale_factor and add_offset are only respected for float32 and float64 data. Recommended compression is "pfor_delta_2d" as it often achieves the best compression ratios on spatio-temporally correlated data, but it will be lossy for floating-point arrays because scale/offset encoding is applied before integer compression.

Parameters:

data – NumPy ndarray to be written.
chunks – Chunk sizes for each dimension of the array.
scale_factor – Scale factor for data compression (default: 1.0).
add_offset – Offset value for data compression (default: 0.0).
compression – Compression algorithm to use (default: "pfor_delta_2d"). Supported values: "pfor_delta_2d", "fpx_xor_2d", "pfor_delta_2d_int16", "pfor_delta_2d_int16_logarithmic".
name – Name of the variable to be written (default: "data").
children – Child variables from the same writer (default: None).

write_array returns an omfiles.OmWriterVariable, which is a write-time handle used to build hierarchy relationships and to select the root variable passed to close(). It is not the same as omfiles.OmVariable, which represents already-materialized metadata when reading.

Returns:: omfiles.OmWriterVariable representing the written array in the file structure.
Raises:: ValueError – If the data type is unsupported or if parameters are invalid.

write_array_streaming(chunks, chunk_iterator, dtype, scale_factor=1.0, add_offset=0.0, compression='pfor_delta_2d', name='data', children=None)

Write an array to the .om file by streaming chunks from a Python iterator.

This method is designed for writing large arrays that do not fit in memory. Instead of providing the full array, you provide the full array dimensions and an iterator that yields NumPy array chunks.

Chunks MUST be yielded in row-major order (C-order) of the chunk grid. Each yielded chunk must be a NumPy ndarray with the declared dtype. Each chunk’s shape determines how many internal file chunks it covers.

Parameters:

dimensions – Shape of the full array (for example [1000, 2000]).
chunks – Chunk sizes for each dimension (for example [100, 200]).
chunk_iterator – Iterator yielding NumPy ndarrays, one per chunk region.
dtype – NumPy dtype of the streamed array (for example np.dtype(np.float32)).
scale_factor – Scale factor for data compression (default: 1.0).
add_offset – Offset value for data compression (default: 0.0).
compression – Compression algorithm to use (default: "pfor_delta_2d").
name – Name of the variable (default: "data").
children – Child variables from the same writer (default: None).

Returns:

omfiles.OmWriterVariable representing the written array in the file structure.

Raises:

ValueError – If the dtype is unsupported or parameters are invalid.
RuntimeError – If there is an error during compression or I/O.

write_group(name, children)

Create a new group in the .om file.

This is essentially a variable with no data, which serves as a container for other variables.

Parameters:

name – Name of the group.
children – Child variables from the same writer.

Returns:

omfiles.OmWriterVariable representing the written group in the file structure.

Raises:

ValueError – If a child handle belongs to a different writer.
RuntimeError – If inline metadata placement is requested before child metadata has been resolved, or if there is an I/O error.

write_scalar(name, children=None)

Write a scalar value to the .om file.

Supported values are Python str plus Python or NumPy scalar values of type int8, int16, int32, int64, uint8, uint16, uint32, uint64, float32, and float64.

Parameters:

value – Scalar value to write.
name – Name of the scalar variable.
children – Child variables from the same writer (default: None).

Child handles must come from the same writer. In metadata_placement="inline" mode they must already be resolved because metadata is emitted immediately. In metadata_placement="tail" mode they may be resolved later during close().

Returns:

omfiles.OmWriterVariable representing the written scalar in the file structure.

Raises:

ValueError – If the value type is unsupported (for example booleans).
RuntimeError – If there is an error writing to the file.

class omfiles.OmVariable

Bases: object

Represents a variable in an OM file.

__module__ = 'builtins'

__repr__(): Return repr(self).

name: The name of the variable.

offset: The offset of the variable in the OM file.

size: The size of the variable in bytes in the OM file.

class omfiles.OmWriterVariable

Bases: object

Represents a variable handle during writing.

__module__ = 'builtins'

name: The name of the variable.

omfiles.types

Types used throughout the library.

class omfiles.types.LatLon(lat: float, lon: float)

Bases: NamedTuple

Represents a latitude and longitude pair.

lat: float: Alias for field number 0

lon: float: Alias for field number 1

class omfiles.types.XYIndex(x: int, y: int)

Bases: NamedTuple

Represents a 2D index in a grid.

x: int: Alias for field number 0

y: int: Alias for field number 1

omfiles.xarray

OmFileReader backend for Xarray.

class omfiles.xarray.OmBackendArray(reader)

Bases: BackendArray

OmBackendArray is an xarray backend implementation for the OmFileReader.

property dtype

property shape

class omfiles.xarray.OmDataStore(root_variable)

Bases: AbstractDataStore

close()

get_attrs()

get_variables()

root_variable: OmFileReader

variables_store: dict[str, OmVariable]

class omfiles.xarray.OmXarrayEntrypoint

Bases: BackendEntrypoint

description: ClassVar[str] = 'Use .om files in Xarray'

guess_can_open(filename_or_obj): Backend open_dataset method used by Xarray in open_dataset().

open_dataset(filename_or_obj, *, drop_variables=None)

Backend open_dataset method used by Xarray in open_dataset().

Return type:: Dataset

url: ClassVar[str] = 'https://github.com/open-meteo/om-file-format/'