API Reference

The API reference is autogenerated directly from docstrings in the source code using sphinx.

The omfiles package provides access to the following classes and modules:

omfiles

Provides classes and utilities for reading, writing, and manipulating OM files.

class omfiles.OmFileReader(source)

Bases: object

An OmFileReader class for reading .om files synchronously.

An OmFileReader object can represent a multidimensional array variable, a scalar variable (an attribute), or a group. An OmFileReader can have an arbitrary number of child readers, each representing a child node in a tree-hierarchy. Supports reading from local files via memory mapping or from remote files through fsspec compatibility.

Variables in OM-Files do not have named dimensions! That means you have to know what the dimensions represent in advance or you need to explicitly encode them as some kind of attribute.

__enter__()

Enter a context manager block.

Returns:: Self for use in context manager.
Return type:: OmFileReader

__exit__(_exc_type=None, _exc_value=None, _traceback=None)

Exit a context manager block, closing the reader.

Parameters:

_exc_type (type, optional) – The exception type, if an exception was raised.
_exc_value (Exception, optional) – The exception value, if an exception was raised.
_traceback (traceback, optional) – The traceback, if an exception was raised.

Returns:

False (exceptions are not suppressed).

Return type:

bool

__getitem__(key, /): Return self[key].

__module__ = 'omfiles.omfiles'

classmethod __new__(*args, **kwargs)

chunks

The chunk shape of the variable.

Returns:: The chunk shape of the variable as a tuple.
Return type:: tuple[int, …]

close()

Close the reader and release resources.

This method releases all resources associated with the reader. After closing, any operation on the reader will raise a ValueError.

It is safe to call this method multiple times.

closed

Check if the reader is closed.

Returns:: True if the reader is closed, False otherwise.
Return type:: bool

compression_name

Get the compression type of the variable.

Returns:: Compression type of the variable.
Return type:: str

dtype

Get the data type of the data stored in the .om file.

Returns:: Data type of the data.
Return type:: numpy.dtype | type

static from_fsspec(fs_obj, path)

Create an OmFileReader from a fsspec fs object.

Parameters:

fs_obj (fsspec.spec.AbstractFileSystem) – A fsspec file system object which needs to have the methods cat_file and size.
path (str) – The path to the file within the file system.

Returns:

A new reader instance.

Return type:

OmFileReader

static from_path(file_path)

Create an OmFileReader from a file path.

Parameters:: file_path (str) – Path to the .om file to read.
Returns:: OmFileReader instance.
Return type:: OmFileReader

get_child_by_index(index)

Get a child reader at the specified index.

Returns:: Child reader at the specified index if exists.
Return type:: OmFileReader

get_child_by_name(name)

Get a child reader by name.

Returns:: Child reader with the specified name if exists.
Return type:: OmFileReader

is_array

Check if the variable is an array.

Returns:: True if the variable is an array, False otherwise.
Return type:: bool

is_group

Check if the variable is a group (a variable with data type None).

Returns:: True if the variable is a group, False otherwise.
Return type:: bool

is_scalar

Check if the variable is a scalar.

Returns:: True if the variable is a scalar, False otherwise.
Return type:: bool

name

Get the name of the variable stored in the .om file.

Returns:: Name of the variable or an empty string if not available.
Return type:: str

num_children

Number of children of the variable.

Returns:: Number of children of the variable.
Return type:: int

read_array(ranges)

Read data from the open variable.om file using numpy-style indexing.

Currently only slices with step 1 are supported.

The returned array will have singleton dimensions removed (squeezed). For example, if you index a 3D array with [1,:,2], the result will be a 1D array since dimensions 0 and 2 have size 1.

Parameters:: ranges (omfiles.types.BasicSelection) – Index expression to select data from the array. Supports basic numpy indexing.
Returns:: NDArray containing the requested data with squeezed singleton dimensions.
Return type:: numpy.typing.NDArray[numpy.int8 | numpy.int16 | numpy.int32 | numpy.int64 | numpy.uint8 | numpy.uint16 | numpy.uint32 | numpy.uint64 | numpy.float32 | numpy.float64]
Raises:: ValueError – If the requested ranges are invalid or if there’s an error reading the data.

read_scalar()

Read the scalar value of the variable.

Returns:: The scalar value as a Python object (str, int, or float).
Return type:: object
Raises:: ValueError – If the variable is not a scalar.

shape

The shape of the variable.

Returns:: The shape of the variable as a tuple.
Return type:: tuple[int, …]

class omfiles.OmFileReaderAsync

Bases: object

An OmFileReaderAsync class for reading .om files asynchronously.

An OmFileReaderAsync object can represent a multidimensional array variable, a scalar variable (an attribute), or a group. An OmFileReaderAsync can have an arbitrary number of child readers, each representing a child node in a tree-hierarchy. Supports reading from local files via memory mapping or from remote files through fsspec compatibility.

Variables in OM-Files do not have named dimensions! That means you have to know what the dimensions represent in advance or you need to explicitly encode them as some kind of attribute.

__enter__()

Enter a context manager block.

Returns:: Self for use in context manager.
Return type:: OmFileReaderAsync

__exit__(_exc_type=None, _exc_value=None, _traceback=None)

Exit a context manager block, closing the reader.

Parameters:

_exc_type (type, optional) – The exception type, if an exception was raised.
_exc_value (Exception, optional) – The exception value, if an exception was raised.
_traceback (traceback, optional) – The traceback, if an exception was raised.

Returns:

False (exceptions are not suppressed).

Return type:

bool

__module__ = 'omfiles.omfiles'

classmethod __new__(*args, **kwargs)

chunks

The chunk shape of the variable.

Returns:: The chunk shape of the variable as a tuple.
Return type:: tuple[int, …]

close()

Close the reader and release any resources.

Properly closes the underlying file resources.

Returns:: None
Raises:: RuntimeError – If the reader cannot be closed due to concurrent access.

closed

Check if the reader is closed.

Returns:: True if the reader is closed, False otherwise.
Return type:: bool

compression_name

Get the compression type of the variable.

Returns:: Compression type of the variable.
Return type:: str

dtype

Get the data type of the data stored in the .om file.

Returns:: Data type of the data.
Return type:: numpy.dtype | type

static from_fsspec(fs_obj, path)

Create a new async reader from an fsspec fs object.

Parameters:

fs_obj (fsspec.spec.AbstractFileSystem) – A fsspec file system object which needs to have the async methods _cat_file and _size.
path (str) – The path to the file within the file system.

Returns:

A new reader instance.

Return type:

OmFileReaderAsync

Raises:

TypeError – If the provided file object is not a valid fsspec file.
IOError – If there’s an error reading the file.

static from_path(file_path)

Create a new async reader from a local file path.

Parameters:: file_path (str) – Path to the OM file to read.
Returns:: A new reader instance.
Return type:: OmFileReaderAsync
Raises:: IOError – If the file cannot be opened or read.

get_child_by_index(index)

Get a child reader at the specified index.

Returns:: Child reader at the specified index if exists.
Return type:: OmFileReaderAsync

get_child_by_name(name)

Get a child reader by name.

Returns:: Child reader with the specified name if exists.
Return type:: OmFileReaderAsync

is_array

Check if the variable is an array.

Returns:: True if the variable is an array, False otherwise.
Return type:: bool

is_group

Check if the variable is a group (a variable with data type None).

Returns:: True if the variable is a group, False otherwise.
Return type:: bool

is_scalar

Check if the variable is a scalar.

Returns:: True if the variable is a scalar, False otherwise.
Return type:: bool

name

Get the name of the variable stored in the .om file.

Returns:: Name of the variable or an empty string if not available.
Return type:: str

num_children

Number of children of the variable.

Returns:: Number of children of the variable.
Return type:: int

read_array(ranges)

Read data from the array concurrently based on specified ranges.

Parameters:

ranges (omfiles.types.BasicSelection) – Index or slice object specifying the ranges to read.

Returns:

Array data of the appropriate numpy type.

Return type:

OmFileTypedArray

Raises:

ValueError – If the reader is closed.
TypeError – If the data type is not supported.

read_scalar()

Read the scalar value of the variable.

Returns:: The scalar value as a Python object (str, int, or float).
Return type:: object
Raises:: ValueError – If the variable is not a scalar.

shape

The shape of the variable.

Returns:: The shape of the variable as a tuple.
Return type:: tuple[int, …]

class omfiles.OmFileWriter(file_path)

Bases: object

A Python wrapper for the Rust OmFileWriter implementation.

__module__ = 'omfiles.omfiles'

classmethod __new__(*args, **kwargs)

static at_path(path)

Initialize an OmFileWriter to write to a file at the specified path.

Parameters:: path – Path where the .om file will be created
Returns:: A new writer instance
Return type:: OmFileWriter

close(root_variable)

Finalize and close the .om file by writing the trailer with the root variable.

Parameters:

root_variable (omfiles.OmVariable) – The OmVariable that serves as the root/entry point of the file hierarchy. All other variables should be accessible through this root variable.

Returns:

None on success.

Raises:

ValueError – If the writer has already been closed
RuntimeError – If a thread lock error occurs or if there’s an error writing to the file

closed: Check if the writer is closed.

static from_fsspec(fs_obj, path)

Create an OmFileWriter from a fsspec filesystem object.

Parameters:

fs_obj – A fsspec filesystem object that supports write operations
path – The path to the file within the file system

Returns:

A new writer instance

Return type:

OmFileWriter

write_array(chunks, scale_factor=1.0, add_offset=0.0, compression='pfor_delta_2d', name='data', children=[])

Write a numpy array to the .om file with specified chunking and scaling parameters.

scale_factor and add_offset are only respected and required for float32 and float64 data types. Recommended compression is “pfor_delta_2d” as it achieves best compression ratios (on spatio-temporally correlated data), but it will be lossy when applied to floating-point data types because of the scale-offset encoding applied to convert float values to integer values.

Parameters:

data – Input array to be written. Supported dtypes are: float32, float64, int8, uint8, int16, uint16, int32, uint32, int64, uint64,
chunks – Chunk sizes for each dimension of the array
scale_factor – Scale factor for data compression (default: 1.0)
add_offset – Offset value for data compression (default: 0.0)
compression – Compression algorithm to use (default: “pfor_delta_2d”) Supported values: “pfor_delta_2d”, “fpx_xor_2d”, “pfor_delta_2d_int16”, “pfor_delta_2d_int16_logarithmic”
name – Name of the variable to be written (default: “data”)
children – List of child variables (default: [])

Returns:

omfiles.OmVariable representing the written group in the file structure

Raises:

ValueError – If the data type is unsupported or if parameters are invalid

write_group(name, children)

Create a new group in the .om file.

This is essentially a variable with no data, which serves as a container for other variables.

Parameters:

name – Name of the group
children – List of child variables

Returns:

omfiles.OmVariable representing the written group in the file structure

Raises:

RuntimeError – If there’s an error writing to the file

write_scalar(name, children=None)

Write a scalar value to the .om file.

Parameters:

value – Scalar value to write. Supported types are: int8, int16, int32, int64, uint8, uint16, uint32, uint64, float32, float64, String
name – Name of the scalar variable
children – List of child variables (default: None)

Returns:

omfiles.OmVariable representing the written scalar in the file structure

Raises:

ValueError – If the value type is unsupported (e.g., booleans)
RuntimeError – If there’s an error writing to the file

class omfiles.OmVariable

Bases: object

Represents a variable in an OM file.

__module__ = 'omfiles.omfiles'

classmethod __new__(*args, **kwargs)

__repr__(): Return repr(self).

name: The name of the variable.

offset: The offset of the variable in the OM file.

size: The size of the variable in bytes in the OM file.

omfiles.types

Types used by omfiles.

omfiles.types.BasicSelection

either a single selector or a tuple of selectors (also used for BlockIndex).

Type:: A selection for an array

omfiles.types.BasicSelector

integer, slice, or ellipsis.

Type:: A single index selector for an array dimension

alias of int | slice | EllipsisType

omfiles.xarray

OmFileReader backend for Xarray.

class omfiles.xarray.OmBackendArray(reader)

Bases: BackendArray

OmBackendArray is an xarray backend implementation for the OmFileReader.

property dtype

property shape

class omfiles.xarray.OmDataStore(root_variable)

Bases: AbstractDataStore

close()

get_attrs()

get_variables()

root_variable: OmFileReader

variables_store: dict[str, OmVariable]

class omfiles.xarray.OmXarrayEntrypoint

Bases: BackendEntrypoint

description: ClassVar[str] = 'Use .om files in Xarray'

guess_can_open(filename_or_obj): Backend open_dataset method used by Xarray in open_dataset().

open_dataset(filename_or_obj, *, drop_variables=None)

Backend open_dataset method used by Xarray in open_dataset().

Return type:: Dataset

url: ClassVar[str] = 'https://github.com/open-meteo/om-file-format/'