# User-defined file readers

OVITO already comes with a collection of built-in file readers. But if you need to import a custom format, or some new file format not yet supported by the software, OVITO’s programming interface gives you the possibility to write your own file reader in the Python language.

To implement a custom file reader you need to define a new Python class, similar to the advanced programming interface for Python modifiers:

    from ovito.data import DataCollection
    from ovito.io import FileReaderInterface, import_file
    from typing import Callable, Any

    class MyFileReader(FileReaderInterface):

        @staticmethod
        def detect(filename: str):
            ...

        def scan(self, filename: str, register_frame: Callable[..., None]):
            ...

        def parse(self, data: DataCollection, filename: str, frame_info: Any, **kwargs: Any):
            ...

You can freely choose the class name (we use `MyFileReader` as an example here) and the class must derive from the base `ovito.io.FileReaderInterface`.

Once your class has been registered, OVITO Pro and the `import_file()` function will try to open files with the help of all installed file readers by calling their `detect()` methods. The first one that returns `True` from its `detect()` method will be used by the system to actually import the requested file.

Example file reader

In the following three sections, we will implement the `detect()`, `scan()`, and `parse()` methods to import the following simple text-based data format:

    Header of MyFileFormat
    Timestep 0: 6 particles
    <x0> <y0> <z0>
    <x1> <y1> <z1>
    <x2> <y2> <z2>
    <x3> <y3> <z3>
    <x4> <y4> <z4>
    <x5> <y5> <z5>
    Timestep 10: 3 particles
    <x0> <y0> <z0>
    <x1> <y1> <z1>
    <x2> <y2> <z2>
    Timestep 20: 4 particles
    <x0> <y0> <z0>
    <x1> <y1> <z1>
    <x2> <y2> <z2>
    <x3> <y3> <z3>

This contrived file format stores multiple trajectory frames of a particle-based model and consists of a header line at the top of the file followed by multiple frame records, each marked by its own header line. Within the data sections, placeholders such as `<x0> <y0> <z0>` denote the xyz coordinates of particles.

Tip

While these sections present a file reader for particle data, the same programming interface can also be used to implement importers for other kinds of data in OVITO, such as surface meshes, voxel grids, or bonds.

## The `detect` method

The `detect()` method is a `static` method of the file reader class and takes the path of the file to be inspected as the only input parameter (no `self` parameter).

This method is called by OVITO whenever the user tries to import a new file to determine whether that file can be parsed by your file reader. That means your implementation should return `True` if your reader class can process a given file and `False` otherwise. For efficiency, the decision should be made as quickly as possible, i.e. by reading and inspecting just the first few lines of the file, in order to not slow down the import of files that will be handled by other file readers.

Let’s consider the following example, where our file reader looks for text files containing the string “Header of MyFileFormat” on the first line:

    from ovito.data import DataCollection
    from ovito.io import FileReaderInterface, import_file
    from typing import Callable, Any

    class MyFileReader(FileReaderInterface):

        @staticmethod
        def detect(filename: str):
            try:
                with open(filename, "r") as f:
                    line = f.readline()
                    return line.strip() == "Header of MyFileFormat"
            except OSError:
                return False

        def scan(self, filename: str, register_frame: Callable[..., None]):
            ...

        def parse(self, data: DataCollection, filename: str, frame_info: Any, **kwargs: Any):
            ...

Our implementation of the `detect()` method opens the file, reads one line, and returns `True` in case it matches the key string we are looking for.

## The `scan` method

The `scan()` method is an optional method that should be implemented only if the files to be read by your reader can store multiple frames of a trajectory. It will be called by the system to index all frames in the imported file, populate the timeline in OVITO, and enable quick random access to individual trajectory frames.

An implementation of the `scan()` method usually reads the whole file, discovers all frames, and communicates each frame’s metadata to the OVITO system. This happens via invocation of the `register_frame` callback function for each discovered frame. The callback is provided by the system and has the following signature:

    register_frame(frame_info: Any = None, label: Optional[str] = None)

`frame_info` can be (almost) any type of Python value and is used by your file reader to describe the storage location of each frame in the file. One might, for example, use the line number or the byte offset where each frame begins in the file as `frame_info`. Or, for a database format, one might use the unique record key of a frame as its `frame_info`, which can later help to access the data of the frame efficiently. The `frame_info` values will be stored by the OVITO system as part of the trajectory index and will be made available later again to your file reader’s `parse()` method when loading specific frames from the file.

The `label` parameter is optional and specifies a human-readable text to be used as a descriptive label for the trajectory frame in the OVITO timeline. It has purely informational character, e.g., the simulation timestep.

In our example file format, each frame begins on a new line with the format “Timestep <T>: <N> particles”. Here, T denotes the simulation timestep, and N the number of particles in the simulation snapshot. We might write the following `scan()` method, which specifically searches for these frame headers using a regular expression:

    from ovito.data import DataCollection
    from ovito.io import FileReaderInterface, import_file
    from typing import Callable, Any
    import re

    class MyFileReader(FileReaderInterface):

        @staticmethod
        def detect(filename: str):
            try:
                with open(filename, "r") as f:
                    line = f.readline()
                    return line.strip() == "Header of MyFileFormat"
            except OSError:
                return False

        def scan(self, filename: str, register_frame: Callable[..., None]):
            expr = r"(Timestep \d+): (\d+) particles"
            with open(filename, "r") as f:
                for line_number, line in enumerate(f):
                    match = re.match(expr, line.strip())
                    if match:
                        number_particles = int(match.group(2))
                        label = match.group(1)
                        register_frame(frame_info=(line_number, number_particles), label=label)

        def parse(self, data: DataCollection, filename: str, frame_info: Any, **kwargs: Any):
            ...

Here, both the line number at which a frame starts and the number of particles it contains are stored as tuple in the `frame_info` for later use. The string “Timestep …” is specified as a label when registering trajectory frames with OVITO.

## The `parse` method

The `parse()` method is the main function you need to implement for a file reader. It will be called by OVITO to load actual data from the file, one trajectory frame at a time, and has the following basic signature:

    def parse(self, data: DataCollection, filename: str, **kwargs):

The first time your `parse()` implementation gets called by the system, it receives an empty `DataCollection` object, which should be populated with the information loaded from the input file. This typically involves creating one or more data objects, e.g. `Particles`, `SimulationCell`, `SurfaceMesh`, `TriangleMesh`, within the `DataCollection`, or populating the `DataCollection.attributes` dictionary with auxiliary metadata parsed from the file.

On subsequent invocations of `parse()`, the `DataCollection` provided by the system may already contain objects from a previous trajectory frame, and your implementation should update or add only information that has changed in the current frame. That means, for example, that particle types shouldn’t be recreated by the file reader every time. Rather, existing data in the collection should be touched only selectively by the file reader to preserve any changes the user has made in the GUI in the meantime. This applies, for instance, to parameters of particle types such as color, radius, and name but also settings of visual elements, which can be concurrently edited by the user in the GUI.

Tip

The Python API of OVITO provides special functions that create new data objects only if needed and otherwise preserve existing information and visualization settings associated with these objects:

- `DataCollection.create_cell()`

- `DataCollection.create_particles()`

- `Particles.create_bonds()`

- `PropertyContainer.create_property()`

- `Property.add_type_name()` and `Property.add_type_id()` (see example)

- `DataCollection.grids.create(identifier: str, vis_params: dict = None, **kwargs)`

- `DataCollection.surfaces.create(identifier: str, vis_params: dict = None, **kwargs)` (see `example`)

- `DataCollection.tables.create(identifier: str, vis_params: dict = None, **kwargs)`

In case you are developing a file reader for a trajectory file format, you can use the following extended signature of the `parse()` method:

    def parse(self, data: DataCollection, filename: str, frame_index: int, frame_info: Any, **kwargs):

The `frame_info` value, which was generated by the file reader’s scan() method introduced above, and the zero-based trajectory frame to be loaded are passed to your `parse()` method by the system. Any further keyword arguments from the system go into the `kwargs` dictionary.

Important

The trailing `**kwargs` parameter must always be part of the method’s parameters list to accept all further arguments, which may be provided by future versions of OVITO. It’s there for forward compatibility reasons and to receive all unused arguments your `parse` method is not interested in.

Please have another look at our example file format defined above, for which we will now implement a parsing method. The following `parse()` implementation skips through the initial lines of the input file until it reaches the one where the requested frame begins. Then a `Particles` object with a data array for the Position property is created before the xyz coordinates of the particles are parsed from the file line by line:

     1from ovito.data import DataCollection
     2from ovito.io import FileReaderInterface, import_file
     3from typing import Callable, Any
     4import re
     5
     6class MyFileReader(FileReaderInterface):
     7
     8    @staticmethod
     9    def detect(filename: str):
    10        try:
    11            with open(filename, "r") as f:
    12                line = f.readline()
    13                return line.strip() == "Header of MyFileFormat"
    14        except OSError:
    15            return False
    16
    17    def scan(self, filename: str, register_frame: Callable[..., None]):
    18        expr = r"(Timestep \d+): (\d+) particles"
    19        with open(filename, "r") as f:
    20            for line_number, line in enumerate(f):
    21                match = re.match(expr, line.strip())
    22                if match:
    23                    num_particles = int(match.group(2))
    24                    label = match.group(1)
    25                    register_frame(frame_info=(line_number, num_particles), label=label)
    26
    27    def parse(self, data: DataCollection, filename: str, frame_info: tuple[int, int], **kwargs: Any):
    28        starting_line_number, num_particles = frame_info
    29
    30        with open(filename, "r") as f:
    31            for _ in range(starting_line_number + 1):
    32                f.readline()
    33
    34            particles = data.create_particles(count=num_particles)
    35            positions = particles.create_property("Position")
    36
    37            for i in range(num_particles):
    38                positions[i] = [float(coord) for coord in f.readline().strip().split()]

While this first example introduced the basic principles of user-defined file readers in OVITO, code example FR1 will present a more thorough implementation of a custom file reader, focusing on how to load more particle properties and the simulation cell geometry into OVITO.

## Testing your file reader

To test your file reader outside of OVITO Pro, you can add a Python main program to the `.py` file in which you define the file reader class and invoke the general `import_file()` function:

    class MyFileReader(FileReaderInterface):
        ...

    if __name__ == "__main__":
        pipeline = import_file("myfile.dat", input_format=MyFileReader)
        for frame in range(pipeline.source.num_frames):
            data = pipeline.compute(frame)

Note that we pass the custom file reader class to the `import_file()` function. This circumvents the automatic detection of the file format (your `detect()` method won’t be called by the system!) and the `scan()` and `parse()` methods will be invoked immediately. The for-loop iterates over all trajectory frames registered by the file reader and loads them one by one.

The approach described above is good for testing or if you just want to use your custom file reader from a standalone Python program. For full integration into OVITO Pro and to make your file reader participate in the automatic file detection system, it needs to be installed and registered as a discoverable extension. This process is outlined in the section Packaging and installation of user extensions for OVITO.

## User parameters

Your Python file reader can optionally expose adjustable user parameters based on the Traits framework, which are displayed automatically in the user interface of OVITO Pro. The system works analogously to Python modifier classes that expose user-defined parameters.

The `import_file()` function will forward any additional keyword arguments to your file reader class and initialize its parameter traits with matching names.

See also

- Packaging and installation of user extensions for OVITO

- `ovito.io.FileReaderInterface`