I/O

A new, comprehensive way of doing data input and output has been introduced in MSMBuidler 3.6. The previous dataset method will still be supported and may be appropriate in certain cases, especially if your data can’t all fit in memory.

Dictionary of trajectories

MSMBuilder learns from and transforms a collection of sequences. While the time-ordering of each sequence is important, the order of the sequences themselves has no special meaning. For I/O, we treat collections of sequences as a dictionary mapping between arbitrary keys and the sequences (which are probably 2D numpy arrays). Because our sequences are time-series, we call them “trajectories”, although they may not be in normal Cartesian space.

The io module assumes a python dictionary. Ideally, we would use our dictionary keys as filenames for individual sequences saved as individual .npy files on disk. In practice, python dictionaries can be any python object and filenames must be unique strings. The io module have a mapping from python-object keys to filenames. It’s important to note that the converse (going from filename to python object) does not need to be codified. That’s because (in contrast to the dataset approach) we persist the set of keys in a separate metadata file. This file is saved using python’s pickle protocol and can contain arbitrary python objects [1].

[1]We don’t just serialize the whole python dictionary of sequences using pickle, because it chokes on big numpy arrays.

Mapping keys to filenames

By default, msmbuilder.io can handle mapping of “normal” dictionary keys to filenames. This should work well with strings, integers, or tuples of the above.

Metadata

Per-sequence information should be persisted in a pandas DataFrame. They index of the dataframe should be they keys used in the trajectory. This dataframe is required for the saving and loading functions, as it serves as the canonical list of keys.

Estimators

Estimators are persisted using the generic pickle protocol.

Saving and loading

load_trajs
save_trajs
load_meta
save_meta
backup
render_meta
load_generic
save_generic
itertrajs
preload_tops
preload_top

Gathering Metadata

Gathering trajectory metadata should come at the start of an MSM project after you have collected and pre-processed your molecular dynamics trajectories. We provide utilities for parsing metadata for common ways of organizing a set of molecular dynamics trajectories

gather_metadata
GenericParser
NumberedRunsParser
HierarchyParser

Project Templates

The msmb TempalteProject command-line command generates a set of example scripts to serve as a framework for an MSM project. You can use this functionality programatically.

TemplateProject

The templates are stored in msmbuilder/project_templates. They are jinja2 templates.

  • Python files can optionally be converted into IPython notebooks during template rendering. Indicate where cell breaks should happen with ## Description goes here

  • The hierarchy of the template project is not read from the msmbuilder/project_templates source filesystem hierarchy. It’s explicitly listed as a Python expression in msmbuilder.io. If you add a new template file, make sure you list it in msmbuilder.io or it will not be rendered.

  • Templates can contain yaml “front matter”. For some reason, jinja2 doesn’t support this, so it is parsed explicitly by MSMBuilder. Include the yaml as the last element in the file’s docstring under a numpydoc heading “Meta”:

    Meta
----
depends:
  - meta.pandas.pickl
arbitrary_key:
  - arbitrary data

  • MSMBuilder defines some variables for use in your templates including {{header}} and {{date}}. For a complete list, check the rendering code.

  • Plotting scripts should include the following macros before any imports:

    # ? include "plot_header.template"
# ? from "plot_macros.template" import xdg_open with context

    This will set up matplotlib to use the correct backend. Add:

    # {{xdg_open('filename.pdf')}}

    to have a call to xdg-open inserted based on user configuration.