aitomic.nomad_nmr

Tools to interact with NOMAD NMR.

A NOMAD NMR deployment is used by NMR labs to manage their machines and store their data in a central place and in a FAIR manner. It automatically provides features such as a monitoring system and a data repository which includes metadata and access control.

The NOMAD NMR server provides a REST API to interact with it, which this module relies upon. The primary goal of this module is to provide an interface for downloading large datasets from the NOMAD server and turn them into data frames which can be used for machine learning.

Examples

Gettiing an NMR peak data frame

If you have data in the NOMAD server, chances are you want to use it for some kind of data analysis. The easiest thing to do is to get a polars.DataFrame with all your NMR peaks. Here we produce polars.DataFrame holding all the peaks, including their spectrum of origin, ppm and volume:

from aitomic import bruker, nomad_nmr

client = nomad_nmr.Client.login(
    "http://demo.nomad-nmr.uk",
    username="demo",
    password="dem0User",
)
experiments = client.auto_experiments()
peak_df = bruker.nmr_peaks_df_1d(experiments.download())
peak_df = nomad_nmr.add_metadata(client, peak_df)
┌─────────────────────────────────┬──────────┬──────────────┬────────────────────────────────┬───┬──────────────┬──────────┬──────────────────────────┬─────────────┐
│ spectrum                        ┆ ppm      ┆ integral     ┆ auto_experiment_id             ┆ … ┆ submitted_at ┆ username ┆ group_id_right           ┆ group_name  │
│ ---                             ┆ ---      ┆ ---          ┆ ---                            ┆   ┆ ---          ┆ ---      ┆ ---                      ┆ ---         │
│ str                             ┆ f64      ┆ f64          ┆ str                            ┆   ┆ null         ┆ str      ┆ str                      ┆ str         │
╞═════════════════════════════════╪══════════╪══════════════╪════════════════════════════════╪═══╪══════════════╪══════════╪══════════════════════════╪═════════════╡
│ 2410081201-0-1-lukasturcani/10… ┆ 8.344768 ┆ 20680.796875 ┆ 2410081201-0-1-lukasturcani-10 ┆ … ┆ null         ┆ test3    ┆ 672fdae0eb3b1c3c17062fee ┆ test-admins │
│ 2410081201-0-1-lukasturcani/10… ┆ 8.339878 ┆ 31792.195312 ┆ 2410081201-0-1-lukasturcani-10 ┆ … ┆ null         ┆ test3    ┆ 672fdae0eb3b1c3c17062fee ┆ test-admins │
│ 2410081201-0-1-lukasturcani/10… ┆ 8.338044 ┆ 20503.757812 ┆ 2410081201-0-1-lukasturcani-10 ┆ … ┆ null         ┆ test3    ┆ 672fdae0eb3b1c3c17062fee ┆ test-admins │
│ 2410081201-0-1-lukasturcani/10… ┆ 8.336821 ┆ 10042.96875  ┆ 2410081201-0-1-lukasturcani-10 ┆ … ┆ null         ┆ test3    ┆ 672fdae0eb3b1c3c17062fee ┆ test-admins │
│ 2410081201-0-1-lukasturcani/10… ┆ 8.323985 ┆ 10558.703125 ┆ 2410081201-0-1-lukasturcani-10 ┆ … ┆ null         ┆ test3    ┆ 672fdae0eb3b1c3c17062fee ┆ test-admins │
│ …                               ┆ …        ┆ …            ┆ …                              ┆ … ┆ …            ┆ …        ┆ …                        ┆ …           │
│ 2410161546-0-1-admin/10/pdata/… ┆ 1.398485 ┆ 10062.0      ┆ 2410161546-0-1-admin-10        ┆ … ┆ null         ┆ test1    ┆ 672fdae0eb3b1c3c17062fed ┆ group-1     │
│ 2410161546-0-1-admin/10/pdata/… ┆ 1.238337 ┆ 4.8948e7     ┆ 2410161546-0-1-admin-10        ┆ … ┆ null         ┆ test1    ┆ 672fdae0eb3b1c3c17062fed ┆ group-1     │
│ 2410161546-0-1-admin/10/pdata/… ┆ 1.051905 ┆ 31991.0      ┆ 2410161546-0-1-admin-10        ┆ … ┆ null         ┆ test1    ┆ 672fdae0eb3b1c3c17062fed ┆ group-1     │
│ 2410161546-0-1-admin/10/pdata/… ┆ 1.048848 ┆ 41602.6875   ┆ 2410161546-0-1-admin-10        ┆ … ┆ null         ┆ test1    ┆ 672fdae0eb3b1c3c17062fed ┆ group-1     │
│ 2410161546-0-1-admin/10/pdata/… ┆ 0.858137 ┆ 146085.9375  ┆ 2410161546-0-1-admin-10        ┆ … ┆ null         ┆ test1    ┆ 672fdae0eb3b1c3c17062fed ┆ group-1     │
└─────────────────────────────────┴──────────┴──────────────┴────────────────────────────────┴───┴──────────────┴──────────┴──────────────────────────┴─────────────┘

See also

Downloading auto experiment data

from aitomic import nomad_nmr
from pathlib import Path

client = nomad_nmr.Client.login(
    "http://demo.nomad-nmr.uk",
    username="demo",
    password="dem0User",
)
experiments = client.auto_experiments()
Path("experiments.zip").write_bytes(experiments.download())

See also

Gettting auto experiment data as a data frame

This example shows you have to get all the data held by AutoExperiments as a data frame, note that this does not download the spectrum data itself:

experiments = client.auto_experiments()
df = experiments.to_df()
┌──────────────────────────┬───────────────────────┬───────────────────┬─────────────────┬────────────┬────────────┬──────────────────────────┬──────────────────────────┬──────────────────────────┬─────────┬─────────────────────────┐
│ auto_experiment_id       ┆ dataset_name          ┆ experiment_number ┆ parameter_set   ┆ parameters ┆ title      ┆ instrument_id            ┆ user_id                  ┆ group_id                 ┆ solvent ┆ submitted_at            │
│ ---                      ┆ ---                   ┆ ---               ┆ ---             ┆ ---        ┆ ---        ┆ ---                      ┆ ---                      ┆ ---                      ┆ ---     ┆ ---                     │
│ str                      ┆ str                   ┆ str               ┆ str             ┆ null       ┆ str        ┆ str                      ┆ str                      ┆ str                      ┆ str     ┆ datetime[μs, UTC]       │
╞══════════════════════════╪═══════════════════════╪═══════════════════╪═════════════════╪════════════╪════════════╪══════════════════════════╪══════════════════════════╪══════════════════════════╪═════════╪═════════════════════════╡
│ 2106231050-2-1-test1-10  ┆ 2106231050-2-1-test1  ┆ 10                ┆ parameter-set-1 ┆ null       ┆ Test Exp 1 ┆ 672658eff9f290068dc027bd ┆ 672658eff9f290068dc027c5 ┆ 672658eff9f290068dc027c3 ┆ CDCl3   ┆ null                    │
│ 2106231050-2-1-test1-11  ┆ 2106231050-2-1-test1  ┆ 11                ┆ parameter-set-1 ┆ null       ┆ Test Exp 1 ┆ 672658eff9f290068dc027bd ┆ 672658eff9f290068dc027c5 ┆ 672658eff9f290068dc027c3 ┆ CDCl3   ┆ null                    │
│ 2106231055-3-2-test2-10  ┆ 2106231055-3-2-test2  ┆ 10                ┆ parameter-set-2 ┆ null       ┆ Test Exp 3 ┆ 672658eff9f290068dc027be ┆ 672658eff9f290068dc027c6 ┆ 672658eff9f290068dc027c3 ┆ C6D6    ┆ null                    │
│ 2106231100-10-2-test3-10 ┆ 2106231100-10-2-test3 ┆ 10                ┆ parameter-set-3 ┆ null       ┆ Test Exp 4 ┆ 672658eff9f290068dc027bf ┆ 672658eff9f290068dc027c7 ┆ 672658eff9f290068dc027c3 ┆ C6D6    ┆ null                    │
│ 2106240012-10-2-test2-10 ┆ 2106240012-10-2-test2 ┆ 10                ┆ parameter-set-3 ┆ null       ┆ Test Exp 5 ┆ 672658eff9f290068dc027bf ┆ 672658eff9f290068dc027c7 ┆ 672658eff9f290068dc027c3 ┆ C6D6    ┆ null                    │
│ 2106241100-10-2-test3-10 ┆ 2106241100-10-2-test3 ┆ 10                ┆ parameter-set-3 ┆ null       ┆ Test Exp 6 ┆ 672658eff9f290068dc027bf ┆ 672658eff9f290068dc027c7 ┆ 672658eff9f290068dc027c4 ┆ CDCl3   ┆ null                    │
│ 2106241100-10-2-test4-1  ┆ 2106241100-10-2-test4 ┆ 1                 ┆ parameter-set-3 ┆ null       ┆ Test Exp 7 ┆ 672658eff9f290068dc027bf ┆ 672658eff9f290068dc027c7 ┆ 672658eff9f290068dc027c4 ┆ CDCl3   ┆ 2024-01-01 00:00:00 UTC │
└──────────────────────────┴───────────────────────┴───────────────────┴─────────────────┴────────────┴────────────┴──────────────────────────┴──────────────────────────┴──────────────────────────┴─────────┴─────────────────────────┘

See also

Joining data frames

Sometimes you may have two different data frames but want to join their data. For example, if you use nomad_nmr.AutoExperiments.to_df() you will have a data frame with all the user ids. However, you will probably want to filter data not by the user id but by the username. First lets create a data frame of auto experiments:

experiments = client.auto_experiments().to_df()

Then lets create a data frame of user data:

users = client.users().to_df()

and join the two data frames:

experiments = experiments.join(users, on="user_id")

Now we can filter the data frame by the username:

experiments.filter(pl.col("username") == "fred")
┌──────────────────────────┬───────────────────────┬───────────────────┬─────────────────┬────────────┬────────────┬──────────────────────────┬──────────────────────────┬──────────────────────────┬─────────┬─────────────────────────┐
│ auto_experiment_id       ┆ dataset_name          ┆ experiment_number ┆ parameter_set   ┆ parameters ┆ title      ┆ username                 ┆ user_id                  ┆ group_id                 ┆ solvent ┆ submitted_at            │
│ ---                      ┆ ---                   ┆ ---               ┆ ---             ┆ ---        ┆ ---        ┆ ---                      ┆ ---                      ┆ ---                      ┆ ---     ┆ ---                     │
│ str                      ┆ str                   ┆ str               ┆ str             ┆ null       ┆ str        ┆ str                      ┆ str                      ┆ str                      ┆ str     ┆ datetime[μs, UTC]       │
╞══════════════════════════╪═══════════════════════╪═══════════════════╪═════════════════╪════════════╪════════════╪══════════════════════════╪══════════════════════════╪══════════════════════════╪═════════╪═════════════════════════╡
│ 2106231050-2-1-test1-10  ┆ 2106231050-2-1-test1  ┆ 10                ┆ parameter-set-1 ┆ null       ┆ Test Exp 1 ┆ fred                     ┆ 672658eff9f290068dc027c5 ┆ 672658eff9f290068dc027c3 ┆ CDCl3   ┆ null                    │
│ 2106231050-2-1-test1-11  ┆ 2106231050-2-1-test1  ┆ 11                ┆ parameter-set-1 ┆ null       ┆ Test Exp 1 ┆ fred                     ┆ 672658eff9f290068dc027c5 ┆ 672658eff9f290068dc027c3 ┆ CDCl3   ┆ null                    │
└──────────────────────────┴───────────────────────┴───────────────────┴─────────────────┴────────────┴────────────┴──────────────────────────┴──────────────────────────┴──────────────────────────┴─────────┴─────────────────────────┘

In addition to users, you can also do this process with groups!

See also

Downloading auto experiment data matching a query

from aitomic import nomad_nmr
from pathlib import Path

client = nomad_nmr.Client.login(
    "http://demo.nomad-nmr.uk",
    username="demo",
    password="dem0User",
)
experiments = client.auto_experiments(
    query=nomad_nmr.AutoExperimentQuery(
        solvent="DMSO",
        title=["test", "test-1"]
    )
)
Path("experiments.zip").write_bytes(experiments.download())

See also

Additional filtering

Sometimes the filtering allowed by AutoExperimentQuery is not enough. In this case, you can use the AutoExperiments.inner attribute to filter the experiments yourself, and then download only the experiments you want:

from aitomic import nomad_nmr
from pathlib import Path

client = nomad_nmr.Client.login(
    "http://demo.nomad-nmr.uk",
    username="demo",
    password="dem0User",
)
experiments = client.auto_experiments(
    query=nomad_nmr.AutoExperimentQuery(
        solvent="DMSO",
    )
)
experiments.inner = [
    experiment
    for experiment in experiments
    if "special-study" in experiment.title
]
Path("experiments.zip").write_bytes(experiments.download())

See also

Functions

Classes

AuthToken

Authentication token for the NOMAD server.

AutoExperiment

Data about an auto experiment stored in NOMAD.

AutoExperimentQuery

Query for auto experiments.

AutoExperiments

A collection of auto experiments.

Client

Client for interacting with a NOMAD server.

Group

Data about a NOMAD group.

Groups

A collection of groups.

User

Data about a NOMAD user.

Users

A collection of users.