Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FIREMinimizationIntegrator fails on explicitly solvated system using CPU platform #686

Open
IAlibay opened this issue Apr 18, 2023 · 3 comments
Open

FIREMinimizationIntegrator fails on explicitly solvated system using CPU platform #686

IAlibay opened this issue Apr 18, 2023 · 3 comments

Comments

@IAlibay
Copy link
Contributor

IAlibay commented Apr 18, 2023

Context

This is related to why #684 needed fixing, and hopefully some of the alchemical pains here will go away with #672

Bug report

We noticed that the FIREMinimizationIntegrator applied to an explicitly solvated system will yield NaNs when using a CPU platform, but will run fine on CUDA, OpenCL, and Reference.

Minimum reproducible code:

from openmmtools.integrators import FIREMinimizationIntegrator
from openmmtools import testsystems
import openmm

t = testsystems.AlanineDipeptideExplicit()
system, positions = t.system, t.positions
integrator = FIREMinimizationIntegrator()
platform = openmm.Platform.getPlatformByName('CPU')
context = openmm.Context(system, integrator, platform)
context.setPositions(positions)
integrator.step(10000)
state = context.getState(getEnergy=True)
state.getPotentialEnergy()

Error output:

OpenMMException                           Traceback (most recent call last)
Cell In[1], line 11
      9 context = openmm.Context(system, integrator, platform)
     10 context.setPositions(positions)
---> 11 integrator.step(10000)
     12 state = context.getState(getEnergy=True)
     13 state.getPotentialEnergy()

File ~/software/mambaforge/install/envs/openfe-latest/lib/python3.10/site-packages/openmm/openmm.py:13583, in CustomIntegrator.step(self, steps)
  13573 def step(self, steps):
  13574     r"""
  13575     step(self, steps)
  13576     Advance a simulation through time by taking a series of time steps.
   (...)
  13581         the number of time steps to take
  13582     """
> 13583     return _openmm.CustomIntegrator_step(self, steps)

OpenMMException: Particle coordinate is NaN.  For more information, see https://github.com/openmm/openmm/wiki/Frequently-Asked-Questions#nan

mamba list output:

# packages in environment at /home/ialibay/software/mambaforge/install/envs/openfe-latest:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
amberlite                 22.0                     pypi_0    pypi
ambertools                22.0            py310h206695f_3    conda-forge
amberutils                21.0                     pypi_0    pypi
anyio                     3.6.2              pyhd8ed1ab_0    conda-forge
argon2-cffi               21.3.0             pyhd8ed1ab_0    conda-forge
argon2-cffi-bindings      21.2.0          py310h5764c6d_3    conda-forge
arpack                    3.7.0                hdefa2d7_2    conda-forge
asttokens                 2.2.1              pyhd8ed1ab_0    conda-forge
astunparse                1.6.3              pyhd8ed1ab_0    conda-forge
attrs                     22.2.0             pyh71513ae_0    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.4              pyhd8ed1ab_0    conda-forge
beautifulsoup4            4.12.2             pyha770c72_0    conda-forge
bleach                    6.0.0              pyhd8ed1ab_0    conda-forge
blosc                     1.21.3               hafa529b_0    conda-forge
boost                     1.74.0          py310h7c3ba0c_5    conda-forge
boost-cpp                 1.74.0               h75c5d50_8    conda-forge
brotli                    1.0.9                h166bdaf_8    conda-forge
brotli-bin                1.0.9                h166bdaf_8    conda-forge
brotlipy                  0.7.0           py310h5764c6d_1005    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.18.1               h7f98852_0    conda-forge
ca-certificates           2022.12.7            ha878542_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cachetools                5.3.0              pyhd8ed1ab_0    conda-forge
cairo                     1.16.0            ha61ee94_1014    conda-forge
certifi                   2022.12.7          pyhd8ed1ab_0    conda-forge
cffi                      1.15.1          py310h255011f_3    conda-forge
cftime                    1.6.2           py310hde88566_1    conda-forge
charset-normalizer        3.1.0              pyhd8ed1ab_0    conda-forge
click                     8.1.3           unix_pyhd8ed1ab_2    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
comm                      0.1.3              pyhd8ed1ab_0    conda-forge
contourpy                 1.0.7           py310hdf3cbec_0    conda-forge
coverage                  7.2.3           py310h1fa729e_0    conda-forge
cryptography              40.0.2          py310h34c0648_0    conda-forge
cudatoolkit               11.8.0              h37601d7_11    conda-forge
curl                      8.0.1                h588be90_0    conda-forge
cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
cython                    0.29.34         py310heca2aa9_0    conda-forge
debugpy                   1.6.7           py310heca2aa9_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
entrypoints               0.4                pyhd8ed1ab_0    conda-forge
executing                 1.2.0              pyhd8ed1ab_0    conda-forge
expat                     2.5.0                hcb278e6_1    conda-forge
fftw                      3.3.10          nompi_hc118613_107    conda-forge
flit-core                 3.8.0              pyhd8ed1ab_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.2               h14ed4e7_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.39.3          py310h1fa729e_0    conda-forge
freetype                  2.12.1               hca18f0e_1    conda-forge
gettext                   0.21.1               h27087fc_0    conda-forge
greenlet                  2.0.2           py310heca2aa9_0    conda-forge
gufe                      0.7.2              pyhd8ed1ab_0    conda-forge
hdf4                      4.2.15               h9772cbc_5    conda-forge
hdf5                      1.12.2          nompi_h4df4325_101    conda-forge
icu                       70.1                 h27087fc_0    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
importlib-metadata        6.5.0              pyha770c72_0    conda-forge
importlib_metadata        6.5.0                hd8ed1ab_0    conda-forge
importlib_resources       5.12.0             pyhd8ed1ab_0    conda-forge
ipykernel                 6.22.0             pyh210e3f2_0    conda-forge
ipython                   8.12.0             pyh41d4057_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
ipywidgets                7.7.5              pyhd8ed1ab_0    conda-forge
jedi                      0.18.2             pyhd8ed1ab_0    conda-forge
jinja2                    3.1.2              pyhd8ed1ab_1    conda-forge
jpeg                      9e                   h0b41bf4_3    conda-forge
jsonschema                4.17.3             pyhd8ed1ab_0    conda-forge
jupyter_client            8.2.0              pyhd8ed1ab_0    conda-forge
jupyter_core              5.3.0           py310hff52083_0    conda-forge
jupyter_events            0.6.3              pyhd8ed1ab_0    conda-forge
jupyter_server            2.5.0              pyhd8ed1ab_0    conda-forge
jupyter_server_terminals  0.4.4              pyhd8ed1ab_1    conda-forge
jupyterlab_pygments       0.2.2              pyhd8ed1ab_0    conda-forge
jupyterlab_widgets        1.1.4              pyhd8ed1ab_0    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.4           py310hbf28c38_1    conda-forge
krb5                      1.20.1               h81ceb04_0    conda-forge
lcms2                     2.15                 hfd0df8a_0    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libaec                    1.0.6                hcb278e6_1    conda-forge
libblas                   3.9.0           16_linux64_openblas    conda-forge
libbrotlicommon           1.0.9                h166bdaf_8    conda-forge
libbrotlidec              1.0.9                h166bdaf_8    conda-forge
libbrotlienc              1.0.9                h166bdaf_8    conda-forge
libcblas                  3.9.0           16_linux64_openblas    conda-forge
libcurl                   8.0.1                h588be90_0    conda-forge
libdeflate                1.17                 h0b41bf4_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libexpat                  2.5.0                hcb278e6_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 12.2.0              h65d4601_19    conda-forge
libgfortran-ng            12.2.0              h69a702a_19    conda-forge
libgfortran5              12.2.0              h337968e_19    conda-forge
libglib                   2.76.1               ha491796_0    conda-forge
libgomp                   12.2.0              h65d4601_19    conda-forge
libiconv                  1.17                 h166bdaf_0    conda-forge
liblapack                 3.9.0           16_linux64_openblas    conda-forge
libllvm11                 11.1.0               he0ac6c6_5    conda-forge
libnetcdf                 4.8.1           nompi_h261ec11_106    conda-forge
libnghttp2                1.52.0               h61bc06f_0    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libopenblas               0.3.21          pthreads_h78a6416_3    conda-forge
libpng                    1.6.39               h753d276_0    conda-forge
libsodium                 1.0.18               h36c2ea0_1    conda-forge
libsqlite                 3.40.0               h753d276_0    conda-forge
libssh2                   1.10.0               hf14f497_3    conda-forge
libstdcxx-ng              12.2.0              h46fd767_19    conda-forge
libtiff                   4.5.0                h6adf6a1_2    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libwebp-base              1.3.0                h0b41bf4_0    conda-forge
libxcb                    1.13              h7f98852_1004    conda-forge
libxml2                   2.10.3               hca2bb57_4    conda-forge
libxslt                   1.1.37               h873f0b0_0    conda-forge
libzip                    1.9.2                hc929e4a_1    conda-forge
libzlib                   1.2.13               h166bdaf_4    conda-forge
llvmlite                  0.39.1          py310h58363a5_1    conda-forge
lomap2                    2.3.0              pyhd8ed1ab_0    conda-forge
lxml                      4.9.2           py310hbdc0903_0    conda-forge
lz4-c                     1.9.4                hcb278e6_0    conda-forge
lzo                       2.10              h516909a_1000    conda-forge
markupsafe                2.1.2           py310h1fa729e_0    conda-forge
matplotlib-base           3.7.1           py310he60537e_0    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
mdtraj                    1.9.7           py310h902c554_4    conda-forge
mistune                   2.0.5              pyhd8ed1ab_0    conda-forge
mmpbsa-py                 16.0                     pypi_0    pypi
mpiplus                   v0.0.1          pyhd8ed1ab_1004    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
nbclassic                 0.5.5              pyhb4ecaf3_1    conda-forge
nbclient                  0.7.3              pyhd8ed1ab_0    conda-forge
nbconvert                 7.3.1              pyhd8ed1ab_0    conda-forge
nbconvert-core            7.3.1              pyhd8ed1ab_0    conda-forge
nbconvert-pandoc          7.3.1              pyhd8ed1ab_0    conda-forge
nbformat                  5.8.0              pyhd8ed1ab_0    conda-forge
ncurses                   6.3                  h27087fc_1    conda-forge
nest-asyncio              1.5.6              pyhd8ed1ab_0    conda-forge
netcdf-fortran            4.6.0           nompi_he1eeb6f_102    conda-forge
netcdf4                   1.6.2           nompi_py310h55e1e36_100    conda-forge
networkx                  3.1                pyhd8ed1ab_0    conda-forge
nomkl                     1.0                  h5ca1d4c_0    conda-forge
nose                      1.3.7                   py_1006    conda-forge
notebook                  6.5.4              pyha770c72_0    conda-forge
notebook-shim             0.2.2              pyhd8ed1ab_0    conda-forge
numba                     0.56.4          py310h0e39c9b_1    conda-forge
numexpr                   2.8.4           py310h690d005_100    conda-forge
numpy                     1.23.5          py310h53a5b5f_0    conda-forge
ocl-icd                   2.3.1                h7f98852_0    conda-forge
ocl-icd-system            1.0.0                         1    conda-forge
openfe                    0.7.2              pyhd8ed1ab_0    conda-forge
openfe-benchmarks         0.0.0                    pypi_0    pypi
openff-amber-ff-ports     0.0.3              pyh6c4a22f_0    conda-forge
openff-forcefields        2023.04.1          pyh1a96a4e_0    conda-forge
openff-interchange        0.2.3              pyhd8ed1ab_1    conda-forge
openff-interchange-base   0.2.3              pyhd8ed1ab_1    conda-forge
openff-models             0.0.4              pyh1a96a4e_0    conda-forge
openff-toolkit            0.12.1             pyhd8ed1ab_2    conda-forge
openff-toolkit-base       0.12.1             pyhd8ed1ab_2    conda-forge
openff-units              0.2.0              pyh1a96a4e_0    conda-forge
openff-utilities          0.1.8              pyh1a96a4e_0    conda-forge
openjpeg                  2.5.0                hfec8fc6_2    conda-forge
openmm                    8.0.0           py310h5728c26_0    conda-forge
openmmforcefields         0.11.2             pyhd8ed1ab_1    conda-forge
openmmtools               0.22.0             pyhd8ed1ab_0    conda-forge
openssl                   3.1.0                h0b41bf4_0    conda-forge
packaging                 23.1               pyhd8ed1ab_0    conda-forge
packmol                   20.010               h86c2bf4_0    conda-forge
packmol-memgen            1.2.3rc0                 pypi_0    pypi
pandas                    2.0.0           py310h9b08913_0    conda-forge
pandoc                    2.19.2               h32600fe_2    conda-forge
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
panedr                    0.7.1              pyhd8ed1ab_0    conda-forge
parmed                    3.4.4           py310heca2aa9_0    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
pbr                       5.11.1             pyhd8ed1ab_0    conda-forge
pcre2                     10.40                hc3806b6_0    conda-forge
pdb4amber                 22.0                     pypi_0    pypi
pdbfixer                  1.8.1              pyh6c4a22f_0    conda-forge
perl                      5.32.1          2_h7f98852_perl5    conda-forge
pexpect                   4.8.0              pyh1a96a4e_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    9.4.0           py310h023d228_1    conda-forge
pint                      0.20.1             pyhd8ed1ab_0    conda-forge
pip                       23.1               pyhd8ed1ab_0    conda-forge
pixman                    0.40.0               h36c2ea0_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_0    conda-forge
platformdirs              3.2.0              pyhd8ed1ab_0    conda-forge
plugcli                   0.1.0              pyhd8ed1ab_0    conda-forge
pooch                     1.7.0              pyha770c72_3    conda-forge
prometheus_client         0.16.0             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.38             pyha770c72_0    conda-forge
prompt_toolkit            3.0.38               hd8ed1ab_0    conda-forge
psutil                    5.9.5           py310h1fa729e_0    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
py3dmol                   2.0.1.post1              pypi_0    pypi
pycairo                   1.23.0          py310hb8a676c_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pydantic                  1.10.7          py310h1fa729e_0    conda-forge
pyedr                     0.7.1              pyhd8ed1ab_0    conda-forge
pygments                  2.15.0             pyhd8ed1ab_0    conda-forge
pymbar                    3.1.1           py310hde88566_2    conda-forge
pyopenssl                 23.1.1             pyhd8ed1ab_0    conda-forge
pyparsing                 3.0.9              pyhd8ed1ab_0    conda-forge
pyrsistent                0.19.3          py310h1fa729e_0    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
pytables                  3.7.0           py310hb60b9b2_3    conda-forge
python                    3.10.10         he550d4f_0_cpython    conda-forge
python-constraint         1.4.0                      py_0    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-fastjsonschema     2.16.3             pyhd8ed1ab_0    conda-forge
python-json-logger        2.0.7              pyhd8ed1ab_0    conda-forge
python-tzdata             2023.3             pyhd8ed1ab_0    conda-forge
python_abi                3.10                    3_cp310    conda-forge
pytraj                    2.0.6                    pypi_0    pypi
pytz                      2023.3             pyhd8ed1ab_0    conda-forge
pyyaml                    6.0             py310h5764c6d_5    conda-forge
pyzmq                     25.0.2          py310h059b190_0    conda-forge
rdkit                     2022.09.1       py310h10c98a6_0    conda-forge
readline                  8.2                  h8228510_1    conda-forge
reportlab                 3.6.12          py310h46e8714_2    conda-forge
requests                  2.28.2             pyhd8ed1ab_1    conda-forge
rfc3339-validator         0.1.4              pyhd8ed1ab_0    conda-forge
rfc3986-validator         0.1.1              pyh9f0ad1d_0    conda-forge
sander                    22.0                     pypi_0    pypi
scipy                     1.10.1          py310h8deb116_0    conda-forge
send2trash                1.8.0              pyhd8ed1ab_0    conda-forge
setuptools                67.6.1             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
smirnoff99frosst          1.1.0              pyh44b312d_0    conda-forge
snappy                    1.1.10               h9fff704_0    conda-forge
sniffio                   1.3.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.3.2.post1        pyhd8ed1ab_0    conda-forge
sqlalchemy                2.0.9           py310h1fa729e_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
terminado                 0.17.1             pyh41d4057_0    conda-forge
tinycss2                  1.2.1              pyhd8ed1ab_0    conda-forge
tinydb                    4.7.1              pyhd8ed1ab_0    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
tornado                   6.3             py310h1fa729e_0    conda-forge
tqdm                      4.65.0             pyhd8ed1ab_1    conda-forge
traitlets                 5.9.0              pyhd8ed1ab_0    conda-forge
typing-extensions         4.5.0                hd8ed1ab_0    conda-forge
typing_extensions         4.5.0              pyha770c72_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
unicodedata2              15.0.0          py310h5764c6d_0    conda-forge
urllib3                   1.26.15            pyhd8ed1ab_0    conda-forge
validators                0.20.0             pyhd8ed1ab_0    conda-forge
wcwidth                   0.2.6              pyhd8ed1ab_0    conda-forge
webencodings              0.5.1                      py_1    conda-forge
websocket-client          1.5.1              pyhd8ed1ab_0    conda-forge
wheel                     0.40.0             pyhd8ed1ab_0    conda-forge
widgetsnbextension        3.6.4              pyhd8ed1ab_0    conda-forge
xmltodict                 0.13.0             pyhd8ed1ab_0    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.0.10               h7f98852_0    conda-forge
xorg-libsm                1.2.3             hd9c2040_1000    conda-forge
xorg-libx11               1.8.4                h0b41bf4_0    conda-forge
xorg-libxau               1.0.9                h7f98852_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
xorg-libxrender           0.9.10            h7f98852_1003    conda-forge
xorg-libxt                1.2.1                h7f98852_2    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
zeromq                    4.3.4                h9c3ff4c_1    conda-forge
zipp                      3.15.0             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               h166bdaf_4    conda-forge
zstd                      1.5.2                h3eb15da_6    conda-forge
@IAlibay IAlibay mentioned this issue Apr 18, 2023
Closed
@ijpulidos
Copy link
Contributor

Oh, I've been trying to get an example on how to reproduce this error because other users have also encountered it. Can you please share the scripts and data to reproduce this? It should help us test and debug behavior when we work on #672 . Thanks for reporting!

@ijpulidos ijpulidos mentioned this issue Apr 18, 2023
Merged
5 tasks
@ijpulidos
Copy link
Contributor

Oh, haha, I am blind, i see the code now. Sorry!

@IAlibay
Copy link
Contributor Author

IAlibay commented Apr 18, 2023 •
edited

As per a slack conversation with @ijpulidos, I tested also this on OpenMM 7.7 and it also fails there (i.e. it fails on both 8 and 7.7)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ijpulidos @IAlibay