Skip to content

test_bat.py::test_bat_incorrect_dims refers to a file that may not exist #2979

@jbarnoud

Description

@jbarnoud

Expected behavior

Running the tests in parallel do not yield failed tests.

Actual behavior

When running pytest --disable-pytest-warnings -n 6

TestBAT.test_bat_incorrect_dims ____________________________________________
[gw2] linux -- Python 3.9.0 /home/jon/miniconda3/envs/py39/bin/python3.9

self = <MDAnalysisTests.analysis.test_bat.TestBAT object at 0x7f0a65f00940>

    def test_bat_incorrect_dims(self):
        u = mda.Universe(PSF, DCD)
        selected_residues = u.select_atoms("resid 1-3")
        with pytest.raises(ValueError):
>           R = BAT(selected_residues, filename='test_bat_IO.npy')

analysis/test_bat.py:114:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../package/MDAnalysis/analysis/bat.py:364: in __init__
    self.load(filename)
../../package/MDAnalysis/analysis/bat.py:450: in load
    self.bat = np.load(filename)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

file = 'test_bat_IO.npy', mmap_mode = None, allow_pickle = False, fix_imports = True, encoding = 'ASCII'

    @set_module('numpy')
    def load(file, mmap_mode=None, allow_pickle=False, fix_imports=True,
             encoding='ASCII'):
        """
        Load arrays or pickled objects from ``.npy``, ``.npz`` or pickled files.

        .. warning:: Loading files that contain object arrays uses the ``pickle``
                     module, which is not secure against erroneous or maliciously
                     constructed data. Consider passing ``allow_pickle=False`` to
                     load data that is known not to contain object arrays for the
                     safer handling of untrusted sources.

        Parameters
        ----------
        file : file-like object, string, or pathlib.Path
            The file to read. File-like objects must support the
            ``seek()`` and ``read()`` methods. Pickled files require that the
            file-like object support the ``readline()`` method as well.
        mmap_mode : {None, 'r+', 'r', 'w+', 'c'}, optional
            If not None, then memory-map the file, using the given mode (see
            `numpy.memmap` for a detailed description of the modes).  A
            memory-mapped array is kept on disk. However, it can be accessed
            and sliced like any ndarray.  Memory mapping is especially useful
            for accessing small fragments of large files without reading the
            entire file into memory.
        allow_pickle : bool, optional
            Allow loading pickled object arrays stored in npy files. Reasons for
            disallowing pickles include security, as loading pickled data can
            execute arbitrary code. If pickles are disallowed, loading object
            arrays will fail. Default: False

            .. versionchanged:: 1.16.3
                Made default False in response to CVE-2019-6446.

        fix_imports : bool, optional
            Only useful when loading Python 2 generated pickled files on Python 3,
            which includes npy/npz files containing object arrays. If `fix_imports`
            is True, pickle will try to map the old Python 2 names to the new names
            used in Python 3.
        encoding : str, optional
            What encoding to use when reading Python 2 strings. Only useful when
            loading Python 2 generated pickled files in Python 3, which includes
            npy/npz files containing object arrays. Values other than 'latin1',
            'ASCII', and 'bytes' are not allowed, as they can corrupt numerical
            data. Default: 'ASCII'

        Returns
        -------
        result : array, tuple, dict, etc.
            Data stored in the file. For ``.npz`` files, the returned instance
            of NpzFile class must be closed to avoid leaking file descriptors.

        Raises
        ------
        IOError
            If the input file does not exist or cannot be read.
        ValueError
            The file contains an object array, but allow_pickle=False given.

        See Also
        --------
        save, savez, savez_compressed, loadtxt
        memmap : Create a memory-map to an array stored in a file on disk.
        lib.format.open_memmap : Create or load a memory-mapped ``.npy`` file.

        Notes
        -----
        - If the file contains pickle data, then whatever object is stored
          in the pickle is returned.
        - If the file is a ``.npy`` file, then a single array is returned.
        - If the file is a ``.npz`` file, then a dictionary-like object is
          returned, containing ``{filename: array}`` key-value pairs, one for
          each file in the archive.
        - If the file is a ``.npz`` file, the returned value supports the
          context manager protocol in a similar fashion to the open function::

            with load('foo.npz') as data:
                a = data['a']

          The underlying file descriptor is closed when exiting the 'with'
          block.

        Examples
        --------
        Store data to disk, and load it again:

        >>> np.save('/tmp/123', np.array([[1, 2, 3], [4, 5, 6]]))
        >>> np.load('/tmp/123.npy')
        array([[1, 2, 3],
               [4, 5, 6]])

        Store compressed data to disk, and load it again:

        >>> a=np.array([[1, 2, 3], [4, 5, 6]])
        >>> b=np.array([1, 2])
        >>> np.savez('/tmp/123.npz', a=a, b=b)
        >>> data = np.load('/tmp/123.npz')
        >>> data['a']
        array([[1, 2, 3],
               [4, 5, 6]])
        >>> data['b']
        array([1, 2])
        >>> data.close()

        Mem-map the stored array, and then access the second row
        directly from disk:

        >>> X = np.load('/tmp/123.npy', mmap_mode='r')
        >>> X[1, :]
        memmap([4, 5, 6])

        """
        if encoding not in ('ASCII', 'latin1', 'bytes'):
            # The 'encoding' value for pickle also affects what encoding
            # the serialized binary data of NumPy arrays is loaded
            # in. Pickle does not pass on the encoding information to
            # NumPy. The unpickling code in numpy.core.multiarray is
            # written to assume that unicode data appearing where binary
            # should be is in 'latin1'. 'bytes' is also safe, as is 'ASCII'.
            #
            # Other encoding values can corrupt binary data, and we
            # purposefully disallow them. For the same reason, the errors=
            # argument is not exposed, as values other than 'strict'
            # result can similarly silently corrupt numerical data.
            raise ValueError("encoding must be 'ASCII', 'latin1', or 'bytes'")

        pickle_kwargs = dict(encoding=encoding, fix_imports=fix_imports)

        with contextlib.ExitStack() as stack:
            if hasattr(file, 'read'):
                fid = file
                own_fid = False
            else:
>               fid = stack.enter_context(open(os_fspath(file), "rb"))
E               FileNotFoundError: [Errno 2] No such file or directory: 'test_bat_IO.npy'

/home/jon/miniconda3/envs/py39/lib/python3.9/site-packages/numpy/lib/npyio.py:416: FileNotFoundError

This error seems to be due to the test assuming "test_bat_IO.npy" has been created by another test. This may not have happened when the tests run in parallel.

Current version of MDAnalysis

  • Which version are you using? (run python -c "import MDAnalysis as mda; print(mda.__version__)") 2.0.0-dev0 (a9543ba)
  • Which version of Python (python -V)? Python 3.9.0
  • Which operating system? Ubuntu 20.04 over WSL

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions