Add InterRDF_s by VOD555 · Pull Request #70 · MDAnalysis/pmda

VOD555 · 2018-10-10T17:00:36Z

Fixes #60

Changes made in this Pull Request:

Add parallel version of InterRDF_s(in pmda.rdf)
Add test for InterRDF_s

PR Checklist

Tests?
Docs?
CHANGELOG updated?
Issue raised/referenced?

orbeckst · 2018-10-25T18:26:53Z

@VOD555 if Travis fails, please check. This time the reason was missing cython for MDA devel installations. I merged master and this might fix it but in general, say something if CI fails due to issues outside the PR so that this can be fixed and your PR be moved along. In general, no-one bothers reviewing if there's still a red CI fail mark because experienced PR authors are expected to fix all their failures on their own. Exceptions are "cries for help" along the lines "Please have a look, I need more eyes to find this issue or solve this problem."

orbeckst · 2018-10-26T00:24:50Z

For the linter error: use the same trick as in

pmda/pmda/rdf.py

Line 90 in a718250

# pylint: disable=redefined-builtin

codecov · 2018-10-27T02:07:36Z

Codecov Report

Merging #70 into master will increase coverage by 0.39%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #70      +/-   ##
==========================================
+ Coverage   96.75%   97.14%   +0.39%     
==========================================
  Files           8        8              
  Lines         431      491      +60     
  Branches       60       70      +10     
==========================================
+ Hits          417      477      +60     
  Misses          8        8              
  Partials        6        6

Impacted Files	Coverage Δ
pmda/rdf.py	`100% <100%> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aaa478c...60a000d. Read the comment docs.

orbeckst

Looking good. Minor comments

add to CHANGELOG
test with different n_blocks

pmda/rdf.py

orbeckst · 2018-10-27T22:23:23Z

pmda/test/test_rdf_s.py

+
+@pytest.fixture(scope='module')
+def rdf_s(u, sels):
+    return InterRDF_s(u, sels).run()


Can you test with different n_blocks?

I don't know if this is actually testing multiple blocks or not. We need to know that the "accumulator" is working with multiple blocks.

Should work with @pytest.mark.parametrize.

VOD555 · 2018-10-29T05:05:42Z

pmda/test/test_rdf_s.py

+    # should see same results from analysis.rdf.InterRDF_s
+    # and pmda.rdf.InterRDF_s
+    nrdf = rdf.InterRDF_s(u, sels).run()
+    prdf = InterRDF_s(u, sels).run(n_jobs=3)


@orbeckst Here is one that is testing multiple blocks. Maybe I should add one test for "n_blocks=1" case

VOD555 · 2018-10-29T06:14:22Z

@orbeckst An error is raised on the use of the get= keyword in the new version of dask(http://docs.dask.org/en/latest/changelog.html)

orbeckst · 2018-10-29T07:44:00Z

That happened quickly!

So we now HAVE to finish PR #66!

EDIT: "quickly" because PR #66 talked about updated to Dask 0.18 and now we have 0.20 and it's an error.

EDIT 2: Fix referenced PR: not #60 but #66

orbeckst

More testing, otherwise looking good.

orbeckst · 2018-11-01T21:11:44Z

pmda/test/test_rdf_s.py

+    # and pmda.rdf.InterRDF_s
+    nrdf = rdf.InterRDF_s(u, sels).run()
+    prdf = InterRDF_s(u, sels).run(n_blocks=n_blocks)
+    assert_almost_equal(nrdf.count[0][0][0], prdf.count[0][0][0])


Why compare just one element? Shouldn't

assert_almost_equal(nrdf.count, prdf.count)

work?

orbeckst · 2018-11-01T21:12:35Z

pmda/test/test_rdf_s.py

+
+
+@pytest.fixture(scope='module')
+def rdf_s(u, sels):


Test with different schedulers:

def rdf_s(u, sels, scheduler)

(The scheduler fixture is defined in conftest.py and should run this calculation with all dask schedulers.)

VOD555 · 2018-11-01T21:58:15Z

Gets error with distributed

=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.6.5, pytest-3.6.2, py-1.5.3, pluggy-0.6.0 -- /home/shujie/anaconda3/envs/py36/bin/python
cachedir: ../../.pytest_cache
rootdir: /home/shujie/pmda, inifile: setup.cfg
plugins: pep8-1.0.6, cov-2.5.1, hypothesis-3.66.1
collected 38 items                                                                                                                                                                                         

test_rdf_s.py::test_nbins PASSED                                                                                                                                                                     [  2%]
test_rdf_s.py::test_range PASSED                                                                                                                                                                     [  5%]
test_rdf_s.py::test_count_size[distributed-1] ERROR                                                                                                                                                  [  7%]
test_rdf_s.py::test_count[distributed-1] ERROR                                                                                                                                                       [ 10%]
test_rdf_s.py::test_double_run[distributed-1] ERROR                                                                                                                                                  [ 13%]
test_rdf_s.py::test_cdf[distributed-1] ERROR                                                                                                                                                         [ 15%]
test_rdf_s.py::test_reduce[distributed-1] ERROR                                                                                                                                                      [ 18%]
test_rdf_s.py::test_count_size[multiprocessing-1] PASSED                                                                                                                                             [ 21%]
test_rdf_s.py::test_count[multiprocessing-1] PASSED                                                                                                                                                  [ 23%]
test_rdf_s.py::test_double_run[multiprocessing-1] PASSED                                                                                                                                             [ 26%]
test_rdf_s.py::test_cdf[multiprocessing-1] PASSED                                                                                                                                                    [ 28%]
test_rdf_s.py::test_reduce[multiprocessing-1] PASSED                                                                                                                                                 [ 31%]
test_rdf_s.py::test_count_size[multiprocessing-2] PASSED                                                                                                                                             [ 34%]
test_rdf_s.py::test_count[multiprocessing-2] PASSED                                                                                                                                                  [ 36%]
test_rdf_s.py::test_double_run[multiprocessing-2] PASSED                                                                                                                                             [ 39%]
test_rdf_s.py::test_cdf[multiprocessing-2] PASSED                                                                                                                                                    [ 42%]
test_rdf_s.py::test_reduce[multiprocessing-2] PASSED                                                                                                                                                 [ 44%]
test_rdf_s.py::test_count_size[distributed-2] ERROR                                                                                                                                                  [ 47%]
test_rdf_s.py::test_count[distributed-2] ERROR                                                                                                                                                       [ 50%]
test_rdf_s.py::test_double_run[distributed-2] ERROR                                                                                                                                                  [ 52%]
test_rdf_s.py::test_cdf[distributed-2] ERROR                                                                                                                                                         [ 55%]
test_rdf_s.py::test_reduce[distributed-2] ERROR                                                                                                                                                      [ 57%]
test_rdf_s.py::test_count_size[single-threaded-2] PASSED                                                                                                                                             [ 60%]
test_rdf_s.py::test_count[single-threaded-2] PASSED                                                                                                                                                  [ 63%]
test_rdf_s.py::test_double_run[single-threaded-2] PASSED                                                                                                                                             [ 65%]
test_rdf_s.py::test_cdf[single-threaded-2] PASSED                                                                                                                                                    [ 68%]
test_rdf_s.py::test_reduce[single-threaded-2] PASSED                                                                                                                                                 [ 71%]
test_rdf_s.py::test_count_size[single-threaded-1] PASSED                                                                                                                                             [ 73%]
test_rdf_s.py::test_count[single-threaded-1] PASSED                                                                                                                                                  [ 76%]
test_rdf_s.py::test_double_run[single-threaded-1] PASSED                                                                                                                                             [ 78%]
test_rdf_s.py::test_cdf[single-threaded-1] PASSED                                                                                                                                                    [ 81%]
test_rdf_s.py::test_reduce[single-threaded-1] PASSED                                                                                                                                                 [ 84%]
test_rdf_s.py::test_same_result[1] PASSED                                                                                                                                                            [ 86%]
test_rdf_s.py::test_same_result[2] PASSED                                                                                                                                                            [ 89%]
test_rdf_s.py::test_same_result[3] PASSED                                                                                                                                                            [ 92%]
test_rdf_s.py::test_same_result[4] PASSED                                                                                                                                                            [ 94%]
test_rdf_s.py::test_density[True-13275.775528444701] PASSED                                                                                                                                          [ 97%]
test_rdf_s.py::test_density[False-0.021915460340071267] PASSED                                                                                                                                       [100%]

================================================================================================== ERRORS ==================================================================================================

VOD555 · 2018-11-01T22:40:59Z

pmda/rdf.py

+        super(InterRDF_s, self).__init__(u, atomgroups)
+
+        # List of pairs of AtomGroups
+        self.ags = ags


Ok, the failure is due to this line. distributed cannot pass atomgroups to blocks.

Seems related to PR #65 .

(Also, thanks for raising #79 )

VOD555 and others added 3 commits October 10, 2018 09:59

Add InterRDF_s

0ca2f0f

add docs and fix some pep8 failures

3582cba

Merge branch 'master' into pInterRDF_s

fc512db

VOD555 added 2 commits October 25, 2018 17:14

Merge branch 'master' of github.com:MDAnalysis/pmda into pInterRDF_s

16e369a

Merge branch 'pInterRDF_s' of github.com:VOD555/pmda into pInterRDF_s

a4d38f8

VOD555 added 3 commits October 25, 2018 17:43

fix pylint error

396b758

fix pep-8

e56c5f7

fix pep8 error in test_rdf_s.py

b17d737

change docs

ea65bc4

VOD555 requested a review from orbeckst October 27, 2018 03:13

VOD555 self-assigned this Oct 27, 2018

VOD555 added the enhancement label Oct 27, 2018

orbeckst requested changes Oct 27, 2018

View reviewed changes

edit format

9024223

VOD555 commented Oct 29, 2018

View reviewed changes

VOD555 added 3 commits October 28, 2018 22:10

add test for different n_blocks

df87438

Merge branch 'master' of github.com:MDAnalysis/pmda into pInterRDF_s

166d78f

add to CHANGELOG

9c49991

orbeckst mentioned this pull request Oct 29, 2018

replace unsupported get keyword with scheduler #48

Closed

VOD555 added 2 commits October 31, 2018 17:31

dask update

f384984

fix conflicts

9f738b6

orbeckst requested changes Nov 1, 2018

View reviewed changes

VOD555 added 2 commits November 1, 2018 14:19

add tests for different n_blocks

44b5549

add scheduler test

eea692f

add scheduler test for test_rdf

d238bb4

VOD555 commented Nov 1, 2018

View reviewed changes

VOD555 added 2 commits November 1, 2018 15:41

change method to pass information of ags

f09b024

fix pep8 error

60a000d

VOD555 mentioned this pull request Nov 1, 2018

distributed.client error with atomgroup or universe type self attribute #79

Open

orbeckst approved these changes Nov 2, 2018

View reviewed changes

orbeckst merged commit f3bbd13 into MDAnalysis:master Nov 2, 2018

Conversation

VOD555 commented Oct 10, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Checklist

Uh oh!

orbeckst commented Oct 25, 2018

Uh oh!

orbeckst commented Oct 26, 2018

Uh oh!

codecov bot commented Oct 27, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

orbeckst left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

VOD555 commented Oct 29, 2018

Uh oh!

orbeckst commented Oct 29, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

orbeckst left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

VOD555 commented Nov 1, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

VOD555 commented Oct 10, 2018 •

edited

Loading

codecov bot commented Oct 27, 2018 •

edited

Loading

orbeckst commented Oct 29, 2018 •

edited

Loading