[Feature Request] add unit tests

[njzjz]

Summary

Add unit tests for the following methods or classes.

Detailed Description

• deepmd.cluster.slurm.get_resource
• deepmd.cluster.slurm._pad_zeros Fix the expanding logic of SLURM_JOB_NODELIST and add unit tests for parallel training. #913
• deepmd.cluster.slurm._expand_ids Fix the expanding logic of SLURM_JOB_NODELIST and add unit tests for parallel training. #913
• deepmd.cluster.slurm._expand_nodelist Fix the expanding logic of SLURM_JOB_NODELIST and add unit tests for parallel training. #913
• deepmd.descriptor.hybrid.DescrptHybrid pass ntypes to sub descriptors in the hybrid descriptor #2611
• deepmd.descriptor.se_a_ebd.DescrptSeAEbd Test se_a_ebd to increase test coverage #2353
• deepmd.descriptor.se_a_ef.DescrptSeAEf
• deepmd.entrypoints.config.valid_dir
• deepmd.entrypoints.config.load_systems
• deepmd.entrypoints.config.get_system_names
• deepmd.entrypoints.config.get_rcut
• deepmd.entrypoints.config.get_batch_size_rule
• deepmd.entrypoints.config.get_stop_batch
• deepmd.entrypoints.config.get_ntypes
• deepmd.entrypoints.config.get_max_density
• deepmd.entrypoints.config.suggest_sel
• deepmd.entrypoints.config.suggest_batch_size
• deepmd.entrypoints.config.suggest_decay
• deepmd.entrypoints.config.config
• deepmd.entrypoints.convert.convert add ut for dp convert-from  #2175
• deepmd.entrypoints.doc.doc_train_input
• deepmd.entrypoints.test.test fix dp test atomic polar; add UTs for dp test #2785
• deepmd.entrypoints.test.rmse fix dp test atomic polar; add UTs for dp test #2785
• deepmd.entrypoints.test.save_txt_file fix dp test atomic polar; add UTs for dp test #2785
• deepmd.entrypoints.test.test_ener fix dp test atomic polar; add UTs for dp test #2785
• deepmd.entrypoints.test.print_ener_sys_avg fix dp test atomic polar; add UTs for dp test #2785
• deepmd.entrypoints.test.run_test fix dp test atomic polar; add UTs for dp test #2785
• deepmd.entrypoints.test.test_wfc
• deepmd.entrypoints.test.print_wfc_sys_avg
• deepmd.entrypoints.test.test_polar fix dp test atomic polar; add UTs for dp test #2785
• deepmd.entrypoints.test.print_polar_sys_avg
• deepmd.entrypoints.test.test_dipole fix dp test atomic polar; add UTs for dp test #2785
• deepmd.entrypoints.test.print_dipole_sys_avg
• deepmd.entrypoints.train.create_done_queue
• deepmd.entrypoints.train.wait_done_queue
• deepmd.entrypoints.train.connect_done_queue
• deepmd.entrypoints.train.fill_done_queue
• deepmd.entrypoints.train.get_rcut
• deepmd.entrypoints.train.get_type_map
• deepmd.entrypoints.train.get_sel
• deepmd.entrypoints.transfer.convert_number
• deepmd.entrypoints.transfer.convert_matrix
• deepmd.fit.polar.PolarFittingLocFrame (removed in clean deprecated codes #2140)
• deepmd.infer.deep_polar.DeepGlobalPolar
• deepmd.infer.deep_wfc.DeepWFC
• deepmd.infer.model_devi.calc_model_devi_e
• deepmd.infer.model_devi._check_tmaps
• deepmd.infer.model_devi.make_model_devi Fix model-devi with mixed_type format #2433
• deepmd.loggers.loggers._MPIRankFilter
• deepmd.loggers.loggers._MPIFileStream
• deepmd.loggers.loggers._MPIHandler
• deepmd.loss.ener.EnerDipoleLoss
• deepmd.utils.compat._warning_input_v0_v1
• deepmd.utils.compat._warning_input_v1_v2
• deepmd.utils.convert.convert_13_to_20
• deepmd.utils.convert.convert_12_to_20
• deepmd.utils.convert.convert_pb_to_pbtxt add ut for dp convert-from  #2175
• deepmd.utils.convert.convert_pbtxt_to_pb add ut for dp convert-from  #2175
• deepmd.utils.convert.convert_dp12_to_dp13 add ut for dp convert-from  #2175
• deepmd.utils.convert.convert_dp13_to_dp20 add ut for dp convert-from  #2175
• deepmd.utils.weight_avg.weighted_average fix dp test atomic polar; add UTs for dp test #2785

[shishaochen]

While improvement comes in #913, these 3 functions seems have problem in unit tests for 100% code coverage.

• deepmd.cluster.slurm._pad_zeros
• deepmd.cluster.slurm._expand_ids
• deepmd.cluster.slurm._expand_nodelist

One reason is that both _expand_nodelist and _expand_ids use commas to separate tokens. Considering execution order, function _expand_ids has no chance to handle argument ids with any comma. Consequently, function _pad_zeros will never be called.

@denghuilu Do you have any document on the schema of environment variable SLURM_JOB_NODELIST? Maybe we can write a better parsing logic. All I can find is Slurm Environmental Variables and slurm.h.in.

[denghuilu]

While improvement comes in #913, these 3 functions seems have problem in unit tests for 100% code coverage.

• deepmd.cluster.slurm._pad_zeros
• deepmd.cluster.slurm._expand_ids
• deepmd.cluster.slurm._expand_nodelist

One reason is that both _expand_nodelist and _expand_ids use commas to separate tokens. Considering execution order, function _expand_ids has no chance to handle argument ids with any comma. Consequently, function _pad_zeros will never be called.

@denghuilu Do you have any document on the schema of environment variable SLURM_JOB_NODELIST? Maybe we can write a better parsing logic. All I can find is Slurm Environmental Variables and slurm.h.in.

I also found the possible definition at https://hpcc.umd.edu/hpcc/help/slurmenv.html. @amcadmus Any suggestions?

[njzjz]

See here: https://slurm.schedmd.com/faq.html

[shishaochen]

@njzjz @denghuilu The tiny package python-hostlist seems professional.

Now code coverages on both deepmd.cluster.local and deepmd.cluster.slurm reach 100% with help of #913.

[njzjz]

Close since most of the tasks have been done.