KeyError: 'state_dict'

[Hatoonli]

I get this error running the test file sh run_pipeline.sh test.vcf hg19 test-output --cuda

File ../selene_sdk/predict/model_predict.py", line 315, in init
trained_model["state_dict"], model)
KeyError: 'state_dict'

[EMossotto]

Same problem - looks like it's a problem with Selene and the following call:

class AnalyzeSequences(object):
    Score sequences and their variants using the predictions made
    by a trained model.

    Parameters
    ----------
    model : torch.nn.Module
        A sequence-based model architecture.
    trained_model_path : str
        The path to the weights file for a trained sequence-based model.
        Architecture must match `model`.
    sequence_length : int
        The length of sequences that the model is expecting.
    features : list(str)
        The names of the features that the model is predicting.
    batch_size : int, optional
        Default is 64. The size of the mini-batches to use.
    use_cuda : bool, optional
        Default is `False`. Specifies whether CUDA-enabled GPUs are available
        for torch to use.
    reference_sequence : class, optional
        Default is `selene_sdk.sequences.Genome`. The type of sequence on
        which this analysis will be performed. Please note that if you need
        to use variant effect prediction, you cannot only pass in the
        class--you must pass in the constructed `selene_sdk.sequences.Sequence`
        object with a particular sequence version (e.g. `Genome("hg19.fa")`).
        This version does NOT have to be the same sequence version that the
        model was trained on. That is, if the sequences in your variants file
        are hg19 but your model was trained on hg38 sequences, you should pass
        in hg19.

    Attributes
    ----------
    model : torch.nn.Module
        A sequence-based model that has already been trained.
    sequence_length : int
        The length of sequences that the model is expecting.
    batch_size : int
        The size of the mini-batches to use.
    features : list(str)
        The names of the features that the model is predicting.
    use_cuda : bool
        Specifies whether to use CUDA or not.
    reference_sequence : class
        The type of sequence on which this analysis will be performed.

    """

    def __init__(self,
                 model,
                 trained_model_path,
                 sequence_length,
                 features,
                 batch_size=64,
                 use_cuda=False,
                 reference_sequence=Genome):
        """
        Constructs a new `AnalyzeSequences` object.
        """
        trained_model = torch.load(
            trained_model_path,
            map_location=lambda storage, location: storage)

        self.model = load_model_from_state_dict(
            trained_model["state_dict"], model)
        self.model.eval()

In particular, the model that gets returned by load_model_from_state_dict in the Selene utils won't work.

    """
    Loads model weights that were saved to a file previously by `torch.save`.
    This is a helper function to reconcile state dict keys where a model was
    saved with/without torch.nn.DataParallel and now must be loaded
    without/with torch.nn.DataParallel.

    Parameters
    ----------
    state_dict : collections.OrderedDict
        The state of the model.
    model : torch.nn.Module
        The PyTorch model, a module composed of submodules.

    Returns
    -------
    torch.nn.Module \
        The model with weights loaded from the state dict.

    Raises
    ------
    ValueError
        If model state dict keys do not match the keys in `state_dict`.

    """
    model_keys = model.state_dict().keys()
    state_dict_keys = state_dict.keys()

    new_state_dict = OrderedDict()
    for (k1, k2) in zip(model_keys, state_dict_keys):
        value = state_dict[k2]
        if k1 == k2:
            new_state_dict[k2] = value
        elif ('module' in k1 and k2 in k1) \
                or ('module' in k2 and k1 in k2):
            new_state_dict[k1] = value
        else:
            raise ValueError("Model state dict keys do not match "
                             "the keys specified in `state_dict` input. "
                             "Cannot load state into the model:\n\n"
                             "\tExpected keys:\n\t{0}\n\n"
                             "\tKeys in the input state dict:\n\t{1}\n".format(
                                 model_keys, state_dict_keys))
    model.load_state_dict(new_state_dict)
    return model

Can you please advise on how to solve this problem?

[jzthree]

It seems that you are using an older version of selene. Can you try upgrading to selene>=0.5.0?

[EMossotto]

Thanks, still struggling to get selene 0.5 installed (looks like there are some conflicts with libggc when installing PyTorch<=1.9).

conda install -c bioconda selene-sdk=0.5.0
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: - 
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed                                                                                                                                                                                        

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versionsThe following specifications were found to be incompatible with your system:

  - feature:/linux-64::__glibc==2.31=0
  - feature:|@/linux-64::__glibc==2.31=0
  - selene-sdk=0.5.0 -> libgcc-ng[version='>=9.3.0'] -> __glibc[version='>=2.17']

Your installed version is: 2.31

I managed to get the test running by letting selene choose the PyTorch version.

conda create --name sei python=3.6
conda activate sei
conda install -c bioconda selene-sdk

Here an environment that works:

_libgcc_mutex             0.1                        main  
_openmp_mutex             4.5                       1_gnu  
blas                      1.0                         mkl  
ca-certificates           2022.3.29            h06a4308_0  
certifi                   2021.5.30        py36h06a4308_0  
cffi                      1.14.6           py36h400218f_0  
cudatoolkit               9.0                  h13b8566_0  
cudnn                     7.6.5                 cuda9.0_0  
cycler                    0.11.0             pyhd3eb1b0_0  
dbus                      1.13.18              hb2f20db_0  
docopt                    0.6.2            py36h06a4308_0  
expat                     2.4.4                h295c915_0  
fontconfig                2.13.1               h6c09931_0  
freetype                  2.11.0               h70c0345_0  
glib                      2.69.1               h4ff587b_1  
gst-plugins-base          1.14.0               h8213a91_2  
gstreamer                 1.14.0               h28cd5cc_2  
h5py                      2.10.0           py36hd6299e0_1  
hdf5                      1.10.6               hb1b8bf9_0  
icu                       58.2                 he6710b0_3  
intel-openmp              2022.0.1          h06a4308_3633  
joblib                    1.0.1              pyhd3eb1b0_0  
jpeg                      9d                   h7f8727e_0  
kiwisolver                1.3.1            py36h2531618_0  
lcms2                     2.12                 h3be6417_0  
ld_impl_linux-64          2.35.1               h7274673_9  
libffi                    3.3                  he6710b0_2  
libgcc-ng                 9.3.0               h5101ec6_17  
libgfortran-ng            7.5.0               ha8ba4b0_17  
libgfortran4              7.5.0               ha8ba4b0_17  
libgomp                   9.3.0               h5101ec6_17  
libpng                    1.6.37               hbc83047_0  
libstdcxx-ng              9.3.0               hd4cf53a_17  
libtiff                   4.2.0                h85742a9_0  
libuuid                   1.0.3                h7f8727e_2  
libwebp-base              1.2.2                h7f8727e_0  
libxcb                    1.14                 h7b6447c_0  
libxml2                   2.9.12               h03d6c58_0  
lz4-c                     1.9.3                h295c915_1  
matplotlib                3.2.2                         0  
matplotlib-base           3.2.2            py36hef1b27d_0  
mkl                       2018.0.3                      1  
mkl_fft                   1.0.6            py36h7dd41cf_0  
mkl_random                1.0.1            py36h4414c95_1  
nccl                      1.3.5                 cuda9.0_0  
ncurses                   6.3                  h7f8727e_2  
ninja                     1.10.2               h5e70eb0_2  
numpy                     1.15.1           py36h1d66e8a_0  
numpy-base                1.15.1           py36h81de0dd_0  
olefile                   0.46                     py36_0  
openjpeg                  2.4.0                h3ad879b_0  
openssl                   1.1.1n               h7f8727e_0  
pandas                    1.0.5            py36h0573a6f_0  
patsy                     0.5.1                    py36_0  
pcre                      8.45                 h295c915_0  
pillow                    8.3.1            py36h2c7a002_0  
pip                       21.2.2           py36h06a4308_0  
plotly                    5.6.0              pyhd3eb1b0_0  
pycparser                 2.21               pyhd3eb1b0_0  
pyfaidx                   0.6.4              pyh5e36f6f_0    bioconda
pyparsing                 3.0.4              pyhd3eb1b0_0  
pyqt                      5.9.2            py36h05f1152_2  
pytabix                   0.0.2            py36hdbcaa40_6    bioconda
python                    3.6.13               h12debd9_1  
python-dateutil           2.8.2              pyhd3eb1b0_0  
pytorch                   0.4.1            py36ha74772b_0  
pytz                      2021.3             pyhd3eb1b0_0  
pyyaml                    5.4.1            py36h27cfd23_1  
qt                        5.9.7                h5867ecd_1  
readline                  8.1.2                h7f8727e_1  
scikit-learn              0.23.2           py36h0573a6f_0  
scipy                     1.1.0            py36hfa4b5c9_1  
seaborn                   0.11.2             pyhd3eb1b0_0  
selene-sdk                0.4.5            py36h516909a_0    bioconda
setuptools                58.0.4           py36h06a4308_0  
sip                       4.19.8           py36hf484d3e_0  
six                       1.16.0             pyhd3eb1b0_1  
sqlite                    3.38.2               hc218d9a_0  
statsmodels               0.11.1           py36h7b6447c_0  
tbb                       2021.5.0             hd09550d_0  
tbb4py                    2021.3.0         py36hd09550d_0  
tenacity                  8.0.1            py36h06a4308_0  
threadpoolctl             2.2.0              pyh0d69192_0  
tk                        8.6.11               h1ccaba5_0  
torchvision               0.2.1                    py36_0  
tornado                   6.1              py36h27cfd23_0  
wheel                     0.37.1             pyhd3eb1b0_0  
xz                        5.2.5                h7b6447c_0  
yaml                      0.2.5                h7b6447c_0  
zlib                      1.2.11               h7f8727e_4  
zstd                      1.4.9                haebb681_0  
 

[jzthree]

OK thanks for updating what worked for you!