numpy.datetime64 precision cause dict indexing failure in index_data.py

[GeorgeGuo1202]

🐛 Bug Description

ns precision numpy.datetime64 is equal to second precision
however when doing indexing of a dict keys, it will cause failure

numpy.datetime64('2017-01-04T00:00:00.000000000')==numpy.datetime64('2017-01-04T00:00:00')
Out[24]: True
self.index_map[numpy.datetime64('2017-01-04T00:00:00.000000000')]
Out[25]: 1
self.index_map[numpy.datetime64('2017-01-04T00:00:00')]
Traceback (most recent call last):
File "F:\work\env\py311\Lib\site-packages\IPython\core\interactiveshell.py", line 3508, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 1, in
self.index_map[numpy.datetime64('2017-01-04T00:00:00')]
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: numpy.datetime64('2017-01-04T00:00:00')

it occurs at qlib/qlib/utils/index_data.py, line 157:
try:
return self.index_map[self._convert_type(item)]
except IndexError as index_e:
raise KeyError(f"{item} can't be found in {self}") from index_e

maybe I did something wrong, hoping for help

[SunsetWolf]

Can you describe, in detail, how this can be surfaced? This will help us to solve the problem.

[akazeakari]

Here is a code reproduction based on the issue description. Although the two numpy.datetime64 values are equal, the difference in precision causes a dict indexing failure.

The following code successfully reproduces the issue on Python 3.9.19, pyqlib 0.9.5.99, and numpy 1.23.5.

import numpy as np
from qlib.utils.index_data import Index, SingleData

index = Index([np.datetime64('2017-01-04T00:00:00.000000000'),
               np.datetime64('2017-01-05T00:00:00.000000000'),
               np.datetime64('2017-01-06T00:00:00.000000000')])

data = SingleData([1, 2, 3], index=index)

# print: True
print(np.datetime64('2017-01-04T00:00:00.000000000') == np.datetime64('2017-01-04T00:00:00'))

# print: 0
print(data.index.index_map[np.datetime64('2017-01-04T00:00:00.000000000')])

# False
print(data.index.index_map[np.datetime64('2017-01-04T00:00:00')])

# print: 0
print(data.index.index_map[np.datetime64(np.datetime64('2017-01-04T00:00:00'), 'ns')])