[ENH] Optimize `example_models` loading using `pickle`.

[daehyun99]

Is your feature request related to a problem? Please describe.

• pgmpy/utils/example_models

Save example models stored in BIF format as Python objects using the pickle library (or joblib).

This reduces the loading time for pgmpy's example models.
It also reduces the CI test execution time in GitHub Actions.

Thanks.

• Example script for measuring model load time

import pickle
import os
from pgmpy.utils import get_example_model
import time

start_time = time.time()
origin_model = get_example_model("diabetes")
end_time = time.time()

print("Model Loading Time(BIF)", end_time - start_time)

# Save as Pickle
file_pkl = f"{"model_name"}_test.pkl"
with open(file_pkl, 'wb') as f:
    pickle.dump(origin_model, f)

start_time = time.time()
with open(file_pkl, 'rb') as f:
    pkl_model = pickle.load(f)
end_time = time.time()

print("Model Loading Time(Python Object)", end_time - start_time)
print("Model Equal Check: ", origin_model.__eq__(pkl_model))

• Output

Model Loading Time(BIF) 55.59602880477905
Model Loading Time(Python Object) 0.01642322540283203
Model Equal Check:  True

Describe the solution you'd like

• Step:

1. Create TestPickleReader and TestPickleWriter classes in the pgmpy/test_readwrite/ path.
2. Create PickleReader and PickleWriter classes in the pgmpy/readwrite/ path.
3. Save the example model as a Python object (*.pkl) using pickle.
4. Modify the filenames path in the get_example_model function to the file path created in step 3.
5. Add code to the get_example_model function to load the Python object using PickleReader.
6. Check if the execution time of the get_example_model function has been reduced.

Describe alternatives you've considered

• pickle
• joblib

[daehyun99]

I think that maybe can be good_first_issue.

[ankurankan]

@daehyun99 What is the size of these pickle objects? If the size is too large, it would significantly increase the size of our package.

[daehyun99]

@ankurankan
I didn't think about the size of our package. It looks like big.

• Total File Size(pkl): 17173.55 KB
• Total File Size(origin): 1664.86 KB

<style> </style>

data	pkl(KB)	origin(KB)
asia	3.01	0.3
cancer	2.08	0.28
earthquake	2.09	0.26
sachs	6.5	1.87
survey	2.42	0.36
alarm	21.94	1.79
barley	1058.16	369.44
child	10.86	1.41
insurance	26.25	3.71
mildew	4313.6	191.11
water	124.51	25.01
hailfinder	59.08	8.86
hepar2	50.62	11.86
win95pts	41.95	4.16
andes	115.15	3.88
diabetes	3995.05	253.14
link	496.15	9.09
munin1	265.3	27.55
munin2	1241.41	135.89
munin3	1277.37	140.19
munin4	1388.65	145.95
pathfinder	942.12	119.04
pigs	270.24	5.9
munin	1394.73	135.46
ecoli70	9.33	10.67
magic-niab	8.96	10.3
magic-irri	13.17	15.37
arth150	20.07	23.04
M-bias	0.52	0.18
confounding	0.5	0.2
mediator	0.46	0.16
paths	0.86	0.59
Sebastiani_2005	2.48	2.23
Polzer_2012	2.52	1.87
Schipf_2010	0.67	0.33
Shrier_2008	1.35	1.17
Acid_1996	1	0.71
Thoemmes_2013	0.79	0.53
Kampen_2014	0.98	0.6
Didelez_2010	0.65	0.4

[ankurankan]

@daehyun99 This seems to make our package 10x in size, and I don't know if that would be a good idea. The other problem with using pickled objects would be that if we make any changes in any of the classes used for representing these models (for example, DiscreteBayesianNetwork or TabularCPD), pickle will fail to load the model. So, we will have to generate the pickle objects every time we make any changes in those classes.

I think a better idea may be to try to optimize our BIFReader implementation. I put that through ChatGPT, and there seems to be quite a bit of room for improvement. What do you think? If you agree, would you mind closing this issue and creating a new one on optimizing the class?

[daehyun99]

@ankurankan

@daehyun99 This seems to make our package 10x in size, and I don't know if that would be a good idea. The other problem with using pickled objects would be that if we make any changes in any of the classes used for representing these models (for example, DiscreteBayesianNetwork or TabularCPD), pickle will fail to load the model. So, we will have to generate the pickle objects every time we make any changes in those classes.

I agree that increasing the package size isn't ideal.

I think a better idea may be to try to optimize our BIFReader implementation. I put that through ChatGPT, and there seems to be quite a bit of room for improvement. What do you think? If you agree, would you mind closing this issue and creating a new one on optimizing the class?

I've taken a look at the BIFReader code, and it seems like there's room for improvement.
I'll open a new issue for this.

Thank you.