Run both load.py and main.py from config.json

[kshefchek]

Currently load.py is driven by config.json, but main.py requires setting the environment variables REDIS_HOST and REDIS_PORT

@YaphetKG is there any issue running both from the config.json? It would seem fine but I may be missing a docker gotcha

[YaphetKG]

Hi @kshefchek , I don't see a real problem with that, maybe one thing is that we would need to mount config.json as a volume when running in docker, probably default config won't work. Our current deployments rely on the Env variables, but we can easily adjust that. Having one config source sounds good.

[gaurav]

I think the redis host/port information is now present in both config.yaml and redis_config.yaml. Does NodeNorm still need env variables?

[YaphetKG]

@gaurav i don't think so , the Redis port and host are now configured via the redis_config.yaml file, config.yaml's entries are no longer used , for the webserver (https://github.com/TranslatorSRI/NodeNormalization/blob/52461e0940e618984452b3fdf8cc13698ec71390/node_normalizer/server.py#L32) and for the loader (https://github.com/TranslatorSRI/NodeNormalization/blob/52461e0940e618984452b3fdf8cc13698ec71390/node_normalizer/loader.py#L361) is where connections are initialized

[YaphetKG]

one thing to note also is that some of the config options are very verbose (things like if its a cluster vs standalone redis instance, and password etc...) on top of that we have managed to split each db (equivalent id -> main id , main id -> list of eq ids, main id ->bl types, main id -> information content etc... ) into single redis instances instead of putting it all in one single db , to help reduce the size of memory required if we had done so. In that case we would need a very long list of env vars (?) not sure if that would any easier than just changing and mounting the redis-config.yaml file