Skip to content
This repository was archived by the owner on Apr 26, 2024. It is now read-only.

Add *experimental* config option to support multiple databases#6258

Closed
erikjohnston wants to merge 8 commits into
developfrom
erikj/datastore_config
Closed

Add *experimental* config option to support multiple databases#6258
erikjohnston wants to merge 8 commits into
developfrom
erikj/datastore_config

Conversation

@erikjohnston
Copy link
Copy Markdown
Member

Most of this is a refactor to remove the get_db_pool and get_db_conn functions from the homeserver god object.

Based on #6240 and #6245.

Commits independently reviewable.


The new options replace database config, and look like:

databases:
    main_db:
        name: psycopg2
        args:
            database: pg1
    state_db:
        name: psycopg2
        args:
            database: pg2

data_stores:
    main: main_db
    state: state_b

I'm not super happy about this config format, hence why its undocumented and experimental, but its useful to allow us to test. Once I've played around with it a bit more, possibly adding support having multiple state data stores, then it should be easy to change or stabilise.

@erikjohnston erikjohnston requested a review from a team October 25, 2019 15:43
@erikjohnston
Copy link
Copy Markdown
Member Author

erikjohnston commented Oct 25, 2019

I need to figure out what to do about mypy, as I don't think its feasible to fix all the errors there in one go (especially since it looks like it would also requiring documenting some of our deps)

I've disabled the follow import stuff for now. We weren't using it before and it seems a bit overly aggressive.

Copy link
Copy Markdown
Member

@richvdh richvdh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to take this out of the queue until some of the groundwork is in place (and the conflicts are fixed...)

This encapsulates config for a given database and is the way to get new
connections.
This is not currently hooked up to any configuration parameters.
This allows us to test if using multiple databases actually works.
@erikjohnston erikjohnston force-pushed the erikj/datastore_config branch from 4655f4c to 6d015de Compare December 5, 2019 13:19
@erikjohnston erikjohnston deleted the erikj/datastore_config branch January 9, 2020 15:47
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants