Skip to content

Conversation

@qasimgulzar
Copy link
Contributor

@qasimgulzar qasimgulzar commented Apr 8, 2025

Description:

This pull request introduces several changes to improve database compatibility, enhance CI workflows, and update migrations for better schema handling. Key updates include adding support for MySQL and PostgreSQL in CI pipelines, modifying database-related settings to use environment variables, and introducing a migration script to handle database-specific column types.

CI Workflow Enhancements:

  • Updated .github/workflows/ci.yml to include MySQL and PostgreSQL services for testing, with health checks and connection settings. Added matrix testing for both databases.
  • Added a new workflow .github/workflows/migrations-checker-postgresql.yml to validate migrations on both MySQL and PostgreSQL. This ensures compatibility with both database engines.
  • Enhanced the CI pipeline to install database-specific dependencies (mysqlclient and psycopg2-binary) and handle coverage file renaming and merging for database-specific coverage artifacts.

Database Configuration Updates:

  • Modified test_settings.py to use environment variables for database configuration, enabling dynamic switching between SQLite, MySQL, and PostgreSQL.

Migrations Improvements:

  • Updated edx_when/migrations/0005_auto_20190911_1056.py to include a custom migration script for altering the rel_date column in the edx_when_datepolicy table. The script ensures proper data types are used for PostgreSQL (interval) and MySQL (BIGINT). [1] [2]

Miscellaneous:

  • Updated .coveragerc to refine the file omission pattern for migrations, ensuring consistent coverage reporting.
  • Bumped the version of the edx_when package from 2.5.1 to 2.5.2.

Testing instructions:

  1. install updated current branch
  2. Run migrations on fresh database instance
  3. It should work seamlessly with mysql and postgresql both.

@openedx-webhooks
Copy link

Thanks for the pull request, @qasimgulzar!

This repository is currently maintained by @openedx/openedx-unmaintained.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

  • If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
    • This process (including the steps you'll need to take) is documented here.
  • If it doesn't, simply proceed with the next step.
🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

  • Dependencies

    This PR must be merged before / after / at the same time as ...

  • Blockers

    This PR is waiting for OEP-1234 to be accepted.

  • Timeline information

    This PR must be merged by XX date because ...

  • Partner information

    This is for a course on edx.org.

  • Supporting documentation
  • Relevant Open edX discussion forum threads
🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details
Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

  • The size and impact of the changes that it introduces
  • The need for product review
  • Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

@openedx-webhooks openedx-webhooks added the open-source-contribution PR author is not from Axim or 2U label Apr 8, 2025
@github-project-automation github-project-automation bot moved this to Needs Triage in Contributions Apr 8, 2025
@qasimgulzar qasimgulzar changed the title fix: Update the migration to make it compatible with postgresql and m… fix: Update the migration to make it compatible with postgresql Apr 8, 2025
@qasimgulzar qasimgulzar force-pushed the qasim/fix/migrations branch from b60a59e to 871551b Compare April 8, 2025 15:11
@qasimgulzar
Copy link
Contributor Author

@ormsbee I added some guard rails here.

@qasimgulzar qasimgulzar force-pushed the qasim/fix/migrations branch 2 times, most recently from f39408c to 2e7625a Compare April 8, 2025 17:53
@qasimgulzar qasimgulzar force-pushed the qasim/fix/migrations branch from b21c0a1 to 3ea36df Compare April 8, 2025 18:58
@qasimgulzar
Copy link
Contributor Author

@ormsbee I added test matrix and migration validation under this PR.

@mphilbrick211 mphilbrick211 moved this from Needs Triage to Ready for Review in Contributions Apr 14, 2025
@mphilbrick211 mphilbrick211 requested a review from ormsbee April 14, 2025 21:26
Copy link
Contributor

@ormsbee ormsbee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also increment the patch version of edx-when here: https://github.com/openedx/edx-when/blob/master/edx_when/__init__.py

Comment on lines 6 to 33
def alter_rel_date_column(apps, schema_editor):
db_engine = connection.settings_dict['ENGINE']
with connection.cursor() as cursor:
if 'postgres' in db_engine:
cursor.execute(
'ALTER TABLE edx_when_datepolicy '
'ALTER COLUMN rel_date TYPE interval USING rel_date::interval;'
)
elif 'mysql' in db_engine:
cursor.execute(
'ALTER TABLE edx_when_datepolicy '
'MODIFY rel_date BIGINT;'
)


def reverse_alter_rel_date_column(apps, schema_editor):
db_engine = connection.settings_dict['ENGINE']
with connection.cursor() as cursor:
if 'postgres' in db_engine:
cursor.execute(
'ALTER TABLE edx_when_datepolicy '
'ALTER COLUMN rel_date TYPE varchar USING rel_date::varchar;'
)
elif 'mysql' in db_engine:
cursor.execute(
'ALTER TABLE edx_when_datepolicy '
'MODIFY rel_date VARCHAR(255);'
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please document in docstrings why these two functions are necessary at all? Won't Django's standard handling of DurationField create a bigint or interval as necessary?

@qasimgulzar qasimgulzar force-pushed the qasim/fix/migrations branch from a35e11d to 7fc5189 Compare April 27, 2025 14:07
@qasimgulzar qasimgulzar requested a review from ormsbee April 27, 2025 14:08
@ormsbee ormsbee merged commit f827c68 into openedx:master Apr 28, 2025
22 of 23 checks passed
@github-project-automation github-project-automation bot moved this from Ready for Review to Done in Contributions Apr 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

open-source-contribution PR author is not from Axim or 2U

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants