Fixed MC image version and bumped MinIO version

[ObadaS]

A brief description of the purpose of the changes contained in this PR.

As recommended here #1957 , I locked down the version of the MC image and updated the MinIO image at the same time

Issues this PR resolves

#1957

Checklist

• Code review by me
• Hand tested by me
• I'm proud of my work
• Code review by reviewer
• Hand tested by reviewer
• CircleCi tests are passing
• Ready to merge

[Didayolo]

CircleCI test:

=================================== FAILURES ===================================
_______________ TestTasks.test_task_shown_as_validated_properly ________________

self = <api.tests.test_tasks.TestTasks testMethod=test_task_shown_as_validated_properly>

    def test_task_shown_as_validated_properly(self):
        user = UserFactory(username='test')
        solution = SolutionFactory(md5="12345")
        task = TaskFactory(created_by=user, solutions=[solution])
        competition = CompetitionFactory(created_by=user)
        phase = PhaseFactory(competition=competition, tasks=[task])
        submission = SubmissionFactory(md5="12345", phase=phase, status=Submission.FINISHED)
        url = reverse('task-detail', kwargs={'pk': task.id})
        self.client.login(username=user.username, password='test')
    
        # task should be validated because we have a successful submission matching
        # our solution
        resp = self.client.get(url)
        assert resp.status_code == 200
>       assert resp.data["validated"]
E       AssertionError: assert False

src/apps/api/tests/test_tasks.py:25: AssertionError

---

Oh I guess it is related to:

• Removed a test checking for validated tasks since we removed the validation in https://github.com/codalab/codabench/pull/1963 #1966

But then how did the tests pass in master?

[ObadaS]

The patch that was made didn't pass the test
https://app.circleci.com/pipelines/github/codalab/codabench/2625/workflows/06f27df7-ed51-4f52-8c98-561c12f29fa2/jobs/4489

#1963

[Didayolo]

@ObadaS

Now I have this error when trying to run locally develop :

codabench-minio-1  | INFO: WARNING: MINIO_ACCESS_KEY and MINIO_SECRET_KEY are deprecated.
codabench-minio-1  |          Please use MINIO_ROOT_USER and MINIO_ROOT_PASSWORD
codabench-minio-1  | FATAL Unable to use the drive /export: drive not found: Invalid arguments specified

Problem 1 - WARNING

Change the variable names to follow recent MinIO versions. I think we need to update:

• docker-compose.yml
• .env
• Manual intervention for next upgrade

Problem 2 - FATAL

Unable to use the drive /export: drive not found: Invalid arguments specified

I don't know the reason for this, but when I revert the changes of this PR then it works again.

[Didayolo]

@ObadaS What do you think about this? I may try it locally (or something similar)

LLM-proposed solution

From me:

% cat var/minio/.minio.sys/format.json 
{"version":"1","format":"fs","id":"d53ea88f-e58f-4245-86b1-c1a520f47059","fs":{"version":"2"}}%

Answer:

That format":"fs" confirms your old MinIO used the deprecated filesystem backend. You’ll need a one-time migration to xl / xl-single using mc mirror. Here is a clean, copy-paste path.

Step-by-step migration (same machine)

1) Bring up a fresh MinIO (2025) with a NEW empty data dir

Add a second service (different ports + different volume) to your docker-compose.yml:

services:
  minio_new:
    image: minio/minio:RELEASE.2025-04-22T22-12-26Z
    command: server /data
    environment:
      MINIO_ROOT_USER: newroot
      MINIO_ROOT_PASSWORD: newroot-very-strong-pass
    volumes:
      - ./var/minio-new:/data        # <— brand-new empty folder
    ports:
      - "9100:9000"                  # S3 API
      - "9101:9001"                  # Web console

Keep your old MinIO (2020) running on its original volume ./var/minio and ports (e.g., 9000/9001).

Important: do not point the 2025 container at your old ./var/minio. The whole point is to copy into a clean xl/xl-single layout.

2) Set up mc aliases

# If you have mc installed locally:
mc alias set old http://localhost:9000 OLDACCESS OLDSECRET
mc alias set new http://localhost:9100 newroot newroot-very-strong-pass

# Sanity check
mc ls old
mc ls new

(Replace OLDACCESS/OLDSECRET with whatever you used via MINIO_ACCESS_KEY/MINIO_SECRET_KEY in your old compose.)

3) Mirror buckets (copy everything)

You can mirror all buckets in one go with the trailing slashes:

# First pass: bulk copy (no live following)
mc mirror --overwrite --preserve --watch=false old/ new/

If you want near-zero downtime, run a short “follow” phase while your apps are still writing to the old server:

# Optional: keep syncing changes until cutover
mc mirror --overwrite --preserve --watch old/ new/
# (Leave it running for a bit; CTRL+C when you’re ready to cut over)

4) Cut over

1. Stop apps briefly (to freeze writes to old).

2. Run one last, quick sync to catch final changes:

mc mirror --overwrite --preserve --watch=false old/ new/

3. Stop the old MinIO container.

4. Switch ports on the new service to 9000/9001 (or update your app endpoints/ENV).

5. Start/redeploy your apps against the new endpoint and credentials.

---

What gets copied vs. what does not

• ✅ Objects and bucket structure are fully copied by mc mirror.

• ⚠️ Users, policies, bucket notifications/replication, lifecycle rules are not copied by a plain mirror (they live in MinIO’s IAM/config, not in object data). If you had any:

  • List on old: mc admin user list old, mc admin policy list old, mc event list old/<bucket>, mc ilm ls old/<bucket>

  • Recreate on new with the corresponding mc admin user add, mc admin policy create/set, mc event add, mc ilm add commands.

  • If you only ever used the root user, you can skip this.

---

Common pitfalls to avoid

• Don’t try to “upgrade in place” from fs → it will keep failing on modern images.

• Use MINIO_ROOT_USER / MINIO_ROOT_PASSWORD on the new container (the old ACCESS/SECRET vars are deprecated).

• Keep ports distinct during the migration (e.g., old: 9000/9001, new: 9100/9101) to avoid alias confusion.

• Make sure the target directory is empty before starting the new MinIO, so it initializes properly as xl-single.

[Didayolo]

Note: if we don't have safe update instructions, we'll need to revert this PR, as we won't be able to upgrade the servers.

[ObadaS]

I think 'mc mirror' might be the only option we have, I read some other comments on it that says the same : minio/minio#15967.

Here is the official guide to migrate from the old FS to the new one. I think we are migrating FS not Gateway, so users should be choosing the FS migration steps when following the guide