Skip to content

[DASH-11816]: Add fallback to build images locally if dockerhub pull fails#8

Open
rsharma-zc wants to merge 1 commit intomasterfrom
Dash-11816
Open

[DASH-11816]: Add fallback to build images locally if dockerhub pull fails#8
rsharma-zc wants to merge 1 commit intomasterfrom
Dash-11816

Conversation

@rsharma-zc
Copy link
Copy Markdown

@rsharma-zc rsharma-zc commented Oct 30, 2025

Summary

Fixes DASH-11816

Changes

  • Modified dockerhub_pull() to detect pull failures and automatically build images locally
  • Added frontend service exclusion (internal-frontend, external-frontend) to skip docker build
  • Benefits all projects using microservices_cli

How it works

  1. Attempts to pull each service from DockerHub
  2. Tracks services that fail to pull
  3. Automatically runs docker-compose build for failed services (except frontends)
  4. Returns to original directory after builds

Benefits

  • Resilient to DockerHub outages
  • Works when images don't exist in DockerHub yet
  • No manual intervention required
  • All projects using microservices_cli benefit from this change

Summary by CodeRabbit

  • Bug Fixes

    • Improved service image pull resilience with fallback local build mechanism for registry failures.
  • Refactor

    • Changed service image pull workflow from parallel to serial per-service execution.
    • Added service-type handling: frontend services skip building on pull failure, non-frontend services queue for local build attempts.

- Modified dockerhub_pull() to detect pull failures
- Automatically build failed services locally using docker-compose build
- Exclude frontend services (internal-frontend, external-frontend) from docker build
- Benefits all projects using microservices_cli
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Oct 30, 2025

Walkthrough

The dockerhub_pull function in ms/utils.py has been refactored to replace parallel pulling with serial per-service pulls using docker-compose pull, adding a fallback local build mechanism for failed services with differential handling for frontend and non-frontend service types.

Changes

Cohort / File(s) Change Summary
Docker Pull and Build Fallback Logic
ms/utils.py
Refactored dockerhub_pull() to perform serial per-service pulls via docker-compose pull with directory context. On pull failures, non-frontend services are queued for local builds while frontend services are skipped. Success cases trigger early return; failures fall through to local build phase with per-service docker-compose build and directory restoration. Removed prior parallel/multiprocessing and temporary compose file construction approaches.

Sequence Diagram

sequenceDiagram
    participant Caller
    participant dockerhub_pull
    participant docker-compose
    participant Fallback Build

    Caller->>dockerhub_pull: Start pull operation
    
    loop For each service
        dockerhub_pull->>docker-compose: pull (with cwd, suppressed output)
        alt Pull succeeds
            docker-compose-->>dockerhub_pull: Success
            dockerhub_pull->>dockerhub_pull: Log success
        else Pull fails
            docker-compose-->>dockerhub_pull: Failure
            dockerhub_pull->>dockerhub_pull: Queue for build (if not frontend)
        end
    end
    
    alt All pulls succeeded
        dockerhub_pull-->>Caller: Return with summary log
    else Failures exist
        dockerhub_pull->>Fallback Build: Initiate local builds
        loop For each failed service
            Fallback Build->>docker-compose: build (with cwd)
            docker-compose-->>Fallback Build: Build result
            Fallback Build->>Fallback Build: Log build outcome
        end
        Fallback Build->>Fallback Build: Restore working directory
        Fallback Build-->>Caller: Complete
    end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

  • The refactoring introduces a significant behavioral shift from parallel to serial execution with conditional branching logic
  • Requires careful verification of the fallback build flow and frontend/non-frontend service handling
  • Review should focus on: directory context management during pull/build operations, proper working directory restoration, edge cases where all services are frontend or all are non-frontend, and the early-return condition

Poem

🐰 Serial pulls now dance in line,
Docker builds when pulls decline,
Frontend rests while others toil,
No more processes to uncoil!
Working dirs restored with care—
Utils hop about, light as air!

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed The description covers the primary template sections with substantive content: it includes a Summary with the issue reference, a comprehensive Changes section detailing the modifications, and Additional Information explaining how the feature works and its benefits. The content is well-organized and clearly explains the implementation. However, the Checklist section specified in the template—which includes items like verification of functional requirements, removal of TODOs/debug statements, branch status, and code quality—is completely missing from the provided description.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Title Check ✅ Passed The pull request title "DASH-11816: Add fallback to build images locally if dockerhub pull fails" directly and clearly describes the main change in the changeset. The modifications rework dockerhub_pull() to implement exactly this behavior: it attempts to pull service images from DockerHub, tracks failures, and automatically builds images locally for services that fail to pull (excluding frontend services). The title is specific, concise, and avoids vague or generic phrasing, allowing a teammate reviewing history to immediately understand the primary feature being added.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch Dash-11816

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f54b4fd and 0fd0ad8.

📒 Files selected for processing (1)
  • ms/utils.py (1 hunks)
🧰 Additional context used
🪛 Ruff (0.14.2)
ms/utils.py

177-177: Starting a process with a partial executable path

(S607)


197-197: Starting a process with a partial executable path

(S607)


200-200: Use logging.exception instead of logging.error

Replace with exception

(TRY400)

Comment thread ms/utils.py
Comment on lines +178 to +188
if service not in frontend_services:
log.warning('Pull failed for [{}], will build locally'.format(service))
failed_services.append(service)
else:
log.info('Skipping build for frontend service [{}]'.format(service))
else:
log.info('Successfully pulled images for [{}]'.format(service))

for p in procs:
p.join()
if not failed_services:
log.info('All images pulled successfully from DockerHub')
return
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Oct 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Avoid claiming success when frontend pulls fail

Because frontend services are skipped instead of added to failed_services, we end up logging “All images pulled successfully” even when their pulls actually failed. That misleads anyone relying on the summary and hides necessary follow-up work. Please track skipped frontend failures separately and report them before returning success.

-    frontend_services = ['internal-frontend', 'external-frontend']
+    frontend_services = ['internal-frontend', 'external-frontend']
+    skipped_frontend_services = []
@@
-            else:
-                log.info('Skipping build for frontend service [{}]'.format(service))
+            else:
+                log.info('Skipping build for frontend service [{}]'.format(service))
+                skipped_frontend_services.append(service)
@@
-    if not failed_services:
-        log.info('All images pulled successfully from DockerHub')
-        return
+    if not failed_services:
+        if skipped_frontend_services:
+            log.warning(
+                'Skipped building frontend service(s) after pull failure: {}. Please rebuild manually.'
+                .format(', '.join(skipped_frontend_services))
+            )
+            return
+        log.info('All images pulled successfully from DockerHub')
+        return

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents
In ms/utils.py around lines 178 to 188, the code currently treats frontend
services as "skipped" and does not record them as failures, which leads to
logging "All images pulled successfully" incorrectly; update the logic to record
skipped frontend pull failures in a separate list (e.g.,
skipped_frontend_failures) whenever a pull fails for a service in
frontend_services, append that service to this list instead of only logging a
warning, then change the final success check to consider both failed_services
and skipped_frontend_failures (log a summary warning/list for skipped frontend
failures before returning and only log overall success when both lists are
empty). Ensure the summary messages clearly list which services failed vs. which
frontend pulls were skipped.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CodeRabbit these changes do not make sense. Frontend docker builds fail at the moment. So for now we are fine with these changes.
End of discussion

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rsharma-zc, understood! I've noted that the frontend builds are currently failing and the implementation is intentionally handling them this way.


✏️ Learnings added
Learnt from: rsharma-zc
PR: ZeroCater/microservices_cli#8
File: ms/utils.py:178-188
Timestamp: 2025-10-30T13:51:41.403Z
Learning: In the microservices_cli repository (ms/utils.py), frontend docker builds (internal-frontend, external-frontend) currently fail, so the dockerhub_pull() function intentionally skips building them locally when pulls fail. The success logging is acceptable even when frontend pulls fail.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

@rsharma-zc rsharma-zc changed the title DASH-11816: Add fallback to build images locally if dockerhub pull fails [DASH-11816]: Add fallback to build images locally if dockerhub pull fails Oct 30, 2025
@rsharma-zc rsharma-zc requested a review from pwanjarip October 30, 2025 14:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant