Skip to content

Conversation

@arielr-lt
Copy link
Collaborator

@arielr-lt arielr-lt commented Jan 13, 2026

Body:

Summary

Adds Argo Workflows infrastructure to staging environment with an S3-to-Elasticsearch indexing workflow.

Changes

Argo Infrastructure

  • Workflow controller, server, RBAC in credreg-staging namespace
  • PostgreSQL persistence, Let's Encrypt ingress
  • Accessible at https://argo-staging.credentialengineregistry.org/workflows

S3 → ES Indexing Workflow.

  • API Endpoint: POST /workflows/index-all-s3-to-es - indexes all S3 JSON-LDs to Elasticsearch
  • Argo WorkflowTemplate: index-s3-to-es - orchestrates the indexing via Keycloak-authenticated API call
  • Auth: Uses Keycloak Client Credentials flow (service account)

New Files

File Purpose
app/api/v1/workflows.rb API endpoint
app/services/index_s3_graph_to_es.rb S3→ES indexing service
app/policies/workflow_policy.rb Admin authorization
lib/tasks/s3.rake Manual rake task
argo-workflow/index-s3-to-es-workflow-template.yaml Argo WorkflowTemplate
argo-workflow/argo-keycloak-credentials-secret.yaml Keycloak credentials

Contributes to #971

@arielr-lt arielr-lt self-assigned this Jan 13, 2026
@arielr-lt arielr-lt changed the title (#971) - add Argo Workflows manifests (#971) - add Argo Workflows manifests (staging) Jan 13, 2026
Ariel Rolfo added 2 commits January 13, 2026 20:11
- Add API endpoint POST /workflows/index-all-s3-to-es
- Add IndexS3GraphToEs service to index S3 objects to ES
- Add WorkflowPolicy for admin authorization
- Add Argo WorkflowTemplate with Keycloak auth
- Add rake task s3:index_all_to_es for manual runs
- Update README with workflow documentation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants