From 3934e8f1a7d4698f188c45a92e7ead310b932b9f Mon Sep 17 00:00:00 2001 From: Herbert Damker Date: Mon, 14 Apr 2025 23:09:00 +0200 Subject: [PATCH 01/22] Update README.md --- README.md | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 73f6e70..8036143 100644 --- a/README.md +++ b/README.md @@ -5,42 +5,47 @@ - + # {{repo_name}} > [!NOTE] -> What is this repository about and how to use it: +> What is this template repository about and how to use it: > -> * For codeowners of existing CAMARA repository as a sample how a CAMARA repository should look like, e.g. to update legacy repositories. Have a look specifically on the [code of the README.md](https://github.com/camaraproject/Template_API_Repository/blob/main/README.md?plain=1) for different variants and copy relevant parts into your repository. +> * For codeowners of existing CAMARA repository it is a sample how a CAMARA repository should look like, e.g. to update legacy repositories. Have a look specifically on the [code of the README.md](https://github.com/camaraproject/Template_API_Repository/blob/main/README.md?plain=1) for different variants and copy relevant parts into your repository. > * For CAMARA admins to create new (Sandbox) API repositories automated: Use the `setup-new-repo` workflow together with documentation in [templates/README.md](templates/README.md). > > The following is the template README for a new independent Sandbox API repository, other variants are within the commented code. +> +> This note must be deleted in newly created repositories + +--- + + Sandbox API Repository to describe, develop, document, and test the {{repo_name}} Service API(s). The repository does not yet belong to a CAMARA Sub Project. * API Repository [wiki page]({{repo_wiki_page}}) +--- + - +--- - @@ -49,7 +54,7 @@ Incubating API Repository to evolve and maintain the definitions and documentati Repository for xxx of the {{subproject_name}} Working Group" * Working Group [wiki home page]({{repo_wiki_page}}) -!! Update with concrete link + --> ## Scope From f8f5b4484d63d3863a395353eeeabc1f9829ad91 Mon Sep 17 00:00:00 2001 From: Herbert Damker Date: Mon, 14 Apr 2025 23:13:26 +0200 Subject: [PATCH 02/22] Update README.md --- README.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 8036143..db775ce 100644 --- a/README.md +++ b/README.md @@ -19,29 +19,27 @@ > * For codeowners of existing CAMARA repository it is a sample how a CAMARA repository should look like, e.g. to update legacy repositories. Have a look specifically on the [code of the README.md](https://github.com/camaraproject/Template_API_Repository/blob/main/README.md?plain=1) for different variants and copy relevant parts into your repository. > * For CAMARA admins to create new (Sandbox) API repositories automated: Use the `setup-new-repo` workflow together with documentation in [templates/README.md](templates/README.md). > -> The following is the template README for a new independent Sandbox API repository, other variants are within the commented code. +> The following is the template README for a new independent Sandbox repositories, Sandbox repositories within a Sub Project, and Incubated Repositories. Further variants are within the commented code. > > This note must be deleted in newly created repositories --- - - + Sandbox API Repository to describe, develop, document, and test the {{repo_name}} Service API(s). The repository does not yet belong to a CAMARA Sub Project. * API Repository [wiki page]({{repo_wiki_page}}) --- - - + Sandbox API Repository to describe, develop, document, and test the {{repo_name}} Service API(s) within the Sub Project [{{subproject_name}}]({{subproject_wiki_page}}) * API Repository [wiki page]({{repo_wiki_page}}) --- - + Incubating API Repository to evolve and maintain the definitions and documentation of {{repo_name}} Service API(s) within the Sub Project [{{subproject_name}}]({{subproject_wiki_page}}) From ce636b8aa0a516a459fc0191074436e32d07e3b8 Mon Sep 17 00:00:00 2001 From: Herbert Damker Date: Mon, 14 Apr 2025 23:15:31 +0200 Subject: [PATCH 03/22] Update initial-admin.md --- templates/issues/initial-admin.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/templates/issues/initial-admin.md b/templates/issues/initial-admin.md index f9def62..7b71b17 100644 --- a/templates/issues/initial-admin.md +++ b/templates/issues/initial-admin.md @@ -5,7 +5,7 @@ - [ ] Remove the content of /templates and the /.github/workflows/setup-new-repo.yml - [ ] Update the README - choose the right badge (default: Sandbox) - - choose the right variant of text for Independent Sandbox / Sandbox in Sub Project / ... + - choose the right variant for Independent Sandboxes / Sandboxes in Sub Project / ... - check the wiki links for correctness - [ ] Assign the initial issue for CODEWONERS to the appropriate codeowners - [ ] Update the Onboarding Tracker within APIBacklog From af2de0210b8643189d34532588e7e7816152816e Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 15 Apr 2025 09:24:47 +0200 Subject: [PATCH 04/22] Update setup-new-repo.yml adding cleanup step --- .github/workflows/setup-new-repo.yml | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/.github/workflows/setup-new-repo.yml b/.github/workflows/setup-new-repo.yml index 45856dd..f929b17 100644 --- a/.github/workflows/setup-new-repo.yml +++ b/.github/workflows/setup-new-repo.yml @@ -236,3 +236,27 @@ jobs: gh issue comment "$ADMIN_ISSUE_URL" \ --body "✅ Repository setup has been completed by automation. You may now proceed with the checklist." + - name: Cleanup setup artifacts + run: | + if gh api repos/$OWNER/$REPO_NAME/contents/.github/workflows/setup-new-repo.yml > /dev/null 2>&1; then + SHA=$(gh api repos/$OWNER/$REPO_NAME/contents/.github/workflows/setup-new-repo.yml | jq -r '.sha') + gh api repos/$OWNER/$REPO_NAME/contents/.github/workflows/setup-new-repo.yml \ + -X DELETE \ + -F message="Remove setup workflow after initialization" \ + -F sha="$SHA" + else + echo "No setup workflow found in new repo. Skipping deletion." + fi + + if gh api repos/$OWNER/$REPO_NAME/contents/templates > /dev/null 2>&1; then + for file in $(gh api repos/$OWNER/$REPO_NAME/contents/templates | jq -r '.[].path'); do + echo "Deleting $file" + sha=$(gh api repos/$OWNER/$REPO_NAME/contents/$file | jq -r '.sha') + gh api repos/$OWNER/$REPO_NAME/contents/$file \ + -X DELETE \ + -F message="Remove $file from templates folder" \ + -F sha="$sha" + done + else + echo "No templates folder found to clean up. Skipping." + fi From c0c98c1a1babb84a5e022b59d653406693205ecd Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 15 Apr 2025 09:31:23 +0200 Subject: [PATCH 05/22] Update setup-new-repo.yml --- .github/workflows/setup-new-repo.yml | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/.github/workflows/setup-new-repo.yml b/.github/workflows/setup-new-repo.yml index f929b17..2704ddb 100644 --- a/.github/workflows/setup-new-repo.yml +++ b/.github/workflows/setup-new-repo.yml @@ -41,7 +41,16 @@ jobs: - name: Set up GitHub CLI token with personal access token run: | echo "GH_TOKEN=${{ secrets.GH_REPO_CREATE_TOKEN }}" >> $GITHUB_ENV - echo "Token GH_REPO_CREATE_TOKEN ready for use in GitHub CLI" + if [ -z "${{ secrets.GH_REPO_CREATE_TOKEN }}" ]; then + echo "::error::GH_REPO_CREATE_TOKEN is not set. Please configure the secret in the 'repo-setup' environment." + exit 1 + fi + echo "Validating GH token with gh auth status..." + gh auth status || { + echo "::error::GitHub CLI authentication failed." + exit 1 + } + fi - name: Create new repository and set variables run: | @@ -236,7 +245,7 @@ jobs: gh issue comment "$ADMIN_ISSUE_URL" \ --body "✅ Repository setup has been completed by automation. You may now proceed with the checklist." - - name: Cleanup setup artifacts + - name: Cleanup setup artifacts from template repository run: | if gh api repos/$OWNER/$REPO_NAME/contents/.github/workflows/setup-new-repo.yml > /dev/null 2>&1; then SHA=$(gh api repos/$OWNER/$REPO_NAME/contents/.github/workflows/setup-new-repo.yml | jq -r '.sha') From de43198adbbc373e8fcbeb4460e2f99b87e32964 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 15 Apr 2025 09:46:14 +0200 Subject: [PATCH 06/22] Update setup-new-repo.yml --- .github/workflows/setup-new-repo.yml | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/.github/workflows/setup-new-repo.yml b/.github/workflows/setup-new-repo.yml index 2704ddb..b052604 100644 --- a/.github/workflows/setup-new-repo.yml +++ b/.github/workflows/setup-new-repo.yml @@ -57,15 +57,24 @@ jobs: REPO_NAME=${{ github.event.inputs.repo_name }} OWNER=$(echo '${{ github.repository }}' | cut -d'/' -f1) - echo "Checking if repository $OWNER/$REPO_NAME already exists..." - if gh api repos/$OWNER/$REPO_NAME > /dev/null 2>&1; then - echo "::error::Repository $OWNER/$REPO_NAME already exists. Exiting." - exit 1 - fi - echo "Creating new repository: https://github.com/$OWNER/$REPO_NAME" gh repo create "$OWNER/$REPO_NAME" --public --template "$OWNER/$(basename '${{ github.repository }}')" --confirm + # Wait until the repository is fully accessible + for i in {1..5}; do + if gh api repos/$OWNER/$REPO_NAME > /dev/null 2>&1; then + echo "::notice::Repository $OWNER/$REPO_NAME is now available." + break + else + echo "Waiting for repository $OWNER/$REPO_NAME to become available..." + sleep 4 + fi + done + + if ! gh api repos/$OWNER/$REPO_NAME > /dev/null 2>&1; then + echo "::warning::Repository $OWNER/$REPO_NAME did not become available after 5 attempts. Continuing anyway." + fi + echo "REPO_NAME=$REPO_NAME" >> $GITHUB_ENV echo "OWNER=$OWNER" >> $GITHUB_ENV echo "MAINTAINERS_TEAM=${REPO_NAME}_maintainers" >> $GITHUB_ENV From 2b12a1fb034a41d6c2bf99d8aa0f5951e0d7d395 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 15 Apr 2025 09:51:08 +0200 Subject: [PATCH 07/22] Update setup-new-repo.yml --- .github/workflows/setup-new-repo.yml | 6 ------ 1 file changed, 6 deletions(-) diff --git a/.github/workflows/setup-new-repo.yml b/.github/workflows/setup-new-repo.yml index b052604..b11bd3c 100644 --- a/.github/workflows/setup-new-repo.yml +++ b/.github/workflows/setup-new-repo.yml @@ -45,12 +45,6 @@ jobs: echo "::error::GH_REPO_CREATE_TOKEN is not set. Please configure the secret in the 'repo-setup' environment." exit 1 fi - echo "Validating GH token with gh auth status..." - gh auth status || { - echo "::error::GitHub CLI authentication failed." - exit 1 - } - fi - name: Create new repository and set variables run: | From 844f02b748e3a55c1da4943c627935e88715509b Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 15 Apr 2025 10:04:38 +0200 Subject: [PATCH 08/22] Update setup-new-repo.yml --- .github/workflows/setup-new-repo.yml | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/.github/workflows/setup-new-repo.yml b/.github/workflows/setup-new-repo.yml index b11bd3c..9b7b091 100644 --- a/.github/workflows/setup-new-repo.yml +++ b/.github/workflows/setup-new-repo.yml @@ -256,19 +256,26 @@ jobs: -X DELETE \ -F message="Remove setup workflow after initialization" \ -F sha="$SHA" + else + echo "::notice::Workflow file not found in $REPO_NAME — skipping deletion." else echo "No setup workflow found in new repo. Skipping deletion." fi - + if gh api repos/$OWNER/$REPO_NAME/contents/templates > /dev/null 2>&1; then for file in $(gh api repos/$OWNER/$REPO_NAME/contents/templates | jq -r '.[].path'); do - echo "Deleting $file" - sha=$(gh api repos/$OWNER/$REPO_NAME/contents/$file | jq -r '.sha') - gh api repos/$OWNER/$REPO_NAME/contents/$file \ - -X DELETE \ - -F message="Remove $file from templates folder" \ - -F sha="$sha" - done + if gh api repos/$OWNER/$REPO_NAME/contents/$file > /dev/null 2>&1; then + echo "Deleting $file" + sha=$(gh api repos/$OWNER/$REPO_NAME/contents/$file | jq -r '.sha') + gh api repos/$OWNER/$REPO_NAME/contents/$file \ + -X DELETE \ + -F message="Remove $file from templates folder" \ + -F sha="$sha" + echo "::notice::Deleted $file" + else + echo "::warning::File $file not found during cleanup. Skipping." + fi + done else echo "No templates folder found to clean up. Skipping." fi From 80d61c34e4b3ee82b3fa4d1ac6c5f67ba048cee9 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 15 Apr 2025 10:14:20 +0200 Subject: [PATCH 09/22] Update setup-new-repo.yml --- .github/workflows/setup-new-repo.yml | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/.github/workflows/setup-new-repo.yml b/.github/workflows/setup-new-repo.yml index 9b7b091..eb1e2a6 100644 --- a/.github/workflows/setup-new-repo.yml +++ b/.github/workflows/setup-new-repo.yml @@ -1,4 +1,21 @@ # .github/workflows/setup-new-repo.yml +# +# Prerequisites: +# - A GitHub environment named `repo-setup` must exist. +# - The environment must contain a secret `GH_REPO_CREATE_TOKEN`. +# - This token must be a fine-grained personal access token (FGPAT) with the following: +# • Repository access: Allow access to the template repo and target org repos. +# • Permissions: +# - Contents: Read and write +# - Issues: Read and write +# - Metadata: Read-only +# - Administration: Read and write (for repo settings, team setup) +# - Rulesets: Read and write +# - Members: Read-only (for user checks) +# - Actions: (optional) Read and write (for cleanup) +# +# See: https://github.com/settings/tokens for generating tokens +# name: Setup New Repository on: @@ -52,7 +69,8 @@ jobs: OWNER=$(echo '${{ github.repository }}' | cut -d'/' -f1) echo "Creating new repository: https://github.com/$OWNER/$REPO_NAME" - gh repo create "$OWNER/$REPO_NAME" --public --template "$OWNER/$(basename '${{ github.repository }}')" --confirm + # Using 'true' instead of deprecated '--confirm' to skip interactive prompt + gh repo create "$OWNER/$REPO_NAME" --public --template "$OWNER/$(basename '${{ github.repository }}')" true # Wait until the repository is fully accessible for i in {1..5}; do From 7970771f2424c8f118513c2dc0eb3a2834c3b6ec Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 15 Apr 2025 10:27:04 +0200 Subject: [PATCH 10/22] Update setup-new-repo.yml --- .github/workflows/setup-new-repo.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/setup-new-repo.yml b/.github/workflows/setup-new-repo.yml index eb1e2a6..b7d18a8 100644 --- a/.github/workflows/setup-new-repo.yml +++ b/.github/workflows/setup-new-repo.yml @@ -70,7 +70,7 @@ jobs: echo "Creating new repository: https://github.com/$OWNER/$REPO_NAME" # Using 'true' instead of deprecated '--confirm' to skip interactive prompt - gh repo create "$OWNER/$REPO_NAME" --public --template "$OWNER/$(basename '${{ github.repository }}')" true + gh repo create "$OWNER/$REPO_NAME" --public --template "$OWNER/$(basename '${{ github.repository }}')" # Wait until the repository is fully accessible for i in {1..5}; do From 2d2ba198b682b5dbb8d5175e29e26a2c3f815970 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 15 Apr 2025 10:31:42 +0200 Subject: [PATCH 11/22] Update setup-new-repo.yml --- .github/workflows/setup-new-repo.yml | 2 -- 1 file changed, 2 deletions(-) diff --git a/.github/workflows/setup-new-repo.yml b/.github/workflows/setup-new-repo.yml index b7d18a8..f6d618a 100644 --- a/.github/workflows/setup-new-repo.yml +++ b/.github/workflows/setup-new-repo.yml @@ -276,8 +276,6 @@ jobs: -F sha="$SHA" else echo "::notice::Workflow file not found in $REPO_NAME — skipping deletion." - else - echo "No setup workflow found in new repo. Skipping deletion." fi if gh api repos/$OWNER/$REPO_NAME/contents/templates > /dev/null 2>&1; then From 5168326ac6eed4d7f3241cb9b111b7bc3351d93d Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 15 Apr 2025 10:55:34 +0200 Subject: [PATCH 12/22] Update setup-new-repo.yml --- .github/workflows/setup-new-repo.yml | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/.github/workflows/setup-new-repo.yml b/.github/workflows/setup-new-repo.yml index f6d618a..1e45781 100644 --- a/.github/workflows/setup-new-repo.yml +++ b/.github/workflows/setup-new-repo.yml @@ -268,14 +268,23 @@ jobs: - name: Cleanup setup artifacts from template repository run: | - if gh api repos/$OWNER/$REPO_NAME/contents/.github/workflows/setup-new-repo.yml > /dev/null 2>&1; then - SHA=$(gh api repos/$OWNER/$REPO_NAME/contents/.github/workflows/setup-new-repo.yml | jq -r '.sha') - gh api repos/$OWNER/$REPO_NAME/contents/.github/workflows/setup-new-repo.yml \ - -X DELETE \ - -F message="Remove setup workflow after initialization" \ - -F sha="$SHA" + echo "Checking if workflow file exists in $OWNER/$REPO_NAME" + WORKFLOW_RESPONSE=$(gh api repos/$OWNER/$REPO_NAME/contents/.github/workflows/setup-new-repo.yml 2>&1) + if echo "$WORKFLOW_RESPONSE" | jq . > /dev/null 2>&1; then + echo "::notice::Workflow file found. Proceeding with deletion." + SHA=$(echo "$WORKFLOW_RESPONSE" | jq -r '.sha') + + if [ "$SHA" = "null" ] || [ -z "$SHA" ]; then + echo "::warning::SHA not found in response. Skipping deletion." + else + gh api repos/$OWNER/$REPO_NAME/contents/.github/workflows/setup-new-repo.yml \ + -X DELETE \ + -F message="Remove setup workflow after initialization" \ + -F sha="$SHA" || echo "::error::Failed to delete the workflow file." + fi else echo "::notice::Workflow file not found in $REPO_NAME — skipping deletion." + echo "Response from GitHub API: $WORKFLOW_RESPONSE" fi if gh api repos/$OWNER/$REPO_NAME/contents/templates > /dev/null 2>&1; then From 6794ffeab31e955060fdcf58a83aa8f7c2f99a69 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 15 Apr 2025 11:18:52 +0200 Subject: [PATCH 13/22] Update setup-new-repo.yml --- .github/workflows/setup-new-repo.yml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/.github/workflows/setup-new-repo.yml b/.github/workflows/setup-new-repo.yml index 1e45781..8c8d36a 100644 --- a/.github/workflows/setup-new-repo.yml +++ b/.github/workflows/setup-new-repo.yml @@ -269,7 +269,8 @@ jobs: - name: Cleanup setup artifacts from template repository run: | echo "Checking if workflow file exists in $OWNER/$REPO_NAME" - WORKFLOW_RESPONSE=$(gh api repos/$OWNER/$REPO_NAME/contents/.github/workflows/setup-new-repo.yml 2>&1) + WORKFLOW_FILE=".github/workflows/setup-new-repo.yml" + WORKFLOW_RESPONSE=$(gh api repos/$OWNER/$REPO_NAME/contents/$WORKFLOW_FILE 2>&1) if echo "$WORKFLOW_RESPONSE" | jq . > /dev/null 2>&1; then echo "::notice::Workflow file found. Proceeding with deletion." SHA=$(echo "$WORKFLOW_RESPONSE" | jq -r '.sha') @@ -277,7 +278,7 @@ jobs: if [ "$SHA" = "null" ] || [ -z "$SHA" ]; then echo "::warning::SHA not found in response. Skipping deletion." else - gh api repos/$OWNER/$REPO_NAME/contents/.github/workflows/setup-new-repo.yml \ + gh api repos/$OWNER/$REPO_NAME/contents/$WORKFLOW_FILE \ -X DELETE \ -F message="Remove setup workflow after initialization" \ -F sha="$SHA" || echo "::error::Failed to delete the workflow file." From 2504394cd93f359d41163ef79c880441635c6aa4 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 15 Apr 2025 12:26:08 +0200 Subject: [PATCH 14/22] Refactored clean up step --- .github/workflows/setup-new-repo.yml | 52 +++++++++------------------- templates/issues/initial-admin.md | 2 +- 2 files changed, 18 insertions(+), 36 deletions(-) diff --git a/.github/workflows/setup-new-repo.yml b/.github/workflows/setup-new-repo.yml index 8c8d36a..d618885 100644 --- a/.github/workflows/setup-new-repo.yml +++ b/.github/workflows/setup-new-repo.yml @@ -268,40 +268,22 @@ jobs: - name: Cleanup setup artifacts from template repository run: | - echo "Checking if workflow file exists in $OWNER/$REPO_NAME" - WORKFLOW_FILE=".github/workflows/setup-new-repo.yml" - WORKFLOW_RESPONSE=$(gh api repos/$OWNER/$REPO_NAME/contents/$WORKFLOW_FILE 2>&1) - if echo "$WORKFLOW_RESPONSE" | jq . > /dev/null 2>&1; then - echo "::notice::Workflow file found. Proceeding with deletion." - SHA=$(echo "$WORKFLOW_RESPONSE" | jq -r '.sha') - - if [ "$SHA" = "null" ] || [ -z "$SHA" ]; then - echo "::warning::SHA not found in response. Skipping deletion." - else - gh api repos/$OWNER/$REPO_NAME/contents/$WORKFLOW_FILE \ + FILES_TO_DELETE=( + "templates/CODEOWNERS" + "templates/issues/initial-admin.md" + "templates/issues/initial-codeowners.md" + "templates/CODEOWNERS" + ) + + for file in "${FILES_TO_DELETE[@]}"; do + if gh api repos/$OWNER/$REPO_NAME/contents/$file > /dev/null 2>&1; then + sha=$(gh api repos/$OWNER/$REPO_NAME/contents/$file | jq -r '.sha') + gh api repos/$OWNER/$REPO_NAME/contents/$file \ -X DELETE \ - -F message="Remove setup workflow after initialization" \ - -F sha="$SHA" || echo "::error::Failed to delete the workflow file." + -F message="Remove $file from template repository" \ + -F sha="$sha" || echo "::error::Failed to delete $file" + echo "::notice::Deleted $file" + else + echo "::warning::File $file not found during cleanup. Skipping." fi - else - echo "::notice::Workflow file not found in $REPO_NAME — skipping deletion." - echo "Response from GitHub API: $WORKFLOW_RESPONSE" - fi - - if gh api repos/$OWNER/$REPO_NAME/contents/templates > /dev/null 2>&1; then - for file in $(gh api repos/$OWNER/$REPO_NAME/contents/templates | jq -r '.[].path'); do - if gh api repos/$OWNER/$REPO_NAME/contents/$file > /dev/null 2>&1; then - echo "Deleting $file" - sha=$(gh api repos/$OWNER/$REPO_NAME/contents/$file | jq -r '.sha') - gh api repos/$OWNER/$REPO_NAME/contents/$file \ - -X DELETE \ - -F message="Remove $file from templates folder" \ - -F sha="$sha" - echo "::notice::Deleted $file" - else - echo "::warning::File $file not found during cleanup. Skipping." - fi - done - else - echo "No templates folder found to clean up. Skipping." - fi + done diff --git a/templates/issues/initial-admin.md b/templates/issues/initial-admin.md index 7b71b17..d175ef0 100644 --- a/templates/issues/initial-admin.md +++ b/templates/issues/initial-admin.md @@ -2,7 +2,7 @@ - [ ] Review repository settings - [ ] Check that the CODEOWNERS file is valid (initial codeowners have accepted the invitation) -- [ ] Remove the content of /templates and the /.github/workflows/setup-new-repo.yml +- [ ] Remove /.github/workflows/setup-new-repo.yml in the new repository (as long as it is copied into it) - [ ] Update the README - choose the right badge (default: Sandbox) - choose the right variant for Independent Sandboxes / Sandboxes in Sub Project / ... From b2f2a98cb1254f08f98f38dd9f9002f3461b5ddd Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 15 Apr 2025 12:27:10 +0200 Subject: [PATCH 15/22] Update prerrequisites --- .github/workflows/setup-new-repo.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/.github/workflows/setup-new-repo.yml b/.github/workflows/setup-new-repo.yml index d618885..2bc4292 100644 --- a/.github/workflows/setup-new-repo.yml +++ b/.github/workflows/setup-new-repo.yml @@ -12,7 +12,6 @@ # - Administration: Read and write (for repo settings, team setup) # - Rulesets: Read and write # - Members: Read-only (for user checks) -# - Actions: (optional) Read and write (for cleanup) # # See: https://github.com/settings/tokens for generating tokens # From c4b277d7af4c2817fd7beb342fbeb80f4e92e90d Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 15 Apr 2025 12:49:50 +0200 Subject: [PATCH 16/22] Renamed CODEOWNERS_TEMPLATE --- .github/workflows/setup-new-repo.yml | 4 +--- CODEOWNERS | 4 +++- templates/{CODEOWNERS => CODEOWNERS_TEMPLATE} | 0 3 files changed, 4 insertions(+), 4 deletions(-) rename templates/{CODEOWNERS => CODEOWNERS_TEMPLATE} (100%) diff --git a/.github/workflows/setup-new-repo.yml b/.github/workflows/setup-new-repo.yml index 2bc4292..788dcb2 100644 --- a/.github/workflows/setup-new-repo.yml +++ b/.github/workflows/setup-new-repo.yml @@ -10,8 +10,6 @@ # - Issues: Read and write # - Metadata: Read-only # - Administration: Read and write (for repo settings, team setup) -# - Rulesets: Read and write -# - Members: Read-only (for user checks) # # See: https://github.com/settings/tokens for generating tokens # @@ -214,7 +212,7 @@ jobs: - name: Update CODEOWNERS file run: | - sed "s|{{initial_codeowners}}|$CODEOWNERS_LIST|g" templates/CODEOWNERS > CODEOWNERS + sed "s|{{initial_codeowners}}|$CODEOWNERS_LIST|g" templates/CODEOWNERS_TEMPLATE > CODEOWNERS CODEOWNERS_SHA=$(gh api repos/$OWNER/$REPO_NAME/contents/CODEOWNERS | jq -r '.sha') gh api repos/$OWNER/$REPO_NAME/contents/CODEOWNERS \ diff --git a/CODEOWNERS b/CODEOWNERS index 9449381..4739da7 100644 --- a/CODEOWNERS +++ b/CODEOWNERS @@ -1,4 +1,6 @@ -# This file provides an overview of code owners in this repository. +# This file provides an overview of code owners in this repository + +# Note: this is the actual CODEOWNERS file to protect the template repository. The CODEONWERS file in newly created repositories will be build based on /templates/CODEOWNERS_TEMPLATE # Each line is a file pattern followed by one or more owners. # The last matching pattern has the most precedence. diff --git a/templates/CODEOWNERS b/templates/CODEOWNERS_TEMPLATE similarity index 100% rename from templates/CODEOWNERS rename to templates/CODEOWNERS_TEMPLATE From 76c8974293571a5fa0373998c0a16530c995f7a4 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 15 Apr 2025 12:52:26 +0200 Subject: [PATCH 17/22] More detailed repository description --- .github/workflows/setup-new-repo.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/setup-new-repo.yml b/.github/workflows/setup-new-repo.yml index 788dcb2..e11927a 100644 --- a/.github/workflows/setup-new-repo.yml +++ b/.github/workflows/setup-new-repo.yml @@ -145,7 +145,7 @@ jobs: - name: Configure repository settings run: | gh repo edit $OWNER/$REPO_NAME \ - --description "$REPO_NAME" \ + --description "Sandbox API Repository for $REPO_NAME API(s)" \ --homepage "${{ github.event.inputs.repo_wiki_page }}" \ --add-topic sandbox-api-repository gh api -X PATCH repos/$OWNER/$REPO_NAME \ From c644bd626483baf7882188d89f88a064343c67da Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 15 Apr 2025 14:06:54 +0200 Subject: [PATCH 18/22] Added comments within the workflow --- .github/workflows/setup-new-repo.yml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/.github/workflows/setup-new-repo.yml b/.github/workflows/setup-new-repo.yml index e11927a..c50e937 100644 --- a/.github/workflows/setup-new-repo.yml +++ b/.github/workflows/setup-new-repo.yml @@ -63,6 +63,7 @@ jobs: - name: Create new repository and set variables run: | REPO_NAME=${{ github.event.inputs.repo_name }} + # Extract the owner (user or organization) part of the current repository (before the slash) OWNER=$(echo '${{ github.repository }}' | cut -d'/' -f1) echo "Creating new repository: https://github.com/$OWNER/$REPO_NAME" @@ -94,6 +95,7 @@ jobs: run: | if gh api orgs/$OWNER/teams > /dev/null 2>&1; then if ! gh api orgs/$OWNER/teams/$MAINTAINERS_TEAM > /dev/null 2>&1; then + # Retrieve the numeric ID of the 'maintainers' team to use as the parent_team_id MAINTAINERS_PARENT_ID=$(gh api orgs/$OWNER/teams/maintainers | jq -r '.id') MAINTAINERS_PARENT_ID_NUM=$(echo "$MAINTAINERS_PARENT_ID" | grep -o '[0-9]*') echo "Debug: MAINTAINERS_PARENT_ID_NUM=$MAINTAINERS_PARENT_ID_NUM" @@ -163,6 +165,7 @@ jobs: sed -i "s|{{mailinglist_name}}|${{ github.event.inputs.mailinglist_name }}|g" README.md sed -i "s|{{initial_codeowners}}|${{ github.event.inputs.initial_codeowners }}|g" README.md + # Retry loop: waits for README.md to appear in the new repo (max 5 attempts) SHA="" for i in {1..5}; do SHA=$(gh api repos/$OWNER/$REPO_NAME/contents/README.md 2>/dev/null | jq -r '.sha') @@ -227,6 +230,7 @@ jobs: TEMPLATE_REPO=$(basename "${{ github.repository }}") echo "Fetching rulesets from $OWNER/$TEMPLATE_REPO" + # Fetch all rulesets defined in the template repository for later replication RULESETS=$(gh api repos/$OWNER/$TEMPLATE_REPO/rulesets \ -H "Accept: application/vnd.github+json" 2>/dev/null || echo "[]") @@ -265,6 +269,7 @@ jobs: - name: Cleanup setup artifacts from template repository run: | + # Explicit list of files to remove after setup (e.g., templates and placeholders) FILES_TO_DELETE=( "templates/CODEOWNERS" "templates/issues/initial-admin.md" From 8e0d108f102e4d90759ff56f7cb7c3c63bcbb025 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 15 Apr 2025 14:07:09 +0200 Subject: [PATCH 19/22] Updated workflow documentation --- templates/README.md | 166 ++++++++++++++++++++++++++------------------ 1 file changed, 97 insertions(+), 69 deletions(-) diff --git a/templates/README.md b/templates/README.md index dbf836f..82e1624 100644 --- a/templates/README.md +++ b/templates/README.md @@ -1,110 +1,138 @@ -# 📘 Sandbox API Repository Creation Automation for CAMARA +# 📘 Automation for CAMARA API Repository Creation -This document describes how to use the GitHub Action in the `Template_API_Repository` to automate the setup of new API repositories within the CAMARA GitHub organization or within a personal namespace. +The automation workflow "Setup New Repository" simplifies the process of setting up a new CAMARA API repository, ensuring consistency across teams and reducing manual steps. + +This document describes how to use the GitHub Action in the `Template_API_Repository` to automate the setup of new API repositories within the [CAMARA GitHub organization](https://github.com/camaraproject). --- ## 🚀 Purpose -To automate the initial setup of a new repository, including: +To automate the initial administrative setup of a new API repository, ensuring consistency and reducing manual work. This includes: -- Creating a new repository from the template -- Setting metadata and repository settings -- Creating teams and assigning permissions (if in an organization) -- Adding CODEOWNERS based on a template -- Posting issues with initial checklists -- Cleaning up setup artifacts (workflow + templates) - not yet implemented +- Creating a new public repository based on the template +- Setting repo description, topics, and metadata +- Creating and configuring teams +- Assigning repository permissions +- Updating the README with project-specific details +- Adding CODEOWNERS +- Creating initial administrative issues +- Cleaning up template files from the new repository --- ## ⚙️ How to Use -1. Go to the **Actions** tab of your fork or the template repo -2. Trigger the **"Setup New Repository"** workflow manually using **workflow\_dispatch** -3. Fill in the following inputs: - -### 🔢 Inputs - -- `repo_name`: Name of the new repository to create -- `subproject_name`: Optional subproject/working group name -- `repo_wiki_page`: Link to the repository wiki -- `subproject_wiki_page`: Optional link to the subproject wiki -- `mailinglist_name`: Mailing list for project discussion -- `initial_codeowners`: Space-separated GitHub usernames with `@` (e.g., `@alice @bob`) +### 🛠 Prerequisites for Administrators ---- +Before running the workflow, ensure the following are set up: -## 📄 What It Does +1. **Environment Setup**: -### ✅ Repository + - A GitHub environment named `repo-setup` must exist in the template repository. -- Creates a new public repository from the template -- Sets description, homepage (to wiki), and topic `sandbox-api-repository` -- Enables issues and discussions, disables wiki +2. **Fine-Grained Personal Access Token (FGPAT)**: -### ✅ Teams + - A secret named `GH_REPO_CREATE_TOKEN` must be stored in the `repo-setup` environment. + - This token must be a **fine-grained personal access token** with access to both the template and target repositories. -- Creates `repo_name_maintainers` and `repo_name_codeowners` under parent teams (if applicable) -- Adds each codeowner to the CODEOWNERS team (if user exists) + #### 🔐 Required Token Permissions: -### ✅ Files + - **Contents**: Read and write + - **Issues**: Read and write + - **Metadata**: Read-only + - **Administration**: Read and write -- Replaces placeholders in `README.md` -- Generates CODEOWNERS from `templates/CODEOWNERS` -- Posts 2 issues using templates in `templates/issues/*.md` -- Adds a comment to the first issue confirming setup success + #### 🔧 How to Create a Fine-Grained Token: -### ✅ Rulesets + 1. Go to [GitHub Developer Settings – Tokens](https://github.com/settings/tokens) + 2. Click **"Generate new token (fine-grained)"** + 3. Set **Resource owner** to your GitHub username or organization + 4. Choose **repositories to access**: The template repo + all new target repos you’ll manage + 5. Under **Repository permissions**, set: + - Contents → Read and write + - Issues → Read and write + - Metadata → Read-only + - Administration → Read and write + - Generate the token and copy it + 6. Add it as the `GH_REPO_CREATE_TOKEN` secret under **Settings > Environments > repo-setup** -- Sync rulesets from template repository +--- -### ✅ Cleanup (currently not implemented) +### ▶️ Running the Workflow -- Deletes the workflow file itself from the new repo -- Deletes all files inside the `templates/` folder +1. Go to the `Actions` tab of [`Template_API_Repository`](https://github.com/camaraproject/Template_API_Repository/actions) +2. Select the **"Setup New Repository"** workflow +3. Use the `workflow_dispatch` form to input the following values: ---- +#### 🔤 Inputs -## 🔐 Requirements +| Input | Required | Description | +| ---------------------- | -------- | ------------------------------------------------------------------------------------------------------------ | +| `repo_name` | ✅ | Name of the new repository to create | +| `subproject_name` | ❌ | Optional subproject or working group name | +| `repo_wiki_page` | ✅ | URL of the new repository's wiki page | +| `subproject_wiki_page` | ❌ | URL of the subproject’s wiki page (if applicable) | +| `mailinglist_name` | ✅ | Mailing list associated with the repo | +| `initial_codeowners` | ✅ | Space-separated GitHub usernames (with `@`). Ensure each user exists and can be invited to the organization. | -### 🔑 GitHub Personal Access Token (PAT) +#### 💡 Example Input: -- Stored as repository or environment secret: `GH_REPO_CREATE_TOKEN` -- Must have scopes: - - `repo` - - `admin:org` (if using teams) +```text +repo_name: my-new-api +subproject_name: mobility +repo_wiki_page: https://github.com/camaraproject/my-new-api/wiki +subproject_wiki_page: https://github.com/camaraproject/mobility/wiki +mailinglist_name: mobility@lists.camaraproject.org +initial_codeowners: @alice @bob @charlie +``` -### 🛡 Environment Restrictions +--- -- The workflow is restricted to run in a GitHub Environment: `repo-setup` -- That environment must be configured in **Settings → Environments** -- Add the `@camaraproject/admins` team as required reviewers to control access -- Store the `GH_REPO_CREATE_TOKEN` secret inside this environment +## 🧱 What the Workflow Does + +This GitHub Actions workflow is defined in the `Template_API_Repository`, which serves as the starting point for all new API repositories in the CAMARA project. + +- Creates a new public repository using this template +- Waits until the repo is accessible +- Sets metadata: + - Description, homepage URL, topic `sandbox-api-repository` + - Enables discussions and issues; disables wiki +- Creates two teams: + - `repo_name_maintainers` (under `maintainers`) + - `repo_name_codeowners` (under `codeowners`) +- Assigns permissions: + - `triage` to maintainers + - `push` to codeowners + - `maintain` to admins +- Updates `README.md` using placeholder replacement +- Generates and commits a `CODEOWNERS` file from template +- Syncs all rulesets from the template repository +- Opens two issues: + - Initial administrative tasks + - Initial tasks for codeowners +- Deletes listed template files from the new repository --- -## 📦 Required Template Files - -The following must exist in the `Template_API_Repository`: +## ✅ What to Do After the Workflow -``` -templates/CODEOWNERS -templates/issues/initial-admin.md -templates/issues/initial-codeowners.md -``` +- Follow the instructions and checklist in the created issue "New Repository - Initial administrative tasks #1", especially: + - Review the newly created repository and ensure all README placeholders and metadata have been populated correctly + - Confirm that the CODEOWNERS file includes valid GitHub usernames. If users are not part of the organization, team invitations may not succeed + - Delete `.github/workflows/setup-new-repo.yml` manually from the new repository, as it is not removed automatically + - Note: Improvements of this checklist should go into `/templates/issues/initial-admin.md` --- -## 🧪 Testing Tips +## 📌 Future Enhancements -- Use a test repo name like `test-repo-$(date +%s)` to avoid naming collisions -- Run in a personal account if you want to skip org/team features -- Check your repo afterwards for: - - Metadata - - CODEOWNERS file - - Issue templates and comment - - Removed setup files +- Set the correct labels for issues and pull requests within the new repository +- Adding linting workflow (based on reusable workflow from `Tooling` repository) +- Validate usernames/org membership during CODEOWNERS setup +- Dry-run support for validation without creation +- Move the workflow out of the template repository into e.g. `.github` or `Tooling` repository (requires some rewriting of the workflow) --- -For questions, open an issue in the template repository or contact a CAMARA admin. +For questions or feedback, open an issue in [`Template_API_Repository`](https://github.com/camaraproject/Template_API_Repository/issues) or reach out via the mailing list [adm@lists.camaraproject.org](mailto\:adm@lists.camaraproject.org). From 54a53e5b95b23fbb14d2c0c153a8450e41ede0fe Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 15 Apr 2025 14:10:10 +0200 Subject: [PATCH 20/22] Some more comments and improvements of the workflow --- .github/workflows/setup-new-repo.yml | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/.github/workflows/setup-new-repo.yml b/.github/workflows/setup-new-repo.yml index c50e937..3db6a2b 100644 --- a/.github/workflows/setup-new-repo.yml +++ b/.github/workflows/setup-new-repo.yml @@ -52,10 +52,11 @@ jobs: - name: Checkout template repository uses: actions/checkout@v4 - - name: Set up GitHub CLI token with personal access token + - name: Set up GitHub CLI token with personal fine grained access token run: | + # Use GH_REPO_CREATE_TOKEN as the authentication token for all GitHub CLI commands echo "GH_TOKEN=${{ secrets.GH_REPO_CREATE_TOKEN }}" >> $GITHUB_ENV - if [ -z "${{ secrets.GH_REPO_CREATE_TOKEN }}" ]; then + if [ -z "$GH_TOKEN" ]; then echo "::error::GH_REPO_CREATE_TOKEN is not set. Please configure the secret in the 'repo-setup' environment." exit 1 fi @@ -67,7 +68,7 @@ jobs: OWNER=$(echo '${{ github.repository }}' | cut -d'/' -f1) echo "Creating new repository: https://github.com/$OWNER/$REPO_NAME" - # Using 'true' instead of deprecated '--confirm' to skip interactive prompt + # Creates a new public repository using the current repository as a template gh repo create "$OWNER/$REPO_NAME" --public --template "$OWNER/$(basename '${{ github.repository }}')" # Wait until the repository is fully accessible @@ -130,6 +131,10 @@ jobs: fi CODEOWNERS_LIST=$(echo "$CODEOWNERS_LIST" | xargs) + # Loop through each provided GitHub username and invite them to the codeowners team + # Note: users who are not yet members of the organization will be invited to join + # and will need to accept the invitation before they are part of the team and the CODEOWNERS file get fully valid + echo "Inviting users to team $CODEOWNERS_TEAM: [$CODEOWNERS_LIST]" for username in $CODEOWNERS_LIST; do clean_user=$(echo "$username" | sed 's/^@//') echo "Checking if @$clean_user is a valid GitHub user..." @@ -157,7 +162,8 @@ jobs: - name: Update README.md placeholders run: | - # changes the README.md from template repository and updates the README.md in the new repository with it + # Replace placeholders in the template README.md and push the updated version to the new repository + # Note: the README.md file is expected to be in the root of the repository sed -i "s/{{repo_name}}/$REPO_NAME/g" README.md sed -i "s|{{repo_wiki_page}}|${{ github.event.inputs.repo_wiki_page }}|g" README.md sed -i "s|{{subproject_name}}|${{ github.event.inputs.subproject_name }}|g" README.md @@ -215,6 +221,9 @@ jobs: - name: Update CODEOWNERS file run: | + # Replace placeholder in CODEOWNERS template with actual list of codeowners + # Note: also users who are skipped during team invitation will be added to the CODEOWNERS file and + # might need to be corrected manually later and invited manually to the CODEOWNERS team sed "s|{{initial_codeowners}}|$CODEOWNERS_LIST|g" templates/CODEOWNERS_TEMPLATE > CODEOWNERS CODEOWNERS_SHA=$(gh api repos/$OWNER/$REPO_NAME/contents/CODEOWNERS | jq -r '.sha') @@ -274,7 +283,7 @@ jobs: "templates/CODEOWNERS" "templates/issues/initial-admin.md" "templates/issues/initial-codeowners.md" - "templates/CODEOWNERS" + "templates/README.md" ) for file in "${FILES_TO_DELETE[@]}"; do From d43d2c62c111702a6ebab398e165372645cbcce2 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 15 Apr 2025 14:57:03 +0200 Subject: [PATCH 21/22] Revert change of token check --- .github/workflows/setup-new-repo.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/setup-new-repo.yml b/.github/workflows/setup-new-repo.yml index 3db6a2b..4266942 100644 --- a/.github/workflows/setup-new-repo.yml +++ b/.github/workflows/setup-new-repo.yml @@ -56,7 +56,7 @@ jobs: run: | # Use GH_REPO_CREATE_TOKEN as the authentication token for all GitHub CLI commands echo "GH_TOKEN=${{ secrets.GH_REPO_CREATE_TOKEN }}" >> $GITHUB_ENV - if [ -z "$GH_TOKEN" ]; then + if [ -z "${{ secrets.GH_REPO_CREATE_TOKEN }}" ]; then echo "::error::GH_REPO_CREATE_TOKEN is not set. Please configure the secret in the 'repo-setup' environment." exit 1 fi From c3a0194baf30687f417532525b4727703c384967 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 15 Apr 2025 15:15:51 +0200 Subject: [PATCH 22/22] Update documentation regarding token creation --- templates/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/templates/README.md b/templates/README.md index 82e1624..26e8d03 100644 --- a/templates/README.md +++ b/templates/README.md @@ -45,10 +45,10 @@ Before running the workflow, ensure the following are set up: #### 🔧 How to Create a Fine-Grained Token: - 1. Go to [GitHub Developer Settings – Tokens](https://github.com/settings/tokens) + 1. Go to [GitHub Developer Settings – Fine-grained personal access tokens](https://github.com/settings/personal-access-tokens) 2. Click **"Generate new token (fine-grained)"** - 3. Set **Resource owner** to your GitHub username or organization - 4. Choose **repositories to access**: The template repo + all new target repos you’ll manage + 3. Set **Resource owner** to your GitHub username (for testing) or organization (camaraproject) + 4. Choose **repositories to access**: all repositories (otherwise newly created repositories are not covered) 5. Under **Repository permissions**, set: - Contents → Read and write - Issues → Read and write