Skip to content

Automate ALN updates and test workbook generation#5616

Merged
LynnMHouston merged 20 commits into
mainfrom
lh/auto_aln_process
May 12, 2026
Merged

Automate ALN updates and test workbook generation#5616
LynnMHouston merged 20 commits into
mainfrom
lh/auto_aln_process

Conversation

@LynnMHouston
Copy link
Copy Markdown
Contributor

Related tickets

#5615

Description of changes

  • Adds automation for fetching Assistance Listing Numbers (ALNs) from
    the SAM.gov Assistance Listings API; introduces fetch_alns_from_sam.py to retrieve Active and Inactive
    ALNs and generate source CSVs.
  • Integrates ALN fetching into the schema build pipeline via the
    Makefile (make fetch_alns and make all).
  • Adds build_test_workbooks.py to automatically generate Cypress
    test workbooks using fixture data.
  • Introduces test_workbook_fixtures.json as a source of consistent
    test data.
  • Updates make all to include test workbook generation, eliminating
    the need for manual updates.

How to test

  1. Set your SAM.gov API key: export SAM_API_KEY="your-key" (or in
    PowerShell: $env:SAM_API_KEY="your-key")

  2. Run full pipeline: make all

  3. Verify:

    • ALNs are fetched from SAM.gov (look for "Fetched Active/Inactive
      page..." logs)
    • active-alns.csv and inactive-alns.csv are generated
    • cfda-lookup.csv is created via merge
    • Excel workbooks are rebuilt in output/excel/xlsx/
    • Cypress test workbooks are generated in
      cypress/fixtures/test_workbooks/
  4. Optional:

    • Open a generated workbook and confirm expected structure and
      validations
    • Confirm test workbooks contain fixture data

Comment thread backend/schemas/scripts/build_test_workbooks.py Outdated
Comment thread backend/schemas/scripts/build_test_workbooks.py Outdated
Comment thread backend/schemas/scripts/fetch_alns_from_sam.py Outdated
…as new script to automate version number increase
Comment thread backend/venv310/Scripts/jwk_create.py Fixed
…as well as new script to automate version number increase"

This reverts commit fbe6aef.
Comment thread backend/schemas/Makefile
Comment thread backend/schemas/README.md Outdated
Comment thread backend/schemas/Makefile Outdated
Comment thread backend/schemas/source/data/test_workbook_fixtures.json Outdated
Comment thread backend/audit/intakelib/checks/check_version_number.py Outdated
…version found or missing fields/bad json, and SAM key example
@LynnMHouston LynnMHouston self-assigned this May 11, 2026
@LynnMHouston LynnMHouston marked this pull request as ready for review May 11, 2026 17:45
@github-actions
Copy link
Copy Markdown
Contributor

Code Coverage

Package Line Rate Branch Rate Health
. 100% 100%
api 98% 86%
api.serializers 97% 88%
api.views 91% 96%
audit 95% 80%
audit.cross_validation 97% 86%
audit.fixtures 84% 50%
audit.formlib 92% 62%
audit.intakelib 89% 83%
audit.intakelib.checks 92% 86%
audit.intakelib.common 98% 82%
audit.intakelib.transforms 100% 95%
audit.management.commands 78% 17%
audit.migrations 100% 100%
audit.models 91% 69%
audit.templatetags 100% 100%
audit.test_viewlib 100% 100%
audit.views 75% 52%
census_historical_migration 96% 65%
census_historical_migration.migrations 100% 100%
census_historical_migration.sac_general_lib 92% 84%
census_historical_migration.transforms 95% 90%
census_historical_migration.workbooklib 68% 69%
config 78% 37%
curation 94% 86%
curation.curationlib 91% 71%
curation.management.commands 46% 34%
curation.migrations 100% 100%
dissemination 90% 70%
dissemination.analytics 27% 0%
dissemination.forms 80% 30%
dissemination.migrations 97% 25%
dissemination.models 100% 100%
dissemination.report_generation 21% 0%
dissemination.report_generation.excel 32% 0%
dissemination.searchlib 61% 44%
dissemination.templatetags 52% 6%
dissemination.views 67% 47%
djangooidc 53% 38%
djangooidc.tests 100% 94%
report_submission 100% 96%
report_submission.migrations 100% 100%
report_submission.templatetags 74% 100%
report_submission.views 78% 61%
support 94% 75%
support.migrations 100% 100%
support.models 90% 50%
tools 98% 50%
users 95% 86%
users.fixtures 100% 83%
users.management 100% 100%
users.management.commands 100% 100%
users.migrations 100% 100%
Summary 88% (22532 / 25479) 69% (2741 / 3994)

Minimum allowed line rate is 85%

@LynnMHouston LynnMHouston added this pull request to the merge queue May 12, 2026
Merged via the queue into main with commit f3cb2b9 May 12, 2026
17 checks passed
@LynnMHouston LynnMHouston deleted the lh/auto_aln_process branch May 12, 2026 12:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants