Skip to content

Sync changes from crfm helm into medhelm main#8

Closed
iulianigas wants to merge 90 commits intomainfrom
sync/upstream-helm-main
Closed

Sync changes from crfm helm into medhelm main#8
iulianigas wants to merge 90 commits intomainfrom
sync/upstream-helm-main

Conversation

@iulianigas
Copy link
Copy Markdown

@iulianigas iulianigas commented Apr 16, 2026

No description provided.

dependabot Bot and others added 30 commits March 3, 2026 19:46
@dependabot
Bump rollup from 4.55.2 to 4.59.0 in /helm-frontend (stanford-crfm#4094)
a10fcca 
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot
Bump the npm group in /helm-frontend with 2 updates (stanford-crfm#4093)
a92f286 
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@yifanmai
Add Mistral Large 3, Mistral Medium 3.1, Mistral Small 3.2, and Minis…
30cd74c 
…tral 3 (stanford-crfm#4098)
@yifanmai
Add GPT-5.4 (stanford-crfm#4099)
5e5c010
@yifanmai
Add model_deployment_generator decorator (stanford-crfm#4100)
c61a7ef
@dependabot
Bump the npm group in /helm-frontend with 2 updates (stanford-crfm#4097)
a3be634 
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@yifanmai
Update Arabic content generation and finance scenarios (stanford-crfm…
c5bf714 
…#4095)
@yifanmai
Temporary experimental changes to Arabic Enterprise (stanford-crfm#4102)
9de0cbb
@yifanmai
Remove the human-evaluation optional dependency (stanford-crfm#4101)
1e56c94
@yifanmai
Modularize model deployment generators (stanford-crfm#4103)
c69baf7
@dependabot
Bump cryptography from 44.0.1 to 46.0.5 (stanford-crfm#4105)
932245a 
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@yifanmai
Switch llama-4-scout-17b-16e-instruct to use Vertex AI instead of Tog…
0f124a6 
…ether (stanford-crfm#4104)
@yifanmai
Allow using OpenAI models without configuration (stanford-crfm#4107)
34ea961
@yifanmai
Use TogetherChatClient instead of TogetherClient for generated model …
aff1a1a 
…deployments (stanford-crfm#4106)
@yifanmai
Remove tokenization from AnthropicMessagesClient (stanford-crfm#4108)
6f4d25c
@yifanmai
Allow using Anthropic models without configuration (stanford-crfm#4109)
b80479f
@dependabot
Bump @types/node from 25.3.3 to 25.4.0 in /helm-frontend in the npm g…
83c92df 
…roup (stanford-crfm#4110)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@yifanmai
Rename GitHub Action workflows (stanford-crfm#4111)
91d2e5d
@yifanmai
Remove tokenization from OpenAIClient (stanford-crfm#4113)
785d256
@yifanmai
Remove completed TODOs in pyproject.toml (stanford-crfm#4112)
00e9ad4
@yifanmai
Allow using OpenRouter models without configuration (stanford-crfm#4114)
1178c79
@dependabot
Bump pyasn1 from 0.6.2 to 0.6.3 (stanford-crfm#4116)
e2db9e7 
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@yifanmai
Modularize tokenizer config generators (stanford-crfm#4117)
1e08ddf
@yifanmai
Add Mistral model deployments (stanford-crfm#4118)
d639d03
@yifanmai
Add Writer model deployments (stanford-crfm#4119)
92c0216
@yifanmai
Add model deployments for xAI (stanford-crfm#4120)
c8f6561
@dependabot
Bump @types/node from 25.4.0 to 25.5.0 in /helm-frontend in the npm g…
579f617 
…roup (stanford-crfm#4121)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@yifanmai
Add Arabic legal scenario (stanford-crfm#4122)
eefc96b
@yifanmai
Add metric for Arabic legal scenarios (stanford-crfm#4123)
7c2f30b
@dependabot
Bump flatted from 3.2.9 to 3.4.2 in /helm-frontend (stanford-crfm#4124)
5f21a0c 
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
yifanmai and others added 20 commits April 7, 2026 16:33
@yifanmai
Add arabic_legal_rag and arabic_legal_qa to Arabic Enterprise run ent…
0107488 
…ries
@yifanmai
Add metadata for Arabic scenarios (stanford-crfm#4170)
220fd69
@yifanmai
Add metadata to mmlu_clinical_afr_scenario and winogrande_afr_scenario (
1c451fb 
stanford-crfm#4184)
@yifanmai
Switch documentation to use gcloud cp instead of gcloud rsync (stanfo…
f842f19 
…rd-crfm#4188)
@yifanmai
Change Dependabot schedule interval to monthly (stanford-crfm#4190)
5fbdab8
@dependabot
Bump mkdocs-include-markdown-plugin from 4.0.0 to 7.1.8 (stanford-crf…
b4c9d73 
…m#4163)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot
Bump vite from 6.4.1 to 6.4.2 in /helm-frontend (stanford-crfm#4175)
5ba8a5b 
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@yifanmai
Add metadata for Ewok scenario (stanford-crfm#4179)
742c4e8
@dependabot
Bump cryptography from 46.0.6 to 46.0.7 (stanford-crfm#4180)
1071710 
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot
Bump gdown from 5.2.1 to 5.2.2 (stanford-crfm#4195)
5f5f975 
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@yifanmai
Add metadata for audio scenarios (stanford-crfm#4185)
db78ee4
@dependabot
Bump pytest from 7.2.2 to 9.0.3 (stanford-crfm#4194)
094d12b 
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@Chessing234 @claude
Fix _apply_output_mapping_pattern returning wrong match results (stan…
5dede25 
…ford-crfm#4192)

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@Chessing234 @claude
Add link to per_instance_stats.json on frontend (stanford-crfm#4166)
bfd9b96 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@Chessing234 @claude
Fix print_summary() showing relevance scores under COHERENCE (stanfor…
d4ba787 
…d-crfm#4183)

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@yifanmai
Add metadata for VHELM scenarios (stanford-crfm#4189)
10a23d2
@dependabot
Bump the npm group across 1 directory with 4 updates (stanford-crfm#4200
36e3c5c 
)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@github-actions @yifanmai
Build frontend (stanford-crfm#4148)
f8341fe 
Co-authored-by: yifanmai <185227+yifanmai@users.noreply.github.com>
@Chessing234
Fix run_executable returning 0 for failed compilations in CodeInsight…
83eabd3 
…s metric (stanford-crfm#4199)

Co-authored-by: Chessing234 <chessing234@users.noreply.github.com>
@yifanmai
Fix BBQ metrics in HELM Safety schema (stanford-crfm#4202)
3971bb1
@iulianigas iulianigas self-assigned this Apr 16, 2026
@iulianigas iulianigas force-pushed the sync/upstream-helm-main branch from 37f524f to a978225 Compare April 22, 2026 13:02
@iulianigas iulianigas closed this Apr 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@blidiselalin blidiselalin Awaiting requested review from blidiselalin

@chakravarthik27 chakravarthik27 Awaiting requested review from chakravarthik27

Assignees

@iulianigas iulianigas

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@iulianigas @yifanmai @aaabulkhair @Chessing234