Agenda: Milestone 4 - Meeting Minutes

[doctorbanu]

Meeting 18

Date: November 19, 2025 (Wednesday, 1:00 PM EST)
Attendees: Amro, Aseel, Banu, Caesar, Reem, Safia

Summary

• The research question was refined with Evan's help:
  • During the group conversation with Evan on Slack, team initially drafted a precise question focusing on whether optimized open-source models (e.g., via recursive editing, distillation) could become environmentally and functionally viable alternatives to commercial models.
  • However, Evan advised that the ELO2 project should remain open-ended, shifting toward a broader guiding question:
    “How can we achieve similar results to large private models on smaller devices and with less power consumption?”
  • As a result, the final deliverable will be a comprehensive portfolio of experiments, benchmarks, comparisons, and promising directions—rather than a single definitive answer.
• Based on Evan’s feedback, the upcoming Google Form will include both commercial and open-source model responses.
• Initially, the plan was to pair small open-source SLMs with commercial models of similar sizes, but this was not feasible due to limited access. The team instead decided to use accessible commercial LLMs (ChatGPT, Claude, Gemini).
• All questions across all categories will be used in the Google form.
• Finalized model assignments:
  • Aseel → ChatGPT
  • Caesar → Claude Haiku 4.5
  • Amro → Gemini Pro 3
  • Banu → Gemini Fast (Flash 2.5)
  • Reem → Gemini Flash 2.5 Lite (via API/HuggingFace)

Action Plan

• Each member will generate responses for all question prompts using their assigned model.
• All responses must be uploaded to the shared document by tomorrow.
• In tomorrow’s meeting, the team will review all open-source and commercial model outputs and select the final answers to include in the Google Form.

---

Meeting 19

Date: November 20, 2025 (Thursday, 2:30 PM EST)
Attendees: Amro, Aseel, Caesar, Reem, Safia

Summary

• The team revisited the original plan for the evaluation form. Initially, all 21 questions across all task categories were intended to be included.
• However, because a 21-question survey would be too long for participants, the group agreed to select only two questions per category to keep the form manageable.
• During this selection process, the team decided to exclude the Retrieval/RAG category entirely, since its questions require factual lookup (dates, names, quantities), which does not align well with the survey’s goal of evaluating reasoning or generation quality.
• As a result, the form will include four categories—Reasoning, Summarization, Creative Writing, Paraphrasing—with two questions per category, for a total of 8 questions. (Selected Q&A's can be found here.)
• The group reaffirmed that each task category will be represented by one model pair, ensuring all models contribute to the study.
• The team decided to pair each open-source model with the closest commercial model for comparative evaluation.
• The final task–model pairings were confirmed as:
  • Reasoning: Gemma ↔ Claude Haiku 4.5
  • Summarization: LaMini ↔ Gemini Flash
  • Creative Writing: Mistral ↔ Gemini Pro 3
  • Paraphrasing: Qwen ↔ ChatGPT

Action Plan

• Add the selected model responses to the Google Form initially created by Banu and finalize the form.
• Submit the form to Evan for feedback, then incorporate any revisions.
• Publish the finalized form to the cohort group and collect responses until November 30.

---

Meeting 20

Date: November 25, 2025 (Tuesday, 2:30 PM EST)
Attendees: Amro, Caesar, Reem, Banu

Summary

• To discuss what is left while the form is still running, the team reviewed the remaining deliverables for both ELO2/Graduation requirements and the project itself.

  1. For ELO2 and graduation, the deliverables were revisited and confirmed as:

     • Repository
     • Presentation
     • 1000-word final testimonial (individual)
     • 1000-word ELO2 retrospective (individual)
     • Exit Survey (individual)

  2. For Green AI project’s final outputs, the deliverables were identified as:

     • Repository
     • Article
     • Presentation
     • Form analysis

• The team also discussed the expected article format and structure.

  • The article should narrate the project process by explaining motivations and the overall journey, with roughly 5–10% on initial ideas, most of the content focusing on the work done, and a concluding section with findings and potential future directions.

• Reem volunteered to create an infographic or visual summary that can be used for both the article and the presentation.

• It was also noted that on November 28 (Friday), support may be requested from Evan to help boost form participation through an announcement.

• Since the form was published later than anticipated, the team decided to close it on December 2nd instead of November 30th.

Action Plan

• Caesar and Reem to begin drafting the article for team review.
• Amro to work on repository updates and refine the main README draft prepared by Banu earlier.
• Reem to create an infographic using a visualization tool to summarize project results for the article, presentation, and Medium or similar platforms (after the form closes).
• The survey form will be closed on December 2nd, after which data analysis will begin.
• Banu, Safia, and Aseel to work on the presentation, building on the initial draft previously prepared by Banu.
• An announcement request may be sent to Evan on November 28 (Friday) to encourage more form responses.

---

Meeting 21

Date: December 2, 2025 (Tuesday, 12:30 PM EST)
Attendees: Caesar, Reem, Aseel

Summary

• The team discussed the status of the survey form and agreed to close it by the end of the day. After reviewing its current performance, they noted that most initial insights and demographics could already be observed through the Google Forms visualization tools, which provided a general overview of respondent characteristics and early trends.
• The group revisited the remaining requirements and clarified what still needs to be completed for the final deliverables. A key part of the discussion focused on how the results will be structured and presented in both the article and the visual summary. This included considering how to best translate the survey findings into a clear narrative and an accompanying infographic or visualization.
• The team confirmed that an additional meeting would be held on December 3rd to examine the survey results in more depth. During that session, they will identify any notable or unexpected findings that may require special emphasis or separate formats within the final outputs.

Action Plan

• Close the survey form by end of day on December 2.
• Begin outlining how survey results will be integrated into the article and visual materials.
• Continue exploring the data using Google Forms visualizations to prepare for deeper analysis.
• Meet again on December 3rd to review detailed results and determine standout findings or sections that require special formatting.

---