This page is a reference guide only. Use it to identify who should grade each output, review the steps for evaluating a submission, and access the answer key. All scoring and feedback must be submitted through Canvas by EOD Wednesday following the Sunday deadline for ILs. If you have any questions about how to score an output, reach out to your technical lead (Kaitlyn Van Gorkom) on Slack!
Grading Guide Fields
Begin by reviewing your curriculum’s grading guide to identify who should grade each output, review the steps for evaluating a submission, and access the answer key.
Open this toggle to see an explanation of the fields included in the Grading Guide!
Title- The name of the assignment in Canvas.
Week- When the assignment was included in Canvas.
Canvas Assign To- Outputs will be assigned and differentiated by role and familiarity with curriculum. You are only responsible for reviewing the outputs of the roles present in your cohort. They may be assigned by:- Experience w/ Curriculum
- New Curriculum ILs: All new IAs, new instructors, and any returners who have never taught the curriculum as an instructor or IA in the past.
- Same Curriculum ILs: Any returning ILs that have taught the curriculum as an instructor or IA
- Role
- New IA
- New Instructor
- Returning Instructor
- Returning IA
Submission Type- The type of output you can expect to review.- Text Entry (HireFlix): Review the video using the HireFlix link in Airtable and read the IL’s reflection in Canvas.
- Text Entry: Review the written output directly in Canvas.
- Website URL: Review the IL’s code or work by following the link they submitted in Canvas.
- Media Recordings: Review the IL’s video directly in Canvas.
- File Uploads: Review the file uploaded by the IL in Canvas.
- Discussion: Review the IL’s contribution to the discussion using SpeedGrader in Canvas.
Grader- The reviewer is the person responsible for reviewing the output. In most instances, this will be you, Senior Instructors, but at some times may be assigned to the KWK team.
Answer Key- An exemplar or answer key to guide your evaluation of the IL’s output.
How to Review- A step-by-step suggestion of how to review the output.
Training Content Development
AI/ML Grading Guide
Title
Week
Canvas Assign To
Submission Type
Grader
Answer Key
How To Review
Week 1 (March 23 - March 29)
Everyone
Discussion
Senior Instructor
For New Curriculum ILs:
1. Review the answer key.
2. Compare with the New Curriculum IL’s Google Colab.
3. Score their output in Canvas using the attached rubric.
For Same Curriculum ILs:
1. Review the Same Curriculum IL’s feedback.
2. Compare their feedback with your own observations.
3. Score their feedback in Canvas using the attached rubric.
1. Review the answer key.
2. Compare with the New Curriculum IL’s Google Colab.
3. Score their output in Canvas using the attached rubric.
For Same Curriculum ILs:
1. Review the Same Curriculum IL’s feedback.
2. Compare their feedback with your own observations.
3. Score their feedback in Canvas using the attached rubric.
Week 1 (March 23 - March 29)
Everyone
Text Entry (HireFlix)
Senior Instructor
1. Review the answer key.
2. Watch the IL’s HireFlix video (access the link from airtable).
3. Read their Canvas reflection.
4. Score their output in Canvas using the attached rubric.
5. Provide feedback for the IL (consider recording video feedback for the IL to see)!
2. Watch the IL’s HireFlix video (access the link from airtable).
3. Read their Canvas reflection.
4. Score their output in Canvas using the attached rubric.
5. Provide feedback for the IL (consider recording video feedback for the IL to see)!
Week 1 (March 23 - March 29)
New Curriculum
Media Recordings
KWK Team
1. Review the answer key.
2. Compare with the IL’s video.
3. Score their work in Canvas using the attached rubric.
4. Write 1-2 sentences of feedback for the IL, especially if there were technical errors.
2. Compare with the IL’s video.
3. Score their work in Canvas using the attached rubric.
4. Write 1-2 sentences of feedback for the IL, especially if there were technical errors.
Week 1 (March 23 - March 29)
Same Curriculum
Media Recordings
KWK Team
1. Review the answer key.
2. Watch the IL’s video.
3. Score their feedback in Canvas using the attached rubric.
4. Write 1-2 sentences of feedback for the IL, especially if there were technical errors.
2. Watch the IL’s video.
3. Score their feedback in Canvas using the attached rubric.
4. Write 1-2 sentences of feedback for the IL, especially if there were technical errors.
Week 2 (March 30 - April 5)
Everyone
Discussion
Senior Instructor
For New Curriculum ILs:
1. Review the answer key.
2. Compare with the New Curriculum IL’s Google Colab.
3. Score their output in Canvas using the attached rubric.
For Same Curriculum ILs:
1. Review the Same Curriculum IL’s feedback.
2. Compare their feedback with your own observations.
3. Score their feedback in Canvas using the attached rubric.
1. Review the answer key.
2. Compare with the New Curriculum IL’s Google Colab.
3. Score their output in Canvas using the attached rubric.
For Same Curriculum ILs:
1. Review the Same Curriculum IL’s feedback.
2. Compare their feedback with your own observations.
3. Score their feedback in Canvas using the attached rubric.
Week 2 (March 30 - April 5)
Everyone
Text Entry (HireFlix)
Senior Instructor
1. Review the answer key.
2. Watch the IL’s HireFlix video (access the link from airtable).
3. Read their Canvas reflection.
4. Score their output in Canvas using the attached rubric.
5. Provide feedback for the IL (consider recording video feedback for the IL to see)!
2. Watch the IL’s HireFlix video (access the link from airtable).
3. Read their Canvas reflection.
4. Score their output in Canvas using the attached rubric.
5. Provide feedback for the IL (consider recording video feedback for the IL to see)!
Week 2 (March 30 - April 5)
Everyone
Media Recordings
KWK Team
1. Review the task description and the guidance in the key.
2. Watch the IL’s video.
3. Score their output in Canvas using the attached rubric.
4. Write 1-2 sentences of feedback for the IL.
2. Watch the IL’s video.
3. Score their output in Canvas using the attached rubric.
4. Write 1-2 sentences of feedback for the IL.
Week 2 (March 30 - April 5)
Everyone
Text Entry
Senior Instructor
1. Review the answer key.
2. Read the IL’s submission in Canvas.
3. Score their output in Canvas using the attached rubric.
4. Write 1-2 sentences of feedback for the IL.
2. Read the IL’s submission in Canvas.
3. Score their output in Canvas using the attached rubric.
4. Write 1-2 sentences of feedback for the IL.
Week 3 (April 6 - April 12)
Everyone
Discussion
KWK Team
For New Curriculum ILs:
1. Review the answer key.
2. Compare with the New Curriculum IL’s work.
3. Score their output in Canvas using the attached rubric.
For Same Curriculum ILs:
1. Review the Same Curriculum IL’s feedback.
2. Compare their feedback with your own observations.
3. Score their feedback in Canvas using the attached rubric.
1. Review the answer key.
2. Compare with the New Curriculum IL’s work.
3. Score their output in Canvas using the attached rubric.
For Same Curriculum ILs:
1. Review the Same Curriculum IL’s feedback.
2. Compare their feedback with your own observations.
3. Score their feedback in Canvas using the attached rubric.
Week 3 (April 6 - April 12)
New Curriculum
Website URL
Senior Instructor
1. Review the answer key.
2. Compare with the IL’s work.
3. Confirm that the IL completed all Try-It steps.
4. Read the IL’s reflection in the Canvas comments.
5. Score their output in Canvas using the attached rubric.
6. Write 1-2 sentences of feedback for the IL.
2. Compare with the IL’s work.
3. Confirm that the IL completed all Try-It steps.
4. Read the IL’s reflection in the Canvas comments.
5. Score their output in Canvas using the attached rubric.
6. Write 1-2 sentences of feedback for the IL.
Week 3 (April 6 - April 12)
Same Curriculum
Media Recordings
Senior Instructor
1. Review the answer key for Try-It 5.
2. Watch the IL’s video.
3. Confirm explanations of key methods.
4. Identify at least 3 CFUs.
5. Score their output in Canvas using the attached rubric.
6. Write 1-2 sentences of feedback for the IL.
2. Watch the IL’s video.
3. Confirm explanations of key methods.
4. Identify at least 3 CFUs.
5. Score their output in Canvas using the attached rubric.
6. Write 1-2 sentences of feedback for the IL.
Week 3 (April 6 - April 12)
New Curriculum
File Uploads
Media Recordings
Website URL
KWK Team
1. Confirm the submission matches a lesson concept.
2. Identify a clear visual and beginner-friendly analogy.
3. Verify that the limitations are explained.
4. Check for technical accuracy and unexplained jargon.
5. Read their reflection in Canvas.
6. Score their output in Canvas using the attached rubric.
7. Write 1-2 sentences of feedback for the IL.
2. Identify a clear visual and beginner-friendly analogy.
3. Verify that the limitations are explained.
4. Check for technical accuracy and unexplained jargon.
5. Read their reflection in Canvas.
6. Score their output in Canvas using the attached rubric.
7. Write 1-2 sentences of feedback for the IL.
Week 3 (April 6 - April 12)
Everyone
Text Entry (HireFlix)
Senior Instructor
1. Review the answer key.
2. Watch the IL’s HireFlix video (access the link from airtable).
3. Read their Canvas reflection.
4. Score their output in Canvas using the attached rubric.
5. Provide feedback for the IL.
2. Watch the IL’s HireFlix video (access the link from airtable).
3. Read their Canvas reflection.
4. Score their output in Canvas using the attached rubric.
5. Provide feedback for the IL.
Week 4 (April 13 - April 19)
New Curriculum
Media Recordings
KWK Team
1. Review the answer key.
2. Watch this video of the lesson if you need a refresher.
3. Watch the IL's video (5-8 minutes).
4. Confirm all steps are completed.
5. Verify correct function structure.
6. Check that the IL explains the WHY behind each choice.
7. Score the submission in Canvas using the rubric.
8. Write 1-2 sentences of feedback.
2. Watch this video of the lesson if you need a refresher.
3. Watch the IL's video (5-8 minutes).
4. Confirm all steps are completed.
5. Verify correct function structure.
6. Check that the IL explains the WHY behind each choice.
7. Score the submission in Canvas using the rubric.
8. Write 1-2 sentences of feedback.
Week 4 (April 13 - April 19)
Everyone
Discussion
KWK Team
For New Curriculum ILs:
1. Review the answer key.
2. Compare with the New Curriculum IL’s work.
3. Score their output in Canvas using the attached rubric.
For Same Curriculum ILs:
1. Review the Same Curriculum IL’s feedback.
2. Compare their feedback with your own observations.
3. Score their feedback in Canvas using the attached rubric.
1. Review the answer key.
2. Compare with the New Curriculum IL’s work.
3. Score their output in Canvas using the attached rubric.
For Same Curriculum ILs:
1. Review the Same Curriculum IL’s feedback.
2. Compare their feedback with your own observations.
3. Score their feedback in Canvas using the attached rubric.
Week 4 (April 13 - April 19)
Everyone
Text Entry (HireFlix)
Senior Instructor
1. Review the answer key.
2. Watch the IL’s HireFlix video (access the link from airtable).
3. Read their Canvas reflection.
4. Score their output in Canvas using the attached rubric.
5. Provide feedback for the IL.
2. Watch the IL’s HireFlix video (access the link from airtable).
3. Read their Canvas reflection.
4. Score their output in Canvas using the attached rubric.
5. Provide feedback for the IL.
Week 4 (April 13 - April 19)
New Curriculum
File Uploads
Senior Instructor
1. Review the answer key.
2. Open the IL's Google Colab notebook.
3. Verify all exercises are completed.
4. Check for a new knowledge base and a relevant query.
5. Note whether the spicy challenge was attempted.
6. Evaluate code quality (Does it follow best practices?).
7. Score the submission in Canvas using the rubric.
8. Write 1-2 sentences of feedback.
2. Open the IL's Google Colab notebook.
3. Verify all exercises are completed.
4. Check for a new knowledge base and a relevant query.
5. Note whether the spicy challenge was attempted.
6. Evaluate code quality (Does it follow best practices?).
7. Score the submission in Canvas using the rubric.
8. Write 1-2 sentences of feedback.
Week 4 (April 13 - April 19)
Same Curriculum
Media Recordings
Senior Instructor
1. Review the answer key to understand what to look for.
2. Watch the IL's debugging video.
3. Confirm the IL correctly identifies the root cause.
4. Check that they explain why the chunking strategy matters.
5. Verify the IL demonstrates a fix and tests it.
6. Count at least 3 CFUs throughout the video.
7. Score the submission in Canvas using the rubric.
8. Write 1-2 sentences of feedback.
2. Watch the IL's debugging video.
3. Confirm the IL correctly identifies the root cause.
4. Check that they explain why the chunking strategy matters.
5. Verify the IL demonstrates a fix and tests it.
6. Count at least 3 CFUs throughout the video.
7. Score the submission in Canvas using the rubric.
8. Write 1-2 sentences of feedback.
Week 5 (April 20 - April 26)
Everyone
Website URL
Senior Instructor
1. Review the rubric and exemplars.
2. Open the IL’s Hugging Face link and interact with it.
3. Confirm the project reflects RAG implementation.
4. Read the IL’s comments and check for personal connection.
5. Score the submission in Canvas using the rubric.
6. Write 1-2 sentences of feedback.
2. Open the IL’s Hugging Face link and interact with it.
3. Confirm the project reflects RAG implementation.
4. Read the IL’s comments and check for personal connection.
5. Score the submission in Canvas using the rubric.
6. Write 1-2 sentences of feedback.
Week 5 (April 20 - April 26)
Everyone
Quiz
Senior Instructor
1. Review the exemplar answers.
2. Score each response 0-2 in Canvas.
2. Score each response 0-2 in Canvas.
Made with Bullet