Guide to Judging: Team Interviews

Overview

The Team Interview Rubric is used for all initial Team Interviews. Judges may use the Team Interview Tips and Sample Questions and Team Interview Notes to assist in interviews. Judges interview teams that have been assigned to them by the Judge Advisor. Teamwork, professionalism, interview quality, and team conduct is considered when nominating and ranking teams for all Judged Awards.

Initial Team Interviews can be conducted in the team pit area. This allows Judges to observe teams at work and quickly move from team to team. Alternatively, initial Team Interviews may be conducted in a hallway or some other still-public place, such as a library room or cafeteria. This can be a quieter venue for interviews, but care should be taken that the interview format remains intact and does not become a prepared presentation. Keep in mind that a more private setting could come across as intimidating for some teams.

Initial Team Interviews can be conducted without notice to teams, scheduled by the Judge Advisor, or conducted at a time of the team’s choosing (examples include schedules made via a signup sheet or first-come-first-served queue). All teams at the event must have their initial interviews scheduled in the same way, and teams are not allowed to choose a particular set of Judges. A best practice for a self-service model for assigning interviews is allocating teams to one of several groups of Judges based on a queuing method, with modifications made in cases where conflicts of interest arise between a team and a Judge.

Judges need to talk only to the student members of the team. Occasionally, enthusiastic adults may want to answer the Judges’ questions. If this is encountered, politely remind the adult(s) that the Judges are there to interview the students. All teams at an event must have an opportunity to be interviewed at least once. A team may decline to be interviewed. That team would no longer be eligible for any Judged Award with the exception of Volunteer Nominated Awards if they are offered at the event.

Some Judge Advisors may wish to create a list of questions for Judges to ask that are common for all interviews at an event. This could be particularly helpful to ensure that all aspects of the robot and competition are addressed, or to assist inexperienced Judges with the interview process. This should not be construed as a “script”; Judges should be free to follow up questions based on student responses.

Some teams may be hard to find at an event: if they are not in their pit space, another approach may be to find them as they come off the field for their match.

Some teams may want to share parts of their Engineering Notebook during their interview. This is permissible, but depending on how and when notebooks are collected, this may not be possible. Teams should be prepared to answer the Judges’ questions without their notebook.

Award finalists may be cross interviewed by different Judges as a part of the deliberation process. The Judge Advisor will assign additional interviews as needed during the event. Follow-up interviews for any award contenders should be conducted without notice, preferably in the competition or pit areas. This allows Judges to see the team in their workspace and does not give any team an advantage via prior notice.

Considerations for Cultural Differences and Communication Styles

Some students, whether it be from individual or cultural differences, may have varying styles of interacting with Judges during the interview process. Maintaining eye contact, speaking in a loud enough voice to be easily heard, and other engagement norms may differ between students. Judges should do their best to give all teams an opportunity to share their design process during the interview and should strive to not allow factors that are beyond students’ control to bias their evaluation of the team.

Judges should avoid using humor or language that could be interpreted as disparaging. For example, “I can’t believe you came up with this on your own!” might have been intended as a compliment to the team but could be misinterpreted to indicate that the Judges believe the team is violating the Code of Conduct by claiming work that is not their own.

STEP 1 – CONDUCTING THE TEAM INTERVIEW

  • All teams should be interviewed for roughly the same amount of time. The Judge Advisor will create a schedule based on the number of teams and Judges at an event.
  • Typically, a Team Interview lasts about 10-15 minutes. Staying on schedule is important to ensure all teams are interviewed and there is sufficient time to conduct deliberations. Teams that need an interpreter to communicate with Judges may need more time, and should notify the Event Partner upon registration.
  • Team Interviews are based around Judges directly asking students open-ended questions about their robot and design process that give students an opportunity to share their design process, teamwork, and journey throughout the season. Follow-up questions are asked as needed.
  • Teams can use their robot and its associated equipment, Engineering Notebook (if available), and programming device to show their code if desired during the interview. It is the intent of the interview for judges to engage with students and their robot and not with audio/visual aids such as presentations or displays.
  • Judges should take notes during interviews and observations to support their evaluations and assist with deliberations. The Team Interview Notes form can be used to keep track of notes for each team.
  • Judges should consider taking a picture of each robot with the team number visible to help recall details about robot designs mentioned in their notes.
  • If Judges are unable to locate an assigned team for an initial Team Interview after several visits to the team’s pit area, they will leave a Judges’ Note to Missed Teams on the team pit table. Notes will not be left for follow-up interviews.
  • If Judges are unable to locate an assigned team’s pit area, they should contact the Judge Advisor for assistance.
  • Judges should remember that younger students communicate their ideas differently than older students. Judges should use age-appropriate language when asking questions and considering student responses.
  • The Judging Single Page Reference may additionally be used by Judges to look up brief award descriptions and other useful information.

STEP 2 – COMPLETE TEAM INTERVIEW RUBRIC / TEAM INTERVIEW EVALUATION

Important: The Team Interview Rubric is a tool for initial team interview evaluations through quantitative comparison. The final determination of all award candidates and winners are done through further qualitative deliberation among judges based on award descriptions and criteria. As such, a team earning a particular or overall score on a rubric is not an automatic disqualification or threshold for any judged award.

After the interview, each group of Judges should complete the Team Interview Rubric and optionally the Initial Award Candidate Ranking Sheet for each team. Judges should go somewhere private to discuss and fill out these forms and/or compile notes, and should take care that their discussions are not overheard by any other party.

Judges should identify student-centered teams with positive, respectful, and ethical conduct during the team interviews and team observations. Conversely, they should also make note of any teams that are not demonstrating these principles, including teams that are not being directly interviewed.

STEP 3 – IDENTIFY INITIAL CANDIDATE TEAMS WITHIN JUDGE GROUP

When additional Judged Awards are offered at an event (beyond the Excellence, Design, Innovate, and Judges Awards), the Judge Advisor may provide the Initial Award Candidate Ranking Sheet to Judge groups assigned to interview teams for use along with the Team Interview Rubric as they interview their group of teams. This form may also be useful when initial Team Interviews are conducted remotely (see section on Remote Judging) as a way to log nominations from each judging group.

On the Initial Award Candidate Ranking Sheet, Judges will write down the team numbers of the teams they are assigned to interview on the left side and fill in any additional Judged Awards offered at the event. Awards should be listed according to precedence from left to right, with Qualifying Awards in the leftmost columns, followed by the non-qualifying awards. The precedence of Qualifying Awards is listed in the REC Foundation Qualifying Criteria. The Judge groups will then use the spaces provided to indicate a candidate for each of the additional Judged Awards being offered at the event.

As Judges interview teams, they may want to use multiple stars or checks on the Initial Award Candidate Ranking Sheet to show rankings as teams are interviewed. This is done by adding check marks to rank teams. For example, if the first team interviewed received one check mark as a recommendation for an award and the second team interviewed would be a better candidate, the second team would receive one check mark and the first team would receive a second check mark, ranking them first and second, respectively. This continues until all teams are interviewed, and the end result is a ranking of teams.

Below is an example of how this sheet might be filled out by one Judge group, judging a subset of teams at a larger event. In this example the Innovate, Think, and Judges awards have been filled in below.

TEAM NUMBER DESIGN AWARD INNOVATE AWARD THINK AWARD JUDGES AWARD
Communicating the Engineering Design Process Documentation of a specific feature Effective programming and autonomous strategy Special Recognition
TEAM A   ✓✓✓  
TEAM B ✓✓ ✓✓ ✓✓✓
TEAM C ✓✓✓    
TEAM D ✓✓ ✓✓✓ ✓✓
         

This is a simple way for Judges to preliminarily rank their recommendations as they go, with final rankings done after their set of interviews are completed. Additionally, Judges can also make notes on the Team Interview Notes sheet.

Continue to the next section, Guide to Judging: Award Deliberations