Guide to Judging: Event Preparation and Execution

Overview

The process of preparing for judging needs to be taken into consideration during the initial stages of event planning. The size of the event, the number of awards given out, the event agenda, and volunteer recruitment all impact the judging process. Coordination between the Event Partner, the Judge Advisor, and Judge volunteers is crucial for the judging process to operate smoothly and effectively.

In the case of tournaments, judging should conclude on the last day of competition. In the case of leagues, judging must occur close to the date of league finals and after all ranking sessions have been concluded. If remote judging is utilized, that process should take place as close to the final date of the event as possible. This is to ensure that the teams and robots that judges evaluate in initial interviews are as close as possible to what is brought to competition and observed by Judges in person.

Prior to Event – Tasks by Role

Event Partner

  • Recruit a qualified Judge Advisor and Judges for the event well in advance to ensure there are enough Judges to meet the needs of the event.
  • Work with the REC Foundation RSM to ensure that all required awards are listed on Robotevents.com, and corresponding trophies/certificates are procured.
  • Ensure that there is a secure and quiet room with adequate space for the judging staff to deliberate (the Judges’ Room). Only the judging staff and specifically authorized volunteers for the event should have access to this room.
  • Know and understand the roles of the Judges and the Judge Advisor.
  • Ensure that the judging staff has appropriate judging materials, including clipboards, pens, highlighters, sticky-notes, copies of current judging documents such as rubrics and note-taking sheets, and other needed items. These documents cannot be modified or replaced with unofficial versions.

Judge Advisor

  • Pass the Judge Advisor / Judge Training & Certification Course for the current season prior to the event.
  • Has no or minimal conflicts of interest with teams attending the event.
  • Review with the Event Partner the awards to be offered at the event.
  • Work with Event Partner to ensure adequate Judges are recruited and confirm their attendance and skill sets.
  • Manage any potential conflicts of interest that individual Judges may have with teams at the event.
  • Prepare a judging schedule based on the number of teams registered and the agenda for the event.
  • Formulate a clear process for how Engineering Notebooks will be collected and judged.
  • Confirm with the Event Partner that judging staff will have all appropriate and current judging materials and documents, including team lists and match sheets from the event’s Tournament Manager operator. These documents cannot be modified or replaced with unofficial versions.

Judge

  • Review the game video and game description to understand the fundamentals of the game that teams will be playing.
  • Communicate any potential conflicts of interest with teams at the event with the Judge Advisor.
  • Be familiar with the current judging materials including official judging documentation, rubrics, and award descriptions. These documents cannot be modified or replaced with unofficial versions.
  • Complete the Judge Advisor / Judge Training & Certification Course (highly encouraged but not required for local events; required for events that qualify teams to the VEX Robotics World Championship). Note: Exceptions to this requirement should be rare and require approval from the REC Foundation RSM.

Event Day – Tasks by Role

Event Partner

  • Ensure judging staff have all needed materials and access to the secure Judges’ Room.
  • Communicate any schedule changes to the Judge Advisor.
  • Event Partners may not recommend or assign Judged Awards to any team or be involved in award deliberations. EPs may recommend or assign awards given to individuals, such as the Volunteer of the Year Award.
  • The Event Partner should oversee the entering of awards into Tournament Manager to do a final check to ensure no team is being given more than one Judged Award. If a team was assigned multiple Judged Awards, the Event Partner should consult with the Judge Advisor to rectify the situation.

Judge Advisor

  • Review the judging process with Judges prior to the start of the event and answer any questions they may have.
  • Receive submitted Engineering Notebooks.
  • Ensure Judges sign in on the Judge Volunteer Check-In Sheet.
  • Group Judges and assign each group a subset of teams to interview, managing potential conflicts of interest. This may be done prior to the event. Judges should not be placed in a position to interview or deliberate for teams with which they have a conflict.
  • Assign Judges with pre-existing relationships to each other—or with similar backgrounds—to different Judge groups so that teams are interacting with Judges who have different perspectives and backgrounds.
  • Manage time and ensure judging groups are keeping pace to interview all teams within time constraints.
    Lead deliberations for Judged Awards.
  • Ensure no team earns more than one Judged Award.
  • Collect Field Notes to Judge Advisor from event staff prior to final deliberations.
  • Record the results of all Judged Awards and communicate the list of award winners to the Event Partner and/or Tournament Manager operator.
  • Have the Tournament Manager operator print the award scripts to be used at the award ceremony.
  • Maintain confidentiality of any judging deliberations and discussions. Teams should not receive any feedback from the Judges or Judge Advisor, nor should Event Partners receive specific information discussed by Judges, except to report Code of Conduct violations.
  • Collect all judging materials to ensure confidentiality. After the event, these materials should be destroyed. Ensure the process for returning all Engineering Notebooks to teams is completed, if applicable.
    The Judge Advisor should not participate in interviews as part of a judging group unless there is a dire need due to an unforeseen lack of personnel.

Judge

  • Conduct several tasks depending on the needs at the event, including:
  • Communicate any potential conflicts of interest with attending teams to the Judge Advisor
  • Deliberate with other Judges under direction of the Judge Advisor to assign award winners following the guidelines in the official judging documentation.
  • Hand in all judging notes and rubrics to the Judge Advisor.
  • Maintain confidentiality of any judging deliberations and discussions. Teams should not receive any feedback from Judges aside from positive encouragement and thanks at the end of their interview.

Typical In-Person Event Timeline Example

The chart below is an example of how the in-person judging process might operate in parallel with the rest of the competition schedule during a typical one-day event.

If remote judging is conducted, Engineering Notebook evaluations and/or initial team interviews are completed prior to the event. See Section 8 for more details.

Typical In-Person Event Timeline Example

All Judging Done in Person

TIME EVENT ACTIVITY TEAMS JUDGES / JUDGE ADVISOR
Early Morning CHECK-IN Teams check in as present, hand in Engineering Notebooks. Once inspected, teams can run their Skills Challenge Matches. Judge Orientation / Begin Interviews - Judges organized into groups and assigned to interview teams. Interviews can begin as soon as there are Judges assigned to groups, and any questions about the process have been addressed by the Judge Advisor. Notebooks can also start being evaluated at this time. It is advisable to pause interviews during the opening ceremonies / Event Meeting.
INSPECTION
Morning

OPENING CEREMONIES/

EVENT MEETING

Teams attend and ask questions at event meeting.
QUALIFICATION MATCHES Teams are scheduled into Qualification Matches. Teams are interviewed during breaks between their matches.
Lunch Break LUNCH BREAK Lunch Break: If event is running behind, teams may run matches through this time. Working Lunch - Judges should take a rest, discuss progress so far, and each group of Judges can name top picks for awards so far. 
Early Afternoon QUALIFICATION MATCHES Teams are scheduled into Qualification Matches. Finish Team Interviews and begin final deliberations. Judge Advisor should collect the final Skills Challenge and qualification rankings from the Tournament Manager Operator, as well as any field notes. If additional interviews are needed, they should be completed before Qualification Matches are over.
Afternoon

ALLIANCE SELECTION/

ALLIANCE PAIRINGS

Teams undergo alliance selection (V5RC) or alliance pairings (VIQRC) or have a short break before finals (VURC/VAIRC). Final Deliberations - Teams should not be interviewed during this time; decisions must be made with the data at hand. Once all awards are decided, the Judge Advisor takes them to the Event Partner and/or Tournament Manager Operator to be put into Tournament Manager. All Engineering Notebooks should be returned to teams.
End of Day ELIMINATION/FINALS MATCHES Teams participate in Finals Matches and receive awards. Some events may intersperse awards with Finals Matches, others may have an awards ceremony afterwards.
AWARDS/CLOSING CEREMONIES Judge Advisor collects and destroys judging notes and rubrics and clears the Judges’ Room of any identifying information. Judges may be asked to read award scripts, present awards, or just be visible for teams at the closing ceremony. The Event Partner should plan this beforehand.

Continue to the next section, Guide to Judging: Awards