Hi All!
As you’ve probably seen if you’re subscribed to RoboCup Worldwide, Peter Stone has announced that RoboCup 2021 will be completely virtual. The RoboCupRescue Robot League has been planning for this eventuality and we would like to present our plan to conduct RoboCupRescue in a completely remote fashion.
We will have more details about the specifics shortly, including the launch of a new online forum where we can discuss the further development of rules and procedures that will be necessary so stay tuned!
Here’s a high level summary of how we will run this year.
This year we have a unique opportunity to modify how we conduct our league to involve even more researchers, add statistical significance to our evaluations, and advertise more widely the state-of-the-science that RoboCupRescue is contributing.
Although this years coronavirus pandemic makes these changes essential, we plan to turn the necessity of remote collaborations into a real improvement in the league going forward. This can be our new model for global outreach and a more rigorous qualification process for future, more normal, competition years.
These two key upgrades will lower the barrier of entry for new smaller teams and use a distributed model of Preliminaries to determine the Best-In-Class robots in each of our categories for Mobility, Dexterity, Exploration and Mapping:
- Focus team robot development and demonstration on the smallest, easiest tests to fabricate, and cheapest set of tests that are on the critical path toward operationally significant. This means as many teams as possible, including those who could not normally travel to the competition, to build the apparatuses and participate. In addition to more academic teams, we hope that this also enables more makerspaces and responder organizations to field teams.
- Enable crowd-sourced scoring of uncut trial videos by all participating teams so they learn from each other’s implementations and verify each other’s scores. The tests are already compatible with video scoring. We did this for the DARPA Robotics Challenge during qualifications and it worked well. We would add requirements for statistical significance (e.g. requiring 10x 20 minute runs in a particular apparatus, with a particular configuration, with uncut video showing the run as well as details of the apparatus and robot), over the course of an extended time (1 or 2 months leading up to the competition).
- Remote Proctoring: The top teams with live teleconference trials with members of the Organising Committee. This will re-evaluate the systems to extend whatever number of trials we required to get to this point.
- Test Facilities: There are a number of test facilities around the world that host the tests. Emergency responder organizations too. They can potentially accommodate visiting teams and proctor them in the tests for locations and at times when this is safe. This is good for teams that maybe cannot fabricate the tests themselves or don’t feel confident conducting trials.
- Best-In-Class Award Certificates can be given out by RoboCup to any team that exceeds a set threshold across ALL the tests in the category as shown in our typical radar charts. We’ll have such extensive statistical significance for each test across more teams performing more trials that we’ll be able to confidently identify the normalized scoring thresholds for levels of performance, such as Gold, Silver, or Bronze.
Of course there are still many additional updates to the rules and additional procedures that will be necessary to make such a competition a success. We will be providing more updates shortly!
Cheers!
- The RoboCupRescue Robot League Committee