Judge:History - Revision history

Brian: /* Future plans */

2014-12-06T23:16:54Z

‎Future plans

Brian: /* Recent hosting */

2014-12-06T23:15:49Z

‎Recent hosting

Brian at 22:23, 17 October 2011

2011-10-17T22:23:30Z

Brian: /* A brief history of the Judge */

2010-11-07T23:30:47Z

‎A brief history of the Judge

Brian: Created page with '==A brief history of the Judge== During the 2007/2008 school year, Hanson and Guru decided to devote some time to a most worthy cause: writing an autograder for the 11/12 enriche…'

2010-11-04T21:22:05Z

Created page with '==A brief history of the Judge== During the 2007/2008 school year, Hanson and Guru decided to devote some time to a most worthy cause: writing an autograder for the 11/12 enriche…'

New page

==A brief history of the Judge==
During the 2007/2008 school year, Hanson and Guru decided to devote some time to a most worthy cause: writing an autograder for the 11/12 enriched CS class at Woburn. In order to encourage us to develop her students' problem-solving skills as much as possible, Ms. Plachta set high expectations for them, such as earning 1000 points by the end of the year, where a typical problem is worth between 3 and 20 points. Students who wanted to earn a 99 in the course then had to solve between 50 and 300 problems, which translated to lots and lots of marking for Ms. Plachta. Marking programs is very tedious work. She had to download the program, save it in the student's folder, compile the program, locate the test data, run the program once for each case, then compare its output with the expected output. If the program failed, she often tried to figure out why, to give feedback to the student, and if it worked, she had to update the points spreadsheet.

The last task is the only one that should require human intervention, and it was clear that Ms. Plachta was spending far too much time on the other tasks. In the periods of time immediately preceding report card mark submission deadlines she was literally spending day and night marking, marking, marking. As she was in poor health due to the cancer it became clear that this would be too much of a burden for her. The more experienced students in class could give feedback to students whose programs didn't work, but it would have been impossible for them to mark everyone’s programs; it would have been too much work for students who had their own work to do too. And so the PEG Judge was born. Though its creators did not anticipate this, the Judge would have the added benefit of encouraging students to code.

On September 23rd, 2008, the first source file was submitted to the PEG Judge, which at the time supported only C and C++ (with Pascal being added soon afterward). The Judge was hosted on one of the school’s machines, a Dell OptiPlex with a dual-core Intel CPU and 4MB L2 cache using Jacob’s home internet connection with Rogers. As hosting servers on residential internet contracts is typically not allowed, Hanson decided to run the server on some port other than 80, in case the ISP was scanning for unauthorized activity on port 80. After trying various ports he determined that 5050 is the only port above the reserved range (''i.e.'', the range 1-1023 to which, on some operating systems, permission to bind is withheld by the kernel and granted only to users with sufficient privilege) that isn't actually filtered by the school board's firewall. So there we sat at http://pegjudge.ath.cx:5050/ . The Judge at that time was very new; it had a very basic set of features, but it got the job done. Guru wrote most of the back end, particularly the grader itself, and Hanson wrote most of the front end.

The 2008/2009 school year saw a dramatic improvement in the interface as Hanson completely overhauled the front end, leaving the interface cleaner, crisper, and more professional-looking. He also overhauled the back end, switching from the ''ad hoc'' PHP functions being used before to the CodeIgniter framework. (The grader, though, has been tweaked only slightly since the inception of the Judge.)

By the end of that year the Judge in its current form was essentially complete, a good thing as Hanson was off to university and would likely have other things on his mind. As Brian had started using Linux the previous year, he was in an ideal position to take over the administration of the Judge (but he had to trick Hanson into giving him root access). In November he added a wiki to the site and he has also gradually populated the allowed languages list.

With the passing of Ms. Plachta the Judge was never as popular as before, for students were no longer being pushed to earn 1000 points. But there were still some PEG members using it; Canada’s IOI teams trained on it; and Brian and a friend from Waterloo C.I. tried to run a mock DWITE contest. And so the Judge went on. When the Plachta house was sold in June, the Judge was migrated to Brian's laptop, again illicitly. When Brian left for university, he could not risk violating the residence contract, so the Judge was left without hosting. Brian, Jacob, and Hanson were hopeful about convincing the Center for Education in Mathematics and Computing at the University of Waterloo to grant PEG hosting, for this is the organization primarily involved with high achievement in computer science at the high school level, running the Canadian Computing Competition and sending Team Canada to the IOI. Almost all problems from CCCs of years past, both stage 1 and stage 2, were already in the PEG Judge's archive, the interface had taken two years of work, the grader had been tweaked precisely, and, all in all, the PEG Judge was the first in Canada with a level of sophistication on par with those of the major online judges of the time. PEG's request was denied by the CEMC, forcing Brian to look elsewhere.

Luckily, some very good VPS plans exist; [http://www.intovps.com/ IntoVPS], for example, offers 512 MB guaranteed RAM, 30 GB disk space, 500 GB monthly bandwidth, and a quad-core CPU for only $10 USD/month. The PEG Judge was initially migrated to an IntoVPS server; a month later it was again migrated to hosting by [http://www.virpus.com/ Virpus], which offers 1 GB guaranteed RAM, 50 GB disk space, 1600 GB monthly bandwidth, and six CPU cores. As hosting is now legitimate, the :5050 suffix was dropped and the site is now http://pegjudge.ath.cx/

← Older revision		Revision as of 23:16, 6 December 2014
Line 32:		Line 32:
	==Future plans==		==Future plans==
	Brian plans to rewrite the core grader from scratch, to improve support for languages such as Java that invariably use multiple processes (something that the Unix API does not handle well). He also plans to refactor and then open-source the Judge code. It's currently uncertain whether either of these projects will ever see the light of day.		Brian plans to rewrite the core grader from scratch, to improve support for languages such as Java that invariably use multiple processes (something that the Unix API does not handle well). He also plans to refactor and then open-source the Judge code. It's currently uncertain whether either of these projects will ever see the light of day.
		+
		+	Brian also hopes that Google Fiber will one day come to Silicon Valley, where he currently resides, so that he can host the Judge on a dedicated server in his own home.

@@ Line 18: / Line 18: @@
 ==Recent hosting==
-The Judge was migrated a second time at the end of September 2010 to a OpenVZ box hosted by [http://www.intovps.com/ IntoVPS], which offered 512 MB guaranteed RAM, 30 GB disk space, 500 GB monthly bandwidth, and a quad-core CPU for only $10 USD/month. The :5050 suffix on the Judge URL was dropped, and the web server simply bound to port 80, since the hosting was then legitimate; Jacob also acquired the <code>wcipeg.com</code> domain from James Rosselet (a former PEG member), giving the URL of the Judge as the current {{Root}}. A month later, it was again migrated to hosting by [http://www.virpus.com/ Virpus], which offered 1 GB guaranteed RAM, 50 GB disk space, 1600 GB monthly bandwidth, and six CPU cores, as Brian thought this was a better plan. However, Virpus hosting suffered from too much downtime, so in June of 2011 the Judge was migrated a fourth time to a [http://volumedrive.com/ VolumeDrive] box. In October of 2011, the Judge was migrated a fifth time to hosting offered by [http://lovevps.com/ LoveVPS], this time a Xen box, due to Brian's dissatisfaction with VolumeDrive's support and the incomplete virtualization of OpenVZ containers in general.
+The Judge was migrated a second time at the end of September 2010 to a OpenVZ box hosted by [http://www.intovps.com/ IntoVPS], which offered 512 MB guaranteed RAM, 30 GB disk space, 500 GB monthly bandwidth, and a quad-core CPU for only $10 USD/month. The :5050 suffix on the Judge URL was dropped, and the web server simply bound to port 80, since the hosting was then legitimate; Jacob also acquired the <code>wcipeg.com</code> domain from James Rosselet (a former PEG member), giving the URL of the Judge as the current {{Root}}.
 ==Future plans==
 Brian plans to rewrite the core grader from scratch, to improve support for languages such as Java that invariably use multiple processes (something that the Unix API does not handle well). He also plans to refactor and then open-source the Judge code. It's currently uncertain whether either of these projects will ever see the light of day.

@@ Line 1: / Line 1: @@
-==A brief history of the Judge==
+==Background==
-During the 2007/2008 school year, Hanson and Guru decided to devote some time to a most worthy cause: writing an autograder for the 11/12 enriched CS class at Woburn. In order to encourage us to develop her students' problem-solving skills as much as possible, Ms. Plachta set high expectations for them, such as earning 1000 points by the end of the year, where a typical problem is worth between 3 and 20 points. Students who wanted to earn a 99 in the course then had to solve between 50 and 300 problems, which translated to lots and lots of marking for Ms. Plachta. Marking programs is very tedious work. She had to download the program, save it in the student's folder, compile the program, locate the test data, run the program once for each case, then compare its output with the expected output. If the program failed, she often tried to figure out why, to give feedback to the student, and if it worked, she had to update the points spreadsheet.
+Construction of the PEG Judge began during the 2007/2008 school year, as a project by Hanson and Guru, two former [[PEG:PEG leaders|PEG leaders]], to help Ms. Plachta by writing an autograder for the 11/12 enriched CS class at Woburn.
-The last task is the only one that should require human intervention, and it was clear that Ms. Plachta was spending far too much time on the other tasks. In the periods of time immediately preceding report card mark submission deadlines she was literally spending day and night marking, marking, marking. As she was in poor health due to the cancer it became clear that this would be too much of a burden for her. The more experienced students in class could give feedback to students whose programs didn't work, but it would have been impossible for them to mark everyone’s programs; it would have been too much work for students who had their own work to do too. And so the PEG Judge was born. Though its creators did not anticipate this, the Judge would have the added benefit of encouraging students to code.
+Ms. Plachta demanded the highest standards of achievement from her students, in order to push them to reach their full potential. A student could typically earn a 99 in the course by earning a large number of points (on the order of magnitude of 1000) in one school year, where a single CS problem was typically worth between 3 and 20 points (depending on difficulty). This translated to lots and lots of marking for Ms. Plachta, who had to download the source file, save it in the student's folder, compile the program, locate the test data, run the program once for each case, then compare its output with the expected output for each case. If the program failed, she often tried to figure out why, to give feedback to the student, and if it worked, she had to update the points spreadsheet. In the code submission frenzies that preceded report card mark submission deadlines, Ms. Plachta often had to work day and night to finish all her marking.
-On September 23rd, 2008, the first source file was submitted to the PEG Judge, which at the time supported only C and C++ (with Pascal being added soon afterward). The Judge was hosted on one of the school’s machines, a Dell OptiPlex with a dual-core Intel CPU and 4MB L2 cache using Jacob’s home internet connection with Rogers. As hosting servers on residential internet contracts is typically not allowed, Hanson decided to run the server on some port other than 80, in case the ISP was scanning for unauthorized activity on port 80. After trying various ports he determined that 5050 is the only port above the reserved range (''i.e.'', the range 1-1023 to which, on some operating systems, permission to bind is withheld by the kernel and granted only to users with sufficient privilege) that isn't actually filtered by the school board's firewall. So there we sat at http://pegjudge.ath.cx:5050/ . The Judge at that time was very new; it had a very basic set of features, but it got the job done. Guru wrote most of the back end, particularly the grader itself, and Hanson wrote most of the front end.
+When Ms. Plachta became ill with cancer, it became clear that this system was unsustainable. There were students in the class experienced enough to give feedback to other students who were having trouble with getting their code to work, but it would have been impossible for them to mark everyone’s programs; it would have been too much work for these students who had their own work to do too. Meanwhile, computers are very bad at figuring out humans' mistakes, but very good at all the other tasks involved in the marking process (automatic submission, compilation, execution, and grading). And so the PEG Judge was born, and complemented the efforts of Guru, Hanson, Jacob, and others in taking over the teaching of the class.
-The 2008/2009 school year saw a dramatic improvement in the interface as Hanson completely overhauled the front end, leaving the interface cleaner, crisper, and more professional-looking. He also overhauled the back end, switching from the ''ad hoc'' PHP functions being used before to the CodeIgniter framework. (The grader, though, has been tweaked only slightly since the inception of the Judge.)
+==Genesis==
-By the end of that year the Judge in its current form was essentially complete, a good thing as Hanson was off to university and would likely have other things on his mind. As Brian had started using Linux the previous year, he was in an ideal position to take over the administration of the Judge (but he had to trick Hanson into giving him root access). In November he added a wiki to the site and he has also gradually populated the allowed languages list.
+==Development==
-With the passing of Ms. Plachta the Judge was never as popular as before, for students were no longer being pushed to earn 1000 points. But there were still some PEG members using it; Canada’s IOI teams trained on it; and Brian and a friend from Waterloo C.I. tried to run a mock DWITE contest. And so the Judge went on. When the Plachta house was sold in June, the Judge was migrated to Brian's laptop, again illicitly. When Brian left for university, he could not risk violating the residence contract, so the Judge was left without hosting. Brian, Jacob, and Hanson were hopeful about convincing the Center for Education in Mathematics and Computing at the University of Waterloo to grant PEG hosting, for this is the organization primarily involved with high achievement in computer science at the high school level, running the Canadian Computing Competition and sending Team Canada to the IOI. Almost all problems from CCCs of years past, both stage 1 and stage 2, were already in the PEG Judge's archive, the interface had taken two years of work, the grader had been tweaked precisely, and, all in all, the PEG Judge was the first in Canada with a level of sophistication on par with those of the major online judges of the time. PEG's request was denied by the CEMC, forcing Brian to look elsewhere.
+By the end of that year the Judge in its current form was essentially complete, a good thing as Hanson was off to university and would likely have other things on his mind. It even included support for running contests and posting comments. Brian, who had been helping with administration since the beginning of the year, took over at this point as primary administrator of the Judge. Brian's primary contributions were the addition of this wiki to the site and support for various additional popular programming languages.
-Luckily, some very good VPS plans exist; [http://www.intovps.com/ IntoVPS], for example, offers 512 MB guaranteed RAM, 30 GB disk space, 500 GB monthly bandwidth, and a quad-core CPU for only $10 USD/month. The PEG Judge was initially migrated to an IntoVPS server; a month later it was again migrated to hosting by [http://www.virpus.com/ Virpus], which offers 1 GB guaranteed RAM, 50 GB disk space, 1600 GB monthly bandwidth, and six CPU cores. As hosting is now legitimate, the :5050 suffix was dropped and the site is now {{Root}}
+==Later years==

← Older revision		Revision as of 23:30, 7 November 2010
Line 12:		Line 12:
	With the passing of Ms. Plachta the Judge was never as popular as before, for students were no longer being pushed to earn 1000 points. But there were still some PEG members using it; Canada’s IOI teams trained on it; and Brian and a friend from Waterloo C.I. tried to run a mock DWITE contest. And so the Judge went on. When the Plachta house was sold in June, the Judge was migrated to Brian's laptop, again illicitly. When Brian left for university, he could not risk violating the residence contract, so the Judge was left without hosting. Brian, Jacob, and Hanson were hopeful about convincing the Center for Education in Mathematics and Computing at the University of Waterloo to grant PEG hosting, for this is the organization primarily involved with high achievement in computer science at the high school level, running the Canadian Computing Competition and sending Team Canada to the IOI. Almost all problems from CCCs of years past, both stage 1 and stage 2, were already in the PEG Judge's archive, the interface had taken two years of work, the grader had been tweaked precisely, and, all in all, the PEG Judge was the first in Canada with a level of sophistication on par with those of the major online judges of the time. PEG's request was denied by the CEMC, forcing Brian to look elsewhere.		With the passing of Ms. Plachta the Judge was never as popular as before, for students were no longer being pushed to earn 1000 points. But there were still some PEG members using it; Canada’s IOI teams trained on it; and Brian and a friend from Waterloo C.I. tried to run a mock DWITE contest. And so the Judge went on. When the Plachta house was sold in June, the Judge was migrated to Brian's laptop, again illicitly. When Brian left for university, he could not risk violating the residence contract, so the Judge was left without hosting. Brian, Jacob, and Hanson were hopeful about convincing the Center for Education in Mathematics and Computing at the University of Waterloo to grant PEG hosting, for this is the organization primarily involved with high achievement in computer science at the high school level, running the Canadian Computing Competition and sending Team Canada to the IOI. Almost all problems from CCCs of years past, both stage 1 and stage 2, were already in the PEG Judge's archive, the interface had taken two years of work, the grader had been tweaked precisely, and, all in all, the PEG Judge was the first in Canada with a level of sophistication on par with those of the major online judges of the time. PEG's request was denied by the CEMC, forcing Brian to look elsewhere.

−	Luckily, some very good VPS plans exist; [http://www.intovps.com/ IntoVPS], for example, offers 512 MB guaranteed RAM, 30 GB disk space, 500 GB monthly bandwidth, and a quad-core CPU for only $10 USD/month. The PEG Judge was initially migrated to an IntoVPS server; a month later it was again migrated to hosting by [http://www.virpus.com/ Virpus], which offers 1 GB guaranteed RAM, 50 GB disk space, 1600 GB monthly bandwidth, and six CPU cores. As hosting is now legitimate, the :5050 suffix was dropped and the site is now ~~http://pegjudge.ath.cx/~~	+	Luckily, some very good VPS plans exist; [http://www.intovps.com/ IntoVPS], for example, offers 512 MB guaranteed RAM, 30 GB disk space, 500 GB monthly bandwidth, and a quad-core CPU for only $10 USD/month. The PEG Judge was initially migrated to an IntoVPS server; a month later it was again migrated to hosting by [http://www.virpus.com/ Virpus], which offers 1 GB guaranteed RAM, 50 GB disk space, 1600 GB monthly bandwidth, and six CPU cores. As hosting is now legitimate, the :5050 suffix was dropped and the site is now {{Root}}