SOL Scoring System discussion
To inform a decision about the SOL scoring system for 2009, the 2008 scores have been re-calculated using a possible alternative system.
There are two reasons why we may wish to review the SOL scoring system for 2009: (1) the new event structure provides an opportunity for change; (2) the old system, being based on the winners time, is simple but does not allow for the strength of competition.
In SCORE (Jan-Feb 2009), Paul Duley suggests that we use the colour (course) as the basis for calculating SOL points. To see the impact of this, I have re-calculated 2008 SOL scores using a course-based formula. The actual formula used is basically the same as for the 6-Days and SoSOL (the average time on a course earns 1000 points; each standard deviation faster than average gets a competitor an extra 200 points; each standard deviation longer than the average loses 200 points). The message from the re-calculation is that it produces relatively little change, but several age classes would have changed hands with this system in place. Because the combinations of age classes and courses have been carefully chosen, there are very few age classes where the average score is significantly less (or more) than the overall average.
The process of calculating points using this system is only marginally more complex than with the current method. The only change is that it will be important to wait until agreed final results have been confirmed for all classes before embarking on calculations as one changed time (eg a re-instatement after a disqualification) will affect the points for all runners on a course.
This file, sol2008_revisedstandings.pdf shows the re-calculated scores against the previous ones. There are some minor errors in this, and it is provided as a guide only. Paul’s SCORE article invites comments on how we should compute SOL scores in 2009. You can either comment below or send them to Paul or Phil Smith (Competitions Convenor), and I will try to answer any questions about the actual scoring systems, see the contacts page.
PS: with the new formula, it is possible to score less than 0, if you take more than 5 standard deviations longer than the average time. Only one competitor, in one event, achieved this distinction in 2008. To save embarrassment, I have replaced their negative score with 0 in the table!
You can see the original/official 2008 final standings here sol2008_scores.pdf
Posted on 02nd Jan 09
by Trevor Hoey - SOL Co-ordinator
Filed under: Fixtures/Events • News • Scot O League
Mark Nixon says:
It looks good, but I don’t think it works on the M21L, which will always be run as a seperate course. Looking at the adapted results it just seems that the races with the most runners give the winner’s the most points. One could win a well attended but poor standard race and get a high score, and win another poorly attended but elite standard race and get a much lower one. But there is not much that can really be done about that.
Trevor Hoey says:
In reply to these two comments:
1. using log-transformed times makes statistical sense, and helps to reward runners who do well. Using the system based on the average time, excellent runs are not well rewarded. In statistical terms, the distribution of times is skewed towards slow times and this can be removed by a logarithmic transformation. We should check that moving to a more complex system does significantly improve the fairness of the results.
However, log-transformed data are not commonplace nor readily understood. So, a question for competitors (especially juniors) is :
“Is it important that the scoring system is simple and easily understood?” Opinions welcome.
Mark’s point is a good one - across all classes more runners usually means higher points as the better runners tend to go to most events, so higher entries usually mean more runners of lower standard. I think without checking it) that Mel’s logarithmic transformation approach will help to reduce the effect of class size. This is something that can be checked.
Andy Paterson says:
At the top of the leaderboard I think this should improve the competition. Under the current system if someone wins the first 4 SOLS then they have won with no further participation required in the next 3. With no cap on the number of points for a win this may keep the outcome open until the last race more often.
Anne Hickling says:
Scoring systems provide endless amusement for statistical boffins, but there is no ‘right’ system. Those who have moved up the SOL rankings with this new system will no doubt prefer that, and vice versa. Even using course rather than class as the basis for scoring, the numbers are still quite small which will lead to anomalies under any system. Personally I’d go for ‘keep it simple’ and understandable.
An interesting feature of M21L last year was that 4 of the 7 events were won by people who only attended that one event. This is clearly going to distort a scoring system. All events should have equal status is the scoring, ie no one should be disadvantaged by choosing not to go to any particular event. Croeso last year tried to address this with a new scoring system which in my view has some merits.
Other recent items in the same category
SoSOL, Muirshiel, Sun 12th Feb
JK 2012 news
World Trail Orienteering Championships 2012
Scottish Night Champs 2012
Scottish O League 2012
Scottish O League 2011
Tay Christmas event, Sun 11 Dec
Start Clocks
Course of the year 2011
British Schools Orienteering Championships Weekend, 2011













Melvyn Perry says:
This is a good start on looking at how we should score the SOLs. I personally like the idea of scoring across the course as you have discussed. This reduces the likelihood of small numbers being used for points calculations. I have been looking at scoring formulae and also the distribution of results using the 2007 6 Day results. My initial conclusions are that the results are best converted by using the log scale. This allows meaningful use of the standard deviation. Once this is done the outcomes are somewhat similar if the current 1000 pts for a win is used, or if the standard deviation approach is used.