How is the McFarlane Prize Judged?

The McFarlane Prize is awarded by a jury of Australian experts in various fields of web profession. The decision process has two stages. In the first stage, entries are assessed for their adherence to best practices in accessibility and standards based coding (correct and valid use of CSS and HTML). These criteria are outlined in detail here.

The top 15 sites from this phase will be individually assessed in four areas by members of the jury who have expertise in a particular area.

These areas are

Based on the two rounds of judging, the McFarlane Prize shortlist will be announced is mid September.

The final announcement will be made at the Web Directions Conference, September 27th 2007.

Who is on the Jury?

We thank all our judges for their generosity with their time and expertise. The Jury for this years McFarlane Prize is:

Pete Ottery

Pete has 9 years experience as a web designer and has designed some of Australia's largest and highly trafficked news and classified websites. He is currently at News Digital Media as the Group Interface Designer and has previously worked for Fairfax Digital Media and Daemon. 

Pete is our special guest first round judge.

Lachlan Hardy

Lachlan currently works as a Design Engineer at Atlassian where he builds functional designs into all their products. He is a web standards expert who has worked on numerous high-profile sites for News Digital Media, Queensland Department of Primary Industry and Fisheries, several Victorian government projects and the ticketing system for the Melbourne Commonwealth Games. He leads the team behind the highly successful Webjam events promoting web innovation in Australia and has written for ReadWriteWeb. He's an advocate of open web technologies and currently very interested in networking the facets of our online identities. His site is Lachstock.

Lachlan is our coding judge for 2008.

Virginia Murdoch

Virginia Murdoch is a partner in Inventive Labs, a young Melbourne web development company. She started life as an editor and a designer of print magazines, but now designs application interfaces and web sites for companies large and small, and teaches design theory and hands-on technical skills to designers and artists. She is inspired by excellent design in any field.

Virginia is the design judge for 2008

Lisa Herrod

Lisa Herrod is the Principal Usability Consultant at Scenario Seven. The primary focus of her work is web usability, which she believes incorporates much more than just user testing. Drawing on a variety of disciplines, Lisa takes an holistic approach to web usability incorporating user research, accessibility, interaction design and web standards development.

Having started in the web during the last century, Lisa is occasionally caught making jokes about font tags, layout tables and shims. Nobody ever laughs.

Lisa is this year's user experience judge.

Henny Swan

Henny has focused on accessibility and usability since the late 90's when she started out working for a Chinese search engine and her own start-up in China. Now working as Senior Web Accessibility Consultant at RNIB in the UK Henny takes an interest in where accessibility standards overlap with mobile best practice and in paticular internationalisation. She is a member of the Web Accessibility Initiative and Co Lead of the Web Standards Project (WaSP) International Liaison Working Group (ILG). Henny blogs at iheni.com and outside of work can be found kick boxing, entertaining and cooking Chinese food.

What are the Judging Criteria?

In the first round of judging, sites will be objectively assessed for their use of best practices in the areas of

as well by our special first round design judge.

In each of these areas a score will be awarded. The 20 highest scoring sites according to these criteria will be then assessed by our Jury of experts. At the Juries discretion, sites may be added to this list, or fewer than 20 sites may be considered for the second round.

First round criteria

Scores will be allocated as follows in the first round.

Accessibility

The home page of the site will be tested using accessibility testing software such as Cynthia Says. Sites will be assessed for their compliance with Level A and AA of the WCAG1 content accessibility guidelines. 1 point will be deducted for each failure to comply with a Level 1 or 2 guideline.

Appropriate use of valid HTML

The home page of the site will be assessed for how valid its HTML or XHTML is (maximum 20 points), and how appropriately it uses HTML (maximum 10 points).

When assessing validity, pages will be validated with the W3C's HTML validator.

The minimum score for this part is 0

When assessing appropriate use of structural and semantic HTML, the following deductions will be made.

Points will be added for use of more sophisticated aspects of HTML, such as the appropriate use of class and id values, and good use of structured and semantic HTML. Sites will receive a maximum of 5, and minimum of 0 points in this section.

Appropriate use of CSS

The home page of the site will be assessed for whether it uses valid CSS.

A score out of 10 will be awarded.

When assessing validity, pages will be validated with the W3C's CSS validator.

Points will also be added for use of more sophisticated CSS as appropriate - including advanced CSS selectors (e.g. descendent and child selectors), and properties. A maximum of 5 and minimum of 0 will be awarded in this section.

Design

In this round, our guest design Judge will award a score out of 20 for each design, based on criteria such as typography, use of whitespace, color, as well as overal appeal.

Second round Judging

In the second round of judging, each of our expert judges scroes each of the sites which pass a certain threshold in the first round in their area of expertise. These scores are then totalled, to produce a shortlist of six sites.

This shortlist is then discussed by the Judges, who name the winner, and any highly commended sites.

Conflicts of Interest

As experts in their fields, the members of the Jury have worked with many organisations, and on many projects. This gives rise to the possibility that a nominated site has an association, either direct or indirect, with a member of the Jury. Our policy is that should this be the case for shortlisted sites, this association will be explicitly declared.