Theme Standards System

[andrew]

I'd like to ask your opinion.

A few days I blogged about the idea of a theme standards system. I'm considering setting this up as an independent site and I want to know if you as a theme user would find it useful when searching for themes or if the theme authors would find it useful to help set them apart. This is in part my way of adding some specifics to claims that themes are premium.

The standards system would be a series of ratings (1 to 5 stars), each of which relate to a set of features and functionality. In general the ratings would mean:

• 1-star - A solid, entry-level theme, built and tested to agreed standards, using accepted practices.
• 2-star - A theme where the look and feel can be customised
• 3-star - A theme where the layout can be customised
• 4-star - A highly configurable blog theme
• 5-star - A theme enabling signficant cms capabilities

The ratings are also comunlative, meaning that a 2-star theme will comply with the requirements set out under both the 1-star and 2-star ratings.

Each star rating would drill down to a much more specific set of features. To give you an idea a 1-star rating would mean:

• Validated (x)html 4 or html 5 (with IE workarounds)
• validated CSS
• compliance with at least the lowest wcag rating
• All marketed features can be controlled using a user interface included within WordPress or added by the theme that is specifically intended for the provided use.(Note this excludes the use of custom fields for advertised features but they can be used as an optional fallback)
• No manual editing of the theme is required for normal operations, i.e. widgets.
• The theme contains all of the key page template types (this excludes item specific files such as category-x)
• contains all wordpress specific css tricks, photo captions, et al.
• Contains all the standard WordPress theme hooks
• Compliant with nested comments

A lot of those points would be expanded to contain a full list of the template files, css tricks, etc.

2-star would go on to list specific ways of how the look and feel could be amended and the minimum functionality that should be provided, and so on.

Obviously a great deal of discussion would need to happen around the contents of each rating to make sure they were relevant to users.

It would be up to the theme author to decide that his / her theme met all of the requirements and to choose to include a badge in their marketing, and tags in the theme repository (if the theme is included). They would need to register the theme for inclusion in a directory where users of the theme could comment on whether they felt something was missing. This wouldn't be something that is awarded.

So, if you are a theme user, would you find it useful for themes to have a star rating to indicate that it provided a level of functionality?

If you are a theme author, would you be prepared to use a rating as a benchmark of the type of functionality to include in your themes and to include a badge in your marketing? Do you think it would help you to set your themes apart from those that do not want to?

[zyml]

From the looks of the requirements above, I think trying to attain a 1-star rating for the themes will be a terrible headache lol. Probably, the only way to achieve all that easily is to ensure that you have a solid framework to work on.

The mainstream users will not bother whether the theme is validated to XHTML4 or HTML5 or CSS, neither will they bother how unique your code is. What matters to them most is the features, how easy it is to customize the theme and the look and feel of the design.

In the end, mainstream users and theme developers define 'premium' themes differently. You might need to balance between the two.

Just my two cents :)

[dancole]

Here is a cool webpage that rates plugin standards on a basic level, but does it in an automated way: http://plugincheck.bravenewcode.com/

I'd like to see a website that has an automated process of rating WordPress Theme by processing their code. Although, I understand that this could get tricky to some degree, but because of WordPress function calls, it shouldn't be impossible.

http://twitter.com/wpthemecritic is being set up by Nathan Rice as a non-profit site that is going to rate Themes.

I think ideas like this are great, themes really need to be held to a standard and a high one at that. I'd be more than happy to run through rating features with you once you get this idea developed a little farther.

[andrew]

From the looks of the requirements above, I think trying to attain a 1-star rating for the themes will be a terrible headache lol. Probably, the only way to achieve all that easily is to ensure that you have a solid framework to work on.

It is true that a lot of themes don't live up to this but I think most are 'starter' themes, that is, people who are coming at theming without any prior knowledge of building websites. I think this list is pretty straightforward.

http://twitter.com/wpthemecritic is being set up by Nathan Rice as a non-profit site that is going to rate Themes.

I am really interested to see what Nathan comes up and how far he pushes the reviews. What I am working on though isn't a review, it is for authors to tell users that they have chosen to meet a defined feature set.

[Laura]

I think the idea of a theme standards rating is brilliant.

I've only done themes for clients before now, but I'm really keen on the idea of making some good themes for the directory, and if I was to get certification on well-made themes, then it'd just be an incentive to do so.

Maybe a lot of theme users wouldn't understand anything about validation or standards, but maybe you could have a list of the advantages, that are more visible/tangible to the user, on the site to explain why this should be important to them.

I never use other people's themes or frameworks, as it's so easy to find dodgy unsemantic code that you end up rewriting anyway. This could save theme authors that build upon other themes a lot of time too, if they knew they were rated as high quality.

If you need any help maintaining that site, Andrew, please let me know. It'd be something I'd love to be involved in!

[duanestorey]

Hey guys, I'm the guy who wrote most of the code for the WordPress plugin checker.

I just want to point out that it's not really a "ranking" system -- lots of plugins that show up with problems are well-used, well-liked, and work fairly well. But we wanted to provide a resource to plugin developers to check their code against common development mistakes (something I've personally learned after lots of trail and error).

But interesting idea about the theme checker. It wouldn't be very hard to add that functionality as well.

[Leland]

I like the basic idea, although I'm not sure about the "star" system. In my opinion, there's absolutely nothing wrong with a theme which you describe as having 1 star: a solid, entry-level theme, built and tested to agreed standards, using accepted practices. Being a "1 star" might imply to some that the theme may be dysfunctional, one that should be avoided.

I can't imagine a theme author marketing their theme who would say something like "yeah, this is a certified 1 star theme." Even though it may a perfectly functional, solid theme, if I was just a beginner, I would imagine being turned off by this "1 star" label.

Instead of stars, maybe there could be word labels such as: solid, flexible, advanced, things like that. Just a thought.

[andrew]

Being a "1 star" might imply to some that the theme may be dysfunctional, one that should be avoided.

That's a very good point Leland. I certainly see the 1-star as the baseline for a 'good' theme, so I wouldn't want to suggest the opposite.

Perhaps a set of titled ratings would be better.

[greenshady]

I'm with Leland on this one. If you place 5 stars on a theme, you're implying it is a better theme or that a 1-star theme is not good. And, that's not necessarily the case.

Overall, the idea is good, but there'd need to be a better system of classifying the themes.

[nathanrice]

I agree with Leland here ... a 5 star rating system would be a flawed execution. It has the disadvantage of communicating lack of quality with 1 star, even if the theme is solidly built.

I've been brainstorming ideas on how to rate themes on wpthemecritic.com, and it's definitely hard. For instance, a solidly built theme, with no validation errors, up to date WP code, a great design, and full cross-browser compatibility just isn't as impressive today as it was 2 years ago. It needs features.

Which is why I'm not sure it's a good idea to submit all themes to the same standard.

In the case of wpthemecritic.com, I'm going to have to take into account the price, the support after the sale, etc., to rate the theme. Ratings won't have stars or numbers (8 out of 10, or 92 out of 100).

A positive review would look something like this:
"Definitely worth the money" or
"As good or better than most themes that cost money"

A negative review would look something like this:
"P.I.N.O. Premium in name only. Don't waste your money" or
"Yes it's free, but you couldn't pay me to use this theme."

Maybe that's flawed too. Too personal. Too subjective. Who knows?

I'm going to consult some people that I highly respect to advise on the process and see what they say. I'm also open to advice from you guys here in the tavern too. Feel free to let me know how you would like to see themes reviewed on WPThemeCritic.com

Nathan
www.nathanrice.net