As developers, we try our best to create the best products we can.  We really do.  Yes, I know that some features get pushed to the back burner when releases are behind schedule or some bugs get kludged and we hedge our bets on whether the user will hit them or not.  However, for the most part, we do what makes the application look the best.  I mean why not – we are type A personalities after all (usually).

So, it really bugs (read: annoys) me when users criticize the application that they are using without providing any helpful feed back, what they don’t like or even what went wrong!  Help us help you – we can’t do anything unless you provide us some information.  We aren’t expecting you to debug the application yourself or even send us crash logs – just a general, accurate description of the problem.  However, don’t think that the responsibility is completely on the consumer.  Developers have to at least attempt to easily facilitate the collection of this information.

The good example the Apple App store.  For those of you who aren’t familiar with the rating systems on there, you can rate the app after you download it with a 1-5 star designation.  You also can enter a comment along with your rating.  If you break down what kind of reviews people leave, there are several main categories: 1) the faithful user who expounds the virtues of the application and leaves a glowing review, 2) an impossible to please user that will give your app a 3 star rating for giving the old “college try” (i.e. “This app is good but would be better if it had “x, y and z” feature) 3) a user that leaves a good comment, but poor star rating or vice versa 4) a user that reports bugs and gives a low star rating (i.e. “I updated the app and every time I attempt to launch it, the app crashes!  please fix!!!!” and 5) the type of user who’s review consists of no helpful information and a low rating (i.e “this app sucks” or “the worst application ever”)

To give you an idea of how broken the system is, here is a small sampling of some of the typical reviews found from the 5 categories above.  Note: These comments are taken in the original form and have not been edited.

  • “You get what you pay for.  A BIG FAT ZERO.  WHAT A JOKE.” (1 star)
  • “This app is garbage.” (1 star)
  • “Pure crap.  Sorry devs, but this is bad.” (1 star)
  • “The best app ever.” (5 stars)
  • “A great way to save money.” (1 star)
  • “I love the idea. But every time load the app it thinks for 30 seconds and returns to the home screen.  Good luck guys!.” (3 stars)

If you browse the comments for any given app, you can see they are all over the place.  Let me clear however – good apps deserve good ratings and bad apps deserve bad ratings.  However, you are able to rate apps without even downloading them to try them.  As proof of how ridiculous this is, the night before the iPad launched, iTunes users were able to go to the iTunes app store and download iPad apps.  Now, absolutely no one had an iPad to test the applications with, however the ratings were still pouring in.  How could end users possibly evaluate the application without even running?

Another part of the system that is broken is that the only time that the iPhone prompts you to rate applications is when you delete them off your phone.  Typically, if you’re deleting something, something went horribly, horribly wrong (i.e. the application crashes all the time, won’t run, you don’t have a use for it anymore).  At this point, the user’s rating will most likely be negative, which will end up affecting the rating of your application.

Lastly, there is no way to disable ratings, edit ratings or reset ratings.  The only way they get “reset” is if you release a new version, but even then the previous ratings don’t disappear – the iTunes App store only separates the ratings into “This version” and “Previous versions.”  I am not saying that you should be able to “hide” the previously poor versions, but they shouldn’t affect the overall rating because every piece of software has known and unknown bugs.  The “overall” rating is the lifetime rating of the application.  Anyone who understands statistics averages fairly well knows that once you get some bad ratings that bring your overall rating down, it takes a LOT more ratings to improve the average.  For example, suppose we had 10,000 ratings that gave us an overall of 2.5/5 stars.  It would require another 5,000 5-star ratings to raise the app rating 1/2 star to 3/5 stars.  It only takes longer to raise your rating with additional lower ratings.  This is stacked against the developer and isn’t fair.

So how can we improve the process so that both the consumer and the developer is satisfied, where the consumer gets to voice their opinion and the developer gets to fix the product. First of all, lets remove the functionality that allows someone to rate an app as they are removing it.  Let’s also can the the ability to rate apps without even downloading them.  If we were able to prompt the user for a rating after they had used the app n times or after some total accumulated usage time t (tracked by iTunes) then we might get a far more unbiased opinion.  After meeting these requirements, we could then unlock the ability to rate the app or iTunes could even prompt us for our thoughts (similar to how Amazon and eBay do after you’ve placed an order and they follow up with you to check on the service).

Another idea is that even though this might deter a few people from leaving ratings, good AND bad, have a required comment field that would allow someone to elaborate on why an app deserves a particular rating.  If the user can’t take the 2 minutes to fill out a simple rating form, then they can’t be too serious about rating the application.  This could even be taken one step further by allowing the Apple or the developer to create a questionnaire with different fields depending on the rating.  eBay has a very good of example of this by listing categories of behavoirs that are encouraged. In the same vein, you could also introduce categories.  Obviously this last addition is probably overkill because it is more complicated to implement.

Lastly, it might be helpful to have a rating system within a rating system where other users could click a “thumbs up” or “thumbs down” to indicate whether another user’s review was particularly helpful or completely off base.

While no rating system is absolutely perfect, we want to get the best information possible about a potential purchase. While a rating system should be easy for the user to express their opinion, it shouldn’t be so simple as to give them all the power or reduce the rating to a simple binary “Good” or “Bad” rating.

Of course, the other alternative to all this is that, we as developers could just produce bug-free software the first time we have a app release.