I think that building a new web based product is very much like making a nightclub. Its no longer enough (if it ever was) just to have a great sound system or a cool bar concept. Its about having the right name, at the right place, with the right doorman, serving the right drinks, with the right promotor and DJs. You need the right people to stay for the right time – and you need the right things to go wrong. In short you have 100s of small things that make up an awesome experience / night out. Building a web product is the same. Its not the name, nor the concept or the design. Its the 100s of things that combined make up an awesome web product. The concept, making the right flow, understanding how to attract the users and keeping them, making the site spin fast and in tune with the user ect.. For both product and brands the devil is in the details – and success is formed out of 1000s small great ideas.
As wikipedia was being build, ‘crowd sourcing’ became a buzz word and the worlds entrepreneurs started to dream up ideas where the work of a thousand small participations could out-compete major companies. While this method for sure has been helpful for some projects (e.g. wiki, digg, openstreetmap) other projects like ‘user reviews sites’ (yelp, amazon reviews) are now getting to a point where obvious disadvantages are becoming clear. This is especially the case when used for review of restaurants, gadgets, hotels and so on. My three main concerns with user submitted content are:
1. One mans cool is another mans stuffy. Reviews work when there are clear and shared criteria for what ‘quality’ is – but that is often not the case. More often that not, I’ll check out a place online which will have e.g. 25 ’5-stars’ and then 10 ’1-star’. So what does that mean? That the place is an medium place? Probably not. Its most likely the case that people like different things. e.g. some like posh up-market places where others find them stuffy.
2. Secondly I’ll find that peoples willingness to express themselves when they have experienced something negative far outweighs people with good things to say. So I tend more for ‘neutral data’ rather than getting concerned if a place have a few ‘bad experiences’. For example, a good indications of a good youtube clip is often more ‘most viewed’ than ‘most popular’. (especially if most viewed include a qualifier like ‘most viewed with more than 75% of the show completed’)
3. Cheating. Multiple examples have been publicized where a company hire a group of freelancers to trawl around the web and writing positive reviews of their products or places. Making, at least I, pretty skeptical about the validity of user reviews.
For these three reasons, which in my view will only become worse, I think that user reviews needs new development. This can either be with the introduction of the social graph (e.g. only read reviews from people you know), using more passive data collection with qualifies (see post about behavior generated content) to make the data more objective or we invent new methods to highly reviews form people who are more like yourselves.
