“In every mention of web analytics in support of usable design, authors are quick to note that traffic analysis, no matter how reliable and sophisticated, is not able to provide the thoroughness or level of insight that lab testing can achieve. However, JupiterResearch’s 2005 report, “The New Usability Framework: Leverage Technology to Drive Customer Experience Management,” found that by combining professional usability testing with web analytics, customer satisfaction information, and a/b optimization, usability expenses can be cut dramatically, and purportedly without loss in quality.”

1. When it comes to web design, I would ignore Tufte. He has no idea what he is talking about. Knows statistics, not the web. (Look at his web site for proof.)

2. At one of his talks he went on about how great Jakob Nielsen is. But then Tufte bashes usability testing. ‘Whatever.’

3. I semi-enjoyed his books. But he is one of the worst presenters I’ve ever come across. I went to one of his ‘courses’ and he spent the morning giving a tour of his books and showing us some rare books he owns. What a complete waste. I could have just bought his books and read them for less than the ‘course’. I left at lunch break and didn’t come back.

4. In the middle of his morning session he looked at one of the attendees and complained they weren’t following along in his book. He clearly doesn’t know his audience.

5. He gave a sneak peek of his ‘spark lines’ idea. Give me a break. The example he used was a win/loss record of a baseball team. On the top line was the wins and the bottom line the losses. When asked, no one in the audience of a few hundred knew what it was supposed to be. If you wanted to know how many wins there were, you had to count the tiny (and I mean tiny) lines that were difficult to see. If no one know what it is, isn’t that bad communication? Isn’t the standard of 76:34 easier to see that a team has 76 wins and 34 losses? Or you could just count 76 little lines to learn that.

To be fair, Tufte and you (and me) and the fantastic Khoi all have reservations when it comes to PowerPoint. Tufte boldly claims that “PowerPoint is evil”. And I liked him for that, as I like approaches that “over-reach”, in order to make a point. I am all for good rhethorics. I especially liked the way he concludes his article on PowerPoint:

PowerPoint is a competent slide manager and projector. But rather than supplementing a presentation, it has become a substitute for it. Such misuse ignores the most important rule of speaking: Respect your audience.

He always smiles on pictures and looks quite relaxed. Until recently I though the guy must have a lot of humor. I was eager to hear his destruction of MS Word (very very evil too). In the mean time I think he was dead serious with his “PowerPoint is evil” claim. Dealing with him so far sure was not a lot of fun.

We all know that good slide shows can help our thinking. And of course we all know that programs cannot be evil as such. They can only be hard to use or structured in a way that leads to inadequate, bad or dangerously wrong results. But then it’s up to us, to refuse them or make the best out of it. People that make programs, can be evil and make them tools of evil. Sounds kind of paranoid though.

We know that PowerPoint presentations are just plain crap and from my experience some people who use PowerPoint are evil. I once worked with this business consultant who used PowerPoint to… anyway. There are some good people who use PowerPoint or Keynote and manage to create pretty good slideshows, which leads to my conclusion: PowerPoint is 95% evil, Keynote 10%. ;)

3) ACT-R and John’s CPM-GOMS are interesting approaches, but they’re a notch too academic. In practice it is still much faster, cheaper and more efficient to write a couple of tasks on a piece of paper, get 5 people for 20 minutes and observe them using the site or applications. From my experience with user testing I highly doubt that it can be emulated by machine or a single human.

4) Tufte is a master organizer of statistic information, but his expertise in interactive usability seems questionable. He is kind of proud about his photography as well, but I leave that critique to the experts in that field. Usability experts normally base the necessity of user testing on the fact that they a) have a deformation professionelle (know too much to see the obvious) and b) that noone sees everything.

I consider myself a usability expert enough to assess the main defects of a site (as they’re usually the same), but again and again I am surprised when I see what users do or don’t do when testing. He must be a usability genius without any blind spots. And he must be able to forget everything he knows at the same time, because that is putting yourself in the non-experienced user’s perspective. I can do that to a certain degree, but only a genius can do that perfectly. I am not saying he’s not a genius, I am just saying that for us normal professionals user testing is inevitable.

For us normal beings web design is a never ending process based on feedback. I look at my stats daily. I use Google analytics to see where people click. My website is ONE never ending user test. I use the information I get through my website to create better products for my clients. I continuously develop my site based on user feedback (even on the visceral level, as I get great input from graphically more talented people than myself), and – no matter what the professor says – based on my user feedback I still belive that I have made great progress like that. I cannot imagine an end to that process of improving, I cannot imagine that I’d ever say: “I don’t need to hear what people that use my products say about it.” Frankly, I don’t think that’s genius, I think that’s obnoxious. But I might be wrong. The idea that one day I don’t need user feedback anymore is kind of scary. In interaction design, personally, I enjoy the interaction as much as the design part.

Of course I am not the ultimate expert here. It would be interesting to see what Nielsen, Norman, Tognazzini think about that.

5) Maybe it’s more of a generation conflict: He grew up with role models like Einstein, Corbusier, Picasso, Schoenberg, Gould, Pele. My paradigms are Larry Page, Thom Yorke, Thomas Bernhard, Steve Jobs, Jeff Raskin, Larry David alias George Costanza, Jakob Nielsen, The Designer’s Republic, Herzog&DeMeuron, Zidane(!) and AphexTwin.

In response to the above statement – a different perspective.. is it just semantics or a mismatch of ontologies? Maybe being more precise on the problem and context?

1) the visceral design of the web page may or may not be user tested – many sites have fantastic style in both type and graphics (expert graphic designer) and are useable or not 2) usability design should be tested with the user in an interactive and iterative design-development process – user centered design and software development 3) usability is about efficiency of use. If the user’s profile, goal and action sequence is well understood, then you can use a substitute for the user – a software agent – see Anderson’s ACT-R work and Bonnie John’s at CMU 4) there are experts in the field of usability, and when they know what the user knows – then they can act as a super-substitute for the user in the evaluation of the design – better than the agent – I would consider Tufte to be one such expert.