Designing Based on Data

After my last Usability course this semester a designer/peer, Andrea Richeson, were chatting. She asked me if the course had changed the way I think about designers. I did not have a good answer at the time because I fancy myself a bit of a designer: I’ve created several websites (MPC, SE, SSE, SethHolloway.com) and I constantly build/tinker in both virtual and physical spaces. Beyond considering myself a programmer and designer, I found usability testing to be an amazingly practical part of creating something for people. The methods are similar to my favorite software development methodology: agile development—get the requirements, do a small piece, fix what is broken and ensure people like the effort so far, repeat. After some reflection I can say the course did not really change my view on design, it gave me a framework for evaluating and altering design based on user data.

Fresh out of the class with a solid understanding of the art of usability testing, I was taken aback while reading a New York Times article about a designer who left Google and went to Twitter. He left Google because of their rigid adherence to trusting data. I am confused how a designer could disagree with trusting data. If you do not like this approach, you don’t believe in (1) the way they collect data or (2) trusting the users. The article does not explain how Google collects their data nor does it explain the designer’s disdain, but from what I read I think Google’s approach is perfect.

Google receives millions of unique visitors per day and they are known for changing elements on the page (think of the whimsical, dynamic elements like logo changes and the “Feeling Lucky” button), so it is not hard to believe that Google would put up different designs of the same page. Now, a fraction of the millions of users sees one design and another fraction sees the other design. From the web server’s standpoint, data collection is fairly easy—especially with AJAX-enabled dynamic pages: just stream events to the server for logging. Later, a human can analyze the data and make decisions. With sample sizes in the thousands, statistical validity is almost guaranteed!

The approach is wonderful: people’s behavior determines the best design for the job and the data is collected without the user’s knowledge. What’s better than getting people’s feedback without their knowledge? Hidden camera shows have always exposed wonderful insights, but we can only get at the truth when people let their guard down.

That is how I imagine the data is collected and the approach seems infalible, thus I must conclude that the designer does not believe in trusting users despite the flawless data collection methods. Even outlandish designs can yield positive results. I’m sure I’m missing nuances, but I side with Google. I believe in designing based on data because it keeps us moving towards usable, pleasing designs.