June 4, 2002 – Keeping Users Stuck to Your Site

Dr. Eric Schaffer: Hi there. I am Dr. Eric Schaffer. I am CEO of Human Factors International. And I have with me a really cool guy, Colin Hynes who is director of usability at Staples. And he has an awesome set of responsibilities. And he is doing some really cool things. And he's going to share with you some of his view points in terms of how to make a site sticky and in terms of how to measure the ROI of usability work. I mean that's something that's really important, because when you hear usability people talk, they sound like preachers, right? I mean you got (inaudible) but they sound like preachers, because we're trying to get a message across. And at the same time usability work is not a religion. You don't do usability work because it's the morally right thing to do. You do it as a business decision. And what you've been able to do if you give us some concrete evidence about how that really happens. Colin has over a decade of experience in the field. He came to Staples, what was it, 3 years ago?

Mr. Colin Hynes: 3 years yeah.

Dr. Eric Schaffer: Yeah. And you've been doing just a little bit of work here in terms of usability. What was it, you said something like? Was it 500?

Mr. Colin Hynes: We did about 500 folks last year in our usability labs. So it was quite a year and it really shows the focus we have on staying close to our customers and really making them the co-developers of our site, which is really a matter for us here at Staples.

Dr. Eric Schaffer: And it shows. And I think it is so widely recognized like being listed as the best website in Forbes. I mean that wasn't just by chance, right.

Mr. Colin Hynes: No, it really wasn't. I mean it it's nice to get the awards and the accolades, but we don't pay attention to them for too long.

Dr. Eric Schaffer: Because really the customers are worth.

Mr. Colin Hynes: It really is. I mean, they speak fairly loudly through their sales and through their behavior and through just any number of interactions we have with them, we get a really good sense of how they are experiencing our site, how they are experiencing us as a company. And I think that it is sort of (inaudible) proxy where we do the testing and then the customers have a good experience, and then the sales are generated. And you really need to understand what kind of sales you are getting and you really need to understand how you measure that and understand how you can build a better site to be more customer centric. It really meets the needs of the business and meets the needs of the customers as well. So that's something that we are extremely focused on.

Dr. Eric Schaffer: And that's something which you developed over the years really. I remember I actually came in as a consultant with HFI and looked at and worked with the very first Staples.com site. And I remember the site was like there was just way too much red. It was hard to read. And it didn't really fit with the way that customers did their work. And it was pretty rugged.

Mr. Colin Hynes: Yeah. I mean, we've obviously come a long way for sure in order to think that the customer-centric view we've had over these years has really been able to you know to push us into being one of the leaders in not just the office supply space online but also hopefully the ecommerce space online. I think that again it comes back to the customers and really listening to that and really scientifically responding to their needs. And not just saying that we know more. We know exactly how they are going to act and what they're going to do on the site, but really backing it up with solid research. And that's why we do 500 customers a year. And that's why we've done 40 online studies last year alone and we ended up having about 32,000 touch points with customers last year between surveys and online study. So when you get that much information, you know, if you use it correctly it can have a really powerful effect on your site. And I think that hopefully our site proves that.

Dr. Eric Schaffer: You know, so many people talk about being customer focused like that's cheap. Everybody says okay. The executives say, we want to be customer focused now. It's like okay, very good. But you've done much more than that. You've done a huge amount of research. Can you tell us a little bit about the organization you got the process, the kind of resources you've been building to make this true.

Mr. Colin Hynes: Yeah, I think certainly we have a dedicated entire organization for usability and also which comprises information architecture as well as usability and usability testing. But I think it's how you really work with the larger organization almost considering them a virtual organization and really staying close to them. There are so many different pockets in an organization to have this intellectual capital about the customer, whether it's you know the folks who answer phone calls or get the emails in or people who are working in the marketing department and conducting focus groups. There's just so much information there that if you mine it correctly, it can be so fruitful to understanding the complete customer experience picture. So it really is a puzzle. You've what are the customers doing and what are they saying and you have all these pieces of the puzzle and you want to make sure you fit them together so you really are getting an accurate picture. If you take one by themselves, sometimes you can get into really dangerous territories. So what we try to do is we try to work within my organization which scales anywhere from say 5 people to 10 people depending on the work load that we have. But it really does call on working very closely with other parts of the organization to make sure we're leveraging them and they are leveraging us. So I think that again we've been really successful in that regards. It's really all about team work. And I think hopefully our site reflects that.

Dr. Eric Schaffer: Yeah. And HFI has been part of the team. I think we're at 22 projects.

Mr. Colin Hynes: It's been a lot. It's been a great time.

Dr. Eric Schaffer: It's an interesting thing, though, what you bring out because user experience officers is really what a lot of companies are starting to talk about. The idea that you bring up is so perfect of going beyond just okay I'm going to do the usability work. Let me really look across the organization and make usability a key differentiator and I think it is so great how Staples has done that.

Mr. Colin Hynes: Yeah. I mean it really has to be. Because understanding the customer and it really is as you said, across the different touch points is that a lot of times companies can get very vertical and they say we're going to understand this customer in this context under this situation instead of saying will they view us especially as a 3-channel way of stores we're cataloging our online presence. Understanding how all those interact, because people don't think of Staples as for the most part just a web company, but that we are a multi-task company. And they have...;

Dr. Eric Schaffer: Many channels.

Mr. Colin Hynes: And they have multifaceted experiences with us. They might, you know, buy something online and then you know walk into a store and return it or you know they might be shopping for a chair and they might see it online. So you know they might really might not feel let's go into a store. Or they might be using a catalog at the website. So you really can't just say you got the web customers and that's what you are doing and that's it. You really need to understand the folks.

Dr. Eric Schaffer: So you started to look across not just the dot com space but really across other ways in which you touch customers.

Mr. Colin Hynes: Absolutely, because that's how customers look at us and again it's sort of spinning that from a company-centric view point to a customer-centric view point. Because they – how we often look at ourselves and it's funny if you look at some of the websites are old and actually some of the websites are new, unfortunately. Is that they really reflect the organizational structure of the company they represent sort of what the customers are saying that they want out of the websites. So it's just, it's really uncanny to see these sites and actually...;

Dr. Eric Schaffer: (inaudible) So this was one of your competitors before (inaudible)

Mr. Colin Hynes: Okay.

Dr. Eric Schaffer: And if you wanted to find post it notes, you had to go to the division that he sits. I said how do I know the division where he sits? You have to know their whole organizational pie.

Mr. Colin Hynes: It is amazing. And I think that's one of the reasons we set up our online testing capabilities to be able to say show a picture of it and may be a description say here's a list of 10 different possibilities where you might find that. Where would you expect to find it. That is a really customer-centric categorization of that really reflects sort of our looking at the customer saying this is what they want, putting it on their website and really reflecting their needs, and us saying you know we need (inaudible) adhesives. We think we know where they live. We're going to put them here instead of saying the customer sites. It really is a fundamental sort of shift in your vantage point about are you looking at it from your perspective or are you looking at it from the customer's perspective.

Dr. Eric Schaffer: And they're almost never the same, right?

Mr. Colin Hynes: They really aren't and you know you find that out sometimes the hard way but you find it out. And it's better to find out the hard way than not to find it out at all.

Dr. Eric Schaffer: Well, sometimes it just lives in the organization. One time, in fact, it is still happening. There was a bank I was working with in Canada. And if you go into the website, you have to establish your user id and password. And if you go to the IVR system, you need a user id and password. You can't use the same password.

Mr. Colin Hynes: Right.

Dr. Eric Schaffer: And if you use the ATM it's a different password. It's like, well, there's nobody like you looking across the overall experience and saying, no, no. Let's look at what the customer is really saying. Let me rationalize that and let me make something that really works.

Mr. Colin Hynes: Yeah, it really is the case. It is the case.

Dr. Eric Schaffer: Very good. So what Colin has done for us, we have a white paper and you can get it on the humanfactors.com website, download it. It's a really good paper. And what it has in it is not only the theory but also a couple of case problems of really measuring the ROI with this kind of usability work. Now, what you're going to see, Colin has this huge effort going on and he has picked a couple of very specific containers of activity. So it's a specific project where he can actually make a measurement and I think that, that measurement is huge and significant, and yet I think the overall value (inaudible) what you measure in these studies.

Mr. Colin Hynes: Yeah. That's certainly true. I mean we do effect the organization and the customer in so many different ways. It will be possible to try to measure that for sure. I just picked out a couple that are indicative of the kind of work that we do and the kind of return you can get from doing the sort of work. So I think that they are representative to most extent, so.

Dr. Eric Schaffer: Now there's a button that you pin to submit questions. And after this segment where Colin talks specifically about the paper and about ROI, we'll take those questions and try to answer them and chat about it. Now, Colin can answer specific numerical questions about the performance of the Staples site. So you have to understand that there is some code of confidentiality issues, there's a sense of (inaudible). But any kind of questions about Human Factors technology, usability, engineering, card sorts, any different kinds of approaches to drop off rate which is really what Colin's looking at, I think that's going to be open for discussion.

Mr. Colin Hynes: Absolutely.

Dr. Eric Schaffer: Okay. Alright. So let's go ahead and play that tape and we'll get back after that and take your questions.

Mr. Colin Hynes: As Eric and I discussed in the introduction, today we're going to talk about keeping users stuck to your site. When I talk specifically about ecommerce sites today, you have more specifically Staples.com, but certainly some of the things we talk about today will be applicable to non-ecommerce sites as well. So what does keeping users stuck to your site mean? We are going to get into some detail about that through the following agenda. First, we're going to talk about the value of drop off data. Then we're going to talk about the effect of drop off on ROI or return on investment. Thirdly, we'll get into a case study on search on Staples.com and some improvements that we made and some drop off improvement numbers that came out of that redesign. In the second case study, we'll go through registration process on Staples.com and also talk about some quantifiable gains we made in the drop off percentage on the registration process as well. And finally we'll conclude and take some of your questions.

So first, let's start talking about the value of drop off data. And to even understand the value of drop off data, we first need to understand what drop off is. So drop off defined is the page where the user leaves the site. This is sometimes referred to as the last page viewed or visited. It will always account for 100% since all users will leave somewhere on your site, whether they leave before checkout, after checkout is a different question. But unless they keep your site up as a screen saver for all eternity, they will leave it some point in time. And because we are talking about an ecommerce site, the goal often times isn't to keep them on your site for a very, very long time specifically, it is more to make sure that they make it through their task. And if that task happens to be buying something on your site, which often times is an ecommerce, that you allow them and enable them to do that in the easiest way possible that responds to their needs as a user. So when we talk about 100% of users leaving your site, then you really need to understand the degrees of drop off badness. So there might be some areas that will be acceptable for users to drop off from. So after they've made an order. So say they submitted the order page for e.g. or may be they find a store page or even they'll request a catalogue page. And if you look at those two pages, find a store and request a catalogue, those are parts of the site where users might not be buying at that specific point in time. But they might be there for other tasks. So they might be looking to find a store. Or they might be looking to get a catalogue in the mail and that's their task for that day. Most areas, however, are not acceptable for users to be leaving from on an ecommerce site. The closer the eye on the check out process, the less acceptable that drop off is. So if they're half way through checkout and they are leaving your site then that's an area of concern that needs to be addressed. And again certainly as they get further on, the less acceptable that drop off is. So where the users are in the check out process and where they drop off can lead to many hypotheses about their needs and behavior. For e.g. you have to first figure out where in experience are they leaving from? So are they leaving from their home page, or are they leaving from a review order page, which may be your last page before they submit their order and submit their credit card information. Clearly they are at the opposite ends home page and review order page are at the opposite ends of the check out process and they tell you different things depending on where they leave. So you really have to look at what the path is that they took before leaving. So did they go to a leather chair product page and then left from the find a store page. If they did do that and that was their behavior, you might have hypothesized that they were looking for a leather chair. They want to make sure that Staples carried the leather chair they wanted. And then they tended to go into the store to buy that leather chair. Especially with the leather chair, where folks might want to sit in it and feel it. That might be a very plausible scenario. However, drop off alone can only give you what I call blind data. To really dig into the actual causal factors, we must conduct scientific research with our users in order to really determine why they are leaving, what's going on in their head when they are leaving, what we're not doing to be able to support their interaction in the way that we need to be supporting that. So this number of different methods you can use to understand these causal factors. The first will be survey. You could also use focus grouping or field studies where you go into the field, their work places, or their homes and understand how their physical environment contributes to how they are thinking about their interaction and how they are having that interaction with you. And then finally usability testing. These are just a few of many different methodologies that can be used as you try to understand their behavior and how you can respond to that behavior.

So now let's talk a little bit about the effect of drop off on ROI. While I can't get into specific figures about drop off related to Staples.com, I can give you some percentages later on in this presentation. So drop off improvement is just one way to gauge ROI. There are many others and some of those include a reduction in contacts. Also an increase in average order size, which is the amount of the purchase per visit that a user is making. Also an increase in frequencies of order across the customer base. So people buying more often. A reduction in material costs. And finally an employee efficiency improvement. So if we are improving our efficiency of our employees, they are able to be more productive and thus more profitable for the company. So again, the further users are in the buying process, the higher the conversion rate would be. And the closer to bill they are, you assume that the more investment they've made in the experience and higher the likelihood of purchase.

So start through in this next slide an ROI calculation model using drop off. This model is a great one to use for usability practitioners or anybody who is a website operator to really understand the gains that you are making when you make a redesign. And this illustrates a before and after look at ROI when you do a redesign. So you need to have information before you do the redesign and then you need to contrast that with the information after the redesign. So in step 1 we're looking to get the annual sales increase. So where you're looking at is first finding out the decrease in monthly drop off. And that is the one number of visit drop off from that pager area that is happening between the non-redesigned pages and the redesigned pages. Then you take that number and then you multiply that times the conversion rate, which is the percentage of visitors who buy during a single visit. You multiply that times the average order size. Or again how much is the average purchase amount in dollar figures for an average order. And then you multiply that because this is an annual sales figure, you multiply that times 12 months. That gets you your annual sales increase. Once you have that number, you are able to move on to step 2. And step 2 takes that annual sales increase number and subtracts it from the cost of the project. The cost of the project could include multiple different variables, things like opportunity cost, and development cost, and consulting cost, and marketing cost, and other costs as would hard ware cost for e.g. So you need to get that full picture and then subtract that from the sales increase and that gets you your first year ROI impact.

So now we will go through one theoretical ROI calculation to use some real numbers to illustrate the point about using drop off and figuring an ROI off of drop off information. So in step 1 for your annual sales increase, say that you had 10,000 less users leaving a certain area of the site that you have redesigned based on that redesign and making it more efficient for the users. You take those 10,000 users and you multiply that times a 5% conversion rate. And the assumption here is that 5% of those 10,000 users would have converted or would have bought, because you had saved them, and they would have normally dropped off. So then you take that number and you multiply it by $100 average order size. So you are figuring that of the folks that made it through the check out process, the average order size of the users is about $100. And then you multiply that times 12 months to get to your final annual sales increase, which in this case would be $600,000. Then you take that $600,000 and you subtract it from again a very theoretical number of $100,000 cost of project. You take the 600,000 minus the 100,000 and you get a half a million dollars in first year ROI impact for that project. So this illustrates to you how you can take that drop off calculation and really put some numbers around it to really get a sense of what was the profitability of this project.

So now we are going to talk specifically about one case study which is search. And search is an area of our site where a user types in a box, types in a product name or item number into a box and then they click search and they get a search results page. The search box is throughout our site and it's on probably 95% of the pages or so. And it enables users who at any point in their process to find a product that they need so that they can buy that product or research that product. When they click that search button what happens is they get a search results page. That search results page will helpfully bring back the exact products that they wanted. When we were doing a redesign a while back, we redesigned our search results page. After we launched that search results page, we found out that we were getting a high drop off from valid search results pages. When I say valid search results pages, what I mean is pages that bring back a product. So if they type in a search term and nothing comes back because we have nothing in our database that gives them or that matches up to their product search, then what happens is they get a page that says there's nothing brought back. We are not talking about that page for this discussion. What we're talking about as when they type in a word that brings back an actual search term, one or many. So again, we were finding a high drop off from that valid search results page. We were also finding a high drop off from pages linked from the search page. We'll talk a little bit more specifically about that further on in the presentation. And finally, we did a usability test doing a competitive analysis that indicated their issues in our search results page. So once we found that we had this high drop off and we were seeing these issues in these competitive usability analysis, that indicated these issues, we said we need to undertake a research plan to understand specifically what we can do to address these issues and how we can make that area of the site more efficient and more user centered. So what we first did is we did a heuristic review with HFI looking at our site and said based on this information we're getting, how can we make our site better? Secondly, we did best practices analysis of not only ecommerce sites but a lot of other search sites that are just search sites like Yahoo or Google or say an Altavista, because as people approach searches on an ecommerce site, what their mental model or their sort of picture in their head of what they are expecting to see could be based on those searches that they do on non-ecommerce sites. So once again understanding of those best practices not only on say the Amazon.coms of the world but also on the Yahoos and the Googles of the world as well. So after we had this information from the drop off data that we had from usability testing from the heuristic review and the best practices analysis, we redesigned the search results page. Then we did some in-house usability testing on those revised screens. And what we're going to show you next is some screen shots of the before and after and highlights some of the changes and improvements that we made based on this methodology.

So what you see on the right hand side of the page is our previous search results page. The green dotted line that goes through the upper half of this screen shot is where the fold was. The fold is where on an 800 x 600 screen resolution the user is seeing as a first screen. So that's the first thing that they see without scrolling. So in that target resolution which was 800 x 600 for Staples.com, what we saw was that users were coming to this site and expecting to see actual products. When they do a search, they want to see pictures of products and names of products specifically related to their search. However, because of so much information on top of our page, pushing the actual search results down was the product of that design. So when people came to the site, they said, I'm looking for products, I'm not seeing products in my first screen. So what they are actually doing, was they were clicking in that grey box and they were clicking on those category likes. So if someone searched for Hewlett Packard, one of those links might be printers. When they click on that printer link, it'll take them to all the printers, not just specific Hewlett Packard Printers. So that was a real mismatch from what they were looking for and it was causing lots of confusion for the users.

So what we did to take care of that issue and you see on the right hand side the current search results page is we tightened up the area at the top so that products came above the fold. And as you see from the green dotted line, now you have about one and a half products that are above the fold. We also moved the search results summary down to the bottom of the page and we removed the category links. So overall we tightened up this page and we were more aligned with the way that users were expecting to see this page. They researched, see products at the top, we responded to that. From that response of methodology that we used, the results were thus: Qualitatively for usability testing on the new screens, we found that users were able to locate the products more easily. Their errors were decreased significantly and overall satisfaction increased as well. If in the quantitative perspective, the drop off on valid search results pages decreased by 10% and again I can't get into specific numbers and what that means for dollar impact. But suffice it to say that the kind of astronomical traffic that a site like Staples.com gets, a 10% decrease in drop off from the search results redesigned gets you an incredible amount of ROI and pay for this project many times over.

So for our second case study, we are going to talk about the registration process on Staples.com and on improvements we made there via drop off data and other approaches. The registration process on Staples.com is where users go to fill in information like billing and shipping information so that they can go through the registration process so that we know where to send products. It also enables them to put in their username and password so that they can come back on subsequent visits and not need to fill in that information again. So with that phase some of the issues we faced after redesign of the registration process was that we were seeing a high drop off throughout the registration process. It's multi-paged as we'll see further on in this discussion. We were also seeing a high contact rate from users regarding log in and registration. This was basically when uses would call up our call centers and say I'm having problems trying to make it through your registration process. I'm not sure what I should put into this field. I'm not sure how I get through this process. I'm not sure what this information will be used for. There were a number of different issues that users were calling in with related to the registration process. We were also finding that usability testing through that competitive analysis that we also used to gather the search information was indicating some issues for users, which we will talk about further on.

So the research plan that we undertook with registration was similar to the search research plan as well. First we started with a heuristic review that we conducted with HFI. And then we went into best practices analysis. In the best practices analysis in this case our search was talking a lot about what were sort of the best practices for design and layout looking field the best practices for registration was as much centered around design as it was around the kinds of information that sites were asking for and the way that they were asking for that information. We'll talk about why that's relevant in a couple of minutes.

After we'd had done the heuristic review and the best practices analysis, we redesigned the pages and then we got into step 3, which was the in-house usability testing of those revised screens. So specifically, here are some issues that we uncovered and how we made some of the changes and improvements of the site. On the right hand side you see the previous registration process. And again you see that green dotted line which indicates the fold. What we find here is that there is a significant amount of extraneous copy at the top of the form there's making a peer a lot longer than it really was. And again if you look at that fold what you're seeing is only one of the fields was above the fold. There was a lot of introduction copy that could have really been tightened up. There's also hit and billing versus shipping check box that was leading to data in shipping errors. And there was a whole other issue in the way we handled errors which we will talk about in a moment. But this hit and billing versus shipping was creating some of those errors. There was also all the required fields were not marked. And finally on that bottom of the amounts of fields we were asking about is that there was one required and three optional shipping information fields that were right on top of a large optional information area. These two combined, the one required and three optional and the large optional informational field which you see by the arrows at the bottom of the screen shot were just creating a very large page that we found out from users was just overwhelming. Too much copy we were hearing. Too many fields we were hearing and that's where we were wanting to address as part of our heuristic review as part of our redesign.

So the revised screen which you see on this page reflects the changes we made from the usability input. First thing we did was we trimmed the intro copy at the top to make it only relevant information. Secondly, we tightened up the form spacing overall which tightened up the page. Thirdly we relocated the billing as shipping to a more visible location on the right. And we marked all fields as required that were required. And we consolidated, very importantly, we consolidated one required and three optional delivery information fields into one optional information field and we relocated those optional user information fields to the second page. Through our competitive usability analysis what we found is that a lot of the other sites were asking the amount of information that we were on this area of the site. And what we found is that the users were referencing back to those pages and those sites and saying no xyz site does not ask me for this informational strings that you do. So we coupled that with our best practices information and we really trimmed down the amount of fields that we were asking for, only the ones that were absolutely critical for us to get them a package on time and in the manner that they were expecting. So that trimming overall and the tightening up of the forms and the reduction of the copy really creates a lot tighter page, a lot more under whelming page and we found through usability testing that that was very, very positively received.

We mentioned errors earlier on and now let me talk a little bit about how we used to do error handling. Previously and you see on the right hand side our previous error page for registration was that when a user made an error, we played the page back to them that was basically like an island. All it did is list some copy at the top that said you have an error that was encountered from your previous registration and then it listed out the errors that user made and it provided them with a link back to that registration page. And what we were finding is that because this was so separate for users that it made huge tax on their working memory. What the users would have to do is they'd have to get to this page, remember the errors that they made, and then click that back link, go back to the previous page remembering those errors and then correct those errors accordingly. And as you can imagine, there was a lot of load that was put on their working memory, especially when they might have 1, 2, 3, or 4 or more errors potentially that they encountered. So we really found that this was one of the biggest issues that we had in registration.

Also if the user clicked the back button in the browser, not the back button in the text on the page, and when they got back to that page that their fields were cleared out. So they'll have to go forward and fill out those fields as well. So these things coupled together were really creating a large issue on the error page that we had previously. So how we handled that was we played back the errors on the page where the user made the errors. So if you see on the screen shot on the right, that there is a grey box at the top that alerts the user that errors were made in the registration process. This really decreased the memory work load that we were requiring from the user and we also with that thought of decreasing the amount of memory required, we created more context-specific error alerting which down below in the page you see it's bold and red by the fields that errors were encountered. So that way as they scroll down the page, they didn't have to remember the fields that were in error because they were bold and they were red.

On the second page of registration, which is the log in page, this is where the user is filling out some information, where they are putting in username and password and some other information so we can identify them to the system next time they come to the site. So what you're seeing here on this slide on the right hand side is the previous registration page. In the previous registration what was happening is users would come here and put in the user name and they would put in a number of user names that they thought would be easy to remember once that they may use another websites. The problem is with over one million registered users, there are so many user names taken that the user names that users would want here would not be available in many cases. So what happened is they would go through the site below, putting a user name and not being available, getting an error page. Having clicked the back link, coming back to this page, putting in another one, there was this sort of long vicious cycle that the result would often be that the user name that they finally came up with that was available was so un-intuitive that when they came back to the site on subsequent visits, that they would not be able to remember it easily. We also had a reminder question on this page that the users would fill in so that if they came back to the site, and they forgot their password, that we'd be able to ask them this question that they wrote in and then they would put in he answer that they wrote in as well, and if they got the right answer to the question, we'd give them their password. But this was requiring users to come up with a reminder question and then type it in and then come up with an answer to that question. So a lot of cycles would have to go through on this page as you can imagine, to come back with some information that folks could remember and that was serving their needs.

So how we handled that on this next slide as you see on the right hand side is the new registration process. What you see at the top is that we pre-populated the username fields with the email address so this would really save on the thinking and input time and really increase the likelihood of recall. So now what we did is the email address, which is specific to that user, would be pre-populated right there to say hey we know that this is something that you'll remember because it's your email address. We know it's specific to you. We enable the users to be able to change though that information if they wanted. We thought this was a good way to ensure that they wouldn't go through those cycles of trial and error. And the password reminder question we pre-populated with mother's maiden name. And this really saved time and effort in coming up with a question. It also created a halo effect. If you think back to credit cards that you might be applying for or other financial transactions specifically, one of the keys that they use is mother's maiden name. So we thought it was a halo effect of security that we would ask that. Certainly Staples.com is a very secure site and we want to make sure that that came across in everything that we do including this password reminder questions. And as I mentioned, on top of the username this field was editable. So if they liked to change it to something other than mother's maiden name, they could do that as well.

So from these changes there were many incredible results. For us qualitatively through usability testing, we found that users made it through the registration process more quickly and with higher satisfaction. From the quantitative perspective, there was a huge, huge win in registration. The drop off on registration area decreased by 73%. So that means 73% less users who were dropping off prior to the re-design were now dropping off after we re-designed those pages. Just an incredible number if you think about what registration is doing which is helping someone through the checkout process, enabling them to give us information so that they can closer their relationship with us. A 73% decrease was incredible. And again, although I am not able to talk specifically about ROI numbers, you can imagine a 73 decrease was something that really had an incredible impact on sales and incredible impact on our business.

So now we are going to go into the conclusion of my presentation. So some of the key takeaways here are that drop off is valuable in gauging potential missed opportunities. Secondly, the drop off impact is relative based on where the user is in a shopping experience. We talked about the further they are in the experience, the more likelihood there is that they'll be buying. So when you're looking at the drop off and the impact, you need to be able to gauge that. Drop off as a number can be used as a key metric in calculating ROI of initiatives. And finally, that a systematic approach to site evaluation can reduce drop off and increase conversion rate, thus increasing ROI, which as business operators is something that we're all very focused on.

So thank you for listening to my presentation. I realize that you all have been sending in questions throughout this presentation. And now we'd like to get to some of those questions with myself and Eric.

Dr. Eric Schaffer: So here we are at the conclusion of Colin's presentation. I'd like to just point out a couple of things. One is that if you want to see some of those images, we have them in the white paper and you can download them from the humanfactors.com website as well as the logic and the good material that Colin has. Also again if you want, we've been getting some great questions. We're going to answer them in just a couple of minutes and if you want to send in some more, we're more than happy to answer questions, on ROI on organization of usability efforts within companies, or usability engineering methodology in general. Or the future of the India and Pakistan problem. So what Colin's done is really wonderful. Because he has taken usability engineering in all its complexity and he has picked several examples which concretely and unambiguously show the value of doing usability engineering in a business environment. There's real dollars involved in it. It isn't just that it's nice for the customers and they got to have the credit cards. That matters. So credit cards definitely matter and it's exciting to see that taking place. But what I want to bring out is you got three examples here that sound really, really clearly good. But are those like wild good cases? Or is that more like an everyday occurrence?

Mr. Colin Hynes: I'd say that certainly the search case study is probably pretty normal, whatever's normal I guess.

Dr. Eric Schaffer: Don't worry if you can do it regularly.

Mr. Colin Hynes: Yeah, exactly. We can repeat it. I would say that the search was sort of more on par so to get that 8% or so reduction in drop offs. If you look at registration at 73% number is really. I wish we could repeat that. But certainly, you know, that is really that was just a phenomenal gain on that area. You know we go into these and we look to improve and a lot of times we don't frankly have benchmarks to say we hope to improve x amount of percentage and we're hoping to improve.

Dr. Eric Schaffer: You got a gut level feeling you can...

Mr. Colin Hynes: Yeah, you do, I mean the dust settles, then you know lot of times that's when you are trying to make these determinations. A lot of times, unfortunately, you need to get funding for these projects. So you need to take sort of a gauge on where you might end up at the end to get these funding. So, fortunately, at Staples we have such great support internally from the highest levels of the organization that we often don't have to go through those exercises to really rationalize usability before we get into these projects. But certainly, on an ongoing basis you want to share your work just like we all do.

Dr. Eric Schaffer: You were saying that usability is really part of the mission, I mean, customer approach. Can you talk to people about that a little?

Mr. Colin Hynes: Yeah, I think it really starts from the top levels of the organization. While they might not know the intricacies and you know scientific principles of usability, if they can appreciate it, then it really, really helps when you are running your usability business so to speak within a larger organization. So this company embraces it, you know, I mentioned to you before we kind of web cast it. Coming from a retail company where sort of the matter has been the customer is always right and you know certainly the customer isn't always right in every situation, but you try to support that. But I think just having that sort of philosophy really trickles down into a lot of different areas of the business and I've seen it in usability and again it is sort of having that sort of mind set of customer focus which really has helped myself in building up this usability competency and helped my team in really doing some great work and implementing some of our thoughts and some of our findings. So it's been a good ride.

Dr. Eric Schaffer: Very good. Very good. Now there's something that I wanted to bring up with you that I keep thinking about. And that's when you look at this drop off rate, if you are having drop off right on the critical path of doing our check out, it's a done deal. It's very simple, right. But when you look at something where say you are (inaudible) to be a known customer or your signing up for a news letter, how do you get any kind of ROI number from that?

Mr. Colin Hynes: Yeah, I mean I think it's definitely an inter-departmental exercise for sure because you know you might, there's different sort of ramifications of getting that information. There might be what's the acquisition cost of getting information about a customer so that you can accurately you know market to them. so that's just one area of it, but there's just so many different other inputs when the customers want to speak to you. The worst thing you want to do is say we don't want to speak to you. And there's been research about inbound calls versus outbound calls and the effectiveness of inbound calls and really the web is about an inbound call for the most part our people coming to you saying I want to do business with you. I want to in one way or the other might not be transact. It might not be I want to check out on your site. It might be you know I want to sign up for a business rewards credit card. I want to do research so I can go into the store and buy something from you. So you really have to sort of temper your ROI number with the multiple different views from folks who might be using that information in the way that the customers are interacting with you. So it really is you know really depends on what you're looking at.

Dr. Eric Schaffer: So you go to the marketing department and say, I'll make you a deal. Give me $3 a head and I can cut the drop off and see you get that. And so if they want to get more people signed up for a program, if they want to get more credit cards then you can really essentially sell that to the organization and have your usability work not just pay off in conversion but also in all kinds of other good things.

Mr. Colin Hynes: Right. And We're talking about an ecommerce site today, but I would think that there are some folks out there are many folks out there in the audience who aren't either working on an ecommerce site or who's work directly affects an ecommerce site. So you just do have to look at all the different effects that your work is having, whether it's conversion. And that conversion again might not be conversion for buying a product or buying a service. It might be conversion of giving information, filling out a form correctly, and all the different things as we saw in registration. All the different things that can go wrong.

Dr. Eric Schaffer: And what you value from a business setting, one of the interesting, so you got it easy actually. So imagine if you are in government, if you're in the government, you don't care about conversion, right? You're not selling anything. But what you care about is the voter's trust. So we want to be seen as a site that's doing e-governance and really supporting. So depending on what kind of organization you're in, I'm trying to save employee time, on (inaudible).

Mr. Colin Hynes: And I think there are so many different dimensions depending on what your business needs and business goals are. At the end of the day, it really is about understanding what the customer is trying to do, what you're trying to do as a business in trying to satisfy both of those. That's really the whole grill. You can satisfy users and the business at the same time. Certainly some times those, you know those are in conflict will say that if the business goals and user goals always matched up, we'd be giving away products for free. Everything would be zero dollars on our site. So certainly you know.

Dr. Eric Schaffer: There isn't a business.

Mr. Colin Hynes: It would be very quickly so (inaudible)

Dr. Eric Schaffer: Although Staples sells it very reasonably.

Mr. Colin Hynes: Yeah. Absolutely, absolutely. Close to zero.

Dr. Eric Schaffer: Almost zero. Almost free.

Mr. Colin Hynes: Right almost. But certainly there's lots of dimensions yet to look for.

Dr. Eric Schaffer: Yeah. Let me go ahead and take some of the questions. I have one critical question. We're getting tons of great questions, that's good. Very good. But I have one killer question I want to ask, but I'm going to ask for some questions coming in. So, let me go ahead and see what we've got. Okay. Since Staples is an international corporation, do you conduct these tests in other countries as well? How do you find international users utilize the web to do business? It is a lot different from the U.S.?

Mr. Colin Hynes: The answer to the first part of the question, we have done extensive usability testing. We've launched a Staples.de site in Germany. So my team worked very closely in Germany in the field to do usability testing with folks in Germany. So it was an interesting exercise in that you can read a lot about localization and internationalization and you know high contacts and low contacts and power distance and all these different sort of variables in different countries or in different cultures.

Dr. Eric Schaffer: Which is about people having different styles of...

Mr. Colin Hynes: They do and a lot of it has to do with you know their colloquial history and their religious history and how they teach in schools and things like that. So there are these lots of different dimensions you have to look at as far as culturally it is concerning. And we actually when we tested in Germany, we did see some of those things coming out. In that we found that yeah, you know lot of what you hear about German users, at least what the literature says is that they have the sense of needing external control. So they say tell me the way that you want me to go through this site. Don't throw a lot of stuff at me and just sort of prescribe the way I should go through. So that will lend itself to things like what called mono pages where...

Dr. Eric Schaffer: (inaudible)

Mr. Colin Hynes: Yes, exactly. Almost like a wizard-like approach where the U.S. on the other side is very much internal controlled. If you think about sort of the historical backgrounds of where both the Germans came from and the U.S. you know, say that sort of the capitalistic ideology of you know it's all for me and I want to screen them to be able to do what I want to. So you build these interfaces that have a lot of flexibility to go wherever you want. But then if you look at you know German culture, again the research will say that you just tell me which way to go and you mark the path and I'll follow that. And while we didn't find that sort of that block in the way issue come up so much. It wasn't so sort of diametrically opposed. But certainly there was some gray there, but by and large that pretty much played itself out in our test in Germany. So it was really interesting when you do the reading, you say is this really true. And when we went into field, it did bare itself out very well and our design reflects that and we've got you know great reviews after we launched. We had you know lot of great work that went in the testing to really hone it down to really respond to their individual needs. And it certainly is not just about cultural needs like things you know like internal and external control, but it's a lot about the kind of the more sort of prescribed needs of like how can – do they have zip codes? They don't have zip codes. So we can't, what's their zip code format? What's their you know their the way that they can accept credit cards or not? Do they have credit cards? Can you do sweepstakes? Can you do what are their rules and regulations? So there is lot of sort of legal aspects to it too that we have to pay attention to. So when you put that all together, it's a really great challenge and I certainly think that we stepped up to that and most specifically with the Staples.de site.

Dr. Eric Schaffer: That's great. Our Managing Director Apala Chavan, our Managing Director in our Mumbai, India office tells a story about a detergent. And they had ads where they would show filthy clothes and you are driving on the street where you se a bill board of filthy clothes and the clothes with a detergent going into a washing machine. And then beautiful clean clothes. But in the Arabic countries, they read the other way. So they showed the nice clean clothes going into the washer and then coming out filthy. So I mean there are so many tricks like that where we.

Mr. Colin Hynes: Icons are always a big mine field. And hand gesture.

Dr. Eric Schaffer: Any hand gesture means up there somewhere.

Mr. Colin Hynes: Yes. You got it right. So you just have to be so, so careful. So we do pay attention to those things. And hopefully decide how to reflect that.

Dr. Eric Schaffer: Good, very good. Okay. Do you incorporate your employees' web experience in your work? I guess that means when people say (inaudible) we had a VP come in and say will I go on the site?

Mr. Colin Hynes: Yeah, we do, we certainly do. You always have to temper that with are they our core customer? Because if they're not, then it's good in putting, you surely don't dismiss it, but at the same time you have most web sites at least we you have a core set of customers. We don't turn anybody away. Certainly, if you want to go to our site, you'll be...;

Dr. Eric Schaffer: (Inaudible) Staples is open.

Mr. Colin Hynes: It's open for business 24 hours a day.

Dr. Eric Schaffer: (Inaudible)

Mr. Colin Hynes: Certainly. But at the same time you do have a core set of customers if you really want to be a successful site, you target on. So if they meet our target, then you know, we'll listen even more than we do when they don't.

Dr. Eric Schaffer: You know, one of the things I find so dangerous about that is that within a corporation, there's a whole knowledge set perspective. To give you an example of that, I had one bank I was working with. And I went into their site to find a Harley Davidson product. I wanted a Harley Davidson product. Really they had that under, they listed it under affinity cards. Now how a user is going to know what an affinity, I mean that's standard banking. They talked about plastics too. What is a plastic? Well, if you're in the banking industry doing credit cards, you understand and you might not see that a customer wouldn't. So, you know, there is such a disconnect between the internal user and the external.

Mr. Colin Hynes: When you are an ethno-centric, you really would look at the world through our own eyes. And it could be a very dangerous thing and you know it happens very often in less sort of user-centric companies that they think that they are the user, you know, because they've taken this sort of you know taken these sort of tasks before that now I know the user and that isn't always true. It is actually very dangerous. So you have to make sure you bring the customers in. You can't assume that you know what they're going to do and what they're going to say and how they're going to react. You have to back it up with really scientific research and get the customers you know on your web site or with your catalogue or in your stores and really try and understand (inaudible) and do it constantly and don't think because you did something 2 years ago now that means that's – because it changes your business. You know the customers change and.

Dr. Eric Schaffer: The customers remember only on the (inaudible) scrolling down 10%. These people know best scrolling back.

Mr. Colin Hynes: I mean people feel more comfortable. There's you know security concerns that will hopefully be mitigated. There are laws that are passed that can help those things out and they publicize those passing of laws well you know hopefully that trickles in to how to go to a web site and not do you make it any less secure but just the way you can communicate to these people.

Dr. Eric Schaffer: Secure, but they are not going to be hacked. That always seems (inaudible). Can you imagine like you're more worried about giving your credit card over a secure web site than to some pimply faced kid making you know minimum wage who's your waiter. And you hand them your credit card and go off and come back 20 minutes later.

Mr. Colin Hynes: That's a classic case study about it and it's the argument that the web folks used to use. You know you don't walk to a restaurant and give away your credit card and pop there it goes. Or you are talking to a cell phone and you are giving your credit card number over a cell phone or you know or a portable phone where you just you know that's not safe.

Dr. Eric Schaffer: Okay, actually, it's (inaudible). But let's go ahead and take the next question. I'm the only usability person in a company of 500 people. So I end up doing user research, heuristic evaluation, and interaction design, and usability testing for lots and lots of projects. Obviously, I'm a bit stretched and not always that effective. What do you recommend I do to promote getting resources to build a multi-person usability group and institutionalize usability within the company – getting it formally integrated with the software – web site design cycle?

Mr. Colin Hynes: Besides showing them this web cast.

Dr. Eric Schaffer: This will be good.

Mr. Colin Hynes: No, but it's really it's a you know it's a depending on the company's view point, which isn't contained in this question, but trying to understand what kind of support you have in general. Is it a question of understanding or is it a question of folks understand, but yet they are not willing to support. I mean that's obviously a tougher nut to crack for sure. But I think that the thing you can do is consider yourself really a marketer of your services. Consider yourself a PR person within your company without being really you know obnoxious about always trying to get in front of people. But when you have wins, you know, make sure they are quantifiable. And make sure that you go into these people's you know areas and really talk it up and don't feel like you are self promoting, because certainly from the tone of this question, it seems like you probably do need a little bit of self promotion to be able to really sort of get your message out and really tell the good things that you are doing.

Dr. Eric Schaffer: This person sounds (inaudible)

Mr. Colin Hynes: Yeah, they actually do feel a lot isolated, but it is certainly you know when I started, HFI I had done a lot of good work before at Staples. There was sort of a...

Dr. Eric Schaffer: (Inaudible) wake up call.

Mr. Colin Hynes: It really was, it was like a landing strip. So it helped me out a lot, but at the same time you know once the rubber meets the road, and you're working with these folks, you still have to be able to quantify your gains and that's hopefully some of the work, the white paper works out to really sort of get a metric, this one metric to help out. But I think it is really important also to you have to speak the language of business. You have to speak the language of technology. You have to speak the language of you know creativity. You have to speak these languages and it gives you credibility. It really lets people, it enables people to open up and listen to you may be more than they would. And it will really help you out in your cause. So I wish you luck.

Dr. Eric Schaffer: But let me dissect that a little bit, because I think it might help. And the Staples example is as good as any because when HFI came in, we did a lot of work that just was really showing the declining value and conveying it wasn't just opinion, it wasn't just enough to stay customer focused.

Mr. Colin Hynes: Right.

Dr. Eric Schaffer: And something magical happened. We got at the very highest level executive champions. And having an executive champion is the thing that you need most. Once you got that then you get a guy like Colin Hynes or may be yourself who can be the hands on manager or the director of the usability effort. But that executive champion is probably the number one thing that I am hearing here that will make a difference.

Mr. Colin Hynes: Yeah.

Dr. Eric Schaffer: Because that person can provide the leadership, the resources...

Mr. Colin Hynes: You know, it does Eric. Because it opens a lot of doors. And it's that thing like I use my connection to get a door opened up. But then you have to (inaudible)

Dr. Eric Schaffer: You have to be good still.

Mr. Colin Hynes: And you have to perform. So you really have to get really tight about what is your message, what's your perspective, what you can add to this company, what are you doing that's going to help this person's business and help them succeed and the company succeed where your company hadn't in going and saying this isn't about me, this is about the company. And that's the way you know reaching out at customers. That's what it really is about at the end of the day, making your customers more successful, fulfilling their needs, and thus fulfilling the needs of the business. So it's what's it's all about.

Dr. Eric Schaffer: Let me ask a piece of advice form this person. And that's when you are working, do usability work. At all cost always do quality. Never do a piece of work that isn't effective either because it's not what the business needs or because it's not quality work. You'd much rather have 20 opportunities to help and do 2 really well because people are going to look and go that makes a difference. Okay that, yeah, we're going to want more of that. Whereas if you do a lot of it and it's not so good...

Mr. Colin Hynes: Yeah. That's a great point and you know it really is about going deep first and then using that to enable you to go wider later. It is

Dr. Eric Schaffer: It hurts really, but...

Mr. Colin Hynes: It is classy, you know you have to turn down a lot of opportunities. You might say, well, I really could have done great on that opportunity, but if you look at you know one person and 500 people, and say you have 10 projects you're just spreading yourself way too thin and it really is building up these case studies of saying look what happened in registration by a user-centered principles. Look what happened in search and start tallying that back to ROI saying well, gosh, it cost you this much money. We made this much, you know, so it really is like you said, great advice. Go deep and then you start to go wide.

Dr. Eric Schaffer: Colin, what's your opinion? Optimally, if you had a 500-person company, how many usability people should you have in that?

Mr. Colin Hynes: That's a great question. I don't know if there's quite a rule of thumb or it's you know this percentage of the company should be. You don't know what these people are doing. You don't know how many people are...

Dr. Eric Schaffer: Let's assume it's, they are all developers. That's a development company.

Mr. Colin Hynes: Okay, so that's a development community. I just take a wild guess. I would say something probably around 10 or so might be sufficient. 500 people for just a development organization if that's what it is you know there are people writing code and building web pages, that's you know it's pretty (inaudible) and you need a lot of people.

Dr. Eric Schaffer: So you think nearly 10 really (inaudible) heavy weights and you have to do a lot of education.

Mr. Colin Hynes: You do. I would actually recommend to this person one thing that I did which was really helpful was to put together a talking document. So put together a presentation that talks about what is usability and why is it important and you know optimally it works in the company and how we think it should work in this company and the customer gains we can make and really sort of structuring that out so that you have something to point to and speak to instead of just going to them and saying this is really great stuff and waving a flag. You really need those case studies to back it up. And use outside research too. Don't always that just because we haven't done you know a bunch of great projects here, just because we haven't done a lot of great work here, that or a lot of deep projects here, that they don't exist out there. So reference outside sources as well with respect to companies.

Dr. Eric Schaffer: There's a big pile of questions coming in. Okay. Very good. During testing are people frank about comparison shopping (comparing prices) and how do such site visitors factor into your ROI calculations?

Mr. Colin Hynes: The first part of the question, I think it really frankly depends on the person, how frank they are about comparison shopping. You know you'd like to think that everybody is going to be perfectly honest with you, but certainly that's not the case. So we try to bring that out in a number of different ways of asking them questions and if they start feeling uncomfortable in talking about other companies, we want to say, you know, this is what it's all about. It's us learning about how you behave and why you behave in certain ways and it helps us to be a better company and serve you better. So, I think a lot of it is just about how you ask the questions and how you try to extract that information from folks. As far as how do such site visitors factor into your ROI calculation, I'm not actually quite sure how to answer that question.

Dr. Eric Schaffer: That's no problem. Because the thing is your ROI calculation is based on the actual results. Those are people at home in their pajamas shopping. And they're going to do what they honestly do. They don't even really know that you're running a test on it. They're just reacting to Staples.com in a good way.

Mr. Colin Hynes: Yeah, there are some I mean certainly you it's not so much the ROI calculation per say, but and really I think the opportunity of people seeing, going and doing comparison charts against different you know products or you know we were in a study about people leaving a valid search results page and trying to figure out why were people leaving a valid search results page and got a lot of good data about there was you know quite a few people who were going in there and who were finding the product they needed, but they were in the process of research mode. So can we may be keep those customers and not have them go off and again go some place else? Certainly. Are we working towards that? Definitely.

Dr. Eric Schaffer: But your usability changes had an effect on the bottom line. They affected the actual drop off. And so it was a before and after case that's control for.

Mr. Colin Hynes: It is and I really think at the end of the day it is that you want price, I mean certainly to have great prices. But you want your price to be less of a factor if you had great user experience there were studies that we talked about earlier before the web cast about studies where a great user experience has on site has a halo effect over things like prices and download times and things like this. And it's really about the perception of the customer, it is not so much the actual. So I mean if you can get people to say this site is so great that you know I don't even need to comparison shop because I trust their prices and I trust that you know they're going to be right by me, that may be they'll be may be less inclined to comparison shop. And I think that's a kind of goal that we should all strive for is creating that affect.

Dr. Eric Schaffer: Yeah, I think there was a wonderful research recently where we were looking at sites and the download time and people thought there's a site that's 4 times longer to download and they thought it was faster. Because the experience was better.

Mr. Colin Hynes: It's amazing. It really is and it speaks a lot of volumes about the kind of impact, the kind of wide reaching impact that great user experience has for folks on the site, so.

Dr. Eric Schaffer: Yeah. That's like there was a fellow. He was in-charge of a large department building. And the elevators were really slow. And he got bids for fixing the elevator, making it faster. It was like half a million dollars. Oh it's too... So what he did then was he brought a bunch of mirrors and he put them up by the elevator. And now people while they were waiting for the elevators would fix their tie and they didn't mind that the elevators were slow.

Mr. Colin Hynes: Right yeah.

Dr. Eric Schaffer: Good ROI. How does Staples decide if they should re-design their entire site? I agree with your ROI discussion, but if your site is not an ecommerce site, how do you decide?

Mr. Colin Hynes: Well, I think it really depends on a number of factors. A lot of it is, is your business changing? Are you going in a different direction? So you know, say we're not an ecommerce site, but we want to become an auction site or you know an asp or something like that, if you are changing your business fundamentally, certainly you need to respond to that. I think only your customers can tell you if it's not being driven fully by the business needs then I think only your customers can tell you. And it really is a great question. It's a tough one to answer, frankly, but I think that it's going out there and finding out from your customers what are your needs. You know what kind of goals are you hoping to accomplish from this site. Are you able to accomplish those goals? What tasks are you going through the site? What kind of functionality do you use, are we providing, what you expect us to provide? And if the answer is no, it's why not and how can we do that. So it's really, it's a complex question and it requires a complex methodology to get out of it.

Dr. Eric Schaffer: I think one of the things that this question may be getting at is that the Staples site is navigationally a solid good site. It better be, we charge a lot (inaudible). So that navigational structure stands you in great stead. You're not going to need to change the site to fix the structures. So many sites have structures that you know they reflect the business and not what the customer's trying to see there. And so if you're building on a bad structure, there's only so much you can do. Because anything you do that improves the way of wording, color, and highlighting, just putting lipstick on a bull dog, it's not going to look great.

Mr. Colin Hynes: It's totally true. I actually call, I have a term for it called frankensite which is that you sort of take one arm from one part of the company, take another arm, and then you get the head, and you sort of screw the head on. Next thing you know you're looking at this thing that just there's no continuity to it at all. And you know again we spoke about earlier before the middle part of the web cast about reflecting the organization not reflecting the user's needs. And you see it all of the time when you go to these sites. And you can tell, you could probably write their own chart based on their website and just soft of do the boxes and so everything is just by looking at their website. It's a very dangerous thing. It shows your perspective. It really shows your where you're coming from as a company, you know. Where do you put your focus? Is it on you or is it on your customers?

Dr. Eric Schaffer: I have one main page over the internet for a telephonic company. And if you wanted to get business cards printed, where would you go? You'd have to go to the finance department. Because finance was responsible for visiting – how do I know that, right? So, difficult. As long as you are working on a solid navigational structure, the kind of methodology you've got where you're scanning through, finding places where you have drop off, and fixing that is very, very effective. But if you're working on a structure which is fundamentally flawed, then if you keep trying to fix it little piece by little piece, that's not going to work.

Mr. Colin Hynes: That's a house of cards.

Dr. Eric Schaffer: Right. So you are working very appropriately for your good services.

Mr. Colin Hynes: Yeah, absolutely.

Dr. Eric Schaffer: Can we get a copy of the power point presentation? I think that the, Brian .. the power point presentation, first of all, you can get the white paper and download that. Also this web cast will be available again from the humanfactors.com site. And is the power point presentation available separately? It will be available separately, okay. We'll make it available. Okay. I think some people are planning to pack it up and make it their working around paper with your help. How do you sell the usability concepts to a company that does not understand its worth, or why you really need it?

Mr. Colin Hynes: I think we touched on this a little bit earlier about really going you know going deep, not so much going wide and really diluting yourself. It is really important to make sure that you get some real solid wind under you belt to have that executive sponsor, to have a plan. To really almost have a you know a marketing plan so to speak with where you're sort of guiding the ship and saying this is how I want my department in usability to be perceived and this is my path I am going to take to get there. Certainly then executive sponsorship is key to open those doors. But again, when you are in there, you know, it's you and it's you articulating it. So you need to be able to speak the language of the business and really understand where the user is coming from, you know. Own that customer data from an intellectual perspective. Understand what is all this data. Understand why it means something to the business owners and understand how you can you know make it better in places where it might not be so great.

Dr. Eric Schaffer: One of the things that HFI does very routine and this is obviously the space we live in a lot because often companies won't even have an internal, if you have an internal person, you got a lot of (inaudible). But very often HFI is just brought in and there are a lot of different techniques we use. One of the best is an expert review. Because when we come in and we take their site and really systematically decompose the things that need to be better, and then you know there's certain amount of crying and people's eyes light up showing that. And I remember showing one site to, it was a brokerage site. And there's a section which describes all about getting retirement funds in place and it was in (inaudible). So if you are old enough to care about getting your retirement account, you were definitely too old to read that (inaudible).

Mr. Colin Hynes: Understanding your audience is just what it's about.

Dr. Eric Schaffer: Alright. You mentioned the web-to-store, store-to-web, catalog-to-web-to-store purchase cycle that consumers use when making purchase decisions. Is there a factor or multiplier to your ROI calculation method that can be applied to your formula? Jupiter suggests that customers that shop in multiple channels spend up to 30% more, so I wonder if your model is modest or if you've seen a model of the (inaudible) ROI based on off line sales increases as a result of improved usability initiatives?

Mr. Colin Hynes: We do. We actually have coded a sort of a metric that we use, but it was several, probably last time I saw it was 6 months ago or a year ago. So I feel uncomfortable sort of quoting it today, but certainly you have to look at sort of the reverberations that happen. So we know that people are using the website to go shop in a store. We know that people have a catalogue next to them when they're using a website. So they are totally interconnected. I mean you do have the 30% might be a valid number, I'm not sure how close that is to our metric, but it may be somewhat accurate. But certainly you have to take it into consideration, look into your business and see how that 30% or whatever number you use really applies to how closely your customers channel shop. Because there might be businesses where people don't do a lot of cross channel shopping. I can't think of any off the top of my head, but I'm sure it depends on the business, how integrated they are, and may be how land based they are. May be there's you know we have lots and lots of stores so may be you know a company like Staples is that much more integrated with people going into stores and sort of testing our products and things like that because of the penetration of stores in certain areas or you know where catalogues are you know shipped and things like that.

Dr. Eric Schaffer: There certainly are companies that are poorly integrated. For e.g. there's a book store, you go to their book store online, you buy a book, you bring (inaudible) and you buy a book, you bring it to their (inaudible) store and you can't return it. Because it's a different company, different system.

Mr. Colin Hynes: And I think what you just is very telling. They are a different company, but customers don't think that way. They think if you have the same name or even close to the same name, that you know you are the same company. And you have to understand that's the way people look at it. We have kiosks in our which HFI helped out and in the design. We had kiosks in our store where folks can go in and say they want to buy you know 10 you know purple sharpies or something and we only have 8, they can go to the kiosk and they can which has Staples.com on it or a very close something like Staples.com, and they can order those you know 10 sharpies and they can have it delivered it to them overnight the next day. So it's that sort of integration that's great and with the kiosk we do something that no one else does which is that we enable people to in that case so you have you know a rim of paper and you also want to get those sharpies, that you can have your rim of paper, you can order those sharpies online have the sharpies are delivered to you and then actually pay at the kiosk, which is just really amazing that we enable people to sort of print out a receipt from the kiosk, bring that up and pay for the sharpies and pay for the paper all at the register. So I mean, it's really amazing. It just speaks volumes about the integration focus that we have. And it really helps the customer out. And it helps us as a business as well, so.

Dr. Eric Schaffer: And so usability is really the key to getting that integration to happen and you can have a really positive synergy like Staples or really bad things can happen. Okay. Colin, you mentioned that your team is anywhere from 5 to 10 depending on project need. My question is how is your team made up? How many developers, information architects, graphic designers, and so on?

Mr. Colin Hynes: My team is comprised of information architects and usability experts, usability specialists. So there is a lot of cross over and there are talents in lot of us put together, process flows, and site maps and things like that and usability testing. But certainly there are core skills there. And there is an information architect on my staff whose core skill is very sort of heavy duty process flows and site maps and things like that and figuring out if it is really difficult sort of functional conundrums that we are faced with every single day. But he also does usability testing. So it keeps him sharp, it keeps him very close to all the customers. So, I mean it's really made up of I would say three usability specialists, an information architect. We actually have something, we have someone our site who's a recruiter, which recruits that the users who are testing. And if you walk and say where did you find these users, and you know we have a recruiter that knows our customer so, so well. And you know how it is where if you have a usability test, you can have the best conceived usability test ever put together but you don't bring in the right customers, it means nothing. And really you must not do it. So we have a recruiter on staff that is very focused on you know working with this list to make sure we get the right customers in or prospects in depending on the tests and then we run the usability studies accordingly and you know then cycling out of that we have a lot of other folks who support it depending on the need. So we'll hire you know consultants to come in and fill the need. And sometimes it's you know it's sort of a general usability test and sometimes it's very specific. We need a very specific skill set to be able to support the initiatives. So we are pretty flexible there also. I mean it does not comprise developers actually code writers. Well, we all have some sense of technical background and can write some code. We don't have sort of the developing group per say in my group. All the graphic designers are also in another group although you know we do work very, very closely together to solve these different pieces.

Dr. Eric Schaffer: It really does stand out for me how I mean 5, 10 people, it's a modest investment.

Mr. Colin Hynes: It really is.

Dr. Eric Schaffer: And yet it is hugely leveraged.

Mr. Colin Hynes: It really is. Again the ROI is really enormous on this, so.

Dr. Eric Schaffer: Okay. This is regarding the drop off view. How would you know that the person leaving the page at an unacceptable time, that's good (inaudible) okay unacceptable time is leaving because of some flaw in the website information architecture versus someone who has multiple windows open and who is just comparing the products of different companies? Wouldn't this result in errors in the data collection process?

Mr. Colin Hynes: Yeah, I think the answer is yes, it may result in errors in the data collection process. And the answer to the other question of how we get at that, there are certain ways to get at it depending on what you are talking about when you say data collection process. If you are looking truly at site logs, you know there might be some variables in there. But you hope that if you have enough traffic that those would work themselves out. (inaudible) you have enough statistical significance to have the (inaudible) outlined and then you'll get at the core issues that are going on there. This kind of – may be other sites are more have this more sort of popping up windows and things like that we're comparing against other sites or people do that on other sites. But if you're just looking at directionally what's happening on the site what is this data indicating? I would caution people against making changes based purely on log files. Really and that was the message in the tape part of the web cast was that you really have to back this up with, bring folks in and understanding (inaudible) you got to work this up statistically also. But you need to really understand qualitatively what's happening and get these people to say well you actually know what, I remember I was in the store last week and there was a price on this product that was this high so I am going to pop up another window and go see you know with this other store. So it really that's why you need to get really close to your customers. Don't think just because you look in a log file, you can answer all the questions and solve all the problems of your website. It really is an integrated approach of getting directional information and then backing that up with scientific research.

Dr. Eric Schaffer: We'll see if I can capsule this methodology and perhaps answer this question to. So initially what you are doing is you are scanning, looking for places where drop off looks bad. You find the place where drop off looks bad. You're not sure what's going on, because it could be because people are satisfied and they're leaving. It could be because of other factors. You need to dig into that area and research it using all the technology you can.

Mr. Colin Hynes: Absolutely.

Dr. Eric Schaffer: Then once you find that, it's really a usability problem. You can solve that and then validate it with the method you have where you say that this is what I had before and this is what I have after. This is what I had before and this is what I have after. I don't care about this person who is comparison shopping, because I had another person comparison shopping before. I had a person comparison shopping, it's a wash.

Mr. Colin Hynes: Right. Some things you can control some things you can't I mean. So you really try to concentrate on what the core goal of that user is and the core experience and tasks that they perform. So that's the kind of thing that we're doing.

Dr. Eric Schaffer: True. How do you deal with a common remark that putting more of the form on one page is too much and complicated? Probably too much and too complicated.

Mr. Colin Hynes: Yeah, I mean I think that if they are referring specifically to the registration portion of the web cast, is that certainly that you know a lot that comes out of information design you can have a lot on a page and if it's designed correctly, then it doesn't feel like a large factor. The whole perception thing that we talked about which is that it is not how fast the page downloads or and it really is about the perception of the user. So you know I've seen a lot of you know 3 or 4 form pages where you just overloaded with you know all sort of instructional data. You might not even need it at that point where it feels like a busy page. So I would really more – (inaudible) more cautioned against what I speak to what is complicated and what isn't complicated and why are they saying that is the truth because of the amount of fields or is it because of the way the fields are laid out? So it's a great issue of usability.

Dr. Eric Schaffer: I had one case. I was working for this insurance company and for a corporate health policy they had a 250 page long form. And the users said it was too long. That's really surprising.

Mr. Colin Hynes: Really bad, yes, shame on them (inaudible)

Dr. Eric Schaffer: Yeah, well, no what they did they wanted to cut it so what they did was they took the form and they crunched all the fields together and went to no space between anything and dots across it went down to 150. And the users still didn't like it.

Mr. Colin Hynes: Perception, it really is.

Dr. Eric Schaffer: And the fact that there is some research saying that generally speaking if you compare going through very lean form, you'd rather have it all together.

Mr. Colin Hynes: Yeah. I've heard the same (inaudible).

Dr. Eric Schaffer: Yeah, okay. But good design matters. How long of hiatus between visits constitutes a drop off, half an hour, one day etc. Does it vary depending on the task you are evaluating?

Mr. Colin Hynes: We don't actually make the distinction between if they leave our site fully, then that's a drop off. So if they you know are making dinner or something like that and they get half way through their purchase process and then go make dinner and come back, then they continue in that flow, you know there's no timing out or anything like that, then they have not dropped off. It's when they leave a Staples.com hosted page for a non Staples.com hosted page that they are considered a recorded drop off. So right, it does, you know it certainly does vary depending on the task you are evaluating and some that are longer certainly it may require the people to come back multiple times. Hopefully you offer them the same mechanism so if you can save the data for the next time.

Dr. Eric Schaffer: But if you are buying something very large purchase, a computer is a classic example. Then the person will come back many times. After the redesign, what was the length of the survey period to determine the drop off rate improvement?

Mr. Colin Hynes: That's a good question. We actually waited 2 weeks after just a you know there's a lot of folks that come in that they might be used to the site in a certain way and you really want to normalize for these folks who come in that are more in the learning curve and you have to be very careful about how you migrate those customers as well. Because you don't want people who get very sort of invested in the site from a sort of an understanding of how things flow and where things live and things like that and also pull the rug out underneath themselves. And we spent a lot of time and we always do sort of a middle-to-major redesign of how we migrate these customers to understand what we are doing. So we waited 2 weeks. We thought that was about right, frankly. It actually in both cases it did start to drop off right away. But to really get an accurate representation we want to wait for that sort of grace period so to speak to see really what is going to be going forward, so.

Dr. Eric Schaffer: Okay. From start to finish how long did it take you to improve your search and registration (inaudible) changes? Were these times in your initial time frame projections? What kind of feed back have you gotten from customers? Sort of a different question.

Mr. Colin Hynes: To answer the first question, at the top of my head, I believe search took from sort of a seeing it with an idea all the way to it being live in a site took anywhere between 2 to 3 months or so. And lot of that was frankly (inaudible)

Dr. Eric Schaffer: (Inaudible)

Mr. Colin Hynes: It does, that's everything from like us saying you know gosh it might be a drop off issue here. It looks like the numbers are going up. We have to address this too, it being click and switch and it being live in the site. A lot of that was researching. Frankly, if you look at what we did in the search results page, one of the things that was cosmetic was moving things around the page was information design. You would think we were pretty solid for the most part of the search results page it was really taking what we were hearing from the users just designing it a little bit differently. And I think that's another message we haven't talked about that this stuff doesn't have to be you know year long projects with millions of dollars. It can be just doing a research, getting an understanding and then having that translated to actual spring design.

Dr. Eric Schaffer: (Inaudible) one or two little words.

Mr. Colin Hynes: It can be cosmetic, I mean it really doesn't have to be a huge thing that takes you know they are moving mountains to make it happen. So, and as far as registration is concerned I think that was also somewhere in the neighborhood of 3 to 4 months or so. That was more complex from a research perspective. There was a lot going on there to try to figure out getting the yellow pages correct. So that was a little bit more timed, but 73% improvement is pretty worth it (inaudible).

Dr. Eric Schaffer: Now we can't let you get away without answering the full question. Had customers called you up saying thank you for the new search.

Mr. Colin Hynes: Yeah, it's great. I mean we're very, yes, they have and certainly for registration as well. And I think one thing that they were pretty strong on this thing integrated with customer service folks as well and understanding were the phone calls coming in? Were the emails coming in? And what we saw is we didn't just see you know my phone ringing all day and people saying I love your registration process. It was more when we looked at drop off going down, we looked at you know sales going up. We looked at phone calls and emails we were getting relevant to these changes we made going down, those sorts of things. So we saw a number of different indicators. (Inaudible) which we used after some of the purchases to understand how the experience was. These indicators were all positive. So that really told us a lot about how we did.

Dr. Eric Schaffer: So you were more excited about the credit cards coming through and the purchases going up than the phone calls to say thank you.

Mr. Colin Hynes: It's a nice double barrel benefit I guess, yeah.

Dr. Eric Schaffer: Very good. What books and training besides those which HFI offer would you recommend?

Mr. Colin Hynes: Oh gosh. I mean there's just so much out there. I would really look for the most recent research. I think that you know JK Wilson's written several really good books. I look at self promotion books a lot of these folks are dealing with issues of trying to sell usability internally. So I think that even looking outside may be the human factors space in general, it would be may be a marketing book to really figure out you know how you promote yourself online, self promotion online I believe is the name of the book. Those sorts of things would really, I think you need to thinking yourself as sort of a small business person within your company how will you build your small business? How will you market yourself and how will you run your business and really (inaudible) about your approach to building up your group.

Dr. Eric Schaffer: Okay, now I go on. So, I was having an ice cream across from Staples down the street. And I was watching people come out of the store.

Mr. Colin Hynes: Two bags in their hands I hope.

Dr. Eric Schaffer: Yeah.

Mr. Colin Hynes: Okay.

Dr. Eric Schaffer: And very few without anything. And yet in the industry, not just Staples (inaudible) but in the industry overall, having a 5 or 6% conversion rate is considered part of the course. How can that be and is it ever going to be the case that we get something that looks like a store? I mean people don't come to a site that sells off the supplies for the aesthetic experience or to be able to tell people hey I went to Staples, and it is a good site. It's not something. People go there because they want to shop.

Mr. Colin Hynes: Yeah, I mean they are extremely task focused. They actually refer to a lot of people do like to go there and do like to browse. I mean that's great. I mean like we spend a lot of time in our store designs to make it a very pleasant experience so we encourage people to go out and spend as much time as they like there. And people actually do. So that's kind of an interesting finding. But...

Dr. Eric Schaffer: This is at the site or at the (inaudible)

Mr. Colin Hynes: At the (inaudible) We spend a lot of time obviously on our web site. But we also know that the majority of people that gets talking about those core folks are there to get their job done. They don't you know they don't you know wake up at night saying I can't wait tomorrow to order some office supplies. It is something that they need to do. It's a part of the essential function of their business. And we are here to support their business. So we know that they are in there because they have specific needs and they want to get those needs done as quickly and as efficiently as possible to the highest level of satisfaction at the best price and best customer service. And if we can provide that, then we're doing our job. We are providing for them and our business profits as a whole. So, as far as trying to see the correlation between stores and online, you know a lot of it has to do with the media, is that if you are driving out to a store, you might not like jump back into your car and drive who knows how far away to get to another store that might sell the same thing. But it's a lot easier on the web. It's just a click away is what they say and it is true for better or for worse, I guess on a website. But it sort of switch over costs a lot lower. So certainly, you know you have to be cognizant of that. I wouldn't say never say never, but I think it would be very difficult to get it to a point where you know the conversion rates so to speak of, of a website would be at par with the store, of people researching at a store, at online sites, it is a lot easier to research. To the point of the previous question, the pop up window and it is so easy to do that, that you (inaudible) to that. So I don't know if you ever get there, but we hope they are equally high for Staples.

Dr. Eric Schaffer: And you certainly bring all the usability technology to be able to make that a site that's a great example for the whole industry.

Mr. Colin Hynes: Thanks.

Dr. Eric Schaffer: So the question answer session, the white paper, they are available in the future on the humanfactors.com site. And we'd like to thank you all for being part of this web cast. And thank you, Colin. That was great. And just wait, not just yet. I have one thing I want to do before we go.

Mr. Colin Hynes: Is it a T-shirt?

Dr. Eric Schaffer: You got it. We have to get (inaudible)

Mr. Colin Hynes: (Inaudible) closet for (inaudible)

Dr. Eric Schaffer: Sorry. Well, this one's special, because it has the Hines equation.

Mr. Colin Hynes: Oh gosh.

Dr. Eric Schaffer: With your calculation for ROIs. So here's.

Mr. Colin Hynes: One for my team, I should say.

Dr. Eric Schaffer: Yeah. Wait, I have two more for you.

Mr. Colin Hynes: It's a long equation, right.

Dr. Eric Schaffer: Yeah, it's kind of a long equation. So thank you for, here you go, the rest of the T-shirts.

Mr. Colin Hynes: Great.

Dr. Eric Schaffer: Alright.

Mr. Colin Hynes: Thank you.

Dr. Eric Schaffer: Well, thanks for being part of it.

Mr. Colin Hynes: Thanks. I enjoyed it.

Dr. Eric Schaffer: Right. Now you can show them.

Top