July, 2005 – The ROI of Usability and Making Usability Routine

Jay More: Hello everyone. I'm Jay More, President of the HFI. Welcome to another in the series of webcasts by the usability of broadcast network. Today's webcast topic is "The ROI of Usability and Making Usability Routine." I wanted you all to know that there is a 27-page white paper that goes along with this webcast which I am sure most of you have already downloaded. And there is a chart on usability maturity report card for you to download as well. Susan will be referring to this chart in the webcast later on. And at the right-hand corner of your screen is a button for you to press and you can submit a question to this webcast at any time. Now I'd like to introduce Dr.Susan Weinschenk, Chief of Technical Staff of Human Factors International. How are you today, Susan?

Dr. Susan Weinschenck: Good, very good.

Jay More: Good. I know you have been traveling for the last year and a half through North America doing a tour presentation on the ROI of Usability.

Dr. Susan Weinschenck: Yes

Jay More: And I wanted to ask you, why is this such an important topic? Why do so many people come to hear your talk?

Dr. Susan Weinschenck: I think it's because as usability becomes more mainstream, usability specialists have a need to communicate about it and talk about it throughout the organization. And especially up to a management level. And as they have the need to communicate they are finding that in order to talk to this larger audience, they need to talk about the business risk of usability and return on investment for usability. So I think they get excited and they are looking for ideas and information how to do that more effectively.

Jay More: And I know you've had a lot of positive responses for your tour.

Dr. Susan Weinschenck: Yes

Jay More: It makes sense that you want to get this message to a larger audience through this webcast.

Dr. Susan Weinschenck: Yes, this has been wonderful. We have done, yesterday and today, we have covered Asia and Europe and now this broadcast for the Americas.

Jay More: It's been really great. Now without any further delay, we'll go right to the main webcast.

Dr. Susan Weinschenck: Our topic today is "The ROI of Usability and How to Make Usability Routine". And what I want to start with is three points:

The first one is, we really can calculate return on investment for specific usability projects. Two, you really can make usability routine in your organization and three; you can calculate how usability being routine also provides return on investment. So I'm going to assume in our talk today, that you already know the basic benefits of usability engineering. Usability engineering saves a ton of money for your developers, produces technology that is easier to use, reduces the amount of customer complaints for your product, there is a whole scope of benefits that usability provides and I'm going to assume that our audience here really does understand that. So we are going to go beyond that and we are going to talk on the return on investment for doing usability work. How can you calculate that? Can you really provide specific metrics? And specific numbers to make the business case? There are a number of different ways to calculate ROI, there are many examples of this which you may have read about and what I want to do is get you thinking, "What are the metrics that make the most sense to my project and what are the metrics that make the most sense for our organization?" So some of the typical types of metrics that we use when we are talking about calculating ROI are things such as conversion rate – what is the percentage of people who perhaps come to your website that turn into buyers? Can we improve the conversion rate? That is a typical calculation. Or what about the average dollar amount spent on when we are talking about the website, can we have people increase the amount they are spending? You know, it's not just about websites and it's not just about dollar amounts, what about other measures like saving user time? So productivity is one of the typical measures that we have had in usability engineering for a while. Can we save the user's time as they are using the application? Can we save the developer's time as they are working on the project and therefore have a cross savings on how long it takes to develop the software? So many different measures – what about a decrease in drop-off rate? What about saving of training time? How about increasing the usage? One of the things that get a lot of people attention is the idea of self-service. Can we get customers or users to come on to the internet or maybe an intranet site and do some of the works themselves rather than having to call for customer service? So all these different measures are appropriate metrics for usability and I really want to stress that there isn't any one measure. Sometimes people say, "We want to calculate ROI, what is the formula?" Well, there is no one formula. The question is what makes sense for our project and what makes sense for our organization. Now once you decide that, you may have more than one metric to use for your project. Once you decide what the metric is, then you want to go about collecting the information to calculate it and this can take some time to do. Sometimes you may have hard data, you may know how many people come to the website, and you may have previous logs that tell you where people are dropping off. And if you have that hard data, you definitely want to use it. But don't let that stop you, sometimes; we have people say, "We don't know." Well, then you have to start making assumptions. How many people do you expect to come use this intranet site, let's say, and how much would you like to increase the usage? And these are all valid; sometimes people think if we don't have hard, past data, then we can't do an ROI calculation. That's not true. You want your assumptions to be as realistic as possible. And I also told people – make your calculations and assumptions very conservative. Go ahead and be very conservative in the numbers and when you get a really nice return on investment you can say we were using conservative numbers. So that's a very valid thing to do. So get all your assumptions and you'll go through the calculations. I suggest you try this out in one or two of your projects, if you're new at it. I know the first time I ever calculated return on investment was quite a long time ago and I had never done it before. I have to tell you, that I am a cognitive psychologist, like many of you in usability, I've been here for a while and you obtain the usability of the idea of making technology easy to use. The whole idea of return on investment in the business case, that was, perhaps you know, not your motivation. That was true for me as well. The first calculation I ever did, we ran through the numbers and we were calculating saving users' time, saving training time and saving calls to the helpdesk. And we came out, we had it all lined up, and we came out with the return on investment. We were going to spend $70,000 on the usability work and over a period of five years, we were going to save $11 million, which was a huge return on investment. I can tell you the first time I ran through these calculations; I assumed I had pushed the numbers on the calculator wrong. And I went through; we must have gone through, two or three times, the same numbers just to make sure because we were unable to believe that we would get $11 million. I was quite nervous when I went to present this to my clients thinking that they were never going to believe the $11 million number either. We actually had a funny moment when the client, after I had presented this, there was a moment or two of silence and one vice-president stopped us and said "3". I said "3? What does the number 3 mean?" Well, she wanted us to do 3 usability tests, not one. She figured if we could save them $11 million on one, we could save $33 million on 3. Then I had to stop and walk through and tell them that, "No, no, it does not work that way." But obviously, she was really sold and that opened my eyes to this idea of making this business case in calculating ROI. So now, my motivation is still to produce technology that is easy to use but I really see the benefit of talking about it in terms of making it a business case and talking about it in terms of ROI.

I would like to share with you, some ROI calculations that we have done recently for some clients to give you a feeling of the way this works when you come out with the actual numbers. Sometimes, I put a little disclaimer in here, like in the ads, when they say "Your results may vary." Some of the numbers that I am going to show you are pretty high and dramatic in ROI and your numbers may not be that high. Some of the ones that I'm going to share with you are perhaps a little more realistic, more like what you will see. Again, none of this is important in terms of my examples that I am showing you-what's really important is what does it mean to your organization and what would be a significant number to the people you are talking to.

So let's first talk about one of the projects that we worked on here at HFI, which was for staples.com. For Staples, we did a number of different works, we in particular, had a particular place in the website, where we knew they had a high drop-off, where we knew people were leaving the site and we evaluated why that was. We also did some work at the website because we discovered during our user analysis that many people were repeating what they were ordering. They wanted to order what had last time. The flow of the current website when we came to it did not allow that so we made these kinds of design changes. And here's what we came up with, we were able to reduce the drop-off from the site by 73% - a huge reduction. And Staples calculated that was saving 40,000 visitors a month and that that turned into $6 million a month, which is really quite a large number and quite significant. So there's an example where with the usability work that we did, we were able to calculate how much money we saved was bringing in, in the additional revenue.

Here's another example, a little bit different. This has to do with reducing training time. So again, bringing up that point where each organization, they have different interests. Different things that they want to do and it might not be increasing usage or reducing drop-off, it may be something like reducing training time. These numbers come from Syncrude, a crude oil manufacturer in Canada, and there, one of things they were interested in was how much could they reduce training to learn their intranet. They had an intranet site and they brought people in and had them do training and it took a day to train them how to effectively handle the intranet. We re-designed the intranet and what we were able to do was bring that training down to about 15 minutes so that was a huge saving for them. They have calculated that the savings there was about $365,000 a year. So not as large as that number of $6 million from Staples but for them that was significant and well worth the work. That's what you want to compare, what is the cost of doing the usability work, you subtract that from the total amount you saved and that gave us here our $365,000.

Let's look at another example. This was for a utility company and what they were interested in was the calls to the helpdesk. The industry average is that it costs about $10 for every helpdesk call that come in and they wanted to reduce that. They were also talking here about an intranet. We had an interesting interaction on this one. I knew that they were tracking this metric, in the past they had told us that they typically gather 300 calls a day to the helpdesk when they introduced some new application to the intranet. So they had that number and we wanted to track against that number. When we released this new application for their intranet, we were going to track how many calls they were going to get. We had obviously applied user-centric design methodology and we were hoping that we get fewer calls to the helpdesk because it would be easier to use. I called that client and said, "OK, it's been a month. Do you have the weblogs on calls to the helpdesk?" and she looked it up and she said that she would have to put me on hold. She said, "I have to put you on hold, there's a problem with the data." She puts me on hold and then she comes back and says, "Oh, I thought there was a problem with the data because we showed no calls to the helpdesk so I figured something was wrong with the report." She went and checked it and indeed, there was nothing wrong with the report. They had actually received no calls to the helpdesk for the first 30 days so they had gone from 300 calls a day down to 0 calls a day. The calculation for that was that it saved approximately $60,000 in the first month by having the re-designed application.

The last example I'd like to share with you is a little bit unusual. Again, I want you to be thinking what are the metrics that will make the most sense for us? We had one client – an intranet medical site and we asked them before we started the new design, what are the metrics that are important to you? And they gave us something which we had never heard before which was, "All we care about," they said, "is the number of page views." And I said, "The number of page views?" and they said, "Yeah, we get paid by advertising revenue. We get paid by the number of different pages that people go to when they come to the website. And therefore, all we care about is to have a high number of page views." So I said to them, "So, do you want to put like only 1 sentence on a page then they will have to go through a whole lot of pages just to read a paragraph on migraine headaches?" and the client said to me, "Oh, can we do that?" I said, "No, we can't." They were hoping that maybe we could do that so we did have that re-designed, we did some user-friendly pages but they increased their page views by 55% and that was very significant for them. They were thrilled at that and also something that happened was that by increasing their page views that much, they came under the radar of one of their competitors and within about a year after the work that we did for them, they were actually acquired by their competitor which was something they were very excited about. So that was a little different type of measurement but again, I'm pointing out that all that really matters is that the metric really makes sense for you.

So when you're thinking about this, here are some ideas about how to think through the process of the calculation. So if you start with the idea that you have a particular product in mind, and again, this might be a website, an application, an intranet site, it might be a device like a remote control or a medical device. Whatever the product is, identify the product and then identify what you think is or have an evidence of having usability problems with the product. So let's take an example to understand, let's say we have an internet site that we're going to evaluate. We know that it has some problems and specifically in this example, it has some problems with is that there is an application form that the customer has to fill out which is really difficult to use. So that would be example of a problem. Now in real life, when you're filling out this information, you may have more than one problem for each product that you have. So now that we have identified the problem, for each problem we're going to want to specify what is the consequence of having that problem? You know this is something, which in usability, we sometimes don't think about; we tend to be very, very good at finding the problem. But when we have to make the business case, we have to go this next step, what is the consequence of not fixing that problem? Now to us as usability people, that is a horrifying concept that we would let there be a problem and not fix it. But from a business case, well, we have to ask, so what? What's the problem and what's consequence of leaving it at that? What's the next step? What's the consequence of not fixing that customer application form? Well, in this case, it would be that there would a high drop-off rate; it will have errors perhaps when applications are coming in. So that's the consequence, now let's go the next step. What's the cost of having that consequence? What's the cost of not fixing that problem? What is the opportunity that is lost? And in our example here, we're going to talk about we're going to lose 25 customers a day, each customer is worth, you know, perhaps $600 per year in revenue etc. You can go out and do the calculation. Now you can discover for that product, with those problems, if we don't them, we are going to lose this amount of money or we're going to lose the opportunity to have this amount of additional revenue. Or remember, it might not be money. It could be training time, etc. So whatever metric it is that you decided to use. Two more steps - the next one is that we find out how much money we're going to save or how much extra revenue we're going to bring in or how much time we're going to save in training. But now we decide that if we're going to fix that, what does it cost to fix? Really to be fair, we can't just determine how much money we're going to save, we really have to figure out how much it costs us to save that amount of money or how much it costs us to bring in that additional revenue. So at this point, you decide what is the usability fix, as we call it? What is the intervention? Are you going to do usability tests or are you going to re-design? So what's the cost of making the fix for the usability problem? Then the last step, you would then take the revenue that you have and subtract the cost of the fix and then you would actually have your ROI. So that's the way you would go about making a calculation with that and as I said, you may have multiple calculations for each product. So I hope that I've shown you here that really it is possible to calculate the ROI for a specific project and some ideas of how you would do that.

Now I'd like to go on to a little bit different topic because I'm interested not only in calculating ROI but I'm interested, and I hope you are too, in how do you calculate the ROI for making usability routine throughout your organization? You know, it's one thing to work on it project by project and sometimes I call that, the "emergency room of usability". A project comes in and the usability team is like the emergency room physicians, the patient is the product with horrible problems. We're going to patch it up as quickly as we can and head it out of the door because then the next usability problem is coming in. That's a lot of the times the way usability teams have tended to do their usability work and you know, when you work in that situation, it can be very frustrating. You don't really have time to really fix the patient, you only have time to put some band-aids on, and then, what's really discouraging is that the same patient comes in the door six months later because you didn't have really a chance to fix it and there is a gamut of more problems. The idea here is that we want to get away from the emergency room model that we are on and you want to work towards what I call a wellness model. What I hope to show you right now is that actually, having an emergency model and making quick fixes and not having a usability routine is expensive and that if you make use of the usability routine throughout the organization, you use that wellness model, you actually in the long run, will save time and money. So that's the case I'm about to read next, then we can talk about the return on investment and how those numbers work.

So let's talk about making usability routine through the organization. One of the things I'd like to mention is that our CEO, Dr. Eric Schaffer has written a wonderful book on this, "The Institutionalization of Usability", and a lot of this information that I'm talking about comes from the book. If you don't have that book, then obviously, I'm going to suggest that you get it. It really does contain very, very detailed explanation about how you go about in usability routine. Right now, in this presentation, I'm going to cover some of the highlights of that. When we think of the hallmarks of usability of routine, how would you know that an organization has usability routine? There are 3 main things that I would say are the true hallmarks. One is, that usability considerations are part of any technology initiative. So if we're talking about an internet site, we're talking about developing a new web application, whatever is under discussion, also under discussion is how we are going to make usability an integration of the user experience and it's just such a normal part of the process that it's not someone later on has to say, "Oh, maybe we better do some usability work" but discussion of usability is in the very beginning so that would be one hallmark, that's how we would know usability had become routine in that organization. The second one is that user-centered design is an established part of the design methodology. So if as the product is being developed by the IT organization, for instance, and they have their software methodology either integrated into that or right alongside of it is a user-centered design methodology. Again, it's not a question, should we do some usability work? But it's a normal part of development. Everyone understands that these are the user-centered design tasks that must be done as part of the development. Thirdly, that there would be trained, and I'm going to say, certified usability staff within the organization. There is a usability group. There is trained staff already there and it's not a matter of "Let's ask the IT staff to go take a class" and then maybe you have usability on the project but it's actually built in the organization – a group of people who are specialists in usability and usability engineering. So those are the three things I look for and if someone says, "Do you think our organization is routine?" or "How fare are we from getting there?" – Those are the three main areas I always look for.

Now at the website for HFI, we have what we call the "Institutionalization Report Card" and maybe you downloaded this before our webcast and had a chance to fill it out. If you did, you may want to pull that out now as we're going to be talking about some of the items on there. If you did not have the chance to do that, you may want to do that when the webcast is over because you can rate yourself on how you score in terms of how you can make the idea of usability routine. So let's talk about some of the things that are on that report card. Let's talk about why some of these things are important so there are some interesting statistics that we have and also interesting is the fact that some of these statistics come from, have been around for quite a while. 1992, from a book by Pressman is the idea that every $1 that's invested in user-centered design returns somewhere between $2 and $100 back. Now that's a huge range – 2 to 100 but I'll take the $2 so even at that very low number that would mean that every $1 I spent in user-centered design would bring back $2. So it doubles my return and that's a great return. That's just an example at the lowest number. Here's some other interesting numbers – 80% of the software cost comes during, let's call it the "maintenance phase" and this is also from Pressman. That's an interesting thing and really what it means is that we are saying that the project is done and we cut it off but actually most of the money we spent on that project happens after that. No why would that be? Well those of us who've been in business for a while know the project comes back in because it has problems. Additionally, 60% of that phase where it comes back to be re-designed is due to re-work because user requirements were not clear in the beginning. As we know, user-centered design helps us define user requirements. It's one of the main advantages of user-centered design. So it would have an exact impact on the largest cost of a software development project. 63% of software projects are beyond their budget and beyond their time and the top four reasons for that are, if you look at that, and this is a study by Lederer and Prassad, is interesting because they all have to do with what we have discussed in user-centered design. Frequent requests for changes by users, tasks that were overlooked in the initial design, the users don't understand their own requirements and communication not being clear enough between the analysts and the users. These are all things that are taken care of when you're following a user-centered design methodology.

So what the idea is, about following user-centered design methodology, without that usability being routine, there is a tendency to rush through the steps that we know are so important – task analysis, design of a user interface structure – rushing through that and at the end we have larger costs for training, helpdesk and having to do re-design. So what we're looking for in doing user-centered design methodology and making usability routine is let's set forth and do the tasks that we know are important and we will have less money that has to be spent later on.  So definitely, we are front-loading – there are more tasks to do and we need more time up-front but the return is great on the back-end. When I talk about user-centered design methodology, we always have here, that we believe in and that we use, whether you use our methodology or whether you use another one, or one that you have developed on your own, it's a basic user-centered design methodology that involves knowing what the organization wants, knowing what the user wants, doing the line work, applying best practices and standards and most importantly for us, I think for most of you who are usability professionals, is to understand the basic user-centered design methodology. The good news is that our field is quite mature and we know how to do this but we don't always apply it. So what I'm going to say about making usability routine is you must have a user-centered design methodology in place and in fact one of the things that we are finding that is very important is that we have what I call, and IP repository, an Intellectual Property repository for all of your usability work. We have a product usability central that we recommend our clients use, again you can use our product or something you've built yourself. You need to have one place, probably an internet site in your organization where all your usability work is, where all the intellectual property is so this would be your user-design methodology, this would be your standards and guidelines. Let's talk about what should be in there because these are very important parts of making usability routine. In addition to your methodology, you want to make sure it says what to do; you really want to make sure you have the resources in there, do you have forums for data gathering, and do you have central reports for usability testing. Here's where the ROI comes in. Your usability staff is going to do data gathering or is going to design a test protocol. They can't do that from scratch every time or search around for how much money you used last time, which wastes time. To have that all in the usability central repository and your usability staff can pull up exactly the form that they need and re-use it again and again. Now you start to get savings because what would take an hour, it only takes 5 minutes. If you multiply that by the number of projects and the number of people doing the work, we start to get savings over time. So we have to build a central repository structure and it does take time and money initially and you have to get all that set. But once you do, then you have a lot of savings. So you want the methodology, you want the resources for people to do their work; you want standards and guidelines which you've known for a long time. You need them accessible for people there. Let's talk about going beyond standards and guidelines. One of the things that people have found extremely powerful is the idea of templates so not just standards and guidelines but specific templates that people can use. Here's an example of ROI for that. I had a client who said that they had tracked a typical 5 weeks to develop the detailed screens for an intranet application. But what we did is, we set up templates for them so the next time they had to design another intranet application, rather than doing re-design again, they just brought up the template and applied it and they said that their development time had come down from 5 weeks to 5 days. So that's the type of ROI that we are looking for, so filling templates in place and not just standards and guidelines. There's another way of getting a very powerful return on investment of making usability routine in your organization. You could even be prototyping tools based on it. These are all the types of things we use in usability central and increase the ROI for our clients. In addition, rather than just have engineering standards and templates, why not customize them so that they really fit your organization. As your organization's logo, banner, colour scheme etc. can be added. These are the kinds of things you want to build in that make usability really happen easily in an organization.

Some of the statements you hear when usability isn't routine, let's talk about that. Here's another sign of how I know that usability is not routine in an organization. You hear things like, "Let's design some quick screens now, we'll fix them later." I'm sure some of you have heard that one before. "If we do some user testing at the end, that's good enough." These quotes are probably ones that you've heard before. "Just let the users design the new interface, they know what they want." Well, advocating for the users is not the same as letting them design, or "We don't have time to do any usability work. The programmers are sitting there, they have to start programming." "The programmers like designing the interface. Don't take away the fun part of their job." "Our software methodology addresses usability." Well, I would want to look really carefully as it probably doesn't. "If we do usability work, we won't meet our time and budget." These are signs that usability is not built in as a routine part. So those are all some of the statements, I know you've probably seen some of those.

Then let's talk about one other thing I mentioned about, having trained staff. So they're important so that you think about training and having usability specialists on staff. HFI has their certification program, we have over 500 usability specialists now worldwide. People have taken our exam in organizations all over the world. It's very important that you have staff with a certain level of expertise that is proven. So I really recommend that you specify specific people and you get them the training and hopefully they will get the certification. So one of the things that you can do besides institutionalization report card is to look at yourself and what we call the usability maturity model and what we have found is that becoming routine doesn't happen overnight. There are certain things that happen in certain stages. You might want to look at that and evaluate. In fact, I'd like you to start thinking about what's the one next thing that we can do, that would move us towards the next phase of usability maturity and the next phase of making usability routine. So we have this chart in evaluating where you fall, and help you figure out something that you can do to move forward. This is a chart from Dr. Schaffer's book that I like, about how to make usability routine. As I said, this book through all of these steps in detail. I would want to point out to you one thing on here that is very important that is that long line stretching across the top which says "finding an executive champion" because what we have found out is, very typically, a usability team may be in an organization for a year, 2 years, 3 years, 4 years and they're doing wonderful work but they're just spinning round. They never seem to be able to infiltrate the organization and really have usability become routine. Why is that? Why doesn't it grab? What we have found is that, often, the missing ingredient is the executive champion. If you don't have someone at the right level of the organization, who really champions the whole idea of usability, you're only going to be able to go so far. And in order to make usability routine, one must have a champion at a very high level that can network. Now this is not a usability manager, this is not a person actually doing the usability work. This is someone who is talking about usability and is excited about usability and moves it forward everywhere in the organization where it needs to be so if you are stuck in that spinning cycle you may want to think about it. "Do we have a champion?" or "How can we go about finding a champion if we don't have one?" Now I hope that in sharing these ideas about making usability routine, I haven't overwhelmed you. Sometimes when I speak to people about this and they are not very far in their maturity chart or didn't score very well in their institutionalization report card, they get a little panicked and say, "Oh, what am I going to do?" So here's a list to highlight – do these things first and if you even pick one of these to work on in the next 6 months, you would make great strides. So, get an executive champion if you don't have one, set out to make that a high goal for yourself. If you don't have usability staff trained well, get that started, get them going on some classes. Provide briefings to your manager. This is what I call enlightenment about usability. In order to get usability routine, you need to get by a larger group. So make sure you're talking to the right people about the work that you've done. Do one or two "showcase projects", this is the term we use, we take a project that will get a lot of usability and we apply everything we can about usability –user-centered design process, and it becomes a showcase that people talk about and that you can talk about in your management briefing. That's a very powerful thing that you can do. Get a start on your usability infrastructure. If you don't have a user-centered methodology and resources and standards and templates, get started on that. Either purchase a usability central or put together whatever you have, or own, start your infrastructure that way. Then the last thing, develop a strategy, look at your institutionalization report card and your usability maturity model chart and figure out how are we going to move forward? Give yourself time, it may take some time to do it but figure out what you're going to do in the next one to two years to move this forward. So I hope you have gotten some interesting ideas out of our presentation here. How to calculate ROI on a project, how to make usability routine and what the ROI for that is, please stay tuned. We're going to now answer questions that you have been sending in. Thank you.

Jay More: Susan, thanks so much for your presentation, it was very good. You used the analogy of moving from an emergency-room model to a wellness model, in other words, from piecemeal to sustained usability to stay. Why is it so hard for companies to move from piecemeal to the wellness model?

Dr. Susan Weinschenck: Well, you know, there are a couple of reasons. One is that, people get used to doing things a certain way and they have been in that emergency room model for so long that it has actually come to seem normal – that that's the normal mode of doing it. So the first thing is to recognize that it doesn't have to be that way, and that's not necessarily the best thing to do. The other reason I think it is hard is that it requires change. You're asking people to change the way they work here, actually asking the whole organization to change the way it works. That's not necessarily an easy thing to do. It may take a while for the change and the best thing to do is really is one of the things that we talked about in the presentation is to start implementing at least one.

NORTH AMERICA: QUESTIONS & ANSWERS

Jay More: Well, it looks like we've been getting some questions. Are you ready for some?

Dr. Susan Weinschenck: OK

Jay More: OK, first question. Your methods of establishing metrics for usability ROI are helpful. Is there a "peer-recognized" methodology to validate once the ROI metrics are established and calculations (savings, increase in revenue, etc.) are generated?

Dr. Susan Weinschenck: So I think what they are asking here is once you've done, you know you do the ROI calculations and you do it possibly or probably and you do it before you've done the usability work. I think what they're asking, if I get this question right, is how after, how can you validate that? You made your estimate and you said, you know, you were going to save this much money, you know make an extra $50,000. How do you know?

Jay More: You mean like a post-study?

Dr. Susan Weinschenck: Right. Can you check that out later? So the best way, is that you can do that. The way you do that, is actually, interestingly enough, it would be best to plan ahead for testing it afterwards because you need to make sure that you can then collect the data beforehand and then collect the data afterwards and do the comparison. It's fairly straight-forward. If you think that there would be more hits to the website or lower drop-off or less calls to the helpdesk, whatever your measure was, you decide ahead of time what is going to be your timeframe and you go ahead and make that measure.

Jay More: Ready for the next one?

Dr. Susan Weinschenck: Yes, go ahead.

Jay More: You mentioned you ran usability ROI calculations for one of your customers and the savings over 5 years would be $11 million. Was it that much?

Dr. Susan Weinschenck: No actually, this was a bank. I think that these people are asking from the previous question, did we actually test it out and did it turn out to be that much? With this particular client, we did not do the after-validation so I don't know if it was really $11 million that they actually ended up saving. But I can say that the calculations that we did in the presentation, you know, that number was surprising to me when I first came up with it and certainly with the calculations that you make may also be that way. But those were, interestingly enough, realistic numbers. We were very conservative in our numbers. The reason you get large numbers and you will only get large numbers like that if you are talking about an application that people use day in and day out and that involves thousands and thousands of users. That is where you can get that kind of savings. This was not one that we were able to validate afterwards.

Jay More: But it more than paid off the usability study?

Dr. Susan Weinschenck: Yes. The usability study was $40,000 so that way we were very sure that we were going to have that returned.

Jay More: How have you handled situations where others within your organization believe anyone (project leaders, business analysts, etc.) can conduct usability work vs. working with a qualified user design specialist?

Dr. Susan Weinschenck: You know, it's an interesting situation that years ago, and I have been in the field for longer than I would like to admit. Years ago, we actually trained project leaders, business analysts and programmers to do usability work. That was normal years ago. It's only that in the last 10-12 years that that has shifted and we have come to have usability specialists. You know, there is a, little story that I tell around this, the time that I decided that it was very important that we have specialists, I was attending years ago, a program in class for a GUI object oriented programming tool. I didn't know about the tool and thought it would help me make better interface design. In the class, we had all programmers except for me; I was the only usability person. I was trying to do these exercises and my partner through the 3-day training was very frustrated with me because every time we were working on an exercise, I would be worrying about the interface and trying to make it more usable and she said, "You should be worrying about the programming code, the programming code!" Finally at the end of the 3 days, she pounded her fist on the table and said, "We don't have time for this!" It was a big light bulb for me because I realized that she was never able to concentrate on the usability. The programming was so daunting and there was so much to do as a programmer. How can we expect people to worry about the programming and worry about the usability? It is two very special areas of expertise. It would be like if you came to me as a usability person and said, "Hey, on this project, on this site, could you write a couple thousand lines of code?" and I would go, "Are you kidding?" I really feel it's the same way; it's not that these people could not do the work. They could, if they had knowledge of the methodology, if they had done some training and if they had the time and attention. The usability portion, of course, we feel is so important it takes time and attention. It's not something that can be done at the time...

Jay More: But you're not saying that, you're saying that we should train other people as user specialists. You're not saying not to train them and the conflict of making them routine.

Dr. Susan Weinschenck: Oh, yeah, that's even a slightly different question. It brings up a good point about training because everyone needs some education but not the same.

Jay More: No.

Dr. Susan Weinschenck: Some people should get a lot of in-depth training to become certified usability specialists and you have other people who, all they need are a half-day session on what usability is and how to integrate it into a project. So you really want to adjust the amount of training to fit the particular people.

Jay More: Here's a good question, how can we learn more about the Schaffer-Weinschenk method?

Dr. Susan Weinschenck: This is probably the easiest question we're going to get.

Jay More: The Schaffer-Weinschenk method is HFI's usability methodology.

Dr. Susan Weinschenck: Yes.

Jay More: That's an easy question.

Dr. Susan Weinschenck: That's an easy question, yes. Thank you for whoever wrote this and for giving me an easy one. There are a number of different ways you can learn about it, we do have information on our website about the Schaffer-Weinschenk method. Probably the book is a good way, Dr. Eric Schaffer's book on "Institutionalization of Usability" and you will find information about the Schaffer-Weinschenk method in here. So I would say the website and the book are probably the easiest way or of course, attending our classes as well.

Jay More: We can also, for those of you, who are interested, just send us an e-mail and we will send you a wall-sized poster of the Schaffer-Weinschenk method. We produce those and we also have posters of the usability maturity model which Susan referred to so we have posters.

Another question, on the report card, on item G4, this person is referring to the usability maturity model.

Dr. Susan Weinschenck: No, they are talking about the institutionalization report card.

Jay More: Oh, the institutionalization report card, right. Does "10% of the IT staff" refer only to the people who run our servers? We make software, not web-products. For us, does "IT" include the software engineers and developers?

Dr. Susan Weinschenck: OK, and what this person is referring to and we talked about, this number comes from Dr. Schaffer and this is a kind of interesting and thought-provoking number "10% of your IT staff should be usability staff". So let me talk a little more about that. What we are referring to there is that you take all of your IT staff, your programmers, your developers, your QA people, and your business analysts, anyone who is working from your IT side on developing software products on the internet. Take that number, let's say it is 4000, and take 10% of that number, that's 400 and that's really how many people you should have within that department or wherever usability work resides, that's how many people you should have doing usability work. Now it's a huge number and when we give that number out, people look at us in shock and they say, "There's no way we will have 400." So first of all, it's an interesting number because it gives you an idea of how far off you might really be so if you have 4000 IT people and only 1 usability person, you know how way off you are. I don't know if we have any clients who actually hit the 10% number – some of our small clients might but for our large clients that is a large number to hit. It's really good to think about because it gives you an indication of how important usability is to that organization. What is that percent, is it 0.0005%?

Jay More: But in this case, where it is a software company would it include the software engineers and developers, it does, right?

Dr. Susan Weinschenck: It does.

Jay More: And the server people?

Dr. Susan Weinschenck: Yes, anyone who is doing IT related work.

Jay More: This is a good question because it reads right into the point you mentioned about the need to create a usability budget, should you use potential ROI savings to decide which project to work on? I think that leads right into that.

Dr. Susan Weinschenck: I talked in the beginning of the presentation that there are two sides of doing usability that comes out ROI business case perspective. If you were probably looking at something like "Let's design something that's usable because that's the right thing to do" so in this question, should you use savings to decide what to work on, you know that's really which side are you going to make your decisions – based on ROI or you're going to make your decisions on what's best or more usable for the user. I don't know if there's a right answer there. I think it's important to take it into account though and therefore be conscious of how are we making our decisions. How are we going to decide? Are we going to work on the projects that have largest ROI or you know, there are a lot of things, you can work on the project that the in the present the company has the most interest in, that may be your decision. I think any of those are valid but it's really good to know that you choose this project to work on because of this reason and yes, I do think it's a very good idea to look at the ROI savings to prioritize your projects. As you mentioned, with the budget, that can be very useful because now I can say, these are the projects we are going to work on and here's why and then look how much money we are saving.

Jay More: One leads to the other.

Dr. Susan Weinschenck: Right.

Jay More: Do you have finance approve the ROI, and is it tracked ongoing each year? This is almost a questions of how does the finance group work? How are they involved and who in the organization would track the ROI by usability?

Dr. Susan Weinschenck: Yeah, it's an interesting question. I don't have a real clear answer for this but let me tell you some of the things you should think about. When you're deciding on ROI, it is very important to decide on who you are going to include in the conversation. So you have, you know, stake holders and they might be from a particular business field. They might include IT, the usability group; the marketing revenue depends on your organization and your projects that you're talking about. So whoever was involved in this when you were talking about ROI and who were involved while you were making these calculations, I think it matters that they need to have some input into the tracking. So I think it's separate, there's one question of who literally does the tracking and the other question is who has to be part of the group to decide how the tracking should be. So I really can't answer that.

Jay More: Let me take a crack at this, I think that in a product company it could very well be the project manager has a role because they have insight into what's going on and into where the ROI impacts, whether it is marketing, sales, accounting or manufacturing, they see the whole picture. In a situation where it is not a product company, then finance creates the budget and so the budgets cross again, marketing, sales, distribution and manufacture, so it can very well be that finance tracks the ROI but as you said that depending on the company, finance would have all the information.

Dr. Susan Weinschenck: Right. We have lots of questions on this.

Jay More: Lots of questions. In your ROI calculations you mentioned that you must remove the cost of usability. Why not the cost of development, testing etc? Aren't these all costs of making the changes?

Dr. Susan Weinschenck: And indeed, you can. So this is something you really need to think through. It is perhaps a partial way of questioning how formal are ROI calculations that you are doing and for what purpose are doing the ROI calculations. So if you're doing just a very high-level "Let's see if this is even worth doing" and it's an informal calculation and it's for yourself and your team, then you may not need to take into account a lot of the costs. You're just trying to see, for instance, we did one recently, and it was for a client and the question they were thinking was spending a very small amount like $25,000 for an expert review and the question was if it was even worth doing that. The ROI calculation I came up with showed that they would have a return of $2 million for the $25,000, okay that's a very top-of-our-heads, no I can't say top-of-our-heads, we had some data. But it didn't take very long to come up with that number and say okay this was worthwhile, let's go for it. It was not a formal ROI so if you really want to do a formal ROI calculation and you want to come up with the number then I think you do have to take into account the costs of development, testing etc. and we have done that. So the idea is I'm going to make changes to re-design. I'm going to make changes and I have the cost of the usability effort to do the re-design then I have the cost of the programming. See that's what they are asking.

Jay More: ROI, right.

Dr. Susan Weinschenck: so it might take $40,000 for the usability work and it might take $200,000 for the programming and yes should that be subtracted? Yes, I think that's a fair argument and you can subtract that. So sometimes when we work with clients, they say, "No, don't do that, let's focus only on the usability cost of it and not the whole process of programming it" but I think it's very valid and can often be taken into consideration.

Jay More: It's a good question.

Dr. Susan Weinschenck: Yeah.

Jay More: You talk about having a champion for usability at the executive level. In terms of time, how long have you seen it take on average to convince management of usability needs? That's a different one.

Dr. Susan Weinschenck: Yeah, I cannot remember someone ever asking me as precisely. This is an interesting question. You know I have seen that it really varies.

Jay More: It varies.

Dr. Susan Weinschenck: You might have, so let's say, you get your executive champion and the executive champion is up and going and doing the work he or she needs to do to talk about that interest in usability throughout the organization. I have seen it literally take days, literally days, once the right kind of person is there and it just kind of goes, whoosh, which is one of the wonderful things about an executive champion. You can make things move very quickly because that person is able to talk to the right kind of people. I have also seen it take people months so it doesn't necessarily mean it's going to go really very fast. I would say that days are probably unusual, I would say, on average which is what the question was; I would have to go back and think about this but off the top-of-my-head, I'm going to say that on average it takes 2-3 months only because the question was how long does it take to convince management of the need for usability not how long does it take to get everything rolling or institutionalized, I would say 2-3 months would probably be sufficient for an executive champion to make things start to happen.

Jay More: Yeah, I see these two forces at work, first at the executive level, you look at it as the nucleus within the circle and there are pressures on the executive level, analysts reporting on the company's website that do reports and say, "Wait a minute, your website needs work" so the executive level says, "We have to start to do something" and simultaneously outside of the executive level, at the outer part of the circle, there are usability analysts, usability specialists who are working hard to build support for the executive level. When those two forces come together, then it works really fast. I've seen that happen in several companies where the inside gets stimulated by the outer side and the outside is working really hard to make things better and to come together.

Dr. Susan Weinschenck: Yes, and you know, that's the reason why a lot of times when we are working with an organization to help them institutionalize that it's the people, the usability person on the in the team who has been working for it for 3 years really hard and things don't move forward and they get really frustrated and all of a sudden this executive champion comes forward and everything moves quickly. I guess they sometimes feel, "I guess I was ineffective and we needed this high-powered person" but that's not it. All of their work was very important for setting the stage and setting the groundwork for when it became needed.

Jay More: Right, for when it comes together. I just want to let everybody know, thanks so much for all your questions. They're all coming in; we have a lot coming in. So bear with us as we try as many questions as possible. Ready for some more?

Dr. Susan Weinschenck: Yeah, ready for more.

Jay More: Your examples and studies seem to be all about websites. What about devices such as PDAs, cell phones, DVD recorders, and televisions?

Dr. Susan Weinschenck: Yeah and you know, when I was thinking through examples to use, I was trying to pick a combination of websites and internet but we have done a lot of work and a lot of ROI calculations for devices specifically for printers, cell phones and medical devices. Those are the most recent ones we have worked on. So the concept is the same, again, what you have to do is to decide what is the metric that makes sense and you do your calculations the same way. This can be done for devices as well. Anytime you have, you know, any technology and you're talking about usability work on it you can do it.

Jay More: I invite the person who asked the question to feel free to e-mail us for help in figuring out what those variables are in their case.

Another question, are there any methods of doing usability analysis retroactively post project? That relates to the other question.

Dr. Susan Weinschenck: No, I think this is a little bit different because they aren't asking here about validating the ROI, I think they are asking we didn't do an ROI ahead of time but now, we did the usability work and now we're asking, was it worth it?

Jay More: Oh, OK.

Dr. Susan Weinschenck: You can do it afterwards, what you'll need to do is again, of course, figure out the right metric is but then you may also have some pre-change data that you can go back to so you might, let me give you an example, let's say you have an internet site and you've made some changes to reduce the drop-off rate and you say, "Oh look at that, all of a sudden our drop-off rate has improved" and someone might say, "Well, how much did it improve? And can we put a dollar amount on that?" Then you go back and you can say, "Well, it improved this much" and you can cut it back your way to include it into the ROI calculation. So that might not help, obviously it's not going to get funding for that project but it might help you get funding for the next project to do that.

Jay More: Let me go to this question first, we are currently in a state of analysis-paralysis. We have run so many UI studies and have so many options that it is not working out. How do you advise we deal with this? It's a good question.

Dr. Susan Weinschenck: Well, let's see. So I think one of the things I would ask is why do you keep running all those studies? Is it because you're not sure of the data you're getting? Or you have a lot of stakeholders who are perhaps complaining about the data you're getting so you just run another one? You can get to the point where, as they said here "analysis-paralysis" where you have too much data. What I would suggest you do is, you know, stop for a while, really step back and try and look at the data at a higher level. Then if you, see the question says that they have so many opinions so if the data is confusing then that's one thing, you have to figure out why are we getting different data but if it's a question of opinions about the data, I would say that it would be good to get someone who hasn't been involved in the studies. It could be someone from inside the company and you could bring someone from the outside to come look at the data...

Jay More: And give an objective opinion.

Dr. Susan Weinschenck: Talk to all the people who've given opinions and then say, "OK, look here's what I think is going on." Sometimes, I think you get too close to it and then you're not an objective usability specialist anymore, you become too involved.

Jay More: So get outside help?

Dr. Susan Weinschenck: Get outside help and as I said it could be someone in your organization but just not in the group who was doing the study.

Jay More: How important a role does internationalization play in your interface design? Have you had to change your design based on different cultures? Have you heard that question before?

Dr. Susan Weinschenck: Yes, we do a lot of internationalization work and I think it's one of the, for me when I joined the HFI, it was one of the most exciting parts because I had not done as much global and international work before. We do a lot of this work at HFI. We have offices around the world and a very large group in India, in different places in India and in Singapore, who really specialize in this. So you do have to base you design on different cultures. You have to work out ahead of time what the cultural issues are and then you need to work on the design based on that so if someone is interested, this is definitely the place they should contact us and they should just tell us specifically about the project and we have information we could send.

Jay More: Yeah, I'd like to invite the person who asked the question on internationalization to send us, or you can send it directly to me at jay@humanfactors.com. a question on internationalization, Apala Chavan who is our VP for Asia, heads our internationalization group and she's written some wonderful white papers on internationalization and I would be glad to make sure you get those so send an e-mail to us and we will get you that information. It's not on a topic on this webcast but we have white papers on that.

Another question, do usability teams typically own the standards and guidelines. If so, do you have any examples to share on how these are then shared with development teams? That's a good question.

Dr. Susan Weinschenck: Yeah. Standards and guidelines first let me say that someone has to definitely own them. I think the usability teams can own them. In some organizations, there might be, I know in some organizations they have a usability team and then they have an interface design team that's kind of a part of IT. The two can work together but they're not the same. In a case like that, it may be the interface design team that's a part of IT that owns the standards for development. It could be either one. The second part of the question was examples to share on how these are shared with the development team. So if the usability group owns them how do you share them with the development team? First of all, and it's very, very important is that when standards and guidelines are developed, the development team and the IT people are involved in that. It's not good for the usability group to do the standards and guidelines on their own when they are in a vacuum and then say, "Oh, we have these wonderful standards and guidelines and now you should use them" that would be a harder style. So if the development team is involved in the development of the standards and guidelines they will automatically have an interest then all you have to do is a marketing and education campaign for the standards and guidelines. One of the things that we do when we are working with clients is we have classes, we do training, once this the standards and guidelines are developed, we invite all the stakeholders and then have classes for our developers that are a combination of good usability, design, let's say you're working on GUI, so we teach you to effectively design GUI screens. Then we introduce standards or if it's web, so we combine usability and the standards together in a class and teach those to the developers. That would really help them get on board and actually they could be using them. The other thing I want to say is that one of the things that are very powerful to get standards and guidelines really used is to not just do the standards and guidelines but to combine them with templates. Very powerful, so don't have just a picture and a description, that's important, but you actually have templates that people can download, what you have built into the usability central, that people can download and use. Your IT staff will work with what they have, you can give it and they will say, "OK, I will use it", that's easier than reading a book.

Jay More: Thanks, Susan. When justifying ROI, the conclusion is often to bring in consultants to do what is perceived as "one-time work." Rather than mixing usability costs in with software project costs, is it more effective to try to get a budget for starting a dedicated group or team and not hide the cost inside a project budget?

Dr. Susan Weinschenck: A complicated question.

Jay More: But a good one.

Dr. Susan Weinschenck: OK, let me work through it. So the question is- do you want to bring in people to do usability on this project, so it's a one-time thing. So we're going to bring people in who will do usability on this project. So rather than mixing usability costs in with software costs so that's the way it will be then. I have my software project and a part of that is bringing in, so d I want to get a budget to get a separate team that does this all the time and then when I want them, I bring them in and I charge for that. You know, clients do, and our clients do it either way.

Jay More: But in terms of your wellness program, it would sustain.

Dr. Susan Weinschenck: In terms of the wellness program, right. I think, in terms, I would rather you look at it in terms of what's going to move us forward. The best mix, Eric talks about it in his book, he talks about a lot in here about where the usability team should reside. The assumption is that there should be a usability team somewhere so really the best mix is to have a core usability group and that brings us to the question before. Before usability group is responsible for owning the standards and guidelines, making changes to them, the tools for doing user-centered design, and then you have, you either send members out to work in projects or you have other members who live and reside with other department groups-other usability groups. But they tie in back to the central usability core so I do believe that's a good idea that you have dedicated group and you work with that.

Jay More: And this question relates to the last question. Do you think that usability should be centralized within a large company (division independent), or do you think it's better for groups of usability people to report to usability managers within specific divisions?

Dr. Susan Weinschenck: A lot of this depends on what works in your organization. How your organization is set up historically and what you think will work best, so first let me say that so don't try to read the information in the book about different ways to set up a usability group. You don't have to necessarily force-feed that if it's not going to work. Having said that, it really is best to have this central core group and then you can feed people out. As I said, you can have them reside long-term out in the other groups and they can also come back to the main group. I do not think, to go a bit further, it's my experience that a number of separate usability groups – having a usability manager with some of the people over here and a usability manager with some people over there – and the reason I say that is it dilutes the power of usability within the organization. As you have managers you may or may not be doing things the same way. Sometimes they are fighting, "I have usability", "My usability group is larger than yours" so although I may have the people out, I may not have the different usability groups out.

Jay More: When planning and justifying usability efforts, sometimes people will try to get all their usability data at once with a focus group. Do you have any comments about whether we should test usability in focus groups or one-on-one? I don't think you've heard that question.

Dr. Susan Weinschenck: You know, traditionally, usability is not the same in focus groups. There are actually usability techniques that do involve people together but it's a little bit problematic. Traditionally, usability, whether it is usability walkthrough or an informal usability test or a formal final test, we usually do have them one-on-one so it's very tricky to have them in focus groups. The main reason is it's very different to ask someone what they think or what they think they would do. How you feel about something and what would you do – you may not get the same information as when you watch them perform. I've been reading the book "Blink" which I think you are familiar with, and I was reading just this morning, actually, about some studies between Coca-Cola and Pepsi – the taste tests. One of the conclusions is that to ask someone to take a quick sip is not the same as when they sip and drink the whole can. They said, you know when you get a whole bunch of people in a focus group and get them to talk about whether they prefer one thing or the other, now they're all the same thing. What did they really do, did they really drink the whole can and say that they would buy it again? So I think we're talking about the same thing here. In focus groups, if you ask people what they think or what they might do, it's not the same as actually measuring it. So I would say don't do a usability test in a focus group, do the one-on-one, it gives you the best data.

Jay More: Another question, I work in a large organization. Do you have any quick-hit suggestions for educating a new executive champion who has not worked with a UCD team and methodologies before? We've lost a few executive champions due to re-organizations and find ourselves having to re-market our services repeatedly, even though our methodologies are integrated within the system development lifecycle. Time to market is the current focus for our development teams. It sounds like his question is coming from a usability group who is losing its executive champion.

Dr. Susan Weinschenck: Yeah, that must be very, very frustrating. We go through all that work to find the usability champion and educate them and then they're gone and you have to do it over and over again. So when you're educating a usability champion, first of all, you need to make a separation between, and the people who've asked this may already know it but I am going to say this and clarify it for everyone else. An executive champion is not the same as a usability manager. So the executive champion needs a certain level of knowledge and education but they don't necessarily need as much as a usability manager. I think that would be, in order to relieve your burden and not have a high cost of educating them, you really want to think about what is it they really need to know. I think the most important thing we need to know is what is the most important thing for you? I think, you think that an executive champion as someone who is high up in the organization and telling everyone else what to do but I really think that you need to say to them – so if you were my executive champion, I would say, "You know Jay, here's what we need. We're not getting by from this group over here." And just tell them what they need and they are going to need obviously some basic usability information about what user-centered design is and I think they need education about the business perspective like the presentation that we just did. I think that would be you know how I would sit the executive champion down and download this presentation from the archive at our website and have them watch it because they really need to know how to talk about it to other people in the organization.

Jay More: Is a quick-hit suggestion, could that involve reviewing the history of what's been done before?

Dr. Susan Weinschenck: Yes.

Jay More: How should they do that?

Dr. Susan Weinschenck: I think they should have a one-on-one conversation about what's been going on in the organization before. So quick-hit is to talk to them about what's going on, give the really basic premiere of what usability is and have them watch this broadcast – those would be the three I would suggest.

Jay More: And bubble up any ROI information.

Dr. Susan Weinschenck: Yes.

Jay More: The last question and this is an easy question – my apologies if I missed this because I came in late. Will your slide presentation be available for downloading? Thank you for all that you do for usability.

Thanks, the answer is for now, download the white paper and we will archive the webcast very soon and you will be able to get the slides so don't worry about it.

EUROPEAN BROADCAST: QUESTIONS AND ANSWERS

Oh, this is a good question about government. Do you have any specific advice or tips on calculating usability ROI for a government internet site which is a public website that doesn't sell anything?

Dr. Susan Weinschenck: A lot of the examples that we talk about, when we talk about what ROI has to do with e-commerce and selling something and therefore making more money through the website but certainly that's only one example. So what's important with ROI calculations of any kind and especially if you're in a government sector, is to think about again, what are the metrics that are important? So for example, in a government site it might not have anything to do with making money but it might be for instance, how many people are using the site? Self-service at the site, customer satisfaction at the site, so all of these things are valid and you would still go through the same process. But you do is to make sure that you've picked a metric that makes sense for your organization.

Jay More: Thanks Susan. Another question – how can someone like and e-Government make usability routine within their ministries who churn out new e-Government websites and applications. Assume their development teams are not in-house but that they get their work done through tenders and 3rd party vendors. This sounds like it's a question coming from Europe, from government ministries. So this is a question about 3rd party vendors.

Dr. Susan Weinschenck: Right and you know that's very common. A lot of people do that, not only governments, we get that question a lot from clients who say, "We're a business but we also use 3rd party development groups or we buy software off the shelf" and so you have very little control about the usability of that software. So this is going to be very difficult but here's what I recommend. First of all, you want to make sure that for instance a development team or contract developers, you can work with them – you can show them your standard website and insist that they do some training on your user-centered design process in order to become a part of the fold and do it in the way that you want. So that's very important. The other thing that you can do is to make sure - if it's a 3rd party piece of software that you're buying, evaluate it for usability. You may not be able to customize it in a way that makes it truly as user-friendly as you want but you can at least compare. A lot of times, a lot of companies are doing a comparison like this, you know, they might be looking at it whether it's cost-effective, is it's functioning to meet our needs? But you can add that usability part. You can do the user testing on a 3rd party product before you decide to purchase it and how that can form your decision.

Jay More: So in a way, you're saying that vendors need a wellness program too.

Dr. Susan Weinschenck: Vendors definitely need a wellness program and you know, it's sometimes hard to convince them but the ROI for that would be helpful too.

Jay More: Thank you. Here's another question, I work as a usability analyst in my company. It's very difficult for me to convince people every time about the lack of usability in their product, and they are not ready to invest money on usability. How to resolve this issue? Have you heard that question before?

Dr. Susan Weinschenck: No, I haven't before. It's very, very difficult and very frustrating. Whoever sent this in, I know must be very frustrated. So there are a number of things that you could do, first of all, you have to realize that it's difficult and it may not change overnight but here are some of the things I would suggest. First of all, if you're having a hard time getting anyone interested and to move forward in investing money, you know to me that leads to the idea that you don't have an executive champion because it's the executive champion who is able to have those conversations, to get the funding and move things forward. So anything that you can do to find an executive champion as soon as possible would really help, even as you are still continuing to have conversations on your own, trying to interest someone. That would be very important. Then the next thing that I would say, and this actually speaks right to this whole question of ROI, you know as usability professionals we feel it should be sufficient if I come to you and show you the usability issues in a product. I assume you will get as excited and as concerned about them as I am. We just assume everybody is like us, right? But actually for someone who doesn't know about usability, it's not that they don't care, not that they are not a caring person. It's just not really high on our list of priorities. So from changing this from a conversation about "we should fix this properly because it's not usable" and turn it into "we should make some changes in order to improve" and then you fill out the blank with whatever your metric is whether it's customer satisfaction or safety. That changes or reframes the conversation. So if you're having a hard time convincing people to do usability tests or to fix usability problems, don't talk about it that way. Talk about it in terms of how much they do care about the product.

Jay More: So it's not just a problem, it is about specific benefits?

Dr. Susan Weinschenck: That's right.

Jay More: And talk about them in a specific way.

Dr. Susan Weinschenck: That's right.

Jay More: We've got some more questions.

Dr. Susan Weinschenck: OK

Jay More: Is there any difference in implementing a wellness program in a large organization as opposed to a small organization?

Dr. Susan Weinschenck: Interesting question, yes I think there is, well, there is and there isn't. So the basic steps are the same. But how fast you do them, what order you do them in, which ones are difficult and which ones are easy, will be different. If you have a large organization, and then you're talking about requiring a lot of change with a lot of people, there is probable going to be a slower, more evolutionary process. A small organization might be able to change more quickly; we're talking about changing the relationship between the developers and the user design people. We're talking about a handful of people rather than thousands of people so it might be easier to make that change. On the other hand, in a way it might be more difficult for a small organization in that there might not be as much ability to handle the time and the cost of the infrastructure. So a large organization might be able to look after the issue of standards and say, "well, it's going to take us some time to do it and we will need a number of resources but it will be worth it" because we will be able to multiply the results of it by the hundreds and hundreds of applications and programmers' time and in a small organization you don't have that. So you're going to need to be much more targeted in what you decide to do and the order in which you decide to implement it. I would say in a small organization, the user-design centric process and the training of staff are the ones that would be the easiest and have the biggest punch. Things like standards that would take longer and would involve a lot of people. That's something that you know, you may get around to and also perhaps a showcase project that you show it to thousands of people and even if you don't have thousands of people in the organization it still is useful or perhaps not as useful. So I would look forward to things in that wellness program that would really pack the most punch. Things would be like I said training and really looking at a user-centered design methodology.

Jay More: I know that in Eric Schaffer's book, he mentions that in a large organization, 10% of the development team should be usability.

Dr. Susan Weinschenck: Oh, it's such an interesting calculation and when we give talks to people, I always bring this up. That is, that Dr. Schaffer comments that 10% of your organization, if take the total number of your IT staff, your developers, and take 10% of that, that's how many usability people you should have. That can be, you know, a shocking number. If you have 4000 developers, you know, that is 400 usability people and I don't know many people where it quite reaches that 10% number but it is quite an interesting way to think about it. He also talks in the book about the way to adjust that number. So if you have a lot of people then maybe that number can come down a little bit. If you have 10 developers in your organization, you need one usability person at least.

Jay More: That makes sense.

Dr. Susan Weinschenck: Yes, it makes sense. So there's a case of how the 10% could be adjusted.

Jay More: Right. Another question – can we get a copy of the presentation? I would like to present this concept of my in-house usability experts to my company.

Dr. Susan Weinschenck: Yes. There are a couple of ways, first of all, there is a personal white paper which has the presentation, that is all the information and you can download it at anytime from the website. Our presentation here, including the video and our question & answer session will be available at the website when we're all done. So you can go to the website and download the presentation and use it in that way.

Jay More: Another question, oh this is a good question. Where can I find the Usability Maturity Model? I think it's about the chart.

Dr. Susan Weinschenck: The chart, yes. The chart's available in two ways. It is in the white paper which you can download. It is kind of small there so if you'd like a larger one that's easier to read, you go to our website and you can download an image, a larger image.

Jay More: Actually, if anybody would like a poster of the Usability Maturity Model, we've created some posters that you could put on your wall. So go ahead and just e-mail us and ask for the Usability Maturity Model poster and we would be glad to send that to you as well.

Dr. Susan Weinschenck: That's right.

Jay More: Is that alright?

Dr. Susan Weinschenck: Yes, OK.

Jay More: Here's a question – I work in a development organization in a telecommunications company. Can you give any more examples of usability metrics that would be applicable for software applications that are developed for telecom operators (i.e. applications that are not related to internet/intranet metrics but are used to manage telecom systems)?

Dr. Susan Weinschenck: Yes, you know what? Most of the work we do is on applications for the internet and intranet and a lot of the ROI calculations are for those applications. I'm not quite sure of what metrics would be applicable here, I'll give some of my own ideas but I also encourage people if they have questions about specific metrics for their particular product they could contact us and we can help them figure that out. We just recently, I've been working with a client about call centre metrics. So in that example for instance, we were discussing such as, there were really two metrics that were really important, one was could we reduce the amount of time it takes to process a call. So calls are coming into the call centre and they knew that on average it took 240 seconds to complete the call, can that be cut down? If we can make people cut that time down they can take more calls and also increase customer satisfaction. So we sued the measurement of time as our metric. Another thing that we used had to do with the number of calls again so it's not only that you spend less time on calls but you answer their question and therefore they don't have to call back in. That saves a lot of time and money as well and user satisfaction. So that's an example of really just thinking about it. So you need to think for this project and for my organization, what is the metric that makes the most sense? I think sometimes, when people get stuck and they're not sure which metric to use, I think that they are thinking ahead too much. They're thinking, "Well, we're not able to measure that, we need something that we have hard data for." I would encourage you to not think that way. Don't think, "Gee, is any of that data good and what can we use as a metric?" That's a really bad thought. We need to think about what really matters to us. What's the measurement that really matters? Brainstorm, don't judge that at first. You'd be amazed if you make that list and you start thinking creatively. You realize that ok; there is a way we can get at that metric. The other thing is you don't have to have hard data. People think that the data must already exist or that it's going to be way too expensive to gather but you can always estimate it. It doesn't mean that your ROI calculation is invalid; it's certainly good to be able to work with hard numbers. To know, for instance, how many calls are coming into the call centre? But it is very appropriate and valid to use estimated numbers. You have to let people know that it is not hard data but that would still connect the business case.

Jay More: OK, another question. I am a person who is interested in usability. I understand it is the key to success for any project (i.e. web or desktop applications). The point is that usability is always considered either during the requirements stage which is too early to talk much about usability or at the end stage (i.e. while testing) which is very late and pressure is very high to get the product out. How should we go on with doing this? This is an interesting question.

Dr. Susan Weinschenck: It's a big question.

Jay More: It's a big question.

Dr. Susan Weinschenck: I'll try and answer it succinctly. There are a number of different things in this question that I'd like to address. So first of all the point that usability is considered during the requirements stage when it's too early to talk much about usability – now I would say right there that I would re-frame the way you're thinking about that because actually talking about usability during the requirements stage is exactly where you need to be. Usability is about requirements and one of the things about usability or one of the ways in which I think usability has its largest impact is when it informs of user requirements. We have talked a lot to people about where should usability be or go in the software development lifecycle and there are a number of different ways to handle that. One of the most probable things that you can do is do your user-centered design process, you user-centered design. Know what the user wants and also what the organization wants and what we call a user interface structure, a very high level of design, create templates. All these are done before any work is done on the product. Numbers are not involved at all and all that is represented is actually a prototype that you will end up with, which becomes the user requirements which feeds into the development. So I think usability can be done very early and is all about being involved in that process of deciding about what needs to be done. Certainly if you talk about the other idea, which is testing too late, there are other parameters. Usability tests can be done throughout the process. We think that usability testing should the final test in the lab, the product exists, and it is finished and that we're testing it. You can certainly do it that way but that's not the way to do user testing. What you want to do is be is user evaluations throughout very early on so I would say that what this questions shows to me is that people are not truly understanding that the role that usability could have in the organization and could have in the process of developing a product. So this speaks then to doing that education work and the whole idea of institutionalization so that usability, people start to think of usability. Usability should be something which permeates throughout the whole process. So that would be really important to get going in this organization.

Jay More: Thank you Susan. Next question – we're a software manufacturer and a lot of our customers ask for web-based user-interfaces, even for their frequent users, since they are easier to maintain than traditional client software. But to the user it often means a GUI with slower response and less keyboard commands so you will have to use the mouse. How do you resolve that equation and still deliver ROI?

Dr. Susan Weinschenck: Well, you know, I'm not sure by the question, whether they mean. I'm assuming that if you do a web application then you should have done a GUI.

Jay More: It sounds like they are comparing and contrasting a web-based application and a GUI.

Dr. Susan Weinschenck: Yeah. There are a couple of possible questions in here and I will address a few of those permutations since we're not sure exactly how the questions were meant. It's really interesting with web applications and GUI applications, and I think that we tended to think for the last couple of years that the evolution has been, I go back quite a way in this business, so I go back to the character based legacy system. And we have in our minds that there was legacy and then there was GUI and then there were websites and then there were web applications and who knows what the future brings? But actually I don't think it's quite that linear. What we find is legacy, GUI, websites, web applications, and now we're almost back to GUI. We're finding a lot of people's requests for instance are for GUI courses. Starting now, we're going to be building our GUI back into our web applications. So I think this is a very important topic and people are interested in GUIs once again. Hopefully a new generation of GUI would come out as it did in the eighties. If you're trying to make a decision - should we go GUI or should we go web application, certainly there are considerations that go way beyond usability like technological considerations that we aren't going to involve in this discussion but you could do an evaluation. So you could do a prototype. Whatever the product is, as a web application, you could do a prototype of it as a GUI application and then test the usability of those two. Then do an ROI, again, pick your metrics, is it speed or response time, some of the things mentioned in the question and you could compare and you might you use it to help inform. I don't know if I would have made the decision based solely on usability as there are so many other technical issues that are important. But you could certainly add usability into that equation.

Jay More: Certainly in our design process, we often try and help our customers balance the optimization between the keyboard and the mouse.

Dr. Susan Weinschenck: Yeah that's an optimization issue.

Jay More: How do we incorporate usability design in standard development methods (which are already in place for projects) like Rational Unified Process or DSDM? These methods usually do have some sort of UI-designer roles defined, but they are not nearly as advanced and dedicated to usability as we might like. Have you heard that question?

Dr. Susan Weinschenck: Oh, yeah. I was almost about to say don't give me this question, I can go on for hours on this particular topic. So this is something that is very important and it's something we do a lot of work in. The way that the person worded the question is the way I absolutely agree with, these methods almost depend on the particular implementation of the method you are using. So I have seen for instance, RUP being done in a very user-centric way and I have seen RUP done with hardly any attention at all to usability. That's one thing, if you realize that any of these methodologies, you can blend user-centered design into them or cannot, so you want to think about how you're doing it. Certainly though, what's important as with any software development methodology is to look at that methodology and think how are we going to combine or mesh the user-centered design with this? There is no one way to do this; there are different ways to do it. If I were to distribute, I'll just give one example and then we'll leave this topic for another webcast, but here's an example. You can build in through the RUP process the user-centered design technique then it's going to happen at that stage and the two of them can run parallel, they can run together. You build communication points to go back and forth. Or another way to do it to use the idea of user-centered design to develop up front and use that to create user requirements as I mentioned before and that feeds in and is the beginning of the RUP process. I've done both methods with clients and they can both succeed. Which one is the best way to go depends on the way you're implementing RUP or whatever the methodology is and also on your own organization and how you work. So it can be done and I think it's very important. I don't think software development methodologies mean we can't do usability but you cannot assume that there is enough user-centered design which we talk about and that there is enough in there already and just follow the software process and we'll find it covered. You're really going to need to figure out where the intersection is. It's possible to do and very, very powerful.

Jay More: Thanks Susan that was a great answer.

Dr. Susan Weinschenck: We have a lot of questions.

Jay More: A lot of good questions. I want to thank everybody for sending their questions. What do you do when there are multiple design solutions to usability problems and solution 1 has a higher ROI but solution 2 seems to be the better one from a usability point of view?

That's good one.

Dr. Susan Weinschenck: Yeah, it's a very interesting question. So the idea is that we can have, and usually there is more than one design solution for any design, there would be infinite number of solutions each one better than the other. It's very possible to have 1, 2 or 3 possible interfaces or design solutions for a usability problem or for that interface for which you're doing that task. What the person is saying here is that one of the design solutions had the larger ROI however they decided to measure it but from the point of view of what someone thought was easier or more aesthetic, they preferred another solution. So here you have a balancing between the business and the ROI and the sensibilities of the user or the developing person. I guess you have a couple of things, one which is to evaluate and make sure that the ROI metric you chose, if you're using one, maybe it's a couple of metrics. Remember you can use more than one metric while measuring ROI and if you're using one metric and maybe design A is better with that one metric but maybe design B is better on a different metric. So think about the metric you are using maybe you want to check that out again. If you add a few more, again, like 2 or 3 ROI metrics, how did the two designs balance out? That would be one thing that I would suggest. Then when it comes down to it, someone is going to have a make a decision whether you go with the one with the higher ROI value or the one that is more aesthetic and I am using that in the sense that it is not just more visually aesthetic but also aesthetic to the usability person. It's a way we grow to realize that what makes best sense to us from the usability point of view may not be the best design and that can sometimes be a very hard thing to deal with when you're the one who designed it. There's a phrase I like to use when I know two people are on interface design which is, "when you're coming up with a design you should be very passionate about it but not necessarily married to it." The idea is that when you're coming up with designs you really have to be careful that you don't become so attached to your design that even when there's evidence that the other design may be better you're reluctant to give it up.

Jay More: So you're not afraid to make trade-offs?

Dr. Susan Weinschenck: Not afraid.

Jay More: It doesn't work in a vacuum.

Dr. Susan Weinschenck: That's right.

Jay More: And that's a part of wellness.

Dr. Susan Weinschenck: That's a part of wellness. That's a part of the maturity of being a usability engineer I think.

Jay More: How should usability be taught in degree courses when many of the topics are "techie" and programming based? That's a question about usability in the curriculum.

Dr. Susan Weinschenck: Yes and there has been a lot of, in other words, the field has matured a lot but not necessarily a lot of standardization. So if you look at how usability is being taught in the universities for instance, either undergraduate or graduate, you'll find sometimes it's in the Psychology department, sometimes it's in Industrial Engineering and sometimes it is in IT. I think any of these are good. I believe that it would best be taught as separate individual courses not necessarily, you know, a two-week part of a program of the course. I think it needs more than that but I think that these courses, these university courses could be taught in any of the departments. I don't know if there is one or the best way to do it. I think it's helpful though to break it out so that people are learning, so if you take a programming course, they're also taking a course in interface design per se, and that's a separate course. Certainly they can practice by creating programs, but it's on the course.

Jay More: OK, we have one more question.

Dr. Susan Weinschenck: OK

Jay More: Do you agree that for small organizations, it is better to outsource to UI consultants? I think you're going to say it depends.

Dr. Susan Weinschenck: Gosh, let's see if you guessed right. I think that using outside UI consultants would always be a good idea. It's always good to get experts in, to get another opinion. I think that you can combine that however with your own people inside. I think that if you're going to move towards wellness and if you're going to move towards institutionalization, I don't think you can rely on just bringing people in from the outside who do everything without anyone on the inside being educated and understanding what they're doing. So even if you said that we want to outsource all the interface design and all the usability work, if you said and if someone came to us and said that, we'd say "Great!" But even if you said, I would say back, "well that's not enough, you're going to need someone on the inside who has been through some training, some certification, and we're still going to need an executive champion"

Jay More: That doesn't go away.

Dr. Susan Weinschenck: No, that doesn't go away. So I really think you need to do a mix of that.

ASIAN BROADCAST: QUESTIONS AND ANSWERS

Jay More: The first question, does HFI do usability for products? If yes, can you share with us how usability helped ROI in terms of products?

Dr. Susan Weinschenck: Yes you can do ROI calculations for products. You know when we talk about usability; we are talking about the usability of any technology. If you have any technology and any human, there is usability involved. We do a lot of product work. We've done work on products, cell phone work; we've done work on medical devices and work on printers etc. The concepts are the same, basically you have to ask what are the metrics for this product that make sense, for instance, the medical device work that we have done just recently for a client had to do with safety so the important metric was safety. By improving the usability, can we improve the safe use of this medical device? It may not always be dollars; it may not always be conversion rates but the same principles apply. But what you need to do is figure out what is the metric that really makes sense for this product and use that along in the same as we talked about in the presentation.

Jay More: OK, I got lost in our questions. I've had the experience of conducting usability testing, collating results and presenting them yet the CEO is suspicious of the results. Usability testing is supported in theory, but the results are often discounted. Apart from getting the CEO along to the sessions, do you have any other suggestions about illustrating that the results are often definitive?

Dr. Susan Weinschenck: So it sounds like we have a doubting CEO. Well I will give some other suggestions but I have to say that the first idea is one of the best ones. Anytime you have someone who is not open to the data, you know, basically, this is real data; it's not your opinion when you're doing a usability test, and it really helps if they are there to watch at least some of the usability sessions. Now the first thing I would say if you can't do that is to use video and to show video clips of sessions and have him watch that. Now I don't know if the person asking this question is implying that they couldn't even use video, so if you can't have the person at the sessions and you can't have him watch the video, there are still some things that you could do. One thing to do is check if you can convince someone who is influential with the CEO who would not doubt it, do you have another manager that that CEO does listen to, can you get that person to go and talk to the CEO?

Jay More: In other words, get an advocate.

Dr. Susan Weinschenck: Get an advocate and have that advocate be the one to make the presentation rather than just you. Basically the data is there and it's very strong. I would really say that also make sure that you make a presentation. I have seen people you know, handing usability test reports and say, "here go through it." It's very easy to thumb through it and say, "well I find it interesting." But what you want to do is really have a presentation to talk about the implications of the results. I think that that's more powerful.

Jay More: And this leads into the topic you illustrated where you gave the importance of the executive champion.

Dr. Susan Weinschenck: Yes.

Jay More: A really good executive champion brings out the results.

Dr. Susan Weinschenck: Well, a good executive champion could be the one to talk to your CEO.

Jay More: Ah, yes. That's true.

Dr. Susan Weinschenck: And in making that case for you. That's exactly why you want an executive champion. For a lot of us dealing with usability, we may not be, or we may not have that experience to make that presentation in a compelling way but your executive champion will.

Jay More: OK, I've got another question. Assuming that the Usability Central Gold standard templates are base page types / guidelines, how are they customized for each organization? This question refers to HFI's Usability Central Gold product. Can you please talk about issues which might be involved in customization?

Dr. Susan Weinschenck: Some of the audience may be familiar with Usability Central, others may not. We have in our Usability Central product page templates, screen templates; these are templates, patterns that can be used. They represent best practice. What an organization does is use those templates to design their screen. We sell them and say that they can use them but what a lot of clients do is they customize those. So if you're going to customize those, the issues that you have to deal with are really about getting by it, by all the people who would want to use these screens, sometimes we set up a community. Sometimes we design and sometimes we customize some like this. We want them to be used across many, many different applications in a company. That's the idea of reusing them and I've talked to them about it during the presentation. It's really important to do two things really, one is to get by all the people-the departments, groups, IT, groups in the organization who are going to have to use the template. The other is to make sure that when you're customizing them, you're looking ahead. You're not customizing for one little project but you're doing it in such a way that these templates will hold for any application that you want to produce. We try to use for instance, two years out at least.

Jay More: OK, this question looks like it's the old chicken-and-egg question. I need ROI data to prove the value of user-centered design to my company. But since they do not recognize the value yet, I can't do a big enough project to demonstrate ROI. What do you recommend? Chicken & egg, right?

Dr. Susan Weinschenck: Yeah, well you know this is such an interesting one because we just had this happen this week. We're working with a client and they called and said that before they could get approval for the project, they said, "We want to get approval to do an ROI project but we need to do an ROI on the proposal to get the approval." So really what we said is start with anything you can even something really very small. You don't have to do a complicated calculation. Everyone wants to do calculation with real data and of course, that's more powerful but you don't have to, you can estimate. You can say, "Well, if we find five usability problems, we could save 20 seconds per call to the call centre." Just go ahead and do a small, do a very small calculation on a very small project just to get yourself going. Now one thing I do want to say though is that it's good to do a small project but don't do an insignificant project. Pick something that really is of value even if it's small.

Jay More: Thanks Susan. More questions.

Dr. Susan Weinschenck: OK

Jay More: What if we cannot even get the money for a usability test to prove that there is a problem in the first place?

Dr. Susan Weinschenck: Oh, so this is really like square one. So on our maturity model and charts are this is way onto one side. What's really important again is I have to say that you pick one thing, so I think in this case, you've really got to ask about an executive champion. You have to ask where is the support for doing the work which you are doing and instead of staying in that swirl of trying to do something and not being able to, look at that maturity model chart and say, "what is one thing on here that we can really do?" I really think they can use an executive champion. I think you're going to have to find somebody in the organization that will help and support this.

Jay More: Seems like a champion is needed there.

Dr. Susan Weinschenck: Yes. That's what I thought.

Jay More: Another question, the charts you showed near the end of your talk would be very helpful for us to have. Where could I get a copy of these?

Dr. Susan Weinschenck: The charts can be, a small version of them is of course in the white paper that they can download but they can also find the larger and easier to read version at our website. If you go there, you will see all different charts listed and you can just download them.

Jay More: You mentioned that to calculate the ROI we have to include the cost of the usability fix. How do we figure out this number? In other words, how do we figure out how much it would cost to do the usability?

Dr. Susan Weinschenck: Right.

Jay More: That's the question.

Dr. Susan Weinschenck: Right. It does, if you're really going to do the whole calculation or not, about how much you're going to save or make, about what the ration is or the benefits are, you'd have to know that. The good news is that as our industry has matured, we're getting much better at the standard usability work that we do. So really when you think about it, there are standard interventions, if you want to use a fancy word instead of fix, standard interventions that we do so you might do anything, the names differ a little because we have our own terminology here. But for instance, you might do an expert review – a review of the applications to see what the usability problems are, you might do a re-design, you might do a design from scratch, you might do a usability test, and these are all standard things that people do when they are talking about usability work. The good news is that we have been doing this now in this field which is at least 25 years old or more and we have a really good idea of what it takes to get these done and what the costs are. So for instance, here at HFI, we have very good estimates of how much it would cost to do an expert review so what I would suggest if you don't have those benchmark numbers, they could certainly contact us and we could give them an idea. In fact, on one of the charts, on one of the methodology charts, it says about how long some of these pieces would take, we could take that about how long it takes and translate that into how much it costs.

Jay More: But in the context of a wellness program, the usability piecemeal is done on a more sustained basis, how can people know how much that will cost? Could you estimate a sustained effort?

Dr. Susan Weinschenck: You can still estimate that because you can estimate, if you think of the maturity model chart, and what's in there, what do we want, do we want to get this many staff trained? What will that take? We want to have standards and guidelines, how much would that cost? So even though we don't do the work piecemeal, we can still price it.

Jay More: I see.

Dr. Susan Weinschenck: We can still take each individual piece and see how much those pieces are. The nice thing about the maturity chart is that you can go through and see which pieces you're missing, put a cross on those pieces and then decide what are we going to tackle in our first year? Then what are going to tackle in our second year and then budget for what you decide you need to do.

Jay More: You've explained the importance of having an "executive champion" for usability. What's the best way to find an executive champion within our company, and how should we approach them about supporting our usability activities?

Dr. Susan Weinschenck: So we keep coming back to the executive champion. There's a number of different ways. You know, really the key to finding an executive champion is kind of interesting. It is to have them self-select. You actually don't want to find one then have to interview them for the position, you want someone to come forward and say, "I want to be the executive champion." How do you get someone to get excited about being an executive champion? What I would like to say is that you want to cast a wide net, you want to talk about the work that you are already doing if you have some usability going on or talk about the work you want to do. Talk about the ROI to as many people as possible. Talk about making presentation, talk in the elevator. One of the techniques that a lot of our clients have found really effective is that they bring someone from the outside, they bring us in, or an expert in to give a talk about usability and invite what I call "candid executive champions" and see someone will get very excited. Someone will see that this is something that they care about, something that they feel they can really move forward, and they will want to improve the company but if don't talk to a lot of people, it won't happen. So basically you have to be a very outgoing and talk a lot about usability.

Jay More: But I'm sure that during our launched tour on ROI this last year and a half you must have helped many companies locate a usability champion.

Dr. Susan Weinschenck: We have, yeah.

Jay More: They came to the launch.

Dr. Susan Weinschenck: Yes, they came to the launch and sometimes what they would do is bring someone with them who they wanted to introduce to the concept and it helped.

Top