July, 2010 – Certified Usable: How to certify your usability practice and your designs

Apala Chavan: I'm the Vice President for HFI's Asian operations. Today, I'd like to introduce to you Dr. Eric Schaffer, CEO and founder of HFI. Eric has been a visionary in the area of field of usability and UX design. In fact, he literally wrote the book on making making usability a routine practice within organizations. The book is called The Institutionalization of Usability: A Step-by-Step Guide. Under Eric's guidance, HFI introduced a certification program for usability practitioners, but today, he's going to talk about a new program he has developed and HFI is going to introduce. This is a program not for individuals, but to certify the usability practice within an organization, and also to certify the usability of applications and products. So, Eric, Welcome,

Eric Schaffer : Thank you.

Apala Chavan: So, what is this new certification about?

Eric Schaffer: So the certification of an organization is about certification of maturity. So, we have a usability maturity model that we've had out since I wrote the book, before then. Now we're using that model as a foundation for certifying the maturity of an organization. That maturity means that the organization has the right kind of executive championship, the right kind of  organizational structures, the right kind of trained and certified staff, the right kind of culture, [and] the right kind of methodology and tools to do good usability design. So, that's the foundation of the certification of an organization.

Then, the very tough problem of how you certify a design – so, a product, an application, a website – how do you certify that? You certify that by seeing that this design was created within the right kind of organization, by the right kind of people doing the right kind of process, and therefore the has been followed to ensure the usability of that design.

Apala Chavan: So, you know a question that comes to me based on what you were just saying is, “Is the usability practice or of a product or application  - is this sort of a binary state? Is it either there or it's not there, or are there degrees of being certified given to define an objective?”

Eric Schaffer: So, we do have three levels of certification. Those levels correspond to the level of maturity within an organization. We have a usability maturity model, and you have level one, beginning usability. And I don't think anybody wants to certify as a beginner. You have level two – when you're beginning to manage usability, you have an executive champion – but that doesn't really indicate any capability to do usability design. Capability comes when you have the right kinds of pieces in place. So, our certification is for levels three four and five.

Level three – I mean, many people may be ready to be certified at level three – what that means is that you have the essentials in place to do good user-centered design. Just, you want to be able to have enough – at least a reasonable methodology, something in the way of standards, just a handful of people – those kinds of capabilities say that at least you have a serious usability operation.

Then, level four says that you have all of the pieces in place – a complete set of capabilities. Level five is just a small change from that – truly mature. Level five says that you have enough staff in order to routinely do user-centered design, and you have a culture which expects to do that. So, they're not going “Oh, we have a design. Should we do user-centered design?” Rather, they're going “Okay, we're doing a design program, and we have to set up to do those things we normally do to make sure it's usable.” The idea is that as the organization becomes mature, we believe that the quality of that design will mature as well.

Apala Chavan: Is HFI then certifying on applications, or products, as usable? Or, is HFI certifying that all of the best practices have been funneled to ensure maximum usability?

Eric Schaffer: It's really the latter. We're not certifying that every individual is going to believe that the design is usable. In fact, that's not even the goal. We recognize – I remember the second class I ever taught – that I was told we design for the fifth to the ninety-fifth percentile. Now, that's not 100% true, it depends on risks and things like that. But, we don't expect everyone will be able to use it. If we designed for everyone, it would be miserable for most people. So, the certification doesn't mean everyone will go “This is easy, this is great! I can use this without training!” What it means is that it's been optimized from a usability viewpoint, which is really what we want. We don't want something that everybody can use without training. We want something where the best practices have been followed to optimize the overall user experience - short term and long term - in terms of speed and accuracy, self-evidency, satisfaction, safety; all those kinds of issues. What we want is the correct practice, that's what we're certifying. Not in some test everybody could do it – that would be ridiculous.

Apala Chavan: Why now?

Eric Schaffer: There are a couple of pieces of that. One thing is that it's time now in the industry. So, in 2000, when I really started thinking and writing a book on the institutionalization of usability, it was just kind of an idea. We've thought about it, but very few organizations were seriously moving toward usability practices. It was rare to find that. Mostly, it was done on the basis of craftsmanship; individuals with some apprentices that did good work. Today, we're finding literally hundreds of hundreds of organizations around the world. We've got a practice in supporting organizations as they work out their strategy and put together the pieces from which you practice. So we see literally on every continent – form Africa, Australia, China – we see practices all around the world working to set up mature, industrial-strength user centered design capability, which is very exciting.

Today, with that capability in place, it's time to give them the ability to certify. It's also the case that we have the pieces in place in order to make it possible. Part of that is understanding how to certify. Part of that is being able to have models in place – like the usability maturity model, like the usability quotient – there's a set of things we've had to build over the years in order to say, “Yes, we have systematic ways of validating the maturity of an organization.” And, finally, the holy grail is to certify that end offering

Apala Chavan: Who really would benefit from this certification?

Eric Schaffer: I think that we're doing this because the field needs it. When we came out with the certified usability analyst program, we did it because the field needed it. There were a number of organizations I went to who asked for a way to certify individuals. But, when I was done talking with them, I went, “It's gong to be ten years, at least, before we have a certification.” And they went, “Well, we need it now!” and they were actually pretty angry. And I went, “OK, we're gonna spend $90,000 to build the certification.” And we did.

Many people said,“Who are you to build a certification?” At one level, we were the biggest company, and are the biggest company specializing in usability. Like, Microsoft has Microsoft certification. Part of it was there was nobody else to do it. So, we stepped forward. It's been incredibly successful. We've certified three thousand people – that's probably ten percent of the world's population of competent usability professionals.

Apala Chavan: Are these three thousand across the world?

Eric Schaffer: Absolutely. In every continent. We have certified people from Australia, to China, to Africa. In fact, we have certification tests we run in [cuts out]. So, we've worked to make this program successful. And that program is foundational to making the ability to certify organizations and certify offerings.

Apala Chavan: What exactly would the benefit be?

Eric Schaffer: Different people will benefit in different ways. If you want an internal usability practice: If you can say, “Look, we're a certified usability practice. We've met criteria that's been clearly set out for our maturity.” That says something to the rest of the organization. That there's a level of accomplishment.

To some extend also is the executive. For example, I know you've worked very hard to put our Indian operation into an ISO certified framework. And that framework – I don't think it's done much to sell usability work for us – but it's a discipline. And it's worth it as a discipline. This is also a discipline. If you follow this, if you move ahead in maturity, you're following a discipline of good user design practice.

Also, if you're buying or selling things, this is a way of ensuring that it's going to be usable. So, I know a number of department stores that have usability test labs to test appliances. Why would a department store need that? They're already selling stuff. But, they need to do that because they end up buying things and people complain, and they'll actually hand them back out of frustration. So, instead of going through that, they can just go “Oh, I can see it's certified usable, I know that those practices were followed.” So, for a buyer buying a product, buying an application that's been developed in another country, by another vender, being able to go to them and say “this needs to be certified usable.” Then, you're not going to get a final offering out that people can’t figure out how to use. In the end, customers will value it, knowing that they won't buy a DVD player that won't flash “12:00” in the middle of the night. They can say “I know I'm buying an app which will go onto my iPhone and won't confuse or frustrate me.” So, there is value in many ways to many people in doing this.

But, the biggest value to me is that it moves the field ahead.  Because as we want to drive the user-centered design community forward into an optimized level of maturity and capability, this kind of clear set of criteria and certification is going to empower them to do it.

Apala Chavan: In other words, you know all our existing CUA's, even, this should benefit them a lot. But, how, exactly?

Eric Schaffer: So, part of it is that your CUA meant that you were capable. Now, you're capable of moving your organization to certification. You’re capable with just a little bit of training to submit an offering and say “I want this certified as being usable.” And, so that's another level of value for our CUA's, and also for other professionals certified, say by the Board of Certification in Professional [Ergonomics]. Somebody certified with a CPE will have the same exact privileges, because they've demonstrated solidly that they understand the field. That's all we want.

So, the certified practice in usability – they don't need to use anything from HFI. They don't need HFI training, standards, tools, none of that. Wejust want them to demonstrate that they have the maturity in place. So, just like we have people who get their CUA's without any training from HFI, this is the same thing, where they can get the certified practice in usability, they can get individual offering certified, without doing anything with HFI as a whole, [or] as a consultancy. This is purely an offering to the field, which I think will move the field, as a whole, forward.

Apala Chavan: So, just as you went ... when you decided to the certification for usability practitioners, and everybody said, “HFI, why?” - My question is, “Why HFI?”

Eric Schaffer: On one level, we've shown ourselves to be responsible and effective in the CUA program. Having three thousand people certified, being able to administer that fairly, without any kind of bias or other issues happening in that process, has, I think, shown a lot. The other thing is: Who else would do it? If there some other organization in place to do it, and it was already there, I would be all happy, because then I could just consult with companies and help them to become mature, and help them to have better standards, and build their organizational capability. That would be very exciting, but it's not there. And, since it's not there, we're going to put it there – end of story. And, I think that we'll see the same kind of success that we saw with the CUA program.

Apala Chavan: But, I still have this question about... Doesn't a usability test perform the same function as this certification program?

Eric Schaffer: Not at all. Actually, I spend a couple of decades thinking about how we could use a usability test to certify something as usable. You can't do it. Anybody who says they can do it, they just don't understand.

Apala Chavan: Why not?

Eric Schaffer: Imagine for a minute that you're trying to certify wheat. Now, if you certify wheat based on its nutritional content, you can put it in a machine, it grinds it up, it test the nutritional content, and you read and say, “Okay, it meets the criteria, it's certified.” No problem. What if you want to certify the taste of the wheat? You can't put that in the machine. You have to have people try it. No matter what you do with the wheat, some people will like it, some people won't. So, what's the level of taste that you consider to be usable, how many people do you have to run to find out whether it's 79% usable (which is not certifiable) versus 80% usable (which is certifiable).  So, it's impossible to find a point. Does 80% of people mean it's usable? Then, I have to run thousands and thousands of people to tell the difference between 79% and 80%.

Now, here's the real problem. When you run a test, how long to you run a test for?

Apala Chavan: Maybe, one hour.

Eric Schaffer: So, in a one hour test, what I'm doing is measuring initial usage. What if I have a product that's going to be used for months or years? Now, I don't really even care about the first hour. So, I can have a design that's not all that self-evident, but within a week I'm really good at it and it's fast and effective. And another design, which is much better in the first hour, but is terrible a week later.

Apala Chavan: I see your point.

Eric Schaffer: You might be certifying with completely criteria. So, usability test are not a valid way of telling if things are actually usable. What you need to do is to take a completely different approach. But, certification means that you have trained people, certified, following a process within an organization, that has the right kind of capabilities and tools, that they do the process, that they document that they do the process, and that the certified people validate good designs. Then we do an audit to make sure that's true. Part of what HFI will do in this process is ensure that people – when put under the pressure of their organization – won't certify things that shouldn't be.

Apala Chavan: It's clear that usability testing isn't anything to compare.

Eric Schaffer: You can think about it – I did for two decades – but it doesn’t' work.

Apala Chavan: But, what about the other processes. ISO has a lot of design processes. How does this certification compare with ISO 13407?

Eric Schaffer: ISO 13407 is a very basic user-centered design process. It kind of just says “We know who the users are, we've built something thinking about it, then we tested it.” I wouldn't consider that a very strong process. That was the foundation of ISO 18529. ISO 18529 is a really serious methodology – a very good methodology – but it's only really meant for very large scale, enterprise-wide systems. If that's what you're doing, that's a great methodology, and you can get certified with that methodology. In other methodologies, in other circumstances, it would make more sense. So, we love the work that ISO is doing, but sometimes it would be effective in certification of an organization, sometimes not. Even if you look at ISO 18152, which is the maturity model, it's more about what you do than what you have. What we want to look at is, “Are the standards in place, are the tool sets in place, are the organizational structures in place?” So, when we look at maturity, it's not so much about what you do – although that's a good way to look it at, it's a useful document – we need to see that the pieces are in place so that the capability is there. So, ISO is great, and ISO has a lot of useful guidelines that are core in the field. So, the original work by Smith and [Moser] - I actually worked on that, it was like 1980 or something – but the Smith and [Moser] work gave this big long set of guidelines.

Apala Chavan: I remember that – it's a huge checklist, never-ending!

Eric Schaffer: Nobody's ever read it. (Laughter)

Apala Chavan: It's just huge!

Eric Schaffer: What they can do is go to our website and there's a questionnaire. It's very short, about ten minutes. So they can get on the website, it's probably on the white paper that comes with this. And that questionnaire will give an indication of whether their requirements are met or not.

I think that many organizations will find that they already meet the criteria. So, they can check with that, and if it looks good, they can request a full certification form from us. Now, that's bit of a long form, and we'll interact to help them fill it out, in terms of explaining things and answering questions. And then they submit that form, and we'll review it systematically, will do a surveillance on it. So, if they claim that certain people are there, we'll call. If they say that they have an executive champion, we'll make sure they have some reasonable understanding of usability. So, we do a surveillance process. If they pass, then they get certified at a certain level.

Once their certified at a given level - if they have a certified individual who gets just a little bit of training in terms of how to submit a certification request - then they can take something they're designing, and they can submit that to us for certification. Again, there's another form, and that submission has to document that the required [process, deliverable] are in place, and that the quality is approved by the internal certified staff.

If somebody ends up submitting certifications that repeatedly fail, they'll lose their certification from us. They'll lose their ability to submit. And, that's – in a sense – in their interest. Because we don't want organizations to be able to pressure you: “Come on, just certify it, it's good enough.” We want you to have the power of HFI behind you saying it needs to meet certain criteria, or I'm going to lose my certification. In a sense, that strengthens the certified individual and it should help make this a serious reality as opposed to something that somebody can forge.

Apala Chavan: That's very good. How long does the certification last for? Does it need to be renewed?

Eric Schaffer: There is a quick re-certification every year.

Apala Chavan: And it's sort of the same process?.

Eric Schaffer: No. It's a small process to just verify that the pieces are still in place. So, maturity is there, the design hasn't changed, so it's a much smaller process to do a re-certification.

Apala Chavan: Do you think that at this point, there are many organizations who are sort of, already there and can be certified.

Eric Schaffer: I would think there are several hundred.

Apala Chavan: Really?

Eric Schaffer: Yes. Because, globally, we have so many organizations that are putting together serious industrial strength usability operations. They have the people, they have the certified people, they have the operation,s they have the methods. This is not a certification that is asking people to do something strange or idiosyncratic. This just says you're doing the normal best practices. It's the same thing as the CUA. To become a CUA, you need to pass a test. If you're good in the field, if you've had some training – even if it's not HFI training – you can pass the test. We've had lots of people like that.

Apala Chavan: But, in case anybody is interested, do you offer per-certification assistance?

Eric Schaffer: Well, we can do some things there, but what  I recommend now is that people take the quiz and see if they're close – where they stand. I think that we'll find many organizations who can get at least level three certification with very little change. At the same time, I'm hoping that this will help motivate organizations to greater level of rigor and greater levels of capability. That's why we're going through all this effort to put this in place. It's what I think the field needs right now..

Apala Chavan: Well, that was very interesting. Thank you, Eric. I think this is truly going to be another milestone for the industry. And, the insights you shared about the rationale for the certification and what it does for individuals and organizations – I think many of our viewers will be very inspired by that.

If any of you would like more information, you can visit our website www.humanfactors.com. The white paper is there, as is the quiz that will help you get started. In case you have questions and you want to discuss anything at all, you can go to HFI Connect. We've actually set up a forum there, to enable discussion on this certification – connect.humanfactors.com.

If you're interested in getting any of these certifications, please be sure to contact HFI, and I'm sure we'll get back to you right away.

Thank you so much.

Top