March, 2006 – Managing the knowledge behind business decisions through user-centered design – a case study

Jerome Nadel: Hello and welcome to the latest in the series of webcasts presented by HFI's Usability Broadcast Network. We are glad that you could view the slide today to discuss our topic "Managing the Knowledge behind Business Decisions through User-Centric Design." I am Jerome Nadel, Vice President and Executive Managing Director at HFI and it's my pleasure to introduce Giovanni Piazza, the Global Director of Ernst & Young's Knowledge Web Program. Before we begin, I would like to remind you that there is a white paper that accompanies this web cast which can be downloaded from our website. Also we encourage you to submit questions through the presentation with the link at the bottom right-hand corner of your display. The main presentation will involve multiple topics but there is essentially one core theme and that is influence. This is the case study of success. Success in the outcome of redesign but more so success in influencing and effecting change. The main presentation will be divided into three core parts. The first led by Giovanni will be an overview of the Ernst & Young environment. He will describe the need for knowledge management, the architecture that supported it and lastly what worked and what didn't. I will then share a little about the user centric design the anatomy of our collaboration, some findings and lessons learnt and lastly focus on the power and influence of user centric design. In the third section we will go back to Giovanni and he will be sharing with us post implementation, quick wins, integration into their portal which is now live and useful takeaways so with that Giovanni I would like to hand it over to you.

Giovanni Piazza: Thank you very much and thanks for giving me the opportunity of being here with many of you. I would like to share what I call the story of the growth, success and renaissance of the Ernst & Young knowledge web which is our knowledge centric, knowledge sharing global environment. Obviously the first question is the business question why does an organization like Ernst & Young need to manage its knowledge? Well, we need it because there is a hundred and thousand of us spread over 140 countries, almost 700 offices in the network of member firms and characterized by different business units with a myriad of service offerings or service offered by each and every one of these business units and hundreds of communities. Communities of practice, communities of interest, communities of people who grow up together by the common need of delivering world-calls service to our client despite or above or beyond the geographic barriers, the cultural barriers and the need for us to deliver more to more clients faster. Ernst & Young has a long story of knowledge management dating back to 1993 and I could have produce a slide literally crammed with hundreds of milestones and we can have philosophical discussion and whether we can have hundreds of minds possible but that is what we consider from the first team that answer knowledge management to a full blown world-class knowledge management environment that we have today. Again, our core value we are a high-end professional services firm and our core value is to retain the passive knowledge that we developed in the process of delivery of service to our customers, our human capital or we are our own resource-the main resource one of the key words within our organization is people first. So nothing more than knowledge management than a robust knowledge management environment can help us achieve our objective. I have a slide right here that shows the growth of our revenues dating back to the year 2000 is not matched by an analogous growth in headcount. I would be calling it presumptuous of me to give all the credit to knowledge management but serving knowledge management is a big part of this scissor between the revenue growth and the headcount. That means to an extent that knowledge management is a way of socializing the passive and exclusive knowledge works.

Jerome Nadel: Giovanni to those of us or those out there who are less familiar with your world knowledge management might seem somewhat esoteric. So if you can try to provide a concrete example of why a management practitioner would be working for a given client? So this not a luxury to access this perhaps describe a little bit of that and then discuss that from the perspective of these knowledge management professionals.

Giovanni Piazza: Sure, in its broadest terms a knowledge management environment is a place that is characterized by several variables and you asked me an excellent question because the answer was right in the next part. Several components of this environment some of it is technology and I'll come back to this one some of this is strategic people and culture so people need to feel the need of contributing knowledge of sharing the knowledge because other people need it so I can take advantage of them but without going too far on the philosophy track if you want to stay solidly anchored to reality we can talk about reality we can talk about Ernst & Young knowledge where the system allows or offers our practitioners on the ramp on the E&Y information super highway so you are contributing knowledge then there are six lanes and then there is the exit ramp where knowledge is dismissed and it is recycled etc. This is the way and then I am going to give you a more detailed answer about all these little pieces. Today, I am going to talk primarily on some of these that enables, that knowledge web presides over or is more directly responsible. The technology, a rack of systems, there are people to contribute knowledge, to store knowledge, to retrieve knowledge, to access knowledge and the process by which that happens. I have seen in the rearview mirror of my storage of a knowledge manager there are many, many, I can see many, many racks of systems that have been built, magnificent system, that has been built with no consideration whatsoever for the people component, the process component. Why would you do that? If it were part of the culture of the organization, if they were the value proposition, the more I contribute, the more I will be able to take out measurements because there is no barricade to it unless you can measure everything you do within your knowledge environment. The next slide is kind of hard to read. I can spend hours and hours describing the intricacies of architecture. Why don't we start from the bottom-left of the slide? You see that we have several ways of helping our few practitioners contributed their knowledge. Knowledge defined as the work that is delivered to the client in the process of service delivery but also you find the stock leadership, business development material, and internal reflection on the state of the future of the profession - all those things that make me a better Ernst & Young professional than I was yesterday. So we need on our end, and then we have large repositories they are aimed for the practice at large for storing knowledge and also you can see to the left and to the right and in the middle of the slide that we cater to specific communities, subject matter professionals, people who dedicate their professional activity to a specific subject matter community, databases to empower with something ready to go and a community of people who work together or who don't work together but share the same interest. Some communities have the transient character; they were here yesterday but they are not going to be here tomorrow based on the business direction and some communities are more permanent. And then on top of this scattered horizon of several repositories etc. a very robust accessory here. When I talk with my fellow knowledge management professionals, some of them I am often asked, "Do you guys know at Ernst & Young, what do you have? Can you put your finger on your knowledge capital?" and more than one person got startled by my confident answer, "yeah, we know." We know what we have. We know our knowledge collateral, we have x thousand repositories. Each and every one has an owner, I can tell you that is the historical backbone of our knowledge web catalogue. The knowledge web catalogue is the secret sauce because in a sense everything is catalogued there and it fulfills a number of functions and on top of the catalogue we built utilities and tools that allow people to find their way in this maze of databases. Then we have the powerful search engine on top of which there is the top part of the slide, the part that is filled with all those two and three level, search engine to remove from the user horizon. They need to understand, "where do I go? I saw this database yesterday I can't even find it anymore. How do I go?" We have a powerful search engine and we leverage the power of the search engine by building tools and applications on top of it including for instance, community based websites. An agile, nimble and dynamic platform that people on the webcast will hear more of during my piece and your piece. Community home spaces, community-directed, community-oriented and even community-driven websites. So it took a few years - this slide is the summary of around six or seven years worth of work and at the end you know we had what we thought of as one of the most powerful knowledge centric intranets. Starting with the home page, where at the top you see a menu that allows you to navigate to our resources and to our knowledge environment. A large space of that page is dedicated to communication - to establish the connection between the practitioner and the firm and global and local communication, global and local resources and then again the powerful search engines. You can see in the next slides it looks a little bit baroque– I'm making a mess because I'm talking about baroque before talking about renaissance, I promised that this will be a story of success and then renaissance but you know the picture of our search engine will be more than a million documents online at all times - thousands of databases, our knowledge resources indexed that with a very well-built, very profound, four or five level deep taxonomy that allows for-taxonomy was the keyword that I was searching etc. I mentioned about 3 minutes ago about community home spaces, right now we have about a hundred community websites built to offer communities a visual identity, a content identity which is what matters more than a pretty face. A content identity by leveraging a common architecture so that the cost is squashed closed to zero and there is a minimum of real reinvention and there is a maximum of leverage of a consolidated platform.

Jerome Nadel: So simply stated, you were providing a whole set of templates to each of these communities to create their community home space without having to rely on the utilities to access.

Giovanni Piazza: That's correct. Without putting them into a strait-jacket of something that is a one-size fit. So, right here I have described the entire knowledge architecture and the systems and the behaviors that supported it. So I was almost ready to get worried because now my team and I have worked ourselves out of a job, what are we going to do next? Then as usual what always happens-reality kicked in. I was told a couple of times that it takes some guts in showing this slide. The slide that shows two curves, one which is the objective curve or the increase of the 'k' web functional value, release after release after release and the other one that shows how instead the user satisfaction in front of the system peaks at some point and then ends and so there is this apparent paradox between an objective increase in functionality, in functional richness and functional value and the decreasing user satisfaction.

Jerome Nadel: And utilization.

Giovanni Piazza: And utilization of a resource. Now, that was good news in the sense so apparently I have not worked myself out of the job.

Jerome Nadel: No.

Giovanni Piazza: In all other fronts it is a situation that requires some profound thinking and some serious action. So the first thing that we did was to go to our users and then identify the symptoms of the disease and what we heard moved across two axes. The first axis was "There is too much. I don't know where to go? How do I find?" The second axis was "Oh! I didn't I know we had it." When people were asking me can I have a search engine I would show them the search engine and then they would say, "Oh I didn't know we had it!" or "Can we (in home space) I didn't know we had it!" or the catalogue and the powerful possibilities to interrogate again, "I didn't know we had it and now I can find it." So at the intersection of these two axes, we identified our problem not as lack of functionality, not as lack of functional value but as lack of usability. The knowledge web that we had been building for years and that had delivered astounding success to our firm was now beginning to suffer from poor usability. So that was the symptom. So what did we do to react to this symptom? Well, we found an advisor and established a partnership but then we switched our own mindsets and we learnt some lessons because after a year at the forefront at the bleeding edge of things having to re-discuss yourself and your capabilities and your approach to things was part of a if you will, a cultural rejuvenation. I don't want to use too big a term but we had to start from our own roots, start challenging our own selves and our own positions in front of these things and so when we started to be user driven, no matter what we would do next we decided we are going to ask real users. Maybe we're going to re-engineer our system or maybe we're going to re-design or maybe we're going to do something else but no matter what we do we are going to ask real users first. Then what we are going to do we are going to ask them again what we are doing. So we moved away from a hypothetical paradox and tell me what your requirements are then let me go away I'll do something for six months and I'll give it back to you and you are going to like it. So we decided to check ourselves to check ourselves as we were doing things and ask the real users whether they were liking what they were seeing or not. Now we decided not to deploy things that we knew users did not like which is different from "oh, I'm going to do exactly what you want exactly, what you say", there is an important thing here. It was the theme of the beginning of my thought that was "knowledge ware is to support the business drive of the firm." So it's not viewing for viewing's sake. So there is an important distinction between do what they ask you to do and make sure that whatever you deploy is met favorably by the users and we will come back, come back to this later on because as we said usability is a holistic mean to a higher end which is user's satisfaction. A user satisfaction is measured by how well 'k' web helps them achieve their business purpose so there has to be a business purpose upon which the knowledge sharing culture is modeled to allow our practitioners to deliver outstanding value and therefore be satisfied with the experience. So this red, thread, this red business thread is interwoven in our canvas will come back more and more and more. I'm afraid I've been talking too much which is something that I tend to do often so I'll be quiet for a second and let you take over from here, Jerome.

Jerome Nadel: Superb Giovanni. So it was interesting to the beginning of our relationship, in fact you had this epiphany, this revelation through your continuous measurement and said "we have an issue here" and put down a formal article saying that we are looking for a help to assess and better diagnose our problem and we came back to you and said "well we can do more than that, we could collaborate on an assessment to identify issues but why don't we look for redesign solution?" So this slide speaks to the six core activities that went into the engagement that we collaborated on and at last at some level is a table of contents for a brief description of the process we went through, some of the key findings and most importantly, how we were able to go through this informed and validated approach to affect and influence your executive level colleagues to implement change. So with that we began with the structural review and I will be elaborating on that review and some of the findings that came out with that in a bit. Then went out and again, appreciate that this is a 140-countries rich, a 100,000 employee organization so when we went out to do data together and we needed to hit multiple entities of multiple countries. From what we learnt we proposed some redesigns. That redesign was then validated through objective end-user testing and based on what we learned in testing led to refinement which led to the final design that we recommended through this engagement.

Giovanni Piazza: And that was a good beginning of our partnership because we asked you to do something and your answer was "No! I want to do more than that" and I said, "Oh, okay!"

Jerome Nadel: Modifying the initial assignment.

Giovanni Piazza: Yeah go ahead.

Jerome Nadel: The next slide basically takes these six activities and divides it into three simple categories and the first is we think is that it is important for the organization to understand what their success criteria is. So the first phase was very much about establishing success criteria and better understanding of users and you had asked for that in the initial assessment. We followed that however with some provocative redesign that looked at more than just the search utility and the knowledge management but an overall integrated architecture for the intranet because this is the core tool that has been used within your firewall and then the last phase was documentation of the design to allow that to be built up more completely. Moving on we look at the first engagement the first component of the engagement which was the expert review and immediately there was a bit of an adversarial-provocative dynamic in our relationship when we said "What you have done is phenomenal, you have a remarkably complex sophisticated system. You have this taxonomical engine and catalogue that enables you to slice and dice your capital." It's like intellectual capital in many ways however if we look fundamentally at what an intranet is, an intranet is divided into two halves - that of content - that of tools and if we look at tools that is further subdivided into core applications and utilities and we humbly came back to you and said that at the very beginning the 'k' web almost axed like it is the intranet and it is a core valuable component but it is a transient utility that should be available in context of what I am doing but shouldn't be the major thing that I come in to do but it should be there for me and almost be transparent.

Giovanni Piazza: And there were some fair amount of debate and some push back on our part, "How dare you call 'k' web a utility?" etc; but then during the redesign and I don't want to steer your thought on that but during the redesign the differentiation became and let's go on with that.

Jerome Nadel: Because immediately again in the intent over here for the viewing audience is to see at the very beginning within the expert review - be provocative, explore architectural issues beyond the interface. So in this slide we talk about migrating to an integrated intranet structure, talking about the idea of the coexistence between EY home who is the communications led component of the intranet and the CBK, the center for business knowledge which is your baby. We then go on and talk about it again - this transient nature of the search utility that this should be context to all such that we talked about the CHS people or the community home space. If I am a practitioner in Sydney, Australia during a certain type of auditing, I should be able to immediately point two things that are relevant to me and not start at the top of the universe asking me to look up something. So we started a structure of framework suggesting that we could look at more global architecture that would be more integrated. We move on to the next phase which was about the user-centered analysis and this is something that you had requested and I think that what is remarkably impressive here is that we were able through this collaborated engagement to touch 220 users in 8 countries around the world. So this was a tagged team and we should know that you have a usability group so we were supplementing the apparatus using some of our refined methodologies to enable us to be in Australia, Canada, France, Germany, Netherlands, Switzerland, UK, US etc. With that, let's move on to the next slide. We needed to come back and boldly and objectively share some anecdotes of what's wrong and this was a little bit to digest at first and you were very willing to receive this feedback and I remember that we went to a meeting with one of your senior level colleagues, sharing this upfront and there was a bit of a silence in the room when we were giving the feedback from the close buddies who were at the daily gathering and when we came back with this, "Less is more, you have a very sophisticated system but it's being underutilized. People as you stated before, are unaware of what they can do, they are overwhelmed. It is inconsistent. They need training to use it effectively." So we started establishing some hypotheses of what would make it better in redesign and that led us to the initial prototype concept. We won't spend much time talking about the details of design but we needed to go from the top of the intranet down in detail from homepage and global navigation down to area landing pages, moving on into this constructive community home space and then more significantly focusing on search and how we can take the multiple search utilities and elegantly integrate them into a single container that would point contextually to give me what I am looking for. So at some level we were taking the visibility of all of the great technology and putting it behind the hood and focusing on just "give me the results, I don't need too many ways to do the things, make it simple, less is more." User centered design is informed and validated. So the story here is we had come up with some prototype concept design. The next step was to validate the assumptions that we had in that design and compare it in a performance, quantitative and qualitative way against what existed already.

Giovanni Piazza: That's the "ask again, while you are doing things" part of your way.

Jerome Nadel: Absolutely and here again through a collaborative exercise, we went out around the world again and I should know that we were able to deploy the field studies which were global. The usability testing which was global, each took a period of two weeks so we were doing these not sequentially, we were doing them in a parallel around the world.

Giovanni Piazza: See that's what you do to me when you make me remember those weeks around Christmas. This happened around the Christmas time and that was an interesting Christmas and as testified by my reaction while I think about that period.

Jerome Nadel: But the news was good because we were moving in the right direction and so this just highlights on this slide here. There were set up locations for this usability testing and we were focusing again on performance testing of the existing 'k' web set of utilities but as well on the next slide we were focusing on a comparison with the prototypes that were evolving as we were going through this process. It was an iterative change through this aggressive time line and here we focused on things like information architecture. One of the anecdotes that you shared and I repeated is that people couldn't find things, they didn't know what was there and that typically speaks to information architecture. "Where are things stored? And if I'm in a browse mode I should be able to find that I shouldn't miss several that we had to search. It should be where I expected it to be." So in this type of test referred to as 'reversed card sort' we were effectively asking participants in the usability test, where would you go to find and we compared the existing design against the new design and what we were looking for is, you see in the color indication over here we're looking for all greens so when we looked at the prototype, we had several of these cells all filled green and while we saw that for a subset of them for example, an engagement document, we were still a 52% which led to further iteration and the key take away that we will come to and as you pose in the next portion of the presentation was we had the data to justify the new design and to push that change to influence those that needed to make the decisions, "should we move forward or not?" Now we move to validated design so we have been through the process of establishing the functional and end user requirements. We have come up with some redesign hypotheses, we tested those and now we had what was moving towards refined design and one of the things I know you will repeat, the previous global navigation was remarkably complex, multi-layered, multiple options and we were moving to a slim and trim simplified approach that had been validated in the usability testing. Furthermore we were advocating the notion of contextual persistent ubiquitous search. Based on where I am when I query or when I just look for something it is immediately available in a standardized location across the intranet. Much of knowledge management is about search. So we talk here about search simplifies complex filter selection. So we had internally described this search utility this design as the isosceles triangle and we talked about the two sides it's an area where I'm building a search identifying a set of filters against the keyword or the alternative of "you have pre-packaged something for me and when I get that I see the result immediately and then can further refine." So on this slide here we focus on the framework of "What are the steps that enable me to simply build the filters that go into my query?" The next slide and last slide for my section here talks about the idea of coming straight to a results set. Now you have pre-packaged this and here we are looking at the surveillance data from Oxley there were a set of filters applied and I have the results list. I can within the utility further refine the results list that I've gotten and you went for a multiple search approaches into a unified approach. So in conclusion, this user centered design process provided design direction but more significantly, buy-in within the executive management at Ernst & Young. Just by way of anecdote, I remember you inviting me down to Mexico City to present it to your peers at a summit of chief knowledge officers and that might sound bizarre that there would be more than one chief knowledge officer but remember again that this is essentially a federation, a constellation of individual member firms. So there are more than one of each of this type of executives and to get agreement can be a bit of a challenge. We shared the story, the objective data driven story of how we came up with this design the rationale for that design and that led to buy-in with the reality of how do we implement in short term on the platform that we already have with a migratory path to where we want to be ultimately. So it's really about effecting change and influence and with that, you can speak to the audience.

Giovanni Piazza: Yes, well this is very really like an ad which when I'm involved is never true but really like an ad because basically we had a diagnosis that was supported by numbers. We had then a remedy which was a prototype that was validated or not validated by numbers, measures. Then we had a final design that was corroborated by numbers again and then we pursued that essential ingredient of every design specially the user centered design by gaining based on numbers and facts, exactly sponsors, and I remember that when I pursued the determining point when my boss looked at us and he said "I understand. It's not what I think, it's not what you think it's what they-they means the end users think and if we are to switch to this now" and he was instrumental in gaining even more senior executive sponsorship executives. So now it is right time because this happened in the past eighteen months and then we built based on all these design and all these studies and we delivered. I'll keep the brag pipe to the minimum because the printer that is printing out all the questions from our colleagues is literally smoking so I want to leave as much time as possible to Q&A and then you know I was taught as a child that bragging is not really polite and it's not really the good thing to do but here is what we did. We deployed a more integrally, we developed and deployed a more integrated intranet with an entry point with few bytes and chock full with information. We replaced a navigation menu with a new navigation menu made of words and entries that users told us they would understand.

Jerome Nadel: That they demonstrated that they would understand.

Giovanni Piazza: They demonstrated that they would understand. They told us to get rid of the corporate mumbo jumbo, to use words that meant something for them and we tested each and every word with end users.

Jerome Nadel: It's a testimonial to the fact that less can be more.

Giovanni Piazza: Yeah, then we replaced our search engine that occupied several pages of different options search in taxonomy, search only these data bases, search with a minimalistic search box that caused an epiphany for me as well because the old system, I came to the realization was user-agnostic. He would not, if we had hundred functions to offer they were offered in the same unbiased way thus leaving the user with the task of selecting among these hundred things that I can possibly do, which one is the one that I needed today and being user-agnostic maybe respectful in theory but in practice misses the point, because then our users told us in no uncertain terms that 99% of the times they wanted to do one of these things. Look for people information, look for company profiles for my client, my prospect, these are some. I want to know about policies because you know very well what kind of a regulated environment our profession is developing nowadays and so has been the conscientious organization that we are and having quality in everything that we do, as our tagline, we always want to make sure that we are adhering to our own applicable policies. So I want to know about policies, I want to know about my intranet so we became we abandoned our user agnosticism and we isolated the search box, those things that people were more interested in most often. We simplified the result page, before the result page was trying to provide too much information, we were hype and happy and that's your expression Jerome and so I go and present only what we need to present and most of all in architecture you alluded to it – layers. I'm in the global space, I'm in my community space and now in a topic of idiomatic space, so everything I do revolves around this theme and if I don't need to start 30,000 feet high, I can start from my community and find the same set of utilities, find the same set of application, find the same set of clinical resources but specific to the place where I am now. So there isn't the cognitive dissonance that there used to be where every cycle was different, every place was something new to learn, hundred functions needed to be learnt and re-learnt one by one. Now there is a more unified approach and identity, function as well as visual. So how did it go because we went live in December and how did it go? Well? Positive feedback outpaced the negative feedback by a 3:1 margin and there was a user who said this is a thousand times more useful and another one said that it is ten thousand times more useful, I'm living in the expectation of finding somebody who is going to tell me hundred thousand times better than before and the keywords were all there. Everything I need appears to be on one page, it is so much easier to navigate. You see all their heartwarming praise was not what we got for positive feedback, well that happens, but it was the positive thing that in those areas we had identified and we had worked on as the critical high areas. So easy to find what you are looking for and in fact you know the bottom line of our effort was "help people find friends"

Jerome Nadel: Help them do their jobs.

Giovanni Piazza: Help them do their jobs. The usage in comparing our lab reports in December 2004 with the old system in December 2005 with the new system, the traffic on E&Y homepage shot up by 40% and obviously there is a novelty factor, so I looked at the January number, the numbers I had a peek at the raw numbers that we had just finished collecting for the month of February and including the fact that this is a period of high demand for knowledge because large parts of our practice are now what we call the busy season so this is a very favorable period of the year because people are very busy and they need a lot of stuff but the numbers confirm that this appears to be a sustained trend, more usage, higher utilization etc. So about 70% of the comments we got are either positive questions and requests – "Can I have my link there?" which is the best success indicator and for an information and knowledge provider who wants in and I want in. We are a very colorful and you know, I don't want to say opinionated but I can't find another term so I use this term. So there we have a healthy 10% of people who offered negative questions and issues, well always in terms of "why don't you do this to improve?" "Why don't you do that to improve?" So here is where we are right now and we just took a turn and this is by no means the end of the journey, we have several releases, incremental releases in the hub where we keep delivering, hopefully delivering value along the lines of what we started, we've got confirmation from our users that we are definitely on the right track and we are in a lot of things. We learn that usability is not lipstick on a or a paint to use , in American usage that I got to learn, okay it's not just makeup, it's not just you know taking something that is ugly and put some lipstick in it and hope that it gets better. It has to be a part of the fabric of your system development and it has to be part of the way, a critical part of the way you go about assessing what the user needs are and what the system characteristics have to be for your system. So that's why I am saying that you know usability grows under the system scheme. It is part of the way you generate the system from the conceptual stages, always remember that usability is never a science, it's often an art, sometimes it's black magic because you know, I just want to give you an example – we had in our email template, we had two buttons one that said "reply to all" and the other one that said "reply with attachments" that means that if you clicked on reply to all, your reply would go without attachments. Then something happened in our main template as two buttons – one that says reply to all and the other one that says reply without attachments, so people keep using the default and now I get many more attachments than I used before. Now this is a specific case where rationality, where usability defines its own rationale. Or there are other cases where things are seen logical, you know the logical and rationale presentation of a screen with a set of functions doesn't click in users because the eye doesn't go where it is supposed to go so I learnt not to try and reconcile usability with a set of correct propositions if I do A, B, C, D the result is going to be E because at times the result is a lie meaning that the users look at me and say why did you do this? It wasn't good. So definitely Jerome and I really want to go to the questions so I close here at my remarks by closing the loop and by repeating with different words the same statement that I started from. K web is not, is not a system that wants to be admired. Knowledge ware in our intranet we never think we do to support our client serving professional is something that has to tie in with, tie into our business processes and our way of adding value to the business. So we need business decisions that drive user centered design. It is this sentence of the end of the last slide where I want to present and I always like to read it backwards, where we need the business decisions that need to validate the principles of the underlying user centered design. And now I'm really, I'm really anxious to see what the questions are?

Jerome Nadel: That is in fact the core theme of this webcast. It's that user centered design has value in the outcome of usable systems but it is an effective mechanism to make logical, rationale business decisions and that's an appropriate sideway to our first question. Again I encourage you to continue submitting questions and we will answer as many as we can in the time remaining - first question.

QUESTIONS AND ANSWERS

Q1: How do you ensure that your leadership continues to invest in usability? It is an expensive business.

Jerome Nadel: Well, we can speak to the risk to not, well I'll let you take that and I'd like to comment as well, Giovanni.

Giovanni Piazza: Sure there is a joke here, if you think giving an advocation is expensive, try with ignorance right? So what is that at the end of the day if you look at the bottom line, what is more expensive? The investment in a project like ours where we talked with 220 people in 8 cultures etc., or well a project like that generates a successful system or a system that is developed in a vacuum and it is going to be you know like a ball of chain around your ankle for years and does not deliver value. How much money gets misspent or not earned by forcing users to interact with a system that doesn't deliver the entire value and calculating the return on investment is really easy. You look at the traffic before you look be, you look at the traffic after, you can quantify easily how much value you gained for every minute you spend on the intranet if you find what you were looking for and then you may have to come to the billing and if you compare it with the cost of the project I mean it's really, really, it's obvious.

Jerome Nadel: So we often speak to establishing key performance indicators, the idea of demonstrating the return on investment but as well the art of storytelling this process tells a good story. We were able to demonstrate a data drive's design. We identified issues in a purical way. We came up with solutions to those issues and then validated those solutions with end users testing and this was something that resonated well with your executive colleagues and they seem to have embraced that and say "Okay, there's a business case. Let's move on." So it's not a luxury of detail design, it is really a focus on doing business more effectively. And I think that's a shift in the field of usability where usability becomes the strategic imperative not just something that says let's make sure our end users like it - something that has core bottom line business value. The other side on our second question over here somebody asks

Q2: How can you make time for user testing against a mandated aggressive deadline?

Jerome Nadel: So the classical challenge that we are going to deploy on x date, how are we going to do usability within our constrained time line and again one could say in the same way how can you not? So I think again we could speak to the timeline for this exhaustive set of activities that we have done. This was an enormous redesign but taking something to deployment without validating it or building something without having adequate requirements is going to lead to a rebuild. So we often talk about the cost to deploy without really looking at the total cost of ownership and if we look at TCL we say it is not just the first deployment but its post deployment of training and rebuild etc., and that's again more than executive perspective, over to you, Giovanni.

Giovanni Piazza: I mean this is you know as a system architect, I've been a system guy for my entire professional life and I've been a production system guy you know I've never developed anything than wouldn't work for less than 5000 people and I want people to say "Why do you need to do user testing or why do I need to do testing? Why do I need to do coding? Can I skip some coding?" Well no, "But why not?" because the system wouldn't work. Well if I put out a system that technically works and its totally unusable does that fall into the category of working systems? It doesn't deliver in the way of business value, this is typically something that needs to be solved and tackled at the leadership level.

Jerome Nadel: Absolutely so that's and we have the luxury, this is part of our success year as we came in through you and you were immediately very willing to engage in some of this intellectual and strategic debate.

Giovanni Piazza: You are over estimating my role but on a more tactical level there are other things that can be done because user testing often times is a time saving technique. By user testing, I'm using the same words that were in the question by assuming that they said user centered design is the real meaning. User centered design often times help you zero in on those three things, the need to change, to deliver, the 20% that needs to change to deliver a 80% of the business value thus allowing a skilled release manager to cut on those activities and functions that otherwise would be developed just in case. These are not one of my metaphors, "Stop fishing with the net, start fishing with the fork" and try and find you know what is the 20% that delivers 80% of the value?

Jerome Nadel: Well stated, now the next question.

Q3: Do you have any quick tips for influencing business decisions regarding funding usability for corporate internet and intranet sites which are currently considered non-strategic projects, when the company's business is usable websites and software?

Jerome Nadel: So distilling that down again and again, quick tips to influence business decisions regarding funding for projects that are considered non-strategic. This really-one needs to step back to answer this question and suggest that usability should not be an afterthought it should be built in to how everything is done. So we often speak to the institutionalization of usability and having standard methodology that enables you to build usability and do everything that is done. So the notion of picking one project over another to apply usability to, really moves away from that core framework of just having some simple methods first to establish what are our success criteria, what are the key performance indicators that's built to those, if you don't know where you are going, don't run and that makes almost every project somewhat strategic.

Giovanni Piazza: And then you know as we all lived in a corporate environment if a project is not considered strategic at the leadership level, usability or any other visibly structured or structure design, object-oriented, those things are not going to change the perception. The definition of a project as a strategic project is that it resides with the business, with the identification of its business value and once the business value of a system is identified and is bought into, then everything flows. We need to be very, very careful. I know you agree with me at not selling usability like the super bowl of the sorts. Usability is the needs to an end as we said before; the end is to achieve the business values.

Jerome Nadel: There are some very good questions here and the next one is-

Q4: How do you quantify the value created from a knowledge solution? Is it possible to really determine ROI or a k-web or an intranet?

Giovanni Piazza: Yeah, yeah you need to. First of all, you need to identify the problem that you want to solve, right? If you don't identify the problem you want to solve, you won't be able to tell whether you solved it or not. There is a much larger version of this presentation that goes much more in depth on the problem that we were trying to solve and we heard our users loud and clear – "I can't find it, I'm overwhelmed, I don't have time, I work for 69 days in a row including Saturdays and Sundays and I still need to keep up with my all my requirements as a good corporate citizen, help me find things please." So that we knew - the problem we wanted to solve. We didn't want to change to a different culture, we didn't want to deploy a new strategy, we didn't want our people to smile more, we wanted them - we wanted to help them find things. So you develop the system and then empirically as well as scientifically you can calculate whether people can find things more easily. For instance the average, let's say in a hypothetical organization you have to take this class okay, it's a professional certification class and historically, and you take it online, historically, compliance hovers around 60% and you can gather evidence by people telling "It's not that I don't want to, I don't know where it is I can't find it so I give up and I go home and play with my kids" okay? Now you promote the visibility of that class, now by itself the compliance percentage shoots up by 20% so now you are 80% compliant which means that your risk is cut in half and in this day and age, to be able to articulate the fact that your risk management that you know, that your risk liability went down by 50%, well that's pretty quantified and that's just, but if you don't define the problem you want to solve first you may never know whether you found the solution.

Jerome Nadel: That's core, it's a core theme and it's interesting to note, and it's really not through this web cast here but defining KM or knowledge management my understanding of your world is, this is not a luxury, making a mistake costs a lot of money. So we talk about fault tolerance oriented systems that don't allow users to make error. The way knowledge management is manifested at Ernst & Young is making sure that people are following rules so they don't get into trouble. So this again is not a luxury, this is a way of doing business. It is a perfect example of how KM is really core to the effectiveness of the services organization.

Q5: Do all functional groups contribute knowledge or only some? Does this benefit some more than others?

Giovanni Piazza: Well I've learned, an old boss of mine taught me that people behave as their thought and they persist as they are rewarded and this goes into the culture, the cultural part of knowledge management. An organization that really intends to pursue knowledge management needs to be serious in terms of sending the message that you are rewarded if you contribute knowledge, you are rewarded if you share knowledge, you are rewarded if you are able to demonstrate that in x hours you have been able to deliver 3 times as much as before not because you worked harder but because you worked smarter and you could leverage the deliverables that other people created. The chilling is good I think reusing is good if the firm or any organization can send this message and say I am going to reward you based on how smartly you contributed and pooled the knowledge then you see that all groups contribute, if you don't do it , no groups contribute and in the 10 years of history of Ernst & Young that I have had the pleasure and privilege of looking at personally, I can tell you that every time this culture is established in a group the spontaneous motion of that group toward knowledge goes up, every time the reward system is too little or something else that they shouldn't pay, they do everything else including knowledge goes down.

Jerome Nadel: Classical alternative psychology, people do what they are rewarded for.

Q6: Do you have any quick tips for redesigning a corporate intranet site for global parenthetical international usability?

Jerome Nadel: Here's a corporate case of why HFI works with many multinational corporations and one of the things that I very much appreciate about this engagement is you upfront with - without our provocation said we need to make sure they were sensitive at a field study level so the requirements of the various constituencies within our constellation. So with that a couple of things one, make sure you understand the divergent requirements, the unique requirements from the various regions or entities. Secondly build an infrastructure whereby you have inclusion and one of the things you have done at Ernst & Young is your usability group is in fact, distributed. You have people out in various regions who are members not at line perhaps, but members of this usability initiative. Let them be in because again, the core theme as assured at the beginning of this presentation was "Influence". User centered design is informed and validated. It enables you to tell a story that can influence and effect change. Let the delegates out in their constituency be represented when you are making change, be a purical, be data-driven, collect data from the various entities.

Giovanni Piazza: And avoid the good-hearted colonialism, don't send an American expert to the UK or an UK expert to Australia beside the fact that it doesn't make sense economically and financially. Experts, who share the same cultural identity as the subjects of the investigation can reconcile the peculiarities of each culture, opt to a lowest common denominator that applies globally.

Jerome Nadel: And from a perspective of efficiency, you were very impressed with our approach of being able to quickly capture and consolidate all the data that we had obtained and with that we have standard tools and templates that enable different practitioners to do these field studies or usability tests and bring them back in a standard unified format so having an approach in tools and technique enables you to be more efficient. We are running short over time over here, so we'll take another set of questions, last batch of questions.

Q7: Do you think that user centered design gives all of the stake holders, users, executives, technical guys a common language to share and I presume this is move forward, it says "more forward"? If yes, provide some examples.

Giovanni Piazza: If yes, provide some examples?

Jerome Nadel: That's a two part question.

Giovanni Piazza: Do I get a grade at the end?

Jerome Nadel: Do you think that, it gives all of the stake holders, users, executives, technical folks a common language to share and move forward.

Giovanni Piazza: Yes.

Jerome Nadel: Yes.

Giovanni Piazza: Yes.

Jerome Nadel: If there is a...

Giovanni Piazza: As a matter of fact it's a profound unifying fact that if everybody buys it into the paradigm of user centered design then you don't have a business guy like me who says, "Well, I know what my users need, you just listen to me. You don't have the techno guy you know, like my best friends those with whom we generate this system say let's do this because it is technically elegant but we always stay focused on what my boss or my boss's mantra, it's not what you think, it's not what I think, it's what they think." And we can tag team and remind each other it's not what you think, it's not what I think so that creates, that creates the common language

Jerome Nadel: Which is a requirement and one of the things we often do in the beginning of a large engagement is suggest that there be an overview course on user centered design to get the core teams speaking in the same language. So within that framework, building this standardized approach and sharing that openly.

Q8: How do we solicit the feedback? Is it solicited from people who use the system for example, "rate this site" information? I know that you spent a lot of mechanisms around that.

Giovanni Piazza: Every – well, specifically asking people to rate this site for information once they are already there is good to rate the experience but not to rate the overall effectiveness because you will never get the opinion of the people who didn't get there or got there and left disgusted or got there and jumped to somewhere else. Every month we issue 27 different reports that analyze all the user behaviors you know, in pieces and parts of K web okay, on the intranet. So we have a report that tells us how many people went there and didn't do anything else. Then the click through, how many people went to the part of let's say to the community home space and did something from there, and how many people landed? We have some index the attraction versus affection, how many people go to the site and how many people return attracted and then they develop the affection. We have traffic measurements so users, unique users visits, recurring visits etc.

Jerome Nadel: I know that the data that you shared this very qualitative, was very subjective feedback oriented but you just highlighted that the data that you're capturing and the data we had - the data we captured, is not just subjective "I like" preferential type feedback, it needs to be what is being used, application logs, server logs, looking at traffic, looking at repeat visit, that enables you to better understand what is being used and if you begin by establishing key performance indicators, this is what we want to have happen and measure against that falls in line with the philosophy of knowledge management that you have to have measured.

Giovanni Piazza: And then there is the business handshake between qualitative and quantitative, okay? We put out this brand new site let's say, or this brand new resource that is key to the business and quantitatively we see the traffic is wrong. Okay let's isolate the sample and let's start with some high touch qualitative research and so we understand why you know, using the more traditional qualitative research techniques, we understand why the traffic on that site is not what it is supposed to be and then we can fix it you know, again with the new iteration of user centered design.

Jerome Nadel: So we begin with questions and again I encourage you to continue submitting questions as we have got some time yet to go through those so this first one is for you, Giovanni.

Q9: How did you determine usability was the root cause of declining user satisfaction at K Web, did you consider other approaches to fixing it besides user centered design?

Giovanni Piazza: We knew - we realized that we had the usability problem and not a functional problem because people asked for, kept asking for things that were there. So I want a search engine that allows me to do x y and z, sure here you have it. I want a set of utilities that allow my community to get up to speed with a website quickly, there you have it. The problem was the spatial problem. "I can't find it. I don't know where it is and I don't know" that's part of the answer. The other part of the answer is that usability is not a box so when I took 7 years of layers of search engine, 8 years and 6 releases and I peeled the onion back to the core of what people really want out of a search engine. Yeah, I was user driven, I was user centered, and the design was user centered by then it ended up, you know, this user centricity resulted in the profound rethinking of the function itself. So it is one of those cases where usability, as I said is not lipstick on a pig, right? Because if you stay close to the user and you are user centered then function edifications happen consequently.

Jerome Nadel: Well done. I guess we can both take this next question; again I encourage you to continue submitting.

Q10: We're coming in, we're working at an enterprise re-design of our corporate intranet but deadlines won't allow for prototyping. How unfortunate! What is the best way to incorporate user feedback in this time constraint?

So I will take a minute to answer that and you can speak to them within the context of Ernst & Young. Looking at a re-design, there must be some motivation behind that and then looking at how you can bypass some level of prototyping and getting feedback on that. The first thing that we advocate is really an understanding of what is the organization looking for? What are the key performance indicators? What are the success criteria? I feel often we begin at the end-user level and don't start at the executive level, asking if we were in an optimal world what would this utility provide. How does it affect our business in terms of enhanced revenue, in terms of reducing cost? So we begin by establishing core success criteria and then I would suggest, much as we have done here, perform some structured review of what exists against those success criteria? I would strongly encourage that you have some usage data to see what is being used and what is not and suggest that the approach of prototyping does not have to be long and drawn-out. Perhaps we should have spoken to timelines within this engagement. Again, we had a lot of data. We spoke to 220 participants in both usability testing and in the up-front user data gathering. We did those data gathering sessions around the world and usability testing in two-week time lots. So when we look at the complete duration of this re-design, we were around 8-10 weeks.

Giovanni Piazza: Yeah.

Jerome Nadel: In total. Don't deploy before you evaluate some of the concepts and don't just waste time throwing out concepts that are not based on an informed foundation of success criteria and usage data. So I would suggest that the prototyping does not need to be exhaustive but challenge what you have with some limited concept design and quickly validate that - practical, pragmatic and efficient.

Giovanni Piazza: Yeah. What we did at our end was re-negotiate, in a case like this, I would as I did, re-negotiate another deadline but the time slice, say, "Look, if we do prototyping, we can be much more focused and understand what needs redevelopment and therefore the development curve can grow-can shrink because we will know what gives us the best bang for the buck." Then there is also the other question, okay, deadlines won't allow for prototyping. I'm not known for being the most considerate or diplomatic person in the world and so I think I have little or no compunction in going in front of some stakeholder and saying, "Look, what deadline do you want this one to be? This system is going to be with us for years. Would you rather have me deploy by the deadline something that is going to be unsuccessful for a few more years? Or would you like me to take a couple more months and, or a couple more weeks depending on the size of the project, and delivering something that will be a success? Answer this question; I'll do as you say." You know what; you put the leadership a little bit on the spot by saying, "I will do what you tell me to do but you have to tell me." And it's surprising how many deadlines are re-negotiable or how many work packages can be shifted and shrunk and expanded within the same overall deadline.

Jerome Nadel: Well said.

Giovanni Piazza: I am not inviting anybody to risk to be fired, okay, or to take a hard position in front of the leadership.

Jerome Nadel: But it comes back to the core theme of this broadcast and that is that user centered design is about informed and validated process, it's about influencing through objective data and doing that in a practical and efficient way. So with that, in advance of our collaboration and following, you have a usability group and we have another question for you.

Q11: Have other departments at Ernst & Young bought into user centered design or is it the primary practice of the knowledge management group? And then as well, how big is Ernst & Young's internal usability group?

Giovanni Piazza: Yeah, two questions, two questions that are strictly related. I think that we were trailblazers and now we, the knowledge world at Ernst & Young, have become thought leaders because I have seen growing interest in user centered design not always, I have to say it, not always to the extent and the pervasiveness of what we did with K-Web but in a textual part growing into the usability culture. So a lot of our other initiatives have been growing and our internal usability team which is now between 4 and 6 FTEs but in order to infuse usability in everything we do, we decided not to create a vertical silo but to spread the usability competency across as many people as I could and each one of those people dedicates part of his or her work life to usability projects and so we have this nice, it's between 4 and 6 people but I were to, if the leader of my usability team were here, he would say, "The answer to the question – "How big is Ernst & Young internal usability group?" will be it's too small and I keep asking you for 2,3,4,5 more people and you keep saying no."

Jerome Nadel: Well, that's the reality of challenging and we have collaborated on other initiatives as well and the model has very much been, "What is the process? How do you get by on it?" and I would say that this has been a sort of avalanche that this approach, especially with senior level executives, has acknowledged that this is a rational way of making decisions. Final question over here and we have some coming in actually.

Q12: Is the task-centric usability testing enough? What about user satisfaction?

I think you should begin on this piece because you spoke about it as a core component of KM, the idea of continual measurement.

Giovanni Piazza: Yeah, I've been a systems guy for my entire life and I have been a production system guy for my entire life so I can confidently say that people are not happy or unhappy, satisfied or dissatisfied by a system. I did marvelous systems that did nothing for people and I did crappy systems that you know, were good for people because people are satisfied or dissatisfied by the ability of the system to support them in whatever they do. You've heard me, you know, cursing some of these computer things that in cars that that have a mouse that you have to move and you're driving at 120 kilometers per hour and you have a mouse – that's a beautiful system but my user satisfaction when I have to try and not to kill you know, whoever is crossing the street in front of me, is very low because user satisfaction is not a function of the system. It's a function of how does the system let me do what I was trying to do and I want to do that and if the system allows me to do that, that is perform my task, then I am going to be happy.

Jerome Nadel: Which really underlines the approach, once again, of user centered design. If I look at something in a static way, I'm not driving – through your analogy of a car – so you need to let people to perform on the system. Their successful performance will affect their satisfaction. So just looking at something statically is really not an adequate assessment of will I be satisfied may be to use it which really brings us to our last question and this is fair.

Q13: The data that you have shared really spoke to the spike following initial deployment, the question is what happened in January or February or down the road?

Giovanni Piazza: Yeah, I have the numbers for January because we publish monthly 27 different usage reports every month about every single little piece of our intranet and you have to understand that January is what we call the "busy season" for large sectors of our practice where the business reaches its peak and people kiss their families bye-bye and disappear for months and then come back around April. So with you know, considering this, which is always obviously creates a spike, another spike in usage, the January data confirmed and supported the trend highlighted in December. We have just finished collecting the raw data for February and I had a peek but I don't have the reports for February yet. I had a peek at the raw data that tell us that in those areas of the firm where busy season is particularly felt, again in usage, again, I don't want to be arrogant or pretentious and say it that it all, it's all due to the new system. A part of it was the environmental circumstances. I'm just happy to be able to say that what I have been able to do for my client-serving colleagues appears to be helping them do what they do for a living which is deliver outstanding, world-class service to our clients.

Jerome Nadel: Which is really the theme of knowledge management as its exercise is interesting of usability. This concludes our broadcast and Giovanni; I want to thank you for your participation in this webcast. I think that once again, is a great story how user-centric design can influence and affect positive change for institutionalized usability. Thank you very much.

Giovanni Piazza: Thank you.

Top