Contextual inquiry

What

The product team unobtrusively observes participants at work, with their permission, then asks questions.

Why

To learn how and why users do what they do; to discover needs and attitudes that might not emerge in an interview to map how tools, digital and otherwise, interact during complex activities.

How to do it

With permission from a supervisor and from the participant, schedule a time to watch a typical work activity and record data.
While observing, ask the participant to act normally. Pretend you’re a student learning how to do the job. Ask questions to help you understand what the person is doing and why.
At the end of the session, explain what you have learned and check for errors.
Immediately after, write up your notes.

Example from 18F

A pair of 18F team members visited two Department of Labor/Wage Hour Division investigators as they interviewed home health care workers who were subject to unpaid overtime and other infractions. Since it was a sensitive subject, the 18F team did not question the health care workers directly, but instead asked the investigators clarifying questions in private. 18F staff also made sure that photos did not include faces.

Additional resources

Observational research: 5 Tips for Improving Your Approach

Applied in government research

No PRA implications, if done properly. Contextual interviews should be non-standardized, conversational, and based on observation. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

For internal folks, get permission from the right level of management. If participants could be under union agreements, contact the agency’s labor relations team.

Design studio

What

An illustration-based way to facilitate communication (and brainstorming) between a project team and stakeholders.

Why

To create a shared understanding and appreciation of design problems confronting the project team.

How to do it

Invite between six and 12 participants: stakeholders, users, and team members who need to build a shared understanding. Before the meeting, share applicable research, user personas (unless users will be present), and the design prompt for the exercise.
Bring drawing materials. At the start of the meeting, review the design prompt and research you shared.
Distribute drawing materials. Ask participants to individually sketch concepts that address the prompt. Remind them that anyone can draw and artistic accuracy is not the goal of the exercise. 15–20 minutes.
Have participants present their ideas to one another in groups of three and solicit critiques.
Ask the groups to create a design that combines the best aspects of members’ individual contributions.
Regroup as a whole. Have each group of three present their ideas to everyone. Discuss.
After the meeting, note areas of consistent agreement or disagreement. Incorporate areas of consensus into design recommendations and areas of contention into a research plan.

Example from 18F

“User-centered design at 18F: a design studio for natural resource revenues” Chris Cairns , Michelle Hertzfeld , Nick Bristow.

Additional resources

Presentation by Todd Zaki Warfel explaining what a design studio is. Todd Zaki Warfel.

Applied in government research

No PRA implications. If conducted with nine or fewer members of the public, the PRA does not apply, 5 CFR 1320.5(c)4. If participants are employees, the PRA does not apply.

Dot voting

What

A simple voting exercise to identify a group’s collective priorities.

Why

To reach a consensus on priorities of subjective, qualitative data with a group of people. This is especially helpful with larger groups of stakeholders and groups with high risk of disagreement.

How to do it

Bring plenty of sticky notes and colored stickers to the meeting.
Gather everyone on the product team and anyone with a stake in the product.
Quickly review the project’s goals and the conclusions of any prior user research.
Ask team members to take five minutes to write important features or user needs on sticky notes. (One feature per sticky note.)
After five minutes, ask participants to put their stickies on a board. If there are many sticky notes, ask participants to put their features next to similar ones. Remove exact duplicates.
Give participants three to five colored stickers and instruct them to place their stickers on features they feel are most important to meeting the project’s goals and user needs.
Identify the features with the largest number of stickers (votes).

Applied in government research

No PRA implications: dot voting falls under “direct observation”, which is explicitly exempt from the PRA, 5 CFR 1320(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

Empathy map

What

Creating a visualization of user thoughts and attitudes to help align the team on a project’s needs and layout

Why

To cultivate a deep understanding of user needs and feelings regarding a product.

How to do it

Create an empathy map split into four quadrants: “Says,” “Thinks,” “Feels,” and “Does.”
Through user research, fill out the map to help visualize profiles.
Use the map to inform your design process.

Additional resources

Traditional Empathy Map

Empathy Map: The First Step in Design Thinking - Sarah Gibbons

Heuristic evaluation

What

A quick way to find common, large usability problems on a website.

Why

To quickly identify common design problems that make websites hard to use without conducting more involved user research.

How to do it

Recruit a group of three to five people familiar with evaluation methods. These people are not necessarily designers, but are familiar with common usability best practices. They are usually not users.
Ask each person to individually create a list of “heuristics” or general usability best practices. Examples of heuristics from Nielsen’s “10 Usability Heuristics for User Interface Design” include:
1. The website should keep users informed about what is going on, through appropriate feedback within reasonable time.
2. The system should speak the user’s language, with words, phrases and concepts familiar to the user, rather than system-oriented terms.
3. Users often choose system functions by mistake and will need a clearly marked “emergency exit” to leave the unwanted state without having to go through an extended dialogue.
Ask each person to evaluate the website against their list and write down possible problems.
After individual evaluations, gather people to discuss what they found and prioritize potential problems.

Additional resources

“10 Usability Heuristics for User Interface Design.” Jakob Nielsen.
“How to Conduct a Heuristic Evaluation.” Jakob Nielsen.

Applied in government research

No PRA Implications, as heuristic evaluations usually include a small number of evaluators. If conducted with nine or fewer members of the public, the PRA does not apply, 5 CFR 1320.5(c)4. If participants are employees, the PRA does not apply. See the methods for Recruiting and Privacy for more tips on taking input from the public.

KJ method

What

A facilitated exercise in which participants list their individual priorities onto cards, collect them as a group, organize them by relationship, and establish group priorities through individual voting.

Why

To reach a consensus on priorities of subjective, qualitative data with a group of people. This is especially helpful with larger groups of stakeholders and groups with high risk of disagreement.

How to do it

Gather four or more participants for 90 minutes. Provide sticky notes and markers.
Create a focused question about the project’s needs and select a facilitator to run the exercise.
Give participants five minutes to write at least three responses to the question, each on its own note.
Give participants 15 minutes to put their answers on the wall, read everyone else’s, and make additions. Have participants cluster similar answers without discussion.
Ask participants to write names for each cluster on their own - this is mandatory. They may also split clusters.
Put each name on the wall by its cluster. Exclude word-for-word duplicates.
Reiterate the question and have each person rank their three most important clusters. Visually tally points.
Combine duplicates and their points if the entire group agrees they’re identical. Three or four groups usually rank higher than the rest - these are the priorities for the question.

Example from 18F

18F conducted this exercise with 20 Federal Election Commission staff members to define priorities around conflicting requests. We used this method to get data from staff (not the decision makers) about what they saw as the most pressing needs. We synthesized and presented the data back to the decision makers.

Additional resources

“The KJ-Technique: A Group Process for Establishing Priorities.” Jared M. Spool.

Applied in government research

At 18F, KJ participants are almost always federal employees. If there is any chance your KJ workshop could include participants who are not federal employees, consult OMB guidance on the Paperwork Reduction Act and the Privacy Act. Your agency’s Office of General Counsel, and perhaps OIRA desk officers, also can ensure you are following the laws and regulations applicable to federal agencies.

Personas

What

Representations of users based off of qualitative and quantitative user research.

Why

To create reliable and realistic representations of your key audience segments for reference

How to do it

Conduct user research: Answer the following questions: Who are your users and why are they using the system? What behaviors, assumptions, and expectations color their view of the system?
Condense the research: Look for themes/characteristics that are specific, relevant, and universal to the system and its users.
Brainstorm: Organize elements into persona groups that represent your target users. Name or classify each group.
Refine: Combine and prioritize the rough personas. Separate them into primary, secondary, and, if necessary, complementary categories. You should have roughly 3-5 personas and their identified characteristics.
Make them realistic: Develop the appropriate descriptions of each personas background, motivations, and expectations. Do not include a lot of personal information. Be relevant and serious; humor is not appropriate.

Additional resources

«««< HEAD Template of a User Persona =======

“Perfecting Your Personas.” Kim Goodwin.
“Inside Goal-Directed Design: A Two-Part Conversation with Alan Cooper.” Caroline Kraus.

&lt;blockquote&gt;
  &lt;blockquote&gt;
    &lt;blockquote&gt;
      &lt;blockquote&gt;
        &lt;blockquote&gt;
          &lt;blockquote&gt;
            &lt;p&gt;73f05b9b7d4c9a741d75156ee8a139cac7afd9d6&lt;/p&gt;
          &lt;/blockquote&gt;
        &lt;/blockquote&gt;
      &lt;/blockquote&gt;
    &lt;/blockquote&gt;
  &lt;/blockquote&gt;
&lt;/blockquote&gt;

How to use Personas -Usability.gov

Project plan

What

A statement of how and when a project’s objectives are to be achieved, by showing the major products, milestones, activities and resources required on the project.

Why

To get an idea of the timeline and goals of a project.

How to do it

Form a project planning group for your team. This team will first determine the objectives and goals of the project, and once work begins, they will manage to see if the plan is being kept to.
To begin making the plan, do an initial assessment of the issues that need to be worked through and the difficulties that may be encountered in the project.
Whenever possible, involve the community to get feedback on their needs, whether through town halls or focus groups.
As the project continues, manage the plan, and make adjustments if necessary.

Stakeholder and user interviews

What

A wide-spanning set of semi-structured interviews with anyone who has an interest in a project’s success, including users.

Why

To build consensus about the problem statement and research objectives.

How to do it

Create a guide for yourself of some topics you’d like to ask about, and some specific questions as a back up. Questions will often concern the individual’s role, the organization, the individuals’ needs, and metrics for success of the project.
Sit down one-on-one with the participant, or two-on-one with a note-taker or joint interviewer, in a focused environment. Introduce yourself. Explain the premise for the interview as far as you can without biasing their responses.
Follow the conversation where the participant takes it. They will focus on their priorities and interests. Be comfortable with silences, which allow the participant to elaborate. To keep from getting entirely off course, use your interview guide to make sure you cover what you need to. Ask lots of “why is that” and “how do you do that” questions.
If there are other products they use or your product doesn’t have constraints imposed by prior work, observe the stakeholders using a competing product and consider a comparative analysis.

Additional resources

“Interview checklist”
“Tips for capturing the best data from user interviews” Ryan Sibley.
“Build empathy with stakeholder interviews, part 1: Preparation.” Andrew Maier.
“Build empathy with stakeholder interviews, part 2: Conversation.” Andrew Maier.

Applied in government research

No PRA implications. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

UX questionnaire

What

An introductory exercise teams can use to get alignment on the scope of the project and to begin planning the user design of its components.

Why

To determine how the product appeals to users, and to assist in finding necessary changes before decisions have been made.

How to do it

Use our UX Questionnaire and get together as a team to talk through the questions, making sure to record your answers where all team members can access them.
As you go through the Discovery phase, refer back to your answers to help inform the design process.

Additional resources

Design hypothesis

What

Framing your work as a hypothesis means no longer just thinking about the thing you’re making or building, but paying more attention to whether that work is achieving your intended goals and outcomes.

Why

When done collaboratively, hypothesis-building is powerful at getting a team on the same page about what it’s doing and why. It also allows the team to be flexible — if one approach doesn’t result in the outcome you expected, you have implicit permission to change course and try something else.

How to do it

As a team, identify and make explicit the problem you’re trying to solve. What goals or needs aren’t being met? What measurable criteria would indicate progress toward those goals?
As a team, write out the hypothesis for the work you want to do to address the problem(s) you’re trying to solve. You may want to write broad hypotheses at the outset of a project and more specific hypotheses each sprint.

Here’s a common way to structure your hypothesis:

We believe that doing/building/creating [this] for [this user] will result in [this outcome]. We’ll know we’re right when we see [this metric/signal].
Identify the main entry points for the user need you’re addressing. This could be external marketing, the homepage, a microsite, or another page.
Build or do the thing, and measure. If you learned something unexpected, then create a new hypothesis and change course so you can continue working toward your goals.

Example from 18F

Example from the Lean product design guide.

Additional resources

Getting out of the deliverables business.

Applied in government research

No PRA implications. No information is collected from members of the public.

Comparative analysis

What

A detailed review of existing experiences provided either by direct competitors or by related agencies or services.

Why

To identify competitors’ solutions that excel, are lacking, or are missing critical design elements. Comparative analysis can give you a competitive edge by identifying opportunities, gaps in other services, and potential design patterns to adopt or avoid.

How to do it

Identify a list of services that would be either direct or related competitors to your service. Pare the list down to four or five.
Establish which criteria or heuristics you will use to evaluate each competing service.
Break down the analysis of each selected competitor into specific focal areas for evaluation. For example, how relevant are search results?
Use a spreadsheet to capture the evaluation and determine how the targeted services and agencies perform based on the identified heuristics.
Present the analysis, which should showcase areas of opportunities that you can take advantage of and design patterns you might adopt or avoid.

Example from 18F

Draft U.S. Web Design Standards Comparative Analysis

Applied in government research

No PRA implications. No information is collected from members of the public.

Journey mapping

What

A visualization of the major interactions shaping a user’s experience of a product or service.

Why

To provide design teams with a bird’s-eye view of a service that helps them see the sequence of interactions that make up a user’s experience including the complexity, successes, pain points, and emotions users experience along the way.

How to do it

Document the elements of the project’s design context. This includes:
- People involved and their related goals
- Their behaviors in pursuit of their goals
- Information, devices, and services that support their behaviors
- Important moments in how they experience a service or major decisions they make
- The emotions associated with these moments or decisions
Visualize the order in which people exhibit behaviors, use information, make decisions, and feel emotions. Group elements into a table of “phases” related to the personal narrative of each persona. Identify where personas share contextual components.
Discuss the map with stakeholders. Point out insights it offers. Use these insights to establish design principles. Think about how to collapse or accelerate a customer’s journey through the various phases. Incorporate this information into the project’s scope.

Additional resources

Adaptive Path’s Guide to Experience Mapping. Adaptive Path (PDF).
An explanation of journey mapping on Wicked Problems Worth Solving. Austin Center for Design.
“Designing Digital Strategies, Part 1: Cartography.” UX Booth.

Applied in government research

Style tiles

What

A design document that contains various fonts, colors, and UI elements that communicate the visual brand direction for a website or application.

Why

To establish a common visual language between the design team and stakeholders. It also acts as a collaboration artifact that both the design team and stakeholders can use to contribute to the final design direction.

How to do it

Gather all the feedback and information that was provided during the initial kickoff of the project.
Distill the information into different directions a solution could take. Label these directions based on what kinds of interactions and brand identity they represent.
Create the appropriate number of style tiles based on the defined directions, which establish the specific visual language for the different directions.
Gather stakeholder feedback. Iterate on the style tiles, eventually getting down to a single style tile which will be the established visual language for the project going forward.

Additional resources

Tool: Style Tiles: A Visual Web Design Process for Clients and the Responsive Web. Style Tiles.
“Style Tiles and How They Work.” Samantha Warren.

Applied in government research

No PRA implications. No information is collected from members of the public.

Task flow analysis

What

A step-by-step analysis of how a user will interact with a system in order to reach a goal. This analysis is documented in a diagram that traces a user’s possible paths through sequences of tasks and decision points in pursuit of their goal. The tasks and decision points should represent steps taken by the user, as well as steps taken by the system.

Why

To validate a design team’s understanding of users’ goals, common scenarios, and tasks, and to illustrate in a solution-agnostic way the overall flow of tasks through which a user progresses to accomplish a goal. Task flow diagrams also help surface obstacles in the way of users achieving their goal.

How to do it

Based on user research, identify target users’ goals that need to be analyzed.
For each goal, identify common scenarios and the tasks and decisions that the user or system will perform in each scenario. Don’t assume you and your stakeholders share the same understanding of the tasks. The idea is to make the flow of tasks explicit in the diagram, so that you can check your understanding by walking through the diagram with users (steps 4 & 5).
Produce a diagram that includes each task and decision point that a user might encounter on their way toward their goal. While there are several diagrammatic languages that can be used to produce task flow diagrams, the basic look is a flow chart of boxes for tasks and decision points and arrows showing directionality and dependencies among tasks. The diagram should cover the common scenarios identified in step 2.
Present the diagram to a subject matter expert who knows the task(s) well enough to check for accuracy.
In collaboration with users and/or subject matter exprts, annotate the task flow diagram to pinpoint areas of interest, risk, or potential frustration.

Additional resources

“Task Analysis: The Key UX Design Step Everyone Skips.” Larry Marine.
Tool: Task Analysis. Usability.gov

Applied in government research

No PRA implications. No information is collected from members of the public.

Task matrix

What

Also known as an user-task matrix, is a simple approach for exposing frequency and importance by user class or persona.

Why

To indicate what tasks are most critical for user experience.

How to do it

Ask the user questions about the importance and frequency about a task.
Gather all the information and seperate each task into a subtask.
Insert each task into a diagram and ensure its complete.
Review the analysis with the team and check if the tasks are consistent.

Additional resources

“The User Task Matrix.” by Chauncey Wilson

“How to improve your UX designs with Task Analysis”

Prototyping

What

A rudimentary version, either static or functional, of something that exhibits realistic form and function.

Why

To enable direct examination of a design concept’s viability with a number of other methods such as usability testing or a cognitive walkthrough. Static prototypes (often paper) are helpful for gaining feedback on users’ intentions and various design elements. Functional prototypes (often coded) are helpful for observing how users interact with the product.

How to do it

Create a rudimentary version of your product. It can be static or functional. Think in the same way you would about a wireframe: demonstrate structure and relationships among different elements, but don’t worry about stylized elements.
Give the prototype to the user and observe their interactions without instruction.
After this observation, ask them to perform a specific task.
Ask clarifying questions about why they do what they do. Let the user’s behavior guide the questions you ask. It can be helpful to have them narrate their thought process as they go along.
Iterate! Prototypes should be quick and painless to create, and even more quick and painless to discard.

Applied in government research

Sketching

What

A team activity wherein each member sketches 6-8 ideas regarding the project at hand within a 5 minute time limit.

Why

To quickly brainstorm and get ideas for a project.

How to do it

Before the meeting, prepare several sheets of paper with a 2×2 or 2×3 grid. Make sure boxes are large enough to sketch an idea, but not so large that multiple ideas can be put into a single box. Try to have space for at least 10 ideas per round.
When the meeting begins, give each player a sheet and explain how the activity works.
Next, set a timer for 5 minutes.
Tell the players to sit silently and sketch out as many ideas as they can until the timer ends — with the goal of reaching 6-8 ideas. The sketches can and should be very rough — nothing polished in this stage.
When the time runs out, the players should share their sketches with the rest of the group. The group can ask questions of each player, but this is not a time for a larger brainstorming session. Make sure every player presents his/her sketches.
With time permitting, repeat another few rounds of 6-8-5. Players can further develop any ideas that were presented by the group as a whole or can sketch new ideas that emerged since the last round. They can continue to work on separate ideas, or begin working on the same idea. But the 5-minute sketching sprint should always be done silently and independently.

Adapted from Gamestorming

Wireframing

What

A simple visual representation of a product or service interface.

Why

To prioritize substance and relationships over decoration as you begin defining the solution. Wireframing also gives designers a great opportunity to start asking developers early questions about feasibility and structure.

How to do it

Build preliminary blueprints that show structure, placement, and hierarchy for your product. Steer clear of font choices, color, or other elements that would distract both the researcher and the reviewer. Lightweight designs are conceptually easier to reconfigure. A few helpful tools for building wireframes are OmniGraffle and Balsamiq, which purposefully keep the wireframe looking like rough sketches.
Use this opportunity to start listing what UX/UI patterns you will need.
Review your wireframes with specific user scenarios and personas in mind. Can users accomplish their task with the wireframe you are sketching out?
Use the wireframes to get the team’s feedback on feasibility and structure.

Applied in government research

No PRA implications. No information is collected from members of the public.

Multivariate testing

What

A test of variations to multiple sections or features of a page to see which combination of variants has the greatest effect. Different from an A/B test, which tests variation to just one section or feature.

Why

To incorporate different contexts, channels, or user types into addressing a user need. Situating a call to action, content section, or feature set differently can help you build a more effective whole solution from a set of partial solutions.

How to do it

Identify the call to action, content section, or feature that needs to be improved to increase conversion rates or user engagement.
Develop a list of possible issues that may be hurting conversion rates or engagement. Specify in advance what you are optimizing for (possibly through metrics definition).
Design several solutions that aim to address the issues listed. Each solution should attempt to address every issue by using a unique combination of variants so each solution can be compared fairly.
Use a web analytics tool that supports multivariate testing, such as Google Website Optimizer or Visual Website Optimizer, to set up the testing environment. Conduct the test for long enough to produce statistically significant results.
Analyze the testing results to determine which solution produced the best conversion or engagement rates. Review the other solutions, as well, to see if there is information worth examining in with future studies.

Additional resources

Applied in government research

No PRA implications. No one asks the users questions, so the PRA does not apply. See the methods for Recruiting and Privacy for more tips on taking input from the public.

Product review

What

A final review determining the status of the delivered product, examining the timeline, issues, and budget in comparison to the original plan, alongside thoughts about necessary updates or changes.

Why

To see how the project plan held up, and any immediate needs for the product, before or after delivery.

How to do it

Form a project planning group for your team. This team will first determine the objectives and goals of the project, and once work begins, they will manage to see if the plan is being kept to.
To begin making the plan, do an initial assessment of the issues that need to be worked through and the difficulties that may be encountered in the project.
Whenever possible, involve the community to get feedback on their needs, whether through town halls or focus groups.
As the project continues, manage the plan, and make adjustments if necessary.

Usability testing

What

Observing users as they attempt to use a product or service while thinking out loud.

Why

To better understand how intuitive the team’s design is, and how adaptable it is to meeting user needs.

How to do it

Pick what you’ll test. Choose something, such as a sketch, prototype, or even a “competitor’s product” that might help users accomplish their goals.
Plan the test. Schedule a research-planning meeting and invite anyone who has an interest in what you’d like to test (using your discretion, of course). Align the group on the scenarios the test will center around, which users should participate (and how you’ll recruit them), and which members of your team will moderate and observe. Prepare a usability test script (example).
Recruit users and inform their consent. Provide a way for potential participants to sign up for the test. Pass along to participants an agreement explaining what participation will entail. Clarify any logistical expectations, such as screen sharing, and pass along links or files of whatever it is you’re testing.
Run the tests. Moderators should verbally confirm with the participant that it’s okay to record the test, ask participants to think outloud, and otherwise remain silent. Observers should contribute to a rolling issues log. Engage your team in a post-interview debrief after each test.
Discuss the results. Schedule a 90-minute collaborative synthesis meeting to discuss issues you observed, and any questions these tests raise concerning user needs. Conclude the meeting by determining how the team will use what it learned in service of future design decisions.

Example from 18F

Additional resources

Example Design Research Participant Agreement (English/Spanish)
Interview checklist
Example usability test script
Rolling issues log

Applied in government research

No PRA implications. First, any given usability test should involve nine or fewer users. Additionally, the PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. It also specifically excludes tests of knowledge or aptitude, 5 CFR 1320.3(h)7, which is essentially what a usability test tests. See the methods for Recruiting and Privacy for more tips on taking input from the public.