Discover

Build a greater understanding of your problem and the people it impacts.

Contextual inquiry

What

The product team unobtrusively observes participants at work, with their permission, then asks questions.

Why

To learn how and why users do what they do; to discover needs and attitudes that might not emerge in an interview to map how tools, digital and otherwise, interact during complex activities.

Time required

1-2 hours per user

How to do it

  1. With permission from a supervisor and from the participant, schedule a time to watch a typical work activity and record data.
  2. While observing, ask the participant to act normally. Pretend you’re a student learning how to do the job. Ask questions to help you understand what the person is doing and why.
  3. At the end of the session, explain what you have learned and check for errors.
  4. Immediately after, write up your notes.

Example from 18F

A pair of 18F team members visited two Department of Labor/Wage Hour Division investigators as they interviewed home health care workers who were subject to unpaid overtime and other infractions. Since it was a sensitive subject, the 18F team did not question the health care workers directly, but instead asked the investigators clarifying questions in private. 18F staff also made sure that photos did not include faces.

Additional resources

Applied in government research

No PRA implications, if done properly. Contextual interviews should be non-standardized, conversational, and based on observation. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

For internal folks, get permission from the right level of management. If participants could be under union agreements, contact the agency’s labor relations team.

18F

Design studio

What

An illustration-based way to facilitate communication (and brainstorming) between a project team and stakeholders.

Why

To create a shared understanding and appreciation of design problems confronting the project team.

Time required

3–4 hours

How to do it

  1. Invite between six and 12 participants: stakeholders, users, and team members who need to build a shared understanding. Before the meeting, share applicable research, user personas (unless users will be present), and the design prompt for the exercise.
  2. Bring drawing materials. At the start of the meeting, review the design prompt and research you shared.
  3. Distribute drawing materials. Ask participants to individually sketch concepts that address the prompt. Remind them that anyone can draw and artistic accuracy is not the goal of the exercise. 15–20 minutes.
  4. Have participants present their ideas to one another in groups of three and solicit critiques.
  5. Ask the groups to create a design that combines the best aspects of members’ individual contributions.
  6. Regroup as a whole. Have each group of three present their ideas to everyone. Discuss.
  7. After the meeting, note areas of consistent agreement or disagreement. Incorporate areas of consensus into design recommendations and areas of contention into a research plan.

Example from 18F

Additional resources

Applied in government research

No PRA implications. If conducted with nine or fewer members of the public, the PRA does not apply, 5 CFR 1320.5(c)4. If participants are employees, the PRA does not apply.

18F

Dot voting

What

A simple voting exercise to identify a group’s collective priorities.

Why

To reach a consensus on priorities of subjective, qualitative data with a group of people. This is especially helpful with larger groups of stakeholders and groups with high risk of disagreement.

Time required

15 minutes

How to do it

  1. Bring plenty of sticky notes and colored stickers to the meeting.
  2. Gather everyone on the product team and anyone with a stake in the product.
  3. Quickly review the project’s goals and the conclusions of any prior user research.
  4. Ask team members to take five minutes to write important features or user needs on sticky notes. (One feature per sticky note.)
  5. After five minutes, ask participants to put their stickies on a board. If there are many sticky notes, ask participants to put their features next to similar ones. Remove exact duplicates.
  6. Give participants three to five colored stickers and instruct them to place their stickers on features they feel are most important to meeting the project’s goals and user needs.
  7. Identify the features with the largest number of stickers (votes).

Applied in government research

No PRA implications: dot voting falls under “direct observation”, which is explicitly exempt from the PRA, 5 CFR 1320(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Empathy map

What

Creating a visualization of user thoughts and attitudes to help align the team on a project’s needs and layout

Why

To cultivate a deep understanding of user needs and feelings regarding a product.

Time required

20-45 minutes

How to do it

  1. Create an empathy map split into four quadrants: “Says,” “Thinks,” “Feels,” and “Does.”
  2. Through user research, fill out the map to help visualize profiles.
  3. Use the map to inform your design process.

Additional resources

Traditional Empathy Map

Empathy Map: The First Step in Design Thinking - Sarah Gibbons

18F

Heuristic evaluation

What

A quick way to find common, large usability problems on a website.

Why

To quickly identify common design problems that make websites hard to use without conducting more involved user research.

Time required

1–2 hours

How to do it

  1. Recruit a group of three to five people familiar with evaluation methods. These people are not necessarily designers, but are familiar with common usability best practices. They are usually not users.
  2. Ask each person to individually create a list of “heuristics” or general usability best practices. Examples of heuristics from Nielsen’s “10 Usability Heuristics for User Interface Design” include:
    1. The website should keep users informed about what is going on, through appropriate feedback within reasonable time.
    2. The system should speak the user’s language, with words, phrases and concepts familiar to the user, rather than system-oriented terms.
    3. Users often choose system functions by mistake and will need a clearly marked “emergency exit” to leave the unwanted state without having to go through an extended dialogue.
  3. Ask each person to evaluate the website against their list and write down possible problems.
  4. After individual evaluations, gather people to discuss what they found and prioritize potential problems.

Additional resources

Applied in government research

No PRA Implications, as heuristic evaluations usually include a small number of evaluators. If conducted with nine or fewer members of the public, the PRA does not apply, 5 CFR 1320.5(c)4. If participants are employees, the PRA does not apply. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

KJ method

What

A facilitated exercise in which participants list their individual priorities onto cards, collect them as a group, organize them by relationship, and establish group priorities through individual voting.

Why

To reach a consensus on priorities of subjective, qualitative data with a group of people. This is especially helpful with larger groups of stakeholders and groups with high risk of disagreement.

Time required

1–2 hours

How to do it

  1. Gather four or more participants for 90 minutes. Provide sticky notes and markers.
  2. Create a focused question about the project’s needs and select a facilitator to run the exercise.
  3. Give participants five minutes to write at least three responses to the question, each on its own note.
  4. Give participants 15 minutes to put their answers on the wall, read everyone else’s, and make additions. Have participants cluster similar answers without discussion.
  5. Ask participants to write names for each cluster on their own - this is mandatory. They may also split clusters.
  6. Put each name on the wall by its cluster. Exclude word-for-word duplicates.
  7. Reiterate the question and have each person rank their three most important clusters. Visually tally points.
  8. Combine duplicates and their points if the entire group agrees they’re identical. Three or four groups usually rank higher than the rest - these are the priorities for the question.

Example from 18F

18F conducted this exercise with 20 Federal Election Commission staff members to define priorities around conflicting requests. We used this method to get data from staff (not the decision makers) about what they saw as the most pressing needs. We synthesized and presented the data back to the decision makers.

Additional resources

“The KJ-Technique: A Group Process for Establishing Priorities.” Jared M. Spool.

Applied in government research

At 18F, KJ participants are almost always federal employees. If there is any chance your KJ workshop could include participants who are not federal employees, consult OMB guidance on the Paperwork Reduction Act and the Privacy Act. Your agency’s Office of General Counsel, and perhaps OIRA desk officers, also can ensure you are following the laws and regulations applicable to federal agencies.

18F

Personas

What

Representations of users based off of qualitative and quantitative user research.

Why

To create reliable and realistic representations of your key audience segments for reference

Time required

1-2 days

How to do it

  • Conduct user research: Answer the following questions: Who are your users and why are they using the system? What behaviors, assumptions, and expectations color their view of the system?
  • Condense the research: Look for themes/characteristics that are specific, relevant, and universal to the system and its users.
  • Brainstorm: Organize elements into persona groups that represent your target users. Name or classify each group.
  • Refine: Combine and prioritize the rough personas. Separate them into primary, secondary, and, if necessary, complementary categories. You should have roughly 3-5 personas and their identified characteristics.
  • Make them realistic: Develop the appropriate descriptions of each personas background, motivations, and expectations. Do not include a lot of personal information. Be relevant and serious; humor is not appropriate.

Additional resources

«««< HEAD Template of a User Persona =======

&lt;blockquote&gt;
  &lt;blockquote&gt;
    &lt;blockquote&gt;
      &lt;blockquote&gt;
        &lt;blockquote&gt;
          &lt;blockquote&gt;
            &lt;p&gt;73f05b9b7d4c9a741d75156ee8a139cac7afd9d6&lt;/p&gt;
          &lt;/blockquote&gt;
        &lt;/blockquote&gt;
      &lt;/blockquote&gt;
    &lt;/blockquote&gt;
  &lt;/blockquote&gt;
&lt;/blockquote&gt;

How to use Personas -Usability.gov

18F

Project plan

What

A statement of how and when a project’s objectives are to be achieved, by showing the major products, milestones, activities and resources required on the project.

Why

To get an idea of the timeline and goals of a project.

Time required

Project Dependent

How to do it

  1. Form a project planning group for your team. This team will first determine the objectives and goals of the project, and once work begins, they will manage to see if the plan is being kept to.
  2. To begin making the plan, do an initial assessment of the issues that need to be worked through and the difficulties that may be encountered in the project.
  3. Whenever possible, involve the community to get feedback on their needs, whether through town halls or focus groups.
  4. As the project continues, manage the plan, and make adjustments if necessary.
18F

Stakeholder and user interviews

What

A wide-spanning set of semi-structured interviews with anyone who has an interest in a project’s success, including users.

Why

To build consensus about the problem statement and research objectives.

Time required

1–2 hours per interviewee

How to do it

  1. Create a guide for yourself of some topics you’d like to ask about, and some specific questions as a back up. Questions will often concern the individual’s role, the organization, the individuals’ needs, and metrics for success of the project.
  2. Sit down one-on-one with the participant, or two-on-one with a note-taker or joint interviewer, in a focused environment. Introduce yourself. Explain the premise for the interview as far as you can without biasing their responses.
  3. Follow the conversation where the participant takes it. They will focus on their priorities and interests. Be comfortable with silences, which allow the participant to elaborate. To keep from getting entirely off course, use your interview guide to make sure you cover what you need to. Ask lots of “why is that” and “how do you do that” questions.
  4. If there are other products they use or your product doesn’t have constraints imposed by prior work, observe the stakeholders using a competing product and consider a comparative analysis.

Additional resources

Applied in government research

No PRA implications. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

UX questionnaire

What

An introductory exercise teams can use to get alignment on the scope of the project and to begin planning the user design of its components.

Why

To determine how the product appeals to users, and to assist in finding necessary changes before decisions have been made.

Time required

10-30 minutes

How to do it

  1. Use our UX Questionnaire and get together as a team to talk through the questions, making sure to record your answers where all team members can access them.
  2. As you go through the Discovery phase, refer back to your answers to help inform the design process.

Additional resources

18F