Standardizingthe research and design process for NYC's first municipal service-design studio
The Service Design Studio at NYC Opportunity deals with unnecessary hurdles and inefficiencies when conducting user-research interviews. I was honored to collaborate with the Studio to see how we could standardize an efficient process for obtaining, sorting, and synthesizing key insights gathered from user-interviews while helping the Studio fulfill its mission of making public services more accessible to New Yorkers.
We came up with a participatory-designed framework for standardizing the process that the Studio can now use for researching, ideating, and designing solutions in the civic-space.
A special thank you to Service Designer Emily Herrick who provided guidance throughout the project.
Which problems are we going to solve?
Roadblocks throughout the research process
Duplications cause inefficiencies. The Studio's approach to user-research was filled with roadblocks that made collecting and managing user-data tedious and duplicative. For example, a common workflow after conducting a user-interview would be to tag key insights on sticky notes, record them on a cloud-based service (i.e. Google Docs), and then dispose of them in the bin afterwards. That's a lot of wasted post-its.
So many UX tools to choose from
With so many moving parts throughout the research process, the Studio needed a tool to centralize all quotes, insights, and discoveries in one repository and share it with the rest of the team. However, there were too many software vendors to choose from and we needed to pick the right one that would solve for our specific needs.
In order to choose the most appropriate software tool for our team, we needed to fully understand what our design process was like in the first place, and identify any pain-points to solve for with that tool.
Not many actionable takeaways from weekly meetings
Every week, the Studio would meet with key stakeholders on a project to discuss findings gathered during user-interviews. While it was important to have everyone at the table share quotes and insights, the format of the meetings was structured around passive listening and note-taking.
The meetings were in need of a revamped framework that would enable our Studio team and stakeholders to do the following:
- take actionable notes
- ask informed questions
- generate hypotheses
- ideate solutions
The following points outlined the vision for the Studio's success for creating a design process:
Here's how we would go about solving for them:
Before embarking on a design-journey, it's important to know which solutions are currently addressing the issue(s). More specifically, it's helpful to understand what works and what could be improved about those current solutions.
From whom do we want to learn? Who are the key stakeholders involved in the service we're designing for that can provide first-hand perspectives on their pain-points, past attempts to solve for them, and what an ideal solution looks like in the future?
After talking and learning from various stakeholders, we need to make sense of their valuable insights. In this phase- we sort data, recognize patterns and themes, and see trends into what everyone else considers to be important.
After making sense of our stakeholders' painpoints, we are in a better position to brainstorm solutions with their needs in mind. This phase also involves prototyping ideas and testing them with our users in a low-risk way before being heavily-invested in the idea.
How do our prototypes and solutions measure up in the "real world?" Were our ideas validated by users and key stakeholders? This is where we see our solutions at work, based on the metrics and evaluations we set initially.
What solutions are currently out there that work?
Before embarking on this adventure, I scanned the "digital landscape" to see which solutions were out there that could help achieve the Studio's goals for creating a streamlined design process. I came across some powerful, all-in-one UX tools that did amazing things and jotted them down in my handy notebook- but the question remained: "were these even the right tools that could help enhance the Studio's UX process in the first place?"
In order to answer this question, I had to understand what our process was like to begin with.
After digging through the Studio's list of tools, I learned that we use the following for recording and digitizing interview notes:
- To manage data
- mine for emerging patterns and insights
- sort through data
- To transcribe interviews
- include key insights and additional notes
- share notes with the team
- To visualize key-quotes
- generate insights
- synthesize and visualize emerging patterns
Getting the Studio team involved
Recruiting participants for research interviews
I reached out to four of the Studio's designers: two of whom worked full-time and two who were on a short-term fellowship. I liked that there was variation in how long they’ve been at the Studio. In my mind, the recent fellows would most likely have a different take on how things are done as opposed to the more established designers who may have grown accustomed to the same way of doing things.
Fresh minds beget fresh perspectives.
What did I want to understand?
In order to understand which UX Tools would best facilitate the Studio’s research process, I had to understand how the Studio did research in the first place. All of my research efforts were focused on understanding three components: People, Processes, and Tools, and how they all affect each other.
I wanted to center my research efforts on understanding my colleagues better: to know what their design process is like, which tools they use, and which possibilities they'd like to explore for solving their pain-points and accomplishing their goals.
Making sense of the data
After conducting the first round of user interviews, I noticed recurring pain-points that everyone in the team was having, as well as opportunities for solving them.
Here's what the Studio team expressed regarding their shared pain-points:
“We need more structure on how to formalize our tools and processes. A lot is missing because a lot of people are involved on the research side. How we defined insights were varied.”
"Lack of consistency is what's most difficult about managing data"
"It wasn't consistent. The consistency wasn't there, feeling that the synthesis process was different for the next round"
"It would have been good to have some type of reporting structure for our synthesis days. For process that would have been good- I don't know how that would look like"
"A lot of data, and really overwhelming"
"Lets say we limited to 5 insights and opportunities, could we have really captured that instead of overwhelming ourselves with 40-50 per person? Could we have done with with 5-10 insights, I don't know. Maybe not. It was definitely overwhelming."
"After the synthesis and capturing that synthesis, we were really exhausted because it took 4-6 hours a day. How do you take that and take a snapshot of it and have it synthesized for you?"
"There's a big amount of raw data in everyone's head that no one else knows about"
"That’s when most of the research was being conducted. Since I’m the project lead, I was leading some of the synthesis sessions, but when I was gone, the team was using different frameworks every time. And they were also conducting research while they were doing synthesis, and it wasn’t paced very well."
"One thing that was challenging for us- it was time-consuming to transcribe into note-taking form and then bringing it into post-its, that whole process was cumbersome"
"I really wanted to organize this, but just couldn't for little technical reasons like your stickies would start falling out if you stacked too many on top of one another. We don't have our own room so we acn't have stuff up for an extended amount of time. So you're constantly moving paper with paper, so stuff starts falling down, you're losing it"
" We had these 6 hour long days, and we were really tired and all this stuff would be up and hanging and we'd be like, 'oh we'll return to it tomorrow', but we'd have 800 other things so it would stay up and wait till the next week that'd we'd block and then we'd forget. We'd have to start over and we'd ask 'what is this again?' "
Wow, a lot of fatigue sifting through piles of sticky notes for hours at a time. This was definitely a painpoint for our Studio designers, but beneath the difficulties there were opportunities worth uncovering to solve for those problems.
"Having a designated space for your research- the space was such a restraint. This was the only space that allowed for everyone to use to do synthesis"
"Worflow wise, collaboration is super key- having another at least one or two people in the team working with you so that you can bounce ideas off, having that collaboration so that you're not doing everyting by yourself, even in the synthesis aspect."
"When possible, streamlining your notes and typing directly into a computer and then having time blocked off right after an interview to digitize and then that becomes a norm or if that's something you do during debrief with your partner, which is to type your notes right away during the debrief?"
"Have set synthesis days during a sprint, so you know that every week you have a 4 hour block for synthesis that you know you're coming to the session so you can prepare accordingly"
Bringing all minds together
We hosted a brainstorming workshop to address our pain-points and ideate solutions
After sorting the key-findings into painpoints and opportunities, I wanted to bring the Studio together for a workshop in which we could all offer input to solve for those pain-points and capitalize on opportunities.
We began with a round of ice-breakers in which we “aired a grievance” from the past day or week. This helped position our minds to think about negative situations and imagine ways to improve them. Our minds were now in "process-improvement" mode.
The least pressing problem that rarely happens is insight overload. That's a good thing- because as a data-driven team, we can never get enough of data and insights.
What should we "stop, start, and continue" in our design processes?
The team was then invited to deliberate on how to map out an ideal workflow that would ensure consistency and solve for the above pain-points by asking the following questions:
- What do we want to stop doing?
- What do we want to continue doing?
- What do we want to start doing?
Here’s how the team envisioned a standardized process going forward:
- Have a diverse team so that stakeholders can get involved
- Clarify and align on understanding between interview-lead and note-taker
- Surface initial and most revelatory insights and quotes
- Generate fifteen immediate insights on sticky notes
- Take five minutes to debrief
- Interviewer sends a "thank you" note and follow-up information to keep stakeholders invested
- Have a near-transcript record of conversation
- Tag major themes and insights
- Organize info into research themes
- Create discussion-frameworks so that workshops and meetings are actionable, engaging, and not passive.
- Incorporate discussion points on patterns, insights, open questions, and hypotheses/ideas
In order to make our shareout sessions more actionable and less reliant on passive listening, we ideated the following framework for taking notes:
Each person in the meeting is given a ‘Notes Template’ to fill out as they are listening to the interviews being read aloud. The template will be divided into the following sections:
What data has been validated?
Which quotes did you find novel/ interesting? why?
What do you still not know?
What might work? Write your "fixes" down to get them out of your head!
Deciding on outcomes and output
One key takeaway from our brainstorm session was that there was a need to track and share data that was emerging from interviews with stakeholders. Together, we decided that we needed to create a repository of emerging data and insights that would be "open-sourced" with key stakeholders and participants.
This gave way to the "Field Logs" which the Studio now uses to journal each important milestone during the design-process for any given project.
Putting our new design framework to the test with the "Pathways to Prevention" initiative.
The "Pathways to Prevention" initiative is a year-long collaboration between the Administration for Children's Services (ACS) and the Studio at NYC Opportunity to deliver an approach that would allow families to use their "voice and choice" to opt for social-services and thereby prevent their children from entering the foster-care system.
The Studio noticed the following improvements:
The Studio began to take more actionable notes called "Field Logs"
After coming to terms with the fact that we needed to make our share-out meetings more actionable and less dependent on passive-listening, the Studio began to create note templates for all participants in order to:
- ask insightful questions
- identify emerging patterns and insights/trends
- generate hypothesis and ideas
The team was able to come up with more insightful questions to ask families in future interviews
The quality of follow-up questions was higher- helping the team explore solutions that addressed the needs of family stakeholders in a more meaningful way.
Here are some of the questions the team thought of before prototyping solutions:
How might we provide parents with opportunities to make informed choices about key parts of their prevention experience?
How do we ensure families safe enough to share their preferences for prevention services?
I wonder how we might design materials that meet each family’s unique situation?
what content is important to families when signing up for Prevention services?
It became easier to visualize trends and insights during the synthesis phase
The Studio was able to better identify emerging trends from interviews with families. For example, the team learned that families needed to feel safe knowing that any interaction with ACS would not lead to the separation of their child
The team also learned that ACS' communication with families should be trauma-informed and easy to understand.
In our research, we heard that the information available to families about the different types of programs and providers could feel unclear, inconsistent, and difficult to access.
We (the Studio) should be including strengths-based phrasing whenever asking parents to identify things they may be struggling with to provide to their families.
Prevention services are not one-size fits all, and we should tailor how families learn about them to meet their diverse needs.
These types of tools would help case-workers effectively move from the role of investigator to supporter.
The team was better informed when testing out prototypes
By focusing on key emerging-trends gathered during interviews, the team was able to quickly prototype ideas that spoke to the needs of their stakeholders and receive validation from case-workers at ACS.
The DCP staff welcomed the idea of well-designed and colorful materials with standardized, family-focused language.
This step involves presenting key information about Prevention providers so families can make informed choices.
We learned that something as simple as showing dad a photo of their new case planner could go a long way in demystifying what Prevention can do for his family.
Diverse groups of teams and stakeholders were incorporated in the research process
We took our findings from our initial brainstorming session and decided to include diverse stakeholders throughout the research process.
For the Pathways to Prevention project, the Studio invited families, ACS case-workers, and members from the Community Based Strategies team at ACS for more enriched perspectives.
Each Thursday in February and March the Pathways team will meet with families, prevention provider staff, and Child Protection Specialists at ACS to test a lot of prototypes designed to answer the project's guiding question: How might we provide parents with opportunities to make informed choices about key parts of their prevention experience?
A centralized and digitized tool for tracking, sorting, and mining data
Now with a revamped design process in place, we needed a software tool that could help fulfill our goals and address our existing pain-points.
We narrowed our search to the following tools based on a decision-matrix:
- An infinite canvas
- Ability to create "frames" or sections that can contain any piece of data
- Easy to import photos of sticky-notes and include within your canvas
- Collaborate with others
- Easy to group notes into categories
- Not much in terms of statistical analysis
- Freedom to manipulate trello boards
- Create “sprints”, along with interview notes
- Easy to import photos of sticky-notes and include within your canvas
- Tagging notes as you envision it best
- Free service
- Not much you can do with data analysis
- Not designed to be a UX research tool
- You’ll need to get creative and “hack” it to your benefit
- Useful for data-analysis
- Tagging feature allows for data-mining and clustering into similar categories
- Visualizations to help make sense of repeat tags
- Collaborate with others
- Not easy to import existing notes from a Google Doc or Word Doc
- Learning curve for getting used to their tagging system
- Easy to import existing docs for digitizing
- Easy to tag and create tagged-groups
- Visualizations to help make sense of repeat tags
- Collaborate with others
- We have to manually create categories and insights
The Studio decided to go with RealTime Board as a UX tool to centralize and digitize all of our findings going forward.