Design insights from reframing your research question

Can’t believe my master’s program here at Georgia Tech is almost done! These two years have flown by. Been a busy week already too. The CHI Student Design and Research competitions were yesterday so I was working on those over winter break. Finally figured out my courses. Now updating my portfolio with recent projects.

I had an interesting insight this morning in regards to the WIC project. I still need to think about this more, but I have realized that the focus of my project should not be on technology itself. I did user research in the fall and learned about WIC parents’ familiarity with technology, their preferences for technology, and values (such as convenience, desire to learn, and trust in content) that affected these preferences.

The goal for this semester was to focus on testing technology (mobile app, text messaging, digital kiosk, website) to see which one is most effective for delivering information about children’s development. However, I was having difficulty envisioning how to address this question through user testing. I reframed the most basic question: “What is needed in technology to track milestones”? and listed different features. I realized there was a mismatch between some technologies. I also noticed some are much more critical than others in terms of delivering basic information (e.g., “View a list of milestones” is essential, as compared to “notifications”). Others still are more critical in terms of ACTUALLY getting parents to use whatever technology (e.g., seeing example pictures of each milestone may be extremely helpful)

I am going to use the Kano method, a tool that arose from market research in order to find what features most impacted customer loyalty, to determine which features are most important.

Sometimes it really is all about taking a step back and rethinking why you’re doing research in the first place. Great insights can arise 🙂


Qualitative Data Analysis

I am learning so much in grad school, and arguably the skill in which I have improved the most (besides programming) is qualitative data analysis. I started using Excel back in the 6th grade for Science Fair projects. Crazy to think that was in 1999. I took advanced math courses throughout high school and then continued to use my analysis skills when working in psychology labs at Duke and Carnegie Mellon.

For this WIC project, however, there are no sophisticated quantitative analysis, at least that I have done so far, since almost all of my data is qualitative notes based on interviews.

The experience has been very enjoyable however and in a way has reminded me of detective work. I know the “culprit” (lack of awareness of children’s development) and am now looking for contextual clues as to how parents approach (or don’t) the problem. The focus of the interview is asking parents how they find information related to children’s health and why they prefer that method. I also present them with five different options for learning about children’s development (kiosks, booklets, texts, website, smartphone app) and ask them to rank them in order of preference. Again, I ask them why they ranked these tools the way they did.

While clearly each parent brings a unique perspective, three core values are consistently emphasized. Each one is also associated with a preference for learning about children’s development:

  1. Convenience/ease of access (prefer text messaging and/or phone apps)
  2. Interest/ desire to learn (prefer website and/or booklet)
  3. Trustworthiness of information (prefer kiosk in WIC/doctor’s clinic or asking a doctor in person)

It also contrasts interestingly with a text analysis assignment I did in an information visualization course I am taking. We had to find the “hidden terrorist plot” in a set of 50 FAKE (this was not real!!!!) documents using software our professor implemented. Here we had no direction however, so it was frustrating taking a shot at the dark. Additionally, in the scenario presented, we only had 50 documents and could not “look up” any additional. We could of course Google things to look for potential relationships between the entities mentioned in the documents (e.g., looking up where there are major airports in the US since several cities are mentioned in the documents as potential targets). For the WIC project, I had the benefit of interviewing additional parents, which I did, in order to get a clearer picture of patterns of behavior. I was not restricted in any way, besides the fact that I had to limit the duration of these interviews.


Spanish vs English interviews, gender roles in WIC

The WIC interviews have given me an interesting perspective as  a researcher due to my cultural background. While I have mostly lived in the United States, I lived in Medellin, Colombia from when I was 5-8 and speak Spanish at home. I learned both English and Spanish at the same time though my English is much better.

Many of the WIC clinics have a large Hispanic clientele ranging from 20-50+%. While there is no way to “prove” this, I hope that my pretty obvious ethnicity (due to my name and physical appearance) have made parents in general more at ease speaking to me. Besides the janitors, there isn’t a single man who works at the WIC! Add to that the fact that I am coming to the clinic as an outside researcher from Georgia Tech and it’s clear that there is potential bias of how the parents will perceive me.

Additionally, being able to speak Spanish has allowed me to speak with some parents who I would not otherwise be able to interview.

On a different but related note: I’ve read that there are slight personality changes when bilingual speakers use different languages. I’ve noticed that I’m slightly more upbeat yet also less talkative in Spanish. I’ve also been told that my voice is deeper when I speak in Spanish. This last point is subconscious because I never seem to notice, even though multiple people have pointed it out.

Interested related article on this last point:


WIC Reflections – Marietta Interviews part 1

Research always takes about twice as long as you expect it to. Like in the Spring, the IRB has delayed things. I learned from the Spring to start early and had the Georgia Tech IRB by the time school started. This time, however, I’m dealing with double IRBs since WIC has its own. Complicating the matter is that each of the 200+ WIC districts has its own policy for dealing with research. Marietta, the last clinic I visited, gave me approval after 2 weeks. Still waiting on Lawrenceville (hopefully next week) and DeKalb (MIA). 

Yesterday I conducted two interviews at Marietta. They were originally meant to be practice to make sure there weren’t any flaws in the questions. I personally think they went well and only noted one thing to change in the interview. I present a figure with examples of formats for accessing developmental milestone information and include images of what they look like. I failed to include a picture of a kiosk, making it harder to imagine what it looks like.

Most importantly, both interviews took almost exactly 15 minutes. This study is challenging in that we need to interview parents while they are in the clinic since it would be difficult to schedule interviews in a different location. The WIC provides a secure and trusted location for conducting interviews, all critical features since this is already a disadvantaged and protected population. All the clinics have mentioned that families want to get in and out as quickly as possible so they can get back to work, a reasonable goal since sadly money is scarce.

After consulting with the nutritionists and directors, we decided the best time to conduct the interviews is right after the nutrition consultation. This is the second-to-last step involved and is followed by food vouchers. Usually it takes 15-20 minutes to print these, and sometimes slightly longer, so this is the most convenient time to do it.

For the two interviews so far, the nutritionist instructed a parent to come to me after explaining that the study was optional. This step is critical because, again, since we are working with a special population, we want to place even more emphasis on protecting personal rights. This is one of the reasons why the clause about the research study being optional is one of the first on the consent form.

Another interesting note: for the parents I spoke with, the nutritionists let me know that they “were very quiet and wouldn’t say much”. I actually found both to be fairly talkative. Hopefully this means I am doing a good job in seeming personable and not overly rigid? I will continue to consciously try and do that


The importance of reflection while conducting fieldwork

Course schedules were released yesterday. I had been advised to look into a computer science course called “User Interface Design & Evaluation” since it focused on qualitative research and user testing.

I’ve reviewed all of the slides for the semester in the past day and took notes.

In hindsight, it would have been nice to have taken this course last fall, but it didn’t fit into my schedule. Computing for Good was an absolutely essential class for me to take and I could not handle the courseload of five work-intensive courses.

Definitely would recommend the course for. However, given that I am conducting and have had experience learning about and actually conducting user testing and evaluation, I would learn more from other classes.

One valuable insight from the notes is documenting design reflections, which is the reason why I started this blog in the first place. I will be typing up my own reflections from the observations I have and will continue to conduct on this blog as well. The ones from past visits have been hand-written so I’ll add those fairly soon.

It’s important to acknowledge one’s own biases, especially when dealing with qualitative research. I’m sure these reflective posts will help me analyze my insights, both now and in the spring


WIC/Master’s Project progress

It’s been an eye-opening experience so far working on my master’s project. Building on the work from the past two semesters where I worked on designing, developing, and user testing an Android app meant for educating parents about devleopmental milestones, I am now tackling the same problem but with a broader audience: low-income families. In Georgia, the Women, Infants, and Children (WIC) program works with around 300,000 families that are living 180% under the poverty line. These services they offer reach about half of all of the infants in the state. An Android app is great for families that are well off and who have the latest smartphones, but that technology clearly is not ubiquitous.

I’m still surprised at how few people take the Computing for Good (C4G) class. One trend I’ve noticed at Tech is that the students in my program tend to gravitate towards the newest and coolest gadget. I swear, if I see one more person walking around with Glass….

Anyway, given this trend, it’s no shock that many students express little interest in classes like C4G that don’t always involve iPads and the like. Yet I think more than ever that we owe the world the opportunity to spread some of our talents and ideas to those who, for whatever reason, are not in a position to enjoy life as fully as we are. These are people in other countries but also in our own states and cities. Technology is evolving at a pace more rapid than what humans can keep up with. Let’s make sure we help others to learn to help themselves.

Yes, I could go and get a PhD and do research, but I might end up staying in academia to focus on very narrow projects. I am blessed to have a wide array of spells and think I should use these to my advantage, as well as to the world’s advantage . No matter where I end up, I’ll continue this tradition of technology consulting, hopefully also with the CDC and WIC.