Cookie Science 5: ‘Blinding’ your subjects

This article is one of a series of Experiments meant to teach students about how science is done, from generating a hypothesis to designing an experiment to analyzing the results with statistics. You can repeat the steps here and compare your results — or use this as inspiration to design your own experiment. 

Welcome back to Cookie Science! I have a cookie recipe I want to share with my friends. But my friend Natalie suffers from celiac disease. Whenever she eats gluten, her immune system attacks her small intestine (making nasty things happen in her gut).  She copes by avoiding gluten. It’s a protein found in wheat. If I want her to be able to eat my favorite chocolate-chip cookie, I’m going to have to create a gluten-free version. As I experiment with this new recipe, I’m showing you how to use the scientific process to answer questions.

After looking through different cookie recipes, I came up with a hypothesis — a statement that I can test. Most gluten-free cookie recipes do not simply replace wheat flour with gluten-free flour. My hypothesis: Substituting gluten-free flour alone into my cookie dough will not make a cookie that will taste as good as my original recipe.

To test my hypothesis, people must try the new cookies. It’s important that I treat my cookie tasters fairly, so I created a protocol and had a review committee sign off on it. This document describes how I will conduct my experiment. It also states how I will protect from harm all who sample my cookies.

Now, I need to feed people my cookies. But I have to be careful as I hand them out. I can’t make it obvious which cookie is the control cookie — containing normal flour — and which cookies are gluten-free. Knowing which cookies are the altered ones, and why I’m altering their recipe, could influence how people respond to them. In science, we refer to such influence as a bias. People may naturally expect that regular flour should taste better than gluten-free flour. So they might be inclined to rate a cookie with regular flour as more tasty than one with gluten-free flour, even before they have tasted either.

To keep my taste testers from doing that, I have to blind them to which is the test cookie. Here, blinding means to hide important differences about features that will be compared. I can’t just give them a plate with one cookie labeled “control” and another “gluten-free.” I want to make the cookies different enough that they won’t accidentally be mixed up, but I also don’t want people to be able to tell which cookie is the control.

My solution: color-coding. I will not tell my participants what the colors mean. My tasters will then judge a “red” cookie against a “blue” cookie (hopefully without being able to tell by sight what other ingredients might make them different).

It’s also important to make sure that each taster judges cookies in the same way. I can’t just have them write down what they generally think of the cookies (perhaps they thought it had too many chocolate chips or was too big). One reason: Such general comments can’t easily be combined and analyzed.

So my participants must fill out a survey. It uses what’s called a Likert scale. Each Likert question is a statement. It might be: “This cookie is chewy.” Under the statement, people consider a range of possible opinions, from “strongly agree”(the cookie is chewy) to “strongly disagree” (meaning the cookie isn’t chewy at all). Participants circle the description that best describes the cookie’s trait.

My survey includes the following Likert statements.

1.       This cookie is sweet.

2.       This cookie is chewy.

3.       I like this cookie’s texture.

4.       This cookie is dry.

5.       Overall, I like this cookie.

From the taster’s Likert ratings, I will be able to see how closely the flavor and texture of my gluten-free cookie compares to the normal wheat-flour cookie. Many of the statements focus on texture. They ask for assessments of things like how dry each cookie is or if it is chewy.

In reading scientific studies on gluten and gluten-free flours, I learned that gluten has important effects on the texture of dough. For instance, it tends to make it more elastic. Gluten-free flours also produce thinner cookies, with more moisture. To confirm these changes, I am asking questions about each cookie’s texture.

I also have included a question asking people to state how sweet they think each cookie is. I am only changing one variable — the type of flour — but I want to see if this impacts how sweet people think the cookies are.

Finally, I am asking people to rate the cookies overall. There are many different factors that go into a good cookie. In addition, some people might say a cookie is dry and not very sweet. But they might prefer a dry, not-too-sweet cookie. Others may desire one that are soft, moist and sweet.

Now, it’s time to start baking!

Follow Eureka! Lab on Twitter

Power Words

bias   The tendency to hold a particular perspective. Scientists often blind subjects to the details of a test so that the participant’s biases will not affect the results.

blinding     (in research)  The deliberate hiding of some important test information from a participant. For instance, a researcher might put a blindfold on participants before they eat something. This prevents them from being influenced by a food’s appearance. For a test of how well a drug works, researchers may give each person a capsule or shot. But only one group gets the real drug. Others might get an inactive “sugar pill” or injection of salt water. By not knowing if they got the real deal or an imposter, participants’ responses should not be biased for or against a particular result.

celiac disease (or coeliac disease in the United Kingdom)  A disorder in which the immune system attacks the small intestine after it encounters foods containing gluten, a wheat protein compound. People with this disease suffer from stomach pain, constipation, diarrhea and a constant feeling of fatigue. They must avoid gluten-containing products like bread, cake and cookies.

gluten  A pair of proteins — gliadin and glutenin — joined together and found in wheat, rye, spelt and barley. The bound proteins give bread, cake and cookie doughs their elasticity and chewiness. Some people may not be able to comfortably tolerate gluten, however, because of a gluten allergy or celiac disease.

hypothesis  A proposed explanation for a phenomenon. In science, a hypothesis is an idea that hasn’t yet been rigorously tested. Once a hypothesis has been extensively tested and is generally accepted to be the accurate explanation for an observation, it becomes a scientific theory.

Likert scale  One of the most commonly used ways for ranking opinions or statements in surveys involving people. A issues a series of statements, such as “I like X,” “the test was easy,” or “it was too loud.” Participants then rate how well they agree by choosing from a range options that might range from “strongly agree” to “strongly disagree.”

protocol  (in science) A written procedure to describe how an experiment will be conducted. Protocols are written before an experiment is performed and are used to make sure that the experiment meets standards for fairness and good practice. Protocols also allow other people to attempt the same experiment and see if they can replicate previous results.