Lesson 2: Statistical Study

class: center, middle, inverse, title-slide

.title[
# Lesson 2: Statistical Study - Trust or Not
]
.author[
### Fei Ye <br /><br /> Department of Mathematics and Computer Science<br /><br />
]
.date[
### May, 2024
]

---

class: center middle

<div>
<style type="text/css">.xaringan-extra-logo {
width: 110px;
height: 25px;
z-index: 0;
background-image: url(assets/nc_sa.png);
background-size: contain;
background-repeat: no-repeat;
position: absolute;
bottom:12px;left:1%;
}
</style>
<script>(function () {
  let tries = 0
  function addLogo () {
    if (typeof slideshow === 'undefined') {
      tries += 1
      if (tries < 10) {
        setTimeout(addLogo, 100)
      }
    } else {
      document.querySelectorAll('.remark-slide-content:not(.title-slide):not(.hide_logo)')
        .forEach(function (slide) {
          const logo = document.createElement('div')
          logo.classList = 'xaringan-extra-logo'
          logo.href = null
          slide.appendChild(logo)
        })
    }
  }
  document.addEventListener('DOMContentLoaded', addLogo)
})()</script>
</div>

## Unit 5B: Should You Believe a Statistical Study

*Primary Source:* PPT for the book "Using & Understanding Mathematics".

---

## Evaluating a Statistical Study

Most statistical research is carried out with integrity and care. Nevertheless, statistical research is sufficiently complex that bias can arise in many different ways. There are eight guidelines that can help you answer the question "Should I believe a statistical study?"

---

## Summary of Eight Guidelines (1 of 2)

1. Get a Big Picture View of the Study. You should understand the goal of the study,
the population that was under study, and whether the study was observational
or an experiment.

2. Consider the Source. Any potential sources of bias on the part of the
researchers.

3. Look for Bias in the Sample. Decide whether the sampling method was likely to
produce a representative sample.

4. Look for Problems in Defining or Measuring the Variables of Interest. Ambiguity
in the variables can make it difficult to interpret reported results.

---

## Summary of Eight Guidelines (2 of 2)

<ol start="5">
<li><p>
Beware of Confounding Variables. If the study neglected potential confounding
variables, its results may not be valid.
</p></li>

<li><p>
Consider the Setting and Wording in Surveys. Look for anything that might tend
to produce inaccurate or dishonest responses.
</p></li>

<li><p>
Check That Results Are Presented Fairly. Check whether the study really supports
the conclusions that are presented in the media.
</p></li>

<li><p>
Stand Back and Consider the Conclusions. Evaluate whether the study achieved
its goals. If so, do the conclusions make sense and have practical significance?
</p></li>
</ol>

---

## Guideline 1: Get a Big Picture View of the Study

Try to answer the following questions.

1. What was the goal of the study?

2. What was the population under study? Was the population clearly and appropriately defined?

3. Was the study observational or an experiment? If it was an experiment, was it single- or double-blind, and were the treatment and control groups properly randomized? Given the goal, was the type of study appropriate?

---

## Example: Appropriate Type of Study

Researchers gave 100 participants their astrological horoscopes and asked whether the horoscopes appeared to be accurate; 85% of the participants answered yes (the horoscopes were accurate). The researchers concluded that horoscopes are valid most of the time.

Should you believe the conclusion? Why?

**Solution:** The goal of study was to determine whether horoscopes are valid. Like testing whether a drug is effective or not. The question may be better answered by experiment.

Moreover, the researcher could easily influence the result by how they survey the participants.

---

## Practice: Academic Preparation

A study of the academic preparation of graduates for a high school used only SAT mathematics scores of graduates who were accepted by Ivy League schools.

---

## Guideline 2: Consider the source

Statistical studies are supposed to be subjective, but the people who carry them out and fund them may be biased.

---

## Example: Is the drug more effective

A group of researchers funded by a pharmaceutical company conducted a study on the effectiveness of a new drug of the company, and concluded that the new drug of the company is more effective than similar drugs of competing companies.

Should you believe that? Why?

**Solution:** Because the researcher was funded by the company, they may tend to conduct the study in ways that the result will favor the company.

---

## Guideline 3: Look for bias in the sample

- Selection bias occurs whenever researchers select their sample in a way that tends to make it unrepresentative of the population.

- Participation bias occurs primarily with surveys and polls; it arises whenever people choose whether to participate. Because people who feel strongly about an issue are more likely to participate, their opinions may not represent the larger population.

---

## Example: Website Survey

A website has a survey asking readers to give their opinion on a tax proposal.

Should you believe that? Why?

**Solution:** Only visitors of the website may see the survey. So the sample is not representative. Besides, only visitors feel strongly about the issue are likely to participate.

So both selection and participation bias exist in the study.

---

## Practice: Polling Customers

From a poll of people who recently bought cold medicine at all stores of a large drugstore chain, investigators concluded that the mean time between colds for all Americans is 5.6 months.

Identify at least one potential source of bias in the study. Explain why the bias would or would not affect your view of the study.

---

## Guideline 4: Look for problems in defining or measuring the variables of interest

A **variable** is any item or quantity that can vary or take on different values.

The variables of interest in a statistical study are the items or quantities that the study seeks to measure.

---

## Example: Illegal Drug Supply

A commonly quoted statistic is that law enforcement authorities succeed in stopping only about 10% to 20% of the illegal drugs entering the United States. Should you believe this statistic?

**Solution** There are essentially two variables in the study: quantity of illegal drugs intercepted and quantity of illegal drugs NOT intercepted. It should be relatively easy to measure the quantity of illegal drugs that law enforcement officials intercept.

However, because the drugs are illegal, its unlikely that anyone is reporting the quantity of drugs that are not intercepted. How, then, can anyone know that the intercepted drugs are 10% to 20% of the total? In a New York Times analysis, a police officer was quoted as saying that his colleagues refer to this type of statistic as PFA,for pulled from the air.

---

## Guideline 5：Beware of confounding variables

Variables that are not intended to be part of the study can sometimes make it difficult to interpret the results properly. Such variables may have an effect on the relationship between these variables of the study and are called **confounding variables**.

---

## Example: Smoking and lung cancer

A study on a group of the people live in a large city shows an association between smoking and lung cancer. Can we conclude that smoking is the cause of the cancer in the study?

**Solution:** Smoking may be the cause, however, air pollution could also be a cause. As air pollution in a large city may contribute to the lung cancer, so we cannot be sure that smoking is the cause of cancer in this study.

---

## Practice: Cause of Obesity

In a study of obesity among children, researchers monitor the eating and exercise habits of the participating children, carefully recording everything they eat and all their activity?

Identify a confounding variable. Explain why or why not the variable will affect the study.

---

## Guideline 6: Consider the design of surveys

Even when a survey is conducted with proper sampling and with clearly defined terms and questions, it is important to watch out for problems in the setting or wording that might produce inaccurate or dishonest responses.

---

## Example: Do You Want a Tax Cut

The Republican National Committee commissioned a poll to find out whether Americans supported their proposed tax cuts. Asked Do you favor a tax cut?a large majority answered yes. Should we conclude that Americans supported the proposal?

**Solution:** A question like "Do you favor a tax cut?" is biased because it does not give other options. If the question was asked by in a different form "Would you favor a tax cut even if it increased the federal deficit?" the conclusion would be different.

---

## Guideline 7: Check that results are presented fairly

The study may be misrepresented in graphs or concluding statements.

Researchers may misinterpret the results or jump to conclusions not supported by the results.

---

## Example: Does the School Board Need a Statistics Lesson

The school board in Boulder, Colorado, created a hubbub when it announced that 28% of Boulder school children were reading below grade level and hence concluded that methods of teaching reading needed to be changed. The announcement was based on reading tests on which 28% of Boulder school children scored below the national average for their grade. Do these data support the boards conclusion?

**Solution:** The fact that 28% of Boulder children scored below the national average for their grade implies that 72% scored at or above the national average. Therefore, the school boards ominous statement about students reading below grade level makes sense only if grade level means the national average score for a particular grade. This interpretation of grade level is curious because it means that half the students in the nation are always below grade level no matter how high the scores. The conclusion that teaching methods needed to be changed was not justified by these data.

---

## Practice: Increase or decrease tuition

An educational research group that tracks tuition rates finds that tuition at a particular small college is 15% more than it was 10 years ago. At the mean time, the annual inflation rate is 1.8% on average.

Should the college reduce the tuition?

---

## Guideline 8: Stand back and consider the conclusions

Ask yourself the following questions.

1. Did the study achieve its goals?

2. Do the conclusions make sense?

3. Can you rule out alternative explanations for the results?

4. If the conclusions do make sense, do they have any practical significance?

---

## Example: Practical Significance

An experiment is conducted in which the weight losses of people who try a new Fast Diet Supplement are compared to the weight losses of a control group of people who try to lose weight in other ways. After eight weeks, the results show that the treatment group lost an average of 1/2 pound more than the control group. Assuming that it has no dangerous side effects, does this study suggest that the Fast Diet Supplement is a good treatment for people wanting to lose weight?

**Solution:** Compared to the average persons body weight, the difference of one-half pound hardly matters at all. So even if the study is flawless, the results don't seem to have much practical significance.

---

## Practice: Statistical and Practical Significance