Lecture 6: Back Doors

Recap

• We’ve now covered how to create causal diagrams
• (aka Directed Acyclic Graphs or DAGs)
• We simply write out the list of the important variables, and draw causal arrows indicating what causes what
• This allows us to figure out what we need to do to identify our effect of interest

Today

• But HOW? How does it know?
• Today we’ll be covering the process that lets you figure out whether you can identify your effect of interest, and how
• What do we need to condition the data on to limit ourselves just to the variation that identifies our effect of interest?
• It turns out, once we have our diagram, to be pretty straightforward
• So easy a computer can do it!

The Back Door and the Front Door

• The basic way we’re going to be thinking about this is with a metaphor
• When you do data analysis, it’s like observing that someone left their house for the day
• When you do causal inference, it’s like asking how they left their house
• You want to make sure that they came out the front door, and not out the back door, not out the window, not out the chimney

The Back Door and the Front Door

• Let’s go back to this example

The Back Door and the Front Door

• We’re interested in the effect of IP spend on profits. That means that our front door is the ways in which IP spend causally affects profits
• Our back door is any other thing that might drive a correlation between the two - the way that tech affects both

Paths

• In order to formalize this a little more, we need to think about the various paths
• We observe that you got out of your house, but we want to know the paths you might have walked to get there
• So, what are the paths we can walk to get from IP.spend to profits?

Paths

• We can go Ip.spend -> profit
• Or IP.spend <- tech -> profit

The Back Door and the Front Door

• One of these paths is the one we’re interested in!
• Ip.spend -> profit is a front door path
• One of them is not!
• IP.spend <- tech -> profit is a back door path

Now what?

• Now, it’s pretty simple!
• In order to make sure you came through the front door…
• We must close the back door
• We can do this by controlling/adjusting for things that will block that door!
• We can close IP.spend <- tech -> profit by adjusting for tech

So?

• We already knew that we could get our desired effect in this case by controlling for tech.
• But this process lets us figure out what we need to do in a much wider range of situations
• All we need to do is follow the steps!
  • List all the paths
  • See which are back doors
  • Adjust for a set of variables that closes all the back doors!
• (orrrr use a method that singles out the front door - we’ll get there)

Example

• How does wine affect your lifespan?

Paths

• Paths from wine to life:
• wine -> life
• wine -> drugs -> life
• wine <- health -> life
• wine <- income -> life
• wine <- health <- U1 -> income -> life
• wine <- income <- U1 -> health -> life
• Don’t leave any out, even the ones that seem redundant!

Paths

• Front doors/Back doors
• wine -> life
• wine -> drugs -> life
• wine <- health -> life
• wine <- income -> life
• wine <- health <- U1 -> income -> life
• wine <- income <- U1 -> health -> life

Adjusting

• By adjusting/controlling for variables we close these back doors
• If an adjusted variable appears anywhere along the path, we can close that path off
• Once ALL the back door paths are closed, we have blocked all the other ways that a correlation COULD appear except through the front door! We’ve identified the causal effect!
• This is “the back door method” for identifying the effect. There are other methods; we’ll get to them.

Adjusting for Health

• Front doors/Open back doors/Closed back doors
• wine -> life
• wine -> drugs -> life
• wine <- health -> life
• wine <- income -> life
• wine <- health <- U1 -> income -> life
• wine <- income <- U1 -> health -> life

Adjusting for Health

• Clearly, adjusting for health isn’t ENOUGH to identify
• We need to adjust for health AND income
• Conveniently, regression makes it easy to add additional controls

Adjusting for Health and Income

• Front doors/Open back doors/Closed back doors
• wine -> life
• wine -> drugs -> life
• wine <- health -> life
• wine <- income -> life
• wine <- health <- U1 -> income -> life
• wine <- income <- U1 -> health -> life

How about Drugs?

• Should we adjust for drugs?
• No! This whole procedure makes that clear
• It’s on a front door path
• If we adjusted for that, that’s shutting out part of the way that wine DOES affect life

Practice

• We want to know how X affects Y. Find all paths and make a list of what to adjust for to close all back doors

Practice Answers

• Front door paths: X -> Y, X -> E -> Y
• Back doors: X <- A -> Y, X <- B -> Y, X <- A <- U1 -> B -> Y, X <- B <- U1 -> A -> Y, X <- A <- C -> Y, X <- B <- U1 -> A <- C -> Y
• (that last back door is actually pre-closed, we’ll get to that later)
• We can close all back doors by adjusting for A and B.

Controlling

• So… what does it actually mean to control for something?
• Often the way we will do it is just by adding a control variable to a regression. In \(Y = \beta_0 + \beta_1X + \beta_2Z + \varepsilon\), the \(\hat{\beta}_1\) estimate gives the effect of \(X\) on \(Y\) *while controlling for \(Z\), and if adjusting for \(Z\) closes all back doors, we’ve identified the effect of \(X\) on \(Y\)!
• But what does it mean?

Controlling

• The idea of controlling for a variable is that we want to remove all parts of the \(X\)/\(Y\) relationship that is related to that variable
• I.e. we want to remove all variation related to that variable
• A regression control will do this (although it will only do it linearly), but anything that achieves this goal will work!
• For example, if you want to “control for income”, we could add income as a regression control, or we could pick a sample only made up of people with very similar incomes
• No variation in \(Z\): \(Z\) is controlled for!

The Two Main Approaches to Controlling

Predicting Variation (what regression does):

• Use \(Z\) (and the other controls) to predict both \(X\) and \(Y\) as best you can
• Remove all the predictable parts, and use only remaining variation, which is unrelated (orthogonal) to \(Z\)

Selecting Non-Variation (what “matching” does):

• Choose observations that have different values of \(X\) but have values of \(Z\) that are as similar as possible
• With multiple controls, this requires some way of combining them together to get a single “similarity” value

The Two Main Approaches to Controlling

• In this class we’ll be focusing mostly on regression
• Purely because that’s what economists do most of the time
• Regression and matching rely on slightly different assumptions to work, but neither is better than the other
• Newfangled “doubly-robust” methods do both regression AND matching, so that the model only fails if the assumptions of BOTH methods fail
• So then, focusing on the “predicting variation” approach…

Controlling

• Up to now, here’s how we’ve been getting the relationship between X and Y while controlling for W:

1. See what part of X is explained by W, and subtract it out. Call the result the residual part of X.
2. See what part of Y is explained by W, and subtract it out. Call the result the residual part of Y.
3. Get the relationship between the residual part of X and the residual part of Y.

• With the last step including things like getting the correlation, plotting the relationship, calculating the variance explained, or comparing mean Y across values of X

In code

df <- tibble(w = rnorm(100)) %>%
  mutate(x = 2*w + rnorm(100)) %>%
  mutate(y = 1*x + 4*w + rnorm(100))
df <- df %>%
  mutate(x.resid = x - predict(lm(x~w)),
         y.resid = y - predict(lm(y~w)))
m1 <- lm(y~x, data = df)
m2 <- lm(y.resid ~ x.resid, data = df)
m3 <- lm(y~x+w, data = df)
msummary(list(m1,m2,m3), stars = TRUE, gof_omit = 'Adj|AIC|BIC|F|Lik')

In code

In Diagrams

• The relationship between X and Y reflects both X->Y and X<-W->Y
• We remove the part of X and Y that W explains to get rid of X<-W and W->Y, blocking X<-W->Y and leaving X->Y

Graphically

Intuitively

• So this is all about removing variation explanined by the control variable
• That’s why you hear some people refer to controlling as “holding W constant” - we literally remove the variation in W, leaving it “constant”
• Another way of thinking of it is that you’re looking for variation of X and Y within values of W - this is made clear in the animation
• Comparing apples to apples

An Example

• We’ll borrow an example from the Wooldridge econometrics textbook (data available in the wooldridge package)
• LaLonde (1986) is a study of whether a job training program improves earnings in 1978 (re78)
• Specifically, it has data on an experiment of assigning people to a job training program (data jtrain2)
• And also data on people who chose to participate in that program, or didn’t (data jtrain3)
• The goal of causal inference - do something to jtrain3 so it gives us the “correct” result from jtrain2

LaLonde

library(wooldridge)
#EXPERIMENT
data(jtrain2)
jtrain2 %>% group_by(train) %>% summarize(wage = mean(re78))

## # A tibble: 2 x 2
##   train  wage
## * <int> <dbl>
## 1     0  4.55
## 2     1  6.35

#BY CHOICE
data(jtrain3)
jtrain3 %>% group_by(train) %>% summarize(wage = mean(re78))

## # A tibble: 2 x 2
##   train  wage
## * <int> <dbl>
## 1     0 21.6 
## 2     1  6.35

Hmm…

• What back doors might the jtrain3 analysis be facing?
• People who need training want to get it but are likely to get lower wages anyway!

Apples to Apples

• The two data sets are looking at very different groups of people!

library(vtable)
sumtable(select(jtrain2,re75,re78), out = 'return')

##   Variable   N  Mean Std. Dev. Min Pctl. 25 Pctl. 75    Max
## 1     re75 445 1.377     3.151   0        0    1.221 25.142
## 2     re78 445 5.301     6.631   0        0    8.125 60.308

sumtable(select(jtrain3,re75,re78), out = 'return')

##   Variable    N   Mean Std. Dev. Min Pctl. 25 Pctl. 75     Max
## 1     re75 2675 17.851    13.878   0    7.605   25.584 156.653
## 2     re78 2675 20.502    15.633   0    9.243   28.816 121.174

Controlling

• We can’t measure “needs training” directly, but we can sort of control for it by limiting ourselves solely to the kind of people who need it - those who had low wages in 1975

## # A tibble: 2 x 2
##   train  wage
## * <int> <dbl>
## 1     0  4.55
## 2     1  6.35

## # A tibble: 2 x 2
##   train  wage
## * <int> <dbl>
## 1     0  5.62
## 2     1  6.00

Controlling

• Not exactly the same (not surprising - we were pretty arbitrary in how we controlled for need.tr, and we never closed train <- U -> wage, oh and we left out plenty of other back doors: race, age, etc.) but an improvement
• This is a demonstration of controlling by choosing a sample; we could also just control for 1975 wages

Controlling

Bad Controls

• So far so good - we have the concept of what it means to control and some ways we can do it, so we can get apples-to-apples comparisons
• But what should we control for?
• Everything, right? We want to make sure our comparison is as apple-y as possible!
• Well, no, not actually

Bad Controls

• Some controls can take you away from showing you the front door
• We already discussed how it’s not a good idea to block a front-door path.
• An increase in the price of cigarettes might improve your health, but not if we control for the number of cigarettes you smoke!

Bad Controls

• There is another kind of bad control - a collider
• Basically, if you’re listing out paths, and you see a path where the arrows collide by both pointing at the same variable, that path is already blocked
• Like this: X <- W -> C <- Z -> Y
• Note the -> C <-. Those arrow are colliding!
• If we control for the collider C, that path opens back up!

Colliders

• One kind of diagram (of many) where this might pop up:

Colliders

• How could this be?
• Because even if two variables cause the same thing (a -> m, b -> m), that doesn’t make them related. Your parents both caused your genetic makeup, that doesn’t make their genetics related. Knowing dad’s eye color tells you nothing about mom’s.
• But within given values of the collider, they ARE related. If you’re brown-eyed, then observing that your dad has blue eyes tells us that your mom is brown-eyed

Colliders

• So here, x <- a -> m <- b -> y is pre-blocked, no problem. a and b are unrelated, so no back door issue!
• Control for m and now a and b are related, back door path open.

Example

• You want to know if programming skills reduce your social skills
• So you go to a tech company and test all their employees on programming and social skills
• Let’s imagine that the truth is that programming skills and social skills are unrelated
• But you find a negative relationship! What gives?

Example

• Oops! By surveying only the tech company, you controlled for “works in a tech company”
• To do that, you need programming skills, social skills, or both! It’s a collider!

Example

set.seed(14233)
survey <- tibble(prog=rnorm(1000),social=rnorm(1000)) %>%
  mutate(hired = (prog + social > .25))
# Truth
m1 <- lm(social~prog, data = survey)

#Controlling by just surveying those hired
m2 <- lm(social~prog, data = survey %>% filter(hired == 1))

#Surveying everyone and controlling with our normal method
m3 <- lm(social ~ prog + hired, data = survey)

msummary(list(m1,m2,m3), stars = TRUE, gof_omit = 'Adj|BIC|AIC|Lik|F')

Example

Graphically

Colliders

• This doesn’t just create correlations from nothing, it can also distort causal effects that ARE there
• For example, did you know that height is UNrelated to basketball skill… among NBA players?

Colliders

• Sometimes, things can get real tricky
• In some cases, the same variable NEEDS to be controlled for to close a back door path, but it’s a collider on ANOTHER back door path!
• In those cases you just can’t identify the effect, at least not easily
• This pops up in estimates of the gender wage gap - example from Cunningham’s Mixtape: should you control for occupation when looking at gender discrimination in the labor market?

Colliders in the Gender Wage Gap

• We are interested in gender -> discrim -> wage; our treatment is gender -> discrim, the discrimination caused by your gender

Colliders in the Gender Wage Gap

• Front doors/Open back doors/Closed back doors
• gender -> discrim -> wage
• gender -> discrim -> occup -> wage
• discrim <- gender -> occup -> wage
• discrim <- gender -> occup <- abil -> wage
• gender -> discrim -> occup <- abil -> wage

Colliders in the Gender Wage Gap

• No occup control? Ignore nondiscriminatory reasons to choose different occupations by gender
• Control for occup? Open both back doors, create a correlation between abil and discrim where there wasn’t one
• And also close a FRONT door, gender -> discrim -> occup -> wage: discriminatory reasons for gender diffs in occup
• We actually can’t identify the effect we want in this diagram by controlling. It happens!
• Suggests this question goes beyond just controlling for stuff. Real research on this topic gets clever.

Next Time

• Perhaps one of the ways we could get at the problem is by isolating front doors instead of focusing on closing back doors
• Many common causal inference methods combine the two!
• Next time we’ll look at the concept of isolating a front door path, usually using “natural experiments”

Practice

• We want to know how X affects Y. Find all paths and make a list of what to adjust for to close all back doors

Practice Answers

• Front door paths: X -> Y, X -> E -> Y
• Back doors: X <- A -> Y, X <- B -> Y, X <- A <- U1 -> B -> Y, X <- B <- U1 -> A -> Y, X <- A <- C -> Y, X <- B <- U1 -> A <- C -> Y
• (X <- B <- U1 -> A <- C -> Y is pre-closed by a collider)
• We can close all back doors by adjusting for A and B.