Nonrespondent law graduates and other sampling questions
“[Paul] Campos argues that low-earning lawyers may be less likely to participate in SIPP in the first place because of the stigma involved in admitting that, even anonymously.”
By email, Jerry Organ asks related questions about the representativeness of our sample.
“SIPP” is the United States Census Bureau’s Survey of Income and Program Participation, and is one of the primary data sources used in The Economic Value of a Law Degree. Campos worries about stigma and non-response. Thankfully SIPP is specifically designed to deal with these problems and to include impoverished and stigmatized members of the population, including those who receive government aid.
The Census Bureau explains SIPP’s purpose as follows:
”To collect source and amount of income, labor force information, program participation and eligibility data, and general demographic characteristics to measure the effectiveness of existing federal, state, and local programs; to estimate future costs and coverage for government programs, such as food stamps; and to provide improved statistics on the distribution of income and measures of economic well-being in the country.”
The Census Bureau elaborates on the use of SIPP to analyze participation in Food Stamps and other anti-poverty programs here.
Census explains in greater detail how SIPP handles issues related to response bias, non-response bias, and weighting here. SIPP oversamples in poor neighborhoods, imputes when necessary, and adjust the sample weights to approach a nationally representative sample.
It is about a good a survey as one is likely to find conducted by people who care a great deal about nonresponse and accurate estimates.
Additionally, to the extent that any lingering nonresponse bias may cause those with low earnings to be less inclined to participate, this bias will affect both law graduates and bachelor’s degree holders. What we measure in the Economic Value of a Law Degree is the earnings premium, or difference in earnings that is attributable to the law degree. The biases should wash out, or more likely, bias down our estimates of the law degree earnings premium, because bachelors are far more likely than law graduates to live in poverty.
Indeed studies that have compared earnings reported in SIPP to earnings from administrative data (tax and social security administration data) find that SIPP data underestimates earnings premiums because more highly educated and higher income individuals tend to underreport earnings, while less educated and lower income individuals tend to over report. We make no attempt to correct for this downward bias in our earnings premium estimates to offset any lingering selection on unobservables.
Individual response bias issues also won’t affect federal student loan default data, which is administrative data from the Department of Education. As noted in the article and in previous blog posts, former law students default on their student loans much less frequently than former students of bachelor’s degree or other graduate degree programs.