Wednesday, 18 March 2015

Gender and Prestige Bias in Philosophy Job Hiring

1. Introduction: Of the various issues in the philosophy profession, it is widely agreed that the most serious issue concerns the job market and job hiring, and the problems facing those who fail to get an academic position, such as a research-only post-doctoral position or a tenure-track position. A post-doc means several years of pure research opportunity, with no teaching or administration obligations. A tenure-track position brings job security, along with some teaching responsibilities, but these are normally reduced for the initial period and only in one's specialized areas.

Those unable to obtain either a tenure-track position or a post-doc often end up in adjunct positions, with no job security, poor pay and exploitative teaching workloads: e.g., teaching four or five different courses each semester. Adjunct and temporary teaching positions are often for a few months or for a year, and therefore the person (and their family) may have to move from one location/town to another to find work. For many in this position---even some having a strong publication list---there is no escape and it can be career-ending. A few survive this route and are able to catch up with the lucky ones that got post-docs or tenure-track positions.

2. Data on philosophy job hiring: Data was collected a couple of years ago by Dr Carolyn Dicey Jennings, who placed the raw data in a number of spreadsheets. The dataset used for this blogpost used to be here and concerns philosophy job hiring in 2012 and 2013. This data now seems to have been moved and replaced by data pertaining to 2014 (this has not been examined).

3. Data Analysis:[*] The data examined relates to:
The publication patterns and prestige standing of their PhD granting department for the cohort of those hired during 2012 and 2013 into tenure-track and post-doc positions from no prior position (those with prior positions were filtered out).
This data reveals two statistical biases and an intriguing negative correlation:
  • Women hired had published less than men did, in fact about half as much.
  • Having a prestigious background---for example, a PhD granted by a high Leiter-ranked department---benefits jobseekers.
  • Prestige and publication rates are negatively correlated: those hired from high prestige departments had lower publication averages.
The data therefore suggests two statistical biases in philosophy job hiring: a gender bias towards women and a prestige bias against applicants from departments with lower prestige ranking. Below the data is summarized in the two main charts below. As mentioned, all the data refers to the cohort of those hired in the job hiring rounds of 2012 and 2013, from no prior position.

Some further analysis of the statistical significance of these difference is also given below.

4. Distributions of hired applicants with respect to number of publications: The first chart below shows the proportions of those hired against their number-of-publications, with two data series (one for women, one for men). The total sample sizes are as follows
Total sample size = 311.
Number of men = 211.
Number of women = 100.
These are not tiny samples, and are large enough to draw reasonably valid conclusions. Here is a brief summary:

The first main finding is that, by and large, men publish more than women do:
  • The average publication rate for women hired was about 0.8. 
  • The median number of publications for a woman hired was 0. 
  • The average publication rate for men hired was about 1.5. 
  • The median number of publications for a man hired was 1. 
The chart below shows the proportion of those hired as a function of their number of publications. The relevant data is this:

In particular, the column for "0 Publications" shows that a majority (54%) of women hired had no publications, as compared with 40% of men.

This is Chart 1:

For comparison, the publication data for the whole cohort hired in 2012 and 2013, without filtering for priors is as follows (N = 498 hired):

As can be seen, the mean is 1.86 publications per hired applicant. Naturally this is higher than the means observed for applicants hired from no prior positions.

5. Distributions of hired applicants with respect to number of top 15 publications: One can redo the calculations based on the highest ranking philosophy journals. In September 2011, The Brooks Blog reported the top 15 Philosophy journals, as follows: Journal of Philosophy, Philosophical Review, Philosophy & Phenomenological Research, Nous, Mind, Ethics, Philosophical Studies, Synthese, Philosophy & Public Affairs, Analysis, Philosophical Quarterly, American Philosophical Quarterly, Philosophers' Imprint, Monist, Canadian Journal of Philosophy.

The results for publication rates in the Top 15 journals are even more striking. Here is the data table:

For the Top 15 journals, 27% of men hired had at least one such publication, while only 11% of women hired had at least one. For these journals, the average publication rate for men hired was 0.42 publications, while for women hired it was only 0.14 publications.

6. Statistical Significance: Are these differences statically significant? We test the null hypothesis: “there is no gender/being-published association”. I.e., that the probability of being unpublished is independent of gender. The 2x2 contingency table is this:

We use a chi-squared test with Yates correction and (two-tailed) P value. There is one degree of freedom. The results are:

This means that we reject the null hypothesis at P = 0.0316. The association between being (un-)published and gender is statistically significant.

The reader may check this for themself at this website, and here is the online computation:

A similar calculation restricted to publications in the Top 15 journals yields even stronger conclusions. Here is the online calculation, which the reader can check for themself:

The null hypothesis ("no correlation") is rejected now with P-value around 0.001. In other words, if the null hypothesis is true, the probability of seeing this observed data is tiny: 0.1%.

7. Prestige Bands: Next we examined the effect of prestige on publication averages. The Leiter PGR 2014 rankings were used to assign prestige bands to 94 Philosophy departments, based on the PGR mean scores, which run from 0 up to 5.0. These were coded as "Leiter prestige bands" A to F follows:
A: 4.5-5.0
B: 4.0-4.4
C: 3.5-3.9
D: 3.0-3.4
E: 2.5-2.9
F: 2.0-2.4
G+U: 0.0-1.9
(The lowest band is G+U, meaning "G" + "unclassified". It is not clear what the data in this band reveals; it may be unreliable because so many lowly departments are unclassified and hires also do not get announced prominently.)

There are 22 departments in the high prestige bands A-C:


These bands A-C were aggregated and called "Alpha". The bands D-F were then aggregated and called "Beta". I.e., "Alpha" stands for departments with PGR means 3.5-5.0, and "Beta" stands for those with PGR means of 2.0-3.4. (As noted, data relating to the G+U band may be unreliable)

8. Prestige Groups and Publication Averages: One can compute the average number of publications for the bands listed (grouped by gender):


Excluding G+U band, both data series (men and women) show a negative correlation between prestige and average publication rate:
Pearson correlation for prestige vs publication average: r = -0.3 (men)
Pearson correlation for prestige vs publication average: r = -0.4 (women)
One can also compute the publication averages for the whole cohort for the seven bands. The results are:

The G+U band complicates this. Removing this band again shows a negative correlation (r = -0.2). However, including this band, shows a positive correlation (r = 0.2).

Finally, one can compute the publication averages for "Alphas" and "Betas". These figures are:
Alphas: 1.09
Betas: 1.55
Here again, publication rate and prestige are negatively correlated, with lower prestige applicants hired having published more than higher prestige applicants hired (but note the complicating G+U band).

9. Gender-Prestige Groups and Publication Averages: Combining the two prestige bands, "Alpha" and "Beta", with two gender groups, "Male" and "Female", one obtains four Gender-Prestige groups amongst those hired. For each Gender-Prestige group amongst those hired from no prior position, one can compute their average number of publications. The results are:
Beta-males: 1.7
Alpha-males: 1.3
Beta-females: 1.1
Alpha-females: 0.6
Below is Chart 2, along with the sample sizes:

This rather vividly indicates that prestige negatively correlates with average publication rate on being hired: less prestigious hires have published more (in both gender groups, "Alphas" publish less than "Betas").

10. Prestige/Gender Boosts: Finally, can we in some way quantify these biases? There is no simple statistical measure of the degree of prestige or gender "boost" involved in job hiring from no prior position, other than mere taking the ratio of the publication averages involved:
Prestige boost = (av. pub rate for betas)/(av. pub rate for alphas) = 1.55/1.09
Gender boost = (av. pub rate for men)/(av. pub rate for women) = 1.45/0.81
The results are:
Prestige boost = 1.4
Gender boost = 1.8
These "boosts" are rough indicators of the extent that one currently "gains" on the job market from either higher prestige or being female. So, the prestige boost of 1.4 indicates that, on average, a Beta prestige applicant needs to publish 40% more than an Alpha prestige applicant does, to reach the same position. Similarly, the gender boost of 1.8 indicates that, on average, a male applicant needs to publish 80% more than a female applicant does, to reach the same position.

11. Summary: The statistical findings, at least as far as philosophy job hiring in 2012 and 2013 were concerned, indicate the existence of both prestige and gender bias in philosophy job hiring:
  • Against lower prestige male applicants.
  • For high prestige female applicants.
As noted above, the correlation amongst those hired between being unpublished and gender is statistically significant.

12. Discussion: The first point to make is that the indicator of academic merit used here is based on an applicant's number of publications on being hired into their first academic post, from no prior position. One might perhaps try to argue that this is too crude an indicator, and that there are other factors relevant to merit, which may even outweigh a good publication record (transcript; reference letters; submitted work; research-talk; teaching experience). However, to try and argue that publication count is not a major indicator of academic merit is not a reasonable position to take.

The existence of prestige bias---hiring preference for higher prestige applicants---may in itself not necessarily reflect a non-meritocratic bias. It is prima facie plausible that higher prestige does correlate with merit, given that graduate selection/admissions is itself, or aims to be, meritocratic in structure. But how does prestige correlate with merit/ability? How does the prestige rank of one's PhD environment correlate with other crucial factors, such as having access to influential mentors, advisers and letter-writers, who can not only provide very high level research help and guidance, but also bring access to the relevant "network" of inter-linked people in the profession? Women from lower prestige departments have published at roughly double the rate (mean of 1.1) as women from high prestige ones (mean of 0.6). Are high prestige women really that much more gifted? The "prestige boost" given above for the whole cohort is 1.4.

The negative correlation of prestige and publication average is interesting. Perhaps high prestige graduate students are more complacent about finding a position, while lower prestige graduate students know they may have to work even harder, and publish, to catch up with those at more prestigious departments. This explanation is, however, highly speculative.

The existence of gender bias---hiring preference for women over men---is, however, not accounted for as a proxy for merit, unless one assumes that being a women correlates with higher ability. There are several other explanations for the pro-female gender bias in junior job hiring. First, perhaps women are given preferential treatment qua women. In other words, held to lower academic expectations than men are, much as there is sometimes "racism of low expectations". One might call this "White Knight" discrimination (though the discriminators are both male and female). Second, perhaps women are less inclined to submit work for publication for other contextual reasons (e.g., perhaps women are more risk-averse, or less confident). That said, the "gender boost" towards women, for the whole cohort, is higher than the "prestige boost".

Finally, it is worth pointing out that affirmative action in employment matters, in connection with a protected characteristic, is in any case illegal. Employers must hire on the basis of merit, and not gender. In the United States, such a practice would contravene Title VII of the Civil Rights Act (1991). In the United Kingdom, it contravenes The Equality Act 2010.

[*: The statistics for those hired from prior positions was filtered out, using the data marked by Prof. Jennings as "1 = priors", indicating, it is assumed, those who had held a prior academic position. Some of the findings were previously published at a philosophy "metablog". Those were passed on in comments by a commenter at Prof. Brian Leiter's blog. As with almost any dataset, there is an occasional error. However, the data has been checked twice for errors, and the overall findings seem to be reasonably robust under corrections of known errors and small variations in the prestige rankings of the departments. If a reader notes that some data has been incorrectly transcribed from Prof. Jennings' 2012/2013 spreadsheet, they might mention this at Prof. Leiter's blog.]