Classroom Poster Sessions: A win for you and your students

Last week, the Free Will & Science course finished up their poster sessions. It was one of the most enriching classroom experiences I’ve ever witnessed.† In case you’re interested, here’s a post about the why and how of classroom poster sessions — including templates for your own classroom. Continue reading Classroom Poster Sessions: A win for you and your students

Research Questions & Mental Shortcuts: A Warning

Daniel Kahneman talks extensively about how we make reasoning errors because we tend to use mental shortcuts. One mental shortcut is ‘substitution‘. Substitution is what we do when we (often unconsciously) answer an easier question than the one being asked. I find that I sometimes do this in my own research. For instance, when I set out to answer the question, “How can X be rational?” I sometimes end up answering easier questions like, “How does X work?”. In an effort to avoid such mistakes, I will (1) explain the question substitution error, (2) give an example of how we can distinguish between questions, (3) give a personal example of the substitution error, and (4) say what we can do about it.

1.  Substitution

In case you’re not familiar with Kahnemen’s notion of ‘substitution’, here is some clarification. In short, substitution is this: responding to a difficult question by (often unintentionally) answering a different, easier question. People use this mental shortcut all the time. Here are some everyday instances:

Difficult Question Easier Question
How satisfied are you with your life? What is my mood right now?
Should I believe what my parents believe? Can I believe what my parents believe?
What are the merits/demerits of that woman who is running for president? What do I remember people in my community saying about that woman?

For further discussion of mental shortcuts and substitution, see Part 1 of Kahneman’s Thinking Fast and Slow (2012).

Now, how does this mental shortcut apply to research?  Continue reading Research Questions & Mental Shortcuts: A Warning

Monitoring for Bias in Education: What Does That Mean?

“I would use the Department of Education … to monitor our institutions of higher education for extreme political bias and deny federal funding if it exists.”

Ben Carson

1. Everyone has biases, political and otherwise.

So denying funding on the basis of any political bias would be tantamount to denying all federal education funding. That’d be problematic. So — if we assume a charitable interpretation of Carson — that’s surely not the Republican plan (…or is it?). So let’s assume that Carson is not out to defund any educational institution that exhibits just any political bias.

Instead, maybe Carson’s plan is to monitor for particular biases. The idea here would be that only institutions with certain biases should be defunded. But even that would be problematic. After all, Carson is a human. And humans are more likely to notice and take issue with others’ biases (Corner et al 2012; Lord et al 1979) or biases that merely seem like others’ biases (Trouche et al 2015). So Carson might be more attuned to and dismissive of others’ biases than his own. And that is itself a political bias.

To overcome that bias, we would need to make sure that Continue reading Monitoring for Bias in Education: What Does That Mean?

On Grading (Or “How To Get An A”)

I’m a tough grader. But am I unreasonably tough? No. At least that’s the view I’ll arrive at in this post. 🙂 I’ll compare my grading to (a small and biased sample of) others’ grading, mention a few of my own grading experiences, summarize my grading philosophy, and present the basics of how my students can get an A.

1. Groundwork

First, my grade distributions often form bell curves centered in the low B range — sometimes the C range. So unless students’ work is exceptional, students tend to get a B or C. (NB: This is not a justification; it is a description.)

Second, I take myself to have been a pretty serious student. (Flashback: in high school, I used my free periods to write papers …not assigned papers. Just papers that I wanted to write.) With a few exceptions in my entire life, I have paid attention in class, done all the homework, and tried to actually learn the material. Nonetheless, I have definitely earned a few Bs and even Cs.

Yeah. Earned. 

I emphasize this because Continue reading On Grading (Or “How To Get An A”)

On Philosophy Ranking & Reporting: A Custom Solution?

Many academic philosophers and aspiring academic philosophers look to the Philosophical Gourmet Report (PGR) for philosophy ranking — i.e, to rank philosophy PhD programs. For many reasons, academic philosophers are becoming more vocal about their criticism of this philosophy ranking system (e.g., this recent paper in Metaphilosophy). In this post, I will propose a new system based on a variety of the common complaints and suggestions about philosophy’s existing ranking system. In the end, it should be clear that the proposed system could be (i) more useful and visually amenable than existing rankings, (ii) achievable, and (iii) generalizable to the broader academic community.


The complaints about the rankings are voluminous — what else would you expect from philosophers? In lieu of an outline of every blog post and every public statement, I provide a list of major themes that fall into three different categories: the practice of ranking, the current process of ranking, and the current leadership of the ranking.

Complaints About Ranking

  1. Rankings might misrepresent the magnitude of the differences between departments.
  2. Rankings might indicate a false sense of hierarchy and/or prestige.
  3. Ordinal lists just aren’t that informative.

Complaints About Process

  1. The PGR rankings are based on a fixed set of variables, so the rankings are not useful to those who wish to compare departments according to a different sets of variables.
  2. The PGR rankings aren’t representative since very few philosophers are invited to weigh in.
  3. The current process of the PGR rankings relies on qualitative reports, which makes for an unnecessarily large margin for error or dispute.

Complaints About People

  1. The selection of the board is concerning.
  2. The primary organizer and editor of the PGR is a subject of concern for a non-negligible portion of philosophers.

I think that my proposal can address many of these concerns. I will try to explain and visualize my proposal below. But first, let’s take a step back.


Let’s imagine that philosophers never had the PGR or any other ranking system. Now imagine you are commissioned to create a tool that philosophers would use to analyze, compare, and maybe even rank their various academic departments. Before you start the project, you ask yourself: what kind of information would academics (and aspiring academics) want to know about their field? No doubt, academics will disagree about what information they want, in general. But more importantly, they will disagree about how much weight should be assigned to each bit of information. This latter disagreement is the central problem for your project, as I see it.


To address that problem you need a reporting tool that features a customizable set of variables. That is, users need to be able to select the variable(s) they care about, weight them the way they see fit, and ignore whatever variables they want to ignore. If people can do this, then there is no longer a problem with people disagreeing about which information matters or which information matters most. With a tool that allows users to select and weight variables, a community with a diverse set of ideals can get individually suited rankings from one and the same dataset.


Also, based on some of the complaints above, you might want a tool that relies on quantitative metrics about, say, publications, citations, areas of specialization (AOS), demographic information, etc. Gathering quantitative information will help address concerns about purely qualitative data.

Getting this information shouldn’t be too difficult. One option is to have departments submit this information. Many departments already have to report certain information to their own institution. Reporting it again needn’t be an enormous hassle. Although the information you want in your database might go beyond the basic information that most departments keep tabs on. Another option is to have professional and aspiring academics create their own profiles and maintain their own metrics. Once the database has enough users from each department, meaningful reports could be made. I find this second option less attractive since it is based on the continued participation of lots of busy people — think about how many academics don’t have or maintain Google Scholar, ResearchGate, or profiles and who relatively few philosophers have or maintain PhilPapers profiles. A third option is to have bots trolling the web for the information, a.k.a., the Google approach. This third option is probably unrealistic for anyone other than Google. And even if it was realistic, the output would be hopelessly flawed without perpetual (human) monitoring and correction.


Once you decide on how to get your information, you compile it into a single dataset and store it on a server. Now, all you need is a friendly web-based user interface that allows for visualizations of your data. This is the part where you might want some outside help — unless you’re a computer scientist, of course. More than that, to avoid some of the complaints above, you might want an independent party to manage things anyway. Fortunately, there isn’t a shortage of competent web developers who could easily pull this off.


With your fancy new website, users can produce reports, comparisons, and rankings based on all sorts of variables and sets of variables. For example, maybe an aspiring grad student wants to see which departments would be a good fit based on her interests in metaphysics. So, the student searches for the departments with the most metaphysicians — metaphysicists? To get a closer look at each department, she can generate pie charts representing their distribution of specializations:

And maybe the aspiring grad student has certain criteria in mind when it comes to choosing a graduate program. So, she selects a few departments that have attracted her attention. Then she selects a few metrics by which to compare these departments.

And maybe that last report did a poor job of capturing the big picture, so she opts to turn her comparison into a ranking (the variables of which would be treated equally by default, but could also be weighted by the user).


Now imagine how other academics could use this tool. Department chairs could look at longitudinal reports of various metrics in their department. They could even compare their department to departments they admire. Hiring committees could produce reports of their department’s areas of specialization, areas of publication, and areas of citation to reveal areas that are underrepresented in the department or to identify a departmental strength. This might help make the decision about which AOS to highlight in their next job post. You might be thinking of other ways to use the tool. The point is that this tool would serve a variety of purposes — certainly more purposes than the existing rankings.

Even Bigger Picture

Before we close, I want to ask you to briefly imagine how this tool could be used by an even larger audience. Imagine the database includes all academics fields across all academic institutions. A single website could serve as a one-stop reporting tool for all kinds of purposes. This would allow for reports, comparisons, and rankings to be made across departments, fields of study, institutions, etc. If such a tool existed, and it were kept up to date, then it would easily generate enough traffic to fund itself with advertising income — come to think of it, I wonder if Google, or US News would be interested in this kind of project. …or maybe ResearchGate is already working on this.


This tool offers lots of advantages. Consider the following:

  1. Easy to maintain. The database could be a single spreadsheet. When new information is received, the spreadsheet is updated. The rest of the system, once it’s built, remains unchanged.
  2. Easy to use. The visual reports would allow for much more nuanced and meaningful reports than static lists.
  3. Customizable. People with various interests, criteria, and goals could produce optimally useful reports based on the same data by selecting different variables.
  4. Broadly useful. Professional and aspiring academics would find it helpful; even para-academic institutions and non-academics might find it useful. And because it’s not a static ranking, even the people who are against rankings could find a use for it.
  5. Expandable. With few changes, the tool could be used not just by a single academic field, but by the larger academic community.

Oh, and if you were wondering: I’m definitely not the first one to make this kind of proposal. Noëlle McAfee and Robert Vallier have made similar proposals. I got part of this idea from Noëlle. I hadn’t come across Robert’s idea until I was writing this. I wouldn’t be surprised if others have made similar, or better, proposals.

Grad School | Part 5: Contingency plans

You’re trying to figure out whether or not you want to go to grad school. You’ve tried to estimate the value of a PhD in philosophy (Part 1). You’ve considered academic jobs (Part 2). And you’ve considered the nuts and bolts of grad school (Part 3) and the pros and cons of grad school (Part 4). Now it’s time to figure out what to do if — after starting grad school — you find yourself no longer wanting the academic life. It’s time to talk grad school contingency plans.

Part 1 | Part 2 | Part 3 | Part 4 | …

1. Disillusionment

Sounds exciting, right? Hear me out.

In just a few years, I have encountered many grad students who Continue reading Grad School | Part 5: Contingency plans

Grad School | Part 4: What’s Good And Bad About Grad School?

Prior to this post I argued that the value of a PhD is not in its job prospects …or lack thereof (Part 1). I showed that desirable academic jobs are neither ideal or common and that most academic jobs are very undesirable: they pay very little, they expire as frequently as every semester, and they offer no health insurance (Part 2). Then you found out about how most US philosophy PhD programs work (Part 3). If you are considering getting a PhD in philosophy, then you’ll want to have a realistic view of the process. This post attempts to provide such a view. It covers two things:

Part 1 | Part 2 | Part 3 | … | Part 5

1. What’s So Great About Grad School?

Even on a mediocre day, I can honestly say that I am living the dream! Really, there’s a lot to be grateful for in terms of being a grad student in philosophy.

1.1 Admission

Just being admitted to grad school Continue reading Grad School | Part 4: What’s Good And Bad About Grad School?