Recently, physics professor Jed Macosko of Wake Forest University spoke to Mind Matters News about the way access to huge troves of data (Big Data) enables a variety of university ranking systems, depending on what matters to the prospective student. This is a far cry from traditional ranking systems like US News, which assume that all agree on the ranking criteria.

You’re a physics professor who co-founded a new college ranking system. How did you end up here?

I was born in 1972, in Minneapolis. My dad, Chris Macosko, who had a Ph.D. in Chemical Engineering from Princeton University, worked in Chemical Engineering and Materials Science at the University of Minnesota. There were four of us kids and we grew up in a neighborhood with a lot of other children, so it was always a lot of fun on our block.

So, your dad was a professor and now you are one, too…

My cousin Evan Macosko is a professor at the Broad Institute of MIT and Harvard who was featured in Technology Review’s “35 under 35” list. Other than that, it was just all the grad students, post-docs, and fellow faculty that my dad invited over to our house who added to my father’s influence. I didn’t start off life wanting to be a professor. I originally planned on being a cook. But a terrifying episode of Emergency!—in which a chef’s hair got stuck in a mixer and sleeve caught on fire—abruptly dispelled that notion. My dad’s job of lecturing and writing seemed a lot less dangerous! Of course, that initial change of plans, which probably happened when I was five or six, was followed by a long process of figuring out whether to follow exactly in my father’s footsteps or whether to try a slightly different field.

And along the way, you developed an interest in academic influence and how universities are ranked.

In the end, I got both my bachelor’s and Ph.D. in chemistry, the former at MIT and the latter at UC Berkeley. So, my job in the physics department at Wake Forest University, and not in a chemistry department, would seem a bit of a surprise if it wasn’t for the fact that my colleagues are very open-minded in what they consider the purview of physics.

Even my recent dive into Big Data and rankings is a form of physics since any mathematical model of something in the real world can be thought of as within the purview of physics. Our department at Wake Forest University, for example, has a rich history of seeing physics as a broad enterprise. Even now we are exploring an econophysics major as a potential option for our students so that they can take the mathematical models that have traditionally been used to describe the statistical mechanical nature of particles and apply those models to entities that are found in the financial sector. I will have to wait to talk about econophysics and its connections for another interview.

You’ve been described as a biophysicist and you do experiments. How does that connect with university rankings?

But Big Data connects with just about any experimental field these days. I learned about it from someone I met nearly twenty years ago. He got interested in Big Data because his Ph.D. project in entomology was so full of measurements that he maxed out the way people typically handled experimental results. Nowadays, nearly every project could fill up any available hard drive if you let it. So, the question is, “If I collect everything that could possibly be collected, will I be able to mine the data and find out something new?” The answer to that question is often, “No.” But when it comes to finding out which people and which universities have influenced our world the most over various time periods, the answer is a resounding “Yes!”

Is that how you got into rankings?

Yes, I have always been interested in thinking about what university is the best place for a given person. Even as a high school student, I struggled to know which college would be the best for me. Now that my own kids are making that decision, it’s on my mind all the time. There are plenty of rankings out there, but it wasn’t until I met Winston Ewert, a Ph.D. in computer science that I felt like there was a way to rank schools objectively.

How did Dr. Ewert manage that?

Well, he got started by working with Dr. Erik J. Larson, who was mining the words in Wikipedia to find out who was most influential about which topics. Basically, if a person’s name is only a few words before or after the location in the text where a topic is mentioned, that person must have something to do with that topic. By computing the average distance between certain names and topics, the most influential people end up being the ones with the shortest average distance. Dr. Ewert took the list of highly influential people and looked at where they were alumni, professors, and administrators. Those places, he figured, must be highly influential in the same areas as the people. Universities are only as strong as the people who work there and graduate from there, right?

That makes sense. So what do you do with this information?

We try to make these lists available to as many prospective students as possible. When I interviewed University of British Columbia president Santa Ono, he said, “There should be an infinite number of rankings. And one of the things I think is powerful is to be able to use impartial data, to use data which can be refreshed on a regular basis during a year, because things are changing all the time, something that makes use of publicly available information, and something that can be sort of customized.” We try to live up to that ideal.

Is there a danger in using only one ranking system?

Most people would say that they don’t use just one ranking. But in the end, that’s exactly what many people do. One person I interviewed, Jeff Stake, a law professor who in some ways “trolled” US News for their unhealthy rankings on his website called The Ranking Game, said that prospective law students would walk up to various law school tables set up at law school fairs, and in their hands would be the US News ranking issue, opened to the page that listed the law schools. They would look at the name of the law school, check their magazine, and then either walk away or stay and chat, depending on where the school’s ranking fell in their assessment of what schools they thought they could get into.

Why is it a problem if the whole world trusts US News?

Well, there are several problems with it. First, there’s the problem Prof. Stake addressed: the trust that the whole world puts on the US News’ rankings is misplaced. For example, he showed that if you use faculty–student ratio instead of student faculty–ratio, you get a different order in your school ranking.

Wait. You mean that dividing the number of students by the number of professors gives a different type of result than if you divide the number of professors by the number of students? How?

You’d think that since you are feeding in the same two columns of data, it shouldn’t matter which column gets divided by which. But it turns out that the way the math works out changes the rankings, or at least it did 15 years ago when Prof. Stake had his website running.

But even if you could fully trust that US News wasn’t really the “Ranking Game” that Prof. Stake showed it to be, there would still be the problem that President Ono pointed out: Things are changing all the time. A lot of the data that gets fed into the US News ranking, or any of the major ranking lists, is already three years old. So, I like the fact that Dr. Ewert uses Wikipedia and other data sources that are constantly updated.

So, two problems are that major rankings are susceptible to strange math quirks and that they use older data in a fast-changing society. What else?

An even bigger issue, and one the President Ono addressed later in our interview, is that “infinite” rankings is not just about getting the latest data but gearing it to what each student wants. Rankings, he said, “really should be based upon what the student cares about. Some students care about institutions, to have that teacher–scholar model, but also have Division One athletics. And so that ranking should incorporate that into the ranking.”

The prospective law students who showed up at Jeff Stake’s table at those law school fairs were really fooling themselves into going along with a ranking that didn’t take their considerations into account. It would have been far better if they could have entered in what they cared about most and then sorted their schools accordingly.

Is that what your AcademicInfluence.com group allows students to do?

That’s what we try to let them do. It’s not perfect, but it at least puts prospective students in the right mindset from the very start. As soon as someone comes across our site, they will see that there’s not just one ranking of undergraduate programs, but as many rankings as you could ever ask for. You can rank schools by discipline, by geographical area, by ease of admission, by rigor of incoming test scores, by cost, and many more ways. Not only that, but you can look specifically for influential professors in your field of interest that may have a connection to the schools you are considering. There are really a ton of ways to slice the pie at our site.

You use a lot of AI in your work. Tell us what it does for you.

I’ve heard it said that Big Data is to the Information Age as steel was to the Industrial Age. We rely on Big Data to power all the machine learning algorithms that generate the lists that we make available. At AcademicInfluence.com, our logo is an “A” shaped like a doorway with an “I” inside of it, shaped like a person entering that doorway, and the nickname we most often use to talk about our company is AI. Our logo and nickname both reflect the fact that there is intelligence inside of our algorithms and much of that intelligence would be considered artificial intelligence. For example, there is a tool on our website called the College Strategist that allows you to specify five things: your SAT, your ACT, your state, your intended major, and your gender. Our machine learning algorithms take the Big Data freely available on the internet (and other datasets that we purchase or build) and use them to suggest what your reach, target, and safety schools are likely to be. In the early days, we called this tool the “Daniel’s Dream” program because it seemed to be able to not only interpret your future choices but even to tell you what those choices were to begin with. People who had already compiled their list of reach, match, and safety schools were surprised by how close our list matched theirs, when they used our College Strategist tool.

So, given these drawbacks, what do you see happening to college rankings in the next few years?

I’m really not sure how much longer the US News rankings will hold sway. In many areas, such as in law school rankings, US News is still the only game in town. But in other areas, people have turned to ranking lists at a variety of “newcomer” sites, ours being one of them.

We haven’t even been talking about global college and university rankings. In that category, Wikipedia lists three main players and twelve “other global rankings”, again, ours being one of them. So, people already have exercised their right to look for rankings elsewhere.

I think it’s likely that the trend will continue and eventually people will get the ranking that is tailored for their needs. Having more and more data available certainly helps that process. We really have entered a new age where data has become a precious raw material that drives a lot of advances in our ability to do things, decide things, and find what we need.

You may also wish to read: How Erik Larson hit on a method for deciding who is influential. The author of The Myth of Artificial Intelligence decided to apply an algorithm to Wikipedia — but it had to be very specific.

and

The difference between influence and official power. Do you wonder why some people are listened to and not others, regardless of the value of their ideas? Well, read on…