Evaluation of RateMyProfessors.com and its implications

RateMyProfessors.comis a web site that allows university students to rate their professors on various metrics anonymously, and write comments about them as well.

This site has come to my attention about two years ago. I found that it is an interesting concept, offering raw unembellished input, and allowing students to have more say about their teachers.

I was thinking of a similar anonymous rating service for workplaces, something like, RateMyBoss, RateMyDepartment, RateMyCompany or even RateMyColleague!

Of course, there are more issues here than in a University or school setting, such as:

• The relationship can be more than the term or the year, and can span decades.
• Workplace stress is more than in a university, and friction between superiors and subordinates is more prevalent.
• There is also more potential for defamation, and libel, which brings the spectre of law suits.

Sociology Professor Ken Westhues, who teaches at the University of Waterloo, has written an article on the pros and cons of that web site and how this approach is a valid one, but to be taken with a grain of salt.

In a subsequent opinion column, Chet Robie, associate professor of management and organizational behavior at Wilfrid Laurier University commented on some parts of his article.

The Canadian school web site mentioned below is at ratemyteachers.ca.

The article was published in The Record, and reproduced here under fair use.

ratemyprofessors.com

Anonymous ratings of every professor in the country are on display for the whole world to see

KENNETH WESTHUES

(Sep 7, 2004)

How would you like a job where everybody you work with is allowed to rate your performance from 1 to 5, add a pungent comment about your strengths or weaknesses, and post it all on a public bulletin board?

I mean everybody, not just your boss and co-workers, but the people who report to you.

Postings are not required. The people who take time to write them tend to be those with strong feelings about you. They identify you by name, but are themselves anonymous.

One other thing: The bulletin board covered with their opinions about you is miraculously placed where the whole world can see, including your son, mother-in-law, ex-husband, nosy neighbour and prospective future employer.

This is the real world in which professors work today.

The reason is a website called ratemyprofessors.com, begun in 1999 by a whiz kid named John Swapceinski in California's Silicon Valley. Easily and quickly, anybody with web access can rate and comment on any professor in the United States or Canada. Results are available worldwide by a few clicks of a mouse.

The site has become a hit, especially in large, mass-market universities. It has grown exponentially from 11,000 ratings for 3,500 professors in 2001, to two million ratings for 400,000 professors in mid-2004.

Among all institutions on this continent, the University of Waterloo ranks sixth in popularity of this website, with 21,000 ratings for 1,500 faculty. Guelph has 8,000 ratings for 850 faculty. Wilfrid Laurier has 7,000 ratings for 650 faculty. For Conestoga College, there are 1,300 ratings for 200 faculty.

Generally, professors have tried to make the website go away. Many insist they have never looked at it. Others lambaste it in print media as biased, dangerous, frightening, unhelpful, worthless and of only entertainment value. A few try to sabotage it. About one professor a week threatens a lawsuit, according to Swapceinski.

But ratemyprofessors is unlikely to go away. It has flourished from the grass roots because it lets students do what was possible before only by washroom graffiti: say exactly what they think of professors they have had and get the lowdown in advance on other ones. To judge by current statistics, the website has passed a tipping point and will snowball further.

From my study of the website over the past year, here are five things that students, faculty, relatives, nosy neighbours and prospective employers should keep in mind.

First, since dimwits exist among students as well as faculty (possibly in greater proportion), a certain percentage of evaluations for any professor are noise. Maybe the harshest number five per cent, maybe the kindest 10 per cent. There is no way to tell which ratings come from bright, hard-working students and which ones from insolent slackers.

Second, multiple positive or negative appraisals of a professor may all have been posted by a single rater. Cookies and filters are in place to prevent this, but there are ways around them. A determined handful of malicious raters, or even just one, can put a professor's ratings in the cellar. One industrious groupie can send the ratings sky high.

Third, a rater need not have taken a course from the professor. A given evaluation may reflect an off-campus lover's adulation or an ex-lover's spite. There is no way to tell.

Fourth, institutions vary in how carefully comments are monitored. Swapceinski tries to have a site administrator on each campus to screen out the more imaginative hateful jibes that filters miss. Some of these local volunteers take their jobs seriously. Others do not. All are anonymous.

Fifth, despite all these possibilities of error and abuse, the bulk of postings to the site are the plain, unvarnished sentiments of ordinary students who have taken the courses identified and posted just one rating per course.

If I were a student, would I check a professor's page on ratemyprofessors before enrolling in his or her course? Sure. The website is an aid for choosing courses wisely. I would study the evaluations critically, aware that some may be ill-founded or fake. I would try to guess from the comments what kind of student the professor and course seem to satisfy, and see if I fit the bill.

While giving the website profile cautious respect, I would place more trust in the views of friends who have studied with the professor in question. I would also scrutinize course outlines.

In sum, I would treat the website as thousands of real students do, as one among several sources of information to help decide which courses to take.

At the same time, as a professor who has studied the website in detail, I oppose any formal use being made of it. I alerted the UW administration earlier this year to the unreliability of ratemyprofessors.com for any official purpose, despite the site administrators' diligent efforts to prevent abuse of it, and even though the vast majority of postings are honest and well-intentioned.

The risk is simply too high of making wrong decisions about professors' careers and livelihoods. Clever people abound in universities, and this website is too vulnerable to untruthful cleverness. In the case of a tenure candidate, for instance, the deck on this website can too easily be stacked for or against.

For decisions about tenure, promotion and salary, deans and department chairs should instead rely on course evaluations administered in class at the end of term. These are standard procedure at UW, as in most universities. The professor leaves the classroom when students fill out the forms. The results have more claim to validity than results on ratemyprofessors.

Even so, for most professors, the website results conform closely to those obtained on in-class questionnaires. The big difference is that the website is public and easily accessible. As its popularity grows, professors and administrators will find it steadily harder to ignore student opinion in decisions about professors' careers.

The unyielding presence of ratemyprofessors may also induce universities and colleges to publish the results of in-class questionnaires on their own websites. The University of Western Ontario already does this with numerical ratings, but without any comments.

Meanwhile, people in other lines of work should not gloat over the painfully public evaluation professors are now subjected to. The technology that spawned ratemyprofessors has endless possibilities.

Swapceinski himself runs a parallel site for teachers at lower levels. It includes hundreds of ratings for teachers in nearly all the high schools of Waterloo Region.

Ratemydoctor is next, Swapceinski says.

For better or worse, anybody identifiable by name in any occupation may soon be subject to evaluation on the web by anonymous fans and detractors. We might as well get used to it.

Kenneth Westhues is a sociology professor at the University of Waterloo and a winner of its distinguished teacher award.

Anonymous ratings may increase unethical behaviour

CHET ROBIE

(Sep 13, 2004)

I am writing in response to the Sept. 7 article by Kenneth Westhues regarding ratemyprofessor.com.

The headline, Anonymous Ratings Of Every Professor In The Country Are On Display For The Whole World To See, brings up points both for and against the technology.

There were several hidden assumptions in this piece, and perhaps a missing piece of data, that I would like to bring to public attention.

Hidden assumption No. 1: Students should be viewed strictly as customers.

Under this assumption, most, if not all, of what students want should be indulged. Should our institutions be simply student-driven? I think most of the cutting-edge thinkers in higher education now see students as a combination of product, partner, and customer, but not strictly as customer.

We are professionals who are being paid a lot of your hard-earned tax dollars to direct, plan, and ultimately -- we hope -- educate your youth. If we indulge every "order" that every student-interest group has without some vision of our own, we'll surely be lost.

Yes, we should always consider the opinions of students; no, we should not indulge every student want.

Hidden assumption No. 2: With absolute anonymity comes complete honesty.

Just as true, I would say, is the statement that with absolute anonymity comes a total disregard for humility.

I have perused some of the comments on ratemyprofessor.com and many of them can be quite nasty and downright hurtful. For example, one statement talks about the weight problem that a faculty member has -- clearly it is not job-related!

Just because a rating process becomes anonymous, doesn't mean that it suddenly is the panacea for a university.

A better approach would be to train all first-year university students on how to properly rate individuals on job-related dimensions and not fall prey to common errors, which include the halo effect (rating faculty highly on every performance dimension because he or she is "nice," because he or she brings candy to class or plays music in class or lets you out early) or lack of frame of reference (everybody has a different idea of what "displays interest and concern for students" means).

Missing piece of data: Ratemyprofessor.com could lead to unethical behaviour on the part of faculty members.

I recently completed a paper with a colleague for which we surveyed all Canadian business professors. One of the most striking findings was that untenured professors rated giving easy grades to avoid negative evaluations as being less unethical than did their tenured counterparts.

After having been recently tenured, I can understand the stress. Pre-tenure, in many institutions, you've got to teach like crazy, publish like mad and sit on all sorts of committees. A lot of pre-tenured faculty are looking for any way to cut corners. It appears from my survey that cutting the corners on giving negative evaluations is not keeping some of them up at night.

Well, let's add some more pressure to their lives, let's start taking what little control they have over their jobs and start putting some of those negative evaluations up on the web for everyone to see! Now there is really going to be a motivation to avoid negative evaluations.

I suggest you will find that the more people start putting grades online and the more people start using ratemyprofessor.com, and, God forbid, if administrative decisions are actually based on these dubious sources of information, you will also see increases in grade inflation.

We may have to do what the United Kingdom recently did -- increase the number of different types of A grades one could receive!

I believe Westhues is a realist. Online evaluations are here to stay. I hope, however, that we take a thoughtful approach to their use.

For example, policies on who sees the evaluations, how they are used, and student training on how to complete the evaluations would go a long way in making it a more credible source of information on how faculty contribute to the development of our most precious assets.

Chet Robie is an associate professor of management and organizational behaviour at Wilfrid Laurier University. Second opinion articles reflect the views of Record readers on a variety of subjects.