Spring 2005
STAT 718 - Statistical
Rating Methods
Wednesday 10:10-11:00
Instructor: Brian Habing
Office: 203 LeConte
e-mail: habing@stat.sc.edu
MSN messenger: brian@habing.com
AOL or Yahoo messenger: DrStatpsy
Office Phone: (803)777-3578
Home Phone: (803)739-2686 (9am to 10pm only)
Office Hours: Whenever the door is open, by appointment, and
Tuesday & Thursday 9:00-11:00
Website: http://www.stat.sc.edu/~habing/courses/718S05.html
Bulletin Description: STAT 718 – Statistical Rating Methods. (1) (Prereq: Consent of Instructor) Statistical rating and ranking methods using factor analytic, paired comparison, and least squares based models. Applications to the ratings of products, universities, and movies; focusing on ranking college football teams.
Purpose of the Course: This one hour course is designed to introduce graduate students with some experience in mathematical statistics (at the level of STAT 513, STAT 703, or STAT 712) an introduction to the wide-spread, but little studied, use of statistical models to rank and rate various entities. The large number of these methods used in collegiate football provides an excellent opportunity to examine many of the fundamental concepts. A student finishing the course should have a general understanding of the use of factor analytic, paired comparison, and least squares based ranking and rating method. They will also have further developed their ability to read and present technical material and program statistical methods.
Expectations: All students are expected to:
Coursework and Grading: The assignments for the course consist of a presentation on an assigned topics made during the semester and a final group project where a rating/ranking system is defined and implemented. The construction of the rating/ranking system and programming may be done in groups approved by the instructor, but the method’s justification, description, and demonstration must be made individually. Students are also expected to come prepared for class and actively participate in the class discussions.
Class 1 – An overview of various ratings that are commonly seen in the media, and discussion of how we interpret them as consumers.
Class 2 – Introduction to Factor Analysis and Principal Components
Class 3 - Student presentation of implementing factor analysis using R or SAS. Student presentation of an example in marketing.
Class 4 – Introduction to the ELO model for chess and the Terry-Bradley paired comparison model.
Class 5 - Overview of the statistical issues in implementing the ELO and Terry-Bradley models, and discussion of the models weaknesses.
Class 6 – Student presentation on implementing the Terry-Bradley model using R or SAS. Student presentation of an example in a recent round-robin tournament.
Class 7 – Student presentation of least squares multiple regression. Discussion of expanding this model to a rating setting.
Class 8 – Technical issues in the least squares rating method.
Class 9 – Discussion of student attempts at programming least squares rating models.
Class 10 – Student presentations of the paired comparison and least squares models currently used to model college football.
Class 11 – Student presentations of combined paired comparison / least squares models.
Class 12 – In depth look at the mathematics behind implementing combined paired comparison /least squares models.
Class 13 – Student presentations of the rating methodologies used to rate universities and movies.
Class 14 – Discussion of issues being encountered in the students attempts to implement their own rating systems.
Technical References:
Agresti, A. (1990). Categorical Data Analysis. New York: John Wiley & Sons.
Hatcher, L. (1994). A Step-by-Step Approach to Using the SAS System for Factor Analysis and Structural Equations Modeling. Cary, NC: SAS Institute Inc.
Lange, K. (1998). Numerical Analysis for Statisticians. New York: Springer-Verlag.
Neter, J., Kutner, M.H., Nachtsheim, C.J., & Wasserman, W. Applied Linear Statistical Models. Boston: McGraw-Hill.
Sharma, S. (1996). Applied Multivariate Techniques. New York: John Wiley & Sons.
Venables, W.N. & Ripley, B.D. (2002). Modern Applied Statistics with S. New York: Springer Verlag.
On-line Rating Sites:
The Internet Movie Database - http://www.imdb.com
Massey Ratings - http://www.mratings.com/
America’s Best Colleges 2005 - http://www.usnews.com/usnews/edu/college/rankings/rankindex_brief.php
Sports Rating Articles:
Harville, David (2002), ``College football: a
modified least squares approach to rating and prediction'', ASA Proceedings
of the Joint Statistical Meetings, 1383-1390
Stern, Hal (1992),
``Who's number one? -- Rating football teams'', ASA Proceedings of the Section
on Statistics in Sports, 1-6
Harville, David A. (1978), ``Football ratings and predictions via linear models (Com: p87-88)'', ASA Proceedings of the Social Statistics Section, 74-82
Marcus, David J. (2001), ``New table-tennis rating system'', The Statistician, 50 (2) , 191-208
Stefani, Raymond T. (2000), ``A systems overview of sports ratings and rankings'', ASA Proceedings of the Section on Statistics in Sports, 64-69
Knorr-Held, Leonhard (2000), ``Dynamic rating of sports teams'', The Statistician, 49 (2) , 261-276
Bennett, Jay (1999), ``ACE: A multidimensional rating for starting pitchers'', ASA Proceedings of the Section on Statistics in Sports, 1-7
Glickman, Mark , and Jones, Albyn C. (1998), ``The United States Chess Federation rating system: Current issues and recent developments'', ASA Proceedings of the Section on Statistics in Sports, 22-27
Bassett, Gilbert W., Jr (1997), ``Robust sports ratings based on least absolute errors'', The American Statistician, 51 , 99-105
Stefani, Raymond T. (1997), ``Survey of the major world sports rating systems'', Journal of Applied Statistics, 24 , 635-646
Carlin, Bradley P. (1996), ``Improved NCAA basketball tournament modeling via point spread and team strength information'', The American Statistician, 50 , 39-43
Glickman, Mark E. (1995), ``Becoming a chess master -- The development of a rating system for tournament chess players (Disc: p14-15)'', ASA Proceedings of the Section on Statistics in Sports, 6-13
Lebovic, James H. , and Sigelman, Lee (2001), ``The forecasting accuracy and determinants of football rankings'', International Journal of Forecasting, 17 (1) , 105-120
Hal S. Stern, "Statistics and the College Football Championship,"
The American Statistician, Vol. 58, No. 3, August 2004 pp.179-185 (Disc/R: pp. 185-195)
A State-Space Model for National Football League Scores (in Applications and Case Studies), Mark E. Glickman; Hal S. Stern, Journal of the American Statistical Association, Vol. 93, No. 441. (Mar., 1998), pp. 25-35.
Trying Out for the Team: Do Exhibitions Matter? Evidence From the National Football League (in Statistics in Sports), Lee A. Craig; Alastair R. Hall, Journal of the American Statistical Association, Vol. 89, No. 427. (Sep., 1994), pp. 1091-1099.
On the Probability of Winning a Football Game, Hal Stern, The American Statistician, Vol. 45, No. 3. (Aug., 1991), pp. 179-183.
Predictions for National Football League Games Via Linear-Model Methodology (in Applications), David Harville, Journal of the American Statistical Association, Vol. 75, No. 371. (Sep., 1980), pp. 516-524.
The Use of Linear-Model Methodology to Rate High School or College Football Teams (in Applications), David Harville, Journal of the American Statistical Association, Vol. 72, No. 358. (Jun., 1977), pp. 278-289.
Ranking Teams in a League
with Two Divisions of t Teams, Richard A. Groeneveld, The American
Statistician, Vol. 44, No. 4. (Nov., 1990), pp. 277-281.
The Rating of Players in Racquetball Tournaments, David Strauss; Barry C. Arnold, Applied Statistics, Vol. 36, No. 2. (1987), pp. 163-173.
Harville, David A. (2000), ``The selection and/or seeding of college basketball or football teams for postseason competition: A statistician's perspective'', ASA Proceedings of the Section on Statistics in Sports, 1-18
Smith, Tyler , and Schwertman, Neil C. (1999), ``Can the NCAA basketball tournament seeding be used to predict margin of victory?'', The American Statistician, 53 , 94-98
Carlin, Bradley P. (1996), ``Improved NCAA basketball tournament modeling via point spread and team strength information'', The American Statistician, 50 , 39-43
Rothman, David (2002), ``My contribution to the BCS: yes, Virginia, there is a social welfare function in college football'', ASA Proceedings of the Joint Statistical Meetings, 2990-
Carlin, Bradley P. , and Stern, Hal S. (1999), ``Designing a college football playoff system'', Chance, New Directions for Statistics and Computers, 12 (3) , 21-26
Wright, Daniel B. (1997), ``Football standings and measurement levels'', The Statistician, 46 , 105-110
Samuelson, Andy , and Samuelson, Douglas A. (1996), ``Will the new bowl structure settle picking the number 1 college football team?'', ASA Proceedings of the Section on Statistics in Sports, 28-33
Samuelson, Andy , and Samuelson, Douglas A. (1995), ``How would a chess coach choose the number 1 college football team?'', ASA Proceedings of the Section on Statistics in Sports, 45-49
Berry, Scott, M. (2003), "College Football Rankings: The BCS and the CLT," Chance, 16(2), 46-49