Rubrics and Authentic Assessment


Since its initial construction in 2002, the goal of this website has continued to be to share Authentic Assessment tools and Rubrics, to describe the assessment techniques that I have adopted, and to provide learning and assessment resources designed especially with these objectives in mind. Also, the educational goals of this site include providing instructors with ideas and tools for enhancing critical thinking, encouraging both self-esteem and the acceptance of others, and improving interpersonal effectiveness.

The materials and information on this Website are divided into the following five distinct sections:

    Curriculum Assessment: Development and Design
    Instruction Assessment: Practices for Teachers
    Student Assessment: Process and Execution
    Technology Assessment: Practices for Integration and Literacy
    Rubrics and other Authentic Assessment Resources



Bibliography

In addtion to providing a list of, and links to the source materials for this Website, this bibliography is designed to provide additional information about the research articles and tools that continue to be gathered and archived for future research. Hopefully you will find them useful in assessing your teaching, student assessment, instructional technology assessment, program evaluation, and conducting educational research. Resources related to data gathering methods, the analysis and reporting of data, and best practices for assessment, evaluation, teaching, and using instructional technology are also included.

For specific research information about in elearning or educational games, please visit my distance learning curriculum design, or my learning game environments websites.


Assess teaching

Andrade, H. G. 2000. Using rubrics to promote thinking and learning. Educational Leadership 57 (5): 13–18.

Anderson, L.W., & Krathwohl (Eds.). (2010).  A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. New York: Longman.

Arbaugh, J. B., Cleveland-Innes, M., Diaz, S. R., Garrison, D. R., Ice, P., Richardson, J. C., et al. (2008). Developing a community of inquiry instrument: Testing a measure of the Community of Inquiry framework using a multi-institutional sample. The Internet and Higher Education, 11(3-4), 133-136.

Asmus, E. P. (2002). Instrumental Music Rubric. January 2002 Florida Music Educators Association In-Service Meeting in Tampa, Florida

Authentic Education. (2011). What is Understanding by Design? Retrieved from http://www.authenticeducation.org/ubd/ubd.lasso

Bailey, K. M. (1998). Learning about language assessment: dilemmas, decisions, and directions. US: Heinle & Heinle.

Bangert, A. W. (2009). Building a validity argument for the community of inquiry survey instrument. Internet and Higher Education, 12(2), 104-111.

Banta, T., Lund, J., Black, K., & Oblander, F. (1996). Assessment in practice: Putting principles to work on college campuses. San Francisco, CA: Jossey-Bass.

Bell, A. (2008). Serious Games Rubric. University of Wisconsin-Stout.  Retrieved from: http://www.uwstout.edu/soe/profdev/rubrics.cfm

Bennett, R. E. (1997). Reinventing assessment: Speculations on the future of large-scale educational testing. ETS (Educational Testing Service). Policy Information Center: Princeton, NJ.

Bente, G. & Breuer, J. (2009). Making the implicit explicit: embedded measurement in serious games, in Serious Games: Mechanisms and Effects, U. Ritterfield, M. J. Cody, and P. Vorderer, Eds., pp. 322–343, Routledge, New York, NY, USA, 2009.

Binsubaih, A., Maddock, S., & Romano, D. (2006).  A Serious Game for Traffic Accident Investigators.  Interactive Technology and Smart Education, v3(4), 329-346.

Borich (2008). Letting Students Shape Instructional Strategies. Learning Languages, v14 n1, 22-24.

Boyd-Batstone, P. (2006). Focused Anecdotal Records Assessment (ARA): a standards based tool for authentic assessment. In (Shelby Barrentine & Sandra Stokes, Editors) Reading Assessment: Principles and practices for elementary teachers. Newark, DE: International Reading Association Publications.

Boyd-Batstone, P. (2006). Differentiated Early Literacy for English Language Learners: Practical Strategies. Boston, MA: Allyn & Bacon.

Boyd-Batstone, P. (2004). Focused Anecdotal Records Assessment (ARA): a standards based tool for authentic assessment. Reading Teacher, v58 n3,  230-239.

Brown, S. & Glasner, A. (1999). Assessment matters in higher education: Choosing and using diverse approaches. Buckingham: The Society for Research into Higher Education & Open University Press.

Brown, S., Race, P., & Bull, J. (1999). Computer-assisted assessment in higher education. London: Kogan Page Limited.

Buck, T. L. (2011). Technology literacy recommendations: A review of common college and university entry level standards. Journal of Research on Technology in Education, v43, n4, 235-239.

Buck, T. L. (2004). Learning Styles and Web-based Learning: An investigation of field dependence - independence and cognitive abilities in a nonlinear distance learning environment. Dissertation, Walden University.

Course evaluations for Lawyering Process I, Fall 2004. Retrieved June 30, 2006 from the University of Denver, Sturm College of Law Web site: http://www.law.du.edu/thomson/Downloads/Evaluation results - Fall 2004 - 101D.htm

Demetriou, A. (1998). Nooplasis: 10 + 1 Postulates about the Formation of Mind. Learning and Instruction, v8 n4, 271-87.

Dubuclet, K. S. (2008). Teaching Presence: A Focus on the Instructor's Role in Online Collaborative Learning. Louisiana State University, Baton Rouge.

Eble, K. (1971). The recognition and evaluation of teaching. Salt Lake City, Project to Improve College Teaching.

Elliott, S. N. (1995). Creating meaningful performance assessments. ERIC Digest E531. EDRS no: ED381985.

Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical Inquiry in a text-based environment: computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87-105.

Garrison, D. R., & Arbaugh J. B. (2007). Researching the community of inquiry framework: Review, issues, and future directions. The Internet and Higher Education, 10, 157-172.

Garrison, D. & Shale, D. (1987). Mapping the boundaries of distance education: Problems in defining the field. The American Journal of Distance Education (1), 4-13.

Gentile, M. J. (2007).  Authentic assessment of scientific ability, The Internet and Higher Education, 10(3), 202-209.

Goertzen, P., & SirKristjansson, C. (2007). Interpersonal dimensions of community in graduate online learning: Exploring social presence through the lens of Systemic Functional Linguistics, The Internet and Higher Education, 10(3), 212-230.

Hale, J. A. (2008). A guide to curriculum mapping: Planning, implementing, and sustaining the process. Thousand Oaks, CA: Corwin Press.

Harlow, L. L., Burkholder, G. J, Morrow, J. A. (2006). Engaging Students in Learning: An Application with Quantitative Psychology.  Teaching of Psychology, v33 n4 p231-235.

Hart, P. M., & Rowley, J. B. (1996). How Video Case Studies Can Promote Reflective Dialogue.  Educational Leadership, v53 n6 p28-29

Henderson, T. (1997).  Multimedia Handbook of Engaged Learning Projects. Glenview, IL: North Central Regional Educational Laboratory.

Hildebrand, M., Wilson, R.C., & Dienst, E.R. (1971) Evaluating University Teaching. Center for Research and Development in Higher Education, University of California, Berkeley.

Jacobs, H. H. (1997).  Mapping the big picture: Integrating curriculum and assessment K-12. Alexandria, VA: Association for Supervision and Curriculum Development.

Jonassen, D. H. (1992).  Learner-Generated vs. Instructor-Provided Analysis of Semantic Relationships. Journal of Computing in Higher Education, v4 n2, 12-42.

Jones, T. (2002). Options and Considerations for Distance Education: Learner Assessment and Self-assessment, (3)3. TOJDE (Turkish Online Journal of Distance Education). ISSN 1302-6488 Available at: http://tojde.anadolu.edu.tr/tojde7/articles/Jonestxt.htm

Juul, J. (2003). The game, the player, the world: looking for a heart of gameness, in: Marinka Copier and Joost Raessens (eds) Level Up: Digital Games Research Conference Proceedings. Utrecht: Universiteit Utrecht, 2003.

Keegan, D. (1996). Interaction and communication, (Chapter 6, pp.89-107). In Keegan, D., The foundations of distance education. Kent, UK.: Croom Helm..

Khan, K. S., Davies, D. A., & Gupta, J. K. (2001). Formative Self-Assessment Using Multiple True-False Questions on the Internet: Feedback According to Confidence About Correct Knowledge.  Medical Teacher, v23 n2, 158-163.

Kolowich, S. (2011). Blackboard's Next Phase. Inside Higher Ed. 18(3),9-12.

Lyle, V. (2006) Leadership for curriculum mapping. Unpublished Manuscript. Marion School District #2, Marion, Illinois.

Mager, R. F. (2005). Preparing Instructional Objectives.  Belmont, Calif.: Fearon.

Michael, D. & Chen, S. (2005). Proof of learning: assessment in serious games.  Available online:
http://www.gamasutra.com/view/feature/2433/proof_of_learning_assessment_in_.php.

Mueller, Jon (2011). Authentic Assessment Toolbox [online document].  Available online: http://jonathan.mueller.faculty.noctrl.edu/toolbox/

Montgomery, K. (2001). Authentic assessment: A guide for elementary teachers. New York: Longman.

Palomba, C.A. & Banta, T.W. (1999). Assessment essentials: Planning, implementing, and improving assessment in higher education. San Francisco, CA: Jossey-Bass.

Perraton, H. (1988). A theory for distance education. In Distance education: International perspectives, ed. D. Sewart, D. Keegan, and B. Holmberg, 34-45. New York: Routledge.

Piccoli, G., Ahmad, R., & Ives, B. (2001). Web-based virtual learning environments: A research framework and a preliminary assessment of effectiveness in basic it skills training. MIS Quarterly, 25, 4, 401-426.

Udelofen, S. (2005). Keys to curriculum mapping: strategies and tools to make it work. Thousand Oaks, CA: Corwin Press.

Vandervelde, J. (2005) Rubric for Classroom Web Pages. Retrieved June 30, 2006 from the University of Wisconsin--Stout School of Education Web http://www.uwstout.edu/soe/profdev/webpagerubric.html

Visscher-Voerman, I., & Gustafson, K. L. (2004). Paradigm in the theory and practice of education and training design. Educational Technology, Research and Development, 52(2), 69-89.

Vanchar, S. C., South, J. B., Williams, D. D., Allen. S., Wilson, B. G. (2010). Struggling with theory? A qualitative investigation of conceptual tool use in instructional design. Educational Technology Research Development 58(1),39 - 60.

Werbach, K. & Hunter, D. (2012). For the Win. How Game Thinking Can Revolutionize Your Business.  Philadelphia, PA: Wharton Digital Press.

Whitlock B., & Nanavati, J. (2013). A systematic approach to performative and authentic assessment. Reference Services Review, v41, 1, 32-48.

 


Assess students

Aldridge, A. et al. (2000). Online testing: Best practices from the field. Creating a Virtual Learning Community, 2(1). Raleigh: North Carolina Community College System. Retrieved April 23, 2007 from: http://www.nccei.org/blackboard/testingadvice.html

Allen, T. (1998). The taxonomy of educational objectives. Retrieved July 9, 2006 from the Humbolt State University Web site: http://www.humboldt.edu/~tha1/bloomtax.html

Arter, J. (2000). Rubrics, scoring guides, and performance criteria: Classroom tools for assessing and improving student learning. Paper presented at the annual conference of the American Educational Research Association, New Orleans.

Bartel, M. (2004). How to write a test for both creativity and knowledge. Retrieved April 23, 2007 from the Goshen College website: http://www.goshen.edu/~marvinpb/arted/testing/drawtest.html

Bixler, B. (2002). Writing educational goals and objectives. Retrieved July 9, 2006 from the Pennsylvania State University Web site: http://www.personal.psu.edu/staff/b/x/bxb11/Objectives/

Bloom, Engelhart, Furst, Hill, & Krathwhol. (1956). Taxonomy of Educational Objectives, Handbook I: Cognitive Domain. New York: Addison Wesley.

Bloom’s taxonomy. (2003). Retrieved July 9, 2006 from the University of Victoria Counseling Services Web site http://www.coun.uvic.ca/learn/program/hndouts/bloom.html

Clark, D. (2002). Learning domains for Bloom’s taxonomy. Retrieved July 9, 2006 from http://www.nwlink.com/~donclark/hrd/bloom.html

Courts, P. L., & McInerney, K. H. (1993). Assessment in higher education: Politics, pedagogy, and porfolios. Westport, Connecticut: Praeger Publishers.

Davis, B. G. 1993. Tools for Teaching. Jossey-Bass: San Francisco, CA.

Division of Instruction. (n.d.). Portfolio assessment in quest (Quality Education for Every Student). Retrieved October 6, 2006, from Prince Georges County Public Schools Web site: http://www.pgcps.org/~elc/portfolio.html

Gronlund, N.E. (1998). Assessment of student achievement, 6th edition. Needham Heights, MA: Longwood.

Haladyna. T. M. (1999) Developing and validating multiple-choice test items, 2nd edition. Mahwah, NJ: Lawrence Erlbaum Associates.

Haldayna, T.M. (1997). Writing test items to evaluate higher order thinking. Boston: Allyn & Bacon.

Haladyna, T.M., & Downing, S.M. (1989). A taxonomy of multiple-choice item-writing rules. Applied Measurement in Education, 2(1):37-50.

Heywood, J. (1989). Assessment in higher education, 2nd edition. Wiley, New York: Chichester.

Huitt, W. (2000). Bloom et al.’s taxonomy of the cognitive domain. Retrieved July 9, 2006 from Valdosta State University Educational Psychology Web site: http://chiron.valdosta.edu/whuitt/col/cogsys/bloom.html

Ittelson, J., & Lorenzo, G. (2005). An overview of eportfolios. Retrieved October 17, 2006, from The Educause Learning Initiative Web site: http://www.educause.edu/ir/library/pdf/ELI3001.pdf

Jacobs, L.C. and Chase, C.I. (1992). Developing and using tests effectively: A guide for faculty. San Francisco: Jossey-Bass. Krumme, G. (2001). Major categories on the taxonomy of educational objectives. Retrieved July 9, 2006 from the University of Washington Web site: http://faculty.washington.edu/krumme/guides/bloom.html

Klenowski, V. (2002). Developing portoflios for learning assessment. New York, NY: Routledge Falmer.

Lewis, K. (Ed.). (2001) Techniques and strategies for interpreting student evaluations. New directions for teaching and learning, 87(Fall). San Francisco, CA: Jossey-Bass.

McKeachie, W. J. & Svinicki, M. (2006). McKeachie’s teaching tips: Strategies, research, and theory for college and university teachers. College teaching series, 12th edition. Boston: Houghton Mifflin Company.

Mehrens, W. A., & Lehmann, I. J. (1973) Measurement and Evaluation in Education and Psychology. New York: Holt, Rinehart and Winston, 333-334.

On Purpose Associates. (2001). Portfolio assessment. In Funderstanding. Retrieved October 6, 2006, from http://www.funderstanding.com/portfolio_assessment.cfm

Richey, R. C., Klein, J. D. (2007). Design and Development Research. Mahwah, New Jersey: Lawrence Erlbatun Associates, Publishers.

Runte, R. (2001). How to write tests. Retrieved April 23, 2007 from the University of Lethbridge website: http://www.uleth.ca/edu/runte/tests/index.html

Steinkuehler, C., Squire, K., & Barab, S. (2012).  Games, Learning, and Society: Learning and Meaning in the Digital Age (Learning in Doing: Social, Cognitive and Computational Perspectives).  New York, NY: Cambridge University Press.

Stevens, K. (2005). Rural schools as regional centers of e-Learning and the management of digital knowledge: The case of Newfoundland and Labrador. In International Journal of Education and Development using Information and Communication Technology. IJEDICT, v2 (4): 119-127.

Suen, H. K. (1990). Principles of test theories. Hillsdale, NJ: Lawrence Erlbaum Associates.

Taggart, G. L., Phifer, S. J., Nixon, J. A., & Wood, M. (Eds.) (n.d.). Rubrics: Handbook for construction and use. Lancaster, PA: Technomic Publishing Co.

Tarouco, L. & Hack, L. (2001). New tools for assessment in distance education [online document]. Available online:
http://www.pgie.ufrgs.br/Webfolioead/artigo1.html

Tracey, M. W., & Richey R. C. (2007). ID model construction and validation: a multiple intelligences case. Educational Technology Research and Development, 55(4), 369-390.

Walvooord, B. E. & Anderson, V. J. 1998. Effective Grading: A tool for learning and assessment. Jossey-Bass: San Francisco, CA

Writing educational goals and objectives. (2001). Retrieved July 9, 2006 from the University of Mississippi School of Pharmacy Bureau of Pharmaceutical Services Web site: http://www.pharmd.org/thebureau/N.htm

Zimmaro, D. M. (2004). Developing grading rubrics. Retrieved June 30, 2006 from the University of Texas at Austin, Division of Instructional Innovation and Assessment Web site: http://www.utexas.edu/academic/mec/research/pdf/rubricshandout.pdf.

Willis, J. (2009). The graduate student’s handbook.  San José, CA: Psychology Department at San José State University.


Assess instructional technology

Chickering, A. W, & Ehrmann, S.C. (1996). Implementing the seven principles: Technology as lever. AAHE Bulletin, 49(2):3-6.

Chickering, A. W., & Reisser, L. (1993). Education and identity, 2nd edition. San Francisco: Jossey-Bass.

Cradler, J (1996). Implementing technology in education: Recent findings from research and evaluation studies. Policy Brief. San Francisco, CA: Far West Laboratory for the California Department of Education.

Eble, K. (1970). The recognition and evaluation of teaching. Washington, DC: American Association of University Professors.

EDUCAUSE Learning Initiative. Supporting Learning with Technology: Assessment’s Role? Retrieved September 21, 2006 from http://www.educause.edu/content.asp?SECTION_ID=210tt.

Ehrmann, S.C. (1995). Asking the right questions: What does research tell us about technology and higher learning? The magazine of higher learning, 27(2).

Ehrmann, S. C. (1999). Studying teaching, learning and technology: A tool kit from the Flashlight Program. Flashlight Program.

Ehrmann, S. C., Kelley, C.L., & Stewart, H. (2000). Assessing the impact of technology on student learning: Preparing students for what? Full-day preconference seminar presented at the EDUCAUSE conference, Nashville, TN.

Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical Inquiry in a text-based environment: computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87-105.

Garrison, D. R., & Arbaugh J. B. (2007). Researching the community of inquiry framework: Review, issues, and future directions. The Internet and Higher Education, 10, 157-172.

Haertel, G. D. & Means, B. (2003). Evaluating educational technology: Effective research designs for improving learning. New York, NY: Teachers College Press.

Harley, D., Maher, M., Henke, J., & Lawrence, S. (2003). An analysis of technology enhancements in a large lecture course. EDUCAUSE Quarterly. 26(3), 26-33.

Langlois, D.E., & Zales, C.R. (1992). Anatomy of a top teacher. Education Digest, 57(5), 31-34.

Knowles, M. S. (1980). The modern practice of adult education: From pedagogy to androgogy.
Englewood Cliffs, NJ: Cambridge Adult Education.

Lewis, V.K., Shaha, S.H., Farnsworth, B.J., Benson, L.F., & Bah, D.L. (2003). The use of assessment in improving technology-based instruction programs. Journal of Instructional Psychology, 30(2). 110-119.

Liu, L., Johnson, D., Maddux, C. & Henderson, N. (2001). Evaluation and assessment in educational information technology. New York, NY: The Haworth Press.

Raza, A., & Murad, H. S. (2008). Knowledge Democracy and the Implications to Information Access. Multicultural Education & Technology Journal, v2 n1, 37-46.

Richardson, J. C., & Ice, P. (2010). Investigating students level of critical thinking, across instructional strategies in online discussions. Internet and Higher Education, 13(1-2), 52 - 59.

U.S. Department of Education. An Evaluator’s Guide to Evaluating the Use of Technology in Schools and Classrooms: Why Am I Evaluating? Retrieved September 19, 2006 from http://www.ed.gov/pubs/EdTechGuide/whyeval.html.


Evaluate programs

Altschuld, J.W. and B.R. Witkin. (1999). From needs assessment to action: Transforming needs into solution strategies. Thousand Oaks, CA: Sage.

American Evaluation Association. (2004). American Evaluation Association Guiding Principles for Evaluators. Retrieved April 23, 2007 from: http://www.eval.org/Publications/GuidingPrinciplesPrintable.asp

Banta, T.W. (Ed.). (2004). Hallmarks of effective outcomes asessment. San Francisco: Jossey-Bass.

Hernon, P., & Dugan, R.E. (Eds.). (2004). Outcomes assessment in higher education: Views and perspectives. Westport, CT: Libraries Unlimited.

Huba, M.E., & J.E. Freed. (2000). Learner-centered assessment on college campuses: Shifting the focus from teaching to learning. Needham Heights, MA: Allyn & Bacon.

Maki, P.L. (2004). Assessing for learning: Building a sustainable commitment across the institution. Sterling, VA: Stylus Publishing.

McNamara, C. (1998). Basic guide to program evaluation, Retrieved June 30, 2006 from http://www.mapnp.org/library/evaluatn/fnl_eval.htm

Middle States Commission on Higher Education. (2002). Best practices in outcomes assessment. Retrieved April 23, 2007 from: http://www.msache.org/msache/content/pdf_files/best-oa.pdf

Miller, B.A. (2007). Assessing organizational performance in higher education. San Francisco: Jossey-Bass.

Popham, W. J. (1993). Educational evaluation. Needham Heights, MA: Allyn and Bacon.

Posavac, E. & Carey, R. (2003). Program evaluation: Methods and case studies. Upper Saddle River, NJ: Prentice Hall.

Russ-Eft, D. & H. Preskill. (2001). Evaluation in organizations: A systematic approach to enhancing learning, performance, and change. New York: Basic Books.

Scriven, M. (1993). Hard-won lessons in program evaluation. San Francisco, CA: Jossey-Bass.

Westat, J. F. with Frierson, H., Hood, S., and Hughs, G. (2002). The 2002 user-friendly handbook for project evaluation. Arlington, VA: National Science Foundation.

Witkin, B.R. and J.W. Altschuld. (1995). Planning and conducting needs assessment: A practical guide. Thousand Oaks, CA: Sage.


Conduct research

2 x 2 mixed factorial design. Retrieved July 13, 2006 from the University of Missouri - Rolla, Psychology World Web site: http://web.umr.edu/~psyworld/mixed_designs.htm

American Educational Research Association. (2000). Ethical standards of the American educational research association. Retrieved April 23, 2007 from: http://www.aera.net/AboutAERA/Default.aspx?menu_id=90&id=222

American Association for Public Opinion Research. (2000). Standard definitions: Final dispositions of case codes and outcome rates for surveys, 2nd edition. Ann Arbor, MI: AAPOR.

Aron, A. & Aron, E. N. (2002). Statistics for psychology, 3rd edition. Upper Saddle River, N J: Prentice Hall.

Arsham, H. (2005). Questionnaire design and surveys sampling, 9th edition. Retrieved July 13, 2006 from: http://home.ubalt.edu/ntsbarsh/stat-data/Surveys.htm

Bantra, T.W. & Associates (2002). Building a scholarship of assessment. San Fransico, CA: John Wiley & Sons.

Berkowitz, S. (1997). Analyzing qualitative data. In J. Frechtling, L. Sharp, and Westat (Eds.), User-friendly handbook for mixed method evaluations (Chapter 4). Retrieved July 13, 2006 from National Science Foundation, Directorate of Education and Human Resources Web site: http://www.ehr.nsf.gov/EHR/REC/pubs/NSF97-153/CHAP_4.HTM

Bogdan R. B. & Biklin, S. K. (1998). Qualitative research for education: An introduction to theory and methods, 3rd Edition. Needham Heights, MA: Allyn and Bacon.

Chi-square: One way. Retrieved July 13, 2006 from the Georgetown University, Department of Psychology, Research Methods and Statistics Resources Web site: http://www.georgetown.edu/departments/psychology/researchmethods/statistics/inferential/chisquareone.htm.

Coe, R. (2000). What is an 'effect size'? A guide for users. Retrieved July 13, 2006 from the University of Durham, Curriculum, Evaluation and Management Centre, Evidence-Based Education-UK Web site: http://www.cemcentre.org/renderpage.asp?linkid=30325016

Cohen's Kappa: Index of inter-rater reliability. Retrieved July 13, 2006 from University of Nebraska Psychology Department Research Design and Data Analysis Directory Web site: http://www.class.unl.edu/psycrs/handcomp/hckappa.PDF.

Darlington, R.B. (1997). Factor analysis. Retrieved July 13, 2006 from: http://comp9.psych.cornell.edu/Darlington/factor.htm

DeVellis, R. F. (2003).  Scale development: Theory and applications (2nd ed.).  Newbury Park, CA: Sage Publications.

Greiner, J. M. (2004). Trained observer ratings. In J. S. Wholey, H. P. Hatry & K. E. Newcomers (Eds.), Handbook of practical program evaluation, 2nd edition, (pp. 211-256). San Francisco: Jossey-Bass.

Helberg, C. (1995). Pitfalls of data analysis. Retrieved July 13, 2006 from: http://my.execpc.com/4A/B7/helberg/pitfalls/

Johnson, B., & Christensen, L. (2004). Educational research: Quantitative, qualitative, and mixed approaches (2nd ed.). Boston: Allyn and Bacon.

Lane, D. M. (2003). Tests of linear combinations of means, independent groups. Retrieved July 13, 2006 from the Hyperstat Online textbook: http://davidmlane.com/hyperstat/confidence_intervals.html

Linear regression. (n.d.) Retrieved July 13, 2006 from the Yale University Department of Statistics Index of Courses 1997-98 Web site: http://www.stat.yale.edu/Courses/1997-98/101/linreg.htm

Lowry, R. P. (2005). Concepts and applications of inferential statistics. Retrieved June 30, 2006 from: http://faculty.vassar.edu/lowry/webtext.html

Osborne, Jason W. (2000). Advantages of hierarchical linear modeling. Practical assessment, research & evaluation, 7(1). Retrieved July 13, 2006 from: http://PAREonline.net/getvn.asp?v=7&n=1

Seidel, J. V. (1998). Qualitative Data Analysis. Retrieved July 13, 2006 from: http://www.qualisresearch.com/QDA.htm

Spector, P. A. (1992). Summated Rating Scale Construction: An Introduction. Newbury Park, CA, Sage.

T-test. Retrieved July 13, 2006 from the Georgetown University, Department of Psychology, Research Methods and Statistics Resources Web site: http://www1.georgetown.edu/departments/psychology/resources/researchmethods/statistics/8318.html.

Trochim, W. M. K. (2002). The Regression-Discontinuity Design. Retrieved July 13, 2006 from: http://www.socialresearchmethods.net/kb/quasird.htm

Weunschk, K.L. (2003). Inter-rater Agreement. Retrieved July 13, 2006 from: http://core.ecu.edu/psyc/wuenschk/docs30/InterRater.doc.

Worthington, R. L., & Whittaker, T. A. (2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34, 806-838


Assessment and evaluation planning

Bordens, K. S. & Abbott, B. B. (1996). Research design and methods: A process approach, 3rd edition. Mountain View, CA: Mayfield Publishing.

Ryan, C. D. (2004). Authentic Assessment A Professional’s Guide. Westminster, CA: Teacher Created Materials, Inc.

Shea, P., S: Bidjerano, T. (2009). Community of inquiry as a theoretical framework to foster "epistemic engagement" and "cognitive presence" in online education. Computers & Education, 52(3), 543 - 553.

Witkin, B.R. and J.W. Altschuld. (1995). Planning and conducting needs assessment: A practical guide. Thousand Oaks, CA: Sage.


Data gathering methods Content analysis

Weber, R.P. (1990). Basic Content Analysis, 2nd ed. Newbury Park, CA: Sage Publications.

Writing Center at Colorado State University. Overview: Content analysis. Retrieved October 23, 2007 from: http://writing.colostate.edu/guides/research/content/index.cfm


Classroom Assessment Techniques (CATs)

Angelo, T.A. and K.P. Cross. (1993). Classroom assessment techniques: A handbook for college teachers, 2nd edition. San Francisco: Jossey-Bass.


Document analysis

Russ-Eft, D. & H. Preskill (2001). Evaluation in organizations: A systematic approach to enhancing learning, performance, and change. New York: Basic Books.

Vandervelde, J. (2005). Rubric for classroom web pages. Retrieved July 13, 2006 from the University of Wisconsin--Stout School of Education Web site: http://www.uwstout.edu/soe/profdev/webpagerubric.html.


Experiment

Reichardt, C. S. & Mark, M. M. (2004). Quasi-experimentation. In J. S. Wholey, H. P. Hatry, & K. E. Newcomber (Eds.) Handbook of practical program evaluation, 2nd edition, San Franciso: Jossey-Bass.

St. Pierre, R. G. (2004). Using randomized experiments. In J.S. Wholey, J. P. Hatry, & K. E. Newcomber (Eds.) Handbook of practical program evaluation, 2nd edition, San Francisco: Jossey-Bass.


Focus group

Fern, E. F. (2001). Advanced focus group research. Thousand Oaks, CA: Sage.

Goldenkoff, R. (2004). Using focus groups. In J.S. Wholey, H. P. Hatry, & K. E. Newcomer (Eds.), Handbook of practical program evaluation, 2nd edition, (pp. 340-362). San Francisco: Jossey-Bass.

Krueger, R. A. & Casey, M. A. (2000). Focus groups: A practical guide for applied research. Thousand Oaks, CA: Sage.

Silverman, G. (n.d.) How to get beneath the surface in focus groups. Retrieved June 30, 2006 from Market Navigation, Inc. Web site: http://www.mnav.com/bensurf.htm.


Interview

Kvale, S. (1996). Inter Views: An introduction to qualitative research interviewing. Thousand Oaks, CA: Sage.

Sewell, M. (no date) The use of qualitative interviews in evaluation. Retrieved June 30, 2006 from the University of Arizona, School of Family and Consumer Science, Institute for Children, Youth, and Families Web site: http://ag.arizona.edu/fcs/cyfernet/cyfar/Intervu5.htm.


Observation

Center for Teaching Excellence. (undated) Preparing for peer observation Austin, TX: The University of Texas at Austin.

Chism, N. V. N. (1999). Peer review of teaching: A sourcebook. Bolton, MA: Anker Publishing.


Portfolio

Courts, P. L., & McInerney, K. H. (1993). Assessment in higher education: politics, pedagogy, and portolios. Westport, CT: Praeger Publishers.

Division of Instruction. (n.d.). Portfolio assessment. In Quest (Quality Education for Every Student). Retrieved October 6, 2006, from Prince Georges County Public Schools Web site: http://www.pgcps.org/~elc/portfolio.html

Heywood, J. (1989). Assessment in higher education, 2nd edition. New York: Wiley.

Ittelson, J., & Lorenzo, G. (2005). An overview of eportfolios. Retrieved October 17, 2006, from The Educause Learning Initiative Web site: http://www.educause.edu/ir/library/pdf/ELI3001.pdf

Klenowski, V. (2002). Developing portoflios for learning assessment. New York: Routledge Falmer.

On Purpose Associates. (2001). Portfolio Assessment. In Funderstanding. Retrieved October 6, 2006, from http://www.funderstanding.com/_portfolio_assessment.cfm.


Product analysis

Sweet, D. (1993). Student portfolios: Classroom uses. U. S. Department of Education Consumer Guide, Number 8, Archived material. Office of Educational Research and Improvement of the U. S. Department of Education.


Survey

Alreck, P.L. & Settle, R.B. (1985). The survey research handbook. Homewood, IL: Richard D. Irwin, Inc.

Arsham, H. (2005). Questionnaire design and surveys sampling, 9th edition. Retrieved June 30, 2006 from: http://home.ubalt.edu/ntsbarsh/stat-data/Surveys.htm

Babbie, E.R. (1973). Survey research methods. Belmont, CA: Wadsworth Publishing Company.

Kalton, G. (1983). Introduction to survey sampling. Newbury Park, CA: Sage Publications, Inc.

Punch, K. F. (2003). Survey research: The basics. London: Sage Publications Ltd.

Spector, P.E. (1992). Summated rating scale construction: An introduction. Newbury Park, CA: Sage Publications, Inc.


Usability

Barnum, C. M. (2002). Usability testing and research. New York: Longman.

Dumas, J.S., & Redish J. C. (1999). A practical guide to usability testing. Portland, OR: Intellect.

Miami University Students in the Masters of Technical and Scientific Communication Program. (2002). Usability testing: Developing useful and usable products. Retrieved April 23, 2007 from: http://www.units.muohio.edu/mtsc/usabilitytestingrevisedFINAL.pdf.

Nielsen, J. (1994). Gerrilla HCI: Using discount usability engineering to penetrate the intimidation barrier. In Cost-justifying usability (pp. 242-272). R.G. Bias and D.J. Mayhew (Eds.). Boston: Academic Press.

Nielsen, J. (n.d.) How to conduct a heuristic evaluation. Retrieved April 23, 2007 from: http://www.useit.com/papers/heuristic/heuristic_evaluation.html.


Analysis and Reporting Quantitative analysis and statistical procedures

2 x 2 Mixed factorial design. Retrieved June 30, 2006 from the University of Missouri – Rolla, Psychology World Web site: http://web.umr.edu/~psyworld/mixed_designs.htm.

American Association for Public Opinion Research (2000). Standard definitions: Final dispositions of case codes and outcome rates forsSurveys, 2nd edition. Ann Arbor, MI: AAPOR

Aron, A. & Aron, E. N. (2002). Statistics for psychology, 3rd edition. Upper Saddle River, N J: Prentice Hall.

Bogdan R. B. & Biklin, S. K. (1998). Qualitative research for education: An introduction to theory and methods, 3rd edition. Needham Heights, MA: Allyn and Bacon.

Bordens, K. S. & Abbott, B. B. (1996). Research design and methods: A process approach, 3rd edition. Mountain View, CA: Mayfield Publishing.

Chi-square: One way. Retrieved June 30, 2006 from the Georgetown University, Department of Psychology, Research Methods and Statistics Resources Web site: http://www.georgetown.edu/departments/psychology/researchmethods/statistics/inferential/chisquareone.htm.

Coe, R. (2000). What is an ‘effect size’? A guide for users. Retrieved June 30, 2006 from the University of Durham, Curriculum, Evaluation and Management Centre, Evidence-Based Education-UK Web site: http://www.cemcentre.org/renderpage.asp?linkID=30325016.

Cohen, B.H. & R.B. Lea. (2004). Essentials of statistics for the social and behavioral sciences. Hoboken, NJ: John Wiley & Sons, Inc.

Cohen’s Kappa: Index of inter-rater reliability. Retrieved June 30, 2006 from University of Nebraska Psychology Department Research Design and Data Analysis Directory Web site: http://www-class.unl.edu/psycrs/handcomp/hckappa.PDF.

Darlington, R.B. (1997). Factor analysis. Retrieved June 30, 2006 from: http://comp9.psych.cornell.edu/Darlington/factor.htm.

DeVellis, R.F. (1991). Scale development: Theory and applications. Newbury Park, CA: Sage Publications, Inc.

Hartman, J.J. & Hedblom, J.H. (1979). Methods for the social sciences: A handbook for students and non-specialists. Connecticut: Greenwood Press.

Helberg, C. (1995). Pitfalls of data analysis. Retrieved June 30, 2006 from: http://my.execpc.com/4A/B7/helberg/pitfalls/.

Johnson, B., & Christensen, L. (2004). Educational research: Quantitative, qualitative, and mixed approaches, 2nd edition. Boston: Allyn and Bacon.

Lane, D. M. (2003). Confidence intervals. Retrieved June 30, 2006 from the Hyperstat online textbook: http://davidmlane.com/hyperstat/confidence_intervals.html

Lane, D. M. (2003). Tests of linear combinations of means, independent groups. Retrieved April 23, 2007 from the Hyperstat online textbook: http://davidmlane.com/hyperstat/confidence_intervals.html.

Linear regression. (n.d.) Retrieved June 30, 2006 from the Yale University Department of Statistics Index of Courses 1997-98 Web site: http://www.stat.yale.edu/Courses/1997-98/101/linreg.htm.

Lowry, R. P. (2005). Concepts and applications of inferential statistics. Retrieved June 30, 2006 from: http://faculty.vassar.edu/lowry/webtext.html.

Osborne, Jason W. (2000). Advantages of hierarchical linear modeling. Practical assessment, research & evaluation, 7(1).

Posavac, E. & Carey, R. (2003). Program evaluation: Methods and case studies. Upper Saddle River, NJ: Prentice Hall.

Sapsford, R. & Jupp, V. (1996). Data Collection and Analysis. London: Sage Publications.

T-test. Retrieved December 4, 2007 from the Georgetown University, Department of Psychology, Research Methods and Statistics Resources Web site: http://www1.georgetown.edu/departments/psychology/resources/researchmethods/statistics/8318.html.

Trochim, W. M. K. (2002). The regression-discontinuity design. Retrieved June 30, 2006 from: http://www.socialresearchmethods.net/kb/quasird.htm.

Vandervelde, J. (2005) Rubric for classroom web pages. Retrieved June 30, 2006 from the University of Wisconsin--Stout School of Education Web site: http://www.uwstout.edu/soe/profdev/webpagerubric.html.

Weunschk, K.L. (2003). Inter-rater agreement. Retrieved June 30, 2006 from: http://core.ecu.edu/psyc/wuenschk/docs30/InterRater.doc.


Qualitative analysis

Auerbach, C.F. & Silverstein, L.B. (2003). Qualitative data: An introduction to coding and analysis. New York: New York University Press.

Berkowitz, S. (1997). Analyzing qualitative data. In J. Frechtling, L. Sharp, and Westat (Eds.), User-friendly handbook for mixed method evaluations. Retrieved June 30, 2006 from National Science Foundation, Directorate of Education and Human Resources Web site: http://www.ehr.nsf.gov/EHR/REC/pubs/NSF97-153/CHAP_4.HTM

Greiner, J. M. (2004). Trained observer ratings. In J. S. Wholey, H. P. Hatry & K. E. Newcomers (Eds.), Handbook of practical program evaluation, 2nd edition (pp. 211-256). San Francisco: Jossey-Bass.

Patton, M. Q. (1990). Qualitative evaluation and research methods, 2nd edition. Newbury Park, CA: Sage.

Seidel, J. V. (1998). Qualitative data analysis. Retrieved June 30, 2006 from: http://www.qualisresearch.com/QDA.htm

Weber, R. P. (1990). Basic content analysis, 2nd edition. Newbury Park, CA: Sage Publications.


Reporting results

Altschuld, J.W. and B.R. Witkin. (1999). From Needs Assessment to Action: Transforming needs into solution strategies. Thousand Oaks, CA: Sage.

Grob, G. F. (2004). Writing for impact. In J.S. Wholey, J. P. Hatry, & K. E. Newcomber (Eds.) Handbook of practical program evaluation, 2nd edition (pp. 604-627). San Francisco: Jossey-Bass.

Torres, R. T. (2001). Communicating and reporting evaluation activities and findings. In D. Russ & H. Preskill (Eds.) Evaluation in organizations: A systematic approach to enhancing learning, performance, and change (pp. 347-380). Boston: Perseus.


Best practices Educational research

American Educational Research Association. (2000). Ethical standards of the American educational research association. Retrieved April 23, 2007 from: http://www.aera.net/AboutAERA/Default.aspx?menu_id=90&id=222.


Program evaluation

American Evaluation Association. (2004). American Evaluation Association Guiding Principles for Evaluators. Retrieved April 23, 2007 from: http://www.eval.org/Publications/GuidingPrinciplesPrintable.asp.

Middle States Commission on Higher Education. (2002). Best practices in outcomes assessment. Retrieved April 23, 2007 from: http://www.msache.org/msache/content/pdf_files/best-oa.pdf.


Student assessment

Aldridge, A. et al. (2000). Online testing: Best practices from the field. Creating a Virtual Learning Community, 2(1). Raleigh: North Carolina Community College System. Retrieved April 23, 2007 from: http://www.nccei.org/blackboard/testingadvice.html

Bartel, M. (2004). How to write a test for both creativity and knowledge. Retrieved April 23, 2007 from the Goshen College website: http://www.goshen.edu/~marvinpb/arted/testing/drawtest.html.

Gronlund, N.E. (1998). Assessment of student achievement, 6th edition. Needham Heights, MA: Longwood.

Haldayna, T.H. (1997). Writing test items to evaluate higher order thinking. Boston: Allyn & Bacon.

Runte, R. (2001). How to write tests. Retrieved April 23, 2007 from the University of Lethbridge website: http://www.uleth.ca/edu/runte/tests/index.html.


Teaching

Chickering, A., & Ehrmann, S.C. (1996). Implementing the seven principles: Technology as lever. AAHE Bulletin, 49(2):3-6.

Chickering, A.W., & Gamson, Z.F. (1987). Seven principles for good practice in undergraduate education. AAHE Bulletin, 39(7):3-7.

Chickering, A. W., & Reisser, L. (1993). Education and identity, 2nd edition. San Francisco: JosseyBass.

Knowles, M. S. (1980). The modern practice of adult education: From pedagogy to androgogy. Englewood Cliffs, NJ: Cambridge Adult Education.

Langlois, D.E., & Zales, C.R. (1992). Anatomy of a top teacher. Education Digest, 57(5):31-34.

Theall, M. & Franklin, J. (Eds.). (1991). Effective practices for improving teaching. New Directions for Teaching and Learning, 48(Winter). San Francisco: Jossey-Bass.

Williams, J. M. (2008). Jules on Schools. Duluth, MN: Clover Valley Press, LLC.


Using instructional technology

Chickering, A., & Ehrmann, S.C. (1996). Implementing the seven principles: Technology as lever. AAHE Bulletin, 49(2):3-6.

Cradler, J (1996). Implementing technology in education: Recent fidings from research and evaluation studies. Policy Brief. San Francisco: Far West Laboratory for the California Department of Education.

Ehrmann, S.C. (1995). Asking the right questions: What does research tell us about technology and higher learning? The magazine of higher learning, 27(2).

Ehrmann, S. C., Kelley, C.L., & Stewart, H. (2000). Assessing the impact of technology on student learning: Preparing students for what? Full-day preconference seminar presented at the EDUCAUSE conference, Nashville, TN.





image
image
image

image

image