Provost Summary

Area Institutional Effectiveness Summary
March 21, 2003

The chief administrators of ten service units and the deans of the ten colleges report to the Provost. This report summarizes institutional effectiveness in the ten provost's units and in the administration of the ten colleges. In addition, this report also reviews academic assessment in the professional program (DVM) of the College of Veterinary Medicine.

Administrative Offices

Administrative Units within the Office of the Provost include:

Provost's Office
Senior Vice Provost for Academic Affairs and Chief of Staff
Vice Provost for Academic Administration
Vice Provost and Director of NCSU Libraries
Vice Provost for Diversity and African American Affairs
Vice Provost for Distance Education and Learning Technology (DELTA)
Vice Provost for Enrollment Management and Services
Vice Provost for Equal Opportunity and Equity
Vice Provost for Information Technology
Vice Provost for International Affairs

The units assess their services as described in the reports linked above. Overall, institutional effectiveness in these units is Developing. All units are engaged in assessing institutional effectiveness, but few have progressed to levels that could be considered mature. There is a considerable range in the degree to which units engage in systematic assessment and use results to improve their services.

(The Vice Provost for Undergraduate Affairs [UGA] also reports to the Provost. UGA’s institutional effectiveness activities are described in the report on assessment in undergraduate education and are not dealt with in this document.) [This will link to the UG-education assessment summary.]

All units whose institutional effectiveness activities are summarized in this report provide support for the university's academic programs (teaching, research, and outreach). For example, the office of the Senior Vice Provost for Academic Affairs guides policies and procedures for faculty development and faculty governance, the Vice Provost for DELTA supports faculty development in the use of instructional technologies and distance education, and the Vice Provost for International Affairs supports study abroad, international student organizations, and exchange programs.

Assessment activities are varied, including counts and amounts (e.g. Information Technology, Enrollment Management and Services), regular meetings (e.g. Office of the Provost), focus groups (e.g. Libraries), customer surveys (e.g. Academic Administration), and external consultants (e.g. Office of Academic Affairs: Faculty Center for Teaching and Learning). Other ways of gathering feedback are also used; for example, the Learning Technology Service (a unit within Distance Education and Learning Technology) uses a workshop registration system that includes information on registrants’ perceptions of LTS services.

Personnel reviews also contribute to IE efforts in the Office of the Provost. The provost and the office are comprehensively reviewed at least every five years. "Reviews of the Provost and Vice Chancellor and the Programs and Leadership of the Office of the Provost" provides for faculty involvement in the review. Upward evaluations of each vice provost were conducted in 1999-2000 and 2000-2001 and will be repeated on a two or three year cycle. These evaluations involve the administration of a common questionnaire to a constituency selected for each vice provost. All vice provosts' constituencies included their direct reports, the ten college deans, and the other 9 vice provosts. Additional constituencies were identified for each vice provost. For example, because of the frequent interactions with academic department heads on issues of faculty development and faculty governance, the constituency of the senior vice provost for academic affairs included the chairs/heads of the 63 academic departments (does not include Music and Physical Education) and the 36 members of the Faculty Senate. The instrument was developed in consultation with the vice provosts and the deans. The provost shares the results of the surveys individually with each respective vice provost and uses them as part of the basis for discussion of areas of needed improvement.

Units in the Office of the Provost vary in how explicitly assessment results are used. The main office of the Provost, for example, holds regularly scheduled meetings with six groups--key staff, the full staff, individual college deans, groups of department heads, deans’ council, and deans’ steering committee--and feedback garnered in these meetings helps guide office activities in informal ways. In contrast, the NCSU Libraries conducts or participates in regular surveys and focus groups and uses the findings directly to improve services. For example, through University Planning and Analysis surveys of sophomores and seniors in 1998 and 2000, the Libraries identified a deficiency in students' "training to use the library." The Libraries addressed that need by expanding the ways students can request information (in person, by telephone, via email, or through Internet "chat" sessions), and by offering customized instruction on research methods and the effective use of library resources, both in classrooms and online. In addition, the Libraries hired a Librarian for Instruction and Undergraduate Research, who specializes in developing services that address undergraduates' needs. DELTA uses its Remedy database to analyze users’ questions about the WolfWare learning management system; based on trends in users’ requests for help, DELTA staff have redesigned WolfWare instructional tools for improved usability. Based on customer feedback, the Office of Academic Administration designed a new website to make information readily accessible and informative; included in the new website is an interactive questionnaire that allows campus customers to provide immediate feedback about office procedures. Based on records of participation and surveys, the African American Cultural Center (in the Office of Diversity and African American Affairs) planned new program offerings to attract non-black students, faculty, and staff to the Center’s programs.

The units are at various stages of development in their assessment-based institutional effectiveness processes. This is understandable in part because a number of the units (e.g. DELTA, International Affairs, Diversity and African American Affairs) were created within the past three years, and others (e.g. Academic Affairs, Equal Opportunity and Equity, Academic Administration) have recently been reorganized or received new assignments. The Libraries illustrate a well-established, mature approach to IE, with a wide range of assessment procedures conducted on a regular basis producing quantitative as well as qualitative information that is used to improve services. Other units in the Office of the Provost are in earlier stages of developing institutional effectiveness procedures. Taken as a whole, the units in the Office of the Provost demonstrate a growing commitment to assessment. Units know what they are trying to accomplish, understand how to gather information about their performance, and are increasingly using that information to improve their work.

College Deans' Offices

The ten academic deans' offices reviewed in this report are as follows:

Office of the Dean of Agriculture and Life Sciences
Office of the Dean of Design
Office of the Dean of Education
Office of the Dean of Engineering
Office of the Dean of Humanities and Social Sciences
Office of the Dean of Management
Office of the Dean of Natural Resources
Office of the Dean of Physical and Mathematical Sciences
Office of the Dean of Textiles
Office of the Dean of Veterinary Medicine

The administrative self-assessment and institutional effectiveness measures of each of the deans' offices are described in reports linked above. Overall, institutional effectiveness activity in these units is between Beginning and Developing. In the following summary, examples of college-level IE assessment activities are limited to administrative functions. However, in the individual college reports from which these examples were drawn, some academic program development activities are included. This illustrates the difficulties of drawing a sharp line between college administrations and the academic programs they exist to serve.

Certain assessment processes are widespread. For example, most colleges work with one or more professional accreditation societies, and their periodic reviews include assessment of administrative effectiveness. College deans carry out annual evaluations of college-level administrators (e.g. associate deans) and department heads. Department heads are formally evaluated every five years via a departmental committee, individual interviews with faculty and staff, and consultation with external clientele. Centers, institutes, and laboratories are reviewed on a regular basis; this review is described in more detail in the report from Research Administration. A variety of regular meetings—administrative, faculty, and staff—provide feedback to college administrative leaders.

Some colleges have elected to carry out additional institutional effectiveness assessment. For example, in the College of Design the dean holds “dinner with the dean” meetings once or twice each semester to give students a chance to provide direct feedback regarding the effectiveness of the college’s academic programs. The College of Humanities and Social Sciences is considering developing an electronic suggestion box for students using their computing labs. The College of Textiles has annual all-day faculty retreats at which key strategies and programs are reviewed and working groups prepare analyses of the college’s progress, strengths and weaknesses, and give specific suggestions for improvement. However, as these examples illustrate, most college-level assessment activity focuses on academic programs.

Colleges are using assessment findings to modify programs, as shown by the following examples:

An example of an advanced IE effort is found in the College of Engineering. There, department heads met with the dean and the associate deans in 1999 and 2000 to discuss improving the environment for new faculty. These discussions led to development of a week-long workshop for new faculty, which has been offered each of the past three years. The program’s effectiveness is evaluated by immediate participant feedback, a survey of faculty participants in their second year (Survey Sent to New Faculty June 2001 and June 2002), and through review of participants’ teaching and research performance. Improvements in the program have been made each year. The response to the program has been overwhelmingly positive, and it has been expanded to include new faculty in the College of Physical and Mathematical Sciences.
In the College of Agriculture and Life Sciences, feedback from users (including department heads, staff, and faculty) on the college’s administrative web site has been used to streamline numerous administrative procedures in the college; and feedback on the college’s new faculty orientation indicated a need for more guidance on the reappointment, promotion and tenure process, leading the college to develop an RTP workshop to be held in summer 2003.
Based on a student focus group, the College of Management changed the time when the computing lab was opened from 9AM to 7:30 AM, and based on problems identified in the senior survey, the college upgraded its advising for freshmen and sophomores.
In the College of Physical and Mathematical Sciences a student survey on diversity identified diversity awareness deficiencies that have been addressed by adding diversity as a component in its introductory course, and an informal assessment of current practice led the Department of Physics to restructure its labs to incorporate more writing.
In the College of Natural Resources, surveys of faculty, staff, graduate students, undergraduate student leaders, and external stakeholders identified a number of concerns about service issues (e.g. financial matters, turnaround time on routine matters) and led to formation of a committee now addressing staff issues.
In the College of Education, feedback from the teacher education accrediting body has guided the college in constructing departmental web links on assessment, training faculty in the use of a database designed to organize assessment information, sponsoring faculty discussions on the use of assessment evidence for continuous program improvement, and piloting a new student teaching performance-based observation process.

In addition to these explicit assessment efforts, regular personnel reviews help assure that college administrations receive information conducive to improving institutional effectiveness. Deans are comprehensively reviewed at least every five years. "Reviews of Academic Administrators: Periodic Review and Evaluation of College Leadership and Programs" provides for faculty involvement in the review. In 1999-2000, the provost conducted anonymous upward evaluations of each dean as part of their annual performance evaluations. These evaluations were repeated in 2000-2001 and hereafter will follow a two or three year cycle. These evaluations involve administration of a common questionnaire to a constituency selected for each dean. All deans' constituencies included their direct reports, academic department heads, and center directors within their respective colleges. In addition, certain external constituents were identified in consultation with the dean. The instrument was developed in consultation with the vice provosts and the deans. The provost shares the results of the surveys individually with each respective dean and uses them as part of the basis for discussion of areas of needed improvement.

College of Veterinary Medicine—Academic Programs

The College’s Ph.D. programs are reviewed and evaluated by the Graduate School, but its professional program (Doctor of Veterinary Medicine) is evaluated by the college. CVM pursues a range of assessment and institutional effectiveness efforts relative to their academic programs, including student course evaluations, peer-review of teaching, and college-level course evaluations on a three-year cycle. These assessments are used in regularly revising courses and in annual faculty appraisals and for promotion and salary considerations.

Feedback from students is also sought at the end of year two and in the last semester of the senior year, when the dean conducts exit interviews. Feedback from the seniors contributed to the decision to reorganize the curriculum to permit earlier and more clinical experience.

The college also conducts an alumni survey every five years and uses the information provided to make improvements. For example, the 1999 alumni survey indicated that students needed more opportunity to practice surgical skills and more training in business and interpersonal skills, and desired to give more attention to their specialties while still in school. Accordingly, CVM 1) moved the introductory surgery course from year three to year two, 2) added "selectives", courses in the last two weeks of the semester that allow students to focus on their specialties, 3) increased the business training in VMC 962, Professional Development in Veterinary Medicine, and 4) added two selectives on the business aspects of veterinary medicine.

Further information on the academic program assessment conducted in the College of Veterinary Medicine can be found in the attached document on CVM Assessment.