Eastern Kentucky University Faculty Program Evaluation
The Scholar’s Community is a monthly meeting between Eastern Kentucky University’s (EKU) faculty interested in research. The group started a year and a half ago. The group has changed leaders every semester, continually shifting the group’s focus. The group began online because of COVID, with around fifteen members. As of fall 2022, it moved to an in-person meeting. The assistant provost and administration requested an evaluation to determine the group’s viability, as attendance has dropped to five or fewer each month.
Determine the group’s viability. Offer suggestions to keep the group viable.
The Scholar’s Community is a monthly meeting between Eastern Kentucky University (EKU) faculty interested in research. The assistant provost and administration requested an evaluation to determine the group’s viability.
In-depth personal interviews conducted with administration and the current leader to determine the project’s scope and gather facts and information about the group.
Surveys were sent to faculty that have attended the Scholar’s Community or expressed interest in the Scholar’s Community to determine the interest level and desires of individual members.
Zoom was used to conduct the interviews.
Microsoft products were used to create wordage and send communications.
Qualtrics was used to conduct the survey results and provide in-depth data analysis.
Graphics were created in Adobe Photoshop.
The report was created in Adobe InDesign.
The results were exported by Adobe Acrobat and delivered through OneDrive.
The evaluation process taken is a formative process evaluation. As no objectives were set to measure the program initially, and the only clear question administration and leadership is currently seeking is group viability with the measurement of attaining more than six individuals coming per month as the critical measurement for success, it seems prudent to analyze the program starting at Donald Kirkpatrick’s level one assessment, which is “Learner Reactions.” Learner reactions are the degree to which participants find the training favorable, engaging, and relevant.
Level one for Kirkpatrick’s is rather basic, and the Scholar’s Community could attain measurements at higher levels by setting proper objectives; nevertheless, the survey attempted to uncover what levels of Kirkpatrick had been reached without any clear institutional goals or objectives for these types of measurements having been set at the formation of the group.
It is recommended that objectives be clearly defined moving forward. Have the leader write clearly defined objectives of what participants should be able to do because of their participation in the Scholar’s Community. This will help guide the group as a whole. Have participants define their own objectives as to what they would like to attain with their participation in the Scholar’s Community. Seek alignment in these objectives. If publication is the end result, team up experienced researchers with “newbies,” and in the networking time frame of each meeting, have participants outline their goals and how the group can help them with accountability.
Open the group up to graduate students to increase participation and networking opportunities.
Advertise the group across campus with a dedicated email announcement to gain interest, and move those individuals to an email newsletter where one can communicate bi-monthly with reminders of days and times. Clear communication is imperative. Clear communication allows for ease in individuals entering a formed group, and new participants know what to expect.
Change topics monthly based on group interest. Start a spreadsheet where individuals have access to contribute ideas and communicate the topic each month in the newsletter to spark interest. As many participants wanted an informal teaching style on a subject, it would be essential to continue this moving forward.
There is a fine line between preferred day and time and online or in-person format preferences. Explore different combinations to allow people who might not usually be able to participate in person to join online.
Explore the idea of an informal learning path to help new people get started in research. This can make networking and “coffee time” more productive, as new people have a scope and knowledge base to dialogue with. It can also be an excellent way for members to contribute to the knowledge and learning of others as the group evolves.
An option discussed with the prior administration was to have an award for those published based on their participation in the Scholar’s Community. This idea would need to be unpacked and fleshed out between the leaders and the new administration; however, it could increase participation and bring a greater sense of commitment to the group.
Reevaluate the Scholar’s Community in the fall of 2023 against objectives written in the spring of 2023. At this time, there would be clear indicators to determine the future of the Scholar’s Community outside of a number count. This approach will also show the group’s viability and worth and allow a greater degree of statistics to be used, which can impact participation. Suppose a correlation between participation in the Scholar’s Community and publication rate can be shown. In this case, more faculty could be interested in participating.
Finally, one of the biggest key indicators for the group to succeed will be the time invested in the group by the leader in planning, advertising, and incorporating the different desires of the participants into the monthly meetings.
* The images are cut off unless one views them from a desktop.*
The evaluation report was composed of fifty-five pages, which included graphs and charts.
Some of my favorite examples are below. The first set of images are visuals I created for the report. The second set of visuals was created to help administration understand why they want to measure against Kirkpatrick. The third set of visuals is how the questions were designed within Qualtrics. The last set of visuals is a few pages from the evaluation report designed and given to Eastern Kentucky University.