Course Entry Self-Assessment Reflection

I am beginning LDT 506: Evaluation of Learning Systems with little to no prior evaluation experience. On the scale from novice to expert as outlined in our Evaluator Self-Assessment, I identify more closely as a novice when it comes to my experience as an evaluator. In my current position, I have not been called to complete any evaluations as it is outside my area of expertise and not part of my job requirements. Given that I currently work as an instructional designer for an established college in an established university, if I were to engage in evaluative work it would most likely be summative in nature – although in the case of new programs or courses, the evaluative work could be formative (Russ-Eft & Preskill, 2009).

The readings for the initial module of this course provided interesting insights about evaluation that I previously had not considered. I was drawn to the following statement especially:

“Just as evaluation standards provide guidance for making decisions when conducting program evaluation studies, evaluator competencies that specify the knowledge, skills, and dispositions central to effectively accomplishing those standards have the potential to further increase the effectiveness of evaluation efforts.” (Stevahn et al., 2005)

This description of the evaluation process makes me envision competencies as ‘standards for the standards’ – what appears to be a necessity for establishing a robust paradigm in professionalizing the role of the evaluator. I did not anticipate that evaluation would be such an in-depth topic in this degree program, and am eager to learn more about the role it plays in the learning design process.

When taking the self-assessment, I initially hesitated to mark myself as familiar with more than a few of the listed evaluator competencies. While many of them were based on concepts that I find important in any context – ethical action, cultural respect, and social justice, for instance – in the context of evaluation specifically I do not know what these competencies look like or how they are actionable. I expect I will have a better understanding of the broad range of evaluator requirements listed by the IBSTPI and the AEA by the end of this course.

In terms of my strengths as a future evaluator, given my background in social statistics I am more confident in the competencies related to the methodology domain of the AEA competencies – for instance, identification of evidence and sampling procedures, as well as collecting and analyzing data. I am grateful that my background in sociology should help expedite this learning process for me as I familiarize myself with the process of evaluation in the context of learning design. Working with data is less intimidating to me than many of the other competencies that were listed in our self-assessment – for instance, project management and implementing change on an organizational level. As such, in terms of weaknesses, I am least familiar with the competencies related to the planning and management domain of the AEA competencies. I have not had to work directly with evaluative components such as budgetary planning or stakeholder management. As a result, these are both areas I hope to understand better (and find less daunting) by the end of this course.

There were a couple competencies I was surprised to see listed in the AEA’s 2018 Evaluator Competencies. One was in the fifth domain: the ability to “[attend] to the ways power and privilege affect evaluation practice” (AEA, 2018). This type of sensitivity to social context is important to me as a sociologist. Too often, evaluation results are perceived as objective, when in reality implicit bias and the influence of the evaluator’s own race, class, and gender can easily affect conclusions drawn. A related competency I was surprised to see was in the first domain: the ability to “[identify] how evaluation practice can promote social justice and the public good” (2018). Again, I imagine this competency is significant for recognizing the implications that evaluative results can have on larger societal processes. I will be interested to see these how two competencies contribute to and shape my experiences with the deliverables and learning materials in this course.    

Overall, I expect these competencies to be integral in my professional growth as a learning designer, and will need to pursue opportunities to apply them in my current role so that I remain a well-rounded member of my team.  Because I am not our team’s lead instructional designer – and was not a member of our team at its inception – I have not seen the evaluative process take place for our programs on a first-hand basis. Based on the results of this course’s self-assessment, I am hoping I will have the opportunity to work more closely with our lead instructional designer on future evaluative tasks using the knowledge I will gain over the next few months.

References

American Evaluation Association. (2018). The 2018 AEA evaluator competencies. Retrieved from https://www.eval.org/About/Competencies-Standards/AEA-Evaluator-Competencies.

Russ-Eft, D., & Preskill, H. (2009). Evaluation in organizations: A systematic approach to enhancing learning, performance, and change. New York, NY: Basic Books.

Stevahn, L., King, J. A., Ghere, G., & Minnema, J. (2005). Establishing essential competencies for program evaluators. American Journal of Evaluation, 26(1), 43–59.