Student assessment in many Brazilian health professions schools consists mainly of summative exams targeting knowledge. Between 2018 and 2020, a faculty development (FD) project was held for nine Brazilian institutions addressing programmatic assessment systems favouring comprehensive, formative assessment of clinical skills and professionalism. This study aimed at identifying the immediate effects of the FD activities, and determining long-term (one-year) changes in participant’s assessment practices and other effects.

Participated 359 teachers from 30 undergraduate courses (13 different health professions). Local leaders took an initial workshops cycle (20 h), which was followed by 10 similar activities at the various institutions. The program design covered topics on assessment practices and systems using active teaching–learning strategies. In the end, participants evaluated the quality of the FD activities and self-evaluated the knowledge and skills acquired. One year later, participants were invited to complete an online structured questionnaire about the importance of the FD activities on 36 aspects of their work.

From 75 to 100% of the 292 (81.3%) respondents had a very positive perception of the workshops, particularly about program design, delivered activities and facilitator’s role, with no significant differences between the initial and the other cycles. Immediately after the workshops, perceived levels of knowledge and skills were significantly greater (p = 0.0001) than those before, for all topics evaluated. In the follow-up evaluation one year later, more than 90% of the 121 (33.7%) respondents to the reliable instrument (Cronbach’s α = 0,90) attributed high importance of participation to their work in student assessment. More than 70% reported positive impacts on other aspects, such as teaching, relationship with students and colleagues and feelings of greater belonging and personal and professional appreciation.

Our findings indicate that well-designed traditional FD activities focusing solely on student assessment enhance teacher’s knowledge and skills in this area, an effect that persists in the long-term. Other improving effects were observed on overall participants’ performance and may represent broader positive impacts on teachers and their institutions. Additionally, it is likely that these FD activities may contribute to changes in the participating institutions leading to a more constructive and meaningful approach to student assessment.

In recent decades, one of the main trends in the assessment of students in medical schools and other health professions in Europe and North America has been the implementation of assessment systems [1], which are within the scope of Programmatic Assessment [2,3,4,5,6].

Programmatic assessment of undergraduate students is an innovative way to more efficiently apply the most current principles and methods of educational assessment so that the assessment performs its attributions more effectively [2, 3, 7]. It can be considered “the latest product of innovation in the world of evaluation” [8], aiming to maximize assessment for learning and, at the same time, reach robust decisions about student progress [2, 3, 7, 9]. In this model, there is centralized coordination of the assessment program, which is the responsibility of the institution or course and not just the teachers of each of the curriculum units. In addition, this system emphasizes formative assessment, promoting greater uniformity in the application of assessment strategies and procedures so that they are completely aligned with the results proposed in the course curriculum.

This approach differs significantly from traditional assessment models. According to Schuwirth, Van der Vleuten, and Durning [10], in programmatic assessment, each assessment must provide significant feedback for the student, whether qualitative, quantitative or both. Initially, individual assessment events are not intended for decision making for approval or failure purposes. However, they serve as a reference for the students themselves to evaluate their performance and set tangible and possible learning goals to achieve [2,3,4,5,6, 9, 10].

The data obtained in each of the various assessment moments, which are predominantly formative in nature, should be inserted into the student's portfolio, forming a more global view that will allow more grounded decisions about their progress at appropriate times. In this process, the data obtained are periodically reviewed for summative decision-making, integrating information from various sources to reach conclusions about the student's progress. Finally, when necessary, remediation plans are created while continuous dialogue between the student and their mentor is maintained, reinforcing a focus on feedback, skills development, and personal and professional growth [2,3,4,5,6].

Instead of a conventional assessment that leads to either approval or failure, programmatic assessment seeks to address both the levels of competence achieved and their development processes. There is also concern for the qualification of methods so that the methods can have attributes that define a good assessment, such as validity, reliability, acceptability and positive educational effects [1, 2, 7].

However, the construction of programmatic assessment systems is not a simple task. Their implementation is a great challenge, which requires considerable investments in material, human and time resources and, above all, changes in the mind-set of those involved with the teaching and evaluation processes [7, 9, 11]. As pointed out by others, although a common set of principles should be shared [6], the implementation of PA demands a number of changes that will depend heavily on the local contexts, as “one size does not fit all programmes” [6, 12]. In particular, the “gatekeeper” attitude of many faculty members, teachers and preceptors may constitute a non-negligible barrier to successful implementation [13]. These requirements implies in considerable and consistent investments in faculty development.

Faculty development (FD) refers to all individual or group activities aimed at improving knowledge, skills, and behaviours in the roles of teachers and educators, leaders and managers, and researchers [14, 15]. In medical education [14, 15], and more recently in other health professions such as nursing [16], dentistry [14, 15, 17] and pharmacy [18], FD has become an integral part of the professional activities of teachers and faculty members.

It is widely recognized that FD initiatives in the health professions enhance not only teaching effectiveness but also a range of other skills and competencies. A recent review by Kohan et al. [19], which included 119 studies published between 1990 and 2020 and examined FD programs based on the Harden and Crosby teachers’ role framework [20], identified key outcomes such as gains in knowledge and skills related to teaching methods and student assessment. The review also found that FD programs were associated with improved teaching performance, the development of new educational curricula and programs, and enhanced feedback and evaluation processes.

Another review, analysing 12 studies published between 1984 and 2022 across 14 countries [21], reported significant changes in teachers’ behaviour, including increased ability to stimulate and encourage students, improvements in teaching quality, and enhanced leadership and self-evaluation skills. Other studies showed that well planned and delivered programs may have positive effects in enhancing faculty and institutional capacity building and organizational development [22, 23].

Nevertheless, studies exploring the effects of FD programs on outcomes related to personal and professional development, as well as relationships with colleagues and institutions in broader multi-professional contexts remain scarce.

In this century, Brazil has experienced an explosive increase in the number of undergraduate courses in the health professions. For instance, the number of medical courses rose from approximately 100 in 2000 to 335 in 2019 [24], and further increased to 389 in 2023 [25] and 423 in 2025 [26]. This trend has also been observed in other health professions, with over 780 nursing courses, 671 dentistry courses, 783 pharmacy courses, 959 nutrition courses, and 1,266 courses in the rehabilitation field—including 986 in physiotherapy, 118 in occupational therapy, and 162 in speech therapy—projected for 2025 [26].

Regarding medical schools, the federal government actively promoted their expansion to increase the medical workforce and, in particular, to attract physicians to underserved areas within Brazil’s public healthcare system [27], the Sistema Único de Saúde (SUS), which is one of the largest government-funded health systems in the world [28, 29]. However, there were no significant investments in infrastructure, faculty development, or teaching and training conditions [30]. As a result, the vast majority of new medical programs are privately owned and operated by for-profit educational institutions [27, 30]. Although the number of physicians in rural areas has increased, most newly graduated doctors continue to concentrate in affluent urban regions [27]. Moreover, data from the Brazilian Ministry of Education’s program evaluation system indicate that the quality of education in these new private medical schools is significantly lower than in established institutions [27].

In this scenario, some authors have expressed disbelief in the possibility that schools and undergraduate programs could improve their practices regarding student assessment [31], mentioning, as possible solutions, nationwide progress tests [32] or federal licensing exams for newly graduated physicians [32].

To standardize and ensure a minimum level of quality in curricula and educational practices across these programs, the Brazilian Ministry of Education has issued national guidelines for each health profession since 2001. This process began with medicine [33] and was completed by 2004 for all 14 health professions recognized by the Ministry [34]. Starting in 2014, these guidelines underwent revisions, beginning again with the medical course [35]. While these guidelines address many critical aspects, such as program objectives, skills and competencies to be acquired, training areas, teaching strategies, and student support, recommendations regarding student assessment remain vague and underemphasized. In fact, an analytical study of these guidelines revealed that, for many health professions’, there is not a single recommendation for student assessment [34].

Given that Brazilian legislation currently allows graduates to practice without requiring licensing exams or postgraduate training, the Federal Medical Council established an accreditation agency in 2015. Until 2023, this agency evaluated medical schools on a voluntary basis, achieving a full accreditation rate of just over 70% [25]. However, student assessment has not been a focal point in the evaluation criteria, and even so, this aspect has been identified as one of the areas with the lowest levels of sufficiency [25].

In many health professions courses, student assessment is predominantly or exclusively summative and concentrated in the cognitive domains [7, 34], which has led to distortions impairing the educational environment. The need of changes in student assessment in the health professions courses have already been advocated [36], since it is necessary, specifically, to “stop thinking of educational assessment as an instrument of power and of student behaviour control”. Assessment must be understood and practiced as an efficient resource to support learning and as an instrument of management, aiming to improve the overall teaching and learning process [36]. Thus, investment in faculty development programs (FD) is essential since there is abundant evidence of their effectiveness in the acquisition of knowledge and changing teachers’ behaviours [14, 15, 19, 21].

In our institution, a traditional school of medicine, which in recent decades has instituted six more undergraduate courses in health professions, a Centre for Faculty Development for Teaching (in Portuguese: Centro de Desenvolvimento Docente para o Ensino – CDDE) was created, with the mission of promoting educational improvement activities for teachers and preceptors linked to the institution and the local University Hospital, which is being done very successfully [37].

Following an institutional strategic decision to invest in the improvement of student assessment in its courses, the CDDE led in 2018 a specific project aimed at FD for the preparation of teachers with a view to introducing programmatic assessment. A consortium was formed with nine higher education institutions. This project was funded by the Latin American Grants (LAG) initiative of the United States National Board of Medical Examiners (NBME) and aimed to create, in the participating schools and courses, a critical mass of teachers trained to develop projects that lead to the future implementation of programmatic student assessment systems. These would be run by the institution and centrally coordinated, with emphasis on the formative assessment of clinical skills and professionalism, supported by effective FD actions. This project provided a unique opportunity to study a successful FD experience that appeared to result in expressive impacts on teachers’ practices and their courses and institutions. This study aimed at identifying the immediate effects of this FD project, consisting of cycles of workshops focused on student assessment, on participant’s knowledge and skills, as well as determining eventual, long-term (one-year) changes in teacher’s assessment practices and work.

This was a descriptive, predominantly quantitative study of the effects of a cycle of workshops focused on student assessment on participant’s knowledge and skills acquisition and changes in teaching and assessment practices. It was composed of a transversal component, immediately after the FD activities, and a longitudinal one, carried out one year later.

Our medical school, which in the beginning of this century added to the medical program six other undergraduate courses in health professions (physiotherapy, occupational therapy, speech therapy, nutrition, biomedical sciences and biomedical informatics) had established the improvement of student assessment in all of its courses as a strategic goal for the period 2015–2020. In this context, our faculty development centre welcome the news arriving in late 2017 that the NBME had established a specific grant for actions starting in 2018 that would “promote collaboration between regional health professions school consortia for innovative and sustainable projects that relate to assessment and address local needs and priorities” [38].

In order to attend to this call, our faculty development centre made preliminary contacts with a number of universities and higher education institutions across the country that have undergraduate programs in the health professions. They were invited to form a consortium that would be fit for applying for the NBME grant. Due to the need of producing specific documents in a time-constrained context, we were able to gather the approval of eight institutions with more than two undergraduate programs to join our medical school in this consortium and run for a successful application.

The locations of the institutions were as follows: one in the northeast region of Brazil (state of Bahia), one in the southern region (state of Rio Grande do Sul) and the others in the southeast region (states of Minas Gerais and São Paulo). Of these, seven were public, government-funded (state or federal) and two private (one of those linked to the Catholic Church).

The project was developed between November 2018 and January 2020 and consisted of offering 11 face-to-face workshop cycles, the first of which, called the “Centralized Workshop”, was offered at our institution's headquarters to only 27 leaders of participating institutions. This initial activity featured foreign (United States of America and The Netherlands) and national faculty guests who were experts in the topics covered. The next ten cycles, jointly called “Decentralized Workshops”, were offered to managers and teachers of participating institutions in their cities and with local and CDDE facilitators. In our leading institution, two cycles were performed due to the high number of interested teachers and to hold the largest number of health courses. Each cycle lasted two days, with a total workload of nearly 20 h. The workshops followed a content programme centred on student assessment, using active teaching–learning strategies [37, 39].

Our faculty development centre had, in 2018, 10 faculty members, one educationalist and two administrative clerks, one of them with an educational background. The faculty composition had a predominance of medical educators (N = 6), but a clear multidisciplinary and inter-professional feature, with members of other health professionals (nutrition, physiotherapy, occupational therapy, speech therapy). All members took part in both the program design and the choice of key topics to be covered, as well as in the learning strategies to be employed (Appendix Table). After consolidating the program first version, all institutions taking part in the consortium were consulted and agreed on its fitness for their needs and priorities. Again, prior to the delivery of the workshops in the various institutions (“Decentralized Workshops”), local leaders had the opportunity to offer further contributions and suggestions for modifications. Nevertheless, changes actually occurred in only a very few cases and involved mostly the emphasis to be given to one or another topic, expressed in terms of duration of the various workshops.

The topics covered were key concepts on educational assessment; programmatic assessment systems; quality criteria in student assessment and choice of methods; alignment between curriculum and student assessment; assessment of cognitive skills; structured assessment of clinical skills; assessment in the scenarios of professional practice; formative assessment and feedback techniques; student portfolio, academic tutoring and mentoring; successful experiences towards programmatic assessment; and the future of institutional programmatic assessment, where we are and where we can go (Appendix Table). The strategies used included selected text readings, interactive short lectures, discussions in small groups, individual and group exercises, practical exercises with role-playing and plenary discussions.

The project involved 359 teachers and managers of the nine participating university institutions. They worked in 30 undergraduate courses in 13 distinct health professions (Biomedical Informatics, Biomedical Sciences, Dentistry, Gerontology, Medicine, Nursing, Nutrition, Occupational Therapy, Pharmacy, Physiotherapy, Social Work, Speech Therapy and Veterinary Medicine). Other than the 27 leaders that took part in the initial, Centralized workshop, the number of participants in each cycle of Decentralized workshops ranged 28–42 (median: 32), and, with a few sporadic exceptions, all of them attended all activities. These participants were recruited and selected by the local leaders based on the perceived importance of their current functions in their courses.

From these 359 participants, 292 (81.3%) answered to the questionnaire on immediate perceptions of the quality of various aspects of FD workshops, 297 (82.7%) performed the self-assessment of their knowledge before and after the workshops, and 121 (33.7%) responded to the instrument of the late effects of participating in the project. The characteristics of this latter group were shown in Table 1. They were predominantly females (73.6%), age range 51–60 years (37.2%), and had more than 20 years of teaching experience (49.6%). Almost half of them (48.8%) worked in the clinical or professional cycle of the courses and had had experience coordinating the courses (52%).

The instrument used to assess the quality of the FD activities offered in the workshop cycles included a five-point scale ranging from 1 (“very poor”) to 5 (“very good”) so that the participants would evaluate seven aspects (general organization; program and content; activities developed; facilitators; infrastructure of the facility; virtual, electronic learning support environment used (Moodle platform); bibliography). It also contained space for each participant to make free comments on the evaluated aspects.

The instrument utilised to evaluate the participant perception about the knowledge acquired in the workshops consists in a scale of seven points, ranging from 1 (“Minimum Knowledge”) to 7 (“Expertise”). For each topic covered, participants were invited to evaluate the initial pre- (retrospective) and post- (final) instrument (after FD activities) of perceived knowledge about the topics covered. This instrument was adapted from that proposed by Bhanji et al. [40].

One year after the latest workshop cycle was offered, participants were again invited to fill out an online structured questionnaire on the later impacts of their participation in the project. This instrument was specifically designed for this study (see Appendix 1) and consisted of three parts: “general effects on teaching practices”, “effects on knowledge” and “effects on student assessment”. Participants were invited to express themselves how they, at that time, perceived the importance of the activities offered in the workshop cycles in relation to 36 possible outcomes expressed by the questionnaire items, using a five-point scale ranging from 1 to 5 (1—“Not important”; 2—Something intermediate between not important and important; 3 – “Important”; 4—“Something intermediate between important and very important”; 5—“Very important”).

The preliminary version of this instrument was submitted to a panel of seven teachers and managers (“judges”) for review. Two of them were active in basic biomedical sciences and five in the clinical stage of different health profession courses. After analysing the instrument, they made suggestions about the content and its relevance and the layout and clarity of the writing of questions and items. All suggestions were incorporated in a revised version of the instrument. This process was developed so that the questionnaire that was effectively applied was more likely to fulfil face and content validity requirements [41, 42].

The questionnaires to evaluate the participant’s perceptions about the quality of the activities and the knowledge acquired were distributed in paper to be answered in the final session of the FD activities.

The instrument to assess the late effects of the workshops were composed electronically (“Google Forms”) and was sent by email to all participants, with a note explaining the study objectives and the consent form. They were also encouraged to contact the research team whether they need further clarifications. To stimulate participation, a follow-up message was sent to non-responders two weeks after the first email, and it was additionally asked for the local consortium leaders to stimulate their colleagues to respond. Participants must had signed the consent form to gain access to the instrument.

Since we are not interested in an in-depth thematic analysis of participants’ comments after answering the instrument on the quality of the workshops, we analyse them individually simply aiming at classifying each one into the categories: “positive”, “critical” and “neutral”. Initially, one of the authors (RHF) examined each comment and classified it and thereafter, another (LEAT) reviewed the results and a further discussion aiming at reaching consensus resolved the very few discrepancies between them.

Differences between the Centralized cycle of workshops and the various Decentralized events concerning the proportions of participants attributing higher degrees of quality to the various aspects evaluated were analysed using a two-tailed Fisher exact probability test. Differences between perceived levels of knowledge before and after the workshops were analysed using Wilcoxon matched pairs signed rank test. These tests were performed using the GraphPad (version 5.04) software. Differences associated with p values equal or less than 0.05 were considered as statistically significant.

The questionnaire utilized to evaluate participants’ perceptions on the effects of the FD activities one year after the workshops was examined regarding its psychometric properties calculating its internal consistency (Cronbach’s alfa coefficient) and performing a factorial analysis of every item in order to obtain the factorial loads to calculate the composite reliability of each of its dimensions [43]. These analyses were performed using a SPSS (version 28.0 version) software.

The participants' perceptions of assessment procedures constituted a larger study project on the effectiveness of FD activities, which was submitted to the local Institutional Research Ethics Committee (Comitê de Ética em Pesquisa do Hospital das Clínicas da Faculdade de Medicina de Ribeirão Preto da Universidade de São Paulo – CEP) and approved on March 6, 2020 (Case No. 3902020. No. 29464920.6.0000.5440). When the study project was submitted, the CEP waived the application of a term of free and informed consent to the participants who had responded to the first instruments of assessment of the workshops because part of the data had already been collected at the end of the workshop cycles, initially with the sole purpose of improving FD activities. Participants who agreed to respond to the later effects questionnaire were invited to read and sign a specific free and informed consent term. All the data were collected anonymously and treated with confidentiality.

Data presented on Table 2 shows that the majority of the 292 respondents had a clearly positive view of nearly all aspects evaluated. This happened for participants of both the initial Centralized cycle of workshops and those from the Decentralized (Fig. 1). Figure 1 shows that more than 75% of the 292 respondents attributed the maximum degree (“very good”) to five of the seven aspects. Lower degrees of quality were attributed to the infrastructure of the site and the virtual learning environment used as support. There was a tendency for a higher attribution of the maximum quality degree to the “Centralized Workshop”, but the differences between this and the “Decentralized Workshops” were small, except for the quality of developed activities (100% vs. 76.6%). There were no significant differences between the results from both workshops, for any of the aspects evaluated.

Participants’ perceptions (n = 292; 81.3%) of the general quality of centralized and decentralized faculty development workshops

Full size image

Complementing this quantitative assessment of the perception of the quality of the workshops, 155 of the respondents made free comments, of which 86 were clearly positive in nature, 33 were of criticism and 36 were neutral. Table 3 presents examples of each class of comments. Among the criticisms, the most frequent were those related to the scarcity of time and to the conceptual and practical complexity of programmatic assessment.

Perceived levels of knowledge after the workshops were significantly higher than those before for all topics evaluated, for the participants of both the Centralized (p = 0.0001) and the various Decentralized workshop (p levels lower than 0.001 for all topics). The results of a sample of four of these topics, referring to the Centralized workshop and Decentralized workshops, are presented in Fig. 2.

Participants’ perceptions (n = 297; 82.7%) of student assessment topics: median knowledge before and after faculty development workshops. Legend: PA—Programmatic Assessment

Full size image

The questionnaire applied one year after the end of the workshops cycles was found to have suitable levels of reliability in general (Cronbach’s alpha = 0,90) and for all its three dimensions (Table 4).

A clear majority of participants attribute higher degrees of importance when evaluating the effects of the workshops on general teaching practices (Table 5), knowledge acquired (Table 6) and aspects of student assessment (Table 7).

Consequently, as shown in Table 8, all 36 outcomes offered for appreciation were considered important by an expressive proportion of respondents. Among these outcomes, greater importance, characterized by classifications of “something intermediate between important and very important” or “very important”, was attributed by more than 90% of the respondents to nine of them. These include outcomes directly linked to fundamental concepts and practical aspects of student assessment. Seventeen of the 36 items received the same high importance classification by 80% and 90% of the respondents. Among these were aspects of the assessment practice, as well as, more generally, those related to their teaching trajectory, such as the performance in teaching and learning activities and in the management or coordination of curriculum activities.

Between 70 and 80% of the respondents recognized the high importance of 10 other outcomes, including the development of feelings of greater belonging to the institution, of personal and professional appreciation, of closer proximity with students and of improved relationships with colleagues.

This FD project involving different Brazilian schools, with a large number of participants, including managers and teachers from various areas, provided a unique opportunity to analyse the immediate and late effects of this type of professional formative activity. The results obtained indicate that participants had a clearly positive perception of the various aspects related to the provision of workshops and acquired knowledge about the basic concepts related to student assessment and the functioning of institutional programmatic assessment systems, as well as cognitive gains in the practice of assessment strategies and techniques. The data obtained one year after the end of the FD activities suggest a positive impact in other areas of teaching performance, in addition to student assessment, particularly those related to the teacher’s trajectory and their relationship with students, colleagues, and the institution.

Indeed, participant’s perception that the activities focused on student assessment have late effects on other areas of faculty work, such as more proximity to students and the development of feelings of greater belonging to the institution and personal and professional appreciation, among others, could be seen as broader, positive effects of FD. This is in line with our previous study evaluating the first structured FD courses in our institution [37], focused on basic aspects of professional education, such as curriculum design, teaching and learning strategies, student assessment and program evaluation. We found that, besides the expected effect on improving teaching practices, participants also reported increased commitment and enthusiasm towards teaching [37]. These perceptual changes may be seen as favouring the formation of teacher’s professional identity.

These observations align with the considerations of Steinert and Mann [14]. They claim that faculty development programs help teachers and educators “develop skills relevant for their institution and suitable for their position and to their responsibilities, in addition to helping them to maintain and support their vitality as educators, in the present and future”. These data are also consistent with the findings of Steinert et al. [15] in a critical review of more than 100 published studies.

In the same line, expressive proportions of respondents (70% or more) attributed high importance to other outcomes, which suggests that participation in faculty development activities had a positive effect on “establishing partnerships with other colleagues, as well as other groups or areas to improve my professional practice”. This can be interpreted as favouring the establishment of “communities of practice” [44] in the institutions that participated in this initiative, which may favour the maintenance and continuous growth of knowledge, skills, and competencies more or less independently of formal FD activities [14, 15, 44].

A scoping review examining the state of faculty development programs in hospital medicine and other specialties found a number of components that could be associated to successful outcomes [45]. Among them, there are program structure and contents, the roles of facilitators and allotted funding and time for faculty participation, which we are tempted to recognize in our study.

The positive perception of the various aspects of the provided workshops was less so for the “infrastructure of the facilities”, which in fact varied widely, probably due to the different characteristics and physical spaces of the institutions that hosted the FD activities. Positive perception was also consistently lower for the virtual learning environment used as support, which can be explained not only by the low familiarity of the participants with this type of resource but also by participant’s greater familiarity with different systems they normally used in their institutions.

There was a tendency toward a higher frequency of the maximum degree of quality being attributed to the “centralized workshop” than to the “decentralized workshops”, but the differences between them were small. It is interesting to note, however, that all participants attributed the maximum degree of quality to three important aspects of the “Centralized workshop” (“general organization”, “activities developed”, and “facilitators”). This could be explained by two factors: the presence of Brazilian and foreign experts, who participated only in this first cycle of workshops, and the smaller number of participants, which may have favoured better performance.

The results on the perception of knowledge acquired, in relation to that existing before participation in FD activities, showed consistent growth concerning all evaluated topics. Notably, the level of prior knowledge was lower regarding the fundamental concepts of programmatic assessment. In fact, this student assessment model, in addition to being relatively recent [2, 3, 7, 9,10,11], is also considered by many to be relatively complex [11, 46] and is a paradigm that, in many ways, contrasts with more traditional practices in Brazilian schools, focused on summative goals and privileging the cognitive domain [7, 34, 36, 39].

With respect to the perception of existing knowledge before FD activities for various topics, including basic concepts on programmatic assessment, the results revealed higher levels for the Decentralized workshops participants than for the first cycle. The most likely explanation is that, among the participants of the Decentralized workshops, were the local leaders who had already participated in the first cycle, which perhaps helped to disseminate the acquired knowledge in their institutions, even before the local cycle of workshops took place.

Programmatic assessment has been strongly linked to competency-based education [47], but constitutes a paradigm that can benefit students in every curricular settings, given its strong emphasis on high-quality feedback for learning and improving performance [46, 47]. This relatively new paradigm also may ensure robust and fairer summative decisions based on integrating results of mainly work-placed, low-stakes assessment events [11]. In fact, it has been shown that, once a programmatic assessment system is implemented, the learning function via feedback is enhanced [46, 48] and defensible progression decisions can be made utilizing mainly qualitative information [46, 48, 49]. Furthermore, the capacity for early detection and supportive remediation of underperforming students is also potentiated [46].

Since its conception [2, 3], programmatic assessment systems have been implemented in many competency-based programs, such as medicine, veterinary medicine and dentistry, both at the undergraduate and postgraduate levels, in a variety of countries, such as The Netherlands, United Kingdom, Canada, United States of America and New Zealand [50]. In these diverse contexts, different strategies were utilized for dealing with the challenges and complexity of the implementation process.

Indeed, the implementation of programmatic assessment systems is often problematic, due to the variety of factors that may hinder it [9, 11, 46, 47, 51]. Amongst them, other than the need to redefine procedures, lye the imperative to transform the institutional assessment culture. This task may appear less complicated when the changes in assessment are associated to a new program [52] or a renewed curriculum [53], or follows a gradual, but systematic strategy [54]. As pointed out by Govaerts et al. [47], introduction of a new paradigm, such as programmatic assessment, depends on changes in the beliefs and perceptions, and roles of students and teachers, as well as in the norms of institutions, for which purpose it is essential that teachers and managers be engaged in consistent FD activities, such as those we offered to the 359 members and teachers of the nine participating institutions.

In the Brazilian context, due to the strong predominance of a summative student assessment model, focused on the cognitive domain [7, 34, 36, 39], it is likely that the implementation of a new paradigm will face huge difficulties. Nevertheless, incremental changes may be more feasible and effective in improving practices related to assessment. For example, in recent years, many medical [55] and nursing [56] schools in the country has been involved in institutional progress test of knowledge, and it has been recently shown that this initiative may be able to facilitate the introduction of programmatic assessment, by promoting a culture of regular feedback among students, teachers and course coordinators [57].

Additionally, in developing our FD project, we have also evaluated the perception of local leaders about eventual changes in the way the courses organize student assessment [39]. The results showed that the FD workshops were associated with significant changes in organizational and practical aspects in some of the consortium institutions, making student assessment organization and practice closer to those of programmatic assessment, as described in detail in another work [39].

These findings, together with the recognition by most managers and teachers that there was a positive impact of knowledge acquired in several of the specific aspects of student assessment that are proper to programmatic assessment systems, suggest that a considerable contribution was made to reaching in the future the main project goal in many of the participating institutions.

It is necessary to recognize that this study has several limitations. A work including nine different institutions from three different regions of the country and involving 359 teachers and managers working in 30 undergraduate courses of 13 distinct health professions can hardly be reproduced unless it integrates an initiative that is led and funded by organizations or associations of either a nationwide or international scope. Despite the large number of institutions and courses, the choice of these schools was dictated by the circumstances, and the sample represents a very small proportion of Brazilian higher education institutions. For example, eight medical courses participated in this study – seven from public institutions and two from private institutions – a very small fraction of the more than 423 courses currently authorized in the country [26], of which nearly 35% were public and approximately 65% were private. This allows us to question the representativeness and generalizability of the results.

The positive perceptions of participants on the quality of FD workshop cycles, the gains in knowledge, and the importance of this in different aspects of teaching performance can represent an “inclusion bias”. In fact, once foreign financing for the project was obtained, the individual cost for participants and the institutions to which they belonged was very small, which may have generated “socially desirable” answers to the applied instruments [58]. Additionally, and in particular for the evaluation of the late effects, we cannot rule out that a convenience, non-probability or census sampling could have led to a “selection bias”, including 121 participants more willing to emphasize positive aspects, and therefore hinder the results generalizability.

Moreover, the cross-sectional design utilized does not allow causal inferences. Future longitudinal studies might provide a more detailed understanding of the evolution of difficulties over time. The study design and development, with a focus on individual participant’s perceptions in conditions of anonymity, preclude us to know whether the activities worked better for some participants than others, or for some courses or institutions than others. Similarly, we had no data on whether participants embarked in any additional faculty development on the topic since the completion of the program. Future studies may address these aspects.

Another aspect to consider, specifically in relation to the data obtained in the questionnaire applied one year after the closure of the faculty development workshop cycles, was that this period coincided with the peak of the COVID-19 pandemic, which forced schools and teachers to introduce profound modifications in educational activities. It is thus possible that this has also contributed in some way, influencing the positive perspective of participants in relation to the investigated items.

Regardless of these possible limitations, the results show that the FD proposal aimed at the future implementation of student programmatic assessment systems, which had a late evaluation of participant perceptions as a distinctive feature, had positive effects on various aspects of teachers’ performance, transcending the specific field of student assessment. The monitoring of possible changes in schools, as well as the late perceptions of teachers and managers who participated in this project, can corroborate whether the main objective was achieved and whether positive effects on teaching performance remained.

Our findings, as a whole, may have some implications for the field of faculty or teacher development, whose activities have traditionally been delivered effectively through strategies such as workshops, short courses, seminars, and other structured group activities [15, 19, 21]. This field has been gradually evolving, not only through the increasing adoption of strategies like longitudinal programs, workplace-based learning, mentorship, and peer coaching but also through shifting perspectives on the scope and aims of FD [15]. A competency-based approach has been widely recommended [15], with some advocating for a framework centred on teachers’ roles [14, 15]. Additionally, there has been a call to expand the scope of FD to include supporting teachers’ professional identity and fostering their academic and scholarly development [59].

The findings of the present study demonstrate that even a more traditional faculty development (FD) approach, such as a cycle of workshops, employing active learning strategies, can effectively achieve the objective of enhancing teachers’ literacy in student assessment and lay the groundwork for the eventual implementation of programmatic assessment systems in their courses and institutions. Our results suggest that this effectiveness is probably linked to the adequacy of the program design, the chosen learning activities, and the facilitator’s role, as evidenced by the high proportions of participants who rated these aspects favourably.

Although our initiative did not adopt an explicit competency-based design, it focused on the teacher’s roles as both assessor and future manager of institutional programmatic assessment. However, the results of the follow-up evaluation of effects indicate that other competencies or teacher roles, such as teaching and curriculum coordination, could also be improved. Furthermore, the activities appear to have contributed to fostering professional identity, as participants reported feelings of greater belonging to the institution and increased personal and professional appreciation. Similarly, the high proportions of participants who attributed significant importance to effects such as the establishment of partnerships with colleagues and other groups or areas suggest potential positive impacts at the organizational and institutional levels.

Thus, our findings indicate that well-designed and effectively delivered traditional FD activities, such as a cycle of workshops focused on a single aspect of teachers' work – namely, student assessment – can not only achieve their intended objectives but also positively influence other areas, such as teachers' professional identity and their perception of their role within the institution. Awareness of these effects by FD program designers could lead to measures that further enhance the program’s overall impact.

CDDE:

Center of Faculty Development for Teaching (CDDE, in Portuguese)

CAEP:

Center for Educational and Psychological Support for Students (CAEP, in Portuguese)

CEP:

Institutional Research Ethics Committee (Comitê de Ética em Pesquisa do Hospital das Clínicas da Faculdade de Medicina de Ribeirão Preto da Universidade de São Paulo)

FD:

Faculty Development

LAG:

Latin American Grants, initiative of the United States

NBME:

National Board of Medical Examiner

Download references

Rodrigo Humberto Flauzino participated in the study design and was responsible for data collection and analysis, as well as for writing the first draft of the manuscript. Maria Paula Panúncio-Pinto supervised the data qualitative analysis, and revised and approved the submitted versions. Valdes Roberto Bollela and Luiz Ernesto de Almeida Troncon participated in the study’s design, supervised the data collection and analysis, and edited and revised the various versions of the manuscript. All authors agree to be accountable for all aspects of the study in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Correspondence to Rodrigo Humberto Flauzino.

We confirm that our study was conducted in accordance with relevant ethical guidelines, including the Declaration of Helsinki. Additionally, it was approved by the local Institutional Research Ethics Committee (Comitê de Ética em Pesquisa do Hospital das Clínicas da Faculdade de Medicina de Ribeirão Preto da Universidade de São Paulo) (Approval Certificate: Case Number 3902020; CAAE: 29464920.6.0000.5440). Informed consent was obtained from all the participants.

Not applicable.

The authors declare no competing interests.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions