Impact of integrating objective structured clinical examination into academic student assessment: Large-scale experience in a French medical school
Alexandre Matet
(1, 2)
,
Ludovic Fournel
(3)
,
François Gaillard
(4)
,
Laurence Amar
(4)
,
Jean-Benoit Arlet
(4)
,
Stéphanie Baron
(4)
,
Anne-Sophie Bats
(4)
,
Celine Buffel Du Vaure
(5)
,
Caroline Charlier
(6, 7)
,
Victoire de Lastours
(8)
,
Albert Faye
(9)
,
Eve Jablon
(5)
,
Natacha Kadlub
(10)
,
Julien Leguen
(4)
,
David Lebeaux
(4)
,
Alexandre Malmartel
(5)
,
Tristan Mirault
(11, 4)
,
Benjamin Planquette
(12, 4)
,
Alexis Régent
(13)
,
Jean-Laurent Thebault
(5)
,
Alexy Tran Dinh
(14, 15)
,
Alexandre Nuzzo
(16)
,
Guillaume Turc
(17)
,
Gérard Friedlander
(18)
,
Philippe Ruszniewski
(17, 19)
,
Cécile Badoual
(11, 4)
,
Brigitte Ranque
(11, 4)
,
Mehdi Oualha
(20)
,
Marie Courbebaisse
(18, 4)
1
Institut Curie [Paris]
2 CRC (UMR_S_1138 / U1138) - Centre de Recherche des Cordeliers
3 T3S - UMR_S 1124 - Toxicité environnementale, cibles thérapeutiques, signalisation cellulaire
4 HEGP - Hôpital Européen Georges Pompidou [APHP]
5 UFR Médecine UPCité - UFR Médecine [Santé] - Université Paris Cité
6 Biologie des Infections - Biology of Infection
7 Service des Maladies infectieuses et tropicales [CHU Necker]
8 IAME (UMR_S_1137 / U1137) - Infection, Anti-microbiens, Modélisation, Evolution
9 Hôpital Robert Debré
10 Hôpital Necker - Enfants Malades [AP-HP]
11 PARCC (UMR_S 970/ U970) - Paris-Centre de Recherche Cardiovasculaire
12 IThEM - U1140 - Innovations thérapeutiques en hémostase = Innovative Therapies in Haemostasis
13 IC UM3 (UMR 8104 / U1016) - Institut Cochin
14 LVTS (UMR_S_1148 / U1148) - Laboratoire de Recherche Vasculaire Translationnelle
15 AP-HP - Hôpital Bichat - Claude Bernard [Paris]
16 Hôpital Beaujon [AP-HP]
17 IPNP - U1266 Inserm - Institut de psychiatrie et neurosciences de Paris
18 INEM - UM 111 (UMR 8253 / U1151) - Institut Necker Enfants-Malades
19 CRI (UMR_S_1149 / ERL_8252 / U1149) - Centre de recherche sur l'Inflammation
20 URP_7323 - Pharmacologie et évaluations thérapeutiques chez l'enfant et la femme enceinte
2 CRC (UMR_S_1138 / U1138) - Centre de Recherche des Cordeliers
3 T3S - UMR_S 1124 - Toxicité environnementale, cibles thérapeutiques, signalisation cellulaire
4 HEGP - Hôpital Européen Georges Pompidou [APHP]
5 UFR Médecine UPCité - UFR Médecine [Santé] - Université Paris Cité
6 Biologie des Infections - Biology of Infection
7 Service des Maladies infectieuses et tropicales [CHU Necker]
8 IAME (UMR_S_1137 / U1137) - Infection, Anti-microbiens, Modélisation, Evolution
9 Hôpital Robert Debré
10 Hôpital Necker - Enfants Malades [AP-HP]
11 PARCC (UMR_S 970/ U970) - Paris-Centre de Recherche Cardiovasculaire
12 IThEM - U1140 - Innovations thérapeutiques en hémostase = Innovative Therapies in Haemostasis
13 IC UM3 (UMR 8104 / U1016) - Institut Cochin
14 LVTS (UMR_S_1148 / U1148) - Laboratoire de Recherche Vasculaire Translationnelle
15 AP-HP - Hôpital Bichat - Claude Bernard [Paris]
16 Hôpital Beaujon [AP-HP]
17 IPNP - U1266 Inserm - Institut de psychiatrie et neurosciences de Paris
18 INEM - UM 111 (UMR 8253 / U1151) - Institut Necker Enfants-Malades
19 CRI (UMR_S_1149 / ERL_8252 / U1149) - Centre de recherche sur l'Inflammation
20 URP_7323 - Pharmacologie et évaluations thérapeutiques chez l'enfant et la femme enceinte
Alexandre Matet
- Fonction : Auteur
- PersonId : 1072339
François Gaillard
- Fonction : Auteur
- PersonId : 768575
- ORCID : 0000-0002-7248-8418
- IdRef : 232808392
Laurence Amar
- Fonction : Auteur
- PersonId : 759846
- ORCID : 0000-0003-3942-4276
- IdRef : 092993508
Caroline Charlier
- Fonction : Auteur
- PersonId : 764666
- ORCID : 0000-0002-9375-3620
Tristan Mirault
- Fonction : Auteur
- PersonId : 764162
- IdHAL : tristan-mirault
- ORCID : 0000-0001-8690-1093
- IdRef : 144101599
Alexandre Nuzzo
- Fonction : Auteur
- PersonId : 806889
- ORCID : 0000-0002-8952-7620
Guillaume Turc
- Fonction : Auteur
- PersonId : 776614
- ORCID : 0000-0001-5059-4095
- IdRef : 098880470
Résumé
Purpose: Objective structured clinical examinations (OSCE) evaluate clinical reasoning, communication skills, and interpersonal behavior during medical education. In France, clinical training has long relied on bedside clinical practice in academic hospitals. The need for a simulated teaching environment has recently emerged, due to the increasing number of students admitted to medical schools, and the necessity of objectively evaluating practical skills. This study aimed at investigating the relationships between OSCE grades and current evaluation modalities.
Methods:
Three-hundred seventy-nine 4th-year students of University-of-Paris Medical School participated to the first large-scale OSCE at this institution, consisting in three OSCE stations (OSCE#1–3). OSCE#1 and #2 focused on cardiovascular clinical skills and competence, whereas OSCE#3 focused on relational skills while providing explanations before planned cholecystectomy. We investigated correlations of OSCE grades with multiple choice (MCQ)-based written examinations and evaluations of clinical skills and behavior (during hospital traineeships); OSCE grade distribution; and the impact of integrating OSCE grades into the current evaluation in terms of student ranking.
Results: The competence-oriented OSCE#1 and OSCE#2 grades correlated only with MCQ grades (r = 0.19, P<0.001) or traineeship skill grades (r = 0.17, P = 0.001), respectively, and not with traineeship behavior grades (P>0.75). Conversely, the behavior-oriented OSCE#3 grades correlated with traineeship skill and behavior grades (r = 0.19, P<0.001, and r = 0.12, P = 0.032), but not with MCQ grades (P = 0.09). The dispersion of OSCE grades was wider than for MCQ examinations (P<0.001). When OSCE grades were integrated to the final fourth-year grade with an incremental 10%, 20% or 40% coefficient, an increasing proportion of the 379 students had a ranking variation by ±50 ranks (P<0.001). This ranking change mainly affected students among the mid-50% of ranking.
Conclusion: This large-scale French experience showed that OSCE designed to assess a combination of clinical competence and behavioral skills, increases the discriminatory capacity of current evaluations modalities in French medical schools.
Format du dépôt | Fichier |
---|---|
Type de dépôt | Article dans une revue |
Titre |
en
Impact of integrating objective structured clinical examination into academic student assessment: Large-scale experience in a French medical school
|
Résumé |
en
Purpose: Objective structured clinical examinations (OSCE) evaluate clinical reasoning, communication skills, and interpersonal behavior during medical education. In France, clinical training has long relied on bedside clinical practice in academic hospitals. The need for a simulated teaching environment has recently emerged, due to the increasing number of students admitted to medical schools, and the necessity of objectively evaluating practical skills. This study aimed at investigating the relationships between OSCE grades and current evaluation modalities.
Methods:
Three-hundred seventy-nine 4th-year students of University-of-Paris Medical School participated to the first large-scale OSCE at this institution, consisting in three OSCE stations (OSCE#1–3). OSCE#1 and #2 focused on cardiovascular clinical skills and competence, whereas OSCE#3 focused on relational skills while providing explanations before planned cholecystectomy. We investigated correlations of OSCE grades with multiple choice (MCQ)-based written examinations and evaluations of clinical skills and behavior (during hospital traineeships); OSCE grade distribution; and the impact of integrating OSCE grades into the current evaluation in terms of student ranking.
Results: The competence-oriented OSCE#1 and OSCE#2 grades correlated only with MCQ grades (r = 0.19, P<0.001) or traineeship skill grades (r = 0.17, P = 0.001), respectively, and not with traineeship behavior grades (P>0.75). Conversely, the behavior-oriented OSCE#3 grades correlated with traineeship skill and behavior grades (r = 0.19, P<0.001, and r = 0.12, P = 0.032), but not with MCQ grades (P = 0.09). The dispersion of OSCE grades was wider than for MCQ examinations (P<0.001). When OSCE grades were integrated to the final fourth-year grade with an incremental 10%, 20% or 40% coefficient, an increasing proportion of the 379 students had a ranking variation by ±50 ranks (P<0.001). This ranking change mainly affected students among the mid-50% of ranking.
Conclusion: This large-scale French experience showed that OSCE designed to assess a combination of clinical competence and behavioral skills, increases the discriminatory capacity of current evaluations modalities in French medical schools.
|
Auteur(s) |
Alexandre Matet
1, 2
, Ludovic Fournel
3
, François Gaillard
4
, Laurence Amar
4
, Jean-Benoit Arlet
4
, Stéphanie Baron
4
, Anne-Sophie Bats
4
, Celine Buffel Du Vaure
5
, Caroline Charlier
6, 7
, Victoire de Lastours
8
, Albert Faye
9
, Eve Jablon
5
, Natacha Kadlub
10
, Julien Leguen
4
, David Lebeaux
4
, Alexandre Malmartel
5
, Tristan Mirault
11, 4
, Benjamin Planquette
12, 4
, Alexis Régent
13
, Jean-Laurent Thebault
5
, Alexy Tran Dinh
14, 15
, Alexandre Nuzzo
16
, Guillaume Turc
17
, Gérard Friedlander
18
, Philippe Ruszniewski
17, 19
, Cécile Badoual
11, 4
, Brigitte Ranque
11, 4
, Mehdi Oualha
20
, Marie Courbebaisse
18, 4
1
Institut Curie [Paris]
( 301888 )
- 26 rue d'Ulm - 75248 Paris
- France
2
CRC (UMR_S_1138 / U1138) -
Centre de Recherche des Cordeliers
( 1004777 )
- 15 rue de l'Ecole de Médecine - Bâtiment B -
75270 Paris Cedex 06
- France
3
T3S - UMR_S 1124 -
Toxicité environnementale, cibles thérapeutiques, signalisation cellulaire
( 1004846 )
- Centre Universitaire des Saints Pères - 45 Rue des Saints-Pères 75270 Paris Cedex 06
- France
4
HEGP -
Hôpital Européen Georges Pompidou [APHP]
( 300119 )
- 20, rue Leblanc, 75015 Paris
- France
5
UFR Médecine UPCité -
UFR Médecine [Santé] - Université Paris Cité
( 1042190 )
- 15 rue de l’École de Médecine - 75006 Paris
- France
6
Biologie des Infections - Biology of Infection
( 223010 )
- Département de Biologie cellulaire et Infection, 25-28, rue du docteur Roux, 75724 Paris Cedex 15
- France
7
Service des Maladies infectieuses et tropicales [CHU Necker]
( 561559 )
- Paris 15
- France
8
IAME (UMR_S_1137 / U1137) -
Infection, Anti-microbiens, Modélisation, Evolution
( 1005021 )
- Faculté de médecine Paris Diderot Paris 7 - site Bichat - 16 rue Henri Huchard 75890 Paris Cedex 18
- France
9
Hôpital Robert Debré
( 300089 )
-
- France
10
Hôpital Necker - Enfants Malades [AP-HP]
( 414766 )
- 149 Rue de Sèvres 75015 Paris
- France
11
PARCC (UMR_S 970/ U970) -
Paris-Centre de Recherche Cardiovasculaire
( 1004693 )
- 56 rue Leblanc
75015 PARIS
FRANCE
- France
12
IThEM - U1140 -
Innovations thérapeutiques en hémostase = Innovative Therapies in Haemostasis
( 1004816 )
- Site : Faculté des Sciences Pharmaceutiques et biologiques, 4 avenue de l'Observatoire 75270 Paris Cedex 06
Site de Hôpital Européen Georges-Pompidou, 20 rue Leblanc 75015 Paris
- France
13
IC UM3 (UMR 8104 / U1016) -
Institut Cochin
( 1004701 )
- 22 rue Méchain, 75014 Paris
- France
14
LVTS (UMR_S_1148 / U1148) -
Laboratoire de Recherche Vasculaire Translationnelle
( 1005065 )
- Bâtiment INSERM - 46 rue Henri Huchard 75018 Paris
- France
15
AP-HP - Hôpital Bichat - Claude Bernard [Paris]
( 300156 )
- 46 Rue Henri Huchard, 75018 Paris
- France
16
Hôpital Beaujon [AP-HP]
( 300983 )
- 100, boulevard du Général Leclerc 92 Clichy
- France
17
IPNP - U1266 Inserm -
Institut de psychiatrie et neurosciences de Paris
( 1004695 )
- 102-108 Rue de la Santé
75014 PARIS
- France
18
INEM - UM 111 (UMR 8253 / U1151) -
Institut Necker Enfants-Malades
( 1004769 )
- Bâtiment Leriche
14 rue Maria Helena Vieira Da Silva
75014 Paris
- France
19
CRI (UMR_S_1149 / ERL_8252 / U1149) -
Centre de recherche sur l'Inflammation
( 1005007 )
- Faculté de Médecine site Bichat - 16, Rue Henri Huchard - BP 416/4ème étage - 75870 Paris Cedex 18
- France
20
URP_7323 -
Pharmacologie et évaluations thérapeutiques chez l'enfant et la femme enceinte
( 1004822 )
- Site 1 : Team Necker-Enfants Malades : TRELUYER J-Marc
Hôpitaux Universitaires Cochin – Broca – Hôtel Dieu, Site Tarnier 2ème étage, 89 rue d’Assas, 75006 Paris
Site 2 : Team Robert Debré JACQZ-AIGRAIN Evelyne
Hôpital Universitaire Robert Debré, 48 boulevard Serurier, 75019 Paris
- France
|
Langue du document |
Anglais
|
Nom de la revue |
|
Vulgarisation |
Non
|
Comité de lecture |
Oui
|
Audience |
Internationale
|
Volume |
16
|
Numéro |
1
|
Page/Identifiant |
e0245439
|
Licence |
Paternité
|
Date de publication |
2021-01-14
|
Domaine(s) |
|
Mots-clés |
en
Medical education, Teachers, France, Chi square tests, Education, Computer software, Patients, Psychometrics
|
DOI | 10.1371/journal.pone.0245439 |
Origine :
Publication financée par une institution
Loading...