with the implementation of a new licensing regulation for doctors in 2002   the faculties have to organise graded exams to benchmark students course achievements . the course achievements are prerequisites for the admission to the medical state examination and are being printed in the certificate since the new licensing requirements in 2002 . recent studies showed that preparation , structuring , accomplishment and correction of exams in germany are often not ideal . space for improvement exists and even first attempts at improving the process were noted , . we want to demonstrate the exam processing in the department of general medicine and evaluate its progress . given the new importance of faculty internal exams , the faculties had to find exam types and modalities to ensure significant and comparable grades . it was in 2003 when the didaktikzentrum der eidgenssischen technischen hochschule zrich ( ethz ) first published the this guideline was meant to be a helpful checklist and includes criteria of good quality which aim to be indicators for the quality of an exam . those main criteria are objectivity , reliability and validity but the guideline also includes factors such as comparability , scaling and economy . on the basis of the swiss guideline , the gma ( gesellschaft fr medizinische ausbildung ) and the kompetenzzentrum prfungen baden - wrttemberg published the  leitlinien fr fakultts - intere leistungsnachweise whrend des medizinstudiums  in 2008 . the authors formulate suggestions for the design of exams to balance the resources with respect to the amount of people that are required and the technical help that exists for the preparation of the exam in comparison with the high prerequisites . the catalogue has 48 items that are grouped into seven categories : general structural prerequisitesexam designorganisational preparations for the realisation of the examrealisation of the examcorrection of the exam and documentationpublishing the resultsreview process general structural prerequisites organisational preparations for the realisation of the exam realisation of the exam correction of the exam and documentation publishing the results the items are presented to help organising exams effectively and relevantly . for example , the appropriate exam design and a consensus on how to correct and deal with grades is a topic to be discussed before the exam . the catalogue focuses on the designation of a person responsible for the exam , appropriate training and an economical use of technical instruments for the correction and the review process . the review process encompasses a careful revision of the exam , including possible collaboration with faculty outside the department , before and after the actual exam takes place . it would include the calculation of the criteria for good quality ( see figure 1 ( fig . 1 ) ) . to evaluate the progress and changes that have been made in the field of exams since the implementation of a new licensing regulation for doctors , the deutsche fakulttentag initiated a survey in 2007 . the german faculties were asked to describe the organisation , exam processing and different review processes at their university . one of the main results was that , despite the fact that most of the exams were designed as multiple choice - exams , less than half of the faculties used existing central infrastructures for the preparation and correction . a more detailed analysis of the data was published by mltner et al two years later . the authors pointed out that efforts that have been made in the field of exams were measureable but that the results have been very heterogeneous . since the implementation of  mecum  , the name of the new curriculum for medical students in munich , the department of general medicine is involved in medical education with two exams , [ https://e-learning.mecum-online.de/informationambu_allg_l6_l7.pdf ] and a variety of classes . during the clinical part of the medical studies the department offers classes and lectures covering the clinic out - patient [ https://e-learning.mecum-online.de/informationambu_allg_l6_l7.pdf ] the course achievements are benchmarked with the help of a multiple choice ( mc ) exam consisting of 40 questions taking place at the end of year four . each lecturer was responsible for the creation of multiple choice ( mc ) questions covering their topic . the correction of the exams was done by a large number of assistants in order to be able to publish the results in an adequate time . in the course of time a confusing and unordered file record originated . using the items published by the gma   to describe the exam , the department fulfils only 14 of 48 possible items . items were fulfilled if they matched the criteria that were described in detail . on the basis of the new relevance the faculty exams gained , the department of general medicine drew the conclusion that an evaluation of the actual exam concept was necessary . since the implementation of  mecum  , the name of the new curriculum for medical students in munich , the department of general medicine is involved in medical education with two exams , [ https://e-learning.mecum-online.de/informationambu_allg_l6_l7.pdf ] and a variety of classes . during the clinical part of the medical studies the department offers classes and lectures covering the clinic out - patient [ https://e-learning.mecum-online.de/informationambu_allg_l6_l7.pdf ] . the course achievements are benchmarked with the help of a multiple choice ( mc ) exam consisting of 40 questions taking place at the end of year four . each lecturer was responsible for the creation of multiple choice ( mc ) questions covering their topic . the correction of the exams was done by a large number of assistants in order to be able to publish the results in an adequate time . in the course of time a confusing and unordered file record originated . a statistical analysis or a review process was never planned . using the items published by the gma   to describe the exam items were fulfilled if they matched the criteria that were described in detail . on the basis of the new relevance the faculty exams gained , the department of general medicine drew the conclusion that an evaluation of the actual exam concept was necessary . to make the process more efficient , the department worked hard on the introduction of a software to accompany the whole process of exams called itemmanagementsystem fr medizin ( imsm ) . the software had already been tested by universities in berlin , heidelberg and munich before and was used by the department of general medicine for the first time in 2008 . the software was developed as a joint project by the universities mentioned above and insures standardisation and quality management to a high degree . users can log into a web interface that gives full control over all steps of the exam . a well - thought - out user management process allows the integration of co - workers at each of the steps individually . imsm facilitates the co - operation between departments or faculties by giving the parties the possibility to exchange their pool of questions . 2 ) ) but also cases that can consist of multiple questions that can be linked . the current version even supports osce exams and thereby can be used in almost every case . the users can tag the questions to certain categories , e.g. educational objectives , in order to help not losing the overview over a growing database . the educational objectives are chosen to be general but the software supports department specific objectives as well . to generate a new exam the user only has to choose questions out of the database . at all steps of the creation questions the software also allows the user to integrate a faculty logo , versioning or page numbering , for example . technical improvements introducing imsm as a tool to facilitate the exam preparation thereby moving the file - based maintenance to a database - based web interface was not the only change . the answer sheets are scanned centrally to ensure a fast and accurate processing and publishing of the results . this is important for further exams as values like the discriminatory power can be used to improve the quality . the department tries to improve communication with the students by publishing the exact date , place and time the exam will take place as well as a written , detailed catalogue of topics that will be covered by the exam . to ensure quality the co - workers responsible for the questions meet regularly to talk about the questions and their possible answers . the department aims to reach a reliability ( cronbachs alpha ) of 0.8 which because of the small number of questions is not always easy . to make the process more efficient , the department worked hard on the introduction of a software to accompany the whole process of exams called itemmanagementsystem fr medizin ( imsm ) . the software had already been tested by universities in berlin , heidelberg and munich before and was used by the department of general medicine for the first time in 2008 . the software was developed as a joint project by the universities mentioned above and insures standardisation and quality management to a high degree . users can log into a web interface that gives full control over all steps of the exam . a well - thought - out user management process allows the integration of co - workers at each of the steps individually . imsm facilitates the co - operation between departments or faculties by giving the parties the possibility to exchange their pool of questions . 2 ) ) but also cases that can consist of multiple questions that can be linked . the current version even supports osce exams and thereby can be used in almost every case . the users can tag the questions to certain categories , e.g. educational objectives , in order to help not losing the overview over a growing database . the educational objectives are chosen to be general but the software supports department specific objectives as well . to generate a new exam the user only has to choose questions out of the database . at all steps of the creation questions the software also allows the user to integrate a faculty logo , versioning or page numbering , for example . technical improvements introducing imsm as a tool to facilitate the exam preparation thereby moving the file - based maintenance to a database - based web interface was not the only change . the answer sheets are scanned centrally to ensure a fast and accurate processing and publishing of the results . this is important for further exams as values like the discriminatory power can be used to improve the quality . the department tries to improve communication with the students by publishing the exact date , place and time the exam will take place as well as a written , detailed catalogue of topics that will be covered by the exam . to ensure quality the co - workers responsible for the questions meet regularly to talk about the questions and their possible answers . the department aims to reach a reliability ( cronbachs alpha ) of 0.8 which because of the small number of questions is not always easy . the steps that have been undertaken to improve the exam concept of the department of general medicine at the university of munich lead to a significant increase with respect to the fulfilment of the criteria enunciated by the gma ( see figure 3 ( fig . the number of fulfilled items in the category  general structural prerequisites  mainly evolves from the realisation of the following : introduction of a catalogue of educational objectives , formal improvements of the general process ( e.g. structured seating plan and grouping ) , introduction of a system to cope with wrong questions and announcing regularly dates for inspection of the exams by students . in the category exam design  we achieved the best improvement by : introduction of a pre - review process and its results , improvement of the economical use of existing infrastructures by delegating tasks to the central examining section and the introduction of a binding correction scheme . the overall reliability ( cronbachs alpha ) was calculated to be 0.65 and did not change very much . 3 ) shows the positive trend towards improvements in the categories  organisational preparations for the realisation of the exam  and  realisation of the exam  . in each of these categories two items in addition a protocol is created covering any change in the process of correction . with the introduced software and the technical help we were able to speed up the exam correction process and are now able to publish the results within at least two weeks . as mentioned before , the students now have the chance to inspect their exams afterwards . within the category review process  both items are fulfilled : there is a documented session reviewing the exam afterwards . almost immediately any relevant topic will be forwarded to the person that handed in the particular question . the implementation of a new licensing regulation for doctors in 2002 has great impact on the relevance of exams within the faculty . since 2002 the grades can be important when it comes to the admission of work places for scholarships . accordingly the exams must be of good quality in order to allow benchmarking with the faculty and between different universities . to assist in the creation of higher quality exams the didaktikzentrum der ethz followed by the ausschuss prfungen der gma and the kompetenzzentrum prfungen bayern - wrtemberg , published guidelines and quality criteria for the whole process . effort was put into transparent communication , review process including the integration of external department reviews and analytical calculations . the published guideline consists of 48 items the aim of which is to improve the quality and design of exams effectively and relevantly including current and future exams . during internal discussions about the possible steps to improve the exam process of our department we oriented ourselves strongly on the gma guidelines . main changes were the declaration of one co - worker who is responsible for the entire process and educational objectives . today , the department fulfils 30 out of the 48 possible items (  gma - score , see figure 3 ( fig . 3 ) ) . in the category  general structural prerequisites  and  exam design  we focused on the implementation of a catalogue of educational objectives . this catalogue can be seen as a basis for the creation of questions , but is also helpful for the students as the lectures are geared to it . the possibility to use technical help , such as reading the answer sheets automatically , takes a burden of the small department that mainly consists of one 40%-full time position . imsm helps to streamline the necessary workflow as it allows the integration of co - workers at each individual step by a well - though - through user management , especially in the review process ( e.g. detection of cueing ) . there were no measureable improvements in the categories  organisational preparations for the realisation of the exam  and  realisation of the exam  . this is largely the result of the faculty that registers every single student for the exam automatically . only if a student decides not to take part , he or she must contact the faculty . in the future we will advise those people watching the students during the exam to document those who try to cheat and the department will work on a general procedure on how to deal with those cases . it has to be noted that the items being used to calculate the  gma - score do not cover all aspects of an exam , e.g. they do not cover the question of how satisfied the students were with the process . the very positive trends toward a better system are an encouraging sign that the process has improved . we plan to engage co - workers outside the department to assist in the various review processes in the future . furthermore we think it might be of value to get into contact with other departments and faculties to benefit from each other s question pools .
graded exams are prerequisites for the admission to the medical state examination 
 . accordingly the exams must be of good quality in order to allow benchmarking with the faculty and between different universities . 
 criteria for good quality need to be considered - namely objectivity , validity and reliability . 
 the guidelines for the processing of exams published by the gma are supposed to help maintaining those criteria . in 2008 the department of general medicine at the university of munich 
 fulfils only 14 of 18 items . a review process , appropriate training of the staff and the introduction of the imsm software were the main changes that helped to improve the  gma - score to 30 fulfilled items . 
 we see the introduction of the imsm system as our biggest challenge ahead . 
 imsm helps to streamline the necessary workflow and improves their quality ( e.g. by the detection of cueing , item analysis ) . 
 overall , we evaluate the steps to improve the exam process as very positive . 
 we plan to engage co - workers outside the department to assist in the various review processes in the future . 
 furthermore we think it might be of value to get into contact with other departments and faculties to benefit from each other s question pools .