as part of ongoing quality improvement , diligent radiologists frequently seek information on the subsequent clinical course of patients whose studies they have read . commonly , this takes the form of asking referring clinicians about the outcome of further workup . alternatively , a radiologist might make a note of the medical record number ( mrn ) for an interesting or mysterious case so that he or she can later independently look for subsequent operative notes , pathology reports , or imaging results in the electronic medical record ( emr)1 . this process of follow - up plays a vital role in developing a radiologist s diagnostic acumen . every time a radiologist finds out that she was wrong  or right  about an interpretation is an opportunity for self - improvement , both during and after training2 . exploring outcomes also helps with many ancillary missions in a radiology practice . knowing clinical outcomes of shared patients allows radiologists to establish better connections with referring colleagues . learning clinical outcomes is also important for creating good teaching material ; a case which ends with a description of what actually happened to the patient can be very powerful . finally , integrating clinical outcomes can be vital for research , as connections which might not have been apparent reveal themselves after follow - up . however , in spite of the importance of follow - up , it does not fit easily into the busy radiology workflow . as radiologists typically see a large volume of patients with limited continuity of care , the process of data aggregation and outcome inference takes substantial time and effort . as radiology workloads have increased and the volume of data contained in the emr has expanded , the required chart review and data assimilation has become a rate - limiting factor . with current emr portals , diligent radiologists could spend several hours each week following up their patients , and even then only follow the most remarkable cases . as a result , though information systems at our institution have some facilities for noting cases , most members of our department have found them to be unsatisfactory , and they are used only sporadically . therefore , we set out to create a tool to allow radiologists to add to a personal database of cases as they perform their clinical work , to which they can return to review the imaging and search for follow - up clinical information . we have created a case - tracking application that we have dubbed  racetrack  ( for  radiology case tracking  ) . the racetrack system itself is a database - backed web application that integrates with our department s existing radiology information systems , including the radiology information system ( ris ; idxrad , ge healthcare ) , picture archiving and communication system ( pacs ; impax , agfa ) , and dictation software ( radwhere , nuance ) . the racetrack system also interfaces with the queried patient inference dossier ( qpid ) , which aggregates and indexes electronic medical records to allow efficient semantic search . in addition , the system utilizes a single sign - on service administered by our organization for authentication . the core racetrack system is implemented using ruby on rails4 , an open - source web application framework with integrated object - relational mapping . application data , which amounts to radiologist - generated metadata about imaging examinations , is managed by the open - source mysql database engine5 the system is currently deployed as a virtual machine running in a vmware virtualization environment ( vmware , palo alto , ca , usa ) on server hardware ( dell , round rock , tx , usa ) running windows server 2003 ( microsoft , redmond , wa , usa ) . the web - based user interface relies on the open - source jquery javascript library8 . scripts that capture radiologist workflow context were created using the open - source autohotkey utility9 . the architecture of the system is summarized in figure 1 . on the server side , user requests are routed through the apache http server to the ruby - based application . database requests are currently made to the mysql database engine running on the local machine , though the database can be moved to a dedicated server as the application scales to greater use . user authentication is handled via ssl - encrypted http by confirming a username and password against an enterprise - provided web services security service10 . queries for medical record information are managed via http web service requests to the qpid system . users interact with the racetrack system using web browsers via http . on pacs and dictation workstations , scripts extract working context and feed that information into web browsers for more convenient case creation . the racetrack application stack consists of a http server , application server , and database server , which all run in a linux virtual machine . the application communicates with an external authentication system via encrypted http to provide a single enterprise - wide sign - on for users . the racetrack application also communicates with the radiology information system ( ris ) via odbc protocol to retrieve information about radiology examinations and the queried patient inference dossier ( qpid ) server via http to retrieve other medical record information . users interact with the racetrack system using web browsers via http . on pacs and dictation workstations , scripts extract working context and feed that information into web browsers for more convenient case creation . the racetrack application stack consists of a http server , application server , and database server , which all run in a linux virtual machine . the application communicates with an external authentication system via encrypted http to provide a single enterprise - wide sign - on for users . the racetrack application also communicates with the radiology information system ( ris ) via odbc protocol to retrieve information about radiology examinations and the queried patient inference dossier ( qpid ) server via http to retrieve other medical record information . on the client side , users at our institution are standardized on the internet explorer version 6 or 7 web browsers . autohotkey scripts automatically extract the accession number of the current exam from either the pacs environment or the dictation system and open a web browser window with this information pre - filled to minimize manual entry . this system and its integration with other radiology information systems allows radiologists to create , manipulate , and follow their tracked cases with a minimum of effort and interruption of their clinical work ( fig . the process begins when the radiologist presses a  hotkey  in either the pacs or the dictation environment . this opens a simple entry form that is automatically pre - filled with study information gathered from the radiologist s current work context . the radiologist types a brief description of his or her interest in the case , and dismisses the form . the system then creates an entry in the radiologist s personal database of tracked cases . later , to review cases , the radiologist logs in to the racetrack system from an office computer , a personal laptop , or a home computer via a virtual private network . the system displays a personalized table of tracked cases that includes links directly to the patient s emr entry and to a web - based pacs display of the study of interest ( fig . a radiologist who sees an examination of interest during her clinical work ( 1 ) uses a hotkey to bring up a case - entry tool with the accession number pre - entered . she enters a brief description of her interest in the case ( 2 ) , and returns to her work . the racetrack system retrieves ancillary patient and exam information from the radiology information system ( ris ; 3 ) , and stores the description and other metadata in a database for the radiologist ( 4 ) . the system then periodically queries the patient s medical records via the queried patient inference dossier ( qpid ; 5 ) . at her leisure , the radiologist can return to view her database of cases ( 6 ) , which includes potentially useful medical record information such as pathology reports and operative notes ( 7 ) . she can choose to form an annotated link between her description of the case and these medical record items ( 8).fig  3personal database of tracked cases . a radiologist s personal list of cases includes information from the radiology information system ( ris ) , such as examination description and date , and also patient demographic information , including medical record number ( mrn ) . for each case in the list , the accession number is displayed ( blurred in this image for privacy ) as a link directly to the examination in a web - based pacs application . the patient mrn is displayed ( blurred here ) as a link to the patient s electronic medical record entry . the case description links to a more detailed display of the case ( see fig . a radiologist who sees an examination of interest during her clinical work ( 1 ) uses a hotkey to bring up a case - entry tool with the accession number pre - entered . she enters a brief description of her interest in the case ( 2 ) , and returns to her work . the racetrack system retrieves ancillary patient and exam information from the radiology information system ( ris ; 3 ) , and stores the description and other metadata in a database for the radiologist ( 4 ) . the system then periodically queries the patient s medical records via the queried patient inference dossier ( qpid ; 5 ) . at her leisure , the radiologist can return to view her database of cases ( 6 ) , which includes potentially useful medical record information such as pathology reports and operative notes ( 7 ) . she can choose to form an annotated link between her description of the case and these medical record items ( 8) . personal database of tracked cases . a radiologist s personal list of cases includes information from the radiology information system ( ris ) , such as examination description and date , and also patient demographic information , including medical record number ( mrn ) . for each case in the list , the accession number is displayed ( blurred in this image for privacy ) as a link directly to the examination in a web - based pacs application . the patient mrn is displayed ( blurred here ) as a link to the patient s electronic medical record entry . the case description links to a more detailed display of the case ( see fig . further , the integration with emr systems helps radiologists place radiology examinations in a broader clinical context . racetrack periodically searches the patient s record via the qpid system for subsequent relevant clinical information such as new operative notes , pathology reports , or follow - up imaging examinations . when new data are found , they are displayed as part of the case description in the racetrack tool without the radiologist having to pore through medical records manually ( fig . the radiologist can then choose relevant reports and create annotated links to these selected medical record items that are maintained with the radiologist s personal description of the case . this drastically decreases the amount of work required for radiologists to find the subsequent clinical course of their patients and keep track of subsequent diagnoses . 
 fig  4detailed information on a tracked case . the detailed case display integrates information from both the radiology information system ( ris ) , the electronic medical record ( emr ) , and the radiologist s own metadata . in the case displayed , the radiologist has made an annotated link to a pathology report which has diagnosed the tumor seen on the examination as a bronchial carcinoid ( under  relevant clinical information  ) . other potentially relevant radiology exams and medical record entries are available for linking (  other radiology exams  and  other medical record information  ) . the detailed case display integrates information from both the radiology information system ( ris ) , the electronic medical record ( emr ) , and the radiologist s own metadata . in the case displayed , the radiologist has made an annotated link to a pathology report which has diagnosed the tumor seen on the examination as a bronchial carcinoid ( under  relevant clinical information  ) . other potentially relevant radiology exams and medical record entries are available for linking (  other radiology exams  and  other medical record information  ) . as an example , consider a busy radiologist interpreting a series of ct scans in the reading room . on a non - enhanced scan of a middle - aged man s abdomen , he sees a renal mass with an unusual but non - specific appearance and recommends follow - up imaging to further characterize it . since he wishes to find out himself what the subsequent workup of the lesion reveals , he activates the racetrack tool . a form is brought up which is pre - filled with the accession number for the study he is viewing . he types a brief description (  strange renal mass , not definitely cystic  ) , indicates that he wishes to receive follow - up , and dismisses the form . the case is now stored in his personal database of cases , and he can return to his queue of studies to be interpreted . later that week , he logs in to the racetrack system from his office computer and peruses his list of cases . he notices the new entry , remembers the case , and uses the link to the web - based pacs system to review the images . he shows the mass to a colleague , who is equally mystified , which further raises his interest in the case . three weeks later , the radiologist is again reviewing his list of cases , and notices a flag on the case of the renal mass , which indicates that new clinical information is available . he clicks on the case , and finds that there has been a contrast - enhanced renal mr exam ( read by one of his colleagues ) , which demonstrates enhancement in a solid portion of the mass , raising suspicion of neoplasm . he adds an annotated link to the mr report to his record of the case (  mass shows enhancement  ) . finally , a month later , while going through his case list , he finds that there has been a nephrectomy and the surgical pathology report is available : the lesion is diagnosed as an oncocytoma . the radiologist creates another annotated link to the pathology report with the text  oncocytoma  and a  diagnostic flag raised indicating that the item provides the diagnosis for the case . as a result of using the racetrack system , the radiologist has found the clinical and pathological outcome of the workup initiated by his recommendation . he can return to the case as needed for reference . at the time of this writing eight volunteer radiologists act as alpha users , and they have already created more than 600 cases for follow - up . users have used a web - based issue - tracking application to report problems with the application and request enhancements . one radiologist maintains an extensive teaching - file collection ; she reports that the system improves her efficiency in tracking potential teaching cases for later follow - up ( l. avery , personal communication ) . another user has used the system to maintain lists of cases for an irb - approved research project , and reports that the maintained list and easy access to emr systems facilitates finding clinical outcomes for included cases ( m. cushing , personal communication ) . the case - tracking system we have created allows radiologists to receive clinical feedback on more of their cases more efficiently . this changed dynamic is already demonstrating that it can improve ancillary processes in radiology : both teaching file creation and research have been made easier by lowering barriers between radiologists and outcome information . more importantly , as more radiologists in our department use the system , we hope that it will become routine for them to receive diagnostic feedback . we expect that this system will contribute to improved diagnostic quality by providing radiologists more opportunities for self - assessment and better integrating them with longitudinal patient care . we anticipate that radiologists in training will find the tool especially useful as they develop their skills , but expect that practicing radiologists undergoing maintenance of certification will also use the tool to gather material to meet the new requirement for practice quality improvement11,12 . in addition to extending the system to a larger group of users , we intend to make several enhancements . we are also developing a search tool that will allow users to quickly find cases based on text contained in their descriptions and other annotations . as an adjunct to both tagging and searching , we intend to add automatic indexing and tagging against radlex13 to increase the specificity of available semantic information . the system will also soon include the ability to export anonymized representations of cases to presentation software formats ( such as powerpoint ) , to a medical imaging resource center server14 , or to online systems such as the radiological society of north america s myrsna service15 . in addition , we would like to hone the process by which the racetrack system finds and presents potentially relevant entries in the electronic medical record . currently , the system returns all subsequent radiology exams , pathology reports , operative notes , and endoscopy results . our goal is to devise algorithms that rank medical record entries based on how relevant they are to the case and the radiologist s interest in it . in addition to enhancements that improve the ability of radiologists to organize their cases and access subsequent patient information , we anticipate using the racetrack system as a platform for developing new applications for tracking radiology cases . one potential application is to develop a tool radiologists use to flag cases as having a technical quality issue . when a radiologist interprets such an examination , they can create a  technical quality ticket  based on the examination in question . this ticket would include relevant information from the ris without requiring manual re - entry , and would then be added to a queue for technical managers to review . the inclusion of this embedded information will increase the efficiency of exploring and resolving these issues compared to ad hoc workflows . in addition , since the cases are stored in a database , they can be queried to generate metrics and mined for potential sources of systemic problems . we also hope to apply the racetrack system to the workflow surrounding radiologist - recommended follow - up examinations . a recommendation - tracking system could automatically capture these recommendations , and then monitor patients subsequent records for the recommendation being followed ( or not followed ) . this information could then be used to send reminders to a patient s physicians when a recommended study has not been obtained in a timely manner . alternatively , when recommended studies are obtained , the system can bring those studies to the attention of the radiologist who originally recommended them , thereby closing the loop they opened . we have implemented a radiology case - tracking system that allows radiologists to create and maintain personal databases of cases of interest . by integrating this tool with a medical record aggregation and search system , we have reduced the energy barrier for radiologists to follow clinical outcomes of their patients . in addition , we hope to use the system as a platform for developing applications for quality improvement in radiology .
radiologists make many diagnoses , but only sporadically get feedback on the subsequent clinical courses of their patients . 
 we have created a web - based application that empowers radiologists to create and maintain personal databases of cases of interest . 
 this tool integrates with existing information systems to minimize manual input such that radiologists can quickly flag cases for further follow - up without interrupting their clinical work . 
 we have integrated this case - tracking system with an electronic medical record aggregation and search tool . as a result 
 , radiologists can learn the outcomes of their patients with much less effort . 
 we intend this tool to aid radiologists in their own personal quality improvement and to increase the efficiency of both teaching and research . 
 we also hope to develop the system into a platform for systematic , continuous , quantitative monitoring of performance in radiology .