BACKGROUND 
       [0001]    After physicians and other healthcare professionals (referred to herein generally as “healthcare providers”) provide healthcare services to patients, bills for such services must be generated. The process of generating such bills based on the set of clinical reports associated with a patient encounter (referred to as a “chart”) can be a tedious, time-consuming, risky, and error-prone process for a variety of reasons, such as:
       Laws, regulations, and institutional policies prescribe that bills satisfy various rules, such as rules requiring that each item in a bill be justified by adequate supporting evidence. Such rules can be difficult to identify and interpret, and the required evidence can be difficult to find and evaluate.   Bills must be encoded using billing codes specified by technical billing code standards such as ICD-9, ICD-10, and CPT. Such standards can be difficult to understand and apply in particular situations in light of the services provided and the available evidence. Furthermore, as older standards (such as ICD-9) are replaced with newer, more complex, standards (such as ICD-10), the difficulty of understanding the applicable standards is increasing.   Bills often must be generated quickly due to time and budget constraints.   The error rate in bills, including both false positives and false negatives, must be kept to a minimum. False positives (including items in bills that should not be included, such as because they are not justified by available evidence) may violate applicable laws, regulations, and/or institutional policies. False negatives (failing to include items in bills that should be included) lead to lost revenue for the healthcare provider.   Billing codes are typically generated by specialized “billing coders,” who must be trained to select the appropriate codes based on the documentation provided by the healthcare providers. Training a billing coder can be time-consuming and expensive, and even expert billing coders can make mistakes.       
 
         [0007]    These problems are likely to be exacerbated by the transition to the ICD-10 billing code standard. Expert billing coders, who are fluent in ICD-10, are in short supply and are unlikely to meet the demand for such billing coders. 
         [0008]    In order to address this shortfall in supply of expert billing coders, many healthcare providers have either been outsourcing their coding process to service companies or attempting to automate the coding process using Computer Assisted Coding (CAC) technology. 
         [0009]    Both outsourcing and automation have associated drawbacks. For example, because the ability to perform billing coding accurately and completely directly impacts the cash flow and overall profitability of healthcare organizations, such organizations are reluctant to rely on an outsourced workforce. Another drawback of outsourced billing coding is that the ultimate responsibility, and legal liability, for the accuracy of billing coding lies with the healthcare organization, few (if any) outsourced billing coding providers are willing to indemnify a sizable healthcare organization against liability incurred as the result of billing coding errors. As a result, even healthcare organizations that are willing to outsource may not be able to outsource all of their billing coding needs to billing coding providers who can satisfy exacting quality and legal requirements. 
         [0010]    CAC solutions have their own problems. CAC solutions apply Natural Language Processing (NLP) technology to compute the most likely set of billing codes from a set of clinical reports before a human coder reviews the chart. Some CAC solutions can, in addition, create confidence scores that estimate the likelihood that any given code, or the complete coding of a chart, is correct. Some CAC solutions provide the option of bypassing the human coder completely, for at least a subset of charts, if the chart-level confidence score is sufficiently high. The state of the art of such fully-automated coding, however, is not sufficiently accurate to be relied upon in practice for anything but the most simple charts. More complex charts, which are the norm in practice, cannot be accurately coded using fully-automated coding. As a result, in practice it is necessary, in most cases, for a human coder to review the automatically-generated codes for accuracy and to revise such codes as necessary. 
         [0011]    The promise of CAC solutions, even when the codes that they generate must be reviewed by a human coder, is to provide an increase in efficiency in comparison with a system that relies solely on human coders, by providing the initial set of codes for review quickly and accurately enough that the combination of generating codes automatically followed by human review and correction of those codes is more efficient and inexpensive than purely human code generation. In practice, however, CAC systems do not always increase productivity as much as is theoretically possible. Furthermore, deploying CAC systems requires a lengthy and labor-intensive tuning process to adapt the CAC technology to the idiosyncrasies of a healthcare provider. The result is that productivity during the tuning process can be impacted negatively, and the resulting overall productivity may be lower than if no CAC system were used at all. 
         [0012]    What is needed, therefore, are techniques for overcoming the problems of conventional CAC systems, and for otherwise improving the efficiency of generating billing codes. 
       SUMMARY 
       [0013]    A computer system increases the efficiency with which billing codes may be generated based on a chart, such as a medical chart. The computer system provides the chart to a computer-assisted coding (CAC) module, which produces an initial set of billing codes and an initial assessment of the accuracy and/or completeness of the codes. The computer system decides whether to send the initial set of billing codes to an initial human reviewer. If the computer system sends the initial set of billing codes to the initial human reviewer, then the initial human reviewer reviews the chart and the output of the CAC module, and attempts to fix errors in the CAC output. The system provides the chart and the current (initial or modified) codes to a final human reviewer, who may be more highly skilled than the initial human reviewer, for final verification and modification. 
         [0014]    Other features and advantages of various aspects and embodiments of the present invention will become apparent from the following description and from the claims. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0015]      FIG. 1  is a dataflow diagram of a system for generating billing codes according to one embodiment of the present invention; and 
           [0016]      FIG. 2  is a flowchart of a method performed by the system of  FIG. 1  according to one embodiment of the present invention. 
       
    
    
     DETAILED DESCRIPTION 
       [0017]    Embodiments of the present invention include computer systems which may be used to improve the efficiency with which billing codes may be generated. Referring to  FIG. 1 , a dataflow diagram is shown of a system  100  for generating billing codes according to one embodiment of the present invention. Referring to  FIG. 2 , a flowchart is shown of a method  200  performed by the system  100  of  FIG. 1  according to one embodiment of the present invention. 
         [0018]    The system  100  includes a document corpus  102 , which includes a plurality of documents  104   a - n , where n may be any number. The document corpus  102  is merely one example of a “data set” as that term is used herein. The term “document” is used generally herein to include any type of data record, such as a freeform text document (e.g., a plain text document or a document created using a word processing application), a structured document (such as an XML document), a scanned document (e.g., a scan of handwritten progress notes), or a data record in a database. A document may, for example, be an Electronic Medical Record (EMR) or Electronic Health Record (EHR). Structured documents in the document corpus  102  may, for example, have been created using techniques disclosed in U.S. Pat. No. 7,584,103 B2, issued on Sep. 1, 2009, entitled, “Automated Extraction of Semantic Content and Generation of a Structured Document from Speech.” 
         [0019]    The document corpus  102  may include documents of different types, such as text documents and EHRs. Although  FIG. 1  shows just one document corpus  102 , the document corpus  102  may include multiple data sets, such as multiple databases, files stored in multiple file systems, multiple EMR/EHR databases, or any combination thereof. As these examples illustrate, the document corpus  102  may include documents stored on a plurality of storage media (e.g., hard drives) and/or maintained by multiple computer systems. 
         [0020]    For ease of explanation, certain embodiments will be disclosed herein in connection with documents which take the form of clinical reports describing patient encounters. Examples of clinical reports include admission orders, discharge orders, and prescriptions. A plurality of clinical reports relating to a particular patient encounter is referred to herein as a “chart.” Often, the process of generating a set of billing codes involves generating a set of billing codes based on the clinical reports in a particular chart. The resulting billing codes may be represented in any manner, such as according to billing code standards such as any one or more of HL7 CDA v2 XML standard (ANSI-approved since May 2005), SNOMED CT, LOINC, CPT, ICD-9 and ICD-10, and UMLS. 
         [0021]    Assume, solely for purposes of example, that the system  100  includes a chart  106 , which includes documents  104   a - b  from the document corpus. As mentioned above, a chart may, more generally, including one or more documents relating to a particular patient encounter. Although the chart  106  is shown as being contained within the document corpus  102  in  FIG. 1 , the chart  106  need not exist as a data structure within the document corpus  102 . Instead, for example, another component of the system  100  may store data (e.g., in a separate database) indicating that the chart  106  includes the documents  104   a  and  104   b.    
         [0022]    Now assume that the chart  106  is ready to be used to generate a set of billing codes based on the chart  106 . Further assume that the system  100  includes a computer-assisted coding (CAC) module  108 . The CAC module  108  may include any number and type of computer hardware, computer software, networking equipment, and interconnections sufficient to enable the CAC module  108  to perform computer-assisted coding. The CAC module  108  is configured to perform, or to attempt to perform, computer-assisted coding without the involvement of a human, except that a human may provide the chart  106  to the CAC module  108  and interpret the output of the CAC module  108 . 
         [0023]    The system  100  may include a chart routing module  150 , which may receive some or all of the chart  106  as input, and which may determine whether the chart  106  qualifies for fully-automated processing by the CAC module  108  ( FIG. 2 , operation  202 ). The chart routing module  150  may make this determination in any of a variety of ways. For example, the chart routing module  150  may determine whether the chart  106  includes any scanned handwritten notes and determine that the chart  106  does not qualify for fully-automated processing by the CAC module  108  in response to determining that the chart  106  includes at least one scanned handwritten note. As another example, the chart routing module  150  may determine whether the chart  106  has a high confidence of being coded accurately by the CAC module  108 . The chart routing module  150  may make this determination by, for example, determining whether the chart  106  describes a complex medical procedure, and concluding that the chart  106  has a high confidence of being coded accurately by the CAC module  108  only if the chart  106  does not describe a complex medical procedure. The system  100  may determine whether a medical procedure is “complex” in any of a variety of ways, such as by determining whether the procedure lasted more than a predetermined amount of time (e.g., 6 hours), or by determining whether the patient who is the subject of the procedure stayed in the hospital for longer than some predetermined amount of time (e.g., 3 days). As yet another example, the chart routing module  150  may conclude that a medical procedure is “complex” and requires non-automated coding if the lab values of the patient who was the subject of the medical procedure has non-normal lab values. Regardless of how the chart routing module  150  determines that the chart  106  has a high confidence of being coded accurately by the CAC module  108 , the chart routing module  150  may determine that the chart  106  qualifies for fully-automated processing by the CAC module  108  in response to determining that the chart  106  has a high confidence of being coded accurately by the CAC module  108 . 
         [0024]    If the chart routing module  150  determines that the chart  106  qualifies for fully-automated processing, the chart routing module  150  may provide output  106   a  representing the chart  106  to the CAC module  108  for processing ( FIG. 2 , operation  204 ). For ease of explanation, the following description will refer to the CAC module  108  as operating on the chart  106  instead of the chart output  106   a . The CAC module  108  may receive the chart  106  and perform computer-assisted coding on the chart  106  to produce CAC output  110  ( FIG. 2 , operation  206 ). The CAC output  110  may include one or more of the following:
       A set  112  of initial billing codes, including one or more billing codes generated by the CAC module  108  based on the chart  106 .   Initial code assessment data  114 , representing an overall assessment of the completeness and/or correctness of the initial billing codes  112 .   Links  132  to evidence that is relevant to the set  112  of initial billing codes. Such evidence may include, for example, one or both of the following: (1) evidence that was found by the CAC module  108  but that the CAC module did not rely upon to generate the set  112  of billing codes; and (2) evidence that was found by the CAC module  108  and that the CAC module did rely upon to generate the set  112  of billing codes.       
 
         [0028]    The initial code assessment data  114  may include any of a variety of data. For example the initial code assessment data  114  may include any one or more of the following:
       data representing an overall assessment of the completeness of the initial billing codes  112 ;   data representing a completeness confidence score indicating a confidence that the initial billing codes  112  are complete, i.e., that the initial billing codes  112  include all billing codes that can be generated based on the chart  106 ;   data representing a correctness confidence score indicating an overall confidence that the initial billing codes  112  are correct, i.e., that the initial billing codes  112  accurately encode billing information for the healthcare services represented by the chart  106 ;   data representing an overall classification of the initial billing codes  112 , such as “likely completely correct,” “requires review,” or “known deficiencies.”       
 
         [0033]    The initial code assessment data  114  may indicate that the initial billing codes  112  are likely completely correct based on, for example, a function of the completeness confidence score and/or the correctness confidence score. For example, if the correctness confidence score exceeds a first predetermined threshold (e.g., 95%) and the completeness confidence score exceeds a second predetermined threshold (e.g., 90%), then the system  100  may conclude that the initial billing codes  112  are likely completely correct. 
         [0034]    The initial code assessment data  114  may indicate that the initial billing codes  112  contain a known deficiency if, for example, the chart  106  is missing a required report, such as a “Discharge Summary” report. 
         [0035]    The system  100  may also include a review determination module  152 , which may receive some or all of the CAC output  110  as input, and which may determine whether the initial code assessment data  114  indicates that the chart  106  has been classified as requiring review ( FIG. 2 , operation  208 ). If the review determination module  152  determines that the chart  106  has been classified as requiring review, then the review determination module  152  may provide output  154  containing or otherwise representing the chart  106  and the CAC output  110  (or a portion thereof) to an initial human reviewer  116 , such as by transmitting the review output  154  (e.g., the chart  106  and CAC output  110 ) over a network to a computing device  118  used by or otherwise associated with the initial human reviewer  116  ( FIG. 2 , operation  210 ). The initial human reviewer  116  may, for example, be a junior or relatively inexperienced and/or unskilled reviewer (e.g., billing coder). If the review determination module  152  determines that the chart  106  has not been classified as requiring review, then the review determination module  152  may produce output  110 ′ representing the CAC output  110 , and provide the output to a coding output routine module  156 , which is described in more detail below. 
         [0036]    Although not shown in  FIG. 2 or 3 , the review determination module  152  may determine whether providing the CAC output  110  to the initial human reviewer  116  is likely to add value to the CAC output  110 , and only provide the CAC output  110  to the initial human reviewer  116  in response to determining that the initial human reviewer  116  is likely to add value to the CAC output  110 . For example, the review determination module  152  may determine, in operation  208 , whether the chart  106  requires initial human review, and also determine whether review of the CAC output  110  by the initial human reviewer  116  is likely to add value to the CAC output  110 , and then only provide the CAC output  110  to the initial human reviewer  116  if the review determination module  152  determined that the chart  106  requires initial human review and that the initial human reviewer  116  is likely to add value to the CAC output  110 . 
         [0037]    The review determination module  152  may determine whether the initial human reviewer  116  is likely to add value to the CAC output  110  in any of a variety of ways. For example, the review determination module  152  may make this determination in any of the ways disclosed herein by which the chart routing module  150  may determine whether the chart  106  describes a complex medical procedure. The review determination module  152  may, for example, use any such technique to determine whether the CAC output  110  and/or the chart  106  describes a complex medical procedure, and not provide the CAC output  110  to the initial human reviewer  116  in response to determining that the CAC output  110  and/or the chart  106  describes a complex medical procedure. 
         [0038]    As another example, the review determination module  152  may determine whether the initial human reviewer  116  is likely to add value to the CAC output  110  by determining whether the initial human reviewer  116  is sufficiently skilled to add value to the CAC output  110 . The review determination module  152  may, for example, determine whether the initial human reviewer  116  is sufficiently skilled to add value to the CAC output  110 , and not provide the CAC output  110  to the initial human reviewer  116  in response to determining that the initial human reviewer  116  is not sufficiently skilled to add value to the CAC output  110 . The review determination module  152  may determine whether the initial human reviewer  116  is sufficiently skilled to add value to the CAC output  110  in any of a variety of ways. For example, the review determination module  152  may determine whether a skill value associated with the initial human reviewer  116  satisfies a skill criterion (e.g., exceeds a maximum predetermined value), and not provide the CAC output  110  to the initial human reviewer  116  in response to determining that the skill value does not satisfy the skill criterion. 
         [0039]    If the chart routing module  150  previously determined (in operation  202  of  FIG. 2 ) that the chart  106  did not qualify for fully-automated coding, then the chart routing module  150  may provide the chart  106  to the initial human reviewer  116  ( FIG. 2 , operation  220 ). In other words, if the CAC module  108  processes the chart  106  to produce the CAC output  110 , then the CAC output  110  may be provided to the initial human reviewer  116 , whereas if the CAC module  108  does not process the chart  106 , then the chart  106  may be provided to the initial human reviewer  116 . If the chart  106  is provided to the initial human reviewer  116 , then the initial human reviewer  116  may manually code the chart  106  to produce modified coding output  124 , without the use of the CAC module  108 . The following discussion, however, assumes that the automatically-generated CAC output  110  is provided to the initial human reviewer  116 . 
         [0040]    Any of the operations described herein as being performed in connection with the output  110  of the CAC module  108  may alternatively be performed on the output  124  of the initial human reviewer  116  (e.g., if the chart output  106   a  is provided to the human reviewer  116  but not to the CAC module  108 ). Furthermore, any operations described herein as being performed on the output  110  of the CAC module  108  may be performed on both the output  110  of the CAC module  108  and the output  124  of the human reviewer  116 . 
         [0041]    The initial human reviewer  116  may review the initial billing codes  112  for completeness and/or correctness. Before doing so, however, the initial human reviewer  116  may determine whether to review the initial billing codes  112 . For example, the initial human reviewer  116  may determine whether the chart  106  can be coded (i.e., whether the initial billing codes  112  can be modified) with high confidence. If the initial human reviewer  116  determines that the chart  106  cannot be coded with high confidence, then the initial human reviewer  116  may skip the following steps involving reviewing the initial billing codes  112  for completeness and/or correctness. 
         [0042]    As another example, the initial human reviewer  116  may determine whether the total amount of reimbursement represented by the chart  106  and/or the initial billing codes  112  exceeds some predetermined threshold amount, such as an average reimbursement amount or an approved reimbursement amount. If the initial human reviewer  116  determines that the total amount of reimbursement exceeds the predetermined threshold amount, then the initial human reviewer  116  may skip the following steps involving reviewing the initial billing codes  112  for completeness and/or correctness. 
         [0043]    The initial human reviewer  116  may, based on any combination of the chart  106 , the initial billing codes  112 , and the initial code assessment  114 , modify the initial billing codes  112  in an attempt to increase their completeness and improve their correctness, thereby producing a set  120  of modified billing codes  120 . The initial human reviewer  116  may also modify the initial code assessment  114  to indicate, for example, the initial human reviewer  116 &#39;s assessment of the completeness and/or correctness of the modified billing codes  120 , thereby producing a modified code assessment  122 . Both the modified billing codes  120  and the modified code assessment  122  may be part of modified coding output  124  produced by the initial human reviewer  116 . 
         [0044]    The initial human reviewer  116  may also perform additional tasks based on the chart  106  and/or the CAC output  110 . One purpose of these additional tasks may be to assist a subsequent human reviewer in reviewing the chart  106  and/or the modified coding output  124 . For example, the initial human reviewer  116  may perform any one or more of the following additional tasks based on the chart  106  and/or the CAC output  110 :
       sort clinical documents in the chart  106  in order of decreasing relevance;   mark individual documents within the chart  106  as relevant and/or irrelevant; and   mark the text positions of relevant portions of documents within the chart  106 , and/or extract such relevant portions from the documents.       
 
         [0048]    Data representing the results of any such additional actions may be stored within the modified coding output  124 . For example, the modified coding output  124  may include data representing the initial human reviewer  116 &#39;s selected sort order of documents within the chart  106 . The initial human reviewer  116 , via the computing device  118 , may provide output  124 ′ containing or otherwise representing the modified coding output  124  back to the system  100  ( FIG. 2 , operation  212 ). 
         [0049]    The system  100  may include a coding output routing module  156 . The coding output routing module  156  may provide a final human reviewer  126  with final coding output  130 , which may include and/or be derived from either:
       the CAC output  110  or  110 ′, if the CAC output  110  was not provided to or modified by the initial human reviewer  116 ; or   the modified coding output  124  or  124 ′, if the CAC output  110  was modified by the initial human reviewer  116  to produce the modified coding output  124 .       
 
         [0052]    The final coding output  130  may also include the chart  106  and/or data derived from the chart  106 . The coding output routing module  152  may, for example, provide the chart  106  and the final coding output  130  to the final human reviewer  126  by transmitting the chart  106  and the final coding output  130  over a network to a computing device  128  used by or otherwise associated with the final human reviewer  126  ( FIG. 2 , operation  214 ). The final human reviewer  126  may, for example, be a senior or relatively experienced or expert reviewer (e.g., billing coder), and may be a different person than the initial human reviewer  116 . The final coding output  130  may, for example, be:
       the CAC output  110 , if the CAC output  110  was not sent to or modified by the initial human reviewer  116 , in which case the final coding output  130  may include both the initial billing codes  112  and the initial code assessment  114 ; or   the modified coding output  124 , if the initial human reviewer  116  modified the CAC output  110  to produce the modified coding output  124 , in which case the final coding output may include the modified billing codes  120  and the modified code assessment  122 .       
 
         [0055]    The final human reviewer  126  may review the chart  106  and the final coding output  130 , and analyze them for any of a variety of purposes. For example, the final human reviewer  126  may choose to perform any one or more of the following, in any combination:
       allow codes classified as “likely completely correct” to be submitted for billing without further review;   allow codes classified as “likely completely correct” to be submitted for billing without further review, except for a randomly selected sample of codes, which may be processed by a quality assurance procedure to verify their accuracy;   if the chart  106  (or specific codes therein) has been classified as “requiring further review,” then use the modified coding output  124  to code the chart  106  (or the specific codes therein); and   if the chart  106  (or specific codes therein) has been classified as “known deficiencies,” then route the chart (or specified codes therein) to a specialized workflow.       
 
         [0060]    Although  FIG. 1  only shows a single chart  106 , the system  100  of  FIG. 1  may repeat the method  200  of  FIG. 2  for any number of charts. 
         [0061]    Embodiments of the present invention have a variety of advantages, such as the following. In general, embodiments of the present invention address shortcomings of CAC technology, by allowing healthcare providers to obtain the efficiency benefits of CAC technology, while staying in full control of the coding process and without sacrificing quality. In particular, embodiments of the present invention may use a combination of automated (CAC) technology and human reviewers, structured and sequenced in a particular manner, to leverage the efficiency gains of CAC while using human reviewers to ensure accuracy. 
         [0062]    Even more specifically, the use of the initial human reviewer  116  enables the system  100  and method  200  to catch certain errors in the CAC output  110 . Using a combination of the CAC module  108  and the initial human reviewer  116  may provide a higher quality output than that produced by the CAC module  108  alone, and at a lower cost than using a highly-trained human reviewer alone, depending on the relative costs and accuracies of the CAC module  108  and the initial human reviewer  116 . 
         [0063]    Furthermore, the initial human reviewer  116  may be relatively unskilled and be capable of correcting only relatively simple errors. Even so, the system  100  as a whole may be more efficient (measured, for example, in terms of accuracy per unit cost) and/or more accurate overall than the CAC module  108  itself, when the function performed by the final human reviewer  126  is taken into account. For example, if the final human reviewer  126  is an expert billing coder, then the final human reviewer  126  may catch and correct errors produced by the CAC module  108  that were not corrected by the initial human reviewer  116 , thereby increasing the accuracy of the final coding output  130 . Even if the cost of the final human reviewer  126  is relatively high (as measured, e.g., in terms of hourly wages), the overall cost of the system  100  may still be acceptable if the number of codes reviewed, and therefore the amount of time spent, by the final human reviewer  126  is relatively small. The system  100 &#39;s use of the CAC module  108  and the initial human reviewer  116 , and in particular the system  100 &#39;s use of the initial code assessment  114  and the modified code assessment  122 , enables the system  100  to limit the number of codes that the final human reviewer  126  must review, so that the cost of the final human reviewer  126  is kept low and so that the final human reviewer  126  is used to review and correct only relatively complex codes for which the expert skills of the final human reviewer  126  are required. 
         [0064]    In addition to increasing the efficiency of the coding process, the system  100  and method  200  may increase the overall accuracy of the system  100  in comparison to a purely automated system (e.g., the CAC module  108 ). As described above, the CAC module  108  may produce erroneous codes, especially in complex situations. The initial human reviewer  116  and the final human reviewer  126  may correct such codes. As a result, the system  100  may increase the accuracy of the final coding output  130  in comparison to the automatically-generated codes  112  produced by the CAC module  108 . 
         [0065]    One benefit of the system  100 , therefore, is that it uses the CAC module  108  to produce the codes  112  automatically, and that it performs additional steps which increase the accuracy of the final coding output  130  in comparison to the codes  112  produced solely by the CAC module  108 . The system  100  may, therefore, be seen as an improved computer system for generating billing codes. The system  100 , therefore, solves the technical problem of how to increase the accuracy of the codes produced by a computer-automated coding module. 
         [0066]    Furthermore, the system  100  and method  200  enable certain charts to be coded (at least in part) automatically, while also enabling codes to be generated based on charts containing clinical reports that cannot be processed automatically, such as clinical reports in the form of scanned handwritten notes. The system  100  and method  200  may code such clinical reports by routing those reports to the initial human reviewer  126 , who may generate an initial set of codes, and by then routing the initial set of codes to the final human reviewer  126  for review and correction. In this way, the system  100  and method  200  obtain the advantages of both the automated CAC module  108  and of the manual skill of the initial human reviewer  116  and the final human reviewer  126 . 
         [0067]    As described above, one function performed by the chart routing module  150  is to determine whether the chart  106  is to determine whether the chart  106  qualifies for fully-automated processing by the CAC module  108 . Another, related, function performed by the chart routing module  150  is to determine the right time at which to submit the chart  106  to the CAC module  108  and/or to a human coder for coding. For example, the chart routing module  150  may be adapted not to submit the chart  106  (e.g., to the CAC module  108 ) for coding unless and until a discharge summary has been received (e.g., unless and until the chart  106  includes a discharge summary). The chart routing module  150  may further be adapted to submit the chart  106  (e.g., to the CAC module  108 ) after some predetermined maximum amount of time has passed, even if no discharge summary has been received (e.g., even if the chart  106  does not include a discharge summary). This is merely one example of a way in which the chart routing module  150  may determine the right time at which to submit the chart  106  for coding. 
         [0068]    It is to be understood that although the invention has been described above in terms of particular embodiments, the foregoing embodiments are provided as illustrative only, and do not limit or define the scope of the invention. Various other embodiments, including but not limited to the following, are also within the scope of the claims. For example, elements and components described herein may be further divided into additional components or joined together to form fewer components for performing the same functions. 
         [0069]    Any of the functions disclosed herein may be implemented using means for performing those functions. Such means include, but are not limited to, any of the components disclosed herein, such as the computer-related components described below. 
         [0070]    The techniques described above may be implemented, for example, in hardware, one or more computer programs tangibly stored on one or more computer-readable media, firmware, or any combination thereof. The techniques described above may be implemented in one or more computer programs executing on (or executable by) a programmable computer including any combination of any number of the following: a processor, a storage medium readable and/or writable by the processor (including, for example, volatile and non-volatile memory and/or storage elements), an input device, and an output device. Program code may be applied to input entered using the input device to perform the functions described and to generate output using the output device. 
         [0071]    Each computer program within the scope of the claims below may be implemented in any programming language, such as assembly language, machine language, a high-level procedural programming language, or an object-oriented programming language. The programming language may, for example, be a compiled or interpreted programming language. 
         [0072]    Each such computer program may be implemented in a computer program product tangibly embodied in a machine-readable storage device for execution by a computer processor. Method steps of the invention may be performed by one or more computer processors executing a program tangibly embodied on a computer-readable medium to perform functions of the invention by operating on input and generating output. Suitable processors include, by way of example, both general and special purpose microprocessors. Generally, the processor receives (reads) instructions and data from a memory (such as a read-only memory and/or a random access memory) and writes (stores) instructions and data to the memory. Storage devices suitable for tangibly embodying computer program instructions and data include, for example, all forms of non-volatile memory, such as semiconductor memory devices, including EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROMs. Any of the foregoing may be supplemented by, or incorporated in, specially-designed ASICs (application-specific integrated circuits) or FPGAs (Field-Programmable Gate Arrays). A computer can generally also receive (read) programs and data from, and write (store) programs and data to, a non-transitory computer-readable storage medium such as an internal disk (not shown) or a removable disk. These elements will also be found in a conventional desktop or workstation computer as well as other computers suitable for executing computer programs implementing the methods described herein, which may be used in conjunction with any digital print engine or marking engine, display monitor, or other raster output device capable of producing color or gray scale pixels on paper, film, display screen, or other output medium. 
         [0073]    Any data disclosed herein may be implemented, for example, in one or more data structures tangibly stored on a non-transitory computer-readable medium. Embodiments of the invention may store such data in such data structure(s) and read such data from such data structure(s).
A computer system increases the efficiency with which billing codes may be generated based on a chart, such as a medical chart. The computer system provides the chart to a computer-assisted coding (CAC) module, which produces an initial set of billing codes and an initial assessment of the accuracy and/or completeness of the codes. The computer system decides whether to send the initial set of billing codes to an initial human reviewer. If the computer system sends the initial set of billing codes to the initial human reviewer, then the initial human reviewer reviews the chart and the output of the CAC module, and attempts to fix errors in the CAC output. The system provides the chart and the current (initial or modified) codes to a final human reviewer, who may be more highly skilled than the initial human reviewer, for final verification and modification.