Greatly improved regex for direct matching with custom entities

Uploaded pdfs with review files will now include all pages that don't have redactions. Slightly improved deny list matching.

Fixed bug where pages suggested for whole redaction are one lower than requested

Fix bug to identify all handwriting labels. Now only concatenates entity_type boxes if they have different labels.

Fixed redaction of image files

Side review bar is mostly there. A couple of bugs fixed. Can now return identified text in initial review files. Still working on retaining found text throughout review process

Hopefully finally fixed the duplicate image_annotation_object issue

Refactor redaction functionality and enhance UI components: Added support for custom recognizers and whole page redaction options. Updated file handling to include new dropdowns for entity selection and improved dataframes for entity management. Enhanced the annotator with better state management and UI responsiveness. Cleaned up redundant code and improved overall performance in the redaction process.

Enhance file handling and UI features: improved Gradio app layout with fill width option, and integrated new settings for deny, and fully redacted lists (placeholders so far). Updated file conversion functions to handle CSV inputs and added CSV review file generation for redactions. Now retains all original and merged redaction boxes.

Can now toggle colour change for boxes. Large boxes now remove text correctly

Fixed issue where redactions were sometimes not removing text underneath boxes. You can now redact in different colours from review page

Updated packages. Reinstituted multithreading with page load, now with order protected. Smaller spacy model used for speed. Textract calls should now be faster

Started adding in support for custom deny list. Fixed textract call issue. Removed multithreading for now as it mixes up pages

Multithreaded file preparation. Can call Textract without signature detection

Allowed for overwriting of default output folder in choose_and_run_redactor function.

Added option for running redact function through CLI (i.e. not going through Gradio UI or API). Test functions for running this through AWS Lambda.

Only shows AWS options when AWS functions enabled. Can now upload previous review files to continue review later. Some review debugging.

Submitting modified redactions will no longer overwrite default labels

Comprehend now uses custom spacy recognisers on top of defaults. Added zoom functionality to annotator. Fixed some pdf mediabox issues and redacted image output issues.

AWS Comprehend query numbers in logs should now add up correctly

Returned file redaction timeout (before resending request) to 105 seconds default

logs should only be updated once per file run now

Improved time taken reporting and readme

Consolidated AWS Comprehend redaction calls to reduce total number

When on AWS, now loads in a default allow_list to exclude common words from redaction. Improved checks on AWS Comprehend calls.

Changed default options for AWS.

Improved logging

Added support for AWS Comprehend for PII identification. OCR and detection results now written to main output

Allowed for time limits on redact to avoid timeouts. Improved review interface. Now accepts only one file at a time. Upgraded Gradio version

Upgraded packages. Fixed some issues with review process. Better progress reporting for user.

Allowed for PIL to load truncated images to avoid some load errors

Added 'Review redactions' tab to the app. You can now visually inspect suggested redactions and modify/add with a point and click interface.

Adjusted outputs correctly for situations where the pdf mediabox size is different from the visible page size

Redaction tool can now export pdfs with selectable text retained - redacted text is deleted and covered with a black box. Licence change for pymupdf use.

General improvement in quick image matching and merging

Generally improved OCR recognition of texts, corrected postcode regex

Optimised Textract and Tesseract workings

Improved allow list, handwriting/signature identification, logging

Added AWS Textract support. Allowed for OCR logs export.

Updated time sum function to sum correctly

Updated default AWS_FUNCTION value. Logs seconds values from outputs correctly.

Should now correctly extract and sum up total processing time

Enhanced logging of usage. Small buffer added to redaction rectangles as it seems to miss the tops of text often.

Works correctly with images again

Can now select only specific pages in document to redact. Image based redaction should work correctly now.

Handles multiple runs with multiple files correctly now. Logging and feedback improvements.

Updated decision making output files, log locations

Decision process now saved as log files. Other log files and feedback added

Added logging, anonymising all Excel sheets, simple redaction tags, some Dockerfile optimisation

Can now redaction text or csv/xlsx files. Can redact multiple files. Embeds redactions as image-based file by default