I am working on a project that manages employee payrolls using php (no frameworks). This application consists of clients (with their own databases). Eachdatabase consists of the client's employees and their payrolls. The relevant features are:An administrator can process a payslip twice a month for all employees in a particular client. When clicking a process button, a pdf file will be created for each employee and storedin a directory.And administrator can select employees in a table of employees for a particular client, then send each selected employees an email containing their pdf file when a button is clicked. The pdf file is different for each employee.And administrator can select clients in a table of clientss, then for each client, send all employees an email containing their pdf file when a button is clicked. The pdf file is different for each employee.My initial solution for creating pdfs and sending emails is to iterate through the list of employees retrieved from each client database and perform some action when a button is clicked. However, when I select 40 employees to create pdf files for them using fpdf, the application took 4 seconds to finnish. This is unacceptable because each client might contain at most 5000 employees. Same goes with sending emails using phpmailer, it took 25 seconds to send three emails.So, I started looking for a multithreaded solution where each action could be executed at the same time. However, pthreads v3 only works in CLI, we are using php 7. I don't know about using multiple processes, but I've read it could be complecated.Is there a viable and fast solution to achieve what I want in php? Should I use a different language?
Performing multiple actions on each row of table from many databases
php;multithreading;email;pdf
Does it need to happen in real-time or would you be OK with a batch solution?A batch solution would involve a few extra steps. You'll need to track your batches, maybe in new batch and batch_job tables. The admin can create a new batch to generate PDFs, and each execution of the PDF generator would be a single batch. You'd need to have some other process that runs in the background and checks the batch tables to see what needs to be run. There should also be something somewhere in the interface so that a user can see what batches have been created, are running, completed, or failed.Basically, the workflow would go like this:Admin user selects 5000 workers to generate PDFs for.Admin creates a batch with 5000 Generate PDF jobs (each job would have a description describing which worker it needs to generate a PDF for).Background process on the server detects that there is a new batch and begins executing the jobs.Admin user can monitor progress in web UI.Job executor updates batch tables with status as they are completed/failed.Advantages of this solution are that the batch process can run in the background. PDF generation could be done in parallel, in serial, or any other way you choose. Disadvantages: implementing this will probably be more than an hour or two's worth of coding ;) If you anticipate having other business processes that operate on very large sets of things, it might be worth considering this seriously. If your hardware can actually support generating 5000 PDFs in under a minute, and you really want this to be real-time, you could implement a shorter version of the above solution (well it's not real-time but more so than my other suggestion):Admin selects 5000 things to create PDFs. Web app passes all the information about those 5000 to a single process that runs asynchronously in the background (not sure how to do this in PHP, would need some additional research).That process churns away, leaving the user free to do other things.