Dealing with large data: selecting a sample
9300 cases is not a lot, this doesn't even fall within the realm of big data or anything were you would have to apply special techniques.

Also given a more restricted feature set there are almost no computational issues you should encounter unless you work on an actual brick or Atari. As an example benchmark I am able to fit a model for >4M cases with up to 20-30 features (some with ~40 levels) on my 16gb Laptop in &lt;10mins.

So let's focus on the real issue, the complexity of the data i.e. the amount of available features/feature levels and most importantly efficient code.

Assuming you don't have the resources to simply throw some cloud processing power on this problem and call it a day we have to restrict the complexity.

Note that this does not mean "sampling the data" which would indicate using only a subset of cases, as said in the beginning this isn't your problem.

Instead let's workshop ways to cut down computational requirements:

1. Optimize your code

This sounds simple but research and make sure that you are using the best packages for the job, adjusted parameters correctly and cut fat from your code. This is hard work but it is necessary and worth it.

E.g. have you checked that all code is parallelized as much as you can and runs on all CPUs? 

2. Optimize your data structure

Factor levels normally aren't a problem computationally speaking but even so >100 levels might be too hard to understand anyway. Try to transform your data by e.g. grouping certain factor levels together. Also try to pivot parts of your data e.g. by wide to long data.

3. Use your domain knowledge

You surely understand your business and this problem. Don't just throw all the data at it! Think for a minute and remove unnecessary or duplicate columns.

4. Work your way upwards

ML modeling has a simply yardstick, if the validated prediction quality is good, your model is good. If you manage to do that with two variables, nobody cares.

So start with a bare-bones data set that includes ALL cases (still keep split-sample validation in mind here) but only the 2-3 simplest, most relevant predictors (based on domain knowledge) as well as the depended variable.

Fit a model and check it's quality, is it good enough? Yeah your done! If not include the next round of variables...

5. Modulize your EDA and have patience

I cannot stress enough that your data set really isn't that big or complex, so optimize your code first. BUT if basic EDA needed to find the best features still takes too long, do the following:


Structure and modulize your code by e.g. doing a PCA for only chunks of the columns (again use all case but only a subset of variables).
Let it run one-by-one and just have patience if the code runs for an hour, sometimes that is how it goes.
Look at the results, compare and decide how to merge results.


6. Talk to your boss

Yes I said I assume you do not have the resources to throw more power on it but if this is an important business problem there should be resources for cloud computation. Fitting a model on AWS is dirt cheap and will only take seconds, so realistically we are talking about &lt;100$ here to train multiple models.

If you don't have a boss to pay for this stuff or he doesn't want to, try free resources. There are a lot of cloud based services that offer enough free GPU or CPU time that you should be able to fit a model.