Fixed file load with files including capital letters

Added clean data options, improved re-representation options and visualisation. General format changes

Allowed for loading in external topic labels. A few visualisation modifications.

Model save now checks and makes a folder before writing the model

Lots of general fixes. New visualisations, fixed hierarchical vis for zero shot. Added calc all probabilities.

Changed Phi model to smaller StableLM 2 1.6. Fixed a None type detection error.

Switched embeddings model to BGE Small 1.5 as Jina seemed unable to do zero shot topic modelling properly

Split off LLM representation, visualisation, and reduce outliers from main function. Added hierarchical visualisation and logs

More efficient embeddings save and representations load/process. Custom visualisation hover option added, formatting improvements. Version 0.1?

App should now check if embeddings are loaded before topic modelling. And will save only once.

Hopefully fixed install and load of LLM model on systems without a HF_HOME environmental variable

Added option to reduce outliers based on closest topic

Returned TruncatedSVD components to 100 - higher values don't seem to help

Greatly increased low resource process dimensions for higher quality. Visualisations disabled by default to increase speed.

Greatly improved low resource mode speed (at cost of potential quality)

Changed zero shot min similarity to 0.5

Now should save embeddings by default. Added random seed to representation

Fixed llm_config, environmental variable, zero shot topic model errors with quick embeddings

removed cpu_config as causing problems

Model export changed to safetensors. Improved representational model function. Got zero shot topic modelling working

first commit