file name pattern matching and identifying I have a automation system which manages files sent by multiple client inside S3. It so happens client sends file on a regular basis. Let's not consider the content of the file for time being as it can be something of new data or append on old data. 

Typically my files are arranged on a pattern: 

i) client_a/data_type1/20180610/file_name1_{some_random_text}.csv
ii) client_a/data_type1/20180611/file_name1_{some_random_text}.csv
iii) client_a/data_type1/20180610/file_name2_{some_random_text}.csv
iv) client_a/data_type1/20180611/file_name2_{some_random_text}.csv
v) client_b/data_type2/20180610/file_name3_{some_random_text}.csv
vi) client_b/data_type2/20180611/file_name3_{some_random_text}.csv
vii) client_b/data_type1/20180610/file_name4_{some_random_text}.csv
viii) client_b/data_type1/20180611/file_name4_{some_random_text}.csv


This is a sample of the file management. Now I want to be able to be able to identify unique files. For example case i and ii are same files received on different date. But may be separated by some_random_text (usually date or something). 
The primary assumptions here are :


File name length varies as per client and datatype 
Random text maybe at the first or end of file name for different files but   for not same file. i.e file_name1 will always have random text after the name
File name extension will vary on different files.


So I want to build a system which takes realtime input of the files received and do an analysis of the name and path to determine if newer version of he same file is received and place it accordingly (same in the sense data may be added to the file). 

I have very limited understanding of machine learning algorithms as of whole and not sure which set of algorithms this requires. I am thinking of looking at clustering algorithms. I just want to sure i start in the right direction. Suggestions will be highly appreciated.
This does not sound like a machine learning problem. It seems to me that you could come up with a list of rules that determine whether two files belong together.

As S van Balen said, you should take a look at the glob and re packages in Python. (If you are on Linux, a Bash script would probably be much more elegant than a solution in Python).

The following links might be helpful:


Walk through a directory and its subdirectories to get a list of all files: https://stackoverflow.com/a/954522/4743630.
Group strings in a list based on partial matches (for instance: turn ['file_name1_xyz', 'file_name2_xyz', 'file_name1_abc', 'file_name2_def'] into [ ['file_name1_xyz', 'file_name1_abc'], ['file_name2_xyz', 'file_name2_def'] ]): https://stackoverflow.com/questions/40167651/python-iterate-through-a-list-of-strings-and-group-partial-matching-strings.


You mention that files may have different file types. Be careful when converting between types: Make sure for instance that separators and quote characters are treated correctly.