Url string processing: what is the best way? I have ~1000 different news websites and I scraped and saved all the internal url links for each website. For instance, the website dcgazette.com has a 2MB text file with associated urls:

1) https://dcgazette.com/writers-wanted/

2) https://dcgazette.com/2019/fifteen-things-that-caught-my-eye-today-june-20-2019/

3) https://dcgazette.com/page/494/

4) https://dcgazette.com/page/3/

5) https://dcgazette.com/2019/trump-has-reason-to-be-concerned-about-2020/

etc

Notice that urls 2 and 5 are associated with actual articles. Across websites, each has a different way of representing their article urls. But within each website, all article urls have the same format, i.e. "domain/date/title" (for dcgazette.com).

Is there any unsupervised learning algorithm to process these url strings such that I can cluster all article urls together for each domain? The result I am aiming for is extracting all article urls for each domain with as little noise as possibe from non-article urls for all these domains.

An idea I had was separating each url into pieces by splitting on "/" and generating features such as length or type of letters used for each piece and using a k-means clustering algorithm. I'm not sure how this will actually perform however and am open to advice on any methodology that can make this as accurate as possible
I don't think you need any ML, in the best case it's going to be very slow compared to direct processing and it's very likely to cause errors. 

For each address you can parse the url string and extract the domain substring (in your example they are all prefixes, if it's always the case it's very easy). Then you just group them by domain in a map... done.