How to delete alphanumeric words out of a Unicode file
Question: I need to use a dictionary database, but most of it is some alphanumeric
useless stuff, and the interesting fields are either non alphanumeric (such as
chinese characters) or inside some brackets. I searched a lot, learned about a
lot of tools like sed, awk, grep, ect I even thought about creating a Python
script to sort it out, but I never managed to find of a solution.

A line of the database looks like this:

    
    
    助 L1782 DN1921 K407 O431 DO346 MN2313 MP2.0376 E314 IN623 DA633 DS248 DF367 DH330 DT284 DC248 DJ826 DG211 DM1800 P1-5-2 I2g5.1 Q7412.7 DR3945 Yzhu4 Wjo ジョ たす.ける たす.かる す.ける すけ {help} {rescue} {assist} 
    

I need it to be like this :

    
    
    助 ジョ たす.ける たす.かる す.ける すけ {help} {rescue} {assist} 
    

Ho can I do this using any of the tools mentioned above?


Answer: Here is a Python solution if you would still like one:

    
    
    import re
    alpha_brack = re.compile(r"([a-zA-Z0-9.\-]+)|({.*?})")
    
    my_string = """
    助 L1782 DN1921 K407 O431 DO346 MN2313 MP2.0376 E314 IN623 DA633 DS248 DF367 
    DH330 DT284 DC248 DJ826 DG211 DM1800 P1-5-2 I2g5.1 Q7412.7 DR3945 Yzhu4 
    Wjo ジョ たす.ける たす.かる す.ける すけ {help} {rescue} {assist}"""
    
    match = alpha_brack.findall(my_string)
    
    new_string = my_string
    
    for g0, _ in match: # only care about first group!
        new_string = new_string.replace(g0,'',1) # replace only first occurence!
    
    final = re.sub(r'\s{2,}',' ', new_string) # finally, clean up whitespace
    
    print(final)
    

My results:

'助ジョ たすける たすかる すける すけ {help} {rescue} {assist}'

