import re

"""
Explanation: Chapter 6 - Regular Expressions
Original URL: https://github.com/mikekestemont/ghent1516/blob/master/Chapter%206%20-%20Regular%20Expressions.ipynb
Regular expressions are another, powerful way of handling strings in Python. You can use them for all sorts of very common string operations, like searching words and replacing them in texts. In fact, regular expressions are often used across many programming languages and text editors. Because of that, you will often be able to reuse many of the things that you will learn about below. The functionality for using regular expressions in Python is included in the 're' package, which you should be able to import as usual:
End of explanation
"""


s = "In principio erat verbum, et verbum erat apud Deum."

"""
Explanation: On many occasions, you will want to search a string in your scripts: e.g. does the following word appear in a text? Is the format of the following email address valid and does it contain an @-symbol and a least one dot? To carry out such operations, the first thing you need is a string to search:
End of explanation
"""


pattern = re.compile(r"verbum")

"""
Explanation: The next thing we define is the actual regular expression which we will use, or the string that we will use to search the sentence we defined above. We pass this string to the compile() function in the re package, which will allow fast searching later on. Note that we put an r in front of this string when we initialize it, which turns our string into a so-called 'raw string'. While this is not always necessary, it is a good idea to do this consistently when dealing with regular expressions.
End of explanation
"""


text = pattern.sub("XXX", s)
print(text)

help(re.sub)

text = re.sub("ABC", )

"""
Explanation: Next, we can call the sub() function from the re package on this pattern, in order to replace (or 'substitute') our pattern with another word, like this:
End of explanation
"""


pattern2 = re.compile(r"XXX")
text = pattern2.sub("verbum", s)
print(text)

"""
Explanation: Note the order of the arguments passed to sub(): first, the word we would like to replace our pattern with, and secondly our original string. We can just as easily get back our original string:
End of explanation
"""


vowel_pattern = re.compile(r"a|e|o|u|i")
without_vowels = vowel_pattern.sub("X", s)
print(without_vowels)

"""
Explanation: So far nothing special: we are simply replacing one word for another word. The smart ones among you will have noticed that we could have achieved the exact same result using the replace() function, which we came across in an earlier chapter. But now: say you would like to replace all vowels in a string. With regular expressions, this is a piece of cake:
End of explanation
"""


vowel_pattern = re.compile(r"a|A|e|E|o|O|u|U|i|I")
without_vowels = vowel_pattern.sub("X", s)
print(without_vowels)

"""
Explanation: Note how our pattern allows for a special syntax: the pipe symbol which we used allows to express that one character OR another one is fine for the regular expression to match. Oops: the capital letter at the beginning of the sentence hasn't been replaced because we only included lowercase vowels in our pattern definition. Let's add the uppercase vowels to the regex:
End of explanation
"""


ups = re.compile(r"[A-Z]")
lows = re.compile(r"[a-z]")
without_ups = ups.sub("X", s)
print(without_ups)
without_ups = lows.sub("X", s)
print(without_ups)

"""
Explanation: There is in fact an easy way to match all lowercase and uppercase characters in a string, like this:
End of explanation
"""


vowel_pattern = re.compile(r"[aeoui]")
without_vowels = vowel_pattern.sub("M", s)
print(without_vowels)

"""
Explanation: These specific patterns are called 'ranges': they will match any lowercase or uppercase letter. In fact, you can use such a range syntax using squared brackets, to replace the pipe syntax we used earlier.
End of explanation
"""


p = re.compile(r"(ri)|(um)|(Th)")
print(vowel_pattern.sub("A", s))

"""
Explanation: You can also look for more specific, as well as longer letter groups by arranging them with round brackets:
End of explanation
"""


any_char = re.compile(r".")
print(any_char.sub("X", s))

"""
Explanation: There is also a syntax to match any character (except the newline):
End of explanation
"""


dot = re.compile(r"\.")
print(dot.sub("X", s))

"""
Explanation: If you would like your expression to match an actual dot, you have to escape it using a backslash:
End of explanation
"""


s = "In principio [erat] verbum, et verbum erat apud Deum."
brackets_wrong = re.compile(r"[|]")
print(brackets_wrong.sub("X", s))
brackets_right = re.compile(r"(\[)|(\])")
print(brackets_right.sub("X", s))

"""
Explanation: By the way, there exist more characters that you might have to escape using a backslash. This is because they are part of the syntax that use to define regular expressions: if you don't escape them, Python will not know that you are looking for an literal match. Characters that you typically might want to escape include: ( + ? . * ^ $ ( ) [ ] { } | \ ) ,. For example:
End of explanation
"""


pattern = re.compile(r"m{2,4}")
print(pattern.match(""))
print(pattern.match("m"))
print(pattern.match("mm"))
print(pattern.match("mmm"))
print(pattern.match("mmmm"))
print(pattern.match("mmmmm"))
print(pattern.match("mmmmmm"))
print(pattern.match("mmmmammm"))

"""
Explanation: The syntax for regular expression includes a whole range of possibilities which we simply cannot all deal with it here. Because of that we will stick to a number of helpful examples. An interesting feature is that you can specify whether or not a character really has to occur. You can check whether the pattern occurs in a string using the match() function which will return None if it doesn't find the pattern in the string searched:
End of explanation
"""


pattern = re.compile(r"a{5}")
if pattern.match("aaaaa"):
    print("Found it!")
else:
    print("Nope...")
# or:
if pattern.match("aa"):
    print("Found it!")
else:
    print("Nope...")

"""
Explanation: With the curly brackets, you indicate that you are only interested in the letter 'm' if it occurs 2,3 or 4 times in a row in the string you search. Because None is returned if not a single match was found, you can use the outcome of match()in an if-statement. The following example shows how you can also use the curly brackets to match an exact number of occurences (in this case three a's).
End of explanation
"""


paper = "My thesis on  biology     contains a lot of  double spaces.   I will remove  them."
mult = re.compile(r" +")
print(mult.sub(" ", paper))

"""
Explanation: Using a plus sign you can indicate whether you want to match multiple occurrences of a character. A good example from the world of paper writing are double spaces, which can be hard to spot. In the code block below, we replace all multiple occurences of a whitespace character by a single whitespace character. Note that you can use the whitespace character just like any other character (you don't have to escape it). Multiple occurences of the whitespace character will be matched: it doesn't matter how many, as long as there is at least one:
End of explanation
"""


s = "In English some letters occur multiple times in a row."
p1 = re.compile(r"t")
p2 = re.compile(r"t+")
p3 = re.compile(r"x")
p4 = re.compile(r"x*")
print(p1.search(s))
print(p2.search(s))
print(p3.search(s))
print(p4.search(s))

print(p2.search(s))

"""
Explanation: A similar piece of functionality is offered by the asterisk operator: here you can match multiple occurences of the same character in a row OR not a single one. Note the subtle difference with respect to the plus operator, which needs at least a single occurence of the character to match. Here we use the search() function which will search the entire string: the match() function which we used earlier will only look for matches at the very beginning of a string. Keep this in mind! The final pattern below yields a match, although there is not a single 'x' in the sentence. That is because the pattern with the asterisk says: "a single x, or no x at all".
End of explanation
"""


candidates = ["good", "god", "gud", "gd"]
p = re.compile(r"go+d")
for c in candidates:
    print(p.match(c))

"""
Explanation: Interestingly, you also use regular expression to search inside words. Can you explain why the following patterns (don't) match?
End of explanation
"""


s = """This is a text  on three   lines
with  multiple instances of  
double spaces."""
whitespace = re.compile(r"\s+")
print(whitespace.split(s))

"""
Explanation: Speaking of words: it might be interesting to know that you can use regular expressions for advanced string splitting. If you want to split a sentence across all whitespace characters for instance, you can use an espaced \s. This operator will match all whitespace characters, such as tabs, linebreaks, normal spaces etc. If you add a + sign, your pattern will match series of whitespace characters:
End of explanation
"""


s = """This is a text  on three   lines
with  multiple instances of  
double spaces."""
whitespace = re.compile(r"\s*\n\s*")
print(whitespace.split(s))

"""
Explanation: If you would have wanted to split on the linebreaks only (possible followed by e.g. spaces), you could have used the following pattern:
End of explanation
"""


ds = re.compile(r" +")
for line in whitespace.split(s):
    print ds.sub(" ", line)

"""
Explanation: If we want to correct the double spaces, we could now do:
End of explanation
"""


s = "these are vowels and consonants"
consonants = re.compile(r"[^aeuoi]")
print(consonants.sub("X", s))

"""
Explanation: One final feature we should mention is the [^...] syntax: this will match any character that is NOT between the brackets. Remember the vowel_pattern above? Using the caret symbol we can quickly 'invert' this pattern, so that it will match all consonants:
End of explanation
"""


