%cd ../examples/superlists/

!python3 functional_tests.py

"""
Explanation: Using Selenium to Test User Interactions
Where were we at the end of the last chapter? Let’s rerun the test and find out:
End of explanation
"""


%%writefile functional_tests.py
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import unittest

class NewVisitorTest(unittest.TestCase):
    
    def setUp(self):
        self.browser = webdriver.Firefox()
        self.browser.implicitly_wait(3)
        
    def tearDown(self):
        self.browser.quit()
        
    def test_can_start_a_list_and_retrieve_it_later(self):
        # Edith has heard about a cool new online to-do app. She goes 
        # to check out its homepage
        self.browser.get('http://localhost:8000')
        
        # She notices the page title and header mention to-do lists
        self.assertIn('To-Do', self.browser.title)
        header_text = self.browser.find_element_by_tag_name('h1').text
        self.assertIn('To-Do', header_text)
        
        # She is invited to enter a to-do item straight away
        inputbox = self.browser.find_element_by_id('id_new_item')
        self.assertEqual(
            inputbox.get_attribute('placeholder'),
            'Enter a to-do item'
        )
        
        # She types "Buy peacock feathers" into a text box (Edith's hobby
        # is tying fly-fishing lures)
        inputbox.send_keys('Buy peacock feathers')
        
        # When she hits enter, the page updates, and now the page lists
        # "1: Buy peacock feathers" as an item in a to-do list table
        inputbox.send_keys(Keys.ENTER)
        
        table = self.browser.find_element_by_id('id_list_table')
        rows = table.find_elements_by_tag_name('tr')
        self.assertTrue(
            any(row.text == '1: Buy peacock feathers' for row in rows)
        )
        
        # There is still a text box inviting her to add another item. She
        # enters "Use peacock feathers to make a fly" (Edith is very
        # methodical)
        self.fail('Finish the test!')
        
        # The page updates again, and now shows both items on her list

        # Edith wonders whether the site will remember her list. Then she sees
        # that the site has generated a unique URL for her -- there is some
        # explanatory text to that effect.

        # She visits that URL - her to-do list is still there.

        # Satisfied, she goes back to sleep

if __name__ == '__main__':  
    unittest.main(warnings='ignore')  

"""
Explanation: Did you try it, and get an error saying Problem loading page or Unable to connect? So did I. It’s because we forgot to spin up the dev server first using manage.py runserver. Do that, and you’ll get the failure message we’re after.

One of the great things about TDD is that you never have to worry about forgetting what to do next—just rerun your tests and they will tell you what you need to work on.

“Finish the test”, it says, so let’s do just that! Open up functional_tests.py and we’ll extend our FT:
End of explanation
"""


!python3 functional_tests.py

"""
Explanation: We’re using several of the methods that Selenium provides to examine web pages: find_element_by_tag_name, find_element_by_id, and find_elements_by_tag_name (notice the extra s, which means it will return several elements rather than just one). We also use send_keys, which is Selenium’s way of typing into input elements. You’ll also see the Keys class (don’t forget to import it), which lets us send special keys like Enter, but also modifiers like Ctrl.

Watch out for the difference between the Selenium find_element_by... and find_elements_by... functions. One returns an element, and raises an exception if it can’t find it, whereas the other returns a list, which may be empty.

Also, just look at that any function. It’s a little-known Python built-in. I don’t even need to explain it, do I? Python is such a joy.
Although, if you’re one of my readers who doesn’t know Python, what’s happening inside the any is a generator expression, which is like a list comprehension but awesomer. You need to read up on this. If you Google it, you’ll find Guido himself explaining it nicely. Come back and tell me that’s not pure joy!
Let’s see how it gets on:
End of explanation
"""


!python3 manage.py test

"""
Explanation: Decoding that, the test is saying it can’t find an &lt;h1&gt; element on the page. Let’s see what we can do to add that to the HTML of our home page.
Big changes to a functional test are usually a good thing to commit on their own. I failed to do so in my first draft, and I regretted it later when I changed my mind and had the change mixed up with a bunch of others. The more atomic your commits, the better:
$ git diff  # should show changes to functional_tests.py
$ git commit -am "Functional test now checks we can input a to-do item"
The "Don't Test Constants" Rule, and Templates to the Rescue
Let’s take a look at our unit tests, lists/tests.py. Currently we’re looking for specific HTML strings, but that’s not a particularly efficient way of testing HTML. In general, one of the rules of unit testing is Don’t test constants, and testing HTML as text is a lot like testing a constant.
In other words, if you have some code that says:
python
wibble = 3
There’s not much point in a test that says:
from myprogram import wibble
assert wibble == 3
Unit tests are really about testing logic, flow control, and configuration. Making assertions about exactly what sequence of characters we have in our HTML strings isn’t doing that.
What’s more, mangling raw strings in Python really isn’t a great way of dealing with HTML. There’s a much better solution, which is to use templates. Quite apart from anything else, if we can keep HTML to one side in a file whose name ends in .html, we’ll get better syntax highlighting! There are lots of Python templating frameworks out there, and Django has its own which works very well. Let’s use that.
Refactoring to Use a Template
What we want to do now is make our view function return exactly the same HTML, but just using a different process. That’s a refactor—when we try to improve the code without changing its functionality.
That last bit is really important. If you try and add new functionality at the same time as refactoring, you’re much more likely to run into trouble. Refactoring is actually a whole discipline in itself, and it even has a reference book: Martin Fowler’s Refactoring.
The first rule is that you can’t refactor without tests. Thankfully, we’re doing TDD, so we’re way ahead of the game. Let’s check our tests pass; they will be what makes sure that our refactoring is behaviour preserving:
End of explanation
"""


!mkdir lists/templates

%%writefile lists/templates/home.html

<html>
    <title>To-Do lists</title>
</html>

"""
Explanation: Great! We’ll start by taking our HTML string and putting it into its own file. Create a directory called lists/templates to keep templates in, and then open a file at lists/templates/home.html, to which we’ll transfer our HTML:
End of explanation
"""


%%writefile lists/views.py

from django.shortcuts import render

def home_page(request):
    return render(request, 'home.html')

"""
Explanation: Now to change our views
End of explanation
"""


!python3 manage.py test

"""
Explanation: Instead of building our own HttpResponse, we now use the Django render function. It takes the request as its first parameter (for reasons we’ll go into later) and the name of the template to render. Django will automatically search folders called templates inside any of your apps' directories. Then it builds an HttpResponse for you, based on the content of the template.
Templates are a very powerful feature of Django’s, and their main strength consists of substituting Python variables into HTML text. We’re not using this feature yet, but we will in future chapters. That’s why we use render and (later) render_to_string rather than, say, manually reading the file from disk with the built-in open.
Let's see if it works:
End of explanation
"""


...
# Application definition

INSTALLED_APPS = (
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'lists',
)
...


"""
Explanation: Another chance to analyse a traceback:


We start with the error: it can’t find the template.


Then we double-check what test is failing: sure enough, it’s our test of the view HTML.


Then we find the line in our tests that caused the failure: it’s when we call the home_page function.


Finally, we look for the part of our own application code that caused the failure: it’s when we try and call render.


So why can’t Django find the template? It’s right where it’s supposed to be, in the lists/templates folder.
The thing is that we haven’t yet officially registered our lists app with Django. Unfortunately, just running the startapp command and having what is obviously an app in your project folder isn’t quite enough. You have to tell Django that you really mean it, and add it to settings.py as well. Belt and braces. Open it up and look for a variable called INSTALLED_APPS, to which we’ll add lists:
End of explanation
"""


!python3 manage.py test

"""
Explanation: You can see there’s lots of apps already in there by default. We just need to add ours, lists, to the bottom of the list. Don’t forget the trailing comma—it may not be required, but one day you’ll be really annoyed when you forget it and Python concatenates two strings on different lines…
Now we can try running the tests again:
End of explanation
"""


%%writefile lists/tests.py

from django.core.urlresolvers import resolve
from django.test import TestCase
from django.http import HttpRequest

from lists.views import home_page 

class HomePageTest(TestCase):
    
    def test_root_url_resolves_to_home_page_view(self):
        found = resolve('/')  
        self.assertEqual(found.func, home_page)  
        
    def test_home_page_returns_correct_html(self):
        request = HttpRequest() 
        response = home_page(request)  
        self.assertTrue(response.content.strip().startswith(b'<html>'))   #<---- Fix offending newline here
        self.assertIn(b'<title>To-Do lists</title>', response.content)  
        self.assertTrue(response.content.strip().endswith(b'</html>'))    #<---- Fix offending newline here

!python3 manage.py test

"""
Explanation: Darn, not quite.
Depending on whether your text editor insists on adding newlines to the end of files, you may not even see this error. If so, you can safely ignore the next bit, and skip straight to where you can see the listing says OK.
But it did get further! It seems it’s managed to find our template, but the last of the three assertions is failing. Apparently there’s something wrong at the end of the output. I had to do a little print(repr(response.content)) to debug this, but it turns out that the switch to templates has introduced an additional newline (\n) at the end. We can get them to pass like this:
End of explanation
"""


# %load lists/tests.py

from django.core.urlresolvers import resolve
from django.test import TestCase
from django.http import HttpRequest
from django.template.loader import render_to_string

from lists.views import home_page 

class HomePageTest(TestCase):
    
    def test_root_url_resolves_to_home_page_view(self):
        found = resolve('/')  
        self.assertEqual(found.func, home_page)  
        
    def test_home_page_returns_correct_html(self):
        request = HttpRequest()
        response = home_page(request)
        expected_html = render_to_string('home.html')
        self.assertEqual(response.content.decode(), expected_html)

"""
Explanation: Our refactor of the code is now complete, and the tests mean we’re happy that behaviour is preserved. Now we can change the tests so that they’re no longer testing constants; instead, they should just check that we’re rendering the right template. Another Django helper function called render_to_string is our friend here:
End of explanation
"""


%%writefile lists/templates/home.html

<html>
    <head>
        <title>To-Do lists</title>
    </head>
    <body>
        <h1>Your To-Do list</h1>
    </body>
</html>

"""
Explanation: We use .decode() to convert the response.content bytes into a Python unicode string, which allows us to compare strings with strings, instead of bytes with bytes as we did earlier.
The main point, though, is that instead of testing constants we’re testing our implementation. Great!
Django has a test client with tools for testing templates, which we’ll use in later chapters. For now we’ll use the low-level tools to make sure we’re comfortable with how everything works. No magic!
On Refactoring
That was an absolutely trivial example of refactoring. But, as Kent Beck puts it in Test-Driven Development: By Example, "Am I recommending that you actually work this way? No. I’m recommending that you be able to work this way."
In fact, as I was writing this my first instinct was to dive in and change the test first—make it use the render_to_string function straight away, delete the three superfluous assertions, leaving just a check of the contents against the expected render, and then go ahead and make the code change. But notice how that actually would have left space for me to break things: I could have defined the template as containing any arbitrary string, instead of the string with the right &lt;html&gt; and &lt;title&gt; tags.
When refactoring, work on either the code or the tests, but not both at once.
There’s always a tendency to skip ahead a couple of steps, to make a couple of tweaks to the behaviour while you’re refactoring, but pretty soon you’ve got changes to half a dozen different files, you’ve totally lost track of where you are, and nothing works any more. If you don’t want to end up like Refactoring Cat (below), stick to small steps; keep refactoring and functionality changes entirely separate.

We’ll come across “Refactoring Cat” again during this book, as an example of what happens when we get carried away and want to change too many things at once. Think of it as the little cartoon demon counterpart to the Testing Goat, popping up over your other shoulder and giving you bad advice…
It’s a good idea to do a commit after any refactoring:
$ git status # see tests.py, views.py, settings.py, + new templates folder
$ git add .  # will also add the untracked templates folder
$ git diff --staged # review the changes we're about to commit
$ git commit -m "Refactor home page view to use a template"
A Little More of Our Front Page
In the meantime, our functional test is still failing. Let’s now make an actual code change to get it passing. Because our HTML is now in a template, we can feel free to make changes to it, without needing to write any extra unit tests. We wanted an &lt;h1&gt;:
End of explanation
"""


!python3 functional_tests.py

"""
Explanation: Let’s see if our functional test likes it a little better:
End of explanation
"""


