text:
Python is often criticized for being slow. In many cases pure Python is fast enough, there are certain cases where Python may not give you the performance you need. In recent years a fair number of Python programmers have made the jump to Golang for performance reasons. However there are a number of ways you can improve the performance of your Python code such as using PyPy, writing C extensions or trying your hand at Cython.What is Cython?
Cython is a superset of the Python language. This means that the vast majority of Python code, is also valid Cython code. Cython allows users to write Cython modules which are then compile and can be used within in Python code. This means that users can port performance critical code into Cython and instantly see increases in performance. The great thing about Cython code is that you can determine how much to optimize your code. Simply copying and compiling your Python code might see you make performance gains of 8-12%, whereas more serious optimization of your code can lead to significantly better performance.Installing Cython
Installing Cython on Linux is very easy to do and just requires you to use the 'pip install cython' command. Those on Windows devices will likely have a much tougher time, with the simplest solution seeming to be just installing 'Visual Code Community' and selecting both the C++ and Python support options. You can then just install Cython like you would any other Python package.Using Pure Python
We are going to begin with compiling a pure Python function. This is a very simple task, and can achieve some limited performance benefits, with a more noted increase in performance for functions which make use of for and while loops. To begin we simply save the below code into a file called 'looping.pyx'.
This very simple function takes a list of numbers and then multiples each number by it's index, and returning the sum of the results. This code is both valid Python and Cython code. However, it takes no advantage of any Cython optimizations other than the compilation of the code into C.
We run the below command to create Cython module which can be used in Python:
We can then import our Cython module into Python in the following manner:
What kind of performance benefits can we expect from just compiling this Python code into a Cython module?
I ran some tests and on average the Cython compiled version of the code took around 10% less time to run over a set of 10,000 numbers.Adding Types
Cython achieves optimization of code by introducing typing to Python code. Cython supports both a range of Python and C types. Python types tend to be more flexible but give you less in terms of performance benefits. The below example makes use of both C and Python types, however we have to be very careful when using C types. For instance we could throw an overflow error should the list of numbers we pass in be too large and the result of the multiplication being to large to store in a C long.
As you can see we use the Python type 'list' to type annotate our input list. We then define two C types which will be used to store the length of our list and our output. We then loop over our list in exactly the same way as we did in our previous example. This shows just how easy it is to start adding C types to code with the help of Python. It also illustrates how easy it is to mix both C and Python types together in one extension module.
This hybrid code when tested was between 15-30% faster than the pure Python implementation without taking the most aggressive path of optimization and turning everything into a C type. While these savings may seem small, they can really add up on operations which are repeated hundreds of thousands of times.Cython Function Types
Unlike standard Python, Cython has three types of functions. These functions differ in how they are defined and where the can be used.
- Cdef functions – Cdef functions can only be used in Cython code and cannot be imported into Python.
- Cpdef functions – can be used and imported in both Python and Cython. If used in Cython they behave as a Cdef function and if used in Python they behave as if they are standard Python function.
- Def functions – are like your standard Python functions and can be used and imported into Python code
The below code block demonstrates how each of these three function types can be defined.
This allows you to define highly performant Cdef functions for use within Cython modules, while at the same time allowing you to write functions that are totally compatible with Python. Cpdef functions are a good middle ground, in the sense that when they are used in Cython code they are highly optimized while remaining compatible with Python, should you want to import them into a Python module.Conclusion
While this introduction only touches the surface of the Cython language, it should be enough to begin optimizing code using Cython. However, some of the more aggressive optimizations and the full power of C types are well beyond the scope of this post.
When looking to extract information from more difficult to scrape sites many programmers turn to browser automation tools such as Selenium and iMacros. At the time of writing, Selenium is by far the most popular option for those looking to leverage browser automation for information retrieval purposes. However, Selenium is very detectable and site owners would be able to block a large percentage of all Selenium users.Selenium Detection with Chrome
Older versions of Firefox used to inject a webdriver attribute into the HTML document. This means that older versions of Firefox could be very simply detected using the above code snippet. At the time of writing Firefox no longer adds this element to pages when using Selenium.
Additional methods of detecting Selenium when using Firefox have also been suggested. Testing seems to suggest that these do not work with the latest builds of Firefox. However, the webdriver standard suggests that this may eventually be implemented in Firefox again.Selenium Detection with PhantomJS
Sanita Women s Professional Patent Clog Patent Red Professional Patent Patent 3a71601 - bewegungswerkstatt.website
All current versions of PhantomJS, add attributes to the window element. This allows site owners to simply check whether these specific PhantomJS attributes are set and redirect the user away when it turns out that they are using PhantomJS. It should also be noted that support for the PhantomJS project has been rather inconsistent and the project makes use on an outdated webkit version which is also detectable and could present a security list.Avoiding Detection
Your best of avoiding detection when using Selenium would require you to use one of the latest builds of Firefox which don't appear to give off any obvious sign that you are using Firefox. Additionally, it may be worth experimenting with both Safari and Opera which are much less commonly used by those scraping the web. It would also seem likely that Firefox may be giving off some less obvious footprint which would need further investigation to discover.
When web-scraping, you often need to source a number of proxies in order to avoid being banned or get around rate limiting imposed by the website in question. This often see's developers purchasing proxies from some sort of commercial provider, this can become quite costly if you are only need the proxies for a short period of time. So in this post we are going to look at how you might use proxies from freely available proxy lists to scrape the internet.Problems With Free Proxies
- Free Proxies Die Very Quickly
- Free Proxies Get Blocked By Popular Sites
- Free Proxies Frequently Timeout
While free proxies are great in the sense that they are free, they tend to be highly unreliable. This is due to the fact that up-time is inconsistent and these proxies get blocked quickly by popular sites such as Google. Our solution is also going to build in some monitoring of the current status of the proxy in question. Allowing us to avoid using proxies which are currently broken.Scraping Proxies
We are going to use free-proxy-list.net, as our source for this example. But the example could easily be expanded to cover multiple sources of proxies. We simply write a simple method which visits the page and pulls out all the proxies from the page in question using our chosen user-agent. We then store the results in a dictionary, with each proxy acting as a key holding the information relating to that particular proxy. We are not doing any error handling, this will be handled in our ProxyManager class.Proxy Manager
Our proxy manager is a simply class which allows us to get and manage the proxies we find on free-proxy-list.net. We pass in a test URL which will be used to test whether the proxy is working and a user agent to be used for both scraping and testing the proxies in question. We also create a thread pool, so we can more quickly check the status of the proxies we have scraped. We then call our update_proxy_list, returning the proxies we have found on free-proxy-list.net into our dictionary of proxies.Checking Proxies
We can now write a couple of methods to test whether a particular proxy works. The first method takes the proxy and the dictionary of information related to the proxy in question. We immediately set the last checked variable to the current time. We make a request against our test URL, with a relatively short timeout. We also then check the status of the request raising an exception should we receive a non-200 status code. Should anything go wrong, we then set the status of the proxy to dead, otherwise we set the status to alive.
We then write our refresh proxy status which simple calls our check proxy status. We iterate over our dictionary, submitting each proxy and the related info of to a thread. If we didn't use threads to check the status of our proxies, we could be waiting a very long time for our results. We then loop through our results and update the status of proxy in question.Getting A Proxy
We then write two methods for getting ourselves a proxy. Our first method allows us to get a list of proxies by passing in a relevant key and value. This method allows us to get a list of proxies that relate to a particular country or boasts a particular level anonymity. This can be useful should we be interested in particular properties of a proxy.
We also have a simple method that allows us to return a single working proxy. This returns the first working proxy found within our proxy dictionary by looping over all the items in the dictionary, and returning the first proxy where 'alive' is equal to true.Example Usage
Using the library is pretty simple. We just create the class passing in our test URL (using Google.com here) and our selected user-agent. We then call refresh_proxy_status, updating the status of the scraped proxies by running them against our test URL. We can then pull out an individual working proxy. We can then update our proxy list with a fresh scrape of our source should we not be satisfied with the proxies we currently have access to.Full Code
In today's post we are going how to look at how you can extract information from a users Instagram profile. It's surprisingly easy to extract profile information such as the number of followers a user has and information and image files for a users most recent posts. With a bit of effort it would be relatively easy to extract large chunks of data regarding a user. This could then be applied at a very broad scale to extract a large chunk of all public posts featured on Instagram's site.Imports & Setup
We begin by making our imports and writing the dunder init method for our class. Our code requires two packages not included in the standard library, requests for making HTTP Requests and BeautifulSoup to make html parsing more user friendly. If you do not already have these libraries install, you can use the following pip command:
The init method of our class takes two optional keyword arguments, which we simply store in self. This will allow us to override the default user agent list and use a proxy should we wish to avoid detection.
We then write two helper methods. First, we write a very simply method that returns us a random user-agent. Switching user agents is often a best practice when web scraping and can help you avoid detection. Should the caller of our class have provided their own list of user agents we take a random agent from the provided list. Otherwise we will return our default user agent.
Our second helper method is simply a wrapper around requests. We pass in a URL and try to make a request using the provided user agent and proxy. If we are unable to make the request or Instagram responds with a non-200 status code we simply re-raise the error. If everything goes fine, we return the page in questions HTML.Extracting JSON from JavaScript
Instagram serve's all the of information regarding a user in the form of JavaScript object. This means that we can extract all of a users profile information and their recent posts by just making a HTML request to their profile page. We simply need to turn this JavaScript object into JSON, which is very easy to do.
We can write this very hacky, but effective method to extract JSON from a user profile. We apply the static method decorator to this function, as it's possible to use this method without initializing our class. We simply create a soup from the HTML, select body of the content and then pull out the first 'script' tag. We can then simply do a couple text replacements on the script tag, to derive a string which can be loaded into a dictionary object using the json.loads method.Bringing it all together
We then bring it all together in two functions which we can use to extract information from this very large JSON object. We first make a request to the page, before extracting the JSON result. We then use two different selectors to pull out the relevant bits of information, as the default JSON object has lots of information we don't really need.
When extracting profile information we extract all attributes from the "user" object, excluding their recent posts. In the "recent posts" function, we use a slightly different selector and pull out all the information about all of the recent posts made by our targeted user.Example Usage
We can then use the Instagram scraper in a very simply fashion to pull out all the most recent posts from our favorite users in a very simple fashion. You could do lots of things with the resulting data, which could be used in Instagram analytics app for instance or you could simply programmatically download all the images relating to that user.
There is certainly room for improvement and modification. It would also be possible to use Instagram's graph API, to pull out further posts from a particular user or pull out lists of a users recent followers etc. Allowing you to collect large amounts of data, without having to deal with Facebook's restrictive API limitations and policies.Full Code
The asyncio library was introduced to Python from versions, 3.4 onwards. However, the async await syntax was not introduced into the language in Python 3.5. The introduction of this functionality allows us to write asynchronous web crawlers, without having to use threads. Getting used to asynchronous programming can take a while, and in this tutorial we are going to build a fully functional web crawler using asyncio and aiohttp.Fan In & Fan Out Concurrency Pattern
Initializing Our Crawler
We are going to write a web crawler which will continue to crawl a particular site, until we reach a defined maximum depth. We are going to make use of a fan-in/fan-out concurrency pattern. Essentially, this involves gathering together a set of tasks, and then distributing them across a bunch of threads, or in across co-routines in our case. We then gather all the results together again, before processing them, and fanning out a new group of tasks. I would highly recommend Brett Slatkin's 2014 talk, which inspired this particular post.
We begin by importing the libraries required for our asyncio crawler. We are using a couple libraries which are not included in Python's standard library. These required libraries can be installed using the following pip command:
We can then start defining our class. Our crawler takes two positional arguments and one optional keyword argument. We pass in the start URL, which is the URL we begin our crawl with and we also set the maximum depth of the crawl. We also pass in a maximum concurrency level which prevents our crawler from making more than 200 concurrent requests at a single time.
The start URL is then parsed to give us the base URL for the site in question. We also create a set of URLs which have already seen, to ensure that we don't end up crawling the same URL more than once. We also create session using aiohttp.ClientSession so that we can skip, having to create a session every time we scrape a URL. Doing this will throw a warning, but the creation of a client session is synchronous, so it can be safely done outside of a co-routine. We also set up a asyncio BoundedSemaphore using our max concurrency variable, we will use this to prevent our crawler from making too many concurrent requests at one time.Making An Async HTTP Request
We can then write a function make to a asynchronous HTTP request. Making a single asynchronous request is pretty similar to making a standard HTTP request. As you can see we write "async" prior to the function definition. We begin by using an async context manager, using the bounded semaphore created when we initialized our class. This will limit asynchronous requests to whatever we passed in when creating an instance of AsyncCrawler class.
We then use another async context manager within a try/except block to make a request to the URL, and await for the response. Before we finally return the HTML.Extracting URLs
We can then write a standard function to extract all the URLs from a html response. We create DOM (Document Object Model) object from our HTML, using Lxml's HTML sub-module. Once we have extracted our document model, we able to query it using either XPath or CSS selectors. Here we use a simple XPath selector to pull out the 'href' element of every link found on the page in question.
We can then use urllib.parse's urljoin function with our base URL and found href. This gives an absolute URL, automatically resolving any relative URLs that we may have found on the page. If we haven't already crawled this URL and it belongs to the site we are crawling, we add it to our list of found URLs.
The extract async function is a simple wrapper around our HTTP request and find URL functions. Should we encounter any error, we simply ignore it. Otherwise we use the HTML to create a list of URLs found on that page.Fanning In/Out
Our extract_multi_async function is where we fan out. The function takes a list of URLs to be crawled. We begin by creating two empty lists. The first will hold the futures which refer to jobs to be done. While the second holds the results of these completed futures. We begin by adding a call to our self.extract_async function for each URL we have passed into the function. These are futures, in the sense that they are tasks which will be completed in the future.
To gather the results from these futures, we use asyncio's as_completed function, which will iterate over the completed futures and gather the results into our results list. This function will essentially block until all of the futures are completed, meaning that we end up returning a list of completed results.Running Our Crawler
We have a parser function defined here which will by default raise a NotImplementedError. So in order to use our crawler, we will have to sub class our crawler and write our own parsing function. Which will do in a minute.
Our main function kicks everything off. We start off by scraping our start URL, and returning a batch of results. We then iterate over our results pulling out the URL, data, and new URLs from each result. We then send the HTML off to be parsed, before appending the relevant data to our list of results. While adding the new URLs to our to_fetch variable. We keep continuing this process until we have reached our max crawl depth, and return all the results collected during the crawl.Sub Classing & Running the Crawler
Sub-classing the crawler is very simple, as we are able to write any function we wish to handle the HTML data returned by our crawler. The above function simply tries to extract the title from each page found by our crawler.
We can the call the crawler in a similar way to how we would call an individual asyncio function. We first initialize our class, before creating future with the asyncio.Task function passing in our crawl_async function. We then need an event loop to run this function in, which we create and run until the function has completed. We then close the loop and grab the results from our future by calling .result() on our completed future.Full Code
I have previously written a piece looking at how to write a web crawler using Go and popular framework Colly. However, it is relatively simple to write a relatively powerful web crawler in Golang without the help of any frameworks. In this post, we are going to write a web crawler using just Golang and the Goquery package to extract HTML elements. All in all, we can write a fast but relatively basic web crawler in around a 130 lines of code.Defining Our Parser Interface
First, we import all the packages we need from the standard library. We then pull in goquery, which we will use to extract data from the HTML returned by our crawler. If you don't already have goquery, you will need to go grab it with go get.
When then define our our ScrapeResult struct, which contains some very simple data regarding the page. This could easily be expanded to return more useful information or to extract certain valuable information. When then define a Parser interface which allows users of our democrawl package to define their own parser to use with the basic crawling logic.Making HTTP Requests
We are going to write a function which simply attempts to grab a page by making a GET request. The function simply takes in a URL, and makes a request using the default Googlebot agent, to hopefully avoid any detection. Should we encounter no issues, we simply return a pointer to the http.Response. Should something go wrong we return nil and the error thrown by the GET request.Extracting Links And Resolving Relative URLs
Our crawl is going to restrict itself to crawling URLs found on the domain of our start URL. To achieve this, we are going to write two functions. Firstly, we are going to write a function which discovers all the links on a page. Then we will need a function to resolve relative URLs (URLs starting with "/").
Our extract links function takes in a pointer to a goquery Document and returns a slice of string. This is relatively easy to do. We simply create a new slice of strings. Should we have passed in a document, we simply find each link element and extract it's href attribute. This is then added to our slice of URLs.
We then have our resolveRelative function. As the name suggests this function resolves relative links and returns us a slice of all the internal links we found on a page. We simply iterate over our slice of foundUrls, if the URL starts with the sites baseURL we add it straight to our slice. If the URL begins with "/", we do some string formatting to get the absolute URL in question. Should the URL not belong to the domain we are crawling we simply skip it.Crawling A Page
We can then start bring all of our work together with a function that crawls a single page. This function takes a number of arguments, we pass in our base URL and the URL we want to scrape. We also pass in the parser we have defined in our main.go function. We also pass in a channel of empty structs, which we use as a semaphore. This allows us to limit the number of requests we make in parallel, as reading from a channel in the above manner is blocking.
We make our requests, then create a goquery Document from the response. This document is used by both our ParsePage function and our extractLinks function. We then resolve the found URLs, before returning them and the results found by the our parser.Getting Our Base URL
We can pull out our baseURL by using the net/url package's Parse function. This allows us to simply parse our start URL into our main Crawl function. After we parse the URL, we simply join together the scheme and host using basic string formatting.Crawl Function
Our crawl function brings together all the other functions we have written and contains quite a lot of it's own logic. We begin by creating a empty slice of ScrapeResult's. We then create a workList channel which will contain a list of URLs to scrape. We also initialize an integer value and set it to one. We also create a channel of tokens which will be passed into our crawl page function and limit the total concurrency as defined when we launch the crawler. We then parse our start URL, to get our baseDomain which is used in multiple places within our crawling logic.
Our main for loop is rather complicated. But we essentially create a new goroutine for each item, in our work list. This doesn't mean we scrape every page at once, due to the fact that we use our tokens channel as a semaphore. We call our crawlPage function, pulling out the results from our parser and all the internal links found. These foundLinks are then put into our workList and the process continues until we run out of new links to crawl.Our main.go file
We can then write a very simple main.go function where we create an instance of our parser. Then simply call our Crawl function, and watch our crawler go out and collect results. It should be noted that the crawler is very fasted and should be used with very low levels of concurrency in most instances. The democrawl repo can be found on my Github, feel free to use the code and expand and modify it to fit your needs.
This blog features multiple posts regarding building Python web crawlers, but the subject of building a crawler in Golang has never been touched upon. There are a couple of frameworks for building web crawlers in Golang, but today we are going to look at building a web crawler using Colly. When I first started playing with the framework, I was shocked how quick and easy it was to build a highly functional crawler with very few lines of Go code.
In this post we are going to build a crawler, which crawls this site and extracts the URL, title and code snippets from every Python post on the site. To write such a crawler we only need to write a total of 60 lines of code! Colly requires an understanding of CSS Selectors which is beyond the scope of this post, but I recommend you take a look at this cheat sheet.Setting Up A Crawler
To begin with we are going to set up our crawler and create the data structure to store our results in. First, of all we need to install Colly using the go get command. Once this is done we create a new struct which will represent an article, and contains all the fields we are going to be collecting with our simple example crawler.
With this done, we can begin writing our main function. To create a new crawler we must create a NewCollector, which itself returns a Collector instance. The NewCollector function takes a list of functions which are used to initialize our crawler. In our case we are only calling one function within our NewCollector function, which is limiting our crawler to pages found on "edmundmartin.com".
Having done this we then place some limits on our crawler. As Golang, is a very performant and many websites are running on relatively slow servers we probably want to limit the speed of our crawler. Here, we are setting up a limiter which matches everything contains "edmundmartin" in the URL. By setting the parallelism to 1 and setting a delay of a second, we are ensuring that we only crawl one URL a second.Basic Crawling Logic
To collect data from our target site, we need to create a clone of our Colly collector. We also create a slice of our 'Article' struct to store the results we will be collecting. We also add a callback to our crawler which will fire every time we make a new request, this callback just prints the URL which are crawler will be visiting.
We then add another "OnHTML" callback which is fired every time the HTML is returned to us. This is attached to our original Colly collector instance and not the clone of the Collector. Here we pass in CSS Selector, which pulls out all of the href's on the page. We can also use some logic contained within the Colly framework which allows us to resolve to URL in question. If URL contains 'python', we submit it to our cloned to Collector, while if 'python' is absent from the URL we simply visit the page in question. This cloning of our collector allows us to define different OnHTML parsers for each clone of original crawler.Extracting Details From A Post Sanita Women s Professional Patent Clog Patent Red Professional Patent Patent 3a71601
We can now add an 'OnHTML' callback to our 'detailCollector' clone. Again we use a CSS Selector to pull out the content of each post contained on the page. From this we can extract the text contained within the post's "H1" tag. We finally, then pick out all of the 'div' containing the class 'crayon-main', we then iterate over all the elements pulling out our code snippets. We then add our collected data to our slice of Articles.
All there is left to do, is start of the crawler by calling our original collector's 'Visit' function with our start URL. The example crawler should finish within around 20 seconds. Colly makes it very easy to write powerful crawlers with relatively little code. It does however take a little while to get used the callback style of the programming.Full Code
In today's post we are going to walk through how to build a flexible image classifier using TFLearn and Python. For those not familiar with TFLearn, it is a wrapper around the very popular Tensorflow library from Google. Building a image classifier with TFLearn is relatively simple and today we are going to walk through how to build your own image classifier.Imports
We are going to need to import a number of different libraries in order to build our classifier. For users on Windows the easiest way to install the Scipy library is to use the pre-compiled wheel which can be found here. Once you have installed all the required imports, we can start building our ImageClassify class.Initializing Our Class
When initializing our class, we are going to need to know a few pieces of information. We are going to need a list of class names. These are the names of the different objects that our classifier is going to classify. We also need to pass in an image size, the classifier will automatically resize our images into a square image of the specified size. So, if we pass in a value of 100, our classifier will end up resizing our images to be 100×100 pixels in size.
Generally, the larger the image size the better the classification we will end up with. This is provided that your images are larger than the specified value. It should be warned that using larger images will increase the time taken to train the algorithm. We store this value in self.image_size.
We also pass in default values for our learning rate and test split. The learning rate dictates how quickly the machine learning algorithm will discover new features. As a default value 0.001 tends to work well. The test split, defines what percentage of samples we will use to validate our model against. Again, using around ten percent of samples for your test size works pretty well.
We also create empty lists which will end up holding our image data and their respective labelsLabeling An Image
Our extract label function takes an image and extracts the label from the image. We begin by creating an array of zeroes, with a zero for each class to be trained. We then split the file name of the image. Our extract label functions expects images in the following format "class.number.png". Using this format allows us to extract the class name directly from the image. We then look up the index of the class label and set that value in our array of zeros to a 1. Before returning the array itself.Processing Images
Our process image function first calls our label function. We then read the image using skimage's io.imread function. We then resize this image to the size specified when we initialized the class. We then append the image data and the labels to self.image_data and self.labels respectively.
Processing images is simply involves us using our process image function on every single image we provide to our image classification class.Building Our Model
Our build model function simply builds us a convolutional net model, using the parameters we defined when initializing our class. Explaining the working's of the net are probably beyond the scope of this post. But I will just note that creating our model like this allows our classifier to be used with images of any size and datasets with any number of classes. Creating a build model function also makes it easier to load and predict using pre-trained models.Training Our Model
Our train_model function, takes a model name, epochs and a batch size parameter. The epochs parameter determines the number of times the model will be run over the entirety of the dataset. The batch size determines the number of samples to be run through the model at once. Generally, the more epochs the more accurate the model will be. Though too many epochs may mean that your model over fits the dataset and you end up with rather inaccurate predictions when you use the model to make predictions. If accuracy hits 100% and loss goes to 0, this is a very strong indication that you have over fit.
We first begin by creating X, y variables using the self.image_data and self.labels variables. We then use our self.test_split value to split the dataset up into training and test sets. We then call the build model function. We then call the fit method on model using both the train and test sets for validation purposes.
Once we have finished training the model. We save the model using the passed in model name and set self.model to be equivalent to our newly trained model.Loading A Pre-Trained Model & Predicting Images
We can define a very simple function to load a model. This will be useful when we need to predict images sometime after we have trained a model. We can load a model by simply passing in the model's name.
We then need another function to take an image and transform it to something we can use in our prediction function. This is much like our process image function, with the exception that we have no need to label the image.
Our predict image function takes a path to an image file. We call our _image_to_array function this data can then be fed straight into the model. Our model will then output an array of probabilities. We can line these up with the classes which we provided to the Image Classify class. We then pull out the most probable label, before returning this and the list of probabilities.Example Usage: Training A Model
Example Usage: Making A Prediction With Already Trained Model
Full Code & Example Dataset
The full code and an example data set can be found on my Github here. The Github also contains another image classification model which makes use of Google's Googlenet model. This model is very highly accurate but takes a considerable amount of time to train and is likely to need to be run for a greater number of epics.
Machine learning frameworks such as Tensorflow and Keras are currently all the range, and you can find several tutorials demonstrating the usage of CNN (Convolutional Neural Nets) to classify text. Often this can be overkill and, in this post we are going to show you how to classify text using Python's NLTK library. The NLTK (Natural Language Toolkit) provides Python users with a number of different tools to deal with text content and provides some basic classification capabilities.Input Data
In the example, I'm using a set of 10,000 tweets which have been classified as being positive or negative. Our classifier is going to take import in CSV format, with the left column containing the tweet and the right column containing the label. An example of the data can be found below:
Using your own data is very simple and simply requires that your left column contains your text document, while the column on the right contains the correct label. Allowing our classifier to classify a wide range of documents with labels of your choosing. The data used for this example can be downloaded here.The Bag of Words Approach
We are going to use a bag of words approach. Simply put, we just take a certain number of the most common words found throughout our data set and then for each document we check whether the document contains this word. The bag of words approach is conceptually simple and doesn't require us to pad documents to ensure that every document in our sample set is the same length. However, the bag of words approach tends to be less accurate than using a word embedding approach. By simply checking whether a document contains a certain set of words we miss out on a lot of valuable information – including the position of the words in a said document. Despite this we can easily train a classifier which can achieve 80%+ accuracy.Initialising our class and reading our CSV file
Our CSV classifier is going to take several arguments. Firstly, we pass the name of our CSV file. Then as optional parameters we pass featureset_size and a test ratio. By default, our classifier will use the most 1,000 common words found in our dataset to create our feature set. Additionally, we will test the accuracy of our classifier against 10% of the items contained in our data set. We then initialise a few variables which will be used later by our classifier.
We then come on to reading our CSV file. We simply iterate through each line and before splitting by our line by commas. The text after our last comma is the documents label, while everything to the left is the document in question. By applying a regex to our document, we produce a list of words contained in the said document. In the example, I used a very simple regex to pull out the words, but it is possible to replace this with a more complex tokenizer. For each word in the document we append this to the list of words. This will allow us to determine the frequency that words occur in our dataset. We also place the list of words found in our document into the variable where we store all the documents found in our dataset.Extracting Word Features
We then write our functions for handling generating our feature set. Here, we use NLTK's Frequency Dist class to store the frequency in which different words where found throughout the dataset. We then iterate through all of the words in the document, creating a new record should we have not seen the word before and incrementing the count should it have already been found. We then limit our bag of words to be equal to the feature set size we passed when we initialised the class.
Now we have got a list of the most frequently found words, we can write a function to generate features for each of the documents in our dataset. As we are using a bag of words approach we are only interested in whether the document contains each word contained in the 1,000 most frequent words. If we find the word we return True, otherwise we return False. Eventually, we get a dictionary of 1,000 features which will be used train the classifier.Training
We start by shuffling the documents. Some algorithms and classifiers can be sensitive to the order of data. This makes important to shuffle our data before training. We then use our feature set function within a list comprehension which returns us a list of tuples containing our feature set dictionary and the documents label. We then calculate where to split our data into training and test sets. The test set allows us to check how the classifier performs against an unseen dataset. We can then pass our data set to nltk's naïve bayes classifier. The actual training may take some time and will take longer the larger the dataset used. We then check the classifiers accuracy against both the training and test set. In all likelihood the classifier will perform significantly better against the training set.Classifying New Documents
Once we have trained a classifier we can then write a function to classify new documents. If we have not already loaded our CSV file and generated the word features, we will have to do this before classifying our new document. We then simply make generate a new set of features for this document before passing this to our classifier class. We will then pass this to our classifier and call the classify method with the feature set. The function will return the string of the predicted label.Saving and Loading Model
Rather than training the model every time we want to classify the sentence, it would make sense to save the model. We can write two simple functions to allow us to reuse our model whenever we want. The save functions simply saves our classifier and feature words to objects to files, which can then be reloaded by our load model function.Accuracy Algorithm Train Test Naive Bayes Classifier (NLTK) 84.09% 72.89% BernouliNB (Sklearn) 83.93% 79.78% MultinomiaNB (Sklearn) 84.58% 74.67% LogisticRegression (Sklearn) 89.05% 75.33% SGDClassifier (Sklearn) 81.23% 69.32%
The algorithm performs relatively well against our example data. Being able to correctly classify whether a Tweet is positive or negative around 72% of the time. NLTK gives it's users the option to replace the standard Naive Bayes Classifier with a number of other classifiers found in the Sci-kit learn package. I ran the same test swapping in these classifiers for the Naive Bayes Classifier, and a number of these classifiers significantly outperformed the standard naive classifier. As you can see the BernouliNB model performed particularly well, correctly classifying documents around 80% of the time.
The accuracy of the classifier could further be improved by using something called an ensemble classifier. To build an ensemble classifier we would simply build several models using different classifiers and then classify new documents against all of these classifiers. We could then select the answer which was provided by the majority of our classifiers (hard voting classifier). Such a classifier would likely outperform just using on of the above classifiers. The full code below provides a function that allows you to try out other Sklearn classifiers.Example Usage
The class is pretty easy to use. The above code outlines all of the steps required to train a classifier and classify an unseen sentence. More usage examples and the full code can be found on Github here.Full Code
What's Baidu?
Baidu is China's largest search engine and has been since Google left the market in {year}. As companies look to move into the Chinese market, there has been more and more interest in scraping search results from Baidu.Scraping Baidu
Scraping Baidu is a relatively simple task. When scraping results from Baidu there is only minor challenge, the URLs displayed on the Baidu results page are found nowhere in the HTML. Baidu links to the sites displayed on the search results page via their own redirector service. In order to get the full final URL we have to follow these redirects. In this post we are going to walk through how to scrape the Baidu search results page.Imports & Class Definition
In order to scrape Baidu, we only need to import two libraries outside of the standard library. Bs4 helps us parse HTML, while requests provides us with a nicer interface for making HTTP requests with Python.
As we are going to scrape multiple pages of Baidu in this tutorial and for this purpose we are going to initialise a class to hold onto the important information for us.
We initialise a new class of the BaiduBot, with a search term and the number of pages to scrape. We also give ourselves the ability to pass a number of keyword arguments to our class. This allows us to pass a proxy, a custom connection timeout, custom user agent and an optional delay between each of the results page we want to scrape. The keyword arguments may be of a lot of help, if we end up being block by Baidu. When initialising the class we also store our base URL, which we use when scraping the subsequent pages.Making Requests & Parsing HTML
We first define a function to scrape a page of Baidu, here we simply try to make a request and check that the response has a 200 Status. Should Baidu start serving us with non-200 status codes, this likely means that they have detected unusual behaviour from our IP and we should probably back off for a while. If there is no issue with the request, we simply return the response object.
Now that we have a way to make HTML requests, we need to write a method for parsing the results page. Our parser is going to take in the HTML and return us with a list of dictionary objects. Each result is handily contained within a 'div' called 'c-container'. This makes it very easy for us to pick out each result. We can then iterate across all of our returned results, using relatively simply BeautifulSoup selectors. Before appending the result to our results list.Getting the Underlying URL
As previously mentioned the full underlying URL is not displayed anywhere in Baidu's search results. This means we must write a couple of functions to extract the full underlying URL. There may be another way to get this URL, but I'm not aware of it. If you know how, please share the method with me in the comments.
Our resolve_urls function is very similar to our Baidu request function. Instead of a response object we are returning the final URL by simply following the chain of redirects. Should we encounter any sort of error we are simply returning the original URL, as found within the search results. But this issue is relatively rare, so it shouldn't impact our data too much.
The we write another function that allows us to use our resolve_urls function over a set of results, updating the URL within our dictionary with the real underlying URL and the rank of the URL in question.Bringing It All Together
We bring this altogether in our scrape_baidu function. We range over our page count variable. For each loop we run through we multiple by our variable by 10, to get the correct pn variable. The pn variable represents the result index, so our logic ensures we start at 0 and continue on in 10 result increments. We then format our URL using both our search term and this variable. We then simply make the request and parse the page using the functions we have already written. Before appending the results to our final results variable. Should we have passed a delay argument, we will also sleep for a while before scraping the next page. This will help us avoided getting banned should we want to scrape multiple pages and search terms.Full Code


text:
Under current law, if a political appointee's political expense is found to be illegal, there is no legal requirement that they personally pay it back or that the money be refunded to taxpayers.
In practice, such appointees often do refund the money because of public pressure — but not always. Trump's original Health and Human Services Secretary Tom Price only
Womens Ff 0172/S Vk Sunglasses Hvna 53 Fendi cZnIXk4
of his private flights which forced his resignation.
Similar issues have plagued other administration members recently including Treasury Secretary Steve Mnuchin, Interior Secretary Ryan Zinke, and former EPA Chief Scott Pruitt who installed an illegal $43,268 phone booth in his office.
Ties On Sale Chocolate Brown Silk 2017 one size Salvatore Ferragamo Ties On Sale 2017 Silk Db1HFzY
[H.R. 6295] would formally create a law that political appointees would have to pay back illegal expenditures out of their own pocket. The illegality would be determined by either the agency's inspector general or the government-wide Government Accountability Office (GAO).
The bill was introduced in June by Rep. Kyrsten Sinema (D-AZ9) .
Supporters argue that the bill creates an official policy of financial fairness that before was only compelled through public pressure rather than force of law.
"Arizonans are sick and tired of having their hard-earned tax dollars wasted. No taxpayer should ever foot the bill for Washington bureaucrats' illegal expenses," Rep. Sinema said in a press release . "Our bipartisan bill forces administration appointees to pay back the taxpayers for illegal expenses and demands accountability."
GovTrack Insider was unable to locate any public statements of opposition from a sitting member of Congress.
Although it was introduced by a Democrat, the majority of cosponsors are across the aisle: four Republicans and one Democrat .
It awaits a potential vote in the House Oversight and Government Reform Committee.
Like our analyses? Want more? Support our work !
What you put in your body is just as important as what you put on it.
Stacked amongst the clear glass jars of homeopathic remedies, immune-supporting supplements, rose creams…
Any time you're on a detox, toxins and chemicals are carried out of your body with water. It follows that drinking clean water is a critical piece…
Integrative structural specialist and frequent contributor Lauren Roxburgh is a bit of a magician. By manipulating…
So here's the thing about GMOs: They're confusing. In fact, most of the people we polled believe that they're bad, but can't articulate exactly why—and those same…
We're no strangers to the power of sweat when it comes to detoxing, but were curious to learn more about why infrared saunas in…
In the first installment of our series with structural integrative specialist, Lauren Roxburgh, we talked about what fascia is—and how you…
Whether you're 47 or 17, male or female, whether your breakouts are once a month or multiple times a week, the reason they come back and back is…
A fascinating account about how to get all the toxins out of your system once they're in there…
For the uninitiated, a colonic is essentially a way to hydrate and irrigate your colon—a section of your intestines that's…
The mystery of exactly how hormones are going to change your child extends to practically every aspect of…
Autism is a complex, diverse developmental disorder, and there is still so much we don't know about it. But in recent years, there have been incredible advancements. Consider some of the wide-ranging potential causes and risk factors that have been identified: Research has linked autism spectrum disorder (ASD) to various biological , environmental , and genetic elements, as well as to prenatal exposure including certain prescription drugs .
Now researchers from the Institute for Exposomic Research at the Icahn School of Medicine at Mount Sinai have unveiled a new milestone: They discovered a way to predict ASD by looking at baby teeth. The teeth show how babies and children
metabolize metals
, which are essential for neurodevelopment. And disruptions in these biological processes have been
Necktie Black floral lace on turquoise Notch VIGSEL TURQUOISE Notch ZkUDrCJ
You all have touched on so many deep and important points with such a simple and clear approach–thank you, thank you, thank you!
One quick note–I realize that DH centers are pretty much all based at universities, but DH collaboration is not solely contained therein. I would love to see this Bill of Rights apply no matter the nature of the institution(s) involved–whether university, museum, library, archive, or other setting. So where possible, I would caution against adhering too strictly to the vocabulary of and about the academy (e.g., "PI," "Universities are locations of creativity and innovation," etc.) Otherwise, this is wonderful! Thanks so much for your hard work.
Interesting. Do you need to distinguish between authors for purposes of credit versus authors for purposes of copyright? For example, in 2A any collaborator identified as an author would own an equal share in the copyright of that work, regardless of the level of her commitment.
You may wish to make it clear that these recommendations address scholarly credit, but that intellectual property rights need to be spelled out in separate agreements.
Overly rigid employment categories are a big problem for DH collaborators in Academia. At one place I worked, for example, librarians can be PIs. People with software development skills don't get hired as librarians though, because the administration is picky about librarians having to be properly credentialled (must have an MSLS or similar), and librarians are harder to hire than regular staff. So when they look for developers to work in the library, they choose (quite rationally) to create staff positions for them so they can pick from a larger pool.
The practical upshot of this is that programmers hired into this library to do DH work, for example, have only limited prospects, can't initiate projects themselves, and can't be promoted beyond a certain point. They are stuck.
None of this is due to malice on anyone's part. It's simply the case that the traditional institutional structures don't work well or fairly for the kind of scholar-programmer who is ideal for DH projects. Changing it would require lobbying the administration, which would require motivation to do so and also successful models to emulate. I think this document is a great step forward towards providing that.
Echoing what some others said above about documenting contributions, I think the nature of "contribution" needs to be clarified a little, possibly by enumeration (in this document) of some kinds of roles, especially those at the edge of projects. (e.g. Does "I send bug reports regularly" count as being a contributor? Is the definition different for tools projects than for resources/archives projects?)
Subscribe to our newsletters to get updates about Illinois Public Media's role in giving voice to local arts, education, new ideas, and community needs, sent straight to your inbox.

subset: commoncrawl


text:
Not content with causing a tidal wave of excitement when he stripped off for Armani in 2007 David Beckham is at it again.
Graduate Fashion Week 2009: Kingston University BA Show Menswear
Kingston University's show was a much-anticipated one, and the atmosphere was electric as the lights went down.
Philippa Jenkins showed smart adventurer looks, with great accessories, she obviously has a knack for colour. Myriam Saumtally focused on sports items and workwear, giving grey jersey a twist with embroidery and we particularly loved the semi-sheer nylon jacket, with houndstooth fabric just visible underneath.
LCF BA show: Aleksandra Domanevskaya
Last night the London College of Fashion's BA Fashion Design students showed their graduate collections.
Farfetch Private Sale
Farfetch, the website that brings together the best from designer boutiques around Europe, is having a private sale. Admittedly there isn't much for a credit crunch budget, but there are plenty of quality investment pieces with up to a 40% discount.
MTV Movie Awards: Kings of Leon and Bruno
Kings of Leon can always be relied on to scrub up well for awards ceremonies, and at the MTV Music Awards yesterday they did look pretty good with one exception. Jared Followill (far right) lets the side down with his whiskered jeans, let us concentrate instead on the chambray shirt and neat peacoat worn by his bandmates.
Poll: Would you take fashion advice from George Lamb?
The Clothes Show kicked off today and over at CQ we were left wondering what Katie Price had in mind for her equestrian bikini.
Top 5: Skinny Summer Ties
The Times has a feature at the moment detailing the changing size of the common tie as we go through decades of economic change. Apparently John Lewis has announced "a dramatic inch-loss of nearly 25 per cent in width over the past six months" with its most popular ties. Charlie Allen, the famous tailor Brandish recently had an exclusive sit-down with over the new 'n' improved England kit, told the Times that "the impact of economic turmoil on tie design can be traced back through the previous recessions of the 20th century. While postwar Britain and the Swinging Sixties embraced exaggerated prints and widths of up to 5in, the downturns of the 1930s and 1980s saw sizes reduce to as little as an inch."
See after the jump for my Top 5 'Skinny Summer ties'– on the cheap for your enjoyment! (They're all under £10)
Would you wear a congruent tie?
The internet is
aflame smouldering with news of the congruent tie. It's a tie which is made so that the stripes on the knot and the rest of the tie match up.
WWWWD Delves into the Diet of Guy Ritchie!
Well, not really but almost! WWWWD, aka the Worldwide Womenswear Digest, is a parody of popular fashion magazines and blogs. With an unknown founder and a readership that is growing massively with each day, the site gives tips and features on everyone and everything.
River Island's graduate success stories pt3: Russell Williams
Over at Catwalk Queen we're gearing up for Graduate Fashion Week, we've been profiling the progression of graduates and reading their tips for getting into the industry. Russell Williams went to University of Derby and has been the menswear t-shirt designer at River Island since 2008.
Woody Allen vs. American Apparel: Dov, Drama and $5 million!
When American Apparel president Dov Charney tells the press that he's "not sorry of expressing myself," it usually means that he has done something involving hip fashion, knee-high socks and models who are dubiously dewy-skinned. However, in this case, Charney is responding to reports over…
Love The Look: Neville Staple from The Specials
We're so glad Neville Staple has ditched the dodgy mid-90s dreads and is looking shipshape for the (partial) Specials reunion. Proving that the Rude Boy look is timeless and stylish, Neville sports a hat and check shirt. Simple and effective.
Oliver Theyskens kicks back and relxes post-Ricci
"The parties tonight are great, but they don't compare to three days ago, when I sat by the Seine on my own and smoked cigarillos. It's my favorite thing in the world. I smoke this brand La Paz–they're cheap but it's what I always buy. And then, I peed into the Seine. It's the best thing I did all week."
Stella McCartney Menswear? Yes, Please!
Since turning up at the Met's Costume Institute Gala with a trinity of her Hollywood girlfriends, Stella McCartney has had a whirlwind week involving everyone from Simon Doonan (creative director of Barneys) to Michelle Obama.
While chatting to the gloriously over-the-top Doonan, Stella was asked about lending her ethical hand to a menswear collection…
André Leon Talley and Alber Elbaz at the Met Costume Gala
It was quite a night for the men, John Galliano turned up in sheer tailoring, and Justin Timberlake was almost unrecognisable in glasses.
Matthew Williamson launches menswear range for H&M
Following on from the high profile launch of Matthew Williamson's H&M range for women comes his men's collection.
As you can tell the collection takes inspiration from Williamson's mix of traditional Brit and colourful hedonism, with brights, seersucker (a brilliant summer fabric) and tie dye.
3sixteen give us a sneak preview of the A/W 09 collection
Andrew Chen from 3sixteen kindly sent us over a sneak preview of their A/W range. I know most of us are just getting our summer wardrobe sorted, but it never hurts to be prepared!
David Beckham teams up with James Bond for Adidas Originals
David Beckham is already promoting Adidas' athletic and football ranges, and now he is chipping in with design too.
Womenswear gets its cues from Menswear
Chances are if you're a Brandish reader then you're already well aware of the importance of menswear, but today Jean Schneides in WWD did some catwalk investigating and concluded that you need to keep one on the menswear collections at all times, if you want a preview of what the womenswear collections will look like a few weeks or months later.
Agyness Deyn, Chloe Sevigny, Paul McCartney & Morrissey at Coachella 2009
Celebs already make us jealous with their post-Christmas holidays in the Carribean, the Bold and the Beautiful have been out at Coachella, the first festival of the summer.
From David Hasslehoff to Jared Leto the US stars were out in full force, having a lovely time but we also had some Uk slebs representing, with Agyness Deyn and Henry Holland putting in an appearance.


text:
Anderson County came out blazing matching Oak Ridge's intensity and then some in the first quarter Tuesday.
Anderson County came out blazing matching Oak Ridge's intensity and then some in the first quarter Tuesday.
It was enough to make Wildcats coach Aaron Green as angry as he has ever been with his team. Down 10-12, Kevin Steen put up a runner and was fouled. The Oak Ridge guard converted the three-point play and the Wildcats outscored the Mavs 59-25 the rest of the way to get the win, 72-37.
"I was probably the most disappointed I have been in the first half with our energy and our effort," coach Green said. "It was very poor for whatever reason."
Green wrote off the fact the 'Cats had defeated the Mavericks 68-29 as an excuse for the lack of effort.
"I can't comprehend that," he said. "As a coach there is a certain way we are going to play ... I say it all the time focus on the process and don't worry about the end result. It doesn't matter who you are playing."
Up 30-21 at halftime, the Wildcats came out of their funk in the third quarter going on a 16-2 run to end the period.
"Halftime was probably one of the more heated halftimes in my three years here," Green said. "Within four minutes, we were up 24. It was because of effort and enthusiasm. Our kids responded. It is amazing what effort can do for you in a game."
Oak Ridge (17-3, 9-1) continued to pour on the defense to start the fourth going on a 11-0 run in the process to wear on Anderson County (6-14, 0-8).
Jaylin Henderson led all scorers with 18 points — 11 in the third quarter. Rian Hall had 15 and Cory Fagan had 10. Kevin Steen finished with eight.
The Mavs were led by Kevin Elliott who had 14 points, 11 of which came in the first half. Ethan Hayes had six.
Oak Ridge now sets its sights on Karns who comes into Wildcat Arena on Friday with an up set in mind.
Oak Ridge defeated Karns on Dec. 11 60-59 when the Beavers final shot was no good.
Karns guard Devin Sibley had a big game that night.
"Sibley, the last time we played them, had 28," Green said. "He almost single-handedly beat us."
A win would keep the Wildcats a game behind first place Powell, who is traveling to Clinton Friday.
Tip-off for the girls game is set for 6 p.m. with the varsity boys to follow at 7:30 p.m. The junior varsity boys are set for a 4:30 p.m. game.
Robert Holder can be contacted at (865) 220-5508.
Anderson Co. 37, Oak Ridge 72
ACHS- Tyler Hicks 3, Kevin Elliott 14, Zane Smith 2, Ethan Hayes 6, Bryley Williams 5, Gabe Templin 2, Daryel Newman 5.
ORHS- Jaylin Henderson 18, Kevin Steen 8, Rian Hall 15, Ted Mitchell 5, Cory Fagan 10, Rashad Gray 5, Tim Grabenstein 2, Xavier Maddox 2, Isaac Merian 7.


text:
Joe McQuaid's Publisher's Notebook: War photos and salty languageBY JOSEPH W. McQUAID
September 22. 2013 6:08PM
If you haven't seen it, you have until Veterans Day to get yourself to the Currier Museum of Art in Manchester for a very powerful news photography exhibit on the Vietnam war.
This is definitely where a picture is worth a thousand words, minimum. Almost all of the photos are iconic, like the one of the little girl running from a napalm attack. Some clear up misconceptions, like that same napalm photo. (Many people think it was an American air attack. It was in fact a South Vietnamese mixup.)
The Currier did an interesting thing with this exhibit. It asked military veterans to share photos and their comments, many of which are now on display. Likewise, the public has been invited to write comments about photos from a war that took place before many people were born. Go see for yourself. You might be moved to comment, too.
Readers leave comments with us every day. One in the mail last week caught my eye with its opening words of "BS!"
Okay, "BS" was actually spelled out, as two words, in fact. One was "bull."
This caught my eye, as did the fact the writer had the Moxie to identify himself, which I appreciate.
He was upset with two recent changes we've made to how the Sunday News is put together.
He shouldn't be. The changes mean we reduce the Saturday night printing most weeks to just two main sections, including live news and sports. It means less chance of longer press runs on a night when our carriers are delivering the biggest package of the week to readers across the state.
The "BS" complainant didn't like obituaries being placed in the sports section. They are actually on the pages immediately following the sports pages; and the new arrangement addresses another reader complaint: having obituaries spread over several pages and in different sections.
Obituaries are an important part of the paper. We can't know in advance how many we will receive on any given day, but we are trying to keep them together as much as possible.
The realignment also puts our Sunday editorial pages in the same section as business news. This also upset my "BS" writer, though he didn't say why. It seems to me a good place for editorial commentary, but the reader said that "Bill Lobe" was going to be waiting for me in the next life and I wouldn't like it much.
Bill Loeb might not like it much that his name was misspelled.
Rumor has it that autumn has arrived. That means the Deerfield Fair must be coming this weekend. I think I'll take Ike, Mike, and Spike. They might have a few comments to make on the fair.
Write to Joe McQuaid at firstname.lastname@example.com or via Twitter at @deucecrew.


text:
In front of a standing-room-only crowd at the Cedars, the founder of Pat's School of Dance was awarded the prestigious Clifton Shipman Award Monday.
In front of a standing-room-only crowd at the Cedars, the founder of Pat's School of Dance was awarded the prestigious Clifton Shipman Award Monday.
Pat Shepherd beamed as she accepted the highest honor given to business leaders from the Hendersonville Merchants and Business Association. The Business Person of the Year Award — named in honor of the late Clifton Shipman, the group's first recipient in 2002 — is given to a local professional who has become a valuable asset to the business community.
"I was surprised," Shepherd said. But she had a clue when she walked into the Cedars and knew just about everyone in the room. Her entire family was in attendance.
Shepherd had come to perform for the ceremony's entertainment, and she did. She was the evening's headlining star.
"I'm very honored and humbled," she said about receiving the award. "Just to be in that list of people of all of those distinguished professionals is quite an honor."
Shepherd said she could never have done it without the support of her family, students, parents and everyone who has worked at Pat's School of Dance, who has helped make it the institution it is today. She founded Pat's School of Dance in 1971 and continues to lead the agency, now with her daughter Sheraton, in its 42nd year. Her dancers have performed around the world.
"This business would not be here just because of me. It has taken my family, my students, the parents and the community to build this business into what it is," Shepherd said. "It was an honor to get the Clifton Shipman award because my father and he were best friends. … I knew Clifton Shipman very well, and it was an honor to get an award with his name on it."
Shepherd has been a trailblazer in several areas. In May 1987, the U.S. Supreme Court ruled that Rotary clubs could not exclude women from membership on the basis of gender. After the decision, Shepherd told her husband, a longtime Hendersonville Lions Club member, that she wanted to start a Lions chapter of her own. She did.
Shepherd was noted as the first woman in the world to join Lions Club International at the ceremony. She was also the organization's first woman president and founder of its first women's-only club — the Four Seasons Lions Club. The women who were members were all mothers — they all worked and had households to keep up, she said.
Shepherd spent a year traveling around the world and speaking to other clubs. The Four Seasons Lions disbanded after about five years, and Shepherd joined her husband in the Hendersonville club.
Shepherd's dancers have won many overall national titles and awards, including the prestigious titanium and president's cup, eight years in a row, from Dance Educators of America.
Pat's School of Dance has won 34 choreography awards for tap dance productions. Shepherd's students have gone on to dance on many college teams, NFL teams, cruise ships, Busch Gardens, regional and state theaters, Universal Studios, Walt Disney World, national tours and Broadway.
Shepherd's students have performed in the Macy's Thanksgiving Day Parade (1995 and 2002), the Orange Bowl Parade (1998), Walt Disney World (1999 and 2006), Atlantis Resort (1999) and as extras in the filming of "Dirty Dancing" (1987).
Shepherd received early recognition as a dynamic choreographer. At age 19, she produced and choreographed a stage show at the Silver City Amusement Park at Chimney Rock. She has choreographed more than 25 musicals and stage productions at Hendersonville High School and the Flat Rock Playhouse. She also has choreographed performances at the Orange Bowl Parade (1998), Lions Club International Forum (1989), Lions Club U.S./Canada Forum (1990), North Carolina's Junior Miss, Wake Forest University Summer Dance Program and Western Carolina University Dance Program.
In 2009, Shepherd was awarded the key to the city of Hendersonville. She received the Athena International Leadership Award in 2010. She was given the Order of the Long Leaf Pine — one of the state's highest honors for service — in 1982.
She also has received the Lions Club International Presidents Award (1989), the Lions Club International Melvin Jones Fellow (1991), the 100 percent Presidents Award for Lions Club International (1987/1988), Outstanding Young Career Woman of the Year (1981) and Honorary Member of the Thespian Troupe of N.C. 851.
All of the awards, she said, have been "very special and very dear to my heart."
Shepherd carried the Olympic torch during the race to Atlanta for the 1996 Summer Olympic Games. The torch hangs in her studio today.
She was a contestant on "The Price is Right" in 1984 and "won a whole bunch of furniture" in the game show's Squeeze Play.
She founded the "Pat Shepherd Second Mile Scholarship," given to a graduating senior at Pat's School of Dance, and organized "Children Helping Children" in her studio. The community service project continues to help less fortunate children and nonprofits throughout the community.
She served on the committee for the International Lions Club Board, as protocol chairman for the U.S./Canada Lions Leadership Forum, as member of the N.C. Apple Festival Board for more than 25 years and as president of the Apple Festival (1985).
Shepherd is married to John M. Shepherd and is mother of Mark and Sheraton Shepherd. She is a graduate of West Henderson High School and Blanton's Business College.
"I love doing what I do. I'm really very fortunate," she said. "I live a very blessed life to say the least."
Reach Weaver at firstname.lastname@example.com or 828-694-7867.


text:
LONDON (AP) — Atheists could be allowed into Britain's boy and girl scouts after more than a century.
The Scout Association said Tuesday it plans to draft an alternative oath without references to God so that young people who do not have a faith could also join the organization.
The current oath — which will remain in use — reads: "On my honor, I promise that I will do my best, to do my duty to God and to the queen, to help other people and to keep the Scout Law."
The group has offered alternative versions of the words "duty to God" for Muslims, Hindus and Buddhists for over four decades, but there have been no such exceptions made for atheists.
Wayne Bulpitt, the association's chief commissioner in the UK, said the group is asking its members' views on the proposed alternative to keep the group relevant and to encourage membership.
"We are a values-based movement and exploring faith and religion will remain a key element of the Scouting program. That will not change," he said. "However, throughout our 105-year history, we have continued to evolve so that we remain relevant to communities across the UK."
But he said the group plans to keep its oath to the queen, although non-British citizens living in Britain may pledge duty to "the country in which I am now living."
The scouting movement began more than a century ago and now encompasses some 40 million members worldwide. Scouts in each country run independently and make their own policies.
In October, a San Francisco Bay-area teenager was kicked out of the Boy Scouts of America because of the organization's national policy of excluding gay members. The British Scout Association said they have an equal opportunities policy and welcome people of all sexualities.


text: Oh Dave--there is SO much more to lovemaking than that little moment at the end. So very much more. It's a foolish man who thinks that's what it's all about. I am surprised you don't know this. No wonder you hold the act of love in such little regard.seen as completed by those engaged in it due to orgasm


text:
BYU Hawaii environmental studies
Think about what career is best suited for you, and where you want to study. Request more information now.
Think about what career is best suited for you, and where you want to study. Request more information now.
Think about what career is best suited for you, and where you want to study. Request more information now.
Get the facts about BYU Hawaii environmental studies, and registering for elective seminars and lectures. Which college degrees give you the best chances of finding employment after graduation? There are a variety of engineering training programs that may appeal to you, such as mechanics, vehicle design, and alternative energy production. Concentrating on BYU Hawaii environmental studies, for example, may give you an advantage in job recruiting on-campus. Check with the environmental reclamation department for course registration deadlines, and engineering program requirements. Please use the form above to request admissions info for BYU Hawaii.
The following technologies are used to clean up contaminated sites during enviromental reclamation work: Gas Chromatography (GC) is the most widely used chromatographic technique for environmental analyses, and is used onsite in field investigations and by offsite reference laboratories. Immunoassay technologies use antibodies to identify and quantify organic compounds and a limited number of metallic analytes. This technique is used widely for environmental field analysis because the antibodies can be highly specific to the target compounds. Mass Spectrometry is an established analytical technique that identifies organic compounds by measuring the mass of the compound's molecule. Although mass spectrometry can be used for the analysis of metals, non-metallic elements and radionuclides, it is most generally used for organic analysis as a field analytical technique.
Ground Penetrating Radar (GPR) is a geophysical method that has been developed for shallow, high-resolution, subsurface investigations of the earth. GPR uses high frequency pulsed electromagnetic waves to acquire subsurface information. Magnetic devices specifically developed for environmental applications may be used to locate subsurface iron, nickel and cobalt alloys which are typically referred to as ferrous materials. The technology has been widely used for quickly locating buried or subsurface ferrous objects that could pose a potential threat to the environment or hinder reclamation efforts. Beyond major requirements within the environmental reclamation department, you may consider Dallas County Community College environmental modeling as a complementary elective.


text:
January 11. 2014 8:25PM
Ted Siefer's City Hall: New beginning for some, same old same old for others
TUESDAY WAS inauguration day, and the ceremony at the Palace Theatre was aptly formal — the city's elected officials sported white carnations — and full of hopeful and lofty rhetoric. Mayor
Ted Gatsas
' inaugural address had the theme "move forward."
But after the boards of aldermen and school committee held their organizational meetings later in the day, some elected officials were probably no longer feeling the love.
At the end of those organizational meetings, the aldermen and school board members received the mayor's committee assignments, and the decisions might lead some of them to conclude that Gatsas has been making a list and checking it twice.
There was a great deal of shifting around among the committees. For example, Alderman-At-Large
Joe Kelly Levasseur
found himself on two new committees, one of them Bills on Second Reading, which meets infrequently.
Of course, Levasseur, although he shares a Republican Party affiliation with Gatsas, has proved to be the biggest thorn in the mayor's side.
"His speech was about the need to move forward and forget the past, but it's obvious he'll never forget. He's all about retribution and retaliation and getting back at people," Levasseur said.
Of course, given Levasseur's fractious relationship with the mayor, some sour grapes might be expected from him, regardless of what committees he was dealt.
But then Ward 5 Alderman
Ed Osborne
, a 20-year veteran of the board, was also somewhat miffed by his assignments. He lost his chairmanship of the Land and Buildings Committee and was shifted to two new committees, Bills on Second Reading and Accounts. He's not the chairman of either.
"I'm a little upset with it. At least give me Traffic, at least something I'm good at," Osborne told me, adding of Gatsas, "I'm not going to throw mud. That's his business. I appreciate that, but I don't appreciate what I got."
For the conspiracy minded, it might be notable that Land and Buildings was the most thoroughly shaken up. Other committees kept several of the previous members and their chairmen, such as Alderman-At-Large
Dan O'Neil
, who heads the Accounts committee, and
Jim Roy
, chairman of the CIP (Community Improvement Program) committee.
There was at least one stubborn issue that couldn't escape the clutches of Land and Buildings last year: the owner of a Dunkin' Donuts near the southern corner of Livingston Park wanted to buy a sliver of park land for the parking lot and a drive-through. The owner also happened to be a friend and big campaign supporter of the mayor. Thanks to the triumvirate of Levasseur, Osborne and Ward 2's
Ron Ludwig
, who each opposed selling the land for various reasons, the matter never made it out of committee. Levasseur, for one, suspects this played a role in Gatsas' decision to essentially disband the previous committee.
But Gatsas insisted there was no payback in his committee choices.
"I've been very clear, I don't do things based on a political nature," he told me. "Putting committees together is not an easy thing. Some people will be happy and some won't."
Another person one might think would feel dissed by the assignments is political veteran
Keith Hirschmann
, the newly elected Ward 12 alderman. He had previously held the seat in the 1990s, he's served in the state Legislature, and he's one of the only other Republicans on the board.
But Hirschmann said he had "no problem" with not getting a chairmanship or his two assignments: Bills on Second Reading and Land and Buildings.
"I think the committee work will be interesting," he said. "I didn't expect any favors of any kind. I went in there with low expectations, being the new kid coming back after so many years."
Then there's
Art Beaudry
, the Ward 9 school board member. He got bumped from his somewhat unlikely chairmanship of the Information Technology Committee (the guy doesn't use email), and he received just two committee assignments: Athletics and Extra-Curricular Activities, and Student Conduct.
Of course, Beaudry is by far the board's most active member. And while his passion comes through at every meeting, it's safe to say he hasn't endeared himself to his colleagues.
The most significant thing to come out of Tuesday's organizational meeting was the thorough overhaul of the board's rules of order. If the school board were Congress, the document might have been called the "Anti-Beaudry Act of 2014."
There was a fair amount of confusion over what was actually new in the rules. For example, one of the most hotly debated provisions, concerning a ban on personal complaints from the public against "any person connected with the district," had been in the district's policy book since 2002.
But the changes approved nearly unanimously by the board will have one clear effect. They will ditch the venerable Robert's Rules of Order - which was scarcely understood by members of the board - as its parliamentary guidebook in favor of the rules used by the aldermen. The mayor has a firm grasp of these rules, and the change will likely allow him to wield much more control over meetings, particularly when they come to be dominated by a certain school board member.
The board's new vice chairman,
Dave Wihby
, who compiled the new rules, wouldn't go so far as to say Beaudry was their inspiration. But he had this to say: "While some people want to get things done, some want to grandstand. If this reduces the grandstanding, maybe it's a good thing."
Beaudry did not return a call for comment about the new rules or his committee assignments.
On Monday, the school board will hold its first meeting of the new year, so we'll get to see the new rules in action soon enough.
Ted Siefer is the City Hall reporter for the New Hampshire Union Leader and New Hampshire Sunday News. He can be reached at
email@example.com and followed on Twitter: @tbsreporter.


text:
King's CupA fun mixed league with 6-end games. Dedicated to Jim and Marie King, long-time members and supporters of mixed curling. The Spirit of Curling Award will be bestowed on a team that mentors new curlers, exhibits fair play and good sportsmanship, and makes the game enjoyable.
When: Fridays, 4:30 pm, 6:15 pm; 8:00 pm; 2nd half TBA
Chairs: Joyce Shaffer, Jennifer Turner
Registration form
Spirit of Curling Award Winnersfall 2016 (5:30): Joyce Shaffer, Jim Dell'Anno, Laura Lake, Dan Simon
fall 2016 (7:30): Jerry Stevens, Marena Taverne, Dave Preznya, Kimberly Lambing
spring 2016 (5:30): Bob Prenoveau,
spring 2016 (7:30): Joel Scherer,
fall 2015 (5:30): Fred Hicks, Jan Rishel, Pat Milhako, Marianne Turczyn*
fall 2015 (7:30):
spring 2014 (5:30):
spring 2014 (7:30): Bob Burnett, Liz Nolan, Dave Mitchell, Israel Acosta*
Historical ResultsThe King's Cup was a medal event prior to Fall 2014
Previous WinnersThe King's Cup was a medal event prior to Fall 2014
2013: Roger Rowlett, Knyoca Law, Mike DiMeo, Jeff White
2012: Mike Swiercz, Erlene Puleo, Joe Puleo, Dee Pfohl
2011: Bob Caine, Diane Caine, Rich Wielgosz, Mary Boak
2010: Eric Hauser, Laura Jacon, Tom Platt Sr., Ann Kebabian
2009: Bob Burnett, M.J. Walsh, Jerry Stevens, Jim Barefoot
2008: Don Knapp, M.J. Walsh, Bill Turner, Erin Hauser
2007: Karen Rogowski, Mike Kessler, Kim Freely, Pete Callari
2006: Don Knapp, Teale LaBarbera, Alex Ryan*, Rachel Ryan*
2005: Kathy Davis, John Davis, Teale LaBarbara, Bob Prenoveau
2004: Pat Costello, Brenda Card, Tomm Platt Sr., Marty Jones
2003: Vinnie Grove, Heather Swiercz, Tom Platt Sr., Cameo Voltz*
2001: Bill Morehouse, Ginger Zillioux, Doug Pens, Eileen Pens
2000: John Davis, Jane Jones, Tom Fritz, Nancy Guido
*denotes new curler (when known)
Previous winners (all events)
General Mixed Rules
Standing Rules
- This is the fun mixed (co-ed) league running in the entire year.
- The format of the finals will be decided by the event Chair, prior to the start of the first game, either;
- By the win/loss record of the top six teams of the first draw vs. the top six teams of the second draw. Or,
- By the win/loss record for the top twelve teams over both draws.
- Games are six ends, with ties played off by an extra end.
- Ties in the standings will be broken based on plus points accumulated during games played and, if required, total number of ends won.
- A maximum of +10 points will be allowed for a win and –10 points for a loss.
- Substitutions:
- The rule of two men and two women playing alternate positions may be ignored if necessary to permit everyone signing up to participate. Substitutes should be on the same sex and abilities as the regular team member whenever possible. In no case shall any team of four men or four women compete.
- A team is allowed a maximum to two substitutes per games. This limitation and the other general rules on substitution do not apply for players who are away on bonspiels or have an emergency circumstance during league games. However, they do apply during the finals. Under no circumstances can there be 4 subs on one team. Team must schedule a make-up or forfeit.
- Substitutes for all positions must be of equal skill level or less to the player being replaced. Use of the established substitution ranking systems for men's and women shall be employed. Any discrepancy will be resolved by the discretion of the event chair(s). Changing the order of players may be allowed to field a legal team with the advanced approval of the event chairs. Let good sportsmanship and curling honor prevail at all times.
- Two players from the original team must be available to avoid a forfeit.
- Points awarded for a forfeit win will be the average of all other winning teams for the evening. Points awarded for a forfeit loss will be the average of all other losing teams for the evening.


text:
Posted on September 18, 2009
"I have been looking for this book MY WHOLE LIFE!"
A lot of customers will sympathize with this buyer, even if he was only about six years old. There are plenty of people who come to the Book Fair looking for their Holy Grail, from the woman who burst into tears as she was showing me a New York Times Cookbook in her price range to the woman who barreled into the fiction area declaring "I would KILL for Stephen king in hardcover!" (I believed her, and got out of her way.)
It doesn't even have to be that One Special Book, or Holy Grail. It can be a chance encounter with a book you didn't expect would matter to you. One woman carried off five novels she didn't intend to read, simply because she remembered her grandmother reading Grace Livingston Hill. Another took an armful of paperbacks because her mother was always going to garage sales looking for Betty Neels. "I wonder if she's any good," she said, "And at this price I can afford to find out."
Some people shopping in the children's section will admit they're buying the books for themselves. "I know I read The Velvet Room when I was eleven, and I loved it," said one shopper. "But I can't for the life of me remember how it came out." And there's a lot of "I didn't know that was a novel before it was a movie. I wonder what how much alike they are."
People are thrilled to find travel books about the places where their parents met, or old textbooks they didn't use but which have the name of their old elementary school stamped in the front. I myself once opened a two volume Greek-English lexicon of the 1890s to find its original owner lived in my hometown in Iowa. It's a nice town, but I wouldn't expect to find many Greek-English dictionaries there. (And I didn't; it had to go to Chicago before I saw it.)
One woman dropped by to chat one day and picked up a cookbook that was sitting on a table. "I had a copy of this but I lent it to somebody," she said. "There was an apple crisp recipe on page…." She opened the book and fell silent.
Her name was written on the flyleaf. The person she'd lent the book to thirty years earlier had moved out of town and donated a load of books on the way. She bought it, of course. I was glad to be part of the reunion (though she never dropped off any apple crisp.)


text:
Joseph W. McQuaid: Anti-gun shenanigans are outrageous and foolishBy JOSEPH W. McQUAID
New Hampshire Union Leader Publisher
January 16. 2013 11:22PM
Outrageous: President Obama using schoolchildren as White House stage props for his anti-gun message.
Foolish: A New Hampshire legislator carping that a normal activity involving guns, specifically a police chiefs' fundraising raffle, should cease because of the Newtown, Conn., school shooting tragedy.
We thought the national issue for the President was the safety and well-being of young children. In the wake of Newtown, we were told repeatedly not to even bring up that terrible massacre with the little ones. So what does Obama do?
He puts some of them on stage, spotlights the tragedy, and thereby attracts the attention and curiosity of thousands more children around the country. Pleasant dreams, kids.
Here in New Hampshire, Democratic state Rep. Sharon Nordgren of Hanover takes offense that the Chiefs of Police Association didn't cancel a raffle begun last October to raise money for an annual youth police academy.
Nordgren doesn't like it that guns will be raffled off to qualified winners. It is insensitive, says she, not only because of Newtown but because of previous shootings.
"They should be pulling back and trying to not be so upfront," she said.
Nonsense. The chiefs' association is to be applauded for its long-standing sponsorship of the youth training academy and for a perfectly sensible and obviously popular (it is already sold out) gun raffle.
Crime, law and justice Politics Public Safety Social issues Front page editorial


text:
Bubble Knit Lamp Shade
$59.00 - $79.00
Overview & Details
- Girls' lamp shade.
- Bright, textured color lamp shade for kids' bedrooms.
- Small 8.5" x 11" x 9"H.
- Large 12" x 16" x 13"H.
- Available in Cream, Hot Pink or Papaya.
- Color Cream available in size Large only.
- Our Bubble kids' lamp shade coordinates perfectly with our Dahlia or Chloe lamp base (sold separately) or your own lamp base.
This charming lamp shade adds a dash of bright color to a little girl's bedroom. Offering unique texture and fashionable looks, our Crochet Bubble lamp shade for girls coordinates perfectly with our Chloe or Dahlia lamp base, sold separately. Or, use it with your own lamp base to give her current lamp a whole new look. Choose from three fun colors of Hot Pink, Papaya or White.


text:
English translation of 'déployer des trésors d'imagination'
déployer des trésors d'imagination
to be highly imaginative
Copyright © by HarperCollins Publishers. All rights reserved.
Nearby words of 'déployer des trésors d'imagination'
Translation of déployer des trésors d'imagination from the Collins French to English Dictionary
Join the Collins community
All the latest wordy news, linguistic insights, offers and competitions every month.
Read more
Study guides for every stage of your learning
Whether you're in search of a crossword puzzle, a detailed guide to tying knots, or tips on writing the perfect college essay, Harper Reference has you covered for all your study needs.
Read more


text:
Search Results For "intex giant inflatable floating mat 114in x 84in" (0)
We could not find any matches for your search. "intex giant inflatable floating mat 114in x 84in"
Search Tips
- When searching for an item number, only enter the numbers before the dash (ex. 100130-5901)
- If you are shopping from a catalog try searching by keyword(s) in the product description or use our Quick Order.
- Make sure all words are spelled correctly.
- Try using fewer words.
- Try using more general keywords.
- Try different keywords.
Still can't find what you're looking for? Call 1-800-685-4845


text:
Police say Maine brothers were 'eelusive' poachersBy JASON SCHREIBER
Union Leader Correspondent
May 03. 2013 10:15PM
HAMPTON FALLS - Two brothers from Maine, suspected of poaching baby eels to sell for big money overseas, were arrested Friday morning after allegedly trying to slip away when authorities moved in to catch them.
Matthew Kinney, 29, of Bremen, and his brother, Justin, 35, of Mount Vernon, face several charges after they were scooped up by authorities following a land and air search that lasted nearly four hours.
Fish and Game Lt. Michael Eastman said authorities received a tip early Friday morning that two men were catching baby American eels in the Hampton River near Route 1.
Eastman said two conservation officers responded to the area shortly before 5 a.m. and found that the brothers had a "substantial amount" of young eels, which are illegal to harvest in New Hampshire.
When one of the officers attempted to arrest Matthew Kinney, Eastman said, Kinney assaulted the officer, was pepper-sprayed, and then fell into the river and later fled into the woods with a handcuff on one of his wrists. Justin Kinney ran into the marsh grass and was arrested a short time later.
Eastman did not identify the conservation officer who was assaulted, but he said the officer wasn't injured.
Police from Kensington, Hampton Falls and Seabrook, along with Fish and Game officers, New Hampshire State Police, Fish and Game and search dogs and a helicopter, assisted in the hunt for Matthew Kinney.
The dogs eventually sniffed out a track leading from the area where Matthew Kinney was last seen to the Hampton Falls Inn on Route 1. Eastman said authorities learned that the brothers had rented a room at the hotel Thursday night. They found Matthew Kinney inside his hotel room around 9:15 a.m. Friday removing his wet clothing and arrested him.
Eastman said Maine and South Carolina are the only states on the East Coast where the harvesting of young eels, also known as elvers, is legal with a license.
"They were pretty much down here to poach elvers to take back to Maine to sell," Eastman said.
Illegal harvesting of baby eels has become a growing problem in New Hampshire as their selling price has skyrocketed in recent years, prompting state Fish and Game officials to crack down.
According to officials, poachers illegally catch the baby eels here and sell them to a dealer who ships them overseas to be raised and sold for meat once they're grown.
"The demand has pushed the price right up there," Eastman said. He said the number of poaching cases locally has climbed over the last three years.
"This is pretty much like having a duck season where all you can shoot is ducklings. They're taking immature eels, or young eels. What we're seeing is that this poaching is causing the bigger eels, the American eels, to decline," Eastman said.
Eastman wasn't sure how many baby eels were caught by the Kinneys, but he said the amount was typical of what authorities have seen recently.
The arrests come a month after Phillip Parker of Candia was cited by Maine's Marine Patrol for allegedly illegally possessing 41 pounds of baby eels. Maine authorities said an investigation showed that Parker intended to sell the eels without a license. It was the largest case of illegal possession of eels in the fishery's history, according to Maine officials.
Meanwhile, Matthew Kinney is charged with simple assault, disobeying a conservation officer, resisting arrest, taking American eels less than 6 inches long, and taking without a harvest permit. He's being held on $5,000 cash bail.
Justin Kinney is charged with disobeying a conservation officer, providing false information to law enforcement, hindering apprehension, taking American eels less than 6 inches long and taking without a harvest permit. His bail was set at $2,500 cash.
The brothers, both of whom face thousands of dollars in fines if convicted. were being held at the Rockigham County jail and will be arraigned Monday in the 10th Circuit Court in Seabrook.