Base class for feed-parsing I have written a class which I will use as a base class, allowing other classes to extend this class.

How can I improve this? Am I breaking any conventions or styling?

import requests
import feedparser
from BeautifulSoup import BeautifulSoup

class BaseCrawler(object):
    ''' Base Class will be extended by other classes'''

    def __init__(self, url):
        self.url = url

    def test(self):
        return self.url

    def get_feed(self):
        feed = feedparser.parse(self.url)
        return feed

    def post_request(self):
        res = requests.post(self.url, self.data)
        return res

    def get_request(self):
        res = requests.get(self.url)
        self.raw = res.content
        return self.raw

    def build_entity(self):
        raise NotImplementedError("Subclasses should implement this!")

    def process(self):
        return NotImplementedError("Subclasses should implement this!")

    def build_query(self):
        return NotImplementedError("Subclasses should implement this!")
I'm not sure what you're intending to do with this class so I can only offer some general tips.


Make private variables private:

def __init__(self, url):
    self._url = url

Use properties instead of get_ methods. For example:

@property
def request(self):
    return self._requests.get(self._url)

# to make it writeable:
@request.setter
def request(self, value):
    self._requests.set(value)

If you're using Python 3, have a look at abc.ABCMeta for better solution for abstract base classes with abstract methods.