CeWL written in Python This is my implementation of CeWL, a ruby application that returns a list of words which can be used for password crackers.

My main concern is performance, as it takes a couple seconds to run. I presume it's mainly within the get_words function. Any and all feedback and criticism is appreciated and considered.

"""
Cewl Implementation (without page depth)

Usage: python3 cewl.py &lt;max word length&gt; &lt;name of file to create&gt; &lt;valid website url&gt;
"""

import string
import sys
import os
from typing import List

import bs4.element
from bs4 import BeautifulSoup
from bs4.element import Comment
import urllib.request

FILE_DIRECTORY = os.path.dirname(os.path.realpath(__file__))

def write_to_file(filename: str, words: List[str]) -&gt; None:
    """
    Creates a file in the same directory as the file.

    :param str file_name: Filename of word file
    :param List[str] words: Words to write to file

    :return None:
    """
    with open(f"{FILE_DIRECTORY}/{filename}", "w") as file:
        for word in words:
            file.write(f"{word}\n")

def filter_words(words: List[str], max_length: int) -&gt; List[str]:
    """
    Filters each of the words by the criteria passed by the user. Also
    ensures that the word is a "word" (no numbers or special characters).

    :param List[str] words: Words to filter
    :param int max_length: Max word length

    :return List[str]: Filtered words
    """
    return [word for word in words if len(word) &lt;= max_length and word.isalpha()]

def visible_tag(element: bs4.element) -&gt; bool:
    """
    Returns if the passed element is visible to scrape words.

    :param bs4.element element: Element to check

    :return bool: True is visible, False otherwise
    """
    return (
        not element.parent.name in ['style', 'script', 'head', 'title', 'meta', '[document]'] and
        not isinstance(element, Comment)
    )

def get_words(url: str) -&gt; List[str]:
    """
    Returns all the words in the website. Steps this function takes:

    1. Get all text from website
    2. Creates a list with words that have punctuation removed
    3. Removes newlines from each word in the list and lowers it
    4. Removes each "" in the list and joins to a string
    5. Returns a list of each word as an element of the list

    :param str url: Url of website

    :return List[str]: List of all text in website
    """
    body = urllib.request.urlopen(url).read()
    soup = BeautifulSoup(body, 'html.parser')
    texts = soup.findAll(text=True)

    # Steps begin #
    visible_texts = filter(visible_tag, texts)
    words = [str(word).translate(str.maketrans('', '', string.punctuation)) for word in visible_texts]
    words = [word.rstrip().lower() for word in words]
    words = " ".join(filter(None, words))
    return words.split()

def main() -&gt; None:
    """
    Main Method for Cewl.

    :return None:
    """
    try:
        max_word_length = int(sys.argv[1])
        filename = sys.argv[2]
        website = sys.argv[3]
    except IndexError:
        sys.exit("Usage: python3 cewl.py &lt;max word length&gt; &lt;name of file to create&gt; &lt;valid website url&gt;")

    words = get_words(website)
    words = filter_words(words, max_word_length)
    words = list(set(words)) # Remove duplicate words and convert back to list #
    write_to_file(filename, words)

if __name__ == "__main__":
    main()
You have quite a few methods that need to run for all words. Currently you always make sure that you output a list again, but you could use generator expressions to avoid these intermediate lists.

from typing import Generator, Iterable

REMOVE_PUNCTUATION = str.maketrans('', '', string.punctuation)

def filter_words(words: Iterable[str], max_length: int) -&gt; Generator[str, None, None]:
    """
    Filters each of the words by the criteria passed by the user. Also
    ensures that the word is a "word" (no numbers or special characters).

    :param Iterable[str] words: Words to filter
    :param int max_length: Max word length

    :return Generator[str, None, None]: Filtered words
    """
    return (word for word in words if len(word) &lt;= max_length and word.isalpha())

...    

def get_words(url: str) -&gt; Generator[str, None, None]:
    """
    Returns all the words in the website. Steps this function takes:

    1. Get all text from website
    2. Creates a list with words that have punctuation removed
    3. Removes newlines from each word in the list and lowers it
    4. Removes each "" in the list and joins to a string
    5. Returns a list of each word as an element of the list

    :param str url: Url of website

    :return Generator[str, None, None]: Iterable of all text in website
    """
    body = urllib.request.urlopen(url).read()
    soup = BeautifulSoup(body, 'lxml')
    texts = soup.findAll(text=True)

    # Steps begin #
    visible_texts = filter(visible_tag, texts)
    words = (str(word).translate(REMOVE_PUNCTUATION).rstrip().lower()
             for word in visible_texts)
    return filter(None, words)


Note the () instead of []. I also made the trans a global constant instead of re-creating it every iteration, grouped up some of the methods and used the faster LXML parser. The generator expression will be run by the call set(words) in main.

Your write_to_file can also be slightly improved by using writelines:

def write_to_file(filename: str, words: Iterable[str]) -&gt; None:
    """
    Creates a file in the same directory as the file.

    :param str file_name: Filename of word file
    :param Iterable[str] words: Words to write to file

    :return None:
    """
    with open(f"{FILE_DIRECTORY}/{filename}", "w") as file:
        file.writelines(f"{word}\n" for word in words)


You might want to consider using requests instead of urllib, since it is much more usable (although the difference in this case is negligible). Instead of manually parsing sys.argv, you could use argparse, which allows much more complicated argument parsing and automatic help message generation.

Regarding style, PEP8, the official style-guide, recommends using less than 80 characters per line, which your code does not and mine mostly does (stupid signature for generator type...). However, larger values are also acceptable, if applied consistently. More than 100 characters should be avoided nevertheless.