Style Information

Check out Raymond Hettinger’s tips on writing beautiful python.

Working with files

Use os.path.realpath() to elegantly handle command line arguments that might be relative paths. Check out os.path.expanduser() if you want to properly handle ~.

glob.glob('*.txt') will return a list of text files in the working directory.

Get the filename with os.path.basename().

CLI Template

A simple template for creating CLI python applications—can read from input stdin or a given input file.

import argparse
import logging
import sys


def main():
    parser = argparse.ArgumentParser()
    parser.add_argument('infile', nargs='?', type=argparse.FileType('r'), default=sys.stdin)
    parser.add_argument('-v', '--verbose', action='store_const', const=logging.INFO, dest='loglevel',
                        help='increase output verbosity.')
    parser.add_argument('-d', '--debug', action='store_const', const=logging.DEBUG, dest='loglevel',
                        default=logging.WARNING, help='show debug output (even more than -v).')
    
    args = parser.parse_args()

    logging.basicConfig(level=args.loglevel)

    for line in args.infile:
        pass


if __name__ == '__main__':
    main()

Regular Expressions

import re

# Matches seconds, hours, minutes, e.g., 10, 10s, 1m, 1h
value, unit = re.match(r'(?P<value>[0-9]+)(?P<unit>[smh]?)', raw.lower()).groups()

Virtual Environments

Sometimes you just get sick of trying to resolve all of the strange dependencies in order to make that shiny new python package work (Ed. Note: I am looking at you scrapy)—Virtual Environments to the rescue.

Go to your project directory and do the following:

virtualenv venv
source venv/bin/activate

#you should see 'venv' at the beginning of your command prompt now
pip install Scrapy

#test that it worked
which scrapy

#Save the requirements in case you want to remake
pip freeze > requirements.txt

#Get rid of it
deactivate
rm -rf venv

pip install -r requirements.txt

Read more about Virtual Environments here.

Errors!

ImportError. Occasionally (if your primary machine account isn’t an admin), you’ll get the dreaded ImportError: No module named... when trying to import a package, even though pip install tells you everything is fine. You are probably getting this error because you previously thought it was necessary to use sudo to install the package. To fix this, uninstall the package using sudo and reinstall without sudo using the --user option. For example:

su admin
sudo pip uninstall beautifulsoup4

#Switch back to a normal user (one without sudo powers)
pip install --user beautifulsoup4

Make sure that you have also added the appropriate package-binary path to your PATH variable as some package need this, e.g., scrapy.

#Find scrapy
find ~ -name scrapy

#Add that path to your PATH variable (you can also put this in your bash_profile)
export PATH=$PATH:$HOME/Library/Python/2.7/bin