Posts in category workingenv

Adding support for a workingenv sandbox to setuptools/distutils

It seems that all I blog about is workingenv.

This time it's a snippet of code that adds a "sandbox" command to distutils, which automatically creates a workingenv from the setuptools extras_require, install_requires and dependency_links options listed in your setup() call. It only supports *nix systems for now, but could easily be extended to support Windows.

Here's the code: 

import os
from setuptools import setup, find_packages
from distutils.cmd import Command

class sandbox(Command):
    description = 'Create a development sandbox using workingenv'
    user_options = [
        ('path=', None, 'workingenv path'),
        ('extras', None, 'also include "extras" requirements'),
        ]

    def initialize_options(self):
        self.path = 'wenv'
        self.extras = False

    def finalize_options(self):
        pass

    def run(self):
        requires = open('requirements.txt', 'w')
        try:
            requirements = self.distribution.dependency_links + \
                           self.distribution.install_requires
            if self.extras:
                extras = self.distribution.extras_require or {}
                requirements += extras.values()
            requires.write('\n'.join(requirements))
        finally:
            requires.close()
        cwd = os.getcwd()
        import workingenv
        workingenv.main(['--always-unzip', '--requirements=requirements.txt',
                         '--site-packages', '--verbose', self.path])
        os.chdir(cwd)
        os.symlink(self.path + '/bin/activate', 'sandbox')
        print
        print 'XXX: Use ". sandbox" to activate the development sandbox'

setup(
    name='MyCoolPackage',
    version='0.0.0.1',
    packages=find_packages(),
    # Add the sandbox command
    cmdclass={'sandbox': sandbox},
    # Search some extra locations for dependencies
    dependency_links=[
        'http://svn.edgewall.org/repos/genshi/trunk#egg=Genshi-dev',
        'http://trac.pocoo.org/repos/werkzeug/trunk#egg=Werkzeug-dev',
        'http://svn.sqlalchemy.org/sqlalchemy/trunk#egg=SQLAlchemy-dev',
    ],
    install_requires=[
        'setuptools >= 0.6b1',
        'Genshi >= 0.5.dev-r698,==dev',
        'Werkzeug >= 0.1.dev-r3831,==dev',
        'SQLAlchemy >= 0.4.0.dev-r3203,==dev',
        'AuthKit >= 0.3.0pre5',
    ],
)

And here's an example of how to use it: 

$ python setup.py sandbox --help
Common commands: (see '--help-commands' for more)

...

Options for 'sandbox' command:
  --path    workingenv path
  --extras  also include "extras" requirements

...
$ python setup.py sandbox --path=mysandbox --extras
running sandbox
Reading requirement requirements.txt
Making working environment in /home/athomas/p/test/mysandbox
Creating lib/python2.5

...

...Installing http://svn.edgewall.org/repos/genshi/trunk#egg=Genshi-dev,
http://trac.pocoo.org/repos/werkzeug/trunk#egg=Werkzeug-dev,
http://svn.sqlalchemy.org/sqlalchemy/trunk#egg=SQLAlchemy-dev,
setuptools >= 0.6b1, Genshi >= 0.5.dev-r698,==dev, Werkzeug >=
0.1.dev-r3831,==dev, SQLAlchemy >= 0.4.0.dev-r3203,==dev, AuthKit >= 0.3.0pre5
...done.

XXX: Use ". sandbox" to activate the development sandbox

Activating a `workingenv` from Python

It can, under some circumstances, be useful to be able to activate a workingenv from Python. Here's a quick function to achieve that: 

import sys
import os

def activate_workingenv(root):
    """Make modules in a self-contained workingenv available."""

    # Add ./bin directory to path.
    bin_dir = os.path.join(root, './bin')
    try:
        os.environ['PATH'] = os.path.pathsep.join([bin_dir, os.environ['PATH']]) 
    except KeyError:
        os.environ['PATH'] = bin_dir

    # Add ./lib to linker path
    lib_dir = os.path.join(root, './lib')
    try:
        os.environ['LD_LIBRARY_PATH'] = \
            os.path.pathsep.join([lib_dir, os.environ['LD_LIBRARY_PATH']])
    except KeyError:
        os.environ['LD_LIBRARY_PATH'] = lib_dir

    # Find the workingenv Python package root
    python_version = '.'.join(map(str, sys.version_info[:2]))
    package_root = os.path.join(root, './lib/python' + python_version)

    # Find and insert setuptools into sys.path
    sys.path.insert(0, package_root)
    real_setuptools = open(os.path.join(package_root,
                                        'setuptools.pth')).read().strip()
    sys.path.insert(0, os.path.join(package_root, real_setuptools))

    # Load all distributions into the working set.
    from pkg_resources import working_set, Environment

    env = Environment(root)
    env.scan()

    distributions, errors = working_set.find_plugins(env)
    for dist in distributions:
        working_set.add(dist)

    return distributions, errors

It's UNIX-centric due to the use of LD_LIBRARY_PATH, but if you're not using shared libraries it's not really necessary anyway.

Use it like so: 

from activate_workingenv import activate_workingenv
activate_workingenv('./wenv')
import some_module_from_the_workingenv

Automatically activating `workingenv.py` environments on directory change

workingenv.py is a very useful tool for Python development. Quoting from its home page:

This tool creates an environment that is isolated from the rest of the Python installation, eliminating site-packages and any other source of modules, so that only the modules (and versions) you install into the environment will be available. This allows for isolated and controlled environments, as well as reproduceability.

To create, activate and deactivate an environment: 

$ workingenv foo
$ . foo/bin/activate
(foo)$ deactivate
$

This is great, but what's even more so is using On Dir with it.

I need to work on multiple versions of Trac (stable, trunk, branches, etc.) at the same time, I have each version in its own directory beneath ~/projects/trac. Each Trac instance is completely self contained, so installing plugins in one will not affect the others.

So I use the following On Dir config to activate the workingenvs as I cd into each Trac directory. 

enter ~/projects/trac/([^/]*)
    declare -F deactivate > /dev/null && deactivate
    activate=../env/$1/bin/activate
    test -r $activate && . $activate

leave ~/projects/trac
    declare -F deactivate > /dev/null && deactivate

Here's an example of me switching between environments. The last environment remains active until I leave the main Trac directory. 

[aat@stalactite:~]cd projects/trac/trunk
(trunk)[aat@stalactite:~/projects/trac/trunk]cd ..
(trunk)[aat@stalactite:~/projects/trac]workingenv --site-packages ../env/stable
Updating working environment in /home/aat/projects/trac/env/stable
Installing local setuptools.................done.
(trunk)[aat@stalactite:~/projects/trac]cd stable/
(stable)[aat@stalactite:~/projects/trac/stable]cd ..
(stable)[aat@stalactite:~/projects/trac]cd trunk
(trunk)[aat@stalactite:~/projects/trac/trunk]cd ../..
[aat@stalactite:~/projects]

Of course, this can be extended to any project that needs its own independent Python environment, not just Trac.