Showing posts with label python. Show all posts
Showing posts with label python. Show all posts

Saturday, 20 March 2010

A Crime Against Nature

Every so often, while writing Python, I've found myself wishing I could easily dispatch method calls according to the type of the arguments. The urge usually passes quickly, but... oh, the hell with it, there's no point trying to justify what I've done. Just look:

>>> import bondage
>>>
>>> class C(object):
... @bondage.discipline(int)
... def foo(self, arg):
... print 'int'
... @foo.discipline(str)
... def foo(self, arg):
... print 'str'
... @foo.discipline(int, str, int)
... def foo(self, arg1, arg2, arg3):
... print 'int, str, int'
...
>>> c = C()
>>> c.foo(1)
int
>>> c.foo('a')
str
>>> c.foo(1, 'a', 1)
int, str, int
>>> c.foo([])
Traceback (most recent call last):
File "", line 1, in <module>
File "bondage.py", line 18, in <lambda>
return lambda *args: self._dispatch(obj, *args)
File "bondage.py", line 22, in _dispatch
return self._argspecs[argspec](obj, *args)
KeyError: (<type 'list'>,)
>>>

I'd like to make it clear that there is absolutely no excuse for perpetrating this sort of insanity, ever. With that said, here's how I did it:

class discipline(object):

def __init__(self, *argspec):
self._argspecs = {}
self.discipline(*argspec)

def discipline(self, *argspec):
self._argspec = argspec
return self

def __call__(self, f):
self._argspecs[self._argspec] = f
return self

def __get__(self, obj, objtype=None):
return lambda *args: self._dispatch(obj, *args)

def _dispatch(self, obj, *args):
argspec = tuple(map(type, args))
return self._argspecs[argspec](obj, *args)

Obviously it's a stupid implementation, and if you wanted to do this properly you'd have to pay attention to subtypes, and do something clever with numeric types, and... oh, God, what am I saying?

Enough!

If you really want to do this "properly", use some other language where it's already built in, and begone.

Tuesday, 2 March 2010

The Joy of Self

I have a distant and fuzzy memory, from back in the day... when I was but a wee slip of a lad, sallying forth to do battle with million-line C++ monstrosities (and just barely escaping with my sanity intact purple monkey dishwasher), I came upon a Path class. It was probably called CPath: classes were new and shiny, so far as any of us knew at the time, and absolutely deserved a prefix to underline their special status. Look, ma, I'm programming Object Orientedly!

And, yeah, it was horrible. Big, clunky, confusing... I'd like to say that the blistering speed made up for it all, but that would be entirely untrue. It was nasty.

As a result of this, I had never since felt the slightest urge to create a Path class of my own... until today. I was trying to get an overgrown build script under control, and - despite my scars - it suddenly seemed like a good idea.

So I had a go.

And... well, this sort of thing is why I love Python, and is also the clearest illustration I've yet seen of why explicit self is a Good Thing. The following code is, as usual, hacked up from memory and may therefore contain hilariously deadly bugs; caveat lector.

import os, shutil

class Path(str):

def __new__(cls, path):
abs_ = os.path.abspath(path)
norm = os.path.normpath(abs_)
return str.__new__(cls, norm)

exists = property(os.path.exists)
isfile = property(os.path.isfile)
isdir = property(os.path.isdir)
# etc...

move = shutil.move
listdir = os.listdir
# etc...

def __getattr__(self, name):
return self.join(name)

def join(*args):
return Path(os.path.join(*args))

def delete(self):
if self.isdir:
shutil.rmtree(self)
else:
os.remove(self)
# etc...

Now, IMO, this was a massive win: I made the client code a lot less verbose, and hence clearer, and I did most of the work by trivially subclassing a builtin type and dropping in a bunch of standard library functions as methods. The real one has many more bells and whistles (in fact, I think I got a bit carried away) but hopefully you get the idea.

The crucial point is that I couldn't have done it so neatly without explicit self; also, amusingly, most of the explicit selfs in this class are in fact implicit. So there.

Thursday, 14 January 2010

Where's a plumber when you need one?

Assertion: On Win32, there's no point bothering with subprocess.PIPE -- just use tempfile.TemporaryFile instead.

Context: You create a Popen(cmd, stdout=PIPE, stderr=PIPE), and you let it run (with a timeout); sometimes it completes successfully, which is cool, and sometimes it doesn't, in which case you read stdout and stderr and try to figure out what went wrong. This is all fine and dandy until one day you add a *little* bit more logging to the tool you're calling, and it suddenly wedges forever.

Explanation: Of course, this is because you've filled up some buffer, which you should have been periodically emptying. However, you can't just select() and read one byte at a time, because select doesn't work with pipes on Win32; you can't just read() what's there, because it blocks and stops the timeout from working; you don't want to spin off another thread to do your reading, because that involves tedious extra code and feels like killing a fly with a sledgehammer; and you don't want to screw around with readline() because that also involves tedious bookkeeping and extra code.

Solution: So, just do the following (warning, coded from memory):


from subprocess import Popen
from tempfile import TemporaryFile
from time import time, sleep

def assert_runs(cmd, timeout=10):
out = TemporaryFile()
err = TemporaryFile()
end_time = time() + timeout
process = Popen(cmd, stdout=out, stderr=err)
while process.poll() is None:
sleep(0.1)
if time() > end_time:
process.terminate()
if process.returncode != 0:
raise AssertionError('%s FAILED (%s)\nstdout:\n%s\nstderr:\n%s' % (
cmd, process.returncode, out.read(), err.read()))


Indeed, it's icky to fill up your hard disk rather than some internal buffer, but you can get a lot more done before you run out of HD space. Now, this surely feels nasty, but IMO it's slightly less nasty (or, at least, less code) than anything else I've tried. Hopefully you, gentle reader, have an infinitely superior solution that you will detail in the comments. Surprise me!

Please note that "Don't use Windows, har har", and variants thereof, fail to qualify as "surprising" ;-).

Thursday, 24 December 2009

Spare batteries for IronPython

As we all know, Python comes with batteries included in the form of a rich standard library; and, on top of this, there are many awesome and liberally-licensed packages just an easy_install away.

IronPython, of course, includes *most* of the CPython standard library, but if you're a heavy user you might have noticed a few minor holes: in the course of my work on Ironclad, I certainly have. Happily for you I can vaguely remember what I did in the course of bodging them closed with cow manure and chewing gum; here then, for your edification and delectation, is my personal recipe for a delicious reduced-hassle IronPython install, with access to the best and brightest offered by CPython, on win32.

  • Install IronPython 2.6.

  • Download Jeff Hardy's zlib for IronPython and copy IronPython.Zlib.dll into IronPython's DLLs subdirectory (create it if it doesn't exist).

  • Download Jeff Hardy's subprocess.py for IronPython and copy it into IronPython's site-packages subdirectory.

  • Download Ironclad, and copy the ironclad package into IronPython's site-packages subdirectory. Yeah, maybe I'll sort out an installer one day, but don't hold your breath.

  • Install CPython 2.6.

  • Add CPython's Dlls subdirectory to your IRONPYTHONPATH environment variable.

  • Copy csv.py, gzip.py, and the sqlite3 directory from CPython's Lib subdirectory to IronPython's site-packages subdirectory.

  • Copy xml/sax/expatreader.py from CPython's Lib subdirectory to the corresponding location in IronPython's Lib subdirectory.

  • Download FePy's pyexpat.py, copy it to IronPython's Lib/xml/parsers subdirectory, and rename it to expat.py.

  • Download and install NumPy 1.3.0 and SciPy 0.7.1 for CPython, and copy them from CPython's site-packages subdirectory to IronPython's.


...and you're done. Start your ipy sessions with a snappy 'import ironclad', and enjoy.

Incidentally, you could just add CPython's site-packages to your IRONPYTHONPATH, and then you wouldn't have to copy extra packages over; the reason I don't do that is because having matplotlib on your path currently breaks scipy under ironclad -- can't remember exactly why -- and it's nice to have matplotlib installed for CPython.

Oh, and let me know if I've made any mistakes above: I just hacked this post together from slightly aged notes, and I'm too lazy to tear down and rebuild my environment to check that every detail is perfect.