Mailman 3 February 2011 - Python-ideas

@classproperty, @abc.abstractclasspropery, etc.
by K. Richard Pixley Dec. 16, 2020

Dec. 16, 2020

There's a whole matrix of these and I'm wondering why the matrix is currently sparse rather than implementing them all. Or rather, why we can't stack them as: class foo(object): @classmethod @property def bar(cls, ...): ... Essentially the permutation are, I think: {'unadorned'|abc.abstract}{'normal'|static|class}{method|property|non-callable attribute}. concreteness implicit first arg type name comments {unadorned} {unadorned} method def foo(): exists now {unadorned} {unadorned} property @property exists now {unadorned} {unadorned} non-callable attribute x = 2 exists now {unadorned} static method @staticmethod exists now {unadorned} static property @staticproperty proposing {unadorned} static non-callable attribute {degenerate case - variables don't have arguments} unnecessary {unadorned} class method @classmethod exists now {unadorned} class property @classproperty or @classmethod;@property proposing {unadorned} class non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract {unadorned} method @abc.abstractmethod exists now abc.abstract {unadorned} property @abc.abstractproperty exists now abc.abstract {unadorned} non-callable attribute @abc.abstractattribute or @abc.abstract;@attribute proposing abc.abstract static method @abc.abstractstaticmethod exists now abc.abstract static property @abc.staticproperty proposing abc.abstract static non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract class method @abc.abstractclassmethod exists now abc.abstract class property @abc.abstractclassproperty proposing abc.abstract class non-callable attribute {degenerate case - variables don't have arguments} unnecessary I think the meanings of the new ones are pretty straightforward, but in case they are not... @staticproperty - like @property only without an implicit first argument. Allows the property to be called directly from the class without requiring a throw-away instance. @classproperty - like @property, only the implicit first argument to the method is the class. Allows the property to be called directly from the class without requiring a throw-away instance. @abc.abstractattribute - a simple, non-callable variable that must be overridden in subclasses @abc.abstractstaticproperty - like @abc.abstractproperty only for @staticproperty @abc.abstractclassproperty - like @abc.abstractproperty only for @classproperty --rich

10 15

i18n and Python tracebacks
by Andre Roberge Oct. 28, 2012

Oct. 28, 2012

I think it would be a good idea if Python tracebacks could be translated into languages other than English - and it would set a good example. For example, using French as my default local language, instead of >>> 1/0 Traceback (most recent call last): File "<stdin>", line 1, in <module> ZeroDivisionError: integer division or modulo by zero I might get something like >>> 1/0 Suivi d'erreur (appel le plus récent en dernier) : Fichier "<stdin>", à la ligne 1, dans <module> ZeroDivisionError: division entière ou modulo par zéro André

9 15

Cofunctions PEP - Revision 4
by Greg Ewing Oct. 13, 2012

Oct. 13, 2012

Here's an updated version of the PEP reflecting my recent suggestions on how to eliminate 'codef'. PEP: XXX Title: Cofunctions Version: $Revision$ Last-Modified: $Date$ Author: Gregory Ewing <greg.ewing(a)canterbury.ac.nz> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 13-Feb-2009 Python-Version: 3.x Post-History: Abstract ======== A syntax is proposed for defining and calling a special type of generator called a 'cofunction'. It is designed to provide a streamlined way of writing generator-based coroutines, and allow the early detection of certain kinds of error that are easily made when writing such code, which otherwise tend to cause hard-to-diagnose symptoms. This proposal builds on the 'yield from' mechanism described in PEP 380, and describes some of the semantics of cofunctions in terms of it. However, it would be possible to define and implement cofunctions independently of PEP 380 if so desired. Specification ============= Cofunction definitions ---------------------- A cofunction is a special kind of generator, distinguished by the presence of the keyword ``cocall`` (defined below) at least once in its body. It may also contain ``yield`` and/or ``yield from`` expressions, which behave as they do in other generators. From the outside, the distinguishing feature of a cofunction is that it cannot be called the same way as an ordinary function. An exception is raised if an ordinary call to a cofunction is attempted. Cocalls ------- Calls from one cofunction to another are made by marking the call with a new keyword ``cocall``. The expression :: cocall f(*args, **kwds) is evaluated by first checking whether the object ``f`` implements a ``__cocall__`` method. If it does, the cocall expression is equivalent to :: yield from f.__cocall__(*args, **kwds) except that the object returned by __cocall__ is expected to be an iterator, so the step of calling iter() on it is skipped. If ``f`` does not have a ``__cocall__`` method, or the ``__cocall__`` method returns ``NotImplemented``, then the cocall expression is treated as an ordinary call, and the ``__call__`` method of ``f`` is invoked. Objects which implement __cocall__ are expected to return an object obeying the iterator protocol. Cofunctions respond to __cocall__ the same way as ordinary generator functions respond to __call__, i.e. by returning a generator-iterator. Certain objects that wrap other callable objects, notably bound methods, will be given __cocall__ implementations that delegate to the underlying object. Grammar ------- The full syntax of a cocall expression is described by the following grammar lines: :: atom: cocall | <existing alternatives for atom> cocall: 'cocall' atom cotrailer* '(' [arglist] ')' cotrailer: '[' subscriptlist ']' | '.' NAME Note that this syntax allows cocalls to methods and elements of sequences or mappings to be expressed naturally. For example, the following are valid: :: y = cocall self.foo(x) y = cocall funcdict[key](x) y = cocall a.b.c[i].d(x) Also note that the final calling parentheses are mandatory, so that for example the following is invalid syntax: :: y = cocall f # INVALID New builtins, attributes and C API functions -------------------------------------------- To facilitate interfacing cofunctions with non-coroutine code, there will be a built-in function ``costart`` whose definition is equivalent to :: def costart(obj, *args, **kwds): try: m = obj.__cocall__ except AttributeError: result = NotImplemented else: result = m(*args, **kwds) if result is NotImplemented: raise TypeError("Object does not support cocall") return result There will also be a corresponding C API function :: PyObject *PyObject_CoCall(PyObject *obj, PyObject *args, PyObject *kwds) It is left unspecified for now whether a cofunction is a distinct type of object or, like a generator function, is simply a specially-marked function instance. If the latter, a read-only boolean attribute ``__iscofunction__`` should be provided to allow testing whether a given function object is a cofunction. Motivation and Rationale ======================== The ``yield from`` syntax is reasonably self-explanatory when used for the purpose of delegating part of the work of a generator to another function. It can also be used to good effect in the implementation of generator-based coroutines, but it reads somewhat awkwardly when used for that purpose, and tends to obscure the true intent of the code. Furthermore, using generators as coroutines is somewhat error-prone. If one forgets to use ``yield from`` when it should have been used, or uses it when it shouldn't have, the symptoms that result can be extremely obscure and confusing. Finally, sometimes there is a need for a function to be a coroutine even though it does not yield anything, and in these cases it is necessary to resort to kludges such as ``if 0: yield`` to force it to be a generator. The ``cocall`` construct address the first issue by making the syntax directly reflect the intent, that is, that the function being called forms part of a coroutine. The second issue is addressed by making it impossible to mix coroutine and non-coroutine code in ways that don't make sense. If the rules are violated, an exception is raised that points out exactly what and where the problem is. Lastly, the need for dummy yields is eliminated by making it possible for a cofunction to call both cofunctions and ordinary functions with the same syntax, so that an ordinary function can be used in place of a cofunction that yields zero times. Record of Discussion ==================== An earlier version of this proposal required a special keyword ``codef`` to be used in place of ``def`` when defining a cofunction, and disallowed calling an ordinary function using ``cocall``. However, it became evident that these features were not necessary, and the ``codef`` keyword was dropped in the interests of minimising the number of new keywords required. The use of a decorator instead of ``codef`` was also suggested, but the current proposal makes this unnecessary as well. It has been questioned whether some combination of decorators and functions could be used instead of a dedicated ``cocall`` syntax. While this might be possible, to achieve equivalent error-detecting power it would be necessary to write cofunction calls as something like :: yield from cocall(f)(args) making them even more verbose and inelegant than an unadorned ``yield from``. It is also not clear whether it is possible to achieve all of the benefits of the cocall syntax using this kind of approach. Prototype Implementation ======================== An implementation of an earlier version of this proposal in the form of patches to Python 3.1.2 can be found here: http://www.cosc.canterbury.ac.nz/greg.ewing/python/generators/cofunctions.h… If this version of the proposal is received favourably, the implementation will be updated to match. Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

11 29

Bring back callable()
by Antoine Pitrou March 7, 2011

March 7, 2011

Hello, Python 3 has removed callable() under the justification that's it's not very useful and duck typing (EAFP) should be used instead. However, it has since been felt by many people that it was an annoying loss; there are situations where you truly want to know whether something is a callable without actually calling it (for example when writing sophisticated decorators, or simply when you want to inform the user of an API misuse). The substitute of writing `isinstance(x, collections.Callable)` is not good, 1) because it's wordier 2) because collections is really not an intuitive place where to look for a Callable ABC. So, I would advocate bringing back the callable() builtin, which was easy to use, helpful and semantically sane. Regards Antoine.

17 21

str.split() oddness
by Mart Sõmermaa March 7, 2011

March 7, 2011

IMHO, x.join(a).split(x) should be "idempotent" in regard to a. >>> foo = ['a', 'b', 'c'] >>> assert '|'.join(foo).split('|') == foo >>> foo = ['a'] >>> assert '|'.join(foo).split('|') == foo >>> foo = [] >>> assert ' '.join(foo).split() == foo And now the odd exception to the rule: >>> assert '|'.join(foo).split('|') == foo Traceback (most recent call last): File "<stdin>", line 1, in <module> AssertionError That forces one to write special case code when using custom separators. Consider: # clean baz = dict(chunk.split('=') for chunk in baz.split()) # ugly baz = (dict(chunk.split('=') for chunk in baz.split("|")) if baz else {}) Our younger cousin Ruby has no such idiosyncrasies: >> foo = [] >> foo.join('|').split('|') == foo => true What is the reason for that oddity? Can we amend it? Best regards, Mart Sõmermaa

9 18

str.split with multiple individual split characters
by Andy Buckley March 2, 2011

March 2, 2011

Here's another str.split() suggestion, this time an extension (Pythonic, I think) rather than a change of semantics. There are cases where, especially in handling user input, I'd like to be able to treat any of a series of possible delimiters as acceptable. Let's say that I want commas, underscores, and hyphens to all be treated as delimiters (as I did in some code I was writing today). I guessed, based on some other Python std lib behaviours, that this might work: usertokens = userstr.split([",", "_", "-"]) It doesn't work though, since the sep argument *has* to be a string. I think it would be nice for an extension like this to be supported, although I would guess a 90% probability of there being an insightful reason for why it's not such a great idea after all* ;-) Unlike many extensions, I don't think that the general solution to this is *very* quick and idiomatic in current Python. As for a compelling use-case... well, I'm very sympathetic to not adding functions for which there is no demand (I forget the relevant acronym) but this is a case where I suddenly found that I did have that problem to solve and that Python didn't have the nice built-in answer that I semi-expected it to. Extension of single arguments to iterables of them is quite a common Python design feature: one of those things where you think "ooh, this really is a nice, consistent, powerful language" when you find it. So I hope that this suggestion finds some favour. Best wishes, Andy [*] Such as "how do you distinguish between a string, which is iterable over its characters, and a list/tuple/blah of individual strings?" Well, that doesn't strike me as too big a technical issue, but maybe it is.

18 35

class ModuleNotFoundError(ImportError)
by cool-RR March 1, 2011

March 1, 2011

There are many programs out there, including Django, that "carefully import" a module by doing: try: import simplejson except ImportError: import whatever_instead as simplejson # or whatever This is problematic because sometimes an `ImportError` is raised not because the module is missing, but because there's some error in the module, or because the module raises an `ImportError` itself. Then the exception gets totally swallowed, resulting in delightful debugging sessions. What do you think about having an exception `ModuleNotFoundError` which is a subclass of `ImportError`? Then people could `except ModuleNotFoundError` and be sure that it was caused by a module not existing. This will be a much better way of "carefully importing" a module. Would this be backwards-compatible? Ram.

5 7

Coercing str.join() argument elements to str
by Dirkjan Ochtman Feb. 28, 2011

Feb. 28, 2011

I just saw someone mention this on Twitter, and I know that I've been bitten by it before. Seems like it wouldn't break any existing code... The worst that could happen is that someone gets nonsensical strings in new code instead of an exception. >>> ', '.join(range(5)) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: sequence item 0: expected string, int found >>> ', '.join(str(i) for i in range(5)) '0, 1, 2, 3, 4' Has this been discussed before? If not, what would be reasons not to do this? Cheers, Dirkjan

5 4

Fw: New pattern-matching library (was: str.split with multiple individual split characters)
by Mike Meyer Feb. 28, 2011

Feb. 28, 2011

I accidental dropped the list from my reply to his comment. I'm forwarding that and his reply (with permission). <mike Begin forwarded message: Date: Mon, 28 Feb 2011 09:59:48 -0800 From: Guido van Rossum <guido(a)python.org> To: Mike Meyer <mwm(a)mired.org> Subject: Re: [Python-ideas] New pattern-matching library (was: str.split with multiple individual split characters) On Mon, Feb 28, 2011 at 9:40 AM, Mike Meyer <mwm(a)mired.org> wrote: > On Mon, 28 Feb 2011 09:15:36 -0800 > Guido van Rossum <guido(a)python.org> wrote: >> That doesn't mean you shouldn't try -- I've been wrong before. But >> maybe instead of striving for stdlib inclusion (which these days is >> pretty much only accessible for proven successful 3rd party >> libraries), you should try to create a new 3rd party pattern matching >> library. While admittedly this gives it a disadvantage to the re >> module, I really don't think we should experiment in the stdlib, since >> the release cycle and backwards compatibility requirements make the >> necessary experimentation too cumbersome. > > How about if it's bundled with new language functionality? Take Icon's > failure/backtrack/alternation features, add them to Python, then put > together a string parsing facility that leverages that? You'd have to write a thorough PEP, do a reference implementation, and go through a long process of discussion before getting it accepted. It can be done, but nothing of the scale has been done in a long time (most syntax PEPs these days are mere tweaks in comparison). Even before writing the PEP you should probably start with a serious brainstorm. Are you up for it? I support you (or anyone) doing it, and may possibly comment in various stages on ideas or designs, but I can't pull this cart myself, nor can I guarantee success. Maybe you can find a few other like-minded folks to help with the various stages of the design? How much deep knowledge about Python's syntax and semantics do you have? PS. Did you intentionally drop python-ideas from the CC header? -- --Guido van Rossum (python.org/~guido) -- Mike Meyer <mwm(a)mired.org> http://www.mired.org/consulting.html Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org

1 0

A couple of with statement ideas
by Nick Coghlan Feb. 27, 2011

Feb. 27, 2011

I've been considering which broad concerns I want to focus on for 3.3, and have come to the conclusion that I'm going to have plenty to keep me busy between helping to smooth over some of the rough edges left over from the PEP 3118 implementation, as well as championing the module aliasing concept I first brought up a few weeks ago. That means there are a couple of with statement ideas that I still like, but almost certainly won't have the time to champion myself. So, I'm lobbing them over the fence in a half-baked form. If anyone feels strongly enough about them to get them into a PEP-worthy state, go right ahead (that's the whole point of this message). If nobody else cares enough to pick them up... well, that will simply justify my decision that they weren't important enough to worry about. Idea 1: Implicit context managers PEP 343 included the idea of a "__context__" method that permitted the creation of objects with an implicit associated context manager. It was eventually dropped because we couldn't find a good way to explain it to users at the time. Instead, you see things like "decimal.localcontext()" and various other objects that need to be called every time you want to use them as a context manager. If you want an object to have a "native" context manager, you have to either write __enter__ and __exit__ manually, or use a mixin or class decorator that adapts the __enter__/__exit__ interface to a single method (e.g. the ContextManager class and manage_context method from GarlicSim that Ram Rachum posted about here some time ago). Now that everyone has had time to get used to the way context managers work, I believe the concept may usefully make a return using the "implicit context manager" terminology. While the "__context__" name has since been claimed by PEP 3134, the alternative name used in PEP 346 ("__with__") is still a possibility. The idea of bringing back this concept would be to allow an object to implement *either* a __with__ method that returns a context manager, or else implement __enter__/__exit__ directly. The behaviour would be similar to the relationship between iterators and iterables, except that a missing "__with__" implementation would imply the "return self" semantics that iterators must implement explicitly. Anyone picking up this idea should be prepared for a lot of pushback, as the last few years have shown us that the with statement is perfectly usable without this feature, and the GarlicSim example shows that this is already feasible using existing mechanisms (even I am at best lukewarm on the concept). Idea 2: Son of PEP 377 (letting context managers skip the body of the with statement) The niggles that lead me to write PEP 377 still bug me. There are some nested with statements that are perfectly valid as inline code but will throw RuntimeError if you make them into a single context manager via contextlib.contextmanager. That PEP was rightly rejected as having too great an impact on "normal" code for something that is a comparatively exotic corner case. As I noted in issue 5251, I've since thought of an alternative, lower impact solution that may prove more acceptable to Guido and others: an optional "__entered__" method for context managers that, if present, would be executed *inside* the scope of the try/finally block before the body of the with statement itself started executing. Any exceptions raised in that method would be passed to the __exit__ method, thus increasing the flexibility of the context management protocol, while having minimal impact Most significantly, contextlib.GeneratorContextManager could be adjusted to make use of this feature to correctly skip the body of the with statement when the internal generator doesn't yield (attempt to invoke __enter__ a second time would still trigger RuntimeError, though). The translation of the problematic nested with statements into a single generator based context manager would then work correctly, functioning in the same way as the equivalent inline code. I really like this idea, since it provides a genuinely new capability to the context management protocol in a relatively low impact manner. However, as noted in the intro, there are other, more immediately practical, questions I want to deal with first, so who knows if or when I'll be able to devote a significant amount of time to this one. So, if anyone is feeling particularly keen to champion a PEP and dig into the internals of contextlib and CPython's with statement implementation, there's a couple of ideas for you to mull over :) Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

8 12