reorder vars

ddd2f011 removed this.

.gitignore

@@ -1,5 +1,8 @@
 *.pyc
 *.pyo
-build/*
+.DS_Store
-dist/*
+build
+dist
+tmp
 MANIFEST
+sendapatch.se

README.rst

@@ -2,7 +2,7 @@
 
 __ http://memcached.org/
 
-It builds on the famous `libmemcached`__ C client from TangetOrg__, notable for
+It builds on the famous `libmemcached`__ C client from TangentOrg__, notable for
 its speed and flexibility.
 
 __ http://tangent.org/552/libmemcached.html
@@ -11,62 +11,10 @@ __ http://tangent.org/
 `libmemcached` must be installed separately, and be available to the compiler
 and linker.
 
-Basic usage is that of `python-memcached`__, like so::
+For installation instructions, usage notes and reference documentation, see
+pylibmc__'s home at http://sendapatch.se/projects/pylibmc/.
 
-    >>> import pylibmc
+__ http://sendapatch.se/projects/pylibmc/
-    >>> mc = pylibmc.Client(["127.0.0.1:11211"])
-    >>> mc.set("foo", "Hello world!")
-    True
-    >>> mc.get("foo")
-    'Hello world!'
-
-__ http://www.tummy.com/Community/software/python-memcached/
-
-There's also support for some other features not present in other Python
-libraries, like the binary protocol::
-
-    >>> mc = pylibmc.Client(["127.0.0.1"], binary=True)
-
-Behaviors
-=========
-
-`libmemcached` has ways of telling it how to behave. You'll have to refer to
-its documentation on what the different behaviors do.
-
-To change behaviors, quite simply::
-
-    >>> mc.behaviors["hash"] = "fnv1a_32"
-
-For a list of the defined behavior key names, see what the keys of a client is.
-For example::
-
-    >>> mc.behaviors.keys()  # doctest: +NORMALIZE_WHITESPACE
-    ['hash', 'connect timeout', 'cache lookups', 'buffer requests',
-     'verify key', 'support cas', 'poll timeout', 'no block', 'tcp nodelay',
-     'distribution', 'sort hosts']
-
-The ``hash`` and ``distribution`` keys are mapped by the Python module to constant
-integer values used by `libmemcached`. See ``pylibmc.hashers`` and
-``pylibmc.distributions``.
-
-Pooling
-=======
-
-In multithreaded environments, accessing the same memcached client object is
-both unsafe and counter-productive in terms of performance. `libmemcached`'s
-take on this is to introduce pooling on C level, which is correspondingly
-mapped to pooling on Python level in `pylibmc`::
-
-    >>> mc = pylibmc.Client(["127.0.0.1"])
-    >>> pool = pylibmc.ThreadMappedPool(mc)
-    >>> # (in a thread...)
-    >>> with pool.reserve() as mc:
-    ...     mc.set("hello", "world")
-
-For more information on pooling, see `my two`__ `long posts`__ about it.
-
-__ http://lericson.blogg.se/code/2009/september/draft-sept-20-2009.html
-__ http://lericson.blogg.se/code/2009/september/pooling-with-pylibmc-pt-2.html
 
 Comparison to other libraries
 =============================
@@ -78,6 +26,23 @@ Why use `pylibmc`? Because it's fast.
 __ http://lericson.blogg.se/code/2008/november/pylibmc-051.html
 __ http://amix.dk/blog/viewEntry/19471
 
+Installation
+============
+
+Building needs libmemcached and optionally zlib, the path to which can be
+specified using command-line options to ``setup.py``
+
+``--with-libmemcached=DIR``
+    Build against libmemcached in DIR
+``--with-zlib=DIR``
+    Build against zlib in DIR
+``--without-zlib``
+    Disable zlib (disables compression)
+
+So for example, if one were to use MacPorts to install libmemcached, your
+libmemcached would end up in ``/opt/local``, hence
+``--with-libmemcached=/opt/local``.
+
 IRC
 ===
 
@@ -86,55 +51,76 @@ IRC
 Change Log
 ==========
 
+New in version 1.1
+------------------
+
+- Removed deprecated space-based behavior names.
+- Acquire and release the GIL properly, thanks ketralnis__
+- Add support for ``libmemcached 0.40``
+- Included a more useful command-line interface
+- Fixed handling of NUL-byte keys in ``get_multi`` in binary protocol
+- Fixed some valgrind-reported memory warnings
+- Fixed bogus usage of time argument for delete.
+- 1.1.1: Fixed tests under Python 2.5
+
+__ http://www.ketralnis.com/
+
+New in version 1.0
+------------------
+
+- Lots of documentation fixes and other nice things like that.
+- Nailed what appears to be the last outstanding memory leak.
+- Explicitly require libmemcached 0.32 or newer.
+
 New in version 0.9
 ------------------
 
- - Added a ``get_stats`` method, which behaves exactly like
+- Added a ``get_stats`` method, which behaves exactly like
-   `python-memcached`'s equivalent.
+  `python-memcached`'s equivalent.
- - Gives the empty string for empty memcached values like `python-memcached`
+- Gives the empty string for empty memcached values like `python-memcached`
-   does.
+  does.
- - Added exceptions for most `libmemcached` return codes.
+- Added exceptions for most `libmemcached` return codes.
- - Fixed an issue with ``Client.behaviors.update``.
+- Fixed an issue with ``Client.behaviors.update``.
 
 New in version 0.8
 ------------------
 
- - Pooling helpers are now available. See ``pooling.rst`` in the distribution.
+- Pooling helpers are now available. See ``pooling.rst`` in the distribution.
- - The binary protocol is now properly exposed, simply pass ``binary=True`` to
+- The binary protocol is now properly exposed, simply pass ``binary=True`` to
-   the constructor and there you go.
+  the constructor and there you go.
- - Call signatures now match `libmemcached` 0.32, but should work with older
+- Call signatures now match `libmemcached` 0.32, but should work with older
-   versions. Remember to run the tests!
+  versions. Remember to run the tests!
 
 New in version 0.7
 ------------------
 
- - Restructured some of the code, which should yield better performance (if not
+- Restructured some of the code, which should yield better performance (if not
-   for that, it reads better.)
+  for that, it reads better.)
- - Fixed some memory leaks.
+- Fixed some memory leaks.
- - Integrated changes from `amix.dk`, which should make pylibmc work under
+- Integrated changes from `amix.dk`, which should make pylibmc work under
-   Snow Leopard.
+  Snow Leopard.
- - Add support for the boolean datatype.
+- Add support for the boolean datatype.
- - Improved test-runner -- now tests ``build/lib.*/_pylibmc.so`` if available,
+- Improved test-runner -- now tests ``build/lib.*/_pylibmc.so`` if available,
-   and reports some version information.
+  and reports some version information.
- - Support for x86_64 should now work completely.
+- Support for x86_64 should now work completely.
- - Builds with Python 2.4, tests run fine, but not officially supported.
+- Builds with Python 2.4, tests run fine, but not officially supported.
- - Fixed critical bugs in behavior manipulation.
+- Fixed critical bugs in behavior manipulation.
 
 New in version 0.6
 ------------------
 
- - Added compatibility with `libmemcached` 0.26, WRT error return codes.
+- Added compatibility with `libmemcached` 0.26, WRT error return codes.
- - Added `flush_all` and `disconnect_all` methods.
+- Added `flush_all` and `disconnect_all` methods.
- - Now using the latest pickling protocol.
+- Now using the latest pickling protocol.
 
 New in version 0.5
 ------------------
 
- - Fixed lots of memory leaks, and added support for `libmemcached` 0.23.
+- Fixed lots of memory leaks, and added support for `libmemcached` 0.23.
- - Also made the code tighter in terms of compiler pedantics.
+- Also made the code tighter in terms of compiler pedantics.
 
 New in version 0.4
 ------------------
 
- - Renamed the C module to `_pylibmc`, and added lots of `libmemcached` constants
+- Renamed the C module to `_pylibmc`, and added lots of `libmemcached` constants
-   to it, as well as implemented behaviors.
+  to it, as well as implemented behaviors.

_pylibmcmodule.h

@@ -68,11 +68,52 @@ typedef ssize_t Py_ssize_t;
 #define PYLIBMC_FLAG_ZLIB    (1 << 3)
 /* }}} */
 
-#define PYLIBMC_INC  (1 << 0)
-#define PYLIBMC_DEC  (1 << 1)
-
 typedef memcached_return (*_PylibMC_SetCommand)(memcached_st *, const char *,
         size_t, const char *, size_t, time_t, uint32_t);
+typedef memcached_return (*_PylibMC_IncrCommand)(memcached_st *,
+        const char *, size_t, unsigned int, uint64_t*);
+
+typedef struct {
+  char key[MEMCACHED_MAX_KEY];
+  size_t key_len;
+  char *value;
+  size_t value_len;
+  uint32_t flags;
+} pylibmc_mget_result;
+
+typedef struct {
+  char *key;
+  Py_ssize_t key_len;
+  char* value;
+  Py_ssize_t value_len;
+  time_t time;
+  uint32_t flags;
+
+  /* the objects that must be freed after the mset is executed */
+  PyObject* key_obj;
+  PyObject* prefixed_key_obj;
+  PyObject* value_obj;
+
+  /* the success of executing the mset afterwards */
+  int success;
+
+} pylibmc_mset;
+
+typedef struct {
+  char* key;
+  Py_ssize_t key_len;
+  _PylibMC_IncrCommand incr_func;
+  unsigned int delta;
+  uint64_t result;
+} pylibmc_incr;
+
+typedef struct {
+    PyObject *self;
+    PyObject *retval;
+    memcached_server_st *servers;  /* DEPRECATED */
+    memcached_stat_st *stats;
+    int index;
+} _PylibMC_StatsContext;
 
 /* {{{ Exceptions */
 static PyObject *PylibMCExc_MemcachedError;
@@ -197,16 +238,20 @@ static PylibMC_Client *PylibMC_ClientType_new(PyTypeObject *, PyObject *,
 static void PylibMC_ClientType_dealloc(PylibMC_Client *);
 static int PylibMC_Client_init(PylibMC_Client *, PyObject *, PyObject *);
 static PyObject *PylibMC_Client_get(PylibMC_Client *, PyObject *arg);
+static PyObject *PylibMC_Client_gets(PylibMC_Client *, PyObject *arg);
 static PyObject *PylibMC_Client_set(PylibMC_Client *, PyObject *, PyObject *);
 static PyObject *PylibMC_Client_replace(PylibMC_Client *, PyObject *, PyObject *);
 static PyObject *PylibMC_Client_add(PylibMC_Client *, PyObject *, PyObject *);
 static PyObject *PylibMC_Client_prepend(PylibMC_Client *, PyObject *, PyObject *);
 static PyObject *PylibMC_Client_append(PylibMC_Client *, PyObject *, PyObject *);
+static PyObject *PylibMC_Client_cas(PylibMC_Client *, PyObject *, PyObject *);
 static PyObject *PylibMC_Client_delete(PylibMC_Client *, PyObject *);
 static PyObject *PylibMC_Client_incr(PylibMC_Client *, PyObject *);
 static PyObject *PylibMC_Client_decr(PylibMC_Client *, PyObject *);
+static PyObject *PylibMC_Client_incr_multi(PylibMC_Client*, PyObject*, PyObject*);
 static PyObject *PylibMC_Client_get_multi(PylibMC_Client *, PyObject *, PyObject *);
 static PyObject *PylibMC_Client_set_multi(PylibMC_Client *, PyObject *, PyObject *);
+static PyObject *PylibMC_Client_add_multi(PylibMC_Client *, PyObject *, PyObject *);
 static PyObject *PylibMC_Client_delete_multi(PylibMC_Client *, PyObject *, PyObject *);
 static PyObject *PylibMC_Client_get_behaviors(PylibMC_Client *);
 static PyObject *PylibMC_Client_set_behaviors(PylibMC_Client *, PyObject *);
@@ -220,12 +265,32 @@ static PyObject *_PylibMC_Unpickle(const char *, size_t);
 static PyObject *_PylibMC_Pickle(PyObject *);
 static int _PylibMC_CheckKey(PyObject *);
 static int _PylibMC_CheckKeyStringAndSize(char *, Py_ssize_t);
+static int _PylibMC_SerializeValue(PyObject* key_obj,
+                                   PyObject* key_prefix,
+                                   PyObject* value_obj,
+                                   time_t time,
+                                   pylibmc_mset* serialized);
+static void _PylibMC_FreeMset(pylibmc_mset*);
+static PyObject *_PylibMC_RunSetCommandSingle(PylibMC_Client *self,
+        _PylibMC_SetCommand f, char *fname, PyObject *args, PyObject *kwds);
+static PyObject *_PylibMC_RunSetCommandMulti(PylibMC_Client* self,
+        _PylibMC_SetCommand f, char *fname, PyObject *args, PyObject *kwds);
+static bool _PylibMC_RunSetCommand(PylibMC_Client* self,
+                                   _PylibMC_SetCommand f, char *fname,
+                                   pylibmc_mset* msets, size_t nkeys,
+                                   size_t min_compress, size_t custom_flag);
+static int _PylibMC_Deflate(char* value, size_t value_len,
+                            char** result, size_t *result_len);
+static bool _PylibMC_IncrDecr(PylibMC_Client*, pylibmc_incr*, size_t);
+
 /* }}} */
 
 /* {{{ Type's method table */
 static PyMethodDef PylibMC_ClientType_methods[] = {
     {"get", (PyCFunction)PylibMC_Client_get, METH_O,
         "Retrieve a key from a memcached."},
+    {"gets", (PyCFunction)PylibMC_Client_gets, METH_O,
+        "Retrieve a key and cas_id from a memcached."},
     {"set", (PyCFunction)PylibMC_Client_set, METH_VARARGS|METH_KEYWORDS,
         "Set a key unconditionally."},
     {"replace", (PyCFunction)PylibMC_Client_replace, METH_VARARGS|METH_KEYWORDS,
@@ -236,16 +301,22 @@ static PyMethodDef PylibMC_ClientType_methods[] = {
         "Prepend data to  a key."},
     {"append", (PyCFunction)PylibMC_Client_append, METH_VARARGS|METH_KEYWORDS,
         "Append data to a key."},
+    {"cas", (PyCFunction)PylibMC_Client_cas, METH_VARARGS|METH_KEYWORDS,
+        "Attempt to compare-and-store a key by CAS ID."},
     {"delete", (PyCFunction)PylibMC_Client_delete, METH_VARARGS,
         "Delete a key."},
     {"incr", (PyCFunction)PylibMC_Client_incr, METH_VARARGS,
         "Increment a key by a delta."},
     {"decr", (PyCFunction)PylibMC_Client_decr, METH_VARARGS,
         "Decrement a key by a delta."},
+    {"incr_multi", (PyCFunction)PylibMC_Client_incr_multi, METH_VARARGS|METH_KEYWORDS,
+        "Increment more than one key by a delta."},
     {"get_multi", (PyCFunction)PylibMC_Client_get_multi,
         METH_VARARGS|METH_KEYWORDS, "Get multiple keys at once."},
     {"set_multi", (PyCFunction)PylibMC_Client_set_multi,
         METH_VARARGS|METH_KEYWORDS, "Set multiple keys at once."},
+    {"add_multi", (PyCFunction)PylibMC_Client_add_multi,
+        METH_VARARGS|METH_KEYWORDS, "Add multiple keys at once."},
     {"delete_multi", (PyCFunction)PylibMC_Client_delete_multi,
         METH_VARARGS|METH_KEYWORDS, "Delete multiple keys at once."},
     {"get_behaviors", (PyCFunction)PylibMC_Client_get_behaviors, METH_NOARGS,

coders.rst

@@ -0,0 +1,8 @@
+The List of Honored Men
+=======================
+
+* Ludvig Ericson <ludvig@lericson.se>
+* ketralnis <ketralnis@reddit.com>
+* Ruda Moura <ruda.moura@corp.terra.com.br>
+* Noah Silas <noah@mahalo.com>
+* Johan Bergström <johan@bergstroem.nu>

pylibmc-version.h

@@ -1 +1 @@
-#define PYLIBMC_VERSION "1.0-alpha"
+#define PYLIBMC_VERSION "1.1.1"

pylibmc.py

@@ -1,222 +0,0 @@
-"""`python-memcached`-compatible wrapper around `_pylibmc`.
-
-The interface is pretty much exactly the same as python-memcached, with some
-minor differences. If you should happen to spot any, file a bug!
-
->>> import pylibmc
->>> mc = pylibmc.Client(["127.0.0.1"])
->>> b = mc.behaviors
->>> ks = list(sorted(k for k in b.keys() if not k.startswith("_")))
->>> ks  # doctest: +NORMALIZE_WHITESPACE
-['buffer_requests', 'cache_lookups', 'cas', 'connect_timeout', 'distribution',
- 'failure_limit', 'hash', 'ketama', 'ketama_hash', 'ketama_weighted',
- 'no_block', 'receive_timeout', 'send_timeout', 'tcp_nodelay', 'verify_keys']
->>> b["hash"]
-'default'
->>> b["hash"] = 'fnv1a_32'
->>> mc.behaviors["hash"]
-'fnv1a_32'
->>> super(pylibmc.Client, mc).get_behaviors()["hash"]
-6
-"""
-
-import _pylibmc
-from warnings import warn
-
-__all__ = ["hashers", "distributions", "Client"]
-__version__ = _pylibmc.__version__
-support_compression = _pylibmc.support_compression
-
-errors = tuple(e for (n, e) in _pylibmc.exceptions)
-# *Cough* Uhm, not the prettiest of things but this unpacks all exception
-# objects and sets them on the very module object currently constructed.
-import sys
-modself = sys.modules[__name__]
-for name, exc in _pylibmc.exceptions:
-    setattr(modself, name, exc)
-
-hashers, hashers_rvs = {}, {}
-distributions, distributions_rvs = {}, {}
-# Not the prettiest way of doing things, but works well.
-for name in dir(_pylibmc):
-    if name.startswith("hash_"):
-        key, value = name[5:], getattr(_pylibmc, name)
-        hashers[key] = value
-        hashers_rvs[value] = key
-    elif name.startswith("distribution_"):
-        key, value = name[13:].replace("_", " "), getattr(_pylibmc, name)
-        distributions[key] = value
-        distributions_rvs[value] = key
-
-class BehaviorDict(dict):
-    def __init__(self, client, *args, **kwds):
-        super(BehaviorDict, self).__init__(*args, **kwds)
-        self.client = client
-
-    def __setitem__(self, name, value):
-        super(BehaviorDict, self).__setitem__(name, value)
-        self.client.set_behaviors({name: value})
-
-    def update(self, d):
-        super(BehaviorDict, self).update(d)
-        self.client.set_behaviors(d.copy())
-
-class Client(_pylibmc.client):
-    def __init__(self, servers, binary=False):
-        """Initialize a memcached client instance.
-
-        This connects to the servers in *servers*, which will default to being
-        TCP servers. If it looks like a filesystem path, a UNIX socket. If
-        prefixed with `udp:`, a UDP connection.
-
-        If *binary* is True, the binary memcached protocol is used.
-        """
-        addr_tups = []
-        for server in servers:
-            addr = server
-            port = 11211
-            if server.startswith("udp:"):
-                stype = _pylibmc.server_type_udp
-                addr = addr[4:]
-                if ":" in server:
-                    (addr, port) = addr.split(":", 1)
-                    port = int(port)
-            elif ":" in server:
-                stype = _pylibmc.server_type_tcp
-                (addr, port) = server.split(":", 1)
-                port = int(port)
-            elif "/" in server:
-                stype = _pylibmc.server_type_unix
-                port = 0
-            else:
-                stype = _pylibmc.server_type_tcp
-            addr_tups.append((stype, addr, port))
-        super(Client, self).__init__(servers=addr_tups, binary=binary)
-
-    def get_behaviors(self):
-        """Gets the behaviors from the underlying C client instance.
-
-        Reverses the integer constants for `hash` and `distribution` into more
-        understandable string values. See *set_behaviors* for info.
-        """
-        behaviors = super(Client, self).get_behaviors()
-        behaviors["hash"] = hashers_rvs[behaviors["hash"]]
-        behaviors["distribution"] = distributions_rvs[behaviors["distribution"]]
-        return BehaviorDict(self, behaviors)
-
-    def set_behaviors(self, behaviors):
-        """Sets the behaviors on the underlying C client instance.
-
-        Takes care of morphing the `hash` key, if specified, into the
-        corresponding integer constant (which the C client expects.) If,
-        however, an unknown value is specified, it's passed on to the C client
-        (where it most surely will error out.)
-
-        This also happens for `distribution`.
-        """
-        behaviors = behaviors.copy()
-        if any(" " in k for k in behaviors):
-            warn(DeprecationWarning("space-delimited behavior names "
-                                    "are deprecated"))
-            for k in [k for k in behaviors if " " in k]:
-                behaviors[k.replace(" ", "_")] = behaviors.pop(k)
-        if behaviors.get("hash") is not None:
-            behaviors["hash"] = hashers[behaviors["hash"]]
-        if behaviors.get("ketama_hash") is not None:
-            behaviors["ketama_hash"] = hashers[behaviors["ketama_hash"]]
-        if behaviors.get("distribution") is not None:
-            behaviors["distribution"] = distributions[behaviors["distribution"]]
-        return super(Client, self).set_behaviors(behaviors)
-
-    behaviors = property(get_behaviors, set_behaviors)
-    @property
-    def behaviours(self):
-        raise AttributeError("nobody uses british spellings")
-
-from contextlib import contextmanager
-
-class ClientPool(list):
-    """Client pooling helper.
-
-    This is mostly useful in threaded environments, because a client isn't
-    thread-safe at all. Instead, what you want to do is have each thread use
-    its own client, but you don't want to reconnect these all the time.
-
-    The solution is a pool, and this class is a helper for that.
-
-    >>> mc = Client(["127.0.0.1"])
-    >>> pool = ClientPool()
-    >>> pool.fill(mc, 4)
-    >>> with pool.reserve() as mc:
-    ...     mc.set("hi", "ho")
-    ...     mc.delete("hi")
-    ... 
-    True
-    True
-    """
-
-    @contextmanager
-    def reserve(self):
-        """Reserve a client, and put it back into the pool when done."""
-        mc = self.pop()
-        try:
-            yield mc
-        finally:
-            self.append(mc)
-
-    def fill(self, mc, n_slots):
-        """Fill *n_slots* of the pool with clones of *mc*."""
-        for i in xrange(n_slots):
-            self.append(mc.clone())
-
-class ThreadMappedPool(dict):
-    """Much like the *ClientPool*, helps you with pooling.
-
-    In a threaded environment, you'd most likely want to have a client per
-    thread. And there'd be no harm in one thread keeping the same client at all
-    times. So, why not map threads to clients? That's what this class does.
-
-    If a client is reserved, this class checks for a key based on the current
-    thread, and if none exists, clones the master client and inserts that key.
-
-    >>> mc = Client(["127.0.0.1"])
-    >>> pool = ThreadMappedPool(mc)
-    >>> with pool.reserve() as mc:
-    ...     mc.set("hi", "ho")
-    ...     mc.delete("hi")
-    ... 
-    True
-    True
-    """
-
-    def __new__(cls, master):
-        return super(ThreadMappedPool, cls).__new__(cls)
-
-    def __init__(self, master):
-        self.master = master
-
-    @contextmanager
-    def reserve(self):
-        """Reserve a client.
-
-        Creates a new client based on the master client if none exists for the
-        current thread.
-        """
-        key = thread.get_ident()
-        mc = self.pop(key, None)
-        if mc is None:
-            mc = self.master.clone()
-        try:
-            yield mc
-        finally:
-            self[key] = mc
-
-# This makes sure ThreadMappedPool doesn't exist with non-thread Pythons.
-try:
-    import thread
-except ImportError:
-    del ThreadMappedPool
-
-if __name__ == "__main__":
-    import doctest
-    doctest.testmod()

pylibmc/__init__.py

@@ -0,0 +1,73 @@
+"""Snappy libmemcached wrapper
+
+pylibmc is a Python wrapper around TangentOrg's libmemcached library.
+
+The interface is intentionally made as close to python-memcached as possible,
+so that applications can drop-in replace it.
+
+Example usage
+=============
+
+Create a connection and configure it::
+
+    >>> import pylibmc
+    >>> mc = pylibmc.Client(["127.0.0.1"], binary=True)
+    >>> mc.behaviors = {"tcp_nodelay": True, "ketama": True}
+
+Basic operation::
+
+    >>> mc.set("some_key", "Some value")
+    True
+    >>> value = mc.get("some_key")
+    >>> value
+    'Some value'
+    >>> mc.set("another_key", 3)
+    True
+    >>> mc.delete("another_key")
+    True
+    >>> mc.set("key", "1")  # str or int is fine
+    True
+
+Atomic increments and decrements::
+
+    >>> mc.incr("key")
+    2L
+    >>> mc.decr("key")
+    1L
+
+Batch operation::
+
+    >>> mc.get_multi(["key", "another_key"])
+    {'key': '1'}
+    >>> mc.set_multi({"cats": ["on acid", "furry"], "dogs": True})
+    []
+    >>> mc.get_multi(["cats", "dogs"])
+    {'cats': ['on acid', 'furry'], 'dogs': True}
+    >>> mc.delete_multi(["cats", "dogs", "nonextant"])
+    False
+    >>> mc.add_multi({"cats": ["on acid", "furry"], "dogs": True})
+    []
+    >>> mc.get_multi(["cats", "dogs"])
+    {'cats': ['on acid', 'furry'], 'dogs': True}
+    >>> mc.add_multi({"cats": "not set", "dogs": "definitely not set", "bacon": "yummy"})
+    ['cats', 'dogs']
+    >>> mc.get_multi(["cats", "dogs", "bacon"])
+    {'cats': ['on acid', 'furry'], 'bacon': 'yummy', 'dogs': True}
+    >>> mc.delete_multi(["cats", "dogs", "bacon"])
+    True
+
+Further Reading
+===============
+
+See http://sendapatch.se/projects/pylibmc/
+"""
+
+import _pylibmc
+from .consts import hashers, distributions
+from .client import Client
+from .pools import ClientPool, ThreadMappedPool
+
+support_compression = _pylibmc.support_compression
+__version__ = _pylibmc.__version__
+__all__ = ["hashers", "distributions", "Client",
+           "ClientPool", "ThreadMappedPool"]

pylibmc/__main__.py

@@ -0,0 +1,48 @@
+"""Interactive shell"""
+
+import sys
+import code
+import random
+import pylibmc
+
+tips = [
+    "Want to use 127.0.0.1? Just hit Enter immediately.",
+    "This was supposed to be a list of tips but I...",
+    "I don't really know what to write here.",
+    "Really, hit Enter immediately and you'll connect to 127.0.0.1.",
+    "Did you know there's a --binary flag? Try it!",
+    "Want to use binary mode? Pass --binary as a sole argument."
+]
+
+def print_header(outf=sys.stdout):
+    outf.write("pylibmc interactive shell\n\n")
+    outf.write("Input list of servers, terminating by a blank line.\n")
+    outf.write(random.choice(tips) + "\n")
+
+def collect_servers():
+    in_addr = raw_input("Address [127.0.0.1]: ")
+    while in_addr:
+        in_addr = raw_input("Address [<stop>]: ")
+        if in_addr:
+            yield in_addr
+    else:
+        yield "127.0.0.1"
+
+banner = ("\nmc client available as `mc`\n"
+          "memface available as mf")
+def interact(servers, banner=banner, binary=False):
+    mc = pylibmc.Client(servers, binary=binary)
+    mf = pylibmc.Memface(servers, binary=binary)
+    local = {"pylibmc": pylibmc,
+             "mf": mf, "mc": mc}
+    code.interact(banner=banner, local=local)
+
+def main():
+    binary = False
+    if sys.argv[1:] == ["--binary"]:
+        binary = True
+    print_header()
+    interact(list(collect_servers()), binary=binary)
+
+if __name__ == "__main__":
+    main()

pylibmc/client.py

@@ -0,0 +1,90 @@
+"""Python-level wrapper client"""
+
+import _pylibmc
+from .consts import (hashers, distributions,
+                     hashers_rvs, distributions_rvs,
+                     BehaviorDict)
+
+class Client(_pylibmc.client):
+    def __init__(self, servers, binary=False):
+        """Initialize a memcached client instance.
+
+        This connects to the servers in *servers*, which will default to being
+        TCP servers. If it looks like a filesystem path, a UNIX socket. If
+        prefixed with `udp:`, a UDP connection.
+
+        If *binary* is True, the binary memcached protocol is used.
+        """
+        self.binary = binary
+        self.addresses = list(servers)
+        addr_tups = []
+        for server in servers:
+            addr = server
+            port = 11211
+            if server.startswith("udp:"):
+                stype = _pylibmc.server_type_udp
+                addr = addr[4:]
+                if ":" in server:
+                    (addr, port) = addr.split(":", 1)
+                    port = int(port)
+            elif ":" in server:
+                stype = _pylibmc.server_type_tcp
+                (addr, port) = server.split(":", 1)
+                port = int(port)
+            elif "/" in server:
+                stype = _pylibmc.server_type_unix
+                port = 0
+            else:
+                stype = _pylibmc.server_type_tcp
+            addr_tups.append((stype, addr, port))
+        super(Client, self).__init__(servers=addr_tups, binary=binary)
+
+    def __repr__(self):
+        return "%s(%r, binary=%r)" % (self.__class__.__name__,
+                                      self.addresses, self.binary)
+
+    def __str__(self):
+        addrs = ", ".join(map(str, self.addresses))
+        return "<%s for %s, binary=%r>" % (self.__class__.__name__,
+                                           addrs, self.binary)
+
+    def get_behaviors(self):
+        """Gets the behaviors from the underlying C client instance.
+
+        Reverses the integer constants for `hash` and `distribution` into more
+        understandable string values. See *set_behaviors* for info.
+        """
+        bvrs = super(Client, self).get_behaviors()
+        bvrs["hash"] = hashers_rvs[bvrs["hash"]]
+        bvrs["distribution"] = distributions_rvs[bvrs["distribution"]]
+        return BehaviorDict(self, bvrs)
+
+    def set_behaviors(self, behaviors):
+        """Sets the behaviors on the underlying C client instance.
+
+        Takes care of morphing the `hash` key, if specified, into the
+        corresponding integer constant (which the C client expects.) If,
+        however, an unknown value is specified, it's passed on to the C client
+        (where it most surely will error out.)
+
+        This also happens for `distribution`.
+        """
+        behaviors = behaviors.copy()
+        if behaviors.get("hash") is not None:
+            behaviors["hash"] = hashers[behaviors["hash"]]
+        if behaviors.get("ketama_hash") is not None:
+            behaviors["ketama_hash"] = hashers[behaviors["ketama_hash"]]
+        if behaviors.get("distribution") is not None:
+            behaviors["distribution"] = distributions[behaviors["distribution"]]
+        return super(Client, self).set_behaviors(behaviors)
+
+    behaviors = property(get_behaviors, set_behaviors)
+    @property
+    def behaviours(self):
+        raise AttributeError("nobody uses british spellings")
+
+    def clone(self):
+        obj = super(Client, self).clone()
+        obj.addresses = list(self.addresses)
+        obj.binary = self.binary
+        return obj

pylibmc/consts.py

@@ -0,0 +1,40 @@
+"""Constants and functionality related to them"""
+
+import _pylibmc
+
+#: Mapping of 
+errors = tuple(e for (n, e) in _pylibmc.exceptions)
+# *Cough* Uhm, not the prettiest of things but this unpacks all exception
+# objects and sets them on the package module object currently constructed.
+import sys
+modpkg = sys.modules[__package__]
+modself = sys.modules[__package__]
+for name, exc in _pylibmc.exceptions:
+    setattr(modself, name, exc)
+    setattr(modpkg, name, exc)
+
+hashers, hashers_rvs = {}, {}
+distributions, distributions_rvs = {}, {}
+# Not the prettiest way of doing things, but works well.
+for name in dir(_pylibmc):
+    if name.startswith("hash_"):
+        key, value = name[5:], getattr(_pylibmc, name)
+        hashers[key] = value
+        hashers_rvs[value] = key
+    elif name.startswith("distribution_"):
+        key, value = name[13:].replace("_", " "), getattr(_pylibmc, name)
+        distributions[key] = value
+        distributions_rvs[value] = key
+
+class BehaviorDict(dict):
+    def __init__(self, client, *args, **kwds):
+        super(BehaviorDict, self).__init__(*args, **kwds)
+        self.client = client
+
+    def __setitem__(self, name, value):
+        super(BehaviorDict, self).__setitem__(name, value)
+        self.client.set_behaviors({name: value})
+
+    def update(self, d):
+        super(BehaviorDict, self).update(d)
+        self.client.set_behaviors(d.copy())

pylibmc/pools.py

@@ -0,0 +1,97 @@
+"""Pooling"""
+
+from __future__ import with_statement
+
+from contextlib import contextmanager
+from Queue import Queue
+
+class ClientPool(Queue):
+    """Client pooling helper.
+
+    This is mostly useful in threaded environments, because a client isn't
+    thread-safe at all. Instead, what you want to do is have each thread use
+    its own client, but you don't want to reconnect these all the time.
+
+    The solution is a pool, and this class is a helper for that.
+
+    >>> mc = Client(["127.0.0.1"])
+    >>> pool = ClientPool()
+    >>> pool.fill(mc, 4)
+    >>> with pool.reserve() as mc:
+    ...     mc.set("hi", "ho")
+    ...     mc.delete("hi")
+    ... 
+    True
+    True
+    """
+
+    def __init__(self, mc=None, n_slots=None):
+        Queue.__init__(self, n_slots)
+        if mc is not None:
+            self.fill(mc, n_slots)
+
+    @contextmanager
+    def reserve(self, timeout=None):
+        """Context manager for reserving a client from the pool.
+
+        If *timeout* is given, it specifiecs how long to wait for a client to
+        become available.
+        """
+        mc = self.get(True, timeout=timeout)
+        try:
+            yield mc
+        finally:
+            self.put(mc)
+
+    def fill(self, mc, n_slots):
+        """Fill *n_slots* of the pool with clones of *mc*."""
+        for i in xrange(n_slots):
+            self.put(mc.clone())
+
+class ThreadMappedPool(dict):
+    """Much like the *ClientPool*, helps you with pooling.
+
+    In a threaded environment, you'd most likely want to have a client per
+    thread. And there'd be no harm in one thread keeping the same client at all
+    times. So, why not map threads to clients? That's what this class does.
+
+    If a client is reserved, this class checks for a key based on the current
+    thread, and if none exists, clones the master client and inserts that key.
+
+    >>> mc = Client(["127.0.0.1"])
+    >>> pool = ThreadMappedPool(mc)
+    >>> with pool.reserve() as mc:
+    ...     mc.set("hi", "ho")
+    ...     mc.delete("hi")
+    ... 
+    True
+    True
+    """
+
+    def __new__(cls, master):
+        return super(ThreadMappedPool, cls).__new__(cls)
+
+    def __init__(self, master):
+        self.master = master
+
+    @contextmanager
+    def reserve(self):
+        """Reserve a client.
+
+        Creates a new client based on the master client if none exists for the
+        current thread.
+        """
+        key = thread.get_ident()
+        mc = self.pop(key, None)
+        if mc is None:
+            mc = self.master.clone()
+        try:
+            yield mc
+        finally:
+            self[key] = mc
+
+# This makes sure ThreadMappedPool doesn't exist with non-thread Pythons.
+try:
+    import thread
+except ImportError:
+    ThreadMappedPool = None

setup.py

@@ -15,6 +15,14 @@ defs = []
 incdirs = []
 libdirs = []
 
+def append_env(L, e):
+    v = os.environ.get(e)
+    if v and os.path.exists(v):
+        L.append(v)
+
+append_env(pkgdirs, "LIBMEMCACHED")
+append_env(pkgdirs, "ZLIB")
+
 # Hack up sys.argv, yay
 unprocessed = []
 for arg in sys.argv[1:]:
@@ -61,9 +69,9 @@ readme_text = open("README.rst", "U").read()
 version = open("pylibmc-version.h", "U").read().strip().split("\"")[1]
 
 setup(name="pylibmc", version=version,
-      url="http://lericson.blogg.se/code/category/pylibmc.html",
+      url="http://sendapatch.se/projects/pylibmc/",
       author="Ludvig Ericson", author_email="ludvig@lericson.se",
       license="3-clause BSD <http://www.opensource.org/licenses/bsd-license.php>",
       description="Quick and small memcached client for Python",
       long_description=readme_text,
-      ext_modules=[pylibmc_ext], py_modules=["pylibmc"])
+      ext_modules=[pylibmc_ext], packages=["pylibmc"])

tests.py

@@ -1,4 +1,4 @@
-"""Tests. They want YOU!!
+r"""Tests. They want YOU!!
 
 
 Basic functionality.
@@ -57,6 +57,10 @@ Zero-key-test-time!
 False
 >>> c.set_multi({"": "hi"})
 ['']
+>>> c.delete_multi({"a": "b"})
+Traceback (most recent call last):
+  ...
+TypeError: keys must be a sequence, not a mapping
 
 Timed stuff. The reason we, at UNIX times, set it two seconds in the future and
 then sleep for >3 is that memcached might round the time up and down and left
@@ -70,17 +74,32 @@ True
 >>> c.get("hi")
 >>> c.set("hi", "loretta", int(time()) + 2)
 True
+>>> c.set_multi({"hi2": "charlotta"}, 1)
+[]
 >>> c.get("hi")
 'loretta'
+>>> c.get("hi2")
+'charlotta'
 >>> sleep(3.1)
->>> c.get("hi")
+>>> c.get("hi"), c.get("hi2")
->>>
+(None, None)
+
+See issue #1 ``http://github.com/lericson/pylibmc/issues/#issue/1`` -- delete
+should not accept a time argument.
+>>> c.delete("foo", 123)
+Traceback (most recent call last):
+  ...
+TypeError: delete() takes exactly 1 argument (2 given)
+>>> c.delete_multi(["foo"], time=123)
+Traceback (most recent call last):
+  ...
+TypeError: 'time' is an invalid keyword argument for this function
 
 Now for keys with funny types.
 >>> c.set(1, "hi")
 Traceback (most recent call last):
   ...
-TypeError: argument 1 must be string or read-only buffer, not int
+TypeError: argument 1 must be string, not int
 >>> c.get(1)
 Traceback (most recent call last):
   ...
@@ -94,6 +113,10 @@ Traceback (most recent call last):
   ...
 TypeError: key must be an instance of str
 
+This didn't use to work, but now it does.
+>>> c.get_multi([])
+{}
+
 Getting stats is fun!
 >>> for (svr, stats) in c.get_stats():
 ...     print svr
@@ -101,7 +124,7 @@ Getting stats is fun!
 ...     while ks:
 ...         cks, ks = ks[:6], ks[6:]
 ...         print ", ".join(cks)
-localhost:11211 (0)
+127.0.0.1:11211 (0)
 pid, total_items, uptime, version, limit_maxbytes, rusage_user
 bytes_read, rusage_system, cmd_get, curr_connections, threads, total_connections
 cmd_set, curr_items, get_misses, evictions, bytes, connection_structures
@@ -174,6 +197,30 @@ True
 >>> c.delete("hello")
 True
 
+incr_multi
+>>> c.add_multi({'a': 1, 'b': 0, 'c': 4})
+[]
+>>> c.incr_multi(('a', 'b', 'c'), delta=1)
+>>> list(sorted(c.get_multi(('a', 'b', 'c')).items()))
+[('a', 2), ('b', 1), ('c', 5)]
+>>> c.delete_multi(('a', 'b', 'c'))
+True
+>>> c.add_multi({'a': 1, 'b': 0, 'c': 4}, key_prefix='x')
+[]
+>>> c.incr_multi(('a', 'b', 'c'), key_prefix='x', delta=5)
+>>> list(sorted(c.get_multi(('a', 'b', 'c'), key_prefix='x').items()))
+[('a', 6), ('b', 5), ('c', 9)]
+>>> c.delete_multi(('a', 'b', 'c'), key_prefix='x')
+True
+>>> c.add('a', 1)
+True
+>>> c.incr_multi(('a', 'b', 'c'), key_prefix='x', delta=1)
+Traceback (most recent call last):
+ ...
+NotFound: error 16 from memcached_increment: NOT FOUND
+>>> c.delete('xa')
+False
+
 Empty server lists are bad for your health.
 >>> c = _pylibmc.client([])
 Traceback (most recent call last):
@@ -187,23 +234,76 @@ Python-wrapped behaviors dict
 >>> pc.behaviors.update({"hash": "fnv1a_32", "distribution": "consistent"})
 >>> (pc.behaviors["hash"], pc.behaviors["distribution"])
 ('fnv1a_32', 'consistent')
+
+>>> pc = pylibmc.Client(["%s:%d" % test_server[1:]])
+>>> b = pc.behaviors
+>>> ks = list(sorted(k for k in b.keys() if not k.startswith("_")))
+>>> ks  # doctest: +NORMALIZE_WHITESPACE
+['buffer_requests', 'cache_lookups', 'cas', 'connect_timeout', 'distribution',
+ 'failure_limit', 'hash', 'ketama', 'ketama_hash', 'ketama_weighted',
+ 'no_block', 'receive_timeout', 'send_timeout', 'tcp_nodelay', 'verify_keys']
+>>> b["hash"]
+'default'
+>>> b["hash"] = 'fnv1a_32'
+>>> pc.behaviors["hash"]
+'fnv1a_32'
+>>> super(pylibmc.Client, pc).get_behaviors()["hash"]
+6
+
+Ormod's Zero-byte Adventure Story
+>>> bc = _pylibmc.client([test_server], binary=True)
+>>> bc.set("\0\0", "ORMOD")
+True
+>>> bc.get_multi(["\0\0"])
+{'\x00\x00': 'ORMOD'}
+
+Test server/client max length
+>>> mc.get('x'*250)
+>>> mc.get('x'*251)
+Traceback (most recent call last):
+  ...
+ValueError: key too long, max is 250
+
+Test CAS
+>>> mc.behaviors['cas'] = True
+>>> mc.delete('foo') and False
+False
+>>> mc.gets('foo')
+(None, None)
+>>> mc.set('foo', 'bar')
+True
+>>> foostr, cas = mc.gets('foo')
+>>> foostr
+'bar'
+>>> mc.cas('foo', 'quux', cas+1)
+False
+>>> mc.cas('foo', 'baz', cas)
+True
+>>> mc.get('foo')
+'baz'
+>>> mc.behaviors['cas'] = False
+>>> mc.gets('foo')
+Traceback (most recent call last):
+...
+ValueError: gets without cas behavior
+>>> mc.cas('foo', 'bar', 1)
+Traceback (most recent call last):
+...
+ValueError: cas without cas behavior
 """
 
 # Used to test pickling.
 class Foo(object): pass
 
-# Fix up sys.path so as to include the build/lib.*/ directory.
+# Fix up sys.path so as to include the correct build/lib.*/ directory.
 import sys
-import os
+from distutils.dist import Distribution
-from glob import glob
+from distutils.command.build import build
 
-dist_dir = os.path.dirname(__file__)
+build_cmd = build(Distribution({"ext_modules": True}))
-for build_dir in glob(os.path.join(dist_dir, "build", "lib.*")):
+build_cmd.finalize_options()
-    sys.path.insert(0, build_dir)
+lib_dirn = build_cmd.build_lib
-    break
+sys.path.insert(0, lib_dirn)
-else:
-    print >>sys.stderr, "Using system-wide installation of pylibmc!"
-    print >>sys.stderr, "==========================================\n"
 
 import pylibmc, _pylibmc
 import socket
@@ -274,7 +374,12 @@ class TestCmemcached(unittest.TestCase):
         self.assertEqual(result, "I Do")
 # }}}
 
-test_server = (_pylibmc.server_type_tcp, "localhost", 11211)
+from os import environ
+
+test_server = (
+    _pylibmc.server_type_tcp,
+    str(environ.get("MEMCACHED_HOST", "127.0.0.1")),
+    int(environ.get("MEMCACHED_PORT", "11211")))
 
 def get_version(addr):
     (type_, host, port) = addr