Tips and tricks from my Telegram-channel @pythonetc, April 2019

    It is a new selection of tips and tricks about Python and programming from my Telegram-channel @pythonetc.

    Previous publications.

    Storing and sending object via network as bytes is a huge topic. Let’s discuss some tools that are usually used for that in Python and their advantages and disadvantages.

    As an example I’ll try to serialize the Cities object which contains some City objects as well as their order. Here is four method you can use:

    1. JSON. It’s human readable, easy to use, but consumes a lot of memory. The same is true for other formats like YAML or XML.

    class City:
        def to_dict(self):
            return dict(
    class Cities:
        def __init__(self, cities):
            self._cities = cities
        def to_json(self):
            return json.dumps([
                c.to_dict() for c in self._cities

    2. Pickle. Pickle is native for Python, can be customized and consumes less memory than JSON. The downside is you have to use Python to unpickle the data.

    class Cities:
        def pickle(self):
            return pickle.dumps(self)

    3. Protobuf (and other binary serializers such as msgpack). Consumes even less memory, can be used from any other programming languages, but require custom schema:

    syntax = "proto2";
    message City {
        required string name = 1;
        required string country = 2;
        required float lon = 3;
        required float lat = 4;
    message Cities {
        repeated City cities = 1;
    class City:
        def to_protobuf(self):
            result = city_pb2.City()
   = self._name
   = self._country
            result.lon = self._lon
   = self._lat
            return result
    class Cities:
        def to_protobuf(self):
            result = city_pb2.Cities()
                c.to_protobuf() for c in self._cities
            return result

    4. Manual. You can manually pack and unpack data with the struct module. It allow you to consume the absolute minimum amount of memory, but protobuf still can be a better choice since it supports versioning and explicit schemas.

    class City:
        def to_bytes(self):
            name_encoded = self._name.encode('utf8')
            name_length = len(name_encoded)
            country_encoded = self._country.encode('utf8')
            country_length = len(country_encoded)
            return struct.pack(
                name_length, name_encoded,
                country_length, country_encoded,
                self._lon, self._lat,
    class Cities:
        def to_bytes(self):
            return b''.join(
                c.to_bytes() for c in self._cities

    If a function argument has the default value of None and is annotated as T, mypy automatically treats it as Optional[T] (in other words, Union[T, None]).

    That doesn't work with other types, so you can't have something like f(x: A = B()). It also doesn't work with a variable assignment: a: A = None will cause an error.

    def f(x: int = None):
    def g(y: int = 'x'):
    z: int = None
    $ mypy error: Revealed type is 'Union[, None]' error: Incompatible default for argument "y" (default has type "str", argument has type "int") error: Revealed type is '' error: Incompatible types in assignment (expression has type "None", variable has type "int") error: Revealed type is ''

    In Python 3, once the except block is exited, the variables that store caught exceptions are removed from locals() even if they previously existed:

    >>> e = 2
    >>> try:
    ...     1/0
    ... except Exception as e:
    ...     pass
    >>> e
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    NameError: name 'e' is not defined

    If you want to save a reference to the exception, you have to use another variable:

    >>> error = None
    >>> try:
    ...     1/0
    ... except Exception as e:
    ...     error = e
    >>> error
    ZeroDivisionError('division by zero',)

    This is not true for Python 2.

    You may have your own pypi repository. It lets you release packages inside your project and install them with pip as though they are regular packages.

    It is remarkable that you don’t have to install any specific software, but can use a regular http-server instead. Here is how it works for me, for example.

    Let’s have a trivial package named pythonetc.
    from setuptools import setup, find_packages
    def ping():
        return 'pong'

    Let’s release it to the ~/pypi directory:

    $ python sdist bdist_wheel
    $ mv dist ~/pypi/pythonetc

    Now server this on the domain with nginx:

    $ cat /etc/nginx/sites-enabled/pypi
    server {
            listen 80;
            root /home/vadim/pypi;
            index index.html index.htm index.nginx-debian.html;
            location / {
                    autoindex on;
                    try_files $uri $uri/ =404;

    It now can be installed:

    $ pip install -i --trusted-host pythonetc
    Collecting pythonetc
    Installing collected packages: pythonetc
    Successfully installed pythonetc-1.0
    $ python
    Python 3.7.0+ (heads/3.7:0964aac, Mar 29 2019, 00:40:55)
    [GCC 4.9.2] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import pythonetc

    It’s quite often when you have to declare a dictionary with all keys equal to the local variables with the same name. Something like this:


    ECMAScript even has the special form of object literal for such cases (it’s called Object Literal Property Value Shorthand):

    > var a = 1;
    < undefined
    > var b = 2;
    < undefined
    > {a, b}
    < {a: 1, b: 2}

    It is possible to create a similar helper in Python (alas, it looks not even closely as good as the ECMAScript notation):

    def shorthand_dict(lcls, names):
        return {k: lcls[k] for k in names}
    context = dict(user_id=42, user_ip='')
    mode = 'force'
    action_type = 7
    shorthand_dict(locals(), [

    You may wonder why we have to pass locals() as a parameter in the previous example. Is it possible to get the locals of the caller in the callee? It is indeed, but you have to mess with the inpsect module:

    import inspect
    def shorthand_dict(names):
        lcls = inspect.currentframe().f_back.f_locals
        return {k: lcls[k] for k in names}
    context = dict(user_id=42, user_ip='')
    mode = 'force'
    action_type = 7

    You can go even further and use something like this —

    from sorcery import dict_of
    dict_of(context, mode, action_type) Group
    Building the Internet

    Comments 0

    Only users with full accounts can post comments. Log in, please.