Tips and tricks from my Telegram-channel @pythonetc, December 2019

It is a new selection of tips and tricks about Python and programming from my Telegram-channel @pythonetc.

← Previous publications.

Different asyncio tasks obviously have different stacks. You can view at all of them at any moment using asyncio.all_tasks() to get all currently running tasks and task.get_stack() to get a stack for each task.

import linecache
import asyncio
import random


async def producer(queue):
    while True:
        await queue.put(random.random())
        await asyncio.sleep(0.01)

async def avg_printer(queue):
    total = 0
    cnt = 0
    while True:
        while queue.qsize():
            x = await queue.get()
            total += x
            cnt += 1
            queue.task_done()
        print(total / cnt)
        await asyncio.sleep(1)

async def monitor():
    while True:
        await asyncio.sleep(1.9)
        for task in asyncio.all_tasks():
            if task is not asyncio.current_task():
                f  = task.get_stack()[-1]
                last_line = linecache.getline(
                    f.f_code.co_filename,
                    f.f_lineno,
                    f.f_globals,
                )
                print(task)
                print('\t', last_line.strip())

        print()

async def main():
    loop = asyncio.get_event_loop()

    queue = asyncio.Queue()

    loop.create_task(producer(queue))
    loop.create_task(producer(queue))
    loop.create_task(producer(queue))

    loop.create_task(avg_printer(queue))
    loop.create_task(monitor())



loop = asyncio.get_event_loop()
loop.create_task(main())
loop.run_forever()

To avoid messing with the stack object directly and using the linecache module you can call task.print_stack() instead.

You can translate or delete characters of a string (like the tr utility does) with the translate method of str:

>>> 'Hello, world!'.translate({
...     ord(','): ';',
...     ord('o'): '0',
... })
'Hell0; w0rld!'

The only argument of translate is a dictionary mapping character codes to characters (or codes). It’s usually more convenient to create such a dictionary with str.maketrans static method:

>>> 'Hello, world!'.translate(str.maketrans({
...     ',': ';',
...     'o': '0',
... }))
'Hell0; w0rld!'

Or even:

>>> 'Hello, world!'.translate(str.maketrans(
...     ',o', ';0'
... ))
'Hell0; w0rld!'

The third argument is for deleting characters:

>>> tr = str.maketrans(',o', ';0', '!')
>>> tr
{44: 59, 111: 48, 33: None}
>>> 'Hello, world!'.translate(tr)
'Hell0; w0rld'

mypy doesn’t yet support recursive types definitions:

from typing import Optional, Dict
from pathlib import Path

TreeDict = Dict[str, 'TreeDict']


def tree(path: Path) -> TreeDict:
    return {
        f.name: tree(f) if f.is_dir() else None
        for f in path.iterdir()
    }

The error is Cannot resolve name "TreeDict" (possible cyclic definition).

Stay tuned here: https://github.com/python/mypy/issues/731

Ordinary function just needs to call itself to become recursive. It’s not so simple for generators: you usually have to use yield from for recursive generators:

from operator import itemgetter


tree = {
    'imgs': {
        '1.png': None,
        '2.png': None,
        'photos': {
            'me.jpg': None
        },
    },
    'MANIFEST': None,
}


def flatten_tree(tree):
    for name, children in sorted(
        tree.items(),
        key=itemgetter(0)
    ):
        yield name
        if children:
            yield from flatten_tree(children)


print(list(flatten_tree(tree)))

You can use for not only with variables but with any expression. It’s evaluated on every iteration:

>>> log2 = {}
>>> key = 1
>>> for log2[key] in range(100):
...     key *= 2
...
>>> log2[16]
4
>>> log2[1024]
10