It is a new selection of tips and tricks about Python and programming from my Telegram-channel @pythonetc.

← Previous publications

Support of Visual Studio 2019 in PVS-Studio affected a number of components: the plugin itself, the command-line analyzer, the cores of the C++ and C# analyzers, and a few utilities. In this article, I will briefly explain what problems we encountered when implementing support of the IDE and how we addressed them.

Is it always easy for you to prioritize the tasks of the huge project? What if five or more tasks have the main priority and urgency?

Experienced project managers and product owners know that intuition is not enough in such cases. In order to avoid missing deadlines, today, managers are able to apply useful methodologies for determining priorities, as well as modern tools that help to visualize data and not miss anything in their workflows.

What is the best tool to use for drawing vector pictures? For me and probably for many others, the answer is pretty obvious: Illustrator, or, maybe, Inkscape. At least that's what I thought when I was asked to draw about eight hundred diagrams for a physics textbook. Nothing exceptional, just a bunch of black and white illustrations with spheres, springs, pulleys, lenses and so on. By that time it was already known that the book was going to be made in LaTeX and I was given a number of MS Word documents with embedded images. Some of them were scanned pictures from other books, some were pencil drawings. Picturing days and nights of inkscaping this stuff made me feel dizzy, so soon I found myself fantasizing about a more automated solution. For some reason MetaPost became the focus of these fantasies.

Introduction

It’s time to talk about exceptions or, rather, exceptional situations. Before we start, let’s look at the definition. What is an exceptional situation?

This is a situation that makes the execution of current or subsequent code incorrect. I mean different from how it was designed or intended. Such a situation compromises the integrity of an application or its part, e.g. an object. It brings the application into an extraordinary or exceptional state.

But why do we need to define this terminology? Because it will keep us in some boundaries. If we don’t follow the terminology, we can get too far from a designed concept which may result in many ambiguous situations. Let’s see some practical examples:

 struct Number
 {
     public static Number Parse(string source)
     {
         // ...
         if(!parsed)
         {
             throw new ParsingException();
         }
         // ...
     }

     public static bool TryParse(string source, out Number result)
     {
        // ..
        return parsed;
     }
 }

This example seems a little strange, and it is for a reason. I made this code slightly artificial to show the importance of problems appearing in it. First, let’s look at the Parse method. Why should it throw an exception?

In the previous articles we discussed PostgreSQL indexing engine, the interface of access methods, and the following methods: hash indexes, B-trees, GiST, SP-GiST, GIN, and RUM. The topic of this article is BRIN indexes.

BRIN

General concept

Unlike indexes with which we've already got acquainted, the idea of BRIN is to avoid looking through definitely unsuited rows rather than quickly find the matching ones. This is always an inaccurate index: it does not contain TIDs of table rows at all.

Simplistically, BRIN works fine for columns where values correlate with their physical location in the table. In other words, if a query without ORDER BY clause returns the column values virtually in the increasing or decreasing order (and there are no indexes on that column).

This access method was created in scope of Axle, the European project for extremely large analytical databases, with an eye on tables that are several terabyte or dozens of terabytes large. An important feature of BRIN that enables us to create indexes on such tables is a small size and minimal overhead costs of maintenance.

This works as follows. The table is split into ranges that are several pages large (or several blocks large, which is the same) — hence the name: Block Range Index, BRIN. The index stores summary information on the data in each range. As a rule, this is the minimal and maximal values, but it happens to be different, as shown further. Assume that a query is performed that contains the condition for a column; if the sought values do not get into the interval, the whole range can be skipped; but if they do get, all rows in all blocks will have to be looked through to choose the matching ones among them.

It will not be a mistake to treat BRIN not as an index, but as an accelerator of sequential scan. We can regard BRIN as an alternative to partitioning if we consider each range as a «virtual» partition.

Now let's discuss the structure of the index in more detail.

I notice that people often use construction like this:

var length = array.Length;
for (int i = 0; i < length; i++) {
    //do smth
}

They think that having a call to the Array.Length on each iteration will make CLR to take more time to execute the code. To avoid it they store the length value in a local variable.
Let’s find out (once and for all !) if this is a viable thing or using a temporary variable is a waste of time.

MessageBox — useful window for different GUI frameworks, but you can't find it in AvaloniaUI.
Let's try to do it.

February 2019 marked the release of ReactiveUI 9 — the cross-platform framework for building GUI applications on the Microsoft .NET platform. ReactiveUI is a tool for tight integration of reactive extensions with the MVVM design pattern. You could familiarize yourself with the framework via a series of videos or the welcome page of the documentation. The ReactiveUI 9 update includes numerous fixes and improvements, but probably the most crucial and interesting one is integration with the DynamicData framework, allowing you to work with dynamic collections in Reactive fashion. Let’s find out what we can use DynamicData for and how this powerful reactive framework works under the hood!

What is my advice for people who want to become game developers? After 25 years in the industry, I do have some advice to offer.

Across all of Microsoft, we are focusing on empowering developers to build better apps, faster. One way we are accomplishing that is by providing a range of products and services covering all stages of the software development lifecycle. This includes IDEs and DevOps tools, application and data platforms on the cloud, operating systems, Artificial Intelligence and IoT solutions, and more. All of these are centered around developers, both as individuals working in teams and organizations, and as members of developer communities.

GitHub is one of the largest developer communities, and for millions of developers around the world their GitHub identity has become a critical aspect of their digital life. Recognizing that, we’re excited to announce improvements that will help GitHub users get started more easily with our developer services, including Azure DevOps and Azure.

This week we issue a really huge selection of all the essential news you might have missed on Front-end development. Make sure to check the latest updates on CSS, HTML, and JavaScript.

In 2016, I came across Oliver Byrne's “The first six books of the Elements of Euclid.” The main feature of this book is that instead of ordinary letter designations such as “triangle ABC,” it employs inclusions of miniature pictures directly in the text, that is, for example, an image of a triangle. As difficult as it probably was in the XIX century, as easy, with the right tools, it should be to make such a book nowadays. And so I decided to find out by myself whether that's the case.

Hi everyone!

I like Tox and respect the participants of this project and their work. In an effort to help Tox developers and users, I looked into the code and noticed potential problems that could lead to a false sense of security. Since I originally published this article in 2016 (in Russian), many improvements have been made to Tox, and I lead a team that re-wrote secure Tox software from scratch using the Rust programming language (check out Tox-rs). I DO recommend using tox in 2019. Let's take a look what actually made us rewrite Tox in Rust.

Original article of 2016

There is an unhealthy tendency to overestimate the security of E2E systems only on the basis that they are E2E. I will present objective facts supplemented with my own comments for you to draw your own conclusions.

Spoiler: The Tox developers agree with my points and my source code pull request was accepted.

Hi! In this article, I want to explain, in layman’s terms, how steal appears in VMs and tell you about some of the less-than-obvious artifacts that we found during research on the topic that I was involved in as CTO of the Mail.ru Cloud Solutions platform. The platform runs KVM.

Last week, Microsoft held its Build 2019 conference at the Washington State Convention Center in Seattle. Build is a large event with several thousand people from around the world attending to learn all about the current, newest, and future developer-oriented tech coming from Microsoft.

We had the pleasure of meeting so many of you at our booth and answering all your questions!

TL;DR: starting February 2020, DNS servers that don’t support DNS both over UDP and TCP may stop working.

Bangkok, in general, is a strange place to stay. Of course, it is warm there, rather cheap and some might find the cuisine interesting, along with the fact that about half of the world’s population does not need to apply for a visa in advance to get there. However, you still need to get acquainted with the smells, and the city streets are casting cyberpunk scenes more than anything else.

In particular, a photo to the left has been taken not far from the center of Thailand’ capital city, one street away from the Shangri-La hotel, where the 30th DNS-OARC organization meeting took place on May 12 and 13. It is a non-profit organization dedicated to security, stability, and overall development of the DNS — the Domain Name System.

Slides from the DNS-OARC 30 meeting are recommended for everyone interested in how the DNS works, though perhaps the most interesting is what is absent in those slides. Namely, a 45-minute round table with a discussion around the results of DNS Flag Day 2019, which occurred on February, 1, 2019.

And, the most impressive result of a round table is the decision to repeat DNS Flag Day once again.

Hi guys,

Before you is an article guide to open data sets for machine learning. In it, I, for a start, will collect a selection of interesting and fresh (relatively) datasets. And as a bonus, at the end of the article, I will attach useful links on independent search of datasets.

Less words, more data.

A selection of datasets for machine learning:

• Data deaths and battles from the game of thrones — This data set combines three data sources, each based on information from a series of books.
• Global Terrorism Database — Over 180,000 terrorist attacks worldwide, 1970-2017.
• Bitcoin, historical data — Bitcoin data with an interval of 1 minute from selected exchanges, January 2012 — March 2019

This week I spent coding my very first public pet-app based on Telegram chat bot which acts as a Bitcoin wallet and allows to send and receive tips between Telegram users and other so-called “Lightning Apps”. I assume that you are familiar with Bitcoin & Telegram in general, i’ll try to post short and without deep jump into details. More resources about Bitcoin can be found here and Telegram is simply an instant messenger that allows you to create your custom apps (chat-bots) using their platform.

What are the key points of such app?

• Allows to rate other users ideas and answers with real value instead of
  ‘virtual likes’. This brings online conversation to completely new level
• Real example of working micro-payment app which can act with other entities
  over internet using open protocol
• All the modules are open-source projects and can be easy re-used and adjusted
  for your own project. App does not relay on third-party commercial services.
  Even it falls under e-commerce field, which is currently almost closed, the app
  is based on open solutions.

What are the use-cases?

something like this…

We have already discussed PostgreSQL indexing engine, the interface of access methods, and main access methods, such as: hash indexes, B-trees, GiST, SP-GiST, and GIN. In this article, we will watch how gin turns into rum.

RUM

Although the authors claim that gin is a powerful genie, the theme of drinks has eventually won: next-generation GIN has been called RUM.

This access method expands the concept that underlies GIN and enables us to perform full-text search even faster. In this series of articles, this is the only method that is not included in a standard PostgreSQL delivery and is an external extension. Several installation options are available for it:

• Take «yum» or «apt» package from the PGDG repository. For example, if you installed PostgreSQL from «postgresql-10» package, also install «postgresql-10-rum».
• Build from source code on github and install on your own (the instruction is there as well).
• Use as a part of Postgres Pro Enterprise (or at least read the documentation from there).

Limitations of GIN

What limitations of GIN does RUM enable us to transcend?

First, «tsvector» data type contains not only lexemes, but also information on their positions inside the document. As we observed last time, GIN index does not store this information. For this reason, operations to search for phrases, which appeared in version 9.6, are supported by GIN index inefficiently and have to access the original data for recheck.

Second, search systems usually return the results sorted by relevance (whatever that means). We can use ranking functions «ts_rank» and «ts_rank_cd» to this end, but they have to be computed for each row of the result, which is certainly slow.

To a first approximation, RUM access method can be considered as GIN that additionally stores position information and can return the results in a needed order (like GiST can return nearest neighbors). Let's move step by step.