Development

This article is the story how we once decided to improve our internal SelfTester tool that we apply to test the quality of the PVS-Studio analyzer. The improvement was simple and seemed to be useful, but got us into some troubles. Later it turned out that we'd better gave up the idea.

A developer, who first encountered generating emails, has almost no chance to write an application, that will do it correctly. Around 40% of emails, generated by corporate applications, are violating some form of standard, and due to this, there are problems with delivery and display. There are reasons for this: emails are technically more difficult than the web, and operating emails is regulated by a few hundred standards, as well as an uncountable number of generally accepted (and not as much) practices, whereas the email clients are more varied and unpredictable than browsers. Testing may significantly improve the situation, but materials that are dedicated to testing the email system, are practically non-existent.

Mail.ru regularly interacts with its users by email. In our projects, all the components responsible for generating emails and even individual mailings, are subject to mandatory testing. In this article, we will share our experience (learning from our mistakes).

Nowadays, talking to conversational agents is becoming a daily routine, and it is crucial for dialogue systems to generate responses as human-like as possible. As one of the main aspects, primary attention should be given to providing emotionally aware responses to users. In this article, we are going to describe the recurrent neural network architecture for emotion detection in textual conversations, that participated in SemEval-2019 Task 3 “EmoContext”, that is, an annual workshop on semantic evaluation. The task objective is to classify emotion (i.e. happy, sad, angry, and others) in a 3-turn conversational data set.

Nowadays there is no need to implement the physics of objects from scratch for game development because there are a lot of libraries for this purpose. Bullet was actively used in many AAA games, virtual reality projects, various simulations and machine learning. And it is still used, being, for example, one of the Red Dead Redemption and Red Dead Redemption 2 engines. So why not check the Bullet with PVS-Studio to see what errors static analysis can detect in such a large-scale physics simulation project.

Hi everybody! I’m a research engineer at the Mail.ru Group computer vision team. In this article, I’m going to tell a story of how we’ve created AI-based photo restoration project for old military photos. What is «photo restoration»? It consists of three steps:

• we find all the image defects: fractures, scuffs, holes;
  
• we inpaint the discovered defects, based on the pixel values around them;
  
• we colorize the image.
  

Further, I’ll describe every step of photo restoration and tell you how we got our data, what nets we trained, what we accomplished, and what mistakes we made.

Popularity of mobile applications continues to grow. So does OAuth 2.0 protocol on mobile apps. It's not enough to implement standard as is to make OAuth 2.0 protocol secure there. One needs to consider the specifics of mobile applications and apply some additional security mechanisms.

In this article, I want to share the concepts of mobile OAuth 2.0 attacks and security mechanisms used to prevent such issues. Described concepts are not new but there is a lack of the structured information on this topic. The main aim of the article is to fill this gap.

RUVDS and Habr continue the series of interviews with interesting people in IT field. Last time we talked to Richard «Levelord» Gray, level designer of popular games Duke Nukem, American McGee’s Alice, Heavy Metal F.A.K.K.2, SiN, Serious Sam, author of well-known «You’re not supposed to be here» phrase.

Today we welcome Randall Steward «Randy» Pitchford II, president, CEO and co-founder of Gearbox Software video game development company.

Randy started in 3D Realms where contributed to Duke Nukem 3D Atomic Edition and Shadow Warrior. Then he founded Gearbox Software and made Half-Life: Opposing Force, which won D.I.C.E in 2000. Other Gearbox titles include Half-Life: Blue Shift, Half-Life: Decay, Counter-Strike: Condition Zero, James Bond 007: Nightfire, Tony Hawk's Pro Skater 3, Halo: Combat Evolved and of course Borderlands.

The interview team also includes editor of Habr Nikolay Zemlyanskiy, Richard «Levelord» Gray, Randy’s wife Kristy Pitchford and Randy’s son Randy Jr.

It is a new selection of tips and tricks about Python and programming from my Telegram-channel @pythonetc.

← Previous publications

Support of Visual Studio 2019 in PVS-Studio affected a number of components: the plugin itself, the command-line analyzer, the cores of the C++ and C# analyzers, and a few utilities. In this article, I will briefly explain what problems we encountered when implementing support of the IDE and how we addressed them.

What is my advice for people who want to become game developers? After 25 years in the industry, I do have some advice to offer.

In 2016, I came across Oliver Byrne's “The first six books of the Elements of Euclid.” The main feature of this book is that instead of ordinary letter designations such as “triangle ABC,” it employs inclusions of miniature pictures directly in the text, that is, for example, an image of a triangle. As difficult as it probably was in the XIX century, as easy, with the right tools, it should be to make such a book nowadays. And so I decided to find out by myself whether that's the case.

Everyone uniquely perceives texts, regardless of whether this person reads news on the Internet or world-known classic novels. This also applies to a variety of algorithms and machine learning techniques, which understand texts in a more mathematical way, namely, using high-dimensional vector space.

This article is devoted to visualizing high-dimensional Word2Vec word embeddings using t-SNE. The visualization can be useful to understand how Word2Vec works and how to interpret relations between vectors captured from your texts before using them in neural networks or other machine learning algorithms. As training data, we will use articles from Google News and classical literary works by Leo Tolstoy, the Russian writer who is regarded as one of the greatest authors of all time.

We go through the brief overview of t-SNE algorithm, then move to word embeddings calculation using Word2Vec, and finally, proceed to word vectors visualization with t-SNE in 2D and 3D space. We will write our scripts in Python using Jupyter Notebook.

Once in a while we go back to the projects that we have previously checked using PVS-Studio, which results in their descriptions in various articles. Two reasons make these comebacks exciting for us. Firstly, the opportunity to assess the progress of our analyzer. Secondly, monitoring the feedback of the project's authors to our article and the report of errors, which we usually provide them with. Of course, errors can be corrected without our participation. However, it is always nice when our efforts help to make a project better. Roslyn was no exception. The previous article about this project check dates back to December 23, 2015. It's quite a long time, in the view of the progress that our analyzer has made since that time. Since the C# core of the PVS-Studio analyzer is based on Roslyn, it gives us additional interest in this project. As a result, we're as keen as mustard about the code quality of this project. Now let's test it once again and find out some new and interesting issues (but let's hope that nothing significant) that PVS-Studio will be able to find.

As many of our readers know, Qrator.Radar is constantly researching global BGP connectivity, as well as regional. Since the Internet stands for “Interconnected Networks,” to ensure the best possible quality and speed the interconnectivity of individual networks should be rich and diverse, with their growth motivated on a sound competitive basis.

The fault-resistance of an internet connection in any given region or country is tied to the number of alternate routes between ASes. Though, as we stated before in our Internet Segments Reliability reports, some paths are obviously more critical compared to the others (for example, the paths to the Tier-1 transit ISPs or autonomous systems hosting authoritative DNS servers), which means that having as many reachable routes as possible is the only viable way to ensure adequate system scalability, stability and robustness.

This time, we are going to have a closer look at the Russian Federation internet segment. There are reasons to keep an eye on that segment: according to the numbers provided by the RIPE database, there are 6183 autonomous systems in Russia, out of 88664 registered worldwide, which stands for 6.87% of total.

This percentage puts Russia on a second place in the world, right after the USA (30.08% of registered ASes) and before Brazil, owning 6.34% of all autonomous systems. Effects of changes in the Russian connectivity could be observed across many other countries dependant on or adjacent to that connectivity, and ultimately by almost any ISP in the world.

A completely routine tech support ticket has uncovered unexpected bans of IP addresses of Protonmail — a very useful service for people valuing their Internet freedoms — in several regions of Russia. I seriously didn’t want to sensationalize the headline, but the story is so strange and inexplicable I couldn’t resist.

TL;DR

Disclaimer: the situation is still developing. There might not be anything malicious, but most likely there is. I will update the post once new information comes through.

MTS and Rostelecom — two of the biggest Russian ISPs — started to block traffic to SMTP servers of the encrypted email service Protonmail according to an FSB request, with no regard for the official government registry of restricted websites. It seems like it’s been happening for a while, but no one paid special attention to it. Until now.

All involved parties have received relevant requests for information which they’re obligated to reply.

UPD: MTS has provided a scan of the FSB letter, which is the basis for restricting the access. Justification: the ongoing Universiade in Krasnoyarsk and “phone terrorism”. It’s supposed to prevent ProtonMail emails from going to emergency addresses of security services and schools.

UPD: Protonmail was surprised by “these strange Russians” and their methods for battling fraud abuse, as well as suggested a more effective way to do it — via abuse mailbox.

UPD: FSB’s justification doesn’t appear to be true: the bans broke ProtonMail’s incoming mail, rather than outgoing.

UPD: Protonmail shrugged and changed the IP addresses of their MXs taking them out of the blocking after that particular FSB letter. What will happen next is open ended question.

UPD: Apparently, such letter was not the only one and there is still a set of IP addresses of VOIP-services which are blocked without appropriate records in the official registry of restricted websites.

A few days ago, Microsoft made the source code of their Windows Calculator publicly available. Calculator is an application that has traditionally shipped with every Windows version. A number of Microsoft projects went open-source over the recent years, but this time the news was covered even by non-IT media on the very first day. Well, it's a popular yet tiny program in C++. Despite its size, we still managed to find a number of suspicious fragments in its code using the PVS-Studio static analyzer.

RUVDS together with Habr.com continues the series of interviews with interesting people in computer field. Previously we met Boris Yangel, who heads AI development of Yandex’s Alice voice assistant.

Today we bring you an interview with Richard (Levelord) Gray — level designer of such legendary games as Duke Nukem, American McGee Alice, Heavy Metal F.A.K.K.2, SiN, and Serious Sam. And he is the one who coined the famous phrase «You are not supposed to be here». Richard was born and spent most of his life in USA, but several years ago he moved to Moscow to his russian wife and daughter.

These who speak to Richard are Nick Zemlyanskiy, editor of Habr.com, and Nikita Tsaplin, co-founder and managing partner of RUVDS company.


→ Text and video in Russian

TL;DR

Using a custom build script in Node JS, it is possible to manipulate a series of Sketch files, and then, using an internal Sketch tool, automatically export their assets, to generate multiple icon libraries, for multiple platforms and different brands, that support dynamic colourisation of the assets via design tokens, and also AB testing of the assets via naming convention. Easy peasy :)

---

Well, actually it’s not that easy, but it can certainly be done. This post is a detailed explanation of how we did it, and what we discovered along the way.

The problem we were trying to solve

At Badoo we build a dating app. Actually, multiple dating apps. For multiple platforms (iOS, Android, Mobile Web, Desktop Web), across multiple teams.

We use hundreds of icons in our apps. Some of them are the same across different apps, some are very specific to the brands the apps reflect. The icons are continuously evolving, in sync with the evolution of the design. Sometimes completely new icons are added, while others get updated, and still others get dropped (although, they often remain in the codebase).

The Object base type and implementation of interfaces. Boxing

It seems we came through hell and high water and can nail any interview, even the one for .NET CLR team. However, let's not rush to microsoft.com and search for vacancies. Now, we need to understand how value types inherit an object if they contain neither a reference to SyncBlockIndex, not a pointer to a virtual methods table. This will completely explain our system of types and all pieces of a puzzle will find their places. However, we will need more than one sentence.

Now, let's remember again how value types are allocated in memory. They get the place in memory right where they are. Reference types get allocation on the heap of small and large objects. They always give a reference to the place on the heap where the object is. Each value type has such methods as ToString, Equals and GetHashCode. They are virtual and overridable, but don’t allow to inherit a value type by overriding methods. If value types used overridable methods, they would need a virtual methods table to route calls. This would lead to the problems of passing structures to unmanaged world: extra fields would go there. As a result, there are descriptions of value type methods somewhere, but you cannot access them directly via a virtual methods table.

This may bring the idea that the lack of inheritance is artificial

This chapter was translated from Russian jointly by author and by professional translators. You can help us with translation from Russian or English into any other language, primarily into Chinese or German.

Also, if you want thank us, the best way you can do that is to give us a star on github or to fork repository github/sidristij/dotnetbook.

I once had a discussion with a founder of an Israeli startup developing a GPU-based database with a focus on speed. The work stack included Haskell and C++, among others, and the founder was complaining about how hard it is to find competent programmers. Which was part of the reason he came to Moscow.

I carefully asked if they considered using something more popular and new. And even though the answer was rather polite and well-supported with arguments, it still sounded like “Come on, don’t even bring up these toys”.

Until then, all I heard about Haskell could be summarized as “be VERY careful in dealing with it”. To get to know Haskell programmers better, I came to a topical Telegram chat with some questions. I was quite afraid at first, and, as it turned out, I was right.

Haskell doesn’t lend itself to popular explanation, and people seemingly don’t even try. If the topic is ever brought up, it’s only talked about in full depth and as objectively as possible. Someone wrote to me: “One of the defining features of both Haskell itself and its community is that they didn’t try to achieve any kind of mainstream recognition. Instead, they focused on building a logical, principal way of solving real problems over trying to appease the widest audience possible”

Nevertheless, a couple of people did tell me about their experiences, which are shown below.