• Almost Perfect Libraries by Electronic Arts

      Our attention was recently attracted by the Electronic Arts repository on GitHub. It's tiny, and of the twenty-three projects available there, only a few C++ libraries seemed interesting: EASTL, EAStdC, EABase, EAThread, EATest, EAMain, and EAAssert. The projects themselves are tiny too (about 10 files each), so bugs were found only in the «largest» project of 20 files :D But we did find them, and they do look interesting! As I was writing this post, we were also having a lively discussion of EA games and the company's policy :D

      Picture 1

      Read more →
    • Best Copy-Paste Algorithms for C and C++. Haiku OS Cookbook

        Numerous typos and Copy-Paste code became the main topic of the additional article about checking the Haiku code by the PVS-Studio analyzer. Yet this article mostly tells about errors related to thoughtlessness and failed refactoring, rather than to typos. The errors found demonstrate how strong the human factor is in software development.

        Picture 1
        Read more →
      • How to shoot yourself in the foot in C and C++. Haiku OS Cookbook

          The story of how the PVS-Studio static analyzer and the Haiku OS code met goes back to the year 2015. It was an exciting experiment and useful experience for teams of both projects. Why the experiment? At that moment, we didn't have the analyzer for Linux and we wouldn't have it for another year and a half. Anyway, efforts of enthusiasts from our team have been rewarded: we got acquainted with Haiku developers and increased the code quality, widened our error base with rare bugs made by developers and refined the analyzer. Now you can check the Haiku code for errors easily and quickly.
          Picture 1

          Read more →
        • A declarative data-processing pipeline on top of actors? Why not?

            Some time ago, in a discussion on one of SObjectizer's releases, we were asked: "Is it possible to make a DSL to describe a data-processing pipeline?" In other words, is it possible to write something like that:


            A | B | C | D


            and get a working pipeline where messages are going from A to B, and then to C, and then to D. With control that B receives exactly that type that A returns. And C receives exactly that type that B returns. And so on.


            It was an interesting task with a surprisingly simple solution. For example, that's how the creation of a pipeline can look like:


            auto pipeline = make_pipeline(env, stage(A) | stage(B) | stage(C) | stage(D));

            Or, in a more complex case (that will be discussed below):


            auto pipeline = make_pipeline( sobj.environment(),
                    stage(validation) | stage(conversion) | broadcast(
                        stage(archiving),
                        stage(distribution),
                        stage(range_checking) | stage(alarm_detector{}) | broadcast(
                            stage(alarm_initiator),
                            stage( []( const alarm_detected & v ) {
                                    alarm_distribution( cerr, v );
                                } )
                            )
                        ) );

            In this article, we'll speak about the implementation of such pipeline DSL. We'll discuss mostly parts related to stage(), broadcast() and operator|() functions with several examples of usage of C++ templates. So I hope it will be interesting even for readers who don't know about SObjectizer (if you never heard of SObjectizer here is an overview of this tool).

            Read more →
          • Why LLVM may call a never called function?

              I don’t care what your dragon’s said, it’s a lie. Dragons lie. You don’t know what’s waiting for you on the other side.

              Michael Swanwick, The Iron Dragon’s Daughter
              This article is based on the post in the Krister Walfridsson’s blog, “Why undefined behavior may call a never called function?”.

              The article draws a simple conclusion: undefined behavior in a compiler can do anything, even something absolutely unexpected. In this article, I examine the internal mechanism of this optimization works.
              Read more →
            • Just take a look at SObjectizer if you want to use Actors or CSP in your C++ project


                A few words about SObjectizer and its history


                SObjectizer is a rather small C++ framework that simplifies the development of multithreaded applications. SObjectizer allows a developer to use approaches from Actor, Publish-Subscribe and Communicating Sequential Processes (CSP) models. It's an OpenSource project that is distributed under BSD-3-CLAUSE license.


                SObjectizer has a long history. SObjectizer itself was born in 2002 as SObjectizer-4 project. But it was based on ideas from previous SCADA Objectizer that was developed between 1995 and 2000. SObjectizer-4 was open-sourced in 2006, but its evolution was stopped soon after that. A new version of SObjectizer with the name SObjectizer-5 was started in 2010 and was open-sourced in 2013. The evolution of SObjectizer-5 is still in progress and SObjectizer-5 has incorporated many new features since 2013.


                SObjectizer is more or less known in the Russian segment of the Internet, but almost unknown outside of the exUSSR. It's because the SObjectizer was mainly used for local projects in exUSSR-countries and many articles, presentations, and talks about SObjectizer are in Russian.


                A niche for SObjectizer and similar tools


                Multithreading is used in Parallel computing as well as in Concurrent computing. But there is a big difference between Parallel and Concurrent computing. And, as a consequence, there are tools targeted Parallel computing, and there are tools for Concurrent computing, and they are different.

                Read more →
              • Dynamically generating robots.txt for ASP.NET Core sites based on environment

                  I'm putting part of older WebForms portions of my site that still run on bare metal to ASP.NET Core and Azure App Services, and while I'm doing that I realized that I want to make sure my staging sites don't get indexed by Google/Bing.


                  I already have a robots.txt, but I want one that's specific to production and others that are specific to development or staging. I thought about a number of ways to solve this. I could have a static robots.txt and another robots-staging.txt and conditionally copy one over the other during my Azure DevOps CI/CD pipeline.


                  Then I realized the simplest possible thing would be to just make robots.txt be dynamic. I thought about writing custom middleware but that sounded like a hassle and more code that needed. I wanted to see just how simple this could be.


                  Read more →
                • AdBlock has stolen the banner, but banners are not teeth — they will be back

                  More
                  Ads
                • How to speed up LZ4 decompression in ClickHouse?

                    When you run queries in ClickHouse, you might notice that the profiler often shows the LZ_decompress_fast function near the top. What is going on? This question had us wondering how to choose the best compression algorithm.

                    ClickHouse stores data in compressed form. When running queries, ClickHouse tries to do as little as possible, in order to conserve CPU resources. In many cases, all the potentially time-consuming computations are already well optimized, plus the user wrote a well thought-out query. Then all that's left to do is to perform decompression.



                    So why does LZ4 decompression becomes a bottleneck? LZ4 seems like an extremely light algorithm: the data decompression rate is usually from 1 to 3 GB/s per processor core, depending on the data. This is much faster than the typical disk subsystem. Moreover, we use all available CPU cores, and decompression scales linearly across all physical cores.
                    Read more →
                  • It's high time to become part of an open source project

                      JavaScript developers, I am working on an exciting opensource project pursuing two goals:

                      1. Learning best practices in JavaScript/NodeJS
                      2. Helping developers and myself to develop and launch MVPs to validate ideas quickly.

                      As developers, we have tons of ideas and would be awesome to have a simple tool to scaffold a secure project quickly, add a couple of forms, some project specific logic, and here you go — deploy and test your idea.
                      Read more →
                    • A drawing bot for realizing everyday scenes and even stories

                        Drawing bot


                        If you were asked to draw a picture of several people in ski gear, standing in the snow, chances are you’d start with an outline of three or four people reasonably positioned in the center of the canvas, then sketch in the skis under their feet. Though it was not specified, you might decide to add a backpack to each of the skiers to jibe with expectations of what skiers would be sporting. Finally, you’d carefully fill in the details, perhaps painting their clothes blue, scarves pink, all against a white background, rendering these people more realistic and ensuring that their surroundings match the description. Finally, to make the scene more vivid, you might even sketch in some brown stones protruding through the snow to suggest that these skiers are in the mountains.


                        Now there’s a bot that can do all that.

                        Read more →
                      • Long journey to Tox-rs. Part 1

                          Tox logo

                          Hi everyone!


                          I like Tox and respect the participants of this project and their work. In an effort to help Tox developers and users, I looked into the code and noticed potential problems that could lead to a false sense of security. Since I originally published this article in 2016 (in Russian), many improvements have been made to Tox, and I lead a team that re-wrote secure Tox software from scratch using the Rust programming language (check out Tox-rs). I DO recommend using tox in 2019. Let's take a look what actually made us rewrite Tox in Rust.


                          Original article of 2016


                          There is an unhealthy tendency to overestimate the security of E2E systems only on the basis that they are E2E. I will present objective facts supplemented with my own comments for you to draw your own conclusions.


                          Spoiler: The Tox developers agree with my points and my source code pull request was accepted.

                          Here go facts:
                        • How to make possible micro-payments in your app

                          This week I spent coding my very first public pet-app based on Telegram chat bot which acts as a Bitcoin wallet and allows to send and receive tips between Telegram users and other so-called “Lightning Apps”. I assume that you are familiar with Bitcoin & Telegram in general, i’ll try to post short and without deep jump into details. More resources about Bitcoin can be found here and Telegram is simply an instant messenger that allows you to create your custom apps (chat-bots) using their platform.


                          What are the key points of such app?


                          • Allows to rate other users ideas and answers with real value instead of
                            ‘virtual likes’. This brings online conversation to completely new level
                          • Real example of working micro-payment app which can act with other entities
                            over internet using open protocol
                          • All the modules are open-source projects and can be easy re-used and adjusted
                            for your own project. App does not relay on third-party commercial services.
                            Even it falls under e-commerce field, which is currently almost closed, the app
                            is based on open solutions.

                          What are the use-cases?


                          something like this…

                          image
                          Read more →
                        • Cataclysm Dark Days Ahead: Static Analysis and Roguelike Games

                            Picture 5

                            You must have already guessed from the title that today's article will be focusing on bugs in software source code. But not only that. If you are not only interested in C++ and in reading about bugs in other developers' code but also dig unusual video games and wonder what «roguelikes» are and how you play them, then welcome to read on!
                            Read more →
                          • Free Wireguard VPN service on AWS

                            • Translation
                            • Tutorial

                            Free Wireguard VPN service on AWS


                            The reasoning


                            The increase of Internet censorship by authoritarian regimes expands the blockage of useful internet resources making impossible the use of the WEB and in essence violates the fundamental right to freedom of opinion and expression enshrined in the Universal Declaration of Human Rights.


                            Article 19
                            Everyone has the right to freedom of opinion and expression; this right includes freedom to hold opinions without interference and to seek, receive and impart information and ideas through any media and regardless of frontiers.

                            The following is the detailed 6 steps instruction for non-IT people to deploy free* VPN service upon Wireguard technology in Amazon Web Services (AWS) cloud infrastructure, using a 12 months free account, on an Instance (virtual machine) run by Ubuntu Server 18.04 LTS.


                            I tried to make this walkthrough as friendly as possible to people far from IT. The only thing required is assiduity in repeating the steps described below.

                            Read more →
                          • Analyzing the Code of CUBA Platform with PVS-Studio


                              Java developers have access to a number of useful tools that help to write high-quality code such as the powerful IDE IntelliJ IDEA, free analyzers SpotBugs, PMD, and the like. The developers working on CUBA Platform have already been using all of these, and this review will show how the project can benefit even more from the use of the static code analyzer PVS-Studio.
                              Read more →
                            • Building VirtualBox for Windows

                              • Translation
                              • Tutorial

                               Intro


                              It is a well-known fact to many users of the Windows version of VirtualBox (from now on, VB; not to be confused with Visual Basic) that starting with 4.3.14 the developers added the so-called «hardening» designed to prevent malicious injections into VB. Although the intentions were good, the implementation happened to cause numerous conflicts with totally legitimate products such as antiviruses, cryptographic modules and even some updates of the Windows itself, and when such a conflict occurs VB simply stops working. Users have to wait for at least a month till the new VB version is released with the proper exclusions added. Worst case is, the conflicting application or update has to be uninstalled, or VB itself has to be downgraded to the version 4.3.12 which was the latest one without hardening. Numerous requests to add a user-controlled exclusion list, or an option to disable hardening, are all left unanswered. The only reply from developers sounds like «if you don't want it build it from source code yourself». Well, looks like we'll have to.

                              Although the build instructions are described on the official project Wiki, they are incomplete and somewhat outdated, while the build procedure often fails with vague error messages. So when, in the end, I got it working I thought it was worth documenting in full details in a separate article. This instruction is being updated from time to time, and at the moment it is adapted to building VB version 6.0.4. However, if you need information on building earlier versions of VB or auxiliary libraries you can always get it from the history of changes.

                              OK, I'm in
                            • Wireshark 3.x: code analysis under macOS and errors review

                                Picture 1

                                Wireshark Foundation released the final stable-version of the popular network traffic analyzer — Wireshark 3.0.0. The new release fixes several bugs, it is now possible to analyze the new protocols, apart from that the driver on Npcap WinPcap is replaced. Here is where quoting of the announcement ends and our note about bugs in the project starts off. The projects authors definitely haven't done their best in fixing bugs before the release.

                                Let's collect hotfixes right now to give a motive in doing a new release :).

                                Introduction


                                Wireshark is a well-known tool to capture and analyze network traffic. The program works with the vast majority of known protocols, has intuitive and logical graphical interface, an all-powerful system of filters. Wireshark is cross-platform, works in such OSs, as: Windows, Linux, macOS, Solaris, FreeBSD, NetBSD and many others.

                                To do the source code analysis, we used PVS-Studio static code analyzer. To analyze the source code, first we needed to compile the project in an OS. The choice was wide not only due to the cross platform nature of the project, but also because of that of the analyzer. I chose macOS for the analysis. You can also run the analyzer under Windows and Linux.
                                Read more →