• External Interrupts in the x86 system. Part 3. Interrupt routing setup in a chipset, with the example of coreboot

    • Tutorial

    We continue to investigate external device interrupt routing setup in the x86 system.


    In Part 1 (Interrupt controller evolution) we looked at the theory behind interrupt controllers and all the necessary terminology. In Part 2 (Linux kernel boot options) we looked at how in practice the OS chooses between different interrupt controllers. In this part we will investigate how the BIOS sets IRQ to the interrupt controllers routing in a chipset.


    None of the modern BIOS developer companies (AwardBIOS/AMIBIOS/Insyde) open their source code. But luсkily there is coreboot — a project aimed at replacing proprietary BIOS with free firmware code. In its source code we'll see what is needed to setup the interrupt routing in a chipset.



    Read more →
  • Porting packages to buildroot using the Zabbix example

    • Tutorial


    The basics of porting


    Originally, Buildroot offers a limited number of packages. It makes sense — there is everything you need, but any other packages can be added.


    To add a package, create 2 description files, an optional checksum file, and add a link to the package in the general package list. There are hooks at different stages of the build. At the same time, Buildroot can recognize the needed type of packages:

    Read more →
  • How I fix cups-printing in Buildroot

      image


      Intro


      Like I said earlier in previos articles, Buildroot is a great system for embedded Linux development. But sometimes strange things can happen.


      Once upon a workday, I got the following task: add printing system in firmware (Kraftway terminal Linux next generation). Ok, so I had to add cups + cups filter and to build firmware. I set a postscript-printer and got an error "Filter failed". Trivial tasks turned into serious work.


      In this article, I wrote my own way of solving this problem. It may be useful for other developers and IT-specialist and, also, for a deeper understanding of the Buildroot.


      If you are a Buildroot beginner, I recommend reading my previous articles.


      Update 1 may 2020


      Revisioned versions of this patches applied to master.

      Read more →
    • A Second Check of Newton Game Dynamics with PVS-Studio

        Рисунок 1

        Some time ago, somewhere on the Internet, I stumbled upon a physics engine called Newton Game Dynamics. Knowing that engine projects are usually big and complex, I decided to check its code with PVS-Studio for any interesting defects. I was especially enthusiastic about this one because my co-worker Andrey Karpov already checked it in 2014 and a second check would be a good opportunity to demonstrate our analyzer's evolution over the past six years. As of this writing, the latest version of Newton Game Dynamics is dated February 27, 2020, which means it has been actively developing for the past six years too. So, hopefully, this article will be interesting not only to us but to the engine's developers as well – and for them it's a chance to fix some bugs and improve their code.
        Read more →
      • Waking up of the sleeping institution: how we taking off training-wheels from Python in REAPER

        • Translation

        Image for taking attention


        About a week I wonder around the idea of this article, mainly, because of missing the content here and tidiness of the coronavirus-themes. But, when robotic-harvester, hacking zip with bitcoins and other cool articles released — I decided not to publish crude material.


        However, unexpectedly today the maintainer of this review hero showed up from the lockdown and, several hours ago reapy v0.6.0 has been published on PyPi. Under the cut — the last change log, which contains (to my pleasure) no line where I have not been involved one way, or another.


        Finally: why reapy is needed and how Python works inside REAPER.

        Read more →
      • Checking the GCC 10 Compiler with PVS-Studio

          PVS-Studo vs GCC 10

          The GCC compiler is written with copious use of macros. Another check of the GCC code using PVS-Studio once again confirms the opinion of our team that macros are evil in the flesh. Not only does the static analyzer struggle with reviewing such code, but also a developer. GCC developers are certainly used to the project and are well versed in it. Nonetheless, it is very difficult to understand something on the third hand. Actually, due to macros, it was not possible to fully perform code checking. However, the PVS-Studio analyzer, as always, showed that it can find errors even in compilers.
          Read more →
        • How does strange code hide errors? TensorFlow.NET Analysis

            PVS-Studio and TensorFlow.NET

            Static analysis is an extremely useful tool for any developer, as it helps to find in time not only errors, but also suspicious and strange code fragments that may cause bewilderment of programmers who will have to work with it in the future. This idea will be demonstrated by the analysis of the TensorFlow.NET open C# project, developed for working with the popular TensorFlow machine learning library.
            Read more →
          • AdBlock has stolen the banner, but banners are not teeth — they will be back

            More
            Ads
          • Checking the Code of Zephyr Operating System

              PVS-Studio and Zephyr

              Some time ago we announced PVS-Studio's new feature that enabled it to integrate into PlatformIO. Naturally, our team kept in touch with the PlatformIO team while working on that feature, and they suggested that we check the real-time operating system Zephyr to see if we could find any interesting bugs in its code. We thought it was a good idea, and so here's this article about the check results.
              Read more →
            • OpenToonz: inside and out

                Picture 1

                Almost four years have passed since the PVS-Studio team checked the OpenToonz source code. This project is a very powerful tool for creating two-dimensional animation. Since the last check, with its help, such animated works as Mary and the Witch Flower, Batman-Ninja, Promare and others were created. As large studios continue using Toonz, why not check the quality of the source code again?
                Read more →
              • GitHub repositories tracker and its «for convenience» tricks

                Work in several GitHub repositories (repos) brings inconveniences. Repos are separated from each other, open and closed issues live in different lists, linked pull requests (PRs) can't be seen until opening an issue — our team works in more than 70 repos, so I learned that hard. And started to write repos tracker on Python. It's close to the final, I'd like to share it (it's completely free), and few tricks I used in progress.

                What is that


                Scraper tracks several GitHub repos in a single Google Sheet (GS) table. You can see all of the opened and done issues, related PRs, priorities, your teammates comments, use coloring, filtering, sorting and other GS functions. That's a good integration, here is how it looks:

                image

                How does it work, shortly


                There is Spreadsheet() class which contain several Sheet() objects, each of which have it's own configurations, described in config.py. When Scraper updates a sheet, it loads configurations, sees list of repos to track, requests info from GitHub, builds a table and sends it to GS service. Sounds easy, but there were several tough to deal with things, which I've solved mostly with support of my work experience in Google projects, and which I consider as good patterns. Take a pen.
                Read more →
              • buildroot — my own experience with multi-platform distro creation

                  Introduction


                  In my previous article (Monitor linux) I wrote, what is this distro and how it works. Now i will write how to do it. It's may be interesting for everyone, who want to study buildroot.


                  Target goals


                  The result we get from article is the following:


                  • Firmware (non-volatile image with restorable config)
                  • Easy management via web-interface
                  • Cross-platform (qemu x86_64, arm-based SBC like rasberry 4, beagle bone black and asus tinker board)
                  • Support without extra effort
                  Read more →
                • PVS-Studio Is Now in Chocolatey: Checking Chocolatey under Azure DevOps

                    Рисунок 1

                    We continue making the use of PVS-Studio more convenient. Our analyzer is now available in Chocolatey, the package manager for Windows. We believe this will make it easier to deploy PVS-Studio, particularly in cloud services. So right off the bat, we also checked the source code of the same Chocolatey. Azure DevOps took on the role of the CI system.
                    Read more →
                  • Monitor linux — cross platform firmware with zabbix server

                      About


                      This is small cross-platform linux-distro with zabbix server. It's a simple way to deploy powerful monitoring system on ARM platfornms and x86_64.


                      Worked as firmware (non-changeable systemd image with config files), have web-interface for system management like network settings, password and other.


                      Who is interested


                      • System admins/engineers who need to fast deploy of zabbix server.
                      • Everyone, who want to deploy zabbix on ARM.
                      • Enthusiasts
                      Read more →
                    • A Brief Comparison of the SDS Architectures for Virtualization

                      • Translation

                      The search for a suitable storage platform: GlusterFS vs. Ceph vs. Virtuozzo Storage


                      This article outlines the key features and differences of such software-defined storage (SDS) solutions as GlusterFS, Ceph, and Virtuozzo Storage. Its goal is to help you find a suitable storage platform.

                      Gluster



                      Let’s start with GlusterFS that is often used as storage for virtual environments in open-source-based hyper-converged products with SDS. It is also offered by Red Hat alongside Ceph.
                      GlusterFS employs a stack of translators, services that handle file distribution and other tasks. It also uses services like Brick that handle disks and Volume that handle pools of bricks. Next, the DHT (distributed hash table) service distributes files into groups based on hashes.
                      Note: We’ll skip the sharding service due to issues related to it, which are described in linked articles.

                      image

                      When a file is written onto GlusterFS storage, it is placed on a brick in one piece and copied to another brick on another server. The next file will be placed on two or more other bricks. This works well if the files are of about the same size and the volume consists of a single group of bricks. Otherwise the following issues may arise:
                      Read more →
                    • Developing a symbolic-expression library with C#. Differentiation, simplification, equation solving and many more

                        Hello!

                        Why does programming a calculator seem to be a task, which every beginner undertakes? History might have the answer — computers were created for this exact purpose. Unlike the beginners, we will develop a smart calculator, which, although won't reach the complexity of SymPy, will be able to perform such algebraic operations as differentiation, simplification, and equations solving, will have built-in latex support, and have implemented features such as compilation to speed up the computations.

                        What are the articles about?
                        It will superficially tell about assembling an expression, parsing from a string, variable substitution, analytic derivative, equation numerical solving, and definite integration, rendering to LaTeX format, complex numbers, compiling functions, simplifying, expanding brackets, and blah blah blah.
                        For those who urgently need to clone something, repository link.

                        Let's do it!
                        Read more →
                      • Introducing One Ring — an open-source pipeline for all your Spark applications

                          If you utilize Apache Spark, you probably have a few applications that consume some data from external sources and produce some intermediate result, that is about to be consumed by some applications further down the processing chain, and so on until you get a final result.


                          We suspect that because we have a similar pipeline with lots of processes like this one:


                          A process flowchart with more than 50 applications and about 70 datasets
                          Click here for a bit larger version


                          Each rectangle is a Spark application with a set of their own execution parameters, and each arrow is an equally parametrized dataset (externally stored highlighted with a color; note the number of intermediate ones). This example is not the most complex of our processes, it’s fairly a simple one. And we don’t assemble such workflows manually, we generate them from Process Templates (outlined as groups on this flowchart).


                          So here comes the One Ring, a Spark pipelining framework with very robust configuration abilities, which makes it easier to compose and execute a most complex Process as a single large Spark job.


                          And we just made it open source. Perhaps, you’re interested in the details.

                          We got you covered!
                        • Five Methods For Database Obfuscation

                            ClickHouse users already know that its biggest advantage is its high-speed processing of analytical queries. But claims like this need to be confirmed with reliable performance testing. That's what we want to talk about today.



                            We started running tests in 2013, long before the product was available as open source. Back then, just like now, our main concern was data processing speed in Yandex.Metrica. We had been storing that data in ClickHouse since January of 2009. Part of the data had been written to a database starting in 2012, and part was converted from OLAPServer and Metrage (data structures previously used by Yandex.Metrica). For testing, we took the first subset at random from data for 1 billion pageviews. Yandex.Metrica didn't have any queries at that point, so we came up with queries that interested us, using all the possible ways to filter, aggregate, and sort the data.

                            ClickHouse performance was compared with similar systems like Vertica and MonetDB. To avoid bias, testing was performed by an employee who hadn't participated in ClickHouse development, and special cases in the code were not optimized until all the results were obtained. We used the same approach to get a data set for functional testing.

                            After ClickHouse was released as open source in 2016, people began questioning these tests.

                            Read more →
                          • Our Small Contribution to Avalonia UI's Fight for Fewer Platforms

                              Рисунок 2

                              This article is a review of the bugs found in the Avalonia UI project with the static analyzer PVS-Studio. Avalonia UI is an open-source cross-platform XAML-based UI framework. This is one of the most technologically significant projects in the history of .NET as it enables developers to create cross-platform interfaces based on the WPF system. We hope the project's authors will find this article helpful in fixing some of the bugs, and convincing enough to make static analysis part of their development process.
                              Read more →