• Citymobil — a manual for improving availability amid business growth for startups. Part 4



      This is the next article of the series describing how we’re increasing our service availability in Citymobil (you can read the previous parts here: part 1, part 2, part 3). In further parts, I’ll talk about the accidents and outages in detail.

      1. Bad release: database overload


      Let me begin with a specific example of this type of outage. We deployed an optimization: added USE INDEX in an SQL query; during testing as well as in production, it sped up short queries, but the long ones — slowed down. The long queries slowdown was only noticed in production. As a result, a lot of long parallel queries caused the database to be down for an hour. We thoroughly studied the way USE INDEX worked; we described it in the Do’s and Dont’s file and warned the engineers against the incorrect usage. We also analyzed the query and realized that it retrieves mostly historical data and, therefore, can be run on a separate replica for historical requests. Even if this replica goes down due to an overload, the business will keep running.
      Read more →
    • Legacy Outage

        Two days ago, May 5 of the year 2019 we saw a peculiar BGP outage, affecting autonomous systems in the customer cone of one very specific AS with the number 721.

        Right at the beginning, we need to outline a couple of details for our readers:

        1. All Autonomous System Numbers under 1000 are called “lower ASNs,” as they are the first autonomous systems on the Internet, registered by IANA in the early days (the late 80’s) of the global network. Today they mostly represent government departments and organizations, that were somehow involved in Internet research and creation in 70-90s.
        2. Our readers should remember, that the Internet became public only after the United States’ Department of Defense, which funded the initial ARPANET, handed it over to the Defense Communication Agency and, later in 1981, connected it to the CSNET with the TCP (RFC675)/IP (RFC791) over X.25. A couple of years later, in 1986, NSF swapped the CSNET in favor of NSFNET, which grew so fast it made possible ARPANET decommission by 1990.
        3. IANA was established in 1988, and supposedly at that time, existing ASNs were registered by the RIRs. It is no surprise that the organization that funded the initial research and creation of the ARPANET, further transferring it to another department because of its operational size and growth, only after diversifying it into 4 different networks (Wiki mentions MILNET, NIPRNET, SIPRNET and JWICS, above which the military-only NIPRNET did not have controlled security gateways to the public Internet).
        Read more →
      • Python in Visual Studio Code – April 2019 Release

          We are pleased to announce that the April 2019 release of the Python Extension for Visual Studio Code is now available. You can download the Python extension from the Marketplace, or install it directly from the extension gallery in Visual Studio Code. You can learn more about Python support in Visual Studio Code in the documentation.


          In this release we made a series of improvements that are listed in our changelog, closing a total of 84 issues including:


          • Variable Explorer and Data Viewer
          • Enhancements to debug configuration
          • Additional improvements to the Python Language Server

          Keep on reading to learn more!  


          Read more →
        • How to create a dark theme without breaking things: learning with the Yandex Mail team


            My name is Vladimir, and I develop mobile front-end for Yandex Mail. Our apps have had a dark theme for a while, but it was incomplete: only the interface and plain emails were dark. Messages with custom formatting remained light and stood out against the dark interface, hurting our users’ eyes at night.


            Today I'll tell you how we fixed this problem. You will learn about two simple techniques that didn't work for us and the method that finally did the trick — adaptive page recoloring. I'll also share some ideas about adapting images to a dark theme. To be fair, darkening pages with custom CSS is a rather peculiar task, but I believe some of you may find our experience helpful.

            Read more →
          • How to Develop A User-Friendly Application



            Mobile apps are necessary for every business today. They are the tools that bridges the gap between consumers and business, makes it easier for clients to view the trends, and offers an easy chance to the consumers to know the business in person. Having so many advantages already, why it is important to keep app’s user-friendliness as one of the most important points?

            There are a number of reasons for developing a user-friendly mobile application. The users today are more eager to have a superb experience while browsing through the application. And for the same reason the UI and the app UX holds prominence for every mobile app development company. No matter how successfully your app has been designed, it is important to offer a unique browsing experience to the users.
            Read more →
          • Build Visual Studio templates with tags, for efficient user search and grouping

              Visual Studio’s project templates enable you, the developer, to create multiple similar projects more efficiently by defining a common set of starter files. The project templates can be fully customized to meet the needs of a development team, or a group, and can be published to the Visual Studio Marketplace for others to download and use too! Once published, developers can install and access the template through Visual Studio’s New Project Dialog.


              The newly designed New Project Dialog for Visual Studio 2019 was built to help developers get to their code faster. Using a search and filter focused experience, we are aiming to provide better discoverability for specific templates to start your application development

               

              Read more →
            • What is a coding bootcamp?

              A coding bootcamp is a program of technical training teaching the programming skills that employers are looking for. Coding bootcamps allow students with low skills to concentrate on the most significant coding aspects and apply their new coding skills to solve real-world problems.

              The goal of many bootcamp coding attendants is to move into a web development career. They do this by learning to build applications at a professional level – providing the foundation they need to build applications that are ready for production and demonstrating the skills they have to add real value to a potential employer.
              Read more →
            • Quality as Team's responsibility. Our QA experience

              Disclaimer: This is a translation of an article. All rights belongs to author of original article and Miro company.


              I'm a QA Engineer in Miro. Let me tell about our experiment of transferring partially testing tasks to developers and of transforming Test Engineer role into QA (Quality assurance).


              First briefly about our development process. We have daily releases for client side and 3 to 5 weekly releases of server side. Team have 60+ people spitted onto 10 Functional Scrum Teams.


              I'm working in Integration team. Our tasks are:


              • Integration of our service into external products
              • Integration of external products into our service
                For example we have integrated Jira. Jira Cards — visual representation of tasks so it's useful to work with tasks not opening Jira at all.

                image

              How the experiment starts


              All starts with trivial issue. When someone of Test Engineers had sick leave then team performance was degraded significantly. Team was continued working on tasks. However when code was reached testing phase task was hold on. As a result new functionality didn't reach production in time.


              Going onto vacation by Test Engineer is a more complex story. He/she needs to find another Test Engineer who ready to take extra tasks and conduct knowledge sharing. Going onto vacation by two Test Engineers at the sane time is not an applicable luxury.

              Read more →
            • Google News and Leo Tolstoy: visualizing Word2Vec word embeddings using t-SNE


                Everyone uniquely perceives texts, regardless of whether this person reads news on the Internet or world-known classic novels. This also applies to a variety of algorithms and machine learning techniques, which understand texts in a more mathematical way, namely, using high-dimensional vector space.

                This article is devoted to visualizing high-dimensional Word2Vec word embeddings using t-SNE. The visualization can be useful to understand how Word2Vec works and how to interpret relations between vectors captured from your texts before using them in neural networks or other machine learning algorithms. As training data, we will use articles from Google News and classical literary works by Leo Tolstoy, the Russian writer who is regarded as one of the greatest authors of all time.

                We go through the brief overview of t-SNE algorithm, then move to word embeddings calculation using Word2Vec, and finally, proceed to word vectors visualization with t-SNE in 2D and 3D space. We will write our scripts in Python using Jupyter Notebook.

                Read more →
              • Hack the JWT Token

                • Tutorial

                For Educational Purposes Only! Intended for Hackers Penetration testers.

                Issue


                The algorithm HS256 uses the secret key to sign and verify each message. The algorithm RS256 uses the private key to sign the message and uses the public key for authentication.

                If you change the algorithm from RS256 to HS256, the backend code uses the public key as the secret key and then uses the HS256 algorithm to verify the signature. Asymmetric Cipher Algorithm => Symmetric Cipher Algorithm.

                Because the public key can sometimes be obtained by the attacker, the attacker can modify the algorithm in the header to HS256 and then use the RSA public key to sign the data.
                The backend code uses the RSA public key + HS256 algorithm for signature verification.

                Example


                Vulnerability appear when client side validation looks like this:

                const decoded = jwt.verify(
                   token,
                   publickRSAKey,
                   { algorithms: ['HS256'  , 'RS256'] }          //accepted both algorithms 
                )

                Lets assume we have initial token like presented below and " => " will explain modification that attacker can make:

                //header 
                {
                alg: 'RS256'                         =>  'HS256'
                }
                //payload
                {
                sub: '123',
                name: 'Oleh Khomiak',
                admin: 'false'                       => 'true'
                }

                The backend code uses the public key as the secret key and then uses the HS256 algorithm to verify the signature.
                Read more →
              • NodeMCU simple driver model (SDM) showcase: dynamic user interface

                image


                NodeMCU is an interactive firmware, which allows running Lua interpreter on the ESP8266 microcontroller (ESP32 support is in development). Alongside with all the regular hardware interfaces, it has WiFi module and SPIFFS file system.


                This article describes the new module for the NodeMCU — sdm. SDM stands for simple driver model and it provides device-driver model abstraction for the system. In the first part of this article we will discuss the model itself and in the second part will be a showcase of dynamically created web user interface using sdm with some commentaries.

                Read more →
              • Citymobil — a manual for improving availability amid business growth for startups. Part 3



                  This is the next article of the series describing how we’re increasing our service availability in Citymobil (you can read the previous parts here and here). In further parts, I’ll talk about the accidents and outages in detail. But first let me highlight something I should’ve talked about in the first article but didn’t. I found out about it from my readers’ feedback. This article gives me a chance to fix this annoying shortcoming.
                  Read more →
                • Blazor now in official preview

                    With this newest Blazor release we’re pleased to announce that Blazor is now in official preview! Blazor is no longer experimental and we are committing to ship it as a supported web UI framework including support for running client-side in the browser on WebAssembly.


                    A little over a year ago we started the Blazor experimental project with the goal of building a client web UI framework based on .NET and WebAssembly. At the time Blazor was little more than a prototype and there were lots of open questions about the viability of running .NET in the browser. Since then we’ve shipped nine experimental Blazor releases addressing a variety of concerns including component model, data binding, event handling, routing, layouts, app size, hosting models, debugging, and tooling. We’re now at the point where we think Blazor is ready to take its next step.


                    Blazor icon
                    Read more →
                  • Statistics and monitoring of PHP scripts in real time. ClickHouse and Grafana go to Pinba for help

                    • Tutorial
                    In this article I will explain how to use pinba with clickhouse and grafana instead of pinba_engine and pinboard.

                    On the php project pinba is probably the only reliable way to understand what is happening with performance. But usually people start to use pinba only when problems are already observed and it isn't clear where to look in.

                    Often developers have no idea how many RPS each script has. So they begin to optimize starting from places that seem to have problem.

                    Someone is analyzing the nginx logs, and someone is slow queries in the database.

                    Of course pinba would not be superfluous, but there are several reasons why it is not on every project.


                    Read more →
                  • Indexes in PostgreSQL — 7 (GIN)

                    • Translation
                    We have already got acquainted with PostgreSQL indexing engine and the interface of access methods and discussed hash indexes, B-trees, as well as GiST and SP-GiST indexes. And this article will feature GIN index.

                    GIN


                    «Gin?.. Gin is, it seems, such an American liquor?..»
                    «I'm not a drink, oh, inquisitive boy!» again the old man flared up, again he realized himself and again took himself in hand. «I am not a drink, but a powerful and undaunted spirit, and there is no such magic in the world that I would not be able to do.»

                    — Lazar Lagin, «Old Khottabych».

                    Gin stands for Generalized Inverted Index and should be considered as a genie, not a drink.
                    README
                    Read more →
                  • TLS 1.3 enabled, and why you should do the same



                      As we wrote in the 2018-2019 Interconnected Networks Issues and Availability Report at the beginning of this year, TLS 1.3 arrival is inevitable. Some time ago we successfully deployed the 1.3 version of the Transport Layer Security protocol. After gathering and analyzing the data, we are now ready to highlight the most exciting parts of this transition.

                      As IETF TLS Working Group Chairs wrote in the article:
                      “In short, TLS 1.3 is poised to provide a foundation for a more secure and efficient Internet over the next 20 years and beyond.”

                      TLS 1.3 has arrived after 10 years of development. Qrator Labs, as well as the IT industry overall, watched the development process closely from the initial draft through each of the 28 versions while a balanced and manageable protocol was maturing that we are ready to support in 2019. The support is already evident among the market, and we want to keep pace in implementing this robust, proven security protocol.

                      Eric Rescorla, the lone author of TLS 1.3 and the Firefox CTO, told The Register that:
                      “It's a drop-in replacement for TLS 1.2, uses the same keys and certificates, and clients and servers can automatically negotiate TLS 1.3 when they both support it,” he said. “There's pretty good library support already, and Chrome and Firefox both have TLS 1.3 on by default.”
                      Read more →