• Managing hundreds of servers for load testing: autoscaling, custom monitoring, DevOps culture

      In the previous article, I talked about our load testing infrastructure. On average, we use about 100 servers to create a load, about 150 servers to run our service. All these servers need to be created, configured, started, deleted. To do this, we use the same tools as in the production environment to reduce the amount of manual work:

      • Terraform scripts for creating and deleting a test environment;
      • Ansible scripts for configuring, updating, starting servers;
      • In-house Python scripts for dynamic scaling, depending on the load.

      Thanks to the Terraform and Ansible scripts, all operations ranging from creating instances to starting servers are performed with only six commands:

      #launch the required instances in the AWS console
      ansible-playbook deploy-config.yml #update servers versions
      ansible-playbook start-application.yml #start our app on these servers
      ansible-playbook update-test-scenario.yml --ask-vault-pass #update the JMeter test scenario if it was changed
      infrastructure-aws-cluster/jmeter_clients:~# terraform apply #create JMeter servers for creating the load
      playbook start-jmeter-server-cluster.yml #start the JMeter cluster
      ansible-playbook start-stress-test.yml #start the test
      

      Read more →
    • Are Your File Transfer Integrations GDPR Compliant?

      The onslaught of data security breaches today is relentless, with thousands of major breaches each year and 50 percent more breaches in 2019 vs. 2018, according to a report by Risk Based Security. The costs for each breach have burgeoned as well, with the average cost of a data breach at about $3.92 million.


      Securing data from breaches not only spares bottom line and publicity, it's now also a basic legal requirement to comply with rapidly growing data privacy laws.


      While organizations have long had to comply with industry-specific standards, such as HIPAA in healthcare and the Payment Card Industry Data Security Standard (PCI DSS), you now also face new consumer privacy regulations. Including:

      Read more →
    • JSONPath in PostgreSQL: committing patches and selecting apartments


        This article was written in Russian in 2019 after the PostgreSQL 12 feature freeze, and it is still up-to-date. Unfortunately other patches of the SQL/JSON will not get even into version 13.
        Many thanks to Elena Indrupskaya for the translation.

        JSONPath


        All that relates to JSON(B) is relevant and of high demand in the world and in Russia, and it is one of the key development areas in Postgres Professional. The jsonb type, as well as functions and operators to manipulate JSON/JSONB, appeared as early as in PostgreSQL 9.4. They were developed by the team lead by Oleg Bartunov.

        The SQL/2016 standard provides for JSON usage: the standard mentions JSONPath — a set of functionalities to address data inside JSON; JSONTABLE — capabilities for conversion of JSON to usual database tables; a large family of functions and operators. Although JSON has long been supported in Postgres, in 2017 Oleg Bartunov with his colleagues started their work to support the standard. Of all described in the standard, only one patch, but a critical one, got into version 12; it is JSONPath, which we will, therefore, describe here.
        Read more →
      • Laser that cuts inside the cornea: ReLEx procedure at the physical level

        • Translation
        The idea — to take and cut a lens in a transparent cornea — is not new. At first it was done manually, with a scalpel directly on the surface (difficult and very rough, with a sea of side effects). The first laser was used in 1979, then it was a pulsed infrared emitter with an effective pulse length of 4 nanoseconds.


        Step 1: creating a plasma bubble, in fact — a microburst. Step 2: expansion of the shock and heat waves. Step 3: cavitation bubble (plasma expansion). Step 4: the formation of a parallel slice at the expense of several adjacent laser focus points.

        Read more →
      • Reach Out Top Hadoop Consulting Companies To Leverage Big Data In 2020

          image

          Hadoop is divided into different modules, each of which delivers a distinct task crucial for a computer system and is uniquely designed for big data analytics. Apache Software Foundation developed this incredible platform. It is extensively utilized by worldwide developers to build big data Hadoop solutions amazingly and easily.

          Big data offers several perks, some of them are; examining root causes of failures, recognizing the potential of data-driven marketing, improving and enhancing customer engagement, and much more. By offering multiple solutions in a single stream it helps in lowering the cost of the organization.

          In various industries such as Retail, Manufacturing, Financial insurance, Education, Transportation, Agriculture, Healthcare, Energy, etc big data is utilized and that’s why it’s demand is expanding day by day. The Global Hadoop Market is envisioned to grow to $84.6 billion by 2021, with an expected CAGR of 63.4%.
          Read more →
        • Agile English teaching. What is it?



            Modern-day agile English teaching has come to take the place of rigid, cut-and-dried lessons that are fast becoming a thing of the past.

            Let me clarify what I mean by agile teaching that is bound to substitute conventional teaching.

            Some decades ago and up until recently it was perfectly valid to choose a certain textbook and go through it module by module together with your students (be it a group or individual learners). Given the abundance of high-quality materials readily accessible online and offline, it is completely unthinkable to proceed with this outdated approach.
            Read more →
          • Why I keep track of spendings in a personal app made with Git+JS

            • Translation

            Hi, folks, let me share my experience of creating an application to keep track of my spendings. Specifically, let me do it by answering the following questions:


            1. Why keep track of spendings in an application?
            2. Why did I create the application as a personal project?
            3. Why does the project use Git+JS?

            1. Why keep track of spendings in an application?


            I, like many people out there, wanted to become rich and successful. To become rich, one is often advised to run a personal budget, that's what I started to do several years ago. I'd like to point out that running my personal budget hasn't made me rich and successful, and I increased income simply by moving to Moscow.

            Read more →
          • AdBlock has stolen the banner, but banners are not teeth — they will be back

            More
            Ads
          • Is Python The Answer To High-End Mobile Apps?

            image

            Before the development of any software or a mobile app, the first thing that comes in mind is- which programming language is the best for this app. Yes! It is really important to choose the best programming language for particular mobile app development.

            So, among various programming languages, let’s throw light on the significance of Python in Mobile app development or it could also be said that why Python is the best language for mobile app development?

            Before that, let’s have a look at the stat where according to statista.com, Python remains the most popular programming language in 2020, according to GitHub and Google Trends, surpassing longstanding Java and JavaScript in popularity.

            Python was also the biggest gainer in the Tiobe index of the popularity of programming languages in the year 2018, rising by 3.62% points from January 2018 to January 2019, with a rating of approx. 8.2% for this month.
            Read more →
          • Best .Net Development Tools in 2020

              .Net is indeed an important application development platform as it is secure, robust and quite easy to learn and implement. Developers are widely using the .Net framework to build web applications and even modernizing the legacy programming based applications into .Net based ones. .Net developers also use many 3rd party tools to carry out the development work. These tools have proven to provide the best support while the development goes on.

              Here are some of the top useful tools being used by many offshore .net development teams, .net developers, individual .net programmers, etc.
              Read more →
            • Ansible: CoreOS to CentOS, 18 months long journey


                There was a custom configuration management solution.


                I would like to share the story about a project. The project used to use a custom configuration management solution. Migration lasted 18 months. You can ask me 'Why?'. There are some answers below about changing processes, agreements and workflows.

                Read more →
              • Why we fell in love with Vue.js?

                  Implementation a web UI is faced with more and more complex tasks that require the use of more and more complex tools. The trend of leaving the MVC application architecture, as well as the desire of developers to use separate libraries for each layer of tasks, led to changes, against which the Vue.JS framework appeared and became popular. Let's try to understand why Vue.JS appeared and what problems it carefully solves.

                  Read more →
                • How to test Ansible and don't go nuts


                    It is the translation of my speech at DevOps-40 2020-03-18:


                    After the second commit, each code becomes legacy. It happens because the original ideas do not meet actual requirements for the system. It is not bad or good thing. It is the nature of infrastructure & agreements between people. Refactoring should align requirements & actual state. Let me call it Infrastructure as Code refactoring.

                    Read more →
                  • Could Quantum Computing Help Reverse Climate Change?

                      The unique powers of quantum computation may give humanity an important weapon — or several weapons — against climate change, according to one quantum computer pioneer.
                      One of the possible solutions for the excess carbon in the atmosphere and to reach global climate goals is to suck it out. It sounds pretty easy, but, in fact, the technology to do so cheaply and easily isn’t quite here yet, according to Jeremy O’Brien Chief Executive Officer, PsiQuantum, a quantum computing startup.

                      Currently, there is no way to simulate large complex molecules, like carbon dioxide. Current classical computers cannot simulate these types of molecules because the problem grows exponentially with the size or complexity of the simulated molecules, according to O’Brien, who wrote an article outlining the issue at the World Economic Forum’s annual meeting held recently.

                      “Crudely speaking, if simulating a molecule with 10 atoms takes a minute, a molecule with 11 takes two minutes, one with 12 atoms takes four minutes and so on,” he writes. “This exponential scaling quickly renders a traditional computer useless: simulating a molecule with just 70 atoms would take longer than the lifetime of the universe (13 billion years).”
                      Read more →
                    • Porting packages to buildroot using the Zabbix example

                      • Tutorial


                      The basics of porting


                      Originally, Buildroot offers a limited number of packages. It makes sense — there is everything you need, but any other packages can be added.


                      To add a package, create 2 description files, an optional checksum file, and add a link to the package in the general package list. There are hooks at different stages of the build. At the same time, Buildroot can recognize the needed type of packages:

                      Read more →
                    • Traps and pitfalls in modernization of enterprise application using «supersonic subatomic java»

                      This is a post about first steps and first pitfalls in modernization of old enterprise application using "supersonic subatomic java" quarkus as it is positioned by RedHat.


                      Initial setup


                      In the end of 2019 I was invited to work in a project in our company, where an old monolithic application should be split into micro-services. Basic reasoning behind this decision was, that framework which is used in application is near to it's end-of-life. The application should be rewritten in any case. If it should be rewritten in any case, why not to split it into micro-services?


                      Since last 10 years, I was working mostly with java and we had in the project specialists with java knowledge, we have decided to give java-based frameworks a try for back-end functionality.
                      OK, let's use spring-cloud for that purpose, was our first thought. But then, we have had a look onto quarkus, which has been released in the end of 2019. We have decided to give it a try, keeping in mind building of native applications using GraalVM.


                      Native applications from our perspective could give us following benefits:


                      • shorten start time of container
                      • reduce resource consumption of container and application

                      We were aware about possible drawbacks of this solution:


                      • no experience in our team with quarkus framework
                      • significantly less amount of available feedback from community, since this is very young framework
                      Read more →
                    • When the COVID-19 pandemic will end

                        Dear all,


                        I am the head of Data Science at British Transport Police, and one of our department tasks is to efficiently allocate staff, depending on the crime rates, which correlate to passenger flow. As you understand, the passenger flow will undertake significant change as soon as the Government decides to cancel quarantine or stop some limitations. The question naturally arises: when will the pandemic end and how to prepare for a return to normal life.

                        Read more →
                      • What is Baked in the Baker's Dozen?

                          On April 8, PostgreSQL feature freeze took place, so only features committed earlier will get into version PostgreSQL 13. Probably, this version can hardly be considered revolutionary, since it has no conceptual changes. Some of critical patches were late to get into it, such as Table and Functions for the JSON/SQL standard, which had been desirable to be part of PostgreSQL 12, along with the JSONPath patch; plug-in warehouses did not appear either — only the interface is being finalized. The list of improvements is still impressive. We prepared a pretty complete overview of the patches included in the Baker's Dozen.
                          Read more →
                        • About embedded again: searching for bugs in the Embox project

                            Рисунок 2

                            Embox is a cross-platform, multi-tasking real-time operating system for embedded systems. It is designed to work with limited computing resources and allows you to run Linux-based applications on microcontrollers without using Linux itself. Certainly, the same as other applications, Embox couldn't escape from bugs. This article is devoted to the analysis of errors found in the code of the Embox project.
                            Read more →
                          • Blood sugar and COVID-19

                              Novel pandemic is very new for science, often it takes years before scientists prove connection of risk factors and replicate their findings in experimental setups, but it is not the case to wait. I have been observing different facts about COVID-19 and propose “Hypothesis for connection between blood sugar levels and infection”. The only reason I do this now with so many controversies is that I genuinely believe it can save lives. Lives of my friends, and their relatives.

                              In the article below I summarise my knowledge on infection and immunity, back it up by links to COVID observations of doctors and scientific papers.
                              Read more →
                            • Working with light: Starting your career at ITMO University

                                One of our previous articles featured an overview of our photonics department students’ work lives. Today we’re going to expand on this topic by looking at four related MA programs: “Light Guide Photonics and Programmable Electronics”, “LED technologies and optoelectronics”, “Photonic materials” and “Laser technologies”. We sat down with some of the folks currently enrolled in these programs, as well as recent graduates, to talk about the role ITMO University played in kickstarting their careers.

                                Read more →