How to become an author
.NET Knowledge Base
  • All streams
  • Development
  • Administrating
  • Design
  • Management
  • Marketing
  • PopSci
Log in Sign up
140.25
Rating

High performance

Methods for obtaining systems high performance

All posts

Top

Authors

Companies

  • Any rating
  • ≥10
  • ≥25
  • ≥50
  • ≥100
  • codesign November 26, 2020 at 09:49 PM

    How to build a high-performance application on Tarantool from scratch

    • Mail.ru Group corporate blog,
    • High performance,
    • Algorithms,
    • Lua,
    • Tarantool
    • Tutorial
    image

    I came to Mail.ru Group in 2013, and I required a queue for one task. First of all, I decided to check what the company had already got. They told me they had this Tarantool product, and I checked how it worked and decided that adding a queue broker to it could work perfectly well.

    I contacted Kostja Osipov, the senior expert in Tarantool, and the next day he gave me a 250-string script that was capable of managing almost everything I needed. Since that moment, I have been in love with Tarantool. It turned out that a small amount of code written with a quite simple script language was capable of ensuring some totally new performance for this DBMS.

    Today, I’m going to tell you how to instantiate your own queue in Tarantool 2.2.
    Read more →
    • +18
    • 781
    • Comment
  • vovkins June 26, 2020 at 11:31 AM

    Making a Tarantool-Based Investment Business Core for Alfa-Bank

    • Mail.ru Group corporate blog,
    • High performance,
    • System Analysis and Design,
    • Database Administration,
    • Tarantool

      A still from «Our Secret Universe: The Hidden Life of the Cell»

      Investment business is one of the most complex domains in the banking world. It's about not just credits, loans, and deposits — there are also securities, currencies, commodities, derivatives, and all kinds of complex stuff like structured products.

      Recently, people have become increasingly aware of their finances. More and more get involved in securities trading. Individual investment accounts have emerged not so long ago. They allow you to trade in securities and get tax credits or avoid taxes at the same time. All clients coming to us want to manage their portfolios and see their reporting on-line. Most frequently, these are multi-product portfolios, which means that people are clients of different business areas.

      Moreover, the demands of regulators, both Russian and international, also grow.

      To meet the current needs and lay a foundation for future upgrades, we've developed our Tarantool-based investment business core.
      Read more →
      • +14
      • 821
      • Comment
    • dokshina April 13, 2020 at 02:34 PM

      Deploying Tarantool Cartridge applications with zero effort (Part 2)

      • Mail.ru Group corporate blog,
      • High performance,
      • Database Administration,
      • Distributed systems,
      • Tarantool
      • Tutorial


      We have recently talked about how to deploy a Tarantool Cartridge application. However, an application's life doesn't end with deployment, so today we will update our application and figure out how to manage topology, sharding, and authorization, and change the role configuration.

      Feeling interested? Please continue reading under the cut.
      Read more →
      • +15
      • 528
      • Comment
    • fessoga5 February 25, 2020 at 07:19 PM

      Fault Tolerance Web Architecture for Our Cloud Solutions

      • Mail.ru Group corporate blog,
      • High performance,
      • Virtualization,
      • Cloud computing,
      • Cloud services
        image

        Hi Habr,

        I'm Artyom Karamyshev, a system administration team leader at Mail.Ru Cloud Solutions (MCS). We launched many products in 2019. We've aimed to make API services easily scalable, fault-tolerant, and ready to accommodate rapid growth. Our platform is running on OpenStack, and in this article, I describe all the component fault tolerance issues that we've resolved.

        The overall fault tolerance of the platform is consists of its components fault tolerance. So, I'm going to show you step by step tutorial about all levels where we've found the risks.
        Read more →
        • +23
        • 1.4k
        • Comment
      • Please pay attention

      • gen December 18, 2019 at 12:48 PM

        Accelerating PHP connectors for Tarantool using Async, Swoole, and Parallel

        • Mail.ru Group corporate blog,
        • High performance,
        • PHP,
        • SQL,
        • NoSQL


          In the PHP ecosystem, there are currently two connectors for the Tarantool server: the official PECL extension tarantool/tarantool-php written in C, and tarantool-php/client written in PHP. I am the author of the latter one.

          In this article I would like to share the results of performance testing of both these libraries and show how you can achieve 3x-5x performance improvement (on synthetic tests!) with minimal changes in code.
          Read more →
          • +39
          • 1.1k
          • Comment
        • dokshina December 16, 2019 at 12:41 PM

          Deploying Tarantool Cartridge applications with zero effort (Part 1)

          • Mail.ru Group corporate blog,
          • High performance,
          • IT Infrastructure,
          • Database Administration,
          • Distributed systems


            We have already presented Tarantool Cartridge that allows you to develop and pack distributed applications. Now let's learn how to deploy and control these applications. No panic, it's all under control! We have brought together all the best practices of working with Tarantool Cartridge and wrote an Ansible role, which will deploy the package to servers, start and join instances into replica sets, configure authorization, bootstrap vshard, enable automatic failover and patch cluster configuration.

            Interesting, huh? Dive in, check details under the cut.
            Read more →
            • +29
            • 788
            • Comment
          • vasiliy-t October 21, 2019 at 07:09 PM

            Tarantool Kubernetes Operator

            • Mail.ru Group corporate blog,
            • High performance,
            • Database Administration,
            • Kubernetes,
            • Tarantool


              Kubernetes has already become a de-facto standard for running stateless applications, mainly because it can reduce time-to-market for new features. Launching stateful applications, such as databases or stateful microservices, is still a complex task, but companies have to meet the competition and maintain a high delivery rate. So they create a demand for such solutions.

              We want to introduce our solution for launching stateful Tarantool Cartridge clusters: Tarantool Kubernetes Operator, more under the cut.
              Read more →
              • +34
              • 1.1k
              • Comment
            • Ads
              AdBlock has stolen the banner, but banners are not teeth — they will be back

              More
            • mt144 October 16, 2019 at 03:10 PM

              Tarantool Data Grid: Architecture and Features

              • Mail.ru Group corporate blog,
              • High performance,
              • Lua,
              • Data storage,
              • Tarantool


                In 2017, we won the competition for the development of the transaction core for Alfa-Bank's investment business and started working at once. (Vladimir Drynkin, Development Team Lead for Alfa-Bank's Investment Business Transaction Core, spoke about the investment business core at HighLoad++ 2018.) This system was supposed to aggregate transaction data in different formats from various sources, unify the data, save it, and provide access to it.

                In the process of development, the system evolved and extended its functions. At some point, we realized that we created something much more than just application software designed for a well-defined scope of tasks: we created a system for building distributed applications with persistent storage. Our experience served as a basis for the new product, Tarantool Data Grid (TDG).

                I want to talk about TDG architecture and the solutions that we worked out during the development. I will introduce the basic functions and show how our product could become the basis for building turnkey solutions.
                Read more →
                • +37
                • 1.3k
                • Comment
              • Rosik October 9, 2019 at 05:32 PM

                Тarantool Cartridge: Sharding Lua Backend in Three Lines

                • Mail.ru Group corporate blog,
                • High performance,
                • Lua,
                • Distributed systems,
                • Tarantool

                  In Mail.ru Group, we have Tarantool, a Lua-based application server and a database united. It's fast and classy, but the resources of a single server are always limited. Vertical scaling is also not the panacea. That is why Tarantool has some tools for horizontal scaling, or the vshard module [1]. It allows you to spread data across multiple servers, but you'll have to tinker with it for a while to configure it and bolt on the business logic.

                  Good news: we got our share of bumps (for example, [2], [3]) and created another framework, which significantly simplifies the solution to this problem.

                  Тarantool Cartridge is the new framework for developing complex distributed systems. It allows you to concentrate on writing business logic instead of solving infrastructure problems. Under the cut, I will tell you how this framework works and how it could help in writing distributed services.
                  Read more →
                  • +24
                  • 1.4k
                  • Comment
                • kaiuk September 10, 2019 at 02:27 PM

                  Enumerable: How to yield a business value

                  • High performance,
                  • .NET,
                  • C#,
                  • ООP,
                  • Microservices
                    This article is a brief explanation about how using a common language keywords might have an influence on the budget of IT-infrastructure of a project or help to achieve some limitations/restrictions of hosting infrastructure and, moreover, will be a good sing of the quality and maturity of the source code.
                    Read more →
                    • +10
                    • 1.3k
                    • Comment
                  • thatsme July 28, 2019 at 04:18 AM

                    12.3 million of concurrent WebSockets

                    • High performance,
                    • C++,
                    • *nix,
                    • Lua
                    • Recovery Mode

                    One thing about WebSockets is that you need a lot of resources on the client's side to generate high enough load for the server to actually eat up all the CPU resources.


                    There are several challenges you have to overcome because the WebSockets protocol is more CPU demanding on the client's side than on the server's side. At the same time you need a lot of RAM to store information about open connections if you have millions of them.


                    I've been lucky enough to get a couple of new servers for a limited period of time at my disposal for the hardware "burnout" tests. So I decided to use my Lua Application Server — LAppS to do both jobs: test the hardware and perform the LAppS high load tests.


                    Read more →
                    • 0
                    • 5k
                    • Comment
                  • o6CuFl2Q June 25, 2019 at 05:42 PM

                    How to speed up LZ4 decompression in ClickHouse?

                    • Яндекс corporate blog,
                    • High performance,
                    • Open source,
                    • C++,
                    • Big Data
                      When you run queries in ClickHouse, you might notice that the profiler often shows the LZ_decompress_fast function near the top. What is going on? This question had us wondering how to choose the best compression algorithm.

                      ClickHouse stores data in compressed form. When running queries, ClickHouse tries to do as little as possible, in order to conserve CPU resources. In many cases, all the potentially time-consuming computations are already well optimized, plus the user wrote a well thought-out query. Then all that's left to do is to perform decompression.



                      So why does LZ4 decompression becomes a bottleneck? LZ4 seems like an extremely light algorithm: the data decompression rate is usually from 1 to 3 GB/s per processor core, depending on the data. This is much faster than the typical disk subsystem. Moreover, we use all available CPU cores, and decompression scales linearly across all physical cores.
                      Read more →
                      • +19
                      • 5.9k
                      • 1
                    • alexey_zz May 23, 2019 at 02:00 PM

                      From High Ceph Latency to Kernel Patch with eBPF/BCC

                      • Selectel corporate blog,
                      • High performance,
                      • Configuring Linux,
                      • Server Administration


                        There are a lot of tools for debugging kernel and userspace programs in Linux. Most of them have performance impact and cannot easily be run in production environments. A few years ago, eBPF was developed, which provides the ability to trace the kernel and userspace with low overhead, without needing to recompile programs or load kernel modules.

                        There are now plenty of tools that use eBPF and in this article, we’ll explain how to write your own profiling tool using the PythonBCC library. This article is based on a real issue from the production environment. We’ll walk you through solving the problem and show how existing bcc tools could be used in some cases.
                        Read more →
                        • +9
                        • 2.8k
                        • Comment
                      • danikin May 13, 2019 at 12:02 PM

                        Citymobil — a manual for improving availability amid business growth for startups. Part 5

                        • Mail.ru Group corporate blog,
                        • High performance,
                        • System Analysis and Design,
                        • Start-up development,
                        • Product Management


                          This is the final part of the series describing how we’re increasing our service availability in Citymobil (you can read the previous part here). Now I’m going to talk about one more type of outages and the conclusions we made about them, how we modified the development process, what automation we introduced.
                          Read more →
                          • +24
                          • 696
                          • Comment
                        • danikin May 7, 2019 at 12:43 PM

                          Citymobil — a manual for improving availability amid business growth for startups. Part 4

                          • Mail.ru Group corporate blog,
                          • High performance,
                          • System Analysis and Design,
                          • Start-up development,
                          • Product Management


                            This is the next article of the series describing how we’re increasing our service availability in Citymobil (you can read the previous parts here: part 1, part 2, part 3). In further parts, I’ll talk about the accidents and outages in detail.

                            1. Bad release: database overload


                            Let me begin with a specific example of this type of outage. We deployed an optimization: added USE INDEX in an SQL query; during testing as well as in production, it sped up short queries, but the long ones — slowed down. The long queries slowdown was only noticed in production. As a result, a lot of long parallel queries caused the database to be down for an hour. We thoroughly studied the way USE INDEX worked; we described it in the Do’s and Dont’s file and warned the engineers against the incorrect usage. We also analyzed the query and realized that it retrieves mostly historical data and, therefore, can be run on a separate replica for historical requests. Even if this replica goes down due to an overload, the business will keep running.
                            Read more →
                            • +15
                            • 617
                            • Comment
                          • danikin April 29, 2019 at 11:51 AM

                            Citymobil — a manual for improving availability amid business growth for startups. Part 3

                            • Mail.ru Group corporate blog,
                            • High performance,
                            • System Analysis and Design,
                            • Start-up development,
                            • Product Management


                              This is the next article of the series describing how we’re increasing our service availability in Citymobil (you can read the previous parts here and here). In further parts, I’ll talk about the accidents and outages in detail. But first let me highlight something I should’ve talked about in the first article but didn’t. I found out about it from my readers’ feedback. This article gives me a chance to fix this annoying shortcoming.
                              Read more →
                              • +23
                              • 696
                              • Comment
                            • morozovsk April 29, 2019 at 01:15 AM

                              Statistics and monitoring of PHP scripts in real time. ClickHouse and Grafana go to Pinba for help

                              • High performance,
                              • PHP
                              • Tutorial
                              In this article I will explain how to use pinba with clickhouse and grafana instead of pinba_engine and pinboard.

                              On the php project pinba is probably the only reliable way to understand what is happening with performance. But usually people start to use pinba only when problems are already observed and it isn't clear where to look in.

                              Often developers have no idea how many RPS each script has. So they begin to optimize starting from places that seem to have problem.

                              Someone is analyzing the nginx logs, and someone is slow queries in the database.

                              Of course pinba would not be superfluous, but there are several reasons why it is not on every project.


                              Read more →
                              • +14
                              • 3.5k
                              • Comment
                            • danikin April 25, 2019 at 12:03 PM

                              Citymobil — a manual for improving availability amid business growth for startups. Part 2

                              • Mail.ru Group corporate blog,
                              • High performance,
                              • System Analysis and Design,
                              • Start-up development,
                              • Product Management


                                This is a second article out of a series «Citymobil — a manual for improving availability amid business growth for startups». You can read the first part here. Let’s continue to talk about the way we managed to improve the availability of Citymobil services. In the first article, we learned how to count the lost trips. Ok, we are counting them. What now? Now that we are equipped with an understandable tool to measure the lost trips, we can move to the most interesting part — how do we decrease losses? Without slowing down our current growth! Since it seemed to us that the lion’s share of technical problems causing the trips loss had something to do with the backend, we decided to turn our attention to the backend development process first. Jumping ahead of myself, I’m going to say that we were right — the backend became the main site of the battle for the lost trips.
                                Read more →
                                • +21
                                • 608
                                • Comment
                              • danikin April 22, 2019 at 06:52 PM

                                Citymobil — a manual for improving availability amid business growth for startups. Part 1

                                • Mail.ru Group corporate blog,
                                • High performance,
                                • System Analysis and Design,
                                • Start-up development,
                                • Product Management


                                  In this first part of an article series «Citymobil — a manual for improving availability amid business growth for startups» I’m going to break down the way we managed to dramatically scale up the availability of Citymobil services. The article opens with the story about our business, our task, the reason for this task to increase the availability emerged and limitations. Citymobil is a rapid-growing taxi aggregator. In 2018, it increased by more than 15 times in terms of number of successfully completed trips. Some months showed 50% increase compared with the previous month.

                                  The business grew like a weed in every direction (it still does): there was an increase in server load, team size and number of deployments. At the same time the new threats to service availability emerged. The company faced a task of the most importance — how to increase availability without compromising company growth. In this article, I’ll talk about the way we managed to solve this task in a relatively short time.
                                  Read more →
                                  • +24
                                  • 830
                                  • Comment
                                • charlieheart February 6, 2019 at 12:30 PM

                                  t1ha = Fast Positive Hash

                                  • High performance,
                                  • Perfect code,
                                  • C
                                  • From sandbox

                                  Just about the fastest portable 64-bit hash function with decent quality.


                                  This is a translation of the original article by Leonid Yuriev.

                                  Read more →
                                  • +19
                                  • 5.2k
                                  • 1
                                • ← here
                                • there →
                                • 1
                                • 2
                                Ads
                                AdBlock has stolen the banner, but banners are not teeth — they will be back

                                More

                                Community sponsors

                                • VTB
                                  Develops and makes financial services accessible online.
                                • Huawei
                                  ICT expert. Supplies smart devices and builds intelligent infrastructure
                                • Avito
                                  Master of ads in Russia. Handles high loads.
                                How to become a sponsor

                                Authors' contribution

                                • olegbunin 4,429.8
                                • AloneCoder 1,531.2
                                • olegchir 1,320.2
                                • it_man 795.8
                                • ilya42 739.6
                                • pmurzakov 650.4
                                • ARG89 645.2
                                • YuriPanchul 507.0
                                • mi5ha6in 445.0
                                • 23derevo 417.0
                                Top 100 authors
                                No reasons for downvotes yet
                                Reasons for downvotes

                                Top posts

                                • Day
                                • Week
                                • Month
                                • Audio over Bluetooth: most detailed information about profiles, codecs, and devices
                                  +22 152k 14 9
                                • CLIP from OpenAI: what is it and how you can try it out yourself
                                  +2 234 0 0
                                • How to grow as a programmer and software architect
                                  +3 1.6k 3 0
                                • Java vs .Net: Who will Reign in the Future?
                                  +3 20.8k 5 3
                                • How to grow as a programmer and software architect
                                  +3 1.6k 3 0
                                • Passcode Data Protection by Using FPGA and Verilog
                                  +1 253 0 0
                                • CLIP from OpenAI: what is it and how you can try it out yourself
                                  +2 234 0 0
                                • 10(+) years in the Labs
                                  +25 192 6 0
                                • Visualizing Network Topologies: Zero to Hero in Two Days
                                  +2 5.1k 4 0
                                • Russian microcontroller K1986BK025 based on the RISC-V processor core for smart electricity meters
                                  +9 2.3k 4 0
                                • How to grow as a programmer and software architect
                                  +3 1.6k 3 0
                                • Did It Have to Take So Long to Find a Bug?
                                  +1 1.2k 0 0

                                Please pay attention

                                Your account

                                • Log in
                                • Sign up

                                Sections

                                • Posts
                                • Hubs
                                • Companies
                                • Users
                                • Sandbox

                                Info

                                • How it works
                                • For Authors
                                • For Companies
                                • Documents
                                • Agreement
                                • Terms of service

                                Services

                                • Ads
                                • Subscription plans
                                • Content
                                • Seminars
                                • Megaprojects
                                © 2006 – 2021 «Habr»
                                Language settings
                                About
                                Support
                                Mobile version
                                Language settings
                                Interface
                                Content