Development

When you run queries in ClickHouse, you might notice that the profiler often shows the LZ_decompress_fast function near the top. What is going on? This question had us wondering how to choose the best compression algorithm.

ClickHouse stores data in compressed form. When running queries, ClickHouse tries to do as little as possible, in order to conserve CPU resources. In many cases, all the potentially time-consuming computations are already well optimized, plus the user wrote a well thought-out query. Then all that's left to do is to perform decompression.



So why does LZ4 decompression becomes a bottleneck? LZ4 seems like an extremely light algorithm: the data decompression rate is usually from 1 to 3 GB/s per processor core, depending on the data. This is much faster than the typical disk subsystem. Moreover, we use all available CPU cores, and decompression scales linearly across all physical cores.

This is the second post in our series of articles about the results of checking open-source software working with the RDP protocol. Today we are going to take a look at the rdesktop client and xrdp server.

On March 13, a proposal for the RIPE anti-abuse working group was submitted, stating that a BGP hijacking event should be treated as a policy violation. In case of acceptance, if you are an ISP attacked with the hijack, you could submit a special request where you might expose such an autonomous system. If there is enough confirming evidence for an expert group, then such a LIR would be considered an adverse party and further punished. There were some arguments against this proposal.

With this article, we want to show an example of the attack where not only the true attacker was under the question, but the whole list of affected prefixes. Moreover, it again raises concerns about the possible motives for the future attack of this type.

Our team provides quick and effective customer support. User requests are handled solely by programmers since our clients are programmers themselves and they often ask tricky questions. Today I'm going to tell you about a recent request concerning one false positive that even forced me to carry out a small investigation to solve the problem.

Ability to write good unit tests is an important feature of any developer. But how to understand that your unit tests are correct? Good unit test is like a good chess game. In our case chessmen are the approaches which we are going to discuss in this post. There is no best chessman in a chess game because everything depends on the positions (and a player). Likewise, in unit testing you don't have to distinguish only one approach. In other words, you should use all approaches together to get the best result. So, if you want to win this game, then welcome under the cut.

Leadership is not a service, it’s a skill. Professionals working as a software developer for a couple of years are given the chance to be a tech lead. However, remember that ‘with great power comes great responsibility.’

There are several things that you need to take care of while being a tech lead. Obviously, you don’t need to code as much as you need to do while being a software developer. However, there are several other non-coding related things that now are your responsibility to deal with.

10 Tips for Being a Good Tech Lead

Maintaining a tech lead position while not gaining any criticism from the team isn’t possible. This is not due to your incapacity albeit due to human nature. However, the effort can be made to minimize it and becoming better in what you do eventually. After all, you are the leader now.

Just about the fastest portable 64-bit hash function with decent quality.

This is a translation of the original article by Leonid Yuriev.

Long time ago, when a mobile phone costed about 2000$ and one minute of voice call was 50 cents, pagers were really popular. Later cellular phones became cheaper, calls and SMS prices became lower, and finally pagers mostly disappeared.


For people, who owned a pager before, and want to know how it works, this article will be useful.

This project is aimed at creating an experimental device for emulating RFID labels of three widely available components. I simplified the explanation of the process so that it could be easily replicated. I also developed some helpful ideas along the way, including writing a special program for converting a serial number into the transmitted data, which will definitely prove useful.

I manage a team that designs and introduces in-house Kubernetes aaS at Mail.ru Cloud Solutions. And we often see a lack of understanding as to this technology, so I’d like to talk about common strategic mistakes at Kubernetes implementation in major projects.

Most of the problems arise because the technology is quite sophisticated. There are unobvious implementation and operation challenges, as well as poorly used advantages, all of those resulting in money loss. Another issue is the global lack of knowledge and experience with Kubernetes. Learning its use by the book can be tricky, and hiring qualified staff can be challenging. All the hype complicates Kubernetes-related decision making. Curiously enough, Kubernetes is often implemented rather formally – just for it to be there and make their lives better in some way.

Hopefully, this post will help you to make a decision you will feel proud of later (and won’t regret or feel like building a time machine to undo it).

I came to Mail.ru Group in 2013, and I required a queue for one task. First of all, I decided to check what the company had already got. They told me they had this Tarantool product, and I checked how it worked and decided that adding a queue broker to it could work perfectly well.

I contacted Kostja Osipov, the senior expert in Tarantool, and the next day he gave me a 250-string script that was capable of managing almost everything I needed. Since that moment, I have been in love with Tarantool. It turned out that a small amount of code written with a quite simple script language was capable of ensuring some totally new performance for this DBMS.

Today, I’m going to tell you how to instantiate your own queue in Tarantool 2.2.

A few days ago 0xd34df00d has published the translation of the article, describing the possible information about some function if we use it as a "black box", not trying to read its implementation. Of course, this information is quite different from language to language; in the original article, four cases were considered:

• Python — dynamic typing, almost no information from signature, some hints are gained by the tests;
• C — weak static typing, a little more information;
• Haskell — strong static typing, with pure functions by default, a lot more information;
• Idris — dependent typing, compiler can prove the function correctness.

"Here's C and there's Haskell, and what about Rust?" — this was the first question in the following discussion. The reply is here.

PVS-Studio is a tool for detecting software bugs and potential vulnerabilities in the source code of software written in C, C++, C#, and Java. The analyzer runs on Windows, Linux, and macOS.

At the moment, cloud CI systems are a highly-demanded service. In this article, we'll tell you how to integrate analysis of source code into a CI cloud platform with the tools that are already available in PVS-Studio. As an example we'll use the Travis CI service.

Many of our articles are focused on anything, but not the PVS-Studio tool itself. Whereas we do a lot to make its usage convenient for developers. Nevertheless, our efforts are often concealed behind the scenes. I decided to remedy this situation and tell you about the PVS-Studio plugin for Visual Studio. If you use Visual Studio, this article is for you.

Intro

.NET is a relative newcomer in the open-source world, and its popularity is nowhere near mainstream platforms like Node.js. So you can imagine there're few tutorials that deal with .NET and frameworks such as ASP.NET on Heroku. And those that do, probably won't use containers.

Do you see C#/.NET here? Yes, me neither.

My name is Vladimir, and I develop mobile front-end for Yandex Mail. Our apps have had a dark theme for a while, but it was incomplete: only the interface and plain emails were dark. Messages with custom formatting remained light and stood out against the dark interface, hurting our users’ eyes at night.

Today I'll tell you how we fixed this problem. You will learn about two simple techniques that didn't work for us and the method that finally did the trick — adaptive page recoloring. I'll also share some ideas about adapting images to a dark theme. To be fair, darkening pages with custom CSS is a rather peculiar task, but I believe some of you may find our experience helpful.

Today’s taxi market is crowded with ride-sharing apps like Uber, Ola, and Lyft. They are giving serious competition for upcoming ventures. These on-demand apps have initiated the radical shift in the taxi industry enabling the smooth transition from conventional taxi hailing to booking a cab through an app. Customers are opting for hailing taxis through the app because of its reliability to get a taxi and ensuring a much better cab booking experience. Thus, the taxi market is ripe for the taking, and this is the right time for entrepreneurs to make use of the opportunity.

Statistics on the Taxi Industry

The traditional taxi industry is still stuck with inefficient phone call bookings, competitors and they stand to lose at least $20 million on an everyday basis just because customers couldn’t find a cab. Another statistics survey tells us that almost 80% of the commuter prefer Ola and Uber over regular taxis and that Uber has completed nearly 2 billion rides since its inception. The profits soared over $20 Billion in the process. What made Uber and Ola successful ventures is the sound business strategy they employed. So if you are going to establish your venture then understanding the market is imperative.

Google loves easter eggs. It loves them so much, in fact, that you could find them in virtually every product of theirs. The tradition of Android easter eggs began in the very earliest versions of the OS (I think everyone there knows what happens when you go into the general settings and tap the version number a few times).

But sometimes you can find an easter egg in the most unlikely of places. There’s even an urban legend that one day, a programmer Googled “mutex lock”, but instead of search results landed on foo.bar, solved all tasks and landed a job at Google.


The same thing (except without the happy ending) happened to me. Hidden messages where there definitely couldn’t be any, reversing Java code and its native libraries, a secret VM, a Google interview — all of that is below.

This article is about the interrupt delivery process from external devices in the x86 system. It tries to answer questions such as:

• What is PIC and what is it for?
• What is APIC and what is it for? What is the purpose of LAPIC and I/O APIC?
• What are the differences between APIC, xAPIC, and x2APIC?
• What is MSI? What are the differences between MSI and MSI-X?
• What is the role of the $PIR, MPtable, and ACPI tables?

If you want to know the answer for one of these questions, or if you simply want to know about interrupt controller evolution, please, welcome.